-- Logs begin at Tue 2022-09-27 09:07:44 UTC, end at Tue 2022-09-27 11:34:33 UTC. -- Sep 27 09:45:01.896311 ncdv-master-1 systemd[1]: Starting Devstack devstack@n-api.service... Sep 27 09:45:01.908033 ncdv-master-1 devstack@n-api.service[129095]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 27 09:45:01 2022] *** Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: nodename: ncdv-master-1 Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: machine: x86_64 Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: clock source: unix Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: pcre jit disabled Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: detected number of CPU cores: 4 Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: current working directory: / Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: detected binary path: /usr/bin/uwsgi-core Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: your processes number limit is 47680 Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: your memory page size is 4096 bytes Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: detected max file descriptor number: 2048 Sep 27 09:45:01.910557 ncdv-master-1 devstack@n-api.service[129095]: lock engine: pthread robust mutexes Sep 27 09:45:01.912107 ncdv-master-1 devstack@n-api.service[129095]: thunder lock: enabled Sep 27 09:45:01.912107 ncdv-master-1 devstack@n-api.service[129095]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 27 09:45:01.912107 ncdv-master-1 devstack@n-api.service[129095]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 27 09:45:01.956025 ncdv-master-1 devstack@n-api.service[129095]: Python main interpreter initialized at 0x55bd9d3c59c0 Sep 27 09:45:01.956025 ncdv-master-1 devstack@n-api.service[129095]: python threads support enabled Sep 27 09:45:01.956025 ncdv-master-1 devstack@n-api.service[129095]: your server socket listen backlog is limited to 100 connections Sep 27 09:45:01.956025 ncdv-master-1 devstack@n-api.service[129095]: your mercy for graceful operations on workers is 90 seconds Sep 27 09:45:01.956590 ncdv-master-1 devstack@n-api.service[129095]: mapped 403077 bytes (393 KB) for 2 cores Sep 27 09:45:01.956590 ncdv-master-1 devstack@n-api.service[129095]: *** Operational MODE: preforking *** Sep 27 09:45:01.956945 ncdv-master-1 devstack@n-api.service[129095]: *** uWSGI is running in multiple interpreter mode *** Sep 27 09:45:01.956945 ncdv-master-1 devstack@n-api.service[129095]: spawned uWSGI master process (pid: 129095) Sep 27 09:45:01.957200 ncdv-master-1 systemd[1]: Started Devstack devstack@n-api.service. Sep 27 09:45:01.958650 ncdv-master-1 devstack@n-api.service[129095]: spawned uWSGI worker 1 (pid: 129096, cores: 1) Sep 27 09:45:01.958650 ncdv-master-1 devstack@n-api.service[129095]: spawned uWSGI worker 2 (pid: 129097, cores: 1) Sep 27 09:45:01.958930 ncdv-master-1 devstack@n-api.service[129095]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 27 09:45:03.388178 ncdv-master-1 devstack@n-api.service[129096]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 27 09:45:03.412201 ncdv-master-1 devstack@n-api.service[129097]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 27 09:45:04.108737 ncdv-master-1 devstack@n-api.service[129096]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:45:04.108737 ncdv-master-1 devstack@n-api.service[129096]: from cryptography.utils import int_from_bytes Sep 27 09:45:04.109102 ncdv-master-1 devstack@n-api.service[129096]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:45:04.109102 ncdv-master-1 devstack@n-api.service[129096]: from cryptography.utils import int_from_bytes Sep 27 09:45:04.133965 ncdv-master-1 devstack@n-api.service[129097]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:45:04.133965 ncdv-master-1 devstack@n-api.service[129097]: from cryptography.utils import int_from_bytes Sep 27 09:45:04.134335 ncdv-master-1 devstack@n-api.service[129097]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:45:04.134335 ncdv-master-1 devstack@n-api.service[129097]: from cryptography.utils import int_from_bytes Sep 27 09:45:06.764591 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 09:45:06.765336 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 09:45:06.765418 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 09:45:06.765745 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 09:45:06.766000 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 09:45:06.766393 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.766717 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 09:45:06.767116 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.767282 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 09:45:06.767541 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.767541 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.767728 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 09:45:06.767820 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.768002 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 09:45:06.768143 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.768276 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 09:45:06.768618 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.768752 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.768967 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.769097 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.769300 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.769412 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.769560 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.769873 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.769964 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.770202 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.770413 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.770506 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.770756 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.770849 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.771079 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.771172 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.771398 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.771492 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.771699 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.771794 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.771974 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.772065 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.772286 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.772386 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.772607 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.772818 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.772963 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.773306 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.773440 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.773555 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.773666 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.774039 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.774166 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.774287 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.774421 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.774552 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.774643 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.774776 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.774870 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.775094 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.775299 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.775405 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.775604 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.775702 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.775902 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.775997 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.776207 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.776447 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.776636 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.776812 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.776940 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.777063 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.777154 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.777322 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.778047 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.778132 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.778724 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.778805 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.778884 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.778968 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.779059 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.779146 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.779146 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.779319 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.779411 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.779535 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.779731 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.779854 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.780168 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.780253 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.780484 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.780778 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.780869 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.781113 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.781420 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.781512 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.781737 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.781989 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.782077 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.782326 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.782418 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.782574 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.782713 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.782878 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.783062 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.783197 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.783514 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.783605 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.783891 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.784132 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.784286 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.784409 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.784675 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.784777 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.785010 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.785221 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.785308 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.785604 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.785699 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.785907 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.785994 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.786209 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.786409 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.786506 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.786798 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.786928 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.787098 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.787400 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.787485 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.787716 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.787809 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.788027 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.788303 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.788398 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.788634 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.788759 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.788971 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.789163 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.789703 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.789871 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.789963 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.790258 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.790390 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.790596 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.790897 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.790988 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.791200 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.791297 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.791514 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.791599 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.791830 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.792120 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.792207 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.792423 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.792650 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.792830 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.793057 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.793205 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.793379 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.793680 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.793772 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.793927 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.794203 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.794289 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.794513 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.794713 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.794828 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.795131 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.795216 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.795434 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.795527 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.795735 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.795894 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.796029 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.796331 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.796415 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.796632 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.796842 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.796942 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.797215 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.797300 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.797531 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.797617 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.797832 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.797931 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.798133 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.798243 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.798437 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.798740 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.798825 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.799040 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.799127 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.799477 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.799638 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.799794 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.799989 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.800091 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.800384 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.800485 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.800727 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.800831 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.801011 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.801110 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.801325 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.801409 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.801625 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.801721 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.801925 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.802021 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.802242 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.802328 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.802543 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.802766 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.802865 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.803144 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.803304 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.803447 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.803749 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.803833 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.804063 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.804179 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.804359 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.804577 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.804678 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.804983 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.805078 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.805288 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.805532 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.805617 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.805879 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.805970 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.806179 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.806268 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.806479 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.806565 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.806779 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.806875 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.807078 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.807165 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.807377 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.807469 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.807673 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.807763 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.807989 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.808082 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.808304 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.808390 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.808600 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.808718 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.808928 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.809029 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.809229 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.809320 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.809526 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.809625 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.809826 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.809913 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.810142 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.810236 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.810456 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.810549 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.810754 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.810847 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.811054 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.811155 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.811352 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.811447 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.811649 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.811743 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.811962 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.812045 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.812260 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.812354 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.812558 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.812652 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.812899 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.812987 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.813199 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.813343 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.813499 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.813586 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.813810 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.813902 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.814109 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.814196 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.814411 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.814540 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.814716 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.814816 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.815117 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.815222 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.815480 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.815567 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.815788 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.816047 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.816134 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.816411 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.816505 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.816757 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.816863 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.817069 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.817163 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.817381 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.817471 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.817682 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.817769 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.817980 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.818073 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.818279 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.818361 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.818580 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.818700 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.818874 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.819018 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.819187 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.819342 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.819500 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.819626 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.819803 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.819892 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.820102 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.820195 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.820400 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.820518 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.820738 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:45:06.820845 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.821070 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.821163 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.821392 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.821490 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.821708 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.821818 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.822010 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.822100 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.822326 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.822485 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.822617 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.822737 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.822928 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.823032 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.823238 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.823334 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.823546 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.823676 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.823865 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.824000 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.824167 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.824303 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.824473 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.824620 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.824801 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.824931 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.825108 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.825226 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.825414 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.825513 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.825715 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.825801 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.826017 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.826105 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.826319 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.826426 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.826621 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.826716 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.826924 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.827046 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.827222 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.827369 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.827542 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.827679 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.827939 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.828074 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.828225 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.828348 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.828476 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.828612 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.828855 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.828981 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.829119 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.829242 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.829382 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.829514 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.829668 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.829768 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.829972 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.830064 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.830282 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.830381 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.830724 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.830829 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.830908 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.831111 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.831230 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.831431 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.831687 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.831782 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.831995 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.832095 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.832317 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.832406 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.832620 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.832741 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.832950 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.833036 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.833266 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.833435 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.833563 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.833651 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.833866 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.833951 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.834167 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.834264 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.834471 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.834557 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.834805 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.834905 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.835094 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.835181 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.835399 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.835484 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.835704 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.835801 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.836548 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.836725 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.837499 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.837591 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.837673 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.837757 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.837838 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.837906 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.837971 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838035 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838099 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838176 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838244 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838308 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838373 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838437 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838500 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838577 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838644 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838709 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.838947 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.839026 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.839281 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.839387 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.839575 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.839663 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.839873 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.839959 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.840172 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.840420 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.840509 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.840587 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.840855 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.840958 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.841084 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.841173 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.841382 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.841471 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.841684 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.841771 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.849737 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.849859 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.849936 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850004 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850080 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850154 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850231 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850307 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850374 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850441 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850507 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850581 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850649 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850740 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850814 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850886 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.850961 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851053 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851129 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851209 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851276 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851341 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851406 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851481 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851571 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851651 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851742 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851822 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851888 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.851962 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852029 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852095 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852167 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852167 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852303 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852371 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852442 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852516 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852582 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852673 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852759 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852829 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852902 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.852981 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853048 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853112 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853185 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853253 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853317 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853380 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853454 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853520 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853597 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853686 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853765 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853838 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853904 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.853993 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854074 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854140 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854204 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854268 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854332 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854405 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854493 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854561 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854626 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854699 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854785 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854862 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.854949 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855017 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855081 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855158 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855232 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855320 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855390 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855454 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855531 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855605 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855691 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855760 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855824 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855896 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.855962 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856034 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856108 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856176 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856240 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856312 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856378 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856443 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856592 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856723 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856803 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856870 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856935 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.856999 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.857073 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.857147 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.857213 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.857278 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.857371 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.857468 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.857678 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.857773 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.858040 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.858142 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.858297 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.858392 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.858606 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.858688 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.858907 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.858993 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.859218 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.859317 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.859523 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.859617 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.859827 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.859908 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.860126 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.860220 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.860432 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.860518 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.860761 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.860852 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.861068 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.861173 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.861373 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.861461 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.861679 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.861756 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.862005 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.862094 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.862308 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.862422 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.862626 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.862734 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.862935 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.863055 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.863235 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.863364 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.863534 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.863692 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.863836 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.863987 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.864173 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.864301 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.864479 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.864600 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.864802 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.864936 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.865119 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.865261 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.865440 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.865588 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.865747 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.865922 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.866061 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.866193 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.866368 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.866508 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.866675 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.866811 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.866989 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.867114 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.867299 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.867419 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.867605 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.867717 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.867912 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.868005 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.868218 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.868313 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.868526 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.868612 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.868854 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.868942 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.869174 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.869270 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.869482 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.869568 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.869791 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.869885 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.870091 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.870186 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.870394 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.870538 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.870688 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.870774 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.870990 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.871079 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.871305 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.871576 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.871663 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.871913 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.872221 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.880397 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.880579 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.880674 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.880770 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.880843 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.881450 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.881450 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.881604 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.881699 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.881795 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.881884 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.881964 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882050 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882130 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882208 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882286 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882364 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882452 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882533 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882610 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882695 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882778 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882846 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.882934 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883025 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883105 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883183 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883261 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883339 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883433 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883516 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883594 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883668 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883746 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883746 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883886 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.883966 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884051 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884139 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884139 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884301 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884390 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884477 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884557 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884634 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884730 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884824 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884901 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.884979 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885057 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885142 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885231 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885320 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885406 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885486 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885552 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885626 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885708 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885795 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885887 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.885968 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886046 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886046 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886271 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886368 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886449 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886523 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886591 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886656 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886728 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886796 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886861 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.886948 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887036 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887117 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887194 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887288 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887376 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887457 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887542 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887621 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887695 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887771 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887851 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887929 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.887995 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888073 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888155 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888231 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888305 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888394 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888394 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888536 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888612 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888698 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888768 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888768 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888899 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888990 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.888990 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.889129 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.889196 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.889260 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.889333 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.889408 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.889483 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.889551 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.889616 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.889680 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.889745 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.889971 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.890059 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.890277 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.890359 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.890587 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.890674 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.891025 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.891141 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.891225 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.891304 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894025 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894150 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894243 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894326 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894326 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894462 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894462 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894596 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894682 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894770 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894850 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.894936 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895023 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895112 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895201 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895281 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895366 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895451 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895530 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895616 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895697 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895774 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895841 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895911 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.895997 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.896085 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.896173 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.896252 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.896319 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.896384 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.896448 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.896532 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.896612 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.896709 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.896781 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.896945 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.897080 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.897251 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.897553 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.897639 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.897854 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.898096 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.898191 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.898440 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.898589 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.898741 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.898923 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.899060 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.899232 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.899366 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.899545 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.899668 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.899851 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.899970 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.900243 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.900334 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.900601 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.900806 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.900925 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.901900 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.902022 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.902115 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.902198 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.902277 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.902363 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.902464 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.902548 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.902628 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.902713 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.902801 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.902910 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.903068 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.903211 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.903371 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.903527 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.903697 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.903830 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.904008 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.904133 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.904320 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.904447 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.904628 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.904891 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.905042 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.905172 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.905320 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.905453 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.905585 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.905715 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.905869 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.906016 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.906148 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.906299 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.906460 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.906615 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.906760 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.906884 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.907071 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.907217 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.907381 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.907534 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.907679 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.907812 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.907978 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.908134 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.908271 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.908420 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.908552 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.908865 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.909020 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.909135 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.909247 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.909460 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.909582 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.909698 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.909839 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.910012 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.910145 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.910321 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.910445 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.910633 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.910744 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.910942 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.911044 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.911249 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.911340 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.911551 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.911638 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.911876 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.911973 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.912183 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.912271 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.912484 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.912570 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.912806 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.912917 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.913109 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.913197 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.913414 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.913496 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.913716 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.913804 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.914073 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.914157 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.914713 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.914812 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.915027 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.915124 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.915338 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.915426 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.915654 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.915740 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.915959 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.916055 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.916264 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.916351 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.916565 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.916675 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.916893 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.916976 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.917193 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.917312 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.917506 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.917633 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.917796 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.917933 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.918088 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.918213 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.918404 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.918557 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.918696 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.918826 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.919001 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.919141 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.919291 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.919441 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.919616 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.919757 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.919929 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.920064 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.920211 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.920358 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.920497 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.920647 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.920976 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.921113 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.921264 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.921399 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.921517 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.921647 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.921764 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.921897 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.922033 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.922204 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.922323 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.922553 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.922640 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.922862 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.922960 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.923173 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.923333 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.923470 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.923605 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.923749 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.923898 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.924031 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.924171 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.924379 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.924532 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.924759 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.924898 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.925068 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.925340 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.925440 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.925653 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.925771 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.925965 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.926075 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.926273 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.926415 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.926569 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.926717 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.926859 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.926988 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.927192 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.927328 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.927500 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.927652 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.927819 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.927916 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.928174 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.928270 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.928483 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.928634 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.928901 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.929044 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.929166 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.929294 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.929439 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.929563 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.929675 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.929785 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.930089 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.930171 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.930394 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.930480 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.930695 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.930784 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.930995 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.931081 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.931294 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.931380 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.931592 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.931681 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.931897 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.931985 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.932203 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.932289 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.932499 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.932585 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.932825 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.932913 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.933123 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.933215 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.933438 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.933532 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.933736 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.933823 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.934050 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.934146 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.934361 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.934455 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.934683 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.934779 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.935001 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.935097 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.935304 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.935419 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.935622 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.935709 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.935930 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.936031 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.936232 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.936318 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.936532 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.936620 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.936857 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.936946 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.937198 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.937282 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.937498 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.937586 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.937812 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.937899 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.938129 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.938216 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.938446 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.938534 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.938752 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.938841 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.939076 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.939164 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.939378 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.939465 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.939696 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.939785 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.940051 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.940144 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.940376 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.940501 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.940720 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.940851 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.941179 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.941266 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.941488 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.941594 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.941798 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.941886 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.942099 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.942185 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.942403 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.942499 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.942704 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.942790 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.943010 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.943096 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.943311 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.943406 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.943613 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.943699 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.943912 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.944007 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.944227 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.944315 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.944528 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.944614 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.944858 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.944957 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.945164 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.945253 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.945465 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.945549 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.945762 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.945847 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.946059 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.946145 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.946372 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.946458 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.946671 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.946757 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.946988 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.947076 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.947292 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.947382 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.947608 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.947707 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.947919 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.948011 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.948215 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.948300 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.948529 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.948718 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.948856 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.948956 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.949156 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.949244 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.949453 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.949596 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.949748 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.949834 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.950052 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.950138 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.950364 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.950455 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.950669 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.950754 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.950984 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.951078 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.951306 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.951552 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.951639 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.951879 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.951965 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.952191 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.952288 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.952501 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.952591 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.952841 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.952929 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.953151 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.953250 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.953456 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.953532 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.953753 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.953846 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.954052 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.954148 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.954357 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.954444 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.954656 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.954742 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.954953 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.955048 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.955249 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.955335 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.955595 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.955700 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.955969 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.956216 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.956333 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.956524 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.956595 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.956865 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.956960 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.957164 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.957260 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.957476 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.957558 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.957776 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.957871 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.958073 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.958160 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.958369 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.958456 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.958663 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.958751 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.958996 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.959084 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.959291 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.959386 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.959613 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.959718 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.959911 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.959997 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.960236 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.960353 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.960524 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.960607 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.960857 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.960968 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.962980 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.963101 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.963101 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.963264 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.963347 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.963427 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.963494 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.963576 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.963658 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.963738 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.964467 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.964554 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.964804 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.964893 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.965686 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.965770 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.965862 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.965945 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966038 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966109 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966175 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966249 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966329 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966408 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966497 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966578 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966646 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966712 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966791 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966882 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.966964 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967046 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967145 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967283 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967362 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967457 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967535 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967609 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967676 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967743 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967808 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967882 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.967971 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.968041 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.968107 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.968194 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.968297 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.968441 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.968524 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.968607 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.968833 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.968922 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.969146 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.969304 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.969455 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.969724 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.969876 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.970066 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.970216 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.970364 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.970501 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.970669 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.970798 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.970977 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.971110 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.971350 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.971494 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.971840 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.972055 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.972205 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.972346 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.972485 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.972618 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.972780 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.972911 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.972911 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.973181 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.973313 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.973449 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.973589 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.973727 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.973880 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.974015 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.974153 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.974288 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.974427 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.974577 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.974697 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.974813 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.974931 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.975044 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.975208 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.975367 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.975516 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.975649 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.975822 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.976055 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.976202 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.976350 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.976490 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.976723 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.976876 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.977029 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.977167 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.977313 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.977460 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.977586 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.977726 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.977861 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.978014 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.978148 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.978280 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.978430 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.978585 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.978726 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.978879 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.979019 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.979159 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.979310 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.979448 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.979641 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.979789 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.979936 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.980073 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.980245 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.980380 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.980539 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.980732 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.980887 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.981025 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.981167 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.981298 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.981476 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.981677 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.981836 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.982019 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.982179 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.982314 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.982420 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.982634 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.982722 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.982957 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.983050 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.983264 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.983366 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.983565 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.983838 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.983934 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.984164 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.984252 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.984461 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.984550 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.984793 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.984885 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.985104 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.985278 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.985419 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.985556 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.985694 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.985826 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.985962 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.986107 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.986256 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.986345 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.986555 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.986691 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.986907 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.987064 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.987200 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.987321 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.987516 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.987659 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.987822 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.987960 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.988115 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.988268 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.988410 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.988547 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.988718 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.988852 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.989011 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.989151 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.989308 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.989448 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.989608 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.989759 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.989901 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.990038 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.990190 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.990330 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.990487 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.990626 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.990747 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.990867 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.991031 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.991268 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.991423 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.991560 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.991689 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.991822 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.991959 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.992110 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.992246 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.992376 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.992553 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.992811 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.992958 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.993114 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.993255 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.993388 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.993525 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.993650 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.993803 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.993966 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.994107 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.994243 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.994373 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.994532 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.994687 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.994829 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.994963 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.995246 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.995405 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.995725 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.995883 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.996037 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.996179 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.996347 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.996504 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.996646 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.996836 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.996971 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.997101 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.997297 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.997440 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.997593 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.997728 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.997917 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.998067 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.998230 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.998358 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.998534 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.998637 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.998833 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.998941 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.999137 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.999228 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.999439 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.999527 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.999742 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:06.999872 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.000046 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.000144 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.000348 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.000435 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.000651 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.000859 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.000978 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.001139 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.001291 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.001431 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.001564 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.001691 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.001822 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.001950 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.002128 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.002216 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.002476 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.002832 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.003178 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.003329 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.003601 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.003691 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.003917 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.004006 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.004229 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.004317 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.004537 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.004622 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.004869 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.004956 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.005172 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.005284 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.005472 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.005567 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.005772 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.005857 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.006078 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.006216 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.006380 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.006517 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.006689 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.006857 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.006988 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.007115 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.007274 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.007576 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.007667 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.007880 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.008029 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.008177 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.008294 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.008500 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.008637 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.008852 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.008942 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.009168 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.009277 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.009496 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.009623 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.009794 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.009893 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.010115 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.010271 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.010404 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.010552 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.010725 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.010858 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.011038 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.011114 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.011361 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.011514 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.011649 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.011783 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.011961 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.012087 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.012263 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.012418 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.012566 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.012730 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.012885 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.013036 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.013176 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.013287 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.013494 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.013611 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.013790 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.013916 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.014087 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.014318 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.014448 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.014650 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.014801 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.014956 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.015113 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.015246 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.015386 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.015527 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.015792 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.016003 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.016141 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.016267 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.016406 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.016554 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.016767 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.016918 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.017058 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.017202 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.017338 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.017427 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.017541 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.017652 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.017824 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.018001 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.018214 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.018633 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.018732 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.019034 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.019169 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.019382 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.019471 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.019744 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.019859 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.020047 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.020169 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.020364 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.020454 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.020697 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.020886 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.021026 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.021112 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.021339 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.021428 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.021646 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.021749 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.021956 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.022097 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.022275 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.022408 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.022600 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.022753 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.022912 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.023055 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.023222 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.023361 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.023553 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.023698 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.023849 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.023984 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.024153 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.024297 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.024441 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.024576 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.024803 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=129097) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 09:45:07.024935 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.025086 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.025360 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.025684 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.026003 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.026304 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.026632 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.026944 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.027257 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.027570 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.027874 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.028196 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.028500 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.028828 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.029136 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.029443 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.029748 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.030061 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.030369 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.030669 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.030979 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.031288 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.031610 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.031928 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.032230 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.032553 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.032899 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.033254 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.033565 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.033969 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.034356 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.034666 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.034973 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.035273 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:45:07.035514 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=129096) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 09:45:07.091181 ncdv-master-1 devstack@n-api.service[129097]: INFO dbcounter [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Registered counter for database nova_api Sep 27 09:45:07.101648 ncdv-master-1 devstack@n-api.service[129096]: INFO dbcounter [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Registered counter for database nova_api Sep 27 09:45:07.111656 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_db.sqlalchemy.engines [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129097) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:45:07.118843 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writer thread running {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:45:07.123038 ncdv-master-1 devstack@n-api.service[129096]: DEBUG oslo_db.sqlalchemy.engines [None req-394974b4-d242-432e-9067-1ff05206f42b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129096) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:45:07.131488 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writer thread running {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:45:07.177903 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.context [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),ed1088da-caae-4426-8048-9cba90e7755c(cell1) {{(pid=129097) load_cells /opt/stack/nova/nova/context.py:464}} Sep 27 09:45:07.181905 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:45:07.182379 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:45:07.183961 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:45:07.184734 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:45:07.185160 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:45:07.187234 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:45:07.189377 ncdv-master-1 devstack@n-api.service[129097]: INFO dbcounter [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Registered counter for database nova_cell0 Sep 27 09:45:07.191177 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.context [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),ed1088da-caae-4426-8048-9cba90e7755c(cell1) {{(pid=129096) load_cells /opt/stack/nova/nova/context.py:464}} Sep 27 09:45:07.195327 ncdv-master-1 devstack@n-api.service[129096]: DEBUG oslo_concurrency.lockutils [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=129096) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:45:07.195691 ncdv-master-1 devstack@n-api.service[129097]: INFO dbcounter [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Registered counter for database nova_cell1 Sep 27 09:45:07.195817 ncdv-master-1 devstack@n-api.service[129096]: DEBUG oslo_concurrency.lockutils [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=129096) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:45:07.197462 ncdv-master-1 devstack@n-api.service[129096]: DEBUG oslo_concurrency.lockutils [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=129096) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:45:07.198302 ncdv-master-1 devstack@n-api.service[129096]: DEBUG oslo_concurrency.lockutils [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=129096) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:45:07.198748 ncdv-master-1 devstack@n-api.service[129096]: DEBUG oslo_concurrency.lockutils [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=129096) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:45:07.200814 ncdv-master-1 devstack@n-api.service[129096]: DEBUG oslo_concurrency.lockutils [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=129096) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:45:07.204596 ncdv-master-1 devstack@n-api.service[129096]: INFO dbcounter [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Registered counter for database nova_cell0 Sep 27 09:45:07.204809 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_db.sqlalchemy.engines [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129097) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:45:07.205974 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_db.sqlalchemy.engines [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129097) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:45:07.208105 ncdv-master-1 devstack@n-api.service[129096]: INFO dbcounter [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Registered counter for database nova_cell1 Sep 27 09:45:07.212513 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writer thread running {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:45:07.213811 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writer thread running {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:45:07.216625 ncdv-master-1 devstack@n-api.service[129096]: DEBUG oslo_db.sqlalchemy.engines [None req-394974b4-d242-432e-9067-1ff05206f42b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129096) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:45:07.219033 ncdv-master-1 devstack@n-api.service[129096]: DEBUG oslo_db.sqlalchemy.engines [None req-394974b4-d242-432e-9067-1ff05206f42b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129096) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:45:07.224598 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writer thread running {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:45:07.362269 ncdv-master-1 devstack@n-api.service[129097]: INFO dbcounter [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Registered counter for database nova_cell0 Sep 27 09:45:07.367452 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writer thread running {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:45:07.369990 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_db.sqlalchemy.engines [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129097) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:45:07.375208 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writer thread running {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:45:07.383167 ncdv-master-1 devstack@n-api.service[129096]: INFO dbcounter [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Registered counter for database nova_cell0 Sep 27 09:45:07.394312 ncdv-master-1 devstack@n-api.service[129096]: DEBUG oslo_db.sqlalchemy.engines [None req-394974b4-d242-432e-9067-1ff05206f42b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=129096) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:45:07.400384 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writer thread running {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:45:07.844970 ncdv-master-1 devstack@n-api.service[129097]: WARNING keystonemiddleware.auth_token [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 09:45:07.854268 ncdv-master-1 devstack@n-api.service[129097]: WARNING keystonemiddleware.auth_token [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 09:45:07.869855 ncdv-master-1 devstack@n-api.service[129096]: WARNING keystonemiddleware.auth_token [None req-394974b4-d242-432e-9067-1ff05206f42b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 09:45:07.877388 ncdv-master-1 devstack@n-api.service[129096]: WARNING keystonemiddleware.auth_token [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 09:45:08.077890 ncdv-master-1 devstack@n-api.service[129097]: WARNING keystonemiddleware.auth_token [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 09:45:08.084219 ncdv-master-1 devstack@n-api.service[129097]: WARNING keystonemiddleware.auth_token [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 09:45:08.089529 ncdv-master-1 devstack@n-api.service[129097]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55bd9d3c59c0 pid: 129097 (default app) Sep 27 09:45:08.091607 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:45:08.092815 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] 10.222.0.26 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001742 Sep 27 09:45:08.093226 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 788 bytes} [Tue Sep 27 09:45:08 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:45:08.099109 ncdv-master-1 devstack@n-api.service[129096]: WARNING keystonemiddleware.auth_token [None req-394974b4-d242-432e-9067-1ff05206f42b None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 09:45:08.105964 ncdv-master-1 devstack@n-api.service[129096]: WARNING keystonemiddleware.auth_token [None req-394974b4-d242-432e-9067-1ff05206f42b None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 09:45:08.110456 ncdv-master-1 devstack@n-api.service[129096]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55bd9d3c59c0 pid: 129096 (default app) Sep 27 09:46:40.259835 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:46:40.260728 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001462 Sep 27 09:46:40.261306 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 2/2] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:46:40 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:46:40.274084 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writing DB stats nova_api:SELECT=3 {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:40.277094 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writing DB stats nova_cell0:SELECT=3 {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:40.278176 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writing DB stats nova_cell1:SELECT=3 {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:40.279527 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writing DB stats nova_cell0:SELECT=3 {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:41.215303 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-83ecc62b-7f97-47c2-a39a-dcd153c3cf23 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:46:41.250174 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-83ecc62b-7f97-47c2-a39a-dcd153c3cf23 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.981426 Sep 27 09:46:41.250944 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 1/3] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 09:46:40 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 984 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:46:44.233475 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:46:44.235059 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-ab99ab13-2e91-4cbb-ae08-bbb544323da8 None None] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002156 Sep 27 09:46:44.235683 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 3/4] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:46:44 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:46:44.684383 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-19bdd6ab-6e04-43db-b300-ed4cdf438db0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "vcpus": 1, "ram": 256, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "id": "c1", "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "cirros256"}} {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:46:44.736416 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-19bdd6ab-6e04-43db-b300-ed4cdf438db0 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.491965 Sep 27 09:46:44.737035 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 2/5] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 09:46:44 2022] POST /compute/v2.1/flavors => generated 407 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:46:44.751166 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writing DB stats nova_api:SELECT=3 {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:44.752434 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writing DB stats nova_cell1:SELECT=3 {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:44.753670 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writing DB stats nova_cell0:SELECT=3 {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:44.754525 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:46:44.960672 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-309732bd-e5b3-4267-b82d-f59fa86342b8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:46:45.046283 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-309732bd-e5b3-4267-b82d-f59fa86342b8 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.300420 Sep 27 09:46:45.046858 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 4/6] 10.222.0.26 () {64 vars in 1336 bytes} [Tue Sep 27 09:46:44 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:46:48.085911 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-19bdd6ab-6e04-43db-b300-ed4cdf438db0 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:46:48.087528 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-19bdd6ab-6e04-43db-b300-ed4cdf438db0 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002077 Sep 27 09:46:48.088105 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 3/7] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:46:48 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:46:49.076950 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-a78ec9ce-7c99-4592-b0ba-4b87c711b5da admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds512M", "disk": 5, "swap": 0, "ram": 512, "vcpus": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d1", "os-flavor-access:is_public": true, "rxtx_factor": 1.0}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:46:49.123113 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-a78ec9ce-7c99-4592-b0ba-4b87c711b5da admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 1.028030 Sep 27 09:46:49.137907 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-1f29a36f-1031-4c59-be32-4ec391ce04e4 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:46:49.143949 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 5/8] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 09:46:48 2022] POST /compute/v2.1/flavors => generated 404 bytes in 1049 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:46:49.208011 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-1f29a36f-1031-4c59-be32-4ec391ce04e4 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.074539 Sep 27 09:46:49.230384 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 4/9] 10.222.0.26 () {64 vars in 1336 bytes} [Tue Sep 27 09:46:49 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:46:53.063363 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-a78ec9ce-7c99-4592-b0ba-4b87c711b5da admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:46:53.064548 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-a78ec9ce-7c99-4592-b0ba-4b87c711b5da admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001631 Sep 27 09:46:53.065483 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 6/10] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:46:53 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:46:53.566201 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-b37f5819-b077-49ca-9431-ca988376fc30 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "ram": 1024, "os-flavor-access:is_public": true, "name": "ds1G", "id": "d2", "disk": 10, "swap": 0, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:46:53.590685 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-b37f5819-b077-49ca-9431-ca988376fc30 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.518370 Sep 27 09:46:53.598901 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 5/11] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 09:46:53 2022] POST /compute/v2.1/flavors => generated 404 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:46:53.604379 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-095b9ebd-f9cf-4182-9f3b-184e1bdc2b35 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:46:53.637740 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-095b9ebd-f9cf-4182-9f3b-184e1bdc2b35 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.036891 Sep 27 09:46:53.640714 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 7/12] 10.222.0.26 () {64 vars in 1336 bytes} [Tue Sep 27 09:46:53 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:46:56.865814 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-b37f5819-b077-49ca-9431-ca988376fc30 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:46:56.866828 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-b37f5819-b077-49ca-9431-ca988376fc30 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001485 Sep 27 09:46:56.867224 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 6/13] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:46:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:46:57.331107 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-b132807c-7d74-4606-b222-9c8c5d5af8b7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "vcpus": 2, "os-flavor-access:is_public": true, "disk": 10, "rxtx_factor": 1.0, "name": "ds2G", "ram": 2048, "id": "d3", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:46:57.352521 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-b132807c-7d74-4606-b222-9c8c5d5af8b7 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.478193 Sep 27 09:46:57.353225 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 8/14] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 09:46:56 2022] POST /compute/v2.1/flavors => generated 404 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:46:57.365587 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-68e1d1e4-be08-4d89-b185-b8522be2f608 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:46:57.397524 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-68e1d1e4-be08-4d89-b185-b8522be2f608 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035891 Sep 27 09:46:57.398133 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 7/15] 10.222.0.26 () {64 vars in 1336 bytes} [Tue Sep 27 09:46:57 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:47:00.564517 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-b132807c-7d74-4606-b222-9c8c5d5af8b7 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:00.565392 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-b132807c-7d74-4606-b222-9c8c5d5af8b7 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001592 Sep 27 09:47:00.565750 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 9/16] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:47:00 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:01.039543 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-f7c1db72-cc49-4b47-a1c5-2f18e5cb3b96 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "name": "ds4G", "vcpus": 4, "ram": 4096, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d4"}} {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:01.063514 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-f7c1db72-cc49-4b47-a1c5-2f18e5cb3b96 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.490693 Sep 27 09:47:01.072586 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 8/17] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 09:47:00 2022] POST /compute/v2.1/flavors => generated 404 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:47:01.078610 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-58340e65-2e9c-437a-8390-0a1d5e75f5f5 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:01.108938 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-58340e65-2e9c-437a-8390-0a1d5e75f5f5 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035003 Sep 27 09:47:01.109877 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 10/18] 10.222.0.26 () {64 vars in 1336 bytes} [Tue Sep 27 09:47:01 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:47:04.202432 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-f7c1db72-cc49-4b47-a1c5-2f18e5cb3b96 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:04.203466 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-f7c1db72-cc49-4b47-a1c5-2f18e5cb3b96 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 Sep 27 09:47:04.203863 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 9/19] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:47:04 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:04.743993 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-1adfd62a-9ad2-47ee-a020-72885b9aba6d admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:04.777899 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-1adfd62a-9ad2-47ee-a020-72885b9aba6d admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.566583 Sep 27 09:47:04.778596 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 11/20] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 09:47:04 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 09:47:08.374284 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-f7c1db72-cc49-4b47-a1c5-2f18e5cb3b96 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:08.375319 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-f7c1db72-cc49-4b47-a1c5-2f18e5cb3b96 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001520 Sep 27 09:47:08.375986 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 10/21] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:47:08 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:08.886758 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-b29b9523-eec8-4fef-8fa5-3686f1166a2c admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "vcpus": 1, "ram": 512, "disk": 1, "id": "1", "rxtx_factor": 1.0, "name": "m1.tiny", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:08.907585 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-b29b9523-eec8-4fef-8fa5-3686f1166a2c admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.523793 Sep 27 09:47:08.908628 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 12/22] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 09:47:08 2022] POST /compute/v2.1/flavors => generated 402 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:47:08.921311 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-afc4bc44-b55b-4e31-9d23-15b5403c5aa0 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:08.950945 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-afc4bc44-b55b-4e31-9d23-15b5403c5aa0 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033423 Sep 27 09:47:08.951532 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 11/23] 10.222.0.26 () {64 vars in 1333 bytes} [Tue Sep 27 09:47:08 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:47:11.943230 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-b29b9523-eec8-4fef-8fa5-3686f1166a2c admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:11.944396 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-b29b9523-eec8-4fef-8fa5-3686f1166a2c admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001575 Sep 27 09:47:11.944857 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:47:11 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:12.368220 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-4b86b298-28d6-4f73-90d8-4243e27ce958 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "id": "2", "disk": 20, "rxtx_factor": 1.0, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "name": "m1.small", "vcpus": 1}} {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:12.374071 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=6,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:12.390840 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-4b86b298-28d6-4f73-90d8-4243e27ce958 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.438426 Sep 27 09:47:12.391441 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 12/25] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 09:47:11 2022] POST /compute/v2.1/flavors => generated 405 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:47:12.404146 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-57855937-f822-44f1-b6e0-ca3334a4a1be admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:12.431629 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-57855937-f822-44f1-b6e0-ca3334a4a1be admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031437 Sep 27 09:47:12.432189 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 14/26] 10.222.0.26 () {64 vars in 1333 bytes} [Tue Sep 27 09:47:12 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:47:15.552299 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-4b86b298-28d6-4f73-90d8-4243e27ce958 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:15.554157 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-4b86b298-28d6-4f73-90d8-4243e27ce958 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002254 Sep 27 09:47:15.554543 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 13/27] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:47:15 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:16.129781 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-9b42468b-c1be-478c-9c09-99fc5fcd26c9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "name": "m1.medium", "rxtx_factor": 1.0, "disk": 40, "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 4096, "os-flavor-access:is_public": true, "id": "3"}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:16.140827 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writing DB stats nova_api:SELECT=36,nova_api:SAVEPOINT=4,nova_api:INSERT=7,nova_api:RELEASE=4 {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:16.168237 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-9b42468b-c1be-478c-9c09-99fc5fcd26c9 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.595294 Sep 27 09:47:16.172761 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 15/28] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 09:47:15 2022] POST /compute/v2.1/flavors => generated 406 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:47:16.184465 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-9bd76346-63b9-4bd5-873c-b778e697454a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:16.219306 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-9bd76346-63b9-4bd5-873c-b778e697454a admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.037876 Sep 27 09:47:16.225214 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 14/29] 10.222.0.26 () {64 vars in 1333 bytes} [Tue Sep 27 09:47:16 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:47:19.312307 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-9b42468b-c1be-478c-9c09-99fc5fcd26c9 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:19.313339 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-9b42468b-c1be-478c-9c09-99fc5fcd26c9 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001517 Sep 27 09:47:19.314783 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 16/30] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:47:19 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:19.895699 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e64582d2-b18a-482f-98e1-c709fd5cda08 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "name": "m1.large", "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "id": "4", "disk": 80, "vcpus": 4, "swap": 0, "ram": 8192}} {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:19.919960 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-e64582d2-b18a-482f-98e1-c709fd5cda08 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.598742 Sep 27 09:47:19.920516 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 15/31] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 09:47:19 2022] POST /compute/v2.1/flavors => generated 405 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:47:19.933113 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-bc133bd0-37eb-4650-b44f-8843ccc8d2d2 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:19.963488 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-bc133bd0-37eb-4650-b44f-8843ccc8d2d2 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.034111 Sep 27 09:47:19.964188 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 17/32] 10.222.0.26 () {64 vars in 1333 bytes} [Tue Sep 27 09:47:19 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:47:23.023789 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e64582d2-b18a-482f-98e1-c709fd5cda08 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:23.024715 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-e64582d2-b18a-482f-98e1-c709fd5cda08 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001493 Sep 27 09:47:23.025105 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 16/33] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:47:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:23.496605 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-a3fff570-461f-460d-b19f-db646ff16c4e admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 160, "rxtx_factor": 1.0, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 16384, "vcpus": 8, "name": "m1.xlarge", "os-flavor-access:is_public": true, "id": "5"}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:23.522260 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-a3fff570-461f-460d-b19f-db646ff16c4e admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.490500 Sep 27 09:47:23.522808 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 18/34] 10.222.0.26 () {64 vars in 1283 bytes} [Tue Sep 27 09:47:23 2022] POST /compute/v2.1/flavors => generated 408 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:47:23.538439 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-4ef3dfce-1b1a-4fbf-8f20-238680026398 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:23.574164 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-4ef3dfce-1b1a-4fbf-8f20-238680026398 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039397 Sep 27 09:47:23.575016 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 17/35] 10.222.0.26 () {64 vars in 1333 bytes} [Tue Sep 27 09:47:23 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:47:41.565135 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-a3fff570-461f-460d-b19f-db646ff16c4e admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:41.566182 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-a3fff570-461f-460d-b19f-db646ff16c4e admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001636 Sep 27 09:47:41.566576 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 19/36] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:47:41 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:41.577698 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writing DB stats nova_api:INSERT=4,nova_api:SELECT=17,nova_api:SAVEPOINT=2,nova_api:RELEASE=2 {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:41.979482 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-d1512f8a-d049-4b33-94a1-e2725ce0f3be admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:41.998299 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-d1512f8a-d049-4b33-94a1-e2725ce0f3be admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.425138 Sep 27 09:47:41.998829 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 18/37] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 09:47:41 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:47:43.909234 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-a3fff570-461f-460d-b19f-db646ff16c4e admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:43.910254 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-a3fff570-461f-460d-b19f-db646ff16c4e admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001510 Sep 27 09:47:43.910647 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 20/38] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 09:47:43 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:44.545208 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-704e1558-b1be-4eee-86f6-de704c6aaee0 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:44.550882 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.wsgi [None req-704e1558-b1be-4eee-86f6-de704c6aaee0 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-704e1558-b1be-4eee-86f6-de704c6aaee0 admin admin] Request method failure captured: Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: Accept: application/json Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: Accept-Encoding: gzip, deflate Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: Connection: keep-alive Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: Content-Length: 0 Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: Host: 10.222.0.26 Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: Openstack-Api-Version: compute 2.92 Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: Openstack-System-Scope: None Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: User-Agent: python-novaclient Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Auth-Token: gAAAAABjMsa__IBccgDd6LYMzXrttfnfEMe7yRG1z3qWT7Tvrvhy4R6sfU1t6rsxD7jQVwDbJS57zFp7yRX9CKEcAJKJE_ENd72vqm7UEP-JmnwLFEzfCBfm9fyerZxDhhEAOTYTbqvIqmMcW15i9BR060ji5NwBSrYThddx7EYgy9LjrBLuqpU Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Domain-Id: None Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Domain-Name: None Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Identity-Status: Confirmed Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Is-Admin-Project: True Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Domain-Id: default Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Domain-Name: Default Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Name: admin Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Role: reader,admin,member Sep 27 09:47:44.571367 ncdv-master-1 devstack@n-api.service[129096]: X-Roles: reader,admin,member Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7d515506957d41b7867232158c5ad868"}]}] Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: X-Tenant: admin Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: X-Tenant-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: X-Tenant-Name: admin Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: X-User: admin Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: X-User-Domain-Id: default Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: X-User-Domain-Name: Default Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: X-User-Id: 7b007883f8f945d6b63ec8a2752bb899 Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: X-User-Name: admin Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: method: > Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: exception: Flavor 42 could not be found. Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: action_args: {'id': '42'} Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:47:44.572898 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:44.573826 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:44.575195 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-704e1558-b1be-4eee-86f6-de704c6aaee0 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=129096) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:47:44.576543 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-704e1558-b1be-4eee-86f6-de704c6aaee0 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.437881 Sep 27 09:47:44.576543 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 19/39] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 09:47:44 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 439 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:44.584190 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-1c75b213-b233-4bab-b2da-795b847e37a0 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:44.588636 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writing DB stats nova_api:INSERT=3,nova_api:SELECT=11,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:47:44.609587 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-1c75b213-b233-4bab-b2da-795b847e37a0 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.029957 Sep 27 09:47:44.610168 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 21/40] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 09:47:44 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:47:44.622058 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-c9a3937e-b71a-4a49-b272-72198b43f6be admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:44.647133 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-c9a3937e-b71a-4a49-b272-72198b43f6be admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.029961 Sep 27 09:47:44.647750 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 20/41] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 09:47:44 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:47:44.658460 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-ee7dfcaa-e4d3-4a95-84d4-3172eb544167 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:44.664944 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.wsgi [None req-ee7dfcaa-e4d3-4a95-84d4-3172eb544167 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-ee7dfcaa-e4d3-4a95-84d4-3172eb544167 admin admin] Request method failure captured: Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: Accept: application/json Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: Accept-Encoding: gzip, deflate Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: Connection: keep-alive Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: Content-Length: 0 Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: Host: 10.222.0.26 Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: Openstack-Api-Version: compute 2.92 Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: Openstack-System-Scope: None Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: User-Agent: python-novaclient Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Auth-Token: gAAAAABjMsa__IBccgDd6LYMzXrttfnfEMe7yRG1z3qWT7Tvrvhy4R6sfU1t6rsxD7jQVwDbJS57zFp7yRX9CKEcAJKJE_ENd72vqm7UEP-JmnwLFEzfCBfm9fyerZxDhhEAOTYTbqvIqmMcW15i9BR060ji5NwBSrYThddx7EYgy9LjrBLuqpU Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Domain-Id: None Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Domain-Name: None Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Identity-Status: Confirmed Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Is-Admin-Project: True Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Domain-Id: default Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Domain-Name: Default Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Name: admin Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Role: reader,admin,member Sep 27 09:47:44.668645 ncdv-master-1 devstack@n-api.service[129097]: X-Roles: reader,admin,member Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7d515506957d41b7867232158c5ad868"}]}] Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: X-Tenant: admin Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: X-Tenant-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: X-Tenant-Name: admin Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: X-User: admin Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: X-User-Domain-Id: default Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: X-User-Domain-Name: Default Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: X-User-Id: 7b007883f8f945d6b63ec8a2752bb899 Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: X-User-Name: admin Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: method: > Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: exception: Flavor 42 could not be found. Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: action_args: {'id': '42'} Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:47:44.670023 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:44.670896 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:44.672177 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-ee7dfcaa-e4d3-4a95-84d4-3172eb544167 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=129097) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:47:44.673557 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-ee7dfcaa-e4d3-4a95-84d4-3172eb544167 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.015843 Sep 27 09:47:44.673557 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 22/42] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 09:47:44 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:46.514400 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-c9a3937e-b71a-4a49-b272-72198b43f6be admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:46.515416 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-c9a3937e-b71a-4a49-b272-72198b43f6be admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001478 Sep 27 09:47:46.516179 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 21/43] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 09:47:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:47.084385 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-5563da41-9e31-40d4-83e4-9c8e01e1c134 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:47.090221 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.wsgi [None req-5563da41-9e31-40d4-83e4-9c8e01e1c134 admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-5563da41-9e31-40d4-83e4-9c8e01e1c134 admin admin] Request method failure captured: Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: Accept: application/json Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: Accept-Encoding: gzip, deflate Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: Connection: keep-alive Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: Content-Length: 0 Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: Host: 10.222.0.26 Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: Openstack-Api-Version: compute 2.92 Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: Openstack-System-Scope: None Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: User-Agent: python-novaclient Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Auth-Token: gAAAAABjMsbCBFyFNLTAigT5zJpshWyelSpGPb0v9F1c4-fHzDae2-em2re3JkRnNg-9b_Zqtmeb0F7Zu7DcmGkLuMRhhTPz7jEyFrDwTpWT6UJXXp1eETgY3-_Jcqq_ZwF-tYoP-oMYqsUJh7Myyi_5JF4aPu_sX1n-43QYBQUQjGPz80C25H8 Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Domain-Id: None Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Domain-Name: None Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Identity-Status: Confirmed Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Is-Admin-Project: True Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Domain-Id: default Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Domain-Name: Default Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Name: admin Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Role: reader,admin,member Sep 27 09:47:47.091808 ncdv-master-1 devstack@n-api.service[129097]: X-Roles: reader,admin,member Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7d515506957d41b7867232158c5ad868"}]}] Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: X-Tenant: admin Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: X-Tenant-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: X-Tenant-Name: admin Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: X-User: admin Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: X-User-Domain-Id: default Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: X-User-Domain-Name: Default Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: X-User-Id: 7b007883f8f945d6b63ec8a2752bb899 Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: X-User-Name: admin Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: method: > Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: exception: Flavor 84 could not be found. Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: action_args: {'id': '84'} Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:47:47.093495 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:47.094472 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:47.096389 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-5563da41-9e31-40d4-83e4-9c8e01e1c134 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=129097) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:47:47.097693 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-5563da41-9e31-40d4-83e4-9c8e01e1c134 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.428084 Sep 27 09:47:47.097693 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 23/44] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 09:47:46 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 430 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:47.104308 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-afdf4fa5-159f-4780-9993-51d592d3b160 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:47.124113 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-afdf4fa5-159f-4780-9993-51d592d3b160 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.023394 Sep 27 09:47:47.124743 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 22/45] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 09:47:47 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 25 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:47:47.139070 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-222b1a37-fbd3-4648-91ad-3582be246586 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:47.163974 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-222b1a37-fbd3-4648-91ad-3582be246586 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.030918 Sep 27 09:47:47.164523 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 24/46] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 09:47:47 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:47:47.174951 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e9fcacaf-dfea-44d9-866b-aa4d5f7472db admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:47.180253 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.wsgi [None req-e9fcacaf-dfea-44d9-866b-aa4d5f7472db admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e9fcacaf-dfea-44d9-866b-aa4d5f7472db admin admin] Request method failure captured: Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: Accept: application/json Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: Accept-Encoding: gzip, deflate Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: Connection: keep-alive Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: Content-Length: 0 Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: Host: 10.222.0.26 Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: Openstack-Api-Version: compute 2.92 Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: Openstack-System-Scope: None Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: User-Agent: python-novaclient Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Auth-Token: gAAAAABjMsbCBFyFNLTAigT5zJpshWyelSpGPb0v9F1c4-fHzDae2-em2re3JkRnNg-9b_Zqtmeb0F7Zu7DcmGkLuMRhhTPz7jEyFrDwTpWT6UJXXp1eETgY3-_Jcqq_ZwF-tYoP-oMYqsUJh7Myyi_5JF4aPu_sX1n-43QYBQUQjGPz80C25H8 Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Domain-Id: None Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Domain-Name: None Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Identity-Status: Confirmed Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Is-Admin-Project: True Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Domain-Id: default Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Domain-Name: Default Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Name: admin Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Role: reader,admin,member Sep 27 09:47:47.181819 ncdv-master-1 devstack@n-api.service[129096]: X-Roles: reader,admin,member Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7d515506957d41b7867232158c5ad868"}]}] Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: X-Tenant: admin Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: X-Tenant-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: X-Tenant-Name: admin Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: X-User: admin Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: X-User-Domain-Id: default Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: X-User-Domain-Name: Default Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: X-User-Id: 7b007883f8f945d6b63ec8a2752bb899 Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: X-User-Name: admin Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: method: > Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: exception: Flavor 84 could not be found. Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: action_args: {'id': '84'} Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:47:47.183223 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:47.184024 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:47.185301 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e9fcacaf-dfea-44d9-866b-aa4d5f7472db admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=129096) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:47:47.186590 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-e9fcacaf-dfea-44d9-866b-aa4d5f7472db admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.012009 Sep 27 09:47:47.186590 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 23/47] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 09:47:47 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:49.034128 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-222b1a37-fbd3-4648-91ad-3582be246586 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:49.035162 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-222b1a37-fbd3-4648-91ad-3582be246586 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002143 Sep 27 09:47:49.035560 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 25/48] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 09:47:49 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:49.744874 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-952e90b7-eba4-4aac-b683-6f538f5fd1a5 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:49.751567 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.wsgi [None req-952e90b7-eba4-4aac-b683-6f538f5fd1a5 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-952e90b7-eba4-4aac-b683-6f538f5fd1a5 admin admin] Request method failure captured: Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: Accept: application/json Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: Accept-Encoding: gzip, deflate Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: Connection: keep-alive Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: Content-Length: 0 Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: Host: 10.222.0.26 Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: Openstack-Api-Version: compute 2.92 Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: Openstack-System-Scope: None Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: User-Agent: python-novaclient Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Auth-Token: gAAAAABjMsbEs1ewhMGyCMx8vPGmYckWOuz_rppCGQO60nXw5VvPO0VEGWjEMot-nn9NE7OOXNmwORPoA8MsOsubjP1wmqoSsvZSKbOn4ySi4_EIQDJZ9-RI3V6VKqPW4S45xVx5S9aDj39MVTjyUhySZcYrJ8siPRcahk551e6gYeUHcFxjJyk Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Domain-Id: None Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Domain-Name: None Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Identity-Status: Confirmed Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Is-Admin-Project: True Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Domain-Id: default Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Domain-Name: Default Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Project-Name: admin Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Role: reader,admin,member Sep 27 09:47:49.753568 ncdv-master-1 devstack@n-api.service[129096]: X-Roles: reader,admin,member Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7d515506957d41b7867232158c5ad868"}]}] Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: X-Tenant: admin Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: X-Tenant-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: X-Tenant-Name: admin Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: X-User: admin Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: X-User-Domain-Id: default Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: X-User-Domain-Name: Default Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: X-User-Id: 7b007883f8f945d6b63ec8a2752bb899 Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: X-User-Name: admin Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: method: > Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: exception: Flavor 451 could not be found. Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: action_args: {'id': '451'} Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:47:49.755186 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:49.756060 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: ERROR nova.api.openstack.wsgi Sep 27 09:47:49.757153 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-952e90b7-eba4-4aac-b683-6f538f5fd1a5 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=129096) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:47:49.758216 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-952e90b7-eba4-4aac-b683-6f538f5fd1a5 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.567498 Sep 27 09:47:49.758216 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 24/49] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 09:47:49 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 569 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 27 09:47:49.764993 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-0b85e48a-4861-4678-b5ea-8abb74eb7a73 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:49.781145 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-0b85e48a-4861-4678-b5ea-8abb74eb7a73 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019665 Sep 27 09:47:49.781685 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 26/50] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 09:47:49 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:47:49.792688 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:49.808816 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.020130 Sep 27 09:47:49.809378 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 25/51] 10.222.0.26 () {60 vars in 1210 bytes} [Tue Sep 27 09:47:49 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 09:47:49.819794 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-8a92e2f4-3ffc-48b6-a494-8b691d1b975f admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:49.824987 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.wsgi [None req-8a92e2f4-3ffc-48b6-a494-8b691d1b975f admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-8a92e2f4-3ffc-48b6-a494-8b691d1b975f admin admin] Request method failure captured: Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: Accept: application/json Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: Accept-Encoding: gzip, deflate Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: Connection: keep-alive Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: Content-Length: 0 Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: Host: 10.222.0.26 Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: Openstack-Api-Version: compute 2.92 Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: Openstack-System-Scope: None Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: User-Agent: python-novaclient Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Auth-Token: gAAAAABjMsbEs1ewhMGyCMx8vPGmYckWOuz_rppCGQO60nXw5VvPO0VEGWjEMot-nn9NE7OOXNmwORPoA8MsOsubjP1wmqoSsvZSKbOn4ySi4_EIQDJZ9-RI3V6VKqPW4S45xVx5S9aDj39MVTjyUhySZcYrJ8siPRcahk551e6gYeUHcFxjJyk Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Domain-Id: None Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Domain-Name: None Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Identity-Status: Confirmed Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Is-Admin-Project: True Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Openstack-Nova-Api-Version: 2.92 Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Domain-Id: default Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Domain-Name: Default Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Project-Name: admin Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Role: reader,admin,member Sep 27 09:47:49.826510 ncdv-master-1 devstack@n-api.service[129097]: X-Roles: reader,admin,member Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7d515506957d41b7867232158c5ad868"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7d515506957d41b7867232158c5ad868"}]}] Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: X-Tenant: admin Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: X-Tenant-Id: 7d515506957d41b7867232158c5ad868 Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: X-Tenant-Name: admin Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: X-User: admin Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: X-User-Domain-Id: default Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: X-User-Domain-Name: Default Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: X-User-Id: 7b007883f8f945d6b63ec8a2752bb899 Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: X-User-Name: admin Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: method: > Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: exception: Flavor 451 could not be found. Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: action_args: {'id': '451'} Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:47:49.827902 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:49.828958 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: ERROR nova.api.openstack.wsgi Sep 27 09:47:49.830317 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-8a92e2f4-3ffc-48b6-a494-8b691d1b975f admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=129097) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:47:49.831533 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-8a92e2f4-3ffc-48b6-a494-8b691d1b975f admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011811 Sep 27 09:47:49.831533 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 27/52] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 09:47:49 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Sep 27 09:47:51.659959 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:51.660975 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 Sep 27 09:47:51.661383 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 26/53] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 09:47:51 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:52.219307 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-87eb910b-7816-45fe-9d0d-861d1bf46725 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:52.241566 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-87eb910b-7816-45fe-9d0d-861d1bf46725 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.430286 Sep 27 09:47:52.242115 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 28/54] 10.222.0.26 () {64 vars in 1235 bytes} [Tue Sep 27 09:47:51 2022] POST /compute/v2.1/flavors => generated 404 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:47:54.105543 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:54.106557 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001496 Sep 27 09:47:54.106966 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 27/55] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 09:47:54 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:54.674183 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-baa9d2d0-21f1-4914-beaa-1d147a491947 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:54.698795 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-baa9d2d0-21f1-4914-beaa-1d147a491947 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.444843 Sep 27 09:47:54.699354 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 29/56] 10.222.0.26 () {64 vars in 1235 bytes} [Tue Sep 27 09:47:54 2022] POST /compute/v2.1/flavors => generated 405 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:47:56.513998 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:56.515002 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001462 Sep 27 09:47:56.515418 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 28/57] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 09:47:56 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:57.106219 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-abcdeac0-3a0c-474a-b856-60c4bdc661c3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:57.127959 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-abcdeac0-3a0c-474a-b856-60c4bdc661c3 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.463706 Sep 27 09:47:57.128596 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 30/58] 10.222.0.26 () {64 vars in 1235 bytes} [Tue Sep 27 09:47:56 2022] POST /compute/v2.1/flavors => generated 407 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:47:58.957576 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:47:58.958574 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 Sep 27 09:47:58.958965 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 29/59] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 09:47:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:47:59.623116 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-154604ec-e9d4-489f-a43c-2881069563ed admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:47:59.647054 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-154604ec-e9d4-489f-a43c-2881069563ed admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.527356 Sep 27 09:47:59.647867 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 31/60] 10.222.0.26 () {64 vars in 1235 bytes} [Tue Sep 27 09:47:59 2022] POST /compute/v2.1/flavors => generated 409 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:48:01.506972 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:48:01.508013 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001545 Sep 27 09:48:01.508605 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 30/61] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 09:48:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:48:02.101657 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-14103fbf-feef-40a6-83ee-97ed34b51e7e admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:48:02.124284 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-14103fbf-feef-40a6-83ee-97ed34b51e7e admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.465276 Sep 27 09:48:02.124766 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 32/62] 10.222.0.26 () {60 vars in 1202 bytes} [Tue Sep 27 09:48:01 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:49:09.555979 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:49:09.556672 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-e60e47b7-1a81-4b64-a557-54aac036987b admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001485 Sep 27 09:49:09.557023 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 31/63] 10.222.0.26 () {54 vars in 931 bytes} [Tue Sep 27 09:49:09 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:49:09.567075 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:09.992597 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-34f69f70-7504-4382-9d01-09467d626630 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:49:10.013129 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-34f69f70-7504-4382-9d01-09467d626630 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.448354 Sep 27 09:49:10.013657 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 33/64] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 09:49:09 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:49:10.036194 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writing DB stats nova_api:SELECT=14 {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:49:10.039993 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-8b7647a8-5a00-4d7f-adb3-b9d803c52c0b admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:49:10.048339 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-8b7647a8-5a00-4d7f-adb3-b9d803c52c0b admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013440 Sep 27 09:49:10.049031 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 32/65] 10.222.0.26 () {60 vars in 1283 bytes} [Tue Sep 27 09:49:10 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:49:10.061145 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-d5b3413c-ef0f-44ca-aefd-508a76892b3f admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:49:10.069124 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-d5b3413c-ef0f-44ca-aefd-508a76892b3f admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011614 Sep 27 09:49:10.069634 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 34/66] 10.222.0.26 () {60 vars in 1286 bytes} [Tue Sep 27 09:49:10 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:49:10.081674 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-39e6d360-d838-42c9-94a5-956e3eacb103 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:49:10.089244 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-39e6d360-d838-42c9-94a5-956e3eacb103 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011144 Sep 27 09:49:10.089569 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 33/67] 10.222.0.26 () {60 vars in 1286 bytes} [Tue Sep 27 09:49:10 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:49:10.101426 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-ceac55fa-db80-4c8c-92cc-6c7df4322ba6 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:49:10.109001 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-ceac55fa-db80-4c8c-92cc-6c7df4322ba6 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011122 Sep 27 09:49:10.109509 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 35/68] 10.222.0.26 () {60 vars in 1283 bytes} [Tue Sep 27 09:49:10 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:54:45.590069 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-39e6d360-d838-42c9-94a5-956e3eacb103 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:54:45.591296 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-39e6d360-d838-42c9-94a5-956e3eacb103 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001782 Sep 27 09:54:45.591773 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 34/69] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 09:54:45 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:54:45.744286 ncdv-master-1 devstack@n-api.service[129097]: DEBUG dbcounter [-] [129097] Writing DB stats nova_api:SELECT=6 {{(pid=129097) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:54:45.911361 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-f10d904a-fc73-4d25-abec-4b6eb244ecb6 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:54:45.916974 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-f10d904a-fc73-4d25-abec-4b6eb244ecb6 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:54:45.917444 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-f10d904a-fc73-4d25-abec-4b6eb244ecb6 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:54:45.917881 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-f10d904a-fc73-4d25-abec-4b6eb244ecb6 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:54:45.918588 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-f10d904a-fc73-4d25-abec-4b6eb244ecb6 admin admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:54:45.919024 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-f10d904a-fc73-4d25-abec-4b6eb244ecb6 admin admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:54:45.919428 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-f10d904a-fc73-4d25-abec-4b6eb244ecb6 admin admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:54:45.959303 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-f10d904a-fc73-4d25-abec-4b6eb244ecb6 admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.92 time: 0.216423 Sep 27 09:54:45.959987 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 36/70] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 09:54:45 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:54:47.630313 ncdv-master-1 devstack@n-api.service[129096]: DEBUG nova.api.openstack.wsgi [None req-39e6d360-d838-42c9-94a5-956e3eacb103 admin admin] Calling method '>' {{(pid=129096) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:54:47.631883 ncdv-master-1 devstack@n-api.service[129096]: INFO nova.api.openstack.requestlog [None req-39e6d360-d838-42c9-94a5-956e3eacb103 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002010 Sep 27 09:54:47.632260 ncdv-master-1 devstack@n-api.service[129096]: [pid: 129096|app: 0|req: 35/71] 10.222.0.26 () {56 vars in 1074 bytes} [Tue Sep 27 09:54:47 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:54:47.851164 ncdv-master-1 devstack@n-api.service[129097]: DEBUG nova.api.openstack.wsgi [None req-03c9350a-efc5-4ff0-9454-29b9961913c8 admin admin] Calling method '>' {{(pid=129097) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:54:47.854585 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-03c9350a-efc5-4ff0-9454-29b9961913c8 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:54:47.855058 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-03c9350a-efc5-4ff0-9454-29b9961913c8 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:54:47.855477 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-03c9350a-efc5-4ff0-9454-29b9961913c8 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:54:47.856176 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-03c9350a-efc5-4ff0-9454-29b9961913c8 admin admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:54:47.856608 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-03c9350a-efc5-4ff0-9454-29b9961913c8 admin admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:54:47.857015 ncdv-master-1 devstack@n-api.service[129097]: DEBUG oslo_concurrency.lockutils [None req-03c9350a-efc5-4ff0-9454-29b9961913c8 admin admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=129097) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:54:47.880072 ncdv-master-1 devstack@n-api.service[129097]: INFO nova.api.openstack.requestlog [None req-03c9350a-efc5-4ff0-9454-29b9961913c8 admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.92 time: 0.092024 Sep 27 09:54:47.880801 ncdv-master-1 devstack@n-api.service[129097]: [pid: 129097|app: 0|req: 37/72] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 09:54:47 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:55:34.914807 ncdv-master-1 systemd[1]: Stopping Devstack devstack@n-api.service... Sep 27 09:55:34.915730 ncdv-master-1 devstack@n-api.service[129095]: Tue Sep 27 09:55:34 2022 - graceful shutdown triggered... Sep 27 09:55:34.916376 ncdv-master-1 devstack@n-api.service[129096]: Gracefully killing worker 1 (pid: 129096)... Sep 27 09:55:34.916912 ncdv-master-1 devstack@n-api.service[129097]: Gracefully killing worker 2 (pid: 129097)... Sep 27 09:55:35.063230 ncdv-master-1 devstack@n-api.service[129096]: DEBUG dbcounter [-] [129096] Writing DB stats nova_api:SELECT=4 {{(pid=129096) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:55:35.916961 ncdv-master-1 devstack@n-api.service[129095]: worker 1 buried after 1 seconds Sep 27 09:55:35.916961 ncdv-master-1 devstack@n-api.service[129095]: worker 2 buried after 1 seconds Sep 27 09:55:35.916961 ncdv-master-1 devstack@n-api.service[129095]: goodbye to uWSGI. Sep 27 09:55:35.918643 ncdv-master-1 systemd[1]: devstack@n-api.service: Succeeded. Sep 27 09:55:35.919372 ncdv-master-1 systemd[1]: Stopped Devstack devstack@n-api.service. Sep 27 09:55:35.922974 ncdv-master-1 systemd[1]: Starting Devstack devstack@n-api.service... Sep 27 09:55:35.968851 ncdv-master-1 devstack@n-api.service[145959]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 27 09:55:35.972558 ncdv-master-1 devstack@n-api.service[145959]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 27 09:55:35 2022] *** Sep 27 09:55:35.972880 ncdv-master-1 devstack@n-api.service[145959]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 27 09:55:35.973061 ncdv-master-1 devstack@n-api.service[145959]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 27 09:55:35.973237 ncdv-master-1 devstack@n-api.service[145959]: nodename: ncdv-master-1 Sep 27 09:55:35.973237 ncdv-master-1 devstack@n-api.service[145959]: machine: x86_64 Sep 27 09:55:35.973446 ncdv-master-1 devstack@n-api.service[145959]: clock source: unix Sep 27 09:55:35.973446 ncdv-master-1 devstack@n-api.service[145959]: pcre jit disabled Sep 27 09:55:35.973446 ncdv-master-1 devstack@n-api.service[145959]: detected number of CPU cores: 4 Sep 27 09:55:35.973446 ncdv-master-1 devstack@n-api.service[145959]: current working directory: / Sep 27 09:55:35.973446 ncdv-master-1 devstack@n-api.service[145959]: detected binary path: /usr/bin/uwsgi-core Sep 27 09:55:35.973446 ncdv-master-1 devstack@n-api.service[145959]: your processes number limit is 47680 Sep 27 09:55:35.973446 ncdv-master-1 devstack@n-api.service[145959]: your memory page size is 4096 bytes Sep 27 09:55:35.973446 ncdv-master-1 devstack@n-api.service[145959]: detected max file descriptor number: 2048 Sep 27 09:55:35.973446 ncdv-master-1 devstack@n-api.service[145959]: lock engine: pthread robust mutexes Sep 27 09:55:35.974000 ncdv-master-1 devstack@n-api.service[145959]: thunder lock: enabled Sep 27 09:55:35.974000 ncdv-master-1 devstack@n-api.service[145959]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 27 09:55:35.974000 ncdv-master-1 devstack@n-api.service[145959]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 27 09:55:36.020439 ncdv-master-1 devstack@n-api.service[145959]: Python main interpreter initialized at 0x562a3e8bd9c0 Sep 27 09:55:36.020439 ncdv-master-1 devstack@n-api.service[145959]: python threads support enabled Sep 27 09:55:36.020439 ncdv-master-1 devstack@n-api.service[145959]: your server socket listen backlog is limited to 100 connections Sep 27 09:55:36.020439 ncdv-master-1 devstack@n-api.service[145959]: your mercy for graceful operations on workers is 90 seconds Sep 27 09:55:36.020993 ncdv-master-1 devstack@n-api.service[145959]: mapped 403077 bytes (393 KB) for 2 cores Sep 27 09:55:36.021064 ncdv-master-1 devstack@n-api.service[145959]: *** Operational MODE: preforking *** Sep 27 09:55:36.021382 ncdv-master-1 devstack@n-api.service[145959]: *** uWSGI is running in multiple interpreter mode *** Sep 27 09:55:36.021382 ncdv-master-1 devstack@n-api.service[145959]: spawned uWSGI master process (pid: 145959) Sep 27 09:55:36.021620 ncdv-master-1 systemd[1]: Started Devstack devstack@n-api.service. Sep 27 09:55:36.022507 ncdv-master-1 devstack@n-api.service[145959]: spawned uWSGI worker 1 (pid: 145968, cores: 1) Sep 27 09:55:36.022642 ncdv-master-1 devstack@n-api.service[145959]: spawned uWSGI worker 2 (pid: 145969, cores: 1) Sep 27 09:55:36.022799 ncdv-master-1 devstack@n-api.service[145959]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 27 09:55:37.370318 ncdv-master-1 devstack@n-api.service[145968]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 27 09:55:37.385658 ncdv-master-1 devstack@n-api.service[145969]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 27 09:55:38.138510 ncdv-master-1 devstack@n-api.service[145968]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:55:38.138510 ncdv-master-1 devstack@n-api.service[145968]: from cryptography.utils import int_from_bytes Sep 27 09:55:38.138842 ncdv-master-1 devstack@n-api.service[145968]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:55:38.138842 ncdv-master-1 devstack@n-api.service[145968]: from cryptography.utils import int_from_bytes Sep 27 09:55:38.148461 ncdv-master-1 devstack@n-api.service[145969]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:55:38.148461 ncdv-master-1 devstack@n-api.service[145969]: from cryptography.utils import int_from_bytes Sep 27 09:55:38.148839 ncdv-master-1 devstack@n-api.service[145969]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 27 09:55:38.148839 ncdv-master-1 devstack@n-api.service[145969]: from cryptography.utils import int_from_bytes Sep 27 09:55:40.628775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 09:55:40.629629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 09:55:40.629629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 09:55:40.629953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 09:55:40.630381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 09:55:40.630656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.630989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.631323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.631634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.631993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.632304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.632635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.632964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.633295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.633578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.633897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.634231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.634562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.634873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.635190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.635511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.635827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.636158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.636468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.636992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.637535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.637766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.638208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.638562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.638770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.639087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.639425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.639742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.640065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.640416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.642788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.642788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.642788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.642788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.642788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.642788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.642788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.643329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.643412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.643703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.644019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.644329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.644714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.645005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.645316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.645641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.646103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.646402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.646683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 27 09:55:40.647309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.647417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 27 09:55:40.647504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.647504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.647845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 27 09:55:40.647931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.647931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.648094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 27 09:55:40.648169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 27 09:55:40.648246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.648532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.648636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.648760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.648935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.649092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.649315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.649538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.649823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.650080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.650249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.650409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.650578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.650732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.650899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.651046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.651200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.651361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.651514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.651671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.651911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.652075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.652271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.652426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.652591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.652591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.652969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] console_host = ncdv-master-1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.653128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.653319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.653529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.653758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.653905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.654086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.654249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.654397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.654549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.654899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.655070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.655070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.655363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.655521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.655673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.655847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.655992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.656175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.656334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.656491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.656781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.657001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.657166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.657312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.657455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.657598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.657598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.657885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.658082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.658235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.658431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.658609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.658797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.658960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.659118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.659268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.659413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.659413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.659800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.659974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.660129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] host = ncdv-master-1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.660311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.660503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.660685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.660856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.661010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.661010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.661449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.661615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.661769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.661949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.662142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.662316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.662316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.662682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.662828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.662828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.663130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.663281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.663431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.663597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.663761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.663915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.664073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.664235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.664388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.664544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.664726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.665056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.665219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.665373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.665518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.665674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.665832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.665992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.666136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.666301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.666446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.666589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.666783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.666933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.667093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.667254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.667399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.667542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.667704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.667856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.668020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.668166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.668309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.668463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.668617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.668812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.668976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.669133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.669278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.669423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.669592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.669761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.669914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.670062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.670206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.670359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.670504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.670671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.670816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.670967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.671119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.671273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.671514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.671680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.671834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.671987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.672131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.672274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.672416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.672577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.672760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.672917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.673061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.673205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.673365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.673521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.673664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.673816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.673969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.674130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.674298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.674444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.674588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.674732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.674875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.675190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.675703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.675860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.676016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.676169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.676322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.676475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.676634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.676843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.677100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.677299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.677461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.677687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.677850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.677996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.678168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.678314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.678466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.678628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.678774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.678928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.679074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.679217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.679368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.679599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.679747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.679891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.680034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.680195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.680340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.680494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.680639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.680849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.681004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.681156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.681310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.681310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.681603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.681761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.681914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.682058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.682228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.682374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.682517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.682669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.682816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.682967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.683127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.683271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.683415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.683566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.683727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.684069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.684243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.684462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.684609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.684785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.684930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.685307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.685472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.685618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.685761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.685906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.686059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.686220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.686372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.686540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.686695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.686895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.687054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.687199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.687351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.687504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.687658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.687658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.687942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.688087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.688240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.688384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.688548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.688774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.688943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.689098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.689251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.689396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.689550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.689694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.689844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.690003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.690150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.690308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.690463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.690631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.690784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.690937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.691090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.691090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.691373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.691373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.691660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.691813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.691956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.691956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.691956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.691956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.692626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.692804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.692804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.693139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.693392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.693550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.693705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.693851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.693997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.694157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.694322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.694476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.694632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.694777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.694927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.695072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.695231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 27 09:55:40.695596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.695848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.696009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.696154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.696315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.696470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.696734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.696901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.697115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.697270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.697432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.697585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.697738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.697882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.698025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.698175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.698334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.698492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.698637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.698790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.698943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.699098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.699259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.699411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.699556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.699556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.699827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.699984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.700129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.700272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.700415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.700567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.700729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.700901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.701057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.701209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.701362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.701506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.701658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.701864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.702022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.702167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.702312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.702463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.702633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.702779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.702941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.703087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.703240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.703385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.703547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.703703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.703847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.703998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.704152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.704298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.704456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.704601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.704795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.704953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.705105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.705249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.705426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.705717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.705870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.706024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.706169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.706312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.706550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.706697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.706842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.706998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.707182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.707337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.707566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.707737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.707892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.708037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.708226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.708417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.708573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.708750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.708910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.709067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.709232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.709387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.709541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.709695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.709849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.710261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.710415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.710582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.710738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.710893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.710893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.711184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.711184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.711495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.711642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.711785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.711936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.712082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.712241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.712386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.712530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.712530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.712898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.713052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.713213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.713358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.713514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.713659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.713810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.713956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.714119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.714266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.714428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.714587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.714785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.714942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.715096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.715242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.715395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.715594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.715906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.716066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.716222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.716377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.716521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.716692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.716843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.717006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.717153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.717298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.717442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.717593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.717797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.717797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.718097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.718242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.718398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.718552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.718709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.718862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.719006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.719157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.719303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.719447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.719611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.719767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.719920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.720073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.720217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.720419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.720598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.720796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.720959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.721105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.721257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.721411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.721573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.721742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.721897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.722050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.722195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.722337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.722496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.722642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.722786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.722941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.723087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.723239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.723385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.723528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.723691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.723890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.724046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.724205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.724360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.724512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.724723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.724873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.725032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.725188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.725384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.725532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.725690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.725862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.726208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.726370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.726516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.726668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.726822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.726975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.727131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.727131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.727407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.727407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.727679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.727679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.727988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.727988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.728272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.728416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.728567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.728740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.728900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.729063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.729207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.729359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.729513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.729658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.729823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.729978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.730122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.730277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.730430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.730585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.730732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.730882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.731035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.731188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.731344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.731498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.731644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.731809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.731956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.732110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.732263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.732418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.732571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.732750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.732910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.733057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.733208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.733367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.733521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.733667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.733825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.733976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.734136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.734289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.734442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.734587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.734731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.734874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.735041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.735283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.735548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.735786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.736104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.736533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.736916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.737161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.737388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.737618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.737844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.738072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.738327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.738552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.738784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.739000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.739222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.739503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.739734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.739960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.740173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.740390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.740615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.740882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.741107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.741331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.741558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.741772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.741984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.742216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.742448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.742666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.742878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.743088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.743450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.743450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.743881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.744124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.744350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.744578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.744899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.745137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.745354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.745597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.745826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.746051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.746285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.746733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.746968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.747184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.747412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.747641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.747858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.748074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.748297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.748531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.748861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.749081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.749311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.749539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.749753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.749985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.750212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.750438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.750621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.750812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.751021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.751250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.751494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.751722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.751950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.752164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.752376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.752611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.752865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.753081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.753296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.753509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.753721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.753954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.754172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.754410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.754626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.754853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.755145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.755383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.755610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.755836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.756053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.756268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.756493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.757066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.757295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.757519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.757763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.757989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.758253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.758484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.758773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.758993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.759208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.759436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.759653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.759878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.760103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.760329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.760545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.760968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.761204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.761422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.761637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.761851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.762065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.762299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.762528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.762740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.762950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.763135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.763329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.763329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.764254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.764483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.764761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.764985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.765219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.765438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.765664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.765878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.766092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.766316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.766546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.766774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.767236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.767447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.767673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.767898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.768116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.768285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.768448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.768601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.768781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.768993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.769168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.769331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.769502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.769690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.769912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.770141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.770357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.770579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.770828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.771051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.771276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.771568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.771568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.771978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.772188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.772419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.772727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.772727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.773126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.773280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.773439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.773635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.773801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.773969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.774137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.774312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.774544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.774760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.774990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.775204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.775428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.775642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.775874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.776088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.776298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.776519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.776757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.777006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.777504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.777679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.777841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.778006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.778191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.778370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.778523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.778677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.778829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.779057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.779288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.779610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.779827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.780038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.780254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.780467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.780717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.780956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.781170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.781411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.781654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.781894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.782114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.782292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.782292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.782594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.782793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.782950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.783112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.783287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.783487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.783918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.784171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.784396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.784629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.784901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.785130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.785358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.785583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.785804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.786044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.786319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.786594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.786819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.787030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.787248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.787713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.787894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.788056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.788219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.788380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.788542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.788729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.788896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.789125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.789125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.789446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.789686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.789851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.790003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.790155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.790315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.790467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.790634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.790790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.790941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.791101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.791267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.791267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.791572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.791736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.791888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.792040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.792206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.792376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.792529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.792719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.792890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.793053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.793206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.793384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.793545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.793706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.793869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.794087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.794242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.794410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.794410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.794717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.794870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.795022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.795173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.795334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.795495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.795657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.795818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.795981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.796133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.796298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.796450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.796601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.796878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.797048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.797222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.797376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.797561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.797886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.798063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.798224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.798392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.798545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.798697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.798848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.799001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.799001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.799308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.799462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.799623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.799788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.799948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.800110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.800272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.800828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.800981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.801133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.801285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.801463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.801616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.801778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.802007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.802174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.802349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.802512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.802673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.802827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.802986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.803149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.803311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.803472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.803633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.803805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.803968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.804128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.804298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.804466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.804466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.804789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.804943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.805107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.805270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.805426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.805577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.805760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.805917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.806076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.806239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.806413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.806576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.806736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.806888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.807050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.807217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.807379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.807541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.807693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.807693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.808194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.808358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.808521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.808747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.808916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.809069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.809235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.809396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.809558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.809712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.809872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.810024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.810202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.810358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.810518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.810672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.810672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.810973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.811139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.811291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.811451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.811602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.811754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.811913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.812075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.812236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.812398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.812560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.812804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.812991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.813153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.813322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.813475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.813635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.813806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.813969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.814144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.814299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.814459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.814611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.814769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.814943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.815096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.815335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.815507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.815662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.815813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.815979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.816133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.816286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.816446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.816598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.816798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.816798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.817111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.817272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.817433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.817656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.817883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.818227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.818395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.818556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.818717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.818870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.819040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.819194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.819354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.819519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.819672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.819835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.820002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.820156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.820308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.820468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.820629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.820895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.821062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.821225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.821378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.821552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.821720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.821882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.822045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.822199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.822352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.822514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.822668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.822668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.822975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.823145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.823308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.823470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.823637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.823637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.823948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.823948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.824249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.824413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.855640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.855971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.859832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.860171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.860171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.861698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.863488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.863656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.863745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.863806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.863877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.863935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.863991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.864988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.865936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.866939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.867953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.868955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.869965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.870949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.871990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.872996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.873958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.874984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.875993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.876943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.877937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.878942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.879933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.880934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.881955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.882958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.883972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.884611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.891747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.891747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.891747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.891747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.891747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.891747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.891747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.892983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.893753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.894965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.895915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.896003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.896102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.896195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.896288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.896386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.896492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.896614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.900206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.900470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.900615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.900785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.900933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.901080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.901225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.901357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.901489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.901620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.901751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.901894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.901894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.902155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.902287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.902419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.902558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.902558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.902831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.902965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.903098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.903229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.903438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.903574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.903723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.903858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.903991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.904124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.904264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.904399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.904542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.904542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.904823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.904957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.905102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.905245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.905377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.905509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.905641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.905641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.905899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.906033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.906176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.906309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.906441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.908783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.909417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.909499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.909586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.909662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.909734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.909734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.909853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.909923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.910972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.911958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=145969) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 09:55:40.912975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.912975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.913409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.913409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.913409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.913409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.913409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.913409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.913409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.913409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.913873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.913873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.916042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.916042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.916042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.916042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.916042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.916042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.916348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.916871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.917021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.917345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.917689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.918027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.918346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.918662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.918977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.919297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.919626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.919943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.920256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.920566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.920905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.921215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.921524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.921833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.922142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.922455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.922801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.923072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.923380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.923686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.923999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.924298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.924608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.924941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.925250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.926142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.926296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.926496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.926838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.927233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.927554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.927883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.928198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.928524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.928876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.929203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.929531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.929852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.930164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.930500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.930811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.931123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.931437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.931746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.932056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.932366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.932773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.933088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.933396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.933726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.934046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.934369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.934687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.935000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.935333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.935645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.936311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.936311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.936597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.936900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.937217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.937565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.937879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.938197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.938517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.938851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.939186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.939501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.939831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.940162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.940514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.940951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.941308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.941687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.942055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.942373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.942722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 27 09:55:40.942972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=145968) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 27 09:55:40.971544 ncdv-master-1 devstack@n-api.service[145969]: INFO dbcounter [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Registered counter for database nova_api Sep 27 09:55:40.993243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_db.sqlalchemy.engines [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=145969) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:55:41.002028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writer thread running {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:55:41.013961 ncdv-master-1 devstack@n-api.service[145968]: INFO dbcounter [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Registered counter for database nova_api Sep 27 09:55:41.035432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_db.sqlalchemy.engines [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=145968) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:55:41.042572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writer thread running {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:55:41.077325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.context [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),ed1088da-caae-4426-8048-9cba90e7755c(cell1) {{(pid=145969) load_cells /opt/stack/nova/nova/context.py:464}} Sep 27 09:55:41.081700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:55:41.082244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:55:41.084228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:55:41.085138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:55:41.085601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:55:41.087791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:55:41.089911 ncdv-master-1 devstack@n-api.service[145969]: INFO dbcounter [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Registered counter for database nova_cell0 Sep 27 09:55:41.094538 ncdv-master-1 devstack@n-api.service[145969]: INFO dbcounter [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Registered counter for database nova_cell1 Sep 27 09:55:41.103081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_db.sqlalchemy.engines [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=145969) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:55:41.108371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.context [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),ed1088da-caae-4426-8048-9cba90e7755c(cell1) {{(pid=145968) load_cells /opt/stack/nova/nova/context.py:464}} Sep 27 09:55:41.110309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writer thread running {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:55:41.111304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_db.sqlalchemy.engines [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=145969) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:55:41.112475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:55:41.113064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:55:41.114844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:55:41.115819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:55:41.116271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:55:41.118340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:55:41.120567 ncdv-master-1 devstack@n-api.service[145968]: INFO dbcounter [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Registered counter for database nova_cell0 Sep 27 09:55:41.124953 ncdv-master-1 devstack@n-api.service[145968]: INFO dbcounter [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Registered counter for database nova_cell1 Sep 27 09:55:41.137709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_db.sqlalchemy.engines [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=145968) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:55:41.139559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_db.sqlalchemy.engines [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=145968) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:55:41.152076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writer thread running {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:55:41.155957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writer thread running {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:55:41.275734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writer thread running {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:55:41.285948 ncdv-master-1 devstack@n-api.service[145969]: INFO dbcounter [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Registered counter for database nova_cell0 Sep 27 09:55:41.293689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_db.sqlalchemy.engines [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=145969) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:55:41.299337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writer thread running {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:55:41.321790 ncdv-master-1 devstack@n-api.service[145968]: INFO dbcounter [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Registered counter for database nova_cell0 Sep 27 09:55:41.330116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_db.sqlalchemy.engines [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=145968) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 27 09:55:41.335563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writer thread running {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 27 09:55:41.762412 ncdv-master-1 devstack@n-api.service[145969]: WARNING keystonemiddleware.auth_token [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 09:55:41.770350 ncdv-master-1 devstack@n-api.service[145969]: WARNING keystonemiddleware.auth_token [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 09:55:41.796637 ncdv-master-1 devstack@n-api.service[145968]: WARNING keystonemiddleware.auth_token [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 09:55:41.805051 ncdv-master-1 devstack@n-api.service[145968]: WARNING keystonemiddleware.auth_token [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 09:55:42.004542 ncdv-master-1 devstack@n-api.service[145969]: WARNING keystonemiddleware.auth_token [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 09:55:42.010719 ncdv-master-1 devstack@n-api.service[145969]: WARNING keystonemiddleware.auth_token [None req-f209df8b-1225-4111-9c40-eb2e05f7efbc None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 09:55:42.016204 ncdv-master-1 devstack@n-api.service[145969]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562a3e8bd9c0 pid: 145969 (default app) Sep 27 09:55:42.033309 ncdv-master-1 devstack@n-api.service[145968]: WARNING keystonemiddleware.auth_token [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 27 09:55:42.039750 ncdv-master-1 devstack@n-api.service[145968]: WARNING keystonemiddleware.auth_token [None req-444f268e-8235-460b-ab3f-69ff80dada70 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 27 09:55:42.045248 ncdv-master-1 devstack@n-api.service[145968]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562a3e8bd9c0 pid: 145968 (default app) Sep 27 09:56:23.096481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:23.098135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:23.099012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:23.099735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:24.431513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c1c1311-b67c-4567-8f47-0f9809035faa tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:24.437371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c1c1311-b67c-4567-8f47-0f9809035faa tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:24.437829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c1c1311-b67c-4567-8f47-0f9809035faa tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:24.438261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c1c1311-b67c-4567-8f47-0f9809035faa tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:24.438989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c1c1311-b67c-4567-8f47-0f9809035faa tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:24.439438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c1c1311-b67c-4567-8f47-0f9809035faa tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:24.439855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c1c1311-b67c-4567-8f47-0f9809035faa tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:24.482189 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c1c1311-b67c-4567-8f47-0f9809035faa tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 1.401242 Sep 27 09:56:24.482776 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1/1] 10.222.0.26 () {58 vars in 1176 bytes} [Tue Sep 27 09:56:23 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 1404 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:24.814934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-771bf707-1204-4cb4-8fdd-cc0af43a6af7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-2138733149"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:24.841273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-771bf707-1204-4cb4-8fdd-cc0af43a6af7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Generated uuid 726b43fa-ae2e-4da7-b84c-e539a0f1d58a for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:56:24.850921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:24.853430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:24.854158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:24.855213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:24.995084 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-771bf707-1204-4cb4-8fdd-cc0af43a6af7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.495349 Sep 27 09:56:24.996287 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1/2] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:56:24 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:25.010427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1175a615-053a-45c4-9d4e-1d0aeb1b8290 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:25.072635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1175a615-053a-45c4-9d4e-1d0aeb1b8290 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:25.073170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1175a615-053a-45c4-9d4e-1d0aeb1b8290 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:25.073615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1175a615-053a-45c4-9d4e-1d0aeb1b8290 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:26.736442 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1175a615-053a-45c4-9d4e-1d0aeb1b8290 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/1/action" status: 200 len: 242 microversion: 2.1 time: 1.731940 Sep 27 09:56:26.745041 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2/3] 10.222.0.26 () {60 vars in 1191 bytes} [Tue Sep 27 09:56:25 2022] POST /compute/v2.1/os-aggregates/1/action => generated 242 bytes in 1734 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:27.074806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7e29f033-8f7c-41be-9fa7-64869ea55082 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Action: 'create', calling method: >, body: {"flavor": {"name": "tempest-test_flavor-1542132897", "ram": 512, "vcpus": 1, "disk": 10, "id": 506933704, "swap": 1024, "rxtx_factor": 1, "OS-FLV-EXT-DATA:ephemeral": 10}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:27.089523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Calling method '.version_select at 0x7f7f690e9e50>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:27.099358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),ed1088da-caae-4426-8048-9cba90e7755c(cell1) {{(pid=145968) load_cells /opt/stack/nova/nova/compute/api.py:319}} Sep 27 09:56:27.100468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:27.101925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:27.102551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:27.133370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:27.133655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:27.134205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:27.155261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:27.156454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:27.157293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:27.169776 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7e29f033-8f7c-41be-9fa7-64869ea55082 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 413 microversion: 2.1 time: 0.430951 Sep 27 09:56:27.170971 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8f057e92-1c6b-4d59-bf88-ae76769f5a2a tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/detail" status: 200 len: 1423 microversion: 2.41 time: 0.865437 Sep 27 09:56:27.174528 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2/4] 10.222.0.26 () {60 vars in 1208 bytes} [Tue Sep 27 09:56:26 2022] GET /compute/v2.1/os-hypervisors/detail => generated 1423 bytes in 870 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:56:27.182527 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3/5] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:56:26 2022] POST /compute/v2.1/flavors => generated 413 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:27.189382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af06d7d6-1b7e-4cfd-bc29-cf2b9b6ca1f7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:27.203533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af06d7d6-1b7e-4cfd-bc29-cf2b9b6ca1f7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:27.204270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af06d7d6-1b7e-4cfd-bc29-cf2b9b6ca1f7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:27.204914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af06d7d6-1b7e-4cfd-bc29-cf2b9b6ca1f7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:27.247530 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-af06d7d6-1b7e-4cfd-bc29-cf2b9b6ca1f7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] HTTP exception thrown: Aggregate 1 already has host nch1-master-1. Sep 27 09:56:27.272400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a8ef4926-2b8f-40dc-a24c-42847e7df917 tempest-QuotaClassesAdmin257Test-539745935 tempest-QuotaClassesAdmin257Test-539745935-project-admin] Calling method '.version_select at 0x7f7f690fff70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af06d7d6-1b7e-4cfd-bc29-cf2b9b6ca1f7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Request method failure captured: Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/os-aggregates/1/action HTTP/1.1 Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 39 Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjH7QYCLZugeJ0HemgmqlSpvzQrDapyIqrhuEZ_iPs-RyHoyEOHHpkZWyJeIyr1hcj35JJZjvQUwiTglpRf4FVRuf_GUwsYg1BP_B9dV7Pni9aC2IYru9U0UypZozOkcJALyLAHHjO5hrSMbhlJr0kvWgGPYkLcuS5TmJ2XrkKwfYg Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:27.292779 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fe46a7ae8eab451e941f83e19be3b539"}]}] Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 49f35168484b42feb22d106d745e6546 Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: {"add_host": {"host": "nch1-master-1"}} Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:27.293965 ncdv-master-1 devstack@n-api.service[145969]: exception: Aggregate 1 already has host nch1-master-1. Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1', 'body': {'add_host': {'host': 'nch1-master-1'}}} Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = self._query(query) Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi conn.query(q) Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result.read() Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 09:56:27.294735 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1062, "Duplicate entry 'nch1-master-1-1' for key 'aggregate_hosts.uniq_aggregate_hosts0host0aggregate_id'") Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 73, in _host_add_to_db Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi host_ref.save(context.session) Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/models.py", line 38, in save Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi session.flush() Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._flush(objects) Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi transaction.rollback(_capture_exception=True) Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 09:56:27.295703 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi compat.raise_( Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi flush_context.execute() Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi rec.execute(self) Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi util.preloaded.orm_persistence.save_obj( Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _emit_insert_statements( Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = connection._execute_20( Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return meth(self, args_10style, kwargs_10style, execution_options) Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return connection._execute_clauseelement( Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 27 09:56:27.296631 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = self._execute_context( Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._handle_dbapi_exception( Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = self._query(query) Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi conn.query(q) Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 09:56:27.297592 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result.read() Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'nch1-master-1-1' for key 'aggregate_hosts.uniq_aggregate_hosts0host0aggregate_id'") Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [SQL: INSERT INTO aggregate_hosts (created_at, updated_at, host, aggregate_id) VALUES (%(created_at)s, %(updated_at)s, %(host)s, %(aggregate_id)s)] Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2022, 9, 27, 9, 56, 27, 226264), 'updated_at': None, 'host': 'nch1-master-1', 'aggregate_id': 1}] Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:27.298462 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 180, in _add_host Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi aggregate = self.api.add_host_to_aggregate(context, id, host) Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6641, in add_host_to_aggregate Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi aggregate.add_host(host_name) Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 395, in add_host Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _host_add_to_db(self._context, self.id, host) Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 76, in _host_add_to_db Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.AggregateHostExists(host=host, Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.AggregateHostExists: Aggregate 1 already has host nch1-master-1. Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.299270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 187, in _add_host Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPConflict(explanation=e.format_message()) Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Aggregate 1 already has host nch1-master-1. Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:27.300265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:27.304305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 1 already has host nch1-master-1. Sep 27 09:56:27.304305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.304305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af06d7d6-1b7e-4cfd-bc29-cf2b9b6ca1f7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Returning 409 to user: Aggregate 1 already has host nch1-master-1. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:27.304305 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af06d7d6-1b7e-4cfd-bc29-cf2b9b6ca1f7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/1/action" status: 409 len: 95 microversion: 2.1 time: 0.110785 Sep 27 09:56:27.304305 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 4/6] 10.222.0.26 () {60 vars in 1191 bytes} [Tue Sep 27 09:56:27 2022] POST /compute/v2.1/os-aggregates/1/action => generated 95 bytes in 112 msecs (HTTP/1.1 409) 9 headers in 377 bytes (1 switches on core 0) Sep 27 09:56:27.311053 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a8ef4926-2b8f-40dc-a24c-42847e7df917 tempest-QuotaClassesAdmin257Test-539745935 tempest-QuotaClassesAdmin257Test-539745935-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 172 microversion: 2.57 time: 0.135372 Sep 27 09:56:27.311814 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3/7] 10.222.0.26 () {60 vars in 1226 bytes} [Tue Sep 27 09:56:27 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 172 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:56:27.385269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b25acaca-865f-4387-ac99-fab9e98f8e2d tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Calling method '.version_select at 0x7f7f6886e670>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:27.386876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '.version_select at 0x7f7f6853c160>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:27.388020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b25acaca-865f-4387-ac99-fab9e98f8e2d tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:27.388817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b25acaca-865f-4387-ac99-fab9e98f8e2d tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:27.389795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b25acaca-865f-4387-ac99-fab9e98f8e2d tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:27.396515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),ed1088da-caae-4426-8048-9cba90e7755c(cell1) {{(pid=145969) load_cells /opt/stack/nova/nova/compute/api.py:319}} Sep 27 09:56:27.397474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:27.398232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:27.399060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:27.423207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:27.423890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:27.424684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:27.428331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b25acaca-865f-4387-ac99-fab9e98f8e2d tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:27.429081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b25acaca-865f-4387-ac99-fab9e98f8e2d tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:27.430375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b25acaca-865f-4387-ac99-fab9e98f8e2d tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:27.440701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:27.441741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:27.442534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:27.452025 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/detail" status: 200 len: 1503 microversion: 2.1 time: 0.155778 Sep 27 09:56:27.453631 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 5/8] 10.222.0.26 () {58 vars in 1167 bytes} [Tue Sep 27 09:56:27 2022] GET /compute/v2.1/os-hypervisors/detail => generated 1503 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:27.454790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:27.456186 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-736f0e1b-9b3b-433d-8e24-fbf88cc3513f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute//" status: 200 len: 377 microversion: - time: 0.001563 Sep 27 09:56:27.460546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b25acaca-865f-4387-ac99-fab9e98f8e2d tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:27.461512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b25acaca-865f-4387-ac99-fab9e98f8e2d tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:27.462260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b25acaca-865f-4387-ac99-fab9e98f8e2d tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:27.467296 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 6/9] 10.222.0.26 () {56 vars in 886 bytes} [Tue Sep 27 09:56:27 2022] GET /compute/ => generated 377 bytes in 13 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Sep 27 09:56:27.489413 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b25acaca-865f-4387-ac99-fab9e98f8e2d tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.176420 Sep 27 09:56:27.495078 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 4/10] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:56:27 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:27.505584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3516992-ddd3-45db-b972-421241467dea tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:27.535522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3516992-ddd3-45db-b972-421241467dea tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:27.536051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3516992-ddd3-45db-b972-421241467dea tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:27.536497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3516992-ddd3-45db-b972-421241467dea tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:27.656443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95f708e2-24f7-4154-870b-f4ba51010c90 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:27.721655 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-95f708e2-24f7-4154-870b-f4ba51010c90 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] HTTP exception thrown: Flavor 506933704 has no extra specs with key hw:cpu_thread_policy. Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95f708e2-24f7-4154-870b-f4ba51010c90 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Request method failure captured: Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/flavors/506933704/os-extra_specs/hw:cpu_thread_policy HTTP/1.1 Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjHHJPESPjYVWF9FoFst8qe5hqenr60H6iZm5otCBHRiNn6vt6iR_eOm70L_bsr1JhJbRd9VW4zIKk_PiZM4rJfPF5TpCwBHmoRc_zHV8JtjHSbgZ2RlNMxWaH8C2xBw90L4dOTgtXzPHb5mACYE7X5KQpfutDNDUnL0BmPaoxpQqw Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: a5b2045c08fb42b888d2fcb7a9cd396f Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:27.723399 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:27.724504 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a5b2045c08fb42b888d2fcb7a9cd396f"}]}] Sep 27 09:56:27.724504 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:27.724504 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: a5b2045c08fb42b888d2fcb7a9cd396f Sep 27 09:56:27.724504 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:27.724504 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project Sep 27 09:56:27.724504 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:27.724504 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:27.724504 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 25d7b2696d6745fd944d3131269c70a2 Sep 27 09:56:27.724504 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project Sep 27 09:56:27.724504 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:27.724504 ncdv-master-1 devstack@n-api.service[145969]: exception: Flavor 506933704 has no extra specs with key hw:cpu_thread_policy. Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'flavor_id': '506933704', 'id': 'hw:cpu_thread_policy'} Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 115, in show Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return {id: flavor.extra_specs[id]} Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi KeyError: 'hw:cpu_thread_policy' Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 120, in show Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 506933704 has no extra specs with key hw:cpu_thread_policy. Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:27.725143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.728293 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:27.728293 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:27.728293 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:27.728293 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:27.728293 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:27.728293 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 506933704 has no extra specs with key hw:cpu_thread_policy. Sep 27 09:56:27.728293 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:27.728293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95f708e2-24f7-4154-870b-f4ba51010c90 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Returning 404 to user: Flavor 506933704 has no extra specs with key hw:cpu_thread_policy. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:27.729307 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-95f708e2-24f7-4154-870b-f4ba51010c90 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] 10.222.0.26 "GET /compute/v2.1/flavors/506933704/os-extra_specs/hw:cpu_thread_policy" status: 404 len: 112 microversion: 2.1 time: 0.259413 Sep 27 09:56:27.729307 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 7/11] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 09:56:27 2022] GET /compute/v2.1/flavors/506933704/os-extra_specs/hw:cpu_thread_policy => generated 112 bytes in 261 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:27.745129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-89e84e36-efa8-4be3-b4a2-8e5da13ce343 tempest-QuotaClassesAdmin257Test-539745935 tempest-QuotaClassesAdmin257Test-539745935-project-admin] Action: 'update', calling method: .version_select at 0x7f7f684d6820>, body: {"quota_class_set": {"cores": 120, "instances": 110, "key_pairs": 200, "metadata_items": 228, "ram": 51300, "server_group_members": 110, "server_groups": 110}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:28.044251 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d3516992-ddd3-45db-b972-421241467dea tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/1/action" status: 200 len: 227 microversion: 2.1 time: 0.547824 Sep 27 09:56:28.046363 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 5/12] 10.222.0.26 () {60 vars in 1191 bytes} [Tue Sep 27 09:56:27 2022] POST /compute/v2.1/os-aggregates/1/action => generated 227 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:28.068786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-50c37cf3-73cb-44d2-96cf-006c005c354e tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Calling method '.version_select at 0x7f7f68883f70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:28.070819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50c37cf3-73cb-44d2-96cf-006c005c354e tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:28.071308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50c37cf3-73cb-44d2-96cf-006c005c354e tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:28.071743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50c37cf3-73cb-44d2-96cf-006c005c354e tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:28.118196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50c37cf3-73cb-44d2-96cf-006c005c354e tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:28.124842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50c37cf3-73cb-44d2-96cf-006c005c354e tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:28.125329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50c37cf3-73cb-44d2-96cf-006c005c354e tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:28.136932 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-89e84e36-efa8-4be3-b4a2-8e5da13ce343 tempest-QuotaClassesAdmin257Test-539745935 tempest-QuotaClassesAdmin257Test-539745935-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 159 microversion: 2.57 time: 0.405337 Sep 27 09:56:28.143205 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 8/13] 10.222.0.26 () {62 vars in 1247 bytes} [Tue Sep 27 09:56:27 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 159 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:56:28.154507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e5708b4b-f65e-468a-a0b9-1c41680a16cb tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:28.158587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e5708b4b-f65e-468a-a0b9-1c41680a16cb tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Policy check for os_compute_api:os-flavor-extra-specs:create failed with credentials {'is_admin': False, 'user_id': '25d7b2696d6745fd944d3131269c70a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a5b2045c08fb42b888d2fcb7a9cd396f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e5708b4b-f65e-468a-a0b9-1c41680a16cb tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Request method failure captured: Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/flavors/506933704/os-extra_specs HTTP/1.1 Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 66 Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjHHJPESPjYVWF9FoFst8qe5hqenr60H6iZm5otCBHRiNn6vt6iR_eOm70L_bsr1JhJbRd9VW4zIKk_PiZM4rJfPF5TpCwBHmoRc_zHV8JtjHSbgZ2RlNMxWaH8C2xBw90L4dOTgtXzPHb5mACYE7X5KQpfutDNDUnL0BmPaoxpQqw Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: a5b2045c08fb42b888d2fcb7a9cd396f Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:28.166329 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a5b2045c08fb42b888d2fcb7a9cd396f"}]}] Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: a5b2045c08fb42b888d2fcb7a9cd396f Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 25d7b2696d6745fd944d3131269c70a2 Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} Sep 27 09:56:28.167467 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'flavor_id': '506933704', 'body': {'extra_specs': {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'}}} Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 73, in create Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(fes_policies.POLICY_ROOT % 'create', target={}) Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:28.168063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. Sep 27 09:56:28.169652 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.172046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e5708b4b-f65e-468a-a0b9-1c41680a16cb tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-extra-specs:create to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:28.172979 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e5708b4b-f65e-468a-a0b9-1c41680a16cb tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] 10.222.0.26 "POST /compute/v2.1/flavors/506933704/os-extra_specs" status: 403 len: 124 microversion: 2.1 time: 0.032752 Sep 27 09:56:28.173933 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 9/14] 10.222.0.26 () {60 vars in 1221 bytes} [Tue Sep 27 09:56:28 2022] POST /compute/v2.1/flavors/506933704/os-extra_specs => generated 124 bytes in 35 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:28.189456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f9e43098-b943-41e2-b8dd-8ef6eade6921 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:28.226404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50c37cf3-73cb-44d2-96cf-006c005c354e tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:28.227667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50c37cf3-73cb-44d2-96cf-006c005c354e tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:28.228133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50c37cf3-73cb-44d2-96cf-006c005c354e tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:28.240124 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f9e43098-b943-41e2-b8dd-8ef6eade6921 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/1" status: 200 len: 0 microversion: 2.1 time: 0.063926 Sep 27 09:56:28.241079 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 10/15] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:56:28 2022] DELETE /compute/v2.1/os-aggregates/1 => generated 0 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:56:28.248907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0af98fb-3235-47fe-9f67-57f5bfd9b601 tempest-QuotaClassesAdmin257Test-539745935 tempest-QuotaClassesAdmin257Test-539745935-project-admin] Calling method '.version_select at 0x7f7f688515e0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:28.249726 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-50c37cf3-73cb-44d2-96cf-006c005c354e tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/nch1-master-1/servers" status: 200 len: 104 microversion: 2.1 time: 0.201900 Sep 27 09:56:28.250343 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 6/16] 10.222.0.26 () {58 vars in 1212 bytes} [Tue Sep 27 09:56:28 2022] GET /compute/v2.1/os-hypervisors/nch1-master-1/servers => generated 104 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:28.263664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d62a3df-d75f-4760-a670-88361bf3423c tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:28.286689 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0af98fb-3235-47fe-9f67-57f5bfd9b601 tempest-QuotaClassesAdmin257Test-539745935 tempest-QuotaClassesAdmin257Test-539745935-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 176 microversion: 2.57 time: 0.044403 Sep 27 09:56:28.289588 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 11/17] 10.222.0.26 () {60 vars in 1226 bytes} [Tue Sep 27 09:56:28 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 176 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:56:28.308773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-53bc259c-ca0f-42aa-ac71-57833cbc91c6 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-965907758"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:28.315829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.aggregate [None req-53bc259c-ca0f-42aa-ac71-57833cbc91c6 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Generated uuid 4c23f630-ee21-4c95-9ad0-003d148c7ea9 for aggregate {{(pid=145969) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:56:28.355140 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-53bc259c-ca0f-42aa-ac71-57833cbc91c6 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 197 microversion: 2.1 time: 0.056374 Sep 27 09:56:28.357819 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 12/18] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:56:28 2022] POST /compute/v2.1/os-aggregates => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:28.369382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7f2f44ff-108f-4a98-86f1-00ba0541df81 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Calling method '.version_select at 0x7f7f6908e0d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:28.375806 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5d62a3df-d75f-4760-a670-88361bf3423c tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/506933704/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.112838 Sep 27 09:56:28.379805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f2f44ff-108f-4a98-86f1-00ba0541df81 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:28.380160 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 7/19] 10.222.0.26 () {60 vars in 1221 bytes} [Tue Sep 27 09:56:28 2022] POST /compute/v2.1/flavors/506933704/os-extra_specs => generated 66 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:56:28.385813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f2f44ff-108f-4a98-86f1-00ba0541df81 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:28.386276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f2f44ff-108f-4a98-86f1-00ba0541df81 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:28.387250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d5da877-2b6a-4fbd-8d1d-326446f8cc4f tempest-QuotaClassesAdmin257Test-539745935 tempest-QuotaClassesAdmin257Test-539745935-project-admin] Action: 'update', calling method: .version_select at 0x7f7f687cb820>, body: {"quota_class_set": {"cores": 20, "instances": 10, "key_pairs": 100, "metadata_items": 128, "ram": 51200, "server_group_members": 10, "server_groups": 10}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:28.415976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f2f44ff-108f-4a98-86f1-00ba0541df81 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:28.416467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f2f44ff-108f-4a98-86f1-00ba0541df81 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:28.416934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f2f44ff-108f-4a98-86f1-00ba0541df81 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:28.446154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f2f44ff-108f-4a98-86f1-00ba0541df81 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:28.446656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f2f44ff-108f-4a98-86f1-00ba0541df81 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:28.447513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f2f44ff-108f-4a98-86f1-00ba0541df81 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:28.455352 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7f2f44ff-108f-4a98-86f1-00ba0541df81 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.095903 Sep 27 09:56:28.456229 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 13/20] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:56:28 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:28.473879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c1502a4f-859f-4379-a0ee-354502e9a537 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"availability_zone": "tempest-test_az-1086642901", "name": "tempest-test_aggregate-1195796150"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:28.494529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.aggregate [None req-c1502a4f-859f-4379-a0ee-354502e9a537 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Generated uuid e054732a-8606-46ca-8e99-af99ffba6560 for aggregate {{(pid=145969) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:56:28.516763 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5d5da877-2b6a-4fbd-8d1d-326446f8cc4f tempest-QuotaClassesAdmin257Test-539745935 tempest-QuotaClassesAdmin257Test-539745935-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 155 microversion: 2.57 time: 0.133376 Sep 27 09:56:28.518353 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 8/21] 10.222.0.26 () {62 vars in 1247 bytes} [Tue Sep 27 09:56:28 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 155 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:56:28.536223 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c1502a4f-859f-4379-a0ee-354502e9a537 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 222 microversion: 2.1 time: 0.074937 Sep 27 09:56:28.537506 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 14/22] 10.222.0.26 () {60 vars in 1165 bytes} [Tue Sep 27 09:56:28 2022] POST /compute/v2.1/os-aggregates => generated 222 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:28.542862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9493ba67-e19f-42f7-a93f-421f3c4d4054 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:28.544867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9493ba67-e19f-42f7-a93f-421f3c4d4054 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Policy check for os_compute_api:os-flavor-extra-specs:delete failed with credentials {'is_admin': False, 'user_id': '25d7b2696d6745fd944d3131269c70a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a5b2045c08fb42b888d2fcb7a9cd396f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9493ba67-e19f-42f7-a93f-421f3c4d4054 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Request method failure captured: Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes HTTP/1.1 Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjHHJPESPjYVWF9FoFst8qe5hqenr60H6iZm5otCBHRiNn6vt6iR_eOm70L_bsr1JhJbRd9VW4zIKk_PiZM4rJfPF5TpCwBHmoRc_zHV8JtjHSbgZ2RlNMxWaH8C2xBw90L4dOTgtXzPHb5mACYE7X5KQpfutDNDUnL0BmPaoxpQqw Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: a5b2045c08fb42b888d2fcb7a9cd396f Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:28.547416 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:28.548553 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a5b2045c08fb42b888d2fcb7a9cd396f"}]}] Sep 27 09:56:28.548553 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:28.548553 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: a5b2045c08fb42b888d2fcb7a9cd396f Sep 27 09:56:28.548553 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:28.548553 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project Sep 27 09:56:28.548553 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:28.548553 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:28.548553 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 25d7b2696d6745fd944d3131269c70a2 Sep 27 09:56:28.548553 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project Sep 27 09:56:28.548553 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:28.548553 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'flavor_id': '506933704', 'id': 'hw:numa_nodes'} Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 129, in delete Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(fes_policies.POLICY_ROOT % 'delete', target={}) Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:28.549135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.550140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9493ba67-e19f-42f7-a93f-421f3c4d4054 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-extra-specs:delete to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:28.550981 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9493ba67-e19f-42f7-a93f-421f3c4d4054 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] 10.222.0.26 "DELETE /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes" status: 403 len: 124 microversion: 2.1 time: 0.010182 Sep 27 09:56:28.554952 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 15/23] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 09:56:28 2022] DELETE /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes => generated 124 bytes in 17 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:28.560767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-575dc36c-ee7d-462d-9d97-a54f97d489b2 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Calling method '.version_select at 0x7f7f68461820>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:28.562480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-575dc36c-ee7d-462d-9d97-a54f97d489b2 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:28.563035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-575dc36c-ee7d-462d-9d97-a54f97d489b2 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:28.563693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-575dc36c-ee7d-462d-9d97-a54f97d489b2 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:28.587124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-575dc36c-ee7d-462d-9d97-a54f97d489b2 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:28.587627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-575dc36c-ee7d-462d-9d97-a54f97d489b2 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:28.588257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-575dc36c-ee7d-462d-9d97-a54f97d489b2 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:28.595252 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-575dc36c-ee7d-462d-9d97-a54f97d489b2 tempest-HypervisorAdminUnderV252Test-1479040114 tempest-HypervisorAdminUnderV252Test-1479040114-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/nch1-master-1/search" status: 200 len: 104 microversion: 2.1 time: 0.039238 Sep 27 09:56:28.603434 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 16/24] 10.222.0.26 () {58 vars in 1209 bytes} [Tue Sep 27 09:56:28 2022] GET /compute/v2.1/os-hypervisors/nch1-master-1/search => generated 104 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:28.603434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-77e9d991-6fef-4f80-b506-5ce9e220ef7b tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:28.613539 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-77e9d991-6fef-4f80-b506-5ce9e220ef7b tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates" status: 200 len: 516 microversion: 2.1 time: 0.016128 Sep 27 09:56:28.621729 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 17/25] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 09:56:28 2022] GET /compute/v2.1/os-aggregates => generated 516 bytes in 17 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:28.621729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d83a55a5-d8ec-48f8-8fec-5dda58eb08d2 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:28.654546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=77,nova_api:INSERT=13,nova_api:UPDATE=7,nova_api:DELETE=2,nova_api:SAVEPOINT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:28.686502 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d83a55a5-d8ec-48f8-8fec-5dda58eb08d2 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/506933704/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.070790 Sep 27 09:56:28.696450 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 18/26] 10.222.0.26 () {60 vars in 1221 bytes} [Tue Sep 27 09:56:28 2022] POST /compute/v2.1/flavors/506933704/os-extra_specs => generated 66 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:56:28.706359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b09d0fe6-bb5b-4c0c-93f1-21f65fbd90f5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:28.725504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b09d0fe6-bb5b-4c0c-93f1-21f65fbd90f5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:28.726049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b09d0fe6-bb5b-4c0c-93f1-21f65fbd90f5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:28.726652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b09d0fe6-bb5b-4c0c-93f1-21f65fbd90f5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:28.941147 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b09d0fe6-bb5b-4c0c-93f1-21f65fbd90f5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/3/action" status: 200 len: 315 microversion: 2.1 time: 0.243407 Sep 27 09:56:28.943256 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 19/27] 10.222.0.26 () {60 vars in 1191 bytes} [Tue Sep 27 09:56:28 2022] POST /compute/v2.1/os-aggregates/3/action => generated 315 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:28.954273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-66d27511-d7a6-414f-8f3a-786fb5cb81f7 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Action: 'update', calling method: >, body: {"hw:numa_nodes": "1"} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:28.965582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-66d27511-d7a6-414f-8f3a-786fb5cb81f7 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Policy check for os_compute_api:os-flavor-extra-specs:update failed with credentials {'is_admin': False, 'user_id': '25d7b2696d6745fd944d3131269c70a2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a5b2045c08fb42b888d2fcb7a9cd396f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-66d27511-d7a6-414f-8f3a-786fb5cb81f7 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Request method failure captured: Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes HTTP/1.1 Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 22 Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjHHJPESPjYVWF9FoFst8qe5hqenr60H6iZm5otCBHRiNn6vt6iR_eOm70L_bsr1JhJbRd9VW4zIKk_PiZM4rJfPF5TpCwBHmoRc_zHV8JtjHSbgZ2RlNMxWaH8C2xBw90L4dOTgtXzPHb5mACYE7X5KQpfutDNDUnL0BmPaoxpQqw Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: a5b2045c08fb42b888d2fcb7a9cd396f Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:28.969070 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a5b2045c08fb42b888d2fcb7a9cd396f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a5b2045c08fb42b888d2fcb7a9cd396f"}]}] Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: a5b2045c08fb42b888d2fcb7a9cd396f Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 25d7b2696d6745fd944d3131269c70a2 Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: {"hw:numa_nodes": "1"} Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:28.970167 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'flavor_id': '506933704', 'id': 'hw:numa_nodes', 'body': {'hw:numa_nodes': '1'}} Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 91, in update Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(fes_policies.POLICY_ROOT % 'update', target={}) Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:28.971024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. Sep 27 09:56:28.972181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.973313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-66d27511-d7a6-414f-8f3a-786fb5cb81f7 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-extra-specs:update to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:28.973313 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-66d27511-d7a6-414f-8f3a-786fb5cb81f7 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956 tempest-FlavorsExtraSpecsNegativeTestJSON-335984956-project] 10.222.0.26 "PUT /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes" status: 403 len: 124 microversion: 2.1 time: 0.026494 Sep 27 09:56:28.973508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b46255a5-00d6-4725-971e-bc27a1313b8b tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:28.973627 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 20/28] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 09:56:28 2022] PUT /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes => generated 124 bytes in 30 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:28.980119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b46255a5-00d6-4725-971e-bc27a1313b8b tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Policy check for os_compute_api:os-aggregates:add_host failed with credentials {'is_admin': False, 'user_id': '8ea0f9af73d5435088120d793a61bef3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5a71603028ff42b4847878e7ede6aebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:28.989782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-860fa32d-276c-4c0f-8fa1-c014aefc9e3f tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b46255a5-00d6-4725-971e-bc27a1313b8b tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Request method failure captured: Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-aggregates/2/action HTTP/1.1 Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 39 Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjFzty-qLENwwC0WJRfeIxzeZV65xTjvO7a5eLb6pj1D2B1dsP_rK8RVeimECsxHNyUllFJgmBOhq-mF9B6V9NztZc9XT1FhgvcpY2yNe42ftBLJ2Zi5O1gzeQf0ok5TU-3MikfFG4bnjZpU2GZ8UnwGjiq5xJIj1Lmd1RWDnpXTf4 Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:56:28.990692 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5a71603028ff42b4847878e7ede6aebe"}]}] Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 8ea0f9af73d5435088120d793a61bef3 Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: {"add_host": {"host": "nch1-master-1"}} Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:56:28.992334 ncdv-master-1 devstack@n-api.service[145968]: exception: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '2', 'body': {'add_host': {'host': 'nch1-master-1'}}} Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 172, in _add_host Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'add_host', target={}) Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:28.993055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. Sep 27 09:56:28.993977 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:28.994916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b46255a5-00d6-4725-971e-bc27a1313b8b tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:add_host to be performed. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:28.997134 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b46255a5-00d6-4725-971e-bc27a1313b8b tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] 10.222.0.26 "POST /compute/v2.1/os-aggregates/2/action" status: 403 len: 118 microversion: 2.1 time: 0.477160 Sep 27 09:56:29.004039 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 9/29] 10.222.0.26 () {60 vars in 1191 bytes} [Tue Sep 27 09:56:28 2022] POST /compute/v2.1/os-aggregates/2/action => generated 118 bytes in 484 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:29.013028 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-860fa32d-276c-4c0f-8fa1-c014aefc9e3f tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] HTTP exception thrown: Flavor 506933704 has no extra specs with key hw:cpu_thread_policy. Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-860fa32d-276c-4c0f-8fa1-c014aefc9e3f tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Request method failure captured: Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/flavors/506933704/os-extra_specs/hw:cpu_thread_policy HTTP/1.1 Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjKDMNuYmzIaK7z4r5EwC0OelA3-CnyT4cs7mWyH_Au6oAAX_36U2aDPwoB9b-EcBbU0Jn1sWNgepumqrGNf3AaQpfRyfMj6i4t42utPTswc5nY1wBA09PAfsc0LdFlNz-7Kn_Vug5iqHfIlcAyae77nVk24-fBnJbS3MZ6BXK9GjI Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2e256e7d04da47679d60ea6549f50e7b Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:29.016069 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:29.018218 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2e256e7d04da47679d60ea6549f50e7b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2e256e7d04da47679d60ea6549f50e7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2e256e7d04da47679d60ea6549f50e7b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2e256e7d04da47679d60ea6549f50e7b"}]}] Sep 27 09:56:29.018218 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 Sep 27 09:56:29.018218 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2e256e7d04da47679d60ea6549f50e7b Sep 27 09:56:29.018218 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 Sep 27 09:56:29.018218 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin Sep 27 09:56:29.018218 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:29.018218 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:29.018218 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 891c58769fc54d58a36cde92f5b358f4 Sep 27 09:56:29.018218 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin Sep 27 09:56:29.018218 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:29.018218 ncdv-master-1 devstack@n-api.service[145969]: exception: Flavor 506933704 has no extra specs with key hw:cpu_thread_policy. Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'flavor_id': '506933704', 'id': 'hw:cpu_thread_policy'} Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 132, in delete Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi del flavor.extra_specs[id] Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi KeyError: 'hw:cpu_thread_policy' Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 141, in delete Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 506933704 has no extra specs with key hw:cpu_thread_policy. Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.018956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.020016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.020016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.020016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.020016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:29.020016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:29.020016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 506933704 has no extra specs with key hw:cpu_thread_policy. Sep 27 09:56:29.020016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.020016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-860fa32d-276c-4c0f-8fa1-c014aefc9e3f tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Returning 404 to user: Flavor 506933704 has no extra specs with key hw:cpu_thread_policy. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:29.020016 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-860fa32d-276c-4c0f-8fa1-c014aefc9e3f tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/506933704/os-extra_specs/hw:cpu_thread_policy" status: 404 len: 112 microversion: 2.1 time: 0.035673 Sep 27 09:56:29.020016 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 21/30] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 09:56:28 2022] DELETE /compute/v2.1/flavors/506933704/os-extra_specs/hw:cpu_thread_policy => generated 112 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:29.020563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dc111b67-53b7-4a1d-bb25-aebdc433ff42 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:29.041614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-76c0128c-d84e-40c2-af88-c19d1531c93c tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Action: 'update', calling method: >, body: {"hw:cpu_policy": "shared"} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:29.048765 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-76c0128c-d84e-40c2-af88-c19d1531c93c tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] HTTP exception thrown: Request body and URI mismatch Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-76c0128c-d84e-40c2-af88-c19d1531c93c tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Request method failure captured: Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes HTTP/1.1 Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 27 Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjKDMNuYmzIaK7z4r5EwC0OelA3-CnyT4cs7mWyH_Au6oAAX_36U2aDPwoB9b-EcBbU0Jn1sWNgepumqrGNf3AaQpfRyfMj6i4t42utPTswc5nY1wBA09PAfsc0LdFlNz-7Kn_Vug5iqHfIlcAyae77nVk24-fBnJbS3MZ6BXK9GjI Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2e256e7d04da47679d60ea6549f50e7b Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:29.050047 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2e256e7d04da47679d60ea6549f50e7b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2e256e7d04da47679d60ea6549f50e7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2e256e7d04da47679d60ea6549f50e7b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2e256e7d04da47679d60ea6549f50e7b"}]}] Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2e256e7d04da47679d60ea6549f50e7b Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 891c58769fc54d58a36cde92f5b358f4 Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: {"hw:cpu_policy": "shared"} Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:29.051638 ncdv-master-1 devstack@n-api.service[145969]: exception: Request body and URI mismatch Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'flavor_id': '506933704', 'id': 'hw:numa_nodes', 'body': {'hw:cpu_policy': 'shared'}} Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors_extraspecs.py", line 96, in update Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=expl) Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Request body and URI mismatch Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.052492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.054297 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:29.054297 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:29.054297 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Request body and URI mismatch Sep 27 09:56:29.054297 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.061009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-76c0128c-d84e-40c2-af88-c19d1531c93c tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Returning 400 to user: Request body and URI mismatch {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:29.069565 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-76c0128c-d84e-40c2-af88-c19d1531c93c tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] 10.222.0.26 "PUT /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes" status: 400 len: 73 microversion: 2.1 time: 0.042346 Sep 27 09:56:29.078251 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 22/31] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 09:56:29 2022] PUT /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes => generated 73 bytes in 47 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 09:56:29.078430 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dc111b67-53b7-4a1d-bb25-aebdc433ff42 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/2" status: 200 len: 0 microversion: 2.1 time: 0.065772 Sep 27 09:56:29.080987 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 10/32] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:56:29 2022] DELETE /compute/v2.1/os-aggregates/2 => generated 0 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:56:29.088320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-47978ac6-5938-4f13-bbde-9505fd3504e2 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Action: 'update', calling method: >, body: {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:29.095762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da3ca238-adc5-49c2-9446-3212971ca815 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1639046203"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:29.103976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-da3ca238-adc5-49c2-9446-3212971ca815 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Generated uuid 72073e73-25e2-426d-9106-76c555e6fdf5 for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-47978ac6-5938-4f13-bbde-9505fd3504e2 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Request method failure captured: Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes HTTP/1.1 Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 49 Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjKDMNuYmzIaK7z4r5EwC0OelA3-CnyT4cs7mWyH_Au6oAAX_36U2aDPwoB9b-EcBbU0Jn1sWNgepumqrGNf3AaQpfRyfMj6i4t42utPTswc5nY1wBA09PAfsc0LdFlNz-7Kn_Vug5iqHfIlcAyae77nVk24-fBnJbS3MZ6BXK9GjI Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2e256e7d04da47679d60ea6549f50e7b Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:29.106171 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2e256e7d04da47679d60ea6549f50e7b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2e256e7d04da47679d60ea6549f50e7b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2e256e7d04da47679d60ea6549f50e7b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2e256e7d04da47679d60ea6549f50e7b"}]}] Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2e256e7d04da47679d60ea6549f50e7b Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 891c58769fc54d58a36cde92f5b358f4 Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"} Sep 27 09:56:29.107365 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: exception: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'flavor_id': '506933704', 'id': 'hw:numa_nodes', 'body': {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'}} Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'maxProperties' in schema: Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'maxProperties': 1, Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'minProperties': 1, Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': ['string', Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'number']}}, Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'object'} Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance: Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'hw:cpu_policy': 'shared', 'hw:numa_nodes': '1'} Sep 27 09:56:29.107980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.109154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.110141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.110141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.110141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 09:56:29.110141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 09:56:29.110141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties Sep 27 09:56:29.110141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.110141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-47978ac6-5938-4f13-bbde-9505fd3504e2 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Returning 400 to user: {'hw:numa_nodes': '1', 'hw:cpu_policy': 'shared'} has too many properties {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:29.110141 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-47978ac6-5938-4f13-bbde-9505fd3504e2 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] 10.222.0.26 "PUT /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes" status: 400 len: 117 microversion: 2.1 time: 0.023643 Sep 27 09:56:29.113239 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 23/33] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 09:56:29 2022] PUT /compute/v2.1/flavors/506933704/os-extra_specs/hw:numa_nodes => generated 117 bytes in 34 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 09:56:29.124123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec872ce2-af44-4cce-99dd-c286ff8b635c tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:29.167335 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da3ca238-adc5-49c2-9446-3212971ca815 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.082253 Sep 27 09:56:29.173646 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ec872ce2-af44-4cce-99dd-c286ff8b635c tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/506933704" status: 202 len: 0 microversion: 2.1 time: 0.055560 Sep 27 09:56:29.176671 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 11/34] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:56:29 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:29.181897 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 24/35] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 09:56:29 2022] DELETE /compute/v2.1/flavors/506933704 => generated 0 bytes in 68 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:56:29.183340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c33fbcad-d85a-43d8-8410-1c92eeb63bb9 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "tempest-nonexist_host-739691337"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:29.192084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a012cb6e-fbf5-40a5-89e1-9a027d090c16 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:29.196807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c33fbcad-d85a-43d8-8410-1c92eeb63bb9 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:29.197310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c33fbcad-d85a-43d8-8410-1c92eeb63bb9 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:29.197753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c33fbcad-d85a-43d8-8410-1c92eeb63bb9 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:29.208695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c33fbcad-d85a-43d8-8410-1c92eeb63bb9 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:29.209436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c33fbcad-d85a-43d8-8410-1c92eeb63bb9 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:29.209909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c33fbcad-d85a-43d8-8410-1c92eeb63bb9 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:29.229040 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-c33fbcad-d85a-43d8-8410-1c92eeb63bb9 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] HTTP exception thrown: Compute host tempest-nonexist_host-739691337 could not be found. Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c33fbcad-d85a-43d8-8410-1c92eeb63bb9 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Request method failure captured: Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-aggregates/4/action HTTP/1.1 Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 57 Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjH7QYCLZugeJ0HemgmqlSpvzQrDapyIqrhuEZ_iPs-RyHoyEOHHpkZWyJeIyr1hcj35JJZjvQUwiTglpRf4FVRuf_GUwsYg1BP_B9dV7Pni9aC2IYru9U0UypZozOkcJALyLAHHjO5hrSMbhlJr0kvWgGPYkLcuS5TmJ2XrkKwfYg Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:56:29.235673 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fe46a7ae8eab451e941f83e19be3b539"}]}] Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 49f35168484b42feb22d106d745e6546 Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: {"add_host": {"host": "tempest-nonexist_host-739691337"}} Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:56:29.236917 ncdv-master-1 devstack@n-api.service[145968]: exception: Compute host tempest-nonexist_host-739691337 could not be found. Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '4', 'body': {'add_host': {'host': 'tempest-nonexist_host-739691337'}}} Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5999, in _get_service_in_cell_by_host Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host_name) Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'tempest-nonexist_host-739691337' is not mapped to any cell Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.237559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6005, in _get_service_in_cell_by_host Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi service = _find_service_in_cell(context, service_host=host_name) Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6067, in _find_service_in_cell Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 180, in _add_host Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi aggregate = self.api.add_host_to_aggregate(context, id, host) Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6621, in add_host_to_aggregate Sep 27 09:56:29.238484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi service = _get_service_in_cell_by_host(context, host_name) Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6007, in _get_service_in_cell_by_host Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ComputeHostNotFound(host=host_name) Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ComputeHostNotFound: Compute host tempest-nonexist_host-739691337 could not be found. Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 184, in _add_host Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Compute host tempest-nonexist_host-739691337 could not be found. Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.239687 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.241148 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.241148 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.241148 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.241148 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:29.241148 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:29.241148 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Compute host tempest-nonexist_host-739691337 could not be found. Sep 27 09:56:29.241148 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.241148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c33fbcad-d85a-43d8-8410-1c92eeb63bb9 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Returning 404 to user: Compute host tempest-nonexist_host-739691337 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:29.241148 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c33fbcad-d85a-43d8-8410-1c92eeb63bb9 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/4/action" status: 404 len: 110 microversion: 2.1 time: 0.059433 Sep 27 09:56:29.244069 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 12/36] 10.222.0.26 () {60 vars in 1191 bytes} [Tue Sep 27 09:56:29 2022] POST /compute/v2.1/os-aggregates/4/action => generated 110 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:29.251804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-955f521f-dcc8-4073-8c2b-281f827fb4fc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:29.286203 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a012cb6e-fbf5-40a5-89e1-9a027d090c16 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552 tempest-FlavorsExtraSpecsNegativeTestJSON-1049126552-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.101728 Sep 27 09:56:29.287729 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 25/37] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:56:29 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:29.307220 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-955f521f-dcc8-4073-8c2b-281f827fb4fc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/4" status: 200 len: 0 microversion: 2.1 time: 0.061546 Sep 27 09:56:29.308319 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 13/38] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:56:29 2022] DELETE /compute/v2.1/os-aggregates/4 => generated 0 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:56:29.319733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e53d5673-630d-42cc-9958-8aa4ed6d1578 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e53d5673-630d-42cc-9958-8aa4ed6d1578 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Request method failure captured: Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 283 Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjH7QYCLZugeJ0HemgmqlSpvzQrDapyIqrhuEZ_iPs-RyHoyEOHHpkZWyJeIyr1hcj35JJZjvQUwiTglpRf4FVRuf_GUwsYg1BP_B9dV7Pni9aC2IYru9U0UypZozOkcJALyLAHHjO5hrSMbhlJr0kvWgGPYkLcuS5TmJ2XrkKwfYg Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:29.324601 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:29.325712 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fe46a7ae8eab451e941f83e19be3b539"}]}] Sep 27 09:56:29.325712 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.325712 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:29.325712 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.325712 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:29.325712 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:29.325712 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:29.325712 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 49f35168484b42feb22d106d745e6546 Sep 27 09:56:29.325712 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:29.325712 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:29.326223 ncdv-master-1 devstack@n-api.service[145969]: {"aggregate": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} Sep 27 09:56:29.326223 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:29.326223 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 09:56:29.326223 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'aggregate': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'}}} Sep 27 09:56:29.326223 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:29.326223 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.326223 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 09:56:29.326223 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 09:56:29.326223 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 09:56:29.326223 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['aggregate']['properties']['name']: Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['aggregate']: Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:29.326738 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.327554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.328291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 09:56:29.328291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 09:56:29.328291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 09:56:29.328291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.330074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e53d5673-630d-42cc-9958-8aa4ed6d1578 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:29.331263 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e53d5673-630d-42cc-9958-8aa4ed6d1578 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 400 len: 619 microversion: 2.1 time: 0.015967 Sep 27 09:56:29.333343 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 26/39] 10.222.0.26 () {60 vars in 1165 bytes} [Tue Sep 27 09:56:29 2022] POST /compute/v2.1/os-aggregates => generated 619 bytes in 18 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 09:56:29.345587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7841ca75-7f3d-466b-971f-46117c7cf832 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": ""}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7841ca75-7f3d-466b-971f-46117c7cf832 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Request method failure captured: Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 27 Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjH7QYCLZugeJ0HemgmqlSpvzQrDapyIqrhuEZ_iPs-RyHoyEOHHpkZWyJeIyr1hcj35JJZjvQUwiTglpRf4FVRuf_GUwsYg1BP_B9dV7Pni9aC2IYru9U0UypZozOkcJALyLAHHjO5hrSMbhlJr0kvWgGPYkLcuS5TmJ2XrkKwfYg Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:56:29.350679 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fe46a7ae8eab451e941f83e19be3b539"}]}] Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 49f35168484b42feb22d106d745e6546 Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: {"aggregate": {"name": ""}} Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:56:29.352049 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute name. Value: . '' is too short Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'aggregate': {'name': ''}}} Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['aggregate']['properties']['name']: Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['aggregate']: Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi '' Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:29.352785 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Sep 27 09:56:29.353925 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.358472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7841ca75-7f3d-466b-971f-46117c7cf832 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:29.360378 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7841ca75-7f3d-466b-971f-46117c7cf832 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 400 len: 108 microversion: 2.1 time: 0.019231 Sep 27 09:56:29.362103 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 14/40] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:56:29 2022] POST /compute/v2.1/os-aggregates => generated 108 bytes in 22 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 09:56:29.374520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9c5f992-b5c2-4f95-a7c9-2c0d2b93134a tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1389574540"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:29.378701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c9c5f992-b5c2-4f95-a7c9-2c0d2b93134a tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Policy check for os_compute_api:os-aggregates:create failed with credentials {'is_admin': False, 'user_id': '8ea0f9af73d5435088120d793a61bef3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5a71603028ff42b4847878e7ede6aebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9c5f992-b5c2-4f95-a7c9-2c0d2b93134a tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Request method failure captured: Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 60 Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjFzty-qLENwwC0WJRfeIxzeZV65xTjvO7a5eLb6pj1D2B1dsP_rK8RVeimECsxHNyUllFJgmBOhq-mF9B6V9NztZc9XT1FhgvcpY2yNe42ftBLJ2Zi5O1gzeQf0ok5TU-3MikfFG4bnjZpU2GZ8UnwGjiq5xJIj1Lmd1RWDnpXTf4 Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:29.380541 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5a71603028ff42b4847878e7ede6aebe"}]}] Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 8ea0f9af73d5435088120d793a61bef3 Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: {"aggregate": {"name": "tempest-test_aggregate-1389574540"}} Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:29.381685 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'aggregate': {'name': 'tempest-test_aggregate-1389574540'}}} Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 70, in create Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'create', target={}) Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:29.382620 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:29.383554 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. Sep 27 09:56:29.384515 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.390270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9c5f992-b5c2-4f95-a7c9-2c0d2b93134a tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:create to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:29.395646 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9c5f992-b5c2-4f95-a7c9-2c0d2b93134a tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 403 len: 116 microversion: 2.1 time: 0.023193 Sep 27 09:56:29.404023 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 27/41] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:56:29 2022] POST /compute/v2.1/os-aggregates => generated 116 bytes in 34 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:29.415524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8429371b-cd19-4cff-ae35-91019afbfa0f tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1951416643"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:29.422523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-8429371b-cd19-4cff-ae35-91019afbfa0f tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Generated uuid 790c413b-6e78-4d75-bb84-b68a4ed040f1 for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:56:29.478455 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8429371b-cd19-4cff-ae35-91019afbfa0f tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.066706 Sep 27 09:56:29.480747 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 15/42] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:56:29 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:29.498219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-37c30a90-e7b7-4911-95bf-60aed57885e2 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1951416643"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:29.508897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-37c30a90-e7b7-4911-95bf-60aed57885e2 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Generated uuid 3e3afda3-6985-4117-8f8b-c1549518dedd for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:56:29.537998 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-37c30a90-e7b7-4911-95bf-60aed57885e2 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] HTTP exception thrown: Aggregate tempest-test_aggregate-1951416643 already exists. Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-37c30a90-e7b7-4911-95bf-60aed57885e2 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Request method failure captured: Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-aggregates HTTP/1.1 Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 60 Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjH7QYCLZugeJ0HemgmqlSpvzQrDapyIqrhuEZ_iPs-RyHoyEOHHpkZWyJeIyr1hcj35JJZjvQUwiTglpRf4FVRuf_GUwsYg1BP_B9dV7Pni9aC2IYru9U0UypZozOkcJALyLAHHjO5hrSMbhlJr0kvWgGPYkLcuS5TmJ2XrkKwfYg Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:56:29.545161 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fe46a7ae8eab451e941f83e19be3b539"}]}] Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 49f35168484b42feb22d106d745e6546 Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: {"aggregate": {"name": "tempest-test_aggregate-1951416643"}} Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:56:29.546686 ncdv-master-1 devstack@n-api.service[145968]: exception: Aggregate tempest-test_aggregate-1951416643 already exists. Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'aggregate': {'name': 'tempest-test_aggregate-1951416643'}}} Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 78, in create Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi aggregate = self.api.create_aggregate(context, name, avail_zone) Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6458, in create_aggregate Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi aggregate.create() Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 310, in create Sep 27 09:56:29.547499 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_create_in_db(self._context, updates, Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 175, in _aggregate_create_in_db Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.AggregateNameExists(aggregate_name=values['name']) Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNameExists: Aggregate tempest-test_aggregate-1951416643 already exists. Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:29.548757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 80, in create Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPConflict(explanation=e.format_message()) Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Aggregate tempest-test_aggregate-1951416643 already exists. Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate tempest-test_aggregate-1951416643 already exists. Sep 27 09:56:29.549964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.557619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-37c30a90-e7b7-4911-95bf-60aed57885e2 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Returning 409 to user: Aggregate tempest-test_aggregate-1951416643 already exists. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:29.559714 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-37c30a90-e7b7-4911-95bf-60aed57885e2 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 409 len: 111 microversion: 2.1 time: 0.071892 Sep 27 09:56:29.561384 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 16/43] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:56:29 2022] POST /compute/v2.1/os-aggregates => generated 111 bytes in 75 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Sep 27 09:56:29.574824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba3879c5-1a3a-4460-bf75-044277760bd7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:29.618865 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba3879c5-1a3a-4460-bf75-044277760bd7 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/5" status: 200 len: 0 microversion: 2.1 time: 0.052191 Sep 27 09:56:29.621681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 17/44] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:56:29 2022] DELETE /compute/v2.1/os-aggregates/5 => generated 0 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 355 bytes (2 switches on core 0) Sep 27 09:56:29.633195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-98aaea49-acec-4aaa-b957-ef2c28720958 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-888403289"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:29.638724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-98aaea49-acec-4aaa-b957-ef2c28720958 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Generated uuid c64a8f3a-6246-42e8-bb75-cc4e36205ac7 for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:56:29.679475 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-98aaea49-acec-4aaa-b957-ef2c28720958 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 197 microversion: 2.1 time: 0.050562 Sep 27 09:56:29.695789 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 18/45] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:56:29 2022] POST /compute/v2.1/os-aggregates => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:29.710250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c46bd710-1090-4753-9bf5-c91403877c24 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:29.715011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c46bd710-1090-4753-9bf5-c91403877c24 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Policy check for os_compute_api:os-aggregates:delete failed with credentials {'is_admin': False, 'user_id': '8ea0f9af73d5435088120d793a61bef3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5a71603028ff42b4847878e7ede6aebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c46bd710-1090-4753-9bf5-c91403877c24 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Request method failure captured: Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-aggregates/6 HTTP/1.1 Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjFzty-qLENwwC0WJRfeIxzeZV65xTjvO7a5eLb6pj1D2B1dsP_rK8RVeimECsxHNyUllFJgmBOhq-mF9B6V9NztZc9XT1FhgvcpY2yNe42ftBLJ2Zi5O1gzeQf0ok5TU-3MikfFG4bnjZpU2GZ8UnwGjiq5xJIj1Lmd1RWDnpXTf4 Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:56:29.721691 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5a71603028ff42b4847878e7ede6aebe"}]}] Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 8ea0f9af73d5435088120d793a61bef3 Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: exception: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. Sep 27 09:56:29.722834 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '6'} Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 147, in delete Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'delete', target={}) Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:29.724075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. Sep 27 09:56:29.725179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.728409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c46bd710-1090-4753-9bf5-c91403877c24 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:delete to be performed. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:29.732993 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c46bd710-1090-4753-9bf5-c91403877c24 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/6" status: 403 len: 116 microversion: 2.1 time: 0.033374 Sep 27 09:56:29.734224 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 19/46] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:56:29 2022] DELETE /compute/v2.1/os-aggregates/6 => generated 116 bytes in 38 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:29.744713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-de6aaa8c-ecb4-467d-819f-2a0dd9f8998c tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:29.797390 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-de6aaa8c-ecb4-467d-819f-2a0dd9f8998c tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/6" status: 200 len: 0 microversion: 2.1 time: 0.057189 Sep 27 09:56:29.802504 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 20/47] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:56:29 2022] DELETE /compute/v2.1/os-aggregates/6 => generated 0 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 355 bytes (2 switches on core 0) Sep 27 09:56:29.812972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-65bbf92c-823e-4c83-aff8-cb4819b6326d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:29.846477 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-65bbf92c-823e-4c83-aff8-cb4819b6326d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] HTTP exception thrown: Aggregate -1 could not be found. Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-65bbf92c-823e-4c83-aff8-cb4819b6326d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Request method failure captured: Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-aggregates/-1 HTTP/1.1 Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjH7QYCLZugeJ0HemgmqlSpvzQrDapyIqrhuEZ_iPs-RyHoyEOHHpkZWyJeIyr1hcj35JJZjvQUwiTglpRf4FVRuf_GUwsYg1BP_B9dV7Pni9aC2IYru9U0UypZozOkcJALyLAHHjO5hrSMbhlJr0kvWgGPYkLcuS5TmJ2XrkKwfYg Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:56:29.848163 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fe46a7ae8eab451e941f83e19be3b539"}]}] Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 49f35168484b42feb22d106d745e6546 Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: exception: Aggregate -1 could not be found. Sep 27 09:56:29.849814 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '-1'} Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 155, in delete Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.api.delete_aggregate(context, id) Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6517, in delete_aggregate Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi aggregate = objects.Aggregate.get_by_id(context, aggregate_id) Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Sep 27 09:56:29.850645 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate -1 could not be found. Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 157, in delete Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate -1 could not be found. Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.852227 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.854450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.854450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.854450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:29.854450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:29.854450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate -1 could not be found. Sep 27 09:56:29.854450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.854450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-65bbf92c-823e-4c83-aff8-cb4819b6326d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Returning 404 to user: Aggregate -1 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:29.854450 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-65bbf92c-823e-4c83-aff8-cb4819b6326d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/-1" status: 404 len: 78 microversion: 2.1 time: 0.043933 Sep 27 09:56:29.856046 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 21/48] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:56:29 2022] DELETE /compute/v2.1/os-aggregates/-1 => generated 78 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Sep 27 09:56:29.869278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2f88c2e3-510b-4ac1-9efe-46846a3d810d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1654867212"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:29.877471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-2f88c2e3-510b-4ac1-9efe-46846a3d810d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Generated uuid 68b097c9-17bf-4887-bceb-06a707267cf0 for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:56:29.901313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=7,nova_api:DELETE=9,nova_api:SAVEPOINT=2,nova_api:RELEASE=2,nova_api:UPDATE=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:29.927091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2f88c2e3-510b-4ac1-9efe-46846a3d810d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.065970 Sep 27 09:56:29.928164 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 22/49] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:56:29 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:29.944436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41329c20-d43e-4792-b6aa-f72b45e98972 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:29.946326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-41329c20-d43e-4792-b6aa-f72b45e98972 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Policy check for os_compute_api:os-aggregates:show failed with credentials {'is_admin': False, 'user_id': '8ea0f9af73d5435088120d793a61bef3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5a71603028ff42b4847878e7ede6aebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41329c20-d43e-4792-b6aa-f72b45e98972 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Request method failure captured: Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-aggregates/7 HTTP/1.1 Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjFzty-qLENwwC0WJRfeIxzeZV65xTjvO7a5eLb6pj1D2B1dsP_rK8RVeimECsxHNyUllFJgmBOhq-mF9B6V9NztZc9XT1FhgvcpY2yNe42ftBLJ2Zi5O1gzeQf0ok5TU-3MikfFG4bnjZpU2GZ8UnwGjiq5xJIj1Lmd1RWDnpXTf4 Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:56:29.949637 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5a71603028ff42b4847878e7ede6aebe"}]}] Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 8ea0f9af73d5435088120d793a61bef3 Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: exception: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. Sep 27 09:56:29.950996 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '7'} Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 100, in show Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'show', target={}) Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:29.951766 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:29.952966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41329c20-d43e-4792-b6aa-f72b45e98972 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:show to be performed. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:29.954031 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-41329c20-d43e-4792-b6aa-f72b45e98972 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] 10.222.0.26 "GET /compute/v2.1/os-aggregates/7" status: 403 len: 114 microversion: 2.1 time: 0.015234 Sep 27 09:56:29.955216 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 23/50] 10.222.0.26 () {58 vars in 1149 bytes} [Tue Sep 27 09:56:29 2022] GET /compute/v2.1/os-aggregates/7 => generated 114 bytes in 19 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:29.962932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-535b8b94-f653-43f3-a02e-8c472544a916 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:30.001209 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-535b8b94-f653-43f3-a02e-8c472544a916 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/7" status: 200 len: 0 microversion: 2.1 time: 0.042356 Sep 27 09:56:30.007992 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 24/51] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:56:29 2022] DELETE /compute/v2.1/os-aggregates/7 => generated 0 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 355 bytes (2 switches on core 0) Sep 27 09:56:30.014922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-18953c4b-d02a-439a-986b-810f1e28ed3d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:30.023712 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-18953c4b-d02a-439a-986b-810f1e28ed3d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] HTTP exception thrown: Aggregate -1 could not be found. Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-18953c4b-d02a-439a-986b-810f1e28ed3d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Request method failure captured: Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-aggregates/-1 HTTP/1.1 Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjH7QYCLZugeJ0HemgmqlSpvzQrDapyIqrhuEZ_iPs-RyHoyEOHHpkZWyJeIyr1hcj35JJZjvQUwiTglpRf4FVRuf_GUwsYg1BP_B9dV7Pni9aC2IYru9U0UypZozOkcJALyLAHHjO5hrSMbhlJr0kvWgGPYkLcuS5TmJ2XrkKwfYg Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:56:30.025553 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fe46a7ae8eab451e941f83e19be3b539"}]}] Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 49f35168484b42feb22d106d745e6546 Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: exception: Aggregate -1 could not be found. Sep 27 09:56:30.027193 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '-1'} Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 108, in show Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi aggregate = self.api.get_aggregate(context, id) Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6464, in get_aggregate Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return objects.Aggregate.get_by_id(context, aggregate_id) Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate -1 could not be found. Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:30.028275 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 110, in show Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate -1 could not be found. Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate -1 could not be found. Sep 27 09:56:30.029608 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:30.030850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-18953c4b-d02a-439a-986b-810f1e28ed3d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Returning 404 to user: Aggregate -1 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:30.030850 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-18953c4b-d02a-439a-986b-810f1e28ed3d tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates/-1" status: 404 len: 78 microversion: 2.1 time: 0.017767 Sep 27 09:56:30.038353 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 25/52] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:56:30 2022] GET /compute/v2.1/os-aggregates/-1 => generated 78 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:56:30.044475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-01c37f19-6827-4736-bedc-7585611c2ec8 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:30.046113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-01c37f19-6827-4736-bedc-7585611c2ec8 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Policy check for os_compute_api:os-aggregates:index failed with credentials {'is_admin': False, 'user_id': '8ea0f9af73d5435088120d793a61bef3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5a71603028ff42b4847878e7ede6aebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-01c37f19-6827-4736-bedc-7585611c2ec8 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Request method failure captured: Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-aggregates HTTP/1.1 Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjFzty-qLENwwC0WJRfeIxzeZV65xTjvO7a5eLb6pj1D2B1dsP_rK8RVeimECsxHNyUllFJgmBOhq-mF9B6V9NztZc9XT1FhgvcpY2yNe42ftBLJ2Zi5O1gzeQf0ok5TU-3MikfFG4bnjZpU2GZ8UnwGjiq5xJIj1Lmd1RWDnpXTf4 Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:56:30.047669 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5a71603028ff42b4847878e7ede6aebe"}]}] Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 8ea0f9af73d5435088120d793a61bef3 Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: exception: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. Sep 27 09:56:30.049054 ncdv-master-1 devstack@n-api.service[145968]: action_args: {} Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 55, in index Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'index', target={}) Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:30.049671 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:30.050594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-01c37f19-6827-4736-bedc-7585611c2ec8 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:index to be performed. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:30.051794 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-01c37f19-6827-4736-bedc-7585611c2ec8 tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] 10.222.0.26 "GET /compute/v2.1/os-aggregates" status: 403 len: 115 microversion: 2.1 time: 0.009503 Sep 27 09:56:30.055709 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 26/53] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 09:56:30 2022] GET /compute/v2.1/os-aggregates => generated 115 bytes in 16 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:30.459923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AggregatesAdminTestJSON-server-1523307210", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "availability_zone": "tempest-test_az-1086642901"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:30.481862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:30.486268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:30.486740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:30.487549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:30.492532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:30.494412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:31.834699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:56:32.186193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:56:32.186624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:56:32.187435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:56:32.188198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:56:32.188825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:56:32.203932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:32.204498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:32.205111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:32.206116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:32.206751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:32.207254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:32.241912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Getting quotas for project 681fc4282a6c4fd8ba4f078705f4ca5c. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:56:32.247593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Getting quotas for user 023730e34b2d4e2a996bfa068a97413c and project 681fc4282a6c4fd8ba4f078705f4ca5c. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:56:32.260464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:56:32.263683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:56:32.338980 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-49c10264-76ca-441e-9732-619232800661 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.878103 Sep 27 09:56:32.344969 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 28/54] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:56:29 2022] POST /compute/v2.1/servers => generated 380 bytes in 2882 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:56:32.353030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e300b7fe-7f76-4f91-afbf-1fa7edb8a3e6 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:32.353785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e300b7fe-7f76-4f91-afbf-1fa7edb8a3e6 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:33.830585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e300b7fe-7f76-4f91-afbf-1fa7edb8a3e6 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:33.856632 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e300b7fe-7f76-4f91-afbf-1fa7edb8a3e6 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1243 microversion: 2.1 time: 1.509184 Sep 27 09:56:33.857703 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 27/55] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:32 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1243 bytes in 1511 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:34.882561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-daab02dd-6283-4ade-904c-f50d3857d90c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:34.883188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-daab02dd-6283-4ade-904c-f50d3857d90c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:34.920287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daab02dd-6283-4ade-904c-f50d3857d90c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:34.920830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daab02dd-6283-4ade-904c-f50d3857d90c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:34.921286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daab02dd-6283-4ade-904c-f50d3857d90c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:34.922007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daab02dd-6283-4ade-904c-f50d3857d90c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:34.922474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daab02dd-6283-4ade-904c-f50d3857d90c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:34.936211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daab02dd-6283-4ade-904c-f50d3857d90c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:35.304737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-daab02dd-6283-4ade-904c-f50d3857d90c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:35.325186 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-daab02dd-6283-4ade-904c-f50d3857d90c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1291 microversion: 2.1 time: 0.446835 Sep 27 09:56:35.326655 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 29/56] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:34 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1291 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:36.350445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-03828578-426c-4b81-bd7b-218b56b81785 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:36.352293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-03828578-426c-4b81-bd7b-218b56b81785 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:36.369428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03828578-426c-4b81-bd7b-218b56b81785 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:36.370921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03828578-426c-4b81-bd7b-218b56b81785 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:36.372550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03828578-426c-4b81-bd7b-218b56b81785 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:36.375053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03828578-426c-4b81-bd7b-218b56b81785 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:36.375917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03828578-426c-4b81-bd7b-218b56b81785 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:36.377856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03828578-426c-4b81-bd7b-218b56b81785 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:36.658814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03828578-426c-4b81-bd7b-218b56b81785 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:36.688430 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-03828578-426c-4b81-bd7b-218b56b81785 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1297 microversion: 2.1 time: 0.340980 Sep 27 09:56:36.690632 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 28/57] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:36 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1297 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:37.107382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-21b6b561-bf45-416f-8628-48ddb2dfb5c8 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Calling method '.version_select at 0x7f7f682f7790>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:37.111056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21b6b561-bf45-416f-8628-48ddb2dfb5c8 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:37.111558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21b6b561-bf45-416f-8628-48ddb2dfb5c8 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:37.111991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21b6b561-bf45-416f-8628-48ddb2dfb5c8 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:37.112775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21b6b561-bf45-416f-8628-48ddb2dfb5c8 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:37.113241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21b6b561-bf45-416f-8628-48ddb2dfb5c8 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:37.113671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21b6b561-bf45-416f-8628-48ddb2dfb5c8 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:37.195569 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-21b6b561-bf45-416f-8628-48ddb2dfb5c8 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts" status: 200 len: 401 microversion: 2.1 time: 0.738442 Sep 27 09:56:37.206239 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 30/58] 10.222.0.26 () {58 vars in 1128 bytes} [Tue Sep 27 09:56:36 2022] GET /compute/v2.1/os-hosts => generated 401 bytes in 751 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:56:37.224235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eb50fd78-fbbb-4e12-a82d-0a670e8063fc tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Calling method '.version_select at 0x7f7f684d89d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:37.250050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb50fd78-fbbb-4e12-a82d-0a670e8063fc tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:37.256282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb50fd78-fbbb-4e12-a82d-0a670e8063fc tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:37.262286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb50fd78-fbbb-4e12-a82d-0a670e8063fc tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:37.266781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb50fd78-fbbb-4e12-a82d-0a670e8063fc tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:37.269335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb50fd78-fbbb-4e12-a82d-0a670e8063fc tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:37.269900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb50fd78-fbbb-4e12-a82d-0a670e8063fc tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:37.730306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-94ccd4b6-7fbf-4033-8d1a-a9c13c814eec tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:37.730956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-94ccd4b6-7fbf-4033-8d1a-a9c13c814eec tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:37.740426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94ccd4b6-7fbf-4033-8d1a-a9c13c814eec tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:37.741285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94ccd4b6-7fbf-4033-8d1a-a9c13c814eec tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:37.743013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94ccd4b6-7fbf-4033-8d1a-a9c13c814eec tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:37.743989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94ccd4b6-7fbf-4033-8d1a-a9c13c814eec tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:37.744683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94ccd4b6-7fbf-4033-8d1a-a9c13c814eec tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:37.745316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94ccd4b6-7fbf-4033-8d1a-a9c13c814eec tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:37.857480 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eb50fd78-fbbb-4e12-a82d-0a670e8063fc tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts?zone=" status: 200 len: 401 microversion: 2.1 time: 0.641925 Sep 27 09:56:37.870057 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 29/59] 10.222.0.26 () {58 vars in 1139 bytes} [Tue Sep 27 09:56:37 2022] GET /compute/v2.1/os-hosts?zone= => generated 401 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:37.887037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-336726be-0825-45e9-8413-c5ae298d5981 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Calling method '.version_select at 0x7f7f698f5e50>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:37.890693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336726be-0825-45e9-8413-c5ae298d5981 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:37.904937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-94ccd4b6-7fbf-4033-8d1a-a9c13c814eec tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:37.913000 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-94ccd4b6-7fbf-4033-8d1a-a9c13c814eec tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1297 microversion: 2.1 time: 0.194344 Sep 27 09:56:37.913869 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 31/60] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:37 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1297 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:37.917059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336726be-0825-45e9-8413-c5ae298d5981 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:37.917646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336726be-0825-45e9-8413-c5ae298d5981 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.026s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:37.920985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336726be-0825-45e9-8413-c5ae298d5981 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:37.921480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336726be-0825-45e9-8413-c5ae298d5981 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:37.921918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336726be-0825-45e9-8413-c5ae298d5981 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:38.051043 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-336726be-0825-45e9-8413-c5ae298d5981 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts?zone=xxx" status: 200 len: 13 microversion: 2.1 time: 0.179250 Sep 27 09:56:38.072964 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 30/61] 10.222.0.26 () {58 vars in 1145 bytes} [Tue Sep 27 09:56:37 2022] GET /compute/v2.1/os-hosts?zone=xxx => generated 13 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:56:38.778764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea937371-70e3-4157-981b-1bb5b31d0b80 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Calling method '.version_select at 0x7f7f682b8c10>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:39.169329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-55a2bc07-5863-4dd5-8070-2e54b86bb2b5 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Calling method '.version_select at 0x7f7f6886e670>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:39.171757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55a2bc07-5863-4dd5-8070-2e54b86bb2b5 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:39.172378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55a2bc07-5863-4dd5-8070-2e54b86bb2b5 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:39.173175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55a2bc07-5863-4dd5-8070-2e54b86bb2b5 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:39.207074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55a2bc07-5863-4dd5-8070-2e54b86bb2b5 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:39.207690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55a2bc07-5863-4dd5-8070-2e54b86bb2b5 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:39.208447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55a2bc07-5863-4dd5-8070-2e54b86bb2b5 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:39.265464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55a2bc07-5863-4dd5-8070-2e54b86bb2b5 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:39.270651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55a2bc07-5863-4dd5-8070-2e54b86bb2b5 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:39.272111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55a2bc07-5863-4dd5-8070-2e54b86bb2b5 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:39.294672 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-55a2bc07-5863-4dd5-8070-2e54b86bb2b5 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.483049 Sep 27 09:56:39.301152 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 31/62] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:56:38 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:56:39.336548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9420dc45-6e88-49dc-9eae-478e3e559d15 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:39.345812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9420dc45-6e88-49dc-9eae-478e3e559d15 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:39.362302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9420dc45-6e88-49dc-9eae-478e3e559d15 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:39.364084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9420dc45-6e88-49dc-9eae-478e3e559d15 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:39.366056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9420dc45-6e88-49dc-9eae-478e3e559d15 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:39.367120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9420dc45-6e88-49dc-9eae-478e3e559d15 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:39.369293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9420dc45-6e88-49dc-9eae-478e3e559d15 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:39.370554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9420dc45-6e88-49dc-9eae-478e3e559d15 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:39.425807 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea937371-70e3-4157-981b-1bb5b31d0b80 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d" status: 200 len: 371 microversion: 2.1 time: 1.287603 Sep 27 09:56:39.429395 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 32/63] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 09:56:38 2022] GET /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d => generated 371 bytes in 1292 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:39.569587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a07a013-65f7-4ba6-9ace-67060fc2210b tempest-FlavorsV255TestJSON-1203847512 tempest-FlavorsV255TestJSON-1203847512-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsV255TestJSON-flavor-1212575476", "id": 1345800171, "os-flavor-access:is_public": "True"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:39.634053 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a07a013-65f7-4ba6-9ace-67060fc2210b tempest-FlavorsV255TestJSON-1203847512 tempest-FlavorsV255TestJSON-1203847512-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 449 microversion: 2.55 time: 0.201540 Sep 27 09:56:39.637715 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 33/64] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 09:56:39 2022] POST /compute/v2.1/flavors => generated 449 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:56:39.936774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e9b02c3f-81b4-4f49-90bd-b3256a7bdaf2 tempest-HypervisorAdminNegativeUnderV252Test-395320564 tempest-HypervisorAdminNegativeUnderV252Test-395320564-project] Calling method '.version_select at 0x7f7f68214790>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:39.940474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e9b02c3f-81b4-4f49-90bd-b3256a7bdaf2 tempest-HypervisorAdminNegativeUnderV252Test-395320564 tempest-HypervisorAdminNegativeUnderV252Test-395320564-project] Policy check for os_compute_api:os-hypervisors:search failed with credentials {'is_admin': False, 'user_id': '8daac732ab214a56a401a3487f2b820c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '246bd38c478e448cab936abd93fa094e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e9b02c3f-81b4-4f49-90bd-b3256a7bdaf2 tempest-HypervisorAdminNegativeUnderV252Test-395320564 tempest-HypervisorAdminNegativeUnderV252Test-395320564-project] Request method failure captured: Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hypervisors/nch1-master-1/search HTTP/1.1 Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjTy4V_NzQ_SsObJcVuU_XaIk0flqH5nH1WlOghCVjPubqsZOthHu7nxr5dXOZ2j92MHUzHXcUBzlsKeGzQF9HINNCYHwMNEAk2bKuyK5IDb1iN6zmxbHfMFpoPAxUILXw62PTOvx9ssmY700qXqomZlPwv9gVhyqh_9c5pV2Q9vxc Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 246bd38c478e448cab936abd93fa094e Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-395320564 Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:39.948019 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/246bd38c478e448cab936abd93fa094e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/246bd38c478e448cab936abd93fa094e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/246bd38c478e448cab936abd93fa094e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_246bd38c478e448cab936abd93fa094e"}]}] Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-395320564 Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 246bd38c478e448cab936abd93fa094e Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-395320564 Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-395320564-project Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 8daac732ab214a56a401a3487f2b820c Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-395320564-project Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. Sep 27 09:56:39.949500 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'nch1-master-1'} Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 439, in search Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'search', target={}) Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:39.950173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. Sep 27 09:56:39.952109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:39.956035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e9b02c3f-81b4-4f49-90bd-b3256a7bdaf2 tempest-HypervisorAdminNegativeUnderV252Test-395320564 tempest-HypervisorAdminNegativeUnderV252Test-395320564-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:search to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:39.957524 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e9b02c3f-81b4-4f49-90bd-b3256a7bdaf2 tempest-HypervisorAdminNegativeUnderV252Test-395320564 tempest-HypervisorAdminNegativeUnderV252Test-395320564-project] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/nch1-master-1/search" status: 403 len: 117 microversion: 2.1 time: 0.318377 Sep 27 09:56:39.958818 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 34/65] 10.222.0.26 () {58 vars in 1209 bytes} [Tue Sep 27 09:56:39 2022] GET /compute/v2.1/os-hypervisors/nch1-master-1/search => generated 117 bytes in 320 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:39.969231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d6bfd2e9-43da-42f1-bd3b-2a44e4066ad1 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Action: 'update', calling method: .version_select at 0x7f7f68285940>, body: {"quota_set": {"force": true, "cores": 0}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:40.064216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9420dc45-6e88-49dc-9eae-478e3e559d15 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:40.095406 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9420dc45-6e88-49dc-9eae-478e3e559d15 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.789548 Sep 27 09:56:40.098858 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 32/66] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:39 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 796 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:40.175165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6bfd2e9-43da-42f1-bd3b-2a44e4066ad1 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:40.176134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6bfd2e9-43da-42f1-bd3b-2a44e4066ad1 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:40.177151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6bfd2e9-43da-42f1-bd3b-2a44e4066ad1 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:40.178549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6bfd2e9-43da-42f1-bd3b-2a44e4066ad1 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:40.179345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6bfd2e9-43da-42f1-bd3b-2a44e4066ad1 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:40.180288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6bfd2e9-43da-42f1-bd3b-2a44e4066ad1 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:40.330640 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d6bfd2e9-43da-42f1-bd3b-2a44e4066ad1 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d" status: 200 len: 328 microversion: 2.1 time: 0.370339 Sep 27 09:56:40.339484 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 35/67] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 09:56:39 2022] PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d => generated 328 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:40.345587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-41e76890-fc05-41bc-a0f5-edbb15a1da6e tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Calling method '.version_select at 0x7f7f684a98b0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:40.354439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41e76890-fc05-41bc-a0f5-edbb15a1da6e tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:40.357249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41e76890-fc05-41bc-a0f5-edbb15a1da6e tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:40.358133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41e76890-fc05-41bc-a0f5-edbb15a1da6e tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:40.374754 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-41e76890-fc05-41bc-a0f5-edbb15a1da6e tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] HTTP exception thrown: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-41e76890-fc05-41bc-a0f5-edbb15a1da6e tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Request method failure captured: Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hypervisors/nonexistent_hypervisor_name/search HTTP/1.1 Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjWPYRnQ_6KM3wc48n9Q3MoH_pql_auIvhNuI_9EzFIw6gMrJCvSeWKJK2LEjntTZhZcXikyoVC23d8KXNJY6CPpBGqbO2WQ-fu9GW0nlahik4fkxPINJrut2JgPGO1EgXuv_1AsO5NDJmPQIKKLnvWMBohHEbEEE5zcuC3ZEePQuk Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 9f08504febac4265b33fed0f12095b45 Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-1469927779 Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:40.377602 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9f08504febac4265b33fed0f12095b45"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9f08504febac4265b33fed0f12095b45"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9f08504febac4265b33fed0f12095b45"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9f08504febac4265b33fed0f12095b45"}]}] Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-1469927779 Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 9f08504febac4265b33fed0f12095b45 Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-1469927779 Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 4b6ee505ea6345f4bf22f29cc5f359f4 Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: exception: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Sep 27 09:56:40.378749 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'nonexistent_hypervisor_name'} Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 443, in search Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi compute_nodes = self._get_compute_nodes_by_name_pattern(context, id) Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 139, in _get_compute_nodes_by_name_pattern Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:40.379393 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:40.380388 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:40.380388 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:40.380388 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:40.380388 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: No hypervisor matching 'nonexistent_hypervisor_name' could be found. Sep 27 09:56:40.380388 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:40.382773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-41e76890-fc05-41bc-a0f5-edbb15a1da6e tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Returning 404 to user: No hypervisor matching 'nonexistent_hypervisor_name' could be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:40.388947 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-41e76890-fc05-41bc-a0f5-edbb15a1da6e tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/nonexistent_hypervisor_name/search" status: 404 len: 114 microversion: 2.1 time: 0.045693 Sep 27 09:56:40.398081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa4e66ae-dbdf-4fb8-ac48-edf6e5c75ee0 tempest-FlavorsV255TestJSON-1599456155 tempest-FlavorsV255TestJSON-1599456155-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:40.405209 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 36/68] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 09:56:40 2022] GET /compute/v2.1/os-hypervisors/nonexistent_hypervisor_name/search => generated 114 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:40.415639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0084f1d6-fa07-4c67-ad5c-0412bdfc6d42 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Calling method '.version_select at 0x7f7f682cfc10>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:40.418860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0084f1d6-fa07-4c67-ad5c-0412bdfc6d42 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:40.422050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0084f1d6-fa07-4c67-ad5c-0412bdfc6d42 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:40.422745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0084f1d6-fa07-4c67-ad5c-0412bdfc6d42 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:40.441423 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fa4e66ae-dbdf-4fb8-ac48-edf6e5c75ee0 tempest-FlavorsV255TestJSON-1599456155 tempest-FlavorsV255TestJSON-1599456155-project] 10.222.0.26 "GET /compute/v2.1/flavors/1345800171" status: 200 len: 449 microversion: 2.55 time: 0.337805 Sep 27 09:56:40.444963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0084f1d6-fa07-4c67-ad5c-0412bdfc6d42 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:40.456884 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 33/69] 10.222.0.26 () {60 vars in 1199 bytes} [Tue Sep 27 09:56:40 2022] GET /compute/v2.1/flavors/1345800171 => generated 449 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:56:40.463983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eccd3b0d-de96-4754-988b-6938f3b0f942 tempest-FlavorsV255TestJSON-1203847512 tempest-FlavorsV255TestJSON-1203847512-project-admin] Action: 'update', calling method: .version_select at 0x7f7f69603670>, body: {"flavor": {"description": "new"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:40.465080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0084f1d6-fa07-4c67-ad5c-0412bdfc6d42 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:40.471006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0084f1d6-fa07-4c67-ad5c-0412bdfc6d42 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:40.499653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0084f1d6-fa07-4c67-ad5c-0412bdfc6d42 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:40.500492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0084f1d6-fa07-4c67-ad5c-0412bdfc6d42 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:40.503382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0084f1d6-fa07-4c67-ad5c-0412bdfc6d42 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:40.515434 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0084f1d6-fa07-4c67-ad5c-0412bdfc6d42 tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.103920 Sep 27 09:56:40.519381 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 37/70] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:56:40 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:40.529378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2099bcbf-3ba4-4678-bc38-f6dabe4ca6b4 tempest-HypervisorAdminNegativeUnderV252Test-395320564 tempest-HypervisorAdminNegativeUnderV252Test-395320564-project] Calling method '.version_select at 0x7f7f682cfa60>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:40.534042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2099bcbf-3ba4-4678-bc38-f6dabe4ca6b4 tempest-HypervisorAdminNegativeUnderV252Test-395320564 tempest-HypervisorAdminNegativeUnderV252Test-395320564-project] Policy check for os_compute_api:os-hypervisors:servers failed with credentials {'is_admin': False, 'user_id': '8daac732ab214a56a401a3487f2b820c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '246bd38c478e448cab936abd93fa094e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2099bcbf-3ba4-4678-bc38-f6dabe4ca6b4 tempest-HypervisorAdminNegativeUnderV252Test-395320564 tempest-HypervisorAdminNegativeUnderV252Test-395320564-project] Request method failure captured: Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hypervisors/1/servers HTTP/1.1 Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjTy4V_NzQ_SsObJcVuU_XaIk0flqH5nH1WlOghCVjPubqsZOthHu7nxr5dXOZ2j92MHUzHXcUBzlsKeGzQF9HINNCYHwMNEAk2bKuyK5IDb1iN6zmxbHfMFpoPAxUILXw62PTOvx9ssmY700qXqomZlPwv9gVhyqh_9c5pV2Q9vxc Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 246bd38c478e448cab936abd93fa094e Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-395320564 Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:40.538669 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/246bd38c478e448cab936abd93fa094e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/246bd38c478e448cab936abd93fa094e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/246bd38c478e448cab936abd93fa094e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_246bd38c478e448cab936abd93fa094e"}]}] Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-395320564 Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 246bd38c478e448cab936abd93fa094e Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-395320564 Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-395320564-project Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 8daac732ab214a56a401a3487f2b820c Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-395320564-project Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. Sep 27 09:56:40.539882 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1'} Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 478, in servers Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'servers', target={}) Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:40.540489 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. Sep 27 09:56:40.541492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:40.546108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2099bcbf-3ba4-4678-bc38-f6dabe4ca6b4 tempest-HypervisorAdminNegativeUnderV252Test-395320564 tempest-HypervisorAdminNegativeUnderV252Test-395320564-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:servers to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:40.548100 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2099bcbf-3ba4-4678-bc38-f6dabe4ca6b4 tempest-HypervisorAdminNegativeUnderV252Test-395320564 tempest-HypervisorAdminNegativeUnderV252Test-395320564-project] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/1/servers" status: 403 len: 118 microversion: 2.1 time: 0.022631 Sep 27 09:56:40.550714 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 38/71] 10.222.0.26 () {58 vars in 1176 bytes} [Tue Sep 27 09:56:40 2022] GET /compute/v2.1/os-hypervisors/1/servers => generated 118 bytes in 26 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Sep 27 09:56:40.568039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-67a2c5bf-0b01-46ac-adf8-a39b11d068da tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Calling method '.version_select at 0x7f7f68285a60>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:40.572648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67a2c5bf-0b01-46ac-adf8-a39b11d068da tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:40.573967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67a2c5bf-0b01-46ac-adf8-a39b11d068da tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:40.575009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67a2c5bf-0b01-46ac-adf8-a39b11d068da tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:40.588889 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-67a2c5bf-0b01-46ac-adf8-a39b11d068da tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] HTTP exception thrown: No hypervisor matching 'cafa6473-e47a-4f1b-9a7e-72196dd02992' could be found. Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-67a2c5bf-0b01-46ac-adf8-a39b11d068da tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Request method failure captured: Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hypervisors/cafa6473-e47a-4f1b-9a7e-72196dd02992/servers HTTP/1.1 Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjWPYRnQ_6KM3wc48n9Q3MoH_pql_auIvhNuI_9EzFIw6gMrJCvSeWKJK2LEjntTZhZcXikyoVC23d8KXNJY6CPpBGqbO2WQ-fu9GW0nlahik4fkxPINJrut2JgPGO1EgXuv_1AsO5NDJmPQIKKLnvWMBohHEbEEE5zcuC3ZEePQuk Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 9f08504febac4265b33fed0f12095b45 Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HypervisorAdminNegativeUnderV252Test-1469927779 Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:40.597437 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9f08504febac4265b33fed0f12095b45"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9f08504febac4265b33fed0f12095b45"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9f08504febac4265b33fed0f12095b45"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9f08504febac4265b33fed0f12095b45"}]}] Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HypervisorAdminNegativeUnderV252Test-1469927779 Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 9f08504febac4265b33fed0f12095b45 Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HypervisorAdminNegativeUnderV252Test-1469927779 Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 4b6ee505ea6345f4bf22f29cc5f359f4 Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: exception: No hypervisor matching 'cafa6473-e47a-4f1b-9a7e-72196dd02992' could be found. Sep 27 09:56:40.598752 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'cafa6473-e47a-4f1b-9a7e-72196dd02992'} Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 482, in servers Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi compute_nodes = self._get_compute_nodes_by_name_pattern(context, id) Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 139, in _get_compute_nodes_by_name_pattern Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: No hypervisor matching 'cafa6473-e47a-4f1b-9a7e-72196dd02992' could be found. Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:40.599360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:40.600341 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:40.600341 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:40.600341 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:40.600341 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:40.600341 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: No hypervisor matching 'cafa6473-e47a-4f1b-9a7e-72196dd02992' could be found. Sep 27 09:56:40.600341 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:40.600341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-67a2c5bf-0b01-46ac-adf8-a39b11d068da tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] Returning 404 to user: No hypervisor matching 'cafa6473-e47a-4f1b-9a7e-72196dd02992' could be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:40.600341 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-67a2c5bf-0b01-46ac-adf8-a39b11d068da tempest-HypervisorAdminNegativeUnderV252Test-1469927779 tempest-HypervisorAdminNegativeUnderV252Test-1469927779-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/cafa6473-e47a-4f1b-9a7e-72196dd02992/servers" status: 404 len: 123 microversion: 2.1 time: 0.041894 Sep 27 09:56:40.605709 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 39/72] 10.222.0.26 () {58 vars in 1281 bytes} [Tue Sep 27 09:56:40 2022] GET /compute/v2.1/os-hypervisors/cafa6473-e47a-4f1b-9a7e-72196dd02992/servers => generated 123 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:40.631085 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eccd3b0d-de96-4754-988b-6938f3b0f942 tempest-FlavorsV255TestJSON-1203847512 tempest-FlavorsV255TestJSON-1203847512-project-admin] 10.222.0.26 "PUT /compute/v2.1/flavors/1345800171" status: 200 len: 450 microversion: 2.55 time: 0.172476 Sep 27 09:56:40.632909 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 34/73] 10.222.0.26 () {62 vars in 1219 bytes} [Tue Sep 27 09:56:40 2022] PUT /compute/v2.1/flavors/1345800171 => generated 450 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:56:40.652537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e9829d3f-63cd-4b90-8221-c1ae9287c381 tempest-FlavorsV255TestJSON-1599456155 tempest-FlavorsV255TestJSON-1599456155-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:40.718633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-1579777790", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:40.723448 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e9829d3f-63cd-4b90-8221-c1ae9287c381 tempest-FlavorsV255TestJSON-1599456155 tempest-FlavorsV255TestJSON-1599456155-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5725 microversion: 2.55 time: 0.076655 Sep 27 09:56:40.725946 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 35/74] 10.222.0.26 () {60 vars in 1187 bytes} [Tue Sep 27 09:56:40 2022] GET /compute/v2.1/flavors/detail => generated 5725 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:56:40.752087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-40eca4e3-7562-42e7-ba1c-2b0afbe34445 tempest-FlavorsV255TestJSON-1599456155 tempest-FlavorsV255TestJSON-1599456155-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:40.805036 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-40eca4e3-7562-42e7-ba1c-2b0afbe34445 tempest-FlavorsV255TestJSON-1599456155 tempest-FlavorsV255TestJSON-1599456155-project] 10.222.0.26 "GET /compute/v2.1/flavors" status: 200 len: 3157 microversion: 2.55 time: 0.059349 Sep 27 09:56:40.805827 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 36/75] 10.222.0.26 () {60 vars in 1166 bytes} [Tue Sep 27 09:56:40 2022] GET /compute/v2.1/flavors => generated 3157 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:56:40.813444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:56:40.828379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-24249b7a-b9f8-4361-9cdb-71d78c933fed tempest-FlavorsV255TestJSON-1203847512 tempest-FlavorsV255TestJSON-1203847512-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:40.865620 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-24249b7a-b9f8-4361-9cdb-71d78c933fed tempest-FlavorsV255TestJSON-1203847512 tempest-FlavorsV255TestJSON-1203847512-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1345800171" status: 202 len: 0 microversion: 2.55 time: 0.047133 Sep 27 09:56:40.869674 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 37/76] 10.222.0.26 () {60 vars in 1202 bytes} [Tue Sep 27 09:56:40 2022] DELETE /compute/v2.1/flavors/1345800171 => generated 0 bytes in 49 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 27 09:56:40.880380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e98734f2-bedf-41ac-9d35-0f62d9c00e32 tempest-FlavorsV255TestJSON-1203847512 tempest-FlavorsV255TestJSON-1203847512-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:40.993271 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e98734f2-bedf-41ac-9d35-0f62d9c00e32 tempest-FlavorsV255TestJSON-1203847512 tempest-FlavorsV255TestJSON-1203847512-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5285 microversion: 2.55 time: 0.121866 Sep 27 09:56:40.996752 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 38/77] 10.222.0.26 () {60 vars in 1187 bytes} [Tue Sep 27 09:56:40 2022] GET /compute/v2.1/flavors/detail => generated 5285 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:56:41.083326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:56:41.083909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:56:41.084253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:56:41.084699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:56:41.085127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:56:41.105079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:41.105574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:41.106951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:41.107990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:41.108894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:41.117343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:41.125336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d8427ff-2aa1-4ca5-aa8c-6370ec542705 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:41.126287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4d8427ff-2aa1-4ca5-aa8c-6370ec542705 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:41.141605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d8427ff-2aa1-4ca5-aa8c-6370ec542705 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:41.144302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d8427ff-2aa1-4ca5-aa8c-6370ec542705 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:41.145685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d8427ff-2aa1-4ca5-aa8c-6370ec542705 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:41.146917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d8427ff-2aa1-4ca5-aa8c-6370ec542705 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:41.150591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d8427ff-2aa1-4ca5-aa8c-6370ec542705 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:41.153276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d8427ff-2aa1-4ca5-aa8c-6370ec542705 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:41.203624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Getting quotas for project e81a1b6d13954194a01cc43062b0038d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:56:41.219936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Getting quotas for user 862f94043f044a25b740db652bf3dc02 and project e81a1b6d13954194a01cc43062b0038d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:56:41.265357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.utils [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] cores quota exceeded for e81a1b6d13954194a01cc43062b0038d, tried to run 1 instances. Cannot run any more instances of this type. {{(pid=145969) check_num_instances_quota /opt/stack/nova/nova/compute/utils.py:1173}} Sep 27 09:56:41.266461 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] HTTP exception thrown: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Request method failure captured: Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 145 Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjTAINI4RBnQ_9m_V6yCjPSET_pmTLP210iSi_5xRgK6s8MsEgyzudXLAi-JPCP-ujYhycfAuw-56NGJtgYCHyb5LazrYd-v0xXgzRiO-iuo1j-YAozhA5pGbE1YOVlf9VRmghhu4zIUkqkbrUj5UoHTYS9nZ9H2-r7gB0M8eKYpb0 Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e81a1b6d13954194a01cc43062b0038d Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:41.274941 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e81a1b6d13954194a01cc43062b0038d"}]}] Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e81a1b6d13954194a01cc43062b0038d Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-QuotasAdminNegativeTest-1307392554-project Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 862f94043f044a25b740db652bf3dc02 Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-QuotasAdminNegativeTest-1307392554-project Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-1579777790", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} Sep 27 09:56:41.276177 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: exception: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-QuotasAdminNegativeTest-server-1579777790', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452'}}} Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1115, in check_num_instances_quota Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, deltas, Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Sep 27 09:56:41.276884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 588, in limit_check_project_and_user Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['cores'] Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._create_instance( Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1733, in _create_instance Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances( Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1362, in _provision_instances Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi num_instances = compute_utils.check_num_instances_quota( Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1181, in check_num_instances_quota Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:41.277836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:41.278903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:41.279962 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:41.279962 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:41.279962 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:41.279962 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:41.279962 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores Sep 27 09:56:41.279962 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:41.286516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Returning 403 to user: Quota exceeded for cores: Requested 1, but already used 0 of 0 cores {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:41.287606 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0d3905cf-5509-408e-a2b2-b17298ce54ad tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 403 len: 111 microversion: 2.1 time: 0.680625 Sep 27 09:56:41.291087 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 40/78] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:56:40 2022] POST /compute/v2.1/servers => generated 111 bytes in 685 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:41.301027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca418473-de8b-47c1-b14f-5aa2ca43a69d tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Action: 'update', calling method: .version_select at 0x7f7f681cf1f0>, body: {"quota_set": {"cores": 20}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:41.628196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4d8427ff-2aa1-4ca5-aa8c-6370ec542705 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:41.641882 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4d8427ff-2aa1-4ca5-aa8c-6370ec542705 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.520396 Sep 27 09:56:41.654964 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 39/79] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:41 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:56:41.930936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca418473-de8b-47c1-b14f-5aa2ca43a69d tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:41.933360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca418473-de8b-47c1-b14f-5aa2ca43a69d tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:41.934518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca418473-de8b-47c1-b14f-5aa2ca43a69d tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:41.936728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca418473-de8b-47c1-b14f-5aa2ca43a69d tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:41.938671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca418473-de8b-47c1-b14f-5aa2ca43a69d tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:41.940868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca418473-de8b-47c1-b14f-5aa2ca43a69d tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:41.986388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:RELEASE=2,nova_api:SAVEPOINT=1,nova_api:SELECT=88,nova_api:INSERT=6,nova_api:DELETE=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:42.076510 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ca418473-de8b-47c1-b14f-5aa2ca43a69d tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d" status: 200 len: 329 microversion: 2.1 time: 0.780253 Sep 27 09:56:42.077912 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 41/80] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 09:56:41 2022] PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d => generated 329 bytes in 782 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:56:42.093192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8cfc9d39-bf95-466f-9f7e-0521225830a8 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Calling method '.version_select at 0x7f7f698f5e50>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:42.340522 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8cfc9d39-bf95-466f-9f7e-0521225830a8 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d" status: 200 len: 371 microversion: 2.1 time: 0.251318 Sep 27 09:56:42.342712 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 40/81] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 09:56:42 2022] GET /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d => generated 371 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:42.354757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-02752bb0-ed68-4db7-92e6-2e5591176e78 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Action: 'update', calling method: .version_select at 0x7f7f681cfee0>, body: {"quota_set": {"force": true, "instances": 0}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:42.501629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02752bb0-ed68-4db7-92e6-2e5591176e78 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:42.502373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02752bb0-ed68-4db7-92e6-2e5591176e78 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:42.509905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02752bb0-ed68-4db7-92e6-2e5591176e78 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:42.511052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02752bb0-ed68-4db7-92e6-2e5591176e78 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:42.512436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02752bb0-ed68-4db7-92e6-2e5591176e78 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:42.513098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02752bb0-ed68-4db7-92e6-2e5591176e78 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:42.644873 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-02752bb0-ed68-4db7-92e6-2e5591176e78 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d" status: 200 len: 328 microversion: 2.1 time: 0.294116 Sep 27 09:56:42.668798 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 42/82] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 09:56:42 2022] PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d => generated 328 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:42.677875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d5da542a-e3c2-4ca7-ab8c-d362feb8036d tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:42.680472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d5da542a-e3c2-4ca7-ab8c-d362feb8036d tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:42.715429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5da542a-e3c2-4ca7-ab8c-d362feb8036d tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:42.716692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5da542a-e3c2-4ca7-ab8c-d362feb8036d tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:42.717501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5da542a-e3c2-4ca7-ab8c-d362feb8036d tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:42.718852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5da542a-e3c2-4ca7-ab8c-d362feb8036d tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:42.723543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5da542a-e3c2-4ca7-ab8c-d362feb8036d tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:42.725061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5da542a-e3c2-4ca7-ab8c-d362feb8036d tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:42.996754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-1170293595", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:43.166575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:56:43.316137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d5da542a-e3c2-4ca7-ab8c-d362feb8036d tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:43.337093 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d5da542a-e3c2-4ca7-ab8c-d362feb8036d tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.656905 Sep 27 09:56:43.337793 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 41/83] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:42 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:43.754967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:56:43.755597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:56:43.756070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:56:43.756739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:56:43.758199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:56:43.775681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:43.775825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:43.784221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:43.787815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:43.792219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:43.794011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:43.862779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Getting quotas for project e81a1b6d13954194a01cc43062b0038d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:56:43.875804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Getting quotas for user 862f94043f044a25b740db652bf3dc02 and project e81a1b6d13954194a01cc43062b0038d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:56:43.920545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.utils [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] instances quota exceeded for e81a1b6d13954194a01cc43062b0038d, tried to run 1 instances. Cannot run any more instances of this type. {{(pid=145969) check_num_instances_quota /opt/stack/nova/nova/compute/utils.py:1173}} Sep 27 09:56:43.921258 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] HTTP exception thrown: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Request method failure captured: Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 145 Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjTAINI4RBnQ_9m_V6yCjPSET_pmTLP210iSi_5xRgK6s8MsEgyzudXLAi-JPCP-ujYhycfAuw-56NGJtgYCHyb5LazrYd-v0xXgzRiO-iuo1j-YAozhA5pGbE1YOVlf9VRmghhu4zIUkqkbrUj5UoHTYS9nZ9H2-r7gB0M8eKYpb0 Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e81a1b6d13954194a01cc43062b0038d Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:43.923494 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e81a1b6d13954194a01cc43062b0038d"}]}] Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e81a1b6d13954194a01cc43062b0038d Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-QuotasAdminNegativeTest-1307392554-project Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 862f94043f044a25b740db652bf3dc02 Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-QuotasAdminNegativeTest-1307392554-project Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-1170293595", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} Sep 27 09:56:43.924885 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: exception: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-QuotasAdminNegativeTest-server-1170293595', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452'}}} Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1115, in check_num_instances_quota Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, deltas, Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Sep 27 09:56:43.925579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 588, in limit_check_project_and_user Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['instances'] Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._create_instance( Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1733, in _create_instance Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances( Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1362, in _provision_instances Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi num_instances = compute_utils.check_num_instances_quota( Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1181, in check_num_instances_quota Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:43.926873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:43.927908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:43.929418 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:43.929418 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:43.929418 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:43.929418 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:43.929418 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances Sep 27 09:56:43.929418 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:43.929418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Returning 403 to user: Quota exceeded for instances: Requested 1, but already used 0 of 0 instances {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:43.929418 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-721301a2-45dc-459a-a495-8f8daa723f9c tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 403 len: 119 microversion: 2.1 time: 0.936317 Sep 27 09:56:43.931210 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 43/84] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:56:42 2022] POST /compute/v2.1/servers => generated 119 bytes in 941 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:43.936123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6dcf6726-af13-4943-a2b9-b57514d545f5 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Action: 'update', calling method: .version_select at 0x7f7f6845b790>, body: {"quota_set": {"instances": 10}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:44.362708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9ff98d6b-1785-4ec6-a420-7624c7de3502 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:44.363543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9ff98d6b-1785-4ec6-a420-7624c7de3502 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:44.377075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff98d6b-1785-4ec6-a420-7624c7de3502 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:44.378324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff98d6b-1785-4ec6-a420-7624c7de3502 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:44.379846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff98d6b-1785-4ec6-a420-7624c7de3502 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:44.383717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff98d6b-1785-4ec6-a420-7624c7de3502 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:44.385243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff98d6b-1785-4ec6-a420-7624c7de3502 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:44.386827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff98d6b-1785-4ec6-a420-7624c7de3502 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:44.636208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6dcf6726-af13-4943-a2b9-b57514d545f5 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:44.649433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6dcf6726-af13-4943-a2b9-b57514d545f5 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:44.650055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6dcf6726-af13-4943-a2b9-b57514d545f5 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:44.651157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6dcf6726-af13-4943-a2b9-b57514d545f5 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:44.653552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6dcf6726-af13-4943-a2b9-b57514d545f5 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:44.654524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6dcf6726-af13-4943-a2b9-b57514d545f5 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:44.850853 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6dcf6726-af13-4943-a2b9-b57514d545f5 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d" status: 200 len: 329 microversion: 2.1 time: 0.918287 Sep 27 09:56:44.863032 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 42/85] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 09:56:43 2022] PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d => generated 329 bytes in 921 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:56:45.156920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-9c0ce825-f8f1-4ca8-a515-202ab1d9c423 req-4e2dfdf2-c916-45e5-805f-45a0fb780b11 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8febbed7-7a38-4051-8910-bdb62bc11eac", "tag": "ed45a4db-ff3b-4d14-b5f9-f0a60046bd2c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:45.201836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9c0ce825-f8f1-4ca8-a515-202ab1d9c423 req-4e2dfdf2-c916-45e5-805f-45a0fb780b11 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:45.205671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9c0ce825-f8f1-4ca8-a515-202ab1d9c423 req-4e2dfdf2-c916-45e5-805f-45a0fb780b11 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:45.206378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9c0ce825-f8f1-4ca8-a515-202ab1d9c423 req-4e2dfdf2-c916-45e5-805f-45a0fb780b11 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:45.281494 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-9c0ce825-f8f1-4ca8-a515-202ab1d9c423 req-4e2dfdf2-c916-45e5-805f-45a0fb780b11 service nova] Creating event network-changed:ed45a4db-ff3b-4d14-b5f9-f0a60046bd2c for instance 8febbed7-7a38-4051-8910-bdb62bc11eac on nch1-master-1 Sep 27 09:56:45.282292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9c0ce825-f8f1-4ca8-a515-202ab1d9c423 req-4e2dfdf2-c916-45e5-805f-45a0fb780b11 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 09:56:45.282819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9c0ce825-f8f1-4ca8-a515-202ab1d9c423 req-4e2dfdf2-c916-45e5-805f-45a0fb780b11 service nova] Acquired lock "compute-rpcapi-router" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 09:56:45.289125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-510a607a-6c4d-4da9-b358-673cccaeb185 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:45.289929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-510a607a-6c4d-4da9-b358-673cccaeb185 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:45.290606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-510a607a-6c4d-4da9-b358-673cccaeb185 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:45.291526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-510a607a-6c4d-4da9-b358-673cccaeb185 None None] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:45.292159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-510a607a-6c4d-4da9-b358-673cccaeb185 None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:45.292842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-510a607a-6c4d-4da9-b358-673cccaeb185 None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:45.305714 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.compute.rpcapi [None req-510a607a-6c4d-4da9-b358-673cccaeb185 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 27 09:56:45.307008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-510a607a-6c4d-4da9-b358-673cccaeb185 None None] Releasing lock "compute-rpcapi-router" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 09:56:45.331320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9ff98d6b-1785-4ec6-a420-7624c7de3502 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:45.338287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-510a607a-6c4d-4da9-b358-673cccaeb185 None None] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.480532 Sep 27 09:56:45.342325 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 43/86] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:56:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:45.343349 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9ff98d6b-1785-4ec6-a420-7624c7de3502 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.985164 Sep 27 09:56:45.344575 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 44/87] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:44 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 987 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:45.364892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-50d48638-6aa6-49b8-88ca-e7ecf13d3ea5 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Calling method '.version_select at 0x7f7f68431940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:45.933363 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-50d48638-6aa6-49b8-88ca-e7ecf13d3ea5 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d" status: 200 len: 371 microversion: 2.1 time: 0.588946 Sep 27 09:56:45.934709 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 44/88] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 09:56:45 2022] GET /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d => generated 371 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:45.950184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4943f3d3-a387-45a4-abe8-3925170413d2 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Action: 'update', calling method: .version_select at 0x7f7f681d35e0>, body: {"quota_set": {"force": true, "ram": 0}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:46.368794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7a6e82f4-5f22-4c82-8aca-5de422810a11 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:46.369532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7a6e82f4-5f22-4c82-8aca-5de422810a11 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:46.374597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4943f3d3-a387-45a4-abe8-3925170413d2 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:46.375792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4943f3d3-a387-45a4-abe8-3925170413d2 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:46.376244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4943f3d3-a387-45a4-abe8-3925170413d2 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:46.379756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4943f3d3-a387-45a4-abe8-3925170413d2 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:46.380264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4943f3d3-a387-45a4-abe8-3925170413d2 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:46.380720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4943f3d3-a387-45a4-abe8-3925170413d2 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:46.389735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a6e82f4-5f22-4c82-8aca-5de422810a11 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:46.390240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a6e82f4-5f22-4c82-8aca-5de422810a11 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:46.390674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a6e82f4-5f22-4c82-8aca-5de422810a11 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:46.399691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a6e82f4-5f22-4c82-8aca-5de422810a11 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:46.400371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a6e82f4-5f22-4c82-8aca-5de422810a11 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:46.400866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a6e82f4-5f22-4c82-8aca-5de422810a11 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:46.543470 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4943f3d3-a387-45a4-abe8-3925170413d2 tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d" status: 200 len: 325 microversion: 2.1 time: 0.599859 Sep 27 09:56:46.544971 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 45/89] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 09:56:45 2022] PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d => generated 325 bytes in 602 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:46.847713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-287127575", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:46.855865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7a6e82f4-5f22-4c82-8aca-5de422810a11 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:46.870192 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7a6e82f4-5f22-4c82-8aca-5de422810a11 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.505318 Sep 27 09:56:46.894547 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 45/90] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:46 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:47.004039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:56:47.646939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:56:47.648095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:56:47.648534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:56:47.649102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:56:47.649790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:56:47.679520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:47.680496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:47.681410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:47.688126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:47.692047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:47.696213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:47.758252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Getting quotas for project e81a1b6d13954194a01cc43062b0038d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:56:47.769173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Getting quotas for user 862f94043f044a25b740db652bf3dc02 and project e81a1b6d13954194a01cc43062b0038d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:56:47.825963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.utils [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] ram quota exceeded for e81a1b6d13954194a01cc43062b0038d, tried to run 1 instances. Cannot run any more instances of this type. {{(pid=145969) check_num_instances_quota /opt/stack/nova/nova/compute/utils.py:1173}} Sep 27 09:56:47.826618 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] HTTP exception thrown: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Request method failure captured: Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 144 Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjTAINI4RBnQ_9m_V6yCjPSET_pmTLP210iSi_5xRgK6s8MsEgyzudXLAi-JPCP-ujYhycfAuw-56NGJtgYCHyb5LazrYd-v0xXgzRiO-iuo1j-YAozhA5pGbE1YOVlf9VRmghhu4zIUkqkbrUj5UoHTYS9nZ9H2-r7gB0M8eKYpb0 Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e81a1b6d13954194a01cc43062b0038d Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:47.833656 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e81a1b6d13954194a01cc43062b0038d"}]}] Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e81a1b6d13954194a01cc43062b0038d Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-QuotasAdminNegativeTest-1307392554-project Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 862f94043f044a25b740db652bf3dc02 Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-QuotasAdminNegativeTest-1307392554-project Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-QuotasAdminNegativeTest-server-287127575", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} Sep 27 09:56:47.835147 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: exception: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-QuotasAdminNegativeTest-server-287127575', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452'}}} Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1115, in check_num_instances_quota Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, deltas, Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Sep 27 09:56:47.836619 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 588, in limit_check_project_and_user Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['ram'] Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._create_instance( Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1733, in _create_instance Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instances_to_build = self._provision_instances( Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1362, in _provision_instances Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi num_instances = compute_utils.check_num_instances_quota( Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/utils.py", line 1181, in check_num_instances_quota Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:47.838733 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:47.839884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:47.841318 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:47.841318 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:47.841318 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:47.841318 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:47.841318 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram Sep 27 09:56:47.841318 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:47.841318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Returning 403 to user: Quota exceeded for ram: Requested 1024, but already used 0 of 0 ram {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:47.841318 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-23f0e086-e6c8-42b1-a214-e31d9dc9e138 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 403 len: 110 microversion: 2.1 time: 0.992954 Sep 27 09:56:47.841318 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 46/91] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:56:46 2022] POST /compute/v2.1/servers => generated 110 bytes in 994 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:47.849970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-760e6cab-f6a8-499b-86f3-7fc5d42e4b0f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Action: 'update', calling method: .version_select at 0x7f7f683ef310>, body: {"quota_set": {"ram": 51200}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:47.908450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-704c8607-dcb2-44e7-87e4-a2c5da4e685c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:47.909584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-704c8607-dcb2-44e7-87e4-a2c5da4e685c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:47.929845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704c8607-dcb2-44e7-87e4-a2c5da4e685c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:47.930831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704c8607-dcb2-44e7-87e4-a2c5da4e685c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:47.932828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704c8607-dcb2-44e7-87e4-a2c5da4e685c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:47.933933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704c8607-dcb2-44e7-87e4-a2c5da4e685c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:47.936703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704c8607-dcb2-44e7-87e4-a2c5da4e685c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:47.938224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704c8607-dcb2-44e7-87e4-a2c5da4e685c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:48.284466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-704c8607-dcb2-44e7-87e4-a2c5da4e685c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:48.301288 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-704c8607-dcb2-44e7-87e4-a2c5da4e685c tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.403239 Sep 27 09:56:48.317352 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 47/92] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:47 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:48.473904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-760e6cab-f6a8-499b-86f3-7fc5d42e4b0f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:48.474657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-760e6cab-f6a8-499b-86f3-7fc5d42e4b0f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:48.475273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-760e6cab-f6a8-499b-86f3-7fc5d42e4b0f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:48.476423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-760e6cab-f6a8-499b-86f3-7fc5d42e4b0f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:48.478050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-760e6cab-f6a8-499b-86f3-7fc5d42e4b0f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:48.478892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-760e6cab-f6a8-499b-86f3-7fc5d42e4b0f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:48.513654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba8f6481-e3fa-4c3a-904d-94c1b2e73cd2 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Action: 'create', calling method: .version_select at 0x7f7f68214790>, body: {"keypair": {"name": "tempest-keypair-1259535551", "type": "ssh", "user_id": "c7ff8a626cd5460295136c2de34f841c", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG5Dp6TRYyVJ9ZNzEZdYNSW/gxF36kvbMx/9Qavz6JZVZ9Ty5VftoQTg1RFWR1N88JutKPcmN/au1WWBb7z9M14k0VX8RlPlvxgKXw7Qze83HFTUsYAwyhG6N1UN31O4IA=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:48.553179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ba8f6481-e3fa-4c3a-904d-94c1b2e73cd2 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Getting quotas for project db2605a6df2e44a5ab5c838febd8df19. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:56:48.562764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ba8f6481-e3fa-4c3a-904d-94c1b2e73cd2 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Getting quotas for user 83828eb232f74e0092746711eb78977b and project db2605a6df2e44a5ab5c838febd8df19. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:56:48.621702 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ba8f6481-e3fa-4c3a-904d-94c1b2e73cd2 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.10 time: 0.302391 Sep 27 09:56:48.623856 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 48/93] 10.222.0.26 () {62 vars in 1200 bytes} [Tue Sep 27 09:56:48 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 305 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 27 09:56:48.673468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7249f809-d8ee-4ccd-a270-2e0ee622a101 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Action: 'create', calling method: .version_select at 0x7f7f681cf790>, body: {"keypair": {"name": "tempest-keypair-1007558671", "type": "ssh", "user_id": "c7ff8a626cd5460295136c2de34f841c", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNvT6x9tHJmfvhufHRkJnf54WwxsYFszSP5kTu7NZE9n4Am3blSOU4gkNEJ42KkFAVyfwEkocJA2HZRxMMGxkaYCFFOE2falqg0xWpjpSjPdNHg102UrXenaabTK24GPGw=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:48.684245 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-760e6cab-f6a8-499b-86f3-7fc5d42e4b0f tempest-QuotasAdminNegativeTest-2052721599 tempest-QuotasAdminNegativeTest-2052721599-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d" status: 200 len: 329 microversion: 2.1 time: 0.839842 Sep 27 09:56:48.685322 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 46/94] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 09:56:47 2022] PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d => generated 329 bytes in 842 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:48.700019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7249f809-d8ee-4ccd-a270-2e0ee622a101 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Getting quotas for project db2605a6df2e44a5ab5c838febd8df19. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:56:48.701218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d339c99-4606-420d-9f94-e21c27da5405 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Action: 'update', calling method: .version_select at 0x7f7f68431550>, body: {"quota_set": {"ram": 0}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:48.707049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7249f809-d8ee-4ccd-a270-2e0ee622a101 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Getting quotas for user 83828eb232f74e0092746711eb78977b and project db2605a6df2e44a5ab5c838febd8df19. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:56:48.721547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2d339c99-4606-420d-9f94-e21c27da5405 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Policy check for os_compute_api:os-quota-sets:update failed with credentials {'is_admin': False, 'user_id': '862f94043f044a25b740db652bf3dc02', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e81a1b6d13954194a01cc43062b0038d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d339c99-4606-420d-9f94-e21c27da5405 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Request method failure captured: Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: request: PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d HTTP/1.1 Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 25 Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjTAINI4RBnQ_9m_V6yCjPSET_pmTLP210iSi_5xRgK6s8MsEgyzudXLAi-JPCP-ujYhycfAuw-56NGJtgYCHyb5LazrYd-v0xXgzRiO-iuo1j-YAozhA5pGbE1YOVlf9VRmghhu4zIUkqkbrUj5UoHTYS9nZ9H2-r7gB0M8eKYpb0 Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e81a1b6d13954194a01cc43062b0038d Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:56:48.724253 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e81a1b6d13954194a01cc43062b0038d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e81a1b6d13954194a01cc43062b0038d"}]}] Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e81a1b6d13954194a01cc43062b0038d Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-QuotasAdminNegativeTest-1307392554 Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-QuotasAdminNegativeTest-1307392554-project Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 862f94043f044a25b740db652bf3dc02 Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-QuotasAdminNegativeTest-1307392554-project Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: {"quota_set": {"ram": 0}} Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: exception: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. Sep 27 09:56:48.725990 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'e81a1b6d13954194a01cc43062b0038d', 'body': {'quota_set': {'ram': 0}}} Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/quota_sets.py", line 170, in update Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._update(req, id, body, []) Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:48.727025 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/quota_sets.py", line 189, in _update Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi context.can(qs_policies.POLICY_ROOT % 'update', {'project_id': id}) Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:48.728780 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:48.730286 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:48.730286 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:48.730286 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:48.730286 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:48.730286 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. Sep 27 09:56:48.730286 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:48.741170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d339c99-4606-420d-9f94-e21c27da5405 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-quota-sets:update to be performed. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:48.744564 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d339c99-4606-420d-9f94-e21c27da5405 tempest-QuotasAdminNegativeTest-1307392554 tempest-QuotasAdminNegativeTest-1307392554-project] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d" status: 403 len: 116 microversion: 2.1 time: 0.047810 Sep 27 09:56:48.746972 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 47/95] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 09:56:48 2022] PUT /compute/v2.1/os-quota-sets/e81a1b6d13954194a01cc43062b0038d => generated 116 bytes in 51 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Sep 27 09:56:48.770029 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7249f809-d8ee-4ccd-a270-2e0ee622a101 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.10 time: 0.130525 Sep 27 09:56:48.772728 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 49/96] 10.222.0.26 () {62 vars in 1200 bytes} [Tue Sep 27 09:56:48 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 138 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 27 09:56:48.783583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0955cc4-3c47-4d85-be91-20f2edb6f969 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Calling method '.version_select at 0x7f7f683a5820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:48.802700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=90,nova_api:DELETE=5,nova_api:UPDATE=3,nova_api:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:48.826731 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f0955cc4-3c47-4d85-be91-20f2edb6f969 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] 10.222.0.26 "GET /compute/v2.1/os-keypairs/tempest-keypair-1259535551?user_id=c7ff8a626cd5460295136c2de34f841c" status: 200 len: 512 microversion: 2.10 time: 0.047886 Sep 27 09:56:48.832089 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 48/97] 10.222.0.26 () {60 vars in 1340 bytes} [Tue Sep 27 09:56:48 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-1259535551?user_id=c7ff8a626cd5460295136c2de34f841c => generated 512 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:56:48.847795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa98d0b7-b049-4d81-b470-05844b97c110 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Action: 'create', calling method: .version_select at 0x7f7f68285550>, body: {"keypair": {"name": "tempest-KeyPairsV210TestJSON-keypair-1746335049", "type": "ssh", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAPBOLkNkEf00reHKiWVsDkQ9jjj8urUbYMeRh3N79zG+SLuXqalWSYFwXcma27KBH154hCrPfPuSDy0TcMeDToRT0Xc6wH5Q3onvB5/mJdei7xYsHf9v4uHo9df4DO0xA=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:48.919314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=93,nova_cell0:UPDATE=7 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:48.949870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-aa98d0b7-b049-4d81-b470-05844b97c110 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Getting quotas for project db2605a6df2e44a5ab5c838febd8df19. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:56:48.978994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-aa98d0b7-b049-4d81-b470-05844b97c110 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Getting quotas for user 83828eb232f74e0092746711eb78977b and project db2605a6df2e44a5ab5c838febd8df19. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:56:49.024158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=5,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:49.046553 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aa98d0b7-b049-4d81-b470-05844b97c110 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 422 microversion: 2.10 time: 0.203503 Sep 27 09:56:49.049300 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 50/98] 10.222.0.26 () {62 vars in 1200 bytes} [Tue Sep 27 09:56:48 2022] POST /compute/v2.1/os-keypairs => generated 422 bytes in 207 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 27 09:56:49.061350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b1ab0a63-13b1-4ad8-ab2b-6cce7b66585e tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Calling method '.version_select at 0x7f7f683ef3a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:49.068591 ncdv-master-1 devstack@n-api.service[145968]: WARNING oslo_db.sqlalchemy.utils [None req-b1ab0a63-13b1-4ad8-ab2b-6cce7b66585e tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Unique keys not in sort_keys. The sorting order may be unstable. Sep 27 09:56:49.079802 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b1ab0a63-13b1-4ad8-ab2b-6cce7b66585e tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] 10.222.0.26 "GET /compute/v2.1/os-keypairs?user_id=c7ff8a626cd5460295136c2de34f841c" status: 200 len: 726 microversion: 2.10 time: 0.023818 Sep 27 09:56:49.081375 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 49/99] 10.222.0.26 () {60 vars in 1259 bytes} [Tue Sep 27 09:56:49 2022] GET /compute/v2.1/os-keypairs?user_id=c7ff8a626cd5460295136c2de34f841c => generated 726 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:56:49.097331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5aab6efe-ec80-4ad7-9fe1-fe2c6d79e212 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Calling method '.version_select at 0x7f7f681d3670>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:49.139279 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5aab6efe-ec80-4ad7-9fe1-fe2c6d79e212 tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV210TestJSON-keypair-1746335049" status: 204 len: 0 microversion: 2.10 time: 0.049376 Sep 27 09:56:49.140282 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 51/100] 10.222.0.26 () {60 vars in 1325 bytes} [Tue Sep 27 09:56:49 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV210TestJSON-keypair-1746335049 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 09:56:49.149418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b99b493-b1cf-4b72-afd0-65947695f05d tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Calling method '.version_select at 0x7f7f683b4b80>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:49.192974 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3b99b493-b1cf-4b72-afd0-65947695f05d tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1007558671?user_id=c7ff8a626cd5460295136c2de34f841c" status: 204 len: 0 microversion: 2.10 time: 0.048087 Sep 27 09:56:49.193586 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 50/101] 10.222.0.26 () {60 vars in 1343 bytes} [Tue Sep 27 09:56:49 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1007558671?user_id=c7ff8a626cd5460295136c2de34f841c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 09:56:49.207138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce537fab-5ece-4cb8-9617-2fa58ad4818a tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] Calling method '.version_select at 0x7f7f681b3e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:49.244384 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ce537fab-5ece-4cb8-9617-2fa58ad4818a tempest-KeyPairsV210TestJSON-83761298 tempest-KeyPairsV210TestJSON-83761298-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1259535551?user_id=c7ff8a626cd5460295136c2de34f841c" status: 204 len: 0 microversion: 2.10 time: 0.041308 Sep 27 09:56:49.245672 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 52/102] 10.222.0.26 () {60 vars in 1343 bytes} [Tue Sep 27 09:56:49 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1259535551?user_id=c7ff8a626cd5460295136c2de34f841c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 09:56:49.336001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c20b4f5-d637-486c-a96c-db5b95efaeab tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:49.337614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5c20b4f5-d637-486c-a96c-db5b95efaeab tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:49.351246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c20b4f5-d637-486c-a96c-db5b95efaeab tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:49.352716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c20b4f5-d637-486c-a96c-db5b95efaeab tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:49.354459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c20b4f5-d637-486c-a96c-db5b95efaeab tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:49.357734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c20b4f5-d637-486c-a96c-db5b95efaeab tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:49.358459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c20b4f5-d637-486c-a96c-db5b95efaeab tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:49.359210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c20b4f5-d637-486c-a96c-db5b95efaeab tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:49.704828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c20b4f5-d637-486c-a96c-db5b95efaeab tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:49.719383 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c20b4f5-d637-486c-a96c-db5b95efaeab tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.391788 Sep 27 09:56:49.729048 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 51/103] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:49 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:50.040452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0784c35f-ba18-438c-90c4-5fa8f5878347 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:50.060173 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0784c35f-ba18-438c-90c4-5fa8f5878347 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.459227 Sep 27 09:56:50.073889 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 53/104] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 09:56:49 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:56:50.529392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7fb997b2-d81a-43b0-96df-50ff3a6a74ff tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 1024, "vcpus": 1, "disk": 1, "name": "tempest-flavor_with_ephemeral_1-2090292158", "id": 436599854, "OS-FLV-EXT-DATA:ephemeral": 1, "os-flavor-access:is_public": "True"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:50.553424 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7fb997b2-d81a-43b0-96df-50ff3a6a74ff tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 422 microversion: 2.1 time: 0.478333 Sep 27 09:56:50.555409 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 52/105] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:56:50 2022] POST /compute/v2.1/flavors => generated 422 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:50.572568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c1b36d62-df10-40e6-874f-aeb635ed9345 tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:50.588916 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c1b36d62-df10-40e6-874f-aeb635ed9345 tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.023437 Sep 27 09:56:50.602745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67e03d37-a79f-401d-9bd1-854664a8b87c tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 1024, "vcpus": 1, "disk": 1, "name": "tempest-flavor_with_ephemeral_0-496229148", "id": 720147853, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": "True"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:50.606122 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 54/106] 10.222.0.26 () {58 vars in 1182 bytes} [Tue Sep 27 09:56:50 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:56:50.639761 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67e03d37-a79f-401d-9bd1-854664a8b87c tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 421 microversion: 2.1 time: 0.042358 Sep 27 09:56:50.640383 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 53/107] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:56:50 2022] POST /compute/v2.1/flavors => generated 421 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:50.655514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-10829641-b06b-4ad7-a3a6-3e4fce9f957f tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:50.677208 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-10829641-b06b-4ad7-a3a6-3e4fce9f957f tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.027052 Sep 27 09:56:50.695664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-278c6e35-70fd-4590-9d00-4cca6a5ab3ff tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Action: 'create', calling method: .version_select at 0x7f7f683b4040>, body: {"keypair": {"name": "tempest-keypair-211851818", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBsieqgi5ALpmapV8gRvqwhoCa6QnsY4cUtEOwKOBZqCxH/Bqi5iAR7feHQprfnVtUnaX17oaap9Q3n48Oncn+9IWWknxwXjflf8SjtomPgUidPiaITAgdanc8UGMJ6hjg=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:50.701675 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 55/108] 10.222.0.26 () {58 vars in 1182 bytes} [Tue Sep 27 09:56:50 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:56:50.721731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-278c6e35-70fd-4590-9d00-4cca6a5ab3ff tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Getting quotas for project 1881e013b4b844cba1afce9fd21f8dd7. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:56:50.731989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-278c6e35-70fd-4590-9d00-4cca6a5ab3ff tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Getting quotas for user b4cebc85e62046d28fab159c6bb13e15 and project 1881e013b4b844cba1afce9fd21f8dd7. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:56:50.753451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95a7ade1-9e39-43de-9bcc-cd5ab628d175 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:50.754360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-95a7ade1-9e39-43de-9bcc-cd5ab628d175 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:50.767698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95a7ade1-9e39-43de-9bcc-cd5ab628d175 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:50.768482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95a7ade1-9e39-43de-9bcc-cd5ab628d175 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:50.769296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95a7ade1-9e39-43de-9bcc-cd5ab628d175 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:50.772518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95a7ade1-9e39-43de-9bcc-cd5ab628d175 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:50.774685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95a7ade1-9e39-43de-9bcc-cd5ab628d175 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:50.776755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95a7ade1-9e39-43de-9bcc-cd5ab628d175 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:50.812005 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-278c6e35-70fd-4590-9d00-4cca6a5ab3ff tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.121864 Sep 27 09:56:50.813039 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 54/109] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 09:56:50 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:51.207433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-95a7ade1-9e39-43de-9bcc-cd5ab628d175 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:51.222287 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-95a7ade1-9e39-43de-9bcc-cd5ab628d175 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.471686 Sep 27 09:56:51.222904 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 56/110] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:50 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:56:51.581399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7aa94b83-5c83-46e1-836c-5a32189ff8cd tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Calling method '.version_select at 0x7f7f683efb80>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:51.589245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7aa94b83-5c83-46e1-836c-5a32189ff8cd tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:51.589953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7aa94b83-5c83-46e1-836c-5a32189ff8cd tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:51.596377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7aa94b83-5c83-46e1-836c-5a32189ff8cd tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:51.641151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7aa94b83-5c83-46e1-836c-5a32189ff8cd tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:51.645627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7aa94b83-5c83-46e1-836c-5a32189ff8cd tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.045s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:51.647475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7aa94b83-5c83-46e1-836c-5a32189ff8cd tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:51.723465 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7aa94b83-5c83-46e1-836c-5a32189ff8cd tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts" status: 200 len: 401 microversion: 2.1 time: 0.685472 Sep 27 09:56:51.724942 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 55/111] 10.222.0.26 () {58 vars in 1128 bytes} [Tue Sep 27 09:56:51 2022] GET /compute/v2.1/os-hosts => generated 401 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:56:52.106242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5985f456-f775-4ce5-be2e-586998ae533d tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Calling method '.version_select at 0x7f7f682414c0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:52.110405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5985f456-f775-4ce5-be2e-586998ae533d tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Policy check for os_compute_api:os-hosts:list failed with credentials {'is_admin': False, 'user_id': '13c5c05dd6b244dd8ff15ca2c1ae298a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56f793358f224ff98e1f6bc0e95309c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5985f456-f775-4ce5-be2e-586998ae533d tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Request method failure captured: Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hosts HTTP/1.1 Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjgRBFDYGySwQXlTtf_VQ7V1cqSE9FFnLxXyOGL1GVMrzQ5vybkblIs0RpEsZgrhIiNA7WNh9lJk_biB06gSxVXa1z245ObxVNGbwx9xCefp8-PAdg0WIDBuwIG_6UnGPnZn62sYX5cHzTJ79YW0u-BeiXF6D98dkLkluJLLWiShvc Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:52.121917 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_56f793358f224ff98e1f6bc0e95309c5"}]}] Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 13c5c05dd6b244dd8ff15ca2c1ae298a Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-hosts:list to be performed. Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: action_args: {} Sep 27 09:56:52.123313 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 74, in index Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'list', Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:52.124090 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:list to be performed. Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:list to be performed. Sep 27 09:56:52.125326 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.128099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5985f456-f775-4ce5-be2e-586998ae533d tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:list to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.129227 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5985f456-f775-4ce5-be2e-586998ae533d tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] 10.222.0.26 "GET /compute/v2.1/os-hosts" status: 403 len: 109 microversion: 2.1 time: 0.396469 Sep 27 09:56:52.130063 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 57/112] 10.222.0.26 () {58 vars in 1128 bytes} [Tue Sep 27 09:56:51 2022] GET /compute/v2.1/os-hosts => generated 109 bytes in 398 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:52.142399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-05499f8d-7e1e-4813-a82e-de5f3514250b tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Calling method '.version_select at 0x7f7f683b43a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:52.145265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-05499f8d-7e1e-4813-a82e-de5f3514250b tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Policy check for os_compute_api:os-hosts:reboot failed with credentials {'is_admin': False, 'user_id': '13c5c05dd6b244dd8ff15ca2c1ae298a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56f793358f224ff98e1f6bc0e95309c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-05499f8d-7e1e-4813-a82e-de5f3514250b tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Request method failure captured: Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-hosts/ncdv-master-1/reboot HTTP/1.1 Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjgRBFDYGySwQXlTtf_VQ7V1cqSE9FFnLxXyOGL1GVMrzQ5vybkblIs0RpEsZgrhIiNA7WNh9lJk_biB06gSxVXa1z245ObxVNGbwx9xCefp8-PAdg0WIDBuwIG_6UnGPnZn62sYX5cHzTJ79YW0u-BeiXF6D98dkLkluJLLWiShvc Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:56:52.148838 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_56f793358f224ff98e1f6bc0e95309c5"}]}] Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 13c5c05dd6b244dd8ff15ca2c1ae298a Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: exception: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'ncdv-master-1'} Sep 27 09:56:52.150324 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 203, in reboot Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'reboot', Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:52.151067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. Sep 27 09:56:52.152189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.156820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-05499f8d-7e1e-4813-a82e-de5f3514250b tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:reboot to be performed. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.158211 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-05499f8d-7e1e-4813-a82e-de5f3514250b tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] 10.222.0.26 "GET /compute/v2.1/os-hosts/ncdv-master-1/reboot" status: 403 len: 111 microversion: 2.1 time: 0.022432 Sep 27 09:56:52.159504 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 56/113] 10.222.0.26 () {58 vars in 1191 bytes} [Tue Sep 27 09:56:52 2022] GET /compute/v2.1/os-hosts/ncdv-master-1/reboot => generated 111 bytes in 24 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:52.170279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5dd270f-e784-49f3-b64c-7094486d4cab tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Calling method '.version_select at 0x7f7f68167dc0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:52.192487 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-c5dd270f-e784-49f3-b64c-7094486d4cab tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5dd270f-e784-49f3-b64c-7094486d4cab tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Request method failure captured: Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname/reboot HTTP/1.1 Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjivOJP_hRj2xyeLXAuWSBw3_nBOAPtdXsD5CjO3KkoCg1J5V9E36ll8NX_wKr3EwTWOl9iCjrVmQT4M1SblE5faOJyGnaKt_5NwtIk_vJ11sYXh4IYL9eC0x9-E1hLYJRvL70iqsV-Hz22J2uUCQmD6vaQHP-IEKrT7WG22r8DV7w Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:52.201580 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8ecef2fe027b4a95a25f3e5aa6360bc2"}]}] Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2a56e8def15742d8a8a8a210ca57d8fe Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: exception: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'nonexistent_hostname'} Sep 27 09:56:52.202975 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 174, in _host_power_action Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = self.api.host_power_action(context, host_name, action) Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5990, in targeted Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:56:52.204152 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 205, in reboot Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._host_power_action(req, host_name=id, action="reboot") Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 178, in _host_power_action Sep 27 09:56:52.205579 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:56:52.206952 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.206952 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.206952 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.206952 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.206952 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.206952 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.206952 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.206952 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:52.206952 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:52.206952 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.206952 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.208503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5dd270f-e784-49f3-b64c-7094486d4cab tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.209845 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c5dd270f-e784-49f3-b64c-7094486d4cab tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts/nonexistent_hostname/reboot" status: 404 len: 99 microversion: 2.1 time: 0.043070 Sep 27 09:56:52.211577 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 58/114] 10.222.0.26 () {58 vars in 1212 bytes} [Tue Sep 27 09:56:52 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname/reboot => generated 99 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Sep 27 09:56:52.222467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-025f857e-312d-4c51-bcee-5c825214e420 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Calling method '.version_select at 0x7f7f683c9f70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:52.225641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-025f857e-312d-4c51-bcee-5c825214e420 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Policy check for os_compute_api:os-hosts:show failed with credentials {'is_admin': False, 'user_id': '13c5c05dd6b244dd8ff15ca2c1ae298a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56f793358f224ff98e1f6bc0e95309c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-025f857e-312d-4c51-bcee-5c825214e420 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Request method failure captured: Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjgRBFDYGySwQXlTtf_VQ7V1cqSE9FFnLxXyOGL1GVMrzQ5vybkblIs0RpEsZgrhIiNA7WNh9lJk_biB06gSxVXa1z245ObxVNGbwx9xCefp8-PAdg0WIDBuwIG_6UnGPnZn62sYX5cHzTJ79YW0u-BeiXF6D98dkLkluJLLWiShvc Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:56:52.228313 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_56f793358f224ff98e1f6bc0e95309c5"}]}] Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 13c5c05dd6b244dd8ff15ca2c1ae298a Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: exception: Policy doesn't allow os_compute_api:os-hosts:show to be performed. Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'ncdv-master-1'} Sep 27 09:56:52.229660 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 270, in show Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'show', Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:52.230372 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:show to be performed. Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:show to be performed. Sep 27 09:56:52.231358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.233711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-025f857e-312d-4c51-bcee-5c825214e420 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:show to be performed. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.234831 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-025f857e-312d-4c51-bcee-5c825214e420 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] 10.222.0.26 "GET /compute/v2.1/os-hosts/ncdv-master-1" status: 403 len: 109 microversion: 2.1 time: 0.017174 Sep 27 09:56:52.235900 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 57/115] 10.222.0.26 () {58 vars in 1170 bytes} [Tue Sep 27 09:56:52 2022] GET /compute/v2.1/os-hosts/ncdv-master-1 => generated 109 bytes in 19 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Sep 27 09:56:52.253269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4ec4a6bd-f08b-447c-b61a-38bf310597ea tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Calling method '.version_select at 0x7f7f68214e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:52.269731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fe72fa04-84a9-460b-9658-89e9bbe3832e tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:52.270441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fe72fa04-84a9-460b-9658-89e9bbe3832e tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:52.271989 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-4ec4a6bd-f08b-447c-b61a-38bf310597ea tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4ec4a6bd-f08b-447c-b61a-38bf310597ea tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Request method failure captured: Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname HTTP/1.1 Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjivOJP_hRj2xyeLXAuWSBw3_nBOAPtdXsD5CjO3KkoCg1J5V9E36ll8NX_wKr3EwTWOl9iCjrVmQT4M1SblE5faOJyGnaKt_5NwtIk_vJ11sYXh4IYL9eC0x9-E1hLYJRvL70iqsV-Hz22J2uUCQmD6vaQHP-IEKrT7WG22r8DV7w Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:52.280375 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8ecef2fe027b4a95a25f3e5aa6360bc2"}]}] Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2a56e8def15742d8a8a8a210ca57d8fe Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: exception: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'nonexistent_hostname'} Sep 27 09:56:52.281946 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 274, in show Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host_name) Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.282777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 282, in show Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.283988 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.286738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4ec4a6bd-f08b-447c-b61a-38bf310597ea tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.288136 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4ec4a6bd-f08b-447c-b61a-38bf310597ea tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts/nonexistent_hostname" status: 404 len: 99 microversion: 2.1 time: 0.040783 Sep 27 09:56:52.288756 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 59/116] 10.222.0.26 () {58 vars in 1191 bytes} [Tue Sep 27 09:56:52 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname => generated 99 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:56:52.301978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe72fa04-84a9-460b-9658-89e9bbe3832e tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:52.310985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe72fa04-84a9-460b-9658-89e9bbe3832e tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:52.312033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe72fa04-84a9-460b-9658-89e9bbe3832e tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:52.313346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe72fa04-84a9-460b-9658-89e9bbe3832e tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:52.313834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe72fa04-84a9-460b-9658-89e9bbe3832e tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:52.314274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe72fa04-84a9-460b-9658-89e9bbe3832e tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:52.315906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-22b492c1-0d0d-487f-afbe-7cb7b0b88549 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Calling method '.version_select at 0x7f7f681cf8b0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:52.319606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-22b492c1-0d0d-487f-afbe-7cb7b0b88549 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Policy check for os_compute_api:os-hosts:shutdown failed with credentials {'is_admin': False, 'user_id': '13c5c05dd6b244dd8ff15ca2c1ae298a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56f793358f224ff98e1f6bc0e95309c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-22b492c1-0d0d-487f-afbe-7cb7b0b88549 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Request method failure captured: Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hosts/ncdv-master-1/shutdown HTTP/1.1 Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjgRBFDYGySwQXlTtf_VQ7V1cqSE9FFnLxXyOGL1GVMrzQ5vybkblIs0RpEsZgrhIiNA7WNh9lJk_biB06gSxVXa1z245ObxVNGbwx9xCefp8-PAdg0WIDBuwIG_6UnGPnZn62sYX5cHzTJ79YW0u-BeiXF6D98dkLkluJLLWiShvc Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:52.327165 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_56f793358f224ff98e1f6bc0e95309c5"}]}] Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 13c5c05dd6b244dd8ff15ca2c1ae298a Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'ncdv-master-1'} Sep 27 09:56:52.328787 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 195, in shutdown Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'shutdown', Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:52.329518 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. Sep 27 09:56:52.330510 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.333889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-22b492c1-0d0d-487f-afbe-7cb7b0b88549 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:shutdown to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.335649 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-22b492c1-0d0d-487f-afbe-7cb7b0b88549 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] 10.222.0.26 "GET /compute/v2.1/os-hosts/ncdv-master-1/shutdown" status: 403 len: 113 microversion: 2.1 time: 0.040151 Sep 27 09:56:52.336845 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 60/117] 10.222.0.26 () {58 vars in 1197 bytes} [Tue Sep 27 09:56:52 2022] GET /compute/v2.1/os-hosts/ncdv-master-1/shutdown => generated 113 bytes in 42 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:52.360810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-59017047-df0f-4f02-b96b-17922456e9a3 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Calling method '.version_select at 0x7f7f681b3e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:52.375088 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-59017047-df0f-4f02-b96b-17922456e9a3 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-59017047-df0f-4f02-b96b-17922456e9a3 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Request method failure captured: Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname/shutdown HTTP/1.1 Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjivOJP_hRj2xyeLXAuWSBw3_nBOAPtdXsD5CjO3KkoCg1J5V9E36ll8NX_wKr3EwTWOl9iCjrVmQT4M1SblE5faOJyGnaKt_5NwtIk_vJ11sYXh4IYL9eC0x9-E1hLYJRvL70iqsV-Hz22J2uUCQmD6vaQHP-IEKrT7WG22r8DV7w Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:52.377850 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8ecef2fe027b4a95a25f3e5aa6360bc2"}]}] Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2a56e8def15742d8a8a8a210ca57d8fe Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: exception: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'nonexistent_hostname'} Sep 27 09:56:52.379045 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 174, in _host_power_action Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = self.api.host_power_action(context, host_name, action) Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5990, in targeted Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:56:52.379949 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 197, in shutdown Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._host_power_action(req, host_name=id, action="shutdown") Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 178, in _host_power_action Sep 27 09:56:52.381873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-59017047-df0f-4f02-b96b-17922456e9a3 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.386445 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-59017047-df0f-4f02-b96b-17922456e9a3 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts/nonexistent_hostname/shutdown" status: 404 len: 99 microversion: 2.1 time: 0.022729 Sep 27 09:56:52.388643 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 61/118] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 09:56:52 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname/shutdown => generated 99 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Sep 27 09:56:52.403964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9e1d4e5f-f24a-4ad5-83bf-4037412621a1 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Calling method '.version_select at 0x7f7f681d3b80>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:52.407771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9e1d4e5f-f24a-4ad5-83bf-4037412621a1 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Policy check for os_compute_api:os-hosts:start failed with credentials {'is_admin': False, 'user_id': '13c5c05dd6b244dd8ff15ca2c1ae298a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56f793358f224ff98e1f6bc0e95309c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9e1d4e5f-f24a-4ad5-83bf-4037412621a1 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Request method failure captured: Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hosts/ncdv-master-1/startup HTTP/1.1 Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjgRBFDYGySwQXlTtf_VQ7V1cqSE9FFnLxXyOGL1GVMrzQ5vybkblIs0RpEsZgrhIiNA7WNh9lJk_biB06gSxVXa1z245ObxVNGbwx9xCefp8-PAdg0WIDBuwIG_6UnGPnZn62sYX5cHzTJ79YW0u-BeiXF6D98dkLkluJLLWiShvc Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:52.409519 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_56f793358f224ff98e1f6bc0e95309c5"}]}] Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 13c5c05dd6b244dd8ff15ca2c1ae298a Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-hosts:start to be performed. Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'ncdv-master-1'} Sep 27 09:56:52.411142 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 187, in startup Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'start', Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:52.411997 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:start to be performed. Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:start to be performed. Sep 27 09:56:52.413670 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.416960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9e1d4e5f-f24a-4ad5-83bf-4037412621a1 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:start to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.418464 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9e1d4e5f-f24a-4ad5-83bf-4037412621a1 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] 10.222.0.26 "GET /compute/v2.1/os-hosts/ncdv-master-1/startup" status: 403 len: 110 microversion: 2.1 time: 0.028493 Sep 27 09:56:52.420878 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 62/119] 10.222.0.26 () {58 vars in 1194 bytes} [Tue Sep 27 09:56:52 2022] GET /compute/v2.1/os-hosts/ncdv-master-1/startup => generated 110 bytes in 31 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:52.432832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-db6eb13f-b9ef-456c-a106-3ab5bbfd56d1 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Calling method '.version_select at 0x7f7f681cf310>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:52.444388 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-db6eb13f-b9ef-456c-a106-3ab5bbfd56d1 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-db6eb13f-b9ef-456c-a106-3ab5bbfd56d1 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Request method failure captured: Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hosts/nonexistent_hostname/startup HTTP/1.1 Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjivOJP_hRj2xyeLXAuWSBw3_nBOAPtdXsD5CjO3KkoCg1J5V9E36ll8NX_wKr3EwTWOl9iCjrVmQT4M1SblE5faOJyGnaKt_5NwtIk_vJ11sYXh4IYL9eC0x9-E1hLYJRvL70iqsV-Hz22J2uUCQmD6vaQHP-IEKrT7WG22r8DV7w Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:52.447201 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8ecef2fe027b4a95a25f3e5aa6360bc2"}]}] Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2a56e8def15742d8a8a8a210ca57d8fe Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: exception: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'nonexistent_hostname'} Sep 27 09:56:52.448858 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 174, in _host_power_action Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = self.api.host_power_action(context, host_name, action) Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5990, in targeted Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:56:52.449763 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 189, in startup Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._host_power_action(req, host_name=id, action="startup") Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 178, in _host_power_action Sep 27 09:56:52.451109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-db6eb13f-b9ef-456c-a106-3ab5bbfd56d1 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-db6eb13f-b9ef-456c-a106-3ab5bbfd56d1 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts/nonexistent_hostname/startup" status: 404 len: 99 microversion: 2.1 time: 0.022047 Sep 27 09:56:52.452359 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 63/120] 10.222.0.26 () {58 vars in 1215 bytes} [Tue Sep 27 09:56:52 2022] GET /compute/v2.1/os-hosts/nonexistent_hostname/startup => generated 99 bytes in 23 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:56:52.459019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-99e953ce-d671-4ee9-9332-3786968ae472 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Action: 'update', calling method: .version_select at 0x7f7f681d3670>, body: {"status": "enable", "maintenance_mode": "invalid"} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-99e953ce-d671-4ee9-9332-3786968ae472 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Request method failure captured: Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 51 Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjivOJP_hRj2xyeLXAuWSBw3_nBOAPtdXsD5CjO3KkoCg1J5V9E36ll8NX_wKr3EwTWOl9iCjrVmQT4M1SblE5faOJyGnaKt_5NwtIk_vJ11sYXh4IYL9eC0x9-E1hLYJRvL70iqsV-Hz22J2uUCQmD6vaQHP-IEKrT7WG22r8DV7w Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:52.464073 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8ecef2fe027b4a95a25f3e5aa6360bc2"}]}] Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2a56e8def15742d8a8a8a210ca57d8fe Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: {"status": "enable", "maintenance_mode": "invalid"} Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.465675 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'ncdv-master-1', 'body': {'status': 'enable', 'maintenance_mode': 'invalid'}} Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['maintenance_mode']: Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'enum': ['enable', Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'disable', Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'Enable', Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'Disable', Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'ENABLE', Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'DISABLE'], Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'string'} Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance: Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'invalid' Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.466399 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.467771 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.469434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.469434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 09:56:52.469434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 09:56:52.469434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Sep 27 09:56:52.469434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.469434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-99e953ce-d671-4ee9-9332-3786968ae472 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Returning 400 to user: Invalid input for field/attribute maintenance_mode. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.469434 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-99e953ce-d671-4ee9-9332-3786968ae472 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 400 len: 199 microversion: 2.1 time: 0.010785 Sep 27 09:56:52.469434 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 64/121] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 09:56:52 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 199 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 09:56:52.477534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca1821f0-f197-49e5-b003-2f4dce903432 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Action: 'update', calling method: .version_select at 0x7f7f681cf040>, body: {"status": "invalid", "maintenance_mode": "enable"} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca1821f0-f197-49e5-b003-2f4dce903432 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Request method failure captured: Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 51 Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjivOJP_hRj2xyeLXAuWSBw3_nBOAPtdXsD5CjO3KkoCg1J5V9E36ll8NX_wKr3EwTWOl9iCjrVmQT4M1SblE5faOJyGnaKt_5NwtIk_vJ11sYXh4IYL9eC0x9-E1hLYJRvL70iqsV-Hz22J2uUCQmD6vaQHP-IEKrT7WG22r8DV7w Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:52.486480 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8ecef2fe027b4a95a25f3e5aa6360bc2"}]}] Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2a56e8def15742d8a8a8a210ca57d8fe Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: {"status": "invalid", "maintenance_mode": "enable"} Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.488011 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'ncdv-master-1', 'body': {'status': 'invalid', 'maintenance_mode': 'enable'}} Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['status']: Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'enum': ['enable', Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'disable', Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'Enable', Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'Disable', Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'ENABLE', Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'DISABLE'], Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'string'} Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance: Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'invalid' Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.489704 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.491451 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.493647 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.493647 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 09:56:52.493647 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 09:56:52.493647 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] Sep 27 09:56:52.493647 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.495221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca1821f0-f197-49e5-b003-2f4dce903432 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Returning 400 to user: Invalid input for field/attribute status. Value: invalid. 'invalid' is not one of ['enable', 'disable', 'Enable', 'Disable', 'ENABLE', 'DISABLE'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.496558 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ca1821f0-f197-49e5-b003-2f4dce903432 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 400 len: 189 microversion: 2.1 time: 0.024079 Sep 27 09:56:52.499682 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 65/122] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 09:56:52 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 189 bytes in 28 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 09:56:52.512338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c4fac9c-d7ef-476b-8380-ee365e139205 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Action: 'update', calling method: .version_select at 0x7f7f682cfaf0>, body: {"status": "enable", "maintenance_mode": "enable"} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:52.518704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c4fac9c-d7ef-476b-8380-ee365e139205 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Policy check for os_compute_api:os-hosts:update failed with credentials {'is_admin': False, 'user_id': '13c5c05dd6b244dd8ff15ca2c1ae298a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56f793358f224ff98e1f6bc0e95309c5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c4fac9c-d7ef-476b-8380-ee365e139205 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Request method failure captured: Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 50 Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjgRBFDYGySwQXlTtf_VQ7V1cqSE9FFnLxXyOGL1GVMrzQ5vybkblIs0RpEsZgrhIiNA7WNh9lJk_biB06gSxVXa1z245ObxVNGbwx9xCefp8-PAdg0WIDBuwIG_6UnGPnZn62sYX5cHzTJ79YW0u-BeiXF6D98dkLkluJLLWiShvc Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:56:52.520891 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/56f793358f224ff98e1f6bc0e95309c5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_56f793358f224ff98e1f6bc0e95309c5"}]}] Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 56f793358f224ff98e1f6bc0e95309c5 Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-892983742 Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 13c5c05dd6b244dd8ff15ca2c1ae298a Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-892983742-project Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: {"status": "enable", "maintenance_mode": "enable"} Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-hosts:update to be performed. Sep 27 09:56:52.522039 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'ncdv-master-1', 'body': {'status': 'enable', 'maintenance_mode': 'enable'}} Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 112, in update Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(hosts_policies.POLICY_NAME % 'update', Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:52.522836 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hosts:update to be performed. Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hosts:update to be performed. Sep 27 09:56:52.523884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.527792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c4fac9c-d7ef-476b-8380-ee365e139205 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hosts:update to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.529024 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c4fac9c-d7ef-476b-8380-ee365e139205 tempest-HostsAdminNegativeTestJSON-892983742 tempest-HostsAdminNegativeTestJSON-892983742-project] 10.222.0.26 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 403 len: 111 microversion: 2.1 time: 0.022764 Sep 27 09:56:52.529857 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 66/123] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 09:56:52 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 111 bytes in 25 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:56:52.541334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-559a5c37-2a94-44b8-9e27-4d4572abce9b tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Action: 'update', calling method: .version_select at 0x7f7f681d3ca0>, body: {"status": null, "maintenance_mode": null} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-559a5c37-2a94-44b8-9e27-4d4572abce9b tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Request method failure captured: Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/os-hosts/ncdv-master-1 HTTP/1.1 Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 42 Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjivOJP_hRj2xyeLXAuWSBw3_nBOAPtdXsD5CjO3KkoCg1J5V9E36ll8NX_wKr3EwTWOl9iCjrVmQT4M1SblE5faOJyGnaKt_5NwtIk_vJ11sYXh4IYL9eC0x9-E1hLYJRvL70iqsV-Hz22J2uUCQmD6vaQHP-IEKrT7WG22r8DV7w Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:52.547994 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8ecef2fe027b4a95a25f3e5aa6360bc2"}]}] Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2a56e8def15742d8a8a8a210ca57d8fe Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: {"status": null, "maintenance_mode": null} Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute status. Value: None. None is not of type 'string' Sep 27 09:56:52.549516 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'ncdv-master-1', 'body': {'status': None, 'maintenance_mode': None}} Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: None is not of type 'string' Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['status']: Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'enum': ['enable', Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'disable', Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'Enable', Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'Disable', Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'ENABLE', Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'DISABLE'], Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'string'} Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance: Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi None Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.550312 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute status. Value: None. None is not of type 'string' Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.556310 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.557755 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 09:56:52.557755 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 09:56:52.557755 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute status. Value: None. None is not of type 'string' Sep 27 09:56:52.557755 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.557755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-559a5c37-2a94-44b8-9e27-4d4572abce9b tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Returning 400 to user: Invalid input for field/attribute status. Value: None. None is not of type 'string' {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.557755 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-559a5c37-2a94-44b8-9e27-4d4572abce9b tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-hosts/ncdv-master-1" status: 400 len: 127 microversion: 2.1 time: 0.014209 Sep 27 09:56:52.557755 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 67/124] 10.222.0.26 () {60 vars in 1190 bytes} [Tue Sep 27 09:56:52 2022] PUT /compute/v2.1/os-hosts/ncdv-master-1 => generated 127 bytes in 16 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 09:56:52.564281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23ad545f-d363-4dcd-bce5-e54e370e8439 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Action: 'update', calling method: .version_select at 0x7f7f681d3160>, body: {"status": "enable", "maintenance_mode": "enable"} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:52.568186 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.hosts [None req-23ad545f-d363-4dcd-bce5-e54e370e8439 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Enabling host nonexistent_hostname. Sep 27 09:56:52.585344 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-23ad545f-d363-4dcd-bce5-e54e370e8439 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] HTTP exception thrown: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23ad545f-d363-4dcd-bce5-e54e370e8439 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Request method failure captured: Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/os-hosts/nonexistent_hostname HTTP/1.1 Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 50 Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjivOJP_hRj2xyeLXAuWSBw3_nBOAPtdXsD5CjO3KkoCg1J5V9E36ll8NX_wKr3EwTWOl9iCjrVmQT4M1SblE5faOJyGnaKt_5NwtIk_vJ11sYXh4IYL9eC0x9-E1hLYJRvL70iqsV-Hz22J2uUCQmD6vaQHP-IEKrT7WG22r8DV7w Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:56:52.587989 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8ecef2fe027b4a95a25f3e5aa6360bc2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8ecef2fe027b4a95a25f3e5aa6360bc2"}]}] Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8ecef2fe027b4a95a25f3e5aa6360bc2 Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HostsAdminNegativeTestJSON-1460268516 Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2a56e8def15742d8a8a8a210ca57d8fe Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HostsAdminNegativeTestJSON-1460268516-project-admin Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: {"status": "enable", "maintenance_mode": "enable"} Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: exception: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.589627 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'nonexistent_hostname', 'body': {'status': 'enable', 'maintenance_mode': 'enable'}} Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 159, in _set_enabled_status Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = self.api.set_host_enabled(context, host_name, enabled) Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5990, in targeted Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host) Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Sep 27 09:56:52.590563 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:56:52.592024 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 124, in update Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result['status'] = self._set_enabled_status(context, id, status) Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hosts.py", line 163, in _set_enabled_status Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Host 'nonexistent_hostname' is not mapped to any cell Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:56:52.594011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23ad545f-d363-4dcd-bce5-e54e370e8439 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] Returning 404 to user: Host 'nonexistent_hostname' is not mapped to any cell {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:56:52.597106 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-23ad545f-d363-4dcd-bce5-e54e370e8439 tempest-HostsAdminNegativeTestJSON-1460268516 tempest-HostsAdminNegativeTestJSON-1460268516-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-hosts/nonexistent_hostname" status: 404 len: 99 microversion: 2.1 time: 0.033520 Sep 27 09:56:52.597106 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 68/125] 10.222.0.26 () {60 vars in 1211 bytes} [Tue Sep 27 09:56:52 2022] PUT /compute/v2.1/os-hosts/nonexistent_hostname => generated 99 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:56:52.668734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fe72fa04-84a9-460b-9658-89e9bbe3832e tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:52.678955 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fe72fa04-84a9-460b-9658-89e9bbe3832e tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.427535 Sep 27 09:56:52.695661 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 58/126] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:52 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:53.425025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersWithSpecificFlavorTestJSON-server-106879088", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "720147853", "adminPass": "***", "networks": [{"uuid": "ef5e7302-9921-42d2-81b1-295d060e5a12"}], "security_groups": [{"name": "tempest-securitygroup--891093939"}], "key_name": "tempest-keypair-211851818", "user_data": "}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:53.572448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] validate_networks() for [('ef5e7302-9921-42d2-81b1-295d060e5a12', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:56:53.702173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6979564e-fd31-4911-a7ca-cf333a706801 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:53.702885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6979564e-fd31-4911-a7ca-cf333a706801 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:53.710203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6979564e-fd31-4911-a7ca-cf333a706801 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:53.711154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6979564e-fd31-4911-a7ca-cf333a706801 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:53.711919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6979564e-fd31-4911-a7ca-cf333a706801 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:53.713089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6979564e-fd31-4911-a7ca-cf333a706801 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:53.713833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6979564e-fd31-4911-a7ca-cf333a706801 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:53.714701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6979564e-fd31-4911-a7ca-cf333a706801 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:53.935236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6979564e-fd31-4911-a7ca-cf333a706801 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:53.944816 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6979564e-fd31-4911-a7ca-cf333a706801 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.247179 Sep 27 09:56:53.945941 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 59/127] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:53 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:56:54.610453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:56:54.611729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:56:54.612433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:56:54.613617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:56:54.614192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:56:54.626296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:54.626914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:54.628246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:54.629131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:54.629697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:54.630514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:54.640176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=23 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:54.641292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=89 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:54.659318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Getting quotas for project 1881e013b4b844cba1afce9fd21f8dd7. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:56:54.665854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Getting quotas for user b4cebc85e62046d28fab159c6bb13e15 and project 1881e013b4b844cba1afce9fd21f8dd7. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:56:54.682394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:56:54.684990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:56:54.797957 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-90ebe93d-32f8-440b-b8bf-99d25c379d6d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.377551 Sep 27 09:56:54.803033 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 69/128] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:56:53 2022] POST /compute/v2.1/servers => generated 401 bytes in 1381 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:56:54.818761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-edc6ec8e-f62c-4a19-a17d-41d71b2ff37c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:54.820917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-edc6ec8e-f62c-4a19-a17d-41d71b2ff37c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:54.930167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-edc6ec8e-f62c-4a19-a17d-41d71b2ff37c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:54.948724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminNegativeTestJSON-server-186946746", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:54.959161 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-edc6ec8e-f62c-4a19-a17d-41d71b2ff37c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1261 microversion: 2.1 time: 0.150295 Sep 27 09:56:54.959765 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 70/129] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:54 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1261 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:56:54.982918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5411274d-f9bd-4a71-a52f-bb688e522142 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:54.985884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5411274d-f9bd-4a71-a52f-bb688e522142 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:55.007960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5411274d-f9bd-4a71-a52f-bb688e522142 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:55.016390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5411274d-f9bd-4a71-a52f-bb688e522142 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:55.018472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5411274d-f9bd-4a71-a52f-bb688e522142 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:55.019233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5411274d-f9bd-4a71-a52f-bb688e522142 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:55.019719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5411274d-f9bd-4a71-a52f-bb688e522142 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:55.020182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5411274d-f9bd-4a71-a52f-bb688e522142 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:55.062923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:56:55.357794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5411274d-f9bd-4a71-a52f-bb688e522142 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:55.365969 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5411274d-f9bd-4a71-a52f-bb688e522142 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.399467 Sep 27 09:56:55.371427 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 71/130] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:54 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:55.469428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:56:55.469850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:56:55.470522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:56:55.471084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:56:55.471819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:56:55.480001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=31 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:55.484566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:55.485114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:55.485554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:55.486468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:55.486929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:55.487357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:55.494167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=15 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:55.495021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=93 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:56:55.511473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Getting quotas for project b33d41b411144dbbbdb012e694857d27. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:56:55.517894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Getting quotas for user d7d3aef1d4da47b2ab5454c3eb0346a6 and project b33d41b411144dbbbdb012e694857d27. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:56:55.538478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:56:55.541720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:56:55.616886 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5b22babd-a159-4f97-a06c-80fdfd56de30 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.022471 Sep 27 09:56:55.625174 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 60/131] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:56:54 2022] POST /compute/v2.1/servers => generated 380 bytes in 1032 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:56:55.638079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1746ab53-44fe-40d2-b678-7aafbeb0f75e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:55.639037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1746ab53-44fe-40d2-b678-7aafbeb0f75e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:55.710917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1746ab53-44fe-40d2-b678-7aafbeb0f75e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:55.720190 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1746ab53-44fe-40d2-b678-7aafbeb0f75e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1221 microversion: 2.1 time: 0.090361 Sep 27 09:56:55.721718 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 61/132] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:55 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1221 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:55.730363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiagnosticsNegativeTest-server-1140902090", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:55.809027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:56:55.990993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a48fb5d7-f22c-481e-af3a-72227d07954b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:55.992063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a48fb5d7-f22c-481e-af3a-72227d07954b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:56.004246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a48fb5d7-f22c-481e-af3a-72227d07954b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:56.005645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a48fb5d7-f22c-481e-af3a-72227d07954b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:56.006603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a48fb5d7-f22c-481e-af3a-72227d07954b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:56.008031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a48fb5d7-f22c-481e-af3a-72227d07954b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:56.009158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a48fb5d7-f22c-481e-af3a-72227d07954b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:56.011200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a48fb5d7-f22c-481e-af3a-72227d07954b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:56.087616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:56:56.088046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:56:56.088619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:56:56.089069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:56:56.089730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:56:56.104466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:56.105785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:56.106887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:56.108282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:56.110047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:56.116207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:56.121562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a48fb5d7-f22c-481e-af3a-72227d07954b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:56.131033 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a48fb5d7-f22c-481e-af3a-72227d07954b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1265 microversion: 2.1 time: 0.148664 Sep 27 09:56:56.151327 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 62/133] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:55 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1265 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:56:56.170136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Getting quotas for project ec89ada1460d4275812339f2d6a93042. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:56:56.175847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Getting quotas for user b3dd80698e4c471abe7cd81809fa206c and project ec89ada1460d4275812339f2d6a93042. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:56:56.200947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:56:56.204326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:56:56.286549 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8c82d256-d016-478f-9e29-f4a5960a2274 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.804444 Sep 27 09:56:56.300923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0d5c2462-9e0b-4f66-b30b-db9254c238cc tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:56.301573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0d5c2462-9e0b-4f66-b30b-db9254c238cc tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:56.324958 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 72/134] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:56:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 843 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:56:56.392698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-212b4422-fac1-4d6c-9af9-ebfd075753d0 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:56.393573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-212b4422-fac1-4d6c-9af9-ebfd075753d0 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:56.409257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-212b4422-fac1-4d6c-9af9-ebfd075753d0 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:56.409759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-212b4422-fac1-4d6c-9af9-ebfd075753d0 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:56.419408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-212b4422-fac1-4d6c-9af9-ebfd075753d0 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:56.421787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-212b4422-fac1-4d6c-9af9-ebfd075753d0 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:56.423758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-212b4422-fac1-4d6c-9af9-ebfd075753d0 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:56.424295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-212b4422-fac1-4d6c-9af9-ebfd075753d0 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:56.465717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0d5c2462-9e0b-4f66-b30b-db9254c238cc tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:56.478056 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0d5c2462-9e0b-4f66-b30b-db9254c238cc tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1223 microversion: 2.1 time: 0.181622 Sep 27 09:56:56.480851 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 63/135] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:56 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1223 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:56:56.674937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-212b4422-fac1-4d6c-9af9-ebfd075753d0 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:56.685573 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-212b4422-fac1-4d6c-9af9-ebfd075753d0 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.297962 Sep 27 09:56:56.687933 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 73/136] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:56 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:56:56.744571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-460c35c8-5729-47f5-a5e2-dd4945e0b738 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:56.745299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-460c35c8-5729-47f5-a5e2-dd4945e0b738 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:56.752430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-460c35c8-5729-47f5-a5e2-dd4945e0b738 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:56.752952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-460c35c8-5729-47f5-a5e2-dd4945e0b738 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:56.753431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-460c35c8-5729-47f5-a5e2-dd4945e0b738 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:56.754216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-460c35c8-5729-47f5-a5e2-dd4945e0b738 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:56.754702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-460c35c8-5729-47f5-a5e2-dd4945e0b738 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:56.755134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-460c35c8-5729-47f5-a5e2-dd4945e0b738 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:56.841086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-460c35c8-5729-47f5-a5e2-dd4945e0b738 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:56.853419 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-460c35c8-5729-47f5-a5e2-dd4945e0b738 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1217 microversion: 2.1 time: 0.113290 Sep 27 09:56:56.854128 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 74/137] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:56 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1217 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:56:56.940796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f9824de2-030a-4a60-81c7-6ac406fd4e9a req-49085ec9-7972-4d91-bf43-3c18942ffbbf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8febbed7-7a38-4051-8910-bdb62bc11eac", "name": "network-vif-plugged", "status": "completed", "tag": "ed45a4db-ff3b-4d14-b5f9-f0a60046bd2c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:56:56.957943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f9824de2-030a-4a60-81c7-6ac406fd4e9a req-49085ec9-7972-4d91-bf43-3c18942ffbbf service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:56.958461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f9824de2-030a-4a60-81c7-6ac406fd4e9a req-49085ec9-7972-4d91-bf43-3c18942ffbbf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:56.958834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f9824de2-030a-4a60-81c7-6ac406fd4e9a req-49085ec9-7972-4d91-bf43-3c18942ffbbf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:56.984303 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f9824de2-030a-4a60-81c7-6ac406fd4e9a req-49085ec9-7972-4d91-bf43-3c18942ffbbf service nova] Creating event network-vif-plugged:ed45a4db-ff3b-4d14-b5f9-f0a60046bd2c for instance 8febbed7-7a38-4051-8910-bdb62bc11eac on nch1-master-1 Sep 27 09:56:56.990693 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f9824de2-030a-4a60-81c7-6ac406fd4e9a req-49085ec9-7972-4d91-bf43-3c18942ffbbf service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.271281 Sep 27 09:56:56.997259 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 64/138] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:56:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:56:57.154593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9f264c71-9bde-4907-87f2-4aa86fb170ae tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:57.155544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9f264c71-9bde-4907-87f2-4aa86fb170ae tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:57.167435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f264c71-9bde-4907-87f2-4aa86fb170ae tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.168986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f264c71-9bde-4907-87f2-4aa86fb170ae tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.170334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f264c71-9bde-4907-87f2-4aa86fb170ae tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.172792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f264c71-9bde-4907-87f2-4aa86fb170ae tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.173469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f264c71-9bde-4907-87f2-4aa86fb170ae tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.176227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f264c71-9bde-4907-87f2-4aa86fb170ae tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.281296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9f264c71-9bde-4907-87f2-4aa86fb170ae tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:57.290881 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9f264c71-9bde-4907-87f2-4aa86fb170ae tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1331 microversion: 2.1 time: 0.140277 Sep 27 09:56:57.291731 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 75/139] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:57 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1331 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:57.480025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13731a41-39d0-4f6f-95b2-4d6595119588 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Calling method '.version_select at 0x7f7f683c9a60>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:57.482667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13731a41-39d0-4f6f-95b2-4d6595119588 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.483525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13731a41-39d0-4f6f-95b2-4d6595119588 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.484137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13731a41-39d0-4f6f-95b2-4d6595119588 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.500578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13731a41-39d0-4f6f-95b2-4d6595119588 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.501114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13731a41-39d0-4f6f-95b2-4d6595119588 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.501542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13731a41-39d0-4f6f-95b2-4d6595119588 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.507538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-264e2097-cedf-4df5-8f32-68b43b0a208b tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:57.508191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-264e2097-cedf-4df5-8f32-68b43b0a208b tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:57.512403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13731a41-39d0-4f6f-95b2-4d6595119588 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.512996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13731a41-39d0-4f6f-95b2-4d6595119588 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.513424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13731a41-39d0-4f6f-95b2-4d6595119588 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.514306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-264e2097-cedf-4df5-8f32-68b43b0a208b tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.514770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-264e2097-cedf-4df5-8f32-68b43b0a208b tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.515218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-264e2097-cedf-4df5-8f32-68b43b0a208b tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.515921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-264e2097-cedf-4df5-8f32-68b43b0a208b tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.516430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-264e2097-cedf-4df5-8f32-68b43b0a208b tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.516886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-264e2097-cedf-4df5-8f32-68b43b0a208b tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.519494 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13731a41-39d0-4f6f-95b2-4d6595119588 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.28 time: 0.194602 Sep 27 09:56:57.520054 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 65/140] 10.222.0.26 () {60 vars in 1187 bytes} [Tue Sep 27 09:56:57 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:56:57.531121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-01d32421-fbdc-4c86-98ac-6b4ac0affcd2 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Calling method '.version_select at 0x7f7f683c9790>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:57.533050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01d32421-fbdc-4c86-98ac-6b4ac0affcd2 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.533599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01d32421-fbdc-4c86-98ac-6b4ac0affcd2 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.534098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01d32421-fbdc-4c86-98ac-6b4ac0affcd2 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.549454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01d32421-fbdc-4c86-98ac-6b4ac0affcd2 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.550016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01d32421-fbdc-4c86-98ac-6b4ac0affcd2 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.550518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01d32421-fbdc-4c86-98ac-6b4ac0affcd2 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.557887 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-01d32421-fbdc-4c86-98ac-6b4ac0affcd2 tempest-HypervisorAdminV228Test-1148355733 tempest-HypervisorAdminV228Test-1148355733-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/1" status: 200 len: 718 microversion: 2.28 time: 0.030475 Sep 27 09:56:57.558621 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 66/141] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 09:56:57 2022] GET /compute/v2.1/os-hypervisors/1 => generated 718 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 09:56:57.581202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-264e2097-cedf-4df5-8f32-68b43b0a208b tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:57.589478 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-264e2097-cedf-4df5-8f32-68b43b0a208b tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1219 microversion: 2.1 time: 0.085850 Sep 27 09:56:57.590109 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 76/142] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:57 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1219 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:57.708772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cd5c5a3c-4154-44f7-896e-9e96ae1e5e17 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:57.709588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cd5c5a3c-4154-44f7-896e-9e96ae1e5e17 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:57.717859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5c5a3c-4154-44f7-896e-9e96ae1e5e17 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.718513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5c5a3c-4154-44f7-896e-9e96ae1e5e17 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.718998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5c5a3c-4154-44f7-896e-9e96ae1e5e17 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.719764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5c5a3c-4154-44f7-896e-9e96ae1e5e17 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.720258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5c5a3c-4154-44f7-896e-9e96ae1e5e17 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.720923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5c5a3c-4154-44f7-896e-9e96ae1e5e17 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.878423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9232429-e9ee-429e-9322-31dffc09c86c tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:57.879771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9232429-e9ee-429e-9322-31dffc09c86c tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:57.887702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9232429-e9ee-429e-9322-31dffc09c86c tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.888512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9232429-e9ee-429e-9322-31dffc09c86c tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.888991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cd5c5a3c-4154-44f7-896e-9e96ae1e5e17 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:57.893367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9232429-e9ee-429e-9322-31dffc09c86c tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.894375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9232429-e9ee-429e-9322-31dffc09c86c tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:57.894821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9232429-e9ee-429e-9322-31dffc09c86c tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:57.895416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9232429-e9ee-429e-9322-31dffc09c86c tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:57.901676 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cd5c5a3c-4154-44f7-896e-9e96ae1e5e17 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.197097 Sep 27 09:56:57.903097 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 67/143] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:57 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:57.972831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c9232429-e9ee-429e-9322-31dffc09c86c tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:57.997244 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9232429-e9ee-429e-9322-31dffc09c86c tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1273 microversion: 2.1 time: 0.122936 Sep 27 09:56:58.004698 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 77/144] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:57 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1273 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:56:58.316917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3e762f43-90f2-4d2f-80d8-9d09cfa05db5 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:58.317547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3e762f43-90f2-4d2f-80d8-9d09cfa05db5 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:58.324199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e762f43-90f2-4d2f-80d8-9d09cfa05db5 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:58.324779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e762f43-90f2-4d2f-80d8-9d09cfa05db5 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:58.325446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e762f43-90f2-4d2f-80d8-9d09cfa05db5 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:58.326313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e762f43-90f2-4d2f-80d8-9d09cfa05db5 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:58.326926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e762f43-90f2-4d2f-80d8-9d09cfa05db5 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:58.327601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e762f43-90f2-4d2f-80d8-9d09cfa05db5 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:58.402198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3e762f43-90f2-4d2f-80d8-9d09cfa05db5 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:58.414893 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3e762f43-90f2-4d2f-80d8-9d09cfa05db5 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1319 microversion: 2.1 time: 0.102765 Sep 27 09:56:58.415622 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 68/145] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:58 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1319 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:58.618091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ccd6a0d-289c-4d04-9e49-59929c55f9a5 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:58.619006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ccd6a0d-289c-4d04-9e49-59929c55f9a5 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:58.627893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ccd6a0d-289c-4d04-9e49-59929c55f9a5 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:58.628542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ccd6a0d-289c-4d04-9e49-59929c55f9a5 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:58.629082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ccd6a0d-289c-4d04-9e49-59929c55f9a5 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:58.629802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ccd6a0d-289c-4d04-9e49-59929c55f9a5 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:58.630284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ccd6a0d-289c-4d04-9e49-59929c55f9a5 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:58.630710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ccd6a0d-289c-4d04-9e49-59929c55f9a5 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:58.708197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2ccd6a0d-289c-4d04-9e49-59929c55f9a5 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:58.715847 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ccd6a0d-289c-4d04-9e49-59929c55f9a5 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1219 microversion: 2.1 time: 0.103720 Sep 27 09:56:58.716511 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 78/146] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:58 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1219 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:58.928244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07d8a5e8-fe8c-49e7-9e1d-fb570b6204a2 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:58.928978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-07d8a5e8-fe8c-49e7-9e1d-fb570b6204a2 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:58.938331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07d8a5e8-fe8c-49e7-9e1d-fb570b6204a2 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:58.938878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07d8a5e8-fe8c-49e7-9e1d-fb570b6204a2 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:58.940076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07d8a5e8-fe8c-49e7-9e1d-fb570b6204a2 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:58.944204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07d8a5e8-fe8c-49e7-9e1d-fb570b6204a2 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:58.944775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07d8a5e8-fe8c-49e7-9e1d-fb570b6204a2 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:58.945408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07d8a5e8-fe8c-49e7-9e1d-fb570b6204a2 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:59.024587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-00eb8ff0-21ad-4a78-a51e-079b182f0c0e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:59.025263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-00eb8ff0-21ad-4a78-a51e-079b182f0c0e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:59.037855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00eb8ff0-21ad-4a78-a51e-079b182f0c0e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:59.038362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00eb8ff0-21ad-4a78-a51e-079b182f0c0e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:59.038793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00eb8ff0-21ad-4a78-a51e-079b182f0c0e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:59.040504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00eb8ff0-21ad-4a78-a51e-079b182f0c0e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:59.041502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00eb8ff0-21ad-4a78-a51e-079b182f0c0e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:59.042443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00eb8ff0-21ad-4a78-a51e-079b182f0c0e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:59.127312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-00eb8ff0-21ad-4a78-a51e-079b182f0c0e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:59.137293 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-00eb8ff0-21ad-4a78-a51e-079b182f0c0e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1273 microversion: 2.1 time: 0.117737 Sep 27 09:56:59.138004 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 79/147] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:59 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1273 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:59.153433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-07d8a5e8-fe8c-49e7-9e1d-fb570b6204a2 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:59.162121 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-07d8a5e8-fe8c-49e7-9e1d-fb570b6204a2 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1339 microversion: 2.1 time: 0.238574 Sep 27 09:56:59.163106 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 69/148] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:58 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1339 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:59.443389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1da4df26-c6fa-49c7-a3fb-f3c985f0f735 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:59.444015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1da4df26-c6fa-49c7-a3fb-f3c985f0f735 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:59.450569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1da4df26-c6fa-49c7-a3fb-f3c985f0f735 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:59.451287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1da4df26-c6fa-49c7-a3fb-f3c985f0f735 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:59.451730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1da4df26-c6fa-49c7-a3fb-f3c985f0f735 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:59.452885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1da4df26-c6fa-49c7-a3fb-f3c985f0f735 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:59.453665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1da4df26-c6fa-49c7-a3fb-f3c985f0f735 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:59.454236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1da4df26-c6fa-49c7-a3fb-f3c985f0f735 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:59.532597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1da4df26-c6fa-49c7-a3fb-f3c985f0f735 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:59.543757 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1da4df26-c6fa-49c7-a3fb-f3c985f0f735 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1319 microversion: 2.1 time: 0.104454 Sep 27 09:56:59.544352 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 80/149] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:59 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1319 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:56:59.738772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-de129efd-0487-451f-a229-e75d4066cea2 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:56:59.739395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-de129efd-0487-451f-a229-e75d4066cea2 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:56:59.746032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de129efd-0487-451f-a229-e75d4066cea2 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:59.746603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de129efd-0487-451f-a229-e75d4066cea2 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:59.747100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de129efd-0487-451f-a229-e75d4066cea2 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:59.747992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de129efd-0487-451f-a229-e75d4066cea2 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:56:59.748458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de129efd-0487-451f-a229-e75d4066cea2 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:56:59.749085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de129efd-0487-451f-a229-e75d4066cea2 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:56:59.819327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-de129efd-0487-451f-a229-e75d4066cea2 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:56:59.828035 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-de129efd-0487-451f-a229-e75d4066cea2 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1219 microversion: 2.1 time: 0.093113 Sep 27 09:56:59.828982 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 70/150] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:56:59 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:00.163049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b36e3511-2990-486d-8796-fac29058f87f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:00.163808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b36e3511-2990-486d-8796-fac29058f87f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:00.170588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b36e3511-2990-486d-8796-fac29058f87f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.171085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b36e3511-2990-486d-8796-fac29058f87f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.171524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b36e3511-2990-486d-8796-fac29058f87f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.172262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b36e3511-2990-486d-8796-fac29058f87f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.172761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b36e3511-2990-486d-8796-fac29058f87f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.173188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b36e3511-2990-486d-8796-fac29058f87f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.188475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69205a37-8802-4ef3-be1a-e02a7724b911 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:00.189217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69205a37-8802-4ef3-be1a-e02a7724b911 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:00.196711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69205a37-8802-4ef3-be1a-e02a7724b911 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.198261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69205a37-8802-4ef3-be1a-e02a7724b911 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.198928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69205a37-8802-4ef3-be1a-e02a7724b911 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.202999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69205a37-8802-4ef3-be1a-e02a7724b911 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.204358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69205a37-8802-4ef3-be1a-e02a7724b911 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.204967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69205a37-8802-4ef3-be1a-e02a7724b911 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.247964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b36e3511-2990-486d-8796-fac29058f87f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:00.258508 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b36e3511-2990-486d-8796-fac29058f87f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1273 microversion: 2.1 time: 0.099342 Sep 27 09:57:00.260029 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 81/151] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:00 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1273 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:00.356419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69205a37-8802-4ef3-be1a-e02a7724b911 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:00.364439 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69205a37-8802-4ef3-be1a-e02a7724b911 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1489 microversion: 2.1 time: 0.180073 Sep 27 09:57:00.365046 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 71/152] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:00 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1489 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:00.388709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d223cfe7-b96e-4a35-b7c3-7d300c6120c5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:00.389332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d223cfe7-b96e-4a35-b7c3-7d300c6120c5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:00.395569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d223cfe7-b96e-4a35-b7c3-7d300c6120c5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.396055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d223cfe7-b96e-4a35-b7c3-7d300c6120c5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.396484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d223cfe7-b96e-4a35-b7c3-7d300c6120c5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.397382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d223cfe7-b96e-4a35-b7c3-7d300c6120c5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.397836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d223cfe7-b96e-4a35-b7c3-7d300c6120c5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.398260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d223cfe7-b96e-4a35-b7c3-7d300c6120c5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.569505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ccdfaf71-5408-45bd-a1d8-4d2b05bc4b5f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:00.570391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ccdfaf71-5408-45bd-a1d8-4d2b05bc4b5f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:00.579320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccdfaf71-5408-45bd-a1d8-4d2b05bc4b5f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.582529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccdfaf71-5408-45bd-a1d8-4d2b05bc4b5f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.582996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccdfaf71-5408-45bd-a1d8-4d2b05bc4b5f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.583950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccdfaf71-5408-45bd-a1d8-4d2b05bc4b5f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.584413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccdfaf71-5408-45bd-a1d8-4d2b05bc4b5f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.585029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccdfaf71-5408-45bd-a1d8-4d2b05bc4b5f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.681841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ccdfaf71-5408-45bd-a1d8-4d2b05bc4b5f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:00.693696 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ccdfaf71-5408-45bd-a1d8-4d2b05bc4b5f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1319 microversion: 2.1 time: 0.126591 Sep 27 09:57:00.694967 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 72/153] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:00 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1319 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:00.769340 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d223cfe7-b96e-4a35-b7c3-7d300c6120c5 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1640 microversion: 2.1 time: 0.383981 Sep 27 09:57:00.769963 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 82/154] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:00 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1640 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:00.793256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-11d215a8-1a2e-48e5-8bd3-0789532a1326 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:00.793815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-11d215a8-1a2e-48e5-8bd3-0789532a1326 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:00.800567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11d215a8-1a2e-48e5-8bd3-0789532a1326 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.801209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11d215a8-1a2e-48e5-8bd3-0789532a1326 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.801651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11d215a8-1a2e-48e5-8bd3-0789532a1326 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.802858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11d215a8-1a2e-48e5-8bd3-0789532a1326 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.804349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11d215a8-1a2e-48e5-8bd3-0789532a1326 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.804911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11d215a8-1a2e-48e5-8bd3-0789532a1326 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.835666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-11d215a8-1a2e-48e5-8bd3-0789532a1326 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:57:00.851865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c6d92171-d063-4b5e-b86a-461e033cdada tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:00.852735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c6d92171-d063-4b5e-b86a-461e033cdada tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:00.864369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6d92171-d063-4b5e-b86a-461e033cdada tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.864874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6d92171-d063-4b5e-b86a-461e033cdada tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.865312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6d92171-d063-4b5e-b86a-461e033cdada tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.866028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6d92171-d063-4b5e-b86a-461e033cdada tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:00.866483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6d92171-d063-4b5e-b86a-461e033cdada tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:00.866912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6d92171-d063-4b5e-b86a-461e033cdada tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:00.939563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c6d92171-d063-4b5e-b86a-461e033cdada tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:00.947908 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c6d92171-d063-4b5e-b86a-461e033cdada tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1219 microversion: 2.1 time: 0.100311 Sep 27 09:57:00.948623 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 83/155] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:00 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:01.037797 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-11d215a8-1a2e-48e5-8bd3-0789532a1326 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "DELETE /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 204 len: 0 microversion: 2.1 time: 0.248296 Sep 27 09:57:01.039728 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 73/156] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:57:00 2022] DELETE /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 0 bytes in 252 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:57:01.053509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c801fcc4-23ea-4cfe-8500-03a78f20be6a tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:01.054410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c801fcc4-23ea-4cfe-8500-03a78f20be6a tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:01.065221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c801fcc4-23ea-4cfe-8500-03a78f20be6a tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:01.066135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c801fcc4-23ea-4cfe-8500-03a78f20be6a tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:01.066790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c801fcc4-23ea-4cfe-8500-03a78f20be6a tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:01.067971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c801fcc4-23ea-4cfe-8500-03a78f20be6a tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:01.068668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c801fcc4-23ea-4cfe-8500-03a78f20be6a tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:01.069263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c801fcc4-23ea-4cfe-8500-03a78f20be6a tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:01.283002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5fcd8b89-f223-4268-99f5-53b8baed00a8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:01.284062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5fcd8b89-f223-4268-99f5-53b8baed00a8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:01.295334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fcd8b89-f223-4268-99f5-53b8baed00a8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:01.296124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fcd8b89-f223-4268-99f5-53b8baed00a8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:01.296461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fcd8b89-f223-4268-99f5-53b8baed00a8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:01.297237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fcd8b89-f223-4268-99f5-53b8baed00a8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:01.297710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fcd8b89-f223-4268-99f5-53b8baed00a8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:01.298132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fcd8b89-f223-4268-99f5-53b8baed00a8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:01.392778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5fcd8b89-f223-4268-99f5-53b8baed00a8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:01.402707 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5fcd8b89-f223-4268-99f5-53b8baed00a8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1273 microversion: 2.1 time: 0.123632 Sep 27 09:57:01.412171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c801fcc4-23ea-4cfe-8500-03a78f20be6a tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:01.413186 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 74/157] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:01 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1273 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:01.429749 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c801fcc4-23ea-4cfe-8500-03a78f20be6a tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1495 microversion: 2.1 time: 0.383744 Sep 27 09:57:01.430420 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 84/158] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:01 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1495 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:01.719612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-17d1ddc2-d632-4b5d-95c9-752a7ae7e856 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:01.720243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-17d1ddc2-d632-4b5d-95c9-752a7ae7e856 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:01.728084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d1ddc2-d632-4b5d-95c9-752a7ae7e856 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:01.728490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d1ddc2-d632-4b5d-95c9-752a7ae7e856 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:01.728951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d1ddc2-d632-4b5d-95c9-752a7ae7e856 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:01.729805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d1ddc2-d632-4b5d-95c9-752a7ae7e856 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:01.730296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d1ddc2-d632-4b5d-95c9-752a7ae7e856 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:01.730538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d1ddc2-d632-4b5d-95c9-752a7ae7e856 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:01.805904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17d1ddc2-d632-4b5d-95c9-752a7ae7e856 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:01.814177 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-17d1ddc2-d632-4b5d-95c9-752a7ae7e856 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1319 microversion: 2.1 time: 0.099218 Sep 27 09:57:01.814771 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 75/159] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:01 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1319 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:01.972712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b1c5d07b-7f3a-4550-81d6-8a4e1199e387 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:01.973393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b1c5d07b-7f3a-4550-81d6-8a4e1199e387 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:01.979878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1c5d07b-7f3a-4550-81d6-8a4e1199e387 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:01.980537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1c5d07b-7f3a-4550-81d6-8a4e1199e387 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:01.981017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1c5d07b-7f3a-4550-81d6-8a4e1199e387 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:01.981807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1c5d07b-7f3a-4550-81d6-8a4e1199e387 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:01.982304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1c5d07b-7f3a-4550-81d6-8a4e1199e387 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:01.982733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1c5d07b-7f3a-4550-81d6-8a4e1199e387 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:02.067309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b1c5d07b-7f3a-4550-81d6-8a4e1199e387 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:02.075539 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b1c5d07b-7f3a-4550-81d6-8a4e1199e387 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1219 microversion: 2.1 time: 0.107222 Sep 27 09:57:02.076243 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 85/160] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:01 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1219 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:02.425749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3a3074a-4774-4b84-ad21-c5efdb41831e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:02.426378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d3a3074a-4774-4b84-ad21-c5efdb41831e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:02.433016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3a3074a-4774-4b84-ad21-c5efdb41831e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:02.433561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3a3074a-4774-4b84-ad21-c5efdb41831e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:02.433950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3a3074a-4774-4b84-ad21-c5efdb41831e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:02.434720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3a3074a-4774-4b84-ad21-c5efdb41831e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:02.435179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3a3074a-4774-4b84-ad21-c5efdb41831e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:02.435610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3a3074a-4774-4b84-ad21-c5efdb41831e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:02.453477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3eb16ea6-b12d-4a6a-b78f-0fe332834bd1 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:02.454096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3eb16ea6-b12d-4a6a-b78f-0fe332834bd1 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:02.464476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3eb16ea6-b12d-4a6a-b78f-0fe332834bd1 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:02.465003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3eb16ea6-b12d-4a6a-b78f-0fe332834bd1 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:02.465448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3eb16ea6-b12d-4a6a-b78f-0fe332834bd1 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:02.466162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3eb16ea6-b12d-4a6a-b78f-0fe332834bd1 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:02.466606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3eb16ea6-b12d-4a6a-b78f-0fe332834bd1 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:02.467048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3eb16ea6-b12d-4a6a-b78f-0fe332834bd1 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:02.537180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d3a3074a-4774-4b84-ad21-c5efdb41831e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:02.545813 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d3a3074a-4774-4b84-ad21-c5efdb41831e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1273 microversion: 2.1 time: 0.123966 Sep 27 09:57:02.546401 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 76/161] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:02 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1273 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:02.623746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3eb16ea6-b12d-4a6a-b78f-0fe332834bd1 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:02.632163 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3eb16ea6-b12d-4a6a-b78f-0fe332834bd1 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1495 microversion: 2.1 time: 0.183053 Sep 27 09:57:02.633145 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 86/162] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:02 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1495 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:02.838719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4492721c-78b2-4f1b-9cef-074495f27a52 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:02.839372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4492721c-78b2-4f1b-9cef-074495f27a52 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:02.845695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4492721c-78b2-4f1b-9cef-074495f27a52 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:02.846189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4492721c-78b2-4f1b-9cef-074495f27a52 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:02.846628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4492721c-78b2-4f1b-9cef-074495f27a52 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:02.847347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4492721c-78b2-4f1b-9cef-074495f27a52 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:02.848582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4492721c-78b2-4f1b-9cef-074495f27a52 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:02.848582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4492721c-78b2-4f1b-9cef-074495f27a52 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:02.913443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4492721c-78b2-4f1b-9cef-074495f27a52 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:02.921415 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4492721c-78b2-4f1b-9cef-074495f27a52 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1319 microversion: 2.1 time: 0.086608 Sep 27 09:57:02.922003 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 77/163] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:02 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1319 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:03.099418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ebd56271-8e54-44a3-8c80-8d8f2150b928 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:03.100005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ebd56271-8e54-44a3-8c80-8d8f2150b928 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:03.107025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ebd56271-8e54-44a3-8c80-8d8f2150b928 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:03.107663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ebd56271-8e54-44a3-8c80-8d8f2150b928 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:03.108240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ebd56271-8e54-44a3-8c80-8d8f2150b928 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:03.109149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ebd56271-8e54-44a3-8c80-8d8f2150b928 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:03.109767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ebd56271-8e54-44a3-8c80-8d8f2150b928 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:03.110239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ebd56271-8e54-44a3-8c80-8d8f2150b928 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:03.175080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ebd56271-8e54-44a3-8c80-8d8f2150b928 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:03.184979 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ebd56271-8e54-44a3-8c80-8d8f2150b928 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1219 microversion: 2.1 time: 0.089137 Sep 27 09:57:03.196822 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 87/164] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:03 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1219 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:03.570295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2ab1dec9-61a6-4d10-b8a4-af20b6461013 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:03.571480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2ab1dec9-61a6-4d10-b8a4-af20b6461013 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:03.574968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=92,nova_api:DELETE=1,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:03.581949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab1dec9-61a6-4d10-b8a4-af20b6461013 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:03.583199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab1dec9-61a6-4d10-b8a4-af20b6461013 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:03.584047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab1dec9-61a6-4d10-b8a4-af20b6461013 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:03.585435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab1dec9-61a6-4d10-b8a4-af20b6461013 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:03.586513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab1dec9-61a6-4d10-b8a4-af20b6461013 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:03.587032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab1dec9-61a6-4d10-b8a4-af20b6461013 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:03.659335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1c0e7456-6cf0-4bbf-86d5-78ab008917dc tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:03.660567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1c0e7456-6cf0-4bbf-86d5-78ab008917dc tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:03.668252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c0e7456-6cf0-4bbf-86d5-78ab008917dc tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:03.669157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c0e7456-6cf0-4bbf-86d5-78ab008917dc tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:03.669742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c0e7456-6cf0-4bbf-86d5-78ab008917dc tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:03.671002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c0e7456-6cf0-4bbf-86d5-78ab008917dc tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:03.671608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c0e7456-6cf0-4bbf-86d5-78ab008917dc tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:03.672178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c0e7456-6cf0-4bbf-86d5-78ab008917dc tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:03.677881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:03.718483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2ab1dec9-61a6-4d10-b8a4-af20b6461013 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:03.726322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:03.739226 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2ab1dec9-61a6-4d10-b8a4-af20b6461013 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1273 microversion: 2.1 time: 0.173631 Sep 27 09:57:03.739831 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 78/165] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:03 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1273 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:03.770655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1c0e7456-6cf0-4bbf-86d5-78ab008917dc tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '023730e34b2d4e2a996bfa068a97413c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '681fc4282a6c4fd8ba4f078705f4ca5c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:03.781055 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1c0e7456-6cf0-4bbf-86d5-78ab008917dc tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 200 len: 1453 microversion: 2.1 time: 0.128734 Sep 27 09:57:03.795854 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 88/166] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:03 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 1453 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:03.811608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-13d840c0-bfcd-41a0-9771-2aaa43723b2f req-a75f3e98-f186-4d2e-aa80-b43fe81484e9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8febbed7-7a38-4051-8910-bdb62bc11eac", "name": "network-vif-deleted", "tag": "ed45a4db-ff3b-4d14-b5f9-f0a60046bd2c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:03.823882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:INSERT=7,nova_api:SELECT=91,nova_api:DELETE=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:03.826728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-13d840c0-bfcd-41a0-9771-2aaa43723b2f req-a75f3e98-f186-4d2e-aa80-b43fe81484e9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:03.827304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-13d840c0-bfcd-41a0-9771-2aaa43723b2f req-a75f3e98-f186-4d2e-aa80-b43fe81484e9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:03.827999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-13d840c0-bfcd-41a0-9771-2aaa43723b2f req-a75f3e98-f186-4d2e-aa80-b43fe81484e9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:03.850834 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-13d840c0-bfcd-41a0-9771-2aaa43723b2f req-a75f3e98-f186-4d2e-aa80-b43fe81484e9 service nova] Creating event network-vif-deleted:ed45a4db-ff3b-4d14-b5f9-f0a60046bd2c for instance 8febbed7-7a38-4051-8910-bdb62bc11eac on nch1-master-1 Sep 27 09:57:03.851357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-13d840c0-bfcd-41a0-9771-2aaa43723b2f req-a75f3e98-f186-4d2e-aa80-b43fe81484e9 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 09:57:03.851768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-13d840c0-bfcd-41a0-9771-2aaa43723b2f req-a75f3e98-f186-4d2e-aa80-b43fe81484e9 service nova] Acquired lock "compute-rpcapi-router" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 09:57:03.853243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3105e12c-f3df-4163-854e-4c95fdf56b06 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:03.853877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3105e12c-f3df-4163-854e-4c95fdf56b06 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:03.854298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3105e12c-f3df-4163-854e-4c95fdf56b06 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:03.855016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3105e12c-f3df-4163-854e-4c95fdf56b06 None None] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:03.855453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3105e12c-f3df-4163-854e-4c95fdf56b06 None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:03.856009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3105e12c-f3df-4163-854e-4c95fdf56b06 None None] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:03.868871 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.compute.rpcapi [None req-3105e12c-f3df-4163-854e-4c95fdf56b06 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 27 09:57:03.869995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3105e12c-f3df-4163-854e-4c95fdf56b06 None None] Releasing lock "compute-rpcapi-router" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 09:57:03.888586 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3105e12c-f3df-4163-854e-4c95fdf56b06 None None] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089457 Sep 27 09:57:03.893385 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 89/167] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:03.911430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-1416161962", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:03.948322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af398331-d0ab-45db-b55a-e1f3b60ca993 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:03.949321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-af398331-d0ab-45db-b55a-e1f3b60ca993 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:03.956983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af398331-d0ab-45db-b55a-e1f3b60ca993 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:03.957483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af398331-d0ab-45db-b55a-e1f3b60ca993 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:03.957883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af398331-d0ab-45db-b55a-e1f3b60ca993 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:03.958592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af398331-d0ab-45db-b55a-e1f3b60ca993 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:03.959035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af398331-d0ab-45db-b55a-e1f3b60ca993 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:03.959456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af398331-d0ab-45db-b55a-e1f3b60ca993 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:03.968712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:57:04.035193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af398331-d0ab-45db-b55a-e1f3b60ca993 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:04.043386 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af398331-d0ab-45db-b55a-e1f3b60ca993 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1319 microversion: 2.1 time: 0.100774 Sep 27 09:57:04.043939 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 90/168] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:03 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1319 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:04.187194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:57:04.187612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:57:04.188086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:57:04.188522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:57:04.188964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:57:04.198444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:04.198989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:04.199349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:04.200069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:04.200516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:04.200960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:04.208051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec767531-8dfe-4c4d-ae63-231e59ab0236 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:04.209166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ec767531-8dfe-4c4d-ae63-231e59ab0236 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:04.216916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec767531-8dfe-4c4d-ae63-231e59ab0236 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:04.217543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec767531-8dfe-4c4d-ae63-231e59ab0236 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:04.218083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec767531-8dfe-4c4d-ae63-231e59ab0236 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:04.219004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec767531-8dfe-4c4d-ae63-231e59ab0236 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:04.219546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec767531-8dfe-4c4d-ae63-231e59ab0236 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:04.220160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec767531-8dfe-4c4d-ae63-231e59ab0236 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:04.225973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:04.231556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:04.243937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:57:04.246511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:57:04.288816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ec767531-8dfe-4c4d-ae63-231e59ab0236 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:04.297006 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ec767531-8dfe-4c4d-ae63-231e59ab0236 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1219 microversion: 2.1 time: 0.093376 Sep 27 09:57:04.308900 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 91/169] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:04 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1219 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:04.309079 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0b88931c-6b18-48a3-b7ed-09c61ec05b41 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.550716 Sep 27 09:57:04.314844 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 79/170] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:57:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 558 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 09:57:04.320080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-785659e3-da11-4f76-9ff8-bbf262499dd1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:04.321997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-785659e3-da11-4f76-9ff8-bbf262499dd1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:04.380143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-785659e3-da11-4f76-9ff8-bbf262499dd1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:04.388430 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-785659e3-da11-4f76-9ff8-bbf262499dd1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1217 microversion: 2.1 time: 0.072215 Sep 27 09:57:04.389247 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 92/171] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:04 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1217 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:04.772015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-54dac4fa-9b88-4182-bba4-c19ba309bf06 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:04.772746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-54dac4fa-9b88-4182-bba4-c19ba309bf06 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:04.779909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54dac4fa-9b88-4182-bba4-c19ba309bf06 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:04.780400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54dac4fa-9b88-4182-bba4-c19ba309bf06 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:04.780877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54dac4fa-9b88-4182-bba4-c19ba309bf06 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:04.781584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54dac4fa-9b88-4182-bba4-c19ba309bf06 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:04.782050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54dac4fa-9b88-4182-bba4-c19ba309bf06 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:04.782474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54dac4fa-9b88-4182-bba4-c19ba309bf06 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:04.807995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:04.809316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:04.816285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:04.817145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:04.817732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:04.818706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:04.819339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:04.819909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:04.835790 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] HTTP exception thrown: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Request method failure captured: Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac HTTP/1.1 Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjHafamTlS-9C5I4M4UbcBJQ1u_43PEJFApwHB_W-rv2AEnXSfBNxtGxxT7V1JqEtILUIzfjkYY1vB0M4LmZMbE1w_Se9O0Mlxkk272eR82WamQNnu8hc4c2JtOlyCbdWOKn09PcQY06xcTaDMoxpToPO6XfOSKqfn7rXRzxVpve9Q Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 681fc4282a6c4fd8ba4f078705f4ca5c Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AggregatesAdminTestJSON-1518209963 Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:57:04.839881 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/681fc4282a6c4fd8ba4f078705f4ca5c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/681fc4282a6c4fd8ba4f078705f4ca5c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/681fc4282a6c4fd8ba4f078705f4ca5c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_681fc4282a6c4fd8ba4f078705f4ca5c"}]}] Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AggregatesAdminTestJSON-1518209963 Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 681fc4282a6c4fd8ba4f078705f4ca5c Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-1518209963 Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AggregatesAdminTestJSON-1518209963-project Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 023730e34b2d4e2a996bfa068a97413c Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AggregatesAdminTestJSON-1518209963-project Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:04.841353 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '8febbed7-7a38-4051-8910-bdb62bc11eac'} Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:04.842070 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:04.843608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:04.845018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Returning 404 to user: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:04.845018 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c35e0446-58c4-49da-9285-a72694c524ea tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 404 len: 111 microversion: 2.1 time: 0.038154 Sep 27 09:57:04.845738 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 93/172] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:04 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:04.850554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a831d101-7fe7-441a-9ff8-02807d7a9afd tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:04.864945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a831d101-7fe7-441a-9ff8-02807d7a9afd tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:04.865214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-54dac4fa-9b88-4182-bba4-c19ba309bf06 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:04.865857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a831d101-7fe7-441a-9ff8-02807d7a9afd tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:04.866267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a831d101-7fe7-441a-9ff8-02807d7a9afd tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:04.874700 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-54dac4fa-9b88-4182-bba4-c19ba309bf06 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1273 microversion: 2.1 time: 0.106853 Sep 27 09:57:04.875532 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 80/173] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:04 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1273 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:05.012876 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a831d101-7fe7-441a-9ff8-02807d7a9afd tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/3/action" status: 200 len: 300 microversion: 2.1 time: 0.165628 Sep 27 09:57:05.013977 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 94/174] 10.222.0.26 () {60 vars in 1191 bytes} [Tue Sep 27 09:57:04 2022] POST /compute/v2.1/os-aggregates/3/action => generated 300 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:05.029066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d27ebca9-5347-40f2-905c-73dd9d7e2e7b tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:05.057977 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d27ebca9-5347-40f2-905c-73dd9d7e2e7b tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/3" status: 200 len: 0 microversion: 2.1 time: 0.033763 Sep 27 09:57:05.073164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f6d8e3c2-d7ac-421a-9791-d8cad5466e1c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:05.073805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f6d8e3c2-d7ac-421a-9791-d8cad5466e1c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:05.079791 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 81/175] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:57:05 2022] DELETE /compute/v2.1/os-aggregates/3 => generated 0 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 355 bytes (2 switches on core 0) Sep 27 09:57:05.086480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6d8e3c2-d7ac-421a-9791-d8cad5466e1c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:05.086887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-14266e91-e5c8-4c29-9191-a42c8233d5bb tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-945463422"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:05.089129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6d8e3c2-d7ac-421a-9791-d8cad5466e1c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:05.089591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6d8e3c2-d7ac-421a-9791-d8cad5466e1c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:05.091457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-14266e91-e5c8-4c29-9191-a42c8233d5bb tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Generated uuid a12bec03-50e4-4dcb-910a-73f5aa208bee for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:57:05.092813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6d8e3c2-d7ac-421a-9791-d8cad5466e1c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:05.094052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6d8e3c2-d7ac-421a-9791-d8cad5466e1c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:05.094985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6d8e3c2-d7ac-421a-9791-d8cad5466e1c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:05.130852 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-14266e91-e5c8-4c29-9191-a42c8233d5bb tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 197 microversion: 2.1 time: 0.049747 Sep 27 09:57:05.132374 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 82/176] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:57:05 2022] POST /compute/v2.1/os-aggregates => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:05.147629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-164d78b7-a6b3-4acc-92ec-2fd3789087cc tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:05.164788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-164d78b7-a6b3-4acc-92ec-2fd3789087cc tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:05.165636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-164d78b7-a6b3-4acc-92ec-2fd3789087cc tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:05.167202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-164d78b7-a6b3-4acc-92ec-2fd3789087cc tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:05.308397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6d8e3c2-d7ac-421a-9791-d8cad5466e1c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:05.332225 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f6d8e3c2-d7ac-421a-9791-d8cad5466e1c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1319 microversion: 2.1 time: 0.264194 Sep 27 09:57:05.332858 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 95/177] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:05 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1319 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:05.352155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-98536c37-2094-47f3-b84e-85b445d5ae19 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:05.361275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-98536c37-2094-47f3-b84e-85b445d5ae19 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:05.368756 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-164d78b7-a6b3-4acc-92ec-2fd3789087cc tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/8/action" status: 200 len: 241 microversion: 2.1 time: 0.226093 Sep 27 09:57:05.369593 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 83/178] 10.222.0.26 () {60 vars in 1191 bytes} [Tue Sep 27 09:57:05 2022] POST /compute/v2.1/os-aggregates/8/action => generated 241 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:05.376681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98536c37-2094-47f3-b84e-85b445d5ae19 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:05.386576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e37cbbb-4da7-47e2-bbbe-3faff9a3e37f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:05.389126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98536c37-2094-47f3-b84e-85b445d5ae19 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:05.390047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98536c37-2094-47f3-b84e-85b445d5ae19 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:05.391361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98536c37-2094-47f3-b84e-85b445d5ae19 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:05.392212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98536c37-2094-47f3-b84e-85b445d5ae19 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:05.400264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98536c37-2094-47f3-b84e-85b445d5ae19 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:05.411732 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9e37cbbb-4da7-47e2-bbbe-3faff9a3e37f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates/8" status: 200 len: 241 microversion: 2.1 time: 0.032678 Sep 27 09:57:05.416772 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 84/179] 10.222.0.26 () {58 vars in 1149 bytes} [Tue Sep 27 09:57:05 2022] GET /compute/v2.1/os-aggregates/8 => generated 241 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:05.420046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69c4ac3e-6415-4e93-bbe6-0c326f4756de tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:05.420730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69c4ac3e-6415-4e93-bbe6-0c326f4756de tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:05.436057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c4ac3e-6415-4e93-bbe6-0c326f4756de tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:05.436545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c4ac3e-6415-4e93-bbe6-0c326f4756de tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:05.437002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c4ac3e-6415-4e93-bbe6-0c326f4756de tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:05.437713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c4ac3e-6415-4e93-bbe6-0c326f4756de tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:05.438171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c4ac3e-6415-4e93-bbe6-0c326f4756de tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:05.438602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c4ac3e-6415-4e93-bbe6-0c326f4756de tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:05.577856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-98536c37-2094-47f3-b84e-85b445d5ae19 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:05.596732 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-98536c37-2094-47f3-b84e-85b445d5ae19 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1275 microversion: 2.1 time: 0.262289 Sep 27 09:57:05.598246 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 96/180] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:05 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1275 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:05.649687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-880362c2-d580-4c2d-a9ef-be249850e2b6 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:05.666869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69c4ac3e-6415-4e93-bbe6-0c326f4756de tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:05.684703 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69c4ac3e-6415-4e93-bbe6-0c326f4756de tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1269 microversion: 2.1 time: 0.269722 Sep 27 09:57:05.700743 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 85/181] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:05 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1269 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:05.701062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-880362c2-d580-4c2d-a9ef-be249850e2b6 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:05.701062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-880362c2-d580-4c2d-a9ef-be249850e2b6 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:05.701062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-880362c2-d580-4c2d-a9ef-be249850e2b6 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:05.897946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-95655e67-9625-42c7-86d7-031bd4d8eaaa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:05.898968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-95655e67-9625-42c7-86d7-031bd4d8eaaa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:05.912144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95655e67-9625-42c7-86d7-031bd4d8eaaa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:05.913752 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-880362c2-d580-4c2d-a9ef-be249850e2b6 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/8/action" status: 200 len: 226 microversion: 2.1 time: 0.313395 Sep 27 09:57:05.914511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95655e67-9625-42c7-86d7-031bd4d8eaaa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:05.914792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95655e67-9625-42c7-86d7-031bd4d8eaaa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:05.915538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95655e67-9625-42c7-86d7-031bd4d8eaaa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:05.915967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95655e67-9625-42c7-86d7-031bd4d8eaaa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:05.916870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95655e67-9625-42c7-86d7-031bd4d8eaaa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:05.917752 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 97/182] 10.222.0.26 () {60 vars in 1191 bytes} [Tue Sep 27 09:57:05 2022] POST /compute/v2.1/os-aggregates/8/action => generated 226 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:05.939544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-365d99c6-549c-436c-809f-55c38c9c4f94 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:05.977140 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-365d99c6-549c-436c-809f-55c38c9c4f94 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/8" status: 200 len: 0 microversion: 2.1 time: 0.050666 Sep 27 09:57:05.978772 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 98/183] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:57:05 2022] DELETE /compute/v2.1/os-aggregates/8 => generated 0 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:57:05.990703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-831672c5-75fd-430e-826b-112e763d1a7b tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1850083736"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:05.994196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.aggregate [None req-831672c5-75fd-430e-826b-112e763d1a7b tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Generated uuid 111ac66e-a1d7-4683-a8a0-08be81696035 for aggregate {{(pid=145969) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:57:06.030306 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-831672c5-75fd-430e-826b-112e763d1a7b tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.042849 Sep 27 09:57:06.031964 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 99/184] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:57:05 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:06.044551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b8c50b35-07b0-4d6f-8092-349915e60ab4 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:06.061034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-95655e67-9625-42c7-86d7-031bd4d8eaaa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:06.074822 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-95655e67-9625-42c7-86d7-031bd4d8eaaa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1279 microversion: 2.1 time: 0.181239 Sep 27 09:57:06.076601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c50b35-07b0-4d6f-8092-349915e60ab4 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.077495 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 86/185] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:05 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1279 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:06.077796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c50b35-07b0-4d6f-8092-349915e60ab4 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.078494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c50b35-07b0-4d6f-8092-349915e60ab4 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.245855 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b8c50b35-07b0-4d6f-8092-349915e60ab4 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/9/action" status: 200 len: 242 microversion: 2.1 time: 0.206450 Sep 27 09:57:06.247762 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 100/186] 10.222.0.26 () {60 vars in 1191 bytes} [Tue Sep 27 09:57:06 2022] POST /compute/v2.1/os-aggregates/9/action => generated 242 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:06.262106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c346f5a-9182-4f26-90b2-8c8a9cdebe8d tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:06.274213 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c346f5a-9182-4f26-90b2-8c8a9cdebe8d tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates" status: 200 len: 245 microversion: 2.1 time: 0.018536 Sep 27 09:57:06.274812 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 87/187] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 09:57:06 2022] GET /compute/v2.1/os-aggregates => generated 245 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:06.287170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-74e0f0f1-a3cd-40bb-825a-feaaea730693 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:06.298667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74e0f0f1-a3cd-40bb-825a-feaaea730693 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.299233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74e0f0f1-a3cd-40bb-825a-feaaea730693 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.299681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74e0f0f1-a3cd-40bb-825a-feaaea730693 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.361223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ac2197cc-39a8-4b27-ab0e-9d43da4f4e9b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:06.362343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ac2197cc-39a8-4b27-ab0e-9d43da4f4e9b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:06.381494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2197cc-39a8-4b27-ab0e-9d43da4f4e9b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.382012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2197cc-39a8-4b27-ab0e-9d43da4f4e9b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.382674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2197cc-39a8-4b27-ab0e-9d43da4f4e9b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.384113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2197cc-39a8-4b27-ab0e-9d43da4f4e9b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.384315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2197cc-39a8-4b27-ab0e-9d43da4f4e9b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.385032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2197cc-39a8-4b27-ab0e-9d43da4f4e9b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.502144 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-74e0f0f1-a3cd-40bb-825a-feaaea730693 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/9/action" status: 200 len: 227 microversion: 2.1 time: 0.218750 Sep 27 09:57:06.504202 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 101/188] 10.222.0.26 () {60 vars in 1191 bytes} [Tue Sep 27 09:57:06 2022] POST /compute/v2.1/os-aggregates/9/action => generated 227 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:06.523030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0de1b180-99de-4d95-bd5c-ff76619b191b tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:06.568211 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0de1b180-99de-4d95-bd5c-ff76619b191b tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/9" status: 200 len: 0 microversion: 2.1 time: 0.050804 Sep 27 09:57:06.568861 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 102/189] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:57:06 2022] DELETE /compute/v2.1/os-aggregates/9 => generated 0 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:57:06.580230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-072cf117-b073-4c7b-99c7-07b5b6c0fd99 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Calling method '.version_select at 0x7f7f68056e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:06.584070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-072cf117-b073-4c7b-99c7-07b5b6c0fd99 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.584634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-072cf117-b073-4c7b-99c7-07b5b6c0fd99 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.586843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-072cf117-b073-4c7b-99c7-07b5b6c0fd99 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.587737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-072cf117-b073-4c7b-99c7-07b5b6c0fd99 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.588974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-072cf117-b073-4c7b-99c7-07b5b6c0fd99 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.589883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-072cf117-b073-4c7b-99c7-07b5b6c0fd99 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.618626 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-072cf117-b073-4c7b-99c7-07b5b6c0fd99 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts" status: 200 len: 379 microversion: 2.1 time: 0.042129 Sep 27 09:57:06.625083 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 103/190] 10.222.0.26 () {58 vars in 1128 bytes} [Tue Sep 27 09:57:06 2022] GET /compute/v2.1/os-hosts => generated 379 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:06.626009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af9e1371-0be0-4fec-917a-8a1a6381ee30 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:06.626631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-af9e1371-0be0-4fec-917a-8a1a6381ee30 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:06.634424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9e1371-0be0-4fec-917a-8a1a6381ee30 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.634424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9e1371-0be0-4fec-917a-8a1a6381ee30 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.635041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9e1371-0be0-4fec-917a-8a1a6381ee30 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.635802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9e1371-0be0-4fec-917a-8a1a6381ee30 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.636265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9e1371-0be0-4fec-917a-8a1a6381ee30 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.636779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9e1371-0be0-4fec-917a-8a1a6381ee30 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.749396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af9e1371-0be0-4fec-917a-8a1a6381ee30 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:06.757688 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af9e1371-0be0-4fec-917a-8a1a6381ee30 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1281 microversion: 2.1 time: 0.135644 Sep 27 09:57:06.766011 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 104/191] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:06 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1281 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:06.766011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e29aebe8-c360-43c4-b041-0ec320b26903 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Calling method '.version_select at 0x7f7f681b3e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:06.767174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e29aebe8-c360-43c4-b041-0ec320b26903 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.768775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e29aebe8-c360-43c4-b041-0ec320b26903 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.769378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e29aebe8-c360-43c4-b041-0ec320b26903 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.770128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e29aebe8-c360-43c4-b041-0ec320b26903 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.770591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e29aebe8-c360-43c4-b041-0ec320b26903 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.771037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e29aebe8-c360-43c4-b041-0ec320b26903 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.801757 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e29aebe8-c360-43c4-b041-0ec320b26903 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts?zone=internal" status: 200 len: 239 microversion: 2.1 time: 0.041464 Sep 27 09:57:06.811521 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 105/192] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:57:06 2022] GET /compute/v2.1/os-hosts?zone=internal => generated 239 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:06.817796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4bd34555-066f-454e-9cd5-6ae124fd877f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:06.818428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4bd34555-066f-454e-9cd5-6ae124fd877f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:06.828486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd34555-066f-454e-9cd5-6ae124fd877f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.829033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd34555-066f-454e-9cd5-6ae124fd877f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.829475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd34555-066f-454e-9cd5-6ae124fd877f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.830202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd34555-066f-454e-9cd5-6ae124fd877f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.830657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd34555-066f-454e-9cd5-6ae124fd877f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.831471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd34555-066f-454e-9cd5-6ae124fd877f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.918776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ac2197cc-39a8-4b27-ab0e-9d43da4f4e9b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:06.930953 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ac2197cc-39a8-4b27-ab0e-9d43da4f4e9b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.573730 Sep 27 09:57:06.933281 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 88/193] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:06 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:06.938169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5137594e-99df-4c1f-af85-8fd8a1b06329 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Calling method '.version_select at 0x7f7f683c9f70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:06.949646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5137594e-99df-4c1f-af85-8fd8a1b06329 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.950354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5137594e-99df-4c1f-af85-8fd8a1b06329 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.951439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5137594e-99df-4c1f-af85-8fd8a1b06329 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.952497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5137594e-99df-4c1f-af85-8fd8a1b06329 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:06.953387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5137594e-99df-4c1f-af85-8fd8a1b06329 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:06.953719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4bd34555-066f-454e-9cd5-6ae124fd877f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:06.954560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5137594e-99df-4c1f-af85-8fd8a1b06329 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:06.966764 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4bd34555-066f-454e-9cd5-6ae124fd877f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1275 microversion: 2.1 time: 0.153470 Sep 27 09:57:06.967998 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 106/194] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:06 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1275 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:06.987783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ef5fa549-0f61-424e-9464-596e9c658e08 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-610395847"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:07.006311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.aggregate [None req-ef5fa549-0f61-424e-9464-596e9c658e08 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Generated uuid 45612b86-0f37-486d-9837-7c0f3aef08a9 for aggregate {{(pid=145969) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:57:07.020725 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5137594e-99df-4c1f-af85-8fd8a1b06329 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts" status: 200 len: 379 microversion: 2.1 time: 0.086050 Sep 27 09:57:07.022075 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 89/195] 10.222.0.26 () {58 vars in 1128 bytes} [Tue Sep 27 09:57:06 2022] GET /compute/v2.1/os-hosts => generated 379 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:07.032377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ab322a93-0fba-4744-b670-0827a6a8a291 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Calling method '.version_select at 0x7f7f682f64c0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:07.047299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab322a93-0fba-4744-b670-0827a6a8a291 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:07.047802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab322a93-0fba-4744-b670-0827a6a8a291 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:07.048248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab322a93-0fba-4744-b670-0827a6a8a291 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:07.067886 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ef5fa549-0f61-424e-9464-596e9c658e08 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 246 microversion: 2.41 time: 0.097888 Sep 27 09:57:07.070177 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 107/196] 10.222.0.26 () {62 vars in 1205 bytes} [Tue Sep 27 09:57:06 2022] POST /compute/v2.1/os-aggregates => generated 246 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:57:07.085693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6b789164-5e83-4668-95cb-d8c027ec612f tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Action: 'update', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-608689670"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:07.087382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab322a93-0fba-4744-b670-0827a6a8a291 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:07.088413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab322a93-0fba-4744-b670-0827a6a8a291 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:07.089033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab322a93-0fba-4744-b670-0827a6a8a291 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:07.111557 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ab322a93-0fba-4744-b670-0827a6a8a291 tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts/nch1-master-1" status: 200 len: 467 microversion: 2.1 time: 0.082996 Sep 27 09:57:07.112945 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 90/197] 10.222.0.26 () {58 vars in 1170 bytes} [Tue Sep 27 09:57:07 2022] GET /compute/v2.1/os-hosts/nch1-master-1 => generated 467 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:07.119330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f4546bf0-6336-4990-b730-b377e882e52b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:07.120257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f4546bf0-6336-4990-b730-b377e882e52b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:07.127579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4546bf0-6336-4990-b730-b377e882e52b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:07.128380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4546bf0-6336-4990-b730-b377e882e52b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:07.129061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4546bf0-6336-4990-b730-b377e882e52b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:07.130033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4546bf0-6336-4990-b730-b377e882e52b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:07.130671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4546bf0-6336-4990-b730-b377e882e52b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:07.131439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4546bf0-6336-4990-b730-b377e882e52b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:07.256753 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6b789164-5e83-4668-95cb-d8c027ec612f tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-aggregates/10" status: 200 len: 299 microversion: 2.41 time: 0.177564 Sep 27 09:57:07.258257 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 108/198] 10.222.0.26 () {62 vars in 1213 bytes} [Tue Sep 27 09:57:07 2022] PUT /compute/v2.1/os-aggregates/10 => generated 299 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 09:57:07.263935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-702a20d1-3a41-4c56-8d0c-01e8e0f68beb tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Calling method '.version_select at 0x7f7f681d3b80>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:07.274181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-702a20d1-3a41-4c56-8d0c-01e8e0f68beb tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:07.274690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-702a20d1-3a41-4c56-8d0c-01e8e0f68beb tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:07.275129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-702a20d1-3a41-4c56-8d0c-01e8e0f68beb tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:07.292242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-702a20d1-3a41-4c56-8d0c-01e8e0f68beb tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:07.292773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-702a20d1-3a41-4c56-8d0c-01e8e0f68beb tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:07.293268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-702a20d1-3a41-4c56-8d0c-01e8e0f68beb tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:07.339313 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-702a20d1-3a41-4c56-8d0c-01e8e0f68beb tempest-HostsAdminTestJSON-1699872310 tempest-HostsAdminTestJSON-1699872310-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hosts/nch2-master-1" status: 200 len: 729 microversion: 2.1 time: 0.079753 Sep 27 09:57:07.346054 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 109/199] 10.222.0.26 () {58 vars in 1170 bytes} [Tue Sep 27 09:57:07 2022] GET /compute/v2.1/os-hosts/nch2-master-1 => generated 729 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:07.351952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec26a436-5271-4c71-8906-15dd3f8f10fb tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:07.363992 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ec26a436-5271-4c71-8906-15dd3f8f10fb tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates/10" status: 200 len: 299 microversion: 2.41 time: 0.016600 Sep 27 09:57:07.365038 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 110/200] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 09:57:07 2022] GET /compute/v2.1/os-aggregates/10 => generated 299 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:57:07.377995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-00736635-fb53-46ef-8d27-5c64cfdd2db7 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:07.392578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00736635-fb53-46ef-8d27-5c64cfdd2db7 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:07.393110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00736635-fb53-46ef-8d27-5c64cfdd2db7 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:07.393546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00736635-fb53-46ef-8d27-5c64cfdd2db7 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:07.547211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f4546bf0-6336-4990-b730-b377e882e52b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:07.556365 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f4546bf0-6336-4990-b730-b377e882e52b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.442081 Sep 27 09:57:07.557004 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 91/201] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:07 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:07.622288 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-00736635-fb53-46ef-8d27-5c64cfdd2db7 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/10/action" status: 200 len: 314 microversion: 2.41 time: 0.246901 Sep 27 09:57:07.623590 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 111/202] 10.222.0.26 () {62 vars in 1235 bytes} [Tue Sep 27 09:57:07 2022] POST /compute/v2.1/os-aggregates/10/action => generated 314 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:57:07.644005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-18d90941-1850-418c-9491-49a54539d130 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:07.666884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-18d90941-1850-418c-9491-49a54539d130 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:07.667949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-18d90941-1850-418c-9491-49a54539d130 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:07.668639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-18d90941-1850-418c-9491-49a54539d130 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:07.783336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-19f82577-41df-4f67-9d24-029929266745 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:07.783956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-19f82577-41df-4f67-9d24-029929266745 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:07.787860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=88,nova_api:DELETE=7,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:07.800973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19f82577-41df-4f67-9d24-029929266745 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:07.803958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19f82577-41df-4f67-9d24-029929266745 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:07.805096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19f82577-41df-4f67-9d24-029929266745 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:07.805445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19f82577-41df-4f67-9d24-029929266745 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:07.805927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19f82577-41df-4f67-9d24-029929266745 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:07.806361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19f82577-41df-4f67-9d24-029929266745 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:07.861350 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-18d90941-1850-418c-9491-49a54539d130 tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/10/action" status: 200 len: 299 microversion: 2.41 time: 0.223789 Sep 27 09:57:07.869686 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 92/203] 10.222.0.26 () {62 vars in 1235 bytes} [Tue Sep 27 09:57:07 2022] POST /compute/v2.1/os-aggregates/10/action => generated 299 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 09:57:07.879158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-908549fb-04ef-4de8-920c-d08b03f24d5d tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:07.910228 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-908549fb-04ef-4de8-920c-d08b03f24d5d tempest-AggregatesAdminTestV241-1036773056 tempest-AggregatesAdminTestV241-1036773056-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/10" status: 200 len: 0 microversion: 2.41 time: 0.037573 Sep 27 09:57:07.911216 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 93/204] 10.222.0.26 () {60 vars in 1196 bytes} [Tue Sep 27 09:57:07 2022] DELETE /compute/v2.1/os-aggregates/10 => generated 0 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:07.933798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90caaeaa-d5f4-41e1-b225-94bf1d9d7d25 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1109236651"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:07.937926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-90caaeaa-d5f4-41e1-b225-94bf1d9d7d25 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Generated uuid 6eeb42d3-eff5-4abe-a597-f69fad4c2ba8 for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:57:07.980635 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-90caaeaa-d5f4-41e1-b225-94bf1d9d7d25 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.050552 Sep 27 09:57:07.991334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-19f82577-41df-4f67-9d24-029929266745 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:08.003069 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 94/205] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:57:07 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:08.004862 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-19f82577-41df-4f67-9d24-029929266745 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1281 microversion: 2.1 time: 0.225170 Sep 27 09:57:08.007627 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 112/206] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:07 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1281 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:08.025279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9a501e31-674e-4e4c-b6c3-ac5ed5e0f3a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:08.028792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9a501e31-674e-4e4c-b6c3-ac5ed5e0f3a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:08.037429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cf3d4ac1-f9cf-4ff6-8f95-a1b376ee0014 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:08.038064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cf3d4ac1-f9cf-4ff6-8f95-a1b376ee0014 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:08.046289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a501e31-674e-4e4c-b6c3-ac5ed5e0f3a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:08.047265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a501e31-674e-4e4c-b6c3-ac5ed5e0f3a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:08.048818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a501e31-674e-4e4c-b6c3-ac5ed5e0f3a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:08.061573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a501e31-674e-4e4c-b6c3-ac5ed5e0f3a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:08.064669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a501e31-674e-4e4c-b6c3-ac5ed5e0f3a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:08.067314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a501e31-674e-4e4c-b6c3-ac5ed5e0f3a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:08.073704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf3d4ac1-f9cf-4ff6-8f95-a1b376ee0014 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:08.076771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf3d4ac1-f9cf-4ff6-8f95-a1b376ee0014 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:08.078147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf3d4ac1-f9cf-4ff6-8f95-a1b376ee0014 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:08.079668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf3d4ac1-f9cf-4ff6-8f95-a1b376ee0014 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:08.083975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf3d4ac1-f9cf-4ff6-8f95-a1b376ee0014 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:08.084786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf3d4ac1-f9cf-4ff6-8f95-a1b376ee0014 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:08.457913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cf3d4ac1-f9cf-4ff6-8f95-a1b376ee0014 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:08.474100 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cf3d4ac1-f9cf-4ff6-8f95-a1b376ee0014 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.467317 Sep 27 09:57:08.475766 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 95/207] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:08 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:08.484128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6dbeaf96-adff-4991-bd4f-001340b83c4f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:08.499892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6dbeaf96-adff-4991-bd4f-001340b83c4f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:08.500766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6dbeaf96-adff-4991-bd4f-001340b83c4f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:08.501438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6dbeaf96-adff-4991-bd4f-001340b83c4f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:08.661301 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6dbeaf96-adff-4991-bd4f-001340b83c4f tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/11/action" status: 200 len: 243 microversion: 2.1 time: 0.182384 Sep 27 09:57:08.662226 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 96/208] 10.222.0.26 () {60 vars in 1194 bytes} [Tue Sep 27 09:57:08 2022] POST /compute/v2.1/os-aggregates/11/action => generated 243 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:08.668751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c48f0936-7729-43a3-8710-55b0939e072d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:08.669874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c48f0936-7729-43a3-8710-55b0939e072d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:08.677812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f0936-7729-43a3-8710-55b0939e072d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:08.678315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f0936-7729-43a3-8710-55b0939e072d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:08.678935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f0936-7729-43a3-8710-55b0939e072d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:08.679917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f0936-7729-43a3-8710-55b0939e072d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:08.681602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f0936-7729-43a3-8710-55b0939e072d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:08.682641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f0936-7729-43a3-8710-55b0939e072d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:08.744111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9a501e31-674e-4e4c-b6c3-ac5ed5e0f3a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:08.759739 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9a501e31-674e-4e4c-b6c3-ac5ed5e0f3a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1317 microversion: 2.1 time: 0.749820 Sep 27 09:57:08.774993 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 113/209] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:08 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1317 bytes in 766 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:08.802722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cae64c55-acb5-4596-9aa8-38101808c4d9 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:08.824855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cae64c55-acb5-4596-9aa8-38101808c4d9 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:08.825812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cae64c55-acb5-4596-9aa8-38101808c4d9 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:08.826861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cae64c55-acb5-4596-9aa8-38101808c4d9 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:08.985238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c48f0936-7729-43a3-8710-55b0939e072d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:08.994662 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c48f0936-7729-43a3-8710-55b0939e072d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.331091 Sep 27 09:57:08.995710 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 97/210] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:08 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:09.038459 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cae64c55-acb5-4596-9aa8-38101808c4d9 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/11/action" status: 200 len: 228 microversion: 2.1 time: 0.245631 Sep 27 09:57:09.042501 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 114/211] 10.222.0.26 () {60 vars in 1194 bytes} [Tue Sep 27 09:57:08 2022] POST /compute/v2.1/os-aggregates/11/action => generated 228 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:09.056512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f93e3711-7ae7-4466-a4b9-6ba1817eec73 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:09.057438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f93e3711-7ae7-4466-a4b9-6ba1817eec73 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:09.064610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f93e3711-7ae7-4466-a4b9-6ba1817eec73 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:09.065457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f93e3711-7ae7-4466-a4b9-6ba1817eec73 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:09.066137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f93e3711-7ae7-4466-a4b9-6ba1817eec73 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:09.068628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f93e3711-7ae7-4466-a4b9-6ba1817eec73 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:09.069335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f93e3711-7ae7-4466-a4b9-6ba1817eec73 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:09.069979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f93e3711-7ae7-4466-a4b9-6ba1817eec73 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:09.120125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5a03888c-952d-4e22-9442-af5badd0661e req-8977f352-0f88-4855-b6b9-a24f674f34d7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "030e5dcc-dc63-43e6-9254-ef26b66b5cde", "tag": "3dbbf26e-6eb1-444a-bd75-862612ffb4bc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:09.148774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5a03888c-952d-4e22-9442-af5badd0661e req-8977f352-0f88-4855-b6b9-a24f674f34d7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:09.150459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5a03888c-952d-4e22-9442-af5badd0661e req-8977f352-0f88-4855-b6b9-a24f674f34d7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:09.151932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5a03888c-952d-4e22-9442-af5badd0661e req-8977f352-0f88-4855-b6b9-a24f674f34d7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:09.178219 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5a03888c-952d-4e22-9442-af5badd0661e req-8977f352-0f88-4855-b6b9-a24f674f34d7 service nova] Creating event network-changed:3dbbf26e-6eb1-444a-bd75-862612ffb4bc for instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde on nch2-master-1 Sep 27 09:57:09.187855 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5a03888c-952d-4e22-9442-af5badd0661e req-8977f352-0f88-4855-b6b9-a24f674f34d7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.190259 Sep 27 09:57:09.190567 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 98/212] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:09.204596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c76a6731-be9b-4487-b136-e4c991f7a52d tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:09.252911 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c76a6731-be9b-4487-b136-e4c991f7a52d tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/11" status: 200 len: 0 microversion: 2.1 time: 0.060068 Sep 27 09:57:09.254180 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 99/213] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:57:09 2022] DELETE /compute/v2.1/os-aggregates/11 => generated 0 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:57:09.265434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c5e42efd-8b51-4c6b-8978-d3db18f0c2c8 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-311053266"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:09.270014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-c5e42efd-8b51-4c6b-8978-d3db18f0c2c8 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Generated uuid a976a041-1059-4b7c-9b5e-91adc7a474b4 for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:57:09.314788 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c5e42efd-8b51-4c6b-8978-d3db18f0c2c8 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.053536 Sep 27 09:57:09.316232 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 100/214] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:57:09 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:09.323401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6e265b48-7b38-4f35-b24f-db1f17794f20 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1330668380"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:09.328923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-6e265b48-7b38-4f35-b24f-db1f17794f20 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Generated uuid 7fd8c459-d24b-4f48-a72b-d289be82aa91 for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:57:09.337988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=8,nova_api:DELETE=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:09.371846 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6e265b48-7b38-4f35-b24f-db1f17794f20 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.053885 Sep 27 09:57:09.373390 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 101/215] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:57:09 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:09.397994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bafb79d0-2d98-4c8b-b6a2-a3d31da61625 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:09.460201 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bafb79d0-2d98-4c8b-b6a2-a3d31da61625 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/12" status: 200 len: 0 microversion: 2.1 time: 0.072047 Sep 27 09:57:09.466053 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 102/216] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:57:09 2022] DELETE /compute/v2.1/os-aggregates/12 => generated 0 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:57:09.472426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1f8c9952-3c3e-434f-9246-7369651752bf tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:09.493098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f8c9952-3c3e-434f-9246-7369651752bf tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:09.493614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f8c9952-3c3e-434f-9246-7369651752bf tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:09.494089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f8c9952-3c3e-434f-9246-7369651752bf tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:09.671180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f93e3711-7ae7-4466-a4b9-6ba1817eec73 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:09.683610 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f93e3711-7ae7-4466-a4b9-6ba1817eec73 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.636138 Sep 27 09:57:09.685193 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 115/217] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:09 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:09.696302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-be5c5d43-8544-4235-b9fc-b9642652fc83 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:09.716782 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-be5c5d43-8544-4235-b9fc-b9642652fc83 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] HTTP exception thrown: Aggregate 12 could not be found. Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-be5c5d43-8544-4235-b9fc-b9642652fc83 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Request method failure captured: Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-aggregates/12 HTTP/1.1 Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjKqVHHQOm60E8_OXe4GpBUvPDG3GhlC7kAt11OjCJPDsAO9idcqiuKSnLbjIp4RTw3TV0GgfsGZKRRh7lWerdDhpBiAhMDlTO_ul7ePPiex57ggjxxA6KbLdHW6ep8N5Z6dlOhyRsINMdBTQvXFQvOXY44v1jyifYkOW3kLJMU3y4 Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: dc9d0df2395744b0baafe373c8913b91 Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:57:09.718781 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_dc9d0df2395744b0baafe373c8913b91"}]}] Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: dc9d0df2395744b0baafe373c8913b91 Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AggregatesAdminTestJSON-1274375176-project-admin Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: e0808d9ea0734660b695111afab964e3 Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AggregatesAdminTestJSON-1274375176-project-admin Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: exception: Aggregate 12 could not be found. Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '12'} Sep 27 09:57:09.719966 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 108, in show Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi aggregate = self.api.get_aggregate(context, id) Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6464, in get_aggregate Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return objects.Aggregate.get_by_id(context, aggregate_id) Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 12 could not be found. Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:09.720865 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 110, in show Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 12 could not be found. Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 12 could not be found. Sep 27 09:57:09.721902 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:09.732831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-be5c5d43-8544-4235-b9fc-b9642652fc83 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Returning 404 to user: Aggregate 12 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:09.757875 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-be5c5d43-8544-4235-b9fc-b9642652fc83 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates/12" status: 404 len: 78 microversion: 2.1 time: 0.070255 Sep 27 09:57:09.761016 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 116/218] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:57:09 2022] GET /compute/v2.1/os-aggregates/12 => generated 78 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:57:09.775898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bd68d545-6a81-4dd1-acde-3a86311e1e8c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:09.777377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bd68d545-6a81-4dd1-acde-3a86311e1e8c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:09.800471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd68d545-6a81-4dd1-acde-3a86311e1e8c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:09.803829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd68d545-6a81-4dd1-acde-3a86311e1e8c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:09.804564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd68d545-6a81-4dd1-acde-3a86311e1e8c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:09.807195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd68d545-6a81-4dd1-acde-3a86311e1e8c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:09.810250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd68d545-6a81-4dd1-acde-3a86311e1e8c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:09.811477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd68d545-6a81-4dd1-acde-3a86311e1e8c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:09.819020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:09.831379 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1f8c9952-3c3e-434f-9246-7369651752bf tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/13/action" status: 200 len: 243 microversion: 2.1 time: 0.367162 Sep 27 09:57:09.832297 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 103/219] 10.222.0.26 () {60 vars in 1194 bytes} [Tue Sep 27 09:57:09 2022] POST /compute/v2.1/os-aggregates/13/action => generated 243 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:09.847172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cfeb5f79-156e-4021-8d9d-db9eb6534631 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:09.862344 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-cfeb5f79-156e-4021-8d9d-db9eb6534631 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] HTTP exception thrown: Aggregate 12 could not be found. Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cfeb5f79-156e-4021-8d9d-db9eb6534631 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Request method failure captured: Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-aggregates/12 HTTP/1.1 Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjKqVHHQOm60E8_OXe4GpBUvPDG3GhlC7kAt11OjCJPDsAO9idcqiuKSnLbjIp4RTw3TV0GgfsGZKRRh7lWerdDhpBiAhMDlTO_ul7ePPiex57ggjxxA6KbLdHW6ep8N5Z6dlOhyRsINMdBTQvXFQvOXY44v1jyifYkOW3kLJMU3y4 Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: dc9d0df2395744b0baafe373c8913b91 Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:57:09.865001 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_dc9d0df2395744b0baafe373c8913b91"}]}] Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: dc9d0df2395744b0baafe373c8913b91 Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminTestJSON-1274375176-project-admin Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: e0808d9ea0734660b695111afab964e3 Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminTestJSON-1274375176-project-admin Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: exception: Aggregate 12 could not be found. Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '12'} Sep 27 09:57:09.866209 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 155, in delete Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.api.delete_aggregate(context, id) Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6517, in delete_aggregate Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi aggregate = objects.Aggregate.get_by_id(context, aggregate_id) Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:57:09.866969 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 12 could not be found. Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 157, in delete Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 12 could not be found. Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:09.868004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:09.869207 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:09.869207 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:09.869207 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:09.869207 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 12 could not be found. Sep 27 09:57:09.869207 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:09.872151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cfeb5f79-156e-4021-8d9d-db9eb6534631 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Returning 404 to user: Aggregate 12 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:09.873505 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cfeb5f79-156e-4021-8d9d-db9eb6534631 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/12" status: 404 len: 78 microversion: 2.1 time: 0.032511 Sep 27 09:57:09.874306 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 104/220] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:57:09 2022] DELETE /compute/v2.1/os-aggregates/12 => generated 78 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:57:09.884787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-84bc4c99-f093-4b0d-a9db-bacd82f17af5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:09.889222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-84bc4c99-f093-4b0d-a9db-bacd82f17af5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:09.909070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bc4c99-f093-4b0d-a9db-bacd82f17af5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:09.910712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bc4c99-f093-4b0d-a9db-bacd82f17af5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:09.911569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bc4c99-f093-4b0d-a9db-bacd82f17af5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:09.912808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bc4c99-f093-4b0d-a9db-bacd82f17af5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:09.913619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bc4c99-f093-4b0d-a9db-bacd82f17af5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:09.914371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bc4c99-f093-4b0d-a9db-bacd82f17af5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:10.387559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bd68d545-6a81-4dd1-acde-3a86311e1e8c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:10.397624 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bd68d545-6a81-4dd1-acde-3a86311e1e8c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.634675 Sep 27 09:57:10.401576 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 117/221] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:09 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 640 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:10.410001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-39605a54-5f26-4eef-a698-cb2075053cae tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:10.414452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-39605a54-5f26-4eef-a698-cb2075053cae tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Policy check for os_compute_api:os-aggregates:remove_host failed with credentials {'is_admin': False, 'user_id': '8ea0f9af73d5435088120d793a61bef3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5a71603028ff42b4847878e7ede6aebe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-39605a54-5f26-4eef-a698-cb2075053cae tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Request method failure captured: Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/os-aggregates/13/action HTTP/1.1 Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 42 Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjFzty-qLENwwC0WJRfeIxzeZV65xTjvO7a5eLb6pj1D2B1dsP_rK8RVeimECsxHNyUllFJgmBOhq-mF9B6V9NztZc9XT1FhgvcpY2yNe42ftBLJ2Zi5O1gzeQf0ok5TU-3MikfFG4bnjZpU2GZ8UnwGjiq5xJIj1Lmd1RWDnpXTf4 Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:57:10.416387 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5a71603028ff42b4847878e7ede6aebe"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5a71603028ff42b4847878e7ede6aebe"}]}] Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 5a71603028ff42b4847878e7ede6aebe Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-728171188 Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 8ea0f9af73d5435088120d793a61bef3 Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-728171188-project Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: {"remove_host": {"host": "nch1-master-1"}} Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:57:10.417681 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '13', 'body': {'remove_host': {'host': 'nch1-master-1'}}} Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 201, in _remove_host Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(aggr_policies.POLICY_ROOT % 'remove_host', target={}) Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:57:10.418811 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. Sep 27 09:57:10.419956 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:10.422572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-39605a54-5f26-4eef-a698-cb2075053cae tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-aggregates:remove_host to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:10.424115 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-39605a54-5f26-4eef-a698-cb2075053cae tempest-AggregatesAdminNegativeTestJSON-728171188 tempest-AggregatesAdminNegativeTestJSON-728171188-project] 10.222.0.26 "POST /compute/v2.1/os-aggregates/13/action" status: 403 len: 121 microversion: 2.1 time: 0.021095 Sep 27 09:57:10.424725 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 118/222] 10.222.0.26 () {60 vars in 1194 bytes} [Tue Sep 27 09:57:10 2022] POST /compute/v2.1/os-aggregates/13/action => generated 121 bytes in 22 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:10.432749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e0c0562b-4789-4e21-9d7e-b82ce3e1b621 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"availability_zone": "tempest-test_az-2088432005", "name": "tempest-test_aggregate-1113707236"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:10.438951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.aggregate [None req-e0c0562b-4789-4e21-9d7e-b82ce3e1b621 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Generated uuid e5e6839c-859d-46bf-89ed-5aaabffd380c for aggregate {{(pid=145969) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:57:10.485056 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e0c0562b-4789-4e21-9d7e-b82ce3e1b621 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 223 microversion: 2.1 time: 0.058877 Sep 27 09:57:10.485722 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 119/223] 10.222.0.26 () {60 vars in 1165 bytes} [Tue Sep 27 09:57:10 2022] POST /compute/v2.1/os-aggregates => generated 223 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:10.497568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4566cf85-2e92-44e0-9fc0-0e3617d0a1d2 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:10.498460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4566cf85-2e92-44e0-9fc0-0e3617d0a1d2 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:10.506844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4566cf85-2e92-44e0-9fc0-0e3617d0a1d2 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:10.507640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4566cf85-2e92-44e0-9fc0-0e3617d0a1d2 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:10.508562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4566cf85-2e92-44e0-9fc0-0e3617d0a1d2 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:10.510169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4566cf85-2e92-44e0-9fc0-0e3617d0a1d2 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:10.510981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4566cf85-2e92-44e0-9fc0-0e3617d0a1d2 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:10.511643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4566cf85-2e92-44e0-9fc0-0e3617d0a1d2 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:10.575809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-84bc4c99-f093-4b0d-a9db-bacd82f17af5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:10.589240 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-84bc4c99-f093-4b0d-a9db-bacd82f17af5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1317 microversion: 2.1 time: 0.713463 Sep 27 09:57:10.593567 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 105/224] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:09 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1317 bytes in 718 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:10.625589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0261ff57-5433-4a86-b30f-dca57dfe4256 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:10.641197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0261ff57-5433-4a86-b30f-dca57dfe4256 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:10.641791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0261ff57-5433-4a86-b30f-dca57dfe4256 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:10.642486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0261ff57-5433-4a86-b30f-dca57dfe4256 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:10.812671 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0261ff57-5433-4a86-b30f-dca57dfe4256 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/13/action" status: 200 len: 228 microversion: 2.1 time: 0.216160 Sep 27 09:57:10.814560 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 106/225] 10.222.0.26 () {60 vars in 1194 bytes} [Tue Sep 27 09:57:10 2022] POST /compute/v2.1/os-aggregates/13/action => generated 228 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:10.837032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d2f44a8-8b38-4413-af6d-c7c311b11ec0 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:10.847338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4566cf85-2e92-44e0-9fc0-0e3617d0a1d2 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:10.858979 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4566cf85-2e92-44e0-9fc0-0e3617d0a1d2 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.365258 Sep 27 09:57:10.859552 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 120/226] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:10 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:10.870383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-13bb7c55-7629-4d5a-9547-f98a2267b69e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:10.871053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-13bb7c55-7629-4d5a-9547-f98a2267b69e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:10.890569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13bb7c55-7629-4d5a-9547-f98a2267b69e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:10.891374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13bb7c55-7629-4d5a-9547-f98a2267b69e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:10.892153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13bb7c55-7629-4d5a-9547-f98a2267b69e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:10.893224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13bb7c55-7629-4d5a-9547-f98a2267b69e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:10.893896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13bb7c55-7629-4d5a-9547-f98a2267b69e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:10.894597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13bb7c55-7629-4d5a-9547-f98a2267b69e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:10.898349 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d2f44a8-8b38-4413-af6d-c7c311b11ec0 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/14" status: 200 len: 0 microversion: 2.1 time: 0.080672 Sep 27 09:57:10.899714 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 107/227] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:57:10 2022] DELETE /compute/v2.1/os-aggregates/14 => generated 0 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:57:10.910646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41bf76a3-452a-4427-bd2c-59c986569bf8 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:10.960823 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-41bf76a3-452a-4427-bd2c-59c986569bf8 tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/13" status: 200 len: 0 microversion: 2.1 time: 0.059607 Sep 27 09:57:10.969038 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 108/228] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:57:10 2022] DELETE /compute/v2.1/os-aggregates/13 => generated 0 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:57:10.974455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4a53f54b-a963-463b-b593-8ab361f3e067 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:10.985615 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-4a53f54b-a963-463b-b593-8ab361f3e067 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] HTTP exception thrown: Aggregate 14 could not be found. Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4a53f54b-a963-463b-b593-8ab361f3e067 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Request method failure captured: Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-aggregates/14 HTTP/1.1 Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjKqVHHQOm60E8_OXe4GpBUvPDG3GhlC7kAt11OjCJPDsAO9idcqiuKSnLbjIp4RTw3TV0GgfsGZKRRh7lWerdDhpBiAhMDlTO_ul7ePPiex57ggjxxA6KbLdHW6ep8N5Z6dlOhyRsINMdBTQvXFQvOXY44v1jyifYkOW3kLJMU3y4 Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: dc9d0df2395744b0baafe373c8913b91 Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:57:10.987962 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_dc9d0df2395744b0baafe373c8913b91"}]}] Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: dc9d0df2395744b0baafe373c8913b91 Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminTestJSON-1274375176-project-admin Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: e0808d9ea0734660b695111afab964e3 Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminTestJSON-1274375176-project-admin Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: exception: Aggregate 14 could not be found. Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '14'} Sep 27 09:57:10.989417 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 108, in show Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi aggregate = self.api.get_aggregate(context, id) Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6464, in get_aggregate Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return objects.Aggregate.get_by_id(context, aggregate_id) Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 14 could not be found. Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:10.990168 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 110, in show Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 14 could not be found. Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 14 could not be found. Sep 27 09:57:10.991196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:10.992558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4a53f54b-a963-463b-b593-8ab361f3e067 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Returning 404 to user: Aggregate 14 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:10.992558 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4a53f54b-a963-463b-b593-8ab361f3e067 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates/14" status: 404 len: 78 microversion: 2.1 time: 0.020189 Sep 27 09:57:10.992558 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 109/229] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:57:10 2022] GET /compute/v2.1/os-aggregates/14 => generated 78 bytes in 22 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:57:11.001563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-fc4d0d84-9f87-4fdb-97df-d567e8ac99bc req-9a322b6e-cfcd-4a15-b64b-9edcee79e3bf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ee40f956-0a40-43c4-a775-a978e8403d1d", "tag": "a921aca9-77a9-4abe-9b1a-34f56baca86d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:11.017968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-fc4d0d84-9f87-4fdb-97df-d567e8ac99bc req-9a322b6e-cfcd-4a15-b64b-9edcee79e3bf service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:11.019451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-fc4d0d84-9f87-4fdb-97df-d567e8ac99bc req-9a322b6e-cfcd-4a15-b64b-9edcee79e3bf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:11.020294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-fc4d0d84-9f87-4fdb-97df-d567e8ac99bc req-9a322b6e-cfcd-4a15-b64b-9edcee79e3bf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:11.040141 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-fc4d0d84-9f87-4fdb-97df-d567e8ac99bc req-9a322b6e-cfcd-4a15-b64b-9edcee79e3bf service nova] Creating event network-changed:a921aca9-77a9-4abe-9b1a-34f56baca86d for instance ee40f956-0a40-43c4-a775-a978e8403d1d on nch2-master-1 Sep 27 09:57:11.046523 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-fc4d0d84-9f87-4fdb-97df-d567e8ac99bc req-9a322b6e-cfcd-4a15-b64b-9edcee79e3bf service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052340 Sep 27 09:57:11.048061 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 110/230] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:11.056448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d33ebe9-f7e8-49dc-81e6-afe5d1b733db tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1775676260"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:11.063886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-9d33ebe9-f7e8-49dc-81e6-afe5d1b733db tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Generated uuid f82a479c-f93f-4fda-a8e1-6abb29afad3a for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:57:11.110958 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d33ebe9-f7e8-49dc-81e6-afe5d1b733db tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.061111 Sep 27 09:57:11.122382 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 111/231] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:57:11 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:11.131997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf24fc9d-b453-4cd5-8f62-ece8c6c45dd7 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:11.152117 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-bf24fc9d-b453-4cd5-8f62-ece8c6c45dd7 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] HTTP exception thrown: Aggregate 14 could not be found. Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf24fc9d-b453-4cd5-8f62-ece8c6c45dd7 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Request method failure captured: Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-aggregates/14 HTTP/1.1 Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjKqVHHQOm60E8_OXe4GpBUvPDG3GhlC7kAt11OjCJPDsAO9idcqiuKSnLbjIp4RTw3TV0GgfsGZKRRh7lWerdDhpBiAhMDlTO_ul7ePPiex57ggjxxA6KbLdHW6ep8N5Z6dlOhyRsINMdBTQvXFQvOXY44v1jyifYkOW3kLJMU3y4 Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: dc9d0df2395744b0baafe373c8913b91 Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:57:11.155106 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/dc9d0df2395744b0baafe373c8913b91"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_dc9d0df2395744b0baafe373c8913b91"}]}] Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: dc9d0df2395744b0baafe373c8913b91 Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-1274375176 Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminTestJSON-1274375176-project-admin Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: e0808d9ea0734660b695111afab964e3 Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminTestJSON-1274375176-project-admin Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: exception: Aggregate 14 could not be found. Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '14'} Sep 27 09:57:11.156564 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 155, in delete Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.api.delete_aggregate(context, id) Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6517, in delete_aggregate Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi aggregate = objects.Aggregate.get_by_id(context, aggregate_id) Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 274, in get_by_id Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_aggregate = _aggregate_get_from_db(context, aggregate_id) Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:57:11.157434 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/aggregate.py", line 45, in _aggregate_get_from_db Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.AggregateNotFound(aggregate_id=aggregate_id) Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.AggregateNotFound: Aggregate 14 could not be found. Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 157, in delete Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Aggregate 14 could not be found. Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:11.159093 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:11.160532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:11.160532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:11.160532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:11.160532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Aggregate 14 could not be found. Sep 27 09:57:11.160532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.164397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf24fc9d-b453-4cd5-8f62-ece8c6c45dd7 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Returning 404 to user: Aggregate 14 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:11.166344 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bf24fc9d-b453-4cd5-8f62-ece8c6c45dd7 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/14" status: 404 len: 78 microversion: 2.1 time: 0.041430 Sep 27 09:57:11.168729 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 112/232] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:57:11 2022] DELETE /compute/v2.1/os-aggregates/14 => generated 78 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:57:11.179401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nonexist_host"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:11.193273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:11.194010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:11.194748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:11.201972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:11.202967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:11.204067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:11.214058 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.compute.aggregates [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Failed to remove host nonexist_host from aggregate 15. Error: Compute host nonexist_host could not be found.: nova.exception.ComputeHostNotFound: Compute host nonexist_host could not be found. Sep 27 09:57:11.215056 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] HTTP exception thrown: Cannot remove host nonexist_host in aggregate 15 Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Request method failure captured: Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-aggregates/15/action HTTP/1.1 Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 42 Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjH7QYCLZugeJ0HemgmqlSpvzQrDapyIqrhuEZ_iPs-RyHoyEOHHpkZWyJeIyr1hcj35JJZjvQUwiTglpRf4FVRuf_GUwsYg1BP_B9dV7Pni9aC2IYru9U0UypZozOkcJALyLAHHjO5hrSMbhlJr0kvWgGPYkLcuS5TmJ2XrkKwfYg Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:57:11.217822 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fe46a7ae8eab451e941f83e19be3b539"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fe46a7ae8eab451e941f83e19be3b539"}]}] Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: fe46a7ae8eab451e941f83e19be3b539 Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminNegativeTestJSON-331983322 Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 49f35168484b42feb22d106d745e6546 Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: {"remove_host": {"host": "nonexist_host"}} Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:57:11.219302 ncdv-master-1 devstack@n-api.service[145968]: exception: Cannot remove host nonexist_host in aggregate 15 Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '15', 'body': {'remove_host': {'host': 'nonexist_host'}}} Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5999, in _get_service_in_cell_by_host Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi mapping = objects.HostMapping.get_by_host(context, host_name) Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 105, in get_by_host Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_host_from_db(context, host) Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/host_mapping.py", line 100, in _get_by_host_from_db Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.HostMappingNotFound(name=host) Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.HostMappingNotFound: Host 'nonexist_host' is not mapped to any cell Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6005, in _get_service_in_cell_by_host Sep 27 09:57:11.220089 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi service = _find_service_in_cell(context, service_host=host_name) Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6067, in _find_service_in_cell Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 209, in _remove_host Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi aggregate = self.api.remove_host_from_aggregate(context, id, host) Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6683, in remove_host_from_aggregate Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _get_service_in_cell_by_host(context, host_name) Sep 27 09:57:11.221303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6007, in _get_service_in_cell_by_host Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ComputeHostNotFound(host=host_name) Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ComputeHostNotFound: Compute host nonexist_host could not be found. Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/aggregates.py", line 217, in _remove_host Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Cannot remove host nonexist_host in aggregate 15 Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.222407 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:11.223549 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:11.223549 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:11.223549 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:11.223549 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:11.223549 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot remove host nonexist_host in aggregate 15 Sep 27 09:57:11.223549 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:11.223549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Returning 404 to user: Cannot remove host nonexist_host in aggregate 15 {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:11.223549 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-49e4475b-1743-4c1a-af85-5e3b58915cfc tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/15/action" status: 404 len: 94 microversion: 2.1 time: 0.052438 Sep 27 09:57:11.223549 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 113/233] 10.222.0.26 () {60 vars in 1194 bytes} [Tue Sep 27 09:57:11 2022] POST /compute/v2.1/os-aggregates/15/action => generated 94 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:57:11.231527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3466f1d3-3125-40d2-ba98-073dd2850754 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-1912790762"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:11.235406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-3466f1d3-3125-40d2-ba98-073dd2850754 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Generated uuid 5789ebee-0922-448a-a57a-d7ddbc4c145e for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:57:11.266114 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3466f1d3-3125-40d2-ba98-073dd2850754 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 199 microversion: 2.1 time: 0.039158 Sep 27 09:57:11.274940 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 114/234] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:57:11 2022] POST /compute/v2.1/os-aggregates => generated 199 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:11.288010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a0b64683-ab36-4484-92af-717af316cb5e tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:11.320310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-13bb7c55-7629-4d5a-9547-f98a2267b69e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:11.329984 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a0b64683-ab36-4484-92af-717af316cb5e tempest-AggregatesAdminNegativeTestJSON-331983322 tempest-AggregatesAdminNegativeTestJSON-331983322-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/15" status: 200 len: 0 microversion: 2.1 time: 0.053629 Sep 27 09:57:11.331414 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 115/235] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:57:11 2022] DELETE /compute/v2.1/os-aggregates/15 => generated 0 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:57:11.338803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2b06e27c-0137-403f-8910-4e4fa3ee0aca tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:11.350330 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2b06e27c-0137-403f-8910-4e4fa3ee0aca tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates/16" status: 200 len: 228 microversion: 2.1 time: 0.017400 Sep 27 09:57:11.351779 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-13bb7c55-7629-4d5a-9547-f98a2267b69e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.491132 Sep 27 09:57:11.353385 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 121/236] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:10 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:11.376781 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 116/237] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:57:11 2022] GET /compute/v2.1/os-aggregates/16 => generated 228 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:11.382459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-87be58d3-f3cd-46a2-a2ae-5162d691f267 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'action', calling method: >, body: {"set_metadata": {"metadata": {"key": "value"}}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:11.438031 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-87be58d3-f3cd-46a2-a2ae-5162d691f267 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/16/action" status: 200 len: 266 microversion: 2.1 time: 0.064017 Sep 27 09:57:11.439762 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 122/238] 10.222.0.26 () {60 vars in 1194 bytes} [Tue Sep 27 09:57:11 2022] POST /compute/v2.1/os-aggregates/16/action => generated 266 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:11.448816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7299dbc1-3dc0-4498-8f69-0258cf8282dc tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:11.450533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7299dbc1-3dc0-4498-8f69-0258cf8282dc tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:11.451392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b7d6c609-af1d-48a1-a51a-41a9a0eafbe4 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:11.464650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7299dbc1-3dc0-4498-8f69-0258cf8282dc tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:11.465259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7299dbc1-3dc0-4498-8f69-0258cf8282dc tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:11.465965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7299dbc1-3dc0-4498-8f69-0258cf8282dc tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:11.466848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7299dbc1-3dc0-4498-8f69-0258cf8282dc tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:11.467398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7299dbc1-3dc0-4498-8f69-0258cf8282dc tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:11.467831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7299dbc1-3dc0-4498-8f69-0258cf8282dc tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:11.472066 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b7d6c609-af1d-48a1-a51a-41a9a0eafbe4 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates/16" status: 200 len: 242 microversion: 2.1 time: 0.024119 Sep 27 09:57:11.473629 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 123/239] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:57:11 2022] GET /compute/v2.1/os-aggregates/16 => generated 242 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:11.491436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-96ba0055-1558-4f99-8d5c-3cbab3991b54 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:11.548567 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-96ba0055-1558-4f99-8d5c-3cbab3991b54 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/16" status: 200 len: 0 microversion: 2.1 time: 0.062732 Sep 27 09:57:11.562507 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 124/240] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:57:11 2022] DELETE /compute/v2.1/os-aggregates/16 => generated 0 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:57:11.577693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1f975a0e-b213-4238-b58f-b7d20f87eed8 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-523781382", "availability_zone": "tempest-test_az-1835160134"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:11.589890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.aggregate [None req-1f975a0e-b213-4238-b58f-b7d20f87eed8 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Generated uuid c5b654cf-b939-4f5e-b45a-700ade04ab40 for aggregate {{(pid=145969) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:57:11.675800 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1f975a0e-b213-4238-b58f-b7d20f87eed8 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 222 microversion: 2.1 time: 0.105685 Sep 27 09:57:11.679469 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 125/241] 10.222.0.26 () {60 vars in 1165 bytes} [Tue Sep 27 09:57:11 2022] POST /compute/v2.1/os-aggregates => generated 222 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:11.695580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-65c324cb-d962-4d1d-8688-080e26176166 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:11.696724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-65c324cb-d962-4d1d-8688-080e26176166 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:11.711959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65c324cb-d962-4d1d-8688-080e26176166 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:11.716313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65c324cb-d962-4d1d-8688-080e26176166 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:11.718063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65c324cb-d962-4d1d-8688-080e26176166 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:11.719779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65c324cb-d962-4d1d-8688-080e26176166 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:11.721546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65c324cb-d962-4d1d-8688-080e26176166 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:11.726286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65c324cb-d962-4d1d-8688-080e26176166 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:11.840587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7299dbc1-3dc0-4498-8f69-0258cf8282dc tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:11.865229 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7299dbc1-3dc0-4498-8f69-0258cf8282dc tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.436889 Sep 27 09:57:11.868244 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 117/242] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:11 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:11.888262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e9f25f56-0154-4eb8-8d0b-cb3a04d60f96 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'update', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-523781382_new", "availability_zone": "tempest-test_az-1835160134_new"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:12.061222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9f25f56-0154-4eb8-8d0b-cb3a04d60f96 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.062105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9f25f56-0154-4eb8-8d0b-cb3a04d60f96 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.065936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9f25f56-0154-4eb8-8d0b-cb3a04d60f96 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:12.141024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-65c324cb-d962-4d1d-8688-080e26176166 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:12.144902 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e9f25f56-0154-4eb8-8d0b-cb3a04d60f96 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-aggregates/17" status: 200 len: 336 microversion: 2.1 time: 0.267147 Sep 27 09:57:12.148768 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 118/243] 10.222.0.26 () {60 vars in 1173 bytes} [Tue Sep 27 09:57:11 2022] PUT /compute/v2.1/os-aggregates/17 => generated 336 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:12.155190 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-65c324cb-d962-4d1d-8688-080e26176166 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1317 microversion: 2.1 time: 0.469616 Sep 27 09:57:12.165418 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 126/244] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:11 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1317 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:12.168161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67cd85fd-d0ce-443d-ab50-ed4c9ec61b88 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:12.170524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67cd85fd-d0ce-443d-ab50-ed4c9ec61b88 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:12.189870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67cd85fd-d0ce-443d-ab50-ed4c9ec61b88 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.190087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f6260747-d397-4a69-8789-cfdb32b45698 req-a210b8bc-c63a-4bd9-842b-96976fe823b0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "52fadb34-4714-4e01-b7d1-db490dec438e", "tag": "98c20e80-4aac-40db-9819-c3a95358dbc2"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:12.190819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67cd85fd-d0ce-443d-ab50-ed4c9ec61b88 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.191461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67cd85fd-d0ce-443d-ab50-ed4c9ec61b88 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:12.193152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67cd85fd-d0ce-443d-ab50-ed4c9ec61b88 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.193832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67cd85fd-d0ce-443d-ab50-ed4c9ec61b88 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.194475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67cd85fd-d0ce-443d-ab50-ed4c9ec61b88 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:12.217862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f6260747-d397-4a69-8789-cfdb32b45698 req-a210b8bc-c63a-4bd9-842b-96976fe823b0 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.218946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f6260747-d397-4a69-8789-cfdb32b45698 req-a210b8bc-c63a-4bd9-842b-96976fe823b0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.219697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f6260747-d397-4a69-8789-cfdb32b45698 req-a210b8bc-c63a-4bd9-842b-96976fe823b0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:12.236024 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f6260747-d397-4a69-8789-cfdb32b45698 req-a210b8bc-c63a-4bd9-842b-96976fe823b0 service nova] Creating event network-changed:98c20e80-4aac-40db-9819-c3a95358dbc2 for instance 52fadb34-4714-4e01-b7d1-db490dec438e on nch2-master-1 Sep 27 09:57:12.243629 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f6260747-d397-4a69-8789-cfdb32b45698 req-a210b8bc-c63a-4bd9-842b-96976fe823b0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071191 Sep 27 09:57:12.245244 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 127/245] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:12.252633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4a8a8529-79b9-4ca8-bd6e-9dd08cbff4c0 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:12.272240 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4a8a8529-79b9-4ca8-bd6e-9dd08cbff4c0 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates" status: 200 len: 339 microversion: 2.1 time: 0.024589 Sep 27 09:57:12.275896 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 128/246] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 09:57:12 2022] GET /compute/v2.1/os-aggregates => generated 339 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:12.284451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-24ee0bd4-348e-436e-9186-73263dc5eb02 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:12.325110 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-24ee0bd4-348e-436e-9186-73263dc5eb02 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/17" status: 200 len: 0 microversion: 2.1 time: 0.043872 Sep 27 09:57:12.328700 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 129/247] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:57:12 2022] DELETE /compute/v2.1/os-aggregates/17 => generated 0 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 355 bytes (2 switches on core 0) Sep 27 09:57:12.338880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6696bf5f-330c-4cdb-b033-d6499bd79a86 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-test_aggregate-527501244"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:12.343285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.aggregate [None req-6696bf5f-330c-4cdb-b033-d6499bd79a86 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Generated uuid 7b035285-c14d-4b86-a49d-df24b1d9dd57 for aggregate {{(pid=145969) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 09:57:12.377059 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6696bf5f-330c-4cdb-b033-d6499bd79a86 tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 198 microversion: 2.1 time: 0.043073 Sep 27 09:57:12.384338 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 130/248] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 09:57:12 2022] POST /compute/v2.1/os-aggregates => generated 198 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:12.390423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b8d75743-6959-4be0-8f29-e7ff890b5d21 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:12.390654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b8d75743-6959-4be0-8f29-e7ff890b5d21 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:12.397927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8d75743-6959-4be0-8f29-e7ff890b5d21 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.398313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8d75743-6959-4be0-8f29-e7ff890b5d21 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.399074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8d75743-6959-4be0-8f29-e7ff890b5d21 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:12.399590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8d75743-6959-4be0-8f29-e7ff890b5d21 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.400053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8d75743-6959-4be0-8f29-e7ff890b5d21 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.400480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8d75743-6959-4be0-8f29-e7ff890b5d21 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:12.424953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-67cd85fd-d0ce-443d-ab50-ed4c9ec61b88 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:12.436545 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67cd85fd-d0ce-443d-ab50-ed4c9ec61b88 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.277860 Sep 27 09:57:12.437001 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 119/249] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:12 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:12.461770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8794ff29-562f-4a65-8c74-188aa6bea5cc tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:12.470142 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8794ff29-562f-4a65-8c74-188aa6bea5cc tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates" status: 200 len: 230 microversion: 2.1 time: 0.013861 Sep 27 09:57:12.470999 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 120/250] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 09:57:12 2022] GET /compute/v2.1/os-aggregates => generated 230 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:12.483610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-00bad998-d62b-4c05-8692-68ebf9f282ec tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:12.522637 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-00bad998-d62b-4c05-8692-68ebf9f282ec tempest-AggregatesAdminTestJSON-1274375176 tempest-AggregatesAdminTestJSON-1274375176-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/18" status: 200 len: 0 microversion: 2.1 time: 0.041761 Sep 27 09:57:12.523435 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 121/251] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 09:57:12 2022] DELETE /compute/v2.1/os-aggregates/18 => generated 0 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 355 bytes (2 switches on core 0) Sep 27 09:57:12.533727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:12.534231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:12.541503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.542315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.542761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:12.543492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.543955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.544425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:12.549807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:12.562267 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] HTTP exception thrown: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Request method failure captured: Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac HTTP/1.1 Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjHafamTlS-9C5I4M4UbcBJQ1u_43PEJFApwHB_W-rv2AEnXSfBNxtGxxT7V1JqEtILUIzfjkYY1vB0M4LmZMbE1w_Se9O0Mlxkk272eR82WamQNnu8hc4c2JtOlyCbdWOKn09PcQY06xcTaDMoxpToPO6XfOSKqfn7rXRzxVpve9Q Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 681fc4282a6c4fd8ba4f078705f4ca5c Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminTestJSON-1518209963 Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:57:12.565600 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/681fc4282a6c4fd8ba4f078705f4ca5c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/681fc4282a6c4fd8ba4f078705f4ca5c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/681fc4282a6c4fd8ba4f078705f4ca5c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_681fc4282a6c4fd8ba4f078705f4ca5c"}]}] Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminTestJSON-1518209963 Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 681fc4282a6c4fd8ba4f078705f4ca5c Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-1518209963 Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminTestJSON-1518209963-project Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 023730e34b2d4e2a996bfa068a97413c Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminTestJSON-1518209963-project Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:12.566862 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '8febbed7-7a38-4051-8910-bdb62bc11eac'} Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:12.567548 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:12.568677 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:12.569956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Returning 404 to user: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:12.569956 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-23c08ad1-02d2-4124-a680-297b53972610 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "DELETE /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 404 len: 111 microversion: 2.1 time: 0.037508 Sep 27 09:57:12.569956 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 122/252] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:57:12 2022] DELETE /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:12.577150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:12.577948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] [instance: 8febbed7-7a38-4051-8910-bdb62bc11eac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:12.586440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.587447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.587581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:12.588436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.589269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.589807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:12.609314 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] HTTP exception thrown: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Request method failure captured: Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac HTTP/1.1 Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjHafamTlS-9C5I4M4UbcBJQ1u_43PEJFApwHB_W-rv2AEnXSfBNxtGxxT7V1JqEtILUIzfjkYY1vB0M4LmZMbE1w_Se9O0Mlxkk272eR82WamQNnu8hc4c2JtOlyCbdWOKn09PcQY06xcTaDMoxpToPO6XfOSKqfn7rXRzxVpve9Q Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 681fc4282a6c4fd8ba4f078705f4ca5c Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AggregatesAdminTestJSON-1518209963 Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:57:12.611633 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/681fc4282a6c4fd8ba4f078705f4ca5c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/681fc4282a6c4fd8ba4f078705f4ca5c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/681fc4282a6c4fd8ba4f078705f4ca5c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_681fc4282a6c4fd8ba4f078705f4ca5c"}]}] Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AggregatesAdminTestJSON-1518209963 Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 681fc4282a6c4fd8ba4f078705f4ca5c Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AggregatesAdminTestJSON-1518209963 Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AggregatesAdminTestJSON-1518209963-project Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 023730e34b2d4e2a996bfa068a97413c Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AggregatesAdminTestJSON-1518209963-project Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:12.613115 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '8febbed7-7a38-4051-8910-bdb62bc11eac'} Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:12.613934 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. Sep 27 09:57:12.615738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:12.616921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] Returning 404 to user: Instance 8febbed7-7a38-4051-8910-bdb62bc11eac could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:12.616921 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-95d212fe-5e6e-4647-b09e-123347778bd7 tempest-AggregatesAdminTestJSON-1518209963 tempest-AggregatesAdminTestJSON-1518209963-project] 10.222.0.26 "GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac" status: 404 len: 111 microversion: 2.1 time: 0.043838 Sep 27 09:57:12.616921 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 123/253] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:12 2022] GET /compute/v2.1/servers/8febbed7-7a38-4051-8910-bdb62bc11eac => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 09:57:12.634891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b8d75743-6959-4be0-8f29-e7ff890b5d21 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:12.644422 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b8d75743-6959-4be0-8f29-e7ff890b5d21 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.258842 Sep 27 09:57:12.647358 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 131/254] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:12 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:12.894840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d6e88015-9b75-44ab-ba2d-5e3e628e8246 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:12.896102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d6e88015-9b75-44ab-ba2d-5e3e628e8246 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:12.902454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6e88015-9b75-44ab-ba2d-5e3e628e8246 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.902940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6e88015-9b75-44ab-ba2d-5e3e628e8246 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.903575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6e88015-9b75-44ab-ba2d-5e3e628e8246 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:12.904451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6e88015-9b75-44ab-ba2d-5e3e628e8246 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:12.905102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6e88015-9b75-44ab-ba2d-5e3e628e8246 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:12.905728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6e88015-9b75-44ab-ba2d-5e3e628e8246 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:13.071443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-0c35361a-0160-4b87-81ec-b46480f1296d req-77feecec-18cf-4e64-a5f3-b1bc7951a2b9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "552575d5-492a-4680-9972-29347ad5479b", "tag": "be0b1ff7-d8c1-4208-8f00-cdcde277fdef"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:13.093965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0c35361a-0160-4b87-81ec-b46480f1296d req-77feecec-18cf-4e64-a5f3-b1bc7951a2b9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:13.094460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0c35361a-0160-4b87-81ec-b46480f1296d req-77feecec-18cf-4e64-a5f3-b1bc7951a2b9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:13.094996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0c35361a-0160-4b87-81ec-b46480f1296d req-77feecec-18cf-4e64-a5f3-b1bc7951a2b9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:13.124523 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-0c35361a-0160-4b87-81ec-b46480f1296d req-77feecec-18cf-4e64-a5f3-b1bc7951a2b9 service nova] Creating event network-changed:be0b1ff7-d8c1-4208-8f00-cdcde277fdef for instance 552575d5-492a-4680-9972-29347ad5479b on nch1-master-1 Sep 27 09:57:13.137570 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-0c35361a-0160-4b87-81ec-b46480f1296d req-77feecec-18cf-4e64-a5f3-b1bc7951a2b9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072437 Sep 27 09:57:13.138833 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 132/255] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:13.179868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fdd3f169-2c7e-4021-ad32-276374b70fc1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:13.181015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fdd3f169-2c7e-4021-ad32-276374b70fc1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:13.190963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd3f169-2c7e-4021-ad32-276374b70fc1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:13.191454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd3f169-2c7e-4021-ad32-276374b70fc1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:13.196712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd3f169-2c7e-4021-ad32-276374b70fc1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:13.198073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd3f169-2c7e-4021-ad32-276374b70fc1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:13.199430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd3f169-2c7e-4021-ad32-276374b70fc1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:13.200057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd3f169-2c7e-4021-ad32-276374b70fc1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:13.283232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d6e88015-9b75-44ab-ba2d-5e3e628e8246 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:13.293135 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d6e88015-9b75-44ab-ba2d-5e3e628e8246 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.402162 Sep 27 09:57:13.293717 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 124/256] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:12 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:13.463903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d841922-49d5-456a-b71e-c9e4e24eb4ff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:13.464570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9d841922-49d5-456a-b71e-c9e4e24eb4ff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:13.487847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d841922-49d5-456a-b71e-c9e4e24eb4ff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:13.489535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d841922-49d5-456a-b71e-c9e4e24eb4ff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:13.490857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d841922-49d5-456a-b71e-c9e4e24eb4ff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:13.494025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d841922-49d5-456a-b71e-c9e4e24eb4ff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:13.499338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d841922-49d5-456a-b71e-c9e4e24eb4ff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:13.500618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d841922-49d5-456a-b71e-c9e4e24eb4ff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:13.803324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fdd3f169-2c7e-4021-ad32-276374b70fc1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:13.821526 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fdd3f169-2c7e-4021-ad32-276374b70fc1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1317 microversion: 2.1 time: 0.646532 Sep 27 09:57:13.822546 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 133/257] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:13 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1317 bytes in 648 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:13.827399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7d89b8e2-8bec-4c3b-afe1-bbb069a4d9a3 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:13.828283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7d89b8e2-8bec-4c3b-afe1-bbb069a4d9a3 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:13.844364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d89b8e2-8bec-4c3b-afe1-bbb069a4d9a3 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:13.845163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d89b8e2-8bec-4c3b-afe1-bbb069a4d9a3 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:13.845762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d89b8e2-8bec-4c3b-afe1-bbb069a4d9a3 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:13.846691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d89b8e2-8bec-4c3b-afe1-bbb069a4d9a3 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:13.847313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d89b8e2-8bec-4c3b-afe1-bbb069a4d9a3 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:13.847894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d89b8e2-8bec-4c3b-afe1-bbb069a4d9a3 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:14.035795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9d841922-49d5-456a-b71e-c9e4e24eb4ff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:14.052536 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d841922-49d5-456a-b71e-c9e4e24eb4ff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.593220 Sep 27 09:57:14.053185 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 125/258] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:13 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:14.319432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af5a2e22-db77-4f66-a2cf-ea78c2e86b4b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:14.320223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af5a2e22-db77-4f66-a2cf-ea78c2e86b4b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:14.323919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=15 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:14.334277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af5a2e22-db77-4f66-a2cf-ea78c2e86b4b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:14.341506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af5a2e22-db77-4f66-a2cf-ea78c2e86b4b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:14.342072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af5a2e22-db77-4f66-a2cf-ea78c2e86b4b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:14.343408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af5a2e22-db77-4f66-a2cf-ea78c2e86b4b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:14.344254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af5a2e22-db77-4f66-a2cf-ea78c2e86b4b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:14.345088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af5a2e22-db77-4f66-a2cf-ea78c2e86b4b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:14.385593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=18 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:14.649925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7d89b8e2-8bec-4c3b-afe1-bbb069a4d9a3 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:14.665777 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7d89b8e2-8bec-4c3b-afe1-bbb069a4d9a3 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.842030 Sep 27 09:57:14.666376 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 134/259] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:13 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 843 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:14.770438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af5a2e22-db77-4f66-a2cf-ea78c2e86b4b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:14.783570 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af5a2e22-db77-4f66-a2cf-ea78c2e86b4b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.468775 Sep 27 09:57:14.784642 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 126/260] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:14 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:14.846763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5706e42a-a626-41c7-b496-ef74ed3aed4c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:14.847573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5706e42a-a626-41c7-b496-ef74ed3aed4c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:14.863516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e42a-a626-41c7-b496-ef74ed3aed4c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:14.863782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e42a-a626-41c7-b496-ef74ed3aed4c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:14.864145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e42a-a626-41c7-b496-ef74ed3aed4c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:14.864987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e42a-a626-41c7-b496-ef74ed3aed4c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:14.865412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e42a-a626-41c7-b496-ef74ed3aed4c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:14.865840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e42a-a626-41c7-b496-ef74ed3aed4c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:15.105010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c3f02d99-505a-479a-bda9-f2775792955e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:15.106485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c3f02d99-505a-479a-bda9-f2775792955e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:15.122252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3f02d99-505a-479a-bda9-f2775792955e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:15.122799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3f02d99-505a-479a-bda9-f2775792955e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:15.125772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3f02d99-505a-479a-bda9-f2775792955e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:15.129282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3f02d99-505a-479a-bda9-f2775792955e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:15.132115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3f02d99-505a-479a-bda9-f2775792955e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:15.132829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3f02d99-505a-479a-bda9-f2775792955e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:15.185682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5706e42a-a626-41c7-b496-ef74ed3aed4c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:15.201909 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5706e42a-a626-41c7-b496-ef74ed3aed4c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1317 microversion: 2.1 time: 0.358760 Sep 27 09:57:15.202878 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 135/261] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:14 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1317 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:15.456333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c3f02d99-505a-479a-bda9-f2775792955e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:15.471949 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c3f02d99-505a-479a-bda9-f2775792955e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.371951 Sep 27 09:57:15.473197 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 127/262] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:15 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:15.692624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-305cec03-514f-4b2d-a328-a88523b25c58 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:15.694177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-305cec03-514f-4b2d-a328-a88523b25c58 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:15.702299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305cec03-514f-4b2d-a328-a88523b25c58 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:15.702942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305cec03-514f-4b2d-a328-a88523b25c58 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:15.703478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305cec03-514f-4b2d-a328-a88523b25c58 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:15.704399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305cec03-514f-4b2d-a328-a88523b25c58 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:15.705005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305cec03-514f-4b2d-a328-a88523b25c58 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:15.705613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305cec03-514f-4b2d-a328-a88523b25c58 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:15.886772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-305cec03-514f-4b2d-a328-a88523b25c58 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:15.895256 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-305cec03-514f-4b2d-a328-a88523b25c58 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.206500 Sep 27 09:57:15.896066 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 128/263] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:15 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:15.917312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13e48260-2c84-4a97-97f2-8231d0acec4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:15.917949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-13e48260-2c84-4a97-97f2-8231d0acec4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:15.921214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:INSERT=4,nova_api:SELECT=83,nova_api:DELETE=11,nova_api:UPDATE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:15.925752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e48260-2c84-4a97-97f2-8231d0acec4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:15.926574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e48260-2c84-4a97-97f2-8231d0acec4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:15.927553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e48260-2c84-4a97-97f2-8231d0acec4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:15.928618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e48260-2c84-4a97-97f2-8231d0acec4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:15.929285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e48260-2c84-4a97-97f2-8231d0acec4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:15.930187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e48260-2c84-4a97-97f2-8231d0acec4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:15.936692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersAdminTestJSON-server-124457164", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:16.022731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:57:16.102307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-13e48260-2c84-4a97-97f2-8231d0acec4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:16.112720 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13e48260-2c84-4a97-97f2-8231d0acec4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.202017 Sep 27 09:57:16.114433 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 129/264] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:15 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:16.279861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:57:16.280557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:57:16.281989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:57:16.282530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:57:16.283038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:57:16.299180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:16.300312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:16.301324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:16.303127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:16.303867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:16.304611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:16.328262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Getting quotas for project 3132070d235941498e3f64d9d7075f6b. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:16.334339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Getting quotas for user 7ba3c51770094c4ea7fec1e597ae13a0 and project 3132070d235941498e3f64d9d7075f6b. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:16.350768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:57:16.353650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:57:16.419443 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3680ae0f-cd56-4f54-8af9-50e49b9f71a2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.814542 Sep 27 09:57:16.420629 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 136/265] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:57:15 2022] POST /compute/v2.1/servers => generated 380 bytes in 817 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:57:16.426084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f3e2b0a-2310-4084-be04-b69b671cdac4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:16.427564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f3e2b0a-2310-4084-be04-b69b671cdac4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:16.439850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3e2b0a-2310-4084-be04-b69b671cdac4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:16.456744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3e2b0a-2310-4084-be04-b69b671cdac4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:16.457913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3e2b0a-2310-4084-be04-b69b671cdac4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:16.458994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3e2b0a-2310-4084-be04-b69b671cdac4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:16.459634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3e2b0a-2310-4084-be04-b69b671cdac4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:16.460072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3e2b0a-2310-4084-be04-b69b671cdac4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:16.484369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1bb1af75-bdda-4a7d-b25f-dbe349951790 tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Calling method '.version_select at 0x7f7f681e9280>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:16.486308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1bb1af75-bdda-4a7d-b25f-dbe349951790 tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Policy check for os_compute_api:os-hypervisors:list-detail failed with credentials {'is_admin': False, 'user_id': '2e121c7ab3e2401eb5a56962d2815b38', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b19fa985d89f459ba0e811b1779dc602', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1bb1af75-bdda-4a7d-b25f-dbe349951790 tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Request method failure captured: Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-hypervisors/detail HTTP/1.1 Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsj6egAuzMa5ItGgQJWAyGiOAaJT2xqJHzZqIxPPQ_b536W8ln1a0IDV23_nOhn7CyJm4S6rcvYR2KIj88YLQuGBWLt9M--cVDMg3GB1cQCLP_grofp1-sWyhbmmpSyqgvB5moMpqVm5A3PdisUAsFRddrdhq_FM-WR2Oj2x74Qh7dE Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b19fa985d89f459ba0e811b1779dc602 Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:57:16.490267 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b19fa985d89f459ba0e811b1779dc602"}]}] Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b19fa985d89f459ba0e811b1779dc602 Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-HypervisorAdminNegativeTestJSON-1586298082-project Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 2e121c7ab3e2401eb5a56962d2815b38 Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082-project Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: exception: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: action_args: {} Sep 27 09:57:16.491539 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 285, in detail Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._detail(req) Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 289, in _detail Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'list-detail', target={}) Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:57:16.494747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. Sep 27 09:57:16.495760 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:16.496799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1bb1af75-bdda-4a7d-b25f-dbe349951790 tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:list-detail to be performed. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:16.496799 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1bb1af75-bdda-4a7d-b25f-dbe349951790 tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/detail" status: 403 len: 122 microversion: 2.1 time: 0.378315 Sep 27 09:57:16.496799 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 130/266] 10.222.0.26 () {58 vars in 1167 bytes} [Tue Sep 27 09:57:16 2022] GET /compute/v2.1/os-hypervisors/detail => generated 122 bytes in 380 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:16.507145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-840116ee-43ed-4a9f-8639-b829d6cef51e tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:16.507793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-840116ee-43ed-4a9f-8639-b829d6cef51e tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:16.524971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=86,nova_api:DELETE=5,nova_api:INSERT=9 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:16.649710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-840116ee-43ed-4a9f-8639-b829d6cef51e tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:16.661476 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-840116ee-43ed-4a9f-8639-b829d6cef51e tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1219 microversion: 2.1 time: 0.163605 Sep 27 09:57:16.662807 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 131/267] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:16 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1219 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:16.674613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-491f707d-8aa6-426a-b064-1f61c37adfdb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:16.675223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-491f707d-8aa6-426a-b064-1f61c37adfdb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:16.688766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-491f707d-8aa6-426a-b064-1f61c37adfdb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:16.691434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-491f707d-8aa6-426a-b064-1f61c37adfdb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:16.692106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-491f707d-8aa6-426a-b064-1f61c37adfdb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:16.693039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-491f707d-8aa6-426a-b064-1f61c37adfdb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:16.697543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-491f707d-8aa6-426a-b064-1f61c37adfdb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:16.698238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-491f707d-8aa6-426a-b064-1f61c37adfdb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:16.764056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8f3e2b0a-2310-4084-be04-b69b671cdac4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:16.780390 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f3e2b0a-2310-4084-be04-b69b671cdac4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1317 microversion: 2.1 time: 0.354218 Sep 27 09:57:16.781642 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 137/268] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:16 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1317 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:16.790895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4fd7836c-5400-427d-8a17-1288a2d1bd67 tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Calling method '.version_select at 0x7f7f68018820>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:16.792753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4fd7836c-5400-427d-8a17-1288a2d1bd67 tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Policy check for os_compute_api:os-hypervisors:list failed with credentials {'is_admin': False, 'user_id': '2e121c7ab3e2401eb5a56962d2815b38', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b19fa985d89f459ba0e811b1779dc602', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4fd7836c-5400-427d-8a17-1288a2d1bd67 tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Request method failure captured: Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hypervisors HTTP/1.1 Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj6egAuzMa5ItGgQJWAyGiOAaJT2xqJHzZqIxPPQ_b536W8ln1a0IDV23_nOhn7CyJm4S6rcvYR2KIj88YLQuGBWLt9M--cVDMg3GB1cQCLP_grofp1-sWyhbmmpSyqgvB5moMpqVm5A3PdisUAsFRddrdhq_FM-WR2Oj2x74Qh7dE Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: b19fa985d89f459ba0e811b1779dc602 Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:57:16.796515 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b19fa985d89f459ba0e811b1779dc602"}]}] Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: b19fa985d89f459ba0e811b1779dc602 Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HypervisorAdminNegativeTestJSON-1586298082-project Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2e121c7ab3e2401eb5a56962d2815b38 Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082-project Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: action_args: {} Sep 27 09:57:16.797762 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 253, in index Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._index(req) Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 257, in _index Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'list', target={}) Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:57:16.798491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. Sep 27 09:57:16.799548 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:16.802524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4fd7836c-5400-427d-8a17-1288a2d1bd67 tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:list to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:16.803634 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4fd7836c-5400-427d-8a17-1288a2d1bd67 tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 403 len: 115 microversion: 2.1 time: 0.020799 Sep 27 09:57:16.804396 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 138/269] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:57:16 2022] GET /compute/v2.1/os-hypervisors => generated 115 bytes in 22 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:16.828989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e1ace85-4f63-44d5-87a7-c2e6824f5f6f tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Calling method '.version_select at 0x7f7f63faf040>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:16.831942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4e1ace85-4f63-44d5-87a7-c2e6824f5f6f tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Policy check for os_compute_api:os-hypervisors:statistics failed with credentials {'is_admin': False, 'user_id': '2e121c7ab3e2401eb5a56962d2815b38', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b19fa985d89f459ba0e811b1779dc602', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e1ace85-4f63-44d5-87a7-c2e6824f5f6f tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Request method failure captured: Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hypervisors/statistics HTTP/1.1 Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj6egAuzMa5ItGgQJWAyGiOAaJT2xqJHzZqIxPPQ_b536W8ln1a0IDV23_nOhn7CyJm4S6rcvYR2KIj88YLQuGBWLt9M--cVDMg3GB1cQCLP_grofp1-sWyhbmmpSyqgvB5moMpqVm5A3PdisUAsFRddrdhq_FM-WR2Oj2x74Qh7dE Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: b19fa985d89f459ba0e811b1779dc602 Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:57:16.833867 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b19fa985d89f459ba0e811b1779dc602"}]}] Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: b19fa985d89f459ba0e811b1779dc602 Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HypervisorAdminNegativeTestJSON-1586298082-project Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2e121c7ab3e2401eb5a56962d2815b38 Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082-project Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: action_args: {} Sep 27 09:57:16.835059 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 521, in statistics Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'statistics', target={}) Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:57:16.835959 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. Sep 27 09:57:16.837119 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:16.841030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e1ace85-4f63-44d5-87a7-c2e6824f5f6f tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:statistics to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:16.842118 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4e1ace85-4f63-44d5-87a7-c2e6824f5f6f tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/statistics" status: 403 len: 121 microversion: 2.1 time: 0.026606 Sep 27 09:57:16.843197 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 139/270] 10.222.0.26 () {58 vars in 1179 bytes} [Tue Sep 27 09:57:16 2022] GET /compute/v2.1/os-hypervisors/statistics => generated 121 bytes in 28 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:16.932819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-491f707d-8aa6-426a-b064-1f61c37adfdb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:16.944924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-491f707d-8aa6-426a-b064-1f61c37adfdb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.280477 Sep 27 09:57:16.947286 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 132/271] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:16 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:16.953013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-70c5ac50-855d-4901-8441-df94b59de1ab tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:16.953642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-70c5ac50-855d-4901-8441-df94b59de1ab tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:16.961637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70c5ac50-855d-4901-8441-df94b59de1ab tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:16.962750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70c5ac50-855d-4901-8441-df94b59de1ab tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:16.964088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70c5ac50-855d-4901-8441-df94b59de1ab tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:16.965061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70c5ac50-855d-4901-8441-df94b59de1ab tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:16.965710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70c5ac50-855d-4901-8441-df94b59de1ab tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:16.966479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70c5ac50-855d-4901-8441-df94b59de1ab tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.171192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a2107e1b-d6f5-4095-a4b7-5283c60324fe tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Calling method '.version_select at 0x7f7f682cf1f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:17.174014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2107e1b-d6f5-4095-a4b7-5283c60324fe tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.174749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2107e1b-d6f5-4095-a4b7-5283c60324fe tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.178718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2107e1b-d6f5-4095-a4b7-5283c60324fe tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.219376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-70c5ac50-855d-4901-8441-df94b59de1ab tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:17.220694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2107e1b-d6f5-4095-a4b7-5283c60324fe tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.221564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2107e1b-d6f5-4095-a4b7-5283c60324fe tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.222387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2107e1b-d6f5-4095-a4b7-5283c60324fe tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.232574 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-70c5ac50-855d-4901-8441-df94b59de1ab tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.283931 Sep 27 09:57:17.234230 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 133/272] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:16 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:17.245594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eaf2d831-4674-4720-8fd4-f015822f1e8e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:17.246225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eaf2d831-4674-4720-8fd4-f015822f1e8e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:17.252765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf2d831-4674-4720-8fd4-f015822f1e8e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.253476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf2d831-4674-4720-8fd4-f015822f1e8e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.254067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf2d831-4674-4720-8fd4-f015822f1e8e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.254970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf2d831-4674-4720-8fd4-f015822f1e8e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.256005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf2d831-4674-4720-8fd4-f015822f1e8e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.257480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf2d831-4674-4720-8fd4-f015822f1e8e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.265515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2107e1b-d6f5-4095-a4b7-5283c60324fe tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.267560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2107e1b-d6f5-4095-a4b7-5283c60324fe tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.268295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2107e1b-d6f5-4095-a4b7-5283c60324fe tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.283980 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a2107e1b-d6f5-4095-a4b7-5283c60324fe tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.426155 Sep 27 09:57:17.286940 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 140/273] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:57:16 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:17.298586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1439fe7f-95c0-4c53-ade3-f09907c61afc tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Calling method '.version_select at 0x7f7f63fafdc0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:17.300905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1439fe7f-95c0-4c53-ade3-f09907c61afc tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Policy check for os_compute_api:os-hypervisors:uptime failed with credentials {'is_admin': False, 'user_id': '2e121c7ab3e2401eb5a56962d2815b38', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b19fa985d89f459ba0e811b1779dc602', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1439fe7f-95c0-4c53-ade3-f09907c61afc tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Request method failure captured: Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hypervisors/1/uptime HTTP/1.1 Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj6egAuzMa5ItGgQJWAyGiOAaJT2xqJHzZqIxPPQ_b536W8ln1a0IDV23_nOhn7CyJm4S6rcvYR2KIj88YLQuGBWLt9M--cVDMg3GB1cQCLP_grofp1-sWyhbmmpSyqgvB5moMpqVm5A3PdisUAsFRddrdhq_FM-WR2Oj2x74Qh7dE Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: b19fa985d89f459ba0e811b1779dc602 Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:57:17.304691 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b19fa985d89f459ba0e811b1779dc602"}]}] Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: b19fa985d89f459ba0e811b1779dc602 Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HypervisorAdminNegativeTestJSON-1586298082-project Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2e121c7ab3e2401eb5a56962d2815b38 Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082-project Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1'} Sep 27 09:57:17.306472 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 387, in uptime Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'uptime', target={}) Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:57:17.307174 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. Sep 27 09:57:17.308385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.311736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1439fe7f-95c0-4c53-ade3-f09907c61afc tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:uptime to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:17.312896 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1439fe7f-95c0-4c53-ade3-f09907c61afc tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/1/uptime" status: 403 len: 117 microversion: 2.1 time: 0.019665 Sep 27 09:57:17.313989 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 141/274] 10.222.0.26 () {58 vars in 1173 bytes} [Tue Sep 27 09:57:17 2022] GET /compute/v2.1/os-hypervisors/1/uptime => generated 117 bytes in 21 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:17.324539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1e0e1cb4-c2ee-4f8e-8996-f88781a068aa tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Calling method '.version_select at 0x7f7f682148b0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:17.326363 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-1e0e1cb4-c2ee-4f8e-8996-f88781a068aa tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] HTTP exception thrown: Hypervisor with ID '27ba2cae-c571-4b2e-ac51-7f16035817d8' could not be found. Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1e0e1cb4-c2ee-4f8e-8996-f88781a068aa tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Request method failure captured: Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hypervisors/27ba2cae-c571-4b2e-ac51-7f16035817d8/uptime HTTP/1.1 Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj7LeSdRN4KN2Hl0xXZn29q715-wh7yp83MY3G0syy-dKSz7tK2-yC7lHH2fv1kvn2PF1_Npg4lWvawFZDojVb1VzVF2hyfeGUqnf9CmsB-yvaO8MeI4B1yUhuvg7md4fOzh7akT6sldnKBU1sy8Z-9AKf39kuC-e0hlu1uYKgJMlo Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 991c193b9ac144c18fc25251ae253764 Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1159914123 Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:57:17.339087 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/991c193b9ac144c18fc25251ae253764"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/991c193b9ac144c18fc25251ae253764"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/991c193b9ac144c18fc25251ae253764"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_991c193b9ac144c18fc25251ae253764"}]}] Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1159914123 Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 991c193b9ac144c18fc25251ae253764 Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1159914123 Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 5430c5ee5cbf4702b9f5056a47b02ad2 Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: exception: Hypervisor with ID '27ba2cae-c571-4b2e-ac51-7f16035817d8' could not be found. Sep 27 09:57:17.340636 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '27ba2cae-c571-4b2e-ac51-7f16035817d8'} Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 501, in validate_integer Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi value = int(str(value)) Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ValueError: invalid literal for int() with base 10: '27ba2cae-c571-4b2e-ac51-7f16035817d8' Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 614, in validate_integer Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return strutils.validate_integer(value, name, min_value, max_value) Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 505, in validate_integer Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise ValueError(msg) Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ValueError: id must be an integer Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 313, in _validate_id Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi utils.validate_integer(hypervisor_id, 'id') Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 616, in validate_integer Sep 27 09:57:17.341460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InvalidInput(reason=str(e)) Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: id must be an integer Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 389, in uptime Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._validate_id(req, id) Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 317, in _validate_id Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Hypervisor with ID '27ba2cae-c571-4b2e-ac51-7f16035817d8' could not be found. Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:17.343250 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.344425 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.344425 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:17.344425 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:17.344425 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:17.344425 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:17.344425 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Hypervisor with ID '27ba2cae-c571-4b2e-ac51-7f16035817d8' could not be found. Sep 27 09:57:17.344425 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.346271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1e0e1cb4-c2ee-4f8e-8996-f88781a068aa tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Returning 404 to user: Hypervisor with ID '27ba2cae-c571-4b2e-ac51-7f16035817d8' could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:17.349386 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1e0e1cb4-c2ee-4f8e-8996-f88781a068aa tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/27ba2cae-c571-4b2e-ac51-7f16035817d8/uptime" status: 404 len: 123 microversion: 2.1 time: 0.029320 Sep 27 09:57:17.350414 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 142/275] 10.222.0.26 () {58 vars in 1278 bytes} [Tue Sep 27 09:57:17 2022] GET /compute/v2.1/os-hypervisors/27ba2cae-c571-4b2e-ac51-7f16035817d8/uptime => generated 123 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 09:57:17.360749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ae77d328-91df-4b41-a649-2764f9d4e822 tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Calling method '.version_select at 0x7f7f63faf5e0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:17.362912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae77d328-91df-4b41-a649-2764f9d4e822 tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.363761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae77d328-91df-4b41-a649-2764f9d4e822 tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.364353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae77d328-91df-4b41-a649-2764f9d4e822 tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.389613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae77d328-91df-4b41-a649-2764f9d4e822 tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.390231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae77d328-91df-4b41-a649-2764f9d4e822 tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.390671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae77d328-91df-4b41-a649-2764f9d4e822 tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.402042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae77d328-91df-4b41-a649-2764f9d4e822 tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.402539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae77d328-91df-4b41-a649-2764f9d4e822 tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.402983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae77d328-91df-4b41-a649-2764f9d4e822 tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.422282 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ae77d328-91df-4b41-a649-2764f9d4e822 tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.065962 Sep 27 09:57:17.423157 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 143/276] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:57:17 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:17.441135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7498d7b0-37ba-4c57-8158-3a0130cddf4a tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Calling method '.version_select at 0x7f7f63faff70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:17.443529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7498d7b0-37ba-4c57-8158-3a0130cddf4a tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Policy check for os_compute_api:os-hypervisors:show failed with credentials {'is_admin': False, 'user_id': '2e121c7ab3e2401eb5a56962d2815b38', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b19fa985d89f459ba0e811b1779dc602', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7498d7b0-37ba-4c57-8158-3a0130cddf4a tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Request method failure captured: Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hypervisors/1 HTTP/1.1 Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj6egAuzMa5ItGgQJWAyGiOAaJT2xqJHzZqIxPPQ_b536W8ln1a0IDV23_nOhn7CyJm4S6rcvYR2KIj88YLQuGBWLt9M--cVDMg3GB1cQCLP_grofp1-sWyhbmmpSyqgvB5moMpqVm5A3PdisUAsFRddrdhq_FM-WR2Oj2x74Qh7dE Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: b19fa985d89f459ba0e811b1779dc602 Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:57:17.446312 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b19fa985d89f459ba0e811b1779dc602"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b19fa985d89f459ba0e811b1779dc602"}]}] Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: b19fa985d89f459ba0e811b1779dc602 Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082 Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HypervisorAdminNegativeTestJSON-1586298082-project Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2e121c7ab3e2401eb5a56962d2815b38 Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1586298082-project Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1'} Sep 27 09:57:17.447567 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 337, in show Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._show(req, id) Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 341, in _show Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(hv_policies.BASE_POLICY_NAME % 'show', target={}) Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:57:17.448360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. Sep 27 09:57:17.449423 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.450441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7498d7b0-37ba-4c57-8158-3a0130cddf4a tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-hypervisors:show to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:17.452601 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7498d7b0-37ba-4c57-8158-3a0130cddf4a tempest-HypervisorAdminNegativeTestJSON-1586298082 tempest-HypervisorAdminNegativeTestJSON-1586298082-project] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/1" status: 403 len: 115 microversion: 2.1 time: 0.021793 Sep 27 09:57:17.453438 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 144/277] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:57:17 2022] GET /compute/v2.1/os-hypervisors/1 => generated 115 bytes in 23 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:17.463185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-eaf2d831-4674-4720-8fd4-f015822f1e8e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:17.464637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0498f5b9-2522-4844-8c46-74404d4bd55b tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Calling method '.version_select at 0x7f7f68018ee0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:17.466346 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-0498f5b9-2522-4844-8c46-74404d4bd55b tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] HTTP exception thrown: Hypervisor with ID '873d2898-ca10-45e6-89fd-691f99ffe7e2' could not be found. Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0498f5b9-2522-4844-8c46-74404d4bd55b tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Request method failure captured: Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-hypervisors/873d2898-ca10-45e6-89fd-691f99ffe7e2 HTTP/1.1 Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj7LeSdRN4KN2Hl0xXZn29q715-wh7yp83MY3G0syy-dKSz7tK2-yC7lHH2fv1kvn2PF1_Npg4lWvawFZDojVb1VzVF2hyfeGUqnf9CmsB-yvaO8MeI4B1yUhuvg7md4fOzh7akT6sldnKBU1sy8Z-9AKf39kuC-e0hlu1uYKgJMlo Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 991c193b9ac144c18fc25251ae253764 Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-HypervisorAdminNegativeTestJSON-1159914123 Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:57:17.468156 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/991c193b9ac144c18fc25251ae253764"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/991c193b9ac144c18fc25251ae253764"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/991c193b9ac144c18fc25251ae253764"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_991c193b9ac144c18fc25251ae253764"}]}] Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-HypervisorAdminNegativeTestJSON-1159914123 Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 991c193b9ac144c18fc25251ae253764 Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-HypervisorAdminNegativeTestJSON-1159914123 Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 5430c5ee5cbf4702b9f5056a47b02ad2 Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: exception: Hypervisor with ID '873d2898-ca10-45e6-89fd-691f99ffe7e2' could not be found. Sep 27 09:57:17.469398 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '873d2898-ca10-45e6-89fd-691f99ffe7e2'} Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 501, in validate_integer Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi value = int(str(value)) Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ValueError: invalid literal for int() with base 10: '873d2898-ca10-45e6-89fd-691f99ffe7e2' Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 614, in validate_integer Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return strutils.validate_integer(value, name, min_value, max_value) Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/strutils.py", line 505, in validate_integer Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise ValueError(msg) Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ValueError: id must be an integer Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 313, in _validate_id Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi utils.validate_integer(hypervisor_id, 'id') Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/utils.py", line 616, in validate_integer Sep 27 09:57:17.470099 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InvalidInput(reason=str(e)) Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: id must be an integer Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 337, in show Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._show(req, id) Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 343, in _show Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._validate_id(req, id) Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/hypervisors.py", line 317, in _validate_id Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Sep 27 09:57:17.471272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Hypervisor with ID '873d2898-ca10-45e6-89fd-691f99ffe7e2' could not be found. Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Hypervisor with ID '873d2898-ca10-45e6-89fd-691f99ffe7e2' could not be found. Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0498f5b9-2522-4844-8c46-74404d4bd55b tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] Returning 404 to user: Hypervisor with ID '873d2898-ca10-45e6-89fd-691f99ffe7e2' could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0498f5b9-2522-4844-8c46-74404d4bd55b tempest-HypervisorAdminNegativeTestJSON-1159914123 tempest-HypervisorAdminNegativeTestJSON-1159914123-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/873d2898-ca10-45e6-89fd-691f99ffe7e2" status: 404 len: 123 microversion: 2.1 time: 0.010716 Sep 27 09:57:17.472450 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 145/278] 10.222.0.26 () {58 vars in 1257 bytes} [Tue Sep 27 09:57:17 2022] GET /compute/v2.1/os-hypervisors/873d2898-ca10-45e6-89fd-691f99ffe7e2 => generated 123 bytes in 12 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:17.477297 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eaf2d831-4674-4720-8fd4-f015822f1e8e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.241637 Sep 27 09:57:17.478223 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 134/279] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:17 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:17.690050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a9a277d0-e896-4d1c-bd95-084bcfb4d4d7 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:17.691160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a9a277d0-e896-4d1c-bd95-084bcfb4d4d7 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:17.698871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a277d0-e896-4d1c-bd95-084bcfb4d4d7 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.699688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a277d0-e896-4d1c-bd95-084bcfb4d4d7 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.700128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a277d0-e896-4d1c-bd95-084bcfb4d4d7 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.700929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a277d0-e896-4d1c-bd95-084bcfb4d4d7 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.701904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a277d0-e896-4d1c-bd95-084bcfb4d4d7 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.702332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a277d0-e896-4d1c-bd95-084bcfb4d4d7 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.781418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a9a277d0-e896-4d1c-bd95-084bcfb4d4d7 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:17.790222 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a9a277d0-e896-4d1c-bd95-084bcfb4d4d7 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1271 microversion: 2.1 time: 0.107035 Sep 27 09:57:17.792793 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 146/280] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:17 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1271 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:17.828793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-52dd4c88-f5e1-432e-add5-63852651d10e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:17.829419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-52dd4c88-f5e1-432e-add5-63852651d10e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:17.837728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52dd4c88-f5e1-432e-add5-63852651d10e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.838223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52dd4c88-f5e1-432e-add5-63852651d10e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.838655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52dd4c88-f5e1-432e-add5-63852651d10e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.839364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52dd4c88-f5e1-432e-add5-63852651d10e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.839815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52dd4c88-f5e1-432e-add5-63852651d10e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.840302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52dd4c88-f5e1-432e-add5-63852651d10e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.884436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-3b4a579c-a7b2-446a-a854-0961ef6a8ecf req-dcbf2196-0499-428d-842c-613668871aaa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "030e5dcc-dc63-43e6-9254-ef26b66b5cde", "name": "network-vif-plugged", "status": "completed", "tag": "3dbbf26e-6eb1-444a-bd75-862612ffb4bc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:17.905201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3b4a579c-a7b2-446a-a854-0961ef6a8ecf req-dcbf2196-0499-428d-842c-613668871aaa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.906826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3b4a579c-a7b2-446a-a854-0961ef6a8ecf req-dcbf2196-0499-428d-842c-613668871aaa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.907267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3b4a579c-a7b2-446a-a854-0961ef6a8ecf req-dcbf2196-0499-428d-842c-613668871aaa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.940519 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-3b4a579c-a7b2-446a-a854-0961ef6a8ecf req-dcbf2196-0499-428d-842c-613668871aaa service nova] Creating event network-vif-plugged:3dbbf26e-6eb1-444a-bd75-862612ffb4bc for instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde on nch2-master-1 Sep 27 09:57:17.950126 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-3b4a579c-a7b2-446a-a854-0961ef6a8ecf req-dcbf2196-0499-428d-842c-613668871aaa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070713 Sep 27 09:57:17.951164 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 147/281] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:17.974478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1d709c81-68b7-4332-8718-ff65bbce74d8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:17.975176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1d709c81-68b7-4332-8718-ff65bbce74d8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:17.982452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d709c81-68b7-4332-8718-ff65bbce74d8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.983363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d709c81-68b7-4332-8718-ff65bbce74d8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.983798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d709c81-68b7-4332-8718-ff65bbce74d8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:17.984513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d709c81-68b7-4332-8718-ff65bbce74d8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:17.985012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d709c81-68b7-4332-8718-ff65bbce74d8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:17.985612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d709c81-68b7-4332-8718-ff65bbce74d8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:18.178036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-52dd4c88-f5e1-432e-add5-63852651d10e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:18.189463 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-52dd4c88-f5e1-432e-add5-63852651d10e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1317 microversion: 2.1 time: 0.365557 Sep 27 09:57:18.190071 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 135/282] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:17 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1317 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:18.230540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1d709c81-68b7-4332-8718-ff65bbce74d8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:18.244759 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1d709c81-68b7-4332-8718-ff65bbce74d8 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.275127 Sep 27 09:57:18.246329 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 148/283] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:17 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:18.266037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-374a6163-0f61-4837-92fd-3aab13ab30c6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:18.272252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-374a6163-0f61-4837-92fd-3aab13ab30c6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:18.281985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-374a6163-0f61-4837-92fd-3aab13ab30c6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:18.284631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-374a6163-0f61-4837-92fd-3aab13ab30c6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:18.287481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-374a6163-0f61-4837-92fd-3aab13ab30c6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:18.288964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-374a6163-0f61-4837-92fd-3aab13ab30c6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:18.289738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-374a6163-0f61-4837-92fd-3aab13ab30c6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:18.290394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-374a6163-0f61-4837-92fd-3aab13ab30c6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:18.503343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fc9060e9-51f1-47d9-a0aa-efb0c339af5b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:18.504515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fc9060e9-51f1-47d9-a0aa-efb0c339af5b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:18.529637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9060e9-51f1-47d9-a0aa-efb0c339af5b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:18.531171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9060e9-51f1-47d9-a0aa-efb0c339af5b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:18.531633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9060e9-51f1-47d9-a0aa-efb0c339af5b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:18.532454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9060e9-51f1-47d9-a0aa-efb0c339af5b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:18.533172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9060e9-51f1-47d9-a0aa-efb0c339af5b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:18.533277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9060e9-51f1-47d9-a0aa-efb0c339af5b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:18.734284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fc9060e9-51f1-47d9-a0aa-efb0c339af5b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:18.743024 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fc9060e9-51f1-47d9-a0aa-efb0c339af5b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.245624 Sep 27 09:57:18.744730 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 149/284] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:18 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:18.796122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-374a6163-0f61-4837-92fd-3aab13ab30c6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:18.808042 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-374a6163-0f61-4837-92fd-3aab13ab30c6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.556045 Sep 27 09:57:18.822339 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 136/285] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:18 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:18.833413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fc1cc5cc-e1a0-4f48-b65a-7b94cbdd021b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:18.834265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fc1cc5cc-e1a0-4f48-b65a-7b94cbdd021b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:18.840979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1cc5cc-e1a0-4f48-b65a-7b94cbdd021b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:18.842174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1cc5cc-e1a0-4f48-b65a-7b94cbdd021b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:18.842842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1cc5cc-e1a0-4f48-b65a-7b94cbdd021b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:18.843756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1cc5cc-e1a0-4f48-b65a-7b94cbdd021b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:18.844428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1cc5cc-e1a0-4f48-b65a-7b94cbdd021b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:18.845089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1cc5cc-e1a0-4f48-b65a-7b94cbdd021b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:18.951840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc1cc5cc-e1a0-4f48-b65a-7b94cbdd021b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:18.967718 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fc1cc5cc-e1a0-4f48-b65a-7b94cbdd021b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1277 microversion: 2.1 time: 0.143140 Sep 27 09:57:18.973398 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 137/286] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:18 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1277 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:18.989226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea8a51db-0d74-40ac-b366-6c51fa4e77c8 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:18.990787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea8a51db-0d74-40ac-b366-6c51fa4e77c8 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:18.991270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea8a51db-0d74-40ac-b366-6c51fa4e77c8 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:18.992580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea8a51db-0d74-40ac-b366-6c51fa4e77c8 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:18.993933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea8a51db-0d74-40ac-b366-6c51fa4e77c8 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:18.994609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea8a51db-0d74-40ac-b366-6c51fa4e77c8 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:18.995255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea8a51db-0d74-40ac-b366-6c51fa4e77c8 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:19.025117 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea8a51db-0d74-40ac-b366-6c51fa4e77c8 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] 10.222.0.26 "GET /compute/v2.1/os-availability-zone" status: 200 len: 97 microversion: 2.1 time: 0.230548 Sep 27 09:57:19.032752 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 150/287] 10.222.0.26 () {58 vars in 1164 bytes} [Tue Sep 27 09:57:18 2022] GET /compute/v2.1/os-availability-zone => generated 97 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 27 09:57:19.038168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8286eaf4-69c0-469a-8fa9-5ecc6cb37b84 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:19.040308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8286eaf4-69c0-469a-8fa9-5ecc6cb37b84 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:19.040846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8286eaf4-69c0-469a-8fa9-5ecc6cb37b84 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:19.041326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8286eaf4-69c0-469a-8fa9-5ecc6cb37b84 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:19.042106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8286eaf4-69c0-469a-8fa9-5ecc6cb37b84 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:19.042539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8286eaf4-69c0-469a-8fa9-5ecc6cb37b84 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:19.043595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8286eaf4-69c0-469a-8fa9-5ecc6cb37b84 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:19.075140 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8286eaf4-69c0-469a-8fa9-5ecc6cb37b84 tempest-AZAdminV2TestJSON-2079226600 tempest-AZAdminV2TestJSON-2079226600-project-admin] 10.222.0.26 "GET /compute/v2.1/os-availability-zone/detail" status: 200 len: 613 microversion: 2.1 time: 0.040843 Sep 27 09:57:19.075773 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 138/288] 10.222.0.26 () {58 vars in 1185 bytes} [Tue Sep 27 09:57:19 2022] GET /compute/v2.1/os-availability-zone/detail => generated 613 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:19.217069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ac18bfe7-b535-462a-a1d1-227369401a57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:19.217691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ac18bfe7-b535-462a-a1d1-227369401a57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:19.234889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac18bfe7-b535-462a-a1d1-227369401a57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:19.236124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac18bfe7-b535-462a-a1d1-227369401a57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:19.236904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac18bfe7-b535-462a-a1d1-227369401a57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:19.238426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac18bfe7-b535-462a-a1d1-227369401a57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:19.239933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac18bfe7-b535-462a-a1d1-227369401a57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:19.240559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac18bfe7-b535-462a-a1d1-227369401a57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:19.272074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eda082f3-e002-4760-9e4e-aeb3929a380d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:19.272822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eda082f3-e002-4760-9e4e-aeb3929a380d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:19.281985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eda082f3-e002-4760-9e4e-aeb3929a380d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:19.283500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eda082f3-e002-4760-9e4e-aeb3929a380d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:19.284855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eda082f3-e002-4760-9e4e-aeb3929a380d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:19.285811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eda082f3-e002-4760-9e4e-aeb3929a380d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:19.286522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eda082f3-e002-4760-9e4e-aeb3929a380d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:19.287250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eda082f3-e002-4760-9e4e-aeb3929a380d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:19.508477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ac18bfe7-b535-462a-a1d1-227369401a57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:19.523772 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ac18bfe7-b535-462a-a1d1-227369401a57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1317 microversion: 2.1 time: 0.312601 Sep 27 09:57:19.525286 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 151/289] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:19 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1317 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:19.598429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-eda082f3-e002-4760-9e4e-aeb3929a380d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:19.615291 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eda082f3-e002-4760-9e4e-aeb3929a380d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.347921 Sep 27 09:57:19.617477 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 139/290] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:19 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:19.768169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3c67e05e-d5cf-419b-ae9f-8d1783fdeb72 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:19.769048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3c67e05e-d5cf-419b-ae9f-8d1783fdeb72 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:19.790915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c67e05e-d5cf-419b-ae9f-8d1783fdeb72 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:19.791438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c67e05e-d5cf-419b-ae9f-8d1783fdeb72 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:19.791883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c67e05e-d5cf-419b-ae9f-8d1783fdeb72 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:19.792623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c67e05e-d5cf-419b-ae9f-8d1783fdeb72 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:19.796721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c67e05e-d5cf-419b-ae9f-8d1783fdeb72 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:19.798830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c67e05e-d5cf-419b-ae9f-8d1783fdeb72 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:19.817889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:19.845288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4358da5-741e-45fc-a699-649c258ef566 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:19.846922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4358da5-741e-45fc-a699-649c258ef566 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:19.855994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4358da5-741e-45fc-a699-649c258ef566 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:19.857080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4358da5-741e-45fc-a699-649c258ef566 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:19.857766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4358da5-741e-45fc-a699-649c258ef566 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:19.859243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4358da5-741e-45fc-a699-649c258ef566 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:19.860099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4358da5-741e-45fc-a699-649c258ef566 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:19.861369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4358da5-741e-45fc-a699-649c258ef566 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:20.215996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c67e05e-d5cf-419b-ae9f-8d1783fdeb72 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:20.228550 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3c67e05e-d5cf-419b-ae9f-8d1783fdeb72 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.462902 Sep 27 09:57:20.229220 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 152/291] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:19 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:20.240823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-fab9baf4-cb4d-44e5-99e5-07255d9a33ae req-eb0c6c2f-6638-47e9-9239-a19773d7f5f0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "552575d5-492a-4680-9972-29347ad5479b", "name": "network-vif-plugged", "status": "completed", "tag": "be0b1ff7-d8c1-4208-8f00-cdcde277fdef"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:20.264892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fab9baf4-cb4d-44e5-99e5-07255d9a33ae req-eb0c6c2f-6638-47e9-9239-a19773d7f5f0 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:20.265392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fab9baf4-cb4d-44e5-99e5-07255d9a33ae req-eb0c6c2f-6638-47e9-9239-a19773d7f5f0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:20.266342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fab9baf4-cb4d-44e5-99e5-07255d9a33ae req-eb0c6c2f-6638-47e9-9239-a19773d7f5f0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:20.283203 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-fab9baf4-cb4d-44e5-99e5-07255d9a33ae req-eb0c6c2f-6638-47e9-9239-a19773d7f5f0 service nova] Creating event network-vif-plugged:be0b1ff7-d8c1-4208-8f00-cdcde277fdef for instance 552575d5-492a-4680-9972-29347ad5479b on nch1-master-1 Sep 27 09:57:20.292041 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-fab9baf4-cb4d-44e5-99e5-07255d9a33ae req-eb0c6c2f-6638-47e9-9239-a19773d7f5f0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061281 Sep 27 09:57:20.293957 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 153/292] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:20.302906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8788adbe-ea44-4437-8dce-838a6213d4ea tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:20.303873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8788adbe-ea44-4437-8dce-838a6213d4ea tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:20.312369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8788adbe-ea44-4437-8dce-838a6213d4ea tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:20.313294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8788adbe-ea44-4437-8dce-838a6213d4ea tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:20.313905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8788adbe-ea44-4437-8dce-838a6213d4ea tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:20.314914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8788adbe-ea44-4437-8dce-838a6213d4ea tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:20.317128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8788adbe-ea44-4437-8dce-838a6213d4ea tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:20.317939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8788adbe-ea44-4437-8dce-838a6213d4ea tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:20.385449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4358da5-741e-45fc-a699-649c258ef566 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:20.395363 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4358da5-741e-45fc-a699-649c258ef566 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.561089 Sep 27 09:57:20.396238 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 140/293] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:19 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:20.548860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-31617fbc-f75c-49ff-8450-0f23ee9c4e72 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:20.550021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-31617fbc-f75c-49ff-8450-0f23ee9c4e72 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:20.559659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31617fbc-f75c-49ff-8450-0f23ee9c4e72 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:20.566060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31617fbc-f75c-49ff-8450-0f23ee9c4e72 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:20.569141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31617fbc-f75c-49ff-8450-0f23ee9c4e72 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:20.569989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31617fbc-f75c-49ff-8450-0f23ee9c4e72 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:20.570692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31617fbc-f75c-49ff-8450-0f23ee9c4e72 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:20.571135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31617fbc-f75c-49ff-8450-0f23ee9c4e72 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:20.882561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8788adbe-ea44-4437-8dce-838a6213d4ea tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:20.897545 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8788adbe-ea44-4437-8dce-838a6213d4ea tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1319 microversion: 2.1 time: 0.601998 Sep 27 09:57:20.901187 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 154/294] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:20 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1319 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:20.927839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b79b9235-0310-409e-a406-08ad8c1740b5 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:20.930733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b79b9235-0310-409e-a406-08ad8c1740b5 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:20.939806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b79b9235-0310-409e-a406-08ad8c1740b5 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:20.940207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b79b9235-0310-409e-a406-08ad8c1740b5 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:20.940948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b79b9235-0310-409e-a406-08ad8c1740b5 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:20.943019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b79b9235-0310-409e-a406-08ad8c1740b5 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:20.943641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b79b9235-0310-409e-a406-08ad8c1740b5 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:20.944411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b79b9235-0310-409e-a406-08ad8c1740b5 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:21.118617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-31617fbc-f75c-49ff-8450-0f23ee9c4e72 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:21.135733 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-31617fbc-f75c-49ff-8450-0f23ee9c4e72 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1317 microversion: 2.1 time: 0.591090 Sep 27 09:57:21.137674 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 141/295] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:20 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1317 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:21.149592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b79b9235-0310-409e-a406-08ad8c1740b5 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:21.166812 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b79b9235-0310-409e-a406-08ad8c1740b5 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.263483 Sep 27 09:57:21.167845 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 155/296] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:20 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:21.257755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29ee44cf-34af-4143-9d68-f18466abd5a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:21.259826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-29ee44cf-34af-4143-9d68-f18466abd5a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:21.273471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ee44cf-34af-4143-9d68-f18466abd5a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:21.274554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ee44cf-34af-4143-9d68-f18466abd5a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:21.275694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ee44cf-34af-4143-9d68-f18466abd5a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:21.277292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ee44cf-34af-4143-9d68-f18466abd5a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:21.278265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ee44cf-34af-4143-9d68-f18466abd5a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:21.279499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ee44cf-34af-4143-9d68-f18466abd5a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:21.418710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7edc7e52-33bf-4941-a2cd-696a8a647c51 req-b3b9567e-7692-4c54-8fc7-540a95efdb20 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2236fa92-0b7d-4029-a7fd-ffbb09df76bb", "tag": "bf3e8c0e-48d1-4bd7-ba5c-d02f03b4e2bf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:21.439660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7edc7e52-33bf-4941-a2cd-696a8a647c51 req-b3b9567e-7692-4c54-8fc7-540a95efdb20 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:21.440367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7edc7e52-33bf-4941-a2cd-696a8a647c51 req-b3b9567e-7692-4c54-8fc7-540a95efdb20 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:21.440831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7edc7e52-33bf-4941-a2cd-696a8a647c51 req-b3b9567e-7692-4c54-8fc7-540a95efdb20 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:21.457709 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7edc7e52-33bf-4941-a2cd-696a8a647c51 req-b3b9567e-7692-4c54-8fc7-540a95efdb20 service nova] Creating event network-changed:bf3e8c0e-48d1-4bd7-ba5c-d02f03b4e2bf for instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb on nch1-master-1 Sep 27 09:57:21.465039 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7edc7e52-33bf-4941-a2cd-696a8a647c51 req-b3b9567e-7692-4c54-8fc7-540a95efdb20 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049915 Sep 27 09:57:21.467171 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 156/297] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:21.474568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-39a02b25-6327-4d0a-a2e1-c898e886627e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:21.475494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-39a02b25-6327-4d0a-a2e1-c898e886627e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:21.491770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a02b25-6327-4d0a-a2e1-c898e886627e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:21.499709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a02b25-6327-4d0a-a2e1-c898e886627e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:21.503972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a02b25-6327-4d0a-a2e1-c898e886627e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:21.504615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a02b25-6327-4d0a-a2e1-c898e886627e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:21.505125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a02b25-6327-4d0a-a2e1-c898e886627e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:21.505549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a02b25-6327-4d0a-a2e1-c898e886627e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:21.685920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29ee44cf-34af-4143-9d68-f18466abd5a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:21.691849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:21.724901 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-29ee44cf-34af-4143-9d68-f18466abd5a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.473598 Sep 27 09:57:21.726814 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 142/298] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:21 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:21.755968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-39a02b25-6327-4d0a-a2e1-c898e886627e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:21.780737 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-39a02b25-6327-4d0a-a2e1-c898e886627e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.311255 Sep 27 09:57:21.785540 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 157/299] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:21 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:21.929266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92841b43-aabe-479b-8e52-9a306b746c3b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:21.932019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-92841b43-aabe-479b-8e52-9a306b746c3b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:21.947075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92841b43-aabe-479b-8e52-9a306b746c3b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:21.948286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92841b43-aabe-479b-8e52-9a306b746c3b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:21.949367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92841b43-aabe-479b-8e52-9a306b746c3b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:21.950997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92841b43-aabe-479b-8e52-9a306b746c3b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:21.952091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92841b43-aabe-479b-8e52-9a306b746c3b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:21.953110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92841b43-aabe-479b-8e52-9a306b746c3b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:22.159947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-04ea916c-9378-4cdc-bda2-21f0142f604c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:22.160498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-04ea916c-9378-4cdc-bda2-21f0142f604c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:22.171675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ea916c-9378-4cdc-bda2-21f0142f604c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:22.172232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ea916c-9378-4cdc-bda2-21f0142f604c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:22.174012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ea916c-9378-4cdc-bda2-21f0142f604c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:22.176902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ea916c-9378-4cdc-bda2-21f0142f604c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:22.179062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ea916c-9378-4cdc-bda2-21f0142f604c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:22.179567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ea916c-9378-4cdc-bda2-21f0142f604c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:22.494647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-92841b43-aabe-479b-8e52-9a306b746c3b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:22.495592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-04ea916c-9378-4cdc-bda2-21f0142f604c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:22.512291 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-92841b43-aabe-479b-8e52-9a306b746c3b tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1319 microversion: 2.1 time: 0.591196 Sep 27 09:57:22.531623 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-04ea916c-9378-4cdc-bda2-21f0142f604c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1466 microversion: 2.1 time: 0.373735 Sep 27 09:57:22.531623 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 158/300] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:22 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1466 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:22.532033 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 143/301] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:21 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1319 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:22.539522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0194123-aefc-413d-8c3d-808bf4096851 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:22.540345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b0194123-aefc-413d-8c3d-808bf4096851 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:22.540959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a9e1851c-e890-4d60-b119-7fe8c64cf124 req-0651472c-0cea-4de8-817e-698283e5eb32 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ee40f956-0a40-43c4-a775-a978e8403d1d", "name": "network-vif-plugged", "status": "completed", "tag": "a921aca9-77a9-4abe-9b1a-34f56baca86d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:22.551842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a9e1851c-e890-4d60-b119-7fe8c64cf124 req-0651472c-0cea-4de8-817e-698283e5eb32 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:22.553322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a9e1851c-e890-4d60-b119-7fe8c64cf124 req-0651472c-0cea-4de8-817e-698283e5eb32 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:22.555019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a9e1851c-e890-4d60-b119-7fe8c64cf124 req-0651472c-0cea-4de8-817e-698283e5eb32 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:22.564433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0194123-aefc-413d-8c3d-808bf4096851 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:22.565291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0194123-aefc-413d-8c3d-808bf4096851 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:22.566095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0194123-aefc-413d-8c3d-808bf4096851 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:22.567363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0194123-aefc-413d-8c3d-808bf4096851 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:22.568039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0194123-aefc-413d-8c3d-808bf4096851 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:22.568696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0194123-aefc-413d-8c3d-808bf4096851 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:22.595848 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a9e1851c-e890-4d60-b119-7fe8c64cf124 req-0651472c-0cea-4de8-817e-698283e5eb32 service nova] Creating event network-vif-plugged:a921aca9-77a9-4abe-9b1a-34f56baca86d for instance ee40f956-0a40-43c4-a775-a978e8403d1d on nch2-master-1 Sep 27 09:57:22.610606 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a9e1851c-e890-4d60-b119-7fe8c64cf124 req-0651472c-0cea-4de8-817e-698283e5eb32 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.076907 Sep 27 09:57:22.611357 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 144/302] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:22.838446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b0194123-aefc-413d-8c3d-808bf4096851 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:22.849582 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0194123-aefc-413d-8c3d-808bf4096851 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.315054 Sep 27 09:57:22.851636 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 159/303] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:22 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:23.029051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2aab8e21-7d29-4b9a-9f30-416f785d45f4 tempest-AZAdminNegativeTestJSON-1426509792 tempest-AZAdminNegativeTestJSON-1426509792-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:23.030779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2aab8e21-7d29-4b9a-9f30-416f785d45f4 tempest-AZAdminNegativeTestJSON-1426509792 tempest-AZAdminNegativeTestJSON-1426509792-project] Policy check for os_compute_api:os-availability-zone:detail failed with credentials {'is_admin': False, 'user_id': 'f6dc529352674021ab4cc1e44e53c1dd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7b84f72741ed4c84953144b829216730', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2aab8e21-7d29-4b9a-9f30-416f785d45f4 tempest-AZAdminNegativeTestJSON-1426509792 tempest-AZAdminNegativeTestJSON-1426509792-project] Request method failure captured: Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-availability-zone/detail HTTP/1.1 Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMskAK7zmIjppGXiSCswatzHA8U3YhxKxaIy1LxFRuzOUNVo-PG0G5d3ldrQTY-YSaXvrU8GJtP9dSJjOhsIwpBjGK4a5fA0rvu3_HEY3K68CbyA5ncbGHIds4aT4lib6mme1fwomeia0G0GaeuwTYcvW9hJNYD-SeatnJz6bAgLXPIs Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 7b84f72741ed4c84953144b829216730 Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AZAdminNegativeTestJSON-1426509792 Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:57:23.033374 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7b84f72741ed4c84953144b829216730"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7b84f72741ed4c84953144b829216730"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7b84f72741ed4c84953144b829216730"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7b84f72741ed4c84953144b829216730"}]}] Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AZAdminNegativeTestJSON-1426509792 Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 7b84f72741ed4c84953144b829216730 Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AZAdminNegativeTestJSON-1426509792 Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AZAdminNegativeTestJSON-1426509792-project Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: f6dc529352674021ab4cc1e44e53c1dd Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AZAdminNegativeTestJSON-1426509792-project Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: exception: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. Sep 27 09:57:23.035059 ncdv-master-1 devstack@n-api.service[145968]: action_args: {} Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/availability_zone.py", line 117, in detail Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi context.can(az_policies.POLICY_ROOT % 'detail', target={}) Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:57:23.035867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:23.037641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2aab8e21-7d29-4b9a-9f30-416f785d45f4 tempest-AZAdminNegativeTestJSON-1426509792 tempest-AZAdminNegativeTestJSON-1426509792-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-availability-zone:detail to be performed. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:23.038723 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2aab8e21-7d29-4b9a-9f30-416f785d45f4 tempest-AZAdminNegativeTestJSON-1426509792 tempest-AZAdminNegativeTestJSON-1426509792-project] 10.222.0.26 "GET /compute/v2.1/os-availability-zone/detail" status: 403 len: 123 microversion: 2.1 time: 0.422122 Sep 27 09:57:23.038723 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 145/304] 10.222.0.26 () {58 vars in 1185 bytes} [Tue Sep 27 09:57:22 2022] GET /compute/v2.1/os-availability-zone/detail => generated 123 bytes in 423 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:23.041531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-98d57173-ea7d-4fac-afec-cedc995b6444 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:23.042255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-98d57173-ea7d-4fac-afec-cedc995b6444 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:23.049968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d57173-ea7d-4fac-afec-cedc995b6444 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:23.051503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d57173-ea7d-4fac-afec-cedc995b6444 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:23.052030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d57173-ea7d-4fac-afec-cedc995b6444 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:23.053288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d57173-ea7d-4fac-afec-cedc995b6444 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:23.053753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d57173-ea7d-4fac-afec-cedc995b6444 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:23.054179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d57173-ea7d-4fac-afec-cedc995b6444 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:23.229507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-98d57173-ea7d-4fac-afec-cedc995b6444 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:23.243019 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-98d57173-ea7d-4fac-afec-cedc995b6444 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1386 microversion: 2.1 time: 0.206591 Sep 27 09:57:23.243942 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 146/305] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:23 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1386 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:23.252382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33592bfa-292e-45fb-a00c-09822eeb8756 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:23.253310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33592bfa-292e-45fb-a00c-09822eeb8756 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:23.259975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33592bfa-292e-45fb-a00c-09822eeb8756 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:23.260542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33592bfa-292e-45fb-a00c-09822eeb8756 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:23.261370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33592bfa-292e-45fb-a00c-09822eeb8756 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:23.262094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33592bfa-292e-45fb-a00c-09822eeb8756 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:23.262603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33592bfa-292e-45fb-a00c-09822eeb8756 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:23.263390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33592bfa-292e-45fb-a00c-09822eeb8756 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:23.293771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-39fe4a30-5f57-42ac-9235-8e54a0f0e342 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:23.294737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-39fe4a30-5f57-42ac-9235-8e54a0f0e342 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:23.303351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39fe4a30-5f57-42ac-9235-8e54a0f0e342 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:23.304611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39fe4a30-5f57-42ac-9235-8e54a0f0e342 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:23.306181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39fe4a30-5f57-42ac-9235-8e54a0f0e342 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:23.307093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39fe4a30-5f57-42ac-9235-8e54a0f0e342 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:23.309395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39fe4a30-5f57-42ac-9235-8e54a0f0e342 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:23.310292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39fe4a30-5f57-42ac-9235-8e54a0f0e342 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:23.448332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33592bfa-292e-45fb-a00c-09822eeb8756 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:23.459414 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33592bfa-292e-45fb-a00c-09822eeb8756 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.213338 Sep 27 09:57:23.461773 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 147/306] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:23 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:23.545809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cdae3610-0a6a-41f5-8b4e-ee5bc4a71ccc tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:23.547085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cdae3610-0a6a-41f5-8b4e-ee5bc4a71ccc tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:23.559288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdae3610-0a6a-41f5-8b4e-ee5bc4a71ccc tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:23.562629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdae3610-0a6a-41f5-8b4e-ee5bc4a71ccc tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:23.564091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdae3610-0a6a-41f5-8b4e-ee5bc4a71ccc tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:23.565639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdae3610-0a6a-41f5-8b4e-ee5bc4a71ccc tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:23.567711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdae3610-0a6a-41f5-8b4e-ee5bc4a71ccc tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:23.568495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdae3610-0a6a-41f5-8b4e-ee5bc4a71ccc tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:23.663558 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-39fe4a30-5f57-42ac-9235-8e54a0f0e342 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1617 microversion: 2.1 time: 0.808392 Sep 27 09:57:23.664868 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 160/307] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:22 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1617 bytes in 811 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:23.709974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-29b133bc-9abe-4e94-a38b-15f43855da68 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Action: 'create', calling method: .version_select at 0x7f7f682cfee0>, body: {"server_group": {"name": "tempest-ServersOnMultiNodesTest-Server-Group-550489795", "policies": ["affinity"]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:23.750487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-29b133bc-9abe-4e94-a38b-15f43855da68 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:23.762819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-29b133bc-9abe-4e94-a38b-15f43855da68 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:23.850725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cdae3610-0a6a-41f5-8b4e-ee5bc4a71ccc tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:23.859927 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cdae3610-0a6a-41f5-8b4e-ee5bc4a71ccc tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1319 microversion: 2.1 time: 0.320766 Sep 27 09:57:23.860518 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 148/308] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:23 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1319 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:23.885009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0aea0ddd-71e7-40ad-aa23-f7b79835e204 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:23.887277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0aea0ddd-71e7-40ad-aa23-f7b79835e204 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:23.895377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-29b133bc-9abe-4e94-a38b-15f43855da68 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:23.905808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0aea0ddd-71e7-40ad-aa23-f7b79835e204 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:23.908133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0aea0ddd-71e7-40ad-aa23-f7b79835e204 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:23.909768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-29b133bc-9abe-4e94-a38b-15f43855da68 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:23.910712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0aea0ddd-71e7-40ad-aa23-f7b79835e204 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:23.913683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0aea0ddd-71e7-40ad-aa23-f7b79835e204 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:23.915464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0aea0ddd-71e7-40ad-aa23-f7b79835e204 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:23.917085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0aea0ddd-71e7-40ad-aa23-f7b79835e204 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:23.962775 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-29b133bc-9abe-4e94-a38b-15f43855da68 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "POST /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.258124 Sep 27 09:57:23.964577 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 161/309] 10.222.0.26 () {60 vars in 1174 bytes} [Tue Sep 27 09:57:23 2022] POST /compute/v2.1/os-server-groups => generated 187 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:24.198546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0aea0ddd-71e7-40ad-aa23-f7b79835e204 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:24.213869 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0aea0ddd-71e7-40ad-aa23-f7b79835e204 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1321 microversion: 2.1 time: 0.338813 Sep 27 09:57:24.231220 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 149/310] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:23 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1321 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:24.269872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fbbb93d6-4303-4e57-b8c3-e2254d583f1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:24.270921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fbbb93d6-4303-4e57-b8c3-e2254d583f1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:24.284337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbbb93d6-4303-4e57-b8c3-e2254d583f1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:24.285882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbbb93d6-4303-4e57-b8c3-e2254d583f1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:24.286562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbbb93d6-4303-4e57-b8c3-e2254d583f1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:24.288681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbbb93d6-4303-4e57-b8c3-e2254d583f1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:24.290251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbbb93d6-4303-4e57-b8c3-e2254d583f1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:24.291035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbbb93d6-4303-4e57-b8c3-e2254d583f1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:24.511724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fbbb93d6-4303-4e57-b8c3-e2254d583f1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:24.525992 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fbbb93d6-4303-4e57-b8c3-e2254d583f1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1586 microversion: 2.1 time: 0.261234 Sep 27 09:57:24.527321 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 162/311] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:24 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1586 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:24.534052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-1339576663", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "min_count": 2, "return_reservation_id": true}, "os:scheduler_hints": {"group": "ba6345ef-9480-4c71-b5a2-0c2e1d6550fb"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:24.553897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-1633958713", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:24.604364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:57:24.620396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:57:24.896010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:57:24.896429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:57:24.897546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:57:24.898143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:57:24.899126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:57:24.913933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:24.915122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:24.915867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:24.916710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:24.917161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:24.917590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:24.938355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for project e79eae6310354ca58046e59dfac23009. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:24.944222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for user 9544cdb0392f416197a586d99e8a3210 and project e79eae6310354ca58046e59dfac23009. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:24.960716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:57:24.963838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:57:24.990163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:57:24.990583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:57:24.991469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:57:24.992123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:57:24.992942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:57:25.039475 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3b4561a7-2967-4a53-bf42-f87a7587d0cf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.717277 Sep 27 09:57:25.041209 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 150/312] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:57:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 720 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 09:57:25.051647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:25.052134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:25.053156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:25.054867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:25.055598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:25.056280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:25.057242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4c1ec992-a1ba-4300-8b5e-149ab882791e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:25.059218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4c1ec992-a1ba-4300-8b5e-149ab882791e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:25.068180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c1ec992-a1ba-4300-8b5e-149ab882791e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:25.068701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c1ec992-a1ba-4300-8b5e-149ab882791e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:25.069139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c1ec992-a1ba-4300-8b5e-149ab882791e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:25.070454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c1ec992-a1ba-4300-8b5e-149ab882791e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:25.071381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c1ec992-a1ba-4300-8b5e-149ab882791e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:25.072433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c1ec992-a1ba-4300-8b5e-149ab882791e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:25.072925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=14 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:25.088286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:25.093801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:25.111113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Going to run 2 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:57:25.113796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 0e538bab-a9f3-488f-9bef-801f2429f73f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:57:25.183052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Checking whether user_id and queued_for_delete are populated for all projects {{(pid=145969) is_qfd_populated /opt/stack/nova/nova/quota.py:1268}} Sep 27 09:57:25.198135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:25.207469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:25.215449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:25.301709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4c1ec992-a1ba-4300-8b5e-149ab882791e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:25.313864 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4c1ec992-a1ba-4300-8b5e-149ab882791e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.271237 Sep 27 09:57:25.315008 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 151/313] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:25 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:25.334546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:25.336177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5226e363-4a15-463e-b0e1-b8754d170d36 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:25.337982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5226e363-4a15-463e-b0e1-b8754d170d36 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:25.345621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:25.352712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5226e363-4a15-463e-b0e1-b8754d170d36 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:25.355652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5226e363-4a15-463e-b0e1-b8754d170d36 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:25.356137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5226e363-4a15-463e-b0e1-b8754d170d36 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:25.356979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5226e363-4a15-463e-b0e1-b8754d170d36 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:25.357374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5226e363-4a15-463e-b0e1-b8754d170d36 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:25.357802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5226e363-4a15-463e-b0e1-b8754d170d36 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:25.372346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=0e538bab-a9f3-488f-9bef-801f2429f73f,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:57:25.519590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:25.523974 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5226e363-4a15-463e-b0e1-b8754d170d36 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.206971 Sep 27 09:57:25.528011 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 152/314] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 09:57:25 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde/os-interface => generated 285 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:25.543874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:25.544542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-53e6e413-1cc6-4b12-ae77-369cf14f0b46 req-d0715aa0-fe35-4443-80f1-3d0354c75331 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "52fadb34-4714-4e01-b7d1-db490dec438e", "name": "network-vif-plugged", "status": "completed", "tag": "98c20e80-4aac-40db-9819-c3a95358dbc2"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:25.607277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-53e6e413-1cc6-4b12-ae77-369cf14f0b46 req-d0715aa0-fe35-4443-80f1-3d0354c75331 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:25.611151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-53e6e413-1cc6-4b12-ae77-369cf14f0b46 req-d0715aa0-fe35-4443-80f1-3d0354c75331 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:25.613805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-53e6e413-1cc6-4b12-ae77-369cf14f0b46 req-d0715aa0-fe35-4443-80f1-3d0354c75331 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:25.665273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:25.676553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:25.682602 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-53e6e413-1cc6-4b12-ae77-369cf14f0b46 req-d0715aa0-fe35-4443-80f1-3d0354c75331 service nova] Creating event network-vif-plugged:98c20e80-4aac-40db-9819-c3a95358dbc2 for instance 52fadb34-4714-4e01-b7d1-db490dec438e on nch2-master-1 Sep 27 09:57:25.707992 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-53e6e413-1cc6-4b12-ae77-369cf14f0b46 req-d0715aa0-fe35-4443-80f1-3d0354c75331 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.174721 Sep 27 09:57:25.711850 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 153/315] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:25.739411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0ce33dfa-739e-4405-809b-809385bd9043 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:25.748943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0ce33dfa-739e-4405-809b-809385bd9043 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:25.782328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ce33dfa-739e-4405-809b-809385bd9043 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:25.786502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ce33dfa-739e-4405-809b-809385bd9043 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:25.791010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ce33dfa-739e-4405-809b-809385bd9043 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:25.791927 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c90cc7cd-970e-45d4-8d8f-ec652cb04510 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 32 microversion: 2.1 time: 1.263123 Sep 27 09:57:25.792999 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 163/316] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:57:24 2022] POST /compute/v2.1/servers => generated 32 bytes in 1265 msecs (HTTP/1.1 202) 9 headers in 362 bytes (1 switches on core 0) Sep 27 09:57:25.798031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ce33dfa-739e-4405-809b-809385bd9043 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:25.799399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ce33dfa-739e-4405-809b-809385bd9043 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:25.800990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ce33dfa-739e-4405-809b-809385bd9043 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:25.804729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-923ebdd9-328d-4277-8d70-f237f7e10ad1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:25.808255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-923ebdd9-328d-4277-8d70-f237f7e10ad1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:26.003317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-923ebdd9-328d-4277-8d70-f237f7e10ad1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:26.016859 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-923ebdd9-328d-4277-8d70-f237f7e10ad1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1213 microversion: 2.1 time: 0.222329 Sep 27 09:57:26.041929 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 164/317] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:25 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1213 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:26.072712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c06ee87b-245a-47eb-83a7-f3875b40e094 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:26.075370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c06ee87b-245a-47eb-83a7-f3875b40e094 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:26.092730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06ee87b-245a-47eb-83a7-f3875b40e094 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:26.094843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06ee87b-245a-47eb-83a7-f3875b40e094 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:26.096315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06ee87b-245a-47eb-83a7-f3875b40e094 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:26.098051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06ee87b-245a-47eb-83a7-f3875b40e094 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:26.099068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06ee87b-245a-47eb-83a7-f3875b40e094 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:26.099843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06ee87b-245a-47eb-83a7-f3875b40e094 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:26.534221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0ce33dfa-739e-4405-809b-809385bd9043 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:26.540313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c06ee87b-245a-47eb-83a7-f3875b40e094 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:26.551418 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0ce33dfa-739e-4405-809b-809385bd9043 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1319 microversion: 2.1 time: 0.828948 Sep 27 09:57:26.553341 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 154/318] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:25 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1319 bytes in 841 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:26.565160 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c06ee87b-245a-47eb-83a7-f3875b40e094 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1470 microversion: 2.1 time: 0.520704 Sep 27 09:57:26.569813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2568ef28-bd2b-4b90-be95-7cbc481b4bab tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:26.584566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2568ef28-bd2b-4b90-be95-7cbc481b4bab tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:26.585618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2568ef28-bd2b-4b90-be95-7cbc481b4bab tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Searching by: {'deleted': False, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 09:57:26.591194 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 165/319] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:26 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1470 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:26.677212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2568ef28-bd2b-4b90-be95-7cbc481b4bab tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:26.678452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2568ef28-bd2b-4b90-be95-7cbc481b4bab tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:26.681579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2568ef28-bd2b-4b90-be95-7cbc481b4bab tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:26.687297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2568ef28-bd2b-4b90-be95-7cbc481b4bab tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:26.688092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2568ef28-bd2b-4b90-be95-7cbc481b4bab tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:26.688835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2568ef28-bd2b-4b90-be95-7cbc481b4bab tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:26.755951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=12 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:26.824867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-2568ef28-bd2b-4b90-be95-7cbc481b4bab tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Listed batch of 1 results from cell out of 998 limit. Returned 1 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 09:57:26.970605 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2568ef28-bd2b-4b90-be95-7cbc481b4bab tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 1004 microversion: 2.1 time: 0.415237 Sep 27 09:57:26.976790 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 155/320] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 09:57:26 2022] GET /compute/v2.1/servers => generated 1004 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:26.989441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da38a1e6-1f80-47b7-8c6b-a514970fbbe8 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:26.992081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-da38a1e6-1f80-47b7-8c6b-a514970fbbe8 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:27.029534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da38a1e6-1f80-47b7-8c6b-a514970fbbe8 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:27.030744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da38a1e6-1f80-47b7-8c6b-a514970fbbe8 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.029s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:27.031725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da38a1e6-1f80-47b7-8c6b-a514970fbbe8 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:27.035890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da38a1e6-1f80-47b7-8c6b-a514970fbbe8 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:27.037601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da38a1e6-1f80-47b7-8c6b-a514970fbbe8 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:27.038270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da38a1e6-1f80-47b7-8c6b-a514970fbbe8 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:27.304557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5198897-b960-471a-a8b2-5681b8d1352d tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Action: 'create', calling method: >, body: {"flavor": {"name": "tempest-test_flavor-1293837965", "ram": 512, "vcpus": 1, "disk": 10, "id": 1607635059, "swap": 1024, "rxtx_factor": 1, "OS-FLV-EXT-DATA:ephemeral": 10}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:27.344639 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c5198897-b960-471a-a8b2-5681b8d1352d tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 416 microversion: 2.1 time: 0.751834 Sep 27 09:57:27.345674 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 166/321] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:57:26 2022] POST /compute/v2.1/flavors => generated 416 bytes in 754 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:27.748833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-94d07685-3927-4050-b537-93846b635715 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:27.749934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-94d07685-3927-4050-b537-93846b635715 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] [instance: 64bbf027-1c59-48d8-b5ab-99d7fcfc9e48] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:27.755960 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-94d07685-3927-4050-b537-93846b635715 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] HTTP exception thrown: Instance 64bbf027-1c59-48d8-b5ab-99d7fcfc9e48 could not be found. Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-94d07685-3927-4050-b537-93846b635715 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Request method failure captured: Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/64bbf027-1c59-48d8-b5ab-99d7fcfc9e48/action HTTP/1.1 Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 15 Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjmOS39Kb4EOG3AEPcCu2xO0uwyRar2-Cyy_T7I1JH7k-PcwJiU0fMYg8Ar4Mg97Sm4-Dg0c7iDeiwLr2h39DInGmz_mpOiq0MW-es6oWKiXfQe-up3SLSro4Gz4XmJgrNEpO_GHOtaEblKUnHqj-DDtujaiB5ib-gntLvjEFbsIlE Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 06e7347f7bbc45ce8de8e29de5255fa9 Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:57:27.757974 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_06e7347f7bbc45ce8de8e29de5255fa9"}]}] Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 06e7347f7bbc45ce8de8e29de5255fa9 Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersAdminNegativeTestJSON-622394730-project-admin Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: f1c3ed62097a4e89bf360a966c764762 Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersAdminNegativeTestJSON-622394730-project-admin Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: {"migrate": {}} Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:57:27.759962 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 64bbf027-1c59-48d8-b5ab-99d7fcfc9e48 could not be found. Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '64bbf027-1c59-48d8-b5ab-99d7fcfc9e48', 'body': {'migrate': {}}} Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 64bbf027-1c59-48d8-b5ab-99d7fcfc9e48 could not be found. Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:27.761069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 47, in _migrate Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id, Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 64bbf027-1c59-48d8-b5ab-99d7fcfc9e48 could not be found. Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 64bbf027-1c59-48d8-b5ab-99d7fcfc9e48 could not be found. Sep 27 09:57:27.765848 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:27.768677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-94d07685-3927-4050-b537-93846b635715 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Returning 404 to user: Instance 64bbf027-1c59-48d8-b5ab-99d7fcfc9e48 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:27.768677 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-94d07685-3927-4050-b537-93846b635715 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/64bbf027-1c59-48d8-b5ab-99d7fcfc9e48/action" status: 404 len: 111 microversion: 2.1 time: 0.412608 Sep 27 09:57:27.768677 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 167/322] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:57:27 2022] POST /compute/v2.1/servers/64bbf027-1c59-48d8-b5ab-99d7fcfc9e48/action => generated 111 bytes in 414 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:27.783588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5c1be487-b24f-474f-a1c6-00be7f6aacd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:27.784221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5c1be487-b24f-474f-a1c6-00be7f6aacd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:27.792354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c1be487-b24f-474f-a1c6-00be7f6aacd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:27.794173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c1be487-b24f-474f-a1c6-00be7f6aacd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:27.795007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c1be487-b24f-474f-a1c6-00be7f6aacd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:27.795789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c1be487-b24f-474f-a1c6-00be7f6aacd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:27.796469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c1be487-b24f-474f-a1c6-00be7f6aacd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:27.797133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c1be487-b24f-474f-a1c6-00be7f6aacd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:27.876976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-da38a1e6-1f80-47b7-8c6b-a514970fbbe8 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:27.890788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5c1be487-b24f-474f-a1c6-00be7f6aacd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:27.901237 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da38a1e6-1f80-47b7-8c6b-a514970fbbe8 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1323 microversion: 2.1 time: 0.921000 Sep 27 09:57:27.913057 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 156/323] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:26 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1323 bytes in 931 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:27.913291 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5c1be487-b24f-474f-a1c6-00be7f6aacd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1215 microversion: 2.1 time: 0.128543 Sep 27 09:57:27.918287 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 168/324] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:27 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1215 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:27.939537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-56c33e11-eeb7-4723-b46c-0154b3f8da3c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:27.946762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-56c33e11-eeb7-4723-b46c-0154b3f8da3c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:27.947343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4a868eae-15e9-48b9-a4e1-5935807f70a4 tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:28.000308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c33e11-eeb7-4723-b46c-0154b3f8da3c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:28.005368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c33e11-eeb7-4723-b46c-0154b3f8da3c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:28.008495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c33e11-eeb7-4723-b46c-0154b3f8da3c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:28.016295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c33e11-eeb7-4723-b46c-0154b3f8da3c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:28.019551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c33e11-eeb7-4723-b46c-0154b3f8da3c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:28.028211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c33e11-eeb7-4723-b46c-0154b3f8da3c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:28.044124 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4a868eae-15e9-48b9-a4e1-5935807f70a4 tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/1607635059/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.119102 Sep 27 09:57:28.050020 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 169/325] 10.222.0.26 () {60 vars in 1224 bytes} [Tue Sep 27 09:57:27 2022] POST /compute/v2.1/flavors/1607635059/os-extra_specs => generated 66 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:57:28.081326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-55a1e2bc-adfa-4b0a-8206-a9b9a6fb8fda tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:28.081326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-55a1e2bc-adfa-4b0a-8206-a9b9a6fb8fda tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:28.111148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55a1e2bc-adfa-4b0a-8206-a9b9a6fb8fda tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:28.112067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55a1e2bc-adfa-4b0a-8206-a9b9a6fb8fda tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:28.112912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55a1e2bc-adfa-4b0a-8206-a9b9a6fb8fda tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:28.114116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55a1e2bc-adfa-4b0a-8206-a9b9a6fb8fda tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:28.118631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55a1e2bc-adfa-4b0a-8206-a9b9a6fb8fda tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:28.120093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55a1e2bc-adfa-4b0a-8206-a9b9a6fb8fda tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:28.221491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-56c33e11-eeb7-4723-b46c-0154b3f8da3c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:28.240073 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-56c33e11-eeb7-4723-b46c-0154b3f8da3c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1271 microversion: 2.1 time: 0.332795 Sep 27 09:57:28.242193 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 157/326] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:27 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1271 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:28.319683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-55a1e2bc-adfa-4b0a-8206-a9b9a6fb8fda tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:28.340962 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-55a1e2bc-adfa-4b0a-8206-a9b9a6fb8fda tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1319 microversion: 2.1 time: 0.288780 Sep 27 09:57:28.351132 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 170/327] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:28 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1319 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:28.371573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminNegativeTestJSON-server-1338867260", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:28.463820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:57:28.695324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1ba1b6ad-965c-4d8f-9e86-dbf0e38c9bed tempest-FlavorsExtraSpecsTestJSON-1070565397 tempest-FlavorsExtraSpecsTestJSON-1070565397-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:28.717565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1ba1b6ad-965c-4d8f-9e86-dbf0e38c9bed tempest-FlavorsExtraSpecsTestJSON-1070565397 tempest-FlavorsExtraSpecsTestJSON-1070565397-project] 10.222.0.26 "GET /compute/v2.1/flavors/1607635059/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.473319 Sep 27 09:57:28.720366 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 158/328] 10.222.0.26 () {58 vars in 1203 bytes} [Tue Sep 27 09:57:28 2022] GET /compute/v2.1/flavors/1607635059/os-extra_specs => generated 66 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:57:28.740898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4fae94e7-8944-4546-b2a9-8f870be268af tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:28.843453 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4fae94e7-8944-4546-b2a9-8f870be268af tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/1607635059/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.113440 Sep 27 09:57:28.846536 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 159/329] 10.222.0.26 () {60 vars in 1224 bytes} [Tue Sep 27 09:57:28 2022] POST /compute/v2.1/flavors/1607635059/os-extra_specs => generated 66 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:57:28.858521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1e977fb4-0ed7-405a-b849-ec79504e90ec tempest-FlavorsExtraSpecsTestJSON-1070565397 tempest-FlavorsExtraSpecsTestJSON-1070565397-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:28.873917 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1e977fb4-0ed7-405a-b849-ec79504e90ec tempest-FlavorsExtraSpecsTestJSON-1070565397 tempest-FlavorsExtraSpecsTestJSON-1070565397-project] 10.222.0.26 "GET /compute/v2.1/flavors/1607635059/os-extra_specs/hw:numa_nodes" status: 200 len: 22 microversion: 2.1 time: 0.021116 Sep 27 09:57:28.874958 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 160/330] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 09:57:28 2022] GET /compute/v2.1/flavors/1607635059/os-extra_specs/hw:numa_nodes => generated 22 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 27 09:57:28.894847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4a3105cb-fc04-4e73-8f7d-39d70138b47c tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:numa_nodes": "1", "hw:cpu_policy": "shared"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:28.962915 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4a3105cb-fc04-4e73-8f7d-39d70138b47c tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/1607635059/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.080560 Sep 27 09:57:28.963781 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 161/331] 10.222.0.26 () {60 vars in 1224 bytes} [Tue Sep 27 09:57:28 2022] POST /compute/v2.1/flavors/1607635059/os-extra_specs => generated 66 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:57:28.969135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-37dc74c2-ef80-4288-af2d-d4e9686b9cde req-8666a806-9e58-40a2-b0cb-82871affa23d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2236fa92-0b7d-4029-a7fd-ffbb09df76bb", "name": "network-vif-plugged", "status": "completed", "tag": "bf3e8c0e-48d1-4bd7-ba5c-d02f03b4e2bf"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:28.975651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=3,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:28.983725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-37dc74c2-ef80-4288-af2d-d4e9686b9cde req-8666a806-9e58-40a2-b0cb-82871affa23d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:28.984629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-37dc74c2-ef80-4288-af2d-d4e9686b9cde req-8666a806-9e58-40a2-b0cb-82871affa23d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:28.985375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-37dc74c2-ef80-4288-af2d-d4e9686b9cde req-8666a806-9e58-40a2-b0cb-82871affa23d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:28.995453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:57:28.995880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:57:28.996530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:57:28.997119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:57:28.997840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:57:29.021614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:29.022395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:29.023940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:29.027761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:29.028430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:29.031928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:29.046799 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-37dc74c2-ef80-4288-af2d-d4e9686b9cde req-8666a806-9e58-40a2-b0cb-82871affa23d service nova] Creating event network-vif-plugged:bf3e8c0e-48d1-4bd7-ba5c-d02f03b4e2bf for instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb on nch1-master-1 Sep 27 09:57:29.072859 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-37dc74c2-ef80-4288-af2d-d4e9686b9cde req-8666a806-9e58-40a2-b0cb-82871affa23d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.107255 Sep 27 09:57:29.078925 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 162/332] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:29.092428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Getting quotas for project b33d41b411144dbbbdb012e694857d27. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:29.094773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e90541df-52fa-4568-af98-aa41539cb62e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:29.099109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e90541df-52fa-4568-af98-aa41539cb62e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:29.110276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Getting quotas for user d7d3aef1d4da47b2ab5454c3eb0346a6 and project b33d41b411144dbbbdb012e694857d27. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:29.116731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90541df-52fa-4568-af98-aa41539cb62e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:29.120024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90541df-52fa-4568-af98-aa41539cb62e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:29.120611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90541df-52fa-4568-af98-aa41539cb62e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:29.123700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90541df-52fa-4568-af98-aa41539cb62e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:29.124324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90541df-52fa-4568-af98-aa41539cb62e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:29.124902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90541df-52fa-4568-af98-aa41539cb62e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:29.126343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:57:29.128840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:57:29.189181 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7459f20c-40a8-438c-8bac-30a29c5a3d7f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.835343 Sep 27 09:57:29.192446 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 171/333] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:57:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 840 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:57:29.206751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-44b57aab-9d1f-46ee-a696-086c555e2d52 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:29.210218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-44b57aab-9d1f-46ee-a696-086c555e2d52 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:29.233725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44b57aab-9d1f-46ee-a696-086c555e2d52 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:29.234789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44b57aab-9d1f-46ee-a696-086c555e2d52 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:29.237976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44b57aab-9d1f-46ee-a696-086c555e2d52 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:29.239480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44b57aab-9d1f-46ee-a696-086c555e2d52 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:29.241998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44b57aab-9d1f-46ee-a696-086c555e2d52 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:29.244157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44b57aab-9d1f-46ee-a696-086c555e2d52 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:29.543048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-44b57aab-9d1f-46ee-a696-086c555e2d52 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:29.562147 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-44b57aab-9d1f-46ee-a696-086c555e2d52 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1271 microversion: 2.1 time: 0.365811 Sep 27 09:57:29.562736 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 172/334] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:29 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1271 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:29.576266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ed18c067-48e2-4ee9-826e-3a0656dc23ff tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:29.598282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e90541df-52fa-4568-af98-aa41539cb62e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:29.606703 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ed18c067-48e2-4ee9-826e-3a0656dc23ff tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/1607635059/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.034544 Sep 27 09:57:29.607429 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 173/335] 10.222.0.26 () {58 vars in 1203 bytes} [Tue Sep 27 09:57:29 2022] GET /compute/v2.1/flavors/1607635059/os-extra_specs => generated 66 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:57:29.613986 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e90541df-52fa-4568-af98-aa41539cb62e tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1472 microversion: 2.1 time: 0.531597 Sep 27 09:57:29.625327 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 163/336] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:29 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1472 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:29.635729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dc533ada-7b3f-4f11-b783-c462bb15538d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:29.636496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06811b24-82f2-48fe-8527-d9df982914ba tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:29.637126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-06811b24-82f2-48fe-8527-d9df982914ba tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:29.637303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dc533ada-7b3f-4f11-b783-c462bb15538d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:29.643348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06811b24-82f2-48fe-8527-d9df982914ba tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:29.643894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06811b24-82f2-48fe-8527-d9df982914ba tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:29.644377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06811b24-82f2-48fe-8527-d9df982914ba tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:29.645071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06811b24-82f2-48fe-8527-d9df982914ba tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:29.645508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06811b24-82f2-48fe-8527-d9df982914ba tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:29.645939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06811b24-82f2-48fe-8527-d9df982914ba tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:29.689387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=11,nova_api:SAVEPOINT=2,nova_api:RELEASE=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:29.870432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-06811b24-82f2-48fe-8527-d9df982914ba tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:29.905845 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-06811b24-82f2-48fe-8527-d9df982914ba tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1271 microversion: 2.1 time: 0.277016 Sep 27 09:57:29.912182 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 164/337] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:29 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1271 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:29.923920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9ec31362-921c-4397-a9b5-b1f667ef9aaf tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:29.924839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ec31362-921c-4397-a9b5-b1f667ef9aaf tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:29.932404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ec31362-921c-4397-a9b5-b1f667ef9aaf tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:29.933166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ec31362-921c-4397-a9b5-b1f667ef9aaf tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:29.933786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ec31362-921c-4397-a9b5-b1f667ef9aaf tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:29.934652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ec31362-921c-4397-a9b5-b1f667ef9aaf tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:29.935263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ec31362-921c-4397-a9b5-b1f667ef9aaf tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:29.935844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ec31362-921c-4397-a9b5-b1f667ef9aaf tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:29.938928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dc533ada-7b3f-4f11-b783-c462bb15538d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:29.950038 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dc533ada-7b3f-4f11-b783-c462bb15538d tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1222 microversion: 2.1 time: 0.339767 Sep 27 09:57:29.950809 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 174/338] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:29 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1222 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:30.140961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6c6dec71-e024-48f1-ad90-7648a0a14d60 tempest-FlavorMetadataValidation-67789048 tempest-FlavorMetadataValidation-67789048-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_validate_metadata_-650163955", "id": 440560962, "os-flavor-access:is_public": "True"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:30.171450 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6c6dec71-e024-48f1-ad90-7648a0a14d60 tempest-FlavorMetadataValidation-67789048 tempest-FlavorMetadataValidation-67789048-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 467 microversion: 2.86 time: 0.219048 Sep 27 09:57:30.172198 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 175/339] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 09:57:29 2022] POST /compute/v2.1/flavors => generated 467 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:57:30.179480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23a7ff5d-d915-47f9-b125-ebba462c1d37 tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Action: 'update', calling method: >, body: {"hw:numa_nodes": "2"} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:30.209437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ec31362-921c-4397-a9b5-b1f667ef9aaf tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:30.228111 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9ec31362-921c-4397-a9b5-b1f667ef9aaf tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1319 microversion: 2.1 time: 0.314121 Sep 27 09:57:30.250694 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 165/340] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:29 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1319 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:30.252953 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-23a7ff5d-d915-47f9-b125-ebba462c1d37 tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "PUT /compute/v2.1/flavors/1607635059/os-extra_specs/hw:numa_nodes" status: 200 len: 22 microversion: 2.1 time: 0.079122 Sep 27 09:57:30.255951 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 176/341] 10.222.0.26 () {60 vars in 1265 bytes} [Tue Sep 27 09:57:30 2022] PUT /compute/v2.1/flavors/1607635059/os-extra_specs/hw:numa_nodes => generated 22 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:57:30.259360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:30.260402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:30.268250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:30.269448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:30.270560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:30.271610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:30.272226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:30.272881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:30.276409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a82cd46-1ef3-450c-a262-0b486f31faca tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:30.280293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a82cd46-1ef3-450c-a262-0b486f31faca tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:30.301609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a82cd46-1ef3-450c-a262-0b486f31faca tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:30.302908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a82cd46-1ef3-450c-a262-0b486f31faca tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:30.308016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-server-diagnostics failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:30.308520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a82cd46-1ef3-450c-a262-0b486f31faca tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Request method failure captured: Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e/diagnostics HTTP/1.1 Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjmFv64Wa3AMo-UyFAoRx6dDfWP3HsJ0MLiqE89PuE04OPXLxxUGe3OGJL0CHWYPrdzVEw8acmb69Hhn_5M5BQhCtyf_2Ml8br1q0hWHyQ_a2WWp_m5m9n_mkpyO-VJzROa8t3tPEj9sK0Jlcv0ig9qhZuGwFy9nwGx-8ne9utbOdc Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: ec89ada1460d4275812339f2d6a93042 Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerDiagnosticsNegativeTest-328594429 Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:57:30.311111 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:57:30.312643 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec89ada1460d4275812339f2d6a93042"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec89ada1460d4275812339f2d6a93042"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ec89ada1460d4275812339f2d6a93042"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ec89ada1460d4275812339f2d6a93042"}]}] Sep 27 09:57:30.312643 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerDiagnosticsNegativeTest-328594429 Sep 27 09:57:30.312643 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: ec89ada1460d4275812339f2d6a93042 Sep 27 09:57:30.312643 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerDiagnosticsNegativeTest-328594429 Sep 27 09:57:30.312643 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerDiagnosticsNegativeTest-328594429-project Sep 27 09:57:30.312643 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:57:30.312643 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:57:30.312643 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b3dd80698e4c471abe7cd81809fa206c Sep 27 09:57:30.312643 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerDiagnosticsNegativeTest-328594429-project Sep 27 09:57:30.312643 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:57:30.312643 ncdv-master-1 devstack@n-api.service[145968]: exception: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'server_id': '52fadb34-4714-4e01-b7d1-db490dec438e'} Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_diagnostics.py", line 38, in index Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi context.can(sd_policies.BASE_POLICY_NAME, Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:57:30.313557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. Sep 27 09:57:30.314886 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:30.316266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a82cd46-1ef3-450c-a262-0b486f31faca tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:30.317598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-server-diagnostics to be performed. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:30.317786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a82cd46-1ef3-450c-a262-0b486f31faca tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:30.319012 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-acc2a06f-afb5-400b-ac7e-ac0247cf3b13 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e/diagnostics" status: 403 len: 117 microversion: 2.1 time: 0.066685 Sep 27 09:57:30.319445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a82cd46-1ef3-450c-a262-0b486f31faca tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:30.320131 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 166/342] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 09:57:30 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e/diagnostics => generated 117 bytes in 69 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:30.327624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-66ff8cad-3013-4f2a-8ee8-201bca5d3447 req-6c57456b-c8c9-43d7-a357-032bc26ccc17 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "030e5dcc-dc63-43e6-9254-ef26b66b5cde", "tag": "3dbbf26e-6eb1-444a-bd75-862612ffb4bc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:30.342835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-66ff8cad-3013-4f2a-8ee8-201bca5d3447 req-6c57456b-c8c9-43d7-a357-032bc26ccc17 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:30.343579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-66ff8cad-3013-4f2a-8ee8-201bca5d3447 req-6c57456b-c8c9-43d7-a357-032bc26ccc17 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:30.344149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-66ff8cad-3013-4f2a-8ee8-201bca5d3447 req-6c57456b-c8c9-43d7-a357-032bc26ccc17 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:30.363245 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-66ff8cad-3013-4f2a-8ee8-201bca5d3447 req-6c57456b-c8c9-43d7-a357-032bc26ccc17 service nova] Creating event network-changed:3dbbf26e-6eb1-444a-bd75-862612ffb4bc for instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde on nch2-master-1 Sep 27 09:57:30.370506 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-66ff8cad-3013-4f2a-8ee8-201bca5d3447 req-6c57456b-c8c9-43d7-a357-032bc26ccc17 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048424 Sep 27 09:57:30.372615 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 167/343] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:30.378905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c42e104-8651-409e-8a12-73b754175b24 tempest-FlavorMetadataValidation-67789048 tempest-FlavorMetadataValidation-67789048-project-admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw:cpu_policy": "shared", "foo:bar": "baz"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:30.643646 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c42e104-8651-409e-8a12-73b754175b24 tempest-FlavorMetadataValidation-67789048 tempest-FlavorMetadataValidation-67789048-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/440560962/os-extra_specs" status: 200 len: 62 microversion: 2.86 time: 0.268875 Sep 27 09:57:30.650230 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 168/344] 10.222.0.26 () {62 vars in 1262 bytes} [Tue Sep 27 09:57:30 2022] POST /compute/v2.1/flavors/440560962/os-extra_specs => generated 62 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:30.657062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c448b821-7a79-4e83-9ee0-820f80e167fc tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:30.665710 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c448b821-7a79-4e83-9ee0-820f80e167fc tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/1607635059/os-extra_specs" status: 200 len: 66 microversion: 2.1 time: 0.013690 Sep 27 09:57:30.670731 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 169/345] 10.222.0.26 () {58 vars in 1203 bytes} [Tue Sep 27 09:57:30 2022] GET /compute/v2.1/flavors/1607635059/os-extra_specs => generated 66 bytes in 16 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:57:30.672712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da65d404-6b36-4ef0-b26a-85f1c2485864 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:30.673362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-da65d404-6b36-4ef0-b26a-85f1c2485864 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:30.683315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da65d404-6b36-4ef0-b26a-85f1c2485864 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:30.683820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da65d404-6b36-4ef0-b26a-85f1c2485864 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:30.684307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da65d404-6b36-4ef0-b26a-85f1c2485864 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:30.685077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da65d404-6b36-4ef0-b26a-85f1c2485864 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:30.685532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da65d404-6b36-4ef0-b26a-85f1c2485864 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:30.685959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da65d404-6b36-4ef0-b26a-85f1c2485864 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:30.703872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:30.746075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-da65d404-6b36-4ef0-b26a-85f1c2485864 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:57:30.867071 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da65d404-6b36-4ef0-b26a-85f1c2485864 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "DELETE /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 204 len: 0 microversion: 2.1 time: 0.198792 Sep 27 09:57:30.868860 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 170/346] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:57:30 2022] DELETE /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:57:30.874860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-abaa59b9-21c8-471e-8dfd-126313f069b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:30.876386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-abaa59b9-21c8-471e-8dfd-126313f069b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:30.883429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abaa59b9-21c8-471e-8dfd-126313f069b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:30.884315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abaa59b9-21c8-471e-8dfd-126313f069b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:30.884939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abaa59b9-21c8-471e-8dfd-126313f069b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:30.886516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abaa59b9-21c8-471e-8dfd-126313f069b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:30.887416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abaa59b9-21c8-471e-8dfd-126313f069b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:30.888013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abaa59b9-21c8-471e-8dfd-126313f069b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:30.965113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-abaa59b9-21c8-471e-8dfd-126313f069b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:30.984892 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-abaa59b9-21c8-471e-8dfd-126313f069b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1277 microversion: 2.1 time: 0.114019 Sep 27 09:57:30.996707 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 171/347] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:30 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1277 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:31.036759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a1ce532d-9ed7-4633-8498-3c06d7a5fd05 tempest-FlavorMetadataValidation-67789048 tempest-FlavorMetadataValidation-67789048-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:31.065378 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a1ce532d-9ed7-4633-8498-3c06d7a5fd05 tempest-FlavorMetadataValidation-67789048 tempest-FlavorMetadataValidation-67789048-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/440560962" status: 202 len: 0 microversion: 2.86 time: 0.075167 Sep 27 09:57:31.066417 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 172/348] 10.222.0.26 () {60 vars in 1199 bytes} [Tue Sep 27 09:57:30 2022] DELETE /compute/v2.1/flavors/440560962 => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:57:31.078229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6a7e65f7-ea34-46b6-ad04-09275b682510 tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:31.098351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0a82cd46-1ef3-450c-a262-0b486f31faca tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:31.115957 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a82cd46-1ef3-450c-a262-0b486f31faca tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 200 len: 1586 microversion: 2.1 time: 0.854173 Sep 27 09:57:31.152992 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 177/349] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:30 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 1586 bytes in 896 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:31.156766 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6a7e65f7-ea34-46b6-ad04-09275b682510 tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1607635059/os-extra_specs/hw:numa_nodes" status: 200 len: 0 microversion: 2.1 time: 0.086002 Sep 27 09:57:31.158061 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 173/350] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 09:57:31 2022] DELETE /compute/v2.1/flavors/1607635059/os-extra_specs/hw:numa_nodes => generated 0 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:57:31.161671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-786dde8d-f777-4f7b-92bc-468b5ea565d6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:31.163343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-786dde8d-f777-4f7b-92bc-468b5ea565d6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:31.177382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-79aae4cb-faa5-4dc9-ac2d-5be297a9da94 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:31.181938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-79aae4cb-faa5-4dc9-ac2d-5be297a9da94 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:31.185026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-786dde8d-f777-4f7b-92bc-468b5ea565d6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:31.189853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-786dde8d-f777-4f7b-92bc-468b5ea565d6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:31.190654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-786dde8d-f777-4f7b-92bc-468b5ea565d6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:31.191598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-786dde8d-f777-4f7b-92bc-468b5ea565d6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:31.192274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-786dde8d-f777-4f7b-92bc-468b5ea565d6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:31.201446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-786dde8d-f777-4f7b-92bc-468b5ea565d6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:31.203674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79aae4cb-faa5-4dc9-ac2d-5be297a9da94 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:31.204799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79aae4cb-faa5-4dc9-ac2d-5be297a9da94 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:31.207537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79aae4cb-faa5-4dc9-ac2d-5be297a9da94 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:31.209940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79aae4cb-faa5-4dc9-ac2d-5be297a9da94 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:31.209940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79aae4cb-faa5-4dc9-ac2d-5be297a9da94 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:31.210367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79aae4cb-faa5-4dc9-ac2d-5be297a9da94 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:31.574480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-79aae4cb-faa5-4dc9-ac2d-5be297a9da94 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:31.580463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-786dde8d-f777-4f7b-92bc-468b5ea565d6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:31.582959 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-79aae4cb-faa5-4dc9-ac2d-5be297a9da94 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1313 microversion: 2.1 time: 0.422695 Sep 27 09:57:31.605241 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-786dde8d-f777-4f7b-92bc-468b5ea565d6 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1478 microversion: 2.1 time: 0.450470 Sep 27 09:57:31.617398 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 174/351] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:31 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1313 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:31.617398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da302c02-c40e-4ffc-a82c-475d9223f319 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:31.618393 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 178/352] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:31 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1478 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:31.625040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-da302c02-c40e-4ffc-a82c-475d9223f319 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:31.628496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-17659a4a-d2e6-4139-ae21-e50cc702342f tempest-FlavorMetadataValidation-67789048 tempest-FlavorMetadataValidation-67789048-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:31.632034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da302c02-c40e-4ffc-a82c-475d9223f319 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:31.632533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da302c02-c40e-4ffc-a82c-475d9223f319 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:31.633079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da302c02-c40e-4ffc-a82c-475d9223f319 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:31.633820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da302c02-c40e-4ffc-a82c-475d9223f319 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:31.634278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da302c02-c40e-4ffc-a82c-475d9223f319 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:31.634706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da302c02-c40e-4ffc-a82c-475d9223f319 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:31.687421 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-17659a4a-d2e6-4139-ae21-e50cc702342f tempest-FlavorMetadataValidation-67789048 tempest-FlavorMetadataValidation-67789048-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 7149 microversion: 2.86 time: 0.067451 Sep 27 09:57:31.688271 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 179/353] 10.222.0.26 () {60 vars in 1187 bytes} [Tue Sep 27 09:57:31 2022] GET /compute/v2.1/flavors/detail => generated 7149 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:31.707585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b45e3a8a-a132-4f76-b2da-11dc0bc16ff0 tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:31.749520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-da302c02-c40e-4ffc-a82c-475d9223f319 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:31.760921 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b45e3a8a-a132-4f76-b2da-11dc0bc16ff0 tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1607635059/os-extra_specs/hw:cpu_policy" status: 200 len: 0 microversion: 2.1 time: 0.068592 Sep 27 09:57:31.762310 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 180/354] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 09:57:31 2022] DELETE /compute/v2.1/flavors/1607635059/os-extra_specs/hw:cpu_policy => generated 0 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 355 bytes (1 switches on core 0) Sep 27 09:57:31.767161 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da302c02-c40e-4ffc-a82c-475d9223f319 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1282 microversion: 2.1 time: 0.148548 Sep 27 09:57:31.779490 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 175/355] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:31 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1282 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:31.779490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1861371e-f213-4408-b9c6-4d60d89a0815 tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:31.779782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-098881b9-a6ed-4dbe-9380-0c22a60ff469 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:31.779782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-098881b9-a6ed-4dbe-9380-0c22a60ff469 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:31.783854 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1861371e-f213-4408-b9c6-4d60d89a0815 tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/1607635059/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.014252 Sep 27 09:57:31.792424 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 176/356] 10.222.0.26 () {58 vars in 1203 bytes} [Tue Sep 27 09:57:31 2022] GET /compute/v2.1/flavors/1607635059/os-extra_specs => generated 19 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:57:31.794698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-098881b9-a6ed-4dbe-9380-0c22a60ff469 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:31.795084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-098881b9-a6ed-4dbe-9380-0c22a60ff469 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:31.795689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-098881b9-a6ed-4dbe-9380-0c22a60ff469 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:31.796574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-098881b9-a6ed-4dbe-9380-0c22a60ff469 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:31.797219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-098881b9-a6ed-4dbe-9380-0c22a60ff469 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:31.797788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-098881b9-a6ed-4dbe-9380-0c22a60ff469 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:31.810380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-773e78a5-2c7d-4b6e-983e-2b8c099b797f tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:31.859142 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-773e78a5-2c7d-4b6e-983e-2b8c099b797f tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1607635059" status: 202 len: 0 microversion: 2.1 time: 0.055130 Sep 27 09:57:31.859972 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 177/357] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:57:31 2022] DELETE /compute/v2.1/flavors/1607635059 => generated 0 bytes in 57 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:57:31.868194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-04f72c20-f3bc-4912-9e5c-8a723ebeddbe tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:31.914568 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-04f72c20-f3bc-4912-9e5c-8a723ebeddbe tempest-FlavorsExtraSpecsTestJSON-594579390 tempest-FlavorsExtraSpecsTestJSON-594579390-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.049951 Sep 27 09:57:31.915679 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 178/358] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:57:31 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:32.029506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-98fb3021-85aa-4abf-8296-ab47597a5a77 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:32.030839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-98fb3021-85aa-4abf-8296-ab47597a5a77 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:32.039651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98fb3021-85aa-4abf-8296-ab47597a5a77 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:32.040116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98fb3021-85aa-4abf-8296-ab47597a5a77 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:32.040552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98fb3021-85aa-4abf-8296-ab47597a5a77 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:32.049618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98fb3021-85aa-4abf-8296-ab47597a5a77 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:32.050264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98fb3021-85aa-4abf-8296-ab47597a5a77 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:32.050869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98fb3021-85aa-4abf-8296-ab47597a5a77 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:32.326864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-98fb3021-85aa-4abf-8296-ab47597a5a77 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:32.341104 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-98fb3021-85aa-4abf-8296-ab47597a5a77 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1277 microversion: 2.1 time: 0.315632 Sep 27 09:57:32.350200 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 179/359] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:32 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1277 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:32.450168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-098881b9-a6ed-4dbe-9380-0c22a60ff469 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:32.465245 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-098881b9-a6ed-4dbe-9380-0c22a60ff469 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1468 microversion: 2.1 time: 0.699896 Sep 27 09:57:32.466202 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 181/360] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:31 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1468 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:32.613447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e9bb8cd2-45e3-42e3-bc96-02d2749a2d70 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:32.616035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e9bb8cd2-45e3-42e3-bc96-02d2749a2d70 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:32.674271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9bb8cd2-45e3-42e3-bc96-02d2749a2d70 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:32.676396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9bb8cd2-45e3-42e3-bc96-02d2749a2d70 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:32.678909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9bb8cd2-45e3-42e3-bc96-02d2749a2d70 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:32.682332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9bb8cd2-45e3-42e3-bc96-02d2749a2d70 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:32.683667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9bb8cd2-45e3-42e3-bc96-02d2749a2d70 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:32.684426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9bb8cd2-45e3-42e3-bc96-02d2749a2d70 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:32.718555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:32.882151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9ea683af-687f-4dcb-b5f8-9b688913cf7e tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:32.894080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ea683af-687f-4dcb-b5f8-9b688913cf7e tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:32.916357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ea683af-687f-4dcb-b5f8-9b688913cf7e tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:32.924246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ea683af-687f-4dcb-b5f8-9b688913cf7e tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:32.925452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ea683af-687f-4dcb-b5f8-9b688913cf7e tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:32.927037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ea683af-687f-4dcb-b5f8-9b688913cf7e tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:32.928608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ea683af-687f-4dcb-b5f8-9b688913cf7e tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:32.935434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ea683af-687f-4dcb-b5f8-9b688913cf7e tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:33.038415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ea683af-687f-4dcb-b5f8-9b688913cf7e tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:57:33.198560 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9ea683af-687f-4dcb-b5f8-9b688913cf7e tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] 10.222.0.26 "DELETE /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 204 len: 0 microversion: 2.1 time: 0.710261 Sep 27 09:57:33.199191 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 180/361] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:57:32 2022] DELETE /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 0 bytes in 712 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:57:33.207505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7dbdf1d6-6972-4378-aa90-e43554a788b4 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:33.208240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7dbdf1d6-6972-4378-aa90-e43554a788b4 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:33.219065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dbdf1d6-6972-4378-aa90-e43554a788b4 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:33.219739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dbdf1d6-6972-4378-aa90-e43554a788b4 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:33.220691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dbdf1d6-6972-4378-aa90-e43554a788b4 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:33.221970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dbdf1d6-6972-4378-aa90-e43554a788b4 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:33.222687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dbdf1d6-6972-4378-aa90-e43554a788b4 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:33.223322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dbdf1d6-6972-4378-aa90-e43554a788b4 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:33.622056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7dbdf1d6-6972-4378-aa90-e43554a788b4 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:33.631616 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7dbdf1d6-6972-4378-aa90-e43554a788b4 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1478 microversion: 2.1 time: 0.430829 Sep 27 09:57:33.632991 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 181/362] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:33 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1478 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:33.638596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e5e898dc-20f4-4b14-bc30-edfcb81b82f6 req-65e330d8-6527-4c95-88f4-c4d08bf01b16 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "706e3141-43af-4a6b-8789-dea152d0312a", "tag": "ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:33.653577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e5e898dc-20f4-4b14-bc30-edfcb81b82f6 req-65e330d8-6527-4c95-88f4-c4d08bf01b16 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:33.654108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e5e898dc-20f4-4b14-bc30-edfcb81b82f6 req-65e330d8-6527-4c95-88f4-c4d08bf01b16 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:33.654504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e5e898dc-20f4-4b14-bc30-edfcb81b82f6 req-65e330d8-6527-4c95-88f4-c4d08bf01b16 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:33.668440 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e5e898dc-20f4-4b14-bc30-edfcb81b82f6 req-65e330d8-6527-4c95-88f4-c4d08bf01b16 service nova] Creating event network-changed:ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4 for instance 706e3141-43af-4a6b-8789-dea152d0312a on nch1-master-1 Sep 27 09:57:33.673957 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e5e898dc-20f4-4b14-bc30-edfcb81b82f6 req-65e330d8-6527-4c95-88f4-c4d08bf01b16 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038838 Sep 27 09:57:33.675432 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 182/363] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:33.680705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9be8347-038e-4d7c-975b-822f3c7eb1fb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:33.682207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c9be8347-038e-4d7c-975b-822f3c7eb1fb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:33.689511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be8347-038e-4d7c-975b-822f3c7eb1fb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:33.689511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be8347-038e-4d7c-975b-822f3c7eb1fb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:33.689799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be8347-038e-4d7c-975b-822f3c7eb1fb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:33.690525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be8347-038e-4d7c-975b-822f3c7eb1fb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:33.690987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be8347-038e-4d7c-975b-822f3c7eb1fb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:33.691491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be8347-038e-4d7c-975b-822f3c7eb1fb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:33.710881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e9bb8cd2-45e3-42e3-bc96-02d2749a2d70 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:33.742993 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e9bb8cd2-45e3-42e3-bc96-02d2749a2d70 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1313 microversion: 2.1 time: 1.135923 Sep 27 09:57:33.744878 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 182/364] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:32 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1313 bytes in 1139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:33.764439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5111981c-10ed-46c7-afbc-df1a7241fa3c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:33.765462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5111981c-10ed-46c7-afbc-df1a7241fa3c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:33.774456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5111981c-10ed-46c7-afbc-df1a7241fa3c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:33.775627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5111981c-10ed-46c7-afbc-df1a7241fa3c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:33.776236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5111981c-10ed-46c7-afbc-df1a7241fa3c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:33.778870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5111981c-10ed-46c7-afbc-df1a7241fa3c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:33.781171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5111981c-10ed-46c7-afbc-df1a7241fa3c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:33.782384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5111981c-10ed-46c7-afbc-df1a7241fa3c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:33.819205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9be8347-038e-4d7c-975b-822f3c7eb1fb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:33.835627 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9be8347-038e-4d7c-975b-822f3c7eb1fb tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1280 microversion: 2.1 time: 0.158825 Sep 27 09:57:33.835627 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 183/365] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:33 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1280 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:33.842345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cffa6a6d-7b07-4991-ad57-49a853ee6cbb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:33.842957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cffa6a6d-7b07-4991-ad57-49a853ee6cbb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:33.854899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffa6a6d-7b07-4991-ad57-49a853ee6cbb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:33.855405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffa6a6d-7b07-4991-ad57-49a853ee6cbb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:33.856101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffa6a6d-7b07-4991-ad57-49a853ee6cbb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:33.859178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffa6a6d-7b07-4991-ad57-49a853ee6cbb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:33.859683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffa6a6d-7b07-4991-ad57-49a853ee6cbb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:33.860120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffa6a6d-7b07-4991-ad57-49a853ee6cbb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:34.193990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cffa6a6d-7b07-4991-ad57-49a853ee6cbb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:34.207791 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cffa6a6d-7b07-4991-ad57-49a853ee6cbb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1319 microversion: 2.1 time: 0.370948 Sep 27 09:57:34.208850 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 184/366] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:33 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1319 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:34.530729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5111981c-10ed-46c7-afbc-df1a7241fa3c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:34.545135 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5111981c-10ed-46c7-afbc-df1a7241fa3c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1474 microversion: 2.1 time: 0.797729 Sep 27 09:57:34.546370 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 183/367] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:33 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1474 bytes in 801 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:34.659920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2804b4c7-f11a-410b-9ba6-a362362f0b8a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:34.661081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2804b4c7-f11a-410b-9ba6-a362362f0b8a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:34.676136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2804b4c7-f11a-410b-9ba6-a362362f0b8a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:34.677896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2804b4c7-f11a-410b-9ba6-a362362f0b8a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:34.679039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2804b4c7-f11a-410b-9ba6-a362362f0b8a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:34.681915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2804b4c7-f11a-410b-9ba6-a362362f0b8a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:34.683131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2804b4c7-f11a-410b-9ba6-a362362f0b8a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:34.684230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2804b4c7-f11a-410b-9ba6-a362362f0b8a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:34.776368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-52882a96-4b2d-495c-a02e-395a926a5d0b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:34.782858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-52882a96-4b2d-495c-a02e-395a926a5d0b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:34.798794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52882a96-4b2d-495c-a02e-395a926a5d0b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:34.799662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52882a96-4b2d-495c-a02e-395a926a5d0b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:34.800424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52882a96-4b2d-495c-a02e-395a926a5d0b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:34.812700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52882a96-4b2d-495c-a02e-395a926a5d0b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:34.814031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52882a96-4b2d-495c-a02e-395a926a5d0b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:34.817649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52882a96-4b2d-495c-a02e-395a926a5d0b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:34.955490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:35.223974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2804b4c7-f11a-410b-9ba6-a362362f0b8a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:35.235080 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2804b4c7-f11a-410b-9ba6-a362362f0b8a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1436 microversion: 2.1 time: 0.582593 Sep 27 09:57:35.235796 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 185/368] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:34 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1436 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:35.243999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-12f27d08-bccf-4c7c-a49a-cebc64c7de1f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:35.244920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12f27d08-bccf-4c7c-a49a-cebc64c7de1f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:35.274756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12f27d08-bccf-4c7c-a49a-cebc64c7de1f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:35.275265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12f27d08-bccf-4c7c-a49a-cebc64c7de1f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:35.275703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12f27d08-bccf-4c7c-a49a-cebc64c7de1f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:35.276589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12f27d08-bccf-4c7c-a49a-cebc64c7de1f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:35.279332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12f27d08-bccf-4c7c-a49a-cebc64c7de1f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:35.279990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12f27d08-bccf-4c7c-a49a-cebc64c7de1f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:35.360956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-52882a96-4b2d-495c-a02e-395a926a5d0b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:35.385861 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-52882a96-4b2d-495c-a02e-395a926a5d0b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1313 microversion: 2.1 time: 0.618675 Sep 27 09:57:35.402822 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 184/369] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:34 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1313 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:35.413581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ace4be50-1f15-47ef-b941-6d31c509a649 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:35.414740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ace4be50-1f15-47ef-b941-6d31c509a649 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:35.423220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace4be50-1f15-47ef-b941-6d31c509a649 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:35.425092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace4be50-1f15-47ef-b941-6d31c509a649 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:35.426067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace4be50-1f15-47ef-b941-6d31c509a649 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:35.427456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace4be50-1f15-47ef-b941-6d31c509a649 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:35.429147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace4be50-1f15-47ef-b941-6d31c509a649 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:35.430104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace4be50-1f15-47ef-b941-6d31c509a649 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:35.844126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12f27d08-bccf-4c7c-a49a-cebc64c7de1f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:35.884010 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-12f27d08-bccf-4c7c-a49a-cebc64c7de1f tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1322 microversion: 2.1 time: 0.645225 Sep 27 09:57:35.900701 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 186/370] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:35 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1322 bytes in 655 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:35.921705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6bb2d58c-cf8a-458a-b5d1-8134aff56159 req-3e1d0a40-646d-4908-b7b5-4d2a270decf2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0e538bab-a9f3-488f-9bef-801f2429f73f", "tag": "9ff31270-663b-4d9c-a7e1-90a7d2b0c7fb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:35.943649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6bb2d58c-cf8a-458a-b5d1-8134aff56159 req-3e1d0a40-646d-4908-b7b5-4d2a270decf2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:35.944259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6bb2d58c-cf8a-458a-b5d1-8134aff56159 req-3e1d0a40-646d-4908-b7b5-4d2a270decf2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:35.944806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6bb2d58c-cf8a-458a-b5d1-8134aff56159 req-3e1d0a40-646d-4908-b7b5-4d2a270decf2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:35.971497 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6bb2d58c-cf8a-458a-b5d1-8134aff56159 req-3e1d0a40-646d-4908-b7b5-4d2a270decf2 service nova] Creating event network-changed:9ff31270-663b-4d9c-a7e1-90a7d2b0c7fb for instance 0e538bab-a9f3-488f-9bef-801f2429f73f on nch1-master-1 Sep 27 09:57:35.983581 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6bb2d58c-cf8a-458a-b5d1-8134aff56159 req-3e1d0a40-646d-4908-b7b5-4d2a270decf2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087306 Sep 27 09:57:35.984209 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 187/371] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:36.004102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d9cc8746-b9e0-4f9c-ab1f-8b756abb63f2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:36.005556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d9cc8746-b9e0-4f9c-ab1f-8b756abb63f2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:36.016359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9cc8746-b9e0-4f9c-ab1f-8b756abb63f2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:36.017045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9cc8746-b9e0-4f9c-ab1f-8b756abb63f2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:36.017734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9cc8746-b9e0-4f9c-ab1f-8b756abb63f2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:36.018841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9cc8746-b9e0-4f9c-ab1f-8b756abb63f2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:36.020496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9cc8746-b9e0-4f9c-ab1f-8b756abb63f2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:36.023090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9cc8746-b9e0-4f9c-ab1f-8b756abb63f2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:36.359287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ace4be50-1f15-47ef-b941-6d31c509a649 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:36.381897 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ace4be50-1f15-47ef-b941-6d31c509a649 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1319 microversion: 2.1 time: 0.973929 Sep 27 09:57:36.383987 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 185/372] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:35 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1319 bytes in 980 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:36.395585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:36.396682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:36.405860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:36.413066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:36.416600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:36.417597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:36.418152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:36.418501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:36.462277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:36.463223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:36.464094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:36.464768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lazy-loading 'fault' on Instance uuid 52fadb34-4714-4e01-b7d1-db490dec438e {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:57:36.495552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d9cc8746-b9e0-4f9c-ab1f-8b756abb63f2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:36.514369 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d9cc8746-b9e0-4f9c-ab1f-8b756abb63f2 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1474 microversion: 2.1 time: 0.522922 Sep 27 09:57:36.514965 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 188/373] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:35 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1474 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:36.519534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a93c3e2-06f9-4f56-a707-bb25f577e83e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:36.520137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a93c3e2-06f9-4f56-a707-bb25f577e83e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:36.538106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a93c3e2-06f9-4f56-a707-bb25f577e83e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:36.538782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a93c3e2-06f9-4f56-a707-bb25f577e83e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:36.539601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a93c3e2-06f9-4f56-a707-bb25f577e83e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:36.540465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a93c3e2-06f9-4f56-a707-bb25f577e83e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:36.541090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a93c3e2-06f9-4f56-a707-bb25f577e83e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:36.541653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a93c3e2-06f9-4f56-a707-bb25f577e83e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:36.804535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:36.817909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b3dd80698e4c471abe7cd81809fa206c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec89ada1460d4275812339f2d6a93042', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:36.830608 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e64d317a-0784-43fc-8cd2-7f4d077be859 tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 200 len: 1313 microversion: 2.1 time: 0.444189 Sep 27 09:57:36.839582 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 186/374] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:36 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 1313 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:36.845494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-478983bc-41d2-4e3b-9ed7-8c3608cf2fb4 req-e3757dc5-5c21-4184-8293-276de13163bb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2236fa92-0b7d-4029-a7fd-ffbb09df76bb", "name": "network-vif-deleted", "tag": "bf3e8c0e-48d1-4bd7-ba5c-d02f03b4e2bf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:36.872795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-478983bc-41d2-4e3b-9ed7-8c3608cf2fb4 req-e3757dc5-5c21-4184-8293-276de13163bb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:36.879151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-478983bc-41d2-4e3b-9ed7-8c3608cf2fb4 req-e3757dc5-5c21-4184-8293-276de13163bb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:36.880138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-478983bc-41d2-4e3b-9ed7-8c3608cf2fb4 req-e3757dc5-5c21-4184-8293-276de13163bb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:36.907875 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-478983bc-41d2-4e3b-9ed7-8c3608cf2fb4 req-e3757dc5-5c21-4184-8293-276de13163bb service nova] Creating event network-vif-deleted:bf3e8c0e-48d1-4bd7-ba5c-d02f03b4e2bf for instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb on nch1-master-1 Sep 27 09:57:36.936183 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-478983bc-41d2-4e3b-9ed7-8c3608cf2fb4 req-e3757dc5-5c21-4184-8293-276de13163bb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.092803 Sep 27 09:57:36.939790 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 187/375] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:36.981368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6f69fa36-dfa5-44aa-90fe-ff3154bf3ec7 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:36.987378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6f69fa36-dfa5-44aa-90fe-ff3154bf3ec7 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:37.011332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f69fa36-dfa5-44aa-90fe-ff3154bf3ec7 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:37.026448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f69fa36-dfa5-44aa-90fe-ff3154bf3ec7 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:37.028241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f69fa36-dfa5-44aa-90fe-ff3154bf3ec7 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:37.037026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f69fa36-dfa5-44aa-90fe-ff3154bf3ec7 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:37.042399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f69fa36-dfa5-44aa-90fe-ff3154bf3ec7 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:37.045599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f69fa36-dfa5-44aa-90fe-ff3154bf3ec7 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:37.120413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8a93c3e2-06f9-4f56-a707-bb25f577e83e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:37.123062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:37.135079 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a93c3e2-06f9-4f56-a707-bb25f577e83e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1313 microversion: 2.1 time: 0.619004 Sep 27 09:57:37.141309 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 189/376] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:36 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1313 bytes in 626 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:37.418212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a8cd1ea4-4350-4bba-b9af-a83f7aa97197 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:37.419346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a8cd1ea4-4350-4bba-b9af-a83f7aa97197 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:37.461555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8cd1ea4-4350-4bba-b9af-a83f7aa97197 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:37.462376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8cd1ea4-4350-4bba-b9af-a83f7aa97197 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:37.463100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8cd1ea4-4350-4bba-b9af-a83f7aa97197 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:37.464494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8cd1ea4-4350-4bba-b9af-a83f7aa97197 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:37.465268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8cd1ea4-4350-4bba-b9af-a83f7aa97197 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:37.466262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8cd1ea4-4350-4bba-b9af-a83f7aa97197 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:37.724453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6f69fa36-dfa5-44aa-90fe-ff3154bf3ec7 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:37.734256 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6f69fa36-dfa5-44aa-90fe-ff3154bf3ec7 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1322 microversion: 2.1 time: 0.783720 Sep 27 09:57:37.734870 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 188/377] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:36 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1322 bytes in 794 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:37.739775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:37.740693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:37.750420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:37.751024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:37.751603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:37.752558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:37.753870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:37.753870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:37.822385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:37.822886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:37.823320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:37.823788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lazy-loading 'fault' on Instance uuid 2236fa92-0b7d-4029-a7fd-ffbb09df76bb {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:57:37.885919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:37.898833 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-db5d0f6a-7196-4a7b-8d51-1c000db6814d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 200 len: 1309 microversion: 2.1 time: 0.162140 Sep 27 09:57:37.900842 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 189/378] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:37 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 1309 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:37.906333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:37.907327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] [instance: 52fadb34-4714-4e01-b7d1-db490dec438e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:37.920790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:37.921411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:37.921988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:37.922747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:37.923264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:37.923681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:37.942237 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] HTTP exception thrown: Instance 52fadb34-4714-4e01-b7d1-db490dec438e could not be found. Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Request method failure captured: Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e HTTP/1.1 Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjmFv64Wa3AMo-UyFAoRx6dDfWP3HsJ0MLiqE89PuE04OPXLxxUGe3OGJL0CHWYPrdzVEw8acmb69Hhn_5M5BQhCtyf_2Ml8br1q0hWHyQ_a2WWp_m5m9n_mkpyO-VJzROa8t3tPEj9sK0Jlcv0ig9qhZuGwFy9nwGx-8ne9utbOdc Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: ec89ada1460d4275812339f2d6a93042 Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerDiagnosticsNegativeTest-328594429 Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:57:37.944261 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec89ada1460d4275812339f2d6a93042"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec89ada1460d4275812339f2d6a93042"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ec89ada1460d4275812339f2d6a93042"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ec89ada1460d4275812339f2d6a93042"}]}] Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerDiagnosticsNegativeTest-328594429 Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: ec89ada1460d4275812339f2d6a93042 Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerDiagnosticsNegativeTest-328594429 Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerDiagnosticsNegativeTest-328594429-project Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b3dd80698e4c471abe7cd81809fa206c Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerDiagnosticsNegativeTest-328594429-project Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 52fadb34-4714-4e01-b7d1-db490dec438e could not be found. Sep 27 09:57:37.945955 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '52fadb34-4714-4e01-b7d1-db490dec438e'} Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 52fadb34-4714-4e01-b7d1-db490dec438e could not be found. Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:37.947116 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 52fadb34-4714-4e01-b7d1-db490dec438e could not be found. Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 52fadb34-4714-4e01-b7d1-db490dec438e could not be found. Sep 27 09:57:37.948491 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:37.949699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] Returning 404 to user: Instance 52fadb34-4714-4e01-b7d1-db490dec438e could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:37.949699 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4f2f2336-63fe-49fa-9ec9-cc20bd0bdb0a tempest-ServerDiagnosticsNegativeTest-328594429 tempest-ServerDiagnosticsNegativeTest-328594429-project] 10.222.0.26 "GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e" status: 404 len: 111 microversion: 2.1 time: 0.046381 Sep 27 09:57:37.952277 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 190/379] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:37 2022] GET /compute/v2.1/servers/52fadb34-4714-4e01-b7d1-db490dec438e => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:57:38.016385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a8cd1ea4-4350-4bba-b9af-a83f7aa97197 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:38.027153 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a8cd1ea4-4350-4bba-b9af-a83f7aa97197 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1319 microversion: 2.1 time: 0.623600 Sep 27 09:57:38.028696 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 190/380] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:37 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1319 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:38.191679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fa714345-c4ce-485e-a039-74af591637c7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:38.193046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa714345-c4ce-485e-a039-74af591637c7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:38.201851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa714345-c4ce-485e-a039-74af591637c7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:38.202953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa714345-c4ce-485e-a039-74af591637c7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:38.203903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa714345-c4ce-485e-a039-74af591637c7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:38.204908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa714345-c4ce-485e-a039-74af591637c7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:38.205577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa714345-c4ce-485e-a039-74af591637c7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:38.206076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa714345-c4ce-485e-a039-74af591637c7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:38.273081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d0b41106-b183-499b-ace6-397122b3c25b req-09392cbc-c2b7-46fc-bc74-13a604592852 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "52fadb34-4714-4e01-b7d1-db490dec438e", "name": "network-vif-deleted", "tag": "98c20e80-4aac-40db-9819-c3a95358dbc2"}, {"name": "network-changed", "server_uuid": "46cf8ffa-724e-46d6-8952-8985413821b7", "tag": "98beaffe-3e67-4b2e-8532-d90575ae8542"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:38.304362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d0b41106-b183-499b-ace6-397122b3c25b req-09392cbc-c2b7-46fc-bc74-13a604592852 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:38.307662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d0b41106-b183-499b-ace6-397122b3c25b req-09392cbc-c2b7-46fc-bc74-13a604592852 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:38.309708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d0b41106-b183-499b-ace6-397122b3c25b req-09392cbc-c2b7-46fc-bc74-13a604592852 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:38.350726 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d0b41106-b183-499b-ace6-397122b3c25b req-09392cbc-c2b7-46fc-bc74-13a604592852 service nova] Creating event network-vif-deleted:98c20e80-4aac-40db-9819-c3a95358dbc2 for instance 52fadb34-4714-4e01-b7d1-db490dec438e on nch2-master-1 Sep 27 09:57:38.352982 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d0b41106-b183-499b-ace6-397122b3c25b req-09392cbc-c2b7-46fc-bc74-13a604592852 service nova] Creating event network-changed:98beaffe-3e67-4b2e-8532-d90575ae8542 for instance 46cf8ffa-724e-46d6-8952-8985413821b7 on nch1-master-1 Sep 27 09:57:38.372027 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d0b41106-b183-499b-ace6-397122b3c25b req-09392cbc-c2b7-46fc-bc74-13a604592852 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.104525 Sep 27 09:57:38.373289 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 191/381] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:38 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:38.462888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fa714345-c4ce-485e-a039-74af591637c7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:38.471927 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fa714345-c4ce-485e-a039-74af591637c7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1313 microversion: 2.1 time: 0.284125 Sep 27 09:57:38.485387 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 191/382] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:38 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1313 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:38.759736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9ff442f3-e72d-4b65-ab88-e7278f125588 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:38.761277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ff442f3-e72d-4b65-ab88-e7278f125588 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:38.774219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=83,nova_api:SAVEPOINT=3,nova_api:INSERT=2,nova_api:RELEASE=3,nova_api:UPDATE=2,nova_api:DELETE=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:38.778151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ff442f3-e72d-4b65-ab88-e7278f125588 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:38.781315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ff442f3-e72d-4b65-ab88-e7278f125588 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:38.782789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ff442f3-e72d-4b65-ab88-e7278f125588 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:38.784757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ff442f3-e72d-4b65-ab88-e7278f125588 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:38.785451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ff442f3-e72d-4b65-ab88-e7278f125588 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:38.791970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ff442f3-e72d-4b65-ab88-e7278f125588 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:38.924268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:38.925766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:38.935043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:38.940253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:38.941061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:38.942049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:38.942705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:38.943313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:38.972443 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] HTTP exception thrown: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Request method failure captured: Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb HTTP/1.1 Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj5jSnuM9Zwz0foBKu5jABAQUgw8wTL0EwJkY5HQ7LQe4gK6BX8DeDMmIBcZM_rJvG9FhHzzDTGDUDodnCyCIy5tpgEbxXeiVZyoCAX68HqXNzXBsZuFCAfa5iwyQsinEdjAd-M-I62Igeh4KLv-wzx9hI1RjjH2hr9CVIE1XJSZgk Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:57:38.974646 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3132070d235941498e3f64d9d7075f6b"}]}] Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7ba3c51770094c4ea7fec1e597ae13a0 Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:57:38.975875 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2236fa92-0b7d-4029-a7fd-ffbb09df76bb'} Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:38.976555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:57:38.977749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:38.982754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Returning 404 to user: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:38.985356 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6a55c071-847b-42cd-aa2a-43ba5580c859 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 404 len: 111 microversion: 2.1 time: 0.065064 Sep 27 09:57:38.987622 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 192/383] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:38 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 09:57:38.993106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-6023e53b-ec91-4990-b00a-72dc2dd07db9 req-6dc468c8-b77a-4e1b-a73a-0d4c8eaf535a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ea27a3db-7b84-4c05-b57f-77b3e6b397f7", "tag": "43772c65-3acd-4918-ad59-e3de25b39006"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:39.011577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6023e53b-ec91-4990-b00a-72dc2dd07db9 req-6dc468c8-b77a-4e1b-a73a-0d4c8eaf535a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:39.014291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6023e53b-ec91-4990-b00a-72dc2dd07db9 req-6dc468c8-b77a-4e1b-a73a-0d4c8eaf535a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:39.016540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6023e53b-ec91-4990-b00a-72dc2dd07db9 req-6dc468c8-b77a-4e1b-a73a-0d4c8eaf535a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:39.043709 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-6023e53b-ec91-4990-b00a-72dc2dd07db9 req-6dc468c8-b77a-4e1b-a73a-0d4c8eaf535a service nova] Creating event network-changed:43772c65-3acd-4918-ad59-e3de25b39006 for instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 on nch2-master-1 Sep 27 09:57:39.059122 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-6023e53b-ec91-4990-b00a-72dc2dd07db9 req-6dc468c8-b77a-4e1b-a73a-0d4c8eaf535a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069879 Sep 27 09:57:39.060411 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 193/384] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:39.065135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:39.071280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-46b18eff-8536-4bd7-a8e5-ecb3e6046558 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:39.072302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-46b18eff-8536-4bd7-a8e5-ecb3e6046558 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:39.079517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46b18eff-8536-4bd7-a8e5-ecb3e6046558 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:39.080316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46b18eff-8536-4bd7-a8e5-ecb3e6046558 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:39.081034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46b18eff-8536-4bd7-a8e5-ecb3e6046558 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:39.081951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46b18eff-8536-4bd7-a8e5-ecb3e6046558 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:39.082627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46b18eff-8536-4bd7-a8e5-ecb3e6046558 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:39.083234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46b18eff-8536-4bd7-a8e5-ecb3e6046558 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:39.159442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ff442f3-e72d-4b65-ab88-e7278f125588 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:39.170966 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9ff442f3-e72d-4b65-ab88-e7278f125588 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1322 microversion: 2.1 time: 0.416152 Sep 27 09:57:39.172297 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 192/385] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:38 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1322 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:39.255632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersAdminTestJSON-server-554447634", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:39.349688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:57:39.648620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:57:39.649618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:57:39.650722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:57:39.651577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:57:39.652156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:57:39.668492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:39.669256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:39.670062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:39.671036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:39.671896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:39.672953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:39.674443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-46b18eff-8536-4bd7-a8e5-ecb3e6046558 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:39.686547 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-46b18eff-8536-4bd7-a8e5-ecb3e6046558 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1319 microversion: 2.1 time: 0.624198 Sep 27 09:57:39.703368 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 194/386] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:39 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1319 bytes in 626 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:39.703368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4cea019c-3703-467e-b2f6-3b2cdb55f078 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:39.703368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4cea019c-3703-467e-b2f6-3b2cdb55f078 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:39.714095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cea019c-3703-467e-b2f6-3b2cdb55f078 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:39.714095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cea019c-3703-467e-b2f6-3b2cdb55f078 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:39.714095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cea019c-3703-467e-b2f6-3b2cdb55f078 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:39.714666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cea019c-3703-467e-b2f6-3b2cdb55f078 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:39.715131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cea019c-3703-467e-b2f6-3b2cdb55f078 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:39.715557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cea019c-3703-467e-b2f6-3b2cdb55f078 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:39.723967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Getting quotas for project 3132070d235941498e3f64d9d7075f6b. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:39.738372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Getting quotas for user 7ba3c51770094c4ea7fec1e597ae13a0 and project 3132070d235941498e3f64d9d7075f6b. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:39.759645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:57:39.769507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:57:39.859106 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c1c2a6d9-8c3e-4106-86e3-6f5dbafe4605 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.604366 Sep 27 09:57:39.860433 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 193/387] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:57:39 2022] POST /compute/v2.1/servers => generated 380 bytes in 610 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 09:57:39.872708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a87960d2-56bf-45d4-be27-44f178a9bd00 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:39.873563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a87960d2-56bf-45d4-be27-44f178a9bd00 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:39.944954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=46 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:39.983669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a87960d2-56bf-45d4-be27-44f178a9bd00 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:39.995637 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a87960d2-56bf-45d4-be27-44f178a9bd00 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1219 microversion: 2.1 time: 0.127011 Sep 27 09:57:39.998466 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 194/388] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:39 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1219 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:40.089450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4cea019c-3703-467e-b2f6-3b2cdb55f078 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:40.121329 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4cea019c-3703-467e-b2f6-3b2cdb55f078 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1313 microversion: 2.1 time: 0.427958 Sep 27 09:57:40.125737 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 195/389] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:39 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1313 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:40.194994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07132a00-b33c-469a-b301-65917d9901aa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:40.196373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-07132a00-b33c-469a-b301-65917d9901aa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:40.205510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07132a00-b33c-469a-b301-65917d9901aa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:40.206382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07132a00-b33c-469a-b301-65917d9901aa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:40.207330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07132a00-b33c-469a-b301-65917d9901aa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:40.212940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07132a00-b33c-469a-b301-65917d9901aa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:40.213969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07132a00-b33c-469a-b301-65917d9901aa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:40.214914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07132a00-b33c-469a-b301-65917d9901aa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:40.408829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-07132a00-b33c-469a-b301-65917d9901aa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:40.419169 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-07132a00-b33c-469a-b301-65917d9901aa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1322 microversion: 2.1 time: 0.227792 Sep 27 09:57:40.432782 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 195/390] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:40 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1322 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:40.717643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5c6b1296-b08c-4695-a68d-4d27a2d3d755 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:40.718516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5c6b1296-b08c-4695-a68d-4d27a2d3d755 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:40.755482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c6b1296-b08c-4695-a68d-4d27a2d3d755 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:40.758303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c6b1296-b08c-4695-a68d-4d27a2d3d755 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:40.761604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c6b1296-b08c-4695-a68d-4d27a2d3d755 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:40.763677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c6b1296-b08c-4695-a68d-4d27a2d3d755 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:40.764171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c6b1296-b08c-4695-a68d-4d27a2d3d755 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:40.765066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c6b1296-b08c-4695-a68d-4d27a2d3d755 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:40.791279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a1d654da-3d17-42ea-ac2e-0f3e42df4609 req-1de6544f-0239-4008-8439-47cfa8e723f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "706e3141-43af-4a6b-8789-dea152d0312a", "name": "network-vif-plugged", "status": "completed", "tag": "ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:40.820092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a1d654da-3d17-42ea-ac2e-0f3e42df4609 req-1de6544f-0239-4008-8439-47cfa8e723f4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:40.821232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a1d654da-3d17-42ea-ac2e-0f3e42df4609 req-1de6544f-0239-4008-8439-47cfa8e723f4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:40.822892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a1d654da-3d17-42ea-ac2e-0f3e42df4609 req-1de6544f-0239-4008-8439-47cfa8e723f4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:40.854449 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a1d654da-3d17-42ea-ac2e-0f3e42df4609 req-1de6544f-0239-4008-8439-47cfa8e723f4 service nova] Creating event network-vif-plugged:ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4 for instance 706e3141-43af-4a6b-8789-dea152d0312a on nch1-master-1 Sep 27 09:57:40.866056 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a1d654da-3d17-42ea-ac2e-0f3e42df4609 req-1de6544f-0239-4008-8439-47cfa8e723f4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080935 Sep 27 09:57:40.871223 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 196/391] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:41.327994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationNegativeTest-server-2093504611", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:41.342543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5c6b1296-b08c-4695-a68d-4d27a2d3d755 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:41.350698 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5c6b1296-b08c-4695-a68d-4d27a2d3d755 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1319 microversion: 2.1 time: 0.642953 Sep 27 09:57:41.367804 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 196/392] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:40 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1319 bytes in 662 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:41.373760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5978c998-6982-41a1-92ab-4996729a721d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:41.374584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5978c998-6982-41a1-92ab-4996729a721d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:41.390056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5978c998-6982-41a1-92ab-4996729a721d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:41.390957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5978c998-6982-41a1-92ab-4996729a721d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:41.391661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5978c998-6982-41a1-92ab-4996729a721d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:41.392634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5978c998-6982-41a1-92ab-4996729a721d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:41.393954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5978c998-6982-41a1-92ab-4996729a721d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:41.395583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5978c998-6982-41a1-92ab-4996729a721d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:41.410298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:57:41.489100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5978c998-6982-41a1-92ab-4996729a721d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:41.501133 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5978c998-6982-41a1-92ab-4996729a721d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1271 microversion: 2.1 time: 0.131764 Sep 27 09:57:41.502070 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 197/393] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:41 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1271 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:41.586876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c229e614-8ea3-4181-8cb3-a54e7c0f46d6 tempest-FlavorsV261TestJSON-608083278 tempest-FlavorsV261TestJSON-608083278-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsV261TestJSON-flavor-4079222", "id": 147777471, "os-flavor-access:is_public": "True"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:41.623992 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c229e614-8ea3-4181-8cb3-a54e7c0f46d6 tempest-FlavorsV261TestJSON-608083278 tempest-FlavorsV261TestJSON-608083278-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 462 microversion: 2.61 time: 0.120041 Sep 27 09:57:41.625366 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 198/394] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 09:57:41 2022] POST /compute/v2.1/flavors => generated 462 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:57:41.657412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0d3c1c65-d448-47fa-a899-5b7162db3a34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:41.664518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0d3c1c65-d448-47fa-a899-5b7162db3a34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:41.675376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3c1c65-d448-47fa-a899-5b7162db3a34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:41.676539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3c1c65-d448-47fa-a899-5b7162db3a34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:41.678132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3c1c65-d448-47fa-a899-5b7162db3a34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:41.680916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3c1c65-d448-47fa-a899-5b7162db3a34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:41.681677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3c1c65-d448-47fa-a899-5b7162db3a34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:41.682445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d3c1c65-d448-47fa-a899-5b7162db3a34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:42.058837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0d3c1c65-d448-47fa-a899-5b7162db3a34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:42.080410 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0d3c1c65-d448-47fa-a899-5b7162db3a34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1313 microversion: 2.1 time: 0.453530 Sep 27 09:57:42.100587 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 199/395] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:41 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1313 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:42.106577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7c80c131-1700-446c-98a3-f0b0bdd06289 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:42.107238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7c80c131-1700-446c-98a3-f0b0bdd06289 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:42.117013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c80c131-1700-446c-98a3-f0b0bdd06289 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:42.118027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c80c131-1700-446c-98a3-f0b0bdd06289 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:42.118799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c80c131-1700-446c-98a3-f0b0bdd06289 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:42.119740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c80c131-1700-446c-98a3-f0b0bdd06289 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:42.120499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c80c131-1700-446c-98a3-f0b0bdd06289 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:42.121273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c80c131-1700-446c-98a3-f0b0bdd06289 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:42.274222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:57:42.274646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:57:42.275313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:57:42.276042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:57:42.276624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:57:42.311323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:42.312382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:42.317241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:42.318472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:42.319550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:42.320742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:42.393328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Getting quotas for project 6e8a121c6c654613a6a92df03c3754fa. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:42.404765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Getting quotas for user 4319ec4b90e1481b8be19e8f9dde1ea4 and project 6e8a121c6c654613a6a92df03c3754fa. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:42.420639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:57:42.423382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:57:42.496649 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cf756e6f-7a89-4d09-bdc7-3e51db3811eb tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.595496 Sep 27 09:57:42.498080 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 197/396] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:57:40 2022] POST /compute/v2.1/servers => generated 380 bytes in 1599 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 09:57:42.547452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7c80c131-1700-446c-98a3-f0b0bdd06289 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:42.560782 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7c80c131-1700-446c-98a3-f0b0bdd06289 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1322 microversion: 2.1 time: 0.458709 Sep 27 09:57:42.570869 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 200/397] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:42 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1322 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:42.587731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ef7d8b5c-b187-40f5-8edf-7f3b9b4995c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:42.588683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ef7d8b5c-b187-40f5-8edf-7f3b9b4995c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:42.605586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef7d8b5c-b187-40f5-8edf-7f3b9b4995c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:42.606345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef7d8b5c-b187-40f5-8edf-7f3b9b4995c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:42.619602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef7d8b5c-b187-40f5-8edf-7f3b9b4995c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:42.622628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef7d8b5c-b187-40f5-8edf-7f3b9b4995c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:42.626898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef7d8b5c-b187-40f5-8edf-7f3b9b4995c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:42.627737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef7d8b5c-b187-40f5-8edf-7f3b9b4995c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:43.059348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-95ba9083-8fc2-4fc0-aee3-a225208740d7 tempest-FlavorsV261TestJSON-172156436 tempest-FlavorsV261TestJSON-172156436-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:43.080909 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-95ba9083-8fc2-4fc0-aee3-a225208740d7 tempest-FlavorsV261TestJSON-172156436 tempest-FlavorsV261TestJSON-172156436-project] 10.222.0.26 "GET /compute/v2.1/flavors/147777471" status: 200 len: 462 microversion: 2.61 time: 0.580670 Sep 27 09:57:43.081890 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 198/398] 10.222.0.26 () {60 vars in 1196 bytes} [Tue Sep 27 09:57:42 2022] GET /compute/v2.1/flavors/147777471 => generated 462 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:57:43.094150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6aa3ac85-4dd4-4044-9632-9de780400201 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:43.100978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6aa3ac85-4dd4-4044-9632-9de780400201 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:43.183405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6aa3ac85-4dd4-4044-9632-9de780400201 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:43.195643 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6aa3ac85-4dd4-4044-9632-9de780400201 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1219 microversion: 2.1 time: 0.111888 Sep 27 09:57:43.197609 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 199/399] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:43 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1219 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:43.208476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c4edcf8c-6712-4f8d-a36f-2566d81be244 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:43.209129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c4edcf8c-6712-4f8d-a36f-2566d81be244 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:43.219623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4edcf8c-6712-4f8d-a36f-2566d81be244 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:43.220241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4edcf8c-6712-4f8d-a36f-2566d81be244 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:43.220784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4edcf8c-6712-4f8d-a36f-2566d81be244 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:43.221600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4edcf8c-6712-4f8d-a36f-2566d81be244 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:43.222070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4edcf8c-6712-4f8d-a36f-2566d81be244 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:43.222503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4edcf8c-6712-4f8d-a36f-2566d81be244 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:43.321075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c4edcf8c-6712-4f8d-a36f-2566d81be244 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:43.330798 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c4edcf8c-6712-4f8d-a36f-2566d81be244 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1277 microversion: 2.1 time: 0.131722 Sep 27 09:57:43.331705 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 200/400] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:43 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1277 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:43.337433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-1a89f5b6-3905-48cc-8dd3-113e1b4a58cb req-6f5b3fd3-c029-48cb-929d-535c0e8851af service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0e538bab-a9f3-488f-9bef-801f2429f73f", "name": "network-vif-plugged", "status": "completed", "tag": "9ff31270-663b-4d9c-a7e1-90a7d2b0c7fb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:43.348717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1a89f5b6-3905-48cc-8dd3-113e1b4a58cb req-6f5b3fd3-c029-48cb-929d-535c0e8851af service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:43.349523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1a89f5b6-3905-48cc-8dd3-113e1b4a58cb req-6f5b3fd3-c029-48cb-929d-535c0e8851af service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:43.350221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1a89f5b6-3905-48cc-8dd3-113e1b4a58cb req-6f5b3fd3-c029-48cb-929d-535c0e8851af service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:43.365795 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-1a89f5b6-3905-48cc-8dd3-113e1b4a58cb req-6f5b3fd3-c029-48cb-929d-535c0e8851af service nova] Creating event network-vif-plugged:9ff31270-663b-4d9c-a7e1-90a7d2b0c7fb for instance 0e538bab-a9f3-488f-9bef-801f2429f73f on nch1-master-1 Sep 27 09:57:43.373224 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-1a89f5b6-3905-48cc-8dd3-113e1b4a58cb req-6f5b3fd3-c029-48cb-929d-535c0e8851af service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039657 Sep 27 09:57:43.373492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ef7d8b5c-b187-40f5-8edf-7f3b9b4995c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:43.375181 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 201/401] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:43.382127 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ef7d8b5c-b187-40f5-8edf-7f3b9b4995c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1319 microversion: 2.1 time: 0.809836 Sep 27 09:57:43.382749 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 201/402] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:42 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1319 bytes in 811 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:43.394413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3cfa9b78-2996-4b96-9655-acc3e856b4b5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:43.395053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3cfa9b78-2996-4b96-9655-acc3e856b4b5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:43.398144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9418614c-89ce-4ffd-bc27-f3069cda0311 tempest-FlavorsV261TestJSON-608083278 tempest-FlavorsV261TestJSON-608083278-project-admin] Action: 'update', calling method: , body: {"flavor": {"description": "new"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:43.416212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cfa9b78-2996-4b96-9655-acc3e856b4b5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:43.416749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cfa9b78-2996-4b96-9655-acc3e856b4b5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:43.417191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cfa9b78-2996-4b96-9655-acc3e856b4b5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:43.417917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cfa9b78-2996-4b96-9655-acc3e856b4b5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:43.418593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cfa9b78-2996-4b96-9655-acc3e856b4b5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:43.419020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cfa9b78-2996-4b96-9655-acc3e856b4b5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:43.423531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:43.478077 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9418614c-89ce-4ffd-bc27-f3069cda0311 tempest-FlavorsV261TestJSON-608083278 tempest-FlavorsV261TestJSON-608083278-project-admin] 10.222.0.26 "PUT /compute/v2.1/flavors/147777471" status: 200 len: 463 microversion: 2.61 time: 0.100705 Sep 27 09:57:43.487173 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 202/403] 10.222.0.26 () {62 vars in 1216 bytes} [Tue Sep 27 09:57:43 2022] PUT /compute/v2.1/flavors/147777471 => generated 463 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 09:57:43.494516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-48e51a21-37c2-4b45-8777-7d33c9a41e4b tempest-FlavorsV261TestJSON-172156436 tempest-FlavorsV261TestJSON-172156436-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:43.549832 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-48e51a21-37c2-4b45-8777-7d33c9a41e4b tempest-FlavorsV261TestJSON-172156436 tempest-FlavorsV261TestJSON-172156436-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 7147 microversion: 2.61 time: 0.060760 Sep 27 09:57:43.551359 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 203/404] 10.222.0.26 () {60 vars in 1187 bytes} [Tue Sep 27 09:57:43 2022] GET /compute/v2.1/flavors/detail => generated 7147 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 09:57:43.575099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9c58b4c5-4d10-477a-89ac-68820a98f74c tempest-FlavorsV261TestJSON-172156436 tempest-FlavorsV261TestJSON-172156436-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:43.623158 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9c58b4c5-4d10-477a-89ac-68820a98f74c tempest-FlavorsV261TestJSON-172156436 tempest-FlavorsV261TestJSON-172156436-project] 10.222.0.26 "GET /compute/v2.1/flavors" status: 200 len: 3674 microversion: 2.61 time: 0.053011 Sep 27 09:57:43.624515 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 204/405] 10.222.0.26 () {60 vars in 1166 bytes} [Tue Sep 27 09:57:43 2022] GET /compute/v2.1/flavors => generated 3674 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:43.631805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a16fa79d-fbb9-4233-8979-60a1db109ee9 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:43.633205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a16fa79d-fbb9-4233-8979-60a1db109ee9 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:43.642912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a16fa79d-fbb9-4233-8979-60a1db109ee9 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:43.643589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a16fa79d-fbb9-4233-8979-60a1db109ee9 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:43.644755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a16fa79d-fbb9-4233-8979-60a1db109ee9 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:43.645990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a16fa79d-fbb9-4233-8979-60a1db109ee9 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:43.646971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a16fa79d-fbb9-4233-8979-60a1db109ee9 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:43.647694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a16fa79d-fbb9-4233-8979-60a1db109ee9 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:43.745888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3cfa9b78-2996-4b96-9655-acc3e856b4b5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:43.755583 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3cfa9b78-2996-4b96-9655-acc3e856b4b5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1463 microversion: 2.1 time: 0.370740 Sep 27 09:57:43.769195 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 202/406] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:43 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1463 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:43.795671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3a9feb5b-06f4-49ab-b438-53306f1069a0 tempest-FlavorsV261TestJSON-608083278 tempest-FlavorsV261TestJSON-608083278-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:43.844610 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3a9feb5b-06f4-49ab-b438-53306f1069a0 tempest-FlavorsV261TestJSON-608083278 tempest-FlavorsV261TestJSON-608083278-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/147777471" status: 202 len: 0 microversion: 2.61 time: 0.073561 Sep 27 09:57:43.845584 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 203/407] 10.222.0.26 () {60 vars in 1199 bytes} [Tue Sep 27 09:57:43 2022] DELETE /compute/v2.1/flavors/147777471 => generated 0 bytes in 76 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:57:43.877057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a16fa79d-fbb9-4233-8979-60a1db109ee9 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:43.887468 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a16fa79d-fbb9-4233-8979-60a1db109ee9 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1322 microversion: 2.1 time: 0.261363 Sep 27 09:57:43.888905 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 205/408] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:43 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1322 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:43.895148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-85e8a224-287b-4981-b89f-d8ef8a6ef969 tempest-FlavorsV261TestJSON-608083278 tempest-FlavorsV261TestJSON-608083278-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:43.925525 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-85e8a224-287b-4981-b89f-d8ef8a6ef969 tempest-FlavorsV261TestJSON-608083278 tempest-FlavorsV261TestJSON-608083278-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 6694 microversion: 2.61 time: 0.034271 Sep 27 09:57:43.928241 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 206/409] 10.222.0.26 () {60 vars in 1187 bytes} [Tue Sep 27 09:57:43 2022] GET /compute/v2.1/flavors/detail => generated 6694 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:44.231490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aafce39e-5ba0-45ba-ac2b-e4e6b6e6d66b tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:44.233840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aafce39e-5ba0-45ba-ac2b-e4e6b6e6d66b tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:44.243706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafce39e-5ba0-45ba-ac2b-e4e6b6e6d66b tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:44.244309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafce39e-5ba0-45ba-ac2b-e4e6b6e6d66b tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:44.244978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafce39e-5ba0-45ba-ac2b-e4e6b6e6d66b tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:44.245903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafce39e-5ba0-45ba-ac2b-e4e6b6e6d66b tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:44.246429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafce39e-5ba0-45ba-ac2b-e4e6b6e6d66b tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:44.247340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafce39e-5ba0-45ba-ac2b-e4e6b6e6d66b tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:44.374939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aafce39e-5ba0-45ba-ac2b-e4e6b6e6d66b tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:44.390041 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aafce39e-5ba0-45ba-ac2b-e4e6b6e6d66b tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1271 microversion: 2.1 time: 0.174229 Sep 27 09:57:44.404836 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 207/410] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:44 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1271 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:44.409676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0029ee2b-c7ec-4846-a43c-69a025be8c38 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:44.410661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0029ee2b-c7ec-4846-a43c-69a025be8c38 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:44.420429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0029ee2b-c7ec-4846-a43c-69a025be8c38 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:44.421193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0029ee2b-c7ec-4846-a43c-69a025be8c38 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:44.421636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0029ee2b-c7ec-4846-a43c-69a025be8c38 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:44.422583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0029ee2b-c7ec-4846-a43c-69a025be8c38 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:44.423049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0029ee2b-c7ec-4846-a43c-69a025be8c38 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:44.423502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0029ee2b-c7ec-4846-a43c-69a025be8c38 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:44.460089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e9cbff3f-460a-411d-b553-9eeb25257796 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:44.460922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e9cbff3f-460a-411d-b553-9eeb25257796 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:44.505268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9cbff3f-460a-411d-b553-9eeb25257796 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:44.505889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9cbff3f-460a-411d-b553-9eeb25257796 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:44.506339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9cbff3f-460a-411d-b553-9eeb25257796 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:44.507113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9cbff3f-460a-411d-b553-9eeb25257796 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:44.507590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9cbff3f-460a-411d-b553-9eeb25257796 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:44.508058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9cbff3f-460a-411d-b553-9eeb25257796 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:44.791111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0029ee2b-c7ec-4846-a43c-69a025be8c38 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:44.821398 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0029ee2b-c7ec-4846-a43c-69a025be8c38 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1319 microversion: 2.1 time: 0.411793 Sep 27 09:57:44.825358 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 208/411] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:44 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1319 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:44.831302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-175e7efe-748b-43f3-96f8-e924654821ae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:44.832447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-175e7efe-748b-43f3-96f8-e924654821ae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:44.841871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-175e7efe-748b-43f3-96f8-e924654821ae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:44.843685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-175e7efe-748b-43f3-96f8-e924654821ae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:44.844412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-175e7efe-748b-43f3-96f8-e924654821ae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:44.845432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-175e7efe-748b-43f3-96f8-e924654821ae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:44.846278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-175e7efe-748b-43f3-96f8-e924654821ae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:44.847109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-175e7efe-748b-43f3-96f8-e924654821ae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:45.119289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-175e7efe-748b-43f3-96f8-e924654821ae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:45.131433 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-175e7efe-748b-43f3-96f8-e924654821ae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1319 microversion: 2.1 time: 0.304216 Sep 27 09:57:45.132598 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 209/412] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:44 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1319 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:45.339916 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e9cbff3f-460a-411d-b553-9eeb25257796 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1614 microversion: 2.1 time: 1.492487 Sep 27 09:57:45.343747 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 204/413] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:43 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1614 bytes in 1498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:45.350635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2432bf3a-0372-47f6-81eb-cd6c95898038 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:45.352544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2432bf3a-0372-47f6-81eb-cd6c95898038 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:45.363870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2432bf3a-0372-47f6-81eb-cd6c95898038 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:45.364847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2432bf3a-0372-47f6-81eb-cd6c95898038 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:45.365578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2432bf3a-0372-47f6-81eb-cd6c95898038 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:45.366778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2432bf3a-0372-47f6-81eb-cd6c95898038 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:45.368012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2432bf3a-0372-47f6-81eb-cd6c95898038 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:45.369199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2432bf3a-0372-47f6-81eb-cd6c95898038 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:45.385341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdmin275Test-server-637864540", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": "none"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:45.466788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:57:45.470240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:57:45.471452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:57:45.473667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:57:45.474203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:57:45.488417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:45.490340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:45.490893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:45.491768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:45.492557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:45.493423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:45.522394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Getting quotas for project 7e6987197b0a4c7d9b89d48402845f3f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:45.529488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Getting quotas for user aa2474e7686c4a758ed93064edf47867 and project 7e6987197b0a4c7d9b89d48402845f3f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:45.546705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:57:45.550676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:57:45.628821 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a81abdb9-9d29-4409-ae27-2c34f5dc6b8d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.75 time: 0.493826 Sep 27 09:57:45.637892 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 210/414] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 09:57:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 498 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 09:57:45.645065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:45.645555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:45.653228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:45.653893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:45.654580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:45.655218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:45.655677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:45.656098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:45.770473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2432bf3a-0372-47f6-81eb-cd6c95898038 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:45.817190 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2432bf3a-0372-47f6-81eb-cd6c95898038 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1322 microversion: 2.1 time: 0.471858 Sep 27 09:57:45.828123 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 205/415] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:45 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1322 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:45.843316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-218c0d88-670f-4cf0-9c4f-6e5e274e9622 req-2dcc83c3-313a-4a6d-b2e9-ac438e2191eb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ea27a3db-7b84-4c05-b57f-77b3e6b397f7", "name": "network-vif-plugged", "status": "completed", "tag": "43772c65-3acd-4918-ad59-e3de25b39006"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:45.885157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-218c0d88-670f-4cf0-9c4f-6e5e274e9622 req-2dcc83c3-313a-4a6d-b2e9-ac438e2191eb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:45.885878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-218c0d88-670f-4cf0-9c4f-6e5e274e9622 req-2dcc83c3-313a-4a6d-b2e9-ac438e2191eb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:45.886677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-218c0d88-670f-4cf0-9c4f-6e5e274e9622 req-2dcc83c3-313a-4a6d-b2e9-ac438e2191eb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:45.904547 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-218c0d88-670f-4cf0-9c4f-6e5e274e9622 req-2dcc83c3-313a-4a6d-b2e9-ac438e2191eb service nova] Creating event network-vif-plugged:43772c65-3acd-4918-ad59-e3de25b39006 for instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 on nch2-master-1 Sep 27 09:57:45.911873 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-218c0d88-670f-4cf0-9c4f-6e5e274e9622 req-2dcc83c3-313a-4a6d-b2e9-ac438e2191eb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080430 Sep 27 09:57:45.912783 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 206/416] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:45.923003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-84f061f4-7cbf-4a23-8f12-d5c86269637c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:45.923704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-84f061f4-7cbf-4a23-8f12-d5c86269637c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:45.933028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84f061f4-7cbf-4a23-8f12-d5c86269637c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:45.933500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84f061f4-7cbf-4a23-8f12-d5c86269637c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:45.933944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84f061f4-7cbf-4a23-8f12-d5c86269637c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:45.934695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84f061f4-7cbf-4a23-8f12-d5c86269637c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:45.935255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84f061f4-7cbf-4a23-8f12-d5c86269637c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:45.935580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84f061f4-7cbf-4a23-8f12-d5c86269637c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:46.204510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-84f061f4-7cbf-4a23-8f12-d5c86269637c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:46.221845 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-84f061f4-7cbf-4a23-8f12-d5c86269637c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1277 microversion: 2.1 time: 0.304877 Sep 27 09:57:46.223335 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 207/417] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:45 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1277 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:46.226459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': 'f74e77337957458794bd6c942c97b344', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3199e83259042efada3f2a565c624ff', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:46.227764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] flavor_id is None. Assuming migration. {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4196}} Sep 27 09:57:46.228414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 09:57:46.231199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3fb729e7-c9ea-48f3-b634-cab7daf3befc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Action: 'update', calling method: >, body: {"server": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:46.235363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3fb729e7-c9ea-48f3-b634-cab7daf3befc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:46.289655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=2,nova_api:SAVEPOINT=2,nova_api:RELEASE=2,nova_api:UPDATE=1,nova_api:DELETE=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:46.461640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3fb729e7-c9ea-48f3-b634-cab7daf3befc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:46.476357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3fb729e7-c9ea-48f3-b634-cab7daf3befc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:46.487110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3fb729e7-c9ea-48f3-b634-cab7daf3befc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:46.499805 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3fb729e7-c9ea-48f3-b634-cab7daf3befc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "PUT /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1358 microversion: 2.75 time: 0.274848 Sep 27 09:57:46.508217 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58e6bcce-99bd-4a10-bcc2-ee7790b26c90 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a/action" status: 202 len: 0 microversion: 2.1 time: 0.874773 Sep 27 09:57:46.509251 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 208/418] 10.222.0.26 () {62 vars in 1297 bytes} [Tue Sep 27 09:57:46 2022] PUT /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1358 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:46.509980 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 211/419] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:57:45 2022] POST /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a/action => generated 0 bytes in 878 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:57:46.514827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-78f2e0d8-c84b-46f9-b178-5612ea7fda6a tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:46.515698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-78f2e0d8-c84b-46f9-b178-5612ea7fda6a tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:46.526665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6ec22369-5dad-445f-ad4e-2f9b2a0a1306 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:46.529524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6ec22369-5dad-445f-ad4e-2f9b2a0a1306 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:46.534253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f2e0d8-c84b-46f9-b178-5612ea7fda6a tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:46.534799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f2e0d8-c84b-46f9-b178-5612ea7fda6a tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:46.535253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f2e0d8-c84b-46f9-b178-5612ea7fda6a tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:46.536027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f2e0d8-c84b-46f9-b178-5612ea7fda6a tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:46.536485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f2e0d8-c84b-46f9-b178-5612ea7fda6a tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:46.536944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f2e0d8-c84b-46f9-b178-5612ea7fda6a tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:46.560298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec22369-5dad-445f-ad4e-2f9b2a0a1306 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:46.569805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec22369-5dad-445f-ad4e-2f9b2a0a1306 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:46.572211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec22369-5dad-445f-ad4e-2f9b2a0a1306 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:46.573384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec22369-5dad-445f-ad4e-2f9b2a0a1306 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:46.576404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec22369-5dad-445f-ad4e-2f9b2a0a1306 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:46.577290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec22369-5dad-445f-ad4e-2f9b2a0a1306 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:46.870226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-78f2e0d8-c84b-46f9-b178-5612ea7fda6a tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:46.885678 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-78f2e0d8-c84b-46f9-b178-5612ea7fda6a tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1319 microversion: 2.1 time: 0.374779 Sep 27 09:57:46.889205 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 209/420] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:46 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1319 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:46.894970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-af1f41b4-07c4-4ae0-b846-01720fa2da79 req-69b771d0-a3b7-4e79-8305-1dbf611879ef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc", "tag": "9bd9c08f-8f18-447f-9137-e5d05f5c4f09"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:46.919430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-af1f41b4-07c4-4ae0-b846-01720fa2da79 req-69b771d0-a3b7-4e79-8305-1dbf611879ef service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:46.923920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-af1f41b4-07c4-4ae0-b846-01720fa2da79 req-69b771d0-a3b7-4e79-8305-1dbf611879ef service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:46.924372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-af1f41b4-07c4-4ae0-b846-01720fa2da79 req-69b771d0-a3b7-4e79-8305-1dbf611879ef service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:46.958667 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-af1f41b4-07c4-4ae0-b846-01720fa2da79 req-69b771d0-a3b7-4e79-8305-1dbf611879ef service nova] Creating event network-changed:9bd9c08f-8f18-447f-9137-e5d05f5c4f09 for instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc on nch2-master-1 Sep 27 09:57:46.973229 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-af1f41b4-07c4-4ae0-b846-01720fa2da79 req-69b771d0-a3b7-4e79-8305-1dbf611879ef service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082045 Sep 27 09:57:46.974438 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 210/421] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:46.987225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0914f7f2-1bef-4177-a24d-51203ea5e93b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:46.990988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0914f7f2-1bef-4177-a24d-51203ea5e93b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:47.026934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0914f7f2-1bef-4177-a24d-51203ea5e93b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:47.029957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0914f7f2-1bef-4177-a24d-51203ea5e93b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:47.032397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0914f7f2-1bef-4177-a24d-51203ea5e93b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:47.046883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0914f7f2-1bef-4177-a24d-51203ea5e93b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:47.053958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0914f7f2-1bef-4177-a24d-51203ea5e93b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:47.054556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0914f7f2-1bef-4177-a24d-51203ea5e93b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:47.067317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6ec22369-5dad-445f-ad4e-2f9b2a0a1306 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:47.081134 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6ec22369-5dad-445f-ad4e-2f9b2a0a1306 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1319 microversion: 2.1 time: 0.569492 Sep 27 09:57:47.081739 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 212/422] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:46 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1319 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:47.108490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:47.110189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:47.117897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:47.118630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:47.119252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:47.120162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:47.120864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:47.121583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:47.319262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:47.332508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:47.335737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:47.355746 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-84e1eb1a-cd06-443b-a8d8-af1b3d240f23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1410 microversion: 2.75 time: 0.253303 Sep 27 09:57:47.377037 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 213/423] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:57:47 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1410 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:47.377037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53066619-6012-4ced-bea8-522015409e91 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:47.377037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53066619-6012-4ced-bea8-522015409e91 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:47.385894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=9,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:47.396898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53066619-6012-4ced-bea8-522015409e91 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:47.397618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53066619-6012-4ced-bea8-522015409e91 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:47.399458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53066619-6012-4ced-bea8-522015409e91 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:47.402500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53066619-6012-4ced-bea8-522015409e91 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:47.403305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53066619-6012-4ced-bea8-522015409e91 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:47.405065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53066619-6012-4ced-bea8-522015409e91 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:47.441763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0914f7f2-1bef-4177-a24d-51203ea5e93b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:47.450907 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0914f7f2-1bef-4177-a24d-51203ea5e93b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1472 microversion: 2.1 time: 0.474203 Sep 27 09:57:47.468326 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 211/424] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:46 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1472 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:47.476646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce983151-37ad-4d1b-bfcf-d6e5a01cfe54 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:47.477562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ce983151-37ad-4d1b-bfcf-d6e5a01cfe54 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:47.484737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce983151-37ad-4d1b-bfcf-d6e5a01cfe54 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:47.485293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce983151-37ad-4d1b-bfcf-d6e5a01cfe54 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:47.485838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce983151-37ad-4d1b-bfcf-d6e5a01cfe54 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:47.486886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce983151-37ad-4d1b-bfcf-d6e5a01cfe54 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:47.487670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce983151-37ad-4d1b-bfcf-d6e5a01cfe54 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:47.488106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce983151-37ad-4d1b-bfcf-d6e5a01cfe54 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:47.735737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ce983151-37ad-4d1b-bfcf-d6e5a01cfe54 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:47.746410 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ce983151-37ad-4d1b-bfcf-d6e5a01cfe54 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1319 microversion: 2.1 time: 0.276803 Sep 27 09:57:47.761618 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 212/425] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:47 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1319 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:47.917845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-848b7816-d371-43d2-a7b9-1e7132d3329d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:47.918101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-53066619-6012-4ced-bea8-522015409e91 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:47.919132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-848b7816-d371-43d2-a7b9-1e7132d3329d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:47.929197 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53066619-6012-4ced-bea8-522015409e91 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1322 microversion: 2.1 time: 0.567209 Sep 27 09:57:47.929817 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 214/426] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:47 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1322 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:47.934803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-848b7816-d371-43d2-a7b9-1e7132d3329d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:47.935495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-848b7816-d371-43d2-a7b9-1e7132d3329d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:47.936912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-848b7816-d371-43d2-a7b9-1e7132d3329d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:47.938164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-848b7816-d371-43d2-a7b9-1e7132d3329d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:47.938832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-848b7816-d371-43d2-a7b9-1e7132d3329d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:47.939490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-848b7816-d371-43d2-a7b9-1e7132d3329d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:47.940043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-80a28ef3-61d2-4710-8cca-43bb5d80eca0 req-17c174c7-3ac7-40e3-bd16-54c67e4114d3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "46cf8ffa-724e-46d6-8952-8985413821b7", "name": "network-vif-plugged", "status": "completed", "tag": "98beaffe-3e67-4b2e-8532-d90575ae8542"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:47.952365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-80a28ef3-61d2-4710-8cca-43bb5d80eca0 req-17c174c7-3ac7-40e3-bd16-54c67e4114d3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:47.952886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-80a28ef3-61d2-4710-8cca-43bb5d80eca0 req-17c174c7-3ac7-40e3-bd16-54c67e4114d3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:47.953318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-80a28ef3-61d2-4710-8cca-43bb5d80eca0 req-17c174c7-3ac7-40e3-bd16-54c67e4114d3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:47.970163 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-80a28ef3-61d2-4710-8cca-43bb5d80eca0 req-17c174c7-3ac7-40e3-bd16-54c67e4114d3 service nova] Creating event network-vif-plugged:98beaffe-3e67-4b2e-8532-d90575ae8542 for instance 46cf8ffa-724e-46d6-8952-8985413821b7 on nch1-master-1 Sep 27 09:57:47.975850 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-80a28ef3-61d2-4710-8cca-43bb5d80eca0 req-17c174c7-3ac7-40e3-bd16-54c67e4114d3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039398 Sep 27 09:57:47.977188 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 215/427] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:48.109610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2ab9f88d-03b0-4e71-8122-17bd55ef9cae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:48.113192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2ab9f88d-03b0-4e71-8122-17bd55ef9cae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:48.133771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab9f88d-03b0-4e71-8122-17bd55ef9cae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:48.137571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab9f88d-03b0-4e71-8122-17bd55ef9cae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:48.142736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab9f88d-03b0-4e71-8122-17bd55ef9cae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:48.146771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab9f88d-03b0-4e71-8122-17bd55ef9cae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:48.149997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab9f88d-03b0-4e71-8122-17bd55ef9cae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:48.152608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ab9f88d-03b0-4e71-8122-17bd55ef9cae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:48.185439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:48.393838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-848b7816-d371-43d2-a7b9-1e7132d3329d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:48.406419 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-848b7816-d371-43d2-a7b9-1e7132d3329d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1319 microversion: 2.1 time: 0.493914 Sep 27 09:57:48.407360 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 213/428] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:47 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1319 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:48.413461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:48.414987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:48.422976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:48.424157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:48.424677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:48.425739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:48.426394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:48.426793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:48.585126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:48.597446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2ab9f88d-03b0-4e71-8122-17bd55ef9cae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:48.605081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:48.607235 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2ab9f88d-03b0-4e71-8122-17bd55ef9cae tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1319 microversion: 2.1 time: 0.502292 Sep 27 09:57:48.608176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:48.616980 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af2e1393-078e-46fc-834d-9d81ea0138d1 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1416 microversion: 2.75 time: 0.208051 Sep 27 09:57:48.618796 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 214/429] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:57:48 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1416 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:48.622928 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 216/430] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:48 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1319 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:48.629604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8817faf2-2bd9-4ac2-b047-b5075e21a4be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:48.631750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8817faf2-2bd9-4ac2-b047-b5075e21a4be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:48.639794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8817faf2-2bd9-4ac2-b047-b5075e21a4be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:48.640267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8817faf2-2bd9-4ac2-b047-b5075e21a4be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:48.640831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8817faf2-2bd9-4ac2-b047-b5075e21a4be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:48.641579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8817faf2-2bd9-4ac2-b047-b5075e21a4be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:48.642223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8817faf2-2bd9-4ac2-b047-b5075e21a4be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:48.642737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8817faf2-2bd9-4ac2-b047-b5075e21a4be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:48.759232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7c62aeb3-6262-44d1-9f9b-61647ca666b4 req-fab2e410-1fb4-4d59-9064-fab0456acd86 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "454cd0f6-eddc-4e88-9437-91ecce9cf66c", "tag": "e4cac2e6-28e9-4db8-86d8-5e557c0d5c8a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:48.775140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7c62aeb3-6262-44d1-9f9b-61647ca666b4 req-fab2e410-1fb4-4d59-9064-fab0456acd86 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:48.777148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7c62aeb3-6262-44d1-9f9b-61647ca666b4 req-fab2e410-1fb4-4d59-9064-fab0456acd86 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:48.778612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7c62aeb3-6262-44d1-9f9b-61647ca666b4 req-fab2e410-1fb4-4d59-9064-fab0456acd86 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:48.810297 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7c62aeb3-6262-44d1-9f9b-61647ca666b4 req-fab2e410-1fb4-4d59-9064-fab0456acd86 service nova] Creating event network-changed:e4cac2e6-28e9-4db8-86d8-5e557c0d5c8a for instance 454cd0f6-eddc-4e88-9437-91ecce9cf66c on nch2-master-1 Sep 27 09:57:48.821011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8817faf2-2bd9-4ac2-b047-b5075e21a4be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:48.823805 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7c62aeb3-6262-44d1-9f9b-61647ca666b4 req-fab2e410-1fb4-4d59-9064-fab0456acd86 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068885 Sep 27 09:57:48.826891 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 217/431] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:48.837214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4305deb7-9772-4c5d-a7a8-e501275150b9 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:48.843927 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8817faf2-2bd9-4ac2-b047-b5075e21a4be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1472 microversion: 2.1 time: 0.223550 Sep 27 09:57:48.845050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4305deb7-9772-4c5d-a7a8-e501275150b9 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:48.856934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305deb7-9772-4c5d-a7a8-e501275150b9 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:48.857658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305deb7-9772-4c5d-a7a8-e501275150b9 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:48.858657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305deb7-9772-4c5d-a7a8-e501275150b9 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:48.859687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305deb7-9772-4c5d-a7a8-e501275150b9 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:48.860418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305deb7-9772-4c5d-a7a8-e501275150b9 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:48.861197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305deb7-9772-4c5d-a7a8-e501275150b9 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:48.862243 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 215/432] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:48 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1472 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:48.966310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a86edd91-5c93-41c9-9fd1-bd6d95276ebc tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:48.967208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a86edd91-5c93-41c9-9fd1-bd6d95276ebc tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:48.977027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a86edd91-5c93-41c9-9fd1-bd6d95276ebc tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:48.977923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a86edd91-5c93-41c9-9fd1-bd6d95276ebc tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:48.978431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a86edd91-5c93-41c9-9fd1-bd6d95276ebc tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:48.979246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a86edd91-5c93-41c9-9fd1-bd6d95276ebc tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:48.979710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a86edd91-5c93-41c9-9fd1-bd6d95276ebc tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:48.980138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a86edd91-5c93-41c9-9fd1-bd6d95276ebc tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.090944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4305deb7-9772-4c5d-a7a8-e501275150b9 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:49.106219 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4305deb7-9772-4c5d-a7a8-e501275150b9 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1319 microversion: 2.1 time: 0.275824 Sep 27 09:57:49.107755 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 218/433] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:48 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1319 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:49.262159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a86edd91-5c93-41c9-9fd1-bd6d95276ebc tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:49.273118 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a86edd91-5c93-41c9-9fd1-bd6d95276ebc tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1472 microversion: 2.1 time: 0.311433 Sep 27 09:57:49.273753 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 216/434] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:48 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1472 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:49.297102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0a7c97b1-7648-4400-8686-66ff80748a8d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:49.297668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0a7c97b1-7648-4400-8686-66ff80748a8d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:49.305417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a7c97b1-7648-4400-8686-66ff80748a8d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.305949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a7c97b1-7648-4400-8686-66ff80748a8d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.306514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a7c97b1-7648-4400-8686-66ff80748a8d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.307453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a7c97b1-7648-4400-8686-66ff80748a8d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.307915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a7c97b1-7648-4400-8686-66ff80748a8d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.308345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a7c97b1-7648-4400-8686-66ff80748a8d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.339067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-0a7c97b1-7648-4400-8686-66ff80748a8d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lazy-loading 'flavor' on Instance uuid ea27a3db-7b84-4c05-b57f-77b3e6b397f7 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:57:49.433938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-48143213-5107-4af2-9658-463a40e17a16 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:49.435396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-48143213-5107-4af2-9658-463a40e17a16 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:49.446190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48143213-5107-4af2-9658-463a40e17a16 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.447378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48143213-5107-4af2-9658-463a40e17a16 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.448144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48143213-5107-4af2-9658-463a40e17a16 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.449588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48143213-5107-4af2-9658-463a40e17a16 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.450371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48143213-5107-4af2-9658-463a40e17a16 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.451403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48143213-5107-4af2-9658-463a40e17a16 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.474102 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0a7c97b1-7648-4400-8686-66ff80748a8d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7/action" status: 202 len: 0 microversion: 2.1 time: 0.180584 Sep 27 09:57:49.475023 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 219/435] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:57:49 2022] POST /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7/action => generated 0 bytes in 182 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:57:49.485141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-32f29b2c-41bd-426f-b81e-a6e7deb4922a tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:49.485965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-32f29b2c-41bd-426f-b81e-a6e7deb4922a tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:49.500233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32f29b2c-41bd-426f-b81e-a6e7deb4922a tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.500752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32f29b2c-41bd-426f-b81e-a6e7deb4922a tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.501376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32f29b2c-41bd-426f-b81e-a6e7deb4922a tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.502454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32f29b2c-41bd-426f-b81e-a6e7deb4922a tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.503097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32f29b2c-41bd-426f-b81e-a6e7deb4922a tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.503525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32f29b2c-41bd-426f-b81e-a6e7deb4922a tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.692019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-48143213-5107-4af2-9658-463a40e17a16 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:49.704267 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-48143213-5107-4af2-9658-463a40e17a16 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1319 microversion: 2.1 time: 0.276776 Sep 27 09:57:49.705761 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 217/436] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:49 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1319 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:49.727520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d7413d54-b187-4e3a-a321-9342169ab200 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:49.733631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d7413d54-b187-4e3a-a321-9342169ab200 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:49.740573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7413d54-b187-4e3a-a321-9342169ab200 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.741159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7413d54-b187-4e3a-a321-9342169ab200 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.741598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7413d54-b187-4e3a-a321-9342169ab200 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.742868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7413d54-b187-4e3a-a321-9342169ab200 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.743576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7413d54-b187-4e3a-a321-9342169ab200 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.744043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7413d54-b187-4e3a-a321-9342169ab200 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.857285 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-32f29b2c-41bd-426f-b81e-a6e7deb4922a tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1631 microversion: 2.1 time: 0.376652 Sep 27 09:57:49.857963 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 220/437] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:49 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1631 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:49.863509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:49.864233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:49.873116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.874862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.875464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.876320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.876904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.877432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.891940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d7413d54-b187-4e3a-a321-9342169ab200 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:49.900520 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d7413d54-b187-4e3a-a321-9342169ab200 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1319 microversion: 2.1 time: 0.192516 Sep 27 09:57:49.901134 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 218/438] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:49 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1319 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:49.958334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:49.968313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:49.969822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:49.975641 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ab6dc9ed-a6e1-405b-9415-12e403166431 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1416 microversion: 2.75 time: 0.116365 Sep 27 09:57:49.977842 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 221/439] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:57:49 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1416 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:49.982852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f73e52b-bb50-4262-a3c8-81b2541dcdf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:49.983637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8f73e52b-bb50-4262-a3c8-81b2541dcdf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:49.990991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f73e52b-bb50-4262-a3c8-81b2541dcdf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.991510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f73e52b-bb50-4262-a3c8-81b2541dcdf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.991953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f73e52b-bb50-4262-a3c8-81b2541dcdf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:49.993576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f73e52b-bb50-4262-a3c8-81b2541dcdf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:49.994074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f73e52b-bb50-4262-a3c8-81b2541dcdf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:49.994511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f73e52b-bb50-4262-a3c8-81b2541dcdf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.089682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b042061c-a821-47aa-8268-ec944f6b3bbb tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Calling method '.version_select at 0x7f7f63eaef70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.091965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b042061c-a821-47aa-8268-ec944f6b3bbb tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.093061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b042061c-a821-47aa-8268-ec944f6b3bbb tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.093551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b042061c-a821-47aa-8268-ec944f6b3bbb tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.113074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b042061c-a821-47aa-8268-ec944f6b3bbb tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.113616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b042061c-a821-47aa-8268-ec944f6b3bbb tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.114056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b042061c-a821-47aa-8268-ec944f6b3bbb tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.129762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b042061c-a821-47aa-8268-ec944f6b3bbb tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.130486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b042061c-a821-47aa-8268-ec944f6b3bbb tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.131043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b042061c-a821-47aa-8268-ec944f6b3bbb tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.140031 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b042061c-a821-47aa-8268-ec944f6b3bbb tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.237738 Sep 27 09:57:50.141069 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 219/440] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:57:49 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:50.147378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a2b1ec41-83da-496f-909f-f871ff133869 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.147378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a2b1ec41-83da-496f-909f-f871ff133869 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:50.156214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2b1ec41-83da-496f-909f-f871ff133869 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.157018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2b1ec41-83da-496f-909f-f871ff133869 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.157649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2b1ec41-83da-496f-909f-f871ff133869 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.158592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2b1ec41-83da-496f-909f-f871ff133869 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.159042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2b1ec41-83da-496f-909f-f871ff133869 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.159577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2b1ec41-83da-496f-909f-f871ff133869 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.173220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8f73e52b-bb50-4262-a3c8-81b2541dcdf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:50.182506 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8f73e52b-bb50-4262-a3c8-81b2541dcdf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1472 microversion: 2.1 time: 0.203463 Sep 27 09:57:50.183078 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 222/441] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:49 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1472 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:50.188161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e2966923-dce2-4333-bd07-b96e3b3828b7 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Calling method '.version_select at 0x7f7f682168b0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.190176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2966923-dce2-4333-bd07-b96e3b3828b7 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.190929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2966923-dce2-4333-bd07-b96e3b3828b7 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.191427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2966923-dce2-4333-bd07-b96e3b3828b7 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.210644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2966923-dce2-4333-bd07-b96e3b3828b7 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.211627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2966923-dce2-4333-bd07-b96e3b3828b7 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.212561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2966923-dce2-4333-bd07-b96e3b3828b7 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.225823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2966923-dce2-4333-bd07-b96e3b3828b7 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.226382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2966923-dce2-4333-bd07-b96e3b3828b7 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.226904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2966923-dce2-4333-bd07-b96e3b3828b7 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.233885 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e2966923-dce2-4333-bd07-b96e3b3828b7 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/detail" status: 200 len: 1505 microversion: 2.1 time: 0.049576 Sep 27 09:57:50.234489 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 223/442] 10.222.0.26 () {58 vars in 1167 bytes} [Tue Sep 27 09:57:50 2022] GET /compute/v2.1/os-hypervisors/detail => generated 1505 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:50.254496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58c4d951-78a6-42e2-9934-36760cf2fa7a tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Calling method '.version_select at 0x7f7f680523a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.256520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c4d951-78a6-42e2-9934-36760cf2fa7a tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.257081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c4d951-78a6-42e2-9934-36760cf2fa7a tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.257636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c4d951-78a6-42e2-9934-36760cf2fa7a tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.343377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a2b1ec41-83da-496f-909f-f871ff133869 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:50.352389 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a2b1ec41-83da-496f-909f-f871ff133869 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1319 microversion: 2.1 time: 0.210389 Sep 27 09:57:50.361907 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 220/443] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:50 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1319 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:50.471711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c4d951-78a6-42e2-9934-36760cf2fa7a tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.472222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c4d951-78a6-42e2-9934-36760cf2fa7a tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.472705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c4d951-78a6-42e2-9934-36760cf2fa7a tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.484524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c4d951-78a6-42e2-9934-36760cf2fa7a tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.485058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c4d951-78a6-42e2-9934-36760cf2fa7a tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.485496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c4d951-78a6-42e2-9934-36760cf2fa7a tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.493152 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58c4d951-78a6-42e2-9934-36760cf2fa7a tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.244524 Sep 27 09:57:50.500071 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 224/444] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:57:50 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:50.508909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3c643d53-0377-4d38-8460-5c781bef462d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Calling method '.version_select at 0x7f7f63ef8b80>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.512226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c643d53-0377-4d38-8460-5c781bef462d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.513536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c643d53-0377-4d38-8460-5c781bef462d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.514036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c643d53-0377-4d38-8460-5c781bef462d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.534065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c643d53-0377-4d38-8460-5c781bef462d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.534627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c643d53-0377-4d38-8460-5c781bef462d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.535071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c643d53-0377-4d38-8460-5c781bef462d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.542257 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3c643d53-0377-4d38-8460-5c781bef462d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/1" status: 200 len: 758 microversion: 2.1 time: 0.039685 Sep 27 09:57:50.542925 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 221/445] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:57:50 2022] GET /compute/v2.1/os-hypervisors/1 => generated 758 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:50.559308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53d592e6-73fd-403a-92ac-37363dffb3e1 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Calling method '.version_select at 0x7f7f682705e0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.561052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53d592e6-73fd-403a-92ac-37363dffb3e1 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.561537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53d592e6-73fd-403a-92ac-37363dffb3e1 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.561977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53d592e6-73fd-403a-92ac-37363dffb3e1 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.580842 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53d592e6-73fd-403a-92ac-37363dffb3e1 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/statistics" status: 200 len: 267 microversion: 2.1 time: 0.025821 Sep 27 09:57:50.581457 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 225/446] 10.222.0.26 () {58 vars in 1179 bytes} [Tue Sep 27 09:57:50 2022] GET /compute/v2.1/os-hypervisors/statistics => generated 267 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:50.596149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d7a99957-4ce8-474c-95a4-3e07f51eb59b tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Calling method '.version_select at 0x7f7f63ff24c0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.597986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a99957-4ce8-474c-95a4-3e07f51eb59b tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.598486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a99957-4ce8-474c-95a4-3e07f51eb59b tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.598968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a99957-4ce8-474c-95a4-3e07f51eb59b tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.616701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a99957-4ce8-474c-95a4-3e07f51eb59b tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.617489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a99957-4ce8-474c-95a4-3e07f51eb59b tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.617949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a99957-4ce8-474c-95a4-3e07f51eb59b tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.622016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:50.632542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a99957-4ce8-474c-95a4-3e07f51eb59b tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.633055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a99957-4ce8-474c-95a4-3e07f51eb59b tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.633813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a99957-4ce8-474c-95a4-3e07f51eb59b tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.640969 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d7a99957-4ce8-474c-95a4-3e07f51eb59b tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.049466 Sep 27 09:57:50.641892 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 222/447] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:57:50 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:50.654464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b42a8c9d-c56f-475a-8eae-0af1c19d9ccd tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Calling method '.version_select at 0x7f7f682701f0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.656323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b42a8c9d-c56f-475a-8eae-0af1c19d9ccd tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.656841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b42a8c9d-c56f-475a-8eae-0af1c19d9ccd tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.657277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b42a8c9d-c56f-475a-8eae-0af1c19d9ccd tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.671678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b42a8c9d-c56f-475a-8eae-0af1c19d9ccd tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.672196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b42a8c9d-c56f-475a-8eae-0af1c19d9ccd tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.672638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b42a8c9d-c56f-475a-8eae-0af1c19d9ccd tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.686450 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b42a8c9d-c56f-475a-8eae-0af1c19d9ccd tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/1" status: 200 len: 758 microversion: 2.1 time: 0.037249 Sep 27 09:57:50.687065 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 226/448] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:57:50 2022] GET /compute/v2.1/os-hypervisors/1 => generated 758 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:50.703772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f4b05b38-ea3b-4036-b6ac-0206285f1c87 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Calling method '.version_select at 0x7f7f63ef8af0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.705642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b05b38-ea3b-4036-b6ac-0206285f1c87 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.706135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b05b38-ea3b-4036-b6ac-0206285f1c87 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.706608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b05b38-ea3b-4036-b6ac-0206285f1c87 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.726087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b05b38-ea3b-4036-b6ac-0206285f1c87 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.726497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b05b38-ea3b-4036-b6ac-0206285f1c87 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.727070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b05b38-ea3b-4036-b6ac-0206285f1c87 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.735265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b71c43ef-1f2b-4096-b5a6-1342ee6f4e61 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.736154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b71c43ef-1f2b-4096-b5a6-1342ee6f4e61 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:50.737172 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f4b05b38-ea3b-4036-b6ac-0206285f1c87 tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/2" status: 200 len: 758 microversion: 2.1 time: 0.037145 Sep 27 09:57:50.738025 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 223/449] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 09:57:50 2022] GET /compute/v2.1/os-hypervisors/2 => generated 758 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:50.751793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71c43ef-1f2b-4096-b5a6-1342ee6f4e61 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.752348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71c43ef-1f2b-4096-b5a6-1342ee6f4e61 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.753003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71c43ef-1f2b-4096-b5a6-1342ee6f4e61 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.753927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71c43ef-1f2b-4096-b5a6-1342ee6f4e61 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.754361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71c43ef-1f2b-4096-b5a6-1342ee6f4e61 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.754819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71c43ef-1f2b-4096-b5a6-1342ee6f4e61 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.756899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-49a82929-c873-46ee-8d91-b36d5a8be23d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Calling method '.version_select at 0x7f7f682cfee0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.760617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49a82929-c873-46ee-8d91-b36d5a8be23d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.761336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49a82929-c873-46ee-8d91-b36d5a8be23d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.762012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49a82929-c873-46ee-8d91-b36d5a8be23d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.786426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49a82929-c873-46ee-8d91-b36d5a8be23d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.786975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49a82929-c873-46ee-8d91-b36d5a8be23d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.788239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49a82929-c873-46ee-8d91-b36d5a8be23d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.805972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49a82929-c873-46ee-8d91-b36d5a8be23d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.807322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49a82929-c873-46ee-8d91-b36d5a8be23d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.808082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49a82929-c873-46ee-8d91-b36d5a8be23d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.857543 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-49a82929-c873-46ee-8d91-b36d5a8be23d tempest-HypervisorAdminTestJSON-83621435 tempest-HypervisorAdminTestJSON-83621435-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/1/uptime" status: 200 len: 175 microversion: 2.1 time: 0.104913 Sep 27 09:57:50.859269 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 224/450] 10.222.0.26 () {58 vars in 1173 bytes} [Tue Sep 27 09:57:50 2022] GET /compute/v2.1/os-hypervisors/1/uptime => generated 175 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:50.889956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5ca302fd-cf25-4740-97ec-77edce67d975 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:50.891180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5ca302fd-cf25-4740-97ec-77edce67d975 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:50.905372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ca302fd-cf25-4740-97ec-77edce67d975 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.906346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ca302fd-cf25-4740-97ec-77edce67d975 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.907921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ca302fd-cf25-4740-97ec-77edce67d975 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:50.909265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ca302fd-cf25-4740-97ec-77edce67d975 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:50.909998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ca302fd-cf25-4740-97ec-77edce67d975 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:50.910864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ca302fd-cf25-4740-97ec-77edce67d975 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.059020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b71c43ef-1f2b-4096-b5a6-1342ee6f4e61 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:51.072743 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b71c43ef-1f2b-4096-b5a6-1342ee6f4e61 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1319 microversion: 2.1 time: 0.344608 Sep 27 09:57:51.073548 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 227/451] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:50 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1319 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:51.080292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a128e4c-66af-4f49-a800-bd1cfc33e6a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:51.083107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3a128e4c-66af-4f49-a800-bd1cfc33e6a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:51.099844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a128e4c-66af-4f49-a800-bd1cfc33e6a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.100389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a128e4c-66af-4f49-a800-bd1cfc33e6a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.100930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a128e4c-66af-4f49-a800-bd1cfc33e6a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.109285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a128e4c-66af-4f49-a800-bd1cfc33e6a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.109990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a128e4c-66af-4f49-a800-bd1cfc33e6a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.110473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a128e4c-66af-4f49-a800-bd1cfc33e6a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.239911 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5ca302fd-cf25-4740-97ec-77edce67d975 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1631 microversion: 2.1 time: 0.355199 Sep 27 09:57:51.242022 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 225/452] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:50 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1631 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:51.248947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:51.262825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:51.283888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.284382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.284888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.286441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.287386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.288184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.363792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3a128e4c-66af-4f49-a800-bd1cfc33e6a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:51.377546 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3a128e4c-66af-4f49-a800-bd1cfc33e6a9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1469 microversion: 2.1 time: 0.301162 Sep 27 09:57:51.378376 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 228/453] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:51 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1469 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:51.393037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-38e5c03a-5b48-4cbd-bc4c-a64cc7868a04 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:51.393037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-38e5c03a-5b48-4cbd-bc4c-a64cc7868a04 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:51.408339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38e5c03a-5b48-4cbd-bc4c-a64cc7868a04 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.408927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38e5c03a-5b48-4cbd-bc4c-a64cc7868a04 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.410983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38e5c03a-5b48-4cbd-bc4c-a64cc7868a04 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.411742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38e5c03a-5b48-4cbd-bc4c-a64cc7868a04 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.412213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38e5c03a-5b48-4cbd-bc4c-a64cc7868a04 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.415414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38e5c03a-5b48-4cbd-bc4c-a64cc7868a04 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.419378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:51.440555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:51.444022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:51.459626 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32a44df0-2e12-41ab-875e-4e43ef04a89f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1437 microversion: 2.75 time: 0.209129 Sep 27 09:57:51.461617 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 226/454] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:57:51 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1437 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:51.467187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1bf42196-3f95-44b6-bc2c-cd09c0aadfe6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:51.468123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1bf42196-3f95-44b6-bc2c-cd09c0aadfe6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:51.478450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bf42196-3f95-44b6-bc2c-cd09c0aadfe6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.480139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bf42196-3f95-44b6-bc2c-cd09c0aadfe6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.480608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bf42196-3f95-44b6-bc2c-cd09c0aadfe6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.481387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bf42196-3f95-44b6-bc2c-cd09c0aadfe6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.481851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bf42196-3f95-44b6-bc2c-cd09c0aadfe6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.485185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bf42196-3f95-44b6-bc2c-cd09c0aadfe6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.739257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-38e5c03a-5b48-4cbd-bc4c-a64cc7868a04 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:51.755572 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-38e5c03a-5b48-4cbd-bc4c-a64cc7868a04 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1472 microversion: 2.1 time: 0.375859 Sep 27 09:57:51.771202 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 229/455] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:51 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1472 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:51.783553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d03302b0-353b-4a28-b969-1b4ab261cbd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:51.785532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d03302b0-353b-4a28-b969-1b4ab261cbd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 0e538bab-a9f3-488f-9bef-801f2429f73f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:51.797604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d03302b0-353b-4a28-b969-1b4ab261cbd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.801267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d03302b0-353b-4a28-b969-1b4ab261cbd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.802057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d03302b0-353b-4a28-b969-1b4ab261cbd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.803319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d03302b0-353b-4a28-b969-1b4ab261cbd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.803940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d03302b0-353b-4a28-b969-1b4ab261cbd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.805615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d03302b0-353b-4a28-b969-1b4ab261cbd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.861674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1bf42196-3f95-44b6-bc2c-cd09c0aadfe6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:51.870937 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1bf42196-3f95-44b6-bc2c-cd09c0aadfe6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1319 microversion: 2.1 time: 0.407243 Sep 27 09:57:51.871213 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 227/456] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:51 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1319 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:51.876611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:51.879504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:51.886949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.887841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.888587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:51.889530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:51.889993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:51.890428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.021611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:57:52.022135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:57:52.022691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:57:52.023169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:57:52.024016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:57:52.037804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d03302b0-353b-4a28-b969-1b4ab261cbd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:52.052396 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d03302b0-353b-4a28-b969-1b4ab261cbd5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f" status: 200 len: 1468 microversion: 2.1 time: 0.279708 Sep 27 09:57:52.055131 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 230/457] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:51 2022] GET /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f => generated 1468 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:52.079006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:52.086050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:52.087396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Searching by: {'reservation_id': 'r-8yw1lks1', 'deleted': False, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 09:57:52.095763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.096551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.097232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.098270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.098921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.099578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.173026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:52.206647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.207233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.207826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.208828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.209460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.213203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.327714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:52.333368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 09:57:52.340079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:52.342154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:52.350121 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7142b3df-84d5-4f83-aa7d-1cdbf4c75cbf tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "POST /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb/action" status: 202 len: 1494 microversion: 2.75 time: 0.478008 Sep 27 09:57:52.362159 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 228/458] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 09:57:51 2022] POST /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb/action => generated 1494 bytes in 490 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Sep 27 09:57:52.383628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80578b74-4685-4be6-86e2-e97473131587 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:52.383962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-80578b74-4685-4be6-86e2-e97473131587 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:52.391453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80578b74-4685-4be6-86e2-e97473131587 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.392172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80578b74-4685-4be6-86e2-e97473131587 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.392901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80578b74-4685-4be6-86e2-e97473131587 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.393799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80578b74-4685-4be6-86e2-e97473131587 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.394552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80578b74-4685-4be6-86e2-e97473131587 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.395102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80578b74-4685-4be6-86e2-e97473131587 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.484139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:52.492061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.492839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.493439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.662845 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9da30d94-191f-4ea3-844e-5325820f7235 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/detail?reservation_id=r-8yw1lks1" status: 200 len: 2930 microversion: 2.1 time: 0.587837 Sep 27 09:57:52.663775 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 231/459] 10.222.0.26 () {58 vars in 1197 bytes} [Tue Sep 27 09:57:52 2022] GET /compute/v2.1/servers/detail?reservation_id=r-8yw1lks1 => generated 2930 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:52.669396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ffef9607-adf0-4be7-b6bd-dcc8cc45505c tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:52.670034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ffef9607-adf0-4be7-b6bd-dcc8cc45505c tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:52.679319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffef9607-adf0-4be7-b6bd-dcc8cc45505c tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.680176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffef9607-adf0-4be7-b6bd-dcc8cc45505c tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.681068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffef9607-adf0-4be7-b6bd-dcc8cc45505c tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.681884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffef9607-adf0-4be7-b6bd-dcc8cc45505c tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.682609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffef9607-adf0-4be7-b6bd-dcc8cc45505c tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.683190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffef9607-adf0-4be7-b6bd-dcc8cc45505c tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.704723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:52.862967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-80578b74-4685-4be6-86e2-e97473131587 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:52.876065 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-80578b74-4685-4be6-86e2-e97473131587 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1319 microversion: 2.1 time: 0.505356 Sep 27 09:57:52.890245 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 229/460] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:52 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1319 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:52.903627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:52.905443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:52.913387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.914498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.914947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.915833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.916386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.916899 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ffef9607-adf0-4be7-b6bd-dcc8cc45505c tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1614 microversion: 2.1 time: 0.251609 Sep 27 09:57:52.917041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:52.917585 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 232/461] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:52 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1614 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:52.939532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0cf9a251-8d39-47e1-8e09-54f609de49b9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:52.981907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.compute.server_groups [None req-0cf9a251-8d39-47e1-8e09-54f609de49b9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Querying cell ed1088da-caae-4426-8048-9cba90e7755c(cell1) for 2 instances {{(pid=145968) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Sep 27 09:57:52.982695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0cf9a251-8d39-47e1-8e09-54f609de49b9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:52.983201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0cf9a251-8d39-47e1-8e09-54f609de49b9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:52.983632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0cf9a251-8d39-47e1-8e09-54f609de49b9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.019055 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0cf9a251-8d39-47e1-8e09-54f609de49b9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/os-server-groups/ba6345ef-9480-4c71-b5a2-0c2e1d6550fb" status: 200 len: 265 microversion: 2.1 time: 0.084036 Sep 27 09:57:53.026553 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 233/462] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 09:57:52 2022] GET /compute/v2.1/os-server-groups/ba6345ef-9480-4c71-b5a2-0c2e1d6550fb => generated 265 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:53.026553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af4c815e-644d-408f-adef-beb1da649513 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:53.026553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af4c815e-644d-408f-adef-beb1da649513 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:53.033323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4c815e-644d-408f-adef-beb1da649513 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.033920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4c815e-644d-408f-adef-beb1da649513 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.034428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4c815e-644d-408f-adef-beb1da649513 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.035320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4c815e-644d-408f-adef-beb1da649513 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.035719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4c815e-644d-408f-adef-beb1da649513 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.036176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4c815e-644d-408f-adef-beb1da649513 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.064832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:53.076382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:53.077949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:53.083757 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e225aa17-4994-4cb0-8259-42953c9c1838 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.191330 Sep 27 09:57:53.085057 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 230/463] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:57:52 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:53.095313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af0053a5-075b-400b-839e-5e77280851c2 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:53.096360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-af0053a5-075b-400b-839e-5e77280851c2 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:53.104586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af0053a5-075b-400b-839e-5e77280851c2 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.105499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af0053a5-075b-400b-839e-5e77280851c2 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.106213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af0053a5-075b-400b-839e-5e77280851c2 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.107391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af0053a5-075b-400b-839e-5e77280851c2 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.108820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af0053a5-075b-400b-839e-5e77280851c2 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.109521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af0053a5-075b-400b-839e-5e77280851c2 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.201695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af4c815e-644d-408f-adef-beb1da649513 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:53.218281 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af4c815e-644d-408f-adef-beb1da649513 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1472 microversion: 2.1 time: 0.197162 Sep 27 09:57:53.218915 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 234/464] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:53 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1472 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:53.224627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:53.225735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:53.238148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.239113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.239660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.240543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.241214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.241788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.273842 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] HTTP exception thrown: Cannot 'migrate' instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 while it is in vm_state suspended Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Request method failure captured: Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7/action HTTP/1.1 Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 15 Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjmOS39Kb4EOG3AEPcCu2xO0uwyRar2-Cyy_T7I1JH7k-PcwJiU0fMYg8Ar4Mg97Sm4-Dg0c7iDeiwLr2h39DInGmz_mpOiq0MW-es6oWKiXfQe-up3SLSro4Gz4XmJgrNEpO_GHOtaEblKUnHqj-DDtujaiB5ib-gntLvjEFbsIlE Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 06e7347f7bbc45ce8de8e29de5255fa9 Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:57:53.276320 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_06e7347f7bbc45ce8de8e29de5255fa9"}]}] Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 06e7347f7bbc45ce8de8e29de5255fa9 Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersAdminNegativeTestJSON-622394730-project-admin Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: f1c3ed62097a4e89bf360a966c764762 Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersAdminNegativeTestJSON-622394730-project-admin Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: {"migrate": {}} Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:57:53.278626 ncdv-master-1 devstack@n-api.service[145968]: exception: Cannot 'migrate' instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 while it is in vm_state suspended Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'ea27a3db-7b84-4c05-b57f-77b3e6b397f7', 'body': {'migrate': {}}} Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 58, in _migrate Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.resize(req.environ['nova.context'], instance, Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 in vm_state suspended. Cannot resize while the instance is in this state. Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.280142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 69, in _migrate Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'migrate' instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 while it is in vm_state suspended Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:53.282431 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:53.284303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:53.284303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:53.284303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'migrate' instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 while it is in vm_state suspended Sep 27 09:57:53.284303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.284303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Returning 409 to user: Cannot 'migrate' instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 while it is in vm_state suspended {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:53.284303 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-251470a4-4d3b-42d6-b750-a2cde75075fb tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7/action" status: 409 len: 148 microversion: 2.1 time: 0.060143 Sep 27 09:57:53.284303 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 235/465] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:57:53 2022] POST /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7/action => generated 148 bytes in 63 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:57:53.290795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa6d24ba-5cd9-49c5-9c45-b018758de2c4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:53.291694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fa6d24ba-5cd9-49c5-9c45-b018758de2c4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] [instance: 0e538bab-a9f3-488f-9bef-801f2429f73f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:53.306471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa6d24ba-5cd9-49c5-9c45-b018758de2c4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.307065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa6d24ba-5cd9-49c5-9c45-b018758de2c4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.307768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa6d24ba-5cd9-49c5-9c45-b018758de2c4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.308757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa6d24ba-5cd9-49c5-9c45-b018758de2c4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.309417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa6d24ba-5cd9-49c5-9c45-b018758de2c4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.310019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa6d24ba-5cd9-49c5-9c45-b018758de2c4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.494322 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fa6d24ba-5cd9-49c5-9c45-b018758de2c4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f" status: 200 len: 1619 microversion: 2.1 time: 0.209084 Sep 27 09:57:53.494908 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 236/466] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:53 2022] GET /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f => generated 1619 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:53.499565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f7a8587b-1fb5-4f18-b001-e3579db7ba5d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "invalid"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f7a8587b-1fb5-4f18-b001-e3579db7ba5d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Request method failure captured: Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d/action HTTP/1.1 Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 39 Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjmOS39Kb4EOG3AEPcCu2xO0uwyRar2-Cyy_T7I1JH7k-PcwJiU0fMYg8Ar4Mg97Sm4-Dg0c7iDeiwLr2h39DInGmz_mpOiq0MW-es6oWKiXfQe-up3SLSro4Gz4XmJgrNEpO_GHOtaEblKUnHqj-DDtujaiB5ib-gntLvjEFbsIlE Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 06e7347f7bbc45ce8de8e29de5255fa9 Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:57:53.504620 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_06e7347f7bbc45ce8de8e29de5255fa9"}]}] Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 06e7347f7bbc45ce8de8e29de5255fa9 Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersAdminNegativeTestJSON-622394730-project-admin Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: f1c3ed62097a4e89bf360a966c764762 Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersAdminNegativeTestJSON-622394730-project-admin Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: {"os-resetState": {"state": "invalid"}} Sep 27 09:57:53.506187 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'ee40f956-0a40-43c4-a775-a978e8403d1d', 'body': {'os-resetState': {'state': 'invalid'}}} Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['active', 'error'] Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['os-resetState']['properties']['state']: Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'enum': ['active', 'error'], 'type': 'string'} Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['os-resetState']: Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'invalid' Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:53.506991 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 09:57:53.508516 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] Sep 27 09:57:53.510023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.510897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f7a8587b-1fb5-4f18-b001-e3579db7ba5d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Returning 400 to user: Invalid input for field/attribute state. Value: invalid. 'invalid' is not one of ['active', 'error'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:53.512285 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f7a8587b-1fb5-4f18-b001-e3579db7ba5d tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d/action" status: 400 len: 144 microversion: 2.1 time: 0.016438 Sep 27 09:57:53.513519 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 237/467] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:57:53 2022] POST /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d/action => generated 144 bytes in 18 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 09:57:53.524621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-644b92ea-d2a4-4321-ad36-77c1e06f927b tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:53.525598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-644b92ea-d2a4-4321-ad36-77c1e06f927b tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:53.532874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-644b92ea-d2a4-4321-ad36-77c1e06f927b tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.533586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-644b92ea-d2a4-4321-ad36-77c1e06f927b tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.534175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-644b92ea-d2a4-4321-ad36-77c1e06f927b tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.535082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-644b92ea-d2a4-4321-ad36-77c1e06f927b tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.535693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-644b92ea-d2a4-4321-ad36-77c1e06f927b tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.536271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-644b92ea-d2a4-4321-ad36-77c1e06f927b tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.670063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af0053a5-075b-400b-839e-5e77280851c2 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:53.682145 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af0053a5-075b-400b-839e-5e77280851c2 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1319 microversion: 2.1 time: 0.595613 Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 231/468] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:53 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1319 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-70e0de86-a6b7-45b4-98f2-d5eee4502488 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": 1}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-70e0de86-a6b7-45b4-98f2-d5eee4502488 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Request method failure captured: Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d/action HTTP/1.1 Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 31 Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjmOS39Kb4EOG3AEPcCu2xO0uwyRar2-Cyy_T7I1JH7k-PcwJiU0fMYg8Ar4Mg97Sm4-Dg0c7iDeiwLr2h39DInGmz_mpOiq0MW-es6oWKiXfQe-up3SLSro4Gz4XmJgrNEpO_GHOtaEblKUnHqj-DDtujaiB5ib-gntLvjEFbsIlE Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 06e7347f7bbc45ce8de8e29de5255fa9 Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:57:53.696725 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_06e7347f7bbc45ce8de8e29de5255fa9"}]}] Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 06e7347f7bbc45ce8de8e29de5255fa9 Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersAdminNegativeTestJSON-622394730-project-admin Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: f1c3ed62097a4e89bf360a966c764762 Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersAdminNegativeTestJSON-622394730-project-admin Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: {"os-resetState": {"state": 1}} Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:57:53.699131 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'ee40f956-0a40-43c4-a775-a978e8403d1d', 'body': {'os-resetState': {'state': 1}}} Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 1 is not of type 'string' Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['os-resetState']['properties']['state']: Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'enum': ['active', 'error'], 'type': 'string'} Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['os-resetState']: Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 1 Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:53.700007 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' Sep 27 09:57:53.703680 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.705662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-70e0de86-a6b7-45b4-98f2-d5eee4502488 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Returning 400 to user: Invalid input for field/attribute state. Value: 1. 1 is not of type 'string' {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:53.705662 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-70e0de86-a6b7-45b4-98f2-d5eee4502488 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d/action" status: 400 len: 120 microversion: 2.1 time: 0.011097 Sep 27 09:57:53.705662 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 232/469] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:57:53 2022] POST /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d/action => generated 120 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 09:57:53.708568 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-644b92ea-d2a4-4321-ad36-77c1e06f927b tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 200 len: 1620 microversion: 2.1 time: 0.188038 Sep 27 09:57:53.716699 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 238/470] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:53 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 1620 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4ce2c92d-4fb2-44d5-9137-ff04e808afa0 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4ce2c92d-4fb2-44d5-9137-ff04e808afa0 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Failed to fetch instance by id 999 {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2981}} Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-4ce2c92d-4fb2-44d5-9137-ff04e808afa0 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] HTTP exception thrown: Instance 999 could not be found. Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4ce2c92d-4fb2-44d5-9137-ff04e808afa0 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Request method failure captured: Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/999/action HTTP/1.1 Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 37 Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjmOS39Kb4EOG3AEPcCu2xO0uwyRar2-Cyy_T7I1JH7k-PcwJiU0fMYg8Ar4Mg97Sm4-Dg0c7iDeiwLr2h39DInGmz_mpOiq0MW-es6oWKiXfQe-up3SLSro4Gz4XmJgrNEpO_GHOtaEblKUnHqj-DDtujaiB5ib-gntLvjEFbsIlE Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:57:53.717072 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 06e7347f7bbc45ce8de8e29de5255fa9 Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/06e7347f7bbc45ce8de8e29de5255fa9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_06e7347f7bbc45ce8de8e29de5255fa9"}]}] Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 06e7347f7bbc45ce8de8e29de5255fa9 Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-622394730 Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersAdminNegativeTestJSON-622394730-project-admin Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: f1c3ed62097a4e89bf360a966c764762 Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersAdminNegativeTestJSON-622394730-project-admin Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:57:53.718477 ncdv-master-1 devstack@n-api.service[145969]: {"os-resetState": {"state": "error"}} Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 999 could not be found. Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '999', 'body': {'os-resetState': {'state': 'error'}}} Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 999 could not be found. Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:57:53.719467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/admin_actions.py", line 66, in _reset_state Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 999 could not be found. Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 999 could not be found. Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:57:53.722731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4ce2c92d-4fb2-44d5-9137-ff04e808afa0 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] Returning 404 to user: Instance 999 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:57:53.723914 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4ce2c92d-4fb2-44d5-9137-ff04e808afa0 tempest-ServersAdminNegativeTestJSON-622394730 tempest-ServersAdminNegativeTestJSON-622394730-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/999/action" status: 404 len: 78 microversion: 2.1 time: 0.013329 Sep 27 09:57:53.725052 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 233/471] 10.222.0.26 () {60 vars in 1179 bytes} [Tue Sep 27 09:57:53 2022] POST /compute/v2.1/servers/999/action => generated 78 bytes in 21 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:57:53.730210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-35e580ce-b6f7-4174-bf51-0ae22b965bff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:53.730834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-35e580ce-b6f7-4174-bf51-0ae22b965bff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:53.738340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35e580ce-b6f7-4174-bf51-0ae22b965bff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.738950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35e580ce-b6f7-4174-bf51-0ae22b965bff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.739484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35e580ce-b6f7-4174-bf51-0ae22b965bff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.740292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35e580ce-b6f7-4174-bf51-0ae22b965bff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.740881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35e580ce-b6f7-4174-bf51-0ae22b965bff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.741435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35e580ce-b6f7-4174-bf51-0ae22b965bff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.744160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-19027420-f483-4d0f-8445-13e8f9b82432 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Action: 'create', calling method: .version_select at 0x7f7f63dab280>, body: {"server_group": {"name": "tempest-ServersOnMultiNodesTest-Server-Group-1537478748", "policies": ["anti-affinity"]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:53.772518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-19027420-f483-4d0f-8445-13e8f9b82432 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:53.778536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-35e580ce-b6f7-4174-bf51-0ae22b965bff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:57:53.779629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-19027420-f483-4d0f-8445-13e8f9b82432 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:53.856840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-19027420-f483-4d0f-8445-13e8f9b82432 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:53.877999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-19027420-f483-4d0f-8445-13e8f9b82432 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:53.878861 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-35e580ce-b6f7-4174-bf51-0ae22b965bff tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "DELETE /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 204 len: 0 microversion: 2.1 time: 0.151914 Sep 27 09:57:53.879853 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 239/472] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:57:53 2022] DELETE /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:57:53.889470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-804aa19a-e14c-49e7-b3b5-564a5da895f4 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:53.890358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-804aa19a-e14c-49e7-b3b5-564a5da895f4 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:53.899376 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-19027420-f483-4d0f-8445-13e8f9b82432 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "POST /compute/v2.1/os-server-groups" status: 200 len: 193 microversion: 2.1 time: 0.165686 Sep 27 09:57:53.901190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-804aa19a-e14c-49e7-b3b5-564a5da895f4 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.901701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-804aa19a-e14c-49e7-b3b5-564a5da895f4 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.902288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-804aa19a-e14c-49e7-b3b5-564a5da895f4 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.903026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-804aa19a-e14c-49e7-b3b5-564a5da895f4 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.903614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-804aa19a-e14c-49e7-b3b5-564a5da895f4 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.904043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-804aa19a-e14c-49e7-b3b5-564a5da895f4 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.912158 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 234/473] 10.222.0.26 () {60 vars in 1174 bytes} [Tue Sep 27 09:57:53 2022] POST /compute/v2.1/os-server-groups => generated 193 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:57:53.918194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c962578d-8cb7-4b35-9840-fe9d3923e56d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:53.919177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c962578d-8cb7-4b35-9840-fe9d3923e56d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:53.927623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c962578d-8cb7-4b35-9840-fe9d3923e56d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.928691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c962578d-8cb7-4b35-9840-fe9d3923e56d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.929371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c962578d-8cb7-4b35-9840-fe9d3923e56d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:53.930504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c962578d-8cb7-4b35-9840-fe9d3923e56d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:53.931498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c962578d-8cb7-4b35-9840-fe9d3923e56d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:53.933088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c962578d-8cb7-4b35-9840-fe9d3923e56d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:54.085203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-804aa19a-e14c-49e7-b3b5-564a5da895f4 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:54.114009 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-804aa19a-e14c-49e7-b3b5-564a5da895f4 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1469 microversion: 2.1 time: 0.229290 Sep 27 09:57:54.114609 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 240/474] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:53 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1469 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:54.120326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:54.120977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:54.134569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:54.134955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:54.135432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:54.136301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:54.136910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:54.137435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:54.153007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c962578d-8cb7-4b35-9840-fe9d3923e56d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:54.169326 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c962578d-8cb7-4b35-9840-fe9d3923e56d tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1319 microversion: 2.1 time: 0.253482 Sep 27 09:57:54.172601 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 235/475] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:53 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1319 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:54.180836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-1787115070", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "min_count": 2, "return_reservation_id": true}, "os:scheduler_hints": {"group": "86f44e48-288d-42e4-91ee-4d99b6fa5532"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:54.265243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:57:54.512863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:54.525138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:54.527879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:54.542709 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-074b252a-8ebc-469c-a4fc-8046d4256679 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.426193 Sep 27 09:57:54.544243 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 241/476] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:57:54 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:54.563149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-10b99746-a742-4028-bd4f-877657a22664 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:54.569557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-10b99746-a742-4028-bd4f-877657a22664 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:54.577076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10b99746-a742-4028-bd4f-877657a22664 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:54.577572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10b99746-a742-4028-bd4f-877657a22664 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:54.578192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10b99746-a742-4028-bd4f-877657a22664 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:54.579874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10b99746-a742-4028-bd4f-877657a22664 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:54.580872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10b99746-a742-4028-bd4f-877657a22664 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:54.581310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10b99746-a742-4028-bd4f-877657a22664 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:54.586232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:57:54.587675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:57:54.588778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:57:54.589293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:57:54.589780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:57:54.599723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=98,nova_api:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:54.614298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:54.614907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:54.615363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:54.616204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:54.616729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:54.617113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:54.647753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:54.654319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:54.675194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Going to run 2 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:57:54.678642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: f769af8b-472a-4950-b4ae-8c0183b23c1e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:57:54.748979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:54.756963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:54.765002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-10b99746-a742-4028-bd4f-877657a22664 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:54.774845 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-10b99746-a742-4028-bd4f-877657a22664 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.216363 Sep 27 09:57:54.775589 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 242/477] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:54 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:54.781797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-429110b0-b7ac-4019-8ad2-c5f264c40c89 req-67fc069f-109a-43a9-b9c2-8a5324617a96 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc", "name": "network-vif-plugged", "status": "completed", "tag": "9bd9c08f-8f18-447f-9137-e5d05f5c4f09"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:54.794386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-429110b0-b7ac-4019-8ad2-c5f264c40c89 req-67fc069f-109a-43a9-b9c2-8a5324617a96 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:54.794940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-429110b0-b7ac-4019-8ad2-c5f264c40c89 req-67fc069f-109a-43a9-b9c2-8a5324617a96 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:54.799889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-429110b0-b7ac-4019-8ad2-c5f264c40c89 req-67fc069f-109a-43a9-b9c2-8a5324617a96 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:54.815788 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-429110b0-b7ac-4019-8ad2-c5f264c40c89 req-67fc069f-109a-43a9-b9c2-8a5324617a96 service nova] Creating event network-vif-plugged:9bd9c08f-8f18-447f-9137-e5d05f5c4f09 for instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc on nch2-master-1 Sep 27 09:57:54.823510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:54.824451 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-429110b0-b7ac-4019-8ad2-c5f264c40c89 req-67fc069f-109a-43a9-b9c2-8a5324617a96 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047105 Sep 27 09:57:54.825105 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 243/478] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:54.834991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8823c031-4c31-4e7b-bc89-4bb350905d14 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:54.835860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8823c031-4c31-4e7b-bc89-4bb350905d14 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:54.836740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:54.843336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8823c031-4c31-4e7b-bc89-4bb350905d14 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:54.844863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8823c031-4c31-4e7b-bc89-4bb350905d14 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:54.845028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8823c031-4c31-4e7b-bc89-4bb350905d14 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:54.849512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8823c031-4c31-4e7b-bc89-4bb350905d14 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:54.849512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8823c031-4c31-4e7b-bc89-4bb350905d14 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:54.849512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8823c031-4c31-4e7b-bc89-4bb350905d14 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:54.852902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=f769af8b-472a-4950-b4ae-8c0183b23c1e,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:57:54.911832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:54.917487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:54.962724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:57:54.969004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:57:55.010855 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-03428d3d-9513-4fe9-a026-7fcd3a9f7048 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 32 microversion: 2.1 time: 0.836292 Sep 27 09:57:55.011825 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 236/479] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:57:54 2022] POST /compute/v2.1/servers => generated 32 bytes in 838 msecs (HTTP/1.1 202) 9 headers in 362 bytes (1 switches on core 0) Sep 27 09:57:55.022474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c17a61f0-b35a-4b28-b5d0-cda912a457fe tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:55.035713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c17a61f0-b35a-4b28-b5d0-cda912a457fe tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:55.041696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c17a61f0-b35a-4b28-b5d0-cda912a457fe tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Searching by: {'deleted': False, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 09:57:55.059875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8823c031-4c31-4e7b-bc89-4bb350905d14 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:55.070700 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8823c031-4c31-4e7b-bc89-4bb350905d14 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1319 microversion: 2.1 time: 0.241084 Sep 27 09:57:55.080468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c17a61f0-b35a-4b28-b5d0-cda912a457fe tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.080468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c17a61f0-b35a-4b28-b5d0-cda912a457fe tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.085749 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 244/480] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:54 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1319 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:55.088079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c17a61f0-b35a-4b28-b5d0-cda912a457fe tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.092734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c17a61f0-b35a-4b28-b5d0-cda912a457fe tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.093568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c17a61f0-b35a-4b28-b5d0-cda912a457fe tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.093876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c17a61f0-b35a-4b28-b5d0-cda912a457fe tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.139481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3024ead8-b70d-43dd-8eb9-0e9e9d9ca33e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:55.140318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3024ead8-b70d-43dd-8eb9-0e9e9d9ca33e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:55.151577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3024ead8-b70d-43dd-8eb9-0e9e9d9ca33e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.153554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3024ead8-b70d-43dd-8eb9-0e9e9d9ca33e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.154831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3024ead8-b70d-43dd-8eb9-0e9e9d9ca33e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.155684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3024ead8-b70d-43dd-8eb9-0e9e9d9ca33e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.157097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3024ead8-b70d-43dd-8eb9-0e9e9d9ca33e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.162740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3024ead8-b70d-43dd-8eb9-0e9e9d9ca33e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.209783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-c17a61f0-b35a-4b28-b5d0-cda912a457fe tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Listed batch of 3 results from cell out of 998 limit. Returned 3 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 09:57:55.336976 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c17a61f0-b35a-4b28-b5d0-cda912a457fe tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 1666 microversion: 2.1 time: 0.318839 Sep 27 09:57:55.336976 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 237/481] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 09:57:55 2022] GET /compute/v2.1/servers => generated 1666 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:55.348753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6c3c6f6f-bcd0-4a88-a976-e161af08b342 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:55.349507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6c3c6f6f-bcd0-4a88-a976-e161af08b342 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:55.357084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3c6f6f-bcd0-4a88-a976-e161af08b342 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.357674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3c6f6f-bcd0-4a88-a976-e161af08b342 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.358192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3c6f6f-bcd0-4a88-a976-e161af08b342 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.358993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3c6f6f-bcd0-4a88-a976-e161af08b342 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.360798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3c6f6f-bcd0-4a88-a976-e161af08b342 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.361458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3c6f6f-bcd0-4a88-a976-e161af08b342 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.365019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3024ead8-b70d-43dd-8eb9-0e9e9d9ca33e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:55.373944 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3024ead8-b70d-43dd-8eb9-0e9e9d9ca33e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1469 microversion: 2.1 time: 0.238816 Sep 27 09:57:55.388433 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 245/482] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:55 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1469 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:55.397188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ab3fa428-7adb-4156-8c31-82b43f4d7379 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:55.398372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ab3fa428-7adb-4156-8c31-82b43f4d7379 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:55.458979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ab3fa428-7adb-4156-8c31-82b43f4d7379 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:55.472736 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ab3fa428-7adb-4156-8c31-82b43f4d7379 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1219 microversion: 2.1 time: 0.082424 Sep 27 09:57:55.477808 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 246/483] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:55 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1219 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:55.567832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6c3c6f6f-bcd0-4a88-a976-e161af08b342 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:55.590955 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6c3c6f6f-bcd0-4a88-a976-e161af08b342 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1319 microversion: 2.1 time: 0.250564 Sep 27 09:57:55.591957 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 238/484] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:55 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1319 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:55.600251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:55.601425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:55.626608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.627363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.628213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.629405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.630098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.630720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.689324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Calling method '.version_select at 0x7f7f68216700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:55.694060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.694545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.694990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.709895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.710379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.710915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.720946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:55.731844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:55.733349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:55.738838 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-69771292-3fab-4841-a862-c02d2726b676 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.145800 Sep 27 09:57:55.739432 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 239/485] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:57:55 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:55.744890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.745388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.745830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.756943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.757687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.758194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.789264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.789795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.790236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.799537 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-66b03dde-4ab8-48e2-81b3-9f86221ab9b9 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/detail?with_servers=True" status: 200 len: 2388 microversion: 2.53 time: 0.306823 Sep 27 09:57:55.800616 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 247/486] 10.222.0.26 () {60 vars in 1243 bytes} [Tue Sep 27 09:57:55 2022] GET /compute/v2.1/os-hypervisors/detail?with_servers=True => generated 2388 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:55.800768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-966dab3d-79cd-4810-a607-d527615b6554 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:55.813787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-966dab3d-79cd-4810-a607-d527615b6554 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:55.818312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Calling method '.version_select at 0x7f7f63f36820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:55.820867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-966dab3d-79cd-4810-a607-d527615b6554 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.821523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-966dab3d-79cd-4810-a607-d527615b6554 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.821966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-966dab3d-79cd-4810-a607-d527615b6554 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.822228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.822684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-966dab3d-79cd-4810-a607-d527615b6554 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.822807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.823135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-966dab3d-79cd-4810-a607-d527615b6554 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.823244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.823557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-966dab3d-79cd-4810-a607-d527615b6554 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.840523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.841303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.841958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.846078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:55.887383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.888150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.889000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.905052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.905852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.906771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.911328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:55.947000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.947466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.947872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.955157 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-93fefc8c-5c51-4077-84b7-036250b43ebd tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors?with_servers=True" status: 200 len: 1081 microversion: 2.53 time: 0.140526 Sep 27 09:57:55.955753 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 248/487] 10.222.0.26 () {60 vars in 1222 bytes} [Tue Sep 27 09:57:55 2022] GET /compute/v2.1/os-hypervisors?with_servers=True => generated 1081 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:55.969667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-85025b91-b148-4102-b92c-b0b73c72dac3 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Calling method '.version_select at 0x7f7f68052d30>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:55.973638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85025b91-b148-4102-b92c-b0b73c72dac3 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.974581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85025b91-b148-4102-b92c-b0b73c72dac3 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:55.975268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85025b91-b148-4102-b92c-b0b73c72dac3 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:55.998623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85025b91-b148-4102-b92c-b0b73c72dac3 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:55.999874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85025b91-b148-4102-b92c-b0b73c72dac3 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:56.000574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85025b91-b148-4102-b92c-b0b73c72dac3 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:56.009882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-966dab3d-79cd-4810-a607-d527615b6554 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:56.011410 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-85025b91-b148-4102-b92c-b0b73c72dac3 tempest-HypervisorAdminV253TestBase-640324434 tempest-HypervisorAdminV253TestBase-640324434-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors/eb6179c5-d095-4b3b-8878-a5e743ce9d07" status: 200 len: 792 microversion: 2.53 time: 0.045644 Sep 27 09:57:56.012215 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 249/488] 10.222.0.26 () {60 vars in 1298 bytes} [Tue Sep 27 09:57:55 2022] GET /compute/v2.1/os-hypervisors/eb6179c5-d095-4b3b-8878-a5e743ce9d07 => generated 792 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 09:57:56.027188 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-966dab3d-79cd-4810-a607-d527615b6554 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.230294 Sep 27 09:57:56.027866 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 240/489] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:55 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:56.098860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-076c115f-3f45-44a2-9bf2-6c46987cedcd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:56.099603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-076c115f-3f45-44a2-9bf2-6c46987cedcd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:56.103061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:57:56.107274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-076c115f-3f45-44a2-9bf2-6c46987cedcd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:56.107949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-076c115f-3f45-44a2-9bf2-6c46987cedcd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:56.108393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-076c115f-3f45-44a2-9bf2-6c46987cedcd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:56.109187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-076c115f-3f45-44a2-9bf2-6c46987cedcd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:56.109626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-076c115f-3f45-44a2-9bf2-6c46987cedcd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:56.110080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-076c115f-3f45-44a2-9bf2-6c46987cedcd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:56.276698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-076c115f-3f45-44a2-9bf2-6c46987cedcd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:56.291886 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-076c115f-3f45-44a2-9bf2-6c46987cedcd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1319 microversion: 2.1 time: 0.196218 Sep 27 09:57:56.293260 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 250/490] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:56 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1319 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:56.405709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3dd62dbc-c037-4402-99c0-b8af637a7408 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:56.406471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3dd62dbc-c037-4402-99c0-b8af637a7408 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:56.419126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dd62dbc-c037-4402-99c0-b8af637a7408 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:56.419803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dd62dbc-c037-4402-99c0-b8af637a7408 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:56.421127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dd62dbc-c037-4402-99c0-b8af637a7408 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:56.422867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dd62dbc-c037-4402-99c0-b8af637a7408 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:56.423481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dd62dbc-c037-4402-99c0-b8af637a7408 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:56.424169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dd62dbc-c037-4402-99c0-b8af637a7408 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:56.497007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6bd9c6f8-3c8f-4d49-bc10-c33be06dfbf0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:56.497627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6bd9c6f8-3c8f-4d49-bc10-c33be06dfbf0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:56.503813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bd9c6f8-3c8f-4d49-bc10-c33be06dfbf0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:56.504299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bd9c6f8-3c8f-4d49-bc10-c33be06dfbf0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:56.504770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bd9c6f8-3c8f-4d49-bc10-c33be06dfbf0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:56.505657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bd9c6f8-3c8f-4d49-bc10-c33be06dfbf0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:56.506274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bd9c6f8-3c8f-4d49-bc10-c33be06dfbf0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:56.506698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bd9c6f8-3c8f-4d49-bc10-c33be06dfbf0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:56.575420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3dd62dbc-c037-4402-99c0-b8af637a7408 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:56.591362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6bd9c6f8-3c8f-4d49-bc10-c33be06dfbf0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:56.592397 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3dd62dbc-c037-4402-99c0-b8af637a7408 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1469 microversion: 2.1 time: 0.191050 Sep 27 09:57:56.593580 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 241/491] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:56 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1469 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:56.598995 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6bd9c6f8-3c8f-4d49-bc10-c33be06dfbf0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1223 microversion: 2.1 time: 0.105802 Sep 27 09:57:56.604859 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 251/492] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:56 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1223 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:57:56.629028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8eee5d46-ae83-4a09-975f-2c80f5d44733 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:56.629639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8eee5d46-ae83-4a09-975f-2c80f5d44733 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:56.635984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8eee5d46-ae83-4a09-975f-2c80f5d44733 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:56.636469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8eee5d46-ae83-4a09-975f-2c80f5d44733 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:56.636978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8eee5d46-ae83-4a09-975f-2c80f5d44733 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:56.637683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8eee5d46-ae83-4a09-975f-2c80f5d44733 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:56.638133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8eee5d46-ae83-4a09-975f-2c80f5d44733 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:56.638553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8eee5d46-ae83-4a09-975f-2c80f5d44733 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:56.766467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:56.767079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:56.777289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:56.778171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:56.779180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:56.780859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:56.781922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:56.782886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:56.926981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:56.946180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:56.951036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:56.964091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-30e3a189-0980-444b-b327-576e31e61949 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.201081 Sep 27 09:57:56.987834 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 252/493] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:57:56 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:57.026247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8eee5d46-ae83-4a09-975f-2c80f5d44733 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:57.058541 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8eee5d46-ae83-4a09-975f-2c80f5d44733 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1319 microversion: 2.1 time: 0.433303 Sep 27 09:57:57.062497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4cd642ad-3712-4e16-9393-1c59dd219b7e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:57.065385 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 242/494] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:56 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1319 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:57.067727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4cd642ad-3712-4e16-9393-1c59dd219b7e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:57.079463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd642ad-3712-4e16-9393-1c59dd219b7e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:57.080985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd642ad-3712-4e16-9393-1c59dd219b7e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:57.081777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd642ad-3712-4e16-9393-1c59dd219b7e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:57.083064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd642ad-3712-4e16-9393-1c59dd219b7e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:57.083871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd642ad-3712-4e16-9393-1c59dd219b7e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:57.084935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd642ad-3712-4e16-9393-1c59dd219b7e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:57.321721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6971737e-07ef-4383-825b-694cde134cdf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:57.323328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6971737e-07ef-4383-825b-694cde134cdf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:57.334597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6971737e-07ef-4383-825b-694cde134cdf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:57.335793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6971737e-07ef-4383-825b-694cde134cdf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:57.336851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6971737e-07ef-4383-825b-694cde134cdf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:57.338320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6971737e-07ef-4383-825b-694cde134cdf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:57.339994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6971737e-07ef-4383-825b-694cde134cdf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:57.341279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6971737e-07ef-4383-825b-694cde134cdf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:57.362952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4cd642ad-3712-4e16-9393-1c59dd219b7e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:57.381312 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4cd642ad-3712-4e16-9393-1c59dd219b7e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.330073 Sep 27 09:57:57.388142 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 253/495] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:57 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:57.619088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5eeab9c8-5a86-49d3-953d-605876d3e5fa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:57.620074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5eeab9c8-5a86-49d3-953d-605876d3e5fa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:57.620407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6971737e-07ef-4383-825b-694cde134cdf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:57.627779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eeab9c8-5a86-49d3-953d-605876d3e5fa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:57.628282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eeab9c8-5a86-49d3-953d-605876d3e5fa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:57.628803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eeab9c8-5a86-49d3-953d-605876d3e5fa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:57.629541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eeab9c8-5a86-49d3-953d-605876d3e5fa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:57.630267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eeab9c8-5a86-49d3-953d-605876d3e5fa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:57.630518 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6971737e-07ef-4383-825b-694cde134cdf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1319 microversion: 2.1 time: 0.316327 Sep 27 09:57:57.630649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eeab9c8-5a86-49d3-953d-605876d3e5fa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:57.645279 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 243/496] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:57 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1319 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:57.653149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fdd2bc74-3fa8-4b52-be61-56b56999832e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:57.653725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fdd2bc74-3fa8-4b52-be61-56b56999832e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:57.660453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd2bc74-3fa8-4b52-be61-56b56999832e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:57.661373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd2bc74-3fa8-4b52-be61-56b56999832e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:57.662061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd2bc74-3fa8-4b52-be61-56b56999832e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:57.662979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd2bc74-3fa8-4b52-be61-56b56999832e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:57.663660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd2bc74-3fa8-4b52-be61-56b56999832e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:57.664533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdd2bc74-3fa8-4b52-be61-56b56999832e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:57.774497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fdd2bc74-3fa8-4b52-be61-56b56999832e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:57.784728 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fdd2bc74-3fa8-4b52-be61-56b56999832e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1271 microversion: 2.1 time: 0.138183 Sep 27 09:57:57.789149 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 244/497] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:57 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1271 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:57.865476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5eeab9c8-5a86-49d3-953d-605876d3e5fa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:57.875631 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5eeab9c8-5a86-49d3-953d-605876d3e5fa tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1469 microversion: 2.1 time: 0.261686 Sep 27 09:57:57.876272 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 254/498] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:57 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1469 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:58.014450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:58.015583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:58.022916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.024019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.024816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.025674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.026416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.027329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.096897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba5f7441-5223-49ad-be72-e931bea732dd tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:58.097663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba5f7441-5223-49ad-be72-e931bea732dd tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:58.109664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba5f7441-5223-49ad-be72-e931bea732dd tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.110257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba5f7441-5223-49ad-be72-e931bea732dd tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.110769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba5f7441-5223-49ad-be72-e931bea732dd tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.111653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba5f7441-5223-49ad-be72-e931bea732dd tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.113790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba5f7441-5223-49ad-be72-e931bea732dd tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.114337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba5f7441-5223-49ad-be72-e931bea732dd tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.118002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:58.126725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:58.128880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:58.135896 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4a697a51-4478-4133-abd8-530e78f84a4a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.125257 Sep 27 09:57:58.136575 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 245/499] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:57:58 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:57:58.325016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba5f7441-5223-49ad-be72-e931bea732dd tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:58.336174 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba5f7441-5223-49ad-be72-e931bea732dd tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1319 microversion: 2.1 time: 0.246174 Sep 27 09:57:58.352675 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 255/500] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:58 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1319 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:58.417894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aff14920-7e23-4cf9-9444-67d6e5d03d4d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:58.418709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aff14920-7e23-4cf9-9444-67d6e5d03d4d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:58.426609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aff14920-7e23-4cf9-9444-67d6e5d03d4d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.427401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aff14920-7e23-4cf9-9444-67d6e5d03d4d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.428792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aff14920-7e23-4cf9-9444-67d6e5d03d4d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.431813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aff14920-7e23-4cf9-9444-67d6e5d03d4d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.432585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aff14920-7e23-4cf9-9444-67d6e5d03d4d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.433218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aff14920-7e23-4cf9-9444-67d6e5d03d4d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.653546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ed7f37fa-7562-4c29-b354-2c5939df2659 req-8140174b-2590-415e-b673-6c8fa5917e1a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "454cd0f6-eddc-4e88-9437-91ecce9cf66c", "name": "network-vif-plugged", "status": "completed", "tag": "e4cac2e6-28e9-4db8-86d8-5e557c0d5c8a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:58.666062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ed7f37fa-7562-4c29-b354-2c5939df2659 req-8140174b-2590-415e-b673-6c8fa5917e1a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.666867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ed7f37fa-7562-4c29-b354-2c5939df2659 req-8140174b-2590-415e-b673-6c8fa5917e1a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.671737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ed7f37fa-7562-4c29-b354-2c5939df2659 req-8140174b-2590-415e-b673-6c8fa5917e1a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.715435 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ed7f37fa-7562-4c29-b354-2c5939df2659 req-8140174b-2590-415e-b673-6c8fa5917e1a service nova] Creating event network-vif-plugged:e4cac2e6-28e9-4db8-86d8-5e557c0d5c8a for instance 454cd0f6-eddc-4e88-9437-91ecce9cf66c on nch2-master-1 Sep 27 09:57:58.723245 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ed7f37fa-7562-4c29-b354-2c5939df2659 req-8140174b-2590-415e-b673-6c8fa5917e1a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074728 Sep 27 09:57:58.724358 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 256/501] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:58.738784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4b34e68d-c620-4e69-99ea-f13e7fb5cd48 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:58.741641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4b34e68d-c620-4e69-99ea-f13e7fb5cd48 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:58.749566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b34e68d-c620-4e69-99ea-f13e7fb5cd48 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.750283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b34e68d-c620-4e69-99ea-f13e7fb5cd48 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.751004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b34e68d-c620-4e69-99ea-f13e7fb5cd48 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.751881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b34e68d-c620-4e69-99ea-f13e7fb5cd48 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.752621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b34e68d-c620-4e69-99ea-f13e7fb5cd48 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.753080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b34e68d-c620-4e69-99ea-f13e7fb5cd48 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.936550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aff14920-7e23-4cf9-9444-67d6e5d03d4d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:58.945712 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aff14920-7e23-4cf9-9444-67d6e5d03d4d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.537947 Sep 27 09:57:58.961279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4b34e68d-c620-4e69-99ea-f13e7fb5cd48 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:58.962699 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 246/502] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:58 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:58.969110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-65ca1a10-54c4-48f7-a6ce-0e09e542e267 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:58.969849 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4b34e68d-c620-4e69-99ea-f13e7fb5cd48 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1319 microversion: 2.1 time: 0.240366 Sep 27 09:57:58.970005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-65ca1a10-54c4-48f7-a6ce-0e09e542e267 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:58.982454 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 257/503] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:58 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1319 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:58.982548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65ca1a10-54c4-48f7-a6ce-0e09e542e267 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.982629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-521ac4cb-9472-40ad-9127-c3ee77903a17 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:58.982741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65ca1a10-54c4-48f7-a6ce-0e09e542e267 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.982741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65ca1a10-54c4-48f7-a6ce-0e09e542e267 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.982741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65ca1a10-54c4-48f7-a6ce-0e09e542e267 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.982948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-521ac4cb-9472-40ad-9127-c3ee77903a17 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:58.983020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65ca1a10-54c4-48f7-a6ce-0e09e542e267 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.983020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65ca1a10-54c4-48f7-a6ce-0e09e542e267 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.987969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521ac4cb-9472-40ad-9127-c3ee77903a17 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.987969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521ac4cb-9472-40ad-9127-c3ee77903a17 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.987969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521ac4cb-9472-40ad-9127-c3ee77903a17 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:58.987969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521ac4cb-9472-40ad-9127-c3ee77903a17 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:58.988355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521ac4cb-9472-40ad-9127-c3ee77903a17 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:58.989246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521ac4cb-9472-40ad-9127-c3ee77903a17 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:59.067901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-65ca1a10-54c4-48f7-a6ce-0e09e542e267 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:59.079960 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-65ca1a10-54c4-48f7-a6ce-0e09e542e267 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1271 microversion: 2.1 time: 0.115830 Sep 27 09:57:59.094720 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 247/504] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:58 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1271 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:59.163826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:59.164931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:59.173498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-521ac4cb-9472-40ad-9127-c3ee77903a17 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:59.183522 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-521ac4cb-9472-40ad-9127-c3ee77903a17 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1469 microversion: 2.1 time: 0.210995 Sep 27 09:57:59.184570 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 258/505] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:58 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1469 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:59.193479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:59.194402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:59.195061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:59.196909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:59.197532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:59.198449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:59.332471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:59.346646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:59.349007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:59.356177 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-22d729b2-1989-4a7e-9844-2f496db789b4 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.196622 Sep 27 09:57:59.357817 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 248/506] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:57:59 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 09:57:59.367447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33df1a5d-8bc5-4476-bffe-6a0231914f19 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:59.368935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33df1a5d-8bc5-4476-bffe-6a0231914f19 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:59.383089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33df1a5d-8bc5-4476-bffe-6a0231914f19 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:59.383586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33df1a5d-8bc5-4476-bffe-6a0231914f19 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:59.384024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33df1a5d-8bc5-4476-bffe-6a0231914f19 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:59.384775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33df1a5d-8bc5-4476-bffe-6a0231914f19 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:59.385239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33df1a5d-8bc5-4476-bffe-6a0231914f19 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:59.385663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33df1a5d-8bc5-4476-bffe-6a0231914f19 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:59.640606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33df1a5d-8bc5-4476-bffe-6a0231914f19 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:57:59.651713 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33df1a5d-8bc5-4476-bffe-6a0231914f19 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1469 microversion: 2.1 time: 0.296637 Sep 27 09:57:59.652827 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 259/507] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:59 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1469 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:57:59.676933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7f43981a-cbab-40e1-b933-91f39c20dce6 tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:59.680098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7f43981a-cbab-40e1-b933-91f39c20dce6 tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:59.694013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f43981a-cbab-40e1-b933-91f39c20dce6 tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:59.698335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f43981a-cbab-40e1-b933-91f39c20dce6 tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:59.698853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f43981a-cbab-40e1-b933-91f39c20dce6 tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:59.699619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f43981a-cbab-40e1-b933-91f39c20dce6 tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:59.703963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f43981a-cbab-40e1-b933-91f39c20dce6 tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:59.705890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f43981a-cbab-40e1-b933-91f39c20dce6 tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:59.818727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0bfc0f2c-5d08-418b-94ef-80dd89690bb6 req-5dc68ad4-4f96-4b32-afd7-ba5202ff944e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ea27a3db-7b84-4c05-b57f-77b3e6b397f7", "name": "network-vif-deleted", "tag": "43772c65-3acd-4918-ad59-e3de25b39006"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:57:59.833254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0bfc0f2c-5d08-418b-94ef-80dd89690bb6 req-5dc68ad4-4f96-4b32-afd7-ba5202ff944e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:59.834669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0bfc0f2c-5d08-418b-94ef-80dd89690bb6 req-5dc68ad4-4f96-4b32-afd7-ba5202ff944e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:59.835358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0bfc0f2c-5d08-418b-94ef-80dd89690bb6 req-5dc68ad4-4f96-4b32-afd7-ba5202ff944e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:57:59.860078 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0bfc0f2c-5d08-418b-94ef-80dd89690bb6 req-5dc68ad4-4f96-4b32-afd7-ba5202ff944e service nova] Creating event network-vif-deleted:43772c65-3acd-4918-ad59-e3de25b39006 for instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 on nch2-master-1 Sep 27 09:57:59.869947 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0bfc0f2c-5d08-418b-94ef-80dd89690bb6 req-5dc68ad4-4f96-4b32-afd7-ba5202ff944e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054858 Sep 27 09:57:59.871244 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 260/508] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:57:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:57:59.934462 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7f43981a-cbab-40e1-b933-91f39c20dce6 tempest-DeleteServersAdminTestJSON-1639236994 tempest-DeleteServersAdminTestJSON-1639236994-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc/action" status: 202 len: 0 microversion: 2.1 time: 0.262619 Sep 27 09:57:59.953280 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 249/509] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:57:59 2022] POST /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc/action => generated 0 bytes in 282 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:57:59.972292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:59.973943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:57:59.981315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-527c59d1-3b4d-42f7-8463-686d4b613b97 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:57:59.995650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:57:59.996280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:57:59.996425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-527c59d1-3b4d-42f7-8463-686d4b613b97 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:00.009618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.010855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.011862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.012331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.014505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-527c59d1-3b4d-42f7-8463-686d4b613b97 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.015407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-527c59d1-3b4d-42f7-8463-686d4b613b97 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.016042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-527c59d1-3b4d-42f7-8463-686d4b613b97 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.017022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-527c59d1-3b4d-42f7-8463-686d4b613b97 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.021788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-527c59d1-3b4d-42f7-8463-686d4b613b97 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.022723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-527c59d1-3b4d-42f7-8463-686d4b613b97 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.097828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.098612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.099088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.099544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lazy-loading 'fault' on Instance uuid 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:00.243816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:00.253469 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d0c3643a-b82c-420f-b42e-6f83c949eca5 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1452 microversion: 2.1 time: 0.300363 Sep 27 09:58:00.254735 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 261/510] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:59 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1452 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:00.260472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d34608e2-5efc-4027-a454-7fd4708acee7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:00.261581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d34608e2-5efc-4027-a454-7fd4708acee7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:00.269143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d34608e2-5efc-4027-a454-7fd4708acee7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.269862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d34608e2-5efc-4027-a454-7fd4708acee7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.270597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d34608e2-5efc-4027-a454-7fd4708acee7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.271578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d34608e2-5efc-4027-a454-7fd4708acee7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.272425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d34608e2-5efc-4027-a454-7fd4708acee7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.273061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d34608e2-5efc-4027-a454-7fd4708acee7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.419087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-527c59d1-3b4d-42f7-8463-686d4b613b97 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:00.427389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d34608e2-5efc-4027-a454-7fd4708acee7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:00.428533 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-527c59d1-3b4d-42f7-8463-686d4b613b97 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.458050 Sep 27 09:58:00.429142 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 250/511] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:57:59 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:00.436552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a3c15d06-da9c-40ec-ae9c-ad17bf173415 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:00.437106 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d34608e2-5efc-4027-a454-7fd4708acee7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1319 microversion: 2.1 time: 0.181047 Sep 27 09:58:00.437223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a3c15d06-da9c-40ec-ae9c-ad17bf173415 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:00.437860 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 262/512] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:00 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1319 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:00.442812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7fa54971-feef-4aa6-a3f1-a8933d5c439a tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:00.443579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7fa54971-feef-4aa6-a3f1-a8933d5c439a tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:00.451089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa54971-feef-4aa6-a3f1-a8933d5c439a tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.451606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa54971-feef-4aa6-a3f1-a8933d5c439a tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.453158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa54971-feef-4aa6-a3f1-a8933d5c439a tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.453933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa54971-feef-4aa6-a3f1-a8933d5c439a tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.454472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:00.454665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa54971-feef-4aa6-a3f1-a8933d5c439a tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.455100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa54971-feef-4aa6-a3f1-a8933d5c439a tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.460582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3c15d06-da9c-40ec-ae9c-ad17bf173415 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.461459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3c15d06-da9c-40ec-ae9c-ad17bf173415 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.462427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3c15d06-da9c-40ec-ae9c-ad17bf173415 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.463206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3c15d06-da9c-40ec-ae9c-ad17bf173415 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.463666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3c15d06-da9c-40ec-ae9c-ad17bf173415 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.464288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3c15d06-da9c-40ec-ae9c-ad17bf173415 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.527843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7fa54971-feef-4aa6-a3f1-a8933d5c439a tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:00.536125 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7fa54971-feef-4aa6-a3f1-a8933d5c439a tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1298 microversion: 2.1 time: 0.097250 Sep 27 09:58:00.536746 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 263/513] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:00 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1298 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:00.578546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a3c15d06-da9c-40ec-ae9c-ad17bf173415 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:00.588953 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a3c15d06-da9c-40ec-ae9c-ad17bf173415 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1277 microversion: 2.1 time: 0.157460 Sep 27 09:58:00.589825 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 251/514] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:00 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1277 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:00.595904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1af8481-7233-4826-9994-bb043bf372c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:00.596684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a1af8481-7233-4826-9994-bb043bf372c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:00.622348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1af8481-7233-4826-9994-bb043bf372c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.623123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1af8481-7233-4826-9994-bb043bf372c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.623624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1af8481-7233-4826-9994-bb043bf372c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.624415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1af8481-7233-4826-9994-bb043bf372c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.624958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1af8481-7233-4826-9994-bb043bf372c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.625468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1af8481-7233-4826-9994-bb043bf372c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.652009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a1af8481-7233-4826-9994-bb043bf372c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:58:00.688329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3322f92-a570-439e-9aaa-8c1575360020 tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833 tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:00.692112 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b3322f92-a570-439e-9aaa-8c1575360020 tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833 tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833-project-admin] HTTP exception thrown: Invalid timestamp for date invalid_time Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3322f92-a570-439e-9aaa-8c1575360020 tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833 tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833-project-admin] Request method failure captured: Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-instance_usage_audit_log/invalid_time HTTP/1.1 Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMskoiuuKpbUBi9aWK5xOtjIdBXVVHm1Ts48Z0ERpXRDiGPkjqIP-MPmW-vkWEuTospC2_qgpS2dlcN0amiFu_6lIUk-deCEH0UwSLC39Q7KhvwPwWQ6VOnHh_G_I0N03W622tjXXBS2yPSNk9HWU4KS1FKrnu5CQqXvo1TsRzjzhR-s Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 982ca491585d4e288c4e330ad9d157b5 Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833 Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:58:00.700506 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:58:00.701887 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/982ca491585d4e288c4e330ad9d157b5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/982ca491585d4e288c4e330ad9d157b5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/982ca491585d4e288c4e330ad9d157b5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_982ca491585d4e288c4e330ad9d157b5"}]}] Sep 27 09:58:00.701887 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833 Sep 27 09:58:00.701887 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 982ca491585d4e288c4e330ad9d157b5 Sep 27 09:58:00.701887 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833 Sep 27 09:58:00.701887 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833-project-admin Sep 27 09:58:00.701887 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:58:00.701887 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:58:00.701887 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 4fd3b1cd2c534d3e973ba2534e43bf92 Sep 27 09:58:00.701887 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833-project-admin Sep 27 09:58:00.701887 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:58:00.701887 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid timestamp for date invalid_time Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'invalid_time'} Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 51, in show Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi before_date = datetime.datetime.strptime(str(id), Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/_strptime.py", line 568, in _strptime_datetime Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi tt, fraction, gmtoff_fraction = _strptime(data_string, format) Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/_strptime.py", line 349, in _strptime Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise ValueError("time data %r does not match format %r" % Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ValueError: time data 'invalid_time' does not match format '%Y-%m-%d %H:%M:%S' Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:00.702553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 55, in show Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=msg) Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid timestamp for date invalid_time Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid timestamp for date invalid_time Sep 27 09:58:00.703682 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:00.705404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3322f92-a570-439e-9aaa-8c1575360020 tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833 tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833-project-admin] Returning 400 to user: Invalid timestamp for date invalid_time {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:00.706313 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3322f92-a570-439e-9aaa-8c1575360020 tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833 tempest-InstanceUsageAuditLogNegativeTestJSON-1851452833-project-admin] 10.222.0.26 "GET /compute/v2.1/os-instance_usage_audit_log/invalid_time" status: 400 len: 83 microversion: 2.1 time: 0.168636 Sep 27 09:58:00.707073 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 264/515] 10.222.0.26 () {58 vars in 1224 bytes} [Tue Sep 27 09:58:00 2022] GET /compute/v2.1/os-instance_usage_audit_log/invalid_time => generated 83 bytes in 170 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:00.712802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:00.713679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:00.722014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.722528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.722956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.723682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.724145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.724584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.748345 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a1af8481-7233-4826-9994-bb043bf372c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "DELETE /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 204 len: 0 microversion: 2.1 time: 0.156935 Sep 27 09:58:00.748981 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 252/516] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:58:00 2022] DELETE /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:58:00.809462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:00.819095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:00.823466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:00.829304 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9ab2a88b-627d-4eb8-93ad-8862876b1147 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.120994 Sep 27 09:58:00.844466 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 265/517] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:00 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:00.845719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:00.847578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:00.854952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.855657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.856247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.857036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.857509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.857943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.894545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:00.894948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:00.895391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:00.896117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lazy-loading 'fault' on Instance uuid 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:00.920077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0e300b67-063e-4c34-87ee-40302d22e214 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:00.922431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0e300b67-063e-4c34-87ee-40302d22e214 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project] Policy check for os_compute_api:os-instance-usage-audit-log:list failed with credentials {'is_admin': False, 'user_id': '8bc909e7c4214abd83b1e0bd9659e430', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5e69aef703eb40a694c1f6f936dd5e05', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0e300b67-063e-4c34-87ee-40302d22e214 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project] Request method failure captured: Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-instance_usage_audit_log HTTP/1.1 Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMskn2XyQzVuNXavs-7qlbHAs8znlCycW4-C1H6VDjTugWCWWtttOTOWcZ7lMphy7SAUSBKXhtUQbrN-j7HMnw7af_DSP22Uh5ylbaq_kFgqhUcwjnVxozHuqfYryCiCdZaOUNlvQvSfAA0ooLWe4r51vSdkG1eUbGARS0DGl9BsCrBg Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 5e69aef703eb40a694c1f6f936dd5e05 Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:58:00.924936 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:58:00.926485 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5e69aef703eb40a694c1f6f936dd5e05"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5e69aef703eb40a694c1f6f936dd5e05"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5e69aef703eb40a694c1f6f936dd5e05"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5e69aef703eb40a694c1f6f936dd5e05"}]}] Sep 27 09:58:00.926485 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 Sep 27 09:58:00.926485 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 5e69aef703eb40a694c1f6f936dd5e05 Sep 27 09:58:00.926485 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 Sep 27 09:58:00.926485 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project Sep 27 09:58:00.926485 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:58:00.926485 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:58:00.926485 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 8bc909e7c4214abd83b1e0bd9659e430 Sep 27 09:58:00.926485 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project Sep 27 09:58:00.926485 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: action_args: {} Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 38, in index Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(iual_policies.BASE_POLICY_NAME % 'list', target={}) Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:58:00.927207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. Sep 27 09:58:00.928777 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:00.931717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0e300b67-063e-4c34-87ee-40302d22e214 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:list to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:00.933701 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0e300b67-063e-4c34-87ee-40302d22e214 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project] 10.222.0.26 "GET /compute/v2.1/os-instance_usage_audit_log" status: 403 len: 128 microversion: 2.1 time: 0.179585 Sep 27 09:58:00.934524 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 253/518] 10.222.0.26 () {58 vars in 1185 bytes} [Tue Sep 27 09:58:00 2022] GET /compute/v2.1/os-instance_usage_audit_log => generated 128 bytes in 185 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:00.945177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58c1afc8-940a-4a93-9fde-654a5fc833d8 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:00.947623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-58c1afc8-940a-4a93-9fde-654a5fc833d8 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project] Policy check for os_compute_api:os-instance-usage-audit-log:show failed with credentials {'is_admin': False, 'user_id': '8bc909e7c4214abd83b1e0bd9659e430', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5e69aef703eb40a694c1f6f936dd5e05', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58c1afc8-940a-4a93-9fde-654a5fc833d8 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project] Request method failure captured: Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-instance_usage_audit_log/2022-09-27%2009:58:00 HTTP/1.1 Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMskn2XyQzVuNXavs-7qlbHAs8znlCycW4-C1H6VDjTugWCWWtttOTOWcZ7lMphy7SAUSBKXhtUQbrN-j7HMnw7af_DSP22Uh5ylbaq_kFgqhUcwjnVxozHuqfYryCiCdZaOUNlvQvSfAA0ooLWe4r51vSdkG1eUbGARS0DGl9BsCrBg Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 5e69aef703eb40a694c1f6f936dd5e05 Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:58:00.949681 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:58:00.950953 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5e69aef703eb40a694c1f6f936dd5e05"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5e69aef703eb40a694c1f6f936dd5e05"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5e69aef703eb40a694c1f6f936dd5e05"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5e69aef703eb40a694c1f6f936dd5e05"}]}] Sep 27 09:58:00.950953 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 Sep 27 09:58:00.950953 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 5e69aef703eb40a694c1f6f936dd5e05 Sep 27 09:58:00.950953 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 Sep 27 09:58:00.950953 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project Sep 27 09:58:00.950953 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:58:00.950953 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:58:00.950953 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 8bc909e7c4214abd83b1e0bd9659e430 Sep 27 09:58:00.950953 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project Sep 27 09:58:00.950953 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2022-09-27 09:58:00'} Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_usage_audit_log.py", line 45, in show Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(iual_policies.BASE_POLICY_NAME % 'show', target={}) Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:58:00.951550 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. Sep 27 09:58:00.953045 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:00.954039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58c1afc8-940a-4a93-9fde-654a5fc833d8 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-instance-usage-audit-log:show to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:00.954121 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-58c1afc8-940a-4a93-9fde-654a5fc833d8 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644 tempest-InstanceUsageAuditLogNegativeTestJSON-1041633644-project] 10.222.0.26 "GET /compute/v2.1/os-instance_usage_audit_log/2022-09-27 09:58:00" status: 403 len: 128 microversion: 2.1 time: 0.013774 Sep 27 09:58:00.955283 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 254/519] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 09:58:00 2022] GET /compute/v2.1/os-instance_usage_audit_log/2022-09-27%2009%3A58%3A00 => generated 128 bytes in 17 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Sep 27 09:58:01.025550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:01.028143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:01.037724 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-94d26012-aa31-4eda-a392-e47d308e9c05 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1458 microversion: 2.1 time: 0.198419 Sep 27 09:58:01.039270 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 266/520] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:00 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1458 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:01.452347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1c9f8aec-b102-4e66-b746-50f4d1e7bd2e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:01.452994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1c9f8aec-b102-4e66-b746-50f4d1e7bd2e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:01.460169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-83b87805-0668-456d-9396-2da24190bdde tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:01.460817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-83b87805-0668-456d-9396-2da24190bdde tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:01.471577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c9f8aec-b102-4e66-b746-50f4d1e7bd2e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:01.474036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83b87805-0668-456d-9396-2da24190bdde tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:01.474538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83b87805-0668-456d-9396-2da24190bdde tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:01.474984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83b87805-0668-456d-9396-2da24190bdde tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:01.475810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83b87805-0668-456d-9396-2da24190bdde tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:01.476161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83b87805-0668-456d-9396-2da24190bdde tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:01.476647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83b87805-0668-456d-9396-2da24190bdde tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:01.481178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c9f8aec-b102-4e66-b746-50f4d1e7bd2e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:01.481804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c9f8aec-b102-4e66-b746-50f4d1e7bd2e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:01.482591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c9f8aec-b102-4e66-b746-50f4d1e7bd2e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:01.483057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c9f8aec-b102-4e66-b746-50f4d1e7bd2e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:01.483658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c9f8aec-b102-4e66-b746-50f4d1e7bd2e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:01.671307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-83b87805-0668-456d-9396-2da24190bdde tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:01.683292 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-83b87805-0668-456d-9396-2da24190bdde tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1469 microversion: 2.1 time: 0.227355 Sep 27 09:58:01.684439 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 267/521] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:01 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1469 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:01.690119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:01.690940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:01.698327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:01.699294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:01.699960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:01.701399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:01.702063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:01.702755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:01.752132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:01.753300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:01.753969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:01.754668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lazy-loading 'fault' on Instance uuid ea27a3db-7b84-4c05-b57f-77b3e6b397f7 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:01.784708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1c9f8aec-b102-4e66-b746-50f4d1e7bd2e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:01.789653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=2,nova_cell1:UPDATE=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:01.796209 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1c9f8aec-b102-4e66-b746-50f4d1e7bd2e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.348290 Sep 27 09:58:01.797373 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 255/522] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:01 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:01.802352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e2f15c2c-fcbb-4b18-ad7b-ab56e723a438 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:01.803824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e2f15c2c-fcbb-4b18-ad7b-ab56e723a438 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:01.810304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:01.814920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2f15c2c-fcbb-4b18-ad7b-ab56e723a438 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:01.816438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2f15c2c-fcbb-4b18-ad7b-ab56e723a438 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:01.816438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2f15c2c-fcbb-4b18-ad7b-ab56e723a438 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:01.818042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2f15c2c-fcbb-4b18-ad7b-ab56e723a438 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:01.818707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2f15c2c-fcbb-4b18-ad7b-ab56e723a438 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:01.819284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2f15c2c-fcbb-4b18-ad7b-ab56e723a438 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:01.824601 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e2a8d296-fbef-45a7-92eb-661197cc19c1 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1312 microversion: 2.1 time: 0.138739 Sep 27 09:58:01.826817 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 268/523] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:01 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1312 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:02.071107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e2f15c2c-fcbb-4b18-ad7b-ab56e723a438 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:02.082618 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e2f15c2c-fcbb-4b18-ad7b-ab56e723a438 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1319 microversion: 2.1 time: 0.283181 Sep 27 09:58:02.084451 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 256/524] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:01 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1319 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:02.103740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:02.104436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:02.113760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:02.114592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:02.115468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:02.116413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:02.117364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:02.117953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:02.142711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "tempest-host-1485258121", "block_migration": true, "disk_over_commit": false}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:02.145706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:02.152734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:02.152993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:02.153655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:02.154569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:02.155213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:02.155819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:02.156501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:02.196123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:02.209543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:02.212739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:02.223807 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0821f24a-0b9f-4a8e-96af-af12a925111a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.137819 Sep 27 09:58:02.224469 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 257/525] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:02 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:02.230945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-6f370ff3-f39e-4760-8b11-26dde15079f6 req-793b2054-f1c2-4e8e-a3ba-afa0476f5216 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f769af8b-472a-4950-b4ae-8c0183b23c1e", "tag": "676ed625-8a34-4b31-8384-30e066fc7f52"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:02.248443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6f370ff3-f39e-4760-8b11-26dde15079f6 req-793b2054-f1c2-4e8e-a3ba-afa0476f5216 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:02.249037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6f370ff3-f39e-4760-8b11-26dde15079f6 req-793b2054-f1c2-4e8e-a3ba-afa0476f5216 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:02.249702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6f370ff3-f39e-4760-8b11-26dde15079f6 req-793b2054-f1c2-4e8e-a3ba-afa0476f5216 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:02.253017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lazy-loading 'flavor' on Instance uuid 454cd0f6-eddc-4e88-9437-91ecce9cf66c {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:02.264702 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-6f370ff3-f39e-4760-8b11-26dde15079f6 req-793b2054-f1c2-4e8e-a3ba-afa0476f5216 service nova] Creating event network-changed:676ed625-8a34-4b31-8384-30e066fc7f52 for instance f769af8b-472a-4950-b4ae-8c0183b23c1e on nch1-master-1 Sep 27 09:58:02.270236 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-6f370ff3-f39e-4760-8b11-26dde15079f6 req-793b2054-f1c2-4e8e-a3ba-afa0476f5216 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044491 Sep 27 09:58:02.270851 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 258/526] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:02.272945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Going to try to live migrate instance to tempest-host-1485258121 {{(pid=145968) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Sep 27 09:58:02.280012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:02.280769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:02.285859 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] HTTP exception thrown: Compute host tempest-host-1485258121 could not be found. Sep 27 09:58:02.287626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:02.288149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:02.288642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:02.289611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:02.290225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:02.290511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Request method failure captured: Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c/action HTTP/1.1 Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 107 Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMskU61IIDD7saYvuizpK3FtZNHWD167DDy06ZOamEJt6aa4cpqWyRYItVDjmgFdQVTen0VlFaWfuYiNTxOAw51LUiwqSN6DdQPhIFKmG2E5WrYO4d5o9C3I_ypK-ab8O0HQwtfEXSKgh_C6Fu-BQNPmhVKUa8GKl084ZOMqQKaE94Wg Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: d1a7023a446f417e9764a1c113453548 Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-LiveMigrationNegativeTest-886264320 Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:58:02.292740 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d1a7023a446f417e9764a1c113453548"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d1a7023a446f417e9764a1c113453548"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d1a7023a446f417e9764a1c113453548"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d1a7023a446f417e9764a1c113453548"}]}] Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-LiveMigrationNegativeTest-886264320 Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: d1a7023a446f417e9764a1c113453548 Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-886264320 Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-LiveMigrationNegativeTest-886264320-project-admin Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: d8b69329ff4d4a79b7cd0d056ae722e1 Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-LiveMigrationNegativeTest-886264320-project-admin Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: {"os-migrateLive": {"host": "tempest-host-1485258121", "block_migration": true, "disk_over_commit": false}} Sep 27 09:58:02.294088 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: exception: Compute host tempest-host-1485258121 could not be found. Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '454cd0f6-eddc-4e88-9437-91ecce9cf66c', 'body': {'os-migrateLive': {'host': 'tempest-host-1485258121', 'block_migration': True, 'disk_over_commit': False}}} Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 123, in _migrate_live Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.live_migrate(context, instance, block_migration, Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 309, in inner Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 278, in inner Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 259, in inner Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 09:58:02.295048 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5488, in live_migrate Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nodes = objects.ComputeNodeList.get_all_by_host(context, host_name) Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/compute_node.py", line 476, in get_all_by_host Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_computes = cls._db_compute_node_get_all_by_host(context, host, Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 179, in wrapper Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/compute_node.py", line 472, in _db_compute_node_get_all_by_host Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return db.compute_node_get_all_by_host(context, host) Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 241, in wrapper Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(context, *args, **kwargs) Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/main/api.py", line 740, in compute_node_get_all_by_host Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ComputeHostNotFound(host=host) Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ComputeHostNotFound: Compute host tempest-host-1485258121 could not be found. Sep 27 09:58:02.297519 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 150, in _migrate_live Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Compute host tempest-host-1485258121 could not be found. Sep 27 09:58:02.299005 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Compute host tempest-host-1485258121 could not be found. Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Returning 400 to user: Compute host tempest-host-1485258121 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5cae925e-d417-4d75-b3eb-90ded20172c6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c/action" status: 400 len: 100 microversion: 2.1 time: 0.466454 Sep 27 09:58:02.300696 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 269/527] 10.222.0.26 () {60 vars in 1279 bytes} [Tue Sep 27 09:58:01 2022] POST /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c/action => generated 100 bytes in 468 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 09:58:02.311171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-80af3184-88e9-4a8d-a59d-a921c1166bec tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:02.313155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-80af3184-88e9-4a8d-a59d-a921c1166bec tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:02.324535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80af3184-88e9-4a8d-a59d-a921c1166bec tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:02.325493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80af3184-88e9-4a8d-a59d-a921c1166bec tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:02.326179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80af3184-88e9-4a8d-a59d-a921c1166bec tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:02.327332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80af3184-88e9-4a8d-a59d-a921c1166bec tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:02.328318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80af3184-88e9-4a8d-a59d-a921c1166bec tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:02.328899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80af3184-88e9-4a8d-a59d-a921c1166bec tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:02.538455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:02.540113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:02.540113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:02.540769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lazy-loading 'fault' on Instance uuid 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:02.560610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-80af3184-88e9-4a8d-a59d-a921c1166bec tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:02.568941 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-80af3184-88e9-4a8d-a59d-a921c1166bec tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1469 microversion: 2.1 time: 0.266512 Sep 27 09:58:02.569523 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 270/528] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:02 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1469 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:02.710153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationNegativeTest-server-706736013", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:02.769667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:58:03.002709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:03.014336 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-670b98ab-2a70-4701-9ab7-8cfdc9ed3302 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1458 microversion: 2.1 time: 0.738561 Sep 27 09:58:03.014989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:58:03.016597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:58:03.016597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:58:03.016597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:58:03.017254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:58:03.028682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:03.028682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:03.028682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:03.029884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:03.029884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:03.030584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:03.030758 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 259/529] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:02 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1458 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:03.055548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Getting quotas for project 6e8a121c6c654613a6a92df03c3754fa. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:58:03.056711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4043b540-f4d3-4280-81bc-017bfc738bee tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:03.057649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4043b540-f4d3-4280-81bc-017bfc738bee tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:03.061923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Getting quotas for user 4319ec4b90e1481b8be19e8f9dde1ea4 and project 6e8a121c6c654613a6a92df03c3754fa. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:58:03.071212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043b540-f4d3-4280-81bc-017bfc738bee tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:03.071712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043b540-f4d3-4280-81bc-017bfc738bee tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:03.072152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043b540-f4d3-4280-81bc-017bfc738bee tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:03.072906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043b540-f4d3-4280-81bc-017bfc738bee tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:03.073370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043b540-f4d3-4280-81bc-017bfc738bee tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:03.073983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043b540-f4d3-4280-81bc-017bfc738bee tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:03.077149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:58:03.079577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:58:03.145057 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b858d5ea-256c-4a04-b04e-b6755c9c3077 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.438691 Sep 27 09:58:03.147043 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 271/530] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 441 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:58:03.157912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:03.158900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:03.169054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:03.170449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:03.171050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:03.172792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:03.173551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:03.174408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:03.257249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:03.258175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:03.259146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:03.259809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lazy-loading 'fault' on Instance uuid ea27a3db-7b84-4c05-b57f-77b3e6b397f7 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:03.365043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:03.399195 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cd443ec1-ac77-4e49-99e3-4475565c8b69 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 200 len: 1312 microversion: 2.1 time: 0.247683 Sep 27 09:58:03.402001 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 272/531] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:03 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 1312 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:03.422168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-361c5eca-3d2d-443b-bfde-a7cc0d93e89d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:03.422862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-361c5eca-3d2d-443b-bfde-a7cc0d93e89d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:03.437549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-361c5eca-3d2d-443b-bfde-a7cc0d93e89d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:03.438255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-361c5eca-3d2d-443b-bfde-a7cc0d93e89d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:03.438696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-361c5eca-3d2d-443b-bfde-a7cc0d93e89d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:03.439423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-361c5eca-3d2d-443b-bfde-a7cc0d93e89d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:03.439882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-361c5eca-3d2d-443b-bfde-a7cc0d93e89d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:03.440358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-361c5eca-3d2d-443b-bfde-a7cc0d93e89d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:03.481348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4043b540-f4d3-4280-81bc-017bfc738bee tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:03.498819 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4043b540-f4d3-4280-81bc-017bfc738bee tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.466411 Sep 27 09:58:03.499710 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 260/532] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:03 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:03.532237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b2710c09-55ac-4f6f-a8e2-68c9c2768ddf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:03.532904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b2710c09-55ac-4f6f-a8e2-68c9c2768ddf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:03.701449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-361c5eca-3d2d-443b-bfde-a7cc0d93e89d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:03.712127 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-361c5eca-3d2d-443b-bfde-a7cc0d93e89d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1319 microversion: 2.1 time: 0.305331 Sep 27 09:58:03.715292 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 273/533] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:03 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1319 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:03.727873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:03.733229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:03.739725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:03.740489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:03.741706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:03.742607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:03.743859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:03.744364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:03.763291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b2710c09-55ac-4f6f-a8e2-68c9c2768ddf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:03.772549 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b2710c09-55ac-4f6f-a8e2-68c9c2768ddf tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1218 microversion: 2.1 time: 0.270841 Sep 27 09:58:03.794258 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 261/534] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:03 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1218 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:03.834870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:03.846866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:03.849917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:03.856542 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-724d6bb4-6c7a-4412-aceb-6094eb07ed23 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.134444 Sep 27 09:58:03.857928 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 274/535] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:03 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:04.048186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:04.049135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:04.059807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.060495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.061207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.062249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.063047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.063525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.107292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.108203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.108677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.110179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lazy-loading 'fault' on Instance uuid 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:04.413184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:04.424866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:04.429324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ea27a3db-7b84-4c05-b57f-77b3e6b397f7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:04.451772 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c155e160-d95b-4f93-be45-43c2afc102be tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1458 microversion: 2.1 time: 0.403208 Sep 27 09:58:04.454774 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 262/536] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:04 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1458 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:04.467297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.467802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.468264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.469037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.469500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.469933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.485508 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] HTTP exception thrown: Instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 could not be found. Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Request method failure captured: Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 HTTP/1.1 Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjkKnEzPQ8NhXvM0Jes3Z8mp4M91AxwhKsOv1TjyFg3pda3-7f4esRqTqojocxEG1nIYOvYRER-AUcD_JoYsO9KRIT5FR0JUmQ8B7Aln6yQd90BTZsZr-kO2vG-fosa1uJx5kJs3YOXTc2sCNUzdkFSkhRINF7_qNv_4vE5YuBHx-w Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b33d41b411144dbbbdb012e694857d27 Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-591068969 Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:58:04.487087 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b33d41b411144dbbbdb012e694857d27"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b33d41b411144dbbbdb012e694857d27"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b33d41b411144dbbbdb012e694857d27"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b33d41b411144dbbbdb012e694857d27"}]}] Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersAdminNegativeTestJSON-591068969 Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b33d41b411144dbbbdb012e694857d27 Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-591068969 Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersAdminNegativeTestJSON-591068969-project Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: d7d3aef1d4da47b2ab5454c3eb0346a6 Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersAdminNegativeTestJSON-591068969-project Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 could not be found. Sep 27 09:58:04.488333 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'ea27a3db-7b84-4c05-b57f-77b3e6b397f7'} Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 could not be found. Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:04.489124 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 could not be found. Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 could not be found. Sep 27 09:58:04.490468 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:04.493171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Returning 404 to user: Instance ea27a3db-7b84-4c05-b57f-77b3e6b397f7 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:04.493171 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b665507c-9e8a-45d9-90cb-186be950555b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7" status: 404 len: 111 microversion: 2.1 time: 0.067757 Sep 27 09:58:04.493171 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 275/537] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:04 2022] GET /compute/v2.1/servers/ea27a3db-7b84-4c05-b57f-77b3e6b397f7 => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 09:58:04.497663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f9f3aa8b-4da2-4f08-a4dc-30165f3b5370 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:04.498196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f9f3aa8b-4da2-4f08-a4dc-30165f3b5370 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:04.504762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9f3aa8b-4da2-4f08-a4dc-30165f3b5370 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.505286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9f3aa8b-4da2-4f08-a4dc-30165f3b5370 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.505589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9f3aa8b-4da2-4f08-a4dc-30165f3b5370 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.506383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9f3aa8b-4da2-4f08-a4dc-30165f3b5370 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.506794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9f3aa8b-4da2-4f08-a4dc-30165f3b5370 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.507205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9f3aa8b-4da2-4f08-a4dc-30165f3b5370 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.523946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a101dd62-9485-4c8d-8e25-d26489a2693a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:04.524973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a101dd62-9485-4c8d-8e25-d26489a2693a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:04.547000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a101dd62-9485-4c8d-8e25-d26489a2693a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.547901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a101dd62-9485-4c8d-8e25-d26489a2693a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.548819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a101dd62-9485-4c8d-8e25-d26489a2693a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.549716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a101dd62-9485-4c8d-8e25-d26489a2693a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.550377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a101dd62-9485-4c8d-8e25-d26489a2693a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.550962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a101dd62-9485-4c8d-8e25-d26489a2693a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.572180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f9f3aa8b-4da2-4f08-a4dc-30165f3b5370 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:58:04.742274 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f9f3aa8b-4da2-4f08-a4dc-30165f3b5370 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "DELETE /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 204 len: 0 microversion: 2.1 time: 0.247852 Sep 27 09:58:04.743282 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 263/538] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:58:04 2022] DELETE /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:58:04.756630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-8245675e-86d2-4d49-aba5-005f6cb0152b req-fb6f5056-f220-430b-b11b-e72f588702c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc", "name": "network-vif-deleted", "tag": "9bd9c08f-8f18-447f-9137-e5d05f5c4f09"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:04.770158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8245675e-86d2-4d49-aba5-005f6cb0152b req-fb6f5056-f220-430b-b11b-e72f588702c2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.771516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8245675e-86d2-4d49-aba5-005f6cb0152b req-fb6f5056-f220-430b-b11b-e72f588702c2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.772401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8245675e-86d2-4d49-aba5-005f6cb0152b req-fb6f5056-f220-430b-b11b-e72f588702c2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.787223 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8245675e-86d2-4d49-aba5-005f6cb0152b req-fb6f5056-f220-430b-b11b-e72f588702c2 service nova] Creating event network-vif-deleted:9bd9c08f-8f18-447f-9137-e5d05f5c4f09 for instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc on nch2-master-1 Sep 27 09:58:04.811747 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-8245675e-86d2-4d49-aba5-005f6cb0152b req-fb6f5056-f220-430b-b11b-e72f588702c2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063802 Sep 27 09:58:04.813245 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 264/539] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:04.820237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1cfa8c42-9bf1-43f3-acbd-3b67d15c0390 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:04.821515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1cfa8c42-9bf1-43f3-acbd-3b67d15c0390 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:04.831164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cfa8c42-9bf1-43f3-acbd-3b67d15c0390 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.832216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cfa8c42-9bf1-43f3-acbd-3b67d15c0390 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.833096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cfa8c42-9bf1-43f3-acbd-3b67d15c0390 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.834055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cfa8c42-9bf1-43f3-acbd-3b67d15c0390 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.834756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cfa8c42-9bf1-43f3-acbd-3b67d15c0390 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.835503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cfa8c42-9bf1-43f3-acbd-3b67d15c0390 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.900678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a101dd62-9485-4c8d-8e25-d26489a2693a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:04.915100 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a101dd62-9485-4c8d-8e25-d26489a2693a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.395970 Sep 27 09:58:04.915699 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 276/540] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:04 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:04.921609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f4d8328c-93f5-416f-8072-501377c4770b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:04.922504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f4d8328c-93f5-416f-8072-501377c4770b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:04.937652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4d8328c-93f5-416f-8072-501377c4770b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.938588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4d8328c-93f5-416f-8072-501377c4770b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.939224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4d8328c-93f5-416f-8072-501377c4770b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:04.940156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4d8328c-93f5-416f-8072-501377c4770b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:04.942164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4d8328c-93f5-416f-8072-501377c4770b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:04.943230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4d8328c-93f5-416f-8072-501377c4770b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:05.156516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:05.213677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1cfa8c42-9bf1-43f3-acbd-3b67d15c0390 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:05.228096 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1cfa8c42-9bf1-43f3-acbd-3b67d15c0390 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1319 microversion: 2.1 time: 0.412886 Sep 27 09:58:05.233735 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 265/541] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:04 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1319 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:05.241121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3382f115-7d1d-4e3f-b6f4-044c493df075 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:05.241759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3382f115-7d1d-4e3f-b6f4-044c493df075 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:05.259750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3382f115-7d1d-4e3f-b6f4-044c493df075 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:05.265539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3382f115-7d1d-4e3f-b6f4-044c493df075 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:05.266030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3382f115-7d1d-4e3f-b6f4-044c493df075 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:05.267162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3382f115-7d1d-4e3f-b6f4-044c493df075 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:05.268101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3382f115-7d1d-4e3f-b6f4-044c493df075 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:05.268701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3382f115-7d1d-4e3f-b6f4-044c493df075 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:05.281063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f4d8328c-93f5-416f-8072-501377c4770b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:05.289252 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f4d8328c-93f5-416f-8072-501377c4770b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1476 microversion: 2.1 time: 0.372151 Sep 27 09:58:05.294047 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 277/542] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:04 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1476 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:05.301447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:05.304267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:05.319253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:05.324384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:05.326736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:05.328204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:05.328851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:05.329517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:05.401349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3382f115-7d1d-4e3f-b6f4-044c493df075 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:05.410134 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3382f115-7d1d-4e3f-b6f4-044c493df075 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1288 microversion: 2.1 time: 0.175165 Sep 27 09:58:05.413215 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 266/543] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:05 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1288 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:05.417673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:05.448241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:05.451079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:05.457135 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fc21f6a7-872b-4081-96f1-399366f27d86 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.161585 Sep 27 09:58:05.457726 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 278/544] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:05 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:05.481696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:05.482303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:05.489564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:05.490044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:05.490694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:05.491397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:05.491950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:05.493325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:05.539381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:05.540933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:05.541733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:05.542341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lazy-loading 'fault' on Instance uuid 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:05.661429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:05.673809 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cf8ac15a-bfa9-4583-b813-60673e34d058 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1309 microversion: 2.1 time: 0.200264 Sep 27 09:58:05.686760 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 267/545] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:05 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1309 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:05.939501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cbe213f0-b527-4961-8553-a42b4fefbacd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:05.940119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cbe213f0-b527-4961-8553-a42b4fefbacd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:05.946961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbe213f0-b527-4961-8553-a42b4fefbacd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:05.947451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbe213f0-b527-4961-8553-a42b4fefbacd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:05.948099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbe213f0-b527-4961-8553-a42b4fefbacd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:05.948872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbe213f0-b527-4961-8553-a42b4fefbacd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:05.949485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbe213f0-b527-4961-8553-a42b4fefbacd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:05.950055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbe213f0-b527-4961-8553-a42b4fefbacd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:06.185410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cbe213f0-b527-4961-8553-a42b4fefbacd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:06.196706 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cbe213f0-b527-4961-8553-a42b4fefbacd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.260584 Sep 27 09:58:06.197917 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 279/546] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:05 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:06.256092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-345257cb-e78b-4454-a764-c4e240bce7c1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:06.257433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-345257cb-e78b-4454-a764-c4e240bce7c1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:06.264452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-345257cb-e78b-4454-a764-c4e240bce7c1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:06.269809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-345257cb-e78b-4454-a764-c4e240bce7c1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:06.270704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-345257cb-e78b-4454-a764-c4e240bce7c1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:06.272018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-345257cb-e78b-4454-a764-c4e240bce7c1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:06.272551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-345257cb-e78b-4454-a764-c4e240bce7c1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:06.273353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-345257cb-e78b-4454-a764-c4e240bce7c1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:06.320488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15d44bf4-bac8-44f2-b30a-b6f736e6c860 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:06.321500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-15d44bf4-bac8-44f2-b30a-b6f736e6c860 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:06.333705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d44bf4-bac8-44f2-b30a-b6f736e6c860 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:06.344983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d44bf4-bac8-44f2-b30a-b6f736e6c860 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:06.345679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d44bf4-bac8-44f2-b30a-b6f736e6c860 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:06.346591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d44bf4-bac8-44f2-b30a-b6f736e6c860 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:06.349690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d44bf4-bac8-44f2-b30a-b6f736e6c860 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:06.350347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d44bf4-bac8-44f2-b30a-b6f736e6c860 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:06.649956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-15d44bf4-bac8-44f2-b30a-b6f736e6c860 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:06.662065 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-15d44bf4-bac8-44f2-b30a-b6f736e6c860 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1476 microversion: 2.1 time: 0.345409 Sep 27 09:58:06.663723 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 280/547] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:06 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1476 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:06.683637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67a0f666-4b34-4500-9a93-de7424eb790d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:06.684269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67a0f666-4b34-4500-9a93-de7424eb790d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:06.693280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a0f666-4b34-4500-9a93-de7424eb790d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:06.693882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a0f666-4b34-4500-9a93-de7424eb790d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:06.694404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a0f666-4b34-4500-9a93-de7424eb790d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:06.695211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a0f666-4b34-4500-9a93-de7424eb790d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:06.696742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a0f666-4b34-4500-9a93-de7424eb790d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:06.697262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a0f666-4b34-4500-9a93-de7424eb790d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:06.764326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-345257cb-e78b-4454-a764-c4e240bce7c1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:06.777630 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-345257cb-e78b-4454-a764-c4e240bce7c1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1319 microversion: 2.1 time: 0.525644 Sep 27 09:58:06.780743 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 268/548] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:06 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1319 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:06.787181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:06.788068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:06.799100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:06.799710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:06.800423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:06.801655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:06.802815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:06.805488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:06.947202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-67a0f666-4b34-4500-9a93-de7424eb790d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:06.984376 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67a0f666-4b34-4500-9a93-de7424eb790d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1318 microversion: 2.1 time: 0.319251 Sep 27 09:58:06.985238 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 281/549] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:06 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1318 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:07.004690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:07.006047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:07.013322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.014021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.014587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:07.015475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.016062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.016618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:07.076764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:07.088852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:07.091613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.092492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.093106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:07.093733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lazy-loading 'fault' on Instance uuid 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:07.094330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:07.103356 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5bafd1c-43a2-48fb-a417-d2847772a9fc tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.321231 Sep 27 09:58:07.109055 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 269/550] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:06 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:07.122025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-045e7b3a-fdc7-4a65-bfcc-e983e7eca5bb req-e1c758e9-4e6a-4397-a584-5fce6b8e676c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "67574e92-7d2a-4f14-8448-5ff3278cb803", "tag": "aa655eea-e771-44bf-9f90-0e21077a0242"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:07.150809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-045e7b3a-fdc7-4a65-bfcc-e983e7eca5bb req-e1c758e9-4e6a-4397-a584-5fce6b8e676c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.151296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-045e7b3a-fdc7-4a65-bfcc-e983e7eca5bb req-e1c758e9-4e6a-4397-a584-5fce6b8e676c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.151733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-045e7b3a-fdc7-4a65-bfcc-e983e7eca5bb req-e1c758e9-4e6a-4397-a584-5fce6b8e676c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:07.172209 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-045e7b3a-fdc7-4a65-bfcc-e983e7eca5bb req-e1c758e9-4e6a-4397-a584-5fce6b8e676c service nova] Creating event network-changed:aa655eea-e771-44bf-9f90-0e21077a0242 for instance 67574e92-7d2a-4f14-8448-5ff3278cb803 on nch2-master-1 Sep 27 09:58:07.177257 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-045e7b3a-fdc7-4a65-bfcc-e983e7eca5bb req-e1c758e9-4e6a-4397-a584-5fce6b8e676c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066696 Sep 27 09:58:07.177976 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 270/551] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:07.188844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7ba3c51770094c4ea7fec1e597ae13a0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3132070d235941498e3f64d9d7075f6b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:07.197783 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-12c4b6ae-cfd0-4e61-8ea3-7d5c235a5192 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 200 len: 1309 microversion: 2.1 time: 0.211290 Sep 27 09:58:07.198635 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 282/552] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:06 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 1309 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:07.226902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-70cb2df8-9ec4-4e3b-93d3-d5d54ba9b079 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:07.228835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-70cb2df8-9ec4-4e3b-93d3-d5d54ba9b079 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:07.254457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-4902b643-febd-4d2b-8a03-8fc209ae72aa req-99d3314d-a51e-4003-ae8d-9576e3a657ba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f769af8b-472a-4950-b4ae-8c0183b23c1e", "name": "network-vif-plugged", "status": "completed", "tag": "676ed625-8a34-4b31-8384-30e066fc7f52"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:07.262992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70cb2df8-9ec4-4e3b-93d3-d5d54ba9b079 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.264028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70cb2df8-9ec4-4e3b-93d3-d5d54ba9b079 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.265204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70cb2df8-9ec4-4e3b-93d3-d5d54ba9b079 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:07.266170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70cb2df8-9ec4-4e3b-93d3-d5d54ba9b079 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.266795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70cb2df8-9ec4-4e3b-93d3-d5d54ba9b079 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.267588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70cb2df8-9ec4-4e3b-93d3-d5d54ba9b079 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:07.288179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4902b643-febd-4d2b-8a03-8fc209ae72aa req-99d3314d-a51e-4003-ae8d-9576e3a657ba service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.289549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4902b643-febd-4d2b-8a03-8fc209ae72aa req-99d3314d-a51e-4003-ae8d-9576e3a657ba service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.290317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4902b643-febd-4d2b-8a03-8fc209ae72aa req-99d3314d-a51e-4003-ae8d-9576e3a657ba service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:07.316081 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-4902b643-febd-4d2b-8a03-8fc209ae72aa req-99d3314d-a51e-4003-ae8d-9576e3a657ba service nova] Creating event network-vif-plugged:676ed625-8a34-4b31-8384-30e066fc7f52 for instance f769af8b-472a-4950-b4ae-8c0183b23c1e on nch1-master-1 Sep 27 09:58:07.334097 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-4902b643-febd-4d2b-8a03-8fc209ae72aa req-99d3314d-a51e-4003-ae8d-9576e3a657ba service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098867 Sep 27 09:58:07.335543 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 283/553] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:07.387721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-9a8c13db-6b74-40e9-8cbe-67385dbcf176 req-eea06f8b-cc4e-4c01-b9a4-949ddefd9037 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a59c15ae-bd3b-4f14-9311-c8f5f52c44d4", "tag": "92de2508-8fe5-415a-987e-c0ff172d47d7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:07.401745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9a8c13db-6b74-40e9-8cbe-67385dbcf176 req-eea06f8b-cc4e-4c01-b9a4-949ddefd9037 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.402992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9a8c13db-6b74-40e9-8cbe-67385dbcf176 req-eea06f8b-cc4e-4c01-b9a4-949ddefd9037 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.403516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9a8c13db-6b74-40e9-8cbe-67385dbcf176 req-eea06f8b-cc4e-4c01-b9a4-949ddefd9037 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:07.408794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:07.420579 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-9a8c13db-6b74-40e9-8cbe-67385dbcf176 req-eea06f8b-cc4e-4c01-b9a4-949ddefd9037 service nova] Creating event network-changed:92de2508-8fe5-415a-987e-c0ff172d47d7 for instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 on nch1-master-1 Sep 27 09:58:07.426812 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-9a8c13db-6b74-40e9-8cbe-67385dbcf176 req-eea06f8b-cc4e-4c01-b9a4-949ddefd9037 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044692 Sep 27 09:58:07.427909 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 284/554] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:07.691281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-70cb2df8-9ec4-4e3b-93d3-d5d54ba9b079 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:07.700393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-353a9310-cdba-4c86-a3b0-56f2a0d27b3b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:07.702042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-353a9310-cdba-4c86-a3b0-56f2a0d27b3b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:07.704329 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-70cb2df8-9ec4-4e3b-93d3-d5d54ba9b079 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.482816 Sep 27 09:58:07.705341 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 271/555] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:07 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:07.735787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:07.748371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353a9310-cdba-4c86-a3b0-56f2a0d27b3b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.749068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353a9310-cdba-4c86-a3b0-56f2a0d27b3b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.749628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353a9310-cdba-4c86-a3b0-56f2a0d27b3b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:07.750447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353a9310-cdba-4c86-a3b0-56f2a0d27b3b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.750951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353a9310-cdba-4c86-a3b0-56f2a0d27b3b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.751376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353a9310-cdba-4c86-a3b0-56f2a0d27b3b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:07.808852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ec7ea6a-85e8-4131-a8a8-2db4f74d9950 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:07.810776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ec7ea6a-85e8-4131-a8a8-2db4f74d9950 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:07.830350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec7ea6a-85e8-4131-a8a8-2db4f74d9950 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.845047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec7ea6a-85e8-4131-a8a8-2db4f74d9950 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.849670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec7ea6a-85e8-4131-a8a8-2db4f74d9950 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:07.856153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec7ea6a-85e8-4131-a8a8-2db4f74d9950 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:07.856731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec7ea6a-85e8-4131-a8a8-2db4f74d9950 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:07.858710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec7ea6a-85e8-4131-a8a8-2db4f74d9950 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.056160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-353a9310-cdba-4c86-a3b0-56f2a0d27b3b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:08.082138 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-353a9310-cdba-4c86-a3b0-56f2a0d27b3b tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1476 microversion: 2.1 time: 0.394268 Sep 27 09:58:08.084361 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 285/556] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:07 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1476 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:08.095075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fe70b9c4-b2db-4f7d-86b0-573b29ee1a91 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:08.097529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fe70b9c4-b2db-4f7d-86b0-573b29ee1a91 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:08.108535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe70b9c4-b2db-4f7d-86b0-573b29ee1a91 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.109259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe70b9c4-b2db-4f7d-86b0-573b29ee1a91 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.109874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe70b9c4-b2db-4f7d-86b0-573b29ee1a91 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.110773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe70b9c4-b2db-4f7d-86b0-573b29ee1a91 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.111438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe70b9c4-b2db-4f7d-86b0-573b29ee1a91 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.112054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe70b9c4-b2db-4f7d-86b0-573b29ee1a91 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.145376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8ec7ea6a-85e8-4131-a8a8-2db4f74d9950 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:08.149954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:08.168696 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8ec7ea6a-85e8-4131-a8a8-2db4f74d9950 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1319 microversion: 2.1 time: 0.361631 Sep 27 09:58:08.169529 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 272/557] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:07 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1319 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:08.190579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:08.191211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:08.214576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.215377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.216267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.219797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.220603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.221349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.322985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:08.336122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:08.337924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:08.343872 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ffcbee33-f2c0-48c1-afd8-614cc26b188a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.157829 Sep 27 09:58:08.344558 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 273/558] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:08 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:08.349331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:08.350075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:08.357343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.357925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.358426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.359204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.359673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.360222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.375445 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] HTTP exception thrown: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Request method failure captured: Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc HTTP/1.1 Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj5jSnuM9Zwz0foBKu5jABAQUgw8wTL0EwJkY5HQ7LQe4gK6BX8DeDMmIBcZM_rJvG9FhHzzDTGDUDodnCyCIy5tpgEbxXeiVZyoCAX68HqXNzXBsZuFCAfa5iwyQsinEdjAd-M-I62Igeh4KLv-wzx9hI1RjjH2hr9CVIE1XJSZgk Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:58:08.377302 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3132070d235941498e3f64d9d7075f6b"}]}] Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7ba3c51770094c4ea7fec1e597ae13a0 Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.378838 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc'} Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:08.379687 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.380858 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.381884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Returning 404 to user: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:08.381884 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3488f0ab-3d5e-41fa-8511-32fa25c21126 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 404 len: 111 microversion: 2.1 time: 0.034623 Sep 27 09:58:08.381884 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 274/559] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:08 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:08.408283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:08.410091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:08.435841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.436520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.437012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.437791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.438261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.438697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.467305 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] HTTP exception thrown: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Request method failure captured: Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc HTTP/1.1 Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj5jSnuM9Zwz0foBKu5jABAQUgw8wTL0EwJkY5HQ7LQe4gK6BX8DeDMmIBcZM_rJvG9FhHzzDTGDUDodnCyCIy5tpgEbxXeiVZyoCAX68HqXNzXBsZuFCAfa5iwyQsinEdjAd-M-I62Igeh4KLv-wzx9hI1RjjH2hr9CVIE1XJSZgk Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:58:08.469179 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3132070d235941498e3f64d9d7075f6b"}]}] Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7ba3c51770094c4ea7fec1e597ae13a0 Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.470737 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc'} Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:08.471456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.472603 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.475480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Returning 404 to user: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:08.476799 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6bd5779e-2712-4980-920f-56d2809410c0 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "DELETE /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 404 len: 111 microversion: 2.1 time: 0.087091 Sep 27 09:58:08.478228 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 275/560] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:58:08 2022] DELETE /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 111 bytes in 89 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 09:58:08.492990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:08.493643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:08.515806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.516957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.517487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.518417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.519048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.519937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.551909 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] HTTP exception thrown: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Request method failure captured: Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc HTTP/1.1 Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj5jSnuM9Zwz0foBKu5jABAQUgw8wTL0EwJkY5HQ7LQe4gK6BX8DeDMmIBcZM_rJvG9FhHzzDTGDUDodnCyCIy5tpgEbxXeiVZyoCAX68HqXNzXBsZuFCAfa5iwyQsinEdjAd-M-I62Igeh4KLv-wzx9hI1RjjH2hr9CVIE1XJSZgk Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:58:08.553822 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3132070d235941498e3f64d9d7075f6b"}]}] Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7ba3c51770094c4ea7fec1e597ae13a0 Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.555371 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc'} Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:08.556248 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. Sep 27 09:58:08.557561 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.559031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Returning 404 to user: Instance 72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:08.559031 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-154e1339-6eef-4174-a3cd-cc351a63c4c6 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc" status: 404 len: 111 microversion: 2.1 time: 0.072385 Sep 27 09:58:08.567223 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 276/561] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:08 2022] GET /compute/v2.1/servers/72be5b11-bccf-4bf4-b3c1-1ea08a0bbecc => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:08.574234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:08.574747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:08.594712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.595219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.595663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.596391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.596880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.599484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.640125 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] HTTP exception thrown: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Request method failure captured: Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb HTTP/1.1 Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj5jSnuM9Zwz0foBKu5jABAQUgw8wTL0EwJkY5HQ7LQe4gK6BX8DeDMmIBcZM_rJvG9FhHzzDTGDUDodnCyCIy5tpgEbxXeiVZyoCAX68HqXNzXBsZuFCAfa5iwyQsinEdjAd-M-I62Igeh4KLv-wzx9hI1RjjH2hr9CVIE1XJSZgk Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:58:08.643536 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3132070d235941498e3f64d9d7075f6b"}]}] Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7ba3c51770094c4ea7fec1e597ae13a0 Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:58:08.644896 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2236fa92-0b7d-4029-a7fd-ffbb09df76bb'} Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:08.645610 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:58:08.647047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.648409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Returning 404 to user: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:08.648409 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aacce15b-1a99-4dc3-8b0d-a477a1982041 tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "DELETE /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 404 len: 111 microversion: 2.1 time: 0.076395 Sep 27 09:58:08.648409 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 277/562] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:58:08 2022] DELETE /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:08.661083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:08.662102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] [instance: 2236fa92-0b7d-4029-a7fd-ffbb09df76bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:08.680483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.681290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.682051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.682815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.686244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.686770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.707520 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] HTTP exception thrown: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Request method failure captured: Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb HTTP/1.1 Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsj5jSnuM9Zwz0foBKu5jABAQUgw8wTL0EwJkY5HQ7LQe4gK6BX8DeDMmIBcZM_rJvG9FhHzzDTGDUDodnCyCIy5tpgEbxXeiVZyoCAX68HqXNzXBsZuFCAfa5iwyQsinEdjAd-M-I62Igeh4KLv-wzx9hI1RjjH2hr9CVIE1XJSZgk Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:58:08.711253 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3132070d235941498e3f64d9d7075f6b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3132070d235941498e3f64d9d7075f6b"}]}] Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 3132070d235941498e3f64d9d7075f6b Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersAdminTestJSON-222543234 Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7ba3c51770094c4ea7fec1e597ae13a0 Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersAdminTestJSON-222543234-project Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:58:08.712580 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2236fa92-0b7d-4029-a7fd-ffbb09df76bb'} Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:08.713342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. Sep 27 09:58:08.714486 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:08.715511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] Returning 404 to user: Instance 2236fa92-0b7d-4029-a7fd-ffbb09df76bb could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:08.716191 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ec9e393d-641e-4a40-9b8a-f04cc1eee72c tempest-DeleteServersAdminTestJSON-222543234 tempest-DeleteServersAdminTestJSON-222543234-project] 10.222.0.26 "GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb" status: 404 len: 111 microversion: 2.1 time: 0.060745 Sep 27 09:58:08.717494 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 278/563] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:08 2022] GET /compute/v2.1/servers/2236fa92-0b7d-4029-a7fd-ffbb09df76bb => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:08.764287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d4b57f3-3656-4c6c-b4d7-bd6b3e289d89 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:08.764955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4d4b57f3-3656-4c6c-b4d7-bd6b3e289d89 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:08.787380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d4b57f3-3656-4c6c-b4d7-bd6b3e289d89 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.788618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d4b57f3-3656-4c6c-b4d7-bd6b3e289d89 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.788618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d4b57f3-3656-4c6c-b4d7-bd6b3e289d89 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.789459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d4b57f3-3656-4c6c-b4d7-bd6b3e289d89 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:08.790091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d4b57f3-3656-4c6c-b4d7-bd6b3e289d89 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:08.790645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d4b57f3-3656-4c6c-b4d7-bd6b3e289d89 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:08.815130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fe70b9c4-b2db-4f7d-86b0-573b29ee1a91 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:08.840970 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fe70b9c4-b2db-4f7d-86b0-573b29ee1a91 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1318 microversion: 2.1 time: 0.755111 Sep 27 09:58:08.859598 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 286/564] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:08 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1318 bytes in 774 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:09.117930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4d4b57f3-3656-4c6c-b4d7-bd6b3e289d89 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:09.120130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-71b24f19-1313-4b22-8b68-5e903f758d5e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:09.121127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-71b24f19-1313-4b22-8b68-5e903f758d5e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:09.128009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71b24f19-1313-4b22-8b68-5e903f758d5e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:09.128962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71b24f19-1313-4b22-8b68-5e903f758d5e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:09.130328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71b24f19-1313-4b22-8b68-5e903f758d5e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:09.131272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71b24f19-1313-4b22-8b68-5e903f758d5e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:09.131489 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4d4b57f3-3656-4c6c-b4d7-bd6b3e289d89 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.385113 Sep 27 09:58:09.132304 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 279/565] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:08 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:09.133156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71b24f19-1313-4b22-8b68-5e903f758d5e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:09.133810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71b24f19-1313-4b22-8b68-5e903f758d5e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:09.197383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d5353664-eec1-4533-ada0-3ff73236838b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:09.198325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d5353664-eec1-4533-ada0-3ff73236838b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:09.217565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5353664-eec1-4533-ada0-3ff73236838b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:09.218079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5353664-eec1-4533-ada0-3ff73236838b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:09.218550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5353664-eec1-4533-ada0-3ff73236838b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:09.219242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5353664-eec1-4533-ada0-3ff73236838b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:09.223584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5353664-eec1-4533-ada0-3ff73236838b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:09.224236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5353664-eec1-4533-ada0-3ff73236838b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:09.306240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-71b24f19-1313-4b22-8b68-5e903f758d5e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:09.316420 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-71b24f19-1313-4b22-8b68-5e903f758d5e tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1434 microversion: 2.1 time: 0.214379 Sep 27 09:58:09.317674 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 287/566] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:09 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1434 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:09.323481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c2a44cf3-1032-4a07-8de5-7beb18d372a4 req-263e3f48-22f0-4597-9183-8c7f53508eed service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ee40f956-0a40-43c4-a775-a978e8403d1d", "name": "network-vif-deleted", "tag": "a921aca9-77a9-4abe-9b1a-34f56baca86d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:09.334718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c2a44cf3-1032-4a07-8de5-7beb18d372a4 req-263e3f48-22f0-4597-9183-8c7f53508eed service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:09.336457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c2a44cf3-1032-4a07-8de5-7beb18d372a4 req-263e3f48-22f0-4597-9183-8c7f53508eed service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:09.338434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c2a44cf3-1032-4a07-8de5-7beb18d372a4 req-263e3f48-22f0-4597-9183-8c7f53508eed service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:09.356386 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c2a44cf3-1032-4a07-8de5-7beb18d372a4 req-263e3f48-22f0-4597-9183-8c7f53508eed service nova] Creating event network-vif-deleted:a921aca9-77a9-4abe-9b1a-34f56baca86d for instance ee40f956-0a40-43c4-a775-a978e8403d1d on nch2-master-1 Sep 27 09:58:09.375387 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c2a44cf3-1032-4a07-8de5-7beb18d372a4 req-263e3f48-22f0-4597-9183-8c7f53508eed service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054445 Sep 27 09:58:09.376098 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 288/567] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:09.390739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:09.394468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:09.411343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:09.413466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:09.415043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:09.418063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:09.418734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:09.419454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:09.547913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d5353664-eec1-4533-ada0-3ff73236838b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:09.565140 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d5353664-eec1-4533-ada0-3ff73236838b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1319 microversion: 2.1 time: 0.375940 Sep 27 09:58:09.572173 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 280/568] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:09 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1319 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:09.620988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:09.643824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:09.660732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:09.671419 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7866cfd0-9218-46b0-bd2d-a9d1ab56e5a5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.290248 Sep 27 09:58:09.672032 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 289/569] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:09 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:09.864364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a5d578be-15b0-4c84-a931-f728e841f8cd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:09.865219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a5d578be-15b0-4c84-a931-f728e841f8cd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:09.873590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5d578be-15b0-4c84-a931-f728e841f8cd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:09.875895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5d578be-15b0-4c84-a931-f728e841f8cd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:09.877422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5d578be-15b0-4c84-a931-f728e841f8cd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:09.883269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5d578be-15b0-4c84-a931-f728e841f8cd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:09.884043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5d578be-15b0-4c84-a931-f728e841f8cd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:09.884708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5d578be-15b0-4c84-a931-f728e841f8cd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:10.160766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9c63cddc-b70e-4e2d-afd5-a7e858300cc3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:10.166210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9c63cddc-b70e-4e2d-afd5-a7e858300cc3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:10.179125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c63cddc-b70e-4e2d-afd5-a7e858300cc3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:10.180151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c63cddc-b70e-4e2d-afd5-a7e858300cc3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:10.182007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c63cddc-b70e-4e2d-afd5-a7e858300cc3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:10.183540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c63cddc-b70e-4e2d-afd5-a7e858300cc3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:10.187988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c63cddc-b70e-4e2d-afd5-a7e858300cc3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:10.188800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c63cddc-b70e-4e2d-afd5-a7e858300cc3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:10.351696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a5d578be-15b0-4c84-a931-f728e841f8cd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:10.366566 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a5d578be-15b0-4c84-a931-f728e841f8cd tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1318 microversion: 2.1 time: 0.506142 Sep 27 09:58:10.367266 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 281/570] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:09 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1318 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:10.372113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:10.377766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:10.397740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:10.399108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:10.399777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:10.401121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:10.401828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:10.402639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:10.507483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:10.508181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:10.508623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:10.517514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lazy-loading 'fault' on Instance uuid ee40f956-0a40-43c4-a775-a978e8403d1d {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:10.654108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9c63cddc-b70e-4e2d-afd5-a7e858300cc3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:10.666323 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9c63cddc-b70e-4e2d-afd5-a7e858300cc3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.514580 Sep 27 09:58:10.666923 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 290/571] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:10 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:10.682758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-10daf250-fe4d-4fe1-9fdf-3622881261cf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:10.685230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-10daf250-fe4d-4fe1-9fdf-3622881261cf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:10.701594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10daf250-fe4d-4fe1-9fdf-3622881261cf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:10.705797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10daf250-fe4d-4fe1-9fdf-3622881261cf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:10.707744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10daf250-fe4d-4fe1-9fdf-3622881261cf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:10.710979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10daf250-fe4d-4fe1-9fdf-3622881261cf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:10.711659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10daf250-fe4d-4fe1-9fdf-3622881261cf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:10.713222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10daf250-fe4d-4fe1-9fdf-3622881261cf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:10.720045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd7d3aef1d4da47b2ab5454c3eb0346a6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b33d41b411144dbbbdb012e694857d27', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:10.735270 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fafae553-3cca-4da5-b41d-de78a9f2b3de tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 200 len: 1311 microversion: 2.1 time: 0.365974 Sep 27 09:58:10.737854 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 282/572] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:10 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 1311 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:10.760587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:10.761831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:10.777149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:10.778240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:10.779384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:10.780796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:10.781667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:10.782456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:11.516207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-10daf250-fe4d-4fe1-9fdf-3622881261cf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:11.563386 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-10daf250-fe4d-4fe1-9fdf-3622881261cf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1319 microversion: 2.1 time: 0.885384 Sep 27 09:58:11.564748 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 291/573] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:10 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1319 bytes in 887 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:11.571341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-17f8a552-9311-457b-a992-ff0ca71e945d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:11.572422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-17f8a552-9311-457b-a992-ff0ca71e945d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:11.579232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17f8a552-9311-457b-a992-ff0ca71e945d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:11.580993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17f8a552-9311-457b-a992-ff0ca71e945d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:11.582265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17f8a552-9311-457b-a992-ff0ca71e945d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:11.583915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17f8a552-9311-457b-a992-ff0ca71e945d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:11.585152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17f8a552-9311-457b-a992-ff0ca71e945d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:11.587980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17f8a552-9311-457b-a992-ff0ca71e945d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:11.717605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:11.735912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:11.737853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:11.745426 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-50f8ee28-171b-4baf-9da4-cc99a7c159f5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 1.005861 Sep 27 09:58:11.747827 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 283/574] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:10 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 1009 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:11.757630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-12f0c7ea-5ee5-4f1e-b500-a5c929da969d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:11.760739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-12f0c7ea-5ee5-4f1e-b500-a5c929da969d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:11.775506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12f0c7ea-5ee5-4f1e-b500-a5c929da969d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:11.776434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12f0c7ea-5ee5-4f1e-b500-a5c929da969d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:11.777395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12f0c7ea-5ee5-4f1e-b500-a5c929da969d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:11.778368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12f0c7ea-5ee5-4f1e-b500-a5c929da969d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:11.779056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12f0c7ea-5ee5-4f1e-b500-a5c929da969d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:11.779830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12f0c7ea-5ee5-4f1e-b500-a5c929da969d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:12.054712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17f8a552-9311-457b-a992-ff0ca71e945d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:12.068076 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-17f8a552-9311-457b-a992-ff0ca71e945d tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1318 microversion: 2.1 time: 0.501432 Sep 27 09:58:12.070390 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 292/575] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:11 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1318 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:12.086518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:12.087544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] [instance: ee40f956-0a40-43c4-a775-a978e8403d1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:12.097785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:12.099292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:12.100013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:12.101293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:12.102046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:12.102730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:12.121485 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] HTTP exception thrown: Instance ee40f956-0a40-43c4-a775-a978e8403d1d could not be found. Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Request method failure captured: Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d HTTP/1.1 Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjkKnEzPQ8NhXvM0Jes3Z8mp4M91AxwhKsOv1TjyFg3pda3-7f4esRqTqojocxEG1nIYOvYRER-AUcD_JoYsO9KRIT5FR0JUmQ8B7Aln6yQd90BTZsZr-kO2vG-fosa1uJx5kJs3YOXTc2sCNUzdkFSkhRINF7_qNv_4vE5YuBHx-w Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b33d41b411144dbbbdb012e694857d27 Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-591068969 Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:58:12.123563 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b33d41b411144dbbbdb012e694857d27"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b33d41b411144dbbbdb012e694857d27"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b33d41b411144dbbbdb012e694857d27"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b33d41b411144dbbbdb012e694857d27"}]}] Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersAdminNegativeTestJSON-591068969 Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b33d41b411144dbbbdb012e694857d27 Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-591068969 Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersAdminNegativeTestJSON-591068969-project Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: d7d3aef1d4da47b2ab5454c3eb0346a6 Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersAdminNegativeTestJSON-591068969-project Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance ee40f956-0a40-43c4-a775-a978e8403d1d could not be found. Sep 27 09:58:12.125010 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'ee40f956-0a40-43c4-a775-a978e8403d1d'} Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ee40f956-0a40-43c4-a775-a978e8403d1d could not be found. Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:12.125729 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ee40f956-0a40-43c4-a775-a978e8403d1d could not be found. Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ee40f956-0a40-43c4-a775-a978e8403d1d could not be found. Sep 27 09:58:12.126882 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:12.129827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] Returning 404 to user: Instance ee40f956-0a40-43c4-a775-a978e8403d1d could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:12.132140 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-30cea321-2914-438d-9997-3deac64c1443 tempest-ServersAdminNegativeTestJSON-591068969 tempest-ServersAdminNegativeTestJSON-591068969-project] 10.222.0.26 "GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d" status: 404 len: 111 microversion: 2.1 time: 0.060083 Sep 27 09:58:12.133950 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 293/576] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:12 2022] GET /compute/v2.1/servers/ee40f956-0a40-43c4-a775-a978e8403d1d => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:12.364932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-12f0c7ea-5ee5-4f1e-b500-a5c929da969d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:12.379766 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-12f0c7ea-5ee5-4f1e-b500-a5c929da969d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.630365 Sep 27 09:58:12.380391 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 284/577] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:11 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:12.587940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-798da3ab-44da-46f3-8bff-16e62db32b7e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:12.588677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-798da3ab-44da-46f3-8bff-16e62db32b7e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:12.606283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798da3ab-44da-46f3-8bff-16e62db32b7e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:12.607375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798da3ab-44da-46f3-8bff-16e62db32b7e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:12.608103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798da3ab-44da-46f3-8bff-16e62db32b7e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:12.609285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798da3ab-44da-46f3-8bff-16e62db32b7e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:12.611413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798da3ab-44da-46f3-8bff-16e62db32b7e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:12.612103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798da3ab-44da-46f3-8bff-16e62db32b7e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:12.785957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:12.786566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:12.807189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:12.807703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:12.808133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:12.808927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:12.809401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:12.809834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:13.041725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:13.072727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=32 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:13.112354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:13.132229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:13.137535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:13.158838 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e87a8454-f6a9-426d-861e-0e10f5b26cb9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.376834 Sep 27 09:58:13.165919 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 285/578] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:12 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:13.186900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa5f9139-a390-49f9-a432-a93ec565dcb5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:13.187539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aa5f9139-a390-49f9-a432-a93ec565dcb5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:13.213069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa5f9139-a390-49f9-a432-a93ec565dcb5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:13.213604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa5f9139-a390-49f9-a432-a93ec565dcb5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:13.214047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa5f9139-a390-49f9-a432-a93ec565dcb5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:13.214785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa5f9139-a390-49f9-a432-a93ec565dcb5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:13.215249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa5f9139-a390-49f9-a432-a93ec565dcb5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:13.215744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa5f9139-a390-49f9-a432-a93ec565dcb5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:13.350332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-798da3ab-44da-46f3-8bff-16e62db32b7e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:13.384509 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-798da3ab-44da-46f3-8bff-16e62db32b7e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1319 microversion: 2.1 time: 0.801412 Sep 27 09:58:13.406218 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 294/579] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:12 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1319 bytes in 817 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:13.432519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-261e9cc9-ad7c-41ef-9795-dd7576cc5bb1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:13.434924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-261e9cc9-ad7c-41ef-9795-dd7576cc5bb1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:13.472985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-261e9cc9-ad7c-41ef-9795-dd7576cc5bb1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:13.478562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-261e9cc9-ad7c-41ef-9795-dd7576cc5bb1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:13.482531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-261e9cc9-ad7c-41ef-9795-dd7576cc5bb1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:13.486409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-261e9cc9-ad7c-41ef-9795-dd7576cc5bb1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:13.489245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-261e9cc9-ad7c-41ef-9795-dd7576cc5bb1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:13.492843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-261e9cc9-ad7c-41ef-9795-dd7576cc5bb1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:13.771633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=34 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:13.822122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aa5f9139-a390-49f9-a432-a93ec565dcb5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:13.842932 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aa5f9139-a390-49f9-a432-a93ec565dcb5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1318 microversion: 2.1 time: 0.674577 Sep 27 09:58:13.867011 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 286/580] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:13 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1318 bytes in 701 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:13.911419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-261e9cc9-ad7c-41ef-9795-dd7576cc5bb1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:13.936719 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-261e9cc9-ad7c-41ef-9795-dd7576cc5bb1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.525664 Sep 27 09:58:13.946865 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 295/581] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:13 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:14.021106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1211dd52-b555-439a-99ea-c5a8c9fa0a59 req-4596b5d1-a5bc-4037-b4c4-39cbcc8893a6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "67574e92-7d2a-4f14-8448-5ff3278cb803", "name": "network-vif-plugged", "status": "completed", "tag": "aa655eea-e771-44bf-9f90-0e21077a0242"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:14.027415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:14.043622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1211dd52-b555-439a-99ea-c5a8c9fa0a59 req-4596b5d1-a5bc-4037-b4c4-39cbcc8893a6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:14.049165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1211dd52-b555-439a-99ea-c5a8c9fa0a59 req-4596b5d1-a5bc-4037-b4c4-39cbcc8893a6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:14.050062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1211dd52-b555-439a-99ea-c5a8c9fa0a59 req-4596b5d1-a5bc-4037-b4c4-39cbcc8893a6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:14.082380 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1211dd52-b555-439a-99ea-c5a8c9fa0a59 req-4596b5d1-a5bc-4037-b4c4-39cbcc8893a6 service nova] Creating event network-vif-plugged:aa655eea-e771-44bf-9f90-0e21077a0242 for instance 67574e92-7d2a-4f14-8448-5ff3278cb803 on nch2-master-1 Sep 27 09:58:14.103194 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1211dd52-b555-439a-99ea-c5a8c9fa0a59 req-4596b5d1-a5bc-4037-b4c4-39cbcc8893a6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074154 Sep 27 09:58:14.104896 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 287/582] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:14.213303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:14.214186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:14.235780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:14.238335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:14.239123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:14.259575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:14.260083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:14.260542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:14.413617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32aeeb68-24cf-4711-ab52-e80b28ae49fd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:14.414445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32aeeb68-24cf-4711-ab52-e80b28ae49fd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:14.425660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32aeeb68-24cf-4711-ab52-e80b28ae49fd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:14.426170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32aeeb68-24cf-4711-ab52-e80b28ae49fd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:14.426623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32aeeb68-24cf-4711-ab52-e80b28ae49fd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:14.427380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32aeeb68-24cf-4711-ab52-e80b28ae49fd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:14.427857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32aeeb68-24cf-4711-ab52-e80b28ae49fd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:14.428716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32aeeb68-24cf-4711-ab52-e80b28ae49fd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:14.740697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:14.740944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32aeeb68-24cf-4711-ab52-e80b28ae49fd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:14.752193 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32aeeb68-24cf-4711-ab52-e80b28ae49fd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1319 microversion: 2.1 time: 0.343277 Sep 27 09:58:14.755301 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 288/583] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:14 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1319 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:14.771039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:14.774529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:14.785785 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6980be45-85ea-4d14-ac56-1aca555ac2e5 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.579383 Sep 27 09:58:14.797734 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 296/584] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:14 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 09:58:14.896456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b66908c5-cd8a-49d4-ad46-5347fc9308a7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:14.901229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b66908c5-cd8a-49d4-ad46-5347fc9308a7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:14.935941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b66908c5-cd8a-49d4-ad46-5347fc9308a7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:14.937185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b66908c5-cd8a-49d4-ad46-5347fc9308a7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:14.939471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b66908c5-cd8a-49d4-ad46-5347fc9308a7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:14.943335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b66908c5-cd8a-49d4-ad46-5347fc9308a7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:14.947238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b66908c5-cd8a-49d4-ad46-5347fc9308a7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:14.948182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b66908c5-cd8a-49d4-ad46-5347fc9308a7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:14.997714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7126f283-d791-4028-85d1-855c839bb012 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:15.003392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7126f283-d791-4028-85d1-855c839bb012 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:15.030458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7126f283-d791-4028-85d1-855c839bb012 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:15.031045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7126f283-d791-4028-85d1-855c839bb012 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:15.031510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7126f283-d791-4028-85d1-855c839bb012 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:15.034807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7126f283-d791-4028-85d1-855c839bb012 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:15.035301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7126f283-d791-4028-85d1-855c839bb012 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:15.035744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7126f283-d791-4028-85d1-855c839bb012 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:15.412508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b66908c5-cd8a-49d4-ad46-5347fc9308a7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:15.426330 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b66908c5-cd8a-49d4-ad46-5347fc9308a7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1318 microversion: 2.1 time: 0.537729 Sep 27 09:58:15.438155 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 289/585] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:14 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1318 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:15.724387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7126f283-d791-4028-85d1-855c839bb012 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:15.764238 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7126f283-d791-4028-85d1-855c839bb012 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1477 microversion: 2.1 time: 0.773380 Sep 27 09:58:15.765446 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 297/586] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:14 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1477 bytes in 779 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:15.786436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bb38a0a3-efc2-4e61-8800-ec7668e56123 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:15.787584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bb38a0a3-efc2-4e61-8800-ec7668e56123 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:15.805877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb38a0a3-efc2-4e61-8800-ec7668e56123 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:15.806507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb38a0a3-efc2-4e61-8800-ec7668e56123 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:15.806960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb38a0a3-efc2-4e61-8800-ec7668e56123 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:15.808303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb38a0a3-efc2-4e61-8800-ec7668e56123 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:15.808807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb38a0a3-efc2-4e61-8800-ec7668e56123 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:15.809249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb38a0a3-efc2-4e61-8800-ec7668e56123 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:15.820886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:15.821938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:15.849106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:15.850025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:15.850963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:15.857019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:15.858216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:15.859203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:16.040188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:16.067643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:16.071279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:16.087835 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a2e69058-e0ab-4417-b43d-b86c6cefe10d tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.272614 Sep 27 09:58:16.089663 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 298/587] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:15 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:16.149946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bb38a0a3-efc2-4e61-8800-ec7668e56123 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:16.163461 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bb38a0a3-efc2-4e61-8800-ec7668e56123 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1319 microversion: 2.1 time: 0.383945 Sep 27 09:58:16.174273 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 290/588] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:15 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1319 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:16.215290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-518cb74b-c448-4a18-be03-8359fd06b1f3 req-594c976e-d1ac-437b-a286-548dee0ff471 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a59c15ae-bd3b-4f14-9311-c8f5f52c44d4", "name": "network-vif-plugged", "status": "completed", "tag": "92de2508-8fe5-415a-987e-c0ff172d47d7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:16.246867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-518cb74b-c448-4a18-be03-8359fd06b1f3 req-594c976e-d1ac-437b-a286-548dee0ff471 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:16.254154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-518cb74b-c448-4a18-be03-8359fd06b1f3 req-594c976e-d1ac-437b-a286-548dee0ff471 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:16.254849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-518cb74b-c448-4a18-be03-8359fd06b1f3 req-594c976e-d1ac-437b-a286-548dee0ff471 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:16.271285 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-518cb74b-c448-4a18-be03-8359fd06b1f3 req-594c976e-d1ac-437b-a286-548dee0ff471 service nova] Creating event network-vif-plugged:92de2508-8fe5-415a-987e-c0ff172d47d7 for instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 on nch1-master-1 Sep 27 09:58:16.279137 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-518cb74b-c448-4a18-be03-8359fd06b1f3 req-594c976e-d1ac-437b-a286-548dee0ff471 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073066 Sep 27 09:58:16.282423 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 299/589] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:16.462924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ebe7f0d6-9651-4f4b-bd7a-227f1d2e7199 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:16.464126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ebe7f0d6-9651-4f4b-bd7a-227f1d2e7199 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:16.476902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebe7f0d6-9651-4f4b-bd7a-227f1d2e7199 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:16.478291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebe7f0d6-9651-4f4b-bd7a-227f1d2e7199 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:16.479015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebe7f0d6-9651-4f4b-bd7a-227f1d2e7199 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:16.484579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebe7f0d6-9651-4f4b-bd7a-227f1d2e7199 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:16.487795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebe7f0d6-9651-4f4b-bd7a-227f1d2e7199 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:16.488813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebe7f0d6-9651-4f4b-bd7a-227f1d2e7199 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:16.755829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ebe7f0d6-9651-4f4b-bd7a-227f1d2e7199 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:16.768359 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ebe7f0d6-9651-4f4b-bd7a-227f1d2e7199 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1318 microversion: 2.1 time: 0.313462 Sep 27 09:58:16.780821 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 300/590] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:16 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1318 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:16.802583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c67b8df7-7046-4087-86f5-bff7d0511241 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:16.804004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c67b8df7-7046-4087-86f5-bff7d0511241 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:16.811701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c67b8df7-7046-4087-86f5-bff7d0511241 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:16.812495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c67b8df7-7046-4087-86f5-bff7d0511241 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:16.813397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c67b8df7-7046-4087-86f5-bff7d0511241 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:16.814926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c67b8df7-7046-4087-86f5-bff7d0511241 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:16.815555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c67b8df7-7046-4087-86f5-bff7d0511241 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:16.816335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c67b8df7-7046-4087-86f5-bff7d0511241 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:17.000385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c67b8df7-7046-4087-86f5-bff7d0511241 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:17.008915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-670d1cdc-f866-440b-8ad5-1c44b40a9004 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": "1024", "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-356534851", "id": 1781803420, "os-flavor-access:is_public": "True"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:17.011955 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c67b8df7-7046-4087-86f5-bff7d0511241 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1476 microversion: 2.1 time: 0.215002 Sep 27 09:58:17.013048 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 301/591] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:16 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1476 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:17.040001 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-670d1cdc-f866-440b-8ad5-1c44b40a9004 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 429 microversion: 2.1 time: 0.644805 Sep 27 09:58:17.044063 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 291/592] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:16 2022] POST /compute/v2.1/flavors => generated 429 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:17.061169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f903b1d-9d63-4932-8954-00cb0e818161 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:17.094333 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9f903b1d-9d63-4932-8954-00cb0e818161 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1781803420" status: 202 len: 0 microversion: 2.1 time: 0.040791 Sep 27 09:58:17.094928 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 302/593] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:17 2022] DELETE /compute/v2.1/flavors/1781803420 => generated 0 bytes in 43 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:17.104974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6988ce2a-efc5-4943-9260-fe0dc3adbef3 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:17.128897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:17.129839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:17.140029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:17.142730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:17.143671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:17.145554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:17.147979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:17.149332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:17.159310 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6988ce2a-efc5-4943-9260-fe0dc3adbef3 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.058721 Sep 27 09:58:17.162218 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 292/594] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:17 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:17.198631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23f4a49d-44ed-4a4a-814d-96c20b4a5364 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-1054013540", "id": 276886037, "swap": 1024, "rxtx_factor": 2, "OS-FLV-EXT-DATA:ephemeral": 10, "os-flavor-access:is_public": "True"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:17.236179 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-23f4a49d-44ed-4a4a-814d-96c20b4a5364 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 414 microversion: 2.1 time: 0.054867 Sep 27 09:58:17.240715 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 293/595] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:17 2022] POST /compute/v2.1/flavors => generated 414 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:17.249552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6dd15130-a2b2-42a3-b5ee-d608f95fad89 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:17.254428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6dd15130-a2b2-42a3-b5ee-d608f95fad89 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:17.273167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dd15130-a2b2-42a3-b5ee-d608f95fad89 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:17.274009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dd15130-a2b2-42a3-b5ee-d608f95fad89 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:17.275361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dd15130-a2b2-42a3-b5ee-d608f95fad89 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:17.278592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dd15130-a2b2-42a3-b5ee-d608f95fad89 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:17.279414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dd15130-a2b2-42a3-b5ee-d608f95fad89 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:17.280913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dd15130-a2b2-42a3-b5ee-d608f95fad89 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:17.305560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:17.323338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:17.325076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:17.336754 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af67bb50-7851-43f4-a868-1001d0d047aa tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.213105 Sep 27 09:58:17.349359 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 303/596] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:17 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:17.363020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4de6b90a-377c-4721-94ce-12b6337708f8 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:17.388845 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4de6b90a-377c-4721-94ce-12b6337708f8 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 6218 microversion: 2.1 time: 0.038226 Sep 27 09:58:17.390152 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 304/597] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:17 2022] GET /compute/v2.1/flavors/detail => generated 6218 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:17.408960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2517a692-980f-4e21-b99d-ea2b2c407dc4 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:17.429929 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2517a692-980f-4e21-b99d-ea2b2c407dc4 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/276886037" status: 202 len: 0 microversion: 2.1 time: 0.024269 Sep 27 09:58:17.430739 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 305/598] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 09:58:17 2022] DELETE /compute/v2.1/flavors/276886037 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:17.442200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a1bcd084-6b35-49f8-834c-8c3d39c561c4 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:17.468907 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a1bcd084-6b35-49f8-834c-8c3d39c561c4 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.031067 Sep 27 09:58:17.469588 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 306/599] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:17 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:17.492774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3322855-a111-4270-a5a8-070e20da424e tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-269612803", "id": 353680647, "os-flavor-access:is_public": "True"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:17.541107 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3322855-a111-4270-a5a8-070e20da424e tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 425 microversion: 2.1 time: 0.051257 Sep 27 09:58:17.544717 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 307/600] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:17 2022] POST /compute/v2.1/flavors => generated 425 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:17.555281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8566c3e5-a7f5-43a0-a3c5-f3faae40411f req-a8739b85-4886-4562-b799-f5ff3e6c6dcb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "706e3141-43af-4a6b-8789-dea152d0312a", "name": "network-vif-unplugged", "status": "completed", "tag": "ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:17.590628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8566c3e5-a7f5-43a0-a3c5-f3faae40411f req-a8739b85-4886-4562-b799-f5ff3e6c6dcb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:17.592823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8566c3e5-a7f5-43a0-a3c5-f3faae40411f req-a8739b85-4886-4562-b799-f5ff3e6c6dcb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:17.593370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8566c3e5-a7f5-43a0-a3c5-f3faae40411f req-a8739b85-4886-4562-b799-f5ff3e6c6dcb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:17.611334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6dd15130-a2b2-42a3-b5ee-d608f95fad89 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:17.626116 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8566c3e5-a7f5-43a0-a3c5-f3faae40411f req-a8739b85-4886-4562-b799-f5ff3e6c6dcb service nova] Creating event network-vif-unplugged:ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4 for instance 706e3141-43af-4a6b-8789-dea152d0312a on nch2-master-1 Sep 27 09:58:17.626405 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6dd15130-a2b2-42a3-b5ee-d608f95fad89 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1469 microversion: 2.1 time: 0.387882 Sep 27 09:58:17.627312 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 294/601] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:17 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1469 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:17.641076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:17.655146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e17fc39f-bded-480e-9ffc-e58b892786a1 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:17.666068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-8566c3e5-a7f5-43a0-a3c5-f3faae40411f req-a8739b85-4886-4562-b799-f5ff3e6c6dcb service nova] Instance 706e3141-43af-4a6b-8789-dea152d0312a is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 09:58:17.699512 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e17fc39f-bded-480e-9ffc-e58b892786a1 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/353680647" status: 202 len: 0 microversion: 2.1 time: 0.056707 Sep 27 09:58:17.701241 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 295/602] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 09:58:17 2022] DELETE /compute/v2.1/flavors/353680647 => generated 0 bytes in 59 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:17.702125 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8566c3e5-a7f5-43a0-a3c5-f3faae40411f req-a8739b85-4886-4562-b799-f5ff3e6c6dcb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.157260 Sep 27 09:58:17.702780 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 308/603] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:17 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:17.707692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f9272b3b-4a67-4270-a8f0-a8c04c34e9c4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:17.712272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f9272b3b-4a67-4270-a8f0-a8c04c34e9c4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: f769af8b-472a-4950-b4ae-8c0183b23c1e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:17.721628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ddd9b3ab-a358-46d6-924b-a70418690b5a tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:17.729533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9272b3b-4a67-4270-a8f0-a8c04c34e9c4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:17.730229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9272b3b-4a67-4270-a8f0-a8c04c34e9c4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:17.730754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9272b3b-4a67-4270-a8f0-a8c04c34e9c4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:17.732602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9272b3b-4a67-4270-a8f0-a8c04c34e9c4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:17.736155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9272b3b-4a67-4270-a8f0-a8c04c34e9c4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:17.736853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9272b3b-4a67-4270-a8f0-a8c04c34e9c4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:17.760255 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ddd9b3ab-a358-46d6-924b-a70418690b5a tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.052823 Sep 27 09:58:17.770361 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 309/604] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:17 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:17.783833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1e15bdb0-ea77-490e-822b-6f4aaef75e7e tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-180868152", "id": null, "os-flavor-access:is_public": "True"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:17.811633 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1e15bdb0-ea77-490e-822b-6f4aaef75e7e tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 506 microversion: 2.1 time: 0.032000 Sep 27 09:58:17.819705 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 310/605] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:17 2022] POST /compute/v2.1/flavors => generated 506 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:17.826874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82327654-d738-4b58-9814-9a57e45d401a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:17.827650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-82327654-d738-4b58-9814-9a57e45d401a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:17.837623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82327654-d738-4b58-9814-9a57e45d401a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:17.838168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82327654-d738-4b58-9814-9a57e45d401a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:17.839581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82327654-d738-4b58-9814-9a57e45d401a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:17.839581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82327654-d738-4b58-9814-9a57e45d401a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:17.839826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82327654-d738-4b58-9814-9a57e45d401a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:17.840229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82327654-d738-4b58-9814-9a57e45d401a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:18.043145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-82327654-d738-4b58-9814-9a57e45d401a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:18.052226 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-82327654-d738-4b58-9814-9a57e45d401a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1318 microversion: 2.1 time: 0.233992 Sep 27 09:58:18.056933 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 311/606] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:17 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1318 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:18.066737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1bb9fb31-574e-4bde-8d8e-85fd25b7fcf9 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:18.108176 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1bb9fb31-574e-4bde-8d8e-85fd25b7fcf9 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/3bc83a95-4e0d-4806-a23f-5831e6450f13" status: 202 len: 0 microversion: 2.1 time: 0.049653 Sep 27 09:58:18.114540 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 312/607] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:58:18 2022] DELETE /compute/v2.1/flavors/3bc83a95-4e0d-4806-a23f-5831e6450f13 => generated 0 bytes in 57 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:18.123929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f997b41-0517-4534-a8e7-e22a1a7175fb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:18.127668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7f997b41-0517-4534-a8e7-e22a1a7175fb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:18.143903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f997b41-0517-4534-a8e7-e22a1a7175fb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:18.146876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f997b41-0517-4534-a8e7-e22a1a7175fb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:18.147880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f997b41-0517-4534-a8e7-e22a1a7175fb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:18.150647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f997b41-0517-4534-a8e7-e22a1a7175fb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:18.153389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f997b41-0517-4534-a8e7-e22a1a7175fb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:18.154118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f997b41-0517-4534-a8e7-e22a1a7175fb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:18.175546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f9272b3b-4a67-4270-a8f0-a8c04c34e9c4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:18.190447 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f9272b3b-4a67-4270-a8f0-a8c04c34e9c4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e" status: 200 len: 1469 microversion: 2.1 time: 0.487885 Sep 27 09:58:18.191051 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 296/608] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:17 2022] GET /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e => generated 1469 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:18.220158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-595b8b13-3bed-4f39-b243-0840c15ed8af tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:18.276954 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-595b8b13-3bed-4f39-b243-0840c15ed8af tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.068383 Sep 27 09:58:18.277981 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 297/609] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:18 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:18.285906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:18.293221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:18.294190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Searching by: {'reservation_id': 'r-p98p9fo6', 'deleted': False, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 09:58:18.308576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:18.309135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:18.309775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:18.310701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:18.311498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:18.311935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:18.408920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7f997b41-0517-4534-a8e7-e22a1a7175fb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:18.418504 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7f997b41-0517-4534-a8e7-e22a1a7175fb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1476 microversion: 2.1 time: 0.302503 Sep 27 09:58:18.422988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:18.424253 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 313/610] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:18 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1476 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:18.429661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9366daec-8438-4ac8-80d1-432d4242a02a tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-635533792", "id": "1412a9e4-2098-4d2e-955d-3667bceb92c6", "os-flavor-access:is_public": "True"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:18.465037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 09:58:18.470878 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9366daec-8438-4ac8-80d1-432d4242a02a tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 506 microversion: 2.1 time: 0.045165 Sep 27 09:58:18.471772 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 314/611] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:18 2022] POST /compute/v2.1/flavors => generated 506 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:18.483705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:18.484631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:18.500271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:18.501139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:18.501750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:18.502660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:18.503295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:18.503889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:18.576844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:18.594768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:18.595567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:18.596215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:18.635872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:18.647588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:18.649442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:18.658578 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-79ee8bdb-eea8-4b46-bcaa-445b759cba4f tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.185581 Sep 27 09:58:18.659198 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 315/612] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:18 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:18.669364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-816746b5-08ff-417d-8d8c-e546a1bd1f79 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:18.713808 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-816746b5-08ff-417d-8d8c-e546a1bd1f79 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1412a9e4-2098-4d2e-955d-3667bceb92c6" status: 202 len: 0 microversion: 2.1 time: 0.053084 Sep 27 09:58:18.715775 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 316/613] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:58:18 2022] DELETE /compute/v2.1/flavors/1412a9e4-2098-4d2e-955d-3667bceb92c6 => generated 0 bytes in 56 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:18.723867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25a063e2-8450-4776-af95-ff5722e1d2bc tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:18.730357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=85,nova_api:DELETE=12,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:18.751601 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-25a063e2-8450-4776-af95-ff5722e1d2bc tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.030534 Sep 27 09:58:18.766105 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 317/614] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:18 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:18.779001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e91671c-6054-4740-864b-05fb6534ff18 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-587397318", "id": 1890779964, "os-flavor-access:is_public": "True"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:18.807426 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e91671c-6054-4740-864b-05fb6534ff18 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 413 microversion: 2.1 time: 0.032812 Sep 27 09:58:18.815283 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 318/615] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:18 2022] POST /compute/v2.1/flavors => generated 413 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:18.820516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fb494277-c703-4546-8200-d4d5504979a3 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:18.828176 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fb494277-c703-4546-8200-d4d5504979a3 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/1890779964" status: 200 len: 413 microversion: 2.1 time: 0.011694 Sep 27 09:58:18.828798 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 319/616] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 09:58:18 2022] GET /compute/v2.1/flavors/1890779964 => generated 413 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:19.191273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c3d8f66d-dba9-405c-93e3-7f685f84288f tempest-FlavorsAdminTestJSON-1731705636 tempest-FlavorsAdminTestJSON-1731705636-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:19.265012 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c3d8f66d-dba9-405c-93e3-7f685f84288f tempest-FlavorsAdminTestJSON-1731705636 tempest-FlavorsAdminTestJSON-1731705636-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 6217 microversion: 2.1 time: 0.426978 Sep 27 09:58:19.285832 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 320/617] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:18 2022] GET /compute/v2.1/flavors/detail => generated 6217 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:19.297603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9be5725-2af2-465a-ae4e-58ba0be2fd33 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:19.298247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c9be5725-2af2-465a-ae4e-58ba0be2fd33 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:19.313102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be5725-2af2-465a-ae4e-58ba0be2fd33 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:19.313610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be5725-2af2-465a-ae4e-58ba0be2fd33 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:19.314051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be5725-2af2-465a-ae4e-58ba0be2fd33 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:19.314783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be5725-2af2-465a-ae4e-58ba0be2fd33 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:19.315241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be5725-2af2-465a-ae4e-58ba0be2fd33 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:19.323890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9be5725-2af2-465a-ae4e-58ba0be2fd33 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:19.326265 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9910da79-0acc-412b-a67d-d90bd0017b39 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/detail?reservation_id=r-p98p9fo6" status: 200 len: 2931 microversion: 2.1 time: 1.045322 Sep 27 09:58:19.329809 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 298/618] 10.222.0.26 () {58 vars in 1197 bytes} [Tue Sep 27 09:58:18 2022] GET /compute/v2.1/servers/detail?reservation_id=r-p98p9fo6 => generated 2931 bytes in 1051 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:19.343614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b9804001-a8d6-4412-8b3a-e2b3c6ac8325 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:19.386335 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b9804001-a8d6-4412-8b3a-e2b3c6ac8325 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1890779964" status: 202 len: 0 microversion: 2.1 time: 0.049298 Sep 27 09:58:19.387667 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 299/619] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:19 2022] DELETE /compute/v2.1/flavors/1890779964 => generated 0 bytes in 57 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:19.400268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-97538bf3-1762-4752-b8e1-83592d39114a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:19.425379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.server_groups [None req-97538bf3-1762-4752-b8e1-83592d39114a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Querying cell ed1088da-caae-4426-8048-9cba90e7755c(cell1) for 2 instances {{(pid=145969) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Sep 27 09:58:19.426373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97538bf3-1762-4752-b8e1-83592d39114a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:19.427057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97538bf3-1762-4752-b8e1-83592d39114a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:19.427683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97538bf3-1762-4752-b8e1-83592d39114a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:19.479960 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-97538bf3-1762-4752-b8e1-83592d39114a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/os-server-groups/86f44e48-288d-42e4-91ee-4d99b6fa5532" status: 200 len: 271 microversion: 2.1 time: 0.090134 Sep 27 09:58:19.485596 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 300/620] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 09:58:19 2022] GET /compute/v2.1/os-server-groups/86f44e48-288d-42e4-91ee-4d99b6fa5532 => generated 271 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:19.490598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e74a4d3d-9f49-4801-b2e0-0c3afe90e20e tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:19.513499 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e74a4d3d-9f49-4801-b2e0-0c3afe90e20e tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.026381 Sep 27 09:58:19.514428 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 301/621] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:19 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:19.519353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ba7d608-ee09-4267-98d3-5281f4c9404c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:19.520226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ba7d608-ee09-4267-98d3-5281f4c9404c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:19.530802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ba7d608-ee09-4267-98d3-5281f4c9404c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:19.533269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ba7d608-ee09-4267-98d3-5281f4c9404c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:19.534125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ba7d608-ee09-4267-98d3-5281f4c9404c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:19.535406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ba7d608-ee09-4267-98d3-5281f4c9404c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:19.537634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ba7d608-ee09-4267-98d3-5281f4c9404c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:19.538747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ba7d608-ee09-4267-98d3-5281f4c9404c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:19.619551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9be5725-2af2-465a-ae4e-58ba0be2fd33 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:19.631857 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9be5725-2af2-465a-ae4e-58ba0be2fd33 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1468 microversion: 2.1 time: 0.344200 Sep 27 09:58:19.632620 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 321/622] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:19 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1468 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:19.638725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dfd16b82-1dc9-40cc-b273-7dcd69a8a0d4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:19.640605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dfd16b82-1dc9-40cc-b273-7dcd69a8a0d4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:19.647701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfd16b82-1dc9-40cc-b273-7dcd69a8a0d4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:19.648204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfd16b82-1dc9-40cc-b273-7dcd69a8a0d4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:19.649293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfd16b82-1dc9-40cc-b273-7dcd69a8a0d4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:19.650063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfd16b82-1dc9-40cc-b273-7dcd69a8a0d4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:19.650535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfd16b82-1dc9-40cc-b273-7dcd69a8a0d4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:19.651043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfd16b82-1dc9-40cc-b273-7dcd69a8a0d4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:19.959537 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dfd16b82-1dc9-40cc-b273-7dcd69a8a0d4 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 200 len: 1620 microversion: 2.1 time: 0.324641 Sep 27 09:58:19.965820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2ba7d608-ee09-4267-98d3-5281f4c9404c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:19.967060 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 322/623] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:19 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 1620 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:19.972016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29aa4e17-ca62-4715-ac4b-7d7484078249 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAdminTestJSON-flavor-257663283", "id": 1592566954, "os-flavor-access:is_public": "False"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:20.000013 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ba7d608-ee09-4267-98d3-5281f4c9404c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1476 microversion: 2.1 time: 0.484169 Sep 27 09:58:20.004474 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 302/624] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:19 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1476 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:20.011403 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-29aa4e17-ca62-4715-ac4b-7d7484078249 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 429 microversion: 2.1 time: 0.040408 Sep 27 09:58:20.021375 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 323/625] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:19 2022] POST /compute/v2.1/flavors => generated 429 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:20.021698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a3b7e6ec-2fcd-4e62-b48b-a464734d1831 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:20.021698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a3b7e6ec-2fcd-4e62-b48b-a464734d1831 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:20.030656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:20.031506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:20.034530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3b7e6ec-2fcd-4e62-b48b-a464734d1831 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:20.037063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3b7e6ec-2fcd-4e62-b48b-a464734d1831 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:20.037964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3b7e6ec-2fcd-4e62-b48b-a464734d1831 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:20.041299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3b7e6ec-2fcd-4e62-b48b-a464734d1831 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:20.041923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3b7e6ec-2fcd-4e62-b48b-a464734d1831 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:20.043851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:20.044930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3b7e6ec-2fcd-4e62-b48b-a464734d1831 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:20.047219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:20.049111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:20.053280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:20.054962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:20.055553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:20.103225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-a3b7e6ec-2fcd-4e62-b48b-a464734d1831 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lazy-loading 'flavor' on Instance uuid a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:20.236011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:20.263598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:20.264926 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a3b7e6ec-2fcd-4e62-b48b-a464734d1831 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4/action" status: 202 len: 0 microversion: 2.1 time: 0.258132 Sep 27 09:58:20.269464 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 303/626] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:58:20 2022] POST /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4/action => generated 0 bytes in 261 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:20.282758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-c6a75027-c1af-4420-a235-0cd8c22df6b9 req-4d8c6579-52af-421d-9cc6-7339384626f9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "706e3141-43af-4a6b-8789-dea152d0312a", "tag": "ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:20.289011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:20.311180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c6a75027-c1af-4420-a235-0cd8c22df6b9 req-4d8c6579-52af-421d-9cc6-7339384626f9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:20.312467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c6a75027-c1af-4420-a235-0cd8c22df6b9 req-4d8c6579-52af-421d-9cc6-7339384626f9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:20.313186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c6a75027-c1af-4420-a235-0cd8c22df6b9 req-4d8c6579-52af-421d-9cc6-7339384626f9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:20.330147 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c6a75027-c1af-4420-a235-0cd8c22df6b9 req-4d8c6579-52af-421d-9cc6-7339384626f9 service nova] Creating event network-changed:ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4 for instance 706e3141-43af-4a6b-8789-dea152d0312a on nch2-master-1 Sep 27 09:58:20.331589 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-480306e8-43db-4f3d-ae46-68fd67f44db9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.310607 Sep 27 09:58:20.346318 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 324/627] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:20 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:20.351256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-c6a75027-c1af-4420-a235-0cd8c22df6b9 req-4d8c6579-52af-421d-9cc6-7339384626f9 service nova] Instance 706e3141-43af-4a6b-8789-dea152d0312a is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 09:58:20.360314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1cc66a54-03a4-4e39-ba54-00413d26625d tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:20.361123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1cc66a54-03a4-4e39-ba54-00413d26625d tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] [instance: f769af8b-472a-4950-b4ae-8c0183b23c1e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:20.371195 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-c6a75027-c1af-4420-a235-0cd8c22df6b9 req-4d8c6579-52af-421d-9cc6-7339384626f9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.103370 Sep 27 09:58:20.375073 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 304/628] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:20.379929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc66a54-03a4-4e39-ba54-00413d26625d tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:20.382518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc66a54-03a4-4e39-ba54-00413d26625d tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:20.383684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d8517cb1-44d8-4bb2-97fd-bef465975e7b tempest-FlavorsAdminTestJSON-1731705636 tempest-FlavorsAdminTestJSON-1731705636-project] Action: 'create', calling method: >, body: {"server": {"name": "test", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "1592566954"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:20.384940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc66a54-03a4-4e39-ba54-00413d26625d tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:20.397616 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-d8517cb1-44d8-4bb2-97fd-bef465975e7b tempest-FlavorsAdminTestJSON-1731705636 tempest-FlavorsAdminTestJSON-1731705636-project] HTTP exception thrown: Flavor 1592566954 could not be found. Sep 27 09:58:20.401220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc66a54-03a4-4e39-ba54-00413d26625d tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d8517cb1-44d8-4bb2-97fd-bef465975e7b tempest-FlavorsAdminTestJSON-1731705636 tempest-FlavorsAdminTestJSON-1731705636-project] Request method failure captured: Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 107 Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsk2d8hqr_HHwjkSa7CEBVbiyXs23k_K1PQLwoDcrzWFz731sG3U6Z8yMMusK0LxRqm1FbNHRRhy2E32XokK0gUTwfzqRgxnG5035NoQjPgu1VJ8TY12v-up0VteKYOqsGY8SeFPOaP1DXu6XdWoad2XSbl_TBZoJ8ZsDnIQChZ6LMk Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 25d46074b23b4a0c802e05437436fe3a Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FlavorsAdminTestJSON-1731705636 Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:58:20.401754 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/25d46074b23b4a0c802e05437436fe3a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/25d46074b23b4a0c802e05437436fe3a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/25d46074b23b4a0c802e05437436fe3a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_25d46074b23b4a0c802e05437436fe3a"}]}] Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FlavorsAdminTestJSON-1731705636 Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 25d46074b23b4a0c802e05437436fe3a Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FlavorsAdminTestJSON-1731705636 Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FlavorsAdminTestJSON-1731705636-project Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b2288601ff7742ab8cfddd8e3c254982 Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FlavorsAdminTestJSON-1731705636-project Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "test", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "1592566954"}} Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:58:20.403410 ncdv-master-1 devstack@n-api.service[145969]: exception: Flavor 1592566954 could not be found. Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'test', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '1592566954'}}} Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 778, in create Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id( Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 1592566954 could not be found. Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:20.404362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor 1592566954 could not be found. Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:20.405908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:20.407472 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:20.407472 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:20.407472 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:20.407472 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 1592566954 could not be found. Sep 27 09:58:20.407472 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:20.407472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d8517cb1-44d8-4bb2-97fd-bef465975e7b tempest-FlavorsAdminTestJSON-1731705636 tempest-FlavorsAdminTestJSON-1731705636-project] Returning 400 to user: Flavor 1592566954 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:20.409164 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d8517cb1-44d8-4bb2-97fd-bef465975e7b tempest-FlavorsAdminTestJSON-1731705636 tempest-FlavorsAdminTestJSON-1731705636-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 81 microversion: 2.1 time: 0.032151 Sep 27 09:58:20.410304 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 305/629] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:20 2022] POST /compute/v2.1/servers => generated 81 bytes in 34 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 09:58:20.413490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc66a54-03a4-4e39-ba54-00413d26625d tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.023s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:20.413947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc66a54-03a4-4e39-ba54-00413d26625d tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:20.420215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-07c7b4fb-133c-40ae-a30c-75e8e3a91b02 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:20.421341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-07c7b4fb-133c-40ae-a30c-75e8e3a91b02 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:20.430362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07c7b4fb-133c-40ae-a30c-75e8e3a91b02 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:20.431224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07c7b4fb-133c-40ae-a30c-75e8e3a91b02 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:20.431868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07c7b4fb-133c-40ae-a30c-75e8e3a91b02 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:20.435312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07c7b4fb-133c-40ae-a30c-75e8e3a91b02 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:20.436170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07c7b4fb-133c-40ae-a30c-75e8e3a91b02 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:20.437001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07c7b4fb-133c-40ae-a30c-75e8e3a91b02 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:20.809968 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1cc66a54-03a4-4e39-ba54-00413d26625d tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e" status: 200 len: 1620 microversion: 2.1 time: 0.460915 Sep 27 09:58:20.812949 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 325/630] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:20 2022] GET /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e => generated 1620 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:20.833340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-674bef59-8b5e-46ae-95d0-25ec47533a02 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:20.932306 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-674bef59-8b5e-46ae-95d0-25ec47533a02 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1592566954" status: 202 len: 0 microversion: 2.1 time: 0.117466 Sep 27 09:58:20.932646 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 326/631] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:20 2022] DELETE /compute/v2.1/flavors/1592566954 => generated 0 bytes in 119 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:20.982013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-61c5b0fd-f217-40f5-aac4-a88c450fa4ac tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:21.008403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-07c7b4fb-133c-40ae-a30c-75e8e3a91b02 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:21.017227 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-07c7b4fb-133c-40ae-a30c-75e8e3a91b02 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1476 microversion: 2.1 time: 0.605554 Sep 27 09:58:21.034151 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 306/632] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:20 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1476 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:21.038774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2b7d0aad-086b-498a-b9f8-dd5f9a417f0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:21.039717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2b7d0aad-086b-498a-b9f8-dd5f9a417f0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:21.046142 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-61c5b0fd-f217-40f5-aac4-a88c450fa4ac tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.101051 Sep 27 09:58:21.047085 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 327/633] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:20 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:21.050261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b7d0aad-086b-498a-b9f8-dd5f9a417f0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:21.050987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b7d0aad-086b-498a-b9f8-dd5f9a417f0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:21.051649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b7d0aad-086b-498a-b9f8-dd5f9a417f0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:21.052579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b7d0aad-086b-498a-b9f8-dd5f9a417f0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:21.057898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b7d0aad-086b-498a-b9f8-dd5f9a417f0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:21.058615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b7d0aad-086b-498a-b9f8-dd5f9a417f0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:21.073649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0310686c-8c27-454a-941e-5fb5347bef7f tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-1948167850", "id": 2140663648, "os-flavor-access:is_public": "False"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:21.112058 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0310686c-8c27-454a-941e-5fb5347bef7f tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 415 microversion: 2.1 time: 0.043504 Sep 27 09:58:21.114360 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 328/634] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:21 2022] POST /compute/v2.1/flavors => generated 415 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:21.127191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c0525d5e-0391-4a22-b9cd-601f5e42e67e tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-2069738708", "id": 1909239318, "os-flavor-access:is_public": "True"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:21.189862 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c0525d5e-0391-4a22-b9cd-601f5e42e67e tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 414 microversion: 2.1 time: 0.068312 Sep 27 09:58:21.191756 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 329/635] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:21 2022] POST /compute/v2.1/flavors => generated 414 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:21.197171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-2091061388", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}, "os:scheduler_hints": {"different_host": "552575d5-492a-4680-9972-29347ad5479b"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:21.280983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:58:21.518760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2b7d0aad-086b-498a-b9f8-dd5f9a417f0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:21.528696 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2b7d0aad-086b-498a-b9f8-dd5f9a417f0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1476 microversion: 2.1 time: 0.501785 Sep 27 09:58:21.529284 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 307/636] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:21 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1476 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:21.543283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3d844e11-2ca8-4f82-acaa-9df9ee3bac4b tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:21.572551 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3d844e11-2ca8-4f82-acaa-9df9ee3bac4b tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=f" status: 200 len: 418 microversion: 2.1 time: 0.038934 Sep 27 09:58:21.574051 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 308/637] 10.222.0.26 () {58 vars in 1169 bytes} [Tue Sep 27 09:58:21 2022] GET /compute/v2.1/flavors/detail?is_public=f => generated 418 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:21.579452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:21.580335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:21.588634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:21.590071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:21.590746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:21.591724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:21.592555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:21.593713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:21.751602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:21.774807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:21.776923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:21.805748 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-45ecd134-c551-4e46-be49-8a7dda07dee7 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.225427 Sep 27 09:58:21.809248 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 309/638] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:21 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:21.846542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6eefecbf-da8a-4456-8fbd-6bd1fe90201b tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:21.861175 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6eefecbf-da8a-4456-8fbd-6bd1fe90201b tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=false" status: 200 len: 418 microversion: 2.1 time: 0.050253 Sep 27 09:58:21.862226 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 310/639] 10.222.0.26 () {58 vars in 1177 bytes} [Tue Sep 27 09:58:21 2022] GET /compute/v2.1/flavors/detail?is_public=false => generated 418 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:21.881910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5d90613-47dc-4d44-b368-8cdcb58847f4 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:21.897836 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c5d90613-47dc-4d44-b368-8cdcb58847f4 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=no" status: 200 len: 418 microversion: 2.1 time: 0.022778 Sep 27 09:58:21.905268 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 311/640] 10.222.0.26 () {58 vars in 1171 bytes} [Tue Sep 27 09:58:21 2022] GET /compute/v2.1/flavors/detail?is_public=no => generated 418 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:21.917729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-532b4179-759f-4c6c-9581-d1b02726cb3e tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:21.929260 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-532b4179-759f-4c6c-9581-d1b02726cb3e tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=0" status: 200 len: 418 microversion: 2.1 time: 0.020679 Sep 27 09:58:21.930422 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 312/641] 10.222.0.26 () {58 vars in 1169 bytes} [Tue Sep 27 09:58:21 2022] GET /compute/v2.1/flavors/detail?is_public=0 => generated 418 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:21.943213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dc8d6677-c2a5-41a3-b30e-7a9e382b28f9 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:21.971168 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dc8d6677-c2a5-41a3-b30e-7a9e382b28f9 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=t" status: 200 len: 6218 microversion: 2.1 time: 0.031534 Sep 27 09:58:21.972121 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 313/642] 10.222.0.26 () {58 vars in 1169 bytes} [Tue Sep 27 09:58:21 2022] GET /compute/v2.1/flavors/detail?is_public=t => generated 6218 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:21.992783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-31e80602-7845-4b13-a9d7-c79dda8f1e60 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:22.019232 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-31e80602-7845-4b13-a9d7-c79dda8f1e60 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=true" status: 200 len: 6218 microversion: 2.1 time: 0.030723 Sep 27 09:58:22.027946 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 314/643] 10.222.0.26 () {58 vars in 1175 bytes} [Tue Sep 27 09:58:21 2022] GET /compute/v2.1/flavors/detail?is_public=true => generated 6218 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:22.043207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9582b4b0-a7ac-49f0-b828-93599b3179a4 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:22.047547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:58:22.047973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:58:22.048448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:58:22.048894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:58:22.049489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:58:22.074185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:22.074495 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9582b4b0-a7ac-49f0-b828-93599b3179a4 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=yes" status: 200 len: 6218 microversion: 2.1 time: 0.037376 Sep 27 09:58:22.077075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:22.079700 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 315/644] 10.222.0.26 () {58 vars in 1173 bytes} [Tue Sep 27 09:58:22 2022] GET /compute/v2.1/flavors/detail?is_public=yes => generated 6218 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:22.082503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:22.085994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-11c8eb95-5c7c-4366-84aa-e7cae0c0690c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:22.087130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:22.087689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-11c8eb95-5c7c-4366-84aa-e7cae0c0690c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:22.095117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:22.095906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:22.099231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-11c8eb95-5c7c-4366-84aa-e7cae0c0690c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:22.099705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-11c8eb95-5c7c-4366-84aa-e7cae0c0690c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:22.102876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-11c8eb95-5c7c-4366-84aa-e7cae0c0690c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:22.105498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-11c8eb95-5c7c-4366-84aa-e7cae0c0690c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:22.105926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-11c8eb95-5c7c-4366-84aa-e7cae0c0690c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:22.106376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-11c8eb95-5c7c-4366-84aa-e7cae0c0690c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:22.157395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:58:22.188151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:58:22.218341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:58:22.220824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:58:22.319229 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-df15a94d-4a2d-4c45-9c44-84410256928f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.126172 Sep 27 09:58:22.329859 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 330/645] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:21 2022] POST /compute/v2.1/servers => generated 380 bytes in 1137 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:58:22.340833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cf954f1b-b761-4ac0-aa35-37b8f05aa61d tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:22.403777 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cf954f1b-b761-4ac0-aa35-37b8f05aa61d tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=1" status: 200 len: 6218 microversion: 2.1 time: 0.072355 Sep 27 09:58:22.407090 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 331/646] 10.222.0.26 () {58 vars in 1169 bytes} [Tue Sep 27 09:58:22 2022] GET /compute/v2.1/flavors/detail?is_public=1 => generated 6218 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:22.419106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4206f10-1d35-4e05-b17e-73740a33e02c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:22.427235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d4206f10-1d35-4e05-b17e-73740a33e02c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:22.438635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-11c8eb95-5c7c-4366-84aa-e7cae0c0690c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:22.449750 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-11c8eb95-5c7c-4366-84aa-e7cae0c0690c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1459 microversion: 2.1 time: 0.368548 Sep 27 09:58:22.451165 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 316/647] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:22 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1459 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:22.461435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-919107b9-237e-49b6-8a4a-2869a274aec0 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:22.503763 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-919107b9-237e-49b6-8a4a-2869a274aec0 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1909239318" status: 202 len: 0 microversion: 2.1 time: 0.050648 Sep 27 09:58:22.504804 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 317/648] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:22 2022] DELETE /compute/v2.1/flavors/1909239318 => generated 0 bytes in 52 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:22.511899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8946e92-c897-43b3-8aac-a602bb3daf56 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:22.513214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f8946e92-c897-43b3-8aac-a602bb3daf56 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:22.522459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8946e92-c897-43b3-8aac-a602bb3daf56 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:22.523719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8946e92-c897-43b3-8aac-a602bb3daf56 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:22.524505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8946e92-c897-43b3-8aac-a602bb3daf56 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:22.525514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8946e92-c897-43b3-8aac-a602bb3daf56 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:22.526746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8946e92-c897-43b3-8aac-a602bb3daf56 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:22.527345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8946e92-c897-43b3-8aac-a602bb3daf56 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:22.600846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d4206f10-1d35-4e05-b17e-73740a33e02c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:22.612434 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4206f10-1d35-4e05-b17e-73740a33e02c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1217 microversion: 2.1 time: 0.200886 Sep 27 09:58:22.613472 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 332/649] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:22 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1217 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:22.620140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d611c69-11dd-42b7-af2d-2af186bb04f8 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:22.689602 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d611c69-11dd-42b7-af2d-2af186bb04f8 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.074866 Sep 27 09:58:22.691331 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 333/650] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:22 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:22.702646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ff10e600-e9da-4a41-979f-fb582d989970 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:22.703655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ff10e600-e9da-4a41-979f-fb582d989970 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:22.713692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff10e600-e9da-4a41-979f-fb582d989970 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:22.714481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff10e600-e9da-4a41-979f-fb582d989970 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:22.715076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff10e600-e9da-4a41-979f-fb582d989970 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:22.716368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff10e600-e9da-4a41-979f-fb582d989970 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:22.717387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff10e600-e9da-4a41-979f-fb582d989970 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:22.718008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff10e600-e9da-4a41-979f-fb582d989970 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:23.475105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ff10e600-e9da-4a41-979f-fb582d989970 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:23.487532 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ff10e600-e9da-4a41-979f-fb582d989970 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1474 microversion: 2.1 time: 0.794931 Sep 27 09:58:23.505989 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 334/651] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:22 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1474 bytes in 814 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:23.523191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7cefd48f-4ae3-4fda-ac56-eb5f562b5eed tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:23.554071 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7cefd48f-4ae3-4fda-ac56-eb5f562b5eed tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/2140663648" status: 202 len: 0 microversion: 2.1 time: 0.046021 Sep 27 09:58:23.555116 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 335/652] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:23 2022] DELETE /compute/v2.1/flavors/2140663648 => generated 0 bytes in 48 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:23.564148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:23.565221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:23.599555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:23.600447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:23.602038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:23.603300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:23.603999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:23.604866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:23.731857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:23.746874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:23.749345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:23.763207 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a9831cf5-b83f-4846-81e4-06cb66d50ba3 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1446 microversion: 2.75 time: 0.206455 Sep 27 09:58:23.764971 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 336/653] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:23 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1446 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:23.778856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c551389c-95d4-4528-bc1e-6fe56cea7804 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:23.841469 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c551389c-95d4-4528-bc1e-6fe56cea7804 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.074982 Sep 27 09:58:23.842941 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 337/654] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:23 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:23.851721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-be4fd564-d58e-430f-b0a6-df1e182596d3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:23.858458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-be4fd564-d58e-430f-b0a6-df1e182596d3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:23.878012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be4fd564-d58e-430f-b0a6-df1e182596d3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:23.879633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be4fd564-d58e-430f-b0a6-df1e182596d3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:23.886880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be4fd564-d58e-430f-b0a6-df1e182596d3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:23.895320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be4fd564-d58e-430f-b0a6-df1e182596d3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:23.896169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be4fd564-d58e-430f-b0a6-df1e182596d3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:23.896808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be4fd564-d58e-430f-b0a6-df1e182596d3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:23.987236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-be4fd564-d58e-430f-b0a6-df1e182596d3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:23.997319 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-be4fd564-d58e-430f-b0a6-df1e182596d3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1221 microversion: 2.1 time: 0.152492 Sep 27 09:58:23.999463 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 338/655] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:23 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1221 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:24.009530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eedea950-137b-4b2a-8e07-d036f1d6dbcc tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-1702858650", "id": 1146120974, "os-flavor-access:is_public": "False"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:24.044459 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f8946e92-c897-43b3-8aac-a602bb3daf56 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1610 microversion: 2.1 time: 1.538268 Sep 27 09:58:24.050189 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 318/656] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:22 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1610 bytes in 1540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:24.051799 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eedea950-137b-4b2a-8e07-d036f1d6dbcc tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 415 microversion: 2.1 time: 0.045567 Sep 27 09:58:24.063854 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 339/657] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:23 2022] POST /compute/v2.1/flavors => generated 415 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:24.068297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4b87e398-0e43-4990-ae8b-58c46156261d tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:24.074373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2b7bee2c-7f2c-409d-98a9-e02f91ef68e6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:24.079551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7bee2c-7f2c-409d-98a9-e02f91ef68e6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:24.080150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7bee2c-7f2c-409d-98a9-e02f91ef68e6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:24.080789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7bee2c-7f2c-409d-98a9-e02f91ef68e6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:24.081780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7bee2c-7f2c-409d-98a9-e02f91ef68e6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:24.082398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7bee2c-7f2c-409d-98a9-e02f91ef68e6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:24.082901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7bee2c-7f2c-409d-98a9-e02f91ef68e6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:24.116769 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2b7bee2c-7f2c-409d-98a9-e02f91ef68e6 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.047900 Sep 27 09:58:24.118308 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 340/658] 10.222.0.26 () {58 vars in 1176 bytes} [Tue Sep 27 09:58:24 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:24.128394 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4b87e398-0e43-4990-ae8b-58c46156261d tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.056491 Sep 27 09:58:24.131556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:24.132147 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 319/659] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:24 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:24.134285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:24.149843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-68105bdb-ade8-4874-bcf8-5dce8631e977 tempest-FlavorsAdminTestJSON-1731705636 tempest-FlavorsAdminTestJSON-1731705636-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:24.153136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:24.153703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:24.154164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:24.155856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:24.156299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:24.156766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:24.191746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Lazy-loading 'flavor' on Instance uuid a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:24.207286 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] HTTP exception thrown: Cannot 'os-migrateLive' instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 while it is in vm_state suspended Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Request method failure captured: Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4/action HTTP/1.1 Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 97 Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMskU61IIDD7saYvuizpK3FtZNHWD167DDy06ZOamEJt6aa4cpqWyRYItVDjmgFdQVTen0VlFaWfuYiNTxOAw51LUiwqSN6DdQPhIFKmG2E5WrYO4d5o9C3I_ypK-ab8O0HQwtfEXSKgh_C6Fu-BQNPmhVKUa8GKl084ZOMqQKaE94Wg Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: d1a7023a446f417e9764a1c113453548 Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-LiveMigrationNegativeTest-886264320 Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:58:24.209259 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d1a7023a446f417e9764a1c113453548"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d1a7023a446f417e9764a1c113453548"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d1a7023a446f417e9764a1c113453548"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d1a7023a446f417e9764a1c113453548"}]}] Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-LiveMigrationNegativeTest-886264320 Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: d1a7023a446f417e9764a1c113453548 Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-886264320 Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-LiveMigrationNegativeTest-886264320-project-admin Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: d8b69329ff4d4a79b7cd0d056ae722e1 Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-LiveMigrationNegativeTest-886264320-project-admin Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": true, "disk_over_commit": false}} Sep 27 09:58:24.210619 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: exception: Cannot 'os-migrateLive' instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 while it is in vm_state suspended Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a59c15ae-bd3b-4f14-9311-c8f5f52c44d4', 'body': {'os-migrateLive': {'host': 'nch2-master-1', 'block_migration': True, 'disk_over_commit': False}}} Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 123, in _migrate_live Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.live_migrate(context, instance, block_migration, Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 309, in inner Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 278, in inner Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 259, in inner Sep 27 09:58:24.212490 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 in vm_state suspended. Cannot live_migrate while the instance is in this state. Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:24.214470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/migrate_server.py", line 152, in _migrate_live Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'os-migrateLive' instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 while it is in vm_state suspended Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'os-migrateLive' instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 while it is in vm_state suspended Sep 27 09:58:24.217023 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:24.219410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] Returning 409 to user: Cannot 'os-migrateLive' instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 while it is in vm_state suspended {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:24.219410 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cafe8d2a-acc7-4ec9-9473-59efc6bb6601 tempest-LiveMigrationNegativeTest-886264320 tempest-LiveMigrationNegativeTest-886264320-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4/action" status: 409 len: 155 microversion: 2.1 time: 0.083464 Sep 27 09:58:24.219410 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 341/660] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:58:24 2022] POST /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4/action => generated 155 bytes in 85 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Sep 27 09:58:24.219836 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-68105bdb-ade8-4874-bcf8-5dce8631e977 tempest-FlavorsAdminTestJSON-1731705636 tempest-FlavorsAdminTestJSON-1731705636-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.069689 Sep 27 09:58:24.219836 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 320/661] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:24 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:24.224499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8bf0b417-fcb0-48a1-acab-de23259bad24 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:24.225131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8bf0b417-fcb0-48a1-acab-de23259bad24 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:24.240438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ee61babe-a87c-4778-8f25-640c9fe25c08 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:24.241071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf0b417-fcb0-48a1-acab-de23259bad24 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:24.241697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf0b417-fcb0-48a1-acab-de23259bad24 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:24.251415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf0b417-fcb0-48a1-acab-de23259bad24 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:24.252330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf0b417-fcb0-48a1-acab-de23259bad24 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:24.252850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf0b417-fcb0-48a1-acab-de23259bad24 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:24.253299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf0b417-fcb0-48a1-acab-de23259bad24 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:24.275269 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ee61babe-a87c-4778-8f25-640c9fe25c08 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1146120974" status: 202 len: 0 microversion: 2.1 time: 0.040227 Sep 27 09:58:24.276482 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 321/662] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:24 2022] DELETE /compute/v2.1/flavors/1146120974 => generated 0 bytes in 42 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 09:58:24.285811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd476d18-1f04-4c5d-ae7f-05f44e69c342 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:24.315575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=2,nova_api:DELETE=12 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:24.316717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8bf0b417-fcb0-48a1-acab-de23259bad24 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:58:24.349422 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd476d18-1f04-4c5d-ae7f-05f44e69c342 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.068072 Sep 27 09:58:24.351688 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 322/663] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:24 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:24.377879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9161928e-26a6-4ac3-b09c-3419c8896a9b tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-test_flavor_-756927913", "id": 743532626, "os-flavor-access:is_public": "True"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:24.428193 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9161928e-26a6-4ac3-b09c-3419c8896a9b tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 410 microversion: 2.1 time: 0.056195 Sep 27 09:58:24.432214 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 323/664] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:24 2022] POST /compute/v2.1/flavors => generated 410 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (3 switches on core 0) Sep 27 09:58:24.451112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0e39c8c3-3839-4b70-ad6c-3f673d0cf8f8 tempest-FlavorsAdminTestJSON-1731705636 tempest-FlavorsAdminTestJSON-1731705636-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:24.510186 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8bf0b417-fcb0-48a1-acab-de23259bad24 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 204 len: 0 microversion: 2.1 time: 0.289197 Sep 27 09:58:24.514523 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 342/665] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:58:24 2022] DELETE /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:58:24.526554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8742ec23-df77-4639-960b-58e777b84fc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:24.529061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8742ec23-df77-4639-960b-58e777b84fc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:24.531879 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0e39c8c3-3839-4b70-ad6c-3f673d0cf8f8 tempest-FlavorsAdminTestJSON-1731705636 tempest-FlavorsAdminTestJSON-1731705636-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 6214 microversion: 2.1 time: 0.090152 Sep 27 09:58:24.533323 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 324/666] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:24 2022] GET /compute/v2.1/flavors/detail => generated 6214 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:24.548078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-944a152c-2482-4c03-bf75-a7fdfb6061da tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:24.549023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-944a152c-2482-4c03-bf75-a7fdfb6061da tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:24.558245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=8,nova_api:DELETE=6,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:24.569395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8742ec23-df77-4639-960b-58e777b84fc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:24.569672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-944a152c-2482-4c03-bf75-a7fdfb6061da tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:24.569950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8742ec23-df77-4639-960b-58e777b84fc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:24.571217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-944a152c-2482-4c03-bf75-a7fdfb6061da tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:24.571579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8742ec23-df77-4639-960b-58e777b84fc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:24.572044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-944a152c-2482-4c03-bf75-a7fdfb6061da tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:24.574620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8742ec23-df77-4639-960b-58e777b84fc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:24.575171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8742ec23-df77-4639-960b-58e777b84fc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:24.575805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8742ec23-df77-4639-960b-58e777b84fc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:24.576460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-944a152c-2482-4c03-bf75-a7fdfb6061da tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:24.577157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-944a152c-2482-4c03-bf75-a7fdfb6061da tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:24.577749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-944a152c-2482-4c03-bf75-a7fdfb6061da tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:25.030639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8742ec23-df77-4639-960b-58e777b84fc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:25.044741 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8742ec23-df77-4639-960b-58e777b84fc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1474 microversion: 2.1 time: 0.528925 Sep 27 09:58:25.049891 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 343/667] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:24 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1474 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:25.055770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5a12c0c3-cb0a-4149-a99c-63bb2eec21d0 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:25.094148 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5a12c0c3-cb0a-4149-a99c-63bb2eec21d0 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/743532626" status: 202 len: 0 microversion: 2.1 time: 0.042585 Sep 27 09:58:25.095597 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 344/668] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 09:58:25 2022] DELETE /compute/v2.1/flavors/743532626 => generated 0 bytes in 45 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:25.112489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:25.116074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:25.132017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:25.135000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:25.135802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:25.138499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:25.139112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:25.139652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:25.344688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:25.366209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:25.373840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-944a152c-2482-4c03-bf75-a7fdfb6061da tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:25.375949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:25.396355 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-944a152c-2482-4c03-bf75-a7fdfb6061da tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1465 microversion: 2.1 time: 0.860696 Sep 27 09:58:25.412722 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a40ab3d2-d4b2-4c61-b2bb-85723d67d3d9 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1464 microversion: 2.75 time: 0.298976 Sep 27 09:58:25.413518 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 325/669] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:24 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1465 bytes in 879 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:25.422155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6fcf3b22-4382-4766-ab96-812490384a58 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:25.423215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6fcf3b22-4382-4766-ab96-812490384a58 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:25.423445 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 345/670] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:25 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1464 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:25.431195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fcf3b22-4382-4766-ab96-812490384a58 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:25.432108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fcf3b22-4382-4766-ab96-812490384a58 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:25.432985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fcf3b22-4382-4766-ab96-812490384a58 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:25.434107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fcf3b22-4382-4766-ab96-812490384a58 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:25.434832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fcf3b22-4382-4766-ab96-812490384a58 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:25.435514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fcf3b22-4382-4766-ab96-812490384a58 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:25.436905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b264fdd7-2d79-402f-973b-ba20aba1a7b5 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:25.496937 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b264fdd7-2d79-402f-973b-ba20aba1a7b5 tempest-FlavorsAdminTestJSON-1767315060 tempest-FlavorsAdminTestJSON-1767315060-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.070748 Sep 27 09:58:25.498781 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 346/671] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:25 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:25.665944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6fcf3b22-4382-4766-ab96-812490384a58 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:25.686287 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6fcf3b22-4382-4766-ab96-812490384a58 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1269 microversion: 2.1 time: 0.270937 Sep 27 09:58:25.691322 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 326/672] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:25 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1269 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:26.072346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ca27f3a3-5fe9-4f28-9d0b-46fbbf348e0a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:26.073048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ca27f3a3-5fe9-4f28-9d0b-46fbbf348e0a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:26.090897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca27f3a3-5fe9-4f28-9d0b-46fbbf348e0a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:26.092322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca27f3a3-5fe9-4f28-9d0b-46fbbf348e0a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:26.093096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca27f3a3-5fe9-4f28-9d0b-46fbbf348e0a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:26.093971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca27f3a3-5fe9-4f28-9d0b-46fbbf348e0a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:26.094522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca27f3a3-5fe9-4f28-9d0b-46fbbf348e0a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:26.094953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca27f3a3-5fe9-4f28-9d0b-46fbbf348e0a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:26.106023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:26.426790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0f38d74e-6d80-4d6d-9c47-aac20a6b693c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:26.427557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0f38d74e-6d80-4d6d-9c47-aac20a6b693c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:26.445085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f38d74e-6d80-4d6d-9c47-aac20a6b693c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:26.446167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f38d74e-6d80-4d6d-9c47-aac20a6b693c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:26.446886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f38d74e-6d80-4d6d-9c47-aac20a6b693c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:26.447829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f38d74e-6d80-4d6d-9c47-aac20a6b693c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:26.448543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f38d74e-6d80-4d6d-9c47-aac20a6b693c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:26.449209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f38d74e-6d80-4d6d-9c47-aac20a6b693c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:26.771134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0f38d74e-6d80-4d6d-9c47-aac20a6b693c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:26.786046 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0f38d74e-6d80-4d6d-9c47-aac20a6b693c tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1465 microversion: 2.1 time: 0.363096 Sep 27 09:58:26.789797 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 327/673] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:26 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1465 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:26.794362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ca27f3a3-5fe9-4f28-9d0b-46fbbf348e0a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:26.803299 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ca27f3a3-5fe9-4f28-9d0b-46fbbf348e0a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1471 microversion: 2.1 time: 0.737132 Sep 27 09:58:26.804296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:26.805751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:26.813528 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 347/674] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:26 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1471 bytes in 748 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:26.819274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-db4115fd-d271-4a18-a122-0a580c75f54e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:26.819933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-db4115fd-d271-4a18-a122-0a580c75f54e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:26.820901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:26.821799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:26.822468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:26.823693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:26.824363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:26.825022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:26.839500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db4115fd-d271-4a18-a122-0a580c75f54e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:26.840384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db4115fd-d271-4a18-a122-0a580c75f54e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:26.841127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db4115fd-d271-4a18-a122-0a580c75f54e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:26.842187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db4115fd-d271-4a18-a122-0a580c75f54e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:26.842810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db4115fd-d271-4a18-a122-0a580c75f54e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:26.845636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db4115fd-d271-4a18-a122-0a580c75f54e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:26.970617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-db4115fd-d271-4a18-a122-0a580c75f54e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:26.989449 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-db4115fd-d271-4a18-a122-0a580c75f54e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1275 microversion: 2.1 time: 0.173389 Sep 27 09:58:26.993159 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 348/675] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:26 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1275 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:26.998517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d91bf11-ba6f-4e7e-b415-d4a7978ff100 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:27.003969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d91bf11-ba6f-4e7e-b415-d4a7978ff100 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:27.008916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:27.017486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d91bf11-ba6f-4e7e-b415-d4a7978ff100 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:27.023074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:27.029177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d91bf11-ba6f-4e7e-b415-d4a7978ff100 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:27.029873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d91bf11-ba6f-4e7e-b415-d4a7978ff100 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:27.030330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:27.031021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d91bf11-ba6f-4e7e-b415-d4a7978ff100 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:27.031740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d91bf11-ba6f-4e7e-b415-d4a7978ff100 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:27.032394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d91bf11-ba6f-4e7e-b415-d4a7978ff100 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:27.042289 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-30ff303f-bdcd-4d52-b4c9-eefe0b7ae68a tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1452 microversion: 2.75 time: 0.250925 Sep 27 09:58:27.047003 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 328/676] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:26 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1452 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:27.180057 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d91bf11-ba6f-4e7e-b415-d4a7978ff100 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "POST /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a/action" status: 204 len: 0 microversion: 2.1 time: 0.184155 Sep 27 09:58:27.181494 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 349/677] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:58:26 2022] POST /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a/action => generated 0 bytes in 187 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:58:27.190390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-19e6d8cf-4edf-4faa-bb6d-c12bbbf6931e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:27.191102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-19e6d8cf-4edf-4faa-bb6d-c12bbbf6931e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:27.213508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19e6d8cf-4edf-4faa-bb6d-c12bbbf6931e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:27.214291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19e6d8cf-4edf-4faa-bb6d-c12bbbf6931e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:27.215164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19e6d8cf-4edf-4faa-bb6d-c12bbbf6931e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:27.217767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19e6d8cf-4edf-4faa-bb6d-c12bbbf6931e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:27.218493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19e6d8cf-4edf-4faa-bb6d-c12bbbf6931e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:27.219738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19e6d8cf-4edf-4faa-bb6d-c12bbbf6931e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:27.814628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6d161ccd-f9b0-4194-b7bc-4880be19b809 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:27.815521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6d161ccd-f9b0-4194-b7bc-4880be19b809 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:27.830903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d161ccd-f9b0-4194-b7bc-4880be19b809 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:27.831868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d161ccd-f9b0-4194-b7bc-4880be19b809 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:27.832525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d161ccd-f9b0-4194-b7bc-4880be19b809 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:27.833536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d161ccd-f9b0-4194-b7bc-4880be19b809 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:27.834240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d161ccd-f9b0-4194-b7bc-4880be19b809 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:27.834866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d161ccd-f9b0-4194-b7bc-4880be19b809 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:27.856992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-19e6d8cf-4edf-4faa-bb6d-c12bbbf6931e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:27.874909 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-19e6d8cf-4edf-4faa-bb6d-c12bbbf6931e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1471 microversion: 2.1 time: 0.688305 Sep 27 09:58:27.878003 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 329/678] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:27 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1471 bytes in 692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:28.017998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7233d6a2-5434-456d-ab9e-b7cfa96f57ef tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:28.019085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7233d6a2-5434-456d-ab9e-b7cfa96f57ef tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:28.039053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7233d6a2-5434-456d-ab9e-b7cfa96f57ef tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:28.039840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7233d6a2-5434-456d-ab9e-b7cfa96f57ef tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:28.040649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7233d6a2-5434-456d-ab9e-b7cfa96f57ef tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:28.042378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7233d6a2-5434-456d-ab9e-b7cfa96f57ef tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:28.043282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7233d6a2-5434-456d-ab9e-b7cfa96f57ef tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:28.044022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7233d6a2-5434-456d-ab9e-b7cfa96f57ef tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:28.057432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:28.296410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6d161ccd-f9b0-4194-b7bc-4880be19b809 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:28.319943 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6d161ccd-f9b0-4194-b7bc-4880be19b809 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1465 microversion: 2.1 time: 0.508916 Sep 27 09:58:28.321262 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 350/679] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:27 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1465 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:28.335698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:28.338755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:28.341413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7233d6a2-5434-456d-ab9e-b7cfa96f57ef tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:28.350333 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7233d6a2-5434-456d-ab9e-b7cfa96f57ef tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1275 microversion: 2.1 time: 0.338614 Sep 27 09:58:28.353061 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 330/680] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:28 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1275 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:28.367655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:28.368441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:28.369070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:28.370085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:28.370708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:28.371293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:28.686071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:28.696474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:28.698071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:28.716063 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6180f040-9d39-483a-b271-7f526a43e088 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1452 microversion: 2.75 time: 0.392230 Sep 27 09:58:28.732825 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 351/681] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:28 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1452 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:28.882331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5c7e3ac9-74fd-444d-9112-e0b2abc847a3 req-8c0f89fd-d724-4064-a16f-0a575fda43a7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a59c15ae-bd3b-4f14-9311-c8f5f52c44d4", "name": "network-vif-deleted", "tag": "92de2508-8fe5-415a-987e-c0ff172d47d7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:28.892315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:28.895714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:28.909197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb8107dd-ac95-466e-83d0-e22ef7ddc067 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:28.910517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cb8107dd-ac95-466e-83d0-e22ef7ddc067 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:28.912820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5c7e3ac9-74fd-444d-9112-e0b2abc847a3 req-8c0f89fd-d724-4064-a16f-0a575fda43a7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:28.913462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5c7e3ac9-74fd-444d-9112-e0b2abc847a3 req-8c0f89fd-d724-4064-a16f-0a575fda43a7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:28.914999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5c7e3ac9-74fd-444d-9112-e0b2abc847a3 req-8c0f89fd-d724-4064-a16f-0a575fda43a7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:28.920545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb8107dd-ac95-466e-83d0-e22ef7ddc067 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:28.922030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb8107dd-ac95-466e-83d0-e22ef7ddc067 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:28.922746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb8107dd-ac95-466e-83d0-e22ef7ddc067 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:28.923827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb8107dd-ac95-466e-83d0-e22ef7ddc067 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:28.924876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb8107dd-ac95-466e-83d0-e22ef7ddc067 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:28.926722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb8107dd-ac95-466e-83d0-e22ef7ddc067 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:28.946913 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5c7e3ac9-74fd-444d-9112-e0b2abc847a3 req-8c0f89fd-d724-4064-a16f-0a575fda43a7 service nova] Creating event network-vif-deleted:92de2508-8fe5-415a-987e-c0ff172d47d7 for instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 on nch1-master-1 Sep 27 09:58:28.953563 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5c7e3ac9-74fd-444d-9112-e0b2abc847a3 req-8c0f89fd-d724-4064-a16f-0a575fda43a7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075301 Sep 27 09:58:28.954790 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 331/682] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:29.346879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:29.347707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:29.356093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:29.356931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:29.357527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:29.358497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:29.359117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:29.359906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:29.463000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:29.463815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:29.464442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:29.465112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lazy-loading 'fault' on Instance uuid a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:29.616560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb8107dd-ac95-466e-83d0-e22ef7ddc067 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:29.631460 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cb8107dd-ac95-466e-83d0-e22ef7ddc067 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1471 microversion: 2.1 time: 0.731497 Sep 27 09:58:29.645129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:29.649716 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 352/683] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:28 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1471 bytes in 751 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:29.680230 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3cffa6d1-31cf-4cb3-bad0-809ad59fcc8f tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 200 len: 1308 microversion: 2.1 time: 0.336665 Sep 27 09:58:29.681807 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 332/684] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:29 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 1308 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:29.690242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f5c83267-cfd5-47c1-b2c9-9c26c35a1e8d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:29.700908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f5c83267-cfd5-47c1-b2c9-9c26c35a1e8d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:29.739712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5c83267-cfd5-47c1-b2c9-9c26c35a1e8d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:29.740484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5c83267-cfd5-47c1-b2c9-9c26c35a1e8d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:29.742530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5c83267-cfd5-47c1-b2c9-9c26c35a1e8d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:29.743834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5c83267-cfd5-47c1-b2c9-9c26c35a1e8d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:29.744472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5c83267-cfd5-47c1-b2c9-9c26c35a1e8d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:29.745080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5c83267-cfd5-47c1-b2c9-9c26c35a1e8d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:29.752305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:29.753309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:29.778206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:29.780550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:29.783034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:29.785057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:29.785675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:29.787143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:30.030701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:30.046951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:30.049927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:30.076098 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-866092b4-8d21-45ee-9b7f-17744454c29c tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1452 microversion: 2.75 time: 0.334839 Sep 27 09:58:30.085467 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 333/685] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:29 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1452 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 09:58:30.788845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-07037f55-0504-4886-9b3e-3eddf2d9162a tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Action: 'create', calling method: .version_select at 0x7f7f63a83820>, body: {"pool": null} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:31.293645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f5c83267-cfd5-47c1-b2c9-9c26c35a1e8d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:31.306348 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f5c83267-cfd5-47c1-b2c9-9c26c35a1e8d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1317 microversion: 2.1 time: 1.654420 Sep 27 09:58:31.307058 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 353/686] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:29 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1317 bytes in 1656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:31.326433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-65d7f8a6-45de-4023-a2e7-08c957cc3de9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:31.327064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-65d7f8a6-45de-4023-a2e7-08c957cc3de9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:31.354176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65d7f8a6-45de-4023-a2e7-08c957cc3de9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:31.354791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65d7f8a6-45de-4023-a2e7-08c957cc3de9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:31.355329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65d7f8a6-45de-4023-a2e7-08c957cc3de9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:31.356075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65d7f8a6-45de-4023-a2e7-08c957cc3de9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:31.356538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65d7f8a6-45de-4023-a2e7-08c957cc3de9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:31.357433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65d7f8a6-45de-4023-a2e7-08c957cc3de9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:32.276169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-65d7f8a6-45de-4023-a2e7-08c957cc3de9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:32.292887 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-65d7f8a6-45de-4023-a2e7-08c957cc3de9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1471 microversion: 2.1 time: 0.982721 Sep 27 09:58:32.293532 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 354/687] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:31 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1471 bytes in 986 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:32.326308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:32.332201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: a59c15ae-bd3b-4f14-9311-c8f5f52c44d4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:32.362362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:32.363457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:32.367186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:32.370013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:32.374900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:32.377684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:32.446829 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] HTTP exception thrown: Instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 could not be found. Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Request method failure captured: Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 HTTP/1.1 Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMskSIxx2wr0SHypAeNgCTUvOeRPeSrzkwGS52FZpml8wBVEX5PaqILAt2hnqyFu-emYlg1O3IdjnGkCJZAtKHmyyDJU_BoNH0wJCM0pWpFGAdf_qX5mjOVGAz9VTuFFVeYOyOiirmWg8oOM7bwUHHtH7p0n7-De9qzZ894PhQ7HJXYQ Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6e8a121c6c654613a6a92df03c3754fa Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-LiveMigrationNegativeTest-1033344067 Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:58:32.453556 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6e8a121c6c654613a6a92df03c3754fa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6e8a121c6c654613a6a92df03c3754fa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6e8a121c6c654613a6a92df03c3754fa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6e8a121c6c654613a6a92df03c3754fa"}]}] Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-LiveMigrationNegativeTest-1033344067 Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6e8a121c6c654613a6a92df03c3754fa Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-1033344067 Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-LiveMigrationNegativeTest-1033344067-project Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 4319ec4b90e1481b8be19e8f9dde1ea4 Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-LiveMigrationNegativeTest-1033344067-project Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 could not be found. Sep 27 09:58:32.455180 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a59c15ae-bd3b-4f14-9311-c8f5f52c44d4'} Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 could not be found. Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:32.456202 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 could not be found. Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 could not be found. Sep 27 09:58:32.457670 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:32.461922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Returning 404 to user: Instance a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:32.464483 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-113e086f-d4e9-4405-95fe-45e005c7be34 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4" status: 404 len: 111 microversion: 2.1 time: 0.147748 Sep 27 09:58:32.468336 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 355/688] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:32 2022] GET /compute/v2.1/servers/a59c15ae-bd3b-4f14-9311-c8f5f52c44d4 => generated 111 bytes in 174 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:32.478354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-094b3913-4268-401a-83ae-9f3e71d85dfc req-e808abb3-71bb-4c53-8e1c-da1730f34fb2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "706e3141-43af-4a6b-8789-dea152d0312a", "name": "network-vif-plugged", "status": "completed", "tag": "ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:32.529976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-094b3913-4268-401a-83ae-9f3e71d85dfc req-e808abb3-71bb-4c53-8e1c-da1730f34fb2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:32.532545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-094b3913-4268-401a-83ae-9f3e71d85dfc req-e808abb3-71bb-4c53-8e1c-da1730f34fb2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:32.533483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-094b3913-4268-401a-83ae-9f3e71d85dfc req-e808abb3-71bb-4c53-8e1c-da1730f34fb2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:32.563365 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-094b3913-4268-401a-83ae-9f3e71d85dfc req-e808abb3-71bb-4c53-8e1c-da1730f34fb2 service nova] Creating event network-vif-plugged:ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4 for instance 706e3141-43af-4a6b-8789-dea152d0312a on nch2-master-1 Sep 27 09:58:32.613943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-094b3913-4268-401a-83ae-9f3e71d85dfc req-e808abb3-71bb-4c53-8e1c-da1730f34fb2 service nova] Instance 706e3141-43af-4a6b-8789-dea152d0312a is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 09:58:32.680610 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-094b3913-4268-401a-83ae-9f3e71d85dfc req-e808abb3-71bb-4c53-8e1c-da1730f34fb2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.210447 Sep 27 09:58:32.681797 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 356/689] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:32.688856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:32.689787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:32.692017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:32.699626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:32.700428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:32.701170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:32.702354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:32.703162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:32.703910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:33.172859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:33.190533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:33.192431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:33.200050 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-704848a1-39c3-4068-8634-bd27d17056af tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1437 microversion: 2.75 time: 0.517000 Sep 27 09:58:33.200635 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 357/690] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:32 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1437 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:33.218845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6bf730a8-5026-46fd-965b-706994dbc75c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:33.219665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6bf730a8-5026-46fd-965b-706994dbc75c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:33.227914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf730a8-5026-46fd-965b-706994dbc75c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:33.229170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf730a8-5026-46fd-965b-706994dbc75c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:33.229843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf730a8-5026-46fd-965b-706994dbc75c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:33.231352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf730a8-5026-46fd-965b-706994dbc75c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:33.232095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf730a8-5026-46fd-965b-706994dbc75c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:33.232541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf730a8-5026-46fd-965b-706994dbc75c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:33.455737 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-07037f55-0504-4886-9b3e-3eddf2d9162a tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] 10.222.0.26 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 3.343892 Sep 27 09:58:33.465440 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 334/691] 10.222.0.26 () {60 vars in 1170 bytes} [Tue Sep 27 09:58:30 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 3345 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:33.465440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b514d44-27f4-4fd7-9261-28c249509196 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:33.465440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3b514d44-27f4-4fd7-9261-28c249509196 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:33.477517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b514d44-27f4-4fd7-9261-28c249509196 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:33.478030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b514d44-27f4-4fd7-9261-28c249509196 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:33.478470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b514d44-27f4-4fd7-9261-28c249509196 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:33.479200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b514d44-27f4-4fd7-9261-28c249509196 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:33.479656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b514d44-27f4-4fd7-9261-28c249509196 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:33.480080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b514d44-27f4-4fd7-9261-28c249509196 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:33.550939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3b514d44-27f4-4fd7-9261-28c249509196 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:58:33.733878 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b514d44-27f4-4fd7-9261-28c249509196 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "DELETE /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 204 len: 0 microversion: 2.1 time: 0.275773 Sep 27 09:58:33.736509 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 335/692] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:58:33 2022] DELETE /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:58:33.837169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6bf730a8-5026-46fd-965b-706994dbc75c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:33.845554 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6bf730a8-5026-46fd-965b-706994dbc75c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1317 microversion: 2.1 time: 0.636677 Sep 27 09:58:33.855441 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 358/693] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:33 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1317 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:34.050451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c147afb7-5fbe-43f7-b9b1-4fdb6056fa2b tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessTestJSON-flavor-1388069081", "id": 265395823, "os-flavor-access:is_public": "False"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:34.090744 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c147afb7-5fbe-43f7-b9b1-4fdb6056fa2b tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 428 microversion: 2.1 time: 0.352030 Sep 27 09:58:34.099677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:34.100252 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 336/694] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:33 2022] POST /compute/v2.1/flavors => generated 428 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:34.108544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8754e96f-5ddb-464f-834b-5046762c41c0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:34.109470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8754e96f-5ddb-464f-834b-5046762c41c0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:34.127336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8754e96f-5ddb-464f-834b-5046762c41c0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:34.127845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8754e96f-5ddb-464f-834b-5046762c41c0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:34.128289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8754e96f-5ddb-464f-834b-5046762c41c0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:34.129074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8754e96f-5ddb-464f-834b-5046762c41c0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:34.129626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8754e96f-5ddb-464f-834b-5046762c41c0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:34.130058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8754e96f-5ddb-464f-834b-5046762c41c0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:34.268524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:34.271847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:34.278917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:34.279412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:34.279846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:34.283322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:34.283806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:34.284234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:34.347170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8754e96f-5ddb-464f-834b-5046762c41c0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:34.360197 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8754e96f-5ddb-464f-834b-5046762c41c0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1463 microversion: 2.1 time: 0.260161 Sep 27 09:58:34.360800 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 337/695] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:34 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1463 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:34.374829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b740655-6dca-4720-9a3d-c156110124e4 tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Calling method '.version_select at 0x7f7f63c625e0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:34.399541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:58:34.399958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:58:34.400424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:58:34.401067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:58:34.401474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:58:34.478324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:34.497600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:34.500833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:34.501453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:34.502353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:34.504334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:34.505398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:34.676320 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b740655-6dca-4720-9a3d-c156110124e4 tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] 10.222.0.26 "GET /compute/v2.1/os-floating-ips/bfbf338f-40ba-4c67-81fe-6f8e617570e4" status: 200 len: 142 microversion: 2.1 time: 0.314204 Sep 27 09:58:34.677170 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 338/696] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 09:58:34 2022] GET /compute/v2.1/os-floating-ips/bfbf338f-40ba-4c67-81fe-6f8e617570e4 => generated 142 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:34.682812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cbc0fdca-a23b-46d5-ba21-1ea3d26b4326 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:34.683442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cbc0fdca-a23b-46d5-ba21-1ea3d26b4326 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:34.690386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbc0fdca-a23b-46d5-ba21-1ea3d26b4326 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:34.690908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbc0fdca-a23b-46d5-ba21-1ea3d26b4326 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:34.691335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbc0fdca-a23b-46d5-ba21-1ea3d26b4326 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:34.692087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbc0fdca-a23b-46d5-ba21-1ea3d26b4326 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:34.692547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbc0fdca-a23b-46d5-ba21-1ea3d26b4326 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:34.693000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbc0fdca-a23b-46d5-ba21-1ea3d26b4326 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:34.698444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:34.787492 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-65e616c8-e438-4f82-ac28-0eaa518854b8 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb/action" status: 202 len: 1940 microversion: 2.75 time: 0.940446 Sep 27 09:58:34.788908 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 359/697] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 09:58:33 2022] POST /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb/action => generated 1940 bytes in 942 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Sep 27 09:58:34.795107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8eb73fc6-28c1-4250-964d-b081dc1e5925 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "e1a6a4c63e014170bae36e6b6a3cc0ba"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:34.911078 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8eb73fc6-28c1-4250-964d-b081dc1e5925 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/265395823/action" status: 200 len: 96 microversion: 2.1 time: 0.119951 Sep 27 09:58:34.919054 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 360/698] 10.222.0.26 () {60 vars in 1197 bytes} [Tue Sep 27 09:58:34 2022] POST /compute/v2.1/flavors/265395823/action => generated 96 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:58:34.921023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cbc0fdca-a23b-46d5-ba21-1ea3d26b4326 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:34.924901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ee66ad8e-fcad-4b00-acea-2a9b3338c65c req-3fcff9b6-b7cf-4e57-a1ba-9cb12ecdfade service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "185c1774-d54d-4d4e-bce6-315fb25d6db7", "tag": "74ca1e33-fb78-46e8-bca5-e323d313f753"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:34.936852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ee66ad8e-fcad-4b00-acea-2a9b3338c65c req-3fcff9b6-b7cf-4e57-a1ba-9cb12ecdfade service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:34.937898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ee66ad8e-fcad-4b00-acea-2a9b3338c65c req-3fcff9b6-b7cf-4e57-a1ba-9cb12ecdfade service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:34.938629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ee66ad8e-fcad-4b00-acea-2a9b3338c65c req-3fcff9b6-b7cf-4e57-a1ba-9cb12ecdfade service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:34.940549 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cbc0fdca-a23b-46d5-ba21-1ea3d26b4326 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1475 microversion: 2.1 time: 0.262092 Sep 27 09:58:34.942643 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 339/699] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:34 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1475 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:34.953659 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ee66ad8e-fcad-4b00-acea-2a9b3338c65c req-3fcff9b6-b7cf-4e57-a1ba-9cb12ecdfade service nova] Creating event network-changed:74ca1e33-fb78-46e8-bca5-e323d313f753 for instance 185c1774-d54d-4d4e-bce6-315fb25d6db7 on nch2-master-1 Sep 27 09:58:34.959521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d5519308-9ba5-4b75-b1a2-dbccfb98ebab tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:34.960238 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ee66ad8e-fcad-4b00-acea-2a9b3338c65c req-3fcff9b6-b7cf-4e57-a1ba-9cb12ecdfade service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038604 Sep 27 09:58:34.960630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d5519308-9ba5-4b75-b1a2-dbccfb98ebab tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:34.966246 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 361/700] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:34.973581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-823ae585-b398-434b-aa5c-c26d8270374c tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Calling method '.version_select at 0x7f7f63ac3b80>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:34.983087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5519308-9ba5-4b75-b1a2-dbccfb98ebab tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:34.983638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5519308-9ba5-4b75-b1a2-dbccfb98ebab tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:34.984078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5519308-9ba5-4b75-b1a2-dbccfb98ebab tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:34.984826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5519308-9ba5-4b75-b1a2-dbccfb98ebab tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:34.987677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5519308-9ba5-4b75-b1a2-dbccfb98ebab tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:34.989132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5519308-9ba5-4b75-b1a2-dbccfb98ebab tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:35.357862 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d5519308-9ba5-4b75-b1a2-dbccfb98ebab tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1614 microversion: 2.1 time: 0.412738 Sep 27 09:58:35.358953 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-823ae585-b398-434b-aa5c-c26d8270374c tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] 10.222.0.26 "GET /compute/v2.1/os-floating-ips" status: 200 len: 145 microversion: 2.1 time: 0.390244 Sep 27 09:58:35.372574 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 340/701] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:34 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1614 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:35.372574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c6ccbee2-78a5-46d7-abdf-cd4d93942c28 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:35.372574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c6ccbee2-78a5-46d7-abdf-cd4d93942c28 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:35.374801 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 362/702] 10.222.0.26 () {58 vars in 1149 bytes} [Tue Sep 27 09:58:34 2022] GET /compute/v2.1/os-floating-ips => generated 145 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:35.379726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6ccbee2-78a5-46d7-abdf-cd4d93942c28 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:35.379726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6ccbee2-78a5-46d7-abdf-cd4d93942c28 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:35.379726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6ccbee2-78a5-46d7-abdf-cd4d93942c28 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:35.379726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6ccbee2-78a5-46d7-abdf-cd4d93942c28 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:35.380116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6ccbee2-78a5-46d7-abdf-cd4d93942c28 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:35.380341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6ccbee2-78a5-46d7-abdf-cd4d93942c28 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:35.389300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7ad93302-cf2b-48e5-9ca5-b7ec58adfde6 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:35.390357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7ad93302-cf2b-48e5-9ca5-b7ec58adfde6 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:35.398037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ad93302-cf2b-48e5-9ca5-b7ec58adfde6 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:35.399030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ad93302-cf2b-48e5-9ca5-b7ec58adfde6 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:35.399848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ad93302-cf2b-48e5-9ca5-b7ec58adfde6 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:35.401135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ad93302-cf2b-48e5-9ca5-b7ec58adfde6 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:35.401815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ad93302-cf2b-48e5-9ca5-b7ec58adfde6 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:35.402491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ad93302-cf2b-48e5-9ca5-b7ec58adfde6 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:35.434314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:35.649134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7ad93302-cf2b-48e5-9ca5-b7ec58adfde6 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:35.658366 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7ad93302-cf2b-48e5-9ca5-b7ec58adfde6 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1317 microversion: 2.1 time: 0.282243 Sep 27 09:58:35.658971 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 363/703] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:35 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1317 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:35.783807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-c6ccbee2-78a5-46d7-abdf-cd4d93942c28 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:35.804588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d31c81d5-92dc-4e97-b3dc-c3079993ee63 tempest-FlavorsAccessTestJSON-1206729466 tempest-FlavorsAccessTestJSON-1206729466-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:35.860123 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c6ccbee2-78a5-46d7-abdf-cd4d93942c28 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.498847 Sep 27 09:58:35.861732 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 341/704] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:35 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:35.864285 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d31c81d5-92dc-4e97-b3dc-c3079993ee63 tempest-FlavorsAccessTestJSON-1206729466 tempest-FlavorsAccessTestJSON-1206729466-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 6232 microversion: 2.1 time: 0.203881 Sep 27 09:58:35.867369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5cca1858-fa91-473f-ae40-b837bf6d6f33 tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Calling method '.version_select at 0x7f7f63b8a9d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:35.877128 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 364/705] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:35 2022] GET /compute/v2.1/flavors/detail => generated 6232 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:35.885887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53be5369-f685-475b-91d2-c34d2b77ae2f tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Calling method '.version_select at 0x7f7f63b2d310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:35.890056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53be5369-f685-475b-91d2-c34d2b77ae2f tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:35.890852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53be5369-f685-475b-91d2-c34d2b77ae2f tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:35.891512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53be5369-f685-475b-91d2-c34d2b77ae2f tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:35.904041 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53be5369-f685-475b-91d2-c34d2b77ae2f tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "GET /compute/v2.1/os-migrations" status: 200 len: 409 microversion: 2.1 time: 0.025542 Sep 27 09:58:35.905061 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 365/706] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 09:58:35 2022] GET /compute/v2.1/os-migrations => generated 409 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:35.938570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c3ad15b8-87aa-4689-90df-e2f4eaff580e tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "e1a6a4c63e014170bae36e6b6a3cc0ba"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:36.091091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c3ad15b8-87aa-4689-90df-e2f4eaff580e tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/265395823/action" status: 200 len: 21 microversion: 2.1 time: 0.175947 Sep 27 09:58:36.097576 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 366/707] 10.222.0.26 () {60 vars in 1197 bytes} [Tue Sep 27 09:58:35 2022] POST /compute/v2.1/flavors/265395823/action => generated 21 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:58:36.104625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e64f4d72-da26-4238-80a1-7d14f06d558a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:36.105656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e64f4d72-da26-4238-80a1-7d14f06d558a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:36.115277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64f4d72-da26-4238-80a1-7d14f06d558a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:36.116306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64f4d72-da26-4238-80a1-7d14f06d558a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:36.116986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64f4d72-da26-4238-80a1-7d14f06d558a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:36.118773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64f4d72-da26-4238-80a1-7d14f06d558a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:36.119406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64f4d72-da26-4238-80a1-7d14f06d558a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:36.121033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64f4d72-da26-4238-80a1-7d14f06d558a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:36.384553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e64f4d72-da26-4238-80a1-7d14f06d558a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:36.396671 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e64f4d72-da26-4238-80a1-7d14f06d558a tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1475 microversion: 2.1 time: 0.297671 Sep 27 09:58:36.397612 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 367/708] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:36 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1475 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:36.403800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-1103710048", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:36.473257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:58:36.768297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:58:36.768758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:58:36.769234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:58:36.769911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:58:36.770235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:58:36.781064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:36.781549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:36.781982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:36.782716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:36.783174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:36.783602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:36.806742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for project e79eae6310354ca58046e59dfac23009. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:58:36.813689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for user 9544cdb0392f416197a586d99e8a3210 and project e79eae6310354ca58046e59dfac23009. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:58:36.834603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:58:36.837334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:58:36.895998 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d2eca69c-4a59-414a-8d35-0ae03f67c20b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.497410 Sep 27 09:58:36.896996 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 368/709] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:36 2022] POST /compute/v2.1/servers => generated 380 bytes in 499 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:58:36.902412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-09ba60a2-c54c-48e1-aae7-ef0b68c5993e tempest-FlavorsAccessTestJSON-1206729466 tempest-FlavorsAccessTestJSON-1206729466-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:36.931589 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-09ba60a2-c54c-48e1-aae7-ef0b68c5993e tempest-FlavorsAccessTestJSON-1206729466 tempest-FlavorsAccessTestJSON-1206729466-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.033080 Sep 27 09:58:36.946721 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 369/710] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:36 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:36.946721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-475e7a96-8ba6-4116-8385-02e3a950421a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:36.946721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-475e7a96-8ba6-4116-8385-02e3a950421a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:36.946721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475e7a96-8ba6-4116-8385-02e3a950421a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:36.946721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475e7a96-8ba6-4116-8385-02e3a950421a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:36.947916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475e7a96-8ba6-4116-8385-02e3a950421a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:36.947916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475e7a96-8ba6-4116-8385-02e3a950421a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:36.947916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475e7a96-8ba6-4116-8385-02e3a950421a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:36.948246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475e7a96-8ba6-4116-8385-02e3a950421a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:37.152135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-475e7a96-8ba6-4116-8385-02e3a950421a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:37.161785 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-475e7a96-8ba6-4116-8385-02e3a950421a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1317 microversion: 2.1 time: 0.227760 Sep 27 09:58:37.166382 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 370/711] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:36 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1317 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:37.183677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b7abe82f-91b5-46ec-88d6-138d1f9dda95 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:37.185025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b7abe82f-91b5-46ec-88d6-138d1f9dda95 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:37.192039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7abe82f-91b5-46ec-88d6-138d1f9dda95 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:37.192891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7abe82f-91b5-46ec-88d6-138d1f9dda95 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:37.193545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7abe82f-91b5-46ec-88d6-138d1f9dda95 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:37.194484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7abe82f-91b5-46ec-88d6-138d1f9dda95 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:37.195143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7abe82f-91b5-46ec-88d6-138d1f9dda95 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:37.195982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7abe82f-91b5-46ec-88d6-138d1f9dda95 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:37.365990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-b7abe82f-91b5-46ec-88d6-138d1f9dda95 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:37.403908 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b7abe82f-91b5-46ec-88d6-138d1f9dda95 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.236134 Sep 27 09:58:37.405031 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 371/712] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:37 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:37.409824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-97bd3b6d-5770-43aa-8a23-86e3e9b54525 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:37.415929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-97bd3b6d-5770-43aa-8a23-86e3e9b54525 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:37.529301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-97bd3b6d-5770-43aa-8a23-86e3e9b54525 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:37.547903 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-97bd3b6d-5770-43aa-8a23-86e3e9b54525 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1213 microversion: 2.1 time: 0.141525 Sep 27 09:58:37.553304 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 372/713] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:37 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1213 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:37.563448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d1bf4f93-2251-42da-823f-de04c424dbbd tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:37.610155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=91,nova_api:DELETE=5,nova_api:INSERT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:37.620218 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d1bf4f93-2251-42da-823f-de04c424dbbd tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/265395823" status: 202 len: 0 microversion: 2.1 time: 0.060788 Sep 27 09:58:37.621141 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 373/714] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 09:58:37 2022] DELETE /compute/v2.1/flavors/265395823 => generated 0 bytes in 67 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:37.631693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0b308398-1a10-4b0b-8c37-132b879940c7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:37.632356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0b308398-1a10-4b0b-8c37-132b879940c7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:37.663734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b308398-1a10-4b0b-8c37-132b879940c7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:37.664274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b308398-1a10-4b0b-8c37-132b879940c7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:37.664792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b308398-1a10-4b0b-8c37-132b879940c7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:37.666273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b308398-1a10-4b0b-8c37-132b879940c7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:37.666273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b308398-1a10-4b0b-8c37-132b879940c7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:37.666478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b308398-1a10-4b0b-8c37-132b879940c7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:37.797720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0b308398-1a10-4b0b-8c37-132b879940c7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:37.808048 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0b308398-1a10-4b0b-8c37-132b879940c7 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1433 microversion: 2.1 time: 0.180825 Sep 27 09:58:37.816741 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 374/715] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:37 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1433 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:37.824207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a965c28c-600b-448c-87e4-443ff736726b req-63e53433-b26a-4d83-8988-1748a0b7e6ac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "454cd0f6-eddc-4e88-9437-91ecce9cf66c", "name": "network-vif-deleted", "tag": "e4cac2e6-28e9-4db8-86d8-5e557c0d5c8a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:37.841521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a965c28c-600b-448c-87e4-443ff736726b req-63e53433-b26a-4d83-8988-1748a0b7e6ac service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:37.841999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a965c28c-600b-448c-87e4-443ff736726b req-63e53433-b26a-4d83-8988-1748a0b7e6ac service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:37.843939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a965c28c-600b-448c-87e4-443ff736726b req-63e53433-b26a-4d83-8988-1748a0b7e6ac service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:37.864349 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a965c28c-600b-448c-87e4-443ff736726b req-63e53433-b26a-4d83-8988-1748a0b7e6ac service nova] Creating event network-vif-deleted:e4cac2e6-28e9-4db8-86d8-5e557c0d5c8a for instance 454cd0f6-eddc-4e88-9437-91ecce9cf66c on nch2-master-1 Sep 27 09:58:37.871523 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a965c28c-600b-448c-87e4-443ff736726b req-63e53433-b26a-4d83-8988-1748a0b7e6ac service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052711 Sep 27 09:58:37.872563 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 375/716] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:37.881983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-014ea187-4fd3-455e-933c-5ba977e83643 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:37.921022 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-014ea187-4fd3-455e-933c-5ba977e83643 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.046995 Sep 27 09:58:37.922162 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 376/717] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:37 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:37.947721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d22edb86-9104-4a04-97a1-7c0dbbcce304 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessTestJSON-flavor-1038314807", "id": 1673022625, "os-flavor-access:is_public": "False"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:37.974841 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d22edb86-9104-4a04-97a1-7c0dbbcce304 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 431 microversion: 2.1 time: 0.033451 Sep 27 09:58:37.978263 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 377/718] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:37 2022] POST /compute/v2.1/flavors => generated 431 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:37.990231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e6f33bba-3b94-47ba-be8e-005ab509a977 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:38.008583 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e6f33bba-3b94-47ba-be8e-005ab509a977 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/1673022625/os-flavor-access" status: 200 len: 21 microversion: 2.1 time: 0.022531 Sep 27 09:58:38.009547 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 378/719] 10.222.0.26 () {58 vars in 1209 bytes} [Tue Sep 27 09:58:37 2022] GET /compute/v2.1/flavors/1673022625/os-flavor-access => generated 21 bytes in 25 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:58:38.019640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-02dbde82-6416-4e22-aba1-ae26f06f7134 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:38.041026 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-02dbde82-6416-4e22-aba1-ae26f06f7134 tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1673022625" status: 202 len: 0 microversion: 2.1 time: 0.025160 Sep 27 09:58:38.045166 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 379/720] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:38 2022] DELETE /compute/v2.1/flavors/1673022625 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 09:58:38.049937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2fd4ce96-8234-4a90-b711-aa0457f4f0cf tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:38.074707 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2fd4ce96-8234-4a90-b711-aa0457f4f0cf tempest-FlavorsAccessTestJSON-754582926 tempest-FlavorsAccessTestJSON-754582926-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.028315 Sep 27 09:58:38.080850 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 380/721] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:38 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:38.186770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d04f3d6-6e4a-4350-a23c-fd68d47337bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:38.187524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2d04f3d6-6e4a-4350-a23c-fd68d47337bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:38.199629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d04f3d6-6e4a-4350-a23c-fd68d47337bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:38.200410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d04f3d6-6e4a-4350-a23c-fd68d47337bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:38.201074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d04f3d6-6e4a-4350-a23c-fd68d47337bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:38.202098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d04f3d6-6e4a-4350-a23c-fd68d47337bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:38.202718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d04f3d6-6e4a-4350-a23c-fd68d47337bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:38.203298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d04f3d6-6e4a-4350-a23c-fd68d47337bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:38.379747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2d04f3d6-6e4a-4350-a23c-fd68d47337bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:38.391050 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5cca1858-fa91-473f-ae40-b837bf6d6f33 tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/bfbf338f-40ba-4c67-81fe-6f8e617570e4" status: 202 len: 0 microversion: 2.1 time: 2.528033 Sep 27 09:58:38.392501 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d04f3d6-6e4a-4350-a23c-fd68d47337bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1317 microversion: 2.1 time: 0.210023 Sep 27 09:58:38.401377 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 342/722] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 09:58:35 2022] DELETE /compute/v2.1/os-floating-ips/bfbf338f-40ba-4c67-81fe-6f8e617570e4 => generated 0 bytes in 2539 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:38.405587 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 381/723] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:38 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1317 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:38.414776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e5d2b1d6-964c-4872-80aa-2636b9d62e7a tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Action: 'create', calling method: .version_select at 0x7f7f63a839d0>, body: {"pool": null} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:38.435298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a71d10dc-f9f5-4d46-9a22-b176f043b816 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:38.435959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a71d10dc-f9f5-4d46-9a22-b176f043b816 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:38.442494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a71d10dc-f9f5-4d46-9a22-b176f043b816 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:38.442992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a71d10dc-f9f5-4d46-9a22-b176f043b816 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:38.443426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a71d10dc-f9f5-4d46-9a22-b176f043b816 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:38.444137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a71d10dc-f9f5-4d46-9a22-b176f043b816 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:38.444593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a71d10dc-f9f5-4d46-9a22-b176f043b816 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:38.445035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a71d10dc-f9f5-4d46-9a22-b176f043b816 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:38.539079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-a71d10dc-f9f5-4d46-9a22-b176f043b816 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:38.561278 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a71d10dc-f9f5-4d46-9a22-b176f043b816 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.129321 Sep 27 09:58:38.561776 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 382/724] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:38 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 09:58:38.582848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-136998f2-3dde-4f2c-aeb0-43d77cdf4cf5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:38.583596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-136998f2-3dde-4f2c-aeb0-43d77cdf4cf5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:38.591862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-136998f2-3dde-4f2c-aeb0-43d77cdf4cf5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:38.592238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-136998f2-3dde-4f2c-aeb0-43d77cdf4cf5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:38.592813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-136998f2-3dde-4f2c-aeb0-43d77cdf4cf5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:38.593636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-136998f2-3dde-4f2c-aeb0-43d77cdf4cf5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:38.594379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-136998f2-3dde-4f2c-aeb0-43d77cdf4cf5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:38.594740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-136998f2-3dde-4f2c-aeb0-43d77cdf4cf5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:38.680038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-136998f2-3dde-4f2c-aeb0-43d77cdf4cf5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:38.690325 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-136998f2-3dde-4f2c-aeb0-43d77cdf4cf5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1217 microversion: 2.1 time: 0.111597 Sep 27 09:58:38.691126 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 383/725] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:38 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1217 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:38.835529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dc01da27-dcf1-409b-b05d-7d41ec91eef6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:38.836355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dc01da27-dcf1-409b-b05d-7d41ec91eef6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:38.843756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01da27-dcf1-409b-b05d-7d41ec91eef6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:38.844567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01da27-dcf1-409b-b05d-7d41ec91eef6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:38.845438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01da27-dcf1-409b-b05d-7d41ec91eef6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:38.849928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01da27-dcf1-409b-b05d-7d41ec91eef6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:38.849928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01da27-dcf1-409b-b05d-7d41ec91eef6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:38.849928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01da27-dcf1-409b-b05d-7d41ec91eef6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:38.935783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dc01da27-dcf1-409b-b05d-7d41ec91eef6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:38.967629 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dc01da27-dcf1-409b-b05d-7d41ec91eef6 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1433 microversion: 2.1 time: 0.137000 Sep 27 09:58:38.971686 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 384/726] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:38 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1433 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:39.421722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-54d1bd4f-c7c0-4e2b-a042-51e7aa9ad8b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:39.423564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-54d1bd4f-c7c0-4e2b-a042-51e7aa9ad8b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:39.432987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d1bd4f-c7c0-4e2b-a042-51e7aa9ad8b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:39.433842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d1bd4f-c7c0-4e2b-a042-51e7aa9ad8b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:39.434461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d1bd4f-c7c0-4e2b-a042-51e7aa9ad8b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:39.435673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d1bd4f-c7c0-4e2b-a042-51e7aa9ad8b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:39.436320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d1bd4f-c7c0-4e2b-a042-51e7aa9ad8b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:39.437979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d1bd4f-c7c0-4e2b-a042-51e7aa9ad8b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:39.701938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-54d1bd4f-c7c0-4e2b-a042-51e7aa9ad8b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:39.714527 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-54d1bd4f-c7c0-4e2b-a042-51e7aa9ad8b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1317 microversion: 2.1 time: 0.297305 Sep 27 09:58:39.721477 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 385/727] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:39 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1317 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:39.754888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b126c6f-69c2-4cd5-bb63-33fdaa22b601 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:39.756686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7b126c6f-69c2-4cd5-bb63-33fdaa22b601 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:39.767142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b126c6f-69c2-4cd5-bb63-33fdaa22b601 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:39.768193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b126c6f-69c2-4cd5-bb63-33fdaa22b601 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:39.769600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b126c6f-69c2-4cd5-bb63-33fdaa22b601 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:39.771034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b126c6f-69c2-4cd5-bb63-33fdaa22b601 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:39.772097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b126c6f-69c2-4cd5-bb63-33fdaa22b601 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:39.772908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b126c6f-69c2-4cd5-bb63-33fdaa22b601 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:39.894090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-7b126c6f-69c2-4cd5-bb63-33fdaa22b601 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:39.925399 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7b126c6f-69c2-4cd5-bb63-33fdaa22b601 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.202261 Sep 27 09:58:39.926043 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 386/728] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:39 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:39.933164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fbee0646-a579-421d-8ed5-af2642e3378f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:39.933695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fbee0646-a579-421d-8ed5-af2642e3378f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:39.942020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbee0646-a579-421d-8ed5-af2642e3378f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:39.942852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbee0646-a579-421d-8ed5-af2642e3378f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:39.943637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbee0646-a579-421d-8ed5-af2642e3378f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:39.944494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbee0646-a579-421d-8ed5-af2642e3378f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:39.945241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbee0646-a579-421d-8ed5-af2642e3378f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:39.946072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbee0646-a579-421d-8ed5-af2642e3378f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:40.081140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fbee0646-a579-421d-8ed5-af2642e3378f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:40.091801 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fbee0646-a579-421d-8ed5-af2642e3378f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1209 microversion: 2.1 time: 0.163676 Sep 27 09:58:40.095907 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 387/729] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:39 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1209 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:40.117389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:40.118256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:40.128915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:40.129409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:40.133836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:40.135201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:40.139842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:40.142949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:40.247278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:40.247718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:40.248327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:40.248884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lazy-loading 'fault' on Instance uuid 454cd0f6-eddc-4e88-9437-91ecce9cf66c {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:40.304250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '4319ec4b90e1481b8be19e8f9dde1ea4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6e8a121c6c654613a6a92df03c3754fa', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:40.319191 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1f177abe-f87e-49ea-82d4-86d7646d7c55 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 200 len: 1309 microversion: 2.1 time: 0.212516 Sep 27 09:58:40.319992 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 388/730] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:40 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 1309 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:40.750626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-159491b9-dcf5-4987-8706-a282d1d9ddd4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:40.752810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-159491b9-dcf5-4987-8706-a282d1d9ddd4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:40.761239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-159491b9-dcf5-4987-8706-a282d1d9ddd4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:40.762113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-159491b9-dcf5-4987-8706-a282d1d9ddd4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:40.762704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-159491b9-dcf5-4987-8706-a282d1d9ddd4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:40.763889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-159491b9-dcf5-4987-8706-a282d1d9ddd4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:40.764541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-159491b9-dcf5-4987-8706-a282d1d9ddd4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:40.765223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-159491b9-dcf5-4987-8706-a282d1d9ddd4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:40.979268 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e5d2b1d6-964c-4872-80aa-2636b9d62e7a tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] 10.222.0.26 "POST /compute/v2.1/os-floating-ips" status: 200 len: 141 microversion: 2.1 time: 2.570920 Sep 27 09:58:40.980210 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 343/731] 10.222.0.26 () {60 vars in 1170 bytes} [Tue Sep 27 09:58:38 2022] POST /compute/v2.1/os-floating-ips => generated 141 bytes in 2573 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:40.986619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd19c24f-e93d-4a8d-8e7a-28638dc39f08 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:40.987301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cd19c24f-e93d-4a8d-8e7a-28638dc39f08 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:41.010433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd19c24f-e93d-4a8d-8e7a-28638dc39f08 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:41.011021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd19c24f-e93d-4a8d-8e7a-28638dc39f08 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:41.011457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd19c24f-e93d-4a8d-8e7a-28638dc39f08 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:41.013083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd19c24f-e93d-4a8d-8e7a-28638dc39f08 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:41.013778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd19c24f-e93d-4a8d-8e7a-28638dc39f08 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:41.014359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd19c24f-e93d-4a8d-8e7a-28638dc39f08 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:41.149338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-cd19c24f-e93d-4a8d-8e7a-28638dc39f08 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:41.189964 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd19c24f-e93d-4a8d-8e7a-28638dc39f08 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.208533 Sep 27 09:58:41.190576 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 344/732] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:40 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:41.216995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f3de95ad-966f-41f2-b829-a4245585e26c tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Calling method '.version_select at 0x7f7f63ef83a0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:41.253352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-159491b9-dcf5-4987-8706-a282d1d9ddd4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:41.264097 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-159491b9-dcf5-4987-8706-a282d1d9ddd4 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1317 microversion: 2.1 time: 0.524825 Sep 27 09:58:41.281451 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 389/733] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:40 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1317 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:41.299732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0023bc33-a330-4ae1-9230-a4096335329c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:41.300437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0023bc33-a330-4ae1-9230-a4096335329c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:41.310618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0023bc33-a330-4ae1-9230-a4096335329c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:41.311358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0023bc33-a330-4ae1-9230-a4096335329c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:41.312067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0023bc33-a330-4ae1-9230-a4096335329c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:41.313342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0023bc33-a330-4ae1-9230-a4096335329c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:41.314041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0023bc33-a330-4ae1-9230-a4096335329c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:41.317087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0023bc33-a330-4ae1-9230-a4096335329c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:41.436504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0023bc33-a330-4ae1-9230-a4096335329c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:41.441335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:41.451436 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0023bc33-a330-4ae1-9230-a4096335329c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1271 microversion: 2.1 time: 0.165933 Sep 27 09:58:41.457177 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 390/734] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:41 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1271 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:41.466292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:41.467124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] [instance: 454cd0f6-eddc-4e88-9437-91ecce9cf66c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:41.475147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:41.476751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:41.478162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:41.479040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:41.480141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:41.480787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:41.542162 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] HTTP exception thrown: Instance 454cd0f6-eddc-4e88-9437-91ecce9cf66c could not be found. Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Request method failure captured: Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c HTTP/1.1 Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMskSIxx2wr0SHypAeNgCTUvOeRPeSrzkwGS52FZpml8wBVEX5PaqILAt2hnqyFu-emYlg1O3IdjnGkCJZAtKHmyyDJU_BoNH0wJCM0pWpFGAdf_qX5mjOVGAz9VTuFFVeYOyOiirmWg8oOM7bwUHHtH7p0n7-De9qzZ894PhQ7HJXYQ Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6e8a121c6c654613a6a92df03c3754fa Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-LiveMigrationNegativeTest-1033344067 Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:58:41.548694 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6e8a121c6c654613a6a92df03c3754fa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6e8a121c6c654613a6a92df03c3754fa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6e8a121c6c654613a6a92df03c3754fa"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6e8a121c6c654613a6a92df03c3754fa"}]}] Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-LiveMigrationNegativeTest-1033344067 Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6e8a121c6c654613a6a92df03c3754fa Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-LiveMigrationNegativeTest-1033344067 Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-LiveMigrationNegativeTest-1033344067-project Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 4319ec4b90e1481b8be19e8f9dde1ea4 Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-LiveMigrationNegativeTest-1033344067-project Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 454cd0f6-eddc-4e88-9437-91ecce9cf66c could not be found. Sep 27 09:58:41.550297 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '454cd0f6-eddc-4e88-9437-91ecce9cf66c'} Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 454cd0f6-eddc-4e88-9437-91ecce9cf66c could not be found. Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:41.551231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 454cd0f6-eddc-4e88-9437-91ecce9cf66c could not be found. Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 454cd0f6-eddc-4e88-9437-91ecce9cf66c could not be found. Sep 27 09:58:41.552595 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:41.554193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] Returning 404 to user: Instance 454cd0f6-eddc-4e88-9437-91ecce9cf66c could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:41.554193 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d74c2a27-e74c-4e46-90be-0bf9f55647b5 tempest-LiveMigrationNegativeTest-1033344067 tempest-LiveMigrationNegativeTest-1033344067-project] 10.222.0.26 "GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c" status: 404 len: 111 microversion: 2.1 time: 0.089187 Sep 27 09:58:41.554193 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 391/735] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:41 2022] GET /compute/v2.1/servers/454cd0f6-eddc-4e88-9437-91ecce9cf66c => generated 111 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:41.985937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ed6fe65b-6e32-4c17-a923-2c7292775d35 req-d4dea98f-a0a3-4d66-b964-92b22f0aac73 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "185c1774-d54d-4d4e-bce6-315fb25d6db7", "name": "network-vif-plugged", "status": "completed", "tag": "74ca1e33-fb78-46e8-bca5-e323d313f753"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:42.010707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ed6fe65b-6e32-4c17-a923-2c7292775d35 req-d4dea98f-a0a3-4d66-b964-92b22f0aac73 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:42.013009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ed6fe65b-6e32-4c17-a923-2c7292775d35 req-d4dea98f-a0a3-4d66-b964-92b22f0aac73 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:42.018690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ed6fe65b-6e32-4c17-a923-2c7292775d35 req-d4dea98f-a0a3-4d66-b964-92b22f0aac73 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:42.057392 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ed6fe65b-6e32-4c17-a923-2c7292775d35 req-d4dea98f-a0a3-4d66-b964-92b22f0aac73 service nova] Creating event network-vif-plugged:74ca1e33-fb78-46e8-bca5-e323d313f753 for instance 185c1774-d54d-4d4e-bce6-315fb25d6db7 on nch2-master-1 Sep 27 09:58:42.085705 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ed6fe65b-6e32-4c17-a923-2c7292775d35 req-d4dea98f-a0a3-4d66-b964-92b22f0aac73 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.106703 Sep 27 09:58:42.088951 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 392/736] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:58:42.228197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d114c53-d849-4852-ab67-a86e042bf1f6 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:42.229421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d114c53-d849-4852-ab67-a86e042bf1f6 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:42.246587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d114c53-d849-4852-ab67-a86e042bf1f6 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:42.248254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d114c53-d849-4852-ab67-a86e042bf1f6 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:42.249578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d114c53-d849-4852-ab67-a86e042bf1f6 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:42.250424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d114c53-d849-4852-ab67-a86e042bf1f6 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:42.250897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d114c53-d849-4852-ab67-a86e042bf1f6 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:42.251325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d114c53-d849-4852-ab67-a86e042bf1f6 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:42.441867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-7d114c53-d849-4852-ab67-a86e042bf1f6 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:42.461664 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d114c53-d849-4852-ab67-a86e042bf1f6 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.243718 Sep 27 09:58:42.463191 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 393/737] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:42 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:42.468832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8bda937c-3d07-4933-ab06-405407a664eb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:42.469897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8bda937c-3d07-4933-ab06-405407a664eb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:42.476610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bda937c-3d07-4933-ab06-405407a664eb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:42.477572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bda937c-3d07-4933-ab06-405407a664eb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:42.478256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bda937c-3d07-4933-ab06-405407a664eb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:42.479449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bda937c-3d07-4933-ab06-405407a664eb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:42.481730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bda937c-3d07-4933-ab06-405407a664eb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:42.482432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bda937c-3d07-4933-ab06-405407a664eb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:42.724232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8bda937c-3d07-4933-ab06-405407a664eb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:42.732705 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8bda937c-3d07-4933-ab06-405407a664eb tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1317 microversion: 2.1 time: 0.268086 Sep 27 09:58:42.739226 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 394/738] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:42 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1317 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:42.757296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-400da161-0397-48f4-975e-f5eab50c02c6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:42.758530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-400da161-0397-48f4-975e-f5eab50c02c6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:42.769837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-400da161-0397-48f4-975e-f5eab50c02c6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:42.770954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-400da161-0397-48f4-975e-f5eab50c02c6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:42.772284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-400da161-0397-48f4-975e-f5eab50c02c6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:42.773206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-400da161-0397-48f4-975e-f5eab50c02c6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:42.774776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-400da161-0397-48f4-975e-f5eab50c02c6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:42.775531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-400da161-0397-48f4-975e-f5eab50c02c6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:42.963830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-400da161-0397-48f4-975e-f5eab50c02c6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:42.980137 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-400da161-0397-48f4-975e-f5eab50c02c6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1271 microversion: 2.1 time: 0.238684 Sep 27 09:58:42.981658 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 395/739] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:42 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1271 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:43.494108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-61d036a1-bd02-4f81-81dc-e5800a449864 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:43.495620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-61d036a1-bd02-4f81-81dc-e5800a449864 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:43.503626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61d036a1-bd02-4f81-81dc-e5800a449864 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:43.509734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61d036a1-bd02-4f81-81dc-e5800a449864 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:43.510424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61d036a1-bd02-4f81-81dc-e5800a449864 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:43.511843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61d036a1-bd02-4f81-81dc-e5800a449864 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:43.512482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61d036a1-bd02-4f81-81dc-e5800a449864 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:43.521122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61d036a1-bd02-4f81-81dc-e5800a449864 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:43.682509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-61d036a1-bd02-4f81-81dc-e5800a449864 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:43.727754 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-61d036a1-bd02-4f81-81dc-e5800a449864 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.236174 Sep 27 09:58:43.752149 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 396/740] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:43 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 09:58:43.959639 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f3de95ad-966f-41f2-b829-a4245585e26c tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/04d2d1d6-7371-411d-b1be-c8251d86a8eb" status: 202 len: 0 microversion: 2.1 time: 2.767875 Sep 27 09:58:43.961217 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 345/741] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 09:58:41 2022] DELETE /compute/v2.1/os-floating-ips/04d2d1d6-7371-411d-b1be-c8251d86a8eb => generated 0 bytes in 2770 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:43.966255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-96860e77-c59e-4049-b30f-f992414e0b2c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:43.967163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-96860e77-c59e-4049-b30f-f992414e0b2c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:43.980357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96860e77-c59e-4049-b30f-f992414e0b2c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:43.984065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96860e77-c59e-4049-b30f-f992414e0b2c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:43.984763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96860e77-c59e-4049-b30f-f992414e0b2c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:43.988596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96860e77-c59e-4049-b30f-f992414e0b2c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:43.989365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96860e77-c59e-4049-b30f-f992414e0b2c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:43.989919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96860e77-c59e-4049-b30f-f992414e0b2c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:44.117851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9cc0bc91-7c91-4dcc-8575-8d881fbd460d tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-682129634", "id": 1434076330, "os-flavor-access:is_public": "False"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:44.155153 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9cc0bc91-7c91-4dcc-8575-8d881fbd460d tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 438 microversion: 2.1 time: 0.400862 Sep 27 09:58:44.156489 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 397/742] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:43 2022] POST /compute/v2.1/flavors => generated 438 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:44.162075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d5bccb3-6757-46b4-8d88-688f9b6d3afa tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Calling method '.version_select at 0x7f7f63aa3940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:44.220493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-9d5bccb3-6757-46b4-8d88-688f9b6d3afa tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP 04d2d1d6-7371-411d-b1be-c8251d86a8eb could not be found", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 09:58:44.221592 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-9d5bccb3-6757-46b4-8d88-688f9b6d3afa tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] HTTP exception thrown: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d5bccb3-6757-46b4-8d88-688f9b6d3afa tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Request method failure captured: Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-floating-ips/04d2d1d6-7371-411d-b1be-c8251d86a8eb HTTP/1.1 Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMslFqJBYpYp0ULjh__03avSp50KZwZNCnzaCKWNl_wqKC5aJIOyrpOSX6B88VXwv2OFdh_-KE7SDcIEYV-4XAKCQR7CPbZYWG3S_gBj7bWQ3YLW0Fl81k5L8AQYzxqQZMPLaXWTnhVicG2bYrQprbzChszet-jX5VxiRTHvKIzwkbSE Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 7476bff780f0442e9be3b2c4ff4b7e41 Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FloatingIPsTestJSON-593880026 Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:58:44.233955 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7476bff780f0442e9be3b2c4ff4b7e41"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7476bff780f0442e9be3b2c4ff4b7e41"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7476bff780f0442e9be3b2c4ff4b7e41"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7476bff780f0442e9be3b2c4ff4b7e41"}]}] Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FloatingIPsTestJSON-593880026 Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 7476bff780f0442e9be3b2c4ff4b7e41 Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FloatingIPsTestJSON-593880026 Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FloatingIPsTestJSON-593880026-project Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 0320d122f53c435588a69baafb9bcf98 Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FloatingIPsTestJSON-593880026-project Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: exception: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '04d2d1d6-7371-411d-b1be-c8251d86a8eb'} Sep 27 09:58:44.235560 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2824, in get_floating_ip Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1008, in show_floatingip Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 09:58:44.236512 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP 04d2d1d6-7371-411d-b1be-c8251d86a8eb could not be found Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-185e7f5f-0d7c-4d2e-b833-8cab6e9d200c'] Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 94, in show Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb. Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:44.237839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 97, in show Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:44.239271 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:44.240769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb Sep 27 09:58:44.240769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.241828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d5bccb3-6757-46b4-8d88-688f9b6d3afa tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Returning 404 to user: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:44.242770 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d5bccb3-6757-46b4-8d88-688f9b6d3afa tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] 10.222.0.26 "GET /compute/v2.1/os-floating-ips/04d2d1d6-7371-411d-b1be-c8251d86a8eb" status: 404 len: 111 microversion: 2.1 time: 0.085131 Sep 27 09:58:44.243346 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 398/743] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 09:58:44 2022] GET /compute/v2.1/os-floating-ips/04d2d1d6-7371-411d-b1be-c8251d86a8eb => generated 111 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:44.251068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6b24a4b9-7259-4599-be43-a9d0498f197e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:44.253227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6b24a4b9-7259-4599-be43-a9d0498f197e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:44.263113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b24a4b9-7259-4599-be43-a9d0498f197e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:44.264163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b24a4b9-7259-4599-be43-a9d0498f197e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:44.264847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b24a4b9-7259-4599-be43-a9d0498f197e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:44.265788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b24a4b9-7259-4599-be43-a9d0498f197e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:44.266451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b24a4b9-7259-4599-be43-a9d0498f197e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:44.267236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b24a4b9-7259-4599-be43-a9d0498f197e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:44.331051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-96860e77-c59e-4049-b30f-f992414e0b2c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:44.340978 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-96860e77-c59e-4049-b30f-f992414e0b2c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1317 microversion: 2.1 time: 0.378164 Sep 27 09:58:44.342209 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 346/744] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:43 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1317 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:44.347621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b91001d0-9a37-49ba-84fd-99de1549b097 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "ed5453928ec84da48914a387ca695902"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:44.634221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6b24a4b9-7259-4599-be43-a9d0498f197e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:44.660518 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6b24a4b9-7259-4599-be43-a9d0498f197e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1313 microversion: 2.1 time: 0.414620 Sep 27 09:58:44.661512 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 399/745] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:44 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1313 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:44.665838 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b91001d0-9a37-49ba-84fd-99de1549b097 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/1434076330/action" status: 200 len: 97 microversion: 2.1 time: 0.321634 Sep 27 09:58:44.667288 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 347/746] 10.222.0.26 () {60 vars in 1200 bytes} [Tue Sep 27 09:58:44 2022] POST /compute/v2.1/flavors/1434076330/action => generated 97 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:58:44.669913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bc2ee909-0b43-4d08-bb75-f31af0565b97 tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Calling method '.version_select at 0x7f7f63b2d940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:44.683165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9be162f1-a974-49a0-b9b2-6e88a52b860f tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "ed5453928ec84da48914a387ca695902"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:44.776378 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-9be162f1-a974-49a0-b9b2-6e88a52b860f tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] HTTP exception thrown: Flavor access already exists for flavor 35 and project ed5453928ec84da48914a387ca695902 combination. Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9be162f1-a974-49a0-b9b2-6e88a52b860f tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Request method failure captured: Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/flavors/1434076330/action HTTP/1.1 Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 67 Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMslS0KnBxQQycCrd9FWNxlZCDJ5elQnp5WPoIznL7pQGf2gfMtkzAd_-0m9w5jtd11zEFEU_YUzufaDjnkIUKJEqWG0O7yCIt8YuDfPxptsa2q-F9jv_DUvj85H7atomT71ElwsbngIBPnYb4Adi7xIRbtU-IjCfj0DrqGO8o9Iuj3g Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 38e89dbee95b4e308e77acde9c3a4dfc Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-1411695947 Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 09:58:44.781243 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/38e89dbee95b4e308e77acde9c3a4dfc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/38e89dbee95b4e308e77acde9c3a4dfc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/38e89dbee95b4e308e77acde9c3a4dfc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_38e89dbee95b4e308e77acde9c3a4dfc"}]}] Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-1411695947 Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 38e89dbee95b4e308e77acde9c3a4dfc Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-1411695947 Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 107c3429510a42ae969e20671542338d Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: {"addTenantAccess": {"tenant": "ed5453928ec84da48914a387ca695902"}} Sep 27 09:58:44.782602 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: exception: Flavor access already exists for flavor 35 and project ed5453928ec84da48914a387ca695902 combination. Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1434076330', 'body': {'addTenantAccess': {'tenant': 'ed5453928ec84da48914a387ca695902'}}} Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = self._query(query) Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi conn.query(q) Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result.read() Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 09:58:44.783332 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1062, "Duplicate entry '35-ed5453928ec84da48914a387ca695902' for key 'flavor_projects.uniq_flavor_projects0flavor_id0project_id'") Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 73, in _flavor_add_project Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi project.save(context.session) Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/models.py", line 38, in save Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi session.flush() Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._flush(objects) Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 09:58:44.784436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi transaction.rollback(_capture_exception=True) Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi compat.raise_( Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi flush_context.execute() Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi rec.execute(self) Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi util.preloaded.orm_persistence.save_obj( Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _emit_insert_statements( Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = connection._execute_20( Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return meth(self, args_10style, kwargs_10style, execution_options) Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 27 09:58:44.785718 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return connection._execute_clauseelement( Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = self._execute_context( Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._handle_dbapi_exception( Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = self._query(query) Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi conn.query(q) Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 09:58:44.791829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result.read() Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '35-ed5453928ec84da48914a387ca695902' for key 'flavor_projects.uniq_flavor_projects0flavor_id0project_id'") Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [SQL: INSERT INTO flavor_projects (created_at, updated_at, flavor_id, project_id) VALUES (%(created_at)s, %(updated_at)s, %(flavor_id)s, %(project_id)s)] Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2022, 9, 27, 9, 58, 44, 769499), 'updated_at': None, 'flavor_id': 35, 'project_id': 'ed5453928ec84da48914a387ca695902'}] Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:44.794018 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 79, in _add_tenant_access Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi flavor.add_access(tenant) Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 411, in add_access Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._add_access(project_id) Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 404, in _add_access Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._flavor_add_project(self._context, self.id, project_id) Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 397, in _flavor_add_project Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return _flavor_add_project(context, flavor_id, project_id) Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 75, in _flavor_add_project Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.FlavorAccessExists(flavor_id=flavor_id, Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.FlavorAccessExists: Flavor access already exists for flavor 35 and project ed5453928ec84da48914a387ca695902 combination. Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:44.796239 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 83, in _add_tenant_access Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=err.format_message()) Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Flavor access already exists for flavor 35 and project ed5453928ec84da48914a387ca695902 combination. Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:44.798916 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:44.802342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor access already exists for flavor 35 and project ed5453928ec84da48914a387ca695902 combination. Sep 27 09:58:44.802342 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.802342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9be162f1-a974-49a0-b9b2-6e88a52b860f tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Returning 409 to user: Flavor access already exists for flavor 35 and project ed5453928ec84da48914a387ca695902 combination. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:44.802342 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9be162f1-a974-49a0-b9b2-6e88a52b860f tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/1434076330/action" status: 409 len: 152 microversion: 2.1 time: 0.118901 Sep 27 09:58:44.802342 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 348/747] 10.222.0.26 () {60 vars in 1200 bytes} [Tue Sep 27 09:58:44 2022] POST /compute/v2.1/flavors/1434076330/action => generated 152 bytes in 120 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:58:44.803469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f42df53-198e-4fd7-ae7b-29199624097f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:44.804878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f42df53-198e-4fd7-ae7b-29199624097f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:44.814593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f42df53-198e-4fd7-ae7b-29199624097f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:44.814919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f42df53-198e-4fd7-ae7b-29199624097f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:44.815305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f42df53-198e-4fd7-ae7b-29199624097f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:44.816547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f42df53-198e-4fd7-ae7b-29199624097f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:44.817938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f42df53-198e-4fd7-ae7b-29199624097f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:44.818582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f42df53-198e-4fd7-ae7b-29199624097f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:44.855855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-bc2ee909-0b43-4d08-bb75-f31af0565b97 tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP 04d2d1d6-7371-411d-b1be-c8251d86a8eb could not be found", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 09:58:44.857009 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-bc2ee909-0b43-4d08-bb75-f31af0565b97 tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] HTTP exception thrown: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bc2ee909-0b43-4d08-bb75-f31af0565b97 tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Request method failure captured: Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-floating-ips/04d2d1d6-7371-411d-b1be-c8251d86a8eb HTTP/1.1 Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMslFqJBYpYp0ULjh__03avSp50KZwZNCnzaCKWNl_wqKC5aJIOyrpOSX6B88VXwv2OFdh_-KE7SDcIEYV-4XAKCQR7CPbZYWG3S_gBj7bWQ3YLW0Fl81k5L8AQYzxqQZMPLaXWTnhVicG2bYrQprbzChszet-jX5VxiRTHvKIzwkbSE Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 7476bff780f0442e9be3b2c4ff4b7e41 Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FloatingIPsTestJSON-593880026 Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:58:44.859166 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7476bff780f0442e9be3b2c4ff4b7e41"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7476bff780f0442e9be3b2c4ff4b7e41"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7476bff780f0442e9be3b2c4ff4b7e41"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7476bff780f0442e9be3b2c4ff4b7e41"}]}] Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FloatingIPsTestJSON-593880026 Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 7476bff780f0442e9be3b2c4ff4b7e41 Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FloatingIPsTestJSON-593880026 Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FloatingIPsTestJSON-593880026-project Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 0320d122f53c435588a69baafb9bcf98 Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FloatingIPsTestJSON-593880026-project Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: exception: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '04d2d1d6-7371-411d-b1be-c8251d86a8eb'} Sep 27 09:58:44.860484 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2824, in get_floating_ip Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1008, in show_floatingip Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 09:58:44.861314 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP 04d2d1d6-7371-411d-b1be-c8251d86a8eb could not be found Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-02cca63a-8552-4ba3-bc20-4cffa4f245a5'] Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 158, in delete Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb. Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:44.862511 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 161, in delete Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:44.863774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:44.864956 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb Sep 27 09:58:44.864956 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:44.866584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bc2ee909-0b43-4d08-bb75-f31af0565b97 tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] Returning 404 to user: Floating IP not found for ID 04d2d1d6-7371-411d-b1be-c8251d86a8eb {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:44.868648 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bc2ee909-0b43-4d08-bb75-f31af0565b97 tempest-FloatingIPsTestJSON-593880026 tempest-FloatingIPsTestJSON-593880026-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/04d2d1d6-7371-411d-b1be-c8251d86a8eb" status: 404 len: 111 microversion: 2.1 time: 0.204842 Sep 27 09:58:44.869245 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 400/748] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 09:58:44 2022] DELETE /compute/v2.1/os-floating-ips/04d2d1d6-7371-411d-b1be-c8251d86a8eb => generated 111 bytes in 207 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:44.874255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d2e92840-a51a-432a-8c4a-096c95b63a36 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "ed5453928ec84da48914a387ca695902"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:44.931182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-8f42df53-198e-4fd7-ae7b-29199624097f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:44.972691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f42df53-198e-4fd7-ae7b-29199624097f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.175510 Sep 27 09:58:44.980934 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 349/749] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:44 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:44.991889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5d2b9d7b-6c60-4b7a-9a14-6a4313c22c9a req-857bd61d-6ce6-4270-85d8-f5f7fe69016f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "48f61d6b-3985-49a2-9eb9-dd250711cdf4", "tag": "04d6d3fb-4795-41f4-a54d-61772b8a10a8"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:45.010774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5d2b9d7b-6c60-4b7a-9a14-6a4313c22c9a req-857bd61d-6ce6-4270-85d8-f5f7fe69016f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:45.013053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5d2b9d7b-6c60-4b7a-9a14-6a4313c22c9a req-857bd61d-6ce6-4270-85d8-f5f7fe69016f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:45.017651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5d2b9d7b-6c60-4b7a-9a14-6a4313c22c9a req-857bd61d-6ce6-4270-85d8-f5f7fe69016f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:45.042395 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5d2b9d7b-6c60-4b7a-9a14-6a4313c22c9a req-857bd61d-6ce6-4270-85d8-f5f7fe69016f service nova] Creating event network-changed:04d6d3fb-4795-41f4-a54d-61772b8a10a8 for instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 on nch2-master-1 Sep 27 09:58:45.048536 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5d2b9d7b-6c60-4b7a-9a14-6a4313c22c9a req-857bd61d-6ce6-4270-85d8-f5f7fe69016f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070288 Sep 27 09:58:45.049870 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 350/750] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:45.177932 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d2e92840-a51a-432a-8c4a-096c95b63a36 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/1434076330/action" status: 200 len: 21 microversion: 2.1 time: 0.306700 Sep 27 09:58:45.179756 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 401/751] 10.222.0.26 () {60 vars in 1200 bytes} [Tue Sep 27 09:58:44 2022] POST /compute/v2.1/flavors/1434076330/action => generated 21 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:58:45.190798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5df0abfc-bb8d-4dc1-acc0-92190f50e211 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:45.211292 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5df0abfc-bb8d-4dc1-acc0-92190f50e211 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1434076330" status: 202 len: 0 microversion: 2.1 time: 0.025065 Sep 27 09:58:45.211934 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 351/752] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:45 2022] DELETE /compute/v2.1/flavors/1434076330 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:45.223520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fac8538d-e60c-4234-a14e-45e897ae85e5 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:45.253036 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fac8538d-e60c-4234-a14e-45e897ae85e5 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.036232 Sep 27 09:58:45.267039 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 402/753] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:45 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:45.277137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-956fb891-96c8-4dba-a4ed-ab34862f7c4c tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-1217267449", "id": 1631403328, "os-flavor-access:is_public": "True"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:45.308444 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-956fb891-96c8-4dba-a4ed-ab34862f7c4c tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 438 microversion: 2.1 time: 0.035428 Sep 27 09:58:45.321782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a943d4e0-5839-4158-8edd-fabf4be8ee33 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:45.326268 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 352/754] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:45 2022] POST /compute/v2.1/flavors => generated 438 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:45.329962 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a943d4e0-5839-4158-8edd-fabf4be8ee33 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] HTTP exception thrown: Access list not available for public flavors. Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a943d4e0-5839-4158-8edd-fabf4be8ee33 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Request method failure captured: Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/flavors/1631403328/os-flavor-access HTTP/1.1 Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMslS0KnBxQQycCrd9FWNxlZCDJ5elQnp5WPoIznL7pQGf2gfMtkzAd_-0m9w5jtd11zEFEU_YUzufaDjnkIUKJEqWG0O7yCIt8YuDfPxptsa2q-F9jv_DUvj85H7atomT71ElwsbngIBPnYb4Adi7xIRbtU-IjCfj0DrqGO8o9Iuj3g Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 38e89dbee95b4e308e77acde9c3a4dfc Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-1411695947 Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:58:45.331832 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/38e89dbee95b4e308e77acde9c3a4dfc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/38e89dbee95b4e308e77acde9c3a4dfc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/38e89dbee95b4e308e77acde9c3a4dfc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_38e89dbee95b4e308e77acde9c3a4dfc"}]}] Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-1411695947 Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 38e89dbee95b4e308e77acde9c3a4dfc Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-1411695947 Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 107c3429510a42ae969e20671542338d Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: exception: Access list not available for public flavors. Sep 27 09:58:45.333408 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'flavor_id': '1631403328'} Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 52, in index Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Access list not available for public flavors. Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Access list not available for public flavors. Sep 27 09:58:45.334295 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:45.337395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a943d4e0-5839-4158-8edd-fabf4be8ee33 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Returning 404 to user: Access list not available for public flavors. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:45.338423 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a943d4e0-5839-4158-8edd-fabf4be8ee33 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/1631403328/os-flavor-access" status: 404 len: 91 microversion: 2.1 time: 0.020952 Sep 27 09:58:45.347534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-92d246b0-6d2d-4f17-97c2-c1db816b20a3 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:45.350311 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 403/755] 10.222.0.26 () {58 vars in 1209 bytes} [Tue Sep 27 09:58:45 2022] GET /compute/v2.1/flavors/1631403328/os-flavor-access => generated 91 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 09:58:45.366647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3035363-293c-426a-bc38-10b042546f67 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:45.368129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3035363-293c-426a-bc38-10b042546f67 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:45.375001 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-92d246b0-6d2d-4f17-97c2-c1db816b20a3 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1631403328" status: 202 len: 0 microversion: 2.1 time: 0.031374 Sep 27 09:58:45.382993 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 353/756] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:45 2022] DELETE /compute/v2.1/flavors/1631403328 => generated 0 bytes in 38 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:45.386728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3035363-293c-426a-bc38-10b042546f67 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:45.389952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3035363-293c-426a-bc38-10b042546f67 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:45.390648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3035363-293c-426a-bc38-10b042546f67 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:45.391572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3035363-293c-426a-bc38-10b042546f67 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:45.392564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4db2e1d8-874d-4d27-92b7-cbe8e28dcfce tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:45.393278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3035363-293c-426a-bc38-10b042546f67 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:45.394669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3035363-293c-426a-bc38-10b042546f67 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:45.442433 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4db2e1d8-874d-4d27-92b7-cbe8e28dcfce tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.054801 Sep 27 09:58:45.448951 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 354/757] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:45 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:45.476730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-41a18c34-eb73-478e-b560-6a87a0492ba1 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-1861527517", "id": 2050361396, "os-flavor-access:is_public": "False"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:45.506138 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-41a18c34-eb73-478e-b560-6a87a0492ba1 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 439 microversion: 2.1 time: 0.032847 Sep 27 09:58:45.515230 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 355/758] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:45 2022] POST /compute/v2.1/flavors => generated 439 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:45.749304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-16454f4c-4477-41f2-80a2-fcf3b9e00ba3 tempest-FlavorsAccessNegativeTestJSON-875077973 tempest-FlavorsAccessNegativeTestJSON-875077973-project] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "ed5453928ec84da48914a387ca695902"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:45.754074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-16454f4c-4477-41f2-80a2-fcf3b9e00ba3 tempest-FlavorsAccessNegativeTestJSON-875077973 tempest-FlavorsAccessNegativeTestJSON-875077973-project] Policy check for os_compute_api:os-flavor-access:add_tenant_access failed with credentials {'is_admin': False, 'user_id': 'ac12036ae0a0427b819ccb5b75a25184', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed5453928ec84da48914a387ca695902', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-16454f4c-4477-41f2-80a2-fcf3b9e00ba3 tempest-FlavorsAccessNegativeTestJSON-875077973 tempest-FlavorsAccessNegativeTestJSON-875077973-project] Request method failure captured: Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/flavors/2050361396/action HTTP/1.1 Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 67 Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMslRRmM3XwoAyrFq23ogKCsV3-LhfvTEDF-rc9oS48U1y5Zr1dVTA5MKxI41Sn6PVyVH5xTpf_e9_No_cKuT3TEzLpactEUiSt0efdnQUYw5-oiDOaKV2Q3rq4XAJ_j0tfHPc0AoWedx1-KUQKLfuBpXQumNfNFfgdeJxXoglhCk33w Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: ed5453928ec84da48914a387ca695902 Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-875077973 Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:58:45.755885 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ed5453928ec84da48914a387ca695902"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ed5453928ec84da48914a387ca695902"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ed5453928ec84da48914a387ca695902"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ed5453928ec84da48914a387ca695902"}]}] Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-875077973 Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: ed5453928ec84da48914a387ca695902 Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-875077973 Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FlavorsAccessNegativeTestJSON-875077973-project Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: ac12036ae0a0427b819ccb5b75a25184 Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-875077973-project Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: {"addTenantAccess": {"tenant": "ed5453928ec84da48914a387ca695902"}} Sep 27 09:58:45.757349 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2050361396', 'body': {'addTenantAccess': {'tenant': 'ed5453928ec84da48914a387ca695902'}}} Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 66, in _add_tenant_access Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(fa_policies.POLICY_ROOT % "add_tenant_access", target={}) Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:58:45.758305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. Sep 27 09:58:45.759595 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:45.761143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-16454f4c-4477-41f2-80a2-fcf3b9e00ba3 tempest-FlavorsAccessNegativeTestJSON-875077973 tempest-FlavorsAccessNegativeTestJSON-875077973-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-access:add_tenant_access to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:45.761143 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-16454f4c-4477-41f2-80a2-fcf3b9e00ba3 tempest-FlavorsAccessNegativeTestJSON-875077973 tempest-FlavorsAccessNegativeTestJSON-875077973-project] 10.222.0.26 "POST /compute/v2.1/flavors/2050361396/action" status: 403 len: 130 microversion: 2.1 time: 0.241054 Sep 27 09:58:45.762449 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 356/759] 10.222.0.26 () {60 vars in 1200 bytes} [Tue Sep 27 09:58:45 2022] POST /compute/v2.1/flavors/2050361396/action => generated 130 bytes in 246 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:45.767348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-49ba0400-0b90-471a-b172-e2f2f458f2d4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:45.768091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-49ba0400-0b90-471a-b172-e2f2f458f2d4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:45.770973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=6,nova_api:UPDATE=1,nova_api:DELETE=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:45.777625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49ba0400-0b90-471a-b172-e2f2f458f2d4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:45.778125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49ba0400-0b90-471a-b172-e2f2f458f2d4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:45.778567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49ba0400-0b90-471a-b172-e2f2f458f2d4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:45.779327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49ba0400-0b90-471a-b172-e2f2f458f2d4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:45.779748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49ba0400-0b90-471a-b172-e2f2f458f2d4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:45.780176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49ba0400-0b90-471a-b172-e2f2f458f2d4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:45.934401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-49ba0400-0b90-471a-b172-e2f2f458f2d4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:45.943194 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-49ba0400-0b90-471a-b172-e2f2f458f2d4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1313 microversion: 2.1 time: 0.179662 Sep 27 09:58:45.943801 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 357/760] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:45 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1313 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:45.949813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-742325e2-8060-4ee6-a37f-d5c61bcfc046 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:45.973685 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-742325e2-8060-4ee6-a37f-d5c61bcfc046 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/2050361396" status: 202 len: 0 microversion: 2.1 time: 0.028235 Sep 27 09:58:45.974985 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 358/761] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:45 2022] DELETE /compute/v2.1/flavors/2050361396 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:45.983892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-982ef659-724a-44e2-ae55-f4d832a85067 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:46.006728 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-982ef659-724a-44e2-ae55-f4d832a85067 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.026545 Sep 27 09:58:46.007332 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 359/762] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:45 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:46.021175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9ed517e-1d4a-4252-a7d9-6f20ca9158a5 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:46.022284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9ed517e-1d4a-4252-a7d9-6f20ca9158a5 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:46.032552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ed517e-1d4a-4252-a7d9-6f20ca9158a5 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:46.033354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ed517e-1d4a-4252-a7d9-6f20ca9158a5 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:46.033831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ed517e-1d4a-4252-a7d9-6f20ca9158a5 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:46.034620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ed517e-1d4a-4252-a7d9-6f20ca9158a5 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:46.035022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ed517e-1d4a-4252-a7d9-6f20ca9158a5 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:46.035627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ed517e-1d4a-4252-a7d9-6f20ca9158a5 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:46.122648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-c9ed517e-1d4a-4252-a7d9-6f20ca9158a5 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:46.125989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:46.126530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b3035363-293c-426a-bc38-10b042546f67 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:46.135886 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3035363-293c-426a-bc38-10b042546f67 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1467 microversion: 2.1 time: 0.774004 Sep 27 09:58:46.137286 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 404/763] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:45 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1467 bytes in 776 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:46.143241 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9ed517e-1d4a-4252-a7d9-6f20ca9158a5 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.128649 Sep 27 09:58:46.144175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-358529e6-b179-43e0-a827-a8770de1dcf5 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-1811606796", "id": 1193330472, "os-flavor-access:is_public": "False"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:46.153255 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 360/764] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:46 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:46.180988 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-358529e6-b179-43e0-a827-a8770de1dcf5 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 439 microversion: 2.1 time: 0.041822 Sep 27 09:58:46.182313 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 405/765] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:46 2022] POST /compute/v2.1/flavors => generated 439 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:46.187654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6d056633-bff2-4c83-8c25-5a33ee78a233 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:46.188492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6d056633-bff2-4c83-8c25-5a33ee78a233 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:46.196282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d056633-bff2-4c83-8c25-5a33ee78a233 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:46.196871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d056633-bff2-4c83-8c25-5a33ee78a233 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:46.197315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d056633-bff2-4c83-8c25-5a33ee78a233 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:46.198339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d056633-bff2-4c83-8c25-5a33ee78a233 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:46.199009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d056633-bff2-4c83-8c25-5a33ee78a233 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:46.199436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d056633-bff2-4c83-8c25-5a33ee78a233 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:46.292811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3160e2b-7b76-4f28-bcea-3913e1905aec tempest-NetworksTest-654453589 tempest-NetworksTest-654453589-project-admin] Calling method '.version_select at 0x7f7f680189d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:46.428041 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e3160e2b-7b76-4f28-bcea-3913e1905aec tempest-NetworksTest-654453589 tempest-NetworksTest-654453589-project-admin] 10.222.0.26 "GET /compute/v2.1/os-networks" status: 200 len: 4703 microversion: 2.1 time: 0.271894 Sep 27 09:58:46.432619 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 361/766] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 09:58:46 2022] GET /compute/v2.1/os-networks => generated 4703 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:46.436083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-52416c73-0278-49e5-8ff7-c42d0cd623f5 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Action: 'action', calling method: >, body: {"addTenantAccess": {"tenant": "ed5453928ec84da48914a387ca695902"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:46.537490 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6d056633-bff2-4c83-8c25-5a33ee78a233 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1618 microversion: 2.1 time: 0.353521 Sep 27 09:58:46.538387 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 406/767] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:46 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1618 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:46.542784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a8da3fd2-689f-4256-bf5c-894aa48aa7a3 tempest-NetworksTest-654453589 tempest-NetworksTest-654453589-project-admin] Calling method '.version_select at 0x7f7f63974ca0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:46.627238 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-52416c73-0278-49e5-8ff7-c42d0cd623f5 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/1193330472/action" status: 200 len: 97 microversion: 2.1 time: 0.196687 Sep 27 09:58:46.628112 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 362/768] 10.222.0.26 () {60 vars in 1200 bytes} [Tue Sep 27 09:58:46 2022] POST /compute/v2.1/flavors/1193330472/action => generated 97 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:58:46.640163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c060613c-4c9d-4e88-b8d9-4dc9548b323b tempest-FlavorsAccessNegativeTestJSON-875077973 tempest-FlavorsAccessNegativeTestJSON-875077973-project] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "ed5453928ec84da48914a387ca695902"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:46.643664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c060613c-4c9d-4e88-b8d9-4dc9548b323b tempest-FlavorsAccessNegativeTestJSON-875077973 tempest-FlavorsAccessNegativeTestJSON-875077973-project] Policy check for os_compute_api:os-flavor-access:remove_tenant_access failed with credentials {'is_admin': False, 'user_id': 'ac12036ae0a0427b819ccb5b75a25184', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ed5453928ec84da48914a387ca695902', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c060613c-4c9d-4e88-b8d9-4dc9548b323b tempest-FlavorsAccessNegativeTestJSON-875077973 tempest-FlavorsAccessNegativeTestJSON-875077973-project] Request method failure captured: Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/flavors/1193330472/action HTTP/1.1 Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 70 Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMslRRmM3XwoAyrFq23ogKCsV3-LhfvTEDF-rc9oS48U1y5Zr1dVTA5MKxI41Sn6PVyVH5xTpf_e9_No_cKuT3TEzLpactEUiSt0efdnQUYw5-oiDOaKV2Q3rq4XAJ_j0tfHPc0AoWedx1-KUQKLfuBpXQumNfNFfgdeJxXoglhCk33w Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: ed5453928ec84da48914a387ca695902 Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-875077973 Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:58:46.645723 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ed5453928ec84da48914a387ca695902"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ed5453928ec84da48914a387ca695902"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ed5453928ec84da48914a387ca695902"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ed5453928ec84da48914a387ca695902"}]}] Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-875077973 Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: ed5453928ec84da48914a387ca695902 Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-875077973 Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FlavorsAccessNegativeTestJSON-875077973-project Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: ac12036ae0a0427b819ccb5b75a25184 Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-875077973-project Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: {"removeTenantAccess": {"tenant": "ed5453928ec84da48914a387ca695902"}} Sep 27 09:58:46.647334 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1193330472', 'body': {'removeTenantAccess': {'tenant': 'ed5453928ec84da48914a387ca695902'}}} Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 91, in _remove_tenant_access Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can( Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:58:46.648220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. Sep 27 09:58:46.650108 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:58:46.651462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c060613c-4c9d-4e88-b8d9-4dc9548b323b tempest-FlavorsAccessNegativeTestJSON-875077973 tempest-FlavorsAccessNegativeTestJSON-875077973-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-flavor-access:remove_tenant_access to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:46.651831 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c060613c-4c9d-4e88-b8d9-4dc9548b323b tempest-FlavorsAccessNegativeTestJSON-875077973 tempest-FlavorsAccessNegativeTestJSON-875077973-project] 10.222.0.26 "POST /compute/v2.1/flavors/1193330472/action" status: 403 len: 133 microversion: 2.1 time: 0.015663 Sep 27 09:58:46.652651 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 363/769] 10.222.0.26 () {60 vars in 1200 bytes} [Tue Sep 27 09:58:46 2022] POST /compute/v2.1/flavors/1193330472/action => generated 133 bytes in 18 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:46.660819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8c75f5e2-ec40-4a2a-b985-f8ff97c0e29c tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "ed5453928ec84da48914a387ca695902"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:46.744793 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8c75f5e2-ec40-4a2a-b985-f8ff97c0e29c tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/1193330472/action" status: 200 len: 21 microversion: 2.1 time: 0.087371 Sep 27 09:58:46.745402 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 364/770] 10.222.0.26 () {60 vars in 1200 bytes} [Tue Sep 27 09:58:46 2022] POST /compute/v2.1/flavors/1193330472/action => generated 21 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 09:58:46.750059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-164978048", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}, "os:scheduler_hints": {"different_host": ["552575d5-492a-4680-9972-29347ad5479b"]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:46.773511 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a8da3fd2-689f-4256-bf5c-894aa48aa7a3 tempest-NetworksTest-654453589 tempest-NetworksTest-654453589-project-admin] 10.222.0.26 "GET /compute/v2.1/os-networks/cf91963d-b887-4e7e-bee4-e38f7a5f86af" status: 200 len: 652 microversion: 2.1 time: 0.233896 Sep 27 09:58:46.774997 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 407/771] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 09:58:46 2022] GET /compute/v2.1/os-networks/cf91963d-b887-4e7e-bee4-e38f7a5f86af => generated 652 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:46.780355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-123ff771-9f8f-4238-a382-49c6b9f5253b tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:46.794889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:46.798257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:DELETE=9,nova_api:SELECT=88,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:46.806945 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-123ff771-9f8f-4238-a382-49c6b9f5253b tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1193330472" status: 202 len: 0 microversion: 2.1 time: 0.030295 Sep 27 09:58:46.810975 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 408/772] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:46 2022] DELETE /compute/v2.1/flavors/1193330472 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:46.812582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-faf4b885-c4ed-492b-a92f-c093906b00d2 tempest-NetworksTest-654453589 tempest-NetworksTest-654453589-project-admin] Calling method '.version_select at 0x7f7f63aa30d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:46.813806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:58:47.050999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:58:47.051638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:58:47.051910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:58:47.052329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:58:47.052769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:58:47.062360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:47.062842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:47.063294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:47.064021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:47.064482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:47.064966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:47.083762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:58:47.088919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:58:47.102803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:58:47.105421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:58:47.118952 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-faf4b885-c4ed-492b-a92f-c093906b00d2 tempest-NetworksTest-654453589 tempest-NetworksTest-654453589-project-admin] 10.222.0.26 "GET /compute/v2.1/os-networks" status: 200 len: 4703 microversion: 2.1 time: 0.309387 Sep 27 09:58:47.120156 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 409/773] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 09:58:46 2022] GET /compute/v2.1/os-networks => generated 4703 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:47.131308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e92f04fe-1a4f-4b02-8d17-e0092bc96f86 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:47.158333 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e92f04fe-1a4f-4b02-8d17-e0092bc96f86 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.036497 Sep 27 09:58:47.159613 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 410/774] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:47 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:47.164945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-307ef598-ec30-41a5-93b4-44d1c22d5399 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:47.165633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-307ef598-ec30-41a5-93b4-44d1c22d5399 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:47.178619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-307ef598-ec30-41a5-93b4-44d1c22d5399 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:47.181051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-307ef598-ec30-41a5-93b4-44d1c22d5399 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:47.182028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-307ef598-ec30-41a5-93b4-44d1c22d5399 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:47.182877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-307ef598-ec30-41a5-93b4-44d1c22d5399 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:47.183549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-307ef598-ec30-41a5-93b4-44d1c22d5399 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:47.184161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-307ef598-ec30-41a5-93b4-44d1c22d5399 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:47.198884 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e7d21951-e045-445d-89a7-d548e59bfd6a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.450575 Sep 27 09:58:47.202269 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 365/775] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 456 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:58:47.208625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-57bd65c3-839c-44f2-9d3e-56fd6e5c655e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:47.213914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-57bd65c3-839c-44f2-9d3e-56fd6e5c655e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:47.225659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57bd65c3-839c-44f2-9d3e-56fd6e5c655e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:47.226471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57bd65c3-839c-44f2-9d3e-56fd6e5c655e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:47.228164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57bd65c3-839c-44f2-9d3e-56fd6e5c655e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:47.229661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57bd65c3-839c-44f2-9d3e-56fd6e5c655e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:47.230857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57bd65c3-839c-44f2-9d3e-56fd6e5c655e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:47.231452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57bd65c3-839c-44f2-9d3e-56fd6e5c655e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:47.246450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-307ef598-ec30-41a5-93b4-44d1c22d5399 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:58:47.434623 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-307ef598-ec30-41a5-93b4-44d1c22d5399 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "DELETE /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 204 len: 0 microversion: 2.1 time: 0.271537 Sep 27 09:58:47.435734 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 411/776] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:58:47 2022] DELETE /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 0 bytes in 275 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:58:47.463762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-45a5bb80-a0ba-4417-92db-8881cc283cef tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:47.469331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-45a5bb80-a0ba-4417-92db-8881cc283cef tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:47.481059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-57bd65c3-839c-44f2-9d3e-56fd6e5c655e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:47.489757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45a5bb80-a0ba-4417-92db-8881cc283cef tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:47.490537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45a5bb80-a0ba-4417-92db-8881cc283cef tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:47.491703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45a5bb80-a0ba-4417-92db-8881cc283cef tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:47.492404 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-57bd65c3-839c-44f2-9d3e-56fd6e5c655e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1313 microversion: 2.1 time: 0.288558 Sep 27 09:58:47.493674 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 366/777] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:47 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1313 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:47.496057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45a5bb80-a0ba-4417-92db-8881cc283cef tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:47.497312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45a5bb80-a0ba-4417-92db-8881cc283cef tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:47.497866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45a5bb80-a0ba-4417-92db-8881cc283cef tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:47.501518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d35ef08e-cc73-4dc6-9e83-d34e6a6fb37a tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "disk": 10, "name": "tempest-FlavorsAccessNegativeTestJSON-flavor-1295174287", "id": 1697427610, "os-flavor-access:is_public": "False"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:47.544763 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d35ef08e-cc73-4dc6-9e83-d34e6a6fb37a tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 439 microversion: 2.1 time: 0.047803 Sep 27 09:58:47.548968 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 367/778] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:47 2022] POST /compute/v2.1/flavors => generated 439 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:47.574140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:47.580776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d7a61885-1925-4ca5-a4e6-d41034647042 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:47.586124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d7a61885-1925-4ca5-a4e6-d41034647042 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:47.694745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-45a5bb80-a0ba-4417-92db-8881cc283cef tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:47.736859 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-45a5bb80-a0ba-4417-92db-8881cc283cef tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.299405 Sep 27 09:58:47.740237 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 412/779] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:47 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:47.745873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cbd8190d-caed-4a7b-b8a9-9b4095229165 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Action: 'action', calling method: >, body: {"removeTenantAccess": {"tenant": "a68f04c73d0c460386e37324fabab1e5"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:47.806902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d7a61885-1925-4ca5-a4e6-d41034647042 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:47.817272 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d7a61885-1925-4ca5-a4e6-d41034647042 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1216 microversion: 2.1 time: 0.265571 Sep 27 09:58:47.817892 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 368/780] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:47 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1216 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:47.852926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersWithSpecificFlavorTestJSON-server-972944077", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "436599854", "adminPass": "***", "networks": [{"uuid": "ef5e7302-9921-42d2-81b1-295d060e5a12"}], "security_groups": [{"name": "tempest-securitygroup--891093939"}], "key_name": "tempest-keypair-211851818", "user_data": "}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:48.165548 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-cbd8190d-caed-4a7b-b8a9-9b4095229165 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] HTTP exception thrown: Flavor access not found for 39 / a68f04c73d0c460386e37324fabab1e5 combination. Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cbd8190d-caed-4a7b-b8a9-9b4095229165 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Request method failure captured: Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/flavors/1697427610/action HTTP/1.1 Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 70 Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMslS0KnBxQQycCrd9FWNxlZCDJ5elQnp5WPoIznL7pQGf2gfMtkzAd_-0m9w5jtd11zEFEU_YUzufaDjnkIUKJEqWG0O7yCIt8YuDfPxptsa2q-F9jv_DUvj85H7atomT71ElwsbngIBPnYb4Adi7xIRbtU-IjCfj0DrqGO8o9Iuj3g Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 38e89dbee95b4e308e77acde9c3a4dfc Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FlavorsAccessNegativeTestJSON-1411695947 Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:58:48.169223 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/38e89dbee95b4e308e77acde9c3a4dfc"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/38e89dbee95b4e308e77acde9c3a4dfc"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/38e89dbee95b4e308e77acde9c3a4dfc"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_38e89dbee95b4e308e77acde9c3a4dfc"}]}] Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FlavorsAccessNegativeTestJSON-1411695947 Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 38e89dbee95b4e308e77acde9c3a4dfc Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FlavorsAccessNegativeTestJSON-1411695947 Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 107c3429510a42ae969e20671542338d Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: {"removeTenantAccess": {"tenant": "a68f04c73d0c460386e37324fabab1e5"}} Sep 27 09:58:48.170894 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: exception: Flavor access not found for 39 / a68f04c73d0c460386e37324fabab1e5 combination. Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '1697427610', 'body': {'removeTenantAccess': {'tenant': 'a68f04c73d0c460386e37324fabab1e5'}}} Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 104, in _remove_tenant_access Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi flavor.remove_access(tenant) Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 423, in remove_access Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._remove_access(project_id) Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 416, in _remove_access Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._flavor_del_project(self._context, self.id, project_id) Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 401, in _flavor_del_project Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return _flavor_del_project(context, flavor_id, project_id) Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 86, in _flavor_del_project Sep 27 09:58:48.171722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.FlavorAccessNotFound(flavor_id=flavor_id, Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.FlavorAccessNotFound: Flavor access not found for 39 / a68f04c73d0c460386e37324fabab1e5 combination. Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_access.py", line 107, in _remove_tenant_access Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor access not found for 39 / a68f04c73d0c460386e37324fabab1e5 combination. Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:48.173009 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:58:48.175475 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:58:48.175475 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:58:48.175475 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:58:48.175475 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:58:48.175475 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor access not found for 39 / a68f04c73d0c460386e37324fabab1e5 combination. Sep 27 09:58:48.175475 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:58:48.179706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cbd8190d-caed-4a7b-b8a9-9b4095229165 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Returning 404 to user: Flavor access not found for 39 / a68f04c73d0c460386e37324fabab1e5 combination. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:58:48.181489 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cbd8190d-caed-4a7b-b8a9-9b4095229165 tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors/1697427610/action" status: 404 len: 124 microversion: 2.1 time: 0.439657 Sep 27 09:58:48.182691 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 413/781] 10.222.0.26 () {60 vars in 1200 bytes} [Tue Sep 27 09:58:47 2022] POST /compute/v2.1/flavors/1697427610/action => generated 124 bytes in 442 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:58:48.192208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bcffc134-275f-45a8-b247-aabfddad434c tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:48.220920 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bcffc134-275f-45a8-b247-aabfddad434c tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1697427610" status: 202 len: 0 microversion: 2.1 time: 0.032529 Sep 27 09:58:48.222247 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 414/782] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 09:58:48 2022] DELETE /compute/v2.1/flavors/1697427610 => generated 0 bytes in 35 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:48.231348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-37acbe03-3a98-454a-bc44-7241e55ccc5c tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:48.255864 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-37acbe03-3a98-454a-bc44-7241e55ccc5c tempest-FlavorsAccessNegativeTestJSON-1411695947 tempest-FlavorsAccessNegativeTestJSON-1411695947-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.029495 Sep 27 09:58:48.257155 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 415/783] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 09:58:48 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:48.377339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] validate_networks() for [('ef5e7302-9921-42d2-81b1-295d060e5a12', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:58:48.517933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-34bced94-ee37-466e-bfd7-b078b67704d3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:48.518903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-34bced94-ee37-466e-bfd7-b078b67704d3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:48.526531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34bced94-ee37-466e-bfd7-b078b67704d3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:48.527296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34bced94-ee37-466e-bfd7-b078b67704d3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:48.527937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34bced94-ee37-466e-bfd7-b078b67704d3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:48.528842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34bced94-ee37-466e-bfd7-b078b67704d3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:48.529528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34bced94-ee37-466e-bfd7-b078b67704d3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:48.530096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34bced94-ee37-466e-bfd7-b078b67704d3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:49.085616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-34bced94-ee37-466e-bfd7-b078b67704d3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:49.097526 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-34bced94-ee37-466e-bfd7-b078b67704d3 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1313 microversion: 2.1 time: 0.583999 Sep 27 09:58:49.098868 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 416/784] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:48 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1313 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:49.113761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0c87a14e-613b-440f-b0a3-7ca86a27fd5a tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:49.114670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0c87a14e-613b-440f-b0a3-7ca86a27fd5a tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:49.127170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c87a14e-613b-440f-b0a3-7ca86a27fd5a tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:49.130420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c87a14e-613b-440f-b0a3-7ca86a27fd5a tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:49.130420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c87a14e-613b-440f-b0a3-7ca86a27fd5a tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:49.131078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c87a14e-613b-440f-b0a3-7ca86a27fd5a tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:49.135319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c87a14e-613b-440f-b0a3-7ca86a27fd5a tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:49.136416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c87a14e-613b-440f-b0a3-7ca86a27fd5a tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:49.357203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-0c87a14e-613b-440f-b0a3-7ca86a27fd5a tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:49.397779 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0c87a14e-613b-440f-b0a3-7ca86a27fd5a tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.295429 Sep 27 09:58:49.398965 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 417/785] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:49 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:49.408221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-81f86645-044c-4b98-91c6-9a28cd5acf0b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:49.410655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-81f86645-044c-4b98-91c6-9a28cd5acf0b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:49.424557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81f86645-044c-4b98-91c6-9a28cd5acf0b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:49.425756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81f86645-044c-4b98-91c6-9a28cd5acf0b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:49.426734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81f86645-044c-4b98-91c6-9a28cd5acf0b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:49.427684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81f86645-044c-4b98-91c6-9a28cd5acf0b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:49.428386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81f86645-044c-4b98-91c6-9a28cd5acf0b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:49.429601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81f86645-044c-4b98-91c6-9a28cd5acf0b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:49.660118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-81f86645-044c-4b98-91c6-9a28cd5acf0b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:49.670211 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-81f86645-044c-4b98-91c6-9a28cd5acf0b tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1220 microversion: 2.1 time: 0.268907 Sep 27 09:58:49.671820 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 418/786] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:49 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1220 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:49.766488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:58:49.767068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:58:49.767874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:58:49.768427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:58:49.768874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:58:49.785748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:49.786459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:49.788166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:49.790410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:49.791735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:49.795977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:49.846444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Getting quotas for project 1881e013b4b844cba1afce9fd21f8dd7. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:58:49.869832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Getting quotas for user b4cebc85e62046d28fab159c6bb13e15 and project 1881e013b4b844cba1afce9fd21f8dd7. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:58:49.903229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:58:49.907622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=-1,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='blank',tag=None,updated_at=,uuid=,volume_id=None,volume_size=1,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:58:50.028751 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6b4d597e-6798-4ec7-8ebd-74707e508915 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 2.192942 Sep 27 09:58:50.029953 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 369/787] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:58:47 2022] POST /compute/v2.1/servers => generated 401 bytes in 2198 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:58:50.048605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f456a7b5-fef8-4489-92b1-f5913a697d64 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:50.049277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f456a7b5-fef8-4489-92b1-f5913a697d64 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:50.125319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b8ac667f-bcfc-4346-be6f-e592c27c4c08 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:50.126380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b8ac667f-bcfc-4346-be6f-e592c27c4c08 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:50.143296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8ac667f-bcfc-4346-be6f-e592c27c4c08 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:50.144405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8ac667f-bcfc-4346-be6f-e592c27c4c08 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:50.144888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8ac667f-bcfc-4346-be6f-e592c27c4c08 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:50.146113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8ac667f-bcfc-4346-be6f-e592c27c4c08 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:50.146761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8ac667f-bcfc-4346-be6f-e592c27c4c08 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:50.147341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8ac667f-bcfc-4346-be6f-e592c27c4c08 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:50.302986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f456a7b5-fef8-4489-92b1-f5913a697d64 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:50.319591 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f456a7b5-fef8-4489-92b1-f5913a697d64 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1261 microversion: 2.1 time: 0.280239 Sep 27 09:58:50.321876 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 419/788] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:50 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1261 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:50.430963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6df6b024-117c-4086-8252-08521210219b tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:50.432385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6df6b024-117c-4086-8252-08521210219b tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:50.443922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6df6b024-117c-4086-8252-08521210219b tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:50.445332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6df6b024-117c-4086-8252-08521210219b tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:50.445776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6df6b024-117c-4086-8252-08521210219b tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:50.449650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6df6b024-117c-4086-8252-08521210219b tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:50.450400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6df6b024-117c-4086-8252-08521210219b tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:50.451132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6df6b024-117c-4086-8252-08521210219b tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:50.671416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:50.687050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-6df6b024-117c-4086-8252-08521210219b tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:50.713640 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6df6b024-117c-4086-8252-08521210219b tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.287923 Sep 27 09:58:50.714236 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 420/789] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:50 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:50.740991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7bd8cacf-81ba-40a2-81b0-b4a1b20da29d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:50.741543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7bd8cacf-81ba-40a2-81b0-b4a1b20da29d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:50.748443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bd8cacf-81ba-40a2-81b0-b4a1b20da29d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:50.749656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bd8cacf-81ba-40a2-81b0-b4a1b20da29d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:50.750243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bd8cacf-81ba-40a2-81b0-b4a1b20da29d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:50.751116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bd8cacf-81ba-40a2-81b0-b4a1b20da29d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:50.751720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bd8cacf-81ba-40a2-81b0-b4a1b20da29d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:50.752286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bd8cacf-81ba-40a2-81b0-b4a1b20da29d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:50.892514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b8ac667f-bcfc-4346-be6f-e592c27c4c08 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:50.904063 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b8ac667f-bcfc-4346-be6f-e592c27c4c08 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1313 microversion: 2.1 time: 0.783984 Sep 27 09:58:50.917407 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 370/790] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:50 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1313 bytes in 796 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:50.935758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7bd8cacf-81ba-40a2-81b0-b4a1b20da29d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:50.948623 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7bd8cacf-81ba-40a2-81b0-b4a1b20da29d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1268 microversion: 2.1 time: 0.218581 Sep 27 09:58:50.949681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 421/791] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:50 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1268 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:51.343844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b4f39322-a3a7-49c0-95f2-d0944b302167 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:51.344458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b4f39322-a3a7-49c0-95f2-d0944b302167 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:51.351410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f39322-a3a7-49c0-95f2-d0944b302167 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:51.355749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f39322-a3a7-49c0-95f2-d0944b302167 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:51.356918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f39322-a3a7-49c0-95f2-d0944b302167 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:51.358394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f39322-a3a7-49c0-95f2-d0944b302167 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:51.359377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f39322-a3a7-49c0-95f2-d0944b302167 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:51.361916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f39322-a3a7-49c0-95f2-d0944b302167 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:51.566871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b4f39322-a3a7-49c0-95f2-d0944b302167 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:51.584004 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b4f39322-a3a7-49c0-95f2-d0944b302167 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1313 microversion: 2.1 time: 0.243324 Sep 27 09:58:51.598872 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 371/792] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:51 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1313 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:51.744638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-40e2e713-918a-4b6e-b597-60fb44776513 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:51.745309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-40e2e713-918a-4b6e-b597-60fb44776513 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:51.757545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40e2e713-918a-4b6e-b597-60fb44776513 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:51.764529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40e2e713-918a-4b6e-b597-60fb44776513 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:51.767220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40e2e713-918a-4b6e-b597-60fb44776513 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:51.770266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40e2e713-918a-4b6e-b597-60fb44776513 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:51.771494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40e2e713-918a-4b6e-b597-60fb44776513 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:51.773670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40e2e713-918a-4b6e-b597-60fb44776513 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:51.931558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cba85a4f-1b1c-49d0-8170-f7ddaf7a3c5c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:51.932174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cba85a4f-1b1c-49d0-8170-f7ddaf7a3c5c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:51.948599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba85a4f-1b1c-49d0-8170-f7ddaf7a3c5c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:51.949622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba85a4f-1b1c-49d0-8170-f7ddaf7a3c5c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:51.951828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba85a4f-1b1c-49d0-8170-f7ddaf7a3c5c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:51.953845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba85a4f-1b1c-49d0-8170-f7ddaf7a3c5c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:51.957227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba85a4f-1b1c-49d0-8170-f7ddaf7a3c5c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:51.957227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba85a4f-1b1c-49d0-8170-f7ddaf7a3c5c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:52.081294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-40e2e713-918a-4b6e-b597-60fb44776513 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:52.133488 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-40e2e713-918a-4b6e-b597-60fb44776513 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.392271 Sep 27 09:58:52.136462 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 422/793] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:51 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 09:58:52.148118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9679232b-130d-4223-bfb9-4b30d7de6a78 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:52.149635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9679232b-130d-4223-bfb9-4b30d7de6a78 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:52.165538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9679232b-130d-4223-bfb9-4b30d7de6a78 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:52.166040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9679232b-130d-4223-bfb9-4b30d7de6a78 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:52.166495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9679232b-130d-4223-bfb9-4b30d7de6a78 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:52.172426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9679232b-130d-4223-bfb9-4b30d7de6a78 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:52.175349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9679232b-130d-4223-bfb9-4b30d7de6a78 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:52.176176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9679232b-130d-4223-bfb9-4b30d7de6a78 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:52.373142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cba85a4f-1b1c-49d0-8170-f7ddaf7a3c5c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:52.381797 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cba85a4f-1b1c-49d0-8170-f7ddaf7a3c5c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1313 microversion: 2.1 time: 0.455155 Sep 27 09:58:52.386122 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 372/794] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:51 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1313 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:52.516283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-97c1d6f3-9b70-4c21-b489-3a3ab4f5a79f req-cd093161-8718-42b8-8f74-7499db900da9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48f61d6b-3985-49a2-9eb9-dd250711cdf4", "name": "network-vif-plugged", "status": "completed", "tag": "04d6d3fb-4795-41f4-a54d-61772b8a10a8"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:52.539270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-97c1d6f3-9b70-4c21-b489-3a3ab4f5a79f req-cd093161-8718-42b8-8f74-7499db900da9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:52.540156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-97c1d6f3-9b70-4c21-b489-3a3ab4f5a79f req-cd093161-8718-42b8-8f74-7499db900da9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:52.540781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-97c1d6f3-9b70-4c21-b489-3a3ab4f5a79f req-cd093161-8718-42b8-8f74-7499db900da9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:52.564230 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-97c1d6f3-9b70-4c21-b489-3a3ab4f5a79f req-cd093161-8718-42b8-8f74-7499db900da9 service nova] Creating event network-vif-plugged:04d6d3fb-4795-41f4-a54d-61772b8a10a8 for instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 on nch2-master-1 Sep 27 09:58:52.570758 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-97c1d6f3-9b70-4c21-b489-3a3ab4f5a79f req-cd093161-8718-42b8-8f74-7499db900da9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059006 Sep 27 09:58:52.571806 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 373/795] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:58:52.614042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd2aae44-e932-4acc-998a-772ec5d94d4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:52.614959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cd2aae44-e932-4acc-998a-772ec5d94d4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:52.631146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd2aae44-e932-4acc-998a-772ec5d94d4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:52.635036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd2aae44-e932-4acc-998a-772ec5d94d4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:52.635757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd2aae44-e932-4acc-998a-772ec5d94d4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:52.636699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd2aae44-e932-4acc-998a-772ec5d94d4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:52.637189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd2aae44-e932-4acc-998a-772ec5d94d4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:52.637681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd2aae44-e932-4acc-998a-772ec5d94d4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:53.004542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9679232b-130d-4223-bfb9-4b30d7de6a78 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:53.015168 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9679232b-130d-4223-bfb9-4b30d7de6a78 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1274 microversion: 2.1 time: 0.877543 Sep 27 09:58:53.015890 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 423/796] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:52 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1274 bytes in 879 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:53.053575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cd2aae44-e932-4acc-998a-772ec5d94d4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:53.081431 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd2aae44-e932-4acc-998a-772ec5d94d4c tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1321 microversion: 2.1 time: 0.471203 Sep 27 09:58:53.082609 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 374/797] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:52 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1321 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:53.166225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-83d8f499-339e-47c6-9c9e-7ef280ad6714 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:53.166805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-83d8f499-339e-47c6-9c9e-7ef280ad6714 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:53.193547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83d8f499-339e-47c6-9c9e-7ef280ad6714 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:53.194065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83d8f499-339e-47c6-9c9e-7ef280ad6714 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:53.194508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83d8f499-339e-47c6-9c9e-7ef280ad6714 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:53.195388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83d8f499-339e-47c6-9c9e-7ef280ad6714 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:53.195846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83d8f499-339e-47c6-9c9e-7ef280ad6714 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:53.196271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83d8f499-339e-47c6-9c9e-7ef280ad6714 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:53.371893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-83d8f499-339e-47c6-9c9e-7ef280ad6714 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:53.404972 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-83d8f499-339e-47c6-9c9e-7ef280ad6714 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.242534 Sep 27 09:58:53.405573 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 424/798] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:53 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:53.424085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25878cca-c42f-422a-9d32-e66cee493cc5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:53.426531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-25878cca-c42f-422a-9d32-e66cee493cc5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:53.441347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25878cca-c42f-422a-9d32-e66cee493cc5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:53.442020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25878cca-c42f-422a-9d32-e66cee493cc5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:53.443623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25878cca-c42f-422a-9d32-e66cee493cc5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:53.445331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25878cca-c42f-422a-9d32-e66cee493cc5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:53.446233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25878cca-c42f-422a-9d32-e66cee493cc5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:53.446999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25878cca-c42f-422a-9d32-e66cee493cc5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:54.042706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07ec8f53-6389-4d4d-804e-cdc5b0d72481 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:54.049474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-07ec8f53-6389-4d4d-804e-cdc5b0d72481 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:54.063522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07ec8f53-6389-4d4d-804e-cdc5b0d72481 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:54.065237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07ec8f53-6389-4d4d-804e-cdc5b0d72481 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:54.065826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07ec8f53-6389-4d4d-804e-cdc5b0d72481 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:54.067900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07ec8f53-6389-4d4d-804e-cdc5b0d72481 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:54.068947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07ec8f53-6389-4d4d-804e-cdc5b0d72481 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:54.069393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07ec8f53-6389-4d4d-804e-cdc5b0d72481 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:54.263690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-25878cca-c42f-422a-9d32-e66cee493cc5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:54.273030 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-25878cca-c42f-422a-9d32-e66cee493cc5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1313 microversion: 2.1 time: 0.860725 Sep 27 09:58:54.273723 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 375/799] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:53 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1313 bytes in 863 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:54.286488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2634d449-c525-474e-938d-ad8ccdf3ff3d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:54.288956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2634d449-c525-474e-938d-ad8ccdf3ff3d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:54.308011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2634d449-c525-474e-938d-ad8ccdf3ff3d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:54.310569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2634d449-c525-474e-938d-ad8ccdf3ff3d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:54.311212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2634d449-c525-474e-938d-ad8ccdf3ff3d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:54.313066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2634d449-c525-474e-938d-ad8ccdf3ff3d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:54.315540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2634d449-c525-474e-938d-ad8ccdf3ff3d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:54.316180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2634d449-c525-474e-938d-ad8ccdf3ff3d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:54.533239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2634d449-c525-474e-938d-ad8ccdf3ff3d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:54.548470 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2634d449-c525-474e-938d-ad8ccdf3ff3d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1319 microversion: 2.1 time: 0.269190 Sep 27 09:58:54.549161 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 376/800] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:54 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1319 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:54.775407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-08d3fd99-a1b0-4985-843d-cc7f19275346 tempest-SecurityGroupsTestAdminJSON-1600475331 tempest-SecurityGroupsTestAdminJSON-1600475331-project] Action: 'create', calling method: .version_select at 0x7f7f63ab3ca0>, body: {"security_group": {"name": "tempest-securitygroup-1543941440", "description": "tempest-description-760170516"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:55.033184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-07ec8f53-6389-4d4d-804e-cdc5b0d72481 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:55.042977 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-07ec8f53-6389-4d4d-804e-cdc5b0d72481 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1316 microversion: 2.1 time: 1.004052 Sep 27 09:58:55.068380 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 425/801] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:54 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1316 bytes in 1012 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:55.068380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5325cb4e-2907-473e-a4bd-41f4e340106e tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:55.068380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5325cb4e-2907-473e-a4bd-41f4e340106e tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:55.088272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5325cb4e-2907-473e-a4bd-41f4e340106e tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:55.089204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5325cb4e-2907-473e-a4bd-41f4e340106e tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:55.089879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5325cb4e-2907-473e-a4bd-41f4e340106e tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:55.090824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5325cb4e-2907-473e-a4bd-41f4e340106e tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:55.091628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5325cb4e-2907-473e-a4bd-41f4e340106e tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:55.092319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5325cb4e-2907-473e-a4bd-41f4e340106e tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:55.319706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-5325cb4e-2907-473e-a4bd-41f4e340106e tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:55.353000 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5325cb4e-2907-473e-a4bd-41f4e340106e tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.301403 Sep 27 09:58:55.355337 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 426/802] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:55 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:55.362040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7a977d62-e3c7-4864-bfd7-a738cd3769b1 req-16da4e48-04c0-46de-adb6-bc01f40b9850 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "030e5dcc-dc63-43e6-9254-ef26b66b5cde", "name": "network-vif-deleted", "tag": "3dbbf26e-6eb1-444a-bd75-862612ffb4bc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:55.375681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a977d62-e3c7-4864-bfd7-a738cd3769b1 req-16da4e48-04c0-46de-adb6-bc01f40b9850 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:55.376524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a977d62-e3c7-4864-bfd7-a738cd3769b1 req-16da4e48-04c0-46de-adb6-bc01f40b9850 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:55.377311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a977d62-e3c7-4864-bfd7-a738cd3769b1 req-16da4e48-04c0-46de-adb6-bc01f40b9850 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:55.393932 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7a977d62-e3c7-4864-bfd7-a738cd3769b1 req-16da4e48-04c0-46de-adb6-bc01f40b9850 service nova] Creating event network-vif-deleted:3dbbf26e-6eb1-444a-bd75-862612ffb4bc for instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde on nch2-master-1 Sep 27 09:58:55.402391 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7a977d62-e3c7-4864-bfd7-a738cd3769b1 req-16da4e48-04c0-46de-adb6-bc01f40b9850 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044835 Sep 27 09:58:55.403661 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 427/803] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:55.411856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0fdc8b2d-4f1a-4134-8860-0bbccc1a6463 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:55.412977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0fdc8b2d-4f1a-4134-8860-0bbccc1a6463 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:55.422901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fdc8b2d-4f1a-4134-8860-0bbccc1a6463 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:55.423830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fdc8b2d-4f1a-4134-8860-0bbccc1a6463 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:55.424593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fdc8b2d-4f1a-4134-8860-0bbccc1a6463 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:55.425667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fdc8b2d-4f1a-4134-8860-0bbccc1a6463 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:55.426267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fdc8b2d-4f1a-4134-8860-0bbccc1a6463 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:55.426885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fdc8b2d-4f1a-4134-8860-0bbccc1a6463 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:55.748593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0fdc8b2d-4f1a-4134-8860-0bbccc1a6463 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:55.757708 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0fdc8b2d-4f1a-4134-8860-0bbccc1a6463 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1463 microversion: 2.1 time: 0.351788 Sep 27 09:58:55.758297 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 428/804] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:55 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1463 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:55.777636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-074b389a-09e7-4fa3-a305-3318d2f89873 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:55.778320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-074b389a-09e7-4fa3-a305-3318d2f89873 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:55.793726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b389a-09e7-4fa3-a305-3318d2f89873 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:55.794237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b389a-09e7-4fa3-a305-3318d2f89873 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:55.794662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b389a-09e7-4fa3-a305-3318d2f89873 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:55.797106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b389a-09e7-4fa3-a305-3318d2f89873 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:55.797818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b389a-09e7-4fa3-a305-3318d2f89873 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:55.799791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-074b389a-09e7-4fa3-a305-3318d2f89873 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:55.902799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-074b389a-09e7-4fa3-a305-3318d2f89873 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:55.911167 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-074b389a-09e7-4fa3-a305-3318d2f89873 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1319 microversion: 2.1 time: 0.148179 Sep 27 09:58:55.917042 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 429/805] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:55 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:55.922363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:55.924675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:55.941027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:55.942468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:55.942935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:55.943793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:55.944446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:55.945191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:56.057903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:56.076222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Old instance type m1.heat2, new instance type m1.heat {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 09:58:56.087954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:58:56.088645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:58:56.089755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:58:56.090338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:58:56.090973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:58:56.291374 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1b5f4286-902e-455d-a91f-f84affdf4dfb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "POST /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4/action" status: 202 len: 0 microversion: 2.1 time: 0.373018 Sep 27 09:58:56.292721 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 430/806] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:58:55 2022] POST /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4/action => generated 0 bytes in 375 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:58:56.299283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-268b8a5d-d7f2-4ba5-9c36-e9d5db69d340 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:56.300369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-268b8a5d-d7f2-4ba5-9c36-e9d5db69d340 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:56.308621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-268b8a5d-d7f2-4ba5-9c36-e9d5db69d340 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:56.309576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-268b8a5d-d7f2-4ba5-9c36-e9d5db69d340 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:56.310312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-268b8a5d-d7f2-4ba5-9c36-e9d5db69d340 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:56.311198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-268b8a5d-d7f2-4ba5-9c36-e9d5db69d340 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:56.311802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-268b8a5d-d7f2-4ba5-9c36-e9d5db69d340 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:56.312570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-268b8a5d-d7f2-4ba5-9c36-e9d5db69d340 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:56.545272 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-08d3fd99-a1b0-4985-843d-cc7f19275346 tempest-SecurityGroupsTestAdminJSON-1600475331 tempest-SecurityGroupsTestAdminJSON-1600475331-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 220 microversion: 2.1 time: 1.994910 Sep 27 09:58:56.546572 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 377/807] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 09:58:54 2022] POST /compute/v2.1/os-security-groups => generated 220 bytes in 1997 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:56.557054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75e2fe08-bea5-49e3-9f85-27daeea65e3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:56.560150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-75e2fe08-bea5-49e3-9f85-27daeea65e3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:56.608309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75e2fe08-bea5-49e3-9f85-27daeea65e3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:56.615181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75e2fe08-bea5-49e3-9f85-27daeea65e3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:56.620476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75e2fe08-bea5-49e3-9f85-27daeea65e3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:56.623655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75e2fe08-bea5-49e3-9f85-27daeea65e3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:56.624546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75e2fe08-bea5-49e3-9f85-27daeea65e3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:56.628325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75e2fe08-bea5-49e3-9f85-27daeea65e3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:56.656327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-268b8a5d-d7f2-4ba5-9c36-e9d5db69d340 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:56.670602 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-268b8a5d-d7f2-4ba5-9c36-e9d5db69d340 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1316 microversion: 2.1 time: 0.376545 Sep 27 09:58:56.678263 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 431/808] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:56 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1316 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:56.703977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e4941520-b346-4e3f-b5c1-3eddb588cc1d tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:56.705732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e4941520-b346-4e3f-b5c1-3eddb588cc1d tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:56.731831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4941520-b346-4e3f-b5c1-3eddb588cc1d tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:56.735358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4941520-b346-4e3f-b5c1-3eddb588cc1d tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:56.735958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4941520-b346-4e3f-b5c1-3eddb588cc1d tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:56.741269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4941520-b346-4e3f-b5c1-3eddb588cc1d tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:56.743716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4941520-b346-4e3f-b5c1-3eddb588cc1d tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:56.745720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4941520-b346-4e3f-b5c1-3eddb588cc1d tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:56.948367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-e4941520-b346-4e3f-b5c1-3eddb588cc1d tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:56.998526 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e4941520-b346-4e3f-b5c1-3eddb588cc1d tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.316859 Sep 27 09:58:56.999130 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 432/809] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:56 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:57.046147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f980c786-0d74-4163-b0e4-a7abdfc7baf2 tempest-SecurityGroupsTestAdminJSON-1600475331 tempest-SecurityGroupsTestAdminJSON-1600475331-project] Action: 'create', calling method: .version_select at 0x7f7f63aa3f70>, body: {"security_group": {"name": "tempest-securitygroup-743374945", "description": "tempest-description-1984955534"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:57.081268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-75e2fe08-bea5-49e3-9f85-27daeea65e3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:57.090942 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-75e2fe08-bea5-49e3-9f85-27daeea65e3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1472 microversion: 2.1 time: 0.543105 Sep 27 09:58:57.091537 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 378/810] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:56 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1472 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:57.129382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9c90161c-8696-4a15-be01-c2800d2a6542 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:57.130527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9c90161c-8696-4a15-be01-c2800d2a6542 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:57.164959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c90161c-8696-4a15-be01-c2800d2a6542 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:57.168599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c90161c-8696-4a15-be01-c2800d2a6542 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:57.171503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c90161c-8696-4a15-be01-c2800d2a6542 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:57.176552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c90161c-8696-4a15-be01-c2800d2a6542 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:57.178603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c90161c-8696-4a15-be01-c2800d2a6542 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:57.189243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c90161c-8696-4a15-be01-c2800d2a6542 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:57.478055 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f980c786-0d74-4163-b0e4-a7abdfc7baf2 tempest-SecurityGroupsTestAdminJSON-1600475331 tempest-SecurityGroupsTestAdminJSON-1600475331-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 220 microversion: 2.1 time: 0.452758 Sep 27 09:58:57.478538 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 433/811] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 09:58:56 2022] POST /compute/v2.1/os-security-groups => generated 220 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:57.788262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9c90161c-8696-4a15-be01-c2800d2a6542 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:57.827076 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9c90161c-8696-4a15-be01-c2800d2a6542 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 0.718104 Sep 27 09:58:57.828963 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 379/812] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:57 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:57.842540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1a152d8a-0caf-4933-88ef-fb581eee4fc7 req-dc24137c-df38-406d-9a12-67279eef87d2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "984079f7-c95b-4c38-8ea7-5a16c0f50c0a", "tag": "c39020e6-7f01-470b-a8f7-f9d386f10248"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:57.863919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1a152d8a-0caf-4933-88ef-fb581eee4fc7 req-dc24137c-df38-406d-9a12-67279eef87d2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:57.866386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1a152d8a-0caf-4933-88ef-fb581eee4fc7 req-dc24137c-df38-406d-9a12-67279eef87d2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:57.868041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1a152d8a-0caf-4933-88ef-fb581eee4fc7 req-dc24137c-df38-406d-9a12-67279eef87d2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:57.893282 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1a152d8a-0caf-4933-88ef-fb581eee4fc7 req-dc24137c-df38-406d-9a12-67279eef87d2 service nova] Creating event network-changed:c39020e6-7f01-470b-a8f7-f9d386f10248 for instance 984079f7-c95b-4c38-8ea7-5a16c0f50c0a on nch2-master-1 Sep 27 09:58:57.907793 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1a152d8a-0caf-4933-88ef-fb581eee4fc7 req-dc24137c-df38-406d-9a12-67279eef87d2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.076800 Sep 27 09:58:57.913439 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 380/813] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:58:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:57.921866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-85f2876e-0bc6-4af9-aab3-896a1b6be011 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:57.925978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-85f2876e-0bc6-4af9-aab3-896a1b6be011 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:57.936810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85f2876e-0bc6-4af9-aab3-896a1b6be011 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:57.937301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85f2876e-0bc6-4af9-aab3-896a1b6be011 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:57.937736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85f2876e-0bc6-4af9-aab3-896a1b6be011 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:57.938443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85f2876e-0bc6-4af9-aab3-896a1b6be011 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:57.938894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85f2876e-0bc6-4af9-aab3-896a1b6be011 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:57.939324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85f2876e-0bc6-4af9-aab3-896a1b6be011 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:57.975043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-107fbd40-558a-49bd-90aa-177b17b029bf tempest-SecurityGroupsTestAdminJSON-2109519667 tempest-SecurityGroupsTestAdminJSON-2109519667-project-admin] Action: 'create', calling method: .version_select at 0x7f7f639f70d0>, body: {"security_group": {"name": "tempest-securitygroup-1816985194", "description": "tempest-description-607031672"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:58.402479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-85f2876e-0bc6-4af9-aab3-896a1b6be011 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:58.413976 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-85f2876e-0bc6-4af9-aab3-896a1b6be011 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1316 microversion: 2.1 time: 0.498251 Sep 27 09:58:58.428839 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 381/814] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:57 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1316 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:58.450858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1900c108-4423-45a0-84c4-839edec8b696 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:58.452630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1900c108-4423-45a0-84c4-839edec8b696 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:58.474760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1900c108-4423-45a0-84c4-839edec8b696 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:58.475693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1900c108-4423-45a0-84c4-839edec8b696 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:58.476334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1900c108-4423-45a0-84c4-839edec8b696 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:58.478482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1900c108-4423-45a0-84c4-839edec8b696 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:58.479142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1900c108-4423-45a0-84c4-839edec8b696 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:58.480164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1900c108-4423-45a0-84c4-839edec8b696 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:58.596776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-1900c108-4423-45a0-84c4-839edec8b696 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:58:58.649520 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1900c108-4423-45a0-84c4-839edec8b696 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.203627 Sep 27 09:58:58.650156 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 382/815] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:58 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:58:58.667417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8dec1f3-a96c-4d65-a517-80280afff1a1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:58.668046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f8dec1f3-a96c-4d65-a517-80280afff1a1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:58.677519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8dec1f3-a96c-4d65-a517-80280afff1a1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:58.678232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8dec1f3-a96c-4d65-a517-80280afff1a1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:58.678972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8dec1f3-a96c-4d65-a517-80280afff1a1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:58.680286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8dec1f3-a96c-4d65-a517-80280afff1a1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:58.680937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8dec1f3-a96c-4d65-a517-80280afff1a1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:58.681540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8dec1f3-a96c-4d65-a517-80280afff1a1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:58.695506 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-107fbd40-558a-49bd-90aa-177b17b029bf tempest-SecurityGroupsTestAdminJSON-2109519667 tempest-SecurityGroupsTestAdminJSON-2109519667-project-admin] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 220 microversion: 2.1 time: 1.207468 Sep 27 09:58:58.704131 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 434/816] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 09:58:57 2022] POST /compute/v2.1/os-security-groups => generated 220 bytes in 1217 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:58.709290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef0c5646-a0b3-437d-89e1-312da9099066 tempest-SecurityGroupsTestAdminJSON-2109519667 tempest-SecurityGroupsTestAdminJSON-2109519667-project-admin] Action: 'create', calling method: .version_select at 0x7f7f63974d30>, body: {"security_group": {"name": "tempest-securitygroup-1274804151", "description": "tempest-description-1956653496"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:58:58.947793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f8dec1f3-a96c-4d65-a517-80280afff1a1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:58.957324 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f8dec1f3-a96c-4d65-a517-80280afff1a1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1472 microversion: 2.1 time: 0.306067 Sep 27 09:58:58.961186 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 383/817] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:58 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1472 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:58.966324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-62ea36cc-d1cb-49f3-ad58-c7055cb720a9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:58.974861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-62ea36cc-d1cb-49f3-ad58-c7055cb720a9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:58.984280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=88,nova_api:DELETE=4,nova_api:INSERT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:58.991160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62ea36cc-d1cb-49f3-ad58-c7055cb720a9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:58.991762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62ea36cc-d1cb-49f3-ad58-c7055cb720a9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:58.993175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62ea36cc-d1cb-49f3-ad58-c7055cb720a9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:58.996721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62ea36cc-d1cb-49f3-ad58-c7055cb720a9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:58.997536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62ea36cc-d1cb-49f3-ad58-c7055cb720a9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:58.998180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62ea36cc-d1cb-49f3-ad58-c7055cb720a9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:59.233500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-62ea36cc-d1cb-49f3-ad58-c7055cb720a9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:59.245200 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-62ea36cc-d1cb-49f3-ad58-c7055cb720a9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 0.282276 Sep 27 09:58:59.261781 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 384/818] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:58 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:58:59.376154 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef0c5646-a0b3-437d-89e1-312da9099066 tempest-SecurityGroupsTestAdminJSON-2109519667 tempest-SecurityGroupsTestAdminJSON-2109519667-project-admin] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 221 microversion: 2.1 time: 0.670553 Sep 27 09:58:59.386284 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 435/819] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 09:58:58 2022] POST /compute/v2.1/os-security-groups => generated 221 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:58:59.391100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2c7da7d5-89e0-4bd7-ae61-14de53460c58 tempest-SecurityGroupsTestAdminJSON-2109519667 tempest-SecurityGroupsTestAdminJSON-2109519667-project-admin] Calling method '.version_select at 0x7f7f63aefb80>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:59.441569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-96732565-c1e4-4d0a-81e6-abda08219ccc tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:59.442189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-96732565-c1e4-4d0a-81e6-abda08219ccc tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:59.455654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96732565-c1e4-4d0a-81e6-abda08219ccc tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:59.457313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96732565-c1e4-4d0a-81e6-abda08219ccc tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:59.457794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96732565-c1e4-4d0a-81e6-abda08219ccc tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:59.459142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96732565-c1e4-4d0a-81e6-abda08219ccc tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:59.462297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96732565-c1e4-4d0a-81e6-abda08219ccc tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:59.462758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96732565-c1e4-4d0a-81e6-abda08219ccc tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:59.819181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:59.886474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=14 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:58:59.905161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-96732565-c1e4-4d0a-81e6-abda08219ccc tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:58:59.918555 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-96732565-c1e4-4d0a-81e6-abda08219ccc tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1316 microversion: 2.1 time: 0.481657 Sep 27 09:58:59.920412 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 436/820] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:58:59 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1316 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:58:59.927779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1a0c8606-2ad9-419f-adef-cd646a307c4f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:58:59.928873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1a0c8606-2ad9-419f-adef-cd646a307c4f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:58:59.941810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0c8606-2ad9-419f-adef-cd646a307c4f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:59.942597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0c8606-2ad9-419f-adef-cd646a307c4f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:59.944453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0c8606-2ad9-419f-adef-cd646a307c4f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:58:59.945944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0c8606-2ad9-419f-adef-cd646a307c4f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:58:59.946608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0c8606-2ad9-419f-adef-cd646a307c4f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:58:59.947251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0c8606-2ad9-419f-adef-cd646a307c4f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:00.113644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-1a0c8606-2ad9-419f-adef-cd646a307c4f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:59:00.118305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:00.141942 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1a0c8606-2ad9-419f-adef-cd646a307c4f tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1927 microversion: 2.75 time: 0.217931 Sep 27 09:59:00.142944 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 437/821] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:58:59 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1927 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:00.148588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-83b443c1-a434-429f-b4ce-1ad2bc9ab1ca req-14f3cde1-3d02-41cc-9bd0-c854e6140ddb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "832b56e9-4ed3-4b6c-a201-d5af937fd9e7", "tag": "86b49b5d-1243-49df-bcec-4cf58543d8d6"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:00.159795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-83b443c1-a434-429f-b4ce-1ad2bc9ab1ca req-14f3cde1-3d02-41cc-9bd0-c854e6140ddb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:00.160630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-83b443c1-a434-429f-b4ce-1ad2bc9ab1ca req-14f3cde1-3d02-41cc-9bd0-c854e6140ddb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:00.161261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-83b443c1-a434-429f-b4ce-1ad2bc9ab1ca req-14f3cde1-3d02-41cc-9bd0-c854e6140ddb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:00.176404 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-83b443c1-a434-429f-b4ce-1ad2bc9ab1ca req-14f3cde1-3d02-41cc-9bd0-c854e6140ddb service nova] Creating event network-changed:86b49b5d-1243-49df-bcec-4cf58543d8d6 for instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 on nch2-master-1 Sep 27 09:59:00.184405 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-83b443c1-a434-429f-b4ce-1ad2bc9ab1ca req-14f3cde1-3d02-41cc-9bd0-c854e6140ddb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038603 Sep 27 09:59:00.184888 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 438/822] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:00.194865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4013cb16-fe11-43ec-adcd-fa4cadba3f3a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:00.195968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4013cb16-fe11-43ec-adcd-fa4cadba3f3a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:00.205008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4013cb16-fe11-43ec-adcd-fa4cadba3f3a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:00.205500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4013cb16-fe11-43ec-adcd-fa4cadba3f3a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:00.206152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4013cb16-fe11-43ec-adcd-fa4cadba3f3a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:00.207094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4013cb16-fe11-43ec-adcd-fa4cadba3f3a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:00.207737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4013cb16-fe11-43ec-adcd-fa4cadba3f3a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:00.208610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4013cb16-fe11-43ec-adcd-fa4cadba3f3a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:00.317452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:00.600221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4013cb16-fe11-43ec-adcd-fa4cadba3f3a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:00.617158 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4013cb16-fe11-43ec-adcd-fa4cadba3f3a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1472 microversion: 2.1 time: 0.429804 Sep 27 09:59:00.621230 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 439/823] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:00 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1472 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:00.627657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa646ba7-ff5f-4826-bf13-8a8e27dffa1f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:00.629012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fa646ba7-ff5f-4826-bf13-8a8e27dffa1f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:00.645511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa646ba7-ff5f-4826-bf13-8a8e27dffa1f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:00.646403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa646ba7-ff5f-4826-bf13-8a8e27dffa1f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:00.647103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa646ba7-ff5f-4826-bf13-8a8e27dffa1f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:00.648132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa646ba7-ff5f-4826-bf13-8a8e27dffa1f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:00.648953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa646ba7-ff5f-4826-bf13-8a8e27dffa1f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:00.650348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa646ba7-ff5f-4826-bf13-8a8e27dffa1f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:00.942522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fa646ba7-ff5f-4826-bf13-8a8e27dffa1f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:00.956099 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fa646ba7-ff5f-4826-bf13-8a8e27dffa1f tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 0.330635 Sep 27 09:59:00.964462 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 440/824] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:00 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:00.975200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e92d7f43-1a7d-48bb-b6cb-246698b9373d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:00.976229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e92d7f43-1a7d-48bb-b6cb-246698b9373d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:00.993417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e92d7f43-1a7d-48bb-b6cb-246698b9373d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:00.993932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e92d7f43-1a7d-48bb-b6cb-246698b9373d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:00.994379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e92d7f43-1a7d-48bb-b6cb-246698b9373d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:00.995480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e92d7f43-1a7d-48bb-b6cb-246698b9373d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:00.996136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e92d7f43-1a7d-48bb-b6cb-246698b9373d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:00.996888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e92d7f43-1a7d-48bb-b6cb-246698b9373d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:01.608262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e92d7f43-1a7d-48bb-b6cb-246698b9373d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:01.621797 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e92d7f43-1a7d-48bb-b6cb-246698b9373d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1316 microversion: 2.1 time: 0.652169 Sep 27 09:59:01.622710 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 441/825] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:00 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1316 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:01.643611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e198f41f-4a44-41f8-8738-9a37f87839e7 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:01.646112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e198f41f-4a44-41f8-8738-9a37f87839e7 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:01.658903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e198f41f-4a44-41f8-8738-9a37f87839e7 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:01.659394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e198f41f-4a44-41f8-8738-9a37f87839e7 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:01.659916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e198f41f-4a44-41f8-8738-9a37f87839e7 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:01.661381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e198f41f-4a44-41f8-8738-9a37f87839e7 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:01.661608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e198f41f-4a44-41f8-8738-9a37f87839e7 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:01.662184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e198f41f-4a44-41f8-8738-9a37f87839e7 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:01.785058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-e198f41f-4a44-41f8-8738-9a37f87839e7 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:59:01.804954 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e198f41f-4a44-41f8-8738-9a37f87839e7 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1933 microversion: 2.75 time: 0.172528 Sep 27 09:59:01.805607 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 442/826] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:01 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1933 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:01.811227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-658cf9ac-137a-4d74-803e-8d8891a9a794 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:01.812100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-658cf9ac-137a-4d74-803e-8d8891a9a794 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:01.821604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658cf9ac-137a-4d74-803e-8d8891a9a794 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:01.822468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658cf9ac-137a-4d74-803e-8d8891a9a794 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:01.822996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658cf9ac-137a-4d74-803e-8d8891a9a794 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:01.823967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658cf9ac-137a-4d74-803e-8d8891a9a794 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:01.824481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658cf9ac-137a-4d74-803e-8d8891a9a794 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:01.825019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658cf9ac-137a-4d74-803e-8d8891a9a794 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:02.186100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-658cf9ac-137a-4d74-803e-8d8891a9a794 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:02.203509 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-658cf9ac-137a-4d74-803e-8d8891a9a794 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1472 microversion: 2.1 time: 0.393439 Sep 27 09:59:02.214389 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 443/827] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:01 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1472 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:02.235608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ab41bcbb-bdf8-4211-9176-fb51ceb88b26 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:02.236241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ab41bcbb-bdf8-4211-9176-fb51ceb88b26 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:02.238263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:DELETE=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:02.262549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab41bcbb-bdf8-4211-9176-fb51ceb88b26 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:02.263049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab41bcbb-bdf8-4211-9176-fb51ceb88b26 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:02.263714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab41bcbb-bdf8-4211-9176-fb51ceb88b26 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:02.264440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab41bcbb-bdf8-4211-9176-fb51ceb88b26 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:02.264930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab41bcbb-bdf8-4211-9176-fb51ceb88b26 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:02.265355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab41bcbb-bdf8-4211-9176-fb51ceb88b26 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:02.872884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ab41bcbb-bdf8-4211-9176-fb51ceb88b26 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:02.883368 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ab41bcbb-bdf8-4211-9176-fb51ceb88b26 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 0.667701 Sep 27 09:59:02.884109 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 444/828] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:02 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:02.905815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ab1a6076-9ca6-4ada-b4f1-51a0914191c3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:02.917826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ab1a6076-9ca6-4ada-b4f1-51a0914191c3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:02.925286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab1a6076-9ca6-4ada-b4f1-51a0914191c3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:02.926024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab1a6076-9ca6-4ada-b4f1-51a0914191c3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:02.926612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab1a6076-9ca6-4ada-b4f1-51a0914191c3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:02.927488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab1a6076-9ca6-4ada-b4f1-51a0914191c3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:02.928131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab1a6076-9ca6-4ada-b4f1-51a0914191c3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:02.928735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab1a6076-9ca6-4ada-b4f1-51a0914191c3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:03.217568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ab1a6076-9ca6-4ada-b4f1-51a0914191c3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:03.229665 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ab1a6076-9ca6-4ada-b4f1-51a0914191c3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1316 microversion: 2.1 time: 0.327406 Sep 27 09:59:03.230888 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 445/829] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:02 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1316 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:03.236307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f6325c48-cf4c-44f9-830f-9245bea2f328 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:03.237951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f6325c48-cf4c-44f9-830f-9245bea2f328 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:03.246209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6325c48-cf4c-44f9-830f-9245bea2f328 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:03.249812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6325c48-cf4c-44f9-830f-9245bea2f328 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:03.250639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6325c48-cf4c-44f9-830f-9245bea2f328 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:03.251731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6325c48-cf4c-44f9-830f-9245bea2f328 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:03.253583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6325c48-cf4c-44f9-830f-9245bea2f328 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:03.255258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6325c48-cf4c-44f9-830f-9245bea2f328 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:03.477886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-f6325c48-cf4c-44f9-830f-9245bea2f328 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:59:03.504337 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f6325c48-cf4c-44f9-830f-9245bea2f328 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1933 microversion: 2.75 time: 0.271762 Sep 27 09:59:03.505274 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 446/830] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:03 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1933 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:03.511145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f5bf5192-e6a3-4d42-bba6-6f2bdfa2f496 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:03.512002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f5bf5192-e6a3-4d42-bba6-6f2bdfa2f496 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:03.525688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5bf5192-e6a3-4d42-bba6-6f2bdfa2f496 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:03.526485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5bf5192-e6a3-4d42-bba6-6f2bdfa2f496 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:03.527110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5bf5192-e6a3-4d42-bba6-6f2bdfa2f496 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:03.528567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5bf5192-e6a3-4d42-bba6-6f2bdfa2f496 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:03.529737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5bf5192-e6a3-4d42-bba6-6f2bdfa2f496 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:03.530488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5bf5192-e6a3-4d42-bba6-6f2bdfa2f496 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:03.751614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f5bf5192-e6a3-4d42-bba6-6f2bdfa2f496 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:03.772470 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f5bf5192-e6a3-4d42-bba6-6f2bdfa2f496 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1477 microversion: 2.1 time: 0.265678 Sep 27 09:59:03.778208 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 447/831] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:03 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1477 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:03.916005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e3ea67e-d8da-4054-bf6f-47165f479ebb tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:03.917784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2e3ea67e-d8da-4054-bf6f-47165f479ebb tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:03.928402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e3ea67e-d8da-4054-bf6f-47165f479ebb tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:03.930970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e3ea67e-d8da-4054-bf6f-47165f479ebb tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:03.931874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e3ea67e-d8da-4054-bf6f-47165f479ebb tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:03.932985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e3ea67e-d8da-4054-bf6f-47165f479ebb tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:03.933600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e3ea67e-d8da-4054-bf6f-47165f479ebb tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:03.934900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e3ea67e-d8da-4054-bf6f-47165f479ebb tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:04.516876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2e3ea67e-d8da-4054-bf6f-47165f479ebb tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:04.526608 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2e3ea67e-d8da-4054-bf6f-47165f479ebb tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 0.618525 Sep 27 09:59:04.541937 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 448/832] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:03 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:04.551502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33006e02-bddc-4d1c-8ccf-4d503fe6940d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:04.552127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33006e02-bddc-4d1c-8ccf-4d503fe6940d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:04.564675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33006e02-bddc-4d1c-8ccf-4d503fe6940d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:04.565709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33006e02-bddc-4d1c-8ccf-4d503fe6940d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:04.566167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33006e02-bddc-4d1c-8ccf-4d503fe6940d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:04.566901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33006e02-bddc-4d1c-8ccf-4d503fe6940d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:04.567366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33006e02-bddc-4d1c-8ccf-4d503fe6940d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:04.567799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33006e02-bddc-4d1c-8ccf-4d503fe6940d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:04.882730 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2c7da7d5-89e0-4bd7-ae61-14de53460c58 tempest-SecurityGroupsTestAdminJSON-2109519667 tempest-SecurityGroupsTestAdminJSON-2109519667-project-admin] 10.222.0.26 "GET /compute/v2.1/os-security-groups?all_tenants=true" status: 200 len: 9683 microversion: 2.1 time: 5.495012 Sep 27 09:59:04.886008 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 385/833] 10.222.0.26 () {58 vars in 1191 bytes} [Tue Sep 27 09:58:59 2022] GET /compute/v2.1/os-security-groups?all_tenants=true => generated 9683 bytes in 5499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:04.892423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33006e02-bddc-4d1c-8ccf-4d503fe6940d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:04.894070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f775de27-4c06-4826-b2a1-cdd51a94f808 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:04.895895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f775de27-4c06-4826-b2a1-cdd51a94f808 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:04.901766 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33006e02-bddc-4d1c-8ccf-4d503fe6940d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1316 microversion: 2.1 time: 0.358436 Sep 27 09:59:04.905003 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 449/834] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:04 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1316 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:04.906568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f775de27-4c06-4826-b2a1-cdd51a94f808 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:04.907333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f775de27-4c06-4826-b2a1-cdd51a94f808 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:04.908075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f775de27-4c06-4826-b2a1-cdd51a94f808 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:04.909104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f775de27-4c06-4826-b2a1-cdd51a94f808 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:04.910394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f775de27-4c06-4826-b2a1-cdd51a94f808 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:04.911061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f775de27-4c06-4826-b2a1-cdd51a94f808 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:04.924095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-80635fd9-adce-437d-ad09-e7e212fceed0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:04.931517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-80635fd9-adce-437d-ad09-e7e212fceed0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:04.946942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80635fd9-adce-437d-ad09-e7e212fceed0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:04.949172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80635fd9-adce-437d-ad09-e7e212fceed0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:04.949950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80635fd9-adce-437d-ad09-e7e212fceed0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:04.952811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80635fd9-adce-437d-ad09-e7e212fceed0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:04.953516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80635fd9-adce-437d-ad09-e7e212fceed0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:04.957043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80635fd9-adce-437d-ad09-e7e212fceed0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:05.136197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-f775de27-4c06-4826-b2a1-cdd51a94f808 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] Lazy-loading 'services' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:59:05.198126 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f775de27-4c06-4826-b2a1-cdd51a94f808 tempest-ServersAdmin275Test-808735172 tempest-ServersAdmin275Test-808735172-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1918 microversion: 2.75 time: 0.308705 Sep 27 09:59:05.198741 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 386/835] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:04 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1918 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:05.214469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-766e29d3-e702-467c-ba16-29e3189498a2 tempest-SecurityGroupsTestAdminJSON-1600475331 tempest-SecurityGroupsTestAdminJSON-1600475331-project] Calling method '.version_select at 0x7f7f63a83280>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:05.238767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-80635fd9-adce-437d-ad09-e7e212fceed0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:05.247966 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-80635fd9-adce-437d-ad09-e7e212fceed0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1477 microversion: 2.1 time: 0.327692 Sep 27 09:59:05.254299 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 450/836] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:04 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1477 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:05.272291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7c59800b-7a4d-4420-be08-7b18d159e78e tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:05.273022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7c59800b-7a4d-4420-be08-7b18d159e78e tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:05.290606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c59800b-7a4d-4420-be08-7b18d159e78e tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:05.291107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c59800b-7a4d-4420-be08-7b18d159e78e tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:05.291548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c59800b-7a4d-4420-be08-7b18d159e78e tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:05.292261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c59800b-7a4d-4420-be08-7b18d159e78e tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:05.295872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c59800b-7a4d-4420-be08-7b18d159e78e tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:05.296419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c59800b-7a4d-4420-be08-7b18d159e78e tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:05.363768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7c59800b-7a4d-4420-be08-7b18d159e78e tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:59:05.579802 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7c59800b-7a4d-4420-be08-7b18d159e78e tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 204 len: 0 microversion: 2.75 time: 0.322746 Sep 27 09:59:05.580820 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 451/837] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 09:59:05 2022] DELETE /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 0 bytes in 325 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 09:59:05.593779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3461eccf-9419-480d-b15a-ce39fc306022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:05.594484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3461eccf-9419-480d-b15a-ce39fc306022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:05.618061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3461eccf-9419-480d-b15a-ce39fc306022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:05.618597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3461eccf-9419-480d-b15a-ce39fc306022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:05.619041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3461eccf-9419-480d-b15a-ce39fc306022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:05.619784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3461eccf-9419-480d-b15a-ce39fc306022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:05.620242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3461eccf-9419-480d-b15a-ce39fc306022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:05.620690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3461eccf-9419-480d-b15a-ce39fc306022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:06.442355 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-766e29d3-e702-467c-ba16-29e3189498a2 tempest-SecurityGroupsTestAdminJSON-1600475331 tempest-SecurityGroupsTestAdminJSON-1600475331-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups?all_tenants=true" status: 200 len: 1111 microversion: 2.1 time: 1.238270 Sep 27 09:59:06.442952 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 387/838] 10.222.0.26 () {58 vars in 1191 bytes} [Tue Sep 27 09:59:05 2022] GET /compute/v2.1/os-security-groups?all_tenants=true => generated 1111 bytes in 1240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:06.456243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:06.456911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:06.475157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:06.475829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:06.484896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:06.485780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:06.486386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:06.487901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:06.533968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:06.576720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3461eccf-9419-480d-b15a-ce39fc306022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:06.584219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:06.590885 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3461eccf-9419-480d-b15a-ce39fc306022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 1.008650 Sep 27 09:59:06.595333 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 452/839] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:05 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 1014 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:06.609035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0ddaadb-5ce3-44a1-8a2a-6f7a5bca59ca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:06.620342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b0ddaadb-5ce3-44a1-8a2a-6f7a5bca59ca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:06.636539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ddaadb-5ce3-44a1-8a2a-6f7a5bca59ca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:06.636965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ddaadb-5ce3-44a1-8a2a-6f7a5bca59ca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:06.639580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ddaadb-5ce3-44a1-8a2a-6f7a5bca59ca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:06.640316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ddaadb-5ce3-44a1-8a2a-6f7a5bca59ca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:06.640828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ddaadb-5ce3-44a1-8a2a-6f7a5bca59ca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:06.641255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ddaadb-5ce3-44a1-8a2a-6f7a5bca59ca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:06.653239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:06.672716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:06.682392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:06.693252 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5b353ee2-f3b8-4689-b64c-df21bec6a9f6 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1443 microversion: 2.75 time: 0.248949 Sep 27 09:59:06.694108 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 388/840] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:06 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1443 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:06.708825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f28be454-4413-426c-be29-b868a837d5b6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:06.710862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f28be454-4413-426c-be29-b868a837d5b6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:06.727321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f28be454-4413-426c-be29-b868a837d5b6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:06.729412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f28be454-4413-426c-be29-b868a837d5b6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:06.730632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f28be454-4413-426c-be29-b868a837d5b6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:06.732824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f28be454-4413-426c-be29-b868a837d5b6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:06.734278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f28be454-4413-426c-be29-b868a837d5b6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:06.734828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f28be454-4413-426c-be29-b868a837d5b6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:06.914169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b0ddaadb-5ce3-44a1-8a2a-6f7a5bca59ca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:06.934590 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b0ddaadb-5ce3-44a1-8a2a-6f7a5bca59ca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1316 microversion: 2.1 time: 0.337722 Sep 27 09:59:06.935893 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 453/841] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:06 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1316 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:06.945682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-dc703412-2f83-4b8b-a0d5-486ff3284906 req-40808866-61d8-4615-ba09-2d00e979fbcc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "984079f7-c95b-4c38-8ea7-5a16c0f50c0a", "name": "network-vif-plugged", "status": "completed", "tag": "c39020e6-7f01-470b-a8f7-f9d386f10248"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:06.971347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-dc703412-2f83-4b8b-a0d5-486ff3284906 req-40808866-61d8-4615-ba09-2d00e979fbcc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:06.977190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-dc703412-2f83-4b8b-a0d5-486ff3284906 req-40808866-61d8-4615-ba09-2d00e979fbcc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:06.977739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-dc703412-2f83-4b8b-a0d5-486ff3284906 req-40808866-61d8-4615-ba09-2d00e979fbcc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:07.002504 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-dc703412-2f83-4b8b-a0d5-486ff3284906 req-40808866-61d8-4615-ba09-2d00e979fbcc service nova] Creating event network-vif-plugged:c39020e6-7f01-470b-a8f7-f9d386f10248 for instance 984079f7-c95b-4c38-8ea7-5a16c0f50c0a on nch2-master-1 Sep 27 09:59:07.016198 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-dc703412-2f83-4b8b-a0d5-486ff3284906 req-40808866-61d8-4615-ba09-2d00e979fbcc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.078315 Sep 27 09:59:07.016933 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 454/842] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:07.033239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eee117fa-abc5-4c49-a44e-576338a78363 tempest-SecurityGroupsTestAdminJSON-2109519667 tempest-SecurityGroupsTestAdminJSON-2109519667-project-admin] Calling method '.version_select at 0x7f7f63aa3a60>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:07.545162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f28be454-4413-426c-be29-b868a837d5b6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:07.585510 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f28be454-4413-426c-be29-b868a837d5b6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1477 microversion: 2.1 time: 0.889912 Sep 27 09:59:07.612276 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 389/843] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:06 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1477 bytes in 915 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:07.621383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-13ff7cea-c826-42ec-ac43-95e68f51c6c9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:07.623506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-13ff7cea-c826-42ec-ac43-95e68f51c6c9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:07.626183 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eee117fa-abc5-4c49-a44e-576338a78363 tempest-SecurityGroupsTestAdminJSON-2109519667 tempest-SecurityGroupsTestAdminJSON-2109519667-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/0b4da8c6-e61b-432c-9f9a-93c834012e0e" status: 202 len: 0 microversion: 2.1 time: 0.604858 Sep 27 09:59:07.634358 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 455/844] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 09:59:07 2022] DELETE /compute/v2.1/os-security-groups/0b4da8c6-e61b-432c-9f9a-93c834012e0e => generated 0 bytes in 617 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:59:07.643488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13ff7cea-c826-42ec-ac43-95e68f51c6c9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:07.646216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13ff7cea-c826-42ec-ac43-95e68f51c6c9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:07.650314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13ff7cea-c826-42ec-ac43-95e68f51c6c9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:07.651133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13ff7cea-c826-42ec-ac43-95e68f51c6c9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:07.651635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13ff7cea-c826-42ec-ac43-95e68f51c6c9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:07.652050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13ff7cea-c826-42ec-ac43-95e68f51c6c9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:07.659873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-64db337d-47d5-43e4-b2a5-6f0a99650a00 tempest-SecurityGroupsTestAdminJSON-2109519667 tempest-SecurityGroupsTestAdminJSON-2109519667-project-admin] Calling method '.version_select at 0x7f7f63974b80>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:08.347376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-13ff7cea-c826-42ec-ac43-95e68f51c6c9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:08.364638 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-13ff7cea-c826-42ec-ac43-95e68f51c6c9 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 0.748377 Sep 27 09:59:08.380759 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 390/845] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:07 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:08.390875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:08.396406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:08.413861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:08.414841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:08.415520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:08.417641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:08.418439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:08.420018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:08.523246 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-64db337d-47d5-43e4-b2a5-6f0a99650a00 tempest-SecurityGroupsTestAdminJSON-2109519667 tempest-SecurityGroupsTestAdminJSON-2109519667-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/cba2cbac-d545-49f7-91c0-69d6601967ef" status: 202 len: 0 microversion: 2.1 time: 0.886397 Sep 27 09:59:08.524474 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 456/846] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 09:59:07 2022] DELETE /compute/v2.1/os-security-groups/cba2cbac-d545-49f7-91c0-69d6601967ef => generated 0 bytes in 888 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:59:08.531687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0438ee99-cea0-4f1b-9a83-e7ae631dee71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:08.532287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:08.532818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:08.533264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:08.533729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lazy-loading 'fault' on Instance uuid a425429f-5f19-433a-b6f4-0ba745d502bb {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:59:08.533954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0438ee99-cea0-4f1b-9a83-e7ae631dee71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:08.555458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0438ee99-cea0-4f1b-9a83-e7ae631dee71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:08.556278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0438ee99-cea0-4f1b-9a83-e7ae631dee71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:08.560158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0438ee99-cea0-4f1b-9a83-e7ae631dee71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:08.562581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0438ee99-cea0-4f1b-9a83-e7ae631dee71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:08.563391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0438ee99-cea0-4f1b-9a83-e7ae631dee71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:08.564135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0438ee99-cea0-4f1b-9a83-e7ae631dee71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:08.616187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:08.627534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:08.639105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'aa2474e7686c4a758ed93064edf47867', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e6987197b0a4c7d9b89d48402845f3f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:08.646455 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-21fa98f1-c123-448e-b569-e531073d5235 tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 200 len: 1448 microversion: 2.75 time: 0.264039 Sep 27 09:59:08.649095 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 391/847] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:08 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 1448 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:08.687426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bfb7ff58-7fdd-44bc-8d88-63d3a290a041 tempest-SecurityGroupsTestAdminJSON-1600475331 tempest-SecurityGroupsTestAdminJSON-1600475331-project] Calling method '.version_select at 0x7f7f639ae8b0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:08.874516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0438ee99-cea0-4f1b-9a83-e7ae631dee71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:08.883295 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0438ee99-cea0-4f1b-9a83-e7ae631dee71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1316 microversion: 2.1 time: 0.356678 Sep 27 09:59:08.895308 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 457/848] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:08 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1316 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:08.903349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-26b8a4ff-cba9-49e2-b3cb-fac1c4569759 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:08.909185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-26b8a4ff-cba9-49e2-b3cb-fac1c4569759 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:08.931491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26b8a4ff-cba9-49e2-b3cb-fac1c4569759 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:08.932964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26b8a4ff-cba9-49e2-b3cb-fac1c4569759 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:08.933570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26b8a4ff-cba9-49e2-b3cb-fac1c4569759 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:08.935234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26b8a4ff-cba9-49e2-b3cb-fac1c4569759 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:08.935900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26b8a4ff-cba9-49e2-b3cb-fac1c4569759 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:08.936478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26b8a4ff-cba9-49e2-b3cb-fac1c4569759 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:09.217103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-26b8a4ff-cba9-49e2-b3cb-fac1c4569759 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:09.229267 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-26b8a4ff-cba9-49e2-b3cb-fac1c4569759 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1477 microversion: 2.1 time: 0.330843 Sep 27 09:59:09.230516 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 458/849] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:08 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1477 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:09.392310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aa18541c-04a0-451c-bba8-6e537a7f9bee tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:09.393164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aa18541c-04a0-451c-bba8-6e537a7f9bee tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:09.406834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa18541c-04a0-451c-bba8-6e537a7f9bee tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:09.407962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa18541c-04a0-451c-bba8-6e537a7f9bee tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:09.417247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa18541c-04a0-451c-bba8-6e537a7f9bee tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:09.418288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa18541c-04a0-451c-bba8-6e537a7f9bee tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:09.418983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa18541c-04a0-451c-bba8-6e537a7f9bee tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:09.425728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa18541c-04a0-451c-bba8-6e537a7f9bee tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:09.767749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aa18541c-04a0-451c-bba8-6e537a7f9bee tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:09.783618 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aa18541c-04a0-451c-bba8-6e537a7f9bee tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 0.395599 Sep 27 09:59:09.787297 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 459/850] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:09 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:09.798344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:09.800695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] [instance: a425429f-5f19-433a-b6f4-0ba745d502bb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:09.821360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:09.827384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:09.827886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:09.828650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:09.832403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:09.834971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:09.902140 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] HTTP exception thrown: Instance a425429f-5f19-433a-b6f4-0ba745d502bb could not be found. Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Request method failure captured: Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb HTTP/1.1 Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMskWKhbcD8owb07J7SvOTETvUYHxeQVJUsQ--dyfG_K1YKDnNONIUyxgXm0z0C0fAAM-v30XbUnY5Y0qWiIgOOLsL8DqOvLwV9iZRljzls_qlMmBNQRiGiOspTrQ6mPoKWcVhpgGMsJDUiuvWPJzqNOcW497EsLhFt7V1w-YWH0PyC8 Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.75 Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 7e6987197b0a4c7d9b89d48402845f3f Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersAdmin275Test-2131918877 Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:59:09.914821 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e6987197b0a4c7d9b89d48402845f3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e6987197b0a4c7d9b89d48402845f3f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7e6987197b0a4c7d9b89d48402845f3f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7e6987197b0a4c7d9b89d48402845f3f"}]}] Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersAdmin275Test-2131918877 Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 7e6987197b0a4c7d9b89d48402845f3f Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersAdmin275Test-2131918877 Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersAdmin275Test-2131918877-project Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: aa2474e7686c4a758ed93064edf47867 Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersAdmin275Test-2131918877-project Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance a425429f-5f19-433a-b6f4-0ba745d502bb could not be found. Sep 27 09:59:09.916286 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a425429f-5f19-433a-b6f4-0ba745d502bb'} Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a425429f-5f19-433a-b6f4-0ba745d502bb could not be found. Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:59:09.917062 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a425429f-5f19-433a-b6f4-0ba745d502bb could not be found. Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a425429f-5f19-433a-b6f4-0ba745d502bb could not be found. Sep 27 09:59:09.919176 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:09.922349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] Returning 404 to user: Instance a425429f-5f19-433a-b6f4-0ba745d502bb could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:59:09.923400 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a0a042e5-b3f1-4fd4-9b32-51b0d5acd6bd tempest-ServersAdmin275Test-2131918877 tempest-ServersAdmin275Test-2131918877-project] 10.222.0.26 "GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb" status: 404 len: 111 microversion: 2.75 time: 0.134846 Sep 27 09:59:09.933734 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 460/851] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:09 2022] GET /compute/v2.1/servers/a425429f-5f19-433a-b6f4-0ba745d502bb => generated 111 bytes in 143 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 09:59:09.937959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5509faa0-1d0a-43d9-8635-27e9d20159b5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:09.938584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5509faa0-1d0a-43d9-8635-27e9d20159b5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:09.954590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5509faa0-1d0a-43d9-8635-27e9d20159b5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:09.955217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5509faa0-1d0a-43d9-8635-27e9d20159b5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:09.955651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5509faa0-1d0a-43d9-8635-27e9d20159b5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:09.956400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5509faa0-1d0a-43d9-8635-27e9d20159b5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:09.957147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5509faa0-1d0a-43d9-8635-27e9d20159b5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:09.957708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5509faa0-1d0a-43d9-8635-27e9d20159b5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:10.367266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5509faa0-1d0a-43d9-8635-27e9d20159b5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:10.382210 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5509faa0-1d0a-43d9-8635-27e9d20159b5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1316 microversion: 2.1 time: 0.448230 Sep 27 09:59:10.397820 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 461/852] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:09 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1316 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:10.406319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-36b4e7a4-f3b9-4b2d-90e7-0cbab66c9dd1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:10.407418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-36b4e7a4-f3b9-4b2d-90e7-0cbab66c9dd1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:10.414321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36b4e7a4-f3b9-4b2d-90e7-0cbab66c9dd1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:10.415855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36b4e7a4-f3b9-4b2d-90e7-0cbab66c9dd1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:10.416459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36b4e7a4-f3b9-4b2d-90e7-0cbab66c9dd1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:10.417625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36b4e7a4-f3b9-4b2d-90e7-0cbab66c9dd1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:10.418251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36b4e7a4-f3b9-4b2d-90e7-0cbab66c9dd1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:10.418864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36b4e7a4-f3b9-4b2d-90e7-0cbab66c9dd1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:10.443568 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bfb7ff58-7fdd-44bc-8d88-63d3a290a041 tempest-SecurityGroupsTestAdminJSON-1600475331 tempest-SecurityGroupsTestAdminJSON-1600475331-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/62d76c9c-18ea-4564-8944-e0c350fd1acb" status: 202 len: 0 microversion: 2.1 time: 1.780400 Sep 27 09:59:10.444758 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 392/853] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 09:59:08 2022] DELETE /compute/v2.1/os-security-groups/62d76c9c-18ea-4564-8944-e0c350fd1acb => generated 0 bytes in 1782 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 09:59:10.453917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-40e30ecd-006b-4c37-ab95-4b0c8cd2b1d8 tempest-SecurityGroupsTestAdminJSON-1600475331 tempest-SecurityGroupsTestAdminJSON-1600475331-project] Calling method '.version_select at 0x7f7f63963430>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:10.843872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-36b4e7a4-f3b9-4b2d-90e7-0cbab66c9dd1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:10.862516 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-36b4e7a4-f3b9-4b2d-90e7-0cbab66c9dd1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1477 microversion: 2.1 time: 0.462825 Sep 27 09:59:10.863499 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 462/854] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:10 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1477 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:10.889973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a41b1154-e3d8-498c-a8e4-9bb0551e64ef tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:10.890811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a41b1154-e3d8-498c-a8e4-9bb0551e64ef tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:10.900536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a41b1154-e3d8-498c-a8e4-9bb0551e64ef tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:10.913597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a41b1154-e3d8-498c-a8e4-9bb0551e64ef tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:10.915089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a41b1154-e3d8-498c-a8e4-9bb0551e64ef tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:10.916158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a41b1154-e3d8-498c-a8e4-9bb0551e64ef tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:10.916879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a41b1154-e3d8-498c-a8e4-9bb0551e64ef tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:10.917542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a41b1154-e3d8-498c-a8e4-9bb0551e64ef tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:11.383275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a41b1154-e3d8-498c-a8e4-9bb0551e64ef tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:11.418867 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a41b1154-e3d8-498c-a8e4-9bb0551e64ef tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 0.550722 Sep 27 09:59:11.422757 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 463/855] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:10 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:11.437157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8e8cf2b-f7d5-4eed-a2f9-bc91688fdcf5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:11.438180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f8e8cf2b-f7d5-4eed-a2f9-bc91688fdcf5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:11.451732 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-40e30ecd-006b-4c37-ab95-4b0c8cd2b1d8 tempest-SecurityGroupsTestAdminJSON-1600475331 tempest-SecurityGroupsTestAdminJSON-1600475331-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/086750e8-c6e8-4629-a76b-04ca9ed3e421" status: 202 len: 0 microversion: 2.1 time: 1.002527 Sep 27 09:59:11.452989 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 393/856] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 09:59:10 2022] DELETE /compute/v2.1/os-security-groups/086750e8-c6e8-4629-a76b-04ca9ed3e421 => generated 0 bytes in 1005 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 09:59:11.454867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e8cf2b-f7d5-4eed-a2f9-bc91688fdcf5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:11.458090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e8cf2b-f7d5-4eed-a2f9-bc91688fdcf5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:11.458090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e8cf2b-f7d5-4eed-a2f9-bc91688fdcf5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:11.458090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e8cf2b-f7d5-4eed-a2f9-bc91688fdcf5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:11.458090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e8cf2b-f7d5-4eed-a2f9-bc91688fdcf5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:11.468329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e8cf2b-f7d5-4eed-a2f9-bc91688fdcf5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:11.886631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1b2d470c-28a9-4a9b-bb33-3794bebde859 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:11.887894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1b2d470c-28a9-4a9b-bb33-3794bebde859 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:11.901973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b2d470c-28a9-4a9b-bb33-3794bebde859 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:11.902737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b2d470c-28a9-4a9b-bb33-3794bebde859 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:11.903324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b2d470c-28a9-4a9b-bb33-3794bebde859 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:11.904210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b2d470c-28a9-4a9b-bb33-3794bebde859 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:11.908411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b2d470c-28a9-4a9b-bb33-3794bebde859 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:11.909132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b2d470c-28a9-4a9b-bb33-3794bebde859 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:11.933650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f8e8cf2b-f7d5-4eed-a2f9-bc91688fdcf5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:11.951358 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f8e8cf2b-f7d5-4eed-a2f9-bc91688fdcf5 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1465 microversion: 2.1 time: 0.520649 Sep 27 09:59:11.952818 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 464/857] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:11 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1465 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:11.981621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bc6e1cc5-f572-434d-8973-58aa126b3009 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:11.982927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bc6e1cc5-f572-434d-8973-58aa126b3009 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:12.011098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc6e1cc5-f572-434d-8973-58aa126b3009 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:12.012000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc6e1cc5-f572-434d-8973-58aa126b3009 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:12.015341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc6e1cc5-f572-434d-8973-58aa126b3009 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:12.016927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc6e1cc5-f572-434d-8973-58aa126b3009 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:12.024691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc6e1cc5-f572-434d-8973-58aa126b3009 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:12.025314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc6e1cc5-f572-434d-8973-58aa126b3009 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:12.519774 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bc6e1cc5-f572-434d-8973-58aa126b3009 tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1616 microversion: 2.1 time: 0.545355 Sep 27 09:59:12.520903 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 465/858] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:11 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1616 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:12.538014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6fe1d2f3-ac7f-4d01-80c6-0a83a66428d3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:12.539620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6fe1d2f3-ac7f-4d01-80c6-0a83a66428d3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:12.550683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe1d2f3-ac7f-4d01-80c6-0a83a66428d3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:12.551555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe1d2f3-ac7f-4d01-80c6-0a83a66428d3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:12.552158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe1d2f3-ac7f-4d01-80c6-0a83a66428d3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:12.553284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe1d2f3-ac7f-4d01-80c6-0a83a66428d3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:12.557567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe1d2f3-ac7f-4d01-80c6-0a83a66428d3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:12.558631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe1d2f3-ac7f-4d01-80c6-0a83a66428d3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:13.134604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1b2d470c-28a9-4a9b-bb33-3794bebde859 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:13.151423 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1b2d470c-28a9-4a9b-bb33-3794bebde859 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1477 microversion: 2.1 time: 1.260254 Sep 27 09:59:13.154503 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 394/859] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:11 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1477 bytes in 1273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:13.178552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersOnMultiNodesTest-server-2094916227", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}, "os:scheduler_hints": {"same_host": "552575d5-492a-4680-9972-29347ad5479b"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:13.285707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:59:13.948200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6fe1d2f3-ac7f-4d01-80c6-0a83a66428d3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:13.965317 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6fe1d2f3-ac7f-4d01-80c6-0a83a66428d3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 1.443049 Sep 27 09:59:13.966660 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 466/860] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:12 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 1445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:14.179241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8b7486f7-2d38-43c7-b14f-05b370f98889 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:14.180754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8b7486f7-2d38-43c7-b14f-05b370f98889 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:14.196476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b7486f7-2d38-43c7-b14f-05b370f98889 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:14.198477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b7486f7-2d38-43c7-b14f-05b370f98889 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:14.199066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b7486f7-2d38-43c7-b14f-05b370f98889 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:14.200236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b7486f7-2d38-43c7-b14f-05b370f98889 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:14.201978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b7486f7-2d38-43c7-b14f-05b370f98889 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:14.204539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b7486f7-2d38-43c7-b14f-05b370f98889 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:14.993416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8b7486f7-2d38-43c7-b14f-05b370f98889 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:15.015801 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8b7486f7-2d38-43c7-b14f-05b370f98889 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1477 microversion: 2.1 time: 0.833858 Sep 27 09:59:15.020724 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 467/861] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:14 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1477 bytes in 843 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:15.022295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-38f5d854-da73-4b19-ab22-7d2f4da5cf07 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:15.022998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-38f5d854-da73-4b19-ab22-7d2f4da5cf07 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:15.037403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38f5d854-da73-4b19-ab22-7d2f4da5cf07 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:15.037901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38f5d854-da73-4b19-ab22-7d2f4da5cf07 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:15.038333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38f5d854-da73-4b19-ab22-7d2f4da5cf07 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:15.043446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38f5d854-da73-4b19-ab22-7d2f4da5cf07 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:15.043955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38f5d854-da73-4b19-ab22-7d2f4da5cf07 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:15.044383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38f5d854-da73-4b19-ab22-7d2f4da5cf07 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:15.377584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:59:15.378080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:59:15.378703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:59:15.379246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:59:15.379729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:59:15.417170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:15.417772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:15.418292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:15.421259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:15.424067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:15.424547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:15.491651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:59:15.500424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Getting quotas for user 00423eb1e4c24763ae0a546225ac1dc7 and project c1ac7b64c68e4896b2c631baacdbe2be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:59:15.538547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:59:15.541940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:59:15.617558 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-45046467-8ef2-4f24-9ee9-229b82db7381 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.459908 Sep 27 09:59:15.618229 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 395/862] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:59:13 2022] POST /compute/v2.1/servers => generated 380 bytes in 2463 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:59:15.631355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6327e313-9d5b-4878-be66-c1053c9dd1b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:15.632450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6327e313-9d5b-4878-be66-c1053c9dd1b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:15.872040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-38f5d854-da73-4b19-ab22-7d2f4da5cf07 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:15.890027 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-38f5d854-da73-4b19-ab22-7d2f4da5cf07 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 0.871894 Sep 27 09:59:15.890623 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 468/863] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:15 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 873 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:15.939287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6327e313-9d5b-4878-be66-c1053c9dd1b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:15.951653 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6327e313-9d5b-4878-be66-c1053c9dd1b8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1217 microversion: 2.1 time: 0.325452 Sep 27 09:59:15.954942 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 396/864] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:15 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1217 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:16.044150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b339d69-58b0-486f-8056-f206cee58ed8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:16.044823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3b339d69-58b0-486f-8056-f206cee58ed8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:16.056022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b339d69-58b0-486f-8056-f206cee58ed8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:16.058898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b339d69-58b0-486f-8056-f206cee58ed8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:16.059555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b339d69-58b0-486f-8056-f206cee58ed8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:16.062198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b339d69-58b0-486f-8056-f206cee58ed8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:16.062754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b339d69-58b0-486f-8056-f206cee58ed8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:16.065309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b339d69-58b0-486f-8056-f206cee58ed8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:16.744371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3b339d69-58b0-486f-8056-f206cee58ed8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:16.760550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-382fd95a-18a0-44fa-8d2f-7d388c5a19ea req-a414268f-0c7d-40c6-9de1-9c3e24efc766 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "832b56e9-4ed3-4b6c-a201-d5af937fd9e7", "name": "network-vif-plugged", "status": "completed", "tag": "86b49b5d-1243-49df-bcec-4cf58543d8d6"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:16.775435 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3b339d69-58b0-486f-8056-f206cee58ed8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1477 microversion: 2.1 time: 0.735450 Sep 27 09:59:16.778075 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 469/865] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:16 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1477 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:16.800134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-382fd95a-18a0-44fa-8d2f-7d388c5a19ea req-a414268f-0c7d-40c6-9de1-9c3e24efc766 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:16.800498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-382fd95a-18a0-44fa-8d2f-7d388c5a19ea req-a414268f-0c7d-40c6-9de1-9c3e24efc766 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:16.813974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-382fd95a-18a0-44fa-8d2f-7d388c5a19ea req-a414268f-0c7d-40c6-9de1-9c3e24efc766 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:16.855856 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-382fd95a-18a0-44fa-8d2f-7d388c5a19ea req-a414268f-0c7d-40c6-9de1-9c3e24efc766 service nova] Creating event network-vif-plugged:86b49b5d-1243-49df-bcec-4cf58543d8d6 for instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 on nch2-master-1 Sep 27 09:59:16.867219 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-382fd95a-18a0-44fa-8d2f-7d388c5a19ea req-a414268f-0c7d-40c6-9de1-9c3e24efc766 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.110981 Sep 27 09:59:16.868502 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 397/866] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:16.915584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-09810219-3eaf-445e-ad7d-23b1d325f489 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:16.916861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-09810219-3eaf-445e-ad7d-23b1d325f489 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:16.928241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09810219-3eaf-445e-ad7d-23b1d325f489 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:16.930111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09810219-3eaf-445e-ad7d-23b1d325f489 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:16.931365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09810219-3eaf-445e-ad7d-23b1d325f489 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:16.932384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09810219-3eaf-445e-ad7d-23b1d325f489 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:16.935007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09810219-3eaf-445e-ad7d-23b1d325f489 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:16.937024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09810219-3eaf-445e-ad7d-23b1d325f489 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:16.982225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cda4f4a3-ff5b-4f4a-8c84-6e6fe53538cd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:16.983113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cda4f4a3-ff5b-4f4a-8c84-6e6fe53538cd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:17.012969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cda4f4a3-ff5b-4f4a-8c84-6e6fe53538cd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:17.013695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cda4f4a3-ff5b-4f4a-8c84-6e6fe53538cd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:17.014691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cda4f4a3-ff5b-4f4a-8c84-6e6fe53538cd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:17.028854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cda4f4a3-ff5b-4f4a-8c84-6e6fe53538cd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:17.030155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cda4f4a3-ff5b-4f4a-8c84-6e6fe53538cd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:17.031369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cda4f4a3-ff5b-4f4a-8c84-6e6fe53538cd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:17.461955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cda4f4a3-ff5b-4f4a-8c84-6e6fe53538cd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:17.489285 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cda4f4a3-ff5b-4f4a-8c84-6e6fe53538cd tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1213 microversion: 2.1 time: 0.510537 Sep 27 09:59:17.501522 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 398/867] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:16 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1213 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:17.614647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-09810219-3eaf-445e-ad7d-23b1d325f489 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:17.632487 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-09810219-3eaf-445e-ad7d-23b1d325f489 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 0.721341 Sep 27 09:59:17.641350 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 470/868] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:16 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 731 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:17.802070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58c75b64-9b65-4a46-a7a1-62daa3b4d262 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:17.806080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-58c75b64-9b65-4a46-a7a1-62daa3b4d262 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:17.818794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c75b64-9b65-4a46-a7a1-62daa3b4d262 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:17.819202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c75b64-9b65-4a46-a7a1-62daa3b4d262 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:17.821965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c75b64-9b65-4a46-a7a1-62daa3b4d262 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:17.826889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c75b64-9b65-4a46-a7a1-62daa3b4d262 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:17.827601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c75b64-9b65-4a46-a7a1-62daa3b4d262 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:17.828199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c75b64-9b65-4a46-a7a1-62daa3b4d262 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:18.380525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-58c75b64-9b65-4a46-a7a1-62daa3b4d262 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:18.389938 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-58c75b64-9b65-4a46-a7a1-62daa3b4d262 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1477 microversion: 2.1 time: 0.592478 Sep 27 09:59:18.390544 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 399/869] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:17 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1477 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:18.522145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7649d9ab-a053-41f5-a5c3-c2e5abdfe313 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:18.522768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7649d9ab-a053-41f5-a5c3-c2e5abdfe313 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:18.541939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7649d9ab-a053-41f5-a5c3-c2e5abdfe313 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:18.543194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7649d9ab-a053-41f5-a5c3-c2e5abdfe313 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:18.544016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7649d9ab-a053-41f5-a5c3-c2e5abdfe313 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:18.546126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7649d9ab-a053-41f5-a5c3-c2e5abdfe313 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:18.547016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7649d9ab-a053-41f5-a5c3-c2e5abdfe313 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:18.547698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7649d9ab-a053-41f5-a5c3-c2e5abdfe313 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:18.897456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7649d9ab-a053-41f5-a5c3-c2e5abdfe313 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:18.906252 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7649d9ab-a053-41f5-a5c3-c2e5abdfe313 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1275 microversion: 2.1 time: 0.387906 Sep 27 09:59:18.913643 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 471/870] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:18 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1275 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:18.931180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cc93d41f-8644-4d23-aa44-77854a2d2022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:18.932299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cc93d41f-8644-4d23-aa44-77854a2d2022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:18.952350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc93d41f-8644-4d23-aa44-77854a2d2022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:18.962338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc93d41f-8644-4d23-aa44-77854a2d2022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:18.963279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc93d41f-8644-4d23-aa44-77854a2d2022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:18.969053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc93d41f-8644-4d23-aa44-77854a2d2022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:18.969778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc93d41f-8644-4d23-aa44-77854a2d2022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:18.970374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc93d41f-8644-4d23-aa44-77854a2d2022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:19.151995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FloatingIPsAssociationNegativeTestJSON-server-80445180", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "e00d057c-26e1-4ae4-9895-9dbcbbcbbfc3"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:19.297498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] validate_networks() for [('e00d057c-26e1-4ae4-9895-9dbcbbcbbfc3', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:59:19.750757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cc93d41f-8644-4d23-aa44-77854a2d2022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:19.766776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:19.788486 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cc93d41f-8644-4d23-aa44-77854a2d2022 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1386 microversion: 2.1 time: 0.868112 Sep 27 09:59:19.791636 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 472/871] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:18 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1386 bytes in 875 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:19.808476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-620b06d6-afa2-4d8a-a769-d832e414067c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:19.809284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-620b06d6-afa2-4d8a-a769-d832e414067c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:19.823077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620b06d6-afa2-4d8a-a769-d832e414067c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:19.824695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620b06d6-afa2-4d8a-a769-d832e414067c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:19.825223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620b06d6-afa2-4d8a-a769-d832e414067c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:19.826202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620b06d6-afa2-4d8a-a769-d832e414067c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:19.827884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620b06d6-afa2-4d8a-a769-d832e414067c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:19.830163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620b06d6-afa2-4d8a-a769-d832e414067c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:20.252755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:59:20.254908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:59:20.255819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:59:20.256460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:59:20.257138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:59:20.280178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:20.281279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:20.281941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:20.283173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:20.283890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:20.284481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:20.324785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Getting quotas for project 2b3380f3fa2b494da649aaa809e11368. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:59:20.330249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Getting quotas for user 57365b62b3294810b8a030d2a26cc451 and project 2b3380f3fa2b494da649aaa809e11368. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:59:20.385293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:59:20.394609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:59:20.480582 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-175b97e9-2d05-4b4e-9425-6419dacf25cc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.835476 Sep 27 09:59:20.482004 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 400/872] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:59:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 1837 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:59:20.958912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-620b06d6-afa2-4d8a-a769-d832e414067c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:20.968006 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-620b06d6-afa2-4d8a-a769-d832e414067c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1477 microversion: 2.1 time: 1.175222 Sep 27 09:59:20.972988 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 473/873] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:19 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1477 bytes in 1181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:20.985752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-320cd399-9817-46e0-b0cb-77079653c2bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:20.989620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-320cd399-9817-46e0-b0cb-77079653c2bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:21.000494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-320cd399-9817-46e0-b0cb-77079653c2bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:21.002938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-320cd399-9817-46e0-b0cb-77079653c2bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:21.004614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-320cd399-9817-46e0-b0cb-77079653c2bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:21.008898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-320cd399-9817-46e0-b0cb-77079653c2bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:21.009715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-320cd399-9817-46e0-b0cb-77079653c2bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:21.010318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-320cd399-9817-46e0-b0cb-77079653c2bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:21.138396 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0dbacc65-b207-4d74-87b7-39fbfc2bbf48 tempest-TenantUsagesNegativeTestJSON-80152298 tempest-TenantUsagesNegativeTestJSON-80152298-project-admin] 10.222.0.26 "GET /compute/v2.1/os-simple-tenant-usage/?start=2022-09-26T09%3A59%3A19.785087&end=2022-09-28T09%3A59%3A19.785087" status: 404 len: 112 microversion: - time: 0.654953 Sep 27 09:59:21.139198 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 401/874] 10.222.0.26 () {58 vars in 1316 bytes} [Tue Sep 27 09:59:20 2022] GET /compute/v2.1/os-simple-tenant-usage/?start=2022-09-26T09%3A59%3A19.785087&end=2022-09-28T09%3A59%3A19.785087 => generated 112 bytes in 656 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 27 09:59:21.155191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-69be15ba-9ec7-4ce9-b5ee-92f06de58230 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:21.162711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-69be15ba-9ec7-4ce9-b5ee-92f06de58230 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:21.364015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-69be15ba-9ec7-4ce9-b5ee-92f06de58230 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:21.405338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-320cd399-9817-46e0-b0cb-77079653c2bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:21.409298 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-69be15ba-9ec7-4ce9-b5ee-92f06de58230 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1230 microversion: 2.1 time: 0.268692 Sep 27 09:59:21.411855 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 402/875] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:21 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1230 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:21.429384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b4cb3e6d-4a15-4b73-ad1b-721cc6744cc8 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:21.434004 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-320cd399-9817-46e0-b0cb-77079653c2bf tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1275 microversion: 2.1 time: 0.454194 Sep 27 09:59:21.436843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b4cb3e6d-4a15-4b73-ad1b-721cc6744cc8 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:21.438961 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 474/876] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:20 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1275 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:21.451096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c79d2fc9-9aea-46e5-9768-1a966794ec8b tempest-TenantUsagesNegativeTestJSON-80152298 tempest-TenantUsagesNegativeTestJSON-80152298-project-admin] Calling method '.version_select at 0x7f7f638395e0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:21.459218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4cb3e6d-4a15-4b73-ad1b-721cc6744cc8 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:21.462246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4cb3e6d-4a15-4b73-ad1b-721cc6744cc8 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:21.464647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4cb3e6d-4a15-4b73-ad1b-721cc6744cc8 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:21.465670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4cb3e6d-4a15-4b73-ad1b-721cc6744cc8 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:21.468447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4cb3e6d-4a15-4b73-ad1b-721cc6744cc8 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:21.469144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4cb3e6d-4a15-4b73-ad1b-721cc6744cc8 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:21.481383 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-c79d2fc9-9aea-46e5-9768-1a966794ec8b tempest-TenantUsagesNegativeTestJSON-80152298 tempest-TenantUsagesNegativeTestJSON-80152298-project-admin] HTTP exception thrown: Invalid start time. The start time cannot occur after the end time. Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c79d2fc9-9aea-46e5-9768-1a966794ec8b tempest-TenantUsagesNegativeTestJSON-80152298 tempest-TenantUsagesNegativeTestJSON-80152298-project-admin] Request method failure captured: Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-simple-tenant-usage/326f1e4c552b4263b54ccd2836c4f59d?start=2022-09-28T09%3A59%3A19.785087&end=2022-09-26T09%3A59%3A19.785087 HTTP/1.1 Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsl3FV-iLrTPOyNXzQzmvYoZxLjAr4JY9CWgY1R9o8bvIEEZE_onU4LcXtAKpJnRv0rt7mjJgg0QMlsM88wC0dmNqdYMGbTu-_DWKI3TIlDEICyI5Ys_MfpYS-RYiyrtp7WswykCtW0rZJcQJ5348eAVN_HhZ2q-JkG3Cs83XkMxtRg Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 9f7c04508ed14a2f9b8dc8aed2745592 Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TenantUsagesNegativeTestJSON-80152298 Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 09:59:21.485845 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9f7c04508ed14a2f9b8dc8aed2745592"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9f7c04508ed14a2f9b8dc8aed2745592"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9f7c04508ed14a2f9b8dc8aed2745592"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9f7c04508ed14a2f9b8dc8aed2745592"}]}] Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TenantUsagesNegativeTestJSON-80152298 Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 9f7c04508ed14a2f9b8dc8aed2745592 Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TenantUsagesNegativeTestJSON-80152298 Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TenantUsagesNegativeTestJSON-80152298-project-admin Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 5126adbc2c534fb2bf0800ba526f3716 Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TenantUsagesNegativeTestJSON-80152298-project-admin Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid start time. The start time cannot occur after the end time. Sep 27 09:59:21.487616 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '326f1e4c552b4263b54ccd2836c4f59d'} Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 292, in show Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._show(req, id) Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 338, in _show Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi (period_start, period_stop, ignore) = self._get_datetime_range( Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 259, in _get_datetime_range Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Sep 27 09:59:21.488642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid start time. The start time cannot occur after the end time. Sep 27 09:59:21.490231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:21.490231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:59:21.490231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:21.490231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:21.490231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:59:21.490231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:59:21.490231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:59:21.490231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:59:21.490231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid start time. The start time cannot occur after the end time. Sep 27 09:59:21.490231 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:21.493736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c79d2fc9-9aea-46e5-9768-1a966794ec8b tempest-TenantUsagesNegativeTestJSON-80152298 tempest-TenantUsagesNegativeTestJSON-80152298-project-admin] Returning 400 to user: Invalid start time. The start time cannot occur after the end time. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:59:21.495153 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c79d2fc9-9aea-46e5-9768-1a966794ec8b tempest-TenantUsagesNegativeTestJSON-80152298 tempest-TenantUsagesNegativeTestJSON-80152298-project-admin] 10.222.0.26 "GET /compute/v2.1/os-simple-tenant-usage/326f1e4c552b4263b54ccd2836c4f59d?start=2022-09-28T09%3A59%3A19.785087&end=2022-09-26T09%3A59%3A19.785087" status: 400 len: 111 microversion: 2.1 time: 0.055174 Sep 27 09:59:21.499916 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 475/877] 10.222.0.26 () {58 vars in 1412 bytes} [Tue Sep 27 09:59:21 2022] GET /compute/v2.1/os-simple-tenant-usage/326f1e4c552b4263b54ccd2836c4f59d?start=2022-09-28T09%3A59%3A19.785087&end=2022-09-26T09%3A59%3A19.785087 => generated 111 bytes in 60 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 09:59:21.837982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e84cfcb2-7ff9-459b-8b01-28d3e7483fe7 tempest-TenantUsagesNegativeTestJSON-1889527120 tempest-TenantUsagesNegativeTestJSON-1889527120-project] Calling method '.version_select at 0x7f7f6380b8b0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:21.843495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e84cfcb2-7ff9-459b-8b01-28d3e7483fe7 tempest-TenantUsagesNegativeTestJSON-1889527120 tempest-TenantUsagesNegativeTestJSON-1889527120-project] Policy check for os_compute_api:os-simple-tenant-usage:list failed with credentials {'is_admin': False, 'user_id': 'be87e699e5eb458e94e5e9f63ddcac26', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326f1e4c552b4263b54ccd2836c4f59d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e84cfcb2-7ff9-459b-8b01-28d3e7483fe7 tempest-TenantUsagesNegativeTestJSON-1889527120 tempest-TenantUsagesNegativeTestJSON-1889527120-project] Request method failure captured: Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-simple-tenant-usage?start=2022-09-26T09%3A59%3A19.785087&end=2022-09-28T09%3A59%3A19.785087&detailed=1 HTTP/1.1 Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsl1OXjdfBofVFrXqaY29rOrOHgJ8LHqhVlLC3vjccWcGU3hvmr7coDdKClKXtPfCSZaC8hyEf7UYNJp3ovB2V9x1QmdwemovRcEMdvRYNDODPQKRuXGFgC4yZ8X-Fp0RYWiBNaHNp5b06XJJCyyYyONaCZs4Kg6BBheteFf1EGoDO4 Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 326f1e4c552b4263b54ccd2836c4f59d Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TenantUsagesNegativeTestJSON-1889527120 Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:59:21.845269 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/326f1e4c552b4263b54ccd2836c4f59d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/326f1e4c552b4263b54ccd2836c4f59d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/326f1e4c552b4263b54ccd2836c4f59d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_326f1e4c552b4263b54ccd2836c4f59d"}]}] Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TenantUsagesNegativeTestJSON-1889527120 Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 326f1e4c552b4263b54ccd2836c4f59d Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TenantUsagesNegativeTestJSON-1889527120 Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TenantUsagesNegativeTestJSON-1889527120-project Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: be87e699e5eb458e94e5e9f63ddcac26 Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TenantUsagesNegativeTestJSON-1889527120-project Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: exception: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: action_args: {} Sep 27 09:59:21.846796 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 277, in index Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._index(req) Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/simple_tenant_usage.py", line 297, in _index Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi context.can(stu_policies.POLICY_ROOT % 'list') Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 09:59:21.847753 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 09:59:21.849154 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 09:59:21.850523 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. Sep 27 09:59:21.850523 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:21.850523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e84cfcb2-7ff9-459b-8b01-28d3e7483fe7 tempest-TenantUsagesNegativeTestJSON-1889527120 tempest-TenantUsagesNegativeTestJSON-1889527120-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-simple-tenant-usage:list to be performed. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:59:21.858958 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e84cfcb2-7ff9-459b-8b01-28d3e7483fe7 tempest-TenantUsagesNegativeTestJSON-1889527120 tempest-TenantUsagesNegativeTestJSON-1889527120-project] 10.222.0.26 "GET /compute/v2.1/os-simple-tenant-usage?start=2022-09-26T09%3A59%3A19.785087&end=2022-09-28T09%3A59%3A19.785087&detailed=1" status: 403 len: 123 microversion: 2.1 time: 0.338065 Sep 27 09:59:21.859521 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 476/878] 10.222.0.26 () {58 vars in 1335 bytes} [Tue Sep 27 09:59:21 2022] GET /compute/v2.1/os-simple-tenant-usage?start=2022-09-26T09%3A59%3A19.785087&end=2022-09-28T09%3A59%3A19.785087&detailed=1 => generated 123 bytes in 352 msecs (HTTP/1.1 403) 9 headers in 379 bytes (2 switches on core 0) Sep 27 09:59:22.012020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c98ac3ee-3dcb-493c-ae1f-330885431806 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:22.013539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c98ac3ee-3dcb-493c-ae1f-330885431806 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:22.027775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c98ac3ee-3dcb-493c-ae1f-330885431806 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:22.029781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c98ac3ee-3dcb-493c-ae1f-330885431806 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:22.030781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c98ac3ee-3dcb-493c-ae1f-330885431806 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:22.032460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c98ac3ee-3dcb-493c-ae1f-330885431806 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:22.033682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c98ac3ee-3dcb-493c-ae1f-330885431806 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:22.034465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c98ac3ee-3dcb-493c-ae1f-330885431806 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:22.048184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b4cb3e6d-4a15-4b73-ad1b-721cc6744cc8 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:22.070924 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b4cb3e6d-4a15-4b73-ad1b-721cc6744cc8 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1585 microversion: 2.1 time: 0.647533 Sep 27 09:59:22.073451 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 403/879] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:21 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1585 bytes in 654 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:22.099968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bfdace77-cd69-475f-a814-f2865449ccb6 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:22.100578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bfdace77-cd69-475f-a814-f2865449ccb6 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:22.110430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfdace77-cd69-475f-a814-f2865449ccb6 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:22.111299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfdace77-cd69-475f-a814-f2865449ccb6 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:22.111982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfdace77-cd69-475f-a814-f2865449ccb6 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:22.112981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfdace77-cd69-475f-a814-f2865449ccb6 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:22.113805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfdace77-cd69-475f-a814-f2865449ccb6 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:22.115786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfdace77-cd69-475f-a814-f2865449ccb6 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:22.859635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c98ac3ee-3dcb-493c-ae1f-330885431806 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:22.879385 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c98ac3ee-3dcb-493c-ae1f-330885431806 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1477 microversion: 2.1 time: 0.879541 Sep 27 09:59:22.897853 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 477/880] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:21 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1477 bytes in 900 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:22.926595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e903a5d-0467-4ec7-9e5c-be715b444496 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:22.935371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e903a5d-0467-4ec7-9e5c-be715b444496 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:22.955313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e903a5d-0467-4ec7-9e5c-be715b444496 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:22.958828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e903a5d-0467-4ec7-9e5c-be715b444496 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:22.965618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e903a5d-0467-4ec7-9e5c-be715b444496 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:22.966832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e903a5d-0467-4ec7-9e5c-be715b444496 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:22.967484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e903a5d-0467-4ec7-9e5c-be715b444496 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:22.968104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e903a5d-0467-4ec7-9e5c-be715b444496 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:23.162546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0e903a5d-0467-4ec7-9e5c-be715b444496 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:23.173998 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e903a5d-0467-4ec7-9e5c-be715b444496 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1282 microversion: 2.1 time: 0.265214 Sep 27 09:59:23.175117 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 478/881] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:22 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1282 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:23.180077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf44063c-7c97-400a-a678-21574e7ce01d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:23.190012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bf44063c-7c97-400a-a678-21574e7ce01d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:23.201181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf44063c-7c97-400a-a678-21574e7ce01d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:23.201720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf44063c-7c97-400a-a678-21574e7ce01d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:23.202164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf44063c-7c97-400a-a678-21574e7ce01d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:23.202896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf44063c-7c97-400a-a678-21574e7ce01d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:23.203355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf44063c-7c97-400a-a678-21574e7ce01d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:23.205260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf44063c-7c97-400a-a678-21574e7ce01d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:23.287522 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bfdace77-cd69-475f-a814-f2865449ccb6 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7/os-interface" status: 200 len: 284 microversion: 2.1 time: 1.191953 Sep 27 09:59:23.288265 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 404/882] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 09:59:22 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7/os-interface => generated 284 bytes in 1194 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:59:23.909810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a9b4738-0e47-4ddd-8869-0693592ba202 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:23.910889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a9b4738-0e47-4ddd-8869-0693592ba202 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:23.921181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a9b4738-0e47-4ddd-8869-0693592ba202 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:23.921977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a9b4738-0e47-4ddd-8869-0693592ba202 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:23.922730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a9b4738-0e47-4ddd-8869-0693592ba202 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:23.923748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a9b4738-0e47-4ddd-8869-0693592ba202 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:23.924436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a9b4738-0e47-4ddd-8869-0693592ba202 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:23.925485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a9b4738-0e47-4ddd-8869-0693592ba202 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:24.168814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bf44063c-7c97-400a-a678-21574e7ce01d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:24.180634 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bf44063c-7c97-400a-a678-21574e7ce01d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1317 microversion: 2.1 time: 1.004534 Sep 27 09:59:24.182676 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 479/883] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:23 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1317 bytes in 1007 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:24.198462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-778db000-c627-4d96-a408-67043e178fc1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:24.199477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-778db000-c627-4d96-a408-67043e178fc1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:24.213113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778db000-c627-4d96-a408-67043e178fc1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:24.214146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778db000-c627-4d96-a408-67043e178fc1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:24.215543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778db000-c627-4d96-a408-67043e178fc1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:24.220202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778db000-c627-4d96-a408-67043e178fc1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:24.221121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778db000-c627-4d96-a408-67043e178fc1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:24.221736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778db000-c627-4d96-a408-67043e178fc1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:24.411829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-778db000-c627-4d96-a408-67043e178fc1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:24.424096 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-778db000-c627-4d96-a408-67043e178fc1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1288 microversion: 2.1 time: 0.230506 Sep 27 09:59:24.425392 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 480/884] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:24 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1288 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:24.721003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a9b4738-0e47-4ddd-8869-0693592ba202 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:24.751618 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a9b4738-0e47-4ddd-8869-0693592ba202 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1476 microversion: 2.1 time: 0.847300 Sep 27 09:59:24.754270 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 405/885] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:23 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1476 bytes in 851 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:24.878113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-64796595-a0b5-49a2-855a-ee690021adc5 req-ccdcbb5e-ceea-4a0f-a7ac-79c15582e92e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48f61d6b-3985-49a2-9eb9-dd250711cdf4", "name": "network-vif-unplugged", "status": "completed", "tag": "04d6d3fb-4795-41f4-a54d-61772b8a10a8"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:24.891475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-64796595-a0b5-49a2-855a-ee690021adc5 req-ccdcbb5e-ceea-4a0f-a7ac-79c15582e92e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:24.892264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-64796595-a0b5-49a2-855a-ee690021adc5 req-ccdcbb5e-ceea-4a0f-a7ac-79c15582e92e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:24.892903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-64796595-a0b5-49a2-855a-ee690021adc5 req-ccdcbb5e-ceea-4a0f-a7ac-79c15582e92e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:24.907459 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-64796595-a0b5-49a2-855a-ee690021adc5 req-ccdcbb5e-ceea-4a0f-a7ac-79c15582e92e service nova] Creating event network-vif-unplugged:04d6d3fb-4795-41f4-a54d-61772b8a10a8 for instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 on nch1-master-1 Sep 27 09:59:24.914622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-64796595-a0b5-49a2-855a-ee690021adc5 req-ccdcbb5e-ceea-4a0f-a7ac-79c15582e92e service nova] Instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 09:59:24.927531 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-64796595-a0b5-49a2-855a-ee690021adc5 req-ccdcbb5e-ceea-4a0f-a7ac-79c15582e92e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.053001 Sep 27 09:59:24.928889 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 481/886] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:24 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:25.209115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ae525bb0-01ab-4e0d-baea-0de2bb3c51db tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:25.210044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ae525bb0-01ab-4e0d-baea-0de2bb3c51db tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:25.217419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae525bb0-01ab-4e0d-baea-0de2bb3c51db tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:25.218254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae525bb0-01ab-4e0d-baea-0de2bb3c51db tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:25.218943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae525bb0-01ab-4e0d-baea-0de2bb3c51db tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:25.219878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae525bb0-01ab-4e0d-baea-0de2bb3c51db tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:25.220570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae525bb0-01ab-4e0d-baea-0de2bb3c51db tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:25.221238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae525bb0-01ab-4e0d-baea-0de2bb3c51db tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:25.448637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5887e2e-7668-4509-bd67-cbcf909109a3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:25.450115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b5887e2e-7668-4509-bd67-cbcf909109a3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:25.457151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5887e2e-7668-4509-bd67-cbcf909109a3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:25.457896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5887e2e-7668-4509-bd67-cbcf909109a3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:25.458512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5887e2e-7668-4509-bd67-cbcf909109a3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:25.459422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5887e2e-7668-4509-bd67-cbcf909109a3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:25.460041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5887e2e-7668-4509-bd67-cbcf909109a3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:25.460616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5887e2e-7668-4509-bd67-cbcf909109a3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:25.805677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b5887e2e-7668-4509-bd67-cbcf909109a3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:25.827447 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b5887e2e-7668-4509-bd67-cbcf909109a3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1288 microversion: 2.1 time: 0.383144 Sep 27 09:59:25.828270 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 482/887] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:25 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1288 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:25.839602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-031ab6bd-f150-49d6-99cf-c00f217a9e4a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:25.842738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-031ab6bd-f150-49d6-99cf-c00f217a9e4a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:25.854426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-031ab6bd-f150-49d6-99cf-c00f217a9e4a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:25.855245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-031ab6bd-f150-49d6-99cf-c00f217a9e4a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:25.855850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-031ab6bd-f150-49d6-99cf-c00f217a9e4a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:25.856773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-031ab6bd-f150-49d6-99cf-c00f217a9e4a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:25.857393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-031ab6bd-f150-49d6-99cf-c00f217a9e4a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:25.857974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-031ab6bd-f150-49d6-99cf-c00f217a9e4a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:26.627196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-031ab6bd-f150-49d6-99cf-c00f217a9e4a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:26.639814 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-031ab6bd-f150-49d6-99cf-c00f217a9e4a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1476 microversion: 2.1 time: 0.810243 Sep 27 09:59:26.640418 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 483/888] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:25 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1476 bytes in 811 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:26.857194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-593a1951-e485-4f1a-9786-fb8562fbe60d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:26.857814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-593a1951-e485-4f1a-9786-fb8562fbe60d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:26.890194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-593a1951-e485-4f1a-9786-fb8562fbe60d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:26.894822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-593a1951-e485-4f1a-9786-fb8562fbe60d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:26.896062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-593a1951-e485-4f1a-9786-fb8562fbe60d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:26.905972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-593a1951-e485-4f1a-9786-fb8562fbe60d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:26.908286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-593a1951-e485-4f1a-9786-fb8562fbe60d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:26.909117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-593a1951-e485-4f1a-9786-fb8562fbe60d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:27.215946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ae525bb0-01ab-4e0d-baea-0de2bb3c51db tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:27.230014 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ae525bb0-01ab-4e0d-baea-0de2bb3c51db tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1317 microversion: 2.1 time: 2.023506 Sep 27 09:59:27.231023 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 406/889] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:25 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1317 bytes in 2027 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:27.551936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-593a1951-e485-4f1a-9786-fb8562fbe60d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:27.561939 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-593a1951-e485-4f1a-9786-fb8562fbe60d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 0.708929 Sep 27 09:59:27.564728 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 484/890] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:26 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:27.657630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-1779957301", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "43a35817-b5c9-4e15-8250-6b84b0c2b25f"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:27.665566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5079eea9-e2c0-4ab2-86a5-a0db589f85fa req-7fb307c6-643e-4d34-b062-c11c71f7470d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8", "tag": "64043e29-53c7-4b1b-8d01-df27fcfbc107"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:27.691056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5079eea9-e2c0-4ab2-86a5-a0db589f85fa req-7fb307c6-643e-4d34-b062-c11c71f7470d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:27.693267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5079eea9-e2c0-4ab2-86a5-a0db589f85fa req-7fb307c6-643e-4d34-b062-c11c71f7470d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:27.693793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5079eea9-e2c0-4ab2-86a5-a0db589f85fa req-7fb307c6-643e-4d34-b062-c11c71f7470d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:27.730940 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5079eea9-e2c0-4ab2-86a5-a0db589f85fa req-7fb307c6-643e-4d34-b062-c11c71f7470d service nova] Creating event network-changed:64043e29-53c7-4b1b-8d01-df27fcfbc107 for instance 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 on nch1-master-1 Sep 27 09:59:27.744741 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5079eea9-e2c0-4ab2-86a5-a0db589f85fa req-7fb307c6-643e-4d34-b062-c11c71f7470d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083284 Sep 27 09:59:27.747353 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 485/891] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:59:27.760183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] validate_networks() for [('43a35817-b5c9-4e15-8250-6b84b0c2b25f', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:59:27.761569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ca89df6f-eb56-4b82-9106-c4db424371e1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:27.764988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ca89df6f-eb56-4b82-9106-c4db424371e1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:27.780160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca89df6f-eb56-4b82-9106-c4db424371e1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:27.783399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca89df6f-eb56-4b82-9106-c4db424371e1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:27.783870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca89df6f-eb56-4b82-9106-c4db424371e1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:27.784692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca89df6f-eb56-4b82-9106-c4db424371e1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:27.785140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca89df6f-eb56-4b82-9106-c4db424371e1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:27.785588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca89df6f-eb56-4b82-9106-c4db424371e1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:28.463372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ca89df6f-eb56-4b82-9106-c4db424371e1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:28.477744 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ca89df6f-eb56-4b82-9106-c4db424371e1 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1476 microversion: 2.1 time: 0.727861 Sep 27 09:59:28.490793 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 486/892] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:27 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1476 bytes in 743 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:28.497628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-352255fe-5e41-4b83-a862-6bfca1b14e5e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:28.498792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-352255fe-5e41-4b83-a862-6bfca1b14e5e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:28.505654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-352255fe-5e41-4b83-a862-6bfca1b14e5e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:28.506351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-352255fe-5e41-4b83-a862-6bfca1b14e5e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:28.506985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-352255fe-5e41-4b83-a862-6bfca1b14e5e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:28.508140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-352255fe-5e41-4b83-a862-6bfca1b14e5e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:28.508809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-352255fe-5e41-4b83-a862-6bfca1b14e5e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:28.509381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-352255fe-5e41-4b83-a862-6bfca1b14e5e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:28.990978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-352255fe-5e41-4b83-a862-6bfca1b14e5e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:29.005808 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-352255fe-5e41-4b83-a862-6bfca1b14e5e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1317 microversion: 2.1 time: 0.513279 Sep 27 09:59:29.009701 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 487/893] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:28 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1317 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:29.024122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5ae137a4-6fb1-4eee-bbdf-33adc28f10f8 req-142a4cb0-c07e-4643-ba94-0d714c0409df service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "48f61d6b-3985-49a2-9eb9-dd250711cdf4", "tag": "04d6d3fb-4795-41f4-a54d-61772b8a10a8"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:29.094477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5ae137a4-6fb1-4eee-bbdf-33adc28f10f8 req-142a4cb0-c07e-4643-ba94-0d714c0409df service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:29.095715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5ae137a4-6fb1-4eee-bbdf-33adc28f10f8 req-142a4cb0-c07e-4643-ba94-0d714c0409df service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:29.098430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5ae137a4-6fb1-4eee-bbdf-33adc28f10f8 req-142a4cb0-c07e-4643-ba94-0d714c0409df service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:29.145704 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5ae137a4-6fb1-4eee-bbdf-33adc28f10f8 req-142a4cb0-c07e-4643-ba94-0d714c0409df service nova] Creating event network-changed:04d6d3fb-4795-41f4-a54d-61772b8a10a8 for instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 on nch1-master-1 Sep 27 09:59:29.179640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-5ae137a4-6fb1-4eee-bbdf-33adc28f10f8 req-142a4cb0-c07e-4643-ba94-0d714c0409df service nova] Instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 09:59:29.212294 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5ae137a4-6fb1-4eee-bbdf-33adc28f10f8 req-142a4cb0-c07e-4643-ba94-0d714c0409df service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.197557 Sep 27 09:59:29.216842 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 488/894] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:29.245245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3fab198f-b21c-4a81-9b1f-63eb5b52e6a5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:29.246186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3fab198f-b21c-4a81-9b1f-63eb5b52e6a5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:29.277064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fab198f-b21c-4a81-9b1f-63eb5b52e6a5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:29.277667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fab198f-b21c-4a81-9b1f-63eb5b52e6a5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:29.278130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fab198f-b21c-4a81-9b1f-63eb5b52e6a5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:29.278903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fab198f-b21c-4a81-9b1f-63eb5b52e6a5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:29.279373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fab198f-b21c-4a81-9b1f-63eb5b52e6a5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:29.279815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fab198f-b21c-4a81-9b1f-63eb5b52e6a5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:30.171222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3fab198f-b21c-4a81-9b1f-63eb5b52e6a5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:30.182875 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3fab198f-b21c-4a81-9b1f-63eb5b52e6a5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 0.960757 Sep 27 09:59:30.183497 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 489/895] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:29 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 965 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:30.203018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a334236-911d-4bf2-bd61-e2d58c470bf3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:30.203632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3a334236-911d-4bf2-bd61-e2d58c470bf3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:30.225466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a334236-911d-4bf2-bd61-e2d58c470bf3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:30.228788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a334236-911d-4bf2-bd61-e2d58c470bf3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:30.229560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a334236-911d-4bf2-bd61-e2d58c470bf3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:30.238129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a334236-911d-4bf2-bd61-e2d58c470bf3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:30.238129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a334236-911d-4bf2-bd61-e2d58c470bf3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:30.238129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a334236-911d-4bf2-bd61-e2d58c470bf3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:30.308520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:30.549957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3a334236-911d-4bf2-bd61-e2d58c470bf3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:30.591181 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3a334236-911d-4bf2-bd61-e2d58c470bf3 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1585 microversion: 2.1 time: 0.406161 Sep 27 09:59:30.598006 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 490/896] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:30 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1585 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:30.638236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-871ffa49-d8bf-4ec0-96cb-fd48df381d4f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:30.639586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-871ffa49-d8bf-4ec0-96cb-fd48df381d4f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:30.647730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-871ffa49-d8bf-4ec0-96cb-fd48df381d4f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:30.649128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-871ffa49-d8bf-4ec0-96cb-fd48df381d4f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:30.650165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-871ffa49-d8bf-4ec0-96cb-fd48df381d4f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:30.651946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-871ffa49-d8bf-4ec0-96cb-fd48df381d4f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:30.652996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-871ffa49-d8bf-4ec0-96cb-fd48df381d4f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:30.653876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-871ffa49-d8bf-4ec0-96cb-fd48df381d4f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:31.188973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-871ffa49-d8bf-4ec0-96cb-fd48df381d4f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:31.282334 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-871ffa49-d8bf-4ec0-96cb-fd48df381d4f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1476 microversion: 2.1 time: 0.637849 Sep 27 09:59:31.301261 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 491/897] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:30 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1476 bytes in 692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:31.318059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-4f0c3893-6238-4633-b0ac-3c584efcab7b req-5dc4f501-9e94-4dc9-a4d6-fa892b9bc3b5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "832b56e9-4ed3-4b6c-a201-d5af937fd9e7", "tag": "86b49b5d-1243-49df-bcec-4cf58543d8d6"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:31.331799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4f0c3893-6238-4633-b0ac-3c584efcab7b req-5dc4f501-9e94-4dc9-a4d6-fa892b9bc3b5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:31.332547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4f0c3893-6238-4633-b0ac-3c584efcab7b req-5dc4f501-9e94-4dc9-a4d6-fa892b9bc3b5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:31.333212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4f0c3893-6238-4633-b0ac-3c584efcab7b req-5dc4f501-9e94-4dc9-a4d6-fa892b9bc3b5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:31.350840 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-4f0c3893-6238-4633-b0ac-3c584efcab7b req-5dc4f501-9e94-4dc9-a4d6-fa892b9bc3b5 service nova] Creating event network-changed:86b49b5d-1243-49df-bcec-4cf58543d8d6 for instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 on nch2-master-1 Sep 27 09:59:31.365532 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-4f0c3893-6238-4633-b0ac-3c584efcab7b req-5dc4f501-9e94-4dc9-a4d6-fa892b9bc3b5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070993 Sep 27 09:59:31.369281 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 492/898] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:31.372431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:31.381682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82b679e0-8bcd-43af-bc96-2f02534f5f1f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:31.382625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-82b679e0-8bcd-43af-bc96-2f02534f5f1f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:31.433038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82b679e0-8bcd-43af-bc96-2f02534f5f1f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:31.439082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82b679e0-8bcd-43af-bc96-2f02534f5f1f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:31.440016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82b679e0-8bcd-43af-bc96-2f02534f5f1f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:31.441064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82b679e0-8bcd-43af-bc96-2f02534f5f1f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:31.441868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82b679e0-8bcd-43af-bc96-2f02534f5f1f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:31.442487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82b679e0-8bcd-43af-bc96-2f02534f5f1f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:31.492998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:31.945297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-82b679e0-8bcd-43af-bc96-2f02534f5f1f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:31.971619 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-82b679e0-8bcd-43af-bc96-2f02534f5f1f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1317 microversion: 2.1 time: 0.598143 Sep 27 09:59:31.973019 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 493/899] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:31 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1317 bytes in 603 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:31.984031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f3e0ecaa-7958-4351-bd05-6de8663236db tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:31.986280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f3e0ecaa-7958-4351-bd05-6de8663236db tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:32.003565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:32.010919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3e0ecaa-7958-4351-bd05-6de8663236db tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:32.012016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3e0ecaa-7958-4351-bd05-6de8663236db tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:32.012718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3e0ecaa-7958-4351-bd05-6de8663236db tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:32.014299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3e0ecaa-7958-4351-bd05-6de8663236db tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:32.015440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3e0ecaa-7958-4351-bd05-6de8663236db tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:32.023815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3e0ecaa-7958-4351-bd05-6de8663236db tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:32.555221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f3e0ecaa-7958-4351-bd05-6de8663236db tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:32.598090 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f3e0ecaa-7958-4351-bd05-6de8663236db tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 0.617777 Sep 27 09:59:32.610116 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 494/900] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:31 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:32.645834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-541155b2-dd32-486c-9e4a-820d0c51f80f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:32.659517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-541155b2-dd32-486c-9e4a-820d0c51f80f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:32.688618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-541155b2-dd32-486c-9e4a-820d0c51f80f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:32.689148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-541155b2-dd32-486c-9e4a-820d0c51f80f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:32.689638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-541155b2-dd32-486c-9e4a-820d0c51f80f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:32.690794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-541155b2-dd32-486c-9e4a-820d0c51f80f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:32.696444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-541155b2-dd32-486c-9e4a-820d0c51f80f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:32.697317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-541155b2-dd32-486c-9e4a-820d0c51f80f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:32.955362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-541155b2-dd32-486c-9e4a-820d0c51f80f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:32.969686 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-541155b2-dd32-486c-9e4a-820d0c51f80f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1476 microversion: 2.1 time: 0.348687 Sep 27 09:59:32.985724 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 495/901] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:32 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1476 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:33.001132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8c5ab5ca-6ff3-4234-acf2-d1aea7f66452 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:33.001926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8c5ab5ca-6ff3-4234-acf2-d1aea7f66452 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:33.009498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c5ab5ca-6ff3-4234-acf2-d1aea7f66452 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:33.010174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c5ab5ca-6ff3-4234-acf2-d1aea7f66452 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:33.010780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c5ab5ca-6ff3-4234-acf2-d1aea7f66452 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:33.011635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c5ab5ca-6ff3-4234-acf2-d1aea7f66452 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:33.012230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c5ab5ca-6ff3-4234-acf2-d1aea7f66452 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:33.012904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c5ab5ca-6ff3-4234-acf2-d1aea7f66452 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:33.194488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c5ab5ca-6ff3-4234-acf2-d1aea7f66452 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:33.216441 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8c5ab5ca-6ff3-4234-acf2-d1aea7f66452 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1317 microversion: 2.1 time: 0.220557 Sep 27 09:59:33.228969 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 496/902] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:32 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1317 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:33.636346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f32ef826-21f2-4348-a249-47b3d00bb40b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:33.638076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f32ef826-21f2-4348-a249-47b3d00bb40b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:33.644609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f32ef826-21f2-4348-a249-47b3d00bb40b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:33.645626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f32ef826-21f2-4348-a249-47b3d00bb40b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:33.649381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f32ef826-21f2-4348-a249-47b3d00bb40b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:33.650666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f32ef826-21f2-4348-a249-47b3d00bb40b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:33.651393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f32ef826-21f2-4348-a249-47b3d00bb40b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:33.652049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f32ef826-21f2-4348-a249-47b3d00bb40b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:33.838153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f32ef826-21f2-4348-a249-47b3d00bb40b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:33.846454 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f32ef826-21f2-4348-a249-47b3d00bb40b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 0.214677 Sep 27 09:59:33.847041 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 497/903] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:33 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:33.929103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:59:33.930373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:59:33.930373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:59:33.930942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:59:33.931744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:59:33.935581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:33.952901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:33.952901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:33.953135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:33.958388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:33.959098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:33.959648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:33.992068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Getting quotas for project 2369f5ee467643948f618bbdee837521. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:59:33.999725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Getting quotas for user 29d2f7b0e8274c769521d13930be9825 and project 2369f5ee467643948f618bbdee837521. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:59:34.002181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d5aead2-eca0-4c4d-b050-a9757c972fda tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:34.003462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4d5aead2-eca0-4c4d-b050-a9757c972fda tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:34.016585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:59:34.020731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:59:34.021962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d5aead2-eca0-4c4d-b050-a9757c972fda tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:34.024344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d5aead2-eca0-4c4d-b050-a9757c972fda tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:34.024836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d5aead2-eca0-4c4d-b050-a9757c972fda tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:34.025637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d5aead2-eca0-4c4d-b050-a9757c972fda tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:34.026103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d5aead2-eca0-4c4d-b050-a9757c972fda tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:34.026592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d5aead2-eca0-4c4d-b050-a9757c972fda tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:34.112490 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c7dadf74-a4ad-47b7-9acf-49086a37a36f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.25 time: 6.673659 Sep 27 09:59:34.118132 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 407/904] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 09:59:27 2022] POST /compute/v2.1/servers => generated 380 bytes in 6680 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Sep 27 09:59:34.129712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eb53b919-47cd-447d-9009-5eb00fc069f3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:34.131740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eb53b919-47cd-447d-9009-5eb00fc069f3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:34.254970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4d5aead2-eca0-4c4d-b050-a9757c972fda tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:34.273449 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4d5aead2-eca0-4c4d-b050-a9757c972fda tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1476 microversion: 2.1 time: 0.281411 Sep 27 09:59:34.280472 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 498/905] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:33 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1476 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:34.302513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1921a7b8-5f8e-4e4e-a181-e36b888bc70a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:34.306355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1921a7b8-5f8e-4e4e-a181-e36b888bc70a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:34.318675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1921a7b8-5f8e-4e4e-a181-e36b888bc70a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:34.319580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1921a7b8-5f8e-4e4e-a181-e36b888bc70a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:34.319735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1921a7b8-5f8e-4e4e-a181-e36b888bc70a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:34.320746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1921a7b8-5f8e-4e4e-a181-e36b888bc70a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:34.322512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1921a7b8-5f8e-4e4e-a181-e36b888bc70a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:34.323304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1921a7b8-5f8e-4e4e-a181-e36b888bc70a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:34.441927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eb53b919-47cd-447d-9009-5eb00fc069f3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:34.469477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eb53b919-47cd-447d-9009-5eb00fc069f3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:34.472747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eb53b919-47cd-447d-9009-5eb00fc069f3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:34.477876 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eb53b919-47cd-447d-9009-5eb00fc069f3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1262 microversion: 2.25 time: 0.352462 Sep 27 09:59:34.482816 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 408/906] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:34 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1262 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 09:59:34.545649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1921a7b8-5f8e-4e4e-a181-e36b888bc70a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:34.554317 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1921a7b8-5f8e-4e4e-a181-e36b888bc70a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1317 microversion: 2.1 time: 0.266593 Sep 27 09:59:34.556185 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 499/907] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:34 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1317 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:34.889937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f5975580-c3a6-45e9-bb8a-af5362662d9f tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:34.890556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f5975580-c3a6-45e9-bb8a-af5362662d9f tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:34.897164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5975580-c3a6-45e9-bb8a-af5362662d9f tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:34.897510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5975580-c3a6-45e9-bb8a-af5362662d9f tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:34.900933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5975580-c3a6-45e9-bb8a-af5362662d9f tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:34.904839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5975580-c3a6-45e9-bb8a-af5362662d9f tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:34.906127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5975580-c3a6-45e9-bb8a-af5362662d9f tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:34.906581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5975580-c3a6-45e9-bb8a-af5362662d9f tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:35.307078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e2190460-a834-4655-909a-9d44fa77bc71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:35.307712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e2190460-a834-4655-909a-9d44fa77bc71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:35.319171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2190460-a834-4655-909a-9d44fa77bc71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:35.319886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2190460-a834-4655-909a-9d44fa77bc71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:35.320604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2190460-a834-4655-909a-9d44fa77bc71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:35.322445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2190460-a834-4655-909a-9d44fa77bc71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:35.323426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2190460-a834-4655-909a-9d44fa77bc71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:35.324091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2190460-a834-4655-909a-9d44fa77bc71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:35.510173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2190460-a834-4655-909a-9d44fa77bc71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:35.520121 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e2190460-a834-4655-909a-9d44fa77bc71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1474 microversion: 2.1 time: 0.217482 Sep 27 09:59:35.520643 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 500/908] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:35 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1474 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:35.544775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:35.545749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:35.552816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:35.553981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:35.555628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:35.556615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:35.557517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:35.558111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:35.585459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f5975580-c3a6-45e9-bb8a-af5362662d9f tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:35.594401 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f5975580-c3a6-45e9-bb8a-af5362662d9f tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 0.710447 Sep 27 09:59:35.600491 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 409/909] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:34 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 717 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:35.608068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fa890c09-5201-4e90-81fe-4d03f5150e6c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:35.611277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa890c09-5201-4e90-81fe-4d03f5150e6c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:35.630585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa890c09-5201-4e90-81fe-4d03f5150e6c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:35.631449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa890c09-5201-4e90-81fe-4d03f5150e6c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:35.632124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa890c09-5201-4e90-81fe-4d03f5150e6c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:35.633642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa890c09-5201-4e90-81fe-4d03f5150e6c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:35.634801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa890c09-5201-4e90-81fe-4d03f5150e6c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:35.635571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa890c09-5201-4e90-81fe-4d03f5150e6c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:35.677309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:35.687275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:35.693570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:35.694754 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6878bd8b-6797-49c7-bf2f-72785c9b7795 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1314 microversion: 2.25 time: 0.160656 Sep 27 09:59:35.695429 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 501/910] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:35 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1314 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:36.101589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a34a9851-982d-403e-b8d9-d712d4bef459 req-3de4f535-1a4b-4b97-b85c-47b3d4b18ef1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7bc6edb-7fc4-463d-8490-8800e2922d12", "tag": "dbb7e99b-8507-4a96-bb0a-c442351f9baf"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:36.118404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a34a9851-982d-403e-b8d9-d712d4bef459 req-3de4f535-1a4b-4b97-b85c-47b3d4b18ef1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:36.119654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a34a9851-982d-403e-b8d9-d712d4bef459 req-3de4f535-1a4b-4b97-b85c-47b3d4b18ef1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:36.120729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a34a9851-982d-403e-b8d9-d712d4bef459 req-3de4f535-1a4b-4b97-b85c-47b3d4b18ef1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:36.138604 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a34a9851-982d-403e-b8d9-d712d4bef459 req-3de4f535-1a4b-4b97-b85c-47b3d4b18ef1 service nova] Creating event network-changed:dbb7e99b-8507-4a96-bb0a-c442351f9baf for instance a7bc6edb-7fc4-463d-8490-8800e2922d12 on nch1-master-1 Sep 27 09:59:36.145159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fa890c09-5201-4e90-81fe-4d03f5150e6c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:36.158582 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a34a9851-982d-403e-b8d9-d712d4bef459 req-3de4f535-1a4b-4b97-b85c-47b3d4b18ef1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062084 Sep 27 09:59:36.161457 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 502/911] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:36.172943 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fa890c09-5201-4e90-81fe-4d03f5150e6c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1317 microversion: 2.1 time: 0.571082 Sep 27 09:59:36.173894 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 410/912] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:35 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1317 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:36.553467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e7b3bae2-4c58-44e1-b85c-132e627443c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:36.555679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e7b3bae2-4c58-44e1-b85c-132e627443c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:36.576379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7b3bae2-4c58-44e1-b85c-132e627443c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:36.577373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7b3bae2-4c58-44e1-b85c-132e627443c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:36.579030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7b3bae2-4c58-44e1-b85c-132e627443c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:36.581775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7b3bae2-4c58-44e1-b85c-132e627443c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:36.587457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7b3bae2-4c58-44e1-b85c-132e627443c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:36.589128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7b3bae2-4c58-44e1-b85c-132e627443c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:36.638544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6457986a-cf56-4b32-b14c-c4abdc1a9752 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:36.655923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6457986a-cf56-4b32-b14c-c4abdc1a9752 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:36.744122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6457986a-cf56-4b32-b14c-c4abdc1a9752 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:36.745491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6457986a-cf56-4b32-b14c-c4abdc1a9752 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:36.749102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6457986a-cf56-4b32-b14c-c4abdc1a9752 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:36.752320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6457986a-cf56-4b32-b14c-c4abdc1a9752 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:36.754750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6457986a-cf56-4b32-b14c-c4abdc1a9752 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:36.761466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6457986a-cf56-4b32-b14c-c4abdc1a9752 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:36.798519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=83 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:37.185333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e7b3bae2-4c58-44e1-b85c-132e627443c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:37.199615 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e7b3bae2-4c58-44e1-b85c-132e627443c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1474 microversion: 2.1 time: 0.652507 Sep 27 09:59:37.214403 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 503/913] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:36 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1474 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:37.228593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:37.229631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:37.237627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:37.238385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:37.239030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:37.239963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:37.240568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:37.241195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:37.339541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:37.350190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:37.352766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:37.354147 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d29c997a-3f72-4ec7-b252-561da92d0b77 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1320 microversion: 2.25 time: 0.138666 Sep 27 09:59:37.368102 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 504/914] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:37 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1320 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:37.381146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d352bcbe-d273-448c-a2d1-a1b53d5789e7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:37.381940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d352bcbe-d273-448c-a2d1-a1b53d5789e7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:37.402019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d352bcbe-d273-448c-a2d1-a1b53d5789e7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:37.403351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d352bcbe-d273-448c-a2d1-a1b53d5789e7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:37.403948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d352bcbe-d273-448c-a2d1-a1b53d5789e7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:37.404898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d352bcbe-d273-448c-a2d1-a1b53d5789e7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:37.405515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d352bcbe-d273-448c-a2d1-a1b53d5789e7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:37.406086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d352bcbe-d273-448c-a2d1-a1b53d5789e7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:37.653266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d352bcbe-d273-448c-a2d1-a1b53d5789e7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:37.667224 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d352bcbe-d273-448c-a2d1-a1b53d5789e7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1317 microversion: 2.1 time: 0.297523 Sep 27 09:59:37.667828 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 505/915] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:37 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1317 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:37.677061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8d305a4b-045f-4606-98d4-c893644579ba req-c8249d44-baba-41f9-80f7-d01d0a204af4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8", "name": "network-vif-plugged", "status": "completed", "tag": "64043e29-53c7-4b1b-8d01-df27fcfbc107"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:37.695694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8d305a4b-045f-4606-98d4-c893644579ba req-c8249d44-baba-41f9-80f7-d01d0a204af4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:37.697661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8d305a4b-045f-4606-98d4-c893644579ba req-c8249d44-baba-41f9-80f7-d01d0a204af4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:37.698451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8d305a4b-045f-4606-98d4-c893644579ba req-c8249d44-baba-41f9-80f7-d01d0a204af4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:37.712261 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8d305a4b-045f-4606-98d4-c893644579ba req-c8249d44-baba-41f9-80f7-d01d0a204af4 service nova] Creating event network-vif-plugged:64043e29-53c7-4b1b-8d01-df27fcfbc107 for instance 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 on nch1-master-1 Sep 27 09:59:37.721684 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8d305a4b-045f-4606-98d4-c893644579ba req-c8249d44-baba-41f9-80f7-d01d0a204af4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050341 Sep 27 09:59:37.722837 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 506/916] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:38.080698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6457986a-cf56-4b32-b14c-c4abdc1a9752 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:38.095991 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6457986a-cf56-4b32-b14c-c4abdc1a9752 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 1.468915 Sep 27 09:59:38.097430 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 411/917] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:36 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 1472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:38.230949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d505262f-2c6b-401b-8f21-ba22106993df tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:38.231626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d505262f-2c6b-401b-8f21-ba22106993df tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:38.239984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d505262f-2c6b-401b-8f21-ba22106993df tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:38.240479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d505262f-2c6b-401b-8f21-ba22106993df tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:38.241126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d505262f-2c6b-401b-8f21-ba22106993df tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:38.241927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d505262f-2c6b-401b-8f21-ba22106993df tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:38.242595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d505262f-2c6b-401b-8f21-ba22106993df tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:38.243024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d505262f-2c6b-401b-8f21-ba22106993df tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:38.386551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:38.387624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:38.395622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:38.396421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:38.397245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:38.398183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:38.398810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:38.399431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:38.550277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d505262f-2c6b-401b-8f21-ba22106993df tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:38.568558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:38.587699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:38.590773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:38.592189 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d505262f-2c6b-401b-8f21-ba22106993df tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1471 microversion: 2.1 time: 0.365763 Sep 27 09:59:38.592723 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e054e5cf-4c72-434f-a864-3fe22a61532f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1320 microversion: 2.25 time: 0.211367 Sep 27 09:59:38.594219 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 412/918] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:38 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1320 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 09:59:38.622979 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 507/919] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:38 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1471 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:38.657325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9fa5e0db-df54-4e3f-af99-2a479a516591 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:38.658615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9fa5e0db-df54-4e3f-af99-2a479a516591 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:38.681497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fa5e0db-df54-4e3f-af99-2a479a516591 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:38.684275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fa5e0db-df54-4e3f-af99-2a479a516591 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:38.703198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-65510b61-7a8a-4384-bf2e-e969333f4451 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:38.704922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fa5e0db-df54-4e3f-af99-2a479a516591 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:38.706461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-65510b61-7a8a-4384-bf2e-e969333f4451 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:38.708164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fa5e0db-df54-4e3f-af99-2a479a516591 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:38.716022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fa5e0db-df54-4e3f-af99-2a479a516591 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:38.718717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fa5e0db-df54-4e3f-af99-2a479a516591 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:38.739752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65510b61-7a8a-4384-bf2e-e969333f4451 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:38.743694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65510b61-7a8a-4384-bf2e-e969333f4451 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:38.745553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65510b61-7a8a-4384-bf2e-e969333f4451 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:38.748900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65510b61-7a8a-4384-bf2e-e969333f4451 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:38.749929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65510b61-7a8a-4384-bf2e-e969333f4451 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:38.750598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65510b61-7a8a-4384-bf2e-e969333f4451 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:38.925139 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9fa5e0db-df54-4e3f-af99-2a479a516591 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "POST /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4/action" status: 204 len: 0 microversion: 2.1 time: 0.275879 Sep 27 09:59:38.925784 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 413/920] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:59:38 2022] POST /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4/action => generated 0 bytes in 280 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:59:38.937276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7529a10f-ba40-495a-a9ec-c6030c78c9c2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:38.938319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7529a10f-ba40-495a-a9ec-c6030c78c9c2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:38.946251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7529a10f-ba40-495a-a9ec-c6030c78c9c2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:38.947569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7529a10f-ba40-495a-a9ec-c6030c78c9c2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:38.950116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7529a10f-ba40-495a-a9ec-c6030c78c9c2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:38.951183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7529a10f-ba40-495a-a9ec-c6030c78c9c2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:38.951927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7529a10f-ba40-495a-a9ec-c6030c78c9c2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:38.952596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7529a10f-ba40-495a-a9ec-c6030c78c9c2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:39.027642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-65510b61-7a8a-4384-bf2e-e969333f4451 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:39.036296 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-65510b61-7a8a-4384-bf2e-e969333f4451 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1317 microversion: 2.1 time: 0.347601 Sep 27 09:59:39.037633 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 508/921] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:38 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1317 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:39.255320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesNegativeTestJSON-server-1931334902", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "9493ab4b-8945-44ef-ae23-02a42d38c87c"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:39.353143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] validate_networks() for [('9493ab4b-8945-44ef-ae23-02a42d38c87c', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:59:39.872074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:59:39.873440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:59:39.874133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:59:39.874910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:59:39.875676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:59:39.885966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:39.886594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:39.887732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:39.888704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:39.889296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:39.890022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:39.924803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Getting quotas for project e9bd76ccd7824e989252951e381a17cd. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:59:39.932163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Getting quotas for user c815e82b67e8436b9bd4a06fe2d03b49 and project e9bd76ccd7824e989252951e381a17cd. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:59:39.952864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:59:39.957109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:59:40.030762 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f3db8f78-b06b-4049-9f71-d1f76c835730 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.988311 Sep 27 09:59:40.032180 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 509/922] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:59:39 2022] POST /compute/v2.1/servers => generated 380 bytes in 994 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:59:40.044842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-22d056af-8f74-461b-9c8c-5135af85226d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:40.045830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-22d056af-8f74-461b-9c8c-5135af85226d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:40.060200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22d056af-8f74-461b-9c8c-5135af85226d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:40.061013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22d056af-8f74-461b-9c8c-5135af85226d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:40.061710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22d056af-8f74-461b-9c8c-5135af85226d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:40.063277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22d056af-8f74-461b-9c8c-5135af85226d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:40.063964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22d056af-8f74-461b-9c8c-5135af85226d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:40.064612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22d056af-8f74-461b-9c8c-5135af85226d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:40.319185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-22d056af-8f74-461b-9c8c-5135af85226d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:40.333568 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-22d056af-8f74-461b-9c8c-5135af85226d tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 0.299917 Sep 27 09:59:40.348974 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 510/923] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:40 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:40.355841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:40.356798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:40.365987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:40.368944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:40.369926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:40.370909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:40.371789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:40.372708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:40.510902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7529a10f-ba40-495a-a9ec-c6030c78c9c2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:40.520970 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7529a10f-ba40-495a-a9ec-c6030c78c9c2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1471 microversion: 2.1 time: 1.590586 Sep 27 09:59:40.523315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:40.526056 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 414/924] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:38 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1471 bytes in 1596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:40.539063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3d686898-19e0-4ac0-ac1f-64179bbf6fce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:40.539713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3d686898-19e0-4ac0-ac1f-64179bbf6fce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:40.555376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:40.559146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:40.563882 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-59d2d236-4b9f-4ebe-b523-f977c574863f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1320 microversion: 2.25 time: 0.212214 Sep 27 09:59:40.581584 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 511/925] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:40 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1320 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:40.600641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d03bcfc-82bd-404b-9a39-d1341dfa141c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:40.602285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d03bcfc-82bd-404b-9a39-d1341dfa141c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:40.619445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d03bcfc-82bd-404b-9a39-d1341dfa141c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:40.620004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d03bcfc-82bd-404b-9a39-d1341dfa141c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:40.620481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d03bcfc-82bd-404b-9a39-d1341dfa141c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:40.621414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d03bcfc-82bd-404b-9a39-d1341dfa141c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:40.622261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d03bcfc-82bd-404b-9a39-d1341dfa141c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:40.622667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d03bcfc-82bd-404b-9a39-d1341dfa141c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:40.648034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:40.834857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7d03bcfc-82bd-404b-9a39-d1341dfa141c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:40.845992 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d03bcfc-82bd-404b-9a39-d1341dfa141c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1467 microversion: 2.1 time: 0.262566 Sep 27 09:59:40.847008 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 512/926] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:40 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1467 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:40.872935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2ccad7a5-0b54-4262-83b4-374a91912e6e tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:40.873786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2ccad7a5-0b54-4262-83b4-374a91912e6e tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:40.888121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ccad7a5-0b54-4262-83b4-374a91912e6e tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:40.889331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ccad7a5-0b54-4262-83b4-374a91912e6e tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:40.890106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ccad7a5-0b54-4262-83b4-374a91912e6e tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:40.891060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ccad7a5-0b54-4262-83b4-374a91912e6e tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:40.891682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ccad7a5-0b54-4262-83b4-374a91912e6e tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:40.892354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ccad7a5-0b54-4262-83b4-374a91912e6e tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:40.931673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3d686898-19e0-4ac0-ac1f-64179bbf6fce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:40.950555 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3d686898-19e0-4ac0-ac1f-64179bbf6fce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1216 microversion: 2.1 time: 0.422776 Sep 27 09:59:40.965069 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 415/927] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:40 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1216 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:41.111176 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2ccad7a5-0b54-4262-83b4-374a91912e6e tempest-ServersOnMultiNodesTest-1386785862 tempest-ServersOnMultiNodesTest-1386785862-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1618 microversion: 2.1 time: 0.243844 Sep 27 09:59:41.112347 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 513/928] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:40 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1618 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:41.138828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-122c6bf7-02aa-4ee8-bd16-e9ba0203eaca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:41.140408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-122c6bf7-02aa-4ee8-bd16-e9ba0203eaca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:41.147836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122c6bf7-02aa-4ee8-bd16-e9ba0203eaca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:41.148704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122c6bf7-02aa-4ee8-bd16-e9ba0203eaca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:41.149477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122c6bf7-02aa-4ee8-bd16-e9ba0203eaca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:41.150283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122c6bf7-02aa-4ee8-bd16-e9ba0203eaca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:41.154376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122c6bf7-02aa-4ee8-bd16-e9ba0203eaca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:41.155238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122c6bf7-02aa-4ee8-bd16-e9ba0203eaca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:41.241709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-122c6bf7-02aa-4ee8-bd16-e9ba0203eaca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:59:41.358486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-43343cd8-9b23-4553-ac82-7273eef39bc5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:41.359769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-43343cd8-9b23-4553-ac82-7273eef39bc5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:41.382288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43343cd8-9b23-4553-ac82-7273eef39bc5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:41.385766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43343cd8-9b23-4553-ac82-7273eef39bc5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:41.391056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43343cd8-9b23-4553-ac82-7273eef39bc5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:41.393212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43343cd8-9b23-4553-ac82-7273eef39bc5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:41.395133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43343cd8-9b23-4553-ac82-7273eef39bc5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:41.397443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43343cd8-9b23-4553-ac82-7273eef39bc5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:41.466445 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-122c6bf7-02aa-4ee8-bd16-e9ba0203eaca tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "DELETE /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 204 len: 0 microversion: 2.1 time: 0.331357 Sep 27 09:59:41.467548 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 416/929] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:59:41 2022] DELETE /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:59:41.477492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5470a86e-3113-4edc-a57a-dd818ad83ef8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:41.481763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5470a86e-3113-4edc-a57a-dd818ad83ef8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:41.501983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5470a86e-3113-4edc-a57a-dd818ad83ef8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:41.502659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5470a86e-3113-4edc-a57a-dd818ad83ef8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:41.512174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5470a86e-3113-4edc-a57a-dd818ad83ef8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:41.514428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5470a86e-3113-4edc-a57a-dd818ad83ef8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:41.515392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5470a86e-3113-4edc-a57a-dd818ad83ef8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:41.517583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5470a86e-3113-4edc-a57a-dd818ad83ef8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:41.675890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-43343cd8-9b23-4553-ac82-7273eef39bc5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:41.710229 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-43343cd8-9b23-4553-ac82-7273eef39bc5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 0.355552 Sep 27 09:59:41.712108 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 514/930] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:41 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:41.719017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed4247b0-53d0-4dad-be03-73abed0f7109 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:41.720511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed4247b0-53d0-4dad-be03-73abed0f7109 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:41.729795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed4247b0-53d0-4dad-be03-73abed0f7109 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:41.730967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed4247b0-53d0-4dad-be03-73abed0f7109 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:41.732719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed4247b0-53d0-4dad-be03-73abed0f7109 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:41.734480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed4247b0-53d0-4dad-be03-73abed0f7109 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:41.736115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed4247b0-53d0-4dad-be03-73abed0f7109 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:41.738133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed4247b0-53d0-4dad-be03-73abed0f7109 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:41.942974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ed4247b0-53d0-4dad-be03-73abed0f7109 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:41.953955 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed4247b0-53d0-4dad-be03-73abed0f7109 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1471 microversion: 2.1 time: 0.240304 Sep 27 09:59:41.954966 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 515/931] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:41 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1471 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:41.967399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:41.968882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:41.993455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:41.994008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:41.994745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:41.995927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:41.996574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:41.997194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:42.233941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:42.244305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:42.250852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:42.252563 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c532db0b-db76-40f3-9cea-1853595008ff tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.296335 Sep 27 09:59:42.255669 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 516/932] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:41 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:42.279625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cab8f433-f058-4176-bddc-c16619413be4 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:42.280687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cab8f433-f058-4176-bddc-c16619413be4 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:42.311099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cab8f433-f058-4176-bddc-c16619413be4 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:42.312096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cab8f433-f058-4176-bddc-c16619413be4 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:42.312822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cab8f433-f058-4176-bddc-c16619413be4 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:42.314987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cab8f433-f058-4176-bddc-c16619413be4 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:42.315683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cab8f433-f058-4176-bddc-c16619413be4 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:42.316262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cab8f433-f058-4176-bddc-c16619413be4 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:42.669155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cab8f433-f058-4176-bddc-c16619413be4 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:42.679775 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cab8f433-f058-4176-bddc-c16619413be4 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1276 microversion: 2.1 time: 0.420062 Sep 27 09:59:42.696908 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 517/933] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:42 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1276 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:42.702273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7d85d1d4-4a3e-467a-8949-38a0c3941bee req-42af8086-ee2a-4ff8-9c46-710bfb112b57 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "48f61d6b-3985-49a2-9eb9-dd250711cdf4", "name": "network-vif-plugged", "status": "completed", "tag": "04d6d3fb-4795-41f4-a54d-61772b8a10a8"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:42.714265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7d85d1d4-4a3e-467a-8949-38a0c3941bee req-42af8086-ee2a-4ff8-9c46-710bfb112b57 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:42.715031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7d85d1d4-4a3e-467a-8949-38a0c3941bee req-42af8086-ee2a-4ff8-9c46-710bfb112b57 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:42.715651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7d85d1d4-4a3e-467a-8949-38a0c3941bee req-42af8086-ee2a-4ff8-9c46-710bfb112b57 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:42.733122 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7d85d1d4-4a3e-467a-8949-38a0c3941bee req-42af8086-ee2a-4ff8-9c46-710bfb112b57 service nova] Creating event network-vif-plugged:04d6d3fb-4795-41f4-a54d-61772b8a10a8 for instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 on nch1-master-1 Sep 27 09:59:42.741663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-7d85d1d4-4a3e-467a-8949-38a0c3941bee req-42af8086-ee2a-4ff8-9c46-710bfb112b57 service nova] Instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 09:59:42.759148 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7d85d1d4-4a3e-467a-8949-38a0c3941bee req-42af8086-ee2a-4ff8-9c46-710bfb112b57 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060414 Sep 27 09:59:42.760081 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 518/934] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:42.767084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8bd3f3ef-9e9c-4913-b065-7b7d544356d5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:42.768178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8bd3f3ef-9e9c-4913-b065-7b7d544356d5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:42.776201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bd3f3ef-9e9c-4913-b065-7b7d544356d5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:42.776785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bd3f3ef-9e9c-4913-b065-7b7d544356d5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:42.777206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bd3f3ef-9e9c-4913-b065-7b7d544356d5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:42.777972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bd3f3ef-9e9c-4913-b065-7b7d544356d5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:42.778434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bd3f3ef-9e9c-4913-b065-7b7d544356d5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:42.778864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bd3f3ef-9e9c-4913-b065-7b7d544356d5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:43.084353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5470a86e-3113-4edc-a57a-dd818ad83ef8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:43.130930 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5470a86e-3113-4edc-a57a-dd818ad83ef8 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1473 microversion: 2.1 time: 1.651389 Sep 27 09:59:43.146410 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 417/935] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:41 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1473 bytes in 1675 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:43.164063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b797aa3e-f9c8-4799-b3fe-f70494337091 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:43.171143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b797aa3e-f9c8-4799-b3fe-f70494337091 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:43.189909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b797aa3e-f9c8-4799-b3fe-f70494337091 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:43.192368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b797aa3e-f9c8-4799-b3fe-f70494337091 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:43.193610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b797aa3e-f9c8-4799-b3fe-f70494337091 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:43.195801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b797aa3e-f9c8-4799-b3fe-f70494337091 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:43.197660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b797aa3e-f9c8-4799-b3fe-f70494337091 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:43.198265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b797aa3e-f9c8-4799-b3fe-f70494337091 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:43.265325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8bd3f3ef-9e9c-4913-b065-7b7d544356d5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:43.290662 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8bd3f3ef-9e9c-4913-b065-7b7d544356d5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 0.528899 Sep 27 09:59:43.291890 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 519/936] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:42 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:43.297011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:43.305325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:43.319412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:43.320292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:43.321300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:43.322462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:43.324285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:43.325105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:43.703831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:43.721471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:43.723568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:43.729208 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8326d599-e6c3-42e0-b42c-aea0fbec7d3e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.434742 Sep 27 09:59:43.737328 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 520/937] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:43 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:43.753553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-032d7f6c-5dce-4326-bac2-b535a0f799ce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:43.754527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-032d7f6c-5dce-4326-bac2-b535a0f799ce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:43.764704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032d7f6c-5dce-4326-bac2-b535a0f799ce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:43.765487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032d7f6c-5dce-4326-bac2-b535a0f799ce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:43.768007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032d7f6c-5dce-4326-bac2-b535a0f799ce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:43.769060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032d7f6c-5dce-4326-bac2-b535a0f799ce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:43.770560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032d7f6c-5dce-4326-bac2-b535a0f799ce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:43.772131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032d7f6c-5dce-4326-bac2-b535a0f799ce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:43.852633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b797aa3e-f9c8-4799-b3fe-f70494337091 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:43.888328 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b797aa3e-f9c8-4799-b3fe-f70494337091 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1471 microversion: 2.1 time: 0.740438 Sep 27 09:59:43.889395 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 418/938] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:43 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1471 bytes in 742 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:44.086209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-032d7f6c-5dce-4326-bac2-b535a0f799ce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:44.109548 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-032d7f6c-5dce-4326-bac2-b535a0f799ce tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1274 microversion: 2.1 time: 0.370483 Sep 27 09:59:44.124788 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 521/939] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:43 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1274 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:44.171831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5efeb76e-a4b2-41d6-a6ac-8bf45f507c56 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:44.172465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5efeb76e-a4b2-41d6-a6ac-8bf45f507c56 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:44.180776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:44.196277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5efeb76e-a4b2-41d6-a6ac-8bf45f507c56 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:44.198567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5efeb76e-a4b2-41d6-a6ac-8bf45f507c56 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:44.201523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5efeb76e-a4b2-41d6-a6ac-8bf45f507c56 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:44.202537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5efeb76e-a4b2-41d6-a6ac-8bf45f507c56 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:44.205478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5efeb76e-a4b2-41d6-a6ac-8bf45f507c56 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:44.206171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5efeb76e-a4b2-41d6-a6ac-8bf45f507c56 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:44.318321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23985a9c-6306-4012-ae8e-b171f7791d34 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:44.319349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-23985a9c-6306-4012-ae8e-b171f7791d34 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:44.345770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23985a9c-6306-4012-ae8e-b171f7791d34 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:44.346377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23985a9c-6306-4012-ae8e-b171f7791d34 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:44.346827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23985a9c-6306-4012-ae8e-b171f7791d34 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:44.347574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23985a9c-6306-4012-ae8e-b171f7791d34 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:44.348035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23985a9c-6306-4012-ae8e-b171f7791d34 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:44.348624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23985a9c-6306-4012-ae8e-b171f7791d34 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:44.462204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5efeb76e-a4b2-41d6-a6ac-8bf45f507c56 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:44.488045 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5efeb76e-a4b2-41d6-a6ac-8bf45f507c56 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 200 len: 1431 microversion: 2.1 time: 0.330220 Sep 27 09:59:44.506581 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 419/940] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:44 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 1431 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:44.796828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:44.812110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:44.881602 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 09:59:44.882108 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 09:59:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 (ip 10.222.0.26) !!! Sep 27 09:59:44.912503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:44.928213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:44.931840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:44.940786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:44.944026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:44.944854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:45.533167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:45.544143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:45.553635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:45.554999 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3aae80aa-7e28-4d16-af18-194334d365aa tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.790884 Sep 27 09:59:45.569193 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 420/941] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:44 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 807 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:45.581103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-4384e7ce-f257-4558-9e5c-8a1f84956088 req-6e2472a5-63e4-478b-96e5-c58451d26a39 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8", "name": "network-vif-deleted", "tag": "64043e29-53c7-4b1b-8d01-df27fcfbc107"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:45.596059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4384e7ce-f257-4558-9e5c-8a1f84956088 req-6e2472a5-63e4-478b-96e5-c58451d26a39 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:45.596905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4384e7ce-f257-4558-9e5c-8a1f84956088 req-6e2472a5-63e4-478b-96e5-c58451d26a39 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:45.597568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4384e7ce-f257-4558-9e5c-8a1f84956088 req-6e2472a5-63e4-478b-96e5-c58451d26a39 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:45.631114 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-4384e7ce-f257-4558-9e5c-8a1f84956088 req-6e2472a5-63e4-478b-96e5-c58451d26a39 service nova] Creating event network-vif-deleted:64043e29-53c7-4b1b-8d01-df27fcfbc107 for instance 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 on nch1-master-1 Sep 27 09:59:45.640205 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-4384e7ce-f257-4558-9e5c-8a1f84956088 req-6e2472a5-63e4-478b-96e5-c58451d26a39 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069206 Sep 27 09:59:45.644463 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 421/942] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:45.658150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-687695b5-a8e7-4a27-addf-74d2dc2fba7c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:45.659114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-687695b5-a8e7-4a27-addf-74d2dc2fba7c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:45.674724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-687695b5-a8e7-4a27-addf-74d2dc2fba7c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:45.675660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-687695b5-a8e7-4a27-addf-74d2dc2fba7c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:45.676290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-687695b5-a8e7-4a27-addf-74d2dc2fba7c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:45.677790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-687695b5-a8e7-4a27-addf-74d2dc2fba7c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:45.681048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-687695b5-a8e7-4a27-addf-74d2dc2fba7c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:45.681712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-687695b5-a8e7-4a27-addf-74d2dc2fba7c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:45.818983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-23985a9c-6306-4012-ae8e-b171f7791d34 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:45.829087 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-23985a9c-6306-4012-ae8e-b171f7791d34 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 1.516428 Sep 27 09:59:45.834486 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 522/943] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:44 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 1523 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:45.846790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2fa5255a-ac4e-41ef-bd1e-07d92d2d7edd tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:45.848326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2fa5255a-ac4e-41ef-bd1e-07d92d2d7edd tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:45.864284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fa5255a-ac4e-41ef-bd1e-07d92d2d7edd tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:45.866293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fa5255a-ac4e-41ef-bd1e-07d92d2d7edd tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:45.867062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fa5255a-ac4e-41ef-bd1e-07d92d2d7edd tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:45.875069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fa5255a-ac4e-41ef-bd1e-07d92d2d7edd tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:45.879847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fa5255a-ac4e-41ef-bd1e-07d92d2d7edd tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:45.880495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fa5255a-ac4e-41ef-bd1e-07d92d2d7edd tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:46.803534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-687695b5-a8e7-4a27-addf-74d2dc2fba7c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:46.826427 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-687695b5-a8e7-4a27-addf-74d2dc2fba7c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1463 microversion: 2.1 time: 1.179547 Sep 27 09:59:46.828092 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 422/944] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:45 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1463 bytes in 1183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:46.840453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:46.842653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:46.858977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:46.861078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:46.862301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:46.863674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:46.866575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:46.867433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:46.906508 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] HTTP exception thrown: Instance 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 could not be found. Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Request method failure captured: Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 HTTP/1.1 Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjuzA8awi02tVpJBbCxyCTwC-N-auCdITAAm8nxhyCwHUSTrFjjx6tbQkA83IeLeCJH2BD7omoS-ML8vYESEbbQcIu0WhsX0gHiWVx2nVId80d2oExNsA2vGHGUS2rPtxB3q_M7eSz3mw7rO-vyGP61qyPfw7K0p8tFBuwGQZNzBLo Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 09:59:46.908209 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c1ac7b64c68e4896b2c631baacdbe2be"}]}] Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersOnMultiNodesTest-536305491 Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 00423eb1e4c24763ae0a546225ac1dc7 Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 could not be found. Sep 27 09:59:46.909984 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8'} Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 could not be found. Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:59:46.910934 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 could not be found. Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 could not be found. Sep 27 09:59:46.912536 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 09:59:46.921783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Returning 404 to user: Instance 26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:59:46.926221 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d3aa223c-b146-44af-bae2-40c274a53b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8" status: 404 len: 111 microversion: 2.1 time: 0.095060 Sep 27 09:59:46.929039 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 423/945] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:46 2022] GET /compute/v2.1/servers/26ef5f4f-7554-4e7b-82ec-79d2d8ad55c8 => generated 111 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:59:46.943554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:46.944752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:46.969585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:46.970092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:46.972775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:46.973722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:46.974292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:46.976781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:47.147638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2fa5255a-ac4e-41ef-bd1e-07d92d2d7edd tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:47.161258 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2fa5255a-ac4e-41ef-bd1e-07d92d2d7edd tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 1.325365 Sep 27 09:59:47.167163 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 523/946] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:45 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 1332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:47.172612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b7db45fa-0420-4811-8a8c-07a8fde554d2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:47.177902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b7db45fa-0420-4811-8a8c-07a8fde554d2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:47.196210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7db45fa-0420-4811-8a8c-07a8fde554d2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:47.201181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7db45fa-0420-4811-8a8c-07a8fde554d2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:47.201641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7db45fa-0420-4811-8a8c-07a8fde554d2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:47.202462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7db45fa-0420-4811-8a8c-07a8fde554d2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:47.202931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7db45fa-0420-4811-8a8c-07a8fde554d2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:47.206501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7db45fa-0420-4811-8a8c-07a8fde554d2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:47.414764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:47.429966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:47.434081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:47.435276 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-57296e3e-6564-4d27-baed-2ac01d4482bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.504268 Sep 27 09:59:47.440176 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 424/947] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:46 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:47.451805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9fb672f5-a638-42ba-b011-bfcf3c4e17d7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:47.452731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9fb672f5-a638-42ba-b011-bfcf3c4e17d7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:47.469844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fb672f5-a638-42ba-b011-bfcf3c4e17d7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:47.470670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fb672f5-a638-42ba-b011-bfcf3c4e17d7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:47.471297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fb672f5-a638-42ba-b011-bfcf3c4e17d7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:47.472220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fb672f5-a638-42ba-b011-bfcf3c4e17d7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:47.472901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fb672f5-a638-42ba-b011-bfcf3c4e17d7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:47.473682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fb672f5-a638-42ba-b011-bfcf3c4e17d7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:47.658617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b7db45fa-0420-4811-8a8c-07a8fde554d2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:47.696534 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b7db45fa-0420-4811-8a8c-07a8fde554d2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1463 microversion: 2.1 time: 0.518819 Sep 27 09:59:47.699924 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 524/948] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:47 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1463 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:47.719975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5e217c4d-8839-4c89-b51e-54fc2f6af571 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:47.723546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5e217c4d-8839-4c89-b51e-54fc2f6af571 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:47.753969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e217c4d-8839-4c89-b51e-54fc2f6af571 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:47.754640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e217c4d-8839-4c89-b51e-54fc2f6af571 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:47.755212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e217c4d-8839-4c89-b51e-54fc2f6af571 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:47.757095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e217c4d-8839-4c89-b51e-54fc2f6af571 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:47.758187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e217c4d-8839-4c89-b51e-54fc2f6af571 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:47.759102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e217c4d-8839-4c89-b51e-54fc2f6af571 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:47.810705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5e217c4d-8839-4c89-b51e-54fc2f6af571 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:59:48.029531 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5e217c4d-8839-4c89-b51e-54fc2f6af571 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "DELETE /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 204 len: 0 microversion: 2.1 time: 0.326648 Sep 27 09:59:48.033033 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 525/949] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:59:47 2022] DELETE /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:59:48.050205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f4eee511-8560-40ea-8ed7-17ff54c6d371 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Action: 'create', calling method: .version_select at 0x7f7f6392b790>, body: {"keypair": {"name": "tempest-keypair-1595116500", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNrNaQLcUQyJ808z7zYmksxwDjz0lvVfB+c8xzrqKB1P9gwTpeJWSIxuYr6D2Pg7WIeYfNruldX4kRVo02ExzLNNvkDMLOvQfVXpIA6JVS6UKTLbGMDy2M87yRlsUsoa9w=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:48.092530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f4eee511-8560-40ea-8ed7-17ff54c6d371 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for project e79eae6310354ca58046e59dfac23009. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:59:48.108637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f4eee511-8560-40ea-8ed7-17ff54c6d371 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for user 9544cdb0392f416197a586d99e8a3210 and project e79eae6310354ca58046e59dfac23009. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:59:48.170790 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f4eee511-8560-40ea-8ed7-17ff54c6d371 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.135184 Sep 27 09:59:48.171470 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 526/950] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 09:59:48 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:48.182983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-027c7cf0-cf3f-429d-8ded-8cb83007652b req-b4fe6c9d-e469-4460-beb2-b5abacdfd693 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f1ff4a24-1251-4827-955f-159c665a1d57", "tag": "9c2668a6-7fb1-48ea-8f26-e6c77b4e01f9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:48.207965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-027c7cf0-cf3f-429d-8ded-8cb83007652b req-b4fe6c9d-e469-4460-beb2-b5abacdfd693 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:48.208459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-027c7cf0-cf3f-429d-8ded-8cb83007652b req-b4fe6c9d-e469-4460-beb2-b5abacdfd693 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:48.208926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-027c7cf0-cf3f-429d-8ded-8cb83007652b req-b4fe6c9d-e469-4460-beb2-b5abacdfd693 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:48.237761 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-027c7cf0-cf3f-429d-8ded-8cb83007652b req-b4fe6c9d-e469-4460-beb2-b5abacdfd693 service nova] Creating event network-changed:9c2668a6-7fb1-48ea-8f26-e6c77b4e01f9 for instance f1ff4a24-1251-4827-955f-159c665a1d57 on nch2-master-1 Sep 27 09:59:48.251610 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-027c7cf0-cf3f-429d-8ded-8cb83007652b req-b4fe6c9d-e469-4460-beb2-b5abacdfd693 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.078693 Sep 27 09:59:48.252352 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 527/951] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:48.267733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3d562b4e-82ff-4966-81e5-4c6fb7c64c27 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:48.268537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3d562b4e-82ff-4966-81e5-4c6fb7c64c27 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:48.290173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d562b4e-82ff-4966-81e5-4c6fb7c64c27 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:48.290876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d562b4e-82ff-4966-81e5-4c6fb7c64c27 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:48.292799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d562b4e-82ff-4966-81e5-4c6fb7c64c27 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:48.293480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d562b4e-82ff-4966-81e5-4c6fb7c64c27 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:48.294025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d562b4e-82ff-4966-81e5-4c6fb7c64c27 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:48.297370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d562b4e-82ff-4966-81e5-4c6fb7c64c27 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:48.509181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9fb672f5-a638-42ba-b011-bfcf3c4e17d7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:48.524278 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9fb672f5-a638-42ba-b011-bfcf3c4e17d7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 1.078816 Sep 27 09:59:48.526045 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 425/952] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:47 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 1085 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:48.535033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8b7df4e6-0948-46fa-9f68-e2280f985656 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:48.535669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8b7df4e6-0948-46fa-9f68-e2280f985656 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:48.556754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b7df4e6-0948-46fa-9f68-e2280f985656 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:48.557290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b7df4e6-0948-46fa-9f68-e2280f985656 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:48.557742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b7df4e6-0948-46fa-9f68-e2280f985656 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:48.562824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b7df4e6-0948-46fa-9f68-e2280f985656 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:48.564306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b7df4e6-0948-46fa-9f68-e2280f985656 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:48.564805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b7df4e6-0948-46fa-9f68-e2280f985656 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:48.932299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8b7df4e6-0948-46fa-9f68-e2280f985656 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:48.948026 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8b7df4e6-0948-46fa-9f68-e2280f985656 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 0.414684 Sep 27 09:59:48.948388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3d562b4e-82ff-4966-81e5-4c6fb7c64c27 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:48.949392 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 426/953] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:48 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:48.968601 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3d562b4e-82ff-4966-81e5-4c6fb7c64c27 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1471 microversion: 2.1 time: 0.706087 Sep 27 09:59:48.969891 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 528/954] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:48 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1471 bytes in 717 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:48.980135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a561df1b-3463-4956-9214-ecc29b864784 req-df32e62f-8987-470c-aac1-cf9602ec6c3b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "24be6165-1a70-40c5-beb7-67dd67933a85", "tag": "d97cf212-d664-46ca-b7be-65423e5322f3"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:48.987878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:48.990020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:48.993928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:49.003085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:49.003668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:49.004215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:49.005156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:49.005633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:49.006061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:49.009308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a561df1b-3463-4956-9214-ecc29b864784 req-df32e62f-8987-470c-aac1-cf9602ec6c3b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:49.010590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a561df1b-3463-4956-9214-ecc29b864784 req-df32e62f-8987-470c-aac1-cf9602ec6c3b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:49.011227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a561df1b-3463-4956-9214-ecc29b864784 req-df32e62f-8987-470c-aac1-cf9602ec6c3b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:49.016381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:49.048224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:49.085477 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a561df1b-3463-4956-9214-ecc29b864784 req-df32e62f-8987-470c-aac1-cf9602ec6c3b service nova] Creating event network-changed:d97cf212-d664-46ca-b7be-65423e5322f3 for instance 24be6165-1a70-40c5-beb7-67dd67933a85 on nch2-master-1 Sep 27 09:59:49.108226 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a561df1b-3463-4956-9214-ecc29b864784 req-df32e62f-8987-470c-aac1-cf9602ec6c3b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.143365 Sep 27 09:59:49.110854 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 427/955] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:49.380238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:49.392685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:49.395491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:49.401154 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-981d27b0-2dbf-4a69-9ec6-053d1dee3f45 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.417817 Sep 27 09:59:49.402571 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 529/956] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:48 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:49.480200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-214893674", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "66dfc7b2-4ec7-40ad-b073-f4df786a999e"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:49.553220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a86a6524-1ea5-442c-b2b8-4e9b0044b8b7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:49.554162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a86a6524-1ea5-442c-b2b8-4e9b0044b8b7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:49.580148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a86a6524-1ea5-442c-b2b8-4e9b0044b8b7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:49.580713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a86a6524-1ea5-442c-b2b8-4e9b0044b8b7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:49.588167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a86a6524-1ea5-442c-b2b8-4e9b0044b8b7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:49.590366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a86a6524-1ea5-442c-b2b8-4e9b0044b8b7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:49.595234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a86a6524-1ea5-442c-b2b8-4e9b0044b8b7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:49.595935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] validate_networks() for [('66dfc7b2-4ec7-40ad-b073-f4df786a999e', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:59:49.596423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a86a6524-1ea5-442c-b2b8-4e9b0044b8b7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:49.903844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:50.215924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a86a6524-1ea5-442c-b2b8-4e9b0044b8b7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:50.236932 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a86a6524-1ea5-442c-b2b8-4e9b0044b8b7 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 0.687028 Sep 27 09:59:50.241021 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 530/957] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:49 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 694 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:50.249903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-346f9fed-5637-40a4-96e4-3cad4cc13f9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:50.254580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-346f9fed-5637-40a4-96e4-3cad4cc13f9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:50.266099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346f9fed-5637-40a4-96e4-3cad4cc13f9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:50.267203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346f9fed-5637-40a4-96e4-3cad4cc13f9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:50.268203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346f9fed-5637-40a4-96e4-3cad4cc13f9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:50.269838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346f9fed-5637-40a4-96e4-3cad4cc13f9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:50.270769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346f9fed-5637-40a4-96e4-3cad4cc13f9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:50.271542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346f9fed-5637-40a4-96e4-3cad4cc13f9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:50.732839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-346f9fed-5637-40a4-96e4-3cad4cc13f9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:50.744412 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-346f9fed-5637-40a4-96e4-3cad4cc13f9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 0.501541 Sep 27 09:59:50.750904 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 531/958] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:50 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:50.764426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-db856c66-511d-4099-a96e-7bc72ef0fe8c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:50.765472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-db856c66-511d-4099-a96e-7bc72ef0fe8c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:50.782603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db856c66-511d-4099-a96e-7bc72ef0fe8c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:50.783657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db856c66-511d-4099-a96e-7bc72ef0fe8c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:50.784312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db856c66-511d-4099-a96e-7bc72ef0fe8c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:50.795132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db856c66-511d-4099-a96e-7bc72ef0fe8c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:50.796262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db856c66-511d-4099-a96e-7bc72ef0fe8c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:50.796980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db856c66-511d-4099-a96e-7bc72ef0fe8c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:50.936378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=10 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:51.166318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-db856c66-511d-4099-a96e-7bc72ef0fe8c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:51.174691 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-db856c66-511d-4099-a96e-7bc72ef0fe8c tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1471 microversion: 2.1 time: 0.422351 Sep 27 09:59:51.175791 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 532/959] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:50 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1471 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:51.181934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:51.182867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:51.208471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:51.211248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:51.212097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:51.213828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:51.219495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:51.220935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:51.496433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:51.513720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:51.522240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:51.524578 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f61b8d18-20db-4b81-b602-fa0958d64b9a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.347466 Sep 27 09:59:51.551750 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 533/960] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:51 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:51.561667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a3a346a0-b02e-4904-a80f-dd9eeac5ca99 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:51.563470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a3a346a0-b02e-4904-a80f-dd9eeac5ca99 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:51.599666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a346a0-b02e-4904-a80f-dd9eeac5ca99 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:51.601789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a346a0-b02e-4904-a80f-dd9eeac5ca99 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:51.602561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a346a0-b02e-4904-a80f-dd9eeac5ca99 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:51.606700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a346a0-b02e-4904-a80f-dd9eeac5ca99 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:51.609182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a346a0-b02e-4904-a80f-dd9eeac5ca99 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:51.619393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a346a0-b02e-4904-a80f-dd9eeac5ca99 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:51.689332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:59:51.690757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:59:51.692073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:59:51.692972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:59:51.697190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:59:51.742558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:51.745911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:51.751274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:51.761415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:51.761951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:51.762595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:51.801031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Getting quotas for project ec0980b5824b425fbad9b8554cf5c42c. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:59:51.810421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Getting quotas for user dce01d3212934161a38be853b4cd93ae and project ec0980b5824b425fbad9b8554cf5c42c. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:59:51.832315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:59:51.836135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:59:51.931821 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3ec9fb1a-0da8-403d-a138-33bb0f8135b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.606663 Sep 27 09:59:51.933505 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 428/961] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:59:49 2022] POST /compute/v2.1/servers => generated 380 bytes in 2610 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 09:59:51.950402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-6cb16aec-d6ab-443c-b713-2605cd2ad7a1 req-c32edbfb-e1bd-41f9-9ebc-57db2a7d8f73 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7bc6edb-7fc4-463d-8490-8800e2922d12", "name": "network-vif-plugged", "status": "completed", "tag": "dbb7e99b-8507-4a96-bb0a-c442351f9baf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:51.962386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6cb16aec-d6ab-443c-b713-2605cd2ad7a1 req-c32edbfb-e1bd-41f9-9ebc-57db2a7d8f73 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:51.962775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6cb16aec-d6ab-443c-b713-2605cd2ad7a1 req-c32edbfb-e1bd-41f9-9ebc-57db2a7d8f73 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:51.967299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6cb16aec-d6ab-443c-b713-2605cd2ad7a1 req-c32edbfb-e1bd-41f9-9ebc-57db2a7d8f73 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:52.008532 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-6cb16aec-d6ab-443c-b713-2605cd2ad7a1 req-c32edbfb-e1bd-41f9-9ebc-57db2a7d8f73 service nova] Creating event network-vif-plugged:dbb7e99b-8507-4a96-bb0a-c442351f9baf for instance a7bc6edb-7fc4-463d-8490-8800e2922d12 on nch1-master-1 Sep 27 09:59:52.037419 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-6cb16aec-d6ab-443c-b713-2605cd2ad7a1 req-c32edbfb-e1bd-41f9-9ebc-57db2a7d8f73 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.102120 Sep 27 09:59:52.048382 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 429/962] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:52.054689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-565abb8c-22e8-4abf-8a5f-58f837039fde tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:52.062808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-565abb8c-22e8-4abf-8a5f-58f837039fde tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:52.082050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-565abb8c-22e8-4abf-8a5f-58f837039fde tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:52.082579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-565abb8c-22e8-4abf-8a5f-58f837039fde tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:52.083129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-565abb8c-22e8-4abf-8a5f-58f837039fde tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:52.083769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-565abb8c-22e8-4abf-8a5f-58f837039fde tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:52.084243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-565abb8c-22e8-4abf-8a5f-58f837039fde tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:52.084748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-565abb8c-22e8-4abf-8a5f-58f837039fde tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:52.487800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a3a346a0-b02e-4904-a80f-dd9eeac5ca99 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:52.509602 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a3a346a0-b02e-4904-a80f-dd9eeac5ca99 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 0.955403 Sep 27 09:59:52.513032 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 534/963] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:51 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 961 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:52.553117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-28401964-4388-40ad-beb0-ef2b57fbac53 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:52.554411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-28401964-4388-40ad-beb0-ef2b57fbac53 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:52.702997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-28401964-4388-40ad-beb0-ef2b57fbac53 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:52.723722 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-28401964-4388-40ad-beb0-ef2b57fbac53 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1210 microversion: 2.1 time: 0.205950 Sep 27 09:59:52.724738 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 535/964] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:52 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1210 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:52.730036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b8e3e56d-2bbc-49e9-ae3a-4080017e87d7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:52.731221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b8e3e56d-2bbc-49e9-ae3a-4080017e87d7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:52.745122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8e3e56d-2bbc-49e9-ae3a-4080017e87d7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:52.745725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8e3e56d-2bbc-49e9-ae3a-4080017e87d7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:52.747158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8e3e56d-2bbc-49e9-ae3a-4080017e87d7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:52.748607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8e3e56d-2bbc-49e9-ae3a-4080017e87d7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:52.750336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8e3e56d-2bbc-49e9-ae3a-4080017e87d7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:52.751331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8e3e56d-2bbc-49e9-ae3a-4080017e87d7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:52.784314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-565abb8c-22e8-4abf-8a5f-58f837039fde tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:52.795038 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-565abb8c-22e8-4abf-8a5f-58f837039fde tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 0.744910 Sep 27 09:59:52.795620 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 430/965] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:52 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 746 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:52.809803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b2763c56-421a-4e3a-afe6-ad0dfbac8683 req-9175bc23-35e6-4ecc-bab2-105470c7ad4d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "984079f7-c95b-4c38-8ea7-5a16c0f50c0a", "name": "network-vif-deleted", "tag": "c39020e6-7f01-470b-a8f7-f9d386f10248"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:52.845319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b2763c56-421a-4e3a-afe6-ad0dfbac8683 req-9175bc23-35e6-4ecc-bab2-105470c7ad4d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:52.846461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b2763c56-421a-4e3a-afe6-ad0dfbac8683 req-9175bc23-35e6-4ecc-bab2-105470c7ad4d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:52.847110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b2763c56-421a-4e3a-afe6-ad0dfbac8683 req-9175bc23-35e6-4ecc-bab2-105470c7ad4d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:52.860624 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b2763c56-421a-4e3a-afe6-ad0dfbac8683 req-9175bc23-35e6-4ecc-bab2-105470c7ad4d service nova] Creating event network-vif-deleted:c39020e6-7f01-470b-a8f7-f9d386f10248 for instance 984079f7-c95b-4c38-8ea7-5a16c0f50c0a on nch2-master-1 Sep 27 09:59:52.863088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b8e3e56d-2bbc-49e9-ae3a-4080017e87d7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:52.867718 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b2763c56-421a-4e3a-afe6-ad0dfbac8683 req-9175bc23-35e6-4ecc-bab2-105470c7ad4d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070244 Sep 27 09:59:52.871020 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 431/966] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 09:59:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:52.875982 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b8e3e56d-2bbc-49e9-ae3a-4080017e87d7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1429 microversion: 2.1 time: 0.150130 Sep 27 09:59:52.879905 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 536/967] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:52 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1429 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:52.896715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:52.898402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:52.899440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13902a5d-4b14-449e-bfdb-7193ce555e95 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Calling method '.version_select at 0x7f7f63ac3e50>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:52.907150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13902a5d-4b14-449e-bfdb-7193ce555e95 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:52.914300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13902a5d-4b14-449e-bfdb-7193ce555e95 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:52.915083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13902a5d-4b14-449e-bfdb-7193ce555e95 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:52.919945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:52.920619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:52.925654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:52.926575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:52.927149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:52.927676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:52.934077 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13902a5d-4b14-449e-bfdb-7193ce555e95 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "GET /compute/v2.1/os-migrations" status: 200 len: 802 microversion: 2.1 time: 0.052645 Sep 27 09:59:52.935848 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 537/968] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 09:59:52 2022] GET /compute/v2.1/os-migrations => generated 802 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:52.964475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bff8be53-02f7-48be-8950-a3aec8dd93eb tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:52.976818 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bff8be53-02f7-48be-8950-a3aec8dd93eb tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.014429 Sep 27 09:59:52.982789 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 538/969] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 09:59:52 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 24 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 09:59:52.995700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b8fb745-fad3-406c-8348-e48ede8cade9 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Action: 'create', calling method: >, body: {"flavor": {"name": "tempest-test_resize_flavor_-309978019", "ram": 1024, "disk": 1, "vcpus": 1}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:53.048902 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7b8fb745-fad3-406c-8348-e48ede8cade9 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 498 microversion: 2.1 time: 0.057342 Sep 27 09:59:53.050174 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 539/970] 10.222.0.26 () {60 vars in 1146 bytes} [Tue Sep 27 09:59:52 2022] POST /compute/v2.1/flavors => generated 498 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:53.064440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3821bd98-dfe8-49fc-ba5e-b4dc0dd3e22a tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:53.074682 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3821bd98-dfe8-49fc-ba5e-b4dc0dd3e22a tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.1 time: 0.015394 Sep 27 09:59:53.078445 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 540/971] 10.222.0.26 () {58 vars in 1182 bytes} [Tue Sep 27 09:59:53 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 27 09:59:53.203027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:53.216331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:53.218371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:53.220150 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-14165572-fc77-4e91-8873-b0bf8c5b8942 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.346827 Sep 27 09:59:53.221353 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 432/972] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:52 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:53.316223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-1546322808", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "f627fe99-2d47-4507-bbb8-75b3a025ba89"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:53.393005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 09:59:53.536735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a04d6406-63c8-4ecf-83ee-27a756479364 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:53.537970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a04d6406-63c8-4ecf-83ee-27a756479364 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:53.556523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a04d6406-63c8-4ecf-83ee-27a756479364 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:53.558371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a04d6406-63c8-4ecf-83ee-27a756479364 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:53.559404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a04d6406-63c8-4ecf-83ee-27a756479364 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:53.562848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a04d6406-63c8-4ecf-83ee-27a756479364 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:53.563950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a04d6406-63c8-4ecf-83ee-27a756479364 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:53.565173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a04d6406-63c8-4ecf-83ee-27a756479364 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:53.776375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 09:59:53.776976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 09:59:53.777764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 09:59:53.778177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 09:59:53.778590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 09:59:53.801002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:53.801798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:53.802533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:53.803545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:53.804163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:53.804889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:53.830992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for project e79eae6310354ca58046e59dfac23009. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 09:59:53.837950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for user 9544cdb0392f416197a586d99e8a3210 and project e79eae6310354ca58046e59dfac23009. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 09:59:53.856440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 09:59:53.859021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 09:59:53.891467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a04d6406-63c8-4ecf-83ee-27a756479364 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:53.905043 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a04d6406-63c8-4ecf-83ee-27a756479364 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1330 microversion: 2.1 time: 0.372257 Sep 27 09:59:53.906136 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 433/973] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:53 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1330 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:53.921019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2d30ccd6-1de6-45ae-a126-faad4a4f8bd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:53.921868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2d30ccd6-1de6-45ae-a126-faad4a4f8bd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:53.924202 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-96c92681-39ad-4f26-9f6d-e12f21bc3aa9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.613029 Sep 27 09:59:53.926211 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 541/974] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 09:59:53 2022] POST /compute/v2.1/servers => generated 380 bytes in 616 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 09:59:53.930511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d30ccd6-1de6-45ae-a126-faad4a4f8bd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:53.931086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d30ccd6-1de6-45ae-a126-faad4a4f8bd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:53.931898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d30ccd6-1de6-45ae-a126-faad4a4f8bd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:53.932906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d30ccd6-1de6-45ae-a126-faad4a4f8bd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:53.933579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d30ccd6-1de6-45ae-a126-faad4a4f8bd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:53.934162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d30ccd6-1de6-45ae-a126-faad4a4f8bd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:53.938041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4222d7c0-9804-48da-8078-740abce7a6a6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:53.938667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4222d7c0-9804-48da-8078-740abce7a6a6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:53.952901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4222d7c0-9804-48da-8078-740abce7a6a6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:53.953665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4222d7c0-9804-48da-8078-740abce7a6a6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:53.955316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4222d7c0-9804-48da-8078-740abce7a6a6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:53.956283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4222d7c0-9804-48da-8078-740abce7a6a6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:53.957132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4222d7c0-9804-48da-8078-740abce7a6a6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:53.957823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4222d7c0-9804-48da-8078-740abce7a6a6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:54.024812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2d30ccd6-1de6-45ae-a126-faad4a4f8bd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:54.045523 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2d30ccd6-1de6-45ae-a126-faad4a4f8bd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1262 microversion: 2.1 time: 0.137867 Sep 27 09:59:54.046146 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 434/975] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:53 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1262 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:54.060873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-91e5cc36-6ce8-40f4-a54e-9d2f1faa6e3e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:54.060873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-91e5cc36-6ce8-40f4-a54e-9d2f1faa6e3e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:54.060873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91e5cc36-6ce8-40f4-a54e-9d2f1faa6e3e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:54.061315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91e5cc36-6ce8-40f4-a54e-9d2f1faa6e3e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:54.062028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91e5cc36-6ce8-40f4-a54e-9d2f1faa6e3e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:54.062788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91e5cc36-6ce8-40f4-a54e-9d2f1faa6e3e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:54.063399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91e5cc36-6ce8-40f4-a54e-9d2f1faa6e3e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:54.063976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91e5cc36-6ce8-40f4-a54e-9d2f1faa6e3e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:54.189113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91e5cc36-6ce8-40f4-a54e-9d2f1faa6e3e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:54.201868 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-91e5cc36-6ce8-40f4-a54e-9d2f1faa6e3e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1429 microversion: 2.1 time: 0.154619 Sep 27 09:59:54.202498 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 435/976] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:54 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1429 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:54.234728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d46a8724-0f9f-45e6-a83f-b4c8e95bb6b9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:54.236015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d46a8724-0f9f-45e6-a83f-b4c8e95bb6b9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:54.303939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d46a8724-0f9f-45e6-a83f-b4c8e95bb6b9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:54.313351 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d46a8724-0f9f-45e6-a83f-b4c8e95bb6b9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1279 microversion: 2.1 time: 0.088918 Sep 27 09:59:54.316359 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 436/977] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:54 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1279 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:54.334143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4222d7c0-9804-48da-8078-740abce7a6a6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:54.335380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:54.335676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:54.345900 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4222d7c0-9804-48da-8078-740abce7a6a6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 0.418150 Sep 27 09:59:54.346922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:54.348497 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 542/978] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:53 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:54.348971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:54.350778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:54.352685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:54.353335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:54.356136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:54.613274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:54.629421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:54.631107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:54.647551 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-95f473ba-88ce-4c72-a3ea-733d229d60e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.327981 Sep 27 09:59:54.653325 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 437/979] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:54 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:54.929999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-00699211-b418-45c9-93a8-43d6cb6affa3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:54.931737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-00699211-b418-45c9-93a8-43d6cb6affa3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:54.945419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00699211-b418-45c9-93a8-43d6cb6affa3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:54.946186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00699211-b418-45c9-93a8-43d6cb6affa3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:54.946798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00699211-b418-45c9-93a8-43d6cb6affa3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:54.947689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00699211-b418-45c9-93a8-43d6cb6affa3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:54.948296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00699211-b418-45c9-93a8-43d6cb6affa3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:54.948895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00699211-b418-45c9-93a8-43d6cb6affa3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:55.078949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-be356b41-61d2-4bab-b415-1566c005a1db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:55.079572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-be356b41-61d2-4bab-b415-1566c005a1db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:55.090117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:55.099262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be356b41-61d2-4bab-b415-1566c005a1db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:55.099871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be356b41-61d2-4bab-b415-1566c005a1db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:55.101661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be356b41-61d2-4bab-b415-1566c005a1db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:55.103718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be356b41-61d2-4bab-b415-1566c005a1db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:55.104233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be356b41-61d2-4bab-b415-1566c005a1db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:55.110640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be356b41-61d2-4bab-b415-1566c005a1db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:55.300337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-be356b41-61d2-4bab-b415-1566c005a1db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:55.302055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-00699211-b418-45c9-93a8-43d6cb6affa3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:55.316680 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-be356b41-61d2-4bab-b415-1566c005a1db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1280 microversion: 2.1 time: 0.239848 Sep 27 09:59:55.321023 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 438/980] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:55 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1280 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:55.323095 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-00699211-b418-45c9-93a8-43d6cb6affa3 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1533 microversion: 2.1 time: 0.397618 Sep 27 09:59:55.323821 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 543/981] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:54 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1533 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 09:59:55.328693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f7a5cf60-e241-40d2-8313-32f4a3ef4559 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:55.329961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f7a5cf60-e241-40d2-8313-32f4a3ef4559 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:55.339910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a5cf60-e241-40d2-8313-32f4a3ef4559 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:55.343853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a5cf60-e241-40d2-8313-32f4a3ef4559 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:55.344790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a5cf60-e241-40d2-8313-32f4a3ef4559 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:55.346300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a5cf60-e241-40d2-8313-32f4a3ef4559 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:55.346997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a5cf60-e241-40d2-8313-32f4a3ef4559 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:55.347705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a5cf60-e241-40d2-8313-32f4a3ef4559 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:55.351556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e66286b-711a-4f66-8853-dbc4e6c14b99 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:55.352237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2e66286b-711a-4f66-8853-dbc4e6c14b99 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:55.358563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e66286b-711a-4f66-8853-dbc4e6c14b99 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:55.359270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e66286b-711a-4f66-8853-dbc4e6c14b99 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:55.360038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e66286b-711a-4f66-8853-dbc4e6c14b99 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:55.360966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e66286b-711a-4f66-8853-dbc4e6c14b99 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:55.362089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e66286b-711a-4f66-8853-dbc4e6c14b99 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:55.364817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e66286b-711a-4f66-8853-dbc4e6c14b99 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:55.495810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2e66286b-711a-4f66-8853-dbc4e6c14b99 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:55.504584 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2e66286b-711a-4f66-8853-dbc4e6c14b99 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1331 microversion: 2.1 time: 0.160112 Sep 27 09:59:55.505595 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 544/982] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:55 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1331 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:55.519292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d03b4ecd-3d6d-4404-a7eb-b187a7626fdc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Action: 'create', calling method: .version_select at 0x7f7f637dc670>, body: {"pool": "cf91963d-b887-4e7e-bee4-e38f7a5f86af"} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:55.600913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f7a5cf60-e241-40d2-8313-32f4a3ef4559 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:55.618524 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f7a5cf60-e241-40d2-8313-32f4a3ef4559 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1301 microversion: 2.1 time: 0.291331 Sep 27 09:59:55.619524 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 439/983] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:55 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1301 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:55.632002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5d7c22c-d457-4d37-b575-67f6f55abaa7 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:55.633906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c5d7c22c-d457-4d37-b575-67f6f55abaa7 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:55.657847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5d7c22c-d457-4d37-b575-67f6f55abaa7 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:55.660802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5d7c22c-d457-4d37-b575-67f6f55abaa7 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:55.662724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5d7c22c-d457-4d37-b575-67f6f55abaa7 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:55.668109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5d7c22c-d457-4d37-b575-67f6f55abaa7 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:55.668834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5d7c22c-d457-4d37-b575-67f6f55abaa7 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:55.669356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5d7c22c-d457-4d37-b575-67f6f55abaa7 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:56.497482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c5d7c22c-d457-4d37-b575-67f6f55abaa7 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:56.543262 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c5d7c22c-d457-4d37-b575-67f6f55abaa7 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 0.910310 Sep 27 09:59:56.544735 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 440/984] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:55 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:56.551415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:56.552434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:56.565442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:56.567655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:56.568353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:56.569496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:56.571753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:56.572437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:57.566982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:57.596575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:57.607717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:57.621363 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5fc19ef4-7d89-48e2-b549-84cfaa223c70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 1.066578 Sep 27 09:59:57.628813 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 441/985] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:56 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 1083 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:57.648351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-51cc658e-0f64-433e-be12-fc8851b3ec7b tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:57.649008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-51cc658e-0f64-433e-be12-fc8851b3ec7b tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:57.663607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51cc658e-0f64-433e-be12-fc8851b3ec7b tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:57.664829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51cc658e-0f64-433e-be12-fc8851b3ec7b tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:57.665734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51cc658e-0f64-433e-be12-fc8851b3ec7b tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:57.668097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51cc658e-0f64-433e-be12-fc8851b3ec7b tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:57.669323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51cc658e-0f64-433e-be12-fc8851b3ec7b tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:57.670298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51cc658e-0f64-433e-be12-fc8851b3ec7b tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:58.030276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-51cc658e-0f64-433e-be12-fc8851b3ec7b tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:58.044387 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-51cc658e-0f64-433e-be12-fc8851b3ec7b tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1268 microversion: 2.1 time: 0.413467 Sep 27 09:59:58.045723 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 442/986] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:57 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1268 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:58.050442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9a046b22-1ef7-4344-9a0b-3456645de6ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:58.064079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9a046b22-1ef7-4344-9a0b-3456645de6ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:58.071957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a046b22-1ef7-4344-9a0b-3456645de6ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:58.073183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a046b22-1ef7-4344-9a0b-3456645de6ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:58.073888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a046b22-1ef7-4344-9a0b-3456645de6ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:58.075033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a046b22-1ef7-4344-9a0b-3456645de6ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:58.076273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a046b22-1ef7-4344-9a0b-3456645de6ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:58.077071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a046b22-1ef7-4344-9a0b-3456645de6ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:58.123601 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d03b4ecd-3d6d-4404-a7eb-b187a7626fdc tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "POST /compute/v2.1/os-floating-ips" status: 200 len: 141 microversion: 2.1 time: 2.616824 Sep 27 09:59:58.124226 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 545/987] 10.222.0.26 () {60 vars in 1170 bytes} [Tue Sep 27 09:59:55 2022] POST /compute/v2.1/os-floating-ips => generated 141 bytes in 2618 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 09:59:58.133578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:58.133578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:58.142282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:58.143103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:58.143767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:58.145997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:58.146828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:58.147464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:58.184499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:58.185026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:58.185719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:58.186432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lazy-loading 'fault' on Instance uuid 984079f7-c95b-4c38-8ea7-5a16c0f50c0a {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 09:59:58.243172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:58.260718 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-12b517d6-aee5-4215-8b1e-de98d20264a7 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 200 len: 1306 microversion: 2.1 time: 0.135151 Sep 27 09:59:58.277710 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 546/988] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:58 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 1306 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:58.283358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c01eeb5-fee9-48c2-8494-9d630af68505 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:58.285601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2c01eeb5-fee9-48c2-8494-9d630af68505 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:58.294001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c01eeb5-fee9-48c2-8494-9d630af68505 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:58.294724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c01eeb5-fee9-48c2-8494-9d630af68505 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:58.295398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c01eeb5-fee9-48c2-8494-9d630af68505 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:58.296522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c01eeb5-fee9-48c2-8494-9d630af68505 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:58.297830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c01eeb5-fee9-48c2-8494-9d630af68505 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:58.298553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c01eeb5-fee9-48c2-8494-9d630af68505 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:58.326133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9a046b22-1ef7-4344-9a0b-3456645de6ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:58.359229 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9a046b22-1ef7-4344-9a0b-3456645de6ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1337 microversion: 2.1 time: 0.312207 Sep 27 09:59:58.366416 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 443/989] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:58 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1337 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:58.382593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Action: 'action', calling method: .version_select at 0x7f7f693fbee0>, body: {"addFloatingIp": {"address": "172.35.1.66"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 09:59:58.393293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:58.422639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:58.423138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:58.432361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:58.432850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:58.433385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:58.433946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:58.605252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 09:59:58.606353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquired lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 09:59:58.761073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2c01eeb5-fee9-48c2-8494-9d630af68505 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:58.773348 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c01eeb5-fee9-48c2-8494-9d630af68505 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 0.493771 Sep 27 09:59:58.790211 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 547/990] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:58 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:58.797048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:58.798244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:58.806185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:58.808294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:58.809017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:58.809989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:58.810654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:58.811530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:59.153458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:59.175551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:59.178109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:59.180798 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ab664124-1547-4732-899b-44e012928617 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.388436 Sep 27 09:59:59.182010 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 548/991] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 09:59:58 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 09:59:59.189181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b13eae3e-6ceb-43ab-9910-49b46ed10348 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:59.190422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b13eae3e-6ceb-43ab-9910-49b46ed10348 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:59.208651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b13eae3e-6ceb-43ab-9910-49b46ed10348 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:59.210319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b13eae3e-6ceb-43ab-9910-49b46ed10348 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:59.211084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b13eae3e-6ceb-43ab-9910-49b46ed10348 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:59.212388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b13eae3e-6ceb-43ab-9910-49b46ed10348 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:59.213213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b13eae3e-6ceb-43ab-9910-49b46ed10348 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:59.214310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b13eae3e-6ceb-43ab-9910-49b46ed10348 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:59.392302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b13eae3e-6ceb-43ab-9910-49b46ed10348 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:59.401317 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b13eae3e-6ceb-43ab-9910-49b46ed10348 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1310 microversion: 2.1 time: 0.217731 Sep 27 09:59:59.412786 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 549/992] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:59 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1310 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:59.412786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:59.412786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 984079f7-c95b-4c38-8ea7-5a16c0f50c0a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:59.415382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:59.416103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:59.416747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:59.417651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:59.418270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:59.419142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:59.441792 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] HTTP exception thrown: Instance 984079f7-c95b-4c38-8ea7-5a16c0f50c0a could not be found. Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Request method failure captured: Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a HTTP/1.1 Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjuzA8awi02tVpJBbCxyCTwC-N-auCdITAAm8nxhyCwHUSTrFjjx6tbQkA83IeLeCJH2BD7omoS-ML8vYESEbbQcIu0WhsX0gHiWVx2nVId80d2oExNsA2vGHGUS2rPtxB3q_M7eSz3mw7rO-vyGP61qyPfw7K0p8tFBuwGQZNzBLo Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 09:59:59.444479 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c1ac7b64c68e4896b2c631baacdbe2be"}]}] Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersOnMultiNodesTest-536305491 Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 00423eb1e4c24763ae0a546225ac1dc7 Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 984079f7-c95b-4c38-8ea7-5a16c0f50c0a could not be found. Sep 27 09:59:59.450762 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '984079f7-c95b-4c38-8ea7-5a16c0f50c0a'} Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 984079f7-c95b-4c38-8ea7-5a16c0f50c0a could not be found. Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 09:59:59.452369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 984079f7-c95b-4c38-8ea7-5a16c0f50c0a could not be found. Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 984079f7-c95b-4c38-8ea7-5a16c0f50c0a could not be found. Sep 27 09:59:59.457230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 09:59:59.459984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Returning 404 to user: Instance 984079f7-c95b-4c38-8ea7-5a16c0f50c0a could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 09:59:59.459984 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f0d0c67c-e714-43ce-8b1f-70e5b63909ad tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a" status: 404 len: 111 microversion: 2.1 time: 0.042625 Sep 27 09:59:59.459984 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 550/993] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:59 2022] GET /compute/v2.1/servers/984079f7-c95b-4c38-8ea7-5a16c0f50c0a => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 09:59:59.459984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-930f3975-b390-4788-be3b-60babb39a2f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:59.459984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-930f3975-b390-4788-be3b-60babb39a2f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:59.460777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-930f3975-b390-4788-be3b-60babb39a2f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:59.460777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-930f3975-b390-4788-be3b-60babb39a2f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:59.460777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-930f3975-b390-4788-be3b-60babb39a2f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:59.461683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-930f3975-b390-4788-be3b-60babb39a2f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:59.461862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-930f3975-b390-4788-be3b-60babb39a2f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:59.462635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-930f3975-b390-4788-be3b-60babb39a2f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:59.576556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-930f3975-b390-4788-be3b-60babb39a2f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 09:59:59.588687 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-930f3975-b390-4788-be3b-60babb39a2f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1337 microversion: 2.1 time: 0.140378 Sep 27 09:59:59.604692 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 551/994] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:59 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1337 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 09:59:59.604692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e659d490-1a6b-4490-bf50-d410b0fc235d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:59.604692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e659d490-1a6b-4490-bf50-d410b0fc235d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:59.610906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e659d490-1a6b-4490-bf50-d410b0fc235d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:59.611420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e659d490-1a6b-4490-bf50-d410b0fc235d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:59.611868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e659d490-1a6b-4490-bf50-d410b0fc235d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:59.612604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e659d490-1a6b-4490-bf50-d410b0fc235d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:59.613092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e659d490-1a6b-4490-bf50-d410b0fc235d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:59.613621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e659d490-1a6b-4490-bf50-d410b0fc235d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:59.634997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 09:59:59.644160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e659d490-1a6b-4490-bf50-d410b0fc235d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 09:59:59.794837 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e659d490-1a6b-4490-bf50-d410b0fc235d tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "DELETE /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 204 len: 0 microversion: 2.1 time: 0.203153 Sep 27 09:59:59.798961 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 552/995] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 09:59:59 2022] DELETE /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 09:59:59.803837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7764baaa-23f0-4248-95ff-7f9a6381ec9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 09:59:59.805139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7764baaa-23f0-4248-95ff-7f9a6381ec9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 09:59:59.812931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7764baaa-23f0-4248-95ff-7f9a6381ec9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:59.813436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7764baaa-23f0-4248-95ff-7f9a6381ec9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:59.813880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7764baaa-23f0-4248-95ff-7f9a6381ec9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 09:59:59.816621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7764baaa-23f0-4248-95ff-7f9a6381ec9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 09:59:59.817169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7764baaa-23f0-4248-95ff-7f9a6381ec9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 09:59:59.817606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7764baaa-23f0-4248-95ff-7f9a6381ec9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:00.011157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7764baaa-23f0-4248-95ff-7f9a6381ec9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:00.025168 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7764baaa-23f0-4248-95ff-7f9a6381ec9f tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 0.227534 Sep 27 10:00:00.038414 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 553/996] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 09:59:59 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:00.060768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e1c223f-3290-400c-9402-03484d312e79 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:00.062812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9e1c223f-3290-400c-9402-03484d312e79 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:00.089339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e1c223f-3290-400c-9402-03484d312e79 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:00.089837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e1c223f-3290-400c-9402-03484d312e79 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:00.090283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e1c223f-3290-400c-9402-03484d312e79 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:00.091590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e1c223f-3290-400c-9402-03484d312e79 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:00.092056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e1c223f-3290-400c-9402-03484d312e79 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:00.092568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e1c223f-3290-400c-9402-03484d312e79 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:00.288112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9e1c223f-3290-400c-9402-03484d312e79 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:00.299994 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9e1c223f-3290-400c-9402-03484d312e79 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1473 microversion: 2.1 time: 0.258859 Sep 27 10:00:00.312885 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 554/997] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:00 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1473 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:00.312885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:00.312885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:00.317601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:00.318338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:00.318940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:00.319915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:00.326859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:00.327934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:00.535441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:00.548961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:00.550301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:00.551709 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2f399446-e79e-425e-8ccd-50e71ecf4d61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.249701 Sep 27 10:00:00.561740 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 555/998] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:00 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:00.574049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8f634d1-f426-4481-8c36-9df2271fbbee tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:00.575365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f8f634d1-f426-4481-8c36-9df2271fbbee tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:00.582589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f634d1-f426-4481-8c36-9df2271fbbee tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:00.583590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f634d1-f426-4481-8c36-9df2271fbbee tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:00.583872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f634d1-f426-4481-8c36-9df2271fbbee tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:00.584767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f634d1-f426-4481-8c36-9df2271fbbee tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:00.585291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f634d1-f426-4481-8c36-9df2271fbbee tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:00.586430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f634d1-f426-4481-8c36-9df2271fbbee tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:00.750280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f8f634d1-f426-4481-8c36-9df2271fbbee tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:00.759121 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f8f634d1-f426-4481-8c36-9df2271fbbee tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1310 microversion: 2.1 time: 0.194193 Sep 27 10:00:00.759944 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 556/999] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:00 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1310 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:00.764921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86c8a506-48ee-4352-ba33-2b83c09a493f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:00.765408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86c8a506-48ee-4352-ba33-2b83c09a493f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:00.771814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86c8a506-48ee-4352-ba33-2b83c09a493f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:00.772312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86c8a506-48ee-4352-ba33-2b83c09a493f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:00.773066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86c8a506-48ee-4352-ba33-2b83c09a493f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:00.773928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86c8a506-48ee-4352-ba33-2b83c09a493f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:00.774397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86c8a506-48ee-4352-ba33-2b83c09a493f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:00.774825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86c8a506-48ee-4352-ba33-2b83c09a493f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:00.971557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-86c8a506-48ee-4352-ba33-2b83c09a493f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:00.981231 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86c8a506-48ee-4352-ba33-2b83c09a493f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1379 microversion: 2.1 time: 0.220408 Sep 27 10:00:00.982022 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 557/1000] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:00 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1379 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:00.998360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a6bb7a13-2eaa-4754-ae65-9a9013af9860 req-173431b8-fa45-4215-9383-2d11ceb6338f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f1ff4a24-1251-4827-955f-159c665a1d57", "name": "network-vif-plugged", "status": "completed", "tag": "9c2668a6-7fb1-48ea-8f26-e6c77b4e01f9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:01.009130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a6bb7a13-2eaa-4754-ae65-9a9013af9860 req-173431b8-fa45-4215-9383-2d11ceb6338f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:01.009623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a6bb7a13-2eaa-4754-ae65-9a9013af9860 req-173431b8-fa45-4215-9383-2d11ceb6338f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:01.010063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a6bb7a13-2eaa-4754-ae65-9a9013af9860 req-173431b8-fa45-4215-9383-2d11ceb6338f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:01.032825 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a6bb7a13-2eaa-4754-ae65-9a9013af9860 req-173431b8-fa45-4215-9383-2d11ceb6338f service nova] Creating event network-vif-plugged:9c2668a6-7fb1-48ea-8f26-e6c77b4e01f9 for instance f1ff4a24-1251-4827-955f-159c665a1d57 on nch2-master-1 Sep 27 10:00:01.039539 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a6bb7a13-2eaa-4754-ae65-9a9013af9860 req-173431b8-fa45-4215-9383-2d11ceb6338f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051809 Sep 27 10:00:01.041571 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 558/1001] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:01.058607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-14bfd489-8f83-4a3c-80c1-ca13caa644ec tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:01.059559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-14bfd489-8f83-4a3c-80c1-ca13caa644ec tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:01.072188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14bfd489-8f83-4a3c-80c1-ca13caa644ec tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:01.074228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14bfd489-8f83-4a3c-80c1-ca13caa644ec tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:01.074762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14bfd489-8f83-4a3c-80c1-ca13caa644ec tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:01.075599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14bfd489-8f83-4a3c-80c1-ca13caa644ec tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:01.076586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14bfd489-8f83-4a3c-80c1-ca13caa644ec tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:01.076586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14bfd489-8f83-4a3c-80c1-ca13caa644ec tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:01.317607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-14bfd489-8f83-4a3c-80c1-ca13caa644ec tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:01.329303 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-14bfd489-8f83-4a3c-80c1-ca13caa644ec tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 0.278483 Sep 27 10:00:01.335087 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 559/1002] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:01 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:01.340561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-895eecfe-b9f6-4131-8ea1-c12758d127e0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:01.341208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-895eecfe-b9f6-4131-8ea1-c12758d127e0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:01.347875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-895eecfe-b9f6-4131-8ea1-c12758d127e0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:01.348431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-895eecfe-b9f6-4131-8ea1-c12758d127e0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:01.348910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-895eecfe-b9f6-4131-8ea1-c12758d127e0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:01.349621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-895eecfe-b9f6-4131-8ea1-c12758d127e0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:01.350098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-895eecfe-b9f6-4131-8ea1-c12758d127e0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:01.350513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-895eecfe-b9f6-4131-8ea1-c12758d127e0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:01.492711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-895eecfe-b9f6-4131-8ea1-c12758d127e0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:01.501903 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-895eecfe-b9f6-4131-8ea1-c12758d127e0 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1473 microversion: 2.1 time: 0.165262 Sep 27 10:00:01.502848 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 560/1003] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:01 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1473 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:01.578632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:01.579281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:01.586558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:01.587065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:01.587531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:01.588266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:01.588917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:01.589317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:01.735437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:01.745556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:01.747046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:01.748540 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bca04bea-c044-4a3a-aaf6-febcf316a92c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.174164 Sep 27 10:00:01.749211 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 561/1004] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:01 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:00:01.787419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e815567-7693-4261-8d6a-224f9c9d89d9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:01.788034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8e815567-7693-4261-8d6a-224f9c9d89d9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:01.791722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:01.795917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e815567-7693-4261-8d6a-224f9c9d89d9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:01.796601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e815567-7693-4261-8d6a-224f9c9d89d9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:01.796884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e815567-7693-4261-8d6a-224f9c9d89d9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:01.797677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e815567-7693-4261-8d6a-224f9c9d89d9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:01.798136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e815567-7693-4261-8d6a-224f9c9d89d9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:01.798567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e815567-7693-4261-8d6a-224f9c9d89d9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:01.939609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8e815567-7693-4261-8d6a-224f9c9d89d9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:01.951461 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e815567-7693-4261-8d6a-224f9c9d89d9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1310 microversion: 2.1 time: 0.168447 Sep 27 10:00:01.952322 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 562/1005] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:01 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1310 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:02.004309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a5c0cc3-6cc0-4794-a6b2-57e92563b11b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:02.005022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a5c0cc3-6cc0-4794-a6b2-57e92563b11b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:02.011902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a5c0cc3-6cc0-4794-a6b2-57e92563b11b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:02.012478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a5c0cc3-6cc0-4794-a6b2-57e92563b11b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:02.013013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a5c0cc3-6cc0-4794-a6b2-57e92563b11b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:02.013804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a5c0cc3-6cc0-4794-a6b2-57e92563b11b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:02.014323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a5c0cc3-6cc0-4794-a6b2-57e92563b11b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:02.014879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a5c0cc3-6cc0-4794-a6b2-57e92563b11b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:02.162520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8a5c0cc3-6cc0-4794-a6b2-57e92563b11b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:02.171004 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a5c0cc3-6cc0-4794-a6b2-57e92563b11b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1379 microversion: 2.1 time: 0.170751 Sep 27 10:00:02.171619 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 563/1006] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:01 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1379 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:02.352388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-274169ce-5305-4a34-97df-bec48a4ec4a8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:02.353860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-274169ce-5305-4a34-97df-bec48a4ec4a8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:02.362070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-274169ce-5305-4a34-97df-bec48a4ec4a8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:02.363058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-274169ce-5305-4a34-97df-bec48a4ec4a8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:02.363195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-274169ce-5305-4a34-97df-bec48a4ec4a8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:02.363982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-274169ce-5305-4a34-97df-bec48a4ec4a8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:02.364714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-274169ce-5305-4a34-97df-bec48a4ec4a8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:02.365173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-274169ce-5305-4a34-97df-bec48a4ec4a8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:02.521136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-274169ce-5305-4a34-97df-bec48a4ec4a8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:02.532339 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-274169ce-5305-4a34-97df-bec48a4ec4a8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 0.183900 Sep 27 10:00:02.546774 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 564/1007] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:02 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:02.558513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-497742a4-30f9-4894-bab4-0545c7bc84ce tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:02.559568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-497742a4-30f9-4894-bab4-0545c7bc84ce tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:02.568039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-497742a4-30f9-4894-bab4-0545c7bc84ce tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:02.568648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-497742a4-30f9-4894-bab4-0545c7bc84ce tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:02.569286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-497742a4-30f9-4894-bab4-0545c7bc84ce tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:02.570172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-497742a4-30f9-4894-bab4-0545c7bc84ce tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:02.570714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-497742a4-30f9-4894-bab4-0545c7bc84ce tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:02.571325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-497742a4-30f9-4894-bab4-0545c7bc84ce tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:02.879029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-497742a4-30f9-4894-bab4-0545c7bc84ce tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:02.888967 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-497742a4-30f9-4894-bab4-0545c7bc84ce tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 200 len: 1473 microversion: 2.1 time: 0.340402 Sep 27 10:00:02.905074 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 565/1008] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:02 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 1473 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:02.910059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e8d91c7d-876b-491c-a7f1-77a8edf07f7d req-9b0ada12-7c66-403b-8f52-c5e16e12aaf6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "16cd150f-c67a-4a9e-9654-ba6bb9e1517a", "tag": "fa2d1e07-1a2a-40d0-baa6-738c999a9e3a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:02.929095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e8d91c7d-876b-491c-a7f1-77a8edf07f7d req-9b0ada12-7c66-403b-8f52-c5e16e12aaf6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:02.929556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e8d91c7d-876b-491c-a7f1-77a8edf07f7d req-9b0ada12-7c66-403b-8f52-c5e16e12aaf6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:02.930686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e8d91c7d-876b-491c-a7f1-77a8edf07f7d req-9b0ada12-7c66-403b-8f52-c5e16e12aaf6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:02.942881 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e8d91c7d-876b-491c-a7f1-77a8edf07f7d req-9b0ada12-7c66-403b-8f52-c5e16e12aaf6 service nova] Creating event network-changed:fa2d1e07-1a2a-40d0-baa6-738c999a9e3a for instance 16cd150f-c67a-4a9e-9654-ba6bb9e1517a on nch1-master-1 Sep 27 10:00:02.948275 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e8d91c7d-876b-491c-a7f1-77a8edf07f7d req-9b0ada12-7c66-403b-8f52-c5e16e12aaf6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042041 Sep 27 10:00:02.948941 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 566/1009] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:02.954516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:02.955346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:02.962661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:02.963162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:02.963605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:02.964348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:02.964833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:02.965262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:03.244620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Building network info cache for instance {{(pid=145969) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Sep 27 10:00:03.246489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:03.266458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:03.268309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:03.269721 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8928c342-04d8-41ea-a34c-b177cfe5005d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1362 microversion: 2.25 time: 0.319471 Sep 27 10:00:03.270768 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 567/1010] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:02 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1362 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:03.280695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6822e711-ad57-4241-afe0-0fcddf375928 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:03.283154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6822e711-ad57-4241-afe0-0fcddf375928 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:03.285658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:03.297913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6822e711-ad57-4241-afe0-0fcddf375928 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:03.298808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6822e711-ad57-4241-afe0-0fcddf375928 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:03.299502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6822e711-ad57-4241-afe0-0fcddf375928 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:03.300440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6822e711-ad57-4241-afe0-0fcddf375928 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:03.301147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6822e711-ad57-4241-afe0-0fcddf375928 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:03.301789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6822e711-ad57-4241-afe0-0fcddf375928 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:03.566065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6822e711-ad57-4241-afe0-0fcddf375928 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:03.572173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:03.587267 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6822e711-ad57-4241-afe0-0fcddf375928 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1310 microversion: 2.1 time: 0.315156 Sep 27 10:00:03.588513 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 568/1011] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:03 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1310 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:03.594384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a2d05216-02f2-442c-a16e-6988391b92d0 req-22d02a28-7c2a-4d61-a6c4-71989be6da34 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "24be6165-1a70-40c5-beb7-67dd67933a85", "name": "network-vif-plugged", "status": "completed", "tag": "d97cf212-d664-46ca-b7be-65423e5322f3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:03.606943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a2d05216-02f2-442c-a16e-6988391b92d0 req-22d02a28-7c2a-4d61-a6c4-71989be6da34 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:03.607799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a2d05216-02f2-442c-a16e-6988391b92d0 req-22d02a28-7c2a-4d61-a6c4-71989be6da34 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:03.608115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a2d05216-02f2-442c-a16e-6988391b92d0 req-22d02a28-7c2a-4d61-a6c4-71989be6da34 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:03.623260 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a2d05216-02f2-442c-a16e-6988391b92d0 req-22d02a28-7c2a-4d61-a6c4-71989be6da34 service nova] Creating event network-vif-plugged:d97cf212-d664-46ca-b7be-65423e5322f3 for instance 24be6165-1a70-40c5-beb7-67dd67933a85 on nch2-master-1 Sep 27 10:00:03.631253 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a2d05216-02f2-442c-a16e-6988391b92d0 req-22d02a28-7c2a-4d61-a6c4-71989be6da34 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040080 Sep 27 10:00:03.632778 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 569/1012] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:03.641399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6bc13054-c4bc-4631-8b01-35a5728196d0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:03.642531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6bc13054-c4bc-4631-8b01-35a5728196d0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:03.653170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bc13054-c4bc-4631-8b01-35a5728196d0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:03.653970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bc13054-c4bc-4631-8b01-35a5728196d0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:03.654570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bc13054-c4bc-4631-8b01-35a5728196d0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:03.655711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bc13054-c4bc-4631-8b01-35a5728196d0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:03.656860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bc13054-c4bc-4631-8b01-35a5728196d0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:03.657460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bc13054-c4bc-4631-8b01-35a5728196d0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:03.817569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:03.849953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=18 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:04.310240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:04.369879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6bc13054-c4bc-4631-8b01-35a5728196d0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:04.384711 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6bc13054-c4bc-4631-8b01-35a5728196d0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1379 microversion: 2.1 time: 0.749760 Sep 27 10:00:04.385600 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 570/1013] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:03 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1379 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:04.410926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3debd339-087b-4e54-ab98-e6fd3adcb064 req-32ca864b-bf9f-41df-ad8b-0e90b98ae98f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "185c1774-d54d-4d4e-bce6-315fb25d6db7", "name": "network-vif-deleted", "tag": "74ca1e33-fb78-46e8-bca5-e323d313f753"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:04.446631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3debd339-087b-4e54-ab98-e6fd3adcb064 req-32ca864b-bf9f-41df-ad8b-0e90b98ae98f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:04.452720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3debd339-087b-4e54-ab98-e6fd3adcb064 req-32ca864b-bf9f-41df-ad8b-0e90b98ae98f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:04.456590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3debd339-087b-4e54-ab98-e6fd3adcb064 req-32ca864b-bf9f-41df-ad8b-0e90b98ae98f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:04.485686 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3debd339-087b-4e54-ab98-e6fd3adcb064 req-32ca864b-bf9f-41df-ad8b-0e90b98ae98f service nova] Creating event network-vif-deleted:74ca1e33-fb78-46e8-bca5-e323d313f753 for instance 185c1774-d54d-4d4e-bce6-315fb25d6db7 on nch2-master-1 Sep 27 10:00:04.494651 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3debd339-087b-4e54-ab98-e6fd3adcb064 req-32ca864b-bf9f-41df-ad8b-0e90b98ae98f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.094547 Sep 27 10:00:04.495856 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 571/1014] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:04.501887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d2f92e0b-100d-4fa9-a299-18488078ec12 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:04.504699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d2f92e0b-100d-4fa9-a299-18488078ec12 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:04.512217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2f92e0b-100d-4fa9-a299-18488078ec12 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:04.513026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2f92e0b-100d-4fa9-a299-18488078ec12 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:04.513936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2f92e0b-100d-4fa9-a299-18488078ec12 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:04.514853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2f92e0b-100d-4fa9-a299-18488078ec12 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:04.515481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2f92e0b-100d-4fa9-a299-18488078ec12 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:04.516069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2f92e0b-100d-4fa9-a299-18488078ec12 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:04.776231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Updating instance_info_cache with network_info: [{"id": "dbb7e99b-8507-4a96-bb0a-c442351f9baf", "address": "fa:16:3e:3b:fc:8a", "network": {"id": "e00d057c-26e1-4ae4-9895-9dbcbbcbbfc3", "bridge": null, "label": "tempest-FloatingIPsAssociationNegativeTestJSON-219801886-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.1.66", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "2b3380f3fa2b494da649aaa809e11368", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapdbb7e99b-85", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=145969) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:00:04.797428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Releasing lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 10:00:04.798310 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d5ebb8e3-340e-4d46-bdbb-14beaef7189b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action" status: 202 len: 0 microversion: 2.1 time: 6.430858 Sep 27 10:00:04.798922 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 444/1015] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 09:59:58 2022] POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action => generated 0 bytes in 6432 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Sep 27 10:00:04.809129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:04.810589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 185c1774-d54d-4d4e-bce6-315fb25d6db7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:04.835054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:04.836520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:04.837245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:04.838108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:04.838827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:04.839422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:04.860490 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] HTTP exception thrown: Instance 185c1774-d54d-4d4e-bce6-315fb25d6db7 could not be found. Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Request method failure captured: Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 HTTP/1.1 Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjuzA8awi02tVpJBbCxyCTwC-N-auCdITAAm8nxhyCwHUSTrFjjx6tbQkA83IeLeCJH2BD7omoS-ML8vYESEbbQcIu0WhsX0gHiWVx2nVId80d2oExNsA2vGHGUS2rPtxB3q_M7eSz3mw7rO-vyGP61qyPfw7K0p8tFBuwGQZNzBLo Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:00:04.865223 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c1ac7b64c68e4896b2c631baacdbe2be"}]}] Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 00423eb1e4c24763ae0a546225ac1dc7 Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 185c1774-d54d-4d4e-bce6-315fb25d6db7 could not be found. Sep 27 10:00:04.866899 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '185c1774-d54d-4d4e-bce6-315fb25d6db7'} Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 185c1774-d54d-4d4e-bce6-315fb25d6db7 could not be found. Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:04.867766 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 185c1774-d54d-4d4e-bce6-315fb25d6db7 could not be found. Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 185c1774-d54d-4d4e-bce6-315fb25d6db7 could not be found. Sep 27 10:00:04.870569 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:04.875602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Returning 404 to user: Instance 185c1774-d54d-4d4e-bce6-315fb25d6db7 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:04.876756 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e59fa6ee-d11b-463e-b07b-d929d92a1e33 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7" status: 404 len: 111 microversion: 2.1 time: 0.073521 Sep 27 10:00:04.880922 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 445/1016] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:04 2022] GET /compute/v2.1/servers/185c1774-d54d-4d4e-bce6-315fb25d6db7 => generated 111 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:04.888646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:04.889571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:04.896619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:04.897437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:04.899848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:04.901052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:04.901687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:04.902382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:05.085634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:05.115511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:05.117802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:05.119510 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-35011ac6-ce38-48af-8803-599a6829c293 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1557 microversion: 2.25 time: 0.237310 Sep 27 10:00:05.120353 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 446/1017] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:04 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1557 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:05.145512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32b75f18-584f-4b42-8841-90c2794599c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:05.146352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32b75f18-584f-4b42-8841-90c2794599c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:05.165051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b75f18-584f-4b42-8841-90c2794599c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:05.165652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b75f18-584f-4b42-8841-90c2794599c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:05.167967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b75f18-584f-4b42-8841-90c2794599c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:05.168814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b75f18-584f-4b42-8841-90c2794599c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:05.169879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b75f18-584f-4b42-8841-90c2794599c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:05.170615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b75f18-584f-4b42-8841-90c2794599c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:05.191373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:SAVEPOINT=1,nova_cell1:UPDATE=1,nova_cell1:RELEASE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:05.294617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d2f92e0b-100d-4fa9-a299-18488078ec12 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:05.307405 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d2f92e0b-100d-4fa9-a299-18488078ec12 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 0.809961 Sep 27 10:00:05.320686 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 572/1018] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:04 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 813 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:05.320686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9b039fcb-9506-475d-9823-8c30dbdee257 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Action: 'create', calling method: .version_select at 0x7f7f6380b5e0>, body: {"pool": "cf91963d-b887-4e7e-bee4-e38f7a5f86af"} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:05.364434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32b75f18-584f-4b42-8841-90c2794599c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:05.391120 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32b75f18-584f-4b42-8841-90c2794599c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1310 microversion: 2.1 time: 0.256094 Sep 27 10:00:05.394681 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 447/1019] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:05 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1310 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:05.407963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fa6a20cc-14e9-4c2a-b038-e765d47b8633 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:05.410289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa6a20cc-14e9-4c2a-b038-e765d47b8633 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: f769af8b-472a-4950-b4ae-8c0183b23c1e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:05.420867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa6a20cc-14e9-4c2a-b038-e765d47b8633 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:05.421624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa6a20cc-14e9-4c2a-b038-e765d47b8633 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:05.422522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa6a20cc-14e9-4c2a-b038-e765d47b8633 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:05.424030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa6a20cc-14e9-4c2a-b038-e765d47b8633 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:05.425048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa6a20cc-14e9-4c2a-b038-e765d47b8633 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:05.425637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa6a20cc-14e9-4c2a-b038-e765d47b8633 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:05.485017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa6a20cc-14e9-4c2a-b038-e765d47b8633 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: f769af8b-472a-4950-b4ae-8c0183b23c1e] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:00:05.644714 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fa6a20cc-14e9-4c2a-b038-e765d47b8633 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e" status: 204 len: 0 microversion: 2.1 time: 0.247354 Sep 27 10:00:05.650231 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 448/1020] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:00:05 2022] DELETE /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:00:05.659658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-91973d9d-e90d-4672-91d2-c82adc057b0a req-6bcd2925-a6bf-4638-be63-0ec001bfbaac service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7bc6edb-7fc4-463d-8490-8800e2922d12", "tag": "dbb7e99b-8507-4a96-bb0a-c442351f9baf"}, {"name": "network-changed", "server_uuid": "69ea013c-5143-45ff-81a2-3ed97a8659cd", "tag": "96ac21a6-013a-4a5e-9320-82ff90906b76"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:05.678587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-91973d9d-e90d-4672-91d2-c82adc057b0a req-6bcd2925-a6bf-4638-be63-0ec001bfbaac service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:05.679175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-91973d9d-e90d-4672-91d2-c82adc057b0a req-6bcd2925-a6bf-4638-be63-0ec001bfbaac service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:05.679652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-91973d9d-e90d-4672-91d2-c82adc057b0a req-6bcd2925-a6bf-4638-be63-0ec001bfbaac service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:05.702155 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-91973d9d-e90d-4672-91d2-c82adc057b0a req-6bcd2925-a6bf-4638-be63-0ec001bfbaac service nova] Creating event network-changed:dbb7e99b-8507-4a96-bb0a-c442351f9baf for instance a7bc6edb-7fc4-463d-8490-8800e2922d12 on nch1-master-1 Sep 27 10:00:05.702829 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-91973d9d-e90d-4672-91d2-c82adc057b0a req-6bcd2925-a6bf-4638-be63-0ec001bfbaac service nova] Creating event network-changed:96ac21a6-013a-4a5e-9320-82ff90906b76 for instance 69ea013c-5143-45ff-81a2-3ed97a8659cd on nch1-master-1 Sep 27 10:00:05.712439 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-91973d9d-e90d-4672-91d2-c82adc057b0a req-6bcd2925-a6bf-4638-be63-0ec001bfbaac service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.056469 Sep 27 10:00:05.713854 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 449/1021] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:05 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:06.004786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-17019dd6-5cef-4cbb-93f7-f8b0cd2d8e03 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:06.005611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-17019dd6-5cef-4cbb-93f7-f8b0cd2d8e03 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:06.013176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17019dd6-5cef-4cbb-93f7-f8b0cd2d8e03 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:06.013777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17019dd6-5cef-4cbb-93f7-f8b0cd2d8e03 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:06.014283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17019dd6-5cef-4cbb-93f7-f8b0cd2d8e03 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:06.015096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17019dd6-5cef-4cbb-93f7-f8b0cd2d8e03 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:06.015622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17019dd6-5cef-4cbb-93f7-f8b0cd2d8e03 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:06.016104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17019dd6-5cef-4cbb-93f7-f8b0cd2d8e03 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:06.160496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-17019dd6-5cef-4cbb-93f7-f8b0cd2d8e03 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid f1ff4a24-1251-4827-955f-159c665a1d57 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:00:06.175396 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-17019dd6-5cef-4cbb-93f7-f8b0cd2d8e03 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 2050 microversion: 2.25 time: 0.460345 Sep 27 10:00:06.175923 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 450/1022] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:05 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 2050 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:06.181070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c47990b5-65b5-4ff5-b635-52132d66cee9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:06.181795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c47990b5-65b5-4ff5-b635-52132d66cee9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:06.188004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c47990b5-65b5-4ff5-b635-52132d66cee9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:06.188496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c47990b5-65b5-4ff5-b635-52132d66cee9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:06.188954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c47990b5-65b5-4ff5-b635-52132d66cee9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:06.189665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c47990b5-65b5-4ff5-b635-52132d66cee9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:06.190128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c47990b5-65b5-4ff5-b635-52132d66cee9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:06.190553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c47990b5-65b5-4ff5-b635-52132d66cee9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:06.336997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c47990b5-65b5-4ff5-b635-52132d66cee9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:06.345456 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c47990b5-65b5-4ff5-b635-52132d66cee9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1379 microversion: 2.1 time: 0.168294 Sep 27 10:00:06.346104 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 451/1023] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:06 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1379 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:06.351292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-46ad7caa-e30b-4541-b7c0-0d108a0b2c99 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:06.351943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-46ad7caa-e30b-4541-b7c0-0d108a0b2c99 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:06.358436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ad7caa-e30b-4541-b7c0-0d108a0b2c99 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:06.359146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ad7caa-e30b-4541-b7c0-0d108a0b2c99 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:06.359735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ad7caa-e30b-4541-b7c0-0d108a0b2c99 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:06.360602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ad7caa-e30b-4541-b7c0-0d108a0b2c99 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:06.361307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ad7caa-e30b-4541-b7c0-0d108a0b2c99 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:06.362385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ad7caa-e30b-4541-b7c0-0d108a0b2c99 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:06.391465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-46ad7caa-e30b-4541-b7c0-0d108a0b2c99 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:00:06.478497 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-46ad7caa-e30b-4541-b7c0-0d108a0b2c99 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "DELETE /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 204 len: 0 microversion: 2.1 time: 0.130784 Sep 27 10:00:06.479017 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 452/1024] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:00:06 2022] DELETE /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:00:06.484105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a595abcb-21d6-49e0-a404-f9fd5f60be91 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:06.484882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a595abcb-21d6-49e0-a404-f9fd5f60be91 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:06.491291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a595abcb-21d6-49e0-a404-f9fd5f60be91 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:06.492374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a595abcb-21d6-49e0-a404-f9fd5f60be91 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:06.492971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a595abcb-21d6-49e0-a404-f9fd5f60be91 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:06.494661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a595abcb-21d6-49e0-a404-f9fd5f60be91 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:06.495912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a595abcb-21d6-49e0-a404-f9fd5f60be91 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:06.496067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a595abcb-21d6-49e0-a404-f9fd5f60be91 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:06.732169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-a595abcb-21d6-49e0-a404-f9fd5f60be91 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid f1ff4a24-1251-4827-955f-159c665a1d57 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:00:06.760536 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a595abcb-21d6-49e0-a404-f9fd5f60be91 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 2050 microversion: 2.25 time: 0.280243 Sep 27 10:00:06.761652 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 453/1025] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:06 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 2050 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:06.767586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b056fa9-a452-46b3-9ade-8760500b46df tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:06.768672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3b056fa9-a452-46b3-9ade-8760500b46df tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:06.777404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b056fa9-a452-46b3-9ade-8760500b46df tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:06.777995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b056fa9-a452-46b3-9ade-8760500b46df tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:06.778731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b056fa9-a452-46b3-9ade-8760500b46df tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:06.779669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b056fa9-a452-46b3-9ade-8760500b46df tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:06.780607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b056fa9-a452-46b3-9ade-8760500b46df tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:06.781353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b056fa9-a452-46b3-9ade-8760500b46df tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:07.199098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3b056fa9-a452-46b3-9ade-8760500b46df tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:07.208577 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b056fa9-a452-46b3-9ade-8760500b46df tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1316 microversion: 2.1 time: 0.445698 Sep 27 10:00:07.220715 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 454/1026] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:06 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1316 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:07.220715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c40b2251-9b39-40cc-8bbb-0b526332cf70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:07.220715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c40b2251-9b39-40cc-8bbb-0b526332cf70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:07.224695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c40b2251-9b39-40cc-8bbb-0b526332cf70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:07.224695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c40b2251-9b39-40cc-8bbb-0b526332cf70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:07.224695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c40b2251-9b39-40cc-8bbb-0b526332cf70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:07.224695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c40b2251-9b39-40cc-8bbb-0b526332cf70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:07.225135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c40b2251-9b39-40cc-8bbb-0b526332cf70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:07.225135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c40b2251-9b39-40cc-8bbb-0b526332cf70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:07.415034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c40b2251-9b39-40cc-8bbb-0b526332cf70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:07.425183 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c40b2251-9b39-40cc-8bbb-0b526332cf70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1310 microversion: 2.1 time: 0.213963 Sep 27 10:00:07.439913 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 455/1027] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:07 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1310 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:07.445391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b773e6b0-6ca2-44e3-8ab3-e62fc124cd18 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:07.446021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b773e6b0-6ca2-44e3-8ab3-e62fc124cd18 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: f769af8b-472a-4950-b4ae-8c0183b23c1e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:07.452898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b773e6b0-6ca2-44e3-8ab3-e62fc124cd18 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:07.453404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b773e6b0-6ca2-44e3-8ab3-e62fc124cd18 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:07.458635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b773e6b0-6ca2-44e3-8ab3-e62fc124cd18 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:07.459807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b773e6b0-6ca2-44e3-8ab3-e62fc124cd18 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:07.460299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b773e6b0-6ca2-44e3-8ab3-e62fc124cd18 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:07.462907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b773e6b0-6ca2-44e3-8ab3-e62fc124cd18 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:07.776086 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9b039fcb-9506-475d-9823-8c30dbdee257 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 2.464141 Sep 27 10:00:07.786465 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 573/1028] 10.222.0.26 () {60 vars in 1170 bytes} [Tue Sep 27 10:00:05 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 2468 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:07.791792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fba5f2a5-aa51-4b7d-8404-1629bf341ce8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:07.798647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba5f2a5-aa51-4b7d-8404-1629bf341ce8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:07.799178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba5f2a5-aa51-4b7d-8404-1629bf341ce8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:07.799618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba5f2a5-aa51-4b7d-8404-1629bf341ce8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:07.801407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba5f2a5-aa51-4b7d-8404-1629bf341ce8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:07.803985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba5f2a5-aa51-4b7d-8404-1629bf341ce8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:07.804610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba5f2a5-aa51-4b7d-8404-1629bf341ce8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:07.872447 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fba5f2a5-aa51-4b7d-8404-1629bf341ce8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 424 microversion: 2.25 time: 0.093180 Sep 27 10:00:07.878667 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 574/1029] 10.222.0.26 () {60 vars in 1217 bytes} [Tue Sep 27 10:00:07 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 424 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:00:07.885283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c31923db-7806-4e1b-8fd9-a00f0db76533 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:07.890625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c31923db-7806-4e1b-8fd9-a00f0db76533 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:07.898230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31923db-7806-4e1b-8fd9-a00f0db76533 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:07.904796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31923db-7806-4e1b-8fd9-a00f0db76533 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:07.906269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31923db-7806-4e1b-8fd9-a00f0db76533 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:07.907318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31923db-7806-4e1b-8fd9-a00f0db76533 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:07.908026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31923db-7806-4e1b-8fd9-a00f0db76533 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:07.908716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31923db-7806-4e1b-8fd9-a00f0db76533 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:08.000725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b773e6b0-6ca2-44e3-8ab3-e62fc124cd18 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:08.020688 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b773e6b0-6ca2-44e3-8ab3-e62fc124cd18 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e" status: 200 len: 1475 microversion: 2.1 time: 0.579197 Sep 27 10:00:08.036828 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 456/1030] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:07 2022] GET /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e => generated 1475 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:08.036828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.1.103"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:08.040742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:08.053183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:08.053600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:08.054129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:08.055873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:08.056160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:08.056687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:08.089068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 10:00:08.089860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquired lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 10:00:08.167542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c31923db-7806-4e1b-8fd9-a00f0db76533 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:08.179253 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c31923db-7806-4e1b-8fd9-a00f0db76533 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1379 microversion: 2.1 time: 0.299102 Sep 27 10:00:08.181006 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 575/1031] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:07 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1379 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:08.186503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e15ba47-0af7-459a-be1d-a6d1b3d26c55 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": "auto"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:08.189279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8e15ba47-0af7-459a-be1d-a6d1b3d26c55 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:08.196894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e15ba47-0af7-459a-be1d-a6d1b3d26c55 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:08.197613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e15ba47-0af7-459a-be1d-a6d1b3d26c55 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:08.198236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e15ba47-0af7-459a-be1d-a6d1b3d26c55 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:08.199104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e15ba47-0af7-459a-be1d-a6d1b3d26c55 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:08.199706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e15ba47-0af7-459a-be1d-a6d1b3d26c55 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:08.200770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e15ba47-0af7-459a-be1d-a6d1b3d26c55 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:08.264344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-8e15ba47-0af7-459a-be1d-a6d1b3d26c55 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'flavor' on Instance uuid f1ff4a24-1251-4827-955f-159c665a1d57 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:00:08.297233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8e15ba47-0af7-459a-be1d-a6d1b3d26c55 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Going to try to live migrate instance to nch1-master-1 {{(pid=145968) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Sep 27 10:00:09.324210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG neutronclient.v2_0.client [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Error message: {"NeutronError": {"type": "FloatingIPPortAlreadyAssociated", "message": "Cannot associate floating IP 172.35.1.103 (0eff6de9-d72e-4f0a-992e-3cc80c214ce4) with port dbb7e99b-8507-4a96-bb0a-c442351f9baf using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network cf91963d-b887-4e7e-bee4-e38f7a5f86af.", "detail": ""}} {{(pid=145969) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:00:09.324833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Releasing lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 10:00:09.325466 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] HTTP exception thrown: Cannot associate floating IP 172.35.1.103 (0eff6de9-d72e-4f0a-992e-3cc80c214ce4) with port dbb7e99b-8507-4a96-bb0a-c442351f9baf using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network cf91963d-b887-4e7e-bee4-e38f7a5f86af. Sep 27 10:00:09.325466 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-84a35fd5-26ae-4264-8f67-1cb27a3b3916'] Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Request method failure captured: Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action HTTP/1.1 Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 46 Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsl2HBK4xoqppYGzeaAO7kPvHRg88PVWis0qMqDneOM_OKStDvMkG8ww5OZe2eE7SZsEKq1Z20hZloERWHRvWtjcUK_NSSf7U3j3z9zRcLFKE6QCJ9Xdxwm-88X8IE0YnmRQzAKMHsGmX1yTr93BcmrSLa3nD5rXGSVM9JulDbR8Deg Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2b3380f3fa2b494da649aaa809e11368 Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:00:09.330241 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2b3380f3fa2b494da649aaa809e11368"}]}] Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2b3380f3fa2b494da649aaa809e11368 Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 57365b62b3294810b8a030d2a26cc451 Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: {"addFloatingIp": {"address": "172.35.1.103"}} Sep 27 10:00:09.331641 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: exception: Cannot associate floating IP 172.35.1.103 (0eff6de9-d72e-4f0a-992e-3cc80c214ce4) with port dbb7e99b-8507-4a96-bb0a-c442351f9baf using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network cf91963d-b887-4e7e-bee4-e38f7a5f86af. Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-84a35fd5-26ae-4264-8f67-1cb27a3b3916'] Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'a7bc6edb-7fc4-463d-8490-8800e2922d12', 'body': {'addFloatingIp': {'address': '172.35.1.103'}}} Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2703, in associate_floating_ip Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi client.update_floatingip(fip['id'], {'floatingip': param}) Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1016, in update_floatingip Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._update_resource(self.floatingip_path % (floatingip), Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 2548, in _update_resource Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.put(path, **kwargs) Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:00:09.332557 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 365, in put Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.retry_request("PUT", action, body=body, Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:00:09.333801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Cannot associate floating IP 172.35.1.103 (0eff6de9-d72e-4f0a-992e-3cc80c214ce4) with port dbb7e99b-8507-4a96-bb0a-c442351f9baf using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network cf91963d-b887-4e7e-bee4-e38f7a5f86af. Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-84a35fd5-26ae-4264-8f67-1cb27a3b3916'] Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 239, in _add_floating_ip Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.network_api.associate_floating_ip(context, instance, Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 165, in wrapper Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = f(self, context, *args, **kwargs) Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2705, in associate_floating_ip Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpAssociateFailed(str(e)) Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpAssociateFailed: Cannot associate floating IP 172.35.1.103 (0eff6de9-d72e-4f0a-992e-3cc80c214ce4) with port dbb7e99b-8507-4a96-bb0a-c442351f9baf using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network cf91963d-b887-4e7e-bee4-e38f7a5f86af. Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-84a35fd5-26ae-4264-8f67-1cb27a3b3916'] Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:09.335016 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 246, in _add_floating_ip Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=e.format_message()) Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Cannot associate floating IP 172.35.1.103 (0eff6de9-d72e-4f0a-992e-3cc80c214ce4) with port dbb7e99b-8507-4a96-bb0a-c442351f9baf using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network cf91963d-b887-4e7e-bee4-e38f7a5f86af. Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-84a35fd5-26ae-4264-8f67-1cb27a3b3916'] Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:09.338205 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:09.340623 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:09.340623 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:09.340623 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:09.340623 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:09.340623 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:09.340623 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:09.340623 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot associate floating IP 172.35.1.103 (0eff6de9-d72e-4f0a-992e-3cc80c214ce4) with port dbb7e99b-8507-4a96-bb0a-c442351f9baf using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network cf91963d-b887-4e7e-bee4-e38f7a5f86af. Sep 27 10:00:09.340623 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-84a35fd5-26ae-4264-8f67-1cb27a3b3916'] Sep 27 10:00:09.340623 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:09.340623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Returning 400 to user: Cannot associate floating IP 172.35.1.103 (0eff6de9-d72e-4f0a-992e-3cc80c214ce4) with port dbb7e99b-8507-4a96-bb0a-c442351f9baf using fixed IP 172.20.0.8, as that fixed IP already has a floating IP on external network cf91963d-b887-4e7e-bee4-e38f7a5f86af. Sep 27 10:00:09.340623 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-84a35fd5-26ae-4264-8f67-1cb27a3b3916'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:09.342562 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9914a94f-78e4-4f5a-8a93-733d714ced2b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action" status: 400 len: 381 microversion: 2.1 time: 1.305856 Sep 27 10:00:09.342562 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 457/1032] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:00:08 2022] POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action => generated 381 bytes in 1308 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:00:09.353688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-38f3c445-0794-43ce-bfc0-d268507ae2af tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:09.354932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-38f3c445-0794-43ce-bfc0-d268507ae2af tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:09.390069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38f3c445-0794-43ce-bfc0-d268507ae2af tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:09.392441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38f3c445-0794-43ce-bfc0-d268507ae2af tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:09.406012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38f3c445-0794-43ce-bfc0-d268507ae2af tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:09.412950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38f3c445-0794-43ce-bfc0-d268507ae2af tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:09.419985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38f3c445-0794-43ce-bfc0-d268507ae2af tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:09.421142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38f3c445-0794-43ce-bfc0-d268507ae2af tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:10.047883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-38f3c445-0794-43ce-bfc0-d268507ae2af tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:10.065676 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-38f3c445-0794-43ce-bfc0-d268507ae2af tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1504 microversion: 2.1 time: 0.723522 Sep 27 10:00:10.068040 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 458/1033] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:09 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1504 bytes in 732 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:10.090735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bfc072c5-d732-4383-8ccc-06326433cf43 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:10.091905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bfc072c5-d732-4383-8ccc-06326433cf43 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:10.120830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfc072c5-d732-4383-8ccc-06326433cf43 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:10.122803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfc072c5-d732-4383-8ccc-06326433cf43 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:10.126416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfc072c5-d732-4383-8ccc-06326433cf43 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:10.129417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfc072c5-d732-4383-8ccc-06326433cf43 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:10.131227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfc072c5-d732-4383-8ccc-06326433cf43 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:10.133145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfc072c5-d732-4383-8ccc-06326433cf43 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:10.742273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bfc072c5-d732-4383-8ccc-06326433cf43 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:10.761110 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bfc072c5-d732-4383-8ccc-06326433cf43 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1310 microversion: 2.1 time: 0.687708 Sep 27 10:00:10.763100 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 459/1034] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:10 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1310 bytes in 694 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:10.787405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:10.796064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: f769af8b-472a-4950-b4ae-8c0183b23c1e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:10.833232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:10.833733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:10.834169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:10.834964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:10.849896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:10.853259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:11.024710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:11.027236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:11.028021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:11.029814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lazy-loading 'fault' on Instance uuid f769af8b-472a-4950-b4ae-8c0183b23c1e {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:00:11.140815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:11.146950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:11.238413 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-12e9d226-e22d-4658-be01-90136c1fa264 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e" status: 200 len: 1309 microversion: 2.1 time: 0.460053 Sep 27 10:00:11.257034 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 460/1035] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:10 2022] GET /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e => generated 1309 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:11.282257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3e2b8e89-7ed7-4efd-b7d3-76efa5a70f38 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:11.287150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e2b8e89-7ed7-4efd-b7d3-76efa5a70f38 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:11.306281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e2b8e89-7ed7-4efd-b7d3-76efa5a70f38 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:11.309029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e2b8e89-7ed7-4efd-b7d3-76efa5a70f38 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:11.309676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e2b8e89-7ed7-4efd-b7d3-76efa5a70f38 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:11.311536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e2b8e89-7ed7-4efd-b7d3-76efa5a70f38 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:11.312267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e2b8e89-7ed7-4efd-b7d3-76efa5a70f38 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:11.312949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e2b8e89-7ed7-4efd-b7d3-76efa5a70f38 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:11.911432 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e15ba47-0af7-459a-be1d-a6d1b3d26c55 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57/action" status: 202 len: 0 microversion: 2.25 time: 3.729072 Sep 27 10:00:11.914304 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 576/1036] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:00:08 2022] POST /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57/action => generated 0 bytes in 3733 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:00:11.921527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d877fec3-d4a3-4371-8d31-4baff3eea700 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '.version_select at 0x7f7f63797e50>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:11.941273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3e2b8e89-7ed7-4efd-b7d3-76efa5a70f38 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:12.021794 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3e2b8e89-7ed7-4efd-b7d3-76efa5a70f38 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1379 microversion: 2.1 time: 0.755645 Sep 27 10:00:12.038416 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 461/1037] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:11 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1379 bytes in 781 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:12.076823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-8a0a03da-7fc5-4cf6-8633-663b4019b397 req-f410148d-d5dc-40a4-a68f-116831acce57 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f769af8b-472a-4950-b4ae-8c0183b23c1e", "name": "network-vif-deleted", "tag": "676ed625-8a34-4b31-8384-30e066fc7f52"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:12.127293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a0a03da-7fc5-4cf6-8633-663b4019b397 req-f410148d-d5dc-40a4-a68f-116831acce57 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:12.134365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a0a03da-7fc5-4cf6-8633-663b4019b397 req-f410148d-d5dc-40a4-a68f-116831acce57 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:12.135253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a0a03da-7fc5-4cf6-8633-663b4019b397 req-f410148d-d5dc-40a4-a68f-116831acce57 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:12.179403 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8a0a03da-7fc5-4cf6-8633-663b4019b397 req-f410148d-d5dc-40a4-a68f-116831acce57 service nova] Creating event network-vif-deleted:676ed625-8a34-4b31-8384-30e066fc7f52 for instance f769af8b-472a-4950-b4ae-8c0183b23c1e on nch1-master-1 Sep 27 10:00:12.193157 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-8a0a03da-7fc5-4cf6-8633-663b4019b397 req-f410148d-d5dc-40a4-a68f-116831acce57 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.152562 Sep 27 10:00:12.202533 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 462/1038] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:12.221577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6dcd60a5-77a3-4339-a495-a38ca8beb64e tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:12.222463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6dcd60a5-77a3-4339-a495-a38ca8beb64e tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:12.254120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dcd60a5-77a3-4339-a495-a38ca8beb64e tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:12.255139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dcd60a5-77a3-4339-a495-a38ca8beb64e tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:12.255957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dcd60a5-77a3-4339-a495-a38ca8beb64e tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:12.261769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dcd60a5-77a3-4339-a495-a38ca8beb64e tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:12.262667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dcd60a5-77a3-4339-a495-a38ca8beb64e tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:12.263343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dcd60a5-77a3-4339-a495-a38ca8beb64e tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:12.379024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6dcd60a5-77a3-4339-a495-a38ca8beb64e tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:00:12.650589 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6dcd60a5-77a3-4339-a495-a38ca8beb64e tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "DELETE /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 204 len: 0 microversion: 2.1 time: 0.446043 Sep 27 10:00:12.665853 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 463/1039] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:00:12 2022] DELETE /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:00:12.705417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-05c75eb3-b14e-4b15-b640-a8b6d9dce383 req-acdcc6a2-157f-4bc2-b78f-0c03d0e8302f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "67574e92-7d2a-4f14-8448-5ff3278cb803", "name": "network-vif-deleted", "tag": "aa655eea-e771-44bf-9f90-0e21077a0242"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:12.753264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-05c75eb3-b14e-4b15-b640-a8b6d9dce383 req-acdcc6a2-157f-4bc2-b78f-0c03d0e8302f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:12.753890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-05c75eb3-b14e-4b15-b640-a8b6d9dce383 req-acdcc6a2-157f-4bc2-b78f-0c03d0e8302f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:12.757240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-05c75eb3-b14e-4b15-b640-a8b6d9dce383 req-acdcc6a2-157f-4bc2-b78f-0c03d0e8302f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:12.779298 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-05c75eb3-b14e-4b15-b640-a8b6d9dce383 req-acdcc6a2-157f-4bc2-b78f-0c03d0e8302f service nova] Creating event network-vif-deleted:aa655eea-e771-44bf-9f90-0e21077a0242 for instance 67574e92-7d2a-4f14-8448-5ff3278cb803 on nch2-master-1 Sep 27 10:00:12.788117 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-05c75eb3-b14e-4b15-b640-a8b6d9dce383 req-acdcc6a2-157f-4bc2-b78f-0c03d0e8302f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.115323 Sep 27 10:00:12.790197 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 464/1040] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:12.814346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-653755a8-baf5-4dad-9020-3e34ee562f70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:12.815878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-653755a8-baf5-4dad-9020-3e34ee562f70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:12.826863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-653755a8-baf5-4dad-9020-3e34ee562f70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:12.828056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-653755a8-baf5-4dad-9020-3e34ee562f70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:12.831792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-653755a8-baf5-4dad-9020-3e34ee562f70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:12.832958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-653755a8-baf5-4dad-9020-3e34ee562f70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:12.834116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-653755a8-baf5-4dad-9020-3e34ee562f70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:12.839347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-653755a8-baf5-4dad-9020-3e34ee562f70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:13.197994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-653755a8-baf5-4dad-9020-3e34ee562f70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:13.216590 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-653755a8-baf5-4dad-9020-3e34ee562f70 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1310 microversion: 2.1 time: 0.424433 Sep 27 10:00:13.229550 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 465/1041] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:12 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1310 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:13.243931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:13.267043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:13.288102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:13.290588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:13.291240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:13.292200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:13.295789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:13.296431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:13.644091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:13.654054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:13.666191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:13.667592 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-abaad710-4485-4ef1-8b3f-d0bc363afee6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1567 microversion: 2.25 time: 0.436757 Sep 27 10:00:13.674769 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 466/1042] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:13 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1567 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:13.697501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1b8dda33-5b8c-43e3-84f5-9d5e09babc7b req-4fbc4224-383e-4e80-86e9-23460393a2bf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "16cd150f-c67a-4a9e-9654-ba6bb9e1517a", "name": "network-vif-plugged", "status": "completed", "tag": "fa2d1e07-1a2a-40d0-baa6-738c999a9e3a"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:13.725860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1b8dda33-5b8c-43e3-84f5-9d5e09babc7b req-4fbc4224-383e-4e80-86e9-23460393a2bf service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:13.726429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1b8dda33-5b8c-43e3-84f5-9d5e09babc7b req-4fbc4224-383e-4e80-86e9-23460393a2bf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:13.727514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1b8dda33-5b8c-43e3-84f5-9d5e09babc7b req-4fbc4224-383e-4e80-86e9-23460393a2bf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:13.774927 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1b8dda33-5b8c-43e3-84f5-9d5e09babc7b req-4fbc4224-383e-4e80-86e9-23460393a2bf service nova] Creating event network-vif-plugged:fa2d1e07-1a2a-40d0-baa6-738c999a9e3a for instance 16cd150f-c67a-4a9e-9654-ba6bb9e1517a on nch1-master-1 Sep 27 10:00:13.795950 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1b8dda33-5b8c-43e3-84f5-9d5e09babc7b req-4fbc4224-383e-4e80-86e9-23460393a2bf service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.118636 Sep 27 10:00:13.807363 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 467/1043] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:13.828974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:13.831550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: f769af8b-472a-4950-b4ae-8c0183b23c1e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:13.859165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:13.862260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:13.865924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:13.867682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:13.870210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:13.872451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:13.944627 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] HTTP exception thrown: Instance f769af8b-472a-4950-b4ae-8c0183b23c1e could not be found. Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Request method failure captured: Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e HTTP/1.1 Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjuzA8awi02tVpJBbCxyCTwC-N-auCdITAAm8nxhyCwHUSTrFjjx6tbQkA83IeLeCJH2BD7omoS-ML8vYESEbbQcIu0WhsX0gHiWVx2nVId80d2oExNsA2vGHGUS2rPtxB3q_M7eSz3mw7rO-vyGP61qyPfw7K0p8tFBuwGQZNzBLo Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:00:13.949526 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c1ac7b64c68e4896b2c631baacdbe2be"}]}] Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 00423eb1e4c24763ae0a546225ac1dc7 Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance f769af8b-472a-4950-b4ae-8c0183b23c1e could not be found. Sep 27 10:00:13.951241 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'f769af8b-472a-4950-b4ae-8c0183b23c1e'} Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f769af8b-472a-4950-b4ae-8c0183b23c1e could not be found. Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:13.952141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f769af8b-472a-4950-b4ae-8c0183b23c1e could not be found. Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f769af8b-472a-4950-b4ae-8c0183b23c1e could not be found. Sep 27 10:00:13.953601 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:13.955014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Returning 404 to user: Instance f769af8b-472a-4950-b4ae-8c0183b23c1e could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:13.955014 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f8bb8217-b139-4865-81fc-5d269b7cd66f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e" status: 404 len: 111 microversion: 2.1 time: 0.141131 Sep 27 10:00:13.959119 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 468/1044] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:13 2022] GET /compute/v2.1/servers/f769af8b-472a-4950-b4ae-8c0183b23c1e => generated 111 bytes in 148 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:13.974558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-60ec4455-854e-4ddf-ba8d-ff533d6d8dad tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:13.979582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-60ec4455-854e-4ddf-ba8d-ff533d6d8dad tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:14.003868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60ec4455-854e-4ddf-ba8d-ff533d6d8dad tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:14.006681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60ec4455-854e-4ddf-ba8d-ff533d6d8dad tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:14.007343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60ec4455-854e-4ddf-ba8d-ff533d6d8dad tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:14.010194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60ec4455-854e-4ddf-ba8d-ff533d6d8dad tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:14.010847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60ec4455-854e-4ddf-ba8d-ff533d6d8dad tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:14.013683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60ec4455-854e-4ddf-ba8d-ff533d6d8dad tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:14.355211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-60ec4455-854e-4ddf-ba8d-ff533d6d8dad tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:14.365936 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-60ec4455-854e-4ddf-ba8d-ff533d6d8dad tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1510 microversion: 2.1 time: 0.408629 Sep 27 10:00:14.379498 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 469/1045] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:13 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1510 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:14.397626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9c1feeda-cde1-40d0-9e43-76c918f8d0ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:14.398666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9c1feeda-cde1-40d0-9e43-76c918f8d0ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:14.408692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1feeda-cde1-40d0-9e43-76c918f8d0ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:14.410109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1feeda-cde1-40d0-9e43-76c918f8d0ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:14.412433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1feeda-cde1-40d0-9e43-76c918f8d0ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:14.413477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1feeda-cde1-40d0-9e43-76c918f8d0ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:14.414387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1feeda-cde1-40d0-9e43-76c918f8d0ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:14.415063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1feeda-cde1-40d0-9e43-76c918f8d0ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:14.642619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9c1feeda-cde1-40d0-9e43-76c918f8d0ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:14.660526 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9c1feeda-cde1-40d0-9e43-76c918f8d0ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1379 microversion: 2.1 time: 0.266732 Sep 27 10:00:14.663349 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 470/1046] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:14 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1379 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:14.694496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:14.697425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 67574e92-7d2a-4f14-8448-5ff3278cb803] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:14.705567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:14.706797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:14.707386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:14.708263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:14.708807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:14.709779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:14.749096 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] HTTP exception thrown: Instance 67574e92-7d2a-4f14-8448-5ff3278cb803 could not be found. Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Request method failure captured: Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 HTTP/1.1 Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjuzA8awi02tVpJBbCxyCTwC-N-auCdITAAm8nxhyCwHUSTrFjjx6tbQkA83IeLeCJH2BD7omoS-ML8vYESEbbQcIu0WhsX0gHiWVx2nVId80d2oExNsA2vGHGUS2rPtxB3q_M7eSz3mw7rO-vyGP61qyPfw7K0p8tFBuwGQZNzBLo Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:00:14.751052 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c1ac7b64c68e4896b2c631baacdbe2be"}]}] Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 00423eb1e4c24763ae0a546225ac1dc7 Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 67574e92-7d2a-4f14-8448-5ff3278cb803 could not be found. Sep 27 10:00:14.752374 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '67574e92-7d2a-4f14-8448-5ff3278cb803'} Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 67574e92-7d2a-4f14-8448-5ff3278cb803 could not be found. Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:14.753148 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 67574e92-7d2a-4f14-8448-5ff3278cb803 could not be found. Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 67574e92-7d2a-4f14-8448-5ff3278cb803 could not be found. Sep 27 10:00:14.754366 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:14.760337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Returning 404 to user: Instance 67574e92-7d2a-4f14-8448-5ff3278cb803 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:14.766777 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f8a0a6a3-2d04-4756-a64f-a4ed919c47fa tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803" status: 404 len: 111 microversion: 2.1 time: 0.096311 Sep 27 10:00:14.767825 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 471/1047] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:14 2022] GET /compute/v2.1/servers/67574e92-7d2a-4f14-8448-5ff3278cb803 => generated 111 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:14.777084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e618c9db-43d9-4ac0-9cd3-a9d68d453462 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:14.778049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e618c9db-43d9-4ac0-9cd3-a9d68d453462 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:14.791839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e618c9db-43d9-4ac0-9cd3-a9d68d453462 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:14.793406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e618c9db-43d9-4ac0-9cd3-a9d68d453462 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:14.795951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e618c9db-43d9-4ac0-9cd3-a9d68d453462 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:14.796923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e618c9db-43d9-4ac0-9cd3-a9d68d453462 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:14.797573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e618c9db-43d9-4ac0-9cd3-a9d68d453462 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:14.798268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e618c9db-43d9-4ac0-9cd3-a9d68d453462 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:15.083710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e618c9db-43d9-4ac0-9cd3-a9d68d453462 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:15.103606 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e618c9db-43d9-4ac0-9cd3-a9d68d453462 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1310 microversion: 2.1 time: 0.334314 Sep 27 10:00:15.109845 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 472/1048] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:14 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1310 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:15.120909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:15.122116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:15.135183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:15.137540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:15.140106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:15.141530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:15.145630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:15.147014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:15.291644 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d877fec3-d4a3-4371-8d31-4baff3eea700 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/0eff6de9-d72e-4f0a-992e-3cc80c214ce4" status: 202 len: 0 microversion: 2.1 time: 3.375341 Sep 27 10:00:15.296114 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 577/1049] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:00:11 2022] DELETE /compute/v2.1/os-floating-ips/0eff6de9-d72e-4f0a-992e-3cc80c214ce4 => generated 0 bytes in 3381 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:00:15.301970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f9e59818-5cba-4b60-ba34-717aea8cec71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:15.337226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:15.349090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:15.350726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:15.354136 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-94cae6e2-2352-43a0-94bd-d343e9088dc0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1567 microversion: 2.25 time: 0.242820 Sep 27 10:00:15.361464 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 473/1050] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:15 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1567 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:15.361464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Action: 'action', calling method: .version_select at 0x7f7f693fbf70>, body: {"removeFloatingIp": {"address": "172.35.1.66"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:15.396914 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f9e59818-5cba-4b60-ba34-717aea8cec71 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/86f44e48-288d-42e4-91ee-4d99b6fa5532" status: 204 len: 0 microversion: 2.1 time: 0.098260 Sep 27 10:00:15.401349 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 578/1051] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:00:15 2022] DELETE /compute/v2.1/os-server-groups/86f44e48-288d-42e4-91ee-4d99b6fa5532 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:00:15.413467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b2ad2af7-117b-435d-a7bf-d458fb855a11 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:15.415293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b2ad2af7-117b-435d-a7bf-d458fb855a11 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:15.429899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2ad2af7-117b-435d-a7bf-d458fb855a11 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:15.435532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2ad2af7-117b-435d-a7bf-d458fb855a11 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:15.436194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2ad2af7-117b-435d-a7bf-d458fb855a11 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:15.437134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2ad2af7-117b-435d-a7bf-d458fb855a11 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:15.437924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2ad2af7-117b-435d-a7bf-d458fb855a11 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:15.438515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2ad2af7-117b-435d-a7bf-d458fb855a11 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:15.634187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:15.643092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:15.643899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:15.644549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:15.645454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:15.646081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:15.646699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:15.813275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b2ad2af7-117b-435d-a7bf-d458fb855a11 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c815e82b67e8436b9bd4a06fe2d03b49', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e9bd76ccd7824e989252951e381a17cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:15.821341 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b2ad2af7-117b-435d-a7bf-d458fb855a11 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 200 len: 1510 microversion: 2.1 time: 0.416152 Sep 27 10:00:15.830632 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 579/1052] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:15 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 1510 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:15.843558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6fe3c862-fbe5-451b-950b-eec8de1f8bda tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:15.850047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6fe3c862-fbe5-451b-950b-eec8de1f8bda tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 0e538bab-a9f3-488f-9bef-801f2429f73f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:15.861191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe3c862-fbe5-451b-950b-eec8de1f8bda tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:15.865245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe3c862-fbe5-451b-950b-eec8de1f8bda tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:15.865774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe3c862-fbe5-451b-950b-eec8de1f8bda tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:15.866623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe3c862-fbe5-451b-950b-eec8de1f8bda tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:15.867330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe3c862-fbe5-451b-950b-eec8de1f8bda tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:15.867956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fe3c862-fbe5-451b-950b-eec8de1f8bda tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:15.899242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6fe3c862-fbe5-451b-950b-eec8de1f8bda tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 0e538bab-a9f3-488f-9bef-801f2429f73f] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:00:15.987139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 10:00:15.987620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquired lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 10:00:16.047937 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6fe3c862-fbe5-451b-950b-eec8de1f8bda tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f" status: 204 len: 0 microversion: 2.1 time: 0.215010 Sep 27 10:00:16.053715 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 580/1053] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:00:15 2022] DELETE /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:00:16.082769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33175f1f-43a0-47dd-a46d-98436b0af40b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:16.088094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33175f1f-43a0-47dd-a46d-98436b0af40b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:16.095986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33175f1f-43a0-47dd-a46d-98436b0af40b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:16.097597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33175f1f-43a0-47dd-a46d-98436b0af40b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:16.098338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33175f1f-43a0-47dd-a46d-98436b0af40b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:16.099833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33175f1f-43a0-47dd-a46d-98436b0af40b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:16.100533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33175f1f-43a0-47dd-a46d-98436b0af40b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:16.101300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33175f1f-43a0-47dd-a46d-98436b0af40b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:16.718478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33175f1f-43a0-47dd-a46d-98436b0af40b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:16.789498 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33175f1f-43a0-47dd-a46d-98436b0af40b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1379 microversion: 2.1 time: 0.726576 Sep 27 10:00:16.802401 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 581/1054] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:16 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1379 bytes in 747 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:16.821865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4c9f1691-037e-47ed-9855-0664d253a836 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:16.825795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4c9f1691-037e-47ed-9855-0664d253a836 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:16.840359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9f1691-037e-47ed-9855-0664d253a836 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:16.842417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9f1691-037e-47ed-9855-0664d253a836 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:16.843064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9f1691-037e-47ed-9855-0664d253a836 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:16.845526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9f1691-037e-47ed-9855-0664d253a836 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:16.846332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9f1691-037e-47ed-9855-0664d253a836 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:16.847097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9f1691-037e-47ed-9855-0664d253a836 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:16.895668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4c9f1691-037e-47ed-9855-0664d253a836 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:00:17.011965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Building network info cache for instance {{(pid=145969) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Sep 27 10:00:17.051762 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4c9f1691-037e-47ed-9855-0664d253a836 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "DELETE /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 204 len: 0 microversion: 2.1 time: 0.246588 Sep 27 10:00:17.053527 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 582/1055] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:00:16 2022] DELETE /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:00:17.067825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6675f269-ed16-46e8-8cad-5be3f7ca9033 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:17.068802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6675f269-ed16-46e8-8cad-5be3f7ca9033 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:17.088162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6675f269-ed16-46e8-8cad-5be3f7ca9033 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:17.090559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6675f269-ed16-46e8-8cad-5be3f7ca9033 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:17.093270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6675f269-ed16-46e8-8cad-5be3f7ca9033 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:17.096685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6675f269-ed16-46e8-8cad-5be3f7ca9033 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:17.097615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6675f269-ed16-46e8-8cad-5be3f7ca9033 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:17.099749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6675f269-ed16-46e8-8cad-5be3f7ca9033 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:17.435845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6675f269-ed16-46e8-8cad-5be3f7ca9033 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:17.454890 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6675f269-ed16-46e8-8cad-5be3f7ca9033 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1493 microversion: 2.1 time: 0.398642 Sep 27 10:00:17.455497 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 583/1056] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:17 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1493 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:17.471099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:17.477496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:17.502290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:17.502793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:17.503225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:17.503948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:17.504396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:17.504845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:17.814267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:17.846761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:17.858336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:17.862166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:17.863403 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-55af21f4-4c5f-40bb-ac2f-884696946a7a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1567 microversion: 2.25 time: 0.404242 Sep 27 10:00:17.869243 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 584/1057] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:17 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1567 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:17.876828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d6f97e40-fc41-4744-9da7-8147604813e7 req-3b5cc9e6-119b-4fc5-9f96-8dd587d2399f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "24be6165-1a70-40c5-beb7-67dd67933a85", "name": "network-vif-deleted", "tag": "d97cf212-d664-46ca-b7be-65423e5322f3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:17.898321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d6f97e40-fc41-4744-9da7-8147604813e7 req-3b5cc9e6-119b-4fc5-9f96-8dd587d2399f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:17.903440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d6f97e40-fc41-4744-9da7-8147604813e7 req-3b5cc9e6-119b-4fc5-9f96-8dd587d2399f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:17.904113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d6f97e40-fc41-4744-9da7-8147604813e7 req-3b5cc9e6-119b-4fc5-9f96-8dd587d2399f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:17.931517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:17.936087 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d6f97e40-fc41-4744-9da7-8147604813e7 req-3b5cc9e6-119b-4fc5-9f96-8dd587d2399f service nova] Creating event network-vif-deleted:d97cf212-d664-46ca-b7be-65423e5322f3 for instance 24be6165-1a70-40c5-beb7-67dd67933a85 on nch2-master-1 Sep 27 10:00:17.950023 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d6f97e40-fc41-4744-9da7-8147604813e7 req-3b5cc9e6-119b-4fc5-9f96-8dd587d2399f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082116 Sep 27 10:00:17.953169 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 585/1058] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:17.959088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3c10d129-e95e-4a1e-9566-06be0a5d3c79 req-1a825b35-23c3-46ea-bfb9-e50f0d2f612b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69ea013c-5143-45ff-81a2-3ed97a8659cd", "name": "network-vif-plugged", "status": "completed", "tag": "96ac21a6-013a-4a5e-9320-82ff90906b76"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:17.985697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3c10d129-e95e-4a1e-9566-06be0a5d3c79 req-1a825b35-23c3-46ea-bfb9-e50f0d2f612b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:17.986506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3c10d129-e95e-4a1e-9566-06be0a5d3c79 req-1a825b35-23c3-46ea-bfb9-e50f0d2f612b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:17.987061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3c10d129-e95e-4a1e-9566-06be0a5d3c79 req-1a825b35-23c3-46ea-bfb9-e50f0d2f612b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:18.025435 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3c10d129-e95e-4a1e-9566-06be0a5d3c79 req-1a825b35-23c3-46ea-bfb9-e50f0d2f612b service nova] Creating event network-vif-plugged:96ac21a6-013a-4a5e-9320-82ff90906b76 for instance 69ea013c-5143-45ff-81a2-3ed97a8659cd on nch1-master-1 Sep 27 10:00:18.031608 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3c10d129-e95e-4a1e-9566-06be0a5d3c79 req-1a825b35-23c3-46ea-bfb9-e50f0d2f612b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.076677 Sep 27 10:00:18.036970 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 586/1059] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:18.046483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:18.048198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:18.061200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:18.062367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:18.063074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:18.064095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:18.064809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:18.066482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:18.085211 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] HTTP exception thrown: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Request method failure captured: Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 HTTP/1.1 Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsmKyA6ZY_0UTI-fk-5sgLa0Rbl9D4N5Yd9gyv5Zlsn6kQCLWqMkvyPGfGKKTvfqGELUcLn_G_8GPGOB472pT5kzc_oEG-A5r9-j-ZRHjHl6fBzDLJdWXmBc3f7s0HbYKtukrsEo0dqSckAwOOSRnEjkC1AukBmXspnpQvD2iS6Sk4g Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:18.087512 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e9bd76ccd7824e989252951e381a17cd"}]}] Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: c815e82b67e8436b9bd4a06fe2d03b49 Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:18.089634 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '24be6165-1a70-40c5-beb7-67dd67933a85'} Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:18.090454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:18.091655 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:18.104505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Returning 404 to user: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:18.105800 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ea94e5a9-eaa6-4d4e-b433-932a1e100233 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 404 len: 111 microversion: 2.1 time: 0.066345 Sep 27 10:00:18.107670 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 587/1060] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:18 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:18.118742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-24d18ab0-8347-4550-9a04-d26b136f82f3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:18.119900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-24d18ab0-8347-4550-9a04-d26b136f82f3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 0e538bab-a9f3-488f-9bef-801f2429f73f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:18.129937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24d18ab0-8347-4550-9a04-d26b136f82f3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:18.131714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24d18ab0-8347-4550-9a04-d26b136f82f3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:18.132738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24d18ab0-8347-4550-9a04-d26b136f82f3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:18.133720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24d18ab0-8347-4550-9a04-d26b136f82f3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:18.134404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24d18ab0-8347-4550-9a04-d26b136f82f3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:18.135092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24d18ab0-8347-4550-9a04-d26b136f82f3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:18.307181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Updating instance_info_cache with network_info: [{"id": "dbb7e99b-8507-4a96-bb0a-c442351f9baf", "address": "fa:16:3e:3b:fc:8a", "network": {"id": "e00d057c-26e1-4ae4-9895-9dbcbbcbbfc3", "bridge": null, "label": "tempest-FloatingIPsAssociationNegativeTestJSON-219801886-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "2b3380f3fa2b494da649aaa809e11368", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapdbb7e99b-85", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=145969) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:00:18.338067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Releasing lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 10:00:18.339501 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-91d3e0b1-9b85-44de-87e3-89455de79264 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action" status: 202 len: 0 microversion: 2.1 time: 2.983219 Sep 27 10:00:18.344899 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 474/1061] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:00:15 2022] POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action => generated 0 bytes in 2988 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Sep 27 10:00:18.366117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-273c0706-841b-428e-9ec7-22f8e114906c req-c29d8cca-92e7-4386-b14f-285f707d65d2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a7bc6edb-7fc4-463d-8490-8800e2922d12", "tag": "dbb7e99b-8507-4a96-bb0a-c442351f9baf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:18.379717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-273c0706-841b-428e-9ec7-22f8e114906c req-c29d8cca-92e7-4386-b14f-285f707d65d2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:18.380600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-273c0706-841b-428e-9ec7-22f8e114906c req-c29d8cca-92e7-4386-b14f-285f707d65d2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:18.381168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-273c0706-841b-428e-9ec7-22f8e114906c req-c29d8cca-92e7-4386-b14f-285f707d65d2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:18.396683 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-273c0706-841b-428e-9ec7-22f8e114906c req-c29d8cca-92e7-4386-b14f-285f707d65d2 service nova] Creating event network-changed:dbb7e99b-8507-4a96-bb0a-c442351f9baf for instance a7bc6edb-7fc4-463d-8490-8800e2922d12 on nch1-master-1 Sep 27 10:00:18.403305 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-273c0706-841b-428e-9ec7-22f8e114906c req-c29d8cca-92e7-4386-b14f-285f707d65d2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056730 Sep 27 10:00:18.404577 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 475/1062] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:18.441682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-24d18ab0-8347-4550-9a04-d26b136f82f3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:18.456267 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-24d18ab0-8347-4550-9a04-d26b136f82f3 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f" status: 200 len: 1474 microversion: 2.1 time: 0.347299 Sep 27 10:00:18.467481 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 588/1063] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:18 2022] GET /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f => generated 1474 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:18.484755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7cbb4824-c466-478a-895c-75dd5e8c52ae tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:18.486269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7cbb4824-c466-478a-895c-75dd5e8c52ae tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:18.494370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cbb4824-c466-478a-895c-75dd5e8c52ae tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:18.496492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cbb4824-c466-478a-895c-75dd5e8c52ae tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:18.498866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cbb4824-c466-478a-895c-75dd5e8c52ae tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:18.500366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cbb4824-c466-478a-895c-75dd5e8c52ae tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:18.501195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cbb4824-c466-478a-895c-75dd5e8c52ae tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:18.501639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cbb4824-c466-478a-895c-75dd5e8c52ae tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:18.674618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0bded3c5-dfc2-407e-a001-4351794e2b22 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:18.675629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0bded3c5-dfc2-407e-a001-4351794e2b22 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:18.692543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bded3c5-dfc2-407e-a001-4351794e2b22 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:18.693373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bded3c5-dfc2-407e-a001-4351794e2b22 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:18.694001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bded3c5-dfc2-407e-a001-4351794e2b22 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:18.694966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bded3c5-dfc2-407e-a001-4351794e2b22 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:18.695780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bded3c5-dfc2-407e-a001-4351794e2b22 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:18.696373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bded3c5-dfc2-407e-a001-4351794e2b22 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:19.057908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7cbb4824-c466-478a-895c-75dd5e8c52ae tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:19.069809 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7cbb4824-c466-478a-895c-75dd5e8c52ae tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1379 microversion: 2.1 time: 0.594038 Sep 27 10:00:19.093368 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 589/1064] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:18 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1379 bytes in 625 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:19.099223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-1885466752", "metadata": {"image_type": "test"}}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:19.103442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:19.112933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:19.113470 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0bded3c5-dfc2-407e-a001-4351794e2b22 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1644 microversion: 2.1 time: 0.707326 Sep 27 10:00:19.114403 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 476/1065] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:18 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1644 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:19.115153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:19.117544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:19.118557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:19.119243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:19.119585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-793559a9-0054-4635-b57e-b7ccd7130217 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '.version_select at 0x7f7f637f3550>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:19.120134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:19.142564 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] HTTP exception thrown: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Request method failure captured: Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85/action HTTP/1.1 Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 112 Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsmKyA6ZY_0UTI-fk-5sgLa0Rbl9D4N5Yd9gyv5Zlsn6kQCLWqMkvyPGfGKKTvfqGELUcLn_G_8GPGOB472pT5kzc_oEG-A5r9-j-ZRHjHl6fBzDLJdWXmBc3f7s0HbYKtukrsEo0dqSckAwOOSRnEjkC1AukBmXspnpQvD2iS6Sk4g Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:19.144721 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e9bd76ccd7824e989252951e381a17cd"}]}] Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: c815e82b67e8436b9bd4a06fe2d03b49 Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-1885466752", "metadata": {"image_type": "test"}}} Sep 27 10:00:19.146106 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '24be6165-1a70-40c5-beb7-67dd67933a85', 'body': {'createImage': {'name': 'tempest-ImagesNegativeTestJSON-image-1885466752', 'metadata': {'image_type': 'test'}}}} Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:19.146871 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1319, in _action_create_image Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:19.148604 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:19.150260 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:19.150260 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:19.150260 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:19.150260 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:19.150260 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:19.150260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Returning 404 to user: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:19.150260 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-84d1a614-249c-463a-aa9d-5220f4c84c35 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "POST /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85/action" status: 404 len: 111 microversion: 2.1 time: 0.052074 Sep 27 10:00:19.151812 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 590/1066] 10.222.0.26 () {60 vars in 1279 bytes} [Tue Sep 27 10:00:19 2022] POST /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85/action => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:19.168690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:19.168690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:19.174980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:19.177320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:19.180107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:19.182862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:19.183501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:19.185599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:19.642626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:19.656123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:19.705926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:19.709407 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8cae101f-332d-4cd0-9ee6-8a9df1e91359 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1567 microversion: 2.25 time: 0.555946 Sep 27 10:00:19.723789 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 591/1067] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:19 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1567 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:19.750290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d7ce92f2-e556-475c-9510-9aa10373fb37 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:19.751122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d7ce92f2-e556-475c-9510-9aa10373fb37 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:19.775186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7ce92f2-e556-475c-9510-9aa10373fb37 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:19.775706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7ce92f2-e556-475c-9510-9aa10373fb37 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:19.776539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7ce92f2-e556-475c-9510-9aa10373fb37 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:19.777321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7ce92f2-e556-475c-9510-9aa10373fb37 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:19.777791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7ce92f2-e556-475c-9510-9aa10373fb37 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:19.778226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7ce92f2-e556-475c-9510-9aa10373fb37 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:20.146485 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d7ce92f2-e556-475c-9510-9aa10373fb37 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1644 microversion: 2.1 time: 0.422032 Sep 27 10:00:20.172695 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 592/1068] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:19 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1644 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:20.198068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7ea54401-f0ca-411a-bd0f-e853b37824e6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-307476482", "metadata": {"image_type": "test"}}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:20.206888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7ea54401-f0ca-411a-bd0f-e853b37824e6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Failed to fetch instance by id tempest-invalid-1437385206 {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2981}} Sep 27 10:00:20.207548 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-7ea54401-f0ca-411a-bd0f-e853b37824e6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] HTTP exception thrown: Instance tempest-invalid-1437385206 could not be found. Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7ea54401-f0ca-411a-bd0f-e853b37824e6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Request method failure captured: Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/tempest-invalid-1437385206/action HTTP/1.1 Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 111 Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsmKyA6ZY_0UTI-fk-5sgLa0Rbl9D4N5Yd9gyv5Zlsn6kQCLWqMkvyPGfGKKTvfqGELUcLn_G_8GPGOB472pT5kzc_oEG-A5r9-j-ZRHjHl6fBzDLJdWXmBc3f7s0HbYKtukrsEo0dqSckAwOOSRnEjkC1AukBmXspnpQvD2iS6Sk4g Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:20.213138 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e9bd76ccd7824e989252951e381a17cd"}]}] Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: c815e82b67e8436b9bd4a06fe2d03b49 Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: {"createImage": {"name": "tempest-ImagesNegativeTestJSON-image-307476482", "metadata": {"image_type": "test"}}} Sep 27 10:00:20.214756 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance tempest-invalid-1437385206 could not be found. Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'tempest-invalid-1437385206', 'body': {'createImage': {'name': 'tempest-ImagesNegativeTestJSON-image-307476482', 'metadata': {'image_type': 'test'}}}} Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance tempest-invalid-1437385206 could not be found. Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:20.215638 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1319, in _action_create_image Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance tempest-invalid-1437385206 could not be found. Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance tempest-invalid-1437385206 could not be found. Sep 27 10:00:20.217132 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:20.220293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7ea54401-f0ca-411a-bd0f-e853b37824e6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Returning 404 to user: Instance tempest-invalid-1437385206 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:20.222250 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7ea54401-f0ca-411a-bd0f-e853b37824e6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "POST /compute/v2.1/servers/tempest-invalid-1437385206/action" status: 404 len: 101 microversion: 2.1 time: 0.047130 Sep 27 10:00:20.226650 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 593/1069] 10.222.0.26 () {60 vars in 1249 bytes} [Tue Sep 27 10:00:20 2022] POST /compute/v2.1/servers/tempest-invalid-1437385206/action => generated 101 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:20.257222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:20.282149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 0e538bab-a9f3-488f-9bef-801f2429f73f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:20.336612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:20.339119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:20.339610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:20.340376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:20.341151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:20.343493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:20.407697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:20.411301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:20.413847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:20.415714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lazy-loading 'fault' on Instance uuid 0e538bab-a9f3-488f-9bef-801f2429f73f {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:00:20.489674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:20.506342 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-43dffbb8-0c2c-40fe-87b1-6bd24e6f1222 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f" status: 200 len: 1309 microversion: 2.1 time: 0.278219 Sep 27 10:00:20.517138 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 594/1070] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:20 2022] GET /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f => generated 1309 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:20.567872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-79672bd1-1afd-4995-a989-2a4b57390ae5 req-426a8f77-b8fa-4baa-ba83-7b5affcb371e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f1ff4a24-1251-4827-955f-159c665a1d57", "name": "network-vif-unplugged", "status": "completed", "tag": "9c2668a6-7fb1-48ea-8f26-e6c77b4e01f9"}, {"name": "network-changed", "server_uuid": "f1ff4a24-1251-4827-955f-159c665a1d57", "tag": "9c2668a6-7fb1-48ea-8f26-e6c77b4e01f9"}, {"server_uuid": "0e538bab-a9f3-488f-9bef-801f2429f73f", "name": "network-vif-deleted", "tag": "9ff31270-663b-4d9c-a7e1-90a7d2b0c7fb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:20.585436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-79672bd1-1afd-4995-a989-2a4b57390ae5 req-426a8f77-b8fa-4baa-ba83-7b5affcb371e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:20.586151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-79672bd1-1afd-4995-a989-2a4b57390ae5 req-426a8f77-b8fa-4baa-ba83-7b5affcb371e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:20.587310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-79672bd1-1afd-4995-a989-2a4b57390ae5 req-426a8f77-b8fa-4baa-ba83-7b5affcb371e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:20.622886 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-79672bd1-1afd-4995-a989-2a4b57390ae5 req-426a8f77-b8fa-4baa-ba83-7b5affcb371e service nova] Creating event network-vif-unplugged:9c2668a6-7fb1-48ea-8f26-e6c77b4e01f9 for instance f1ff4a24-1251-4827-955f-159c665a1d57 on nch2-master-1 Sep 27 10:00:20.623764 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-79672bd1-1afd-4995-a989-2a4b57390ae5 req-426a8f77-b8fa-4baa-ba83-7b5affcb371e service nova] Creating event network-changed:9c2668a6-7fb1-48ea-8f26-e6c77b4e01f9 for instance f1ff4a24-1251-4827-955f-159c665a1d57 on nch2-master-1 Sep 27 10:00:20.625910 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-79672bd1-1afd-4995-a989-2a4b57390ae5 req-426a8f77-b8fa-4baa-ba83-7b5affcb371e service nova] Creating event network-vif-deleted:9ff31270-663b-4d9c-a7e1-90a7d2b0c7fb for instance 0e538bab-a9f3-488f-9bef-801f2429f73f on nch1-master-1 Sep 27 10:00:20.645936 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-79672bd1-1afd-4995-a989-2a4b57390ae5 req-426a8f77-b8fa-4baa-ba83-7b5affcb371e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 523 microversion: 2.1 time: 0.126673 Sep 27 10:00:20.647738 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 595/1071] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:20 2022] POST /compute/v2.1/os-server-external-events => generated 523 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:20.656751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-94781d01-4248-43c6-944d-e841253a264b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:20.657627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-94781d01-4248-43c6-944d-e841253a264b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:20.664967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94781d01-4248-43c6-944d-e841253a264b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:20.665679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94781d01-4248-43c6-944d-e841253a264b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:20.666500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94781d01-4248-43c6-944d-e841253a264b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:20.667282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94781d01-4248-43c6-944d-e841253a264b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:20.668014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94781d01-4248-43c6-944d-e841253a264b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:20.668284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94781d01-4248-43c6-944d-e841253a264b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.358701 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-793559a9-0054-4635-b57e-b7ccd7130217 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/cbd4acfc-8ba0-42c8-9934-e6be328f4708" status: 202 len: 0 microversion: 2.1 time: 2.243048 Sep 27 10:00:21.359306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-94781d01-4248-43c6-944d-e841253a264b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:21.361479 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 477/1072] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:00:19 2022] DELETE /compute/v2.1/os-floating-ips/cbd4acfc-8ba0-42c8-9934-e6be328f4708 => generated 0 bytes in 2247 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:00:21.367370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cebe54d9-4fe2-4e0e-a7b1-d01f5a5ec6ac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:21.376763 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-94781d01-4248-43c6-944d-e841253a264b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1529 microversion: 2.1 time: 0.726573 Sep 27 10:00:21.379944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cebe54d9-4fe2-4e0e-a7b1-d01f5a5ec6ac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:21.380794 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 596/1073] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:20 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1529 bytes in 733 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:21.381730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cebe54d9-4fe2-4e0e-a7b1-d01f5a5ec6ac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:21.384247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cebe54d9-4fe2-4e0e-a7b1-d01f5a5ec6ac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.388292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cebe54d9-4fe2-4e0e-a7b1-d01f5a5ec6ac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:21.389283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cebe54d9-4fe2-4e0e-a7b1-d01f5a5ec6ac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:21.391587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cebe54d9-4fe2-4e0e-a7b1-d01f5a5ec6ac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.393497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-925f3e5f-8cba-4b9d-8419-f2d255c2cc30 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-1152851586"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:21.400254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-925f3e5f-8cba-4b9d-8419-f2d255c2cc30 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Failed to fetch instance by id aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2981}} Sep 27 10:00:21.401264 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-925f3e5f-8cba-4b9d-8419-f2d255c2cc30 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] HTTP exception thrown: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-925f3e5f-8cba-4b9d-8419-f2d255c2cc30 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Request method failure captured: Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action HTTP/1.1 Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 57 Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsmKyA6ZY_0UTI-fk-5sgLa0Rbl9D4N5Yd9gyv5Zlsn6kQCLWqMkvyPGfGKKTvfqGELUcLn_G_8GPGOB472pT5kzc_oEG-A5r9-j-ZRHjHl6fBzDLJdWXmBc3f7s0HbYKtukrsEo0dqSckAwOOSRnEjkC1AukBmXspnpQvD2iS6Sk4g Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:21.405693 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e9bd76ccd7824e989252951e381a17cd"}]}] Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: c815e82b67e8436b9bd4a06fe2d03b49 Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: {"createImage": {"name": "tempest-test-snap-1152851586"}} Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:00:21.407041 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'body': {'createImage': {'name': 'tempest-test-snap-1152851586'}}} Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:21.407815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1319, in _action_create_image Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Sep 27 10:00:21.410792 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.415414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-925f3e5f-8cba-4b9d-8419-f2d255c2cc30 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Returning 404 to user: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:21.417068 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-925f3e5f-8cba-4b9d-8419-f2d255c2cc30 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action" status: 404 len: 110 microversion: 2.1 time: 0.034853 Sep 27 10:00:21.417988 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 597/1074] 10.222.0.26 () {60 vars in 1275 bytes} [Tue Sep 27 10:00:21 2022] POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action => generated 110 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:21.426136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:21.431230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c01a91df-33fa-4d8c-84a5-b45b962e1ad5 req-249aae98-e9d9-4140-8f4f-0ec87723c191 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "46cf8ffa-724e-46d6-8952-8985413821b7", "name": "network-vif-deleted", "tag": "98beaffe-3e67-4b2e-8532-d90575ae8542"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:21.439454 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cebe54d9-4fe2-4e0e-a7b1-d01f5a5ec6ac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.076566 Sep 27 10:00:21.440075 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 478/1075] 10.222.0.26 () {58 vars in 1176 bytes} [Tue Sep 27 10:00:21 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:21.448029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c01a91df-33fa-4d8c-84a5-b45b962e1ad5 req-249aae98-e9d9-4140-8f4f-0ec87723c191 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:21.448029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c01a91df-33fa-4d8c-84a5-b45b962e1ad5 req-249aae98-e9d9-4140-8f4f-0ec87723c191 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:21.448029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c01a91df-33fa-4d8c-84a5-b45b962e1ad5 req-249aae98-e9d9-4140-8f4f-0ec87723c191 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.458761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:21.459419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:21.460943 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c01a91df-33fa-4d8c-84a5-b45b962e1ad5 req-249aae98-e9d9-4140-8f4f-0ec87723c191 service nova] Creating event network-vif-deleted:98beaffe-3e67-4b2e-8532-d90575ae8542 for instance 46cf8ffa-724e-46d6-8952-8985413821b7 on nch1-master-1 Sep 27 10:00:21.466405 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c01a91df-33fa-4d8c-84a5-b45b962e1ad5 req-249aae98-e9d9-4140-8f4f-0ec87723c191 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040433 Sep 27 10:00:21.466779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:21.467235 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 598/1076] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:21.467394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:21.470919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.471663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:21.472539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:21.473016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.474255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-249652d0-77c7-4127-91f0-e08e14a2cf7b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Action: 'action', calling method: .version_select at 0x7f7f693faee0>, body: {"addFloatingIp": {"address": ""}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-249652d0-77c7-4127-91f0-e08e14a2cf7b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Request method failure captured: Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action HTTP/1.1 Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 34 Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsl2HBK4xoqppYGzeaAO7kPvHRg88PVWis0qMqDneOM_OKStDvMkG8ww5OZe2eE7SZsEKq1Z20hZloERWHRvWtjcUK_NSSf7U3j3z9zRcLFKE6QCJ9Xdxwm-88X8IE0YnmRQzAKMHsGmX1yTr93BcmrSLa3nD5rXGSVM9JulDbR8Deg Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 2b3380f3fa2b494da649aaa809e11368 Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:21.478289 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2b3380f3fa2b494da649aaa809e11368"}]}] Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 2b3380f3fa2b494da649aaa809e11368 Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 57365b62b3294810b8a030d2a26cc451 Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: {"addFloatingIp": {"address": ""}} Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:00:21.479950 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a7bc6edb-7fc4-463d-8490-8800e2922d12', 'body': {'addFloatingIp': {'address': ''}}} Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is not valid under any of the given schemas Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'oneOf' in schema['properties']['addFloatingIp']['properties']['address']: Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'oneOf': [{'format': 'ipv4'}, {'format': 'ipv6'}], 'type': 'string'} Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['addFloatingIp']: Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi '' Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:21.480971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:00:21.482636 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:00:21.483971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas Sep 27 10:00:21.483971 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.483971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-249652d0-77c7-4127-91f0-e08e14a2cf7b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Returning 400 to user: Invalid input for field/attribute address. Value: . '' is not valid under any of the given schemas {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:21.483971 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-249652d0-77c7-4127-91f0-e08e14a2cf7b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action" status: 400 len: 142 microversion: 2.1 time: 0.013117 Sep 27 10:00:21.483971 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 599/1077] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:00:21 2022] POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action => generated 142 bytes in 15 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:00:21.493095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0f223ec8-9f89-428c-b6f1-f49fc851713f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:21.493849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0f223ec8-9f89-428c-b6f1-f49fc851713f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:21.501081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f223ec8-9f89-428c-b6f1-f49fc851713f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:21.501798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f223ec8-9f89-428c-b6f1-f49fc851713f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:21.502388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f223ec8-9f89-428c-b6f1-f49fc851713f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.503292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f223ec8-9f89-428c-b6f1-f49fc851713f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:21.503896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f223ec8-9f89-428c-b6f1-f49fc851713f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:21.504467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f223ec8-9f89-428c-b6f1-f49fc851713f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.674176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:21.689582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:21.691847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:21.693571 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3fc356ae-64fd-4392-87e9-7b171b4f3d7e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1567 microversion: 2.25 time: 0.251288 Sep 27 10:00:21.695606 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 479/1078] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:21 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1567 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:21.701189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a3a19dc3-ba70-45c0-8021-1c9d21347f40 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-1822685617"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:21.703886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a3a19dc3-ba70-45c0-8021-1c9d21347f40 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Failed to fetch instance by id aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2981}} Sep 27 10:00:21.704403 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a3a19dc3-ba70-45c0-8021-1c9d21347f40 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] HTTP exception thrown: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a3a19dc3-ba70-45c0-8021-1c9d21347f40 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Request method failure captured: Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action HTTP/1.1 Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 57 Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsmKyA6ZY_0UTI-fk-5sgLa0Rbl9D4N5Yd9gyv5Zlsn6kQCLWqMkvyPGfGKKTvfqGELUcLn_G_8GPGOB472pT5kzc_oEG-A5r9-j-ZRHjHl6fBzDLJdWXmBc3f7s0HbYKtukrsEo0dqSckAwOOSRnEjkC1AukBmXspnpQvD2iS6Sk4g Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:00:21.706106 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e9bd76ccd7824e989252951e381a17cd"}]}] Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: c815e82b67e8436b9bd4a06fe2d03b49 Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: {"createImage": {"name": "tempest-test-snap-1822685617"}} Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:00:21.707411 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'body': {'createImage': {'name': 'tempest-test-snap-1822685617'}}} Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:21.708302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1319, in _action_create_image Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. Sep 27 10:00:21.709928 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:21.711134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a3a19dc3-ba70-45c0-8021-1c9d21347f40 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Returning 404 to user: Instance aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:21.711134 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a3a19dc3-ba70-45c0-8021-1c9d21347f40 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action" status: 404 len: 112 microversion: 2.1 time: 0.010686 Sep 27 10:00:21.711134 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 480/1079] 10.222.0.26 () {60 vars in 1281 bytes} [Tue Sep 27 10:00:21 2022] POST /compute/v2.1/servers/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/action => generated 112 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:21.714013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0bb2c38f-7b23-4d2b-b553-e8d6554011f9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:21.718607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0bb2c38f-7b23-4d2b-b553-e8d6554011f9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:21.728315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bb2c38f-7b23-4d2b-b553-e8d6554011f9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:21.729180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bb2c38f-7b23-4d2b-b553-e8d6554011f9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:21.729764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bb2c38f-7b23-4d2b-b553-e8d6554011f9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.730661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bb2c38f-7b23-4d2b-b553-e8d6554011f9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:21.731265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bb2c38f-7b23-4d2b-b553-e8d6554011f9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:21.731836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bb2c38f-7b23-4d2b-b553-e8d6554011f9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.747758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0f223ec8-9f89-428c-b6f1-f49fc851713f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:21.759993 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0f223ec8-9f89-428c-b6f1-f49fc851713f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1529 microversion: 2.1 time: 0.270893 Sep 27 10:00:21.761441 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 600/1080] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:21 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1529 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:21.777460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "0.0.0.0"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:21.782344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:21.800994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:21.801496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:21.808289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.810201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:21.810689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:21.811210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:21.889587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=1,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:21.920652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-0bb2c38f-7b23-4d2b-b553-e8d6554011f9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lazy-loading 'flavor' on Instance uuid 16cd150f-c67a-4a9e-9654-ba6bb9e1517a {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:00:21.946574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0bb2c38f-7b23-4d2b-b553-e8d6554011f9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Going to try to live migrate instance to nch2-master-1 {{(pid=145969) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Sep 27 10:00:22.020381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 10:00:22.021701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquired lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 10:00:22.313420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Releasing lock "refresh_cache-a7bc6edb-7fc4-463d-8490-8800e2922d12" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 10:00:22.314001 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] HTTP exception thrown: floating IP not found Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Request method failure captured: Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action HTTP/1.1 Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 41 Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsl2HBK4xoqppYGzeaAO7kPvHRg88PVWis0qMqDneOM_OKStDvMkG8ww5OZe2eE7SZsEKq1Z20hZloERWHRvWtjcUK_NSSf7U3j3z9zRcLFKE6QCJ9Xdxwm-88X8IE0YnmRQzAKMHsGmX1yTr93BcmrSLa3nD5rXGSVM9JulDbR8Deg Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 2b3380f3fa2b494da649aaa809e11368 Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:22.315653 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2b3380f3fa2b494da649aaa809e11368"}]}] Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 2b3380f3fa2b494da649aaa809e11368 Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 57365b62b3294810b8a030d2a26cc451 Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: {"addFloatingIp": {"address": "0.0.0.0"}} Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:00:22.317417 ncdv-master-1 devstack@n-api.service[145968]: exception: floating IP not found Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a7bc6edb-7fc4-463d-8490-8800e2922d12', 'body': {'addFloatingIp': {'address': '0.0.0.0'}}} Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 239, in _add_floating_ip Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.network_api.associate_floating_ip(context, instance, Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 165, in wrapper Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = f(self, context, *args, **kwargs) Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2699, in associate_floating_ip Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi fip = self._get_floating_ip_by_address(client, floating_address) Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 3011, in _get_floating_ip_by_address Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFoundForAddress(address=address) Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFoundForAddress: Floating IP not found for address 0.0.0.0. Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.318193 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 252, in _add_floating_ip Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: floating IP not found Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: floating IP not found Sep 27 10:00:22.319313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.321896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Returning 404 to user: floating IP not found {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:22.321896 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-75d78310-f5e7-4d12-83d1-69643024d256 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action" status: 404 len: 67 microversion: 2.1 time: 0.556043 Sep 27 10:00:22.325249 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 601/1081] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:00:21 2022] POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action => generated 67 bytes in 562 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:00:22.332286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:22.332947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 0e538bab-a9f3-488f-9bef-801f2429f73f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:22.343239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.344053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.347604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.351300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.352790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.353450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.390152 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] HTTP exception thrown: Instance 0e538bab-a9f3-488f-9bef-801f2429f73f could not be found. Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Request method failure captured: Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f HTTP/1.1 Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjuzA8awi02tVpJBbCxyCTwC-N-auCdITAAm8nxhyCwHUSTrFjjx6tbQkA83IeLeCJH2BD7omoS-ML8vYESEbbQcIu0WhsX0gHiWVx2nVId80d2oExNsA2vGHGUS2rPtxB3q_M7eSz3mw7rO-vyGP61qyPfw7K0p8tFBuwGQZNzBLo Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:22.392643 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c1ac7b64c68e4896b2c631baacdbe2be"}]}] Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 00423eb1e4c24763ae0a546225ac1dc7 Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 0e538bab-a9f3-488f-9bef-801f2429f73f could not be found. Sep 27 10:00:22.394317 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '0e538bab-a9f3-488f-9bef-801f2429f73f'} Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0e538bab-a9f3-488f-9bef-801f2429f73f could not be found. Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:22.395309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0e538bab-a9f3-488f-9bef-801f2429f73f could not be found. Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0e538bab-a9f3-488f-9bef-801f2429f73f could not be found. Sep 27 10:00:22.397216 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.400253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Returning 404 to user: Instance 0e538bab-a9f3-488f-9bef-801f2429f73f could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:22.401357 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-889a47fb-a7e1-4d6b-8973-bb6ba1e182c9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f" status: 404 len: 111 microversion: 2.1 time: 0.074984 Sep 27 10:00:22.406239 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 602/1082] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:22 2022] GET /compute/v2.1/servers/0e538bab-a9f3-488f-9bef-801f2429f73f => generated 111 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:22.412249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:22.414781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:22.424019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.424390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.426769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.429284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.429892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.430370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.459636 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] HTTP exception thrown: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Request method failure captured: Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 HTTP/1.1 Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsmKyA6ZY_0UTI-fk-5sgLa0Rbl9D4N5Yd9gyv5Zlsn6kQCLWqMkvyPGfGKKTvfqGELUcLn_G_8GPGOB472pT5kzc_oEG-A5r9-j-ZRHjHl6fBzDLJdWXmBc3f7s0HbYKtukrsEo0dqSckAwOOSRnEjkC1AukBmXspnpQvD2iS6Sk4g Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:22.464670 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e9bd76ccd7824e989252951e381a17cd"}]}] Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: c815e82b67e8436b9bd4a06fe2d03b49 Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:22.466218 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '24be6165-1a70-40c5-beb7-67dd67933a85'} Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:22.467045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:22.468265 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.470919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Returning 404 to user: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:22.471262 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a96a4647-a5cd-4cc5-ae61-fcb0a66504f8 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "DELETE /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 404 len: 111 microversion: 2.1 time: 0.063584 Sep 27 10:00:22.472923 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 603/1083] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:00:22 2022] DELETE /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:22.482036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2bab8a23-e45d-4f00-b211-b639b0b133cd tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:22.502463 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2bab8a23-e45d-4f00-b211-b639b0b133cd tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/f627fe99-2d47-4507-bbb8-75b3a025ba89" status: 202 len: 0 microversion: 2.1 time: 0.024127 Sep 27 10:00:22.503348 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 604/1084] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:00:22 2022] DELETE /compute/v2.1/flavors/f627fe99-2d47-4507-bbb8-75b3a025ba89 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:00:22.509071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2cc6c146-2d10-42a5-be0a-c3d45135bc02 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Action: 'action', calling method: .version_select at 0x7f7f693faf70>, body: {"removeFloatingIp": {"address": "0.0.0.0"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:22.564853 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-2cc6c146-2d10-42a5-be0a-c3d45135bc02 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] HTTP exception thrown: Floating IP not found for address 0.0.0.0. Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2cc6c146-2d10-42a5-be0a-c3d45135bc02 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Request method failure captured: Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action HTTP/1.1 Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 44 Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsl2HBK4xoqppYGzeaAO7kPvHRg88PVWis0qMqDneOM_OKStDvMkG8ww5OZe2eE7SZsEKq1Z20hZloERWHRvWtjcUK_NSSf7U3j3z9zRcLFKE6QCJ9Xdxwm-88X8IE0YnmRQzAKMHsGmX1yTr93BcmrSLa3nD5rXGSVM9JulDbR8Deg Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 2b3380f3fa2b494da649aaa809e11368 Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:22.566835 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2b3380f3fa2b494da649aaa809e11368"}]}] Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 2b3380f3fa2b494da649aaa809e11368 Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 57365b62b3294810b8a030d2a26cc451 Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: {"removeFloatingIp": {"address": "0.0.0.0"}} Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:00:22.568351 ncdv-master-1 devstack@n-api.service[145968]: exception: Floating IP not found for address 0.0.0.0. Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a7bc6edb-7fc4-463d-8490-8800e2922d12', 'body': {'removeFloatingIp': {'address': '0.0.0.0'}}} Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 58, in get_instance_by_floating_ip_addr Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.network_api.get_instance_id_by_floating_address( Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2927, in get_instance_id_by_floating_address Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi fip = self._get_floating_ip_by_address(client, address) Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 3011, in _get_floating_ip_by_address Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFoundForAddress(address=address) Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFoundForAddress: Floating IP not found for address 0.0.0.0. Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:22.570215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 277, in _remove_floating_ip Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = get_instance_by_floating_ip_addr(self, context, address) Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 61, in get_instance_by_floating_ip_addr Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=ex.format_message()) Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for address 0.0.0.0. Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for address 0.0.0.0. Sep 27 10:00:22.573212 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.575635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2cc6c146-2d10-42a5-be0a-c3d45135bc02 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Returning 404 to user: Floating IP not found for address 0.0.0.0. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:22.575635 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2cc6c146-2d10-42a5-be0a-c3d45135bc02 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action" status: 404 len: 88 microversion: 2.1 time: 0.063828 Sep 27 10:00:22.583901 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 605/1085] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:00:22 2022] POST /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12/action => generated 88 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:00:22.590124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:22.593564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 46cf8ffa-724e-46d6-8952-8985413821b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:22.603484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.604231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.604886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.605920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.606546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.607364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.624613 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] HTTP exception thrown: Instance 46cf8ffa-724e-46d6-8952-8985413821b7 could not be found. Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Request method failure captured: Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 HTTP/1.1 Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsjuzA8awi02tVpJBbCxyCTwC-N-auCdITAAm8nxhyCwHUSTrFjjx6tbQkA83IeLeCJH2BD7omoS-ML8vYESEbbQcIu0WhsX0gHiWVx2nVId80d2oExNsA2vGHGUS2rPtxB3q_M7eSz3mw7rO-vyGP61qyPfw7K0p8tFBuwGQZNzBLo Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:22.626815 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c1ac7b64c68e4896b2c631baacdbe2be"}]}] Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 00423eb1e4c24763ae0a546225ac1dc7 Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 46cf8ffa-724e-46d6-8952-8985413821b7 could not be found. Sep 27 10:00:22.628386 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '46cf8ffa-724e-46d6-8952-8985413821b7'} Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 46cf8ffa-724e-46d6-8952-8985413821b7 could not be found. Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:22.629605 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 46cf8ffa-724e-46d6-8952-8985413821b7 could not be found. Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 46cf8ffa-724e-46d6-8952-8985413821b7 could not be found. Sep 27 10:00:22.630847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.633274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Returning 404 to user: Instance 46cf8ffa-724e-46d6-8952-8985413821b7 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:22.633274 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bfb80603-ad55-4dbe-a1c7-a7bcb4566b4a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7" status: 404 len: 111 microversion: 2.1 time: 0.043113 Sep 27 10:00:22.635195 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 606/1086] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:22 2022] GET /compute/v2.1/servers/46cf8ffa-724e-46d6-8952-8985413821b7 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:22.646947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:22.648652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] [instance: 24be6165-1a70-40c5-beb7-67dd67933a85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:22.656230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.656856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.657390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.658134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.658593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.659017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.674042 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] HTTP exception thrown: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Request method failure captured: Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 HTTP/1.1 Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsmKyA6ZY_0UTI-fk-5sgLa0Rbl9D4N5Yd9gyv5Zlsn6kQCLWqMkvyPGfGKKTvfqGELUcLn_G_8GPGOB472pT5kzc_oEG-A5r9-j-ZRHjHl6fBzDLJdWXmBc3f7s0HbYKtukrsEo0dqSckAwOOSRnEjkC1AukBmXspnpQvD2iS6Sk4g Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:22.675595 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e9bd76ccd7824e989252951e381a17cd"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e9bd76ccd7824e989252951e381a17cd"}]}] Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e9bd76ccd7824e989252951e381a17cd Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesNegativeTestJSON-1064732661 Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: c815e82b67e8436b9bd4a06fe2d03b49 Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesNegativeTestJSON-1064732661-project Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:22.677366 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '24be6165-1a70-40c5-beb7-67dd67933a85'} Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:22.678113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. Sep 27 10:00:22.679903 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:22.681400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] Returning 404 to user: Instance 24be6165-1a70-40c5-beb7-67dd67933a85 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:22.681400 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-43893dba-3705-4e80-b9cb-dc4a255dc8f6 tempest-ImagesNegativeTestJSON-1064732661 tempest-ImagesNegativeTestJSON-1064732661-project] 10.222.0.26 "GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85" status: 404 len: 111 microversion: 2.1 time: 0.039104 Sep 27 10:00:22.681400 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 607/1087] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:22 2022] GET /compute/v2.1/servers/24be6165-1a70-40c5-beb7-67dd67933a85 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:22.682586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a6f699f9-b267-4744-b029-dd94006fc575 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:22.706570 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a6f699f9-b267-4744-b029-dd94006fc575 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5814 microversion: 2.1 time: 0.027745 Sep 27 10:00:22.707518 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 608/1088] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:00:22 2022] GET /compute/v2.1/flavors/detail => generated 5814 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:22.730162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-032f0474-6e8c-44b1-b95b-ca800a5cc733 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:22.730673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-032f0474-6e8c-44b1-b95b-ca800a5cc733 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:22.738440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=92,nova_api:DELETE=6,nova_api:UPDATE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:22.741998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032f0474-6e8c-44b1-b95b-ca800a5cc733 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.742522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032f0474-6e8c-44b1-b95b-ca800a5cc733 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.742968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032f0474-6e8c-44b1-b95b-ca800a5cc733 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.743751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032f0474-6e8c-44b1-b95b-ca800a5cc733 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.744203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032f0474-6e8c-44b1-b95b-ca800a5cc733 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.744627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-032f0474-6e8c-44b1-b95b-ca800a5cc733 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.774786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-032f0474-6e8c-44b1-b95b-ca800a5cc733 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:00:22.877999 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-032f0474-6e8c-44b1-b95b-ca800a5cc733 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 204 len: 0 microversion: 2.1 time: 0.168886 Sep 27 10:00:22.884048 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 609/1089] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:00:22 2022] DELETE /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:00:22.892774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-be9fb483-16f5-4724-bfe0-a2e3c85f33e1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:22.940469 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-be9fb483-16f5-4724-bfe0-a2e3c85f33e1 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/ba6345ef-9480-4c71-b5a2-0c2e1d6550fb" status: 204 len: 0 microversion: 2.1 time: 0.053352 Sep 27 10:00:22.944524 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 610/1090] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:00:22 2022] DELETE /compute/v2.1/os-server-groups/ba6345ef-9480-4c71-b5a2-0c2e1d6550fb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:00:22.948470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:22.951657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:22.959470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.960426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.961056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.961948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:22.962528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:22.963066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:22.998619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:23.005330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Old instance type tempest-test_resize_flavor_-309978019, new instance type m1.heat {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:00:23.013221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:00:23.013800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:00:23.014373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:00:23.015043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:00:23.015330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:00:23.087456 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f42dabdf-1198-467c-bad6-5dd875ee4d81 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "POST /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd/action" status: 202 len: 0 microversion: 2.1 time: 0.144319 Sep 27 10:00:23.088270 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 611/1091] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:00:22 2022] POST /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd/action => generated 0 bytes in 146 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:00:23.093103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:23.093711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:23.100096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:23.100586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:23.101508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:23.102237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:23.102762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:23.103118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:23.126233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:23.497814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:23.507473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:23.509618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:23.511142 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3d5bff56-5ed3-487e-a3f9-e4a8dc3648c3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1567 microversion: 2.25 time: 0.421887 Sep 27 10:00:23.511778 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 612/1092] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:23 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1567 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:23.516963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69cb6409-1862-4bb2-9697-3e44c33cbe16 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:23.517838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69cb6409-1862-4bb2-9697-3e44c33cbe16 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:23.530464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69cb6409-1862-4bb2-9697-3e44c33cbe16 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:23.531165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69cb6409-1862-4bb2-9697-3e44c33cbe16 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:23.531660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69cb6409-1862-4bb2-9697-3e44c33cbe16 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:23.532764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69cb6409-1862-4bb2-9697-3e44c33cbe16 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:23.533242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69cb6409-1862-4bb2-9697-3e44c33cbe16 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:23.533673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69cb6409-1862-4bb2-9697-3e44c33cbe16 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:23.760687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69cb6409-1862-4bb2-9697-3e44c33cbe16 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:23.771283 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69cb6409-1862-4bb2-9697-3e44c33cbe16 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1539 microversion: 2.1 time: 0.257797 Sep 27 10:00:23.773853 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 613/1093] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:23 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1539 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:23.780383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-042430ee-57ef-48e5-925d-5c9fbd3bb07f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:23.781362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-042430ee-57ef-48e5-925d-5c9fbd3bb07f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:23.790821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-042430ee-57ef-48e5-925d-5c9fbd3bb07f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:23.791662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-042430ee-57ef-48e5-925d-5c9fbd3bb07f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:23.792409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-042430ee-57ef-48e5-925d-5c9fbd3bb07f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:23.793322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-042430ee-57ef-48e5-925d-5c9fbd3bb07f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:23.793931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-042430ee-57ef-48e5-925d-5c9fbd3bb07f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:23.794508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-042430ee-57ef-48e5-925d-5c9fbd3bb07f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:23.826351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-042430ee-57ef-48e5-925d-5c9fbd3bb07f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:00:23.936909 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-042430ee-57ef-48e5-925d-5c9fbd3bb07f tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "DELETE /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 204 len: 0 microversion: 2.1 time: 0.161306 Sep 27 10:00:23.938054 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 614/1094] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:00:23 2022] DELETE /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:00:23.944619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a3ace5ab-46b0-4b4d-b63c-0787deddf635 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:23.946232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a3ace5ab-46b0-4b4d-b63c-0787deddf635 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:23.955836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3ace5ab-46b0-4b4d-b63c-0787deddf635 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:23.956335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3ace5ab-46b0-4b4d-b63c-0787deddf635 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:23.956801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3ace5ab-46b0-4b4d-b63c-0787deddf635 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:23.960973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3ace5ab-46b0-4b4d-b63c-0787deddf635 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:23.967099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3ace5ab-46b0-4b4d-b63c-0787deddf635 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:23.967771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3ace5ab-46b0-4b4d-b63c-0787deddf635 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:24.262542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a3ace5ab-46b0-4b4d-b63c-0787deddf635 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:24.283169 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a3ace5ab-46b0-4b4d-b63c-0787deddf635 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1538 microversion: 2.1 time: 0.342538 Sep 27 10:00:24.284509 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 615/1095] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:23 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1538 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:24.291764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-004240d7-948c-4c08-93bd-0e6450a75cc9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:24.293242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-004240d7-948c-4c08-93bd-0e6450a75cc9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:24.302189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-004240d7-948c-4c08-93bd-0e6450a75cc9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:24.303095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-004240d7-948c-4c08-93bd-0e6450a75cc9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:24.303704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-004240d7-948c-4c08-93bd-0e6450a75cc9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:24.306285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-004240d7-948c-4c08-93bd-0e6450a75cc9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:24.307176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-004240d7-948c-4c08-93bd-0e6450a75cc9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:24.307909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-004240d7-948c-4c08-93bd-0e6450a75cc9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:24.601417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-004240d7-948c-4c08-93bd-0e6450a75cc9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:24.632862 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-004240d7-948c-4c08-93bd-0e6450a75cc9 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1472 microversion: 2.1 time: 0.346576 Sep 27 10:00:24.636700 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 616/1096] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:24 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1472 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:24.655241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:24.657426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:24.684554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:24.685563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:24.686201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:24.689085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:24.689754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:24.690354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:24.978604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:24.994008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:24.996535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:24.997877 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c935a15-6d85-40a6-9b9c-c06a33599329 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1567 microversion: 2.25 time: 0.357229 Sep 27 10:00:24.998460 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 617/1097] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:24 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1567 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:25.011816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-08c02b2d-890e-4f0f-8973-48ccab4078e5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:25.011816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-08c02b2d-890e-4f0f-8973-48ccab4078e5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:25.023035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c02b2d-890e-4f0f-8973-48ccab4078e5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:25.023035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c02b2d-890e-4f0f-8973-48ccab4078e5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:25.023035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c02b2d-890e-4f0f-8973-48ccab4078e5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:25.025953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c02b2d-890e-4f0f-8973-48ccab4078e5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:25.032756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c02b2d-890e-4f0f-8973-48ccab4078e5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:25.032756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c02b2d-890e-4f0f-8973-48ccab4078e5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:25.432171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-08c02b2d-890e-4f0f-8973-48ccab4078e5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:25.441087 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-08c02b2d-890e-4f0f-8973-48ccab4078e5 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1539 microversion: 2.1 time: 0.435156 Sep 27 10:00:25.445977 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 618/1098] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:24 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1539 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:25.452192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d01dc742-0f56-45b2-a5af-94774de30855 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:25.453161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d01dc742-0f56-45b2-a5af-94774de30855 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:25.459715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d01dc742-0f56-45b2-a5af-94774de30855 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:25.460608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d01dc742-0f56-45b2-a5af-94774de30855 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:25.461838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d01dc742-0f56-45b2-a5af-94774de30855 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:25.463571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d01dc742-0f56-45b2-a5af-94774de30855 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:25.464780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d01dc742-0f56-45b2-a5af-94774de30855 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:25.465470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d01dc742-0f56-45b2-a5af-94774de30855 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:25.803078 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0bb2c38f-7b23-4d2b-b553-e8d6554011f9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a/action" status: 202 len: 0 microversion: 2.1 time: 4.092892 Sep 27 10:00:25.809066 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 481/1099] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:00:21 2022] POST /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a/action => generated 0 bytes in 4099 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:00:25.815537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b03ceaa9-aa7e-4efb-870e-3d11e43cf83e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:25.819710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b03ceaa9-aa7e-4efb-870e-3d11e43cf83e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:25.857834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d01dc742-0f56-45b2-a5af-94774de30855 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:25.860840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b03ceaa9-aa7e-4efb-870e-3d11e43cf83e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:25.862662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b03ceaa9-aa7e-4efb-870e-3d11e43cf83e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:25.863593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b03ceaa9-aa7e-4efb-870e-3d11e43cf83e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:25.867916 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d01dc742-0f56-45b2-a5af-94774de30855 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1538 microversion: 2.1 time: 0.420163 Sep 27 10:00:25.868165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b03ceaa9-aa7e-4efb-870e-3d11e43cf83e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:25.869604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b03ceaa9-aa7e-4efb-870e-3d11e43cf83e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:25.871430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b03ceaa9-aa7e-4efb-870e-3d11e43cf83e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:25.873361 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 619/1100] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:25 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1538 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:25.897016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-556fa3cd-0742-43ae-aaca-ecd51305cf18 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:25.897624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-556fa3cd-0742-43ae-aaca-ecd51305cf18 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:25.921010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556fa3cd-0742-43ae-aaca-ecd51305cf18 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:25.921334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556fa3cd-0742-43ae-aaca-ecd51305cf18 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:25.924029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556fa3cd-0742-43ae-aaca-ecd51305cf18 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:25.924836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556fa3cd-0742-43ae-aaca-ecd51305cf18 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:25.925304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556fa3cd-0742-43ae-aaca-ecd51305cf18 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:25.927996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556fa3cd-0742-43ae-aaca-ecd51305cf18 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:26.438421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b03ceaa9-aa7e-4efb-870e-3d11e43cf83e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:26.447927 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b03ceaa9-aa7e-4efb-870e-3d11e43cf83e tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1472 microversion: 2.1 time: 0.637286 Sep 27 10:00:26.459587 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 482/1101] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:25 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1472 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:26.465030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:26.465650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:26.478909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:26.479406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:26.479927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:26.480670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:26.481136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:26.481635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:26.625893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-556fa3cd-0742-43ae-aaca-ecd51305cf18 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:26.639292 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-556fa3cd-0742-43ae-aaca-ecd51305cf18 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.750648 Sep 27 10:00:26.645320 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 620/1102] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:25 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 771 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:26.658292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4bac9f8-4300-4801-852d-e71364b37bb2 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:26.659984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4bac9f8-4300-4801-852d-e71364b37bb2 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:26.670969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4bac9f8-4300-4801-852d-e71364b37bb2 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:26.676920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4bac9f8-4300-4801-852d-e71364b37bb2 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:26.677872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4bac9f8-4300-4801-852d-e71364b37bb2 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:26.678877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4bac9f8-4300-4801-852d-e71364b37bb2 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:26.679691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4bac9f8-4300-4801-852d-e71364b37bb2 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:26.680585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4bac9f8-4300-4801-852d-e71364b37bb2 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:27.215069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4bac9f8-4300-4801-852d-e71364b37bb2 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:27.227385 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4bac9f8-4300-4801-852d-e71364b37bb2 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1539 microversion: 2.1 time: 0.576600 Sep 27 10:00:27.247506 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 621/1103] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:26 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1539 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:27.257599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c31f5f18-94e7-47d4-bf77-ad09c226d8f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:27.258731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c31f5f18-94e7-47d4-bf77-ad09c226d8f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:27.277156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31f5f18-94e7-47d4-bf77-ad09c226d8f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:27.277783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31f5f18-94e7-47d4-bf77-ad09c226d8f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:27.282487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31f5f18-94e7-47d4-bf77-ad09c226d8f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:27.283618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31f5f18-94e7-47d4-bf77-ad09c226d8f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:27.286208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31f5f18-94e7-47d4-bf77-ad09c226d8f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:27.286958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c31f5f18-94e7-47d4-bf77-ad09c226d8f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:27.336901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:27.352039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:27.353672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:27.372138 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5fef6ed2-0e9a-4ac2-b506-f684d1743d7f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1567 microversion: 2.25 time: 0.911372 Sep 27 10:00:27.372813 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 483/1104] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:26 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1567 bytes in 913 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:27.474927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-82f26aae-0f17-43ee-b324-968f144efc57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:27.475879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-82f26aae-0f17-43ee-b324-968f144efc57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:27.501882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82f26aae-0f17-43ee-b324-968f144efc57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:27.503767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82f26aae-0f17-43ee-b324-968f144efc57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:27.504548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82f26aae-0f17-43ee-b324-968f144efc57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:27.505543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82f26aae-0f17-43ee-b324-968f144efc57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:27.506166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82f26aae-0f17-43ee-b324-968f144efc57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:27.506755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82f26aae-0f17-43ee-b324-968f144efc57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:27.813316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-82f26aae-0f17-43ee-b324-968f144efc57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '00423eb1e4c24763ae0a546225ac1dc7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1ac7b64c68e4896b2c631baacdbe2be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:27.828913 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-82f26aae-0f17-43ee-b324-968f144efc57 tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 200 len: 1430 microversion: 2.1 time: 0.357887 Sep 27 10:00:27.835761 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 484/1105] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:27 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 1430 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:27.842234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-02bbff03-6cd0-45cb-af1f-14ba95974ef6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:27.842831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c31f5f18-94e7-47d4-bf77-ad09c226d8f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:27.843366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-02bbff03-6cd0-45cb-af1f-14ba95974ef6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:27.851879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02bbff03-6cd0-45cb-af1f-14ba95974ef6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:27.853090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02bbff03-6cd0-45cb-af1f-14ba95974ef6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:27.853770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02bbff03-6cd0-45cb-af1f-14ba95974ef6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:27.854928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02bbff03-6cd0-45cb-af1f-14ba95974ef6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:27.855738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02bbff03-6cd0-45cb-af1f-14ba95974ef6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:27.856469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02bbff03-6cd0-45cb-af1f-14ba95974ef6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:27.875018 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c31f5f18-94e7-47d4-bf77-ad09c226d8f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1538 microversion: 2.1 time: 0.621660 Sep 27 10:00:27.877168 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 622/1106] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:27 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1538 bytes in 629 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:27.884305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-9af5377e-ef5d-4481-ba4d-b0efd1d04050 req-ff68c405-03a5-42de-bf3b-450f20aea92c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "552575d5-492a-4680-9972-29347ad5479b", "name": "network-vif-deleted", "tag": "be0b1ff7-d8c1-4208-8f00-cdcde277fdef"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:27.936156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9af5377e-ef5d-4481-ba4d-b0efd1d04050 req-ff68c405-03a5-42de-bf3b-450f20aea92c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:27.936570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9af5377e-ef5d-4481-ba4d-b0efd1d04050 req-ff68c405-03a5-42de-bf3b-450f20aea92c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:27.945238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9af5377e-ef5d-4481-ba4d-b0efd1d04050 req-ff68c405-03a5-42de-bf3b-450f20aea92c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:27.970284 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-9af5377e-ef5d-4481-ba4d-b0efd1d04050 req-ff68c405-03a5-42de-bf3b-450f20aea92c service nova] Creating event network-vif-deleted:be0b1ff7-d8c1-4208-8f00-cdcde277fdef for instance 552575d5-492a-4680-9972-29347ad5479b on nch1-master-1 Sep 27 10:00:27.980794 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-9af5377e-ef5d-4481-ba4d-b0efd1d04050 req-ff68c405-03a5-42de-bf3b-450f20aea92c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.101842 Sep 27 10:00:27.989093 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 623/1107] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:28.251933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-126db602-0a27-4803-abf5-7051efacf35b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:28.252760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-126db602-0a27-4803-abf5-7051efacf35b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:28.274344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-126db602-0a27-4803-abf5-7051efacf35b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:28.275780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-126db602-0a27-4803-abf5-7051efacf35b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:28.276541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-126db602-0a27-4803-abf5-7051efacf35b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:28.277822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-126db602-0a27-4803-abf5-7051efacf35b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:28.281249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-126db602-0a27-4803-abf5-7051efacf35b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:28.282202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-126db602-0a27-4803-abf5-7051efacf35b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:28.494999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-126db602-0a27-4803-abf5-7051efacf35b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:28.504938 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-126db602-0a27-4803-abf5-7051efacf35b tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1539 microversion: 2.1 time: 0.257621 Sep 27 10:00:28.543409 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 624/1108] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:28 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1539 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:28.552434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:28.553274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:28.584914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:28.585904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:28.586452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:28.587577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:28.588548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:28.589540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:29.067956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-02bbff03-6cd0-45cb-af1f-14ba95974ef6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:29.089265 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-02bbff03-6cd0-45cb-af1f-14ba95974ef6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 1.251835 Sep 27 10:00:29.090399 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 485/1109] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:27 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 1254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:29.095677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-becf2f56-4682-4d67-a486-9486905e37aa req-57450bc8-3a6a-429c-b75e-1ea07c05a8fa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7bc6edb-7fc4-463d-8490-8800e2922d12", "name": "network-vif-unplugged", "status": "completed", "tag": "dbb7e99b-8507-4a96-bb0a-c442351f9baf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:29.109558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-becf2f56-4682-4d67-a486-9486905e37aa req-57450bc8-3a6a-429c-b75e-1ea07c05a8fa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:29.110228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-becf2f56-4682-4d67-a486-9486905e37aa req-57450bc8-3a6a-429c-b75e-1ea07c05a8fa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:29.110774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-becf2f56-4682-4d67-a486-9486905e37aa req-57450bc8-3a6a-429c-b75e-1ea07c05a8fa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:29.152063 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-becf2f56-4682-4d67-a486-9486905e37aa req-57450bc8-3a6a-429c-b75e-1ea07c05a8fa service nova] Creating event network-vif-unplugged:dbb7e99b-8507-4a96-bb0a-c442351f9baf for instance a7bc6edb-7fc4-463d-8490-8800e2922d12 on nch1-master-1 Sep 27 10:00:29.180443 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-becf2f56-4682-4d67-a486-9486905e37aa req-57450bc8-3a6a-429c-b75e-1ea07c05a8fa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.088442 Sep 27 10:00:29.182079 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 486/1110] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:29 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:29.187910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:29.194696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] [instance: 552575d5-492a-4680-9972-29347ad5479b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:29.202868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:29.203546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:29.204275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:29.205213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:29.206037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:29.206638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:29.236642 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] HTTP exception thrown: Instance 552575d5-492a-4680-9972-29347ad5479b could not be found. Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Request method failure captured: Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b HTTP/1.1 Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjuzA8awi02tVpJBbCxyCTwC-N-auCdITAAm8nxhyCwHUSTrFjjx6tbQkA83IeLeCJH2BD7omoS-ML8vYESEbbQcIu0WhsX0gHiWVx2nVId80d2oExNsA2vGHGUS2rPtxB3q_M7eSz3mw7rO-vyGP61qyPfw7K0p8tFBuwGQZNzBLo Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:00:29.247627 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c1ac7b64c68e4896b2c631baacdbe2be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c1ac7b64c68e4896b2c631baacdbe2be"}]}] Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: c1ac7b64c68e4896b2c631baacdbe2be Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersOnMultiNodesTest-536305491 Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 00423eb1e4c24763ae0a546225ac1dc7 Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersOnMultiNodesTest-536305491-project Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 552575d5-492a-4680-9972-29347ad5479b could not be found. Sep 27 10:00:29.249376 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '552575d5-492a-4680-9972-29347ad5479b'} Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 552575d5-492a-4680-9972-29347ad5479b could not be found. Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:29.250358 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 552575d5-492a-4680-9972-29347ad5479b could not be found. Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 552575d5-492a-4680-9972-29347ad5479b could not be found. Sep 27 10:00:29.251818 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:29.253459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] Returning 404 to user: Instance 552575d5-492a-4680-9972-29347ad5479b could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:29.253459 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-85079d50-fe09-49a8-9ec4-8a4d2ad7781a tempest-ServersOnMultiNodesTest-536305491 tempest-ServersOnMultiNodesTest-536305491-project] 10.222.0.26 "GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b" status: 404 len: 111 microversion: 2.1 time: 0.065936 Sep 27 10:00:29.253459 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 487/1111] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:29 2022] GET /compute/v2.1/servers/552575d5-492a-4680-9972-29347ad5479b => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:29.275064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-495bbacb-363c-4a83-bab6-b8caff56b60a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:29.275775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-495bbacb-363c-4a83-bab6-b8caff56b60a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:29.333600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495bbacb-363c-4a83-bab6-b8caff56b60a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:29.335739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495bbacb-363c-4a83-bab6-b8caff56b60a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:29.336287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495bbacb-363c-4a83-bab6-b8caff56b60a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:29.343687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495bbacb-363c-4a83-bab6-b8caff56b60a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:29.344211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495bbacb-363c-4a83-bab6-b8caff56b60a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:29.344685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495bbacb-363c-4a83-bab6-b8caff56b60a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:29.396601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:29.415713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:29.419243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:29.423018 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8c229cdd-c83b-4192-a390-014da206f18b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1567 microversion: 2.25 time: 0.878155 Sep 27 10:00:29.424716 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 625/1112] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:28 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1567 bytes in 881 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:29.535525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-caa57042-ee58-474b-b7d5-7f3615484699 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:29.536157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-caa57042-ee58-474b-b7d5-7f3615484699 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:29.543051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa57042-ee58-474b-b7d5-7f3615484699 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:29.543552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa57042-ee58-474b-b7d5-7f3615484699 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:29.549889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa57042-ee58-474b-b7d5-7f3615484699 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:29.551156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa57042-ee58-474b-b7d5-7f3615484699 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:29.551658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa57042-ee58-474b-b7d5-7f3615484699 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:29.552104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa57042-ee58-474b-b7d5-7f3615484699 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:29.942970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-caa57042-ee58-474b-b7d5-7f3615484699 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:29.959949 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-caa57042-ee58-474b-b7d5-7f3615484699 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1539 microversion: 2.1 time: 0.421088 Sep 27 10:00:29.961836 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 626/1113] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:29 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1539 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:30.120897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5e80262-9d54-462c-a113-e85df089044a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:30.122702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b5e80262-9d54-462c-a113-e85df089044a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:30.132958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5e80262-9d54-462c-a113-e85df089044a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:30.133606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5e80262-9d54-462c-a113-e85df089044a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:30.134223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5e80262-9d54-462c-a113-e85df089044a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:30.135135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5e80262-9d54-462c-a113-e85df089044a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:30.135748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5e80262-9d54-462c-a113-e85df089044a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:30.136325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5e80262-9d54-462c-a113-e85df089044a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:30.157089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-495bbacb-363c-4a83-bab6-b8caff56b60a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:30.183086 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-495bbacb-363c-4a83-bab6-b8caff56b60a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.919537 Sep 27 10:00:30.184065 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 488/1114] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:29 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 934 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:30.290052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f912b9ec-7fac-4bef-8237-6ab1bd73b56d req-b0f341ed-26ba-41df-ac54-e122016955f1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a7bc6edb-7fc4-463d-8490-8800e2922d12", "name": "network-vif-deleted", "tag": "dbb7e99b-8507-4a96-bb0a-c442351f9baf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:30.306857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f912b9ec-7fac-4bef-8237-6ab1bd73b56d req-b0f341ed-26ba-41df-ac54-e122016955f1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:30.307712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f912b9ec-7fac-4bef-8237-6ab1bd73b56d req-b0f341ed-26ba-41df-ac54-e122016955f1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:30.308420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f912b9ec-7fac-4bef-8237-6ab1bd73b56d req-b0f341ed-26ba-41df-ac54-e122016955f1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:30.324694 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f912b9ec-7fac-4bef-8237-6ab1bd73b56d req-b0f341ed-26ba-41df-ac54-e122016955f1 service nova] Creating event network-vif-deleted:dbb7e99b-8507-4a96-bb0a-c442351f9baf for instance a7bc6edb-7fc4-463d-8490-8800e2922d12 on nch1-master-1 Sep 27 10:00:30.340528 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f912b9ec-7fac-4bef-8237-6ab1bd73b56d req-b0f341ed-26ba-41df-ac54-e122016955f1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054681 Sep 27 10:00:30.344376 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 489/1115] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:00:30.478763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:30.479588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:30.500737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:30.501223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:30.501921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:30.502897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:30.503342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:30.503755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:30.877226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b5e80262-9d54-462c-a113-e85df089044a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:30.897716 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b5e80262-9d54-462c-a113-e85df089044a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.784878 Sep 27 10:00:30.907088 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 627/1116] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:30 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 794 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:30.989463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:30.990158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:31.004639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:31.006494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:31.007832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:31.008864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:31.009413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:31.009919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:31.015749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:31.093886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:31.100305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:31.100305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:31.102549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lazy-loading 'fault' on Instance uuid a7bc6edb-7fc4-463d-8490-8800e2922d12 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:00:31.149731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:31.164019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '57365b62b3294810b8a030d2a26cc451', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2b3380f3fa2b494da649aaa809e11368', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:31.164990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:31.172869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:31.174409 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9e7650f2-8195-4426-8c2a-d26842ec8245 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1557 microversion: 2.25 time: 0.717616 Sep 27 10:00:31.178585 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 490/1117] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:30 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1557 bytes in 725 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:31.213796 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5fec8693-96d0-4c59-8666-05c15f3e6331 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 200 len: 1320 microversion: 2.1 time: 0.222022 Sep 27 10:00:31.214396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-59c33143-3e57-4b5a-b075-aa7002bca4bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:31.215827 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 628/1118] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:30 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 1320 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:31.221059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-59c33143-3e57-4b5a-b075-aa7002bca4bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:31.223439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-70705747-d7ae-4ef4-868a-0ab32df3ff13 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '.version_select at 0x7f7f63727430>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:31.234351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70705747-d7ae-4ef4-868a-0ab32df3ff13 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:31.236190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70705747-d7ae-4ef4-868a-0ab32df3ff13 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:31.237281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70705747-d7ae-4ef4-868a-0ab32df3ff13 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:31.258244 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-70705747-d7ae-4ef4-868a-0ab32df3ff13 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/os-migrations" status: 200 len: 2612 microversion: 2.25 time: 0.040648 Sep 27 10:00:31.260343 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 629/1119] 10.222.0.26 () {60 vars in 1184 bytes} [Tue Sep 27 10:00:31 2022] GET /compute/v2.1/os-migrations => generated 2612 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:31.267739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59c33143-3e57-4b5a-b075-aa7002bca4bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:31.268461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59c33143-3e57-4b5a-b075-aa7002bca4bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:31.278767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59c33143-3e57-4b5a-b075-aa7002bca4bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:31.282866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59c33143-3e57-4b5a-b075-aa7002bca4bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:31.283547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59c33143-3e57-4b5a-b075-aa7002bca4bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:31.293962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59c33143-3e57-4b5a-b075-aa7002bca4bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:31.294274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cdd89763-a1d7-42c0-aa59-bf9b34ddf66a tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:31.295665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cdd89763-a1d7-42c0-aa59-bf9b34ddf66a tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:31.317801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdd89763-a1d7-42c0-aa59-bf9b34ddf66a tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:31.320333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdd89763-a1d7-42c0-aa59-bf9b34ddf66a tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:31.321505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdd89763-a1d7-42c0-aa59-bf9b34ddf66a tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:31.324233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdd89763-a1d7-42c0-aa59-bf9b34ddf66a tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:31.326420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdd89763-a1d7-42c0-aa59-bf9b34ddf66a tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:31.327155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cdd89763-a1d7-42c0-aa59-bf9b34ddf66a tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:31.422928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:31.588830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-cdd89763-a1d7-42c0-aa59-bf9b34ddf66a tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid f1ff4a24-1251-4827-955f-159c665a1d57 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:00:31.641074 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cdd89763-a1d7-42c0-aa59-bf9b34ddf66a tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 2050 microversion: 2.25 time: 0.350856 Sep 27 10:00:31.643958 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 630/1120] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:31 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 2050 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:31.760966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-59c33143-3e57-4b5a-b075-aa7002bca4bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:31.779438 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-59c33143-3e57-4b5a-b075-aa7002bca4bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.569799 Sep 27 10:00:31.785153 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 491/1121] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:31 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:31.931848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6a4a2db6-02cc-492f-9607-47871dc6faf3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:31.935332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6a4a2db6-02cc-492f-9607-47871dc6faf3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:31.979952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a4a2db6-02cc-492f-9607-47871dc6faf3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:31.980980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a4a2db6-02cc-492f-9607-47871dc6faf3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:31.987077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a4a2db6-02cc-492f-9607-47871dc6faf3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:31.991301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a4a2db6-02cc-492f-9607-47871dc6faf3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:31.992337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a4a2db6-02cc-492f-9607-47871dc6faf3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:31.996298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a4a2db6-02cc-492f-9607-47871dc6faf3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:32.121618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-1331801240", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "43a35817-b5c9-4e15-8250-6b84b0c2b25f"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:32.246175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] validate_networks() for [('43a35817-b5c9-4e15-8250-6b84b0c2b25f', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:00:32.464474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6a4a2db6-02cc-492f-9607-47871dc6faf3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:32.483625 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6a4a2db6-02cc-492f-9607-47871dc6faf3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.555412 Sep 27 10:00:32.484885 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 631/1122] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:31 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:32.503196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:32.504379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] [instance: a7bc6edb-7fc4-463d-8490-8800e2922d12] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:32.512272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:32.512948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:32.515109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:32.516435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:32.517252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:32.517946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:32.555373 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] HTTP exception thrown: Instance a7bc6edb-7fc4-463d-8490-8800e2922d12 could not be found. Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Request method failure captured: Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 HTTP/1.1 Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsl2HBK4xoqppYGzeaAO7kPvHRg88PVWis0qMqDneOM_OKStDvMkG8ww5OZe2eE7SZsEKq1Z20hZloERWHRvWtjcUK_NSSf7U3j3z9zRcLFKE6QCJ9Xdxwm-88X8IE0YnmRQzAKMHsGmX1yTr93BcmrSLa3nD5rXGSVM9JulDbR8Deg Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 2b3380f3fa2b494da649aaa809e11368 Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:32.559375 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2b3380f3fa2b494da649aaa809e11368"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2b3380f3fa2b494da649aaa809e11368"}]}] Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 2b3380f3fa2b494da649aaa809e11368 Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730 Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 57365b62b3294810b8a030d2a26cc451 Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance a7bc6edb-7fc4-463d-8490-8800e2922d12 could not be found. Sep 27 10:00:32.560974 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a7bc6edb-7fc4-463d-8490-8800e2922d12'} Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a7bc6edb-7fc4-463d-8490-8800e2922d12 could not be found. Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:32.561965 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a7bc6edb-7fc4-463d-8490-8800e2922d12 could not be found. Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a7bc6edb-7fc4-463d-8490-8800e2922d12 could not be found. Sep 27 10:00:32.563217 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:32.567806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] Returning 404 to user: Instance a7bc6edb-7fc4-463d-8490-8800e2922d12 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:32.569444 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c26ab86-b163-49ee-b9fb-1d834d0ea9e1 tempest-FloatingIPsAssociationNegativeTestJSON-45707730 tempest-FloatingIPsAssociationNegativeTestJSON-45707730-project] 10.222.0.26 "GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12" status: 404 len: 111 microversion: 2.1 time: 0.081357 Sep 27 10:00:32.574992 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 632/1123] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:32 2022] GET /compute/v2.1/servers/a7bc6edb-7fc4-463d-8490-8800e2922d12 => generated 111 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:32.588481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-50462b4d-36ef-4309-a822-83e5cece2ee0 req-a335393a-4990-41b8-bde4-fd555f243ab2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "16cd150f-c67a-4a9e-9654-ba6bb9e1517a", "name": "network-vif-unplugged", "status": "completed", "tag": "fa2d1e07-1a2a-40d0-baa6-738c999a9e3a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:32.610259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-50462b4d-36ef-4309-a822-83e5cece2ee0 req-a335393a-4990-41b8-bde4-fd555f243ab2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:32.612817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-50462b4d-36ef-4309-a822-83e5cece2ee0 req-a335393a-4990-41b8-bde4-fd555f243ab2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:32.613510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-50462b4d-36ef-4309-a822-83e5cece2ee0 req-a335393a-4990-41b8-bde4-fd555f243ab2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:32.676700 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-50462b4d-36ef-4309-a822-83e5cece2ee0 req-a335393a-4990-41b8-bde4-fd555f243ab2 service nova] Creating event network-vif-unplugged:fa2d1e07-1a2a-40d0-baa6-738c999a9e3a for instance 16cd150f-c67a-4a9e-9654-ba6bb9e1517a on nch1-master-1 Sep 27 10:00:32.698868 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-50462b4d-36ef-4309-a822-83e5cece2ee0 req-a335393a-4990-41b8-bde4-fd555f243ab2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.121970 Sep 27 10:00:32.699486 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 633/1124] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:32 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:32.811571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29867da6-e633-499b-9125-c18453a6e99a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:32.814542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-29867da6-e633-499b-9125-c18453a6e99a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:32.838242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29867da6-e633-499b-9125-c18453a6e99a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:32.842516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29867da6-e633-499b-9125-c18453a6e99a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:32.843087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29867da6-e633-499b-9125-c18453a6e99a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:32.843898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29867da6-e633-499b-9125-c18453a6e99a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:32.844550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29867da6-e633-499b-9125-c18453a6e99a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:32.845098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29867da6-e633-499b-9125-c18453a6e99a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:33.180237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29867da6-e633-499b-9125-c18453a6e99a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:33.200739 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-29867da6-e633-499b-9125-c18453a6e99a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.394755 Sep 27 10:00:33.202101 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 634/1125] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:32 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:33.513304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-368bdcbf-5f6e-4112-b37a-17e86051a453 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:33.514684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-368bdcbf-5f6e-4112-b37a-17e86051a453 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:33.538348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-368bdcbf-5f6e-4112-b37a-17e86051a453 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:33.541175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-368bdcbf-5f6e-4112-b37a-17e86051a453 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:33.541870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-368bdcbf-5f6e-4112-b37a-17e86051a453 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:33.547752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-368bdcbf-5f6e-4112-b37a-17e86051a453 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:33.549670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-368bdcbf-5f6e-4112-b37a-17e86051a453 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:33.550369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-368bdcbf-5f6e-4112-b37a-17e86051a453 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:33.786458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:00:33.787648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:00:33.788640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:00:33.789818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:00:33.790683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:00:33.812563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:33.814865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:33.815546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:33.816560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:33.817236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:33.818084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:33.841266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Getting quotas for project 2369f5ee467643948f618bbdee837521. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:00:33.850863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Getting quotas for user 29d2f7b0e8274c769521d13930be9825 and project 2369f5ee467643948f618bbdee837521. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:00:33.870631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:00:33.880363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:00:33.910515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-368bdcbf-5f6e-4112-b37a-17e86051a453 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:33.925067 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-368bdcbf-5f6e-4112-b37a-17e86051a453 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.418517 Sep 27 10:00:33.937480 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 635/1126] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:33 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:33.987469 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d10db905-b99f-4222-8421-3daf2994b602 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.25 time: 1.871948 Sep 27 10:00:33.988131 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 492/1127] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:00:32 2022] POST /compute/v2.1/servers => generated 380 bytes in 1873 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:00:34.005715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f8976c6-4538-46f7-b947-ebd270ac7c1e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:34.006614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7f8976c6-4538-46f7-b947-ebd270ac7c1e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:34.128278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7f8976c6-4538-46f7-b947-ebd270ac7c1e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:34.139459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7f8976c6-4538-46f7-b947-ebd270ac7c1e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:34.144017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7f8976c6-4538-46f7-b947-ebd270ac7c1e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:34.145382 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7f8976c6-4538-46f7-b947-ebd270ac7c1e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1262 microversion: 2.25 time: 0.148588 Sep 27 10:00:34.150586 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 636/1128] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:33 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1262 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:00:34.234819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0be49496-ec4d-4b82-8d50-678897f43eb9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:34.235494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0be49496-ec4d-4b82-8d50-678897f43eb9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:34.243920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be49496-ec4d-4b82-8d50-678897f43eb9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:34.244642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be49496-ec4d-4b82-8d50-678897f43eb9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:34.245276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be49496-ec4d-4b82-8d50-678897f43eb9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:34.246577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be49496-ec4d-4b82-8d50-678897f43eb9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:34.247408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be49496-ec4d-4b82-8d50-678897f43eb9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:34.248178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be49496-ec4d-4b82-8d50-678897f43eb9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:34.431451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0be49496-ec4d-4b82-8d50-678897f43eb9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:34.440184 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0be49496-ec4d-4b82-8d50-678897f43eb9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.209510 Sep 27 10:00:34.460240 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 637/1129] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:34 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:34.740812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-baf4680f-d968-474c-94e3-ab7de852bba6 tempest-FlavorsV2NegativeTest-1457909390 tempest-FlavorsV2NegativeTest-1457909390-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:34.759036 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-baf4680f-d968-474c-94e3-ab7de852bba6 tempest-FlavorsV2NegativeTest-1457909390 tempest-FlavorsV2NegativeTest-1457909390-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.151178 Sep 27 10:00:34.762075 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 638/1130] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:00:34 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:34.853148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-582cc7f5-b942-431d-bef5-848e35025756 tempest-ListImageFiltersNegativeTestJSON-224299336 tempest-ListImageFiltersNegativeTestJSON-224299336-project] Calling method '.version_select at 0x7f7f637b9940>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:34.871709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3d967c23-c0d1-42e6-ba71-c6d253b76a80 req-3019f165-13c1-49c7-b2c9-868d0c77503d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "16cd150f-c67a-4a9e-9654-ba6bb9e1517a", "tag": "fa2d1e07-1a2a-40d0-baa6-738c999a9e3a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:34.890473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3d967c23-c0d1-42e6-ba71-c6d253b76a80 req-3019f165-13c1-49c7-b2c9-868d0c77503d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:34.891949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3d967c23-c0d1-42e6-ba71-c6d253b76a80 req-3019f165-13c1-49c7-b2c9-868d0c77503d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:34.893021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3d967c23-c0d1-42e6-ba71-c6d253b76a80 req-3019f165-13c1-49c7-b2c9-868d0c77503d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:34.925113 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3d967c23-c0d1-42e6-ba71-c6d253b76a80 req-3019f165-13c1-49c7-b2c9-868d0c77503d service nova] Creating event network-changed:fa2d1e07-1a2a-40d0-baa6-738c999a9e3a for instance 16cd150f-c67a-4a9e-9654-ba6bb9e1517a on nch1-master-1 Sep 27 10:00:34.933436 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3d967c23-c0d1-42e6-ba71-c6d253b76a80 req-3019f165-13c1-49c7-b2c9-868d0c77503d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074593 Sep 27 10:00:34.935007 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 639/1131] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (3 switches on core 0) Sep 27 10:00:34.975338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-46a363b3-6354-4bea-881c-60e53a03c3d4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:34.982457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-46a363b3-6354-4bea-881c-60e53a03c3d4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:35.008635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46a363b3-6354-4bea-881c-60e53a03c3d4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:35.016555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46a363b3-6354-4bea-881c-60e53a03c3d4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:35.030949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46a363b3-6354-4bea-881c-60e53a03c3d4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:35.032923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46a363b3-6354-4bea-881c-60e53a03c3d4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:35.034462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46a363b3-6354-4bea-881c-60e53a03c3d4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:35.035012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46a363b3-6354-4bea-881c-60e53a03c3d4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:35.066790 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-582cc7f5-b942-431d-bef5-848e35025756 tempest-ListImageFiltersNegativeTestJSON-224299336 tempest-ListImageFiltersNegativeTestJSON-224299336-project] HTTP exception thrown: Image not found. Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-582cc7f5-b942-431d-bef5-848e35025756 tempest-ListImageFiltersNegativeTestJSON-224299336 tempest-ListImageFiltersNegativeTestJSON-224299336-project] Request method failure captured: Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/images/d74237c0-271e-45e5-a93c-c7e719a5d410 HTTP/1.1 Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsnCMYTB7235UYZ36q-mJpeInWd0sMonH7xPbC6FyoEp6oac7KfpVVSehi09vLtta0RbYfll-EWRCi3UcaYk6Gv0YGn-AKc-N0jCpinMqZv-mU5zDahemh4u1bek-9rBJx6hV2LmLRaxzYK9llxr8g_Ib9HbW2V1Y7zzZ7MTlht8Ysg Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e2481f3613e4485a8424a23cc716a060 Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ListImageFiltersNegativeTestJSON-224299336 Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:00:35.118718 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2481f3613e4485a8424a23cc716a060"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2481f3613e4485a8424a23cc716a060"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e2481f3613e4485a8424a23cc716a060"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e2481f3613e4485a8424a23cc716a060"}]}] Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ListImageFiltersNegativeTestJSON-224299336 Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e2481f3613e4485a8424a23cc716a060 Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ListImageFiltersNegativeTestJSON-224299336 Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ListImageFiltersNegativeTestJSON-224299336-project Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 9e9418ffc1c249fd8d854fb4dc7be36f Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ListImageFiltersNegativeTestJSON-224299336-project Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: exception: Image not found. Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'd74237c0-271e-45e5-a93c-c7e719a5d410'} Sep 27 10:00:35.120411 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:00:35.121431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID d74237c0-271e-45e5-a93c-c7e719a5d410 Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:00:35.122779 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image d74237c0-271e-45e5-a93c-c7e719a5d410 could not be found. Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:00:35.124208 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:35.126275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-582cc7f5-b942-431d-bef5-848e35025756 tempest-ListImageFiltersNegativeTestJSON-224299336 tempest-ListImageFiltersNegativeTestJSON-224299336-project] Returning 404 to user: Image not found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:35.127999 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-582cc7f5-b942-431d-bef5-848e35025756 tempest-ListImageFiltersNegativeTestJSON-224299336 tempest-ListImageFiltersNegativeTestJSON-224299336-project] 10.222.0.26 "GET /compute/v2.1/images/d74237c0-271e-45e5-a93c-c7e719a5d410" status: 404 len: 62 microversion: 2.1 time: 0.904361 Sep 27 10:00:35.127999 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 493/1132] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:00:34 2022] GET /compute/v2.1/images/d74237c0-271e-45e5-a93c-c7e719a5d410 => generated 62 bytes in 906 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:00:35.176464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:35.177075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:35.208991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:35.212087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:35.214367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:35.231346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:35.239353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:35.241247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:35.546828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-46a363b3-6354-4bea-881c-60e53a03c3d4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:35.560749 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-46a363b3-6354-4bea-881c-60e53a03c3d4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.589728 Sep 27 10:00:35.564888 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 640/1133] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:34 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:35.576682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bee05e93-11a7-4af7-857d-f34adaf6aded tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:35.579613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bee05e93-11a7-4af7-857d-f34adaf6aded tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:35.596687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bee05e93-11a7-4af7-857d-f34adaf6aded tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:35.600601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bee05e93-11a7-4af7-857d-f34adaf6aded tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:35.601555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bee05e93-11a7-4af7-857d-f34adaf6aded tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:35.602611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bee05e93-11a7-4af7-857d-f34adaf6aded tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:35.604052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bee05e93-11a7-4af7-857d-f34adaf6aded tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:35.606408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bee05e93-11a7-4af7-857d-f34adaf6aded tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:35.652017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:35.670043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:35.678802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:35.685504 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a1770ca-2864-4494-ba85-bb677b0f219e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1258 microversion: 2.25 time: 0.514153 Sep 27 10:00:35.690927 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 494/1134] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:35 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1258 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:36.019990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bee05e93-11a7-4af7-857d-f34adaf6aded tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:36.032063 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bee05e93-11a7-4af7-857d-f34adaf6aded tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.465761 Sep 27 10:00:36.036967 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 641/1135] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:35 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:36.607882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9cf1f4be-d931-4081-9817-05b0c4a40018 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:36.610819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9cf1f4be-d931-4081-9817-05b0c4a40018 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:36.637445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cf1f4be-d931-4081-9817-05b0c4a40018 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:36.641188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cf1f4be-d931-4081-9817-05b0c4a40018 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:36.641976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cf1f4be-d931-4081-9817-05b0c4a40018 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:36.643525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cf1f4be-d931-4081-9817-05b0c4a40018 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:36.644751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cf1f4be-d931-4081-9817-05b0c4a40018 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:36.650248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cf1f4be-d931-4081-9817-05b0c4a40018 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:36.719705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:36.723485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:36.768600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:36.776175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:36.784128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:36.786367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:36.787532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:36.788359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:36.966669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:36.978196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:36.981900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:36.983672 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d99f0f3b-d7ad-4116-abc9-0370c5597ba6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1332 microversion: 2.25 time: 0.269824 Sep 27 10:00:36.984410 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 642/1136] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:36 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1332 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:36.993593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d74bac6-b85d-41eb-bde0-b3e00d2a466f tempest-FlavorsV2NegativeTest-1457909390 tempest-FlavorsV2NegativeTest-1457909390-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FlavorsV2NegativeTest-server-1748460849", "imageRef": "e4b0c321-c073-47e5-8488-006e511ed6d9", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:37.142334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-2d74bac6-b85d-41eb-bde0-b3e00d2a466f tempest-FlavorsV2NegativeTest-1457909390 tempest-FlavorsV2NegativeTest-1457909390-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:00:37.194435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9cf1f4be-d931-4081-9817-05b0c4a40018 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:37.202289 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9cf1f4be-d931-4081-9817-05b0c4a40018 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.603595 Sep 27 10:00:37.206329 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 495/1137] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:36 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 608 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:37.211447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-20e80aa2-09c4-470a-a6c7-2879f1e8dadf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:37.221855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-20e80aa2-09c4-470a-a6c7-2879f1e8dadf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:37.240252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e80aa2-09c4-470a-a6c7-2879f1e8dadf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:37.241023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e80aa2-09c4-470a-a6c7-2879f1e8dadf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:37.241667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e80aa2-09c4-470a-a6c7-2879f1e8dadf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:37.242679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e80aa2-09c4-470a-a6c7-2879f1e8dadf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:37.243412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e80aa2-09c4-470a-a6c7-2879f1e8dadf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:37.243988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e80aa2-09c4-470a-a6c7-2879f1e8dadf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:37.493240 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-2d74bac6-b85d-41eb-bde0-b3e00d2a466f tempest-FlavorsV2NegativeTest-1457909390 tempest-FlavorsV2NegativeTest-1457909390-project] HTTP exception thrown: Flavor's memory is too small for requested image. Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d74bac6-b85d-41eb-bde0-b3e00d2a466f tempest-FlavorsV2NegativeTest-1457909390 tempest-FlavorsV2NegativeTest-1457909390-project] Request method failure captured: Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 143 Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnCXvtJLoSJj_TxtCeRG3ofkPn3z6q8Tdgl-xkzlrV98XyUaQJqP3_HoZdm5Yfl1TG3ioAABP67OGIsvMkosmCOFxcI7x54W_S-3A5xGv-JJImcHkgTiTNLK-LNfGDzkWWJUHpE6afw1APsKjKn6E6nHOQPUQLVbTjndLnSr11gpYE Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 9164898c395c4ad49cf36430163a3b72 Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FlavorsV2NegativeTest-1457909390 Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:37.495499 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9164898c395c4ad49cf36430163a3b72"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9164898c395c4ad49cf36430163a3b72"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9164898c395c4ad49cf36430163a3b72"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9164898c395c4ad49cf36430163a3b72"}]}] Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FlavorsV2NegativeTest-1457909390 Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 9164898c395c4ad49cf36430163a3b72 Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FlavorsV2NegativeTest-1457909390 Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FlavorsV2NegativeTest-1457909390-project Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 07ba36721b77427d96352f54367283ab Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FlavorsV2NegativeTest-1457909390-project Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-FlavorsV2NegativeTest-server-1748460849", "imageRef": "e4b0c321-c073-47e5-8488-006e511ed6d9", "flavorRef": "452"}} Sep 27 10:00:37.497225 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: exception: Flavor's memory is too small for requested image. Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-FlavorsV2NegativeTest-server-1748460849', 'imageRef': 'e4b0c321-c073-47e5-8488-006e511ed6d9', 'flavorRef': '452'}}} Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._create_instance( Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1724, in _create_instance Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._checks_for_create_and_rebuild(context, image_id, boot_meta, Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1019, in _checks_for_create_and_rebuild Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._validate_flavor_image(context, image_id, image, Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 683, in _validate_flavor_image Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._validate_flavor_image_nostatus( Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 751, in _validate_flavor_image_nostatus Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.FlavorMemoryTooSmall() Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.FlavorMemoryTooSmall: Flavor's memory is too small for requested image. Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:37.498034 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 868, in create Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor's memory is too small for requested image. Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:37.499324 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:37.501024 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:37.501024 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:37.501024 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:37.501024 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:37.501024 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor's memory is too small for requested image. Sep 27 10:00:37.501024 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:37.501024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d74bac6-b85d-41eb-bde0-b3e00d2a466f tempest-FlavorsV2NegativeTest-1457909390 tempest-FlavorsV2NegativeTest-1457909390-project] Returning 400 to user: Flavor's memory is too small for requested image. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:37.501024 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d74bac6-b85d-41eb-bde0-b3e00d2a466f tempest-FlavorsV2NegativeTest-1457909390 tempest-FlavorsV2NegativeTest-1457909390-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 93 microversion: 2.1 time: 0.511202 Sep 27 10:00:37.501024 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 643/1138] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:00:36 2022] POST /compute/v2.1/servers => generated 93 bytes in 513 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 10:00:38.020381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:38.021487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:38.036089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:38.038766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:38.039989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:38.041664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:38.043192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:38.043642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:38.113631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-20e80aa2-09c4-470a-a6c7-2879f1e8dadf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:38.123257 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-20e80aa2-09c4-470a-a6c7-2879f1e8dadf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.915473 Sep 27 10:00:38.131455 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 496/1139] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:37 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:38.234037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-99bfa6bd-9d6e-4996-b114-71b13c2ef75e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:38.235076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-99bfa6bd-9d6e-4996-b114-71b13c2ef75e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:38.243123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:38.246676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99bfa6bd-9d6e-4996-b114-71b13c2ef75e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:38.250214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99bfa6bd-9d6e-4996-b114-71b13c2ef75e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:38.251005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99bfa6bd-9d6e-4996-b114-71b13c2ef75e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:38.252094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99bfa6bd-9d6e-4996-b114-71b13c2ef75e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:38.252814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:38.254847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99bfa6bd-9d6e-4996-b114-71b13c2ef75e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:38.255292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99bfa6bd-9d6e-4996-b114-71b13c2ef75e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:38.261097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:38.263163 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-466ad21d-4afa-4bf3-8f02-915a3875cdf4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1320 microversion: 2.25 time: 0.255208 Sep 27 10:00:38.264084 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 644/1140] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:38 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1320 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:38.740056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-99bfa6bd-9d6e-4996-b114-71b13c2ef75e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:38.760337 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-99bfa6bd-9d6e-4996-b114-71b13c2ef75e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.531148 Sep 27 10:00:38.763168 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 497/1141] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:38 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:39.162334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb4e3214-964b-4f41-aa64-365308a8f260 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:39.163992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cb4e3214-964b-4f41-aa64-365308a8f260 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:39.190611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb4e3214-964b-4f41-aa64-365308a8f260 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:39.191113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb4e3214-964b-4f41-aa64-365308a8f260 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:39.191680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb4e3214-964b-4f41-aa64-365308a8f260 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:39.192430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb4e3214-964b-4f41-aa64-365308a8f260 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:39.192935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb4e3214-964b-4f41-aa64-365308a8f260 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:39.193376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb4e3214-964b-4f41-aa64-365308a8f260 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:39.299959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:39.301545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:39.316949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:39.343032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.026s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:39.343737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:39.344807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:39.345668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:39.346353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:39.639804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb4e3214-964b-4f41-aa64-365308a8f260 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:39.652770 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cb4e3214-964b-4f41-aa64-365308a8f260 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.495633 Sep 27 10:00:39.657460 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 645/1142] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:39 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:39.791204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e74ab26c-39fd-4bc3-adba-ff9831064182 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:39.791997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e74ab26c-39fd-4bc3-adba-ff9831064182 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:39.799011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=2,nova_api:DELETE=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:39.810755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e74ab26c-39fd-4bc3-adba-ff9831064182 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:39.813523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e74ab26c-39fd-4bc3-adba-ff9831064182 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:39.814217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e74ab26c-39fd-4bc3-adba-ff9831064182 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:39.815183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e74ab26c-39fd-4bc3-adba-ff9831064182 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:39.817777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e74ab26c-39fd-4bc3-adba-ff9831064182 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:39.819510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e74ab26c-39fd-4bc3-adba-ff9831064182 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:39.861686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:39.871970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:39.873913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:39.875570 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7a20c53b-e45e-4e51-b9a7-c3515ea496d7 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1362 microversion: 2.25 time: 0.579705 Sep 27 10:00:39.877140 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 498/1143] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:39 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1362 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:00:40.209188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e74ab26c-39fd-4bc3-adba-ff9831064182 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:40.231547 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e74ab26c-39fd-4bc3-adba-ff9831064182 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.447304 Sep 27 10:00:40.233542 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 646/1144] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:39 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:40.700068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e408ef5-9ae7-42b2-b22e-33d61ac9c80e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:40.701500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4e408ef5-9ae7-42b2-b22e-33d61ac9c80e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:40.746714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e408ef5-9ae7-42b2-b22e-33d61ac9c80e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:40.747394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e408ef5-9ae7-42b2-b22e-33d61ac9c80e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:40.748714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e408ef5-9ae7-42b2-b22e-33d61ac9c80e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:40.750367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e408ef5-9ae7-42b2-b22e-33d61ac9c80e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:40.764026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e408ef5-9ae7-42b2-b22e-33d61ac9c80e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:40.765129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e408ef5-9ae7-42b2-b22e-33d61ac9c80e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:40.905655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:40.907236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:40.920579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:40.921127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:40.921573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:40.925451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:40.927304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:40.928135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:41.199472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4e408ef5-9ae7-42b2-b22e-33d61ac9c80e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:41.206353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:41.213975 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4e408ef5-9ae7-42b2-b22e-33d61ac9c80e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.518993 Sep 27 10:00:41.225414 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 499/1145] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:40 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:41.263470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3f1ecbc7-5205-4fd8-aefa-8a876b69385f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:41.264478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3f1ecbc7-5205-4fd8-aefa-8a876b69385f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:41.287218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f1ecbc7-5205-4fd8-aefa-8a876b69385f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:41.288369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f1ecbc7-5205-4fd8-aefa-8a876b69385f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:41.292371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f1ecbc7-5205-4fd8-aefa-8a876b69385f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:41.296429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f1ecbc7-5205-4fd8-aefa-8a876b69385f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:41.300262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f1ecbc7-5205-4fd8-aefa-8a876b69385f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:41.304082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f1ecbc7-5205-4fd8-aefa-8a876b69385f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:41.547953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:41.574516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:41.591004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:41.593324 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-abb7b2b1-4a86-4c6f-8a7e-b1af69cd0919 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1362 microversion: 2.25 time: 0.693141 Sep 27 10:00:41.620831 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 647/1146] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:40 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1362 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:00:41.910113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3f1ecbc7-5205-4fd8-aefa-8a876b69385f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:41.925829 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3f1ecbc7-5205-4fd8-aefa-8a876b69385f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.668378 Sep 27 10:00:41.932356 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 500/1147] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:41 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:42.087733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c5714eff-d199-4a6c-aea3-efa1600bd374 req-695e9658-d75f-4d2b-b83c-60c761d71bab service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9edcd7b8-f01b-4529-b9c1-3729e79ec760", "tag": "5a182f65-e926-4e9f-97dc-7f90cd3a6dc9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:42.133321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c5714eff-d199-4a6c-aea3-efa1600bd374 req-695e9658-d75f-4d2b-b83c-60c761d71bab service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:42.134415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c5714eff-d199-4a6c-aea3-efa1600bd374 req-695e9658-d75f-4d2b-b83c-60c761d71bab service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:42.135217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c5714eff-d199-4a6c-aea3-efa1600bd374 req-695e9658-d75f-4d2b-b83c-60c761d71bab service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:42.159809 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c5714eff-d199-4a6c-aea3-efa1600bd374 req-695e9658-d75f-4d2b-b83c-60c761d71bab service nova] Creating event network-changed:5a182f65-e926-4e9f-97dc-7f90cd3a6dc9 for instance 9edcd7b8-f01b-4529-b9c1-3729e79ec760 on nch2-master-1 Sep 27 10:00:42.168077 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c5714eff-d199-4a6c-aea3-efa1600bd374 req-695e9658-d75f-4d2b-b83c-60c761d71bab service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.092505 Sep 27 10:00:42.169848 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 648/1148] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:42.254008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2d61ff07-914f-4009-b7ba-82748372ca54 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:42.257265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2d61ff07-914f-4009-b7ba-82748372ca54 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:42.285601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d61ff07-914f-4009-b7ba-82748372ca54 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:42.286951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d61ff07-914f-4009-b7ba-82748372ca54 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:42.289522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d61ff07-914f-4009-b7ba-82748372ca54 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:42.293493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d61ff07-914f-4009-b7ba-82748372ca54 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:42.294479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d61ff07-914f-4009-b7ba-82748372ca54 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:42.296298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d61ff07-914f-4009-b7ba-82748372ca54 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:42.627566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:42.630248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:42.641847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:42.642860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:42.643566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:42.646105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:42.646975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:42.647723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:42.686075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2d61ff07-914f-4009-b7ba-82748372ca54 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:42.714334 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2d61ff07-914f-4009-b7ba-82748372ca54 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.466206 Sep 27 10:00:42.716004 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 501/1149] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:42 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:42.969307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-50e9944e-cbb9-4923-b3b4-511ba03d4418 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:42.971609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-50e9944e-cbb9-4923-b3b4-511ba03d4418 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:42.998887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50e9944e-cbb9-4923-b3b4-511ba03d4418 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:43.002062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50e9944e-cbb9-4923-b3b4-511ba03d4418 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:43.002723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50e9944e-cbb9-4923-b3b4-511ba03d4418 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:43.005542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50e9944e-cbb9-4923-b3b4-511ba03d4418 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:43.007013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50e9944e-cbb9-4923-b3b4-511ba03d4418 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:43.011360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50e9944e-cbb9-4923-b3b4-511ba03d4418 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:43.506291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-50e9944e-cbb9-4923-b3b4-511ba03d4418 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:43.525721 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-50e9944e-cbb9-4923-b3b4-511ba03d4418 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.565196 Sep 27 10:00:43.538844 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 502/1150] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:42 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:43.741516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b1d7bd1b-10b3-408c-8d86-ff30d18fefd6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:43.743430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b1d7bd1b-10b3-408c-8d86-ff30d18fefd6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:43.780997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1d7bd1b-10b3-408c-8d86-ff30d18fefd6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:43.785651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1d7bd1b-10b3-408c-8d86-ff30d18fefd6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:43.786348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1d7bd1b-10b3-408c-8d86-ff30d18fefd6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:43.789709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1d7bd1b-10b3-408c-8d86-ff30d18fefd6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:43.790506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1d7bd1b-10b3-408c-8d86-ff30d18fefd6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:43.797774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1d7bd1b-10b3-408c-8d86-ff30d18fefd6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:43.842831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:43.864593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:44.113437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b1d7bd1b-10b3-408c-8d86-ff30d18fefd6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:44.124414 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b1d7bd1b-10b3-408c-8d86-ff30d18fefd6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.387545 Sep 27 10:00:44.125065 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 503/1151] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:43 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:44.133279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:44.519094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:44.537612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:44.541359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:44.544591 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-21cb2bd8-99c9-4f40-91d7-d2b83db5a335 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1362 microversion: 2.25 time: 1.924005 Sep 27 10:00:44.559247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1882429-e353-4ee2-b949-e29163783eda tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:44.560160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a1882429-e353-4ee2-b949-e29163783eda tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:44.563054 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 649/1152] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:42 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1362 bytes in 1944 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:00:44.579762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1882429-e353-4ee2-b949-e29163783eda tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:44.580328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1882429-e353-4ee2-b949-e29163783eda tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:44.580843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1882429-e353-4ee2-b949-e29163783eda tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:44.581611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1882429-e353-4ee2-b949-e29163783eda tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:44.582080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1882429-e353-4ee2-b949-e29163783eda tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:44.582513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1882429-e353-4ee2-b949-e29163783eda tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:44.968885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a1882429-e353-4ee2-b949-e29163783eda tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:44.982977 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a1882429-e353-4ee2-b949-e29163783eda tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.431656 Sep 27 10:00:44.986129 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 504/1153] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:44 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:45.150101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5fbdea17-054c-4824-9c16-5add2876f3a6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:45.151463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5fbdea17-054c-4824-9c16-5add2876f3a6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:45.172629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fbdea17-054c-4824-9c16-5add2876f3a6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:45.174013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fbdea17-054c-4824-9c16-5add2876f3a6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:45.176430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fbdea17-054c-4824-9c16-5add2876f3a6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:45.179463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fbdea17-054c-4824-9c16-5add2876f3a6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:45.183450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fbdea17-054c-4824-9c16-5add2876f3a6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:45.185422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fbdea17-054c-4824-9c16-5add2876f3a6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:45.239740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:45.380008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6858c5d2-d0dd-4369-b531-9c508b00570b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:45.381430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6858c5d2-d0dd-4369-b531-9c508b00570b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:45.394757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6858c5d2-d0dd-4369-b531-9c508b00570b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:45.397705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6858c5d2-d0dd-4369-b531-9c508b00570b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:45.398439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6858c5d2-d0dd-4369-b531-9c508b00570b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:45.401159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6858c5d2-d0dd-4369-b531-9c508b00570b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:45.402809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6858c5d2-d0dd-4369-b531-9c508b00570b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:45.409737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6858c5d2-d0dd-4369-b531-9c508b00570b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:45.507157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6858c5d2-d0dd-4369-b531-9c508b00570b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:00:45.558238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5fbdea17-054c-4824-9c16-5add2876f3a6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:45.575279 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5fbdea17-054c-4824-9c16-5add2876f3a6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.431219 Sep 27 10:00:45.602209 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 650/1154] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:45 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:45.627725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:45.630652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:45.680760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:45.685863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:45.687376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:45.692497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:45.695834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:45.697176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:45.814642 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6858c5d2-d0dd-4369-b531-9c508b00570b tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "DELETE /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 204 len: 0 microversion: 2.1 time: 0.436604 Sep 27 10:00:45.822373 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 505/1155] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:00:45 2022] DELETE /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:00:45.846086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5d5a4f0-5a4b-44bf-bac5-c890a9f793a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:45.851504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5d5a4f0-5a4b-44bf-bac5-c890a9f793a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:45.871561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5d5a4f0-5a4b-44bf-bac5-c890a9f793a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:45.872404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5d5a4f0-5a4b-44bf-bac5-c890a9f793a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:45.873411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5d5a4f0-5a4b-44bf-bac5-c890a9f793a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:45.874368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5d5a4f0-5a4b-44bf-bac5-c890a9f793a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:45.875010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5d5a4f0-5a4b-44bf-bac5-c890a9f793a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:45.875869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5d5a4f0-5a4b-44bf-bac5-c890a9f793a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:46.126426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:46.139277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:46.140970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:46.143332 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cd9b30c9-84e3-4cea-b25e-cab0b6a8ac19 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1362 microversion: 2.25 time: 0.536025 Sep 27 10:00:46.162911 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 651/1156] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:45 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1362 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:46.170597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1bf1fcbb-f3ed-417c-855e-8e7cc964fb13 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:46.171364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1bf1fcbb-f3ed-417c-855e-8e7cc964fb13 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:46.180714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1bf1fcbb-f3ed-417c-855e-8e7cc964fb13 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:46.180714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1bf1fcbb-f3ed-417c-855e-8e7cc964fb13 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:46.181161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1bf1fcbb-f3ed-417c-855e-8e7cc964fb13 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:46.182989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1bf1fcbb-f3ed-417c-855e-8e7cc964fb13 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:46.183576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1bf1fcbb-f3ed-417c-855e-8e7cc964fb13 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:46.183888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1bf1fcbb-f3ed-417c-855e-8e7cc964fb13 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:46.581143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1bf1fcbb-f3ed-417c-855e-8e7cc964fb13 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:46.601855 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1bf1fcbb-f3ed-417c-855e-8e7cc964fb13 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.437930 Sep 27 10:00:46.603470 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 652/1157] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:46 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:46.953389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5d5a4f0-5a4b-44bf-bac5-c890a9f793a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:46.969249 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5d5a4f0-5a4b-44bf-bac5-c890a9f793a4 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1710 microversion: 2.1 time: 1.141588 Sep 27 10:00:46.970194 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 506/1158] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:45 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1710 bytes in 1144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:46.992304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba958a27-b915-40f5-a32c-704f4a642cd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:46.993234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ba958a27-b915-40f5-a32c-704f4a642cd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:47.000039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba958a27-b915-40f5-a32c-704f4a642cd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:47.000892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba958a27-b915-40f5-a32c-704f4a642cd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:47.001498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba958a27-b915-40f5-a32c-704f4a642cd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:47.002445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba958a27-b915-40f5-a32c-704f4a642cd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:47.003088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba958a27-b915-40f5-a32c-704f4a642cd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:47.003709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba958a27-b915-40f5-a32c-704f4a642cd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:47.237975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b8c1993-1efe-4001-97a6-4906a2806c4d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Action: 'create', calling method: .version_select at 0x7f7f637308b0>, body: {"keypair": {"name": "tempest-keypair-1899311251", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMWng1Yj8k1WN6Hswyd9T6cIjN6Tvu3onzOHYim0E4fwjysMvhwFQpgamg7Mw20X6IL/IhsKen7NEUH4L+PNAEwaANeh85FOjfvo7ej8CigT+M+lDUU28zhBYu4jhuAaaw=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:47.278757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-7b8c1993-1efe-4001-97a6-4906a2806c4d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Getting quotas for project 5de10e92e2024a24880c7205697e5732. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:00:47.292790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-7b8c1993-1efe-4001-97a6-4906a2806c4d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Getting quotas for user 0d563feacff84b4c9927ec759fb365be and project 5de10e92e2024a24880c7205697e5732. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:00:47.339810 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7b8c1993-1efe-4001-97a6-4906a2806c4d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.734394 Sep 27 10:00:47.342140 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 653/1159] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:00:46 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 738 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:47.357528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:47.358741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:47.383251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:47.388528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:47.389197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:47.391756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:47.392264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:47.392738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:47.911730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ba958a27-b915-40f5-a32c-704f4a642cd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:47.942637 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ba958a27-b915-40f5-a32c-704f4a642cd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.956501 Sep 27 10:00:47.953106 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 507/1160] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:46 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 967 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:47.958870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7ca9a993-ed10-4179-b2d4-4ea4e44d6c4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:47.959586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7ca9a993-ed10-4179-b2d4-4ea4e44d6c4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:47.984369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ca9a993-ed10-4179-b2d4-4ea4e44d6c4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:47.984916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ca9a993-ed10-4179-b2d4-4ea4e44d6c4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:47.985368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ca9a993-ed10-4179-b2d4-4ea4e44d6c4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:47.986110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ca9a993-ed10-4179-b2d4-4ea4e44d6c4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:47.986596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ca9a993-ed10-4179-b2d4-4ea4e44d6c4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:47.998340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ca9a993-ed10-4179-b2d4-4ea4e44d6c4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:48.233619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:48.248150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:48.254565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:48.256013 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-06de9002-d9a0-40e9-861d-31edc73dd182 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1362 microversion: 2.25 time: 0.909795 Sep 27 10:00:48.259124 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 654/1161] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:47 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1362 bytes in 916 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:48.265004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99b62b4c-3ac2-44ae-9f21-cac4ca8ffa95 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:48.273999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-99b62b4c-3ac2-44ae-9f21-cac4ca8ffa95 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:48.293242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99b62b4c-3ac2-44ae-9f21-cac4ca8ffa95 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:48.294194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99b62b4c-3ac2-44ae-9f21-cac4ca8ffa95 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:48.294896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99b62b4c-3ac2-44ae-9f21-cac4ca8ffa95 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:48.295712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99b62b4c-3ac2-44ae-9f21-cac4ca8ffa95 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:48.296241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99b62b4c-3ac2-44ae-9f21-cac4ca8ffa95 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:48.296756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99b62b4c-3ac2-44ae-9f21-cac4ca8ffa95 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:48.742392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7ca9a993-ed10-4179-b2d4-4ea4e44d6c4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:48.775125 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7ca9a993-ed10-4179-b2d4-4ea4e44d6c4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 0.820580 Sep 27 10:00:48.776957 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 508/1162] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:47 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 823 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:48.986203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-056583a3-726c-4206-823c-b9480c1aedc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:48.987295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-056583a3-726c-4206-823c-b9480c1aedc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:48.995612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-056583a3-726c-4206-823c-b9480c1aedc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:48.996235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-056583a3-726c-4206-823c-b9480c1aedc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:48.996702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-056583a3-726c-4206-823c-b9480c1aedc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:48.997638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-056583a3-726c-4206-823c-b9480c1aedc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:48.998278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-056583a3-726c-4206-823c-b9480c1aedc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:48.998891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-056583a3-726c-4206-823c-b9480c1aedc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:49.090589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-99b62b4c-3ac2-44ae-9f21-cac4ca8ffa95 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:49.106322 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-99b62b4c-3ac2-44ae-9f21-cac4ca8ffa95 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1710 microversion: 2.1 time: 0.845143 Sep 27 10:00:49.108317 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 655/1163] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:48 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1710 bytes in 848 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:49.286484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:49.287973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:49.297545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:49.298442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:49.299085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:49.300129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:49.300943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:49.301645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:49.732813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:49.744289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:49.746324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:49.748028 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-23ef1cfe-f3c9-4cbb-bc4b-cb4d0c48e20d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1362 microversion: 2.25 time: 0.466209 Sep 27 10:00:49.767180 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 656/1164] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:49 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1362 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:00:49.807633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d0b9b933-66b8-466f-ad22-9008254ca769 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:49.808201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d0b9b933-66b8-466f-ad22-9008254ca769 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:49.821974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0b9b933-66b8-466f-ad22-9008254ca769 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:49.824519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0b9b933-66b8-466f-ad22-9008254ca769 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:49.825333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0b9b933-66b8-466f-ad22-9008254ca769 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:49.826478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0b9b933-66b8-466f-ad22-9008254ca769 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:49.827179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0b9b933-66b8-466f-ad22-9008254ca769 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:49.827814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0b9b933-66b8-466f-ad22-9008254ca769 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:50.010619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-056583a3-726c-4206-823c-b9480c1aedc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:50.024740 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-056583a3-726c-4206-823c-b9480c1aedc2 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 1.042442 Sep 27 10:00:50.025933 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 509/1165] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:48 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 1044 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:50.133838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9450e7b0-4059-424c-91aa-45733df4ad11 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:50.134470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9450e7b0-4059-424c-91aa-45733df4ad11 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:50.143921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9450e7b0-4059-424c-91aa-45733df4ad11 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:50.145140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9450e7b0-4059-424c-91aa-45733df4ad11 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:50.145140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9450e7b0-4059-424c-91aa-45733df4ad11 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:50.145607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9450e7b0-4059-424c-91aa-45733df4ad11 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:50.146066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9450e7b0-4059-424c-91aa-45733df4ad11 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:50.151085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9450e7b0-4059-424c-91aa-45733df4ad11 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:50.915357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d0b9b933-66b8-466f-ad22-9008254ca769 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:50.925283 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d0b9b933-66b8-466f-ad22-9008254ca769 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1503 microversion: 2.1 time: 1.122511 Sep 27 10:00:50.926557 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 657/1166] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:49 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1503 bytes in 1124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:00:50.935956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:50.942639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:50.958739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:50.959743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:50.961277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:50.961277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:50.961671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:50.962018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:51.208310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9450e7b0-4059-424c-91aa-45733df4ad11 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:51.230503 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9450e7b0-4059-424c-91aa-45733df4ad11 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1710 microversion: 2.1 time: 1.096398 Sep 27 10:00:51.240896 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 510/1167] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:50 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1710 bytes in 1114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:51.255454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1e4dd43d-0e4c-4376-87a9-04c8fb879849 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:51.259892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1e4dd43d-0e4c-4376-87a9-04c8fb879849 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:51.329617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4dd43d-0e4c-4376-87a9-04c8fb879849 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:51.331687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4dd43d-0e4c-4376-87a9-04c8fb879849 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.032s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:51.332177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4dd43d-0e4c-4376-87a9-04c8fb879849 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:51.345496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4dd43d-0e4c-4376-87a9-04c8fb879849 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:51.346845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4dd43d-0e4c-4376-87a9-04c8fb879849 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:51.347488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4dd43d-0e4c-4376-87a9-04c8fb879849 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:51.467063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:51.477567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:51.479599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:51.483485 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e3c1ed21-43ee-4ac0-9f67-82a1ba1a18b0 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1362 microversion: 2.25 time: 0.555500 Sep 27 10:00:51.484842 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 658/1168] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:50 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1362 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:51.964515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7df91aea-95b7-4474-8d3e-086509824003 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:51.965288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7df91aea-95b7-4474-8d3e-086509824003 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:51.978670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df91aea-95b7-4474-8d3e-086509824003 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:51.979600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df91aea-95b7-4474-8d3e-086509824003 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:51.980269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df91aea-95b7-4474-8d3e-086509824003 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:51.981249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df91aea-95b7-4474-8d3e-086509824003 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:51.983370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df91aea-95b7-4474-8d3e-086509824003 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:51.984163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df91aea-95b7-4474-8d3e-086509824003 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:52.282689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1e4dd43d-0e4c-4376-87a9-04c8fb879849 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:52.311093 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1e4dd43d-0e4c-4376-87a9-04c8fb879849 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 1.068553 Sep 27 10:00:52.315516 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 511/1169] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:51 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 1074 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:52.356604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2383d647-6425-4ee8-9f3a-b429b51988c7 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:52.357361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2383d647-6425-4ee8-9f3a-b429b51988c7 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:52.359493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:52.378698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2383d647-6425-4ee8-9f3a-b429b51988c7 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:52.379497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2383d647-6425-4ee8-9f3a-b429b51988c7 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:52.381250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2383d647-6425-4ee8-9f3a-b429b51988c7 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:52.382474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2383d647-6425-4ee8-9f3a-b429b51988c7 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:52.383150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2383d647-6425-4ee8-9f3a-b429b51988c7 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:52.383739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2383d647-6425-4ee8-9f3a-b429b51988c7 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:52.598421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2383d647-6425-4ee8-9f3a-b429b51988c7 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:52.627994 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2383d647-6425-4ee8-9f3a-b429b51988c7 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1643 microversion: 2.1 time: 0.288141 Sep 27 10:00:52.639850 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 512/1170] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:52 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1643 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:52.651822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:52.652783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:52.668725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:52.670467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7df91aea-95b7-4474-8d3e-086509824003 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:52.671397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:52.674292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:52.677228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:52.680537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:52.683546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:52.685394 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7df91aea-95b7-4474-8d3e-086509824003 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1493 microversion: 2.1 time: 0.724098 Sep 27 10:00:52.701111 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 659/1171] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:51 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1493 bytes in 741 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:52.720698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8d1e07d8-996b-485c-a08b-7ca3a34ce318 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '.version_select at 0x7f7f63797a60>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:52.733081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d1e07d8-996b-485c-a08b-7ca3a34ce318 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:52.736264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d1e07d8-996b-485c-a08b-7ca3a34ce318 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:52.737913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d1e07d8-996b-485c-a08b-7ca3a34ce318 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:52.778084 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8d1e07d8-996b-485c-a08b-7ca3a34ce318 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/os-migrations" status: 200 len: 1961 microversion: 2.1 time: 0.064710 Sep 27 10:00:52.780293 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 660/1172] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 10:00:52 2022] GET /compute/v2.1/os-migrations => generated 1961 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:52.803793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8043dab7-00cc-455e-8b28-663e08c103c4 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:52.805573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8043dab7-00cc-455e-8b28-663e08c103c4 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:52.820109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8043dab7-00cc-455e-8b28-663e08c103c4 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:52.823876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8043dab7-00cc-455e-8b28-663e08c103c4 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:52.826357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8043dab7-00cc-455e-8b28-663e08c103c4 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:52.830114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8043dab7-00cc-455e-8b28-663e08c103c4 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:52.830871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8043dab7-00cc-455e-8b28-663e08c103c4 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:52.832138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8043dab7-00cc-455e-8b28-663e08c103c4 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:53.869752 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8043dab7-00cc-455e-8b28-663e08c103c4 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1644 microversion: 2.1 time: 1.075820 Sep 27 10:00:53.870814 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 661/1173] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:52 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1644 bytes in 1077 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:53.878428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:53.897012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf6502d0-65e8-455e-bf5d-4fb63250ed4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:53.901445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:53.901642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bf6502d0-65e8-455e-bf5d-4fb63250ed4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:53.902875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:53.904048 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-420845fa-4978-4969-95bd-25f4cc0cb96a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1362 microversion: 2.25 time: 1.262752 Sep 27 10:00:53.904626 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 513/1174] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:52 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1362 bytes in 1264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:53.916582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf6502d0-65e8-455e-bf5d-4fb63250ed4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:53.917406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf6502d0-65e8-455e-bf5d-4fb63250ed4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:53.918848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf6502d0-65e8-455e-bf5d-4fb63250ed4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:53.924481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf6502d0-65e8-455e-bf5d-4fb63250ed4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:53.925345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf6502d0-65e8-455e-bf5d-4fb63250ed4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:53.930598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf6502d0-65e8-455e-bf5d-4fb63250ed4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:53.933857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ecc6d7f8-beff-42a2-ade1-888dba88514c req-eb28ed9e-a6b4-4c3e-be27-c84463ea1537 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9edcd7b8-f01b-4529-b9c1-3729e79ec760", "name": "network-vif-plugged", "status": "completed", "tag": "5a182f65-e926-4e9f-97dc-7f90cd3a6dc9"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:53.952549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ecc6d7f8-beff-42a2-ade1-888dba88514c req-eb28ed9e-a6b4-4c3e-be27-c84463ea1537 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:53.953078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ecc6d7f8-beff-42a2-ade1-888dba88514c req-eb28ed9e-a6b4-4c3e-be27-c84463ea1537 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:53.953527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ecc6d7f8-beff-42a2-ade1-888dba88514c req-eb28ed9e-a6b4-4c3e-be27-c84463ea1537 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:54.019689 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ecc6d7f8-beff-42a2-ade1-888dba88514c req-eb28ed9e-a6b4-4c3e-be27-c84463ea1537 service nova] Creating event network-vif-plugged:5a182f65-e926-4e9f-97dc-7f90cd3a6dc9 for instance 9edcd7b8-f01b-4529-b9c1-3729e79ec760 on nch2-master-1 Sep 27 10:00:54.034647 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ecc6d7f8-beff-42a2-ade1-888dba88514c req-eb28ed9e-a6b4-4c3e-be27-c84463ea1537 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.128699 Sep 27 10:00:54.041630 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 514/1175] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:54.077538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:54.081472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:54.110439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:54.115723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:54.118704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:54.126162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:54.129157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:54.132459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:54.187347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:54.214158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:54.215053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:54.215758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:54.216455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lazy-loading 'fault' on Instance uuid 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:00:54.396387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4cebc85e62046d28fab159c6bb13e15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1881e013b4b844cba1afce9fd21f8dd7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:54.408587 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-776e04bc-730d-4f14-8a26-ba5702e209ce tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 200 len: 1351 microversion: 2.1 time: 0.364384 Sep 27 10:00:54.409572 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 515/1176] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:54 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 1351 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:54.415029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2256c6ca-482e-41c5-985c-3c1cbb36c10d req-61059485-1d15-475a-b845-d64a05cbc130 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "832b56e9-4ed3-4b6c-a201-d5af937fd9e7", "name": "network-vif-deleted", "tag": "86b49b5d-1243-49df-bcec-4cf58543d8d6"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:54.447564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2256c6ca-482e-41c5-985c-3c1cbb36c10d req-61059485-1d15-475a-b845-d64a05cbc130 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:54.448282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2256c6ca-482e-41c5-985c-3c1cbb36c10d req-61059485-1d15-475a-b845-d64a05cbc130 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:54.449955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2256c6ca-482e-41c5-985c-3c1cbb36c10d req-61059485-1d15-475a-b845-d64a05cbc130 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:54.472701 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2256c6ca-482e-41c5-985c-3c1cbb36c10d req-61059485-1d15-475a-b845-d64a05cbc130 service nova] Creating event network-vif-deleted:86b49b5d-1243-49df-bcec-4cf58543d8d6 for instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 on nch2-master-1 Sep 27 10:00:54.479825 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2256c6ca-482e-41c5-985c-3c1cbb36c10d req-61059485-1d15-475a-b845-d64a05cbc130 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068756 Sep 27 10:00:54.480449 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 516/1177] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:00:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:54.497972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-472385339", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "66dfc7b2-4ec7-40ad-b073-f4df786a999e"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:54.595693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] validate_networks() for [('66dfc7b2-4ec7-40ad-b073-f4df786a999e', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:00:55.005580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bf6502d0-65e8-455e-bf5d-4fb63250ed4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:55.023816 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bf6502d0-65e8-455e-bf5d-4fb63250ed4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 1.149278 Sep 27 10:00:55.026240 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 662/1178] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:53 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 1155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:55.048430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachSCSIVolumeTestJSON-server-1922952780", "imageRef": "6bf85e45-997a-41db-a895-d2a21ca8e103", "flavorRef": "452", "config_drive": true, "networks": [{"uuid": "9775dec5-d53e-4b37-a757-f04bf67a3ca7"}], "security_groups": [{"name": "tempest-securitygroup--56592414"}], "key_name": "tempest-keypair-1899311251", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:55.266855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:00:55.267351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:00:55.267961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:00:55.268375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:00:55.269183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:00:55.281911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:55.282415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:55.282857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:55.283679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:55.284196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:55.284556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:55.311046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Getting quotas for project ec0980b5824b425fbad9b8554cf5c42c. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:00:55.324847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Getting quotas for user dce01d3212934161a38be853b4cd93ae and project ec0980b5824b425fbad9b8554cf5c42c. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:00:55.346883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:00:55.349752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:00:55.420115 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9cb8ea6-f96a-4954-88af-dc91bc0083f3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.938062 Sep 27 10:00:55.421546 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 517/1179] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:00:54 2022] POST /compute/v2.1/servers => generated 380 bytes in 940 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:00:55.447664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] validate_networks() for [('9775dec5-d53e-4b37-a757-f04bf67a3ca7', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:00:55.475562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:55.478095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:55.492099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:55.492951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:55.493048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:55.493738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:55.494201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:55.494637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:55.807389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:55.823997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:55.825517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:55.827500 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-91af34f1-868c-4c4b-9cfb-919b612ec7ba tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1362 microversion: 2.25 time: 0.404663 Sep 27 10:00:55.828932 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 518/1180] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:55 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1362 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:55.851192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-86d5710c-aad0-4b32-a851-fe734305ef4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:55.852239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-86d5710c-aad0-4b32-a851-fe734305ef4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:55.970688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-86d5710c-aad0-4b32-a851-fe734305ef4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:55.980599 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-86d5710c-aad0-4b32-a851-fe734305ef4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1210 microversion: 2.1 time: 0.150125 Sep 27 10:00:55.993553 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 519/1181] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:55 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1210 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:56.003747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:56.004587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:56.012340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:56.013146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:56.014050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:56.014950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:56.015578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:56.016176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:56.079889 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] HTTP exception thrown: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Request method failure captured: Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 HTTP/1.1 Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjPZvwhOgf5DgHDxmEY2IJPi1iZBS_eZLXNcitDauBO2wUr4Jdhflau4krIGdAz3-JusYOeXsBz6rTAxZXMhPE8bOjokYQpnoySExNKZdk_jAKyfaoghRDxvUXyW883Qiix_k9J5MyDqyA48wainH4ZxgL6f_gNG383bUHWxRkqMds Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:00:56.089870 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1881e013b4b844cba1afce9fd21f8dd7"}]}] Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4cebc85e62046d28fab159c6bb13e15 Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:00:56.091340 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '832b56e9-4ed3-4b6c-a201-d5af937fd9e7'} Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:56.092175 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:00:56.094281 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.095535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Returning 404 to user: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:56.110624 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6e6becb9-1401-4429-9e74-b5d08d0a0afd tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 404 len: 111 microversion: 2.1 time: 0.109478 Sep 27 10:00:56.112477 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 520/1182] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:55 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 111 bytes in 118 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:56.122493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-07cad221-dc0c-4046-aa91-4ce006669d53 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:56.123457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-07cad221-dc0c-4046-aa91-4ce006669d53 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:56.158686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad221-dc0c-4046-aa91-4ce006669d53 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:56.159308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad221-dc0c-4046-aa91-4ce006669d53 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:56.159753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad221-dc0c-4046-aa91-4ce006669d53 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:56.165718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad221-dc0c-4046-aa91-4ce006669d53 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:56.166451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad221-dc0c-4046-aa91-4ce006669d53 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:56.167139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad221-dc0c-4046-aa91-4ce006669d53 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:56.427919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-07cad221-dc0c-4046-aa91-4ce006669d53 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:56.448469 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-07cad221-dc0c-4046-aa91-4ce006669d53 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.334457 Sep 27 10:00:56.452153 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 521/1183] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:56 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:56.459119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:56.472545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:56.490264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:56.491052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:56.491891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:56.493380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:56.495092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:56.495675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:56.515176 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] HTTP exception thrown: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Request method failure captured: Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde HTTP/1.1 Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjPZvwhOgf5DgHDxmEY2IJPi1iZBS_eZLXNcitDauBO2wUr4Jdhflau4krIGdAz3-JusYOeXsBz6rTAxZXMhPE8bOjokYQpnoySExNKZdk_jAKyfaoghRDxvUXyW883Qiix_k9J5MyDqyA48wainH4ZxgL6f_gNG383bUHWxRkqMds Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:00:56.517127 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1881e013b4b844cba1afce9fd21f8dd7"}]}] Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4cebc85e62046d28fab159c6bb13e15 Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:00:56.518525 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '030e5dcc-dc63-43e6-9254-ef26b66b5cde'} Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:56.519288 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:00:56.520783 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.522057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Returning 404 to user: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:56.522057 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-69281c1c-1ea3-4d9c-aa41-352db0d4eb02 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "DELETE /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 404 len: 111 microversion: 2.1 time: 0.065598 Sep 27 10:00:56.522057 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 522/1184] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:00:56 2022] DELETE /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:00:56.528878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:56.529521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:56.537237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:56.540372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:56.541644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:56.542817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:56.543796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:56.544508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:56.563486 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] HTTP exception thrown: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Request method failure captured: Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde HTTP/1.1 Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjPZvwhOgf5DgHDxmEY2IJPi1iZBS_eZLXNcitDauBO2wUr4Jdhflau4krIGdAz3-JusYOeXsBz6rTAxZXMhPE8bOjokYQpnoySExNKZdk_jAKyfaoghRDxvUXyW883Qiix_k9J5MyDqyA48wainH4ZxgL6f_gNG383bUHWxRkqMds Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:00:56.565496 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1881e013b4b844cba1afce9fd21f8dd7"}]}] Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4cebc85e62046d28fab159c6bb13e15 Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:00:56.568387 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '030e5dcc-dc63-43e6-9254-ef26b66b5cde'} Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:56.569362 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:00:56.570621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:00:56.572432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Returning 404 to user: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:56.573618 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-157c9c1e-7d3b-46d4-bf39-673d5e88be1d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 404 len: 111 microversion: 2.1 time: 0.049431 Sep 27 10:00:56.574705 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 523/1185] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:56 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:00:56.587816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f9e951ff-efa1-4c99-98fe-c1cd9e74bf4d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '.version_select at 0x7f7f639053a0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:56.617825 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f9e951ff-efa1-4c99-98fe-c1cd9e74bf4d tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-211851818" status: 202 len: 0 microversion: 2.1 time: 0.038074 Sep 27 10:00:56.631717 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 524/1186] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:00:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-211851818 => generated 0 bytes in 53 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:00:56.866357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:56.871958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:56.898682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:56.901249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:56.903281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:56.904177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:56.906315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:56.908823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:57.309905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:00:57.503309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:57.516099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:57.525900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:57.528491 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f2e01b05-7db7-4caa-8a4d-9ed43aad61d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1557 microversion: 2.25 time: 0.673443 Sep 27 10:00:57.532336 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 525/1187] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:56 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1557 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:57.540849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4fa9fd70-a9e3-4114-87bd-63a85e677014 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:57.545933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4fa9fd70-a9e3-4114-87bd-63a85e677014 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:57.563594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fa9fd70-a9e3-4114-87bd-63a85e677014 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:57.564824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fa9fd70-a9e3-4114-87bd-63a85e677014 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:57.567918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fa9fd70-a9e3-4114-87bd-63a85e677014 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:57.569105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fa9fd70-a9e3-4114-87bd-63a85e677014 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:57.569836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fa9fd70-a9e3-4114-87bd-63a85e677014 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:57.572526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fa9fd70-a9e3-4114-87bd-63a85e677014 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:57.772671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4fa9fd70-a9e3-4114-87bd-63a85e677014 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:57.781213 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4fa9fd70-a9e3-4114-87bd-63a85e677014 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1270 microversion: 2.1 time: 0.247393 Sep 27 10:00:57.784583 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 526/1188] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:57 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1270 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:57.870767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:00:57.871137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:00:57.871653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:00:57.872198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:00:57.872698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:00:57.891942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:57.892692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:57.894078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:57.895055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:57.895730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:57.901122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:57.943565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Getting quotas for project 5de10e92e2024a24880c7205697e5732. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:00:57.950106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Getting quotas for user 0d563feacff84b4c9927ec759fb365be and project 5de10e92e2024a24880c7205697e5732. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:00:57.974693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:00:57.990508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='6bf85e45-997a-41db-a895-d2a21ca8e103',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:00:58.087155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb4df78b-8afd-4606-b65d-ae4729585075 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f637f90d0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-1380955427", "description": "tempest-description-1953889793"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:58.093630 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a302210f-60dc-4b1c-8793-47fbd43d855a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 3.065900 Sep 27 10:00:58.099667 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 663/1189] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:00:55 2022] POST /compute/v2.1/servers => generated 404 bytes in 3072 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:00:58.120528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed9f8b8e-5d3a-4495-bda5-656c3e9bcd86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:58.121528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed9f8b8e-5d3a-4495-bda5-656c3e9bcd86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:58.136055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed9f8b8e-5d3a-4495-bda5-656c3e9bcd86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:58.136676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed9f8b8e-5d3a-4495-bda5-656c3e9bcd86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:58.137426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed9f8b8e-5d3a-4495-bda5-656c3e9bcd86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:58.138498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed9f8b8e-5d3a-4495-bda5-656c3e9bcd86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:58.139196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed9f8b8e-5d3a-4495-bda5-656c3e9bcd86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:58.139839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed9f8b8e-5d3a-4495-bda5-656c3e9bcd86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:58.505183 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fb4df78b-8afd-4606-b65d-ae4729585075 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 256 microversion: 2.1 time: 0.705611 Sep 27 10:00:58.507349 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 527/1190] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:00:57 2022] POST /compute/v2.1/os-security-groups => generated 256 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:58.522557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5009f468-7be6-4212-8865-e608fef4de04 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:58.526524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5009f468-7be6-4212-8865-e608fef4de04 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:58.548530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5009f468-7be6-4212-8865-e608fef4de04 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:58.549103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5009f468-7be6-4212-8865-e608fef4de04 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:58.549547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5009f468-7be6-4212-8865-e608fef4de04 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:58.551455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5009f468-7be6-4212-8865-e608fef4de04 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:58.553248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5009f468-7be6-4212-8865-e608fef4de04 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:58.553986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5009f468-7be6-4212-8865-e608fef4de04 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:58.828940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-5009f468-7be6-4212-8865-e608fef4de04 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid 9edcd7b8-f01b-4529-b9c1-3729e79ec760 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:00:58.875236 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5009f468-7be6-4212-8865-e608fef4de04 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 2050 microversion: 2.25 time: 0.365053 Sep 27 10:00:58.876847 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 528/1191] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:58 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 2050 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:58.884987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9774da34-b0ed-4a39-89fb-9eb0cf8fdae3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:58.886363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9774da34-b0ed-4a39-89fb-9eb0cf8fdae3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:58.990161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9774da34-b0ed-4a39-89fb-9eb0cf8fdae3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:59.006720 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9774da34-b0ed-4a39-89fb-9eb0cf8fdae3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1246 microversion: 2.1 time: 0.128243 Sep 27 10:00:59.007818 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 529/1192] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:58 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1246 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:59.014523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0204bb7d-65e2-4051-a425-d91318ba04f2 tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:59.056430 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0204bb7d-65e2-4051-a425-d91318ba04f2 tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/720147853" status: 202 len: 0 microversion: 2.1 time: 0.046114 Sep 27 10:00:59.057693 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 530/1193] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 10:00:59 2022] DELETE /compute/v2.1/flavors/720147853 => generated 0 bytes in 49 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:00:59.066035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba7f6d02-4f96-442f-b90c-b20ff15cc13c tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f636aec10>, body: {"security_group_rule": {"parent_group_id": "5744c4c0-cf3d-4604-b2cc-383261adbdb0", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:59.199170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ed9f8b8e-5d3a-4495-bda5-656c3e9bcd86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:59.212146 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed9f8b8e-5d3a-4495-bda5-656c3e9bcd86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 1.110702 Sep 27 10:00:59.214322 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 664/1194] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:58 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 1114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:59.223186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-814b61e1-57c7-40b1-b47e-49f18c7bda8e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:59.224689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-814b61e1-57c7-40b1-b47e-49f18c7bda8e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:59.235753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-814b61e1-57c7-40b1-b47e-49f18c7bda8e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:59.236736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-814b61e1-57c7-40b1-b47e-49f18c7bda8e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:59.237456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-814b61e1-57c7-40b1-b47e-49f18c7bda8e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:59.238435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-814b61e1-57c7-40b1-b47e-49f18c7bda8e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:59.239862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-814b61e1-57c7-40b1-b47e-49f18c7bda8e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:59.240777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-814b61e1-57c7-40b1-b47e-49f18c7bda8e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:59.463152 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ba7f6d02-4f96-442f-b90c-b20ff15cc13c tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 232 microversion: 2.1 time: 0.403497 Sep 27 10:00:59.464423 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 531/1195] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:00:59 2022] POST /compute/v2.1/os-security-group-rules => generated 232 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:00:59.471320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1b30966b-e46e-455d-abba-fcf546a2cd99 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:59.472173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1b30966b-e46e-455d-abba-fcf546a2cd99 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:00:59.491054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b30966b-e46e-455d-abba-fcf546a2cd99 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:59.491694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b30966b-e46e-455d-abba-fcf546a2cd99 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:59.493194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b30966b-e46e-455d-abba-fcf546a2cd99 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:59.494316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b30966b-e46e-455d-abba-fcf546a2cd99 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:59.494790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b30966b-e46e-455d-abba-fcf546a2cd99 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:59.495243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b30966b-e46e-455d-abba-fcf546a2cd99 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:59.524227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-814b61e1-57c7-40b1-b47e-49f18c7bda8e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:00:59.547681 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-814b61e1-57c7-40b1-b47e-49f18c7bda8e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1268 microversion: 2.1 time: 0.327576 Sep 27 10:00:59.549310 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 665/1196] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:00:59 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1268 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:59.570920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a548f725-a027-4730-a92d-3485eb7c30b6 tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:59.662099 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a548f725-a027-4730-a92d-3485eb7c30b6 tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5403 microversion: 2.1 time: 0.104102 Sep 27 10:00:59.665633 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 666/1197] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:00:59 2022] GET /compute/v2.1/flavors/detail => generated 5403 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:00:59.683458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d239aec-88d2-4ee4-aa7f-a05a4ba7b28a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f6392b820>, body: {"security_group_rule": {"parent_group_id": "5744c4c0-cf3d-4604-b2cc-383261adbdb0", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:59.845312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-1b30966b-e46e-455d-abba-fcf546a2cd99 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid 9edcd7b8-f01b-4529-b9c1-3729e79ec760 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:00:59.872595 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1b30966b-e46e-455d-abba-fcf546a2cd99 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 2050 microversion: 2.25 time: 0.406608 Sep 27 10:00:59.873882 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 532/1198] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:00:59 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 2050 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:00:59.879499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-69cadf04-050a-45de-86ab-ece30b83a8de tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:59.904533 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-69cadf04-050a-45de-86ab-ece30b83a8de tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/436599854" status: 202 len: 0 microversion: 2.1 time: 0.029111 Sep 27 10:00:59.905445 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 533/1199] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 10:00:59 2022] DELETE /compute/v2.1/flavors/436599854 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:00:59.917642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d6e49143-71d5-4f40-a6d7-a843d0f295a9 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:59.922762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6e49143-71d5-4f40-a6d7-a843d0f295a9 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:59.923273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6e49143-71d5-4f40-a6d7-a843d0f295a9 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:59.923713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6e49143-71d5-4f40-a6d7-a843d0f295a9 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:59.925378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6e49143-71d5-4f40-a6d7-a843d0f295a9 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:00:59.925861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6e49143-71d5-4f40-a6d7-a843d0f295a9 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:00:59.926509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6e49143-71d5-4f40-a6d7-a843d0f295a9 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:00:59.955338 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-5d239aec-88d2-4ee4-aa7f-a05a4ba7b28a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] HTTP exception thrown: This rule already exists in group 5744c4c0-cf3d-4604-b2cc-383261adbdb0 Sep 27 10:00:59.956527 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d6e49143-71d5-4f40-a6d7-a843d0f295a9 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 424 microversion: 2.25 time: 0.046153 Sep 27 10:00:59.957345 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 534/1200] 10.222.0.26 () {60 vars in 1217 bytes} [Tue Sep 27 10:00:59 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 424 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d239aec-88d2-4ee4-aa7f-a05a4ba7b28a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Request method failure captured: Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 138 Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnZBt2Q4ufyfBqR8j38D6_unm922QvijuZubeV1Qk8gogf1P2WqTN3o2PejQ807D5p3FDkutceU80OokvIPrC4v4-5vXDNOoNSJqfq9Fu6FoR717_4uWMpOOjK2TPwFT6VqpfH_NzAIatJt9NFdrtHW9ec983tF0bwaHAoIP6ClDWA Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5db36951642427481c9b79db0009118 Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:00:59.966734 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5db36951642427481c9b79db0009118"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5db36951642427481c9b79db0009118"}]}] Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5db36951642427481c9b79db0009118 Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 546c5dc3d40c4e72ac683f084429b5f8 Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: {"security_group_rule": {"parent_group_id": "5744c4c0-cf3d-4604-b2cc-383261adbdb0", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} Sep 27 10:00:59.968466 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: exception: This rule already exists in group 5744c4c0-cf3d-4604-b2cc-383261adbdb0 Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'security_group_rule': {'parent_group_id': '5744c4c0-cf3d-4604-b2cc-383261adbdb0', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 22}}} Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 324, in create Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_group_api.create_security_group_rule( Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 186, in create_security_group_rule Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.Invalid(msg) Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: This rule already exists in group 5744c4c0-cf3d-4604-b2cc-383261adbdb0 Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:00:59.969461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 327, in create Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: This rule already exists in group 5744c4c0-cf3d-4604-b2cc-383261adbdb0 Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: This rule already exists in group 5744c4c0-cf3d-4604-b2cc-383261adbdb0 Sep 27 10:00:59.970948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:00:59.975587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d239aec-88d2-4ee4-aa7f-a05a4ba7b28a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Returning 400 to user: This rule already exists in group 5744c4c0-cf3d-4604-b2cc-383261adbdb0 {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:00:59.980335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ac94875d-b04e-4e32-b545-2919ef934b6c tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:00:59.981683 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5d239aec-88d2-4ee4-aa7f-a05a4ba7b28a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 114 microversion: 2.1 time: 0.314229 Sep 27 10:00:59.986056 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 667/1201] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:00:59 2022] POST /compute/v2.1/os-security-group-rules => generated 114 bytes in 319 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:00:59.991616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-969f1d1f-8e25-4eac-9235-b4beb0e6db67 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:00:59.992474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-969f1d1f-8e25-4eac-9235-b4beb0e6db67 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:00.000564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-969f1d1f-8e25-4eac-9235-b4beb0e6db67 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:00.008212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-969f1d1f-8e25-4eac-9235-b4beb0e6db67 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:00.009000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-969f1d1f-8e25-4eac-9235-b4beb0e6db67 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:00.009930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-969f1d1f-8e25-4eac-9235-b4beb0e6db67 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:00.010577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-969f1d1f-8e25-4eac-9235-b4beb0e6db67 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:00.011164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-969f1d1f-8e25-4eac-9235-b4beb0e6db67 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:00.020873 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ac94875d-b04e-4e32-b545-2919ef934b6c tempest-ServersWithSpecificFlavorTestJSON-417161768 tempest-ServersWithSpecificFlavorTestJSON-417161768-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.051829 Sep 27 10:01:00.022342 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 535/1202] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:00:59 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:00.027617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c521d6bb-8dfb-402d-95c4-36bfa754143a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Calling method '.version_select at 0x7f7f636914c0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:00.120019 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-969f1d1f-8e25-4eac-9235-b4beb0e6db67 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760/action" status: 202 len: 0 microversion: 2.25 time: 0.132301 Sep 27 10:01:00.123576 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 668/1203] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:00:59 2022] POST /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760/action => generated 0 bytes in 137 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:01:00.133077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4acdaebe-4064-4df2-b54c-67aeb4e97220 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:00.134183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4acdaebe-4064-4df2-b54c-67aeb4e97220 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:00.141650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4acdaebe-4064-4df2-b54c-67aeb4e97220 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:00.142622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4acdaebe-4064-4df2-b54c-67aeb4e97220 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:00.143255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4acdaebe-4064-4df2-b54c-67aeb4e97220 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:00.144186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4acdaebe-4064-4df2-b54c-67aeb4e97220 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:00.145118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4acdaebe-4064-4df2-b54c-67aeb4e97220 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:00.145759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4acdaebe-4064-4df2-b54c-67aeb4e97220 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:00.232458 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c521d6bb-8dfb-402d-95c4-36bfa754143a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-group-rules/b366054c-8134-4949-ab8c-ac9c45f55df8" status: 202 len: 0 microversion: 2.1 time: 0.208488 Sep 27 10:01:00.233905 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 536/1204] 10.222.0.26 () {58 vars in 1287 bytes} [Tue Sep 27 10:01:00 2022] DELETE /compute/v2.1/os-security-group-rules/b366054c-8134-4949-ab8c-ac9c45f55df8 => generated 0 bytes in 211 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:01:00.248100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:00.248692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:00.276196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:00.277137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:00.277890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:00.280411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:00.282263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:00.283073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:00.304071 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] HTTP exception thrown: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Request method failure captured: Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 HTTP/1.1 Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjPZvwhOgf5DgHDxmEY2IJPi1iZBS_eZLXNcitDauBO2wUr4Jdhflau4krIGdAz3-JusYOeXsBz6rTAxZXMhPE8bOjokYQpnoySExNKZdk_jAKyfaoghRDxvUXyW883Qiix_k9J5MyDqyA48wainH4ZxgL6f_gNG383bUHWxRkqMds Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:01:00.308180 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1881e013b4b844cba1afce9fd21f8dd7"}]}] Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4cebc85e62046d28fab159c6bb13e15 Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:01:00.310172 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '832b56e9-4ed3-4b6c-a201-d5af937fd9e7'} Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:00.311407 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:01:00.313220 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:00.326939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Returning 404 to user: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:00.336272 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6f841e53-1f08-4133-95b9-005804684896 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "DELETE /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 404 len: 111 microversion: 2.1 time: 0.100894 Sep 27 10:01:00.342337 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 537/1205] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:01:00 2022] DELETE /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 111 bytes in 107 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:01:00.348068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b7fc73bb-187f-4e5d-b29e-09e66fb996bf tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:00.348744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b7fc73bb-187f-4e5d-b29e-09e66fb996bf tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:00.372115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7fc73bb-187f-4e5d-b29e-09e66fb996bf tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:00.373002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7fc73bb-187f-4e5d-b29e-09e66fb996bf tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:00.373826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7fc73bb-187f-4e5d-b29e-09e66fb996bf tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:00.374720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7fc73bb-187f-4e5d-b29e-09e66fb996bf tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:00.375333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7fc73bb-187f-4e5d-b29e-09e66fb996bf tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:00.375923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7fc73bb-187f-4e5d-b29e-09e66fb996bf tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:00.667073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-b7fc73bb-187f-4e5d-b29e-09e66fb996bf tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid 9edcd7b8-f01b-4529-b9c1-3729e79ec760 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:01:00.704133 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b7fc73bb-187f-4e5d-b29e-09e66fb996bf tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 2055 microversion: 2.25 time: 0.358214 Sep 27 10:01:00.706406 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 538/1206] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:00 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 2055 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:00.715903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aed14987-8873-45d1-a020-54b8d2b64721 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:00.716952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aed14987-8873-45d1-a020-54b8d2b64721 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:00.725438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aed14987-8873-45d1-a020-54b8d2b64721 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:00.726611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aed14987-8873-45d1-a020-54b8d2b64721 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:00.727552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aed14987-8873-45d1-a020-54b8d2b64721 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:00.728754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aed14987-8873-45d1-a020-54b8d2b64721 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:00.730463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aed14987-8873-45d1-a020-54b8d2b64721 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:00.731395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aed14987-8873-45d1-a020-54b8d2b64721 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:00.799620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4acdaebe-4064-4df2-b54c-67aeb4e97220 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:00.819155 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4acdaebe-4064-4df2-b54c-67aeb4e97220 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1298 microversion: 2.1 time: 0.693471 Sep 27 10:01:00.820086 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 669/1207] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:00 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1298 bytes in 696 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:00.825388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23ebe248-7429-4ef8-ab49-fb9483f76642 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f63730430>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-2055095827", "description": "tempest-description-2010891683"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:01.014749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aed14987-8873-45d1-a020-54b8d2b64721 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:01.052892 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aed14987-8873-45d1-a020-54b8d2b64721 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1543 microversion: 2.1 time: 0.332844 Sep 27 10:01:01.060239 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 539/1208] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:00 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1543 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:01.087086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:01.090278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 832b56e9-4ed3-4b6c-a201-d5af937fd9e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:01.105612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:01.112491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:01.113076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:01.114025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:01.114514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:01.115220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:01.169789 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] HTTP exception thrown: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Request method failure captured: Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 HTTP/1.1 Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjPZvwhOgf5DgHDxmEY2IJPi1iZBS_eZLXNcitDauBO2wUr4Jdhflau4krIGdAz3-JusYOeXsBz6rTAxZXMhPE8bOjokYQpnoySExNKZdk_jAKyfaoghRDxvUXyW883Qiix_k9J5MyDqyA48wainH4ZxgL6f_gNG383bUHWxRkqMds Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:01:01.177152 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1881e013b4b844cba1afce9fd21f8dd7"}]}] Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4cebc85e62046d28fab159c6bb13e15 Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:01:01.179224 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '832b56e9-4ed3-4b6c-a201-d5af937fd9e7'} Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:01.180115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. Sep 27 10:01:01.181880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.183634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Returning 404 to user: Instance 832b56e9-4ed3-4b6c-a201-d5af937fd9e7 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:01.183634 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a89993c7-9dcd-4192-aca2-78e43e909c74 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7" status: 404 len: 111 microversion: 2.1 time: 0.117097 Sep 27 10:01:01.187847 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 540/1209] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:01 2022] GET /compute/v2.1/servers/832b56e9-4ed3-4b6c-a201-d5af937fd9e7 => generated 111 bytes in 122 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:01:01.207166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8a91d61b-2da4-4676-93f0-2d207c72ffdf tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:01.207792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8a91d61b-2da4-4676-93f0-2d207c72ffdf tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:01.222226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a91d61b-2da4-4676-93f0-2d207c72ffdf tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:01.222736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a91d61b-2da4-4676-93f0-2d207c72ffdf tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:01.223173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a91d61b-2da4-4676-93f0-2d207c72ffdf tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:01.223904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a91d61b-2da4-4676-93f0-2d207c72ffdf tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:01.224370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a91d61b-2da4-4676-93f0-2d207c72ffdf tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:01.224836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a91d61b-2da4-4676-93f0-2d207c72ffdf tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:01.577997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8a91d61b-2da4-4676-93f0-2d207c72ffdf tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:01.595424 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8a91d61b-2da4-4676-93f0-2d207c72ffdf tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1310 microversion: 2.1 time: 0.406250 Sep 27 10:01:01.596020 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 541/1210] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:01 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1310 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:01.621765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:01.622538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:01.628994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:01.630007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:01.630629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:01.631518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:01.632142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:01.632783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:01.648067 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] HTTP exception thrown: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Request method failure captured: Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde HTTP/1.1 Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjPZvwhOgf5DgHDxmEY2IJPi1iZBS_eZLXNcitDauBO2wUr4Jdhflau4krIGdAz3-JusYOeXsBz6rTAxZXMhPE8bOjokYQpnoySExNKZdk_jAKyfaoghRDxvUXyW883Qiix_k9J5MyDqyA48wainH4ZxgL6f_gNG383bUHWxRkqMds Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:01:01.650674 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1881e013b4b844cba1afce9fd21f8dd7"}]}] Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4cebc85e62046d28fab159c6bb13e15 Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:01:01.652108 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '030e5dcc-dc63-43e6-9254-ef26b66b5cde'} Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:01.654573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:01:01.655809 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.658753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Returning 404 to user: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:01.658753 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1eb6921f-94c7-436e-9c26-9e73d36c4162 tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "DELETE /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 404 len: 111 microversion: 2.1 time: 0.055150 Sep 27 10:01:01.658753 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 542/1211] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:01:01 2022] DELETE /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:01:01.660862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:01.661495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] [instance: 030e5dcc-dc63-43e6-9254-ef26b66b5cde] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:01.667741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:01.668234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:01.668722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:01.669424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:01.669898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:01.670309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:01.691731 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] HTTP exception thrown: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Request method failure captured: Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde HTTP/1.1 Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsjPZvwhOgf5DgHDxmEY2IJPi1iZBS_eZLXNcitDauBO2wUr4Jdhflau4krIGdAz3-JusYOeXsBz6rTAxZXMhPE8bOjokYQpnoySExNKZdk_jAKyfaoghRDxvUXyW883Qiix_k9J5MyDqyA48wainH4ZxgL6f_gNG383bUHWxRkqMds Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:01:01.693336 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1881e013b4b844cba1afce9fd21f8dd7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1881e013b4b844cba1afce9fd21f8dd7"}]}] Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1881e013b4b844cba1afce9fd21f8dd7 Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898 Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4cebc85e62046d28fab159c6bb13e15 Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersWithSpecificFlavorTestJSON-714727898-project Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:01:01.694819 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '030e5dcc-dc63-43e6-9254-ef26b66b5cde'} Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:01.695867 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. Sep 27 10:01:01.697319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:01.699383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] Returning 404 to user: Instance 030e5dcc-dc63-43e6-9254-ef26b66b5cde could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:01.700595 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5465f79a-69b0-48bb-b625-80fd518e790e tempest-ServersWithSpecificFlavorTestJSON-714727898 tempest-ServersWithSpecificFlavorTestJSON-714727898-project] 10.222.0.26 "GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde" status: 404 len: 111 microversion: 2.1 time: 0.043677 Sep 27 10:01:01.705066 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 543/1212] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:01 2022] GET /compute/v2.1/servers/030e5dcc-dc63-43e6-9254-ef26b66b5cde => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:01:01.732637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3bb68a3a-fbd9-406d-8c18-49e94d86b482 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:01.733171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3bb68a3a-fbd9-406d-8c18-49e94d86b482 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:01.743749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb68a3a-fbd9-406d-8c18-49e94d86b482 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:01.744256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb68a3a-fbd9-406d-8c18-49e94d86b482 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:01.744718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb68a3a-fbd9-406d-8c18-49e94d86b482 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:01.745455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb68a3a-fbd9-406d-8c18-49e94d86b482 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:01.745933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb68a3a-fbd9-406d-8c18-49e94d86b482 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:01.746343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb68a3a-fbd9-406d-8c18-49e94d86b482 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:02.304931 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-23ebe248-7429-4ef8-ab49-fb9483f76642 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 256 microversion: 2.1 time: 1.482702 Sep 27 10:01:02.308630 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 670/1213] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:01:00 2022] POST /compute/v2.1/os-security-groups => generated 256 bytes in 1488 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:02.320645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-51552352-656e-48cb-81c9-25f876e66383 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:02.326490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-51552352-656e-48cb-81c9-25f876e66383 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:02.356471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51552352-656e-48cb-81c9-25f876e66383 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:02.357845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51552352-656e-48cb-81c9-25f876e66383 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:02.359487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51552352-656e-48cb-81c9-25f876e66383 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:02.360482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51552352-656e-48cb-81c9-25f876e66383 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:02.365709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51552352-656e-48cb-81c9-25f876e66383 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:02.367101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51552352-656e-48cb-81c9-25f876e66383 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:02.476222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:03.064818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-3bb68a3a-fbd9-406d-8c18-49e94d86b482 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid 9edcd7b8-f01b-4529-b9c1-3729e79ec760 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:01:03.113910 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3bb68a3a-fbd9-406d-8c18-49e94d86b482 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 2035 microversion: 2.25 time: 1.385376 Sep 27 10:01:03.133429 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 544/1214] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:01 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 2035 bytes in 1406 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:01:03.148175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2888b1f2-2f18-4498-8d5f-5a3b7f463e15 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:03.149145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2888b1f2-2f18-4498-8d5f-5a3b7f463e15 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:03.155803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2888b1f2-2f18-4498-8d5f-5a3b7f463e15 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:03.156563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2888b1f2-2f18-4498-8d5f-5a3b7f463e15 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:03.157262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2888b1f2-2f18-4498-8d5f-5a3b7f463e15 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:03.158195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2888b1f2-2f18-4498-8d5f-5a3b7f463e15 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:03.158910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2888b1f2-2f18-4498-8d5f-5a3b7f463e15 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:03.159685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2888b1f2-2f18-4498-8d5f-5a3b7f463e15 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:04.131010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-51552352-656e-48cb-81c9-25f876e66383 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:04.157840 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-51552352-656e-48cb-81c9-25f876e66383 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1370 microversion: 2.1 time: 1.835902 Sep 27 10:01:04.159113 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 671/1215] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:02 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1370 bytes in 1849 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:04.167155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c984935-5c91-4bc6-baec-540f07bdc7fd tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f63730dc0>, body: {"security_group_rule": {"parent_group_id": "1a0ef65a-e693-4e1b-9b0a-c6fa4a8b8a9c", "ip_protocol": "tcp", "from_port": 678495830, "to_port": 22}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:04.420863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2888b1f2-2f18-4498-8d5f-5a3b7f463e15 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:04.452752 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2888b1f2-2f18-4498-8d5f-5a3b7f463e15 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1542 microversion: 2.1 time: 1.315692 Sep 27 10:01:04.454592 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 545/1216] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:03 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1542 bytes in 1320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:04.463626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cb66079e-575f-40ec-85c0-843e77282174 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:04.476768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cb66079e-575f-40ec-85c0-843e77282174 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:04.491626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb66079e-575f-40ec-85c0-843e77282174 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:04.494354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb66079e-575f-40ec-85c0-843e77282174 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:04.495572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb66079e-575f-40ec-85c0-843e77282174 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:04.499562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb66079e-575f-40ec-85c0-843e77282174 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:04.500330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb66079e-575f-40ec-85c0-843e77282174 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:04.501000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb66079e-575f-40ec-85c0-843e77282174 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:04.551297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:04.889052 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-2c984935-5c91-4bc6-baec-540f07bdc7fd tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] HTTP exception thrown: Invalid port range 678495830:22. Former value cannot be greater than the later Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c984935-5c91-4bc6-baec-540f07bdc7fd tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Request method failure captured: Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 145 Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnZBt2Q4ufyfBqR8j38D6_unm922QvijuZubeV1Qk8gogf1P2WqTN3o2PejQ807D5p3FDkutceU80OokvIPrC4v4-5vXDNOoNSJqfq9Fu6FoR717_4uWMpOOjK2TPwFT6VqpfH_NzAIatJt9NFdrtHW9ec983tF0bwaHAoIP6ClDWA Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:01:04.892858 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5db36951642427481c9b79db0009118"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5db36951642427481c9b79db0009118"}]}] Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 546c5dc3d40c4e72ac683f084429b5f8 Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: {"security_group_rule": {"parent_group_id": "1a0ef65a-e693-4e1b-9b0a-c6fa4a8b8a9c", "ip_protocol": "tcp", "from_port": 678495830, "to_port": 22}} Sep 27 10:01:04.894270 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid port range 678495830:22. Former value cannot be greater than the later Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'security_group_rule': {'parent_group_id': '1a0ef65a-e693-4e1b-9b0a-c6fa4a8b8a9c', 'ip_protocol': 'tcp', 'from_port': 678495830, 'to_port': 22}}} Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 150, in _new_ingress_rule Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InvalidPortRange(from_port=from_port, Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InvalidPortRange: Invalid port range 678495830:22. Former value cannot be greater than the later Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:04.895061 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid port range 678495830:22. Former value cannot be greater than the later Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid port range 678495830:22. Former value cannot be greater than the later Sep 27 10:01:04.896270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:04.897448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c984935-5c91-4bc6-baec-540f07bdc7fd tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Returning 400 to user: Invalid port range 678495830:22. Former value cannot be greater than the later {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:04.897448 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c984935-5c91-4bc6-baec-540f07bdc7fd tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 122 microversion: 2.1 time: 0.734431 Sep 27 10:01:04.897448 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 672/1217] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:01:04 2022] POST /compute/v2.1/os-security-group-rules => generated 122 bytes in 736 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:01:04.999645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-976735228", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "4568aca4-c33f-4379-b9d1-22f70d115851"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:05.105228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] validate_networks() for [('4568aca4-c33f-4379-b9d1-22f70d115851', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:01:05.259240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cb66079e-575f-40ec-85c0-843e77282174 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:05.275161 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cb66079e-575f-40ec-85c0-843e77282174 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1310 microversion: 2.1 time: 0.816818 Sep 27 10:01:05.278483 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 546/1218] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:04 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1310 bytes in 823 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:05.298000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2f818668-524b-4861-ae3e-d2622b5640e8 req-ca84055a-9b42-44da-a545-528aa808e944 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69ea013c-5143-45ff-81a2-3ed97a8659cd", "name": "network-vif-unplugged", "status": "completed", "tag": "96ac21a6-013a-4a5e-9320-82ff90906b76"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:05.322795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2f818668-524b-4861-ae3e-d2622b5640e8 req-ca84055a-9b42-44da-a545-528aa808e944 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:05.325002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2f818668-524b-4861-ae3e-d2622b5640e8 req-ca84055a-9b42-44da-a545-528aa808e944 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:05.325730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2f818668-524b-4861-ae3e-d2622b5640e8 req-ca84055a-9b42-44da-a545-528aa808e944 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:05.361269 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2f818668-524b-4861-ae3e-d2622b5640e8 req-ca84055a-9b42-44da-a545-528aa808e944 service nova] Creating event network-vif-unplugged:96ac21a6-013a-4a5e-9320-82ff90906b76 for instance 69ea013c-5143-45ff-81a2-3ed97a8659cd on nch2-master-1 Sep 27 10:01:05.375689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-2f818668-524b-4861-ae3e-d2622b5640e8 req-ca84055a-9b42-44da-a545-528aa808e944 service nova] Instance 69ea013c-5143-45ff-81a2-3ed97a8659cd is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:01:05.418949 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2f818668-524b-4861-ae3e-d2622b5640e8 req-ca84055a-9b42-44da-a545-528aa808e944 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.138209 Sep 27 10:01:05.423379 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 547/1219] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:05 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:05.519557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa6481cb-b0bd-4bc6-a673-8742bcde6e81 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": "auto"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:05.529677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aa6481cb-b0bd-4bc6-a673-8742bcde6e81 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:05.557678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=3,nova_api:DELETE=7 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:05.564244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa6481cb-b0bd-4bc6-a673-8742bcde6e81 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:05.565835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa6481cb-b0bd-4bc6-a673-8742bcde6e81 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:05.568139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa6481cb-b0bd-4bc6-a673-8742bcde6e81 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:05.570676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa6481cb-b0bd-4bc6-a673-8742bcde6e81 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:05.571400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa6481cb-b0bd-4bc6-a673-8742bcde6e81 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:05.575489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa6481cb-b0bd-4bc6-a673-8742bcde6e81 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:05.641494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-aa6481cb-b0bd-4bc6-a673-8742bcde6e81 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'flavor' on Instance uuid 9edcd7b8-f01b-4529-b9c1-3729e79ec760 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:01:05.675444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aa6481cb-b0bd-4bc6-a673-8742bcde6e81 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Going to try to live migrate instance to nch1-master-1 {{(pid=145969) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Sep 27 10:01:07.909296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:07.969667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:09.001403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=10 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:09.077234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:01:09.078308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:01:09.078795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:01:09.079209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:01:09.079622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:01:09.098490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:09.098986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:09.099441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:09.100182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:09.100749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:09.105538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:09.170747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Getting quotas for project e5a565f65b3849a99b74454791397ca5. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:01:09.177710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Getting quotas for user 6eb3ccfcd9c344e1a7ad55f8398ec87a and project e5a565f65b3849a99b74454791397ca5. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:01:09.201956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:01:09.207060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:01:09.347261 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b73827c0-8d4b-4cbe-aa56-e907327c44f3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.450330 Sep 27 10:01:09.348971 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 673/1220] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:01:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 4453 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:01:09.356987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90b50bb6-a2ed-4935-8003-14208110b722 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f63730940>, body: {"security_group_rule": {"parent_group_id": "tempest-non_int_id-1047274831", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:09.360341 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-90b50bb6-a2ed-4935-8003-14208110b722 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] HTTP exception thrown: Security group id should be uuid Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90b50bb6-a2ed-4935-8003-14208110b722 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Request method failure captured: Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 131 Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnZBt2Q4ufyfBqR8j38D6_unm922QvijuZubeV1Qk8gogf1P2WqTN3o2PejQ807D5p3FDkutceU80OokvIPrC4v4-5vXDNOoNSJqfq9Fu6FoR717_4uWMpOOjK2TPwFT6VqpfH_NzAIatJt9NFdrtHW9ec983tF0bwaHAoIP6ClDWA Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:01:09.363030 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5db36951642427481c9b79db0009118"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5db36951642427481c9b79db0009118"}]}] Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 546c5dc3d40c4e72ac683f084429b5f8 Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: {"security_group_rule": {"parent_group_id": "tempest-non_int_id-1047274831", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:01:09.364958 ncdv-master-1 devstack@n-api.service[145968]: exception: Security group id should be uuid Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'security_group_rule': {'parent_group_id': 'tempest-non_int_id-1047274831', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 22}}} Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 285, in create Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi parent_group_id = security_group_api.validate_id( Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 51, in validate_id Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.Invalid(msg) Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Security group id should be uuid Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:09.365927 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Security group id should be uuid Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group id should be uuid Sep 27 10:01:09.368544 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:09.401964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90b50bb6-a2ed-4935-8003-14208110b722 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Returning 400 to user: Security group id should be uuid {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:09.403258 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-90b50bb6-a2ed-4935-8003-14208110b722 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 76 microversion: 2.1 time: 0.053046 Sep 27 10:01:09.404918 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 674/1221] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:01:09 2022] POST /compute/v2.1/os-security-group-rules => generated 76 bytes in 55 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:09.423238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6228b664-a456-4521-99ef-e201a4ed10d0 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:09.424261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6228b664-a456-4521-99ef-e201a4ed10d0 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:09.435633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6228b664-a456-4521-99ef-e201a4ed10d0 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:09.436601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6228b664-a456-4521-99ef-e201a4ed10d0 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:09.437859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6228b664-a456-4521-99ef-e201a4ed10d0 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:09.438939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6228b664-a456-4521-99ef-e201a4ed10d0 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:09.439549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6228b664-a456-4521-99ef-e201a4ed10d0 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:09.440123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6228b664-a456-4521-99ef-e201a4ed10d0 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:09.931033 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aa6481cb-b0bd-4bc6-a673-8742bcde6e81 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760/action" status: 202 len: 0 microversion: 2.25 time: 4.501189 Sep 27 10:01:09.932624 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 548/1222] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:01:05 2022] POST /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760/action => generated 0 bytes in 4509 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:01:09.969806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-65d801fa-8b63-47fd-ba59-781307773300 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:09.970820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-65d801fa-8b63-47fd-ba59-781307773300 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:09.973142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:10.013603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d801fa-8b63-47fd-ba59-781307773300 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:10.014742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d801fa-8b63-47fd-ba59-781307773300 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:10.016165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d801fa-8b63-47fd-ba59-781307773300 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:10.019570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d801fa-8b63-47fd-ba59-781307773300 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:10.021086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d801fa-8b63-47fd-ba59-781307773300 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:10.026328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d801fa-8b63-47fd-ba59-781307773300 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:10.211770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6228b664-a456-4521-99ef-e201a4ed10d0 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:10.230215 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6228b664-a456-4521-99ef-e201a4ed10d0 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1370 microversion: 2.1 time: 0.818427 Sep 27 10:01:10.256617 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 675/1223] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:09 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1370 bytes in 846 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:10.269288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-28c7b4a5-53ca-4bc5-aed4-8eb538cce223 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:10.277032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-28c7b4a5-53ca-4bc5-aed4-8eb538cce223 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:10.299490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=82,nova_api:INSERT=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:10.302296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28c7b4a5-53ca-4bc5-aed4-8eb538cce223 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:10.306552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28c7b4a5-53ca-4bc5-aed4-8eb538cce223 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:10.307942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28c7b4a5-53ca-4bc5-aed4-8eb538cce223 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:10.310297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28c7b4a5-53ca-4bc5-aed4-8eb538cce223 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:10.312810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28c7b4a5-53ca-4bc5-aed4-8eb538cce223 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:10.314226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28c7b4a5-53ca-4bc5-aed4-8eb538cce223 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:11.369117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-28c7b4a5-53ca-4bc5-aed4-8eb538cce223 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:11.383905 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-28c7b4a5-53ca-4bc5-aed4-8eb538cce223 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1310 microversion: 2.1 time: 1.125168 Sep 27 10:01:11.390479 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 676/1224] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:10 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1310 bytes in 1133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:11.395746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c9494f99-36d4-435c-b4d5-724e72673c92 req-88527ebd-28ec-43b1-9e50-05c1ecde2702 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0fa79578-f299-4e5d-8dea-8bd1ca1c31d1", "tag": "ffaeeb25-b086-4ac5-b059-d12a6497adf1"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:11.424242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9494f99-36d4-435c-b4d5-724e72673c92 req-88527ebd-28ec-43b1-9e50-05c1ecde2702 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:11.426744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9494f99-36d4-435c-b4d5-724e72673c92 req-88527ebd-28ec-43b1-9e50-05c1ecde2702 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:11.429504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9494f99-36d4-435c-b4d5-724e72673c92 req-88527ebd-28ec-43b1-9e50-05c1ecde2702 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:11.454878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-65d801fa-8b63-47fd-ba59-781307773300 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:11.464215 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-65d801fa-8b63-47fd-ba59-781307773300 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1542 microversion: 2.1 time: 1.530146 Sep 27 10:01:11.467004 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c9494f99-36d4-435c-b4d5-724e72673c92 req-88527ebd-28ec-43b1-9e50-05c1ecde2702 service nova] Creating event network-changed:ffaeeb25-b086-4ac5-b059-d12a6497adf1 for instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 on nch2-master-1 Sep 27 10:01:11.480983 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c9494f99-36d4-435c-b4d5-724e72673c92 req-88527ebd-28ec-43b1-9e50-05c1ecde2702 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.088888 Sep 27 10:01:11.481228 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 549/1225] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:09 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1542 bytes in 1548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:11.484035 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 677/1226] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:11.490875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-dc205612-ff63-48e9-bc97-c808b433d29b req-256c05fa-a360-4a06-9782-33c4c0ac606d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "69ea013c-5143-45ff-81a2-3ed97a8659cd", "tag": "96ac21a6-013a-4a5e-9320-82ff90906b76"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:11.496512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-77bdc833-3f69-439d-b749-dbd532859588 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:11.507238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-77bdc833-3f69-439d-b749-dbd532859588 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:11.508745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dc205612-ff63-48e9-bc97-c808b433d29b req-256c05fa-a360-4a06-9782-33c4c0ac606d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:11.509412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dc205612-ff63-48e9-bc97-c808b433d29b req-256c05fa-a360-4a06-9782-33c4c0ac606d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:11.510116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dc205612-ff63-48e9-bc97-c808b433d29b req-256c05fa-a360-4a06-9782-33c4c0ac606d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:11.523386 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-dc205612-ff63-48e9-bc97-c808b433d29b req-256c05fa-a360-4a06-9782-33c4c0ac606d service nova] Creating event network-changed:96ac21a6-013a-4a5e-9320-82ff90906b76 for instance 69ea013c-5143-45ff-81a2-3ed97a8659cd on nch2-master-1 Sep 27 10:01:11.525145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77bdc833-3f69-439d-b749-dbd532859588 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:11.526944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77bdc833-3f69-439d-b749-dbd532859588 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:11.528429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77bdc833-3f69-439d-b749-dbd532859588 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:11.530615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77bdc833-3f69-439d-b749-dbd532859588 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:11.533357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77bdc833-3f69-439d-b749-dbd532859588 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:11.534231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77bdc833-3f69-439d-b749-dbd532859588 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:11.541469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-dc205612-ff63-48e9-bc97-c808b433d29b req-256c05fa-a360-4a06-9782-33c4c0ac606d service nova] Instance 69ea013c-5143-45ff-81a2-3ed97a8659cd is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:01:11.553868 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-dc205612-ff63-48e9-bc97-c808b433d29b req-256c05fa-a360-4a06-9782-33c4c0ac606d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070565 Sep 27 10:01:11.556624 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 550/1227] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:11.562952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-29ea1730-c44f-4472-bf41-45c93cb3a53a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f63760160>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-564684654", "description": "tempest-description-431987254"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:11.812393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-77bdc833-3f69-439d-b749-dbd532859588 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:11.822770 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-77bdc833-3f69-439d-b749-dbd532859588 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1259 microversion: 2.1 time: 0.332849 Sep 27 10:01:11.824172 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 678/1228] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:11 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1259 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:11.832817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:11.837587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:11.857509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:11.858765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:11.859734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:11.861421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:11.862262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:11.864307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:12.071917 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-29ea1730-c44f-4472-bf41-45c93cb3a53a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 254 microversion: 2.1 time: 0.513103 Sep 27 10:01:12.073261 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 551/1229] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:01:11 2022] POST /compute/v2.1/os-security-groups => generated 254 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:12.089873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ff4a41bc-5ea6-4731-8448-db84fadf3d70 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:12.090945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ff4a41bc-5ea6-4731-8448-db84fadf3d70 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:12.101007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4a41bc-5ea6-4731-8448-db84fadf3d70 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:12.101950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4a41bc-5ea6-4731-8448-db84fadf3d70 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:12.103633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4a41bc-5ea6-4731-8448-db84fadf3d70 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:12.105716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4a41bc-5ea6-4731-8448-db84fadf3d70 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:12.130967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4a41bc-5ea6-4731-8448-db84fadf3d70 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:12.132394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4a41bc-5ea6-4731-8448-db84fadf3d70 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.026s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:12.388601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:12.404787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:12.425019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:12.426946 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b1584159-c735-48e7-9666-055b9e05bb61 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1567 microversion: 2.25 time: 0.600986 Sep 27 10:01:12.435811 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 679/1230] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:11 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1567 bytes in 611 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:12.452065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75bf9528-3586-4673-a28f-124db747152d tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f63b2d0d0>, body: {"security_group_rule": {"parent_group_id": "b7f3e77b-c03a-44d6-8ecb-a96b87bb28ea", "ip_protocol": "tempest-999-1096024184", "from_port": 22, "to_port": 22}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:13.003685 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-75bf9528-3586-4673-a28f-124db747152d tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] HTTP exception thrown: Invalid IP protocol tempest-999-1096024184. Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75bf9528-3586-4673-a28f-124db747152d tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Request method failure captured: Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 157 Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnZBt2Q4ufyfBqR8j38D6_unm922QvijuZubeV1Qk8gogf1P2WqTN3o2PejQ807D5p3FDkutceU80OokvIPrC4v4-5vXDNOoNSJqfq9Fu6FoR717_4uWMpOOjK2TPwFT6VqpfH_NzAIatJt9NFdrtHW9ec983tF0bwaHAoIP6ClDWA Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:01:13.005437 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5db36951642427481c9b79db0009118"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5db36951642427481c9b79db0009118"}]}] Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 546c5dc3d40c4e72ac683f084429b5f8 Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: {"security_group_rule": {"parent_group_id": "b7f3e77b-c03a-44d6-8ecb-a96b87bb28ea", "ip_protocol": "tempest-999-1096024184", "from_port": 22, "to_port": 22}} Sep 27 10:01:13.007028 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid IP protocol tempest-999-1096024184. Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'security_group_rule': {'parent_group_id': 'b7f3e77b-c03a-44d6-8ecb-a96b87bb28ea', 'ip_protocol': 'tempest-999-1096024184', 'from_port': 22, 'to_port': 22}}} Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 144, in _new_ingress_rule Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InvalidIpProtocol(protocol=ip_protocol) Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InvalidIpProtocol: Invalid IP protocol tempest-999-1096024184. Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:13.007876 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid IP protocol tempest-999-1096024184. Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid IP protocol tempest-999-1096024184. Sep 27 10:01:13.009194 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:13.010411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75bf9528-3586-4673-a28f-124db747152d tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Returning 400 to user: Invalid IP protocol tempest-999-1096024184. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:13.010411 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-75bf9528-3586-4673-a28f-124db747152d tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 87 microversion: 2.1 time: 0.569477 Sep 27 10:01:13.010411 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 680/1231] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:01:12 2022] POST /compute/v2.1/os-security-group-rules => generated 87 bytes in 571 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 10:01:13.020619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4dc50a4a-a9c9-43ec-9216-622ac1715554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:13.021334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4dc50a4a-a9c9-43ec-9216-622ac1715554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:13.044345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dc50a4a-a9c9-43ec-9216-622ac1715554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:13.044881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dc50a4a-a9c9-43ec-9216-622ac1715554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:13.045320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dc50a4a-a9c9-43ec-9216-622ac1715554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:13.053660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dc50a4a-a9c9-43ec-9216-622ac1715554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:13.054275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dc50a4a-a9c9-43ec-9216-622ac1715554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:13.054727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dc50a4a-a9c9-43ec-9216-622ac1715554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:13.093815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ff4a41bc-5ea6-4731-8448-db84fadf3d70 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:13.114694 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ff4a41bc-5ea6-4731-8448-db84fadf3d70 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1370 microversion: 2.1 time: 1.039794 Sep 27 10:01:13.121766 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 552/1232] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:12 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1370 bytes in 1048 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:13.127707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32216671-bb6f-4d52-8519-657b507846a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:13.130418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32216671-bb6f-4d52-8519-657b507846a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:13.145629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32216671-bb6f-4d52-8519-657b507846a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:13.146371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32216671-bb6f-4d52-8519-657b507846a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:13.147058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32216671-bb6f-4d52-8519-657b507846a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:13.148423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32216671-bb6f-4d52-8519-657b507846a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:13.149260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32216671-bb6f-4d52-8519-657b507846a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:13.155711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32216671-bb6f-4d52-8519-657b507846a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:13.722452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32216671-bb6f-4d52-8519-657b507846a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:13.740360 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32216671-bb6f-4d52-8519-657b507846a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1474 microversion: 2.1 time: 0.616815 Sep 27 10:01:13.741841 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 553/1233] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:13 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1474 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:13.747371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-24930bb0-807c-473e-aaf9-0b57243971ad tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:13.748429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-24930bb0-807c-473e-aaf9-0b57243971ad tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:13.757608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24930bb0-807c-473e-aaf9-0b57243971ad tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:13.758246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24930bb0-807c-473e-aaf9-0b57243971ad tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:13.758934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24930bb0-807c-473e-aaf9-0b57243971ad tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:13.759667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24930bb0-807c-473e-aaf9-0b57243971ad tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:13.760096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24930bb0-807c-473e-aaf9-0b57243971ad tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:13.760713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24930bb0-807c-473e-aaf9-0b57243971ad tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:13.883883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-24930bb0-807c-473e-aaf9-0b57243971ad tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:13.901435 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-24930bb0-807c-473e-aaf9-0b57243971ad tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1265 microversion: 2.1 time: 0.157741 Sep 27 10:01:13.904347 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 554/1234] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:13 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1265 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:13.915038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-265146f3-5598-45a7-aec3-0bcedf75a7cb tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f6366b0d0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-1841808452", "description": "tempest-description-798943271"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:14.005321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4dc50a4a-a9c9-43ec-9216-622ac1715554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:14.017394 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4dc50a4a-a9c9-43ec-9216-622ac1715554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1310 microversion: 2.1 time: 1.008651 Sep 27 10:01:14.019396 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 681/1235] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:13 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1310 bytes in 1011 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:14.024963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:14.025912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:14.036225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:14.037345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:14.038019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:14.039364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:14.040079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:14.040753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:14.440303 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-265146f3-5598-45a7-aec3-0bcedf75a7cb tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 255 microversion: 2.1 time: 0.534750 Sep 27 10:01:14.449539 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 555/1236] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:01:13 2022] POST /compute/v2.1/os-security-groups => generated 255 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:14.454694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-8a44864f-3c22-4abc-9ea4-567ce155e9a5 req-c58c7e76-5488-45e0-869a-772283caf95c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e8ca3e24-d297-4435-abe8-a70d128f4bb4", "tag": "769d6c7f-2742-4b0f-b84e-a0dcd3b8b472"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:14.479301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:14.488242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:14.489594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a44864f-3c22-4abc-9ea4-567ce155e9a5 req-c58c7e76-5488-45e0-869a-772283caf95c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:14.489752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:14.490506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a44864f-3c22-4abc-9ea4-567ce155e9a5 req-c58c7e76-5488-45e0-869a-772283caf95c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:14.490844 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-47d672d6-e73e-4c7c-aa6b-4d3f50aae3bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1567 microversion: 2.25 time: 0.470152 Sep 27 10:01:14.491281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a44864f-3c22-4abc-9ea4-567ce155e9a5 req-c58c7e76-5488-45e0-869a-772283caf95c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:14.500846 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 682/1237] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:14 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1567 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:14.527177 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8a44864f-3c22-4abc-9ea4-567ce155e9a5 req-c58c7e76-5488-45e0-869a-772283caf95c service nova] Creating event network-changed:769d6c7f-2742-4b0f-b84e-a0dcd3b8b472 for instance e8ca3e24-d297-4435-abe8-a70d128f4bb4 on nch1-master-1 Sep 27 10:01:14.532272 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-8a44864f-3c22-4abc-9ea4-567ce155e9a5 req-c58c7e76-5488-45e0-869a-772283caf95c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.081185 Sep 27 10:01:14.532909 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 556/1238] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:14.545190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b4f1110b-5bc4-458a-b2b2-4d77e4ecc986 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:14.545900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b4f1110b-5bc4-458a-b2b2-4d77e4ecc986 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:14.556450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f1110b-5bc4-458a-b2b2-4d77e4ecc986 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:14.557323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f1110b-5bc4-458a-b2b2-4d77e4ecc986 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:14.557889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f1110b-5bc4-458a-b2b2-4d77e4ecc986 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:14.569610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f1110b-5bc4-458a-b2b2-4d77e4ecc986 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:14.570358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f1110b-5bc4-458a-b2b2-4d77e4ecc986 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:14.570955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f1110b-5bc4-458a-b2b2-4d77e4ecc986 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:14.579595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7821ebdc-7050-48f3-b091-adeea88df186 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] Calling method '.version_select at 0x7f7f638f24c0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:14.743966 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7821ebdc-7050-48f3-b091-adeea88df186 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] 10.222.0.26 "GET /compute/v2.1/os-floating-ips" status: 200 len: 20 microversion: 2.1 time: 0.236435 Sep 27 10:01:14.755659 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 683/1239] 10.222.0.26 () {58 vars in 1149 bytes} [Tue Sep 27 10:01:14 2022] GET /compute/v2.1/os-floating-ips => generated 20 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:01:14.768285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-83d20747-15a8-41a8-8326-ee898d09194a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f63dac670>, body: {"security_group_rule": {"parent_group_id": "84099756-7273-454a-b0ae-ff29ceb9c41f", "ip_protocol": "tcp", "from_port": 22, "to_port": 21}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:14.967824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b4f1110b-5bc4-458a-b2b2-4d77e4ecc986 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:15.000034 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b4f1110b-5bc4-458a-b2b2-4d77e4ecc986 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1370 microversion: 2.1 time: 0.465824 Sep 27 10:01:15.007322 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 557/1240] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:14 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1370 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:15.018288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a2f115c7-c544-47cd-abb3-3442b9667d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:15.019155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a2f115c7-c544-47cd-abb3-3442b9667d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:15.028556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2f115c7-c544-47cd-abb3-3442b9667d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:15.029382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2f115c7-c544-47cd-abb3-3442b9667d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:15.030038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2f115c7-c544-47cd-abb3-3442b9667d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:15.030990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2f115c7-c544-47cd-abb3-3442b9667d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:15.031672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2f115c7-c544-47cd-abb3-3442b9667d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:15.032305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2f115c7-c544-47cd-abb3-3442b9667d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:15.298661 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-83d20747-15a8-41a8-8326-ee898d09194a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] HTTP exception thrown: Invalid port range 22:21. Former value cannot be greater than the later Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-83d20747-15a8-41a8-8326-ee898d09194a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Request method failure captured: Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 138 Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnZBt2Q4ufyfBqR8j38D6_unm922QvijuZubeV1Qk8gogf1P2WqTN3o2PejQ807D5p3FDkutceU80OokvIPrC4v4-5vXDNOoNSJqfq9Fu6FoR717_4uWMpOOjK2TPwFT6VqpfH_NzAIatJt9NFdrtHW9ec983tF0bwaHAoIP6ClDWA Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:01:15.305182 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5db36951642427481c9b79db0009118"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5db36951642427481c9b79db0009118"}]}] Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 546c5dc3d40c4e72ac683f084429b5f8 Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: {"security_group_rule": {"parent_group_id": "84099756-7273-454a-b0ae-ff29ceb9c41f", "ip_protocol": "tcp", "from_port": 22, "to_port": 21}} Sep 27 10:01:15.307175 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid port range 22:21. Former value cannot be greater than the later Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'security_group_rule': {'parent_group_id': '84099756-7273-454a-b0ae-ff29ceb9c41f', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 21}}} Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 150, in _new_ingress_rule Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InvalidPortRange(from_port=from_port, Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InvalidPortRange: Invalid port range 22:21. Former value cannot be greater than the later Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:15.308203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid port range 22:21. Former value cannot be greater than the later Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid port range 22:21. Former value cannot be greater than the later Sep 27 10:01:15.310801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:15.318656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-83d20747-15a8-41a8-8326-ee898d09194a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Returning 400 to user: Invalid port range 22:21. Former value cannot be greater than the later {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:15.318656 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-83d20747-15a8-41a8-8326-ee898d09194a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 115 microversion: 2.1 time: 0.561558 Sep 27 10:01:15.319970 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 684/1241] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:01:14 2022] POST /compute/v2.1/os-security-group-rules => generated 115 bytes in 564 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:01:15.332013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-070b4d8b-4f5c-45e0-b981-6566661cd655 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] Action: 'create', calling method: .version_select at 0x7f7f6392bee0>, body: {"pool": "non_exist_pool"} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:15.375021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a2f115c7-c544-47cd-abb3-3442b9667d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:15.385058 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a2f115c7-c544-47cd-abb3-3442b9667d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1474 microversion: 2.1 time: 0.376307 Sep 27 10:01:15.397266 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 558/1242] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:15 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1474 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:15.408566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-921c201f-2bc8-442e-896b-f84852872cb4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:15.409625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-921c201f-2bc8-442e-896b-f84852872cb4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:15.417204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-921c201f-2bc8-442e-896b-f84852872cb4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:15.418131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-921c201f-2bc8-442e-896b-f84852872cb4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:15.418899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-921c201f-2bc8-442e-896b-f84852872cb4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:15.419996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-921c201f-2bc8-442e-896b-f84852872cb4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:15.420802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-921c201f-2bc8-442e-896b-f84852872cb4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:15.421483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-921c201f-2bc8-442e-896b-f84852872cb4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:15.441676 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-070b4d8b-4f5c-45e0-b981-6566661cd655 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] HTTP exception thrown: Floating IP pool not found. Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-070b4d8b-4f5c-45e0-b981-6566661cd655 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] Request method failure captured: Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-floating-ips HTTP/1.1 Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 26 Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnp6gx1sL95fXDp5iC93dChUhgdKrDkRFABGkkywTAGs0s2o3xPY8XHFEeRJRZ6fXlK3RofjUB8WJ4aYD_hxcgzpSF-IAmN8-nQhoWSJnAJpSifCRnpbRj7T5vL8R5Cf6y3WzvNa_tWQMMUU96Zk32t2wVS_6kmDpFO-TD3CYwK9P4 Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 52e5e5d10e5a4da28c68aab50536bc08 Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FloatingIPsNegativeTestJSON-830291640 Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:01:15.443555 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/52e5e5d10e5a4da28c68aab50536bc08"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/52e5e5d10e5a4da28c68aab50536bc08"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/52e5e5d10e5a4da28c68aab50536bc08"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_52e5e5d10e5a4da28c68aab50536bc08"}]}] Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FloatingIPsNegativeTestJSON-830291640 Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 52e5e5d10e5a4da28c68aab50536bc08 Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FloatingIPsNegativeTestJSON-830291640 Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FloatingIPsNegativeTestJSON-830291640-project Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 424d1e6ad1ba435180f9de10d6beea14 Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FloatingIPsNegativeTestJSON-830291640-project Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: {"pool": "non_exist_pool"} Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: exception: Floating IP pool not found. Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'pool': 'non_exist_pool'}} Sep 27 10:01:15.445443 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 127, in create Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi address = self.network_api.allocate_floating_ip(context, pool) Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2972, in allocate_floating_ip Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi pool_id = self._get_floating_ip_pool_id_by_name_or_id(client, pool) Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2962, in _get_floating_ip_pool_id_by_name_or_id Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpPoolNotFound() Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpPoolNotFound: Floating IP pool not found. Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:15.446663 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 142, in create Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP pool not found. Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP pool not found. Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-070b4d8b-4f5c-45e0-b981-6566661cd655 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] Returning 404 to user: Floating IP pool not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:15.452819 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-070b4d8b-4f5c-45e0-b981-6566661cd655 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] 10.222.0.26 "POST /compute/v2.1/os-floating-ips" status: 404 len: 73 microversion: 2.1 time: 0.125159 Sep 27 10:01:15.453966 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 685/1243] 10.222.0.26 () {60 vars in 1170 bytes} [Tue Sep 27 10:01:15 2022] POST /compute/v2.1/os-floating-ips => generated 73 bytes in 127 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:01:15.456423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-192a1832-7f98-450a-bf77-a508a0b62284 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:15.462265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-192a1832-7f98-450a-bf77-a508a0b62284 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:15.477431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-192a1832-7f98-450a-bf77-a508a0b62284 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:15.477926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-192a1832-7f98-450a-bf77-a508a0b62284 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:15.478365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-192a1832-7f98-450a-bf77-a508a0b62284 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:15.479087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-192a1832-7f98-450a-bf77-a508a0b62284 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:15.479557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-192a1832-7f98-450a-bf77-a508a0b62284 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:15.479971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-192a1832-7f98-450a-bf77-a508a0b62284 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:15.923175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-192a1832-7f98-450a-bf77-a508a0b62284 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:15.938243 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-192a1832-7f98-450a-bf77-a508a0b62284 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1310 microversion: 2.1 time: 0.486038 Sep 27 10:01:15.963649 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 686/1244] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:15 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1310 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:15.979104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0d2ab8d2-7fff-4565-8fdc-781c717e3e76 req-17339a76-1e45-4f32-ab27-273b76509662 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e8ca3e24-d297-4435-abe8-a70d128f4bb4", "name": "network-vif-plugged", "status": "completed", "tag": "769d6c7f-2742-4b0f-b84e-a0dcd3b8b472"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:16.025680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0d2ab8d2-7fff-4565-8fdc-781c717e3e76 req-17339a76-1e45-4f32-ab27-273b76509662 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:16.026455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0d2ab8d2-7fff-4565-8fdc-781c717e3e76 req-17339a76-1e45-4f32-ab27-273b76509662 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:16.027133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0d2ab8d2-7fff-4565-8fdc-781c717e3e76 req-17339a76-1e45-4f32-ab27-273b76509662 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:16.044886 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0d2ab8d2-7fff-4565-8fdc-781c717e3e76 req-17339a76-1e45-4f32-ab27-273b76509662 service nova] Creating event network-vif-plugged:769d6c7f-2742-4b0f-b84e-a0dcd3b8b472 for instance e8ca3e24-d297-4435-abe8-a70d128f4bb4 on nch1-master-1 Sep 27 10:01:16.054481 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0d2ab8d2-7fff-4565-8fdc-781c717e3e76 req-17339a76-1e45-4f32-ab27-273b76509662 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085899 Sep 27 10:01:16.064433 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 687/1245] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:16.070067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-350f7286-626b-4b86-b807-ef9ea4b20109 req-89f8c5c8-cbf2-41b5-9539-b7c7ad3f2275 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9edcd7b8-f01b-4529-b9c1-3729e79ec760", "name": "network-vif-unplugged", "status": "completed", "tag": "5a182f65-e926-4e9f-97dc-7f90cd3a6dc9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:16.083226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-921c201f-2bc8-442e-896b-f84852872cb4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:16.089956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-350f7286-626b-4b86-b807-ef9ea4b20109 req-89f8c5c8-cbf2-41b5-9539-b7c7ad3f2275 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:16.098967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-350f7286-626b-4b86-b807-ef9ea4b20109 req-89f8c5c8-cbf2-41b5-9539-b7c7ad3f2275 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:16.099812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-350f7286-626b-4b86-b807-ef9ea4b20109 req-89f8c5c8-cbf2-41b5-9539-b7c7ad3f2275 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:16.112497 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-921c201f-2bc8-442e-896b-f84852872cb4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1307 microversion: 2.1 time: 0.707684 Sep 27 10:01:16.113205 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 559/1246] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:15 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1307 bytes in 715 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:16.126803 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-350f7286-626b-4b86-b807-ef9ea4b20109 req-89f8c5c8-cbf2-41b5-9539-b7c7ad3f2275 service nova] Creating event network-vif-unplugged:5a182f65-e926-4e9f-97dc-7f90cd3a6dc9 for instance 9edcd7b8-f01b-4529-b9c1-3729e79ec760 on nch2-master-1 Sep 27 10:01:16.131062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e0b70153-0a9c-4de8-8474-f33bad6e8c6b tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f636414c0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesNegativeTestJSON-securitygroup-1613296419", "description": "tempest-description-1811079184"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:16.141484 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-350f7286-626b-4b86-b807-ef9ea4b20109 req-89f8c5c8-cbf2-41b5-9539-b7c7ad3f2275 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.073074 Sep 27 10:01:16.143691 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 688/1247] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:16 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:16.162518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c0665bd8-4878-44b5-8015-77df5e0ae6b2 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] Calling method '.version_select at 0x7f7f6392b280>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:16.477821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-c0665bd8-4878-44b5-8015-77df5e0ae6b2 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP acbc5461-c8c8-45c3-a4c2-28b726b2af78 could not be found", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:01:16.478586 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-c0665bd8-4878-44b5-8015-77df5e0ae6b2 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] HTTP exception thrown: Floating IP not found for ID acbc5461-c8c8-45c3-a4c2-28b726b2af78 Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c0665bd8-4878-44b5-8015-77df5e0ae6b2 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] Request method failure captured: Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-floating-ips/acbc5461-c8c8-45c3-a4c2-28b726b2af78 HTTP/1.1 Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnp6gx1sL95fXDp5iC93dChUhgdKrDkRFABGkkywTAGs0s2o3xPY8XHFEeRJRZ6fXlK3RofjUB8WJ4aYD_hxcgzpSF-IAmN8-nQhoWSJnAJpSifCRnpbRj7T5vL8R5Cf6y3WzvNa_tWQMMUU96Zk32t2wVS_6kmDpFO-TD3CYwK9P4 Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 52e5e5d10e5a4da28c68aab50536bc08 Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FloatingIPsNegativeTestJSON-830291640 Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:01:16.480782 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/52e5e5d10e5a4da28c68aab50536bc08"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/52e5e5d10e5a4da28c68aab50536bc08"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/52e5e5d10e5a4da28c68aab50536bc08"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_52e5e5d10e5a4da28c68aab50536bc08"}]}] Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FloatingIPsNegativeTestJSON-830291640 Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 52e5e5d10e5a4da28c68aab50536bc08 Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FloatingIPsNegativeTestJSON-830291640 Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FloatingIPsNegativeTestJSON-830291640-project Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 424d1e6ad1ba435180f9de10d6beea14 Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FloatingIPsNegativeTestJSON-830291640-project Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: exception: Floating IP not found for ID acbc5461-c8c8-45c3-a4c2-28b726b2af78 Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'acbc5461-c8c8-45c3-a4c2-28b726b2af78'} Sep 27 10:01:16.482390 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2824, in get_floating_ip Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1008, in show_floatingip Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:01:16.483396 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP acbc5461-c8c8-45c3-a4c2-28b726b2af78 could not be found Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d94c476f-e673-4368-b808-7b9aac398549'] Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 158, in delete Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID acbc5461-c8c8-45c3-a4c2-28b726b2af78. Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:16.484691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 161, in delete Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID acbc5461-c8c8-45c3-a4c2-28b726b2af78 Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:16.485928 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:16.487258 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID acbc5461-c8c8-45c3-a4c2-28b726b2af78 Sep 27 10:01:16.487258 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:16.487258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c0665bd8-4878-44b5-8015-77df5e0ae6b2 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] Returning 404 to user: Floating IP not found for ID acbc5461-c8c8-45c3-a4c2-28b726b2af78 {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:16.487258 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c0665bd8-4878-44b5-8015-77df5e0ae6b2 tempest-FloatingIPsNegativeTestJSON-830291640 tempest-FloatingIPsNegativeTestJSON-830291640-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/acbc5461-c8c8-45c3-a4c2-28b726b2af78" status: 404 len: 111 microversion: 2.1 time: 0.337235 Sep 27 10:01:16.487258 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 689/1248] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:01:16 2022] DELETE /compute/v2.1/os-floating-ips/acbc5461-c8c8-45c3-a4c2-28b726b2af78 => generated 111 bytes in 338 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:01:16.489993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:16.502703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:16.516024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:16.516515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:16.517060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:16.525959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:16.526846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:16.528355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:16.980908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:16.994201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:16.996355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:16.998177 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-677f24bd-e93f-462c-b512-51c4935b32a3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1567 microversion: 2.25 time: 0.514450 Sep 27 10:01:16.999042 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 690/1249] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:16 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1567 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:17.008244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30d7e6d2-d9af-4e7e-aca4-6317076f2a39 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:17.013521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-30d7e6d2-d9af-4e7e-aca4-6317076f2a39 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:17.020246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d7e6d2-d9af-4e7e-aca4-6317076f2a39 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.020764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d7e6d2-d9af-4e7e-aca4-6317076f2a39 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.021374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d7e6d2-d9af-4e7e-aca4-6317076f2a39 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.022277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d7e6d2-d9af-4e7e-aca4-6317076f2a39 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.022765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d7e6d2-d9af-4e7e-aca4-6317076f2a39 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.023387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d7e6d2-d9af-4e7e-aca4-6317076f2a39 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.219592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-30d7e6d2-d9af-4e7e-aca4-6317076f2a39 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:17.233552 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-30d7e6d2-d9af-4e7e-aca4-6317076f2a39 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1370 microversion: 2.1 time: 0.229090 Sep 27 10:01:17.234526 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 691/1250] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:17 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1370 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:17.239978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cfb5fde3-cbc9-4892-9f75-f3f85726ed0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:17.240882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cfb5fde3-cbc9-4892-9f75-f3f85726ed0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:17.247620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfb5fde3-cbc9-4892-9f75-f3f85726ed0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.248375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfb5fde3-cbc9-4892-9f75-f3f85726ed0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.249023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfb5fde3-cbc9-4892-9f75-f3f85726ed0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.249909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfb5fde3-cbc9-4892-9f75-f3f85726ed0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.250523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfb5fde3-cbc9-4892-9f75-f3f85726ed0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.251105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfb5fde3-cbc9-4892-9f75-f3f85726ed0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.410067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cfb5fde3-cbc9-4892-9f75-f3f85726ed0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:17.420475 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cfb5fde3-cbc9-4892-9f75-f3f85726ed0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1471 microversion: 2.1 time: 0.184686 Sep 27 10:01:17.421494 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 692/1251] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:17 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1471 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:17.427408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b381f758-8d58-4322-a9ae-49212e8cf250 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:17.428355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b381f758-8d58-4322-a9ae-49212e8cf250 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:17.435901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b381f758-8d58-4322-a9ae-49212e8cf250 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.436618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b381f758-8d58-4322-a9ae-49212e8cf250 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.437380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b381f758-8d58-4322-a9ae-49212e8cf250 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.438301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b381f758-8d58-4322-a9ae-49212e8cf250 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.439072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b381f758-8d58-4322-a9ae-49212e8cf250 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.439658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b381f758-8d58-4322-a9ae-49212e8cf250 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.617934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b381f758-8d58-4322-a9ae-49212e8cf250 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:17.627295 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e0b70153-0a9c-4de8-8474-f33bad6e8c6b tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 256 microversion: 2.1 time: 1.512962 Sep 27 10:01:17.628242 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b381f758-8d58-4322-a9ae-49212e8cf250 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1310 microversion: 2.1 time: 0.204551 Sep 27 10:01:17.643954 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 560/1252] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:01:16 2022] POST /compute/v2.1/os-security-groups => generated 256 bytes in 1516 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:17.643954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3e9a0817-fb03-408c-987e-2d88f3de34c2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:17.643954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e9a0817-fb03-408c-987e-2d88f3de34c2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:17.645313 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 693/1253] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:17 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1310 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:17.647395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a0817-fb03-408c-987e-2d88f3de34c2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.648590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a0817-fb03-408c-987e-2d88f3de34c2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.654561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:17.656374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:17.658951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a0817-fb03-408c-987e-2d88f3de34c2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.659855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a0817-fb03-408c-987e-2d88f3de34c2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.660359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a0817-fb03-408c-987e-2d88f3de34c2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.661824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a0817-fb03-408c-987e-2d88f3de34c2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.664805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.666026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.666785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.667670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.668290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.669460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.759902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.760393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.762961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.764030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:17.764793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:17.765538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:17.824634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for project e79eae6310354ca58046e59dfac23009. Resources: {'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:01:17.832548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for user 9544cdb0392f416197a586d99e8a3210 and project e79eae6310354ca58046e59dfac23009. Resources: {'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:01:17.956605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:17.973937 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-62b09e61-6455-42d8-a54f-74512f76478e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "POST /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd/action" status: 202 len: 0 microversion: 2.1 time: 0.326883 Sep 27 10:01:17.974907 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 694/1254] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:01:17 2022] POST /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd/action => generated 0 bytes in 329 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:01:17.982087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-164eac94-d6a8-40c7-acf8-b023cd2351d3 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f63b2d040>, body: {"security_group_rule": {"parent_group_id": "3496c8f9-d667-408e-bcbd-287c1895b4a7", "ip_protocol": "tcp", "from_port": 22, "to_port": 1404143249}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:18.076944 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-164eac94-d6a8-40c7-acf8-b023cd2351d3 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] HTTP exception thrown: Invalid port range 22:1404143249. Valid TCP ports should be between 1-65535 Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-164eac94-d6a8-40c7-acf8-b023cd2351d3 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Request method failure captured: Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 146 Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnZBt2Q4ufyfBqR8j38D6_unm922QvijuZubeV1Qk8gogf1P2WqTN3o2PejQ807D5p3FDkutceU80OokvIPrC4v4-5vXDNOoNSJqfq9Fu6FoR717_4uWMpOOjK2TPwFT6VqpfH_NzAIatJt9NFdrtHW9ec983tF0bwaHAoIP6ClDWA Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:01:18.078695 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5db36951642427481c9b79db0009118"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5db36951642427481c9b79db0009118"}]}] Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 546c5dc3d40c4e72ac683f084429b5f8 Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: {"security_group_rule": {"parent_group_id": "3496c8f9-d667-408e-bcbd-287c1895b4a7", "ip_protocol": "tcp", "from_port": 22, "to_port": 1404143249}} Sep 27 10:01:18.080082 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid port range 22:1404143249. Valid TCP ports should be between 1-65535 Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'security_group_rule': {'parent_group_id': '3496c8f9-d667-408e-bcbd-287c1895b4a7', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 1404143249}}} Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 294, in create Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi new_rule = self._rule_args_to_dict(context, Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 346, in _rule_args_to_dict Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return security_group_api.new_cidr_ingress_rule( Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 103, in new_cidr_ingress_rule Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return _new_ingress_rule( Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 157, in _new_ingress_rule Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InvalidPortRange(from_port=from_port, Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InvalidPortRange: Invalid port range 22:1404143249. Valid TCP ports should be between 1-65535 Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:18.080867 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 301, in create Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid port range 22:1404143249. Valid TCP ports should be between 1-65535 Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid port range 22:1404143249. Valid TCP ports should be between 1-65535 Sep 27 10:01:18.082099 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.083331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-164eac94-d6a8-40c7-acf8-b023cd2351d3 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Returning 400 to user: Invalid port range 22:1404143249. Valid TCP ports should be between 1-65535 {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:18.083331 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-164eac94-d6a8-40c7-acf8-b023cd2351d3 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 400 len: 119 microversion: 2.1 time: 0.103170 Sep 27 10:01:18.083331 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 695/1255] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:01:17 2022] POST /compute/v2.1/os-security-group-rules => generated 119 bytes in 105 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:01:18.089789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6c48c39c-b8cf-4b2e-b038-5f4ac388cd3b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:18.090843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6c48c39c-b8cf-4b2e-b038-5f4ac388cd3b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:18.109394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c48c39c-b8cf-4b2e-b038-5f4ac388cd3b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:18.110897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c48c39c-b8cf-4b2e-b038-5f4ac388cd3b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:18.111384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c48c39c-b8cf-4b2e-b038-5f4ac388cd3b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:18.112617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c48c39c-b8cf-4b2e-b038-5f4ac388cd3b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:18.113120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c48c39c-b8cf-4b2e-b038-5f4ac388cd3b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:18.115391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c48c39c-b8cf-4b2e-b038-5f4ac388cd3b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:18.247353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3e9a0817-fb03-408c-987e-2d88f3de34c2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:18.260132 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3e9a0817-fb03-408c-987e-2d88f3de34c2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1307 microversion: 2.1 time: 0.629175 Sep 27 10:01:18.261550 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 561/1256] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:17 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1307 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:18.266721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:18.267674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:18.275119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:18.276087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:18.276797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:18.277775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:18.278799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:18.279458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:18.329994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6c48c39c-b8cf-4b2e-b038-5f4ac388cd3b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:18.339518 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6c48c39c-b8cf-4b2e-b038-5f4ac388cd3b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1470 microversion: 2.1 time: 0.257593 Sep 27 10:01:18.340720 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 696/1257] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:18 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1470 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:18.346319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed24684a-fb8e-48da-87dd-4336f115b99a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Action: 'create', calling method: .version_select at 0x7f7f63a82ca0>, body: {"security_group_rule": {"parent_group_id": "ac982a20-9863-4cd6-b71c-a701fc2622dc", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:18.389865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-ed24684a-fb8e-48da-87dd-4336f115b99a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group ac982a20-9863-4cd6-b71c-a701fc2622dc does not exist", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:01:18.390584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.security_group_api [None req-ed24684a-fb8e-48da-87dd-4336f115b99a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Neutron security group ac982a20-9863-4cd6-b71c-a701fc2622dc not found {{(pid=145968) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:01:18.391202 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-ed24684a-fb8e-48da-87dd-4336f115b99a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] HTTP exception thrown: Security group ac982a20-9863-4cd6-b71c-a701fc2622dc does not exist Sep 27 10:01:18.391202 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-c33088d1-ad3d-4115-911f-71c4410c07d6'] Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed24684a-fb8e-48da-87dd-4336f115b99a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Request method failure captured: Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-security-group-rules HTTP/1.1 Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 138 Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnZBt2Q4ufyfBqR8j38D6_unm922QvijuZubeV1Qk8gogf1P2WqTN3o2PejQ807D5p3FDkutceU80OokvIPrC4v4-5vXDNOoNSJqfq9Fu6FoR717_4uWMpOOjK2TPwFT6VqpfH_NzAIatJt9NFdrtHW9ec983tF0bwaHAoIP6ClDWA Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:01:18.393292 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5db36951642427481c9b79db0009118"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5db36951642427481c9b79db0009118"}]}] Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 546c5dc3d40c4e72ac683f084429b5f8 Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: {"security_group_rule": {"parent_group_id": "ac982a20-9863-4cd6-b71c-a701fc2622dc", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} Sep 27 10:01:18.394952 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: exception: Security group ac982a20-9863-4cd6-b71c-a701fc2622dc does not exist Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-c33088d1-ad3d-4115-911f-71c4410c07d6'] Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'security_group_rule': {'parent_group_id': 'ac982a20-9863-4cd6-b71c-a701fc2622dc', 'ip_protocol': 'tcp', 'from_port': 22, 'to_port': 22}}} Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:01:18.395799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group ac982a20-9863-4cd6-b71c-a701fc2622dc does not exist Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c33088d1-ad3d-4115-911f-71c4410c07d6'] Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 287, in create Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get( Sep 27 10:01:18.397221 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group ac982a20-9863-4cd6-b71c-a701fc2622dc does not exist Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c33088d1-ad3d-4115-911f-71c4410c07d6'] Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 303, in create Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group ac982a20-9863-4cd6-b71c-a701fc2622dc does not exist Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c33088d1-ad3d-4115-911f-71c4410c07d6'] Sep 27 10:01:18.398681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group ac982a20-9863-4cd6-b71c-a701fc2622dc does not exist Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c33088d1-ad3d-4115-911f-71c4410c07d6'] Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed24684a-fb8e-48da-87dd-4336f115b99a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Returning 404 to user: Security group ac982a20-9863-4cd6-b71c-a701fc2622dc does not exist Sep 27 10:01:18.400327 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-c33088d1-ad3d-4115-911f-71c4410c07d6'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:18.402386 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed24684a-fb8e-48da-87dd-4336f115b99a tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 404 len: 194 microversion: 2.1 time: 0.057844 Sep 27 10:01:18.402562 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 697/1258] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:01:18 2022] POST /compute/v2.1/os-security-group-rules => generated 194 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:01:18.413046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-92e4e3d4-e9cf-411e-bdad-b1fa2c60148e req-9268f1ef-dab1-4642-bb98-21132d4547b1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9edcd7b8-f01b-4529-b9c1-3729e79ec760", "tag": "5a182f65-e926-4e9f-97dc-7f90cd3a6dc9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:18.433991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-92e4e3d4-e9cf-411e-bdad-b1fa2c60148e req-9268f1ef-dab1-4642-bb98-21132d4547b1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:18.434764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-92e4e3d4-e9cf-411e-bdad-b1fa2c60148e req-9268f1ef-dab1-4642-bb98-21132d4547b1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:18.435372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-92e4e3d4-e9cf-411e-bdad-b1fa2c60148e req-9268f1ef-dab1-4642-bb98-21132d4547b1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:18.449104 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-92e4e3d4-e9cf-411e-bdad-b1fa2c60148e req-9268f1ef-dab1-4642-bb98-21132d4547b1 service nova] Creating event network-changed:5a182f65-e926-4e9f-97dc-7f90cd3a6dc9 for instance 9edcd7b8-f01b-4529-b9c1-3729e79ec760 on nch2-master-1 Sep 27 10:01:18.454569 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-92e4e3d4-e9cf-411e-bdad-b1fa2c60148e req-9268f1ef-dab1-4642-bb98-21132d4547b1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050618 Sep 27 10:01:18.455175 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 698/1259] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:18.460874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5a4c0a4d-c7b0-44ca-b1dc-59cfb3a349c9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:18.461779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5a4c0a4d-c7b0-44ca-b1dc-59cfb3a349c9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:18.468685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a4c0a4d-c7b0-44ca-b1dc-59cfb3a349c9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:18.469101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a4c0a4d-c7b0-44ca-b1dc-59cfb3a349c9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:18.469541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a4c0a4d-c7b0-44ca-b1dc-59cfb3a349c9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:18.470244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a4c0a4d-c7b0-44ca-b1dc-59cfb3a349c9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:18.470696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a4c0a4d-c7b0-44ca-b1dc-59cfb3a349c9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:18.471118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a4c0a4d-c7b0-44ca-b1dc-59cfb3a349c9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:18.620711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5a4c0a4d-c7b0-44ca-b1dc-59cfb3a349c9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:18.629766 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5a4c0a4d-c7b0-44ca-b1dc-59cfb3a349c9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1370 microversion: 2.1 time: 0.173162 Sep 27 10:01:18.646243 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 699/1260] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:18 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1370 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:18.651575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7005a062-3a04-4a58-9b80-686f512a5af4 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Calling method '.version_select at 0x7f7f638f2d30>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:18.677051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-7005a062-3a04-4a58-9b80-686f512a5af4 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Error message: {"NeutronError": {"type": "SecurityGroupRuleNotFound", "message": "Security group rule 087a2559-3dec-44f9-abfc-05afbd32314f does not exist", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:01:18.677663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.security_group_api [None req-7005a062-3a04-4a58-9b80-686f512a5af4 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Neutron security group rule 087a2559-3dec-44f9-abfc-05afbd32314f not found {{(pid=145968) get_rule /opt/stack/nova/nova/network/security_group_api.py:463}} Sep 27 10:01:18.678199 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-7005a062-3a04-4a58-9b80-686f512a5af4 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] HTTP exception thrown: Security group rule 087a2559-3dec-44f9-abfc-05afbd32314f does not exist Sep 27 10:01:18.678199 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-2e7758cb-fe7c-4c0c-8307-42c561c54a2e'] Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7005a062-3a04-4a58-9b80-686f512a5af4 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Request method failure captured: Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-security-group-rules/087a2559-3dec-44f9-abfc-05afbd32314f HTTP/1.1 Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnZBt2Q4ufyfBqR8j38D6_unm922QvijuZubeV1Qk8gogf1P2WqTN3o2PejQ807D5p3FDkutceU80OokvIPrC4v4-5vXDNOoNSJqfq9Fu6FoR717_4uWMpOOjK2TPwFT6VqpfH_NzAIatJt9NFdrtHW9ec983tF0bwaHAoIP6ClDWA Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:01:18.680278 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5db36951642427481c9b79db0009118"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5db36951642427481c9b79db0009118"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5db36951642427481c9b79db0009118"}]}] Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5db36951642427481c9b79db0009118 Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021 Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 546c5dc3d40c4e72ac683f084429b5f8 Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupRulesNegativeTestJSON-636602021-project Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: exception: Security group rule 087a2559-3dec-44f9-abfc-05afbd32314f does not exist Sep 27 10:01:18.682345 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-2e7758cb-fe7c-4c0c-8307-42c561c54a2e'] Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '087a2559-3dec-44f9-abfc-05afbd32314f'} Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 459, in get_rule Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi rule = neutron.show_security_group_rule( Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1092, in show_security_group_rule Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_rule_path % (security_group_rule), Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:18.683280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group rule 087a2559-3dec-44f9-abfc-05afbd32314f does not exist Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-2e7758cb-fe7c-4c0c-8307-42c561c54a2e'] Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 359, in delete Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi rule = security_group_api.get_rule(context, id) Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 464, in get_rule Sep 27 10:01:18.684851 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group rule 087a2559-3dec-44f9-abfc-05afbd32314f does not exist Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-2e7758cb-fe7c-4c0c-8307-42c561c54a2e'] Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 365, in delete Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group rule 087a2559-3dec-44f9-abfc-05afbd32314f does not exist Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-2e7758cb-fe7c-4c0c-8307-42c561c54a2e'] Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.686303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group rule 087a2559-3dec-44f9-abfc-05afbd32314f does not exist Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-2e7758cb-fe7c-4c0c-8307-42c561c54a2e'] Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7005a062-3a04-4a58-9b80-686f512a5af4 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Returning 404 to user: Security group rule 087a2559-3dec-44f9-abfc-05afbd32314f does not exist Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-2e7758cb-fe7c-4c0c-8307-42c561c54a2e'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7005a062-3a04-4a58-9b80-686f512a5af4 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-group-rules/087a2559-3dec-44f9-abfc-05afbd32314f" status: 404 len: 199 microversion: 2.1 time: 0.034538 Sep 27 10:01:18.687858 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 700/1261] 10.222.0.26 () {58 vars in 1287 bytes} [Tue Sep 27 10:01:18 2022] DELETE /compute/v2.1/os-security-group-rules/087a2559-3dec-44f9-abfc-05afbd32314f => generated 199 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:01:18.689565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef37b96e-1396-45e2-9642-5d865861a9c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:18.690186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ef37b96e-1396-45e2-9642-5d865861a9c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:18.697818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef37b96e-1396-45e2-9642-5d865861a9c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:18.698318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef37b96e-1396-45e2-9642-5d865861a9c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:18.698758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef37b96e-1396-45e2-9642-5d865861a9c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:18.699488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef37b96e-1396-45e2-9642-5d865861a9c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:18.699953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef37b96e-1396-45e2-9642-5d865861a9c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:18.700377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef37b96e-1396-45e2-9642-5d865861a9c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:18.798820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:18.822785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:18.832316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:18.836121 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3c6c7003-20da-4a9d-982b-ebc47c673ed6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1567 microversion: 2.25 time: 0.573096 Sep 27 10:01:18.837330 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 562/1262] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:18 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1567 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:18.849981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd67919e-91e1-48ce-8c2d-a904fa94e4d5 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Calling method '.version_select at 0x7f7f63671670>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:18.932605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ef37b96e-1396-45e2-9642-5d865861a9c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:18.940814 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef37b96e-1396-45e2-9642-5d865861a9c6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1494 microversion: 2.1 time: 0.256661 Sep 27 10:01:18.946801 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 701/1263] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:18 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1494 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:18.971284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a281be73-e87d-4317-b54e-8680cdde69e8 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:18.972181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a281be73-e87d-4317-b54e-8680cdde69e8 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:18.979025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a281be73-e87d-4317-b54e-8680cdde69e8 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:18.979760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a281be73-e87d-4317-b54e-8680cdde69e8 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:18.980363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a281be73-e87d-4317-b54e-8680cdde69e8 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:18.981520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a281be73-e87d-4317-b54e-8680cdde69e8 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:18.982126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a281be73-e87d-4317-b54e-8680cdde69e8 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:18.982810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a281be73-e87d-4317-b54e-8680cdde69e8 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:19.132833 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a281be73-e87d-4317-b54e-8680cdde69e8 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1645 microversion: 2.1 time: 0.166586 Sep 27 10:01:19.133429 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 702/1264] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:18 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1645 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:01:19.166965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fbf62f37-6f9a-4276-b2f5-4d5d5d578e1d tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:19.167751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fbf62f37-6f9a-4276-b2f5-4d5d5d578e1d tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:19.176577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf62f37-6f9a-4276-b2f5-4d5d5d578e1d tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:19.177185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf62f37-6f9a-4276-b2f5-4d5d5d578e1d tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:19.177718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf62f37-6f9a-4276-b2f5-4d5d5d578e1d tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:19.178527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf62f37-6f9a-4276-b2f5-4d5d5d578e1d tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:19.179121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf62f37-6f9a-4276-b2f5-4d5d5d578e1d tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:19.179812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf62f37-6f9a-4276-b2f5-4d5d5d578e1d tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:19.221404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:19.394898 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fbf62f37-6f9a-4276-b2f5-4d5d5d578e1d tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1645 microversion: 2.1 time: 0.236578 Sep 27 10:01:19.396613 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 703/1265] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:19 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1645 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:19.404174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fb6115c6-ea77-475c-9586-7f5c81121339 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:19.405026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fb6115c6-ea77-475c-9586-7f5c81121339 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:19.413057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb6115c6-ea77-475c-9586-7f5c81121339 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:19.414000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb6115c6-ea77-475c-9586-7f5c81121339 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:19.414632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb6115c6-ea77-475c-9586-7f5c81121339 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:19.416972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb6115c6-ea77-475c-9586-7f5c81121339 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:19.417635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb6115c6-ea77-475c-9586-7f5c81121339 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:19.418419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb6115c6-ea77-475c-9586-7f5c81121339 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:19.668865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fb6115c6-ea77-475c-9586-7f5c81121339 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:19.683230 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fb6115c6-ea77-475c-9586-7f5c81121339 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1307 microversion: 2.1 time: 0.285139 Sep 27 10:01:19.684855 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 704/1266] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:19 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1307 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:19.693172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7522354e-edfc-484f-8e4a-e0bc7cbfc628 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:19.694305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7522354e-edfc-484f-8e4a-e0bc7cbfc628 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:19.705273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7522354e-edfc-484f-8e4a-e0bc7cbfc628 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:19.705773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7522354e-edfc-484f-8e4a-e0bc7cbfc628 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:19.706221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7522354e-edfc-484f-8e4a-e0bc7cbfc628 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:19.706944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7522354e-edfc-484f-8e4a-e0bc7cbfc628 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:19.707406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7522354e-edfc-484f-8e4a-e0bc7cbfc628 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:19.707861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7522354e-edfc-484f-8e4a-e0bc7cbfc628 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:19.870163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7522354e-edfc-484f-8e4a-e0bc7cbfc628 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:19.878914 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7522354e-edfc-484f-8e4a-e0bc7cbfc628 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1470 microversion: 2.1 time: 0.192027 Sep 27 10:01:19.893882 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 705/1267] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:19 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1470 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:19.901771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-afc579dc-0aba-4c15-9c82-401f16395aa5 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:19.905977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afc579dc-0aba-4c15-9c82-401f16395aa5 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:19.906542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afc579dc-0aba-4c15-9c82-401f16395aa5 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:19.907055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afc579dc-0aba-4c15-9c82-401f16395aa5 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:19.907792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afc579dc-0aba-4c15-9c82-401f16395aa5 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:19.908434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afc579dc-0aba-4c15-9c82-401f16395aa5 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:19.908890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afc579dc-0aba-4c15-9c82-401f16395aa5 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:19.934937 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-afc579dc-0aba-4c15-9c82-401f16395aa5 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.039503 Sep 27 10:01:19.935561 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 706/1268] 10.222.0.26 () {58 vars in 1176 bytes} [Tue Sep 27 10:01:19 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:19.940212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5316a7d9-8ef7-488f-bc13-72c33dbc6fa3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:19.940853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5316a7d9-8ef7-488f-bc13-72c33dbc6fa3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:19.948023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5316a7d9-8ef7-488f-bc13-72c33dbc6fa3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:19.948755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5316a7d9-8ef7-488f-bc13-72c33dbc6fa3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:19.949188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5316a7d9-8ef7-488f-bc13-72c33dbc6fa3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:19.949898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5316a7d9-8ef7-488f-bc13-72c33dbc6fa3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:19.950351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5316a7d9-8ef7-488f-bc13-72c33dbc6fa3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:19.950772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5316a7d9-8ef7-488f-bc13-72c33dbc6fa3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.088820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5316a7d9-8ef7-488f-bc13-72c33dbc6fa3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:20.097520 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5316a7d9-8ef7-488f-bc13-72c33dbc6fa3 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1370 microversion: 2.1 time: 0.160742 Sep 27 10:01:20.098122 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 707/1269] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:19 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1370 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:20.102871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a45bf708-54f1-4b70-82e5-363ec8c6351e req-74fb3c1a-75e2-488e-b7c8-9444c809d845 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69ea013c-5143-45ff-81a2-3ed97a8659cd", "name": "network-vif-plugged", "status": "completed", "tag": "96ac21a6-013a-4a5e-9320-82ff90906b76"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:20.115346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a45bf708-54f1-4b70-82e5-363ec8c6351e req-74fb3c1a-75e2-488e-b7c8-9444c809d845 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.115997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a45bf708-54f1-4b70-82e5-363ec8c6351e req-74fb3c1a-75e2-488e-b7c8-9444c809d845 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.116431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a45bf708-54f1-4b70-82e5-363ec8c6351e req-74fb3c1a-75e2-488e-b7c8-9444c809d845 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.129794 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a45bf708-54f1-4b70-82e5-363ec8c6351e req-74fb3c1a-75e2-488e-b7c8-9444c809d845 service nova] Creating event network-vif-plugged:96ac21a6-013a-4a5e-9320-82ff90906b76 for instance 69ea013c-5143-45ff-81a2-3ed97a8659cd on nch2-master-1 Sep 27 10:01:20.136373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-a45bf708-54f1-4b70-82e5-363ec8c6351e req-74fb3c1a-75e2-488e-b7c8-9444c809d845 service nova] Instance 69ea013c-5143-45ff-81a2-3ed97a8659cd is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:01:20.147697 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a45bf708-54f1-4b70-82e5-363ec8c6351e req-74fb3c1a-75e2-488e-b7c8-9444c809d845 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048396 Sep 27 10:01:20.148997 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 708/1270] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:20.154354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:20.155063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:20.161985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.162486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.162924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.164366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.165532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.165965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.311015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:20.319576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:20.321778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:20.322974 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-95d37c90-ef0e-4b2b-9bf3-8ca1ffe71f13 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1567 microversion: 2.25 time: 0.172872 Sep 27 10:01:20.323950 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 709/1271] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:20 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1567 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:20.328116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2978cfb4-e23d-473a-873c-1f101c8959c6 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:20.328579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2978cfb4-e23d-473a-873c-1f101c8959c6 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:20.338568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2978cfb4-e23d-473a-873c-1f101c8959c6 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.339331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2978cfb4-e23d-473a-873c-1f101c8959c6 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.339962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2978cfb4-e23d-473a-873c-1f101c8959c6 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.340816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2978cfb4-e23d-473a-873c-1f101c8959c6 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.342010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2978cfb4-e23d-473a-873c-1f101c8959c6 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.342434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2978cfb4-e23d-473a-873c-1f101c8959c6 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.416560 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2978cfb4-e23d-473a-873c-1f101c8959c6 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4/action" status: 202 len: 0 microversion: 2.1 time: 0.091747 Sep 27 10:01:20.418643 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 710/1272] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:01:20 2022] POST /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4/action => generated 0 bytes in 95 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:01:20.419998 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd67919e-91e1-48ce-8c2d-a904fa94e4d5 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/3496c8f9-d667-408e-bcbd-287c1895b4a7" status: 202 len: 0 microversion: 2.1 time: 1.581281 Sep 27 10:01:20.420867 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 563/1273] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:01:18 2022] DELETE /compute/v2.1/os-security-groups/3496c8f9-d667-408e-bcbd-287c1895b4a7 => generated 0 bytes in 1583 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:01:20.426982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-42543248-fa19-407a-a4a4-3660935e7b69 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:20.427786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-42543248-fa19-407a-a4a4-3660935e7b69 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:20.434969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42543248-fa19-407a-a4a4-3660935e7b69 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.435602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42543248-fa19-407a-a4a4-3660935e7b69 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.436134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42543248-fa19-407a-a4a4-3660935e7b69 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.437011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42543248-fa19-407a-a4a4-3660935e7b69 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.437563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42543248-fa19-407a-a4a4-3660935e7b69 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.438145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42543248-fa19-407a-a4a4-3660935e7b69 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.439505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-541fa483-4de7-4847-8ac1-86a7db74c8e2 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Calling method '.version_select at 0x7f7f63641310>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:20.623752 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-42543248-fa19-407a-a4a4-3660935e7b69 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1650 microversion: 2.1 time: 0.200784 Sep 27 10:01:20.624857 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 711/1274] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:20 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1650 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:20.638659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e9d9d711-f75f-47a4-94ba-d42ecadd7be5 req-402d8b7c-0871-49f8-9eeb-3812953d4b06 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1b526682-dfde-459f-bc61-2df38e0ec4b7", "tag": "bdcd738d-10f6-4db4-ab57-e00ef79f23bb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:20.653363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e9d9d711-f75f-47a4-94ba-d42ecadd7be5 req-402d8b7c-0871-49f8-9eeb-3812953d4b06 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.654382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e9d9d711-f75f-47a4-94ba-d42ecadd7be5 req-402d8b7c-0871-49f8-9eeb-3812953d4b06 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.655081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e9d9d711-f75f-47a4-94ba-d42ecadd7be5 req-402d8b7c-0871-49f8-9eeb-3812953d4b06 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.671503 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e9d9d711-f75f-47a4-94ba-d42ecadd7be5 req-402d8b7c-0871-49f8-9eeb-3812953d4b06 service nova] Creating event network-changed:bdcd738d-10f6-4db4-ab57-e00ef79f23bb for instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 on nch1-master-1 Sep 27 10:01:20.677146 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e9d9d711-f75f-47a4-94ba-d42ecadd7be5 req-402d8b7c-0871-49f8-9eeb-3812953d4b06 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046673 Sep 27 10:01:20.677833 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 712/1275] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:20.710610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5731fc21-b95e-4200-a966-ef05bb17c7fc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:20.711550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5731fc21-b95e-4200-a966-ef05bb17c7fc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:20.718224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5731fc21-b95e-4200-a966-ef05bb17c7fc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.718790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5731fc21-b95e-4200-a966-ef05bb17c7fc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.719291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5731fc21-b95e-4200-a966-ef05bb17c7fc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.720069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5731fc21-b95e-4200-a966-ef05bb17c7fc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.720644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5731fc21-b95e-4200-a966-ef05bb17c7fc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.721150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5731fc21-b95e-4200-a966-ef05bb17c7fc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.875302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5731fc21-b95e-4200-a966-ef05bb17c7fc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:20.883546 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5731fc21-b95e-4200-a966-ef05bb17c7fc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1307 microversion: 2.1 time: 0.177312 Sep 27 10:01:20.884130 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 713/1276] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:20 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1307 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:20.906761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-27e0eb1f-5e47-4231-b02d-9319f946351f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:20.907429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-27e0eb1f-5e47-4231-b02d-9319f946351f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:20.913938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27e0eb1f-5e47-4231-b02d-9319f946351f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.914499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27e0eb1f-5e47-4231-b02d-9319f946351f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.914997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27e0eb1f-5e47-4231-b02d-9319f946351f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:20.915802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27e0eb1f-5e47-4231-b02d-9319f946351f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:20.916258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27e0eb1f-5e47-4231-b02d-9319f946351f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:20.916823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27e0eb1f-5e47-4231-b02d-9319f946351f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:21.083230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-27e0eb1f-5e47-4231-b02d-9319f946351f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:21.094713 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-27e0eb1f-5e47-4231-b02d-9319f946351f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1470 microversion: 2.1 time: 0.191366 Sep 27 10:01:21.100316 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-541fa483-4de7-4847-8ac1-86a7db74c8e2 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/84099756-7273-454a-b0ae-ff29ceb9c41f" status: 202 len: 0 microversion: 2.1 time: 0.670841 Sep 27 10:01:21.102297 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 564/1277] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:01:20 2022] DELETE /compute/v2.1/os-security-groups/84099756-7273-454a-b0ae-ff29ceb9c41f => generated 0 bytes in 674 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:01:21.113535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-44dc73fc-2727-48b0-b64f-1482fde45591 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Calling method '.version_select at 0x7f7f637b9940>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:21.118257 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 714/1278] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:20 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1470 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:01:21.132490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ab959c21-668f-415b-9002-c49530a37537 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:21.133657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ab959c21-668f-415b-9002-c49530a37537 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:21.143348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab959c21-668f-415b-9002-c49530a37537 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:21.143964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab959c21-668f-415b-9002-c49530a37537 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:21.144474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab959c21-668f-415b-9002-c49530a37537 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:21.147701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab959c21-668f-415b-9002-c49530a37537 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:21.148629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab959c21-668f-415b-9002-c49530a37537 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:21.149324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab959c21-668f-415b-9002-c49530a37537 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:21.317161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ab959c21-668f-415b-9002-c49530a37537 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:21.321841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:21.335313 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ab959c21-668f-415b-9002-c49530a37537 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1370 microversion: 2.1 time: 0.215462 Sep 27 10:01:21.336469 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 715/1279] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:21 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1370 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:21.349476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:21.350243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:21.358269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:21.358778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:21.359246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:21.360220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:21.360715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:21.361154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:21.504389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:21.514133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:21.515572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:21.516790 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-846c6dbd-fffe-4f42-b0af-8e9f8e6c80b8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1567 microversion: 2.25 time: 0.171837 Sep 27 10:01:21.517592 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 716/1280] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:21 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1567 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:01:21.648903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8571a1e0-b81f-4695-86f0-4282283c5ca1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:21.649184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8571a1e0-b81f-4695-86f0-4282283c5ca1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:21.655832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8571a1e0-b81f-4695-86f0-4282283c5ca1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:21.656331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8571a1e0-b81f-4695-86f0-4282283c5ca1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:21.656789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8571a1e0-b81f-4695-86f0-4282283c5ca1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:21.657572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8571a1e0-b81f-4695-86f0-4282283c5ca1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:21.658041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8571a1e0-b81f-4695-86f0-4282283c5ca1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:21.658492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8571a1e0-b81f-4695-86f0-4282283c5ca1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:21.802390 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8571a1e0-b81f-4695-86f0-4282283c5ca1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1630 microversion: 2.1 time: 0.157693 Sep 27 10:01:21.803073 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 717/1281] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:21 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1630 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:21.825997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1571e307-c729-42ce-a5f7-ac18d6779756 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:21.828742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1571e307-c729-42ce-a5f7-ac18d6779756 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:21.835026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1571e307-c729-42ce-a5f7-ac18d6779756 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:21.835560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1571e307-c729-42ce-a5f7-ac18d6779756 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:21.835998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1571e307-c729-42ce-a5f7-ac18d6779756 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:21.836739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1571e307-c729-42ce-a5f7-ac18d6779756 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:21.837200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1571e307-c729-42ce-a5f7-ac18d6779756 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:21.837624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1571e307-c729-42ce-a5f7-ac18d6779756 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:21.862936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-1571e307-c729-42ce-a5f7-ac18d6779756 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lazy-loading 'flavor' on Instance uuid e8ca3e24-d297-4435-abe8-a70d128f4bb4 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:01:21.863738 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-44dc73fc-2727-48b0-b64f-1482fde45591 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/b7f3e77b-c03a-44d6-8ecb-a96b87bb28ea" status: 202 len: 0 microversion: 2.1 time: 0.757199 Sep 27 10:01:21.864338 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 565/1282] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:01:21 2022] DELETE /compute/v2.1/os-security-groups/b7f3e77b-c03a-44d6-8ecb-a96b87bb28ea => generated 0 bytes in 759 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:01:21.873359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2c617429-5d2c-478d-a25d-ce318303d987 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Calling method '.version_select at 0x7f7f637608b0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:21.878992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1571e307-c729-42ce-a5f7-ac18d6779756 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Going to try to live migrate instance to nch2-master-1 {{(pid=145968) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Sep 27 10:01:23.496424 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2c617429-5d2c-478d-a25d-ce318303d987 tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/1a0ef65a-e693-4e1b-9b0a-c6fa4a8b8a9c" status: 202 len: 0 microversion: 2.1 time: 1.626867 Sep 27 10:01:23.497439 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 566/1283] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:01:21 2022] DELETE /compute/v2.1/os-security-groups/1a0ef65a-e693-4e1b-9b0a-c6fa4a8b8a9c => generated 0 bytes in 1629 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:01:23.505049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5b062fb9-c3d6-4d13-8b6d-84ddf0920833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:23.506200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5b062fb9-c3d6-4d13-8b6d-84ddf0920833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:23.516678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b062fb9-c3d6-4d13-8b6d-84ddf0920833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:23.517564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b062fb9-c3d6-4d13-8b6d-84ddf0920833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:23.518419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b062fb9-c3d6-4d13-8b6d-84ddf0920833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:23.519393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b062fb9-c3d6-4d13-8b6d-84ddf0920833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:23.520067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b062fb9-c3d6-4d13-8b6d-84ddf0920833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:23.520736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b062fb9-c3d6-4d13-8b6d-84ddf0920833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:24.767536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b062fb9-c3d6-4d13-8b6d-84ddf0920833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:24.796006 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5b062fb9-c3d6-4d13-8b6d-84ddf0920833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1307 microversion: 2.1 time: 1.297295 Sep 27 10:01:24.798139 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 567/1284] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:23 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1307 bytes in 1300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:24.809054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5507e665-7410-4f95-9364-986c22223cf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:24.810220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5507e665-7410-4f95-9364-986c22223cf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:24.822185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5507e665-7410-4f95-9364-986c22223cf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:24.823306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5507e665-7410-4f95-9364-986c22223cf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:24.824632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5507e665-7410-4f95-9364-986c22223cf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:24.826017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5507e665-7410-4f95-9364-986c22223cf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:24.826725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5507e665-7410-4f95-9364-986c22223cf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:24.827425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5507e665-7410-4f95-9364-986c22223cf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:25.477105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5507e665-7410-4f95-9364-986c22223cf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:25.486265 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5507e665-7410-4f95-9364-986c22223cf6 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1470 microversion: 2.1 time: 0.684524 Sep 27 10:01:25.486866 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 568/1285] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:24 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1470 bytes in 686 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:25.491785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-15807be4-7f73-4954-bb83-eed4c3aa965a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:25.492642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-15807be4-7f73-4954-bb83-eed4c3aa965a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:25.499423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15807be4-7f73-4954-bb83-eed4c3aa965a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:25.499423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15807be4-7f73-4954-bb83-eed4c3aa965a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:25.499783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15807be4-7f73-4954-bb83-eed4c3aa965a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:25.500494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15807be4-7f73-4954-bb83-eed4c3aa965a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:25.501021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15807be4-7f73-4954-bb83-eed4c3aa965a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:25.501450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15807be4-7f73-4954-bb83-eed4c3aa965a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:25.506342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:25.889325 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1571e307-c729-42ce-a5f7-ac18d6779756 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4/action" status: 202 len: 0 microversion: 2.1 time: 4.067563 Sep 27 10:01:25.890648 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 718/1286] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:01:21 2022] POST /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4/action => generated 0 bytes in 4070 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:01:25.900910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:25.901809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:25.910657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:25.911397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:25.912222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:25.913168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:25.915097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:25.915737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:25.999831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-15807be4-7f73-4954-bb83-eed4c3aa965a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:26.010754 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-15807be4-7f73-4954-bb83-eed4c3aa965a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1370 microversion: 2.1 time: 0.522770 Sep 27 10:01:26.011354 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 569/1287] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:25 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1370 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:26.025519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dd0608c7-3f38-4f47-a984-ca6199f69acd tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] Calling method '.version_select at 0x7f7f63671f70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:26.121353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:26.130480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:26.132467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:26.133787 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-12191cdd-734f-4fe7-80b7-8402c7819c73 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1567 microversion: 2.25 time: 0.241254 Sep 27 10:01:26.134351 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 719/1288] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:25 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1567 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:26.139714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d8126dfe-3e11-4e02-96fb-4ba4aba8f846 req-f6659e3e-cdcc-4a68-99f8-24b75cd0011e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1b526682-dfde-459f-bc61-2df38e0ec4b7", "name": "network-vif-plugged", "status": "completed", "tag": "bdcd738d-10f6-4db4-ab57-e00ef79f23bb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:26.163097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d8126dfe-3e11-4e02-96fb-4ba4aba8f846 req-f6659e3e-cdcc-4a68-99f8-24b75cd0011e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:26.164427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d8126dfe-3e11-4e02-96fb-4ba4aba8f846 req-f6659e3e-cdcc-4a68-99f8-24b75cd0011e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:26.165247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d8126dfe-3e11-4e02-96fb-4ba4aba8f846 req-f6659e3e-cdcc-4a68-99f8-24b75cd0011e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:26.210735 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d8126dfe-3e11-4e02-96fb-4ba4aba8f846 req-f6659e3e-cdcc-4a68-99f8-24b75cd0011e service nova] Creating event network-vif-plugged:bdcd738d-10f6-4db4-ab57-e00ef79f23bb for instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 on nch1-master-1 Sep 27 10:01:26.217924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d8126dfe-3e11-4e02-96fb-4ba4aba8f846 req-f6659e3e-cdcc-4a68-99f8-24b75cd0011e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081761 Sep 27 10:01:26.218792 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 720/1289] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:26.227755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-81ae6b63-2d7f-4a90-b46c-4667f200227c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:26.228927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-81ae6b63-2d7f-4a90-b46c-4667f200227c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:26.243051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81ae6b63-2d7f-4a90-b46c-4667f200227c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:26.243551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81ae6b63-2d7f-4a90-b46c-4667f200227c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:26.244259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81ae6b63-2d7f-4a90-b46c-4667f200227c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:26.246093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81ae6b63-2d7f-4a90-b46c-4667f200227c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:26.246622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81ae6b63-2d7f-4a90-b46c-4667f200227c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:26.247055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81ae6b63-2d7f-4a90-b46c-4667f200227c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:26.435813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-81ae6b63-2d7f-4a90-b46c-4667f200227c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:26.447970 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-81ae6b63-2d7f-4a90-b46c-4667f200227c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1307 microversion: 2.1 time: 0.226646 Sep 27 10:01:26.452365 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 721/1290] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:26 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1307 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:26.457967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1cee28c5-9044-404d-aa17-a1e5b6d46ed5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:26.458812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1cee28c5-9044-404d-aa17-a1e5b6d46ed5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:26.466357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cee28c5-9044-404d-aa17-a1e5b6d46ed5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:26.467184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cee28c5-9044-404d-aa17-a1e5b6d46ed5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:26.467783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cee28c5-9044-404d-aa17-a1e5b6d46ed5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:26.468874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cee28c5-9044-404d-aa17-a1e5b6d46ed5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:26.469335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cee28c5-9044-404d-aa17-a1e5b6d46ed5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:26.469746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cee28c5-9044-404d-aa17-a1e5b6d46ed5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:26.650784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1cee28c5-9044-404d-aa17-a1e5b6d46ed5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:26.660755 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1cee28c5-9044-404d-aa17-a1e5b6d46ed5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 0.206243 Sep 27 10:01:26.661664 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 722/1291] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:26 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:26.667384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-37d4dce7-2e92-44de-9b34-d91f804bb13a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:26.668072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-37d4dce7-2e92-44de-9b34-d91f804bb13a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:26.675407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37d4dce7-2e92-44de-9b34-d91f804bb13a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:26.676221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37d4dce7-2e92-44de-9b34-d91f804bb13a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:26.676962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37d4dce7-2e92-44de-9b34-d91f804bb13a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:26.678057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37d4dce7-2e92-44de-9b34-d91f804bb13a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:26.678828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37d4dce7-2e92-44de-9b34-d91f804bb13a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:26.679459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37d4dce7-2e92-44de-9b34-d91f804bb13a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:26.933303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-37d4dce7-2e92-44de-9b34-d91f804bb13a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:26.941937 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-37d4dce7-2e92-44de-9b34-d91f804bb13a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1536 microversion: 2.1 time: 0.279072 Sep 27 10:01:26.942681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 723/1292] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:26 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1536 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:27.042156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d61d6f4e-c500-446a-9d17-c6e678ef6bf9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:27.043099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d61d6f4e-c500-446a-9d17-c6e678ef6bf9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:27.046998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:27.055309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d61d6f4e-c500-446a-9d17-c6e678ef6bf9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:27.056173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d61d6f4e-c500-446a-9d17-c6e678ef6bf9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:27.056612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d61d6f4e-c500-446a-9d17-c6e678ef6bf9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:27.057796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d61d6f4e-c500-446a-9d17-c6e678ef6bf9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:27.058255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d61d6f4e-c500-446a-9d17-c6e678ef6bf9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:27.058663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d61d6f4e-c500-446a-9d17-c6e678ef6bf9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:27.270182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d61d6f4e-c500-446a-9d17-c6e678ef6bf9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:27.279558 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d61d6f4e-c500-446a-9d17-c6e678ef6bf9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1370 microversion: 2.1 time: 0.243186 Sep 27 10:01:27.282583 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 724/1293] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:27 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1370 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:01:27.299559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:27.300392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:27.308325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:27.309022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:27.309565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:27.310912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:27.310912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:27.312224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:27.575222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:27.584814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:27.587271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:27.588644 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53005d04-2d54-48d7-b73d-419e7bea8513 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1567 microversion: 2.25 time: 0.297038 Sep 27 10:01:27.605592 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 725/1294] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:27 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1567 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:27.625059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-69acd060-0d26-4d94-ba3b-585ca7b1cfe1 req-cff7bd38-6912-4e79-9426-d3d12831fdf9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69ea013c-5143-45ff-81a2-3ed97a8659cd", "name": "network-vif-unplugged", "status": "completed", "tag": "96ac21a6-013a-4a5e-9320-82ff90906b76"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:27.635530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-69acd060-0d26-4d94-ba3b-585ca7b1cfe1 req-cff7bd38-6912-4e79-9426-d3d12831fdf9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:27.636094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-69acd060-0d26-4d94-ba3b-585ca7b1cfe1 req-cff7bd38-6912-4e79-9426-d3d12831fdf9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:27.636456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-69acd060-0d26-4d94-ba3b-585ca7b1cfe1 req-cff7bd38-6912-4e79-9426-d3d12831fdf9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:27.651018 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-69acd060-0d26-4d94-ba3b-585ca7b1cfe1 req-cff7bd38-6912-4e79-9426-d3d12831fdf9 service nova] Creating event network-vif-unplugged:96ac21a6-013a-4a5e-9320-82ff90906b76 for instance 69ea013c-5143-45ff-81a2-3ed97a8659cd on nch1-master-1 Sep 27 10:01:27.674233 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dd0608c7-3f38-4f47-a984-ca6199f69acd tempest-SecurityGroupRulesNegativeTestJSON-636602021 tempest-SecurityGroupRulesNegativeTestJSON-636602021-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/5744c4c0-cf3d-4604-b2cc-383261adbdb0" status: 202 len: 0 microversion: 2.1 time: 1.654413 Sep 27 10:01:27.683783 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 570/1295] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:01:26 2022] DELETE /compute/v2.1/os-security-groups/5744c4c0-cf3d-4604-b2cc-383261adbdb0 => generated 0 bytes in 1665 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:01:27.691032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a80bcb99-ef58-4280-b5cd-7a34366e8ccb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:27.692091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a80bcb99-ef58-4280-b5cd-7a34366e8ccb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:27.696006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-69acd060-0d26-4d94-ba3b-585ca7b1cfe1 req-cff7bd38-6912-4e79-9426-d3d12831fdf9 service nova] Instance 69ea013c-5143-45ff-81a2-3ed97a8659cd is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:01:27.709603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a80bcb99-ef58-4280-b5cd-7a34366e8ccb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:27.710593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a80bcb99-ef58-4280-b5cd-7a34366e8ccb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:27.711108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a80bcb99-ef58-4280-b5cd-7a34366e8ccb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:27.712128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a80bcb99-ef58-4280-b5cd-7a34366e8ccb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:27.712905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a80bcb99-ef58-4280-b5cd-7a34366e8ccb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:27.713404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a80bcb99-ef58-4280-b5cd-7a34366e8ccb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:27.715508 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-69acd060-0d26-4d94-ba3b-585ca7b1cfe1 req-cff7bd38-6912-4e79-9426-d3d12831fdf9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.108074 Sep 27 10:01:27.717310 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 726/1296] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:27 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:27.725259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af098181-b7e7-416f-9ffa-9471d13ca9d0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:27.727195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af098181-b7e7-416f-9ffa-9471d13ca9d0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:27.735263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af098181-b7e7-416f-9ffa-9471d13ca9d0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:27.736455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af098181-b7e7-416f-9ffa-9471d13ca9d0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:27.737110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af098181-b7e7-416f-9ffa-9471d13ca9d0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:27.738935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af098181-b7e7-416f-9ffa-9471d13ca9d0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:27.739737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af098181-b7e7-416f-9ffa-9471d13ca9d0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:27.740615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af098181-b7e7-416f-9ffa-9471d13ca9d0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:28.202336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af098181-b7e7-416f-9ffa-9471d13ca9d0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:28.213506 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af098181-b7e7-416f-9ffa-9471d13ca9d0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 0.494651 Sep 27 10:01:28.224007 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 727/1297] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:27 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:28.229387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1513df91-8bb8-4eed-a7ae-da4d5e165632 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:28.230013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1513df91-8bb8-4eed-a7ae-da4d5e165632 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:28.238124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1513df91-8bb8-4eed-a7ae-da4d5e165632 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:28.238603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1513df91-8bb8-4eed-a7ae-da4d5e165632 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:28.239064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1513df91-8bb8-4eed-a7ae-da4d5e165632 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:28.239780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1513df91-8bb8-4eed-a7ae-da4d5e165632 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:28.240243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1513df91-8bb8-4eed-a7ae-da4d5e165632 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:28.240771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1513df91-8bb8-4eed-a7ae-da4d5e165632 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:28.533842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a80bcb99-ef58-4280-b5cd-7a34366e8ccb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:28.542774 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a80bcb99-ef58-4280-b5cd-7a34366e8ccb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1307 microversion: 2.1 time: 0.857380 Sep 27 10:01:28.543238 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 571/1298] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:27 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1307 bytes in 859 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:28.549819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-8015f2af-4aef-41dc-b22c-8fe07101df13 req-78a51a37-0a4e-4227-a5f6-7ae11e79bdfe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0fa79578-f299-4e5d-8dea-8bd1ca1c31d1", "name": "network-vif-plugged", "status": "completed", "tag": "ffaeeb25-b086-4ac5-b059-d12a6497adf1"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:28.563876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8015f2af-4aef-41dc-b22c-8fe07101df13 req-78a51a37-0a4e-4227-a5f6-7ae11e79bdfe service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:28.564754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8015f2af-4aef-41dc-b22c-8fe07101df13 req-78a51a37-0a4e-4227-a5f6-7ae11e79bdfe service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:28.565509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8015f2af-4aef-41dc-b22c-8fe07101df13 req-78a51a37-0a4e-4227-a5f6-7ae11e79bdfe service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:28.578809 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8015f2af-4aef-41dc-b22c-8fe07101df13 req-78a51a37-0a4e-4227-a5f6-7ae11e79bdfe service nova] Creating event network-vif-plugged:ffaeeb25-b086-4ac5-b059-d12a6497adf1 for instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 on nch2-master-1 Sep 27 10:01:28.585353 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-8015f2af-4aef-41dc-b22c-8fe07101df13 req-78a51a37-0a4e-4227-a5f6-7ae11e79bdfe service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040305 Sep 27 10:01:28.586149 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 572/1299] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:28.593790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6d1f3230-445a-4148-ba29-ce6e3deef592 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:28.594496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6d1f3230-445a-4148-ba29-ce6e3deef592 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:28.602205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d1f3230-445a-4148-ba29-ce6e3deef592 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:28.603003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d1f3230-445a-4148-ba29-ce6e3deef592 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:28.603601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d1f3230-445a-4148-ba29-ce6e3deef592 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:28.604490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d1f3230-445a-4148-ba29-ce6e3deef592 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:28.605203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d1f3230-445a-4148-ba29-ce6e3deef592 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:28.605846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d1f3230-445a-4148-ba29-ce6e3deef592 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:28.617071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1513df91-8bb8-4eed-a7ae-da4d5e165632 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:28.629244 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1513df91-8bb8-4eed-a7ae-da4d5e165632 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1536 microversion: 2.1 time: 0.403754 Sep 27 10:01:28.643718 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 728/1300] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:28 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1536 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:28.650303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:28.650921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:28.659684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:28.660718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:28.660875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:28.661555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:28.662430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:28.665076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:29.116178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:29.134642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:29.137931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:29.139881 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-56befe4a-d27a-4c53-8d29-b9dfa17ed22d tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1567 microversion: 2.25 time: 0.495077 Sep 27 10:01:29.146312 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 729/1301] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:28 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1567 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:29.247926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-788ce0a5-67dd-49e5-8d71-a94616280950 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:29.249489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-788ce0a5-67dd-49e5-8d71-a94616280950 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:29.256982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-788ce0a5-67dd-49e5-8d71-a94616280950 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:29.257806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-788ce0a5-67dd-49e5-8d71-a94616280950 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:29.258587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-788ce0a5-67dd-49e5-8d71-a94616280950 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:29.259159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-788ce0a5-67dd-49e5-8d71-a94616280950 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:29.259877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-788ce0a5-67dd-49e5-8d71-a94616280950 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:29.260562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-788ce0a5-67dd-49e5-8d71-a94616280950 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:29.334214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6d1f3230-445a-4148-ba29-ce6e3deef592 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:29.343422 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6d1f3230-445a-4148-ba29-ce6e3deef592 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1370 microversion: 2.1 time: 0.755296 Sep 27 10:01:29.344011 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 573/1302] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:28 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1370 bytes in 757 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:29.569746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1d10174f-81b8-4676-82bd-00a20bdc5eee tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:29.570802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1d10174f-81b8-4676-82bd-00a20bdc5eee tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:29.577751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d10174f-81b8-4676-82bd-00a20bdc5eee tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:29.578674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d10174f-81b8-4676-82bd-00a20bdc5eee tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:29.579568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d10174f-81b8-4676-82bd-00a20bdc5eee tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:29.580575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d10174f-81b8-4676-82bd-00a20bdc5eee tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:29.581323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d10174f-81b8-4676-82bd-00a20bdc5eee tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:29.581997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d10174f-81b8-4676-82bd-00a20bdc5eee tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:29.872268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-788ce0a5-67dd-49e5-8d71-a94616280950 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:29.895871 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-788ce0a5-67dd-49e5-8d71-a94616280950 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 0.651586 Sep 27 10:01:29.903054 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 730/1303] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:29 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:29.923438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-db8d4c2a-4640-4c2a-88a6-d3354a7eae2b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:29.924073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-db8d4c2a-4640-4c2a-88a6-d3354a7eae2b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:29.948261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:29.973406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db8d4c2a-4640-4c2a-88a6-d3354a7eae2b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:29.978760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db8d4c2a-4640-4c2a-88a6-d3354a7eae2b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:29.981138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db8d4c2a-4640-4c2a-88a6-d3354a7eae2b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:29.991311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db8d4c2a-4640-4c2a-88a6-d3354a7eae2b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:29.992403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1d10174f-81b8-4676-82bd-00a20bdc5eee tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:29.993258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db8d4c2a-4640-4c2a-88a6-d3354a7eae2b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:29.993709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db8d4c2a-4640-4c2a-88a6-d3354a7eae2b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:30.001896 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1d10174f-81b8-4676-82bd-00a20bdc5eee tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1488 microversion: 2.1 time: 0.436092 Sep 27 10:01:30.003383 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 574/1304] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:29 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1488 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:01:30.008528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b12c925f-192a-455b-958e-eabf87ac7869 req-46367b40-2afd-4e28-83b7-e379f3bffe41 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "69ea013c-5143-45ff-81a2-3ed97a8659cd", "tag": "96ac21a6-013a-4a5e-9320-82ff90906b76"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:30.019194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b12c925f-192a-455b-958e-eabf87ac7869 req-46367b40-2afd-4e28-83b7-e379f3bffe41 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:30.020031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b12c925f-192a-455b-958e-eabf87ac7869 req-46367b40-2afd-4e28-83b7-e379f3bffe41 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:30.020627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b12c925f-192a-455b-958e-eabf87ac7869 req-46367b40-2afd-4e28-83b7-e379f3bffe41 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:30.035350 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b12c925f-192a-455b-958e-eabf87ac7869 req-46367b40-2afd-4e28-83b7-e379f3bffe41 service nova] Creating event network-changed:96ac21a6-013a-4a5e-9320-82ff90906b76 for instance 69ea013c-5143-45ff-81a2-3ed97a8659cd on nch1-master-1 Sep 27 10:01:30.041905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-b12c925f-192a-455b-958e-eabf87ac7869 req-46367b40-2afd-4e28-83b7-e379f3bffe41 service nova] Instance 69ea013c-5143-45ff-81a2-3ed97a8659cd is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:01:30.043995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:30.056702 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b12c925f-192a-455b-958e-eabf87ac7869 req-46367b40-2afd-4e28-83b7-e379f3bffe41 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051650 Sep 27 10:01:30.058825 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 575/1305] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:30.066039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba460af1-a1d8-4b2c-ab59-4f937638e40e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:30.067428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ba460af1-a1d8-4b2c-ab59-4f937638e40e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:30.077513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba460af1-a1d8-4b2c-ab59-4f937638e40e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:30.078354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba460af1-a1d8-4b2c-ab59-4f937638e40e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:30.079090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba460af1-a1d8-4b2c-ab59-4f937638e40e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:30.080336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba460af1-a1d8-4b2c-ab59-4f937638e40e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:30.081044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba460af1-a1d8-4b2c-ab59-4f937638e40e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:30.082130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba460af1-a1d8-4b2c-ab59-4f937638e40e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:30.205604 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ba460af1-a1d8-4b2c-ab59-4f937638e40e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "POST /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7/action" status: 202 len: 0 microversion: 2.1 time: 0.144686 Sep 27 10:01:30.206853 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 576/1306] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:01:30 2022] POST /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7/action => generated 0 bytes in 147 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:01:30.215187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:30.216162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:30.224692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:30.225557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:30.226330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:30.227854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:30.228603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:30.229450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:30.700932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-db8d4c2a-4640-4c2a-88a6-d3354a7eae2b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:30.713990 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-db8d4c2a-4640-4c2a-88a6-d3354a7eae2b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1536 microversion: 2.1 time: 0.809403 Sep 27 10:01:30.714580 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 731/1307] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:29 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1536 bytes in 811 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:30.720219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-484db3e4-6317-4be4-974f-4e7cf625a477 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:30.721456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-484db3e4-6317-4be4-974f-4e7cf625a477 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:30.728709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-484db3e4-6317-4be4-974f-4e7cf625a477 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:30.729485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-484db3e4-6317-4be4-974f-4e7cf625a477 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:30.730132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-484db3e4-6317-4be4-974f-4e7cf625a477 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:30.731419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-484db3e4-6317-4be4-974f-4e7cf625a477 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:30.732143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-484db3e4-6317-4be4-974f-4e7cf625a477 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:30.733133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-484db3e4-6317-4be4-974f-4e7cf625a477 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:31.107138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-484db3e4-6317-4be4-974f-4e7cf625a477 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:31.129717 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-484db3e4-6317-4be4-974f-4e7cf625a477 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1493 microversion: 2.1 time: 0.413187 Sep 27 10:01:31.144841 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 732/1308] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:30 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1493 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:31.144841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a525c7e3-d446-4843-9d40-c60c0f989bc4 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:31.144841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a525c7e3-d446-4843-9d40-c60c0f989bc4 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:31.187285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a525c7e3-d446-4843-9d40-c60c0f989bc4 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:31.191582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a525c7e3-d446-4843-9d40-c60c0f989bc4 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:31.193476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a525c7e3-d446-4843-9d40-c60c0f989bc4 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:31.194927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a525c7e3-d446-4843-9d40-c60c0f989bc4 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:31.195667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a525c7e3-d446-4843-9d40-c60c0f989bc4 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:31.196356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a525c7e3-d446-4843-9d40-c60c0f989bc4 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:31.383242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:31.396561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:31.400219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:31.402565 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0c2696b2-e39a-47e9-9d44-83b05154cd39 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1542 microversion: 2.25 time: 1.193898 Sep 27 10:01:31.418533 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 577/1309] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:30 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1542 bytes in 1210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:31.431054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a4ce0d89-4afd-46a0-b7f0-6e704ff82983 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:31.432628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a4ce0d89-4afd-46a0-b7f0-6e704ff82983 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:31.461908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ce0d89-4afd-46a0-b7f0-6e704ff82983 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:31.479648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ce0d89-4afd-46a0-b7f0-6e704ff82983 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:31.483331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ce0d89-4afd-46a0-b7f0-6e704ff82983 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:31.487723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ce0d89-4afd-46a0-b7f0-6e704ff82983 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:31.490639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ce0d89-4afd-46a0-b7f0-6e704ff82983 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:31.494516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ce0d89-4afd-46a0-b7f0-6e704ff82983 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:31.576008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a525c7e3-d446-4843-9d40-c60c0f989bc4 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:31.586681 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a525c7e3-d446-4843-9d40-c60c0f989bc4 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1557 microversion: 2.1 time: 0.451980 Sep 27 10:01:31.593786 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 733/1310] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:31 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1557 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:31.598701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dad900c1-6126-455c-92d7-a78ad1781355 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '.version_select at 0x7f7f63e36280>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:31.616529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dad900c1-6126-455c-92d7-a78ad1781355 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:31.617625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dad900c1-6126-455c-92d7-a78ad1781355 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:31.618350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dad900c1-6126-455c-92d7-a78ad1781355 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:31.631148 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dad900c1-6126-455c-92d7-a78ad1781355 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/os-migrations" status: 200 len: 3451 microversion: 2.25 time: 0.035989 Sep 27 10:01:31.642571 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 734/1311] 10.222.0.26 () {60 vars in 1184 bytes} [Tue Sep 27 10:01:31 2022] GET /compute/v2.1/os-migrations => generated 3451 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:31.648900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d43f623-1f62-4887-915a-b00af4879282 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:31.652624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d43f623-1f62-4887-915a-b00af4879282 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:31.666635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d43f623-1f62-4887-915a-b00af4879282 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:31.671079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d43f623-1f62-4887-915a-b00af4879282 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:31.671856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d43f623-1f62-4887-915a-b00af4879282 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:31.672880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d43f623-1f62-4887-915a-b00af4879282 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:31.673506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d43f623-1f62-4887-915a-b00af4879282 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:31.674110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d43f623-1f62-4887-915a-b00af4879282 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:31.785362 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d43f623-1f62-4887-915a-b00af4879282 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.141132 Sep 27 10:01:31.786786 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 735/1312] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:01:31 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-interface => generated 285 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:31.801710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6570029-5df3-492e-ab00-92d2afc997f8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:31.803336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c6570029-5df3-492e-ab00-92d2afc997f8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:31.813145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6570029-5df3-492e-ab00-92d2afc997f8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:31.814902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6570029-5df3-492e-ab00-92d2afc997f8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:31.815561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6570029-5df3-492e-ab00-92d2afc997f8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:31.817496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6570029-5df3-492e-ab00-92d2afc997f8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:31.818182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6570029-5df3-492e-ab00-92d2afc997f8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:31.819654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6570029-5df3-492e-ab00-92d2afc997f8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:32.306538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a4ce0d89-4afd-46a0-b7f0-6e704ff82983 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:32.330450 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a4ce0d89-4afd-46a0-b7f0-6e704ff82983 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 0.910685 Sep 27 10:01:32.334843 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 578/1313] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:31 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 916 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:32.349280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cddc4481-ffaa-46a4-95ce-ccfda84bb13c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:32.350173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cddc4481-ffaa-46a4-95ce-ccfda84bb13c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:32.368534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cddc4481-ffaa-46a4-95ce-ccfda84bb13c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:32.370226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cddc4481-ffaa-46a4-95ce-ccfda84bb13c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:32.370869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cddc4481-ffaa-46a4-95ce-ccfda84bb13c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:32.372536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cddc4481-ffaa-46a4-95ce-ccfda84bb13c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:32.373228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cddc4481-ffaa-46a4-95ce-ccfda84bb13c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:32.377013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-c6570029-5df3-492e-ab00-92d2afc997f8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid 9edcd7b8-f01b-4529-b9c1-3729e79ec760 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:01:32.377901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cddc4481-ffaa-46a4-95ce-ccfda84bb13c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:32.445738 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c6570029-5df3-492e-ab00-92d2afc997f8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 2035 microversion: 2.25 time: 0.643359 Sep 27 10:01:32.457465 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 736/1314] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:31 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 2035 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:32.496448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-701ea612-e64c-4be4-ac38-b20036af8e63 req-8ffde9d5-cd1d-4621-8bb1-a7b7dd940518 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e8ca3e24-d297-4435-abe8-a70d128f4bb4", "name": "network-vif-unplugged", "status": "completed", "tag": "769d6c7f-2742-4b0f-b84e-a0dcd3b8b472"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:32.517881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-701ea612-e64c-4be4-ac38-b20036af8e63 req-8ffde9d5-cd1d-4621-8bb1-a7b7dd940518 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:32.519220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-701ea612-e64c-4be4-ac38-b20036af8e63 req-8ffde9d5-cd1d-4621-8bb1-a7b7dd940518 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:32.520230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-701ea612-e64c-4be4-ac38-b20036af8e63 req-8ffde9d5-cd1d-4621-8bb1-a7b7dd940518 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:32.538503 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-701ea612-e64c-4be4-ac38-b20036af8e63 req-8ffde9d5-cd1d-4621-8bb1-a7b7dd940518 service nova] Creating event network-vif-unplugged:769d6c7f-2742-4b0f-b84e-a0dcd3b8b472 for instance e8ca3e24-d297-4435-abe8-a70d128f4bb4 on nch1-master-1 Sep 27 10:01:32.553385 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-701ea612-e64c-4be4-ac38-b20036af8e63 req-8ffde9d5-cd1d-4621-8bb1-a7b7dd940518 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.085710 Sep 27 10:01:32.555110 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 737/1315] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:32 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:32.562144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6d2d2037-971f-4d20-a45f-991ee4e18835 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:32.563157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6d2d2037-971f-4d20-a45f-991ee4e18835 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:32.570221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d2d2037-971f-4d20-a45f-991ee4e18835 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:32.571200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d2d2037-971f-4d20-a45f-991ee4e18835 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:32.571796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d2d2037-971f-4d20-a45f-991ee4e18835 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:32.572759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d2d2037-971f-4d20-a45f-991ee4e18835 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:32.573385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d2d2037-971f-4d20-a45f-991ee4e18835 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:32.573971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d2d2037-971f-4d20-a45f-991ee4e18835 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:32.805654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cddc4481-ffaa-46a4-95ce-ccfda84bb13c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:32.824245 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cddc4481-ffaa-46a4-95ce-ccfda84bb13c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1529 microversion: 2.1 time: 0.487828 Sep 27 10:01:32.824876 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 579/1316] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:32 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1529 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:32.830222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dc4479f0-5ce0-4283-a00a-3cba52461663 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Action: 'create', calling method: .version_select at 0x7f7f6366bb80>, body: {"keypair": {"name": "tempest-keypair-270589461", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM4lOdCYE/21Qlr1NLxgM0oun1kwY3RwbusijGM0dRIc1Zg1J1hjMf6ZlTiQRvbFSSv+QiWH/HJ5vM7RfnPP9WCTZHk+sZU23zLMfJ1Yz3A1fNwoM/axRur2/1fbObl+WA=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:32.869553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-dc4479f0-5ce0-4283-a00a-3cba52461663 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Getting quotas for project 2369f5ee467643948f618bbdee837521. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:01:32.875517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-dc4479f0-5ce0-4283-a00a-3cba52461663 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Getting quotas for user 29d2f7b0e8274c769521d13930be9825 and project 2369f5ee467643948f618bbdee837521. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:01:32.926621 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dc4479f0-5ce0-4283-a00a-3cba52461663 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.25 time: 0.100455 Sep 27 10:01:32.927937 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 580/1317] 10.222.0.26 () {62 vars in 1200 bytes} [Tue Sep 27 10:01:32 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 102 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 27 10:01:32.940802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ed0d94dd-eeb1-4302-9c54-9b058e0a289f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:32.941673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ed0d94dd-eeb1-4302-9c54-9b058e0a289f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:32.959096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed0d94dd-eeb1-4302-9c54-9b058e0a289f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:32.959699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed0d94dd-eeb1-4302-9c54-9b058e0a289f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:32.960587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed0d94dd-eeb1-4302-9c54-9b058e0a289f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:32.961092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed0d94dd-eeb1-4302-9c54-9b058e0a289f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:32.961748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed0d94dd-eeb1-4302-9c54-9b058e0a289f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:32.962288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed0d94dd-eeb1-4302-9c54-9b058e0a289f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:33.105181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6d2d2037-971f-4d20-a45f-991ee4e18835 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:33.113258 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6d2d2037-971f-4d20-a45f-991ee4e18835 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 200 len: 1473 microversion: 2.1 time: 0.556505 Sep 27 10:01:33.134879 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 738/1318] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:32 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 1473 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:33.143993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-920ac097-2295-4cf8-b402-a722ca0a1e5d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-146670574"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:33.146381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-920ac097-2295-4cf8-b402-a722ca0a1e5d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:33.164349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-920ac097-2295-4cf8-b402-a722ca0a1e5d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:33.165006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-920ac097-2295-4cf8-b402-a722ca0a1e5d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:33.168964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-920ac097-2295-4cf8-b402-a722ca0a1e5d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:33.169923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-920ac097-2295-4cf8-b402-a722ca0a1e5d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:33.170562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-920ac097-2295-4cf8-b402-a722ca0a1e5d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:33.170990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-920ac097-2295-4cf8-b402-a722ca0a1e5d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:33.779379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ed0d94dd-eeb1-4302-9c54-9b058e0a289f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:33.841130 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ed0d94dd-eeb1-4302-9c54-9b058e0a289f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1529 microversion: 2.1 time: 0.910501 Sep 27 10:01:33.857082 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 581/1319] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:32 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1529 bytes in 928 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:33.862982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6006991e-0673-46b5-9bbb-6902d7684a49 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:33.868229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6006991e-0673-46b5-9bbb-6902d7684a49 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:33.877087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6006991e-0673-46b5-9bbb-6902d7684a49 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:33.878866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6006991e-0673-46b5-9bbb-6902d7684a49 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:33.879727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6006991e-0673-46b5-9bbb-6902d7684a49 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:33.882632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6006991e-0673-46b5-9bbb-6902d7684a49 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:33.884316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6006991e-0673-46b5-9bbb-6902d7684a49 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:33.885263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6006991e-0673-46b5-9bbb-6902d7684a49 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:34.179533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6006991e-0673-46b5-9bbb-6902d7684a49 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:34.204756 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6006991e-0673-46b5-9bbb-6902d7684a49 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 0.345512 Sep 27 10:01:34.208298 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 582/1320] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:33 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:34.217670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73f96ad6-21ee-4ab8-94c3-d3376d524cf2 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:34.232086 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-73f96ad6-21ee-4ab8-94c3-d3376d524cf2 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] HTTP exception thrown: Flavor f627fe99-2d47-4507-bbb8-75b3a025ba89 could not be found. Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73f96ad6-21ee-4ab8-94c3-d3376d524cf2 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Request method failure captured: Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/flavors/f627fe99-2d47-4507-bbb8-75b3a025ba89 HTTP/1.1 Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMskEJKO1NeanpsGQq7CFVwA06n2KpraCBF0ltgcN32pMeL0yWYkD4Y-0nOd19f5b-BYjUDKPkAFRcUKsYlXaM0qaa2M5sV170J1cC1HqPxf6KEWwZwVpClbilx-YuobQIlW4A_Y65gFHLp87E1LFHbYu2jk1Py1tiXK8R765kIE2SYo Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: a3199e83259042efada3f2a565c624ff Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-MigrationsAdminTest-755919817 Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 10:01:34.243809 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a3199e83259042efada3f2a565c624ff"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a3199e83259042efada3f2a565c624ff"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a3199e83259042efada3f2a565c624ff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a3199e83259042efada3f2a565c624ff"}]}] Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-MigrationsAdminTest-755919817 Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: a3199e83259042efada3f2a565c624ff Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-MigrationsAdminTest-755919817 Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-MigrationsAdminTest-755919817-project-admin Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: f74e77337957458794bd6c942c97b344 Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-MigrationsAdminTest-755919817-project-admin Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: exception: Flavor f627fe99-2d47-4507-bbb8-75b3a025ba89 could not be found. Sep 27 10:01:34.245387 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'f627fe99-2d47-4507-bbb8-75b3a025ba89'} Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_manage.py", line 43, in _delete Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi flavor.destroy() Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 558, in destroy Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_flavor = self._flavor_destroy(self._context, Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 544, in _flavor_destroy Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return _flavor_destroy(context, flavor_id=flavor_id, flavorid=flavorid) Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 185, in _flavor_destroy Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=(flavor_id or flavorid)) Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor f627fe99-2d47-4507-bbb8-75b3a025ba89 could not be found. Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:34.246286 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavor_manage.py", line 45, in _delete Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor f627fe99-2d47-4507-bbb8-75b3a025ba89 could not be found. Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor f627fe99-2d47-4507-bbb8-75b3a025ba89 could not be found. Sep 27 10:01:34.247925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:01:34.250513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73f96ad6-21ee-4ab8-94c3-d3376d524cf2 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Returning 404 to user: Flavor f627fe99-2d47-4507-bbb8-75b3a025ba89 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:34.251835 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-73f96ad6-21ee-4ab8-94c3-d3376d524cf2 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/f627fe99-2d47-4507-bbb8-75b3a025ba89" status: 404 len: 109 microversion: 2.1 time: 0.041355 Sep 27 10:01:34.257013 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 583/1321] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:01:34 2022] DELETE /compute/v2.1/flavors/f627fe99-2d47-4507-bbb8-75b3a025ba89 => generated 109 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:01:34.286099 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-920ac097-2295-4cf8-b402-a722ca0a1e5d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "POST /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7/action" status: 202 len: 0 microversion: 2.1 time: 1.119805 Sep 27 10:01:34.293255 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 739/1322] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:01:33 2022] POST /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7/action => generated 0 bytes in 1154 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 10:01:34.317227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ab24e3ae-94d0-4d34-9d4a-bfe4c19eec2d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6366b1f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:34.392689 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ab24e3ae-94d0-4d34-9d4a-bfe4c19eec2d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.094800 Sep 27 10:01:34.395542 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 584/1323] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:34 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:34.442625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MigrationsAdminTest-server-43890143", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:34.519948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:01:34.629347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-fe49248b-dfd8-4b24-901e-af8aa25e5a91 req-41972212-4317-4058-9a47-5f313e2e5e18 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e8ca3e24-d297-4435-abe8-a70d128f4bb4", "tag": "769d6c7f-2742-4b0f-b84e-a0dcd3b8b472"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:34.640014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fe49248b-dfd8-4b24-901e-af8aa25e5a91 req-41972212-4317-4058-9a47-5f313e2e5e18 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:34.640923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fe49248b-dfd8-4b24-901e-af8aa25e5a91 req-41972212-4317-4058-9a47-5f313e2e5e18 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:34.641412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fe49248b-dfd8-4b24-901e-af8aa25e5a91 req-41972212-4317-4058-9a47-5f313e2e5e18 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:34.656031 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-fe49248b-dfd8-4b24-901e-af8aa25e5a91 req-41972212-4317-4058-9a47-5f313e2e5e18 service nova] Creating event network-changed:769d6c7f-2742-4b0f-b84e-a0dcd3b8b472 for instance e8ca3e24-d297-4435-abe8-a70d128f4bb4 on nch1-master-1 Sep 27 10:01:34.663691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-fe49248b-dfd8-4b24-901e-af8aa25e5a91 req-41972212-4317-4058-9a47-5f313e2e5e18 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041457 Sep 27 10:01:34.666478 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 585/1324] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:34.770390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-3b98a092-ff22-43d7-8233-716b8213e10b req-dfe3424b-0729-44e4-9fc4-f1ab8ffa4606 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69ea013c-5143-45ff-81a2-3ed97a8659cd", "name": "network-vif-plugged", "status": "completed", "tag": "96ac21a6-013a-4a5e-9320-82ff90906b76"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:34.790617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3b98a092-ff22-43d7-8233-716b8213e10b req-dfe3424b-0729-44e4-9fc4-f1ab8ffa4606 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:34.791461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3b98a092-ff22-43d7-8233-716b8213e10b req-dfe3424b-0729-44e4-9fc4-f1ab8ffa4606 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:34.792469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3b98a092-ff22-43d7-8233-716b8213e10b req-dfe3424b-0729-44e4-9fc4-f1ab8ffa4606 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:34.815175 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-3b98a092-ff22-43d7-8233-716b8213e10b req-dfe3424b-0729-44e4-9fc4-f1ab8ffa4606 service nova] Creating event network-vif-plugged:96ac21a6-013a-4a5e-9320-82ff90906b76 for instance 69ea013c-5143-45ff-81a2-3ed97a8659cd on nch1-master-1 Sep 27 10:01:34.826918 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-3b98a092-ff22-43d7-8233-716b8213e10b req-dfe3424b-0729-44e4-9fc4-f1ab8ffa4606 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060350 Sep 27 10:01:34.828437 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 586/1325] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:01:35.236980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7b50a717-bc0c-4f1b-aec9-dc378553a0f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:35.237694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7b50a717-bc0c-4f1b-aec9-dc378553a0f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:35.248970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b50a717-bc0c-4f1b-aec9-dc378553a0f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:35.251862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b50a717-bc0c-4f1b-aec9-dc378553a0f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:35.254876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b50a717-bc0c-4f1b-aec9-dc378553a0f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:35.255652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b50a717-bc0c-4f1b-aec9-dc378553a0f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:35.258360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b50a717-bc0c-4f1b-aec9-dc378553a0f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:35.260726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b50a717-bc0c-4f1b-aec9-dc378553a0f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:35.330619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:01:35.331696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:01:35.332352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:01:35.332980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:01:35.333555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:01:35.353111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:35.353792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:35.354435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:35.355347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:35.356074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:35.357110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:35.379565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for project e79eae6310354ca58046e59dfac23009. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:01:35.386454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Getting quotas for user 9544cdb0392f416197a586d99e8a3210 and project e79eae6310354ca58046e59dfac23009. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:01:35.401112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:01:35.404053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:01:35.484611 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-80b736a8-c53a-4b47-ac01-d27fae323697 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.051514 Sep 27 10:01:35.487455 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 740/1326] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:01:34 2022] POST /compute/v2.1/servers => generated 380 bytes in 1056 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:01:35.498455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7352c642-1574-4872-8fd9-6b59408164cf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c2cc160>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:35.577894 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7352c642-1574-4872-8fd9-6b59408164cf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.088448 Sep 27 10:01:35.579194 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 741/1327] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:35 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:35.585657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bca60f8b-612f-4459-b771-2ea197d34d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:35.586713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bca60f8b-612f-4459-b771-2ea197d34d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:35.706768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7b50a717-bc0c-4f1b-aec9-dc378553a0f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:35.716748 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7b50a717-bc0c-4f1b-aec9-dc378553a0f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 0.486157 Sep 27 10:01:35.721754 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 587/1328] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:35 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:35.828476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bca60f8b-612f-4459-b771-2ea197d34d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:35.849810 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bca60f8b-612f-4459-b771-2ea197d34d71 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1211 microversion: 2.1 time: 0.268324 Sep 27 10:01:35.861032 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 742/1329] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:35 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1211 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:36.028397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f0509e6f-3b46-479f-b797-5684fc56639d req-96d6fba8-919e-49df-b297-4c49cb4a1a58 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0fa79578-f299-4e5d-8dea-8bd1ca1c31d1", "tag": "ffaeeb25-b086-4ac5-b059-d12a6497adf1"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:36.038771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=73,nova_api:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:36.055089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f0509e6f-3b46-479f-b797-5684fc56639d req-96d6fba8-919e-49df-b297-4c49cb4a1a58 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:36.055518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f0509e6f-3b46-479f-b797-5684fc56639d req-96d6fba8-919e-49df-b297-4c49cb4a1a58 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:36.055939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f0509e6f-3b46-479f-b797-5684fc56639d req-96d6fba8-919e-49df-b297-4c49cb4a1a58 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:36.100076 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f0509e6f-3b46-479f-b797-5684fc56639d req-96d6fba8-919e-49df-b297-4c49cb4a1a58 service nova] Creating event network-changed:ffaeeb25-b086-4ac5-b059-d12a6497adf1 for instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 on nch2-master-1 Sep 27 10:01:36.106032 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f0509e6f-3b46-479f-b797-5684fc56639d req-96d6fba8-919e-49df-b297-4c49cb4a1a58 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083309 Sep 27 10:01:36.107114 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 588/1330] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:36.594582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eb3f6d0e-762e-479e-995a-2db5d3ab2034 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c2cc430>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:36.660618 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eb3f6d0e-762e-479e-995a-2db5d3ab2034 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.070274 Sep 27 10:01:36.661900 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 743/1331] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:36 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:36.750502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-63788549-bab1-425f-b90a-9b101e1e71e0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:36.752300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-63788549-bab1-425f-b90a-9b101e1e71e0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:36.769333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63788549-bab1-425f-b90a-9b101e1e71e0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:36.770806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63788549-bab1-425f-b90a-9b101e1e71e0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:36.771481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63788549-bab1-425f-b90a-9b101e1e71e0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:36.773001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63788549-bab1-425f-b90a-9b101e1e71e0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:36.773853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63788549-bab1-425f-b90a-9b101e1e71e0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:36.774556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63788549-bab1-425f-b90a-9b101e1e71e0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:36.885793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0fe49b24-00f7-403a-b1f2-535181a8bc56 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:36.888919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0fe49b24-00f7-403a-b1f2-535181a8bc56 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:36.904085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fe49b24-00f7-403a-b1f2-535181a8bc56 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:36.905175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fe49b24-00f7-403a-b1f2-535181a8bc56 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:36.906231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fe49b24-00f7-403a-b1f2-535181a8bc56 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:36.907556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fe49b24-00f7-403a-b1f2-535181a8bc56 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:36.908175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fe49b24-00f7-403a-b1f2-535181a8bc56 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:36.908902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fe49b24-00f7-403a-b1f2-535181a8bc56 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:37.137612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0fe49b24-00f7-403a-b1f2-535181a8bc56 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:37.175177 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0fe49b24-00f7-403a-b1f2-535181a8bc56 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1215 microversion: 2.1 time: 0.305175 Sep 27 10:01:37.176112 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 744/1332] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:36 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1215 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:37.351849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-63788549-bab1-425f-b90a-9b101e1e71e0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:37.360800 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-63788549-bab1-425f-b90a-9b101e1e71e0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 0.616664 Sep 27 10:01:37.361316 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 589/1333] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:36 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:37.678785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3bce8f46-fd0e-4b91-ae9b-f049f0faa207 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c2cc3a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:37.765667 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3bce8f46-fd0e-4b91-ae9b-f049f0faa207 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.092328 Sep 27 10:01:37.766515 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 745/1334] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:37 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:37.921113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-145147020", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "43a35817-b5c9-4e15-8250-6b84b0c2b25f"}], "security_groups": [{"name": "tempest-securitygroup--881376963"}], "key_name": "tempest-keypair-270589461", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:38.033325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9fbdeffc-ec06-43e7-8610-89f2f6f578bf tempest-InstanceUsageAuditLogTestJSON-892603333 tempest-InstanceUsageAuditLogTestJSON-892603333-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:38.071429 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9fbdeffc-ec06-43e7-8610-89f2f6f578bf tempest-InstanceUsageAuditLogTestJSON-892603333 tempest-InstanceUsageAuditLogTestJSON-892603333-project-admin] 10.222.0.26 "GET /compute/v2.1/os-instance_usage_audit_log/2022-09-27 10:01:37" status: 200 len: 318 microversion: 2.1 time: 0.267332 Sep 27 10:01:38.072431 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 590/1335] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:01:37 2022] GET /compute/v2.1/os-instance_usage_audit_log/2022-09-27%2010%3A01%3A37 => generated 318 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:38.101797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-64f20eab-c0f9-428a-b9bb-dd441a9926bf tempest-InstanceUsageAuditLogTestJSON-892603333 tempest-InstanceUsageAuditLogTestJSON-892603333-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:38.121973 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-64f20eab-c0f9-428a-b9bb-dd441a9926bf tempest-InstanceUsageAuditLogTestJSON-892603333 tempest-InstanceUsageAuditLogTestJSON-892603333-project-admin] 10.222.0.26 "GET /compute/v2.1/os-instance_usage_audit_log" status: 200 len: 319 microversion: 2.1 time: 0.026772 Sep 27 10:01:38.133180 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 591/1336] 10.222.0.26 () {58 vars in 1185 bytes} [Tue Sep 27 10:01:38 2022] GET /compute/v2.1/os-instance_usage_audit_log => generated 319 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:38.147187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] validate_networks() for [('43a35817-b5c9-4e15-8250-6b84b0c2b25f', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:01:38.205512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-70ca0bb4-2e69-435e-8ba6-b04da0d74a86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:38.206346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-70ca0bb4-2e69-435e-8ba6-b04da0d74a86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:38.239415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ca0bb4-2e69-435e-8ba6-b04da0d74a86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:38.240676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ca0bb4-2e69-435e-8ba6-b04da0d74a86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:38.241816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ca0bb4-2e69-435e-8ba6-b04da0d74a86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:38.245710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ca0bb4-2e69-435e-8ba6-b04da0d74a86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:38.263529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ca0bb4-2e69-435e-8ba6-b04da0d74a86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:38.266396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ca0bb4-2e69-435e-8ba6-b04da0d74a86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:38.679675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-70ca0bb4-2e69-435e-8ba6-b04da0d74a86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:38.695089 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-70ca0bb4-2e69-435e-8ba6-b04da0d74a86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1271 microversion: 2.1 time: 0.494999 Sep 27 10:01:38.696377 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 592/1337] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:38 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1271 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:38.705195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2beec02d-c3a8-485e-8c2a-f2b09c13027c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:38.706148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2beec02d-c3a8-485e-8c2a-f2b09c13027c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:38.730573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2beec02d-c3a8-485e-8c2a-f2b09c13027c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:38.734070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2beec02d-c3a8-485e-8c2a-f2b09c13027c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:38.735141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2beec02d-c3a8-485e-8c2a-f2b09c13027c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:38.743163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2beec02d-c3a8-485e-8c2a-f2b09c13027c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:38.743706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2beec02d-c3a8-485e-8c2a-f2b09c13027c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:38.744157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2beec02d-c3a8-485e-8c2a-f2b09c13027c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:39.685636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:01:39.686197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:01:39.686485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:01:39.687185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:01:39.687802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:01:39.707345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:39.724378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:39.725293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:39.726975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:39.731118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:39.734262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:39.809274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Getting quotas for project 2369f5ee467643948f618bbdee837521. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:01:39.823041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Getting quotas for user 29d2f7b0e8274c769521d13930be9825 and project 2369f5ee467643948f618bbdee837521. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:01:39.862429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:01:39.869592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:01:39.892627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2beec02d-c3a8-485e-8c2a-f2b09c13027c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:39.896746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:39.906382 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2beec02d-c3a8-485e-8c2a-f2b09c13027c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 1.208891 Sep 27 10:01:39.913201 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 593/1338] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:38 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 1216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:39.929629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-01f97629-d352-4358-96a0-1639fcf23833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f636d2dc0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:40.006272 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-94acc520-b607-40f9-a749-386862b05b26 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.25 time: 2.090364 Sep 27 10:01:40.010987 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 746/1339] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:01:37 2022] POST /compute/v2.1/servers => generated 405 bytes in 2096 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:01:40.021138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c755b597-b206-45ae-929e-d8a7954ccfcf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:40.025159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c755b597-b206-45ae-929e-d8a7954ccfcf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:40.030729 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-01f97629-d352-4358-96a0-1639fcf23833 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.110255 Sep 27 10:01:40.031581 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 594/1340] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:39 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:40.043904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c755b597-b206-45ae-929e-d8a7954ccfcf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:40.045223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c755b597-b206-45ae-929e-d8a7954ccfcf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:40.046293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d74b670c-24bc-4917-ad00-fbc55ee35c64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:40.047005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c755b597-b206-45ae-929e-d8a7954ccfcf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:40.048495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d74b670c-24bc-4917-ad00-fbc55ee35c64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:40.049947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c755b597-b206-45ae-929e-d8a7954ccfcf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:40.052173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c755b597-b206-45ae-929e-d8a7954ccfcf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:40.052933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c755b597-b206-45ae-929e-d8a7954ccfcf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:40.185726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c755b597-b206-45ae-929e-d8a7954ccfcf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:40.197779 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c755b597-b206-45ae-929e-d8a7954ccfcf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1269 microversion: 2.1 time: 0.185426 Sep 27 10:01:40.202416 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 747/1341] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:40 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1269 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:40.509430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d74b670c-24bc-4917-ad00-fbc55ee35c64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:40.522905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d74b670c-24bc-4917-ad00-fbc55ee35c64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:40.524415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d74b670c-24bc-4917-ad00-fbc55ee35c64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:40.528028 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d74b670c-24bc-4917-ad00-fbc55ee35c64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1284 microversion: 2.25 time: 0.495018 Sep 27 10:01:40.528792 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 595/1342] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:40 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1284 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:40.941897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-91d88bb6-d6ff-4ae2-8cb8-6d456f540c65 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:40.942605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-91d88bb6-d6ff-4ae2-8cb8-6d456f540c65 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:40.970497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91d88bb6-d6ff-4ae2-8cb8-6d456f540c65 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:40.970998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91d88bb6-d6ff-4ae2-8cb8-6d456f540c65 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:40.971439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91d88bb6-d6ff-4ae2-8cb8-6d456f540c65 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:40.972161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91d88bb6-d6ff-4ae2-8cb8-6d456f540c65 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:40.972611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91d88bb6-d6ff-4ae2-8cb8-6d456f540c65 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:40.976024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91d88bb6-d6ff-4ae2-8cb8-6d456f540c65 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:41.047334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4ffd38d1-cd39-4e5a-8b2d-0dc9fd9e3183 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f636d2af0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:41.134866 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4ffd38d1-cd39-4e5a-8b2d-0dc9fd9e3183 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.092657 Sep 27 10:01:41.135576 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 596/1343] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:41 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:01:41.226020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c538062d-e9fd-4810-bac1-693c19a43e78 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:41.226820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c538062d-e9fd-4810-bac1-693c19a43e78 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:41.236909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c538062d-e9fd-4810-bac1-693c19a43e78 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:41.246833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c538062d-e9fd-4810-bac1-693c19a43e78 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:41.249020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c538062d-e9fd-4810-bac1-693c19a43e78 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:41.250061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c538062d-e9fd-4810-bac1-693c19a43e78 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:41.251897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c538062d-e9fd-4810-bac1-693c19a43e78 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:41.253864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c538062d-e9fd-4810-bac1-693c19a43e78 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:41.326472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-91d88bb6-d6ff-4ae2-8cb8-6d456f540c65 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:41.341223 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-91d88bb6-d6ff-4ae2-8cb8-6d456f540c65 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 0.404377 Sep 27 10:01:41.344080 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 748/1344] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:40 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:41.371391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-bd6c1df0-9e8d-4253-a617-1e9bddc5731d req-bc02d1f8-8efa-405b-a593-d1f1689d18d6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1b526682-dfde-459f-bc61-2df38e0ec4b7", "name": "network-vif-plugged", "status": "completed", "tag": "bdcd738d-10f6-4db4-ab57-e00ef79f23bb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:41.408167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bd6c1df0-9e8d-4253-a617-1e9bddc5731d req-bc02d1f8-8efa-405b-a593-d1f1689d18d6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:41.408913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bd6c1df0-9e8d-4253-a617-1e9bddc5731d req-bc02d1f8-8efa-405b-a593-d1f1689d18d6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:41.410118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bd6c1df0-9e8d-4253-a617-1e9bddc5731d req-bc02d1f8-8efa-405b-a593-d1f1689d18d6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:41.425140 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-bd6c1df0-9e8d-4253-a617-1e9bddc5731d req-bc02d1f8-8efa-405b-a593-d1f1689d18d6 service nova] Creating event network-vif-plugged:bdcd738d-10f6-4db4-ab57-e00ef79f23bb for instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 on nch1-master-1 Sep 27 10:01:41.433316 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-bd6c1df0-9e8d-4253-a617-1e9bddc5731d req-bc02d1f8-8efa-405b-a593-d1f1689d18d6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065445 Sep 27 10:01:41.435352 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 749/1345] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:41.566044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:41.566759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:41.594778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c538062d-e9fd-4810-bac1-693c19a43e78 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:41.602756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:41.607839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:41.608149 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c538062d-e9fd-4810-bac1-693c19a43e78 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1269 microversion: 2.1 time: 0.385468 Sep 27 10:01:41.609614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:41.614578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:41.615383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:41.616362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:41.625415 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 597/1346] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:41 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1269 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:01:41.807246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:41.837769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:41.839350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:41.840556 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9b229f88-29a6-40ad-9f77-87dc8e1649e6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1280 microversion: 2.25 time: 0.281725 Sep 27 10:01:41.864726 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 750/1347] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:41 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1280 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:42.163349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c0dd0f16-c037-403d-ac89-ef365c2eeeb5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6366d3a0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:42.277847 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c0dd0f16-c037-403d-ac89-ef365c2eeeb5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.126511 Sep 27 10:01:42.278950 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 598/1348] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:42 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:42.372566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1408041e-ada0-4e49-b9c0-43886d15c063 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:42.373992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1408041e-ada0-4e49-b9c0-43886d15c063 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:42.397031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1408041e-ada0-4e49-b9c0-43886d15c063 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:42.399146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1408041e-ada0-4e49-b9c0-43886d15c063 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:42.399787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1408041e-ada0-4e49-b9c0-43886d15c063 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:42.401641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1408041e-ada0-4e49-b9c0-43886d15c063 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:42.402798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1408041e-ada0-4e49-b9c0-43886d15c063 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:42.404956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1408041e-ada0-4e49-b9c0-43886d15c063 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:42.638885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-266bad27-4dea-4edc-b242-c80e9d54d8ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:42.639505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-266bad27-4dea-4edc-b242-c80e9d54d8ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:42.656435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266bad27-4dea-4edc-b242-c80e9d54d8ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:42.657045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266bad27-4dea-4edc-b242-c80e9d54d8ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:42.657507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266bad27-4dea-4edc-b242-c80e9d54d8ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:42.659304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266bad27-4dea-4edc-b242-c80e9d54d8ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:42.659774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266bad27-4dea-4edc-b242-c80e9d54d8ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:42.660215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266bad27-4dea-4edc-b242-c80e9d54d8ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:43.139899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1408041e-ada0-4e49-b9c0-43886d15c063 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:43.162787 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1408041e-ada0-4e49-b9c0-43886d15c063 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 0.795775 Sep 27 10:01:43.207151 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 751/1349] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:42 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 840 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:43.216329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:43.223991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:43.268807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:43.270454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:43.271124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:43.274360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:43.275063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:43.278115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:43.476468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-266bad27-4dea-4edc-b242-c80e9d54d8ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:43.495799 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-266bad27-4dea-4edc-b242-c80e9d54d8ca tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1311 microversion: 2.1 time: 0.853270 Sep 27 10:01:43.499722 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 599/1350] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:42 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1311 bytes in 866 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:01:43.505096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-191e9237-ba65-4927-9bcc-4d8642678bf3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f636221f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:43.596951 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-191e9237-ba65-4927-9bcc-4d8642678bf3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.091611 Sep 27 10:01:43.598335 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 600/1351] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:43 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:43.753144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:43.770591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:43.773183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:43.777283 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-668e2992-5530-47f7-a998-ac800361f78a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1342 microversion: 2.25 time: 0.567531 Sep 27 10:01:43.777891 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 752/1352] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:43 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1342 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:44.197089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-81e0d835-ba20-4b63-a54a-55e4487838b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:44.197884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-81e0d835-ba20-4b63-a54a-55e4487838b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:44.217500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81e0d835-ba20-4b63-a54a-55e4487838b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:44.220029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81e0d835-ba20-4b63-a54a-55e4487838b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:44.221132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81e0d835-ba20-4b63-a54a-55e4487838b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:44.224844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81e0d835-ba20-4b63-a54a-55e4487838b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:44.225926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81e0d835-ba20-4b63-a54a-55e4487838b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:44.227146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81e0d835-ba20-4b63-a54a-55e4487838b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:44.529503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4070cb5-4bd7-4a04-bb69-eea58489f1c8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:44.530442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4070cb5-4bd7-4a04-bb69-eea58489f1c8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:44.549966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4070cb5-4bd7-4a04-bb69-eea58489f1c8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:44.553019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4070cb5-4bd7-4a04-bb69-eea58489f1c8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:44.553720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4070cb5-4bd7-4a04-bb69-eea58489f1c8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:44.559605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4070cb5-4bd7-4a04-bb69-eea58489f1c8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:44.561257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4070cb5-4bd7-4a04-bb69-eea58489f1c8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:44.567581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4070cb5-4bd7-4a04-bb69-eea58489f1c8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:44.900904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-81e0d835-ba20-4b63-a54a-55e4487838b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:44.915773 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-81e0d835-ba20-4b63-a54a-55e4487838b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 0.722385 Sep 27 10:01:44.916684 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 601/1353] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:44 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 727 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:44.935853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe8fcf3e-e5d3-47e5-b51c-e46309a69df2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6360a5e0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:45.054823 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fe8fcf3e-e5d3-47e5-b51c-e46309a69df2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.136631 Sep 27 10:01:45.063615 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 602/1354] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:44 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:45.075981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:45.078618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:45.087251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:45.088039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:45.088842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:45.090101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:45.090717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:45.091322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:45.520481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:45.535210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:45.541776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:45.545563 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-84584899-81e3-4ee7-83a4-e8fa9c19ffd9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1409 microversion: 2.25 time: 0.479034 Sep 27 10:01:45.553752 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 603/1355] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:45 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1409 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:45.659230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4070cb5-4bd7-4a04-bb69-eea58489f1c8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:45.731956 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4070cb5-4bd7-4a04-bb69-eea58489f1c8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1311 microversion: 2.1 time: 1.200801 Sep 27 10:01:45.732795 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 753/1356] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:44 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1311 bytes in 1212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:01:45.941506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-075520f9-4bbd-4871-8453-d6ac107d85df tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:45.942308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-075520f9-4bbd-4871-8453-d6ac107d85df tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:45.953952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-075520f9-4bbd-4871-8453-d6ac107d85df tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:45.954489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-075520f9-4bbd-4871-8453-d6ac107d85df tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:45.954941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-075520f9-4bbd-4871-8453-d6ac107d85df tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:45.955665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-075520f9-4bbd-4871-8453-d6ac107d85df tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:45.959227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-075520f9-4bbd-4871-8453-d6ac107d85df tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:45.959925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-075520f9-4bbd-4871-8453-d6ac107d85df tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:46.071498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dee85d77-d32e-4d4f-afdd-189fbe7de047 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f68216820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:46.191714 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dee85d77-d32e-4d4f-afdd-189fbe7de047 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.124472 Sep 27 10:01:46.194629 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 754/1357] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:46 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:01:46.475356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-075520f9-4bbd-4871-8453-d6ac107d85df tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:46.490022 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-075520f9-4bbd-4871-8453-d6ac107d85df tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1504 microversion: 2.1 time: 0.552799 Sep 27 10:01:46.490952 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 604/1358] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:45 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1504 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:46.585786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:46.586702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:46.615915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:46.617761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:46.618449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:46.619402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:46.621104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:46.625336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:46.671269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:46.755400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-428ef01f-45f6-485c-98f8-b65e4ea80dec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:46.756125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-428ef01f-45f6-485c-98f8-b65e4ea80dec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:46.766357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-428ef01f-45f6-485c-98f8-b65e4ea80dec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:46.769228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-428ef01f-45f6-485c-98f8-b65e4ea80dec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:46.770995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-428ef01f-45f6-485c-98f8-b65e4ea80dec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:46.772283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-428ef01f-45f6-485c-98f8-b65e4ea80dec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:46.778602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-428ef01f-45f6-485c-98f8-b65e4ea80dec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:46.779425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-428ef01f-45f6-485c-98f8-b65e4ea80dec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:47.826886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-428ef01f-45f6-485c-98f8-b65e4ea80dec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:47.840894 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-428ef01f-45f6-485c-98f8-b65e4ea80dec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1311 microversion: 2.1 time: 1.089525 Sep 27 10:01:47.844686 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 605/1359] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:46 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1311 bytes in 1094 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:47.860520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1d9eabbe-d2c8-48e8-bb9d-f1ca6727bf21 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f636f9940>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:47.943210 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1d9eabbe-d2c8-48e8-bb9d-f1ca6727bf21 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.095022 Sep 27 10:01:47.943812 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 606/1360] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:47 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:47.954578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-89eb4dac-cb9a-4216-a144-1dafec86edbd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:47.955744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-89eb4dac-cb9a-4216-a144-1dafec86edbd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:47.969367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eb4dac-cb9a-4216-a144-1dafec86edbd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:47.974249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eb4dac-cb9a-4216-a144-1dafec86edbd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:47.974715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eb4dac-cb9a-4216-a144-1dafec86edbd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:47.976552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eb4dac-cb9a-4216-a144-1dafec86edbd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:47.977095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eb4dac-cb9a-4216-a144-1dafec86edbd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:47.981496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eb4dac-cb9a-4216-a144-1dafec86edbd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:48.002967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:48.037304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:48.053280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:48.055281 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c157ad84-deba-44f0-982d-9974fad7a240 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1409 microversion: 2.25 time: 1.474102 Sep 27 10:01:48.055880 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 755/1361] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:46 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1409 bytes in 1475 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:48.472051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-aa09db94-31cb-436f-b600-99e84427ebc5 req-ab6bdeea-e89f-4889-8ee1-8400ac6aa1e1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9a2aa469-5c5e-4a03-b09e-8e6b2252d615", "tag": "4c0e9dc5-844c-484c-99a2-79cf10f253fe"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:48.509251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-aa09db94-31cb-436f-b600-99e84427ebc5 req-ab6bdeea-e89f-4889-8ee1-8400ac6aa1e1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:48.517521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-aa09db94-31cb-436f-b600-99e84427ebc5 req-ab6bdeea-e89f-4889-8ee1-8400ac6aa1e1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:48.519495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-aa09db94-31cb-436f-b600-99e84427ebc5 req-ab6bdeea-e89f-4889-8ee1-8400ac6aa1e1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:48.550219 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-aa09db94-31cb-436f-b600-99e84427ebc5 req-ab6bdeea-e89f-4889-8ee1-8400ac6aa1e1 service nova] Creating event network-changed:4c0e9dc5-844c-484c-99a2-79cf10f253fe for instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 on nch2-master-1 Sep 27 10:01:48.569487 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-aa09db94-31cb-436f-b600-99e84427ebc5 req-ab6bdeea-e89f-4889-8ee1-8400ac6aa1e1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.101322 Sep 27 10:01:48.575396 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 756/1362] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:48.937776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a1fcce8c-d0c9-4fcf-a049-b777c2b0aebe tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Action: 'update', calling method: .version_select at 0x7f7f63830dc0>, body: {"quota_set": {"ram": "5120"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:49.016905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-89eb4dac-cb9a-4216-a144-1dafec86edbd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:49.036498 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-89eb4dac-cb9a-4216-a144-1dafec86edbd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1479 microversion: 2.1 time: 1.091422 Sep 27 10:01:49.041862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fcce8c-d0c9-4fcf-a049-b777c2b0aebe tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:49.049521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fcce8c-d0c9-4fcf-a049-b777c2b0aebe tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:49.051273 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 607/1363] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:47 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1479 bytes in 1107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:49.052025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fcce8c-d0c9-4fcf-a049-b777c2b0aebe tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:49.058241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fcce8c-d0c9-4fcf-a049-b777c2b0aebe tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:49.065489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fcce8c-d0c9-4fcf-a049-b777c2b0aebe tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:49.071051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fcce8c-d0c9-4fcf-a049-b777c2b0aebe tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:49.085507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e66196ff-9c58-4af0-8bff-5bfd7c215203 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:49.086587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e66196ff-9c58-4af0-8bff-5bfd7c215203 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:49.100636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66196ff-9c58-4af0-8bff-5bfd7c215203 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:49.109500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66196ff-9c58-4af0-8bff-5bfd7c215203 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:49.111707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66196ff-9c58-4af0-8bff-5bfd7c215203 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:49.113234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66196ff-9c58-4af0-8bff-5bfd7c215203 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:49.116273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66196ff-9c58-4af0-8bff-5bfd7c215203 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:49.117237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66196ff-9c58-4af0-8bff-5bfd7c215203 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:49.260517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:49.270934 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a1fcce8c-d0c9-4fcf-a049-b777c2b0aebe tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/e558ccda20e64593971db841548cc33d" status: 200 len: 240 microversion: 2.36 time: 0.615138 Sep 27 10:01:49.272694 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 757/1364] 10.222.0.26 () {62 vars in 1303 bytes} [Tue Sep 27 10:01:48 2022] PUT /compute/v2.1/os-quota-sets/e558ccda20e64593971db841548cc33d => generated 240 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:01:49.297680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-88ca58bd-5ce5-4f9e-9a07-7253a8adc861 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6392bf70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:49.387180 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-88ca58bd-5ce5-4f9e-9a07-7253a8adc861 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.112767 Sep 27 10:01:49.388188 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 758/1365] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:49 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:49.394318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fd9b4572-87fe-406d-98f5-1625caa200a9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '.version_select at 0x7f7f636d0790>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:49.398850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd9b4572-87fe-406d-98f5-1625caa200a9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:49.399658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd9b4572-87fe-406d-98f5-1625caa200a9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:49.400332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd9b4572-87fe-406d-98f5-1625caa200a9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:49.414313 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fd9b4572-87fe-406d-98f5-1625caa200a9 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/os-migrations" status: 200 len: 2730 microversion: 2.1 time: 0.024355 Sep 27 10:01:49.415878 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 759/1366] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 10:01:49 2022] GET /compute/v2.1/os-migrations => generated 2730 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:49.431063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:49.432273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:49.456296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:49.457418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:49.458210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:49.459378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:49.461134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:49.462580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:50.101074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:50.114327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:50.118676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:50.122655 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58d52e96-3e6f-4046-9aea-c7f261ce59bb tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1409 microversion: 2.25 time: 0.704678 Sep 27 10:01:50.124159 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 760/1367] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:49 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1409 bytes in 708 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:50.141189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f139f412-6390-45e4-ba4a-d48def712f6e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Calling method '.version_select at 0x7f7f636d0040>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:50.253193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f139f412-6390-45e4-ba4a-d48def712f6e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:50.253941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f139f412-6390-45e4-ba4a-d48def712f6e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:50.254474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f139f412-6390-45e4-ba4a-d48def712f6e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:50.255319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f139f412-6390-45e4-ba4a-d48def712f6e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:50.255884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f139f412-6390-45e4-ba4a-d48def712f6e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:50.256408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f139f412-6390-45e4-ba4a-d48def712f6e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:50.303686 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f139f412-6390-45e4-ba4a-d48def712f6e tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/e558ccda20e64593971db841548cc33d/detail" status: 200 len: 672 microversion: 2.36 time: 0.170170 Sep 27 10:01:50.305319 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 761/1368] 10.222.0.26 () {60 vars in 1304 bytes} [Tue Sep 27 10:01:50 2022] GET /compute/v2.1/os-quota-sets/e558ccda20e64593971db841548cc33d/detail => generated 672 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:01:50.322992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e36643e9-40e9-44e9-9b80-39939dc1d902 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:50.324618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e36643e9-40e9-44e9-9b80-39939dc1d902 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:50.345534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e36643e9-40e9-44e9-9b80-39939dc1d902 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:50.347910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e36643e9-40e9-44e9-9b80-39939dc1d902 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:50.349924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e36643e9-40e9-44e9-9b80-39939dc1d902 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:50.353911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e36643e9-40e9-44e9-9b80-39939dc1d902 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:50.358206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e36643e9-40e9-44e9-9b80-39939dc1d902 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:50.359223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e36643e9-40e9-44e9-9b80-39939dc1d902 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:50.513407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=14 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:50.666196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e66196ff-9c58-4af0-8bff-5bfd7c215203 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:50.677355 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e66196ff-9c58-4af0-8bff-5bfd7c215203 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1311 microversion: 2.1 time: 1.623758 Sep 27 10:01:50.699683 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 608/1369] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:49 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1311 bytes in 1647 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:50.844127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-9bff2555-8c3c-47c2-92f2-3ea052ac512d req-bf0f797c-644b-4249-84f5-14c909b9221c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fcd14928-8ef7-425b-89bb-2c714284e840", "tag": "b4c8bc77-41d3-49f6-84f2-59d40e203a22"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:50.884720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9bff2555-8c3c-47c2-92f2-3ea052ac512d req-bf0f797c-644b-4249-84f5-14c909b9221c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:50.885248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9bff2555-8c3c-47c2-92f2-3ea052ac512d req-bf0f797c-644b-4249-84f5-14c909b9221c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:50.885674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9bff2555-8c3c-47c2-92f2-3ea052ac512d req-bf0f797c-644b-4249-84f5-14c909b9221c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:50.927091 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-9bff2555-8c3c-47c2-92f2-3ea052ac512d req-bf0f797c-644b-4249-84f5-14c909b9221c service nova] Creating event network-changed:b4c8bc77-41d3-49f6-84f2-59d40e203a22 for instance fcd14928-8ef7-425b-89bb-2c714284e840 on nch2-master-1 Sep 27 10:01:50.941562 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-9bff2555-8c3c-47c2-92f2-3ea052ac512d req-bf0f797c-644b-4249-84f5-14c909b9221c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.240433 Sep 27 10:01:50.957063 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 609/1370] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:01:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:50.963862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-17c75bc6-1226-4735-88dd-2909be022d2b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63637ca0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:51.067363 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e36643e9-40e9-44e9-9b80-39939dc1d902 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1630 microversion: 2.1 time: 0.756346 Sep 27 10:01:51.068181 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 762/1371] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:50 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1630 bytes in 762 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:51.076866 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-17c75bc6-1226-4735-88dd-2909be022d2b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.118081 Sep 27 10:01:51.077453 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 610/1372] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:50 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:51.090517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Action: 'update', calling method: .version_select at 0x7f7f636d9430>, body: {"quota_set": {"ram": "2048"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:51.102862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e0c4fd0b-eb50-4393-b8a3-591e62863289 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Action: 'create', calling method: .version_select at 0x7f7f6377b1f0>, body: {"keypair": {"name": "tempest-keypair-1620198310", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ+Uu/xkYCzOV8/zcHH8L7MVokN7SSyR0KkGqIQvKYmOZWhSHt5IAHqz3hK0u7qvVZfGaIDjwr1HRDkv63z8pkypJAR1JB++hiY6rTDvRaYS5RrUjO62nhXHBbzN2rG8Bw=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:51.196565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-e0c4fd0b-eb50-4393-b8a3-591e62863289 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Getting quotas for project ec0980b5824b425fbad9b8554cf5c42c. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:01:51.235362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-e0c4fd0b-eb50-4393-b8a3-591e62863289 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Getting quotas for user dce01d3212934161a38be853b4cd93ae and project ec0980b5824b425fbad9b8554cf5c42c. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:01:51.238487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:51.239711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:51.240494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:51.241723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:51.242501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:51.243303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:51.329112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:51.329876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:51.333016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:51.344921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:51.347127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:51.347550 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e0c4fd0b-eb50-4393-b8a3-591e62863289 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.249614 Sep 27 10:01:51.361744 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 611/1373] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:01:51 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:01:51.362598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:51.366927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:51.367675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:51.399314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:51.399816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:51.400260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:51.409171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:51.409709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:51.410165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:51.622559 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c5392dc3-fe4e-4fbc-9102-8110eefa0635 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/e558ccda20e64593971db841548cc33d?user_id=6aeeba5aa2744622abfbe9e2e234f580" status: 200 len: 240 microversion: 2.36 time: 0.549100 Sep 27 10:01:51.630531 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 763/1374] 10.222.0.26 () {62 vars in 1384 bytes} [Tue Sep 27 10:01:51 2022] PUT /compute/v2.1/os-quota-sets/e558ccda20e64593971db841548cc33d?user_id=6aeeba5aa2744622abfbe9e2e234f580 => generated 240 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:01:51.637238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1f6faf2e-311b-44f8-99ad-4da7ef3919e2 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] Calling method '.version_select at 0x7f7f636d04c0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:51.804999 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1f6faf2e-311b-44f8-99ad-4da7ef3919e2 tempest-QuotasAdminTestV236-721261246 tempest-QuotasAdminTestV236-721261246-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/e558ccda20e64593971db841548cc33d?user_id=6aeeba5aa2744622abfbe9e2e234f580" status: 200 len: 282 microversion: 2.36 time: 0.171774 Sep 27 10:01:51.806548 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 764/1375] 10.222.0.26 () {60 vars in 1364 bytes} [Tue Sep 27 10:01:51 2022] GET /compute/v2.1/os-quota-sets/e558ccda20e64593971db841548cc33d?user_id=6aeeba5aa2744622abfbe9e2e234f580 => generated 282 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:01:51.850407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1334686a-b6af-4c99-a88e-de33bc1bad03 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:51.852071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1334686a-b6af-4c99-a88e-de33bc1bad03 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:51.877717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1334686a-b6af-4c99-a88e-de33bc1bad03 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:51.885510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1334686a-b6af-4c99-a88e-de33bc1bad03 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:51.888037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1334686a-b6af-4c99-a88e-de33bc1bad03 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:51.893230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1334686a-b6af-4c99-a88e-de33bc1bad03 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:51.893955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1334686a-b6af-4c99-a88e-de33bc1bad03 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:51.894562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1334686a-b6af-4c99-a88e-de33bc1bad03 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:52.715228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1334686a-b6af-4c99-a88e-de33bc1bad03 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:52.728005 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1334686a-b6af-4c99-a88e-de33bc1bad03 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1311 microversion: 2.1 time: 0.894932 Sep 27 10:01:52.745400 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 765/1376] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:51 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1311 bytes in 913 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:52.759794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eaf5ed48-1d70-4c6b-91ad-d90d79043bef tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c2b4430>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:52.909443 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eaf5ed48-1d70-4c6b-91ad-d90d79043bef tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.162409 Sep 27 10:01:52.914657 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 766/1377] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:52 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:53.416341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:53.434714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:53.442098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:53.444155 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d44cf66c-bbbb-40a7-9e1b-f389051c3e1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1409 microversion: 2.25 time: 2.081269 Sep 27 10:01:53.457682 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 612/1378] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:51 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1409 bytes in 2095 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:53.755010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b57ef42c-b1e4-438f-85f0-7c2c41303e4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:53.755643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b57ef42c-b1e4-438f-85f0-7c2c41303e4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:53.770400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b57ef42c-b1e4-438f-85f0-7c2c41303e4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:53.778460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b57ef42c-b1e4-438f-85f0-7c2c41303e4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:53.780050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b57ef42c-b1e4-438f-85f0-7c2c41303e4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:53.782132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b57ef42c-b1e4-438f-85f0-7c2c41303e4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:53.787808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b57ef42c-b1e4-438f-85f0-7c2c41303e4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:53.788630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b57ef42c-b1e4-438f-85f0-7c2c41303e4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:53.934907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bad65e31-2999-4a5e-8c9b-641670242f01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63674ee0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:54.053527 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bad65e31-2999-4a5e-8c9b-641670242f01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.122805 Sep 27 10:01:54.054228 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 613/1379] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:53 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:54.434478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b57ef42c-b1e4-438f-85f0-7c2c41303e4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:54.451386 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b57ef42c-b1e4-438f-85f0-7c2c41303e4b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1311 microversion: 2.1 time: 0.703138 Sep 27 10:01:54.453169 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 767/1380] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:53 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1311 bytes in 706 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:54.475414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:54.476776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:54.485011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:54.485747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:54.486441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:54.487342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:54.487984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:54.488693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:55.079790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06e90324-18c2-4e64-9979-f67a6384b144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63ae11f0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:55.202728 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-06e90324-18c2-4e64-9979-f67a6384b144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.134102 Sep 27 10:01:55.203348 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 768/1381] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:55 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:55.250697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:55.267612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:55.273593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:55.277811 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ec2a4fd0-2376-416e-8d65-5b481c4bc6e5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1409 microversion: 2.25 time: 0.807402 Sep 27 10:01:55.278818 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 614/1382] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:54 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1409 bytes in 809 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:55.490287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e96c3b12-0d7b-4b09-a36d-2a00342c85eb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:55.492278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e96c3b12-0d7b-4b09-a36d-2a00342c85eb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:55.509537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96c3b12-0d7b-4b09-a36d-2a00342c85eb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:55.512070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96c3b12-0d7b-4b09-a36d-2a00342c85eb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:55.512802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96c3b12-0d7b-4b09-a36d-2a00342c85eb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:55.514405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96c3b12-0d7b-4b09-a36d-2a00342c85eb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:55.516586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96c3b12-0d7b-4b09-a36d-2a00342c85eb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:55.521478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96c3b12-0d7b-4b09-a36d-2a00342c85eb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:55.696536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8bf3e7d-3388-4a03-903a-0a0302763c98 tempest-FloatingIPDetailsNegativeTestJSON-703804521 tempest-FloatingIPDetailsNegativeTestJSON-703804521-project] Calling method '.version_select at 0x7f7f6c2cc5e0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:55.877687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-f8bf3e7d-3388-4a03-903a-0a0302763c98 tempest-FloatingIPDetailsNegativeTestJSON-703804521 tempest-FloatingIPDetailsNegativeTestJSON-703804521-project] Error message: {"NeutronError": {"type": "FloatingIPNotFound", "message": "Floating IP cff4ded2-0912-44bd-9215-dfd3f7b29e09 could not be found", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:01:55.878615 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-f8bf3e7d-3388-4a03-903a-0a0302763c98 tempest-FloatingIPDetailsNegativeTestJSON-703804521 tempest-FloatingIPDetailsNegativeTestJSON-703804521-project] HTTP exception thrown: Floating IP not found for ID cff4ded2-0912-44bd-9215-dfd3f7b29e09 Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8bf3e7d-3388-4a03-903a-0a0302763c98 tempest-FloatingIPDetailsNegativeTestJSON-703804521 tempest-FloatingIPDetailsNegativeTestJSON-703804521-project] Request method failure captured: Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-floating-ips/cff4ded2-0912-44bd-9215-dfd3f7b29e09 HTTP/1.1 Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsoTVKMeqlOZPOjuGNImUY7EXy-WZBPaXj-0GfeRVZC26t6di9ZukYN-40wf6S0mmQxFllNUKeH-yESaJ6uMw8PyUc2yk_-kLbUR0HXa8nXqg0XnvBIx5VvaFy2WqtXzgPjckoMRPXh0Z2hdROntb5G1PeAfkpo3IMbP5Kx56bjfL_s Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 4d37d16753ac4c68aa1f1fac7925d857 Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FloatingIPDetailsNegativeTestJSON-703804521 Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:01:55.881099 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4d37d16753ac4c68aa1f1fac7925d857"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4d37d16753ac4c68aa1f1fac7925d857"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/4d37d16753ac4c68aa1f1fac7925d857"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_4d37d16753ac4c68aa1f1fac7925d857"}]}] Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FloatingIPDetailsNegativeTestJSON-703804521 Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 4d37d16753ac4c68aa1f1fac7925d857 Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FloatingIPDetailsNegativeTestJSON-703804521 Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FloatingIPDetailsNegativeTestJSON-703804521-project Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6d6bc44a06384fa4bc0aba86ea50e52c Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FloatingIPDetailsNegativeTestJSON-703804521-project Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: exception: Floating IP not found for ID cff4ded2-0912-44bd-9215-dfd3f7b29e09 Sep 27 10:01:55.882652 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'cff4ded2-0912-44bd-9215-dfd3f7b29e09'} Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2824, in get_floating_ip Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi fip = client.show_floatingip(id)['floatingip'] Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1008, in show_floatingip Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.floatingip_path % (floatingip), params=_params) Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:55.883453 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Floating IP cff4ded2-0912-44bd-9215-dfd3f7b29e09 could not be found Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-5319f96e-9c46-4642-a047-4e5a6b0a21a4'] Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 94, in show Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi floating_ip = self.network_api.get_floating_ip(context, id) Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 2827, in get_floating_ip Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.FloatingIpNotFound(id=id) Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.FloatingIpNotFound: Floating IP not found for ID cff4ded2-0912-44bd-9215-dfd3f7b29e09. Sep 27 10:01:55.884823 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 97, in show Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=msg) Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Floating IP not found for ID cff4ded2-0912-44bd-9215-dfd3f7b29e09 Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:01:55.886038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:01:55.887375 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:01:55.887375 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP not found for ID cff4ded2-0912-44bd-9215-dfd3f7b29e09 Sep 27 10:01:55.887375 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:01:55.889611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8bf3e7d-3388-4a03-903a-0a0302763c98 tempest-FloatingIPDetailsNegativeTestJSON-703804521 tempest-FloatingIPDetailsNegativeTestJSON-703804521-project] Returning 404 to user: Floating IP not found for ID cff4ded2-0912-44bd-9215-dfd3f7b29e09 {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:01:55.890984 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f8bf3e7d-3388-4a03-903a-0a0302763c98 tempest-FloatingIPDetailsNegativeTestJSON-703804521 tempest-FloatingIPDetailsNegativeTestJSON-703804521-project] 10.222.0.26 "GET /compute/v2.1/os-floating-ips/cff4ded2-0912-44bd-9215-dfd3f7b29e09" status: 404 len: 111 microversion: 2.1 time: 0.550613 Sep 27 10:01:55.891815 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 769/1383] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:01:55 2022] GET /compute/v2.1/os-floating-ips/cff4ded2-0912-44bd-9215-dfd3f7b29e09 => generated 111 bytes in 552 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:01:55.988220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e96c3b12-0d7b-4b09-a36d-2a00342c85eb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:56.007188 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e96c3b12-0d7b-4b09-a36d-2a00342c85eb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1311 microversion: 2.1 time: 0.528786 Sep 27 10:01:56.012185 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 615/1384] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:55 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1311 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:56.220878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8fa46c0b-821a-4080-b74b-db8e92944771 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f636d09d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:56.301924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8fa46c0b-821a-4080-b74b-db8e92944771 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.086143 Sep 27 10:01:56.313566 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 770/1385] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:56 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:56.316504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:56.317787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:56.339273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:56.340328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:56.342092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:56.349483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:56.350471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:56.351204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:56.833882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-InstanceActionsV221TestJSON-server-1219204877", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "e1c686e1-e4ac-4dbd-8335-7bdcd7dc022b"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:56.959277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] validate_networks() for [('e1c686e1-e4ac-4dbd-8335-7bdcd7dc022b', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:01:56.980496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:56.994743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:56.999658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:57.001778 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-114e51ef-de62-40a4-8c71-31f81e37b441 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1409 microversion: 2.25 time: 0.697135 Sep 27 10:01:57.005419 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 616/1386] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:56 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1409 bytes in 702 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:01:57.038187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a05b7cc-dc58-45d8-8223-4b5e9a757d91 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:57.039534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a05b7cc-dc58-45d8-8223-4b5e9a757d91 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:57.056429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a05b7cc-dc58-45d8-8223-4b5e9a757d91 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:57.057265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a05b7cc-dc58-45d8-8223-4b5e9a757d91 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:57.058243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a05b7cc-dc58-45d8-8223-4b5e9a757d91 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:57.061835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a05b7cc-dc58-45d8-8223-4b5e9a757d91 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:57.063653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a05b7cc-dc58-45d8-8223-4b5e9a757d91 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:57.064346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a05b7cc-dc58-45d8-8223-4b5e9a757d91 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:58.088444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0a05b7cc-dc58-45d8-8223-4b5e9a757d91 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:01:58.099818 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a05b7cc-dc58-45d8-8223-4b5e9a757d91 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1311 microversion: 2.1 time: 1.065550 Sep 27 10:01:58.102929 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 617/1387] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:01:57 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1311 bytes in 1070 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:58.121198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2b01dc16-6b1f-4d56-978b-1a1e1c8bcf35 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6377be50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:58.184719 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2b01dc16-6b1f-4d56-978b-1a1e1c8bcf35 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.080112 Sep 27 10:01:58.186560 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 618/1388] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:01:58 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:01:58.193093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-854190203", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "66dfc7b2-4ec7-40ad-b073-f4df786a999e"}], "security_groups": [{"name": "tempest-securitygroup--1979732314"}], "key_name": "tempest-keypair-1620198310", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:01:58.491773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] validate_networks() for [('66dfc7b2-4ec7-40ad-b073-f4df786a999e', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:01:59.341366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:01:59.341871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:01:59.342265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:01:59.342682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:01:59.343097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:01:59.365045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=41 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:01:59.371400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:59.375542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:59.378090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:59.381908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:59.386029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:59.386858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:59.465243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Getting quotas for project 2750135860944b03816344b2e7e58f30. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:01:59.499192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Getting quotas for user b31b5932bb8e4edcadc6954c46c2c386 and project 2750135860944b03816344b2e7e58f30. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:01:59.578659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:01:59.587133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:01:59.667956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:01:59.668454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:01:59.669433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:01:59.670183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:01:59.670615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:01:59.705226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:59.705972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:59.706403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:59.714668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:59.715451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:59.716033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:59.745892 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5eb25f0c-3055-4480-a2f8-c983cb975393 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.21 time: 3.308706 Sep 27 10:01:59.758078 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 771/1389] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:01:56 2022] POST /compute/v2.1/servers => generated 380 bytes in 3322 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:01:59.767599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Getting quotas for project ec0980b5824b425fbad9b8554cf5c42c. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:01:59.769692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:01:59.778280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Getting quotas for user dce01d3212934161a38be853b4cd93ae and project ec0980b5824b425fbad9b8554cf5c42c. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:01:59.789148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:01:59.804103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:01:59.810374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:01:59.818634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:59.819139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:59.819703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:01:59.820451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:01:59.821045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:01:59.821482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:00.046682 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a23fde26-ca4b-4af9-b330-43785aec73f0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.858483 Sep 27 10:02:00.047948 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 619/1390] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:01:58 2022] POST /compute/v2.1/servers => generated 406 bytes in 1860 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:02:00.056148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4aad2b4e-ba60-420c-9d50-99152a85144e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:00.057184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4aad2b4e-ba60-420c-9d50-99152a85144e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:00.068936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aad2b4e-ba60-420c-9d50-99152a85144e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:00.069688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aad2b4e-ba60-420c-9d50-99152a85144e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:00.070669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aad2b4e-ba60-420c-9d50-99152a85144e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:00.072445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aad2b4e-ba60-420c-9d50-99152a85144e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:00.073282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aad2b4e-ba60-420c-9d50-99152a85144e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:00.073889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aad2b4e-ba60-420c-9d50-99152a85144e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:00.468305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4aad2b4e-ba60-420c-9d50-99152a85144e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:00.485475 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4aad2b4e-ba60-420c-9d50-99152a85144e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1311 microversion: 2.1 time: 0.435563 Sep 27 10:02:00.488568 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 620/1391] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:00 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1311 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:00.521369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-703ed030-61fc-4512-8594-f0da885744f6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63641670>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:00.669881 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-703ed030-61fc-4512-8594-f0da885744f6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.176612 Sep 27 10:02:00.670551 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 621/1392] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:00 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:00.688523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bb5badc4-083f-40ff-8e37-5c619ccb7b5d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:00.695497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bb5badc4-083f-40ff-8e37-5c619ccb7b5d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:00.858209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bb5badc4-083f-40ff-8e37-5c619ccb7b5d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:00.870404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bb5badc4-083f-40ff-8e37-5c619ccb7b5d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:00.872466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bb5badc4-083f-40ff-8e37-5c619ccb7b5d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:00.874115 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bb5badc4-083f-40ff-8e37-5c619ccb7b5d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1259 microversion: 2.21 time: 0.190440 Sep 27 10:02:00.876125 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 622/1393] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:00 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1259 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:00.909730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7c0e4757-7635-4a23-b69f-25229532e9bd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:00.912350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7c0e4757-7635-4a23-b69f-25229532e9bd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:00.928522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:00.956442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:00.980849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:00.982434 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ff16b703-264e-4f21-811e-b6fe48a19643 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1409 microversion: 2.25 time: 1.222897 Sep 27 10:02:00.985106 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 772/1394] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:01:59 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1409 bytes in 1225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:01.054974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7c0e4757-7635-4a23-b69f-25229532e9bd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:01.071694 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7c0e4757-7635-4a23-b69f-25229532e9bd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1234 microversion: 2.1 time: 0.179099 Sep 27 10:02:01.076712 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 623/1395] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:00 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1234 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:01.516164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3c0ef9e1-7bad-4c79-95ce-bc23d6a1961f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:01.522223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3c0ef9e1-7bad-4c79-95ce-bc23d6a1961f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:01.556250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c0ef9e1-7bad-4c79-95ce-bc23d6a1961f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:01.575825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c0ef9e1-7bad-4c79-95ce-bc23d6a1961f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:01.576621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c0ef9e1-7bad-4c79-95ce-bc23d6a1961f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:01.578618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c0ef9e1-7bad-4c79-95ce-bc23d6a1961f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:01.580292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c0ef9e1-7bad-4c79-95ce-bc23d6a1961f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:01.581918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c0ef9e1-7bad-4c79-95ce-bc23d6a1961f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:01.687085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-303c758b-9771-49fa-9cdc-6c9c426e58e2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6366d430>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:01.815585 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-303c758b-9771-49fa-9cdc-6c9c426e58e2 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.133467 Sep 27 10:02:01.816820 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 624/1396] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:01 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:01.911698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:01.912941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:01.932176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:01.934506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:01.945856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:01.954573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:01.957305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:01.959014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:02.280900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3c0ef9e1-7bad-4c79-95ce-bc23d6a1961f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:02.292501 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3c0ef9e1-7bad-4c79-95ce-bc23d6a1961f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1311 microversion: 2.1 time: 0.783935 Sep 27 10:02:02.293652 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 773/1397] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:01 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1311 bytes in 787 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:02.802714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e1351dea-572e-4a89-aac9-d0c50405284b req-18658921-5b2b-4fa9-8b48-91b75eb2991d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a2aa469-5c5e-4a03-b09e-8e6b2252d615", "name": "network-vif-plugged", "status": "completed", "tag": "4c0e9dc5-844c-484c-99a2-79cf10f253fe"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:02.868288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e1351dea-572e-4a89-aac9-d0c50405284b req-18658921-5b2b-4fa9-8b48-91b75eb2991d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:02.870000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e1351dea-572e-4a89-aac9-d0c50405284b req-18658921-5b2b-4fa9-8b48-91b75eb2991d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:02.871561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e1351dea-572e-4a89-aac9-d0c50405284b req-18658921-5b2b-4fa9-8b48-91b75eb2991d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:02.899637 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e1351dea-572e-4a89-aac9-d0c50405284b req-18658921-5b2b-4fa9-8b48-91b75eb2991d service nova] Creating event network-vif-plugged:4c0e9dc5-844c-484c-99a2-79cf10f253fe for instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 on nch2-master-1 Sep 27 10:02:02.913516 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e1351dea-572e-4a89-aac9-d0c50405284b req-18658921-5b2b-4fa9-8b48-91b75eb2991d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.618000 Sep 27 10:02:02.919352 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 774/1398] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 625 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:02.930181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:02.931904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:02.959923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:02.961221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:02.962200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:02.963477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:02.964414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:02.965208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:03.384961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:03.404688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:03.419085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:03.420937 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aef0ee2e-d77a-425e-83b3-d70299333958 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1409 microversion: 2.25 time: 0.499789 Sep 27 10:02:03.424549 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 775/1399] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:02 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1409 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:03.442710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2bf73f37-fcf9-4acc-8780-5a58f6d8453f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:03.455086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2bf73f37-fcf9-4acc-8780-5a58f6d8453f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:03.468158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bf73f37-fcf9-4acc-8780-5a58f6d8453f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:03.469236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bf73f37-fcf9-4acc-8780-5a58f6d8453f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:03.471865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bf73f37-fcf9-4acc-8780-5a58f6d8453f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:03.472832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bf73f37-fcf9-4acc-8780-5a58f6d8453f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:03.477730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bf73f37-fcf9-4acc-8780-5a58f6d8453f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:03.478449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bf73f37-fcf9-4acc-8780-5a58f6d8453f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:03.596293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2bf73f37-fcf9-4acc-8780-5a58f6d8453f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:03.615158 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2bf73f37-fcf9-4acc-8780-5a58f6d8453f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1286 microversion: 2.1 time: 0.189242 Sep 27 10:02:03.615756 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 776/1400] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:03 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1286 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:03.634237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5234dfe9-ce13-40fc-aa28-eed332ac2aa0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6392bee0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:03.687332 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5234dfe9-ce13-40fc-aa28-eed332ac2aa0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.070370 Sep 27 10:02:03.687931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:03.689063 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 777/1401] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:03 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:03.735541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:03.737569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:03.740579 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e84148f7-55b5-451c-8a11-b120cef99e66 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1311 microversion: 2.21 time: 1.837835 Sep 27 10:02:03.754664 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 625/1402] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:01 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1311 bytes in 1852 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:02:03.768494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-56f9cc24-4a26-4552-af9d-0478193b3d3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:03.769764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-56f9cc24-4a26-4552-af9d-0478193b3d3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:03.778384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56f9cc24-4a26-4552-af9d-0478193b3d3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:03.779266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56f9cc24-4a26-4552-af9d-0478193b3d3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:03.779917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56f9cc24-4a26-4552-af9d-0478193b3d3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:03.780940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56f9cc24-4a26-4552-af9d-0478193b3d3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:03.781827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56f9cc24-4a26-4552-af9d-0478193b3d3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:03.782511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56f9cc24-4a26-4552-af9d-0478193b3d3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.111606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cedba3de-5522-459d-91ef-18d080b256bf tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:04.121133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cedba3de-5522-459d-91ef-18d080b256bf tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.122501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cedba3de-5522-459d-91ef-18d080b256bf tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.123324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cedba3de-5522-459d-91ef-18d080b256bf tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.124496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cedba3de-5522-459d-91ef-18d080b256bf tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.128555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cedba3de-5522-459d-91ef-18d080b256bf tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.129404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cedba3de-5522-459d-91ef-18d080b256bf tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.159944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=16 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:04.212754 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cedba3de-5522-459d-91ef-18d080b256bf tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.522390 Sep 27 10:02:04.213382 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 778/1403] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:02:03 2022] GET /compute/v2.1/os-services => generated 947 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:04.228137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cd2ee483-6f31-4fb7-b294-7667bc85a29c tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:04.236997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd2ee483-6f31-4fb7-b294-7667bc85a29c tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.237601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd2ee483-6f31-4fb7-b294-7667bc85a29c tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.238359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd2ee483-6f31-4fb7-b294-7667bc85a29c tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.239455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd2ee483-6f31-4fb7-b294-7667bc85a29c tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.240612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd2ee483-6f31-4fb7-b294-7667bc85a29c tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.241307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd2ee483-6f31-4fb7-b294-7667bc85a29c tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.283277 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cd2ee483-6f31-4fb7-b294-7667bc85a29c tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?host=ncdv-master-1" status: 200 len: 581 microversion: 2.1 time: 0.058448 Sep 27 10:02:04.284973 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 779/1404] 10.222.0.26 () {58 vars in 1174 bytes} [Tue Sep 27 10:02:04 2022] GET /compute/v2.1/os-services?host=ncdv-master-1 => generated 581 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:04.303669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-45e49434-74a7-4916-93fa-9fd0ba592d0f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:04.309128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e49434-74a7-4916-93fa-9fd0ba592d0f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.311073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e49434-74a7-4916-93fa-9fd0ba592d0f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.311779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e49434-74a7-4916-93fa-9fd0ba592d0f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.312861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e49434-74a7-4916-93fa-9fd0ba592d0f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.313609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e49434-74a7-4916-93fa-9fd0ba592d0f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.314272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e49434-74a7-4916-93fa-9fd0ba592d0f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.345135 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-45e49434-74a7-4916-93fa-9fd0ba592d0f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.046968 Sep 27 10:02:04.352641 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 780/1405] 10.222.0.26 () {58 vars in 1176 bytes} [Tue Sep 27 10:02:04 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:04.367594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b8964f44-585a-40a7-b60c-8d523a33f95f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:04.371349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8964f44-585a-40a7-b60c-8d523a33f95f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.371848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8964f44-585a-40a7-b60c-8d523a33f95f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.372291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8964f44-585a-40a7-b60c-8d523a33f95f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.373849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8964f44-585a-40a7-b60c-8d523a33f95f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.374508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8964f44-585a-40a7-b60c-8d523a33f95f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.375123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8964f44-585a-40a7-b60c-8d523a33f95f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.407556 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b8964f44-585a-40a7-b60c-8d523a33f95f tempest-ServicesAdminTestJSON-1550520853 tempest-ServicesAdminTestJSON-1550520853-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.043628 Sep 27 10:02:04.409396 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 781/1406] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:02:04 2022] GET /compute/v2.1/os-services => generated 947 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:04.441286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-56f9cc24-4a26-4552-af9d-0478193b3d3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:04.445084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:04.454542 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-56f9cc24-4a26-4552-af9d-0478193b3d3f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1311 microversion: 2.1 time: 0.690845 Sep 27 10:02:04.456990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:04.458643 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 626/1407] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:03 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1311 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:04.471675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:04.481214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.483629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.486511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.490166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.491279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.493346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.643973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c2f12186-55f2-48ed-a26a-5ebc75f0f1b4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:04.645040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c2f12186-55f2-48ed-a26a-5ebc75f0f1b4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:04.659793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2f12186-55f2-48ed-a26a-5ebc75f0f1b4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.664866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2f12186-55f2-48ed-a26a-5ebc75f0f1b4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.665770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2f12186-55f2-48ed-a26a-5ebc75f0f1b4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:04.666763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2f12186-55f2-48ed-a26a-5ebc75f0f1b4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:04.668262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2f12186-55f2-48ed-a26a-5ebc75f0f1b4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:04.671419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2f12186-55f2-48ed-a26a-5ebc75f0f1b4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:05.094594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:05.110412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:05.114474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:05.118983 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1ae87e3c-9bcb-412e-b8ca-f81fafc4ca8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1409 microversion: 2.25 time: 0.670548 Sep 27 10:02:05.121017 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 782/1408] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:04 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1409 bytes in 674 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:02:05.134173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8541a5c9-224b-492e-a58a-5acc2a641777 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6380b9d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:05.249240 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8541a5c9-224b-492e-a58a-5acc2a641777 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.125560 Sep 27 10:02:05.252261 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 783/1409] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:05 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:05.268637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:05.269596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:05.284206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:05.293562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:05.300594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:05.304776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:05.309729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:05.311712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:05.346357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c2f12186-55f2-48ed-a26a-5ebc75f0f1b4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:05.360755 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c2f12186-55f2-48ed-a26a-5ebc75f0f1b4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1292 microversion: 2.1 time: 0.722802 Sep 27 10:02:05.367751 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 627/1410] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:04 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1292 bytes in 731 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:05.387900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-578102ea-9110-4b26-87f7-35b2e735fb5e req-799b0887-55a0-4dd7-be80-d1c92d53916d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fcd14928-8ef7-425b-89bb-2c714284e840", "name": "network-vif-plugged", "status": "completed", "tag": "b4c8bc77-41d3-49f6-84f2-59d40e203a22"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:05.418341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-578102ea-9110-4b26-87f7-35b2e735fb5e req-799b0887-55a0-4dd7-be80-d1c92d53916d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:05.419747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-578102ea-9110-4b26-87f7-35b2e735fb5e req-799b0887-55a0-4dd7-be80-d1c92d53916d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:05.423633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-578102ea-9110-4b26-87f7-35b2e735fb5e req-799b0887-55a0-4dd7-be80-d1c92d53916d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:05.460038 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-578102ea-9110-4b26-87f7-35b2e735fb5e req-799b0887-55a0-4dd7-be80-d1c92d53916d service nova] Creating event network-vif-plugged:b4c8bc77-41d3-49f6-84f2-59d40e203a22 for instance fcd14928-8ef7-425b-89bb-2c714284e840 on nch2-master-1 Sep 27 10:02:05.492583 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-578102ea-9110-4b26-87f7-35b2e735fb5e req-799b0887-55a0-4dd7-be80-d1c92d53916d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.103218 Sep 27 10:02:05.495038 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 628/1411] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:05.522713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea0014e1-6e35-4db2-9490-c6560a23f18c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:05.528448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ea0014e1-6e35-4db2-9490-c6560a23f18c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:05.542403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea0014e1-6e35-4db2-9490-c6560a23f18c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:05.544949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea0014e1-6e35-4db2-9490-c6560a23f18c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:05.545430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea0014e1-6e35-4db2-9490-c6560a23f18c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:05.548514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea0014e1-6e35-4db2-9490-c6560a23f18c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:05.549063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea0014e1-6e35-4db2-9490-c6560a23f18c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:05.550744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea0014e1-6e35-4db2-9490-c6560a23f18c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:05.645716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:05.660270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:05.662769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:05.664801 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7fa1bdbd-6b59-4777-a780-f2a3bfb987f2 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1317 microversion: 2.21 time: 0.410973 Sep 27 10:02:05.665937 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 784/1412] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:05 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1317 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:06.150159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:06.150984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:06.162905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:06.164430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:06.165400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:06.166147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:06.166699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:06.167275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:06.237320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ea0014e1-6e35-4db2-9490-c6560a23f18c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:06.252749 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea0014e1-6e35-4db2-9490-c6560a23f18c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1461 microversion: 2.1 time: 0.755076 Sep 27 10:02:06.269863 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 629/1413] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:05 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1461 bytes in 774 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:06.284766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3320cc59-1b42-4184-ac3c-4e7eb94b382f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63760670>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:06.373107 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3320cc59-1b42-4184-ac3c-4e7eb94b382f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1411 microversion: 2.1 time: 0.101743 Sep 27 10:02:06.374883 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 630/1414] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:06 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1411 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:06.387006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-77211f45-5f6d-4c95-b00e-3ae439355182 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:06.388168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-77211f45-5f6d-4c95-b00e-3ae439355182 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:06.390585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=86,nova_api:INSERT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:06.432254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77211f45-5f6d-4c95-b00e-3ae439355182 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:06.434771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77211f45-5f6d-4c95-b00e-3ae439355182 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:06.435512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77211f45-5f6d-4c95-b00e-3ae439355182 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:06.437974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77211f45-5f6d-4c95-b00e-3ae439355182 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:06.439465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77211f45-5f6d-4c95-b00e-3ae439355182 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:06.443734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77211f45-5f6d-4c95-b00e-3ae439355182 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:06.592237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:06.607458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:06.611608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:06.614339 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-61dc4656-e0ca-4539-b33a-3f102d5d9089 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1409 microversion: 2.25 time: 0.469167 Sep 27 10:02:06.617992 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 785/1415] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:06 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1409 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:06.634245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7a2707e3-83d6-425b-b0d9-e47385a13575 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:06.634759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7a2707e3-83d6-425b-b0d9-e47385a13575 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:06.649106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a2707e3-83d6-425b-b0d9-e47385a13575 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:06.649778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a2707e3-83d6-425b-b0d9-e47385a13575 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:06.650232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a2707e3-83d6-425b-b0d9-e47385a13575 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:06.650989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a2707e3-83d6-425b-b0d9-e47385a13575 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:06.653267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a2707e3-83d6-425b-b0d9-e47385a13575 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:06.655816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a2707e3-83d6-425b-b0d9-e47385a13575 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:07.134304 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-77211f45-5f6d-4c95-b00e-3ae439355182 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1612 microversion: 2.1 time: 0.757971 Sep 27 10:02:07.135587 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 631/1416] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:06 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1612 bytes in 760 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:07.142850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:07.143935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:07.157984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:07.158781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:07.159530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:07.160331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:07.160892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:07.161383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:07.406988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7a2707e3-83d6-425b-b0d9-e47385a13575 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:07.410404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:07.445479 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7a2707e3-83d6-425b-b0d9-e47385a13575 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1360 microversion: 2.1 time: 0.826272 Sep 27 10:02:07.449417 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 786/1417] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:06 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1360 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:07.483449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:07.484121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:07.498063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:07.500929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:07.501443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:07.502199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:07.502664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:07.504791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:07.555356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:07.571702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:07.583309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:07.584816 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-70ccd852-3573-452f-bf1c-3cae910a8b2f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1359 microversion: 2.21 time: 0.448015 Sep 27 10:02:07.585740 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 632/1418] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:07 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1359 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:07.589549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': 'f74e77337957458794bd6c942c97b344', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a3199e83259042efada3f2a565c624ff', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:07.590326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] flavor_id is None. Assuming migration. {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4196}} Sep 27 10:02:07.590780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:02:07.591427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ee15e517-3686-4c16-a359-c8e20a4360bd tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6377bb80>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:07.750122 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ee15e517-3686-4c16-a359-c8e20a4360bd tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1420 microversion: 2.1 time: 0.162956 Sep 27 10:02:07.752119 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 633/1419] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:07 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:07.767601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:07.769041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:07.793870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:07.794770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:07.795619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:07.796725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:07.797383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:07.798004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:08.189237 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3f402c75-6a1a-4a7c-9f28-d8e54c9b5aee tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615/action" status: 202 len: 0 microversion: 2.1 time: 0.724286 Sep 27 10:02:08.197033 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 787/1420] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:02:07 2022] POST /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615/action => generated 0 bytes in 734 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:02:08.202692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ea0865ae-2cad-4171-bbb8-59fb18bdb405 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63830310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:08.283083 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ea0865ae-2cad-4171-bbb8-59fb18bdb405 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 200 len: 1420 microversion: 2.1 time: 0.082324 Sep 27 10:02:08.283717 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 788/1421] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:02:08 2022] GET /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:08.301924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6d4fb637-bed5-4b4a-8d6f-4db000d33b4e req-c80478da-c711-40b7-8bd9-506fe0fefd6e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "11a0d56b-3e1f-40e7-9e3c-30fca67df25e", "tag": "b3618e9e-e34a-4212-aa29-7747a00843a9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:08.316861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6d4fb637-bed5-4b4a-8d6f-4db000d33b4e req-c80478da-c711-40b7-8bd9-506fe0fefd6e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:08.317679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6d4fb637-bed5-4b4a-8d6f-4db000d33b4e req-c80478da-c711-40b7-8bd9-506fe0fefd6e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:08.318315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6d4fb637-bed5-4b4a-8d6f-4db000d33b4e req-c80478da-c711-40b7-8bd9-506fe0fefd6e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:08.335612 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6d4fb637-bed5-4b4a-8d6f-4db000d33b4e req-c80478da-c711-40b7-8bd9-506fe0fefd6e service nova] Creating event network-changed:b3618e9e-e34a-4212-aa29-7747a00843a9 for instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e on nch1-master-1 Sep 27 10:02:08.344685 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6d4fb637-bed5-4b4a-8d6f-4db000d33b4e req-c80478da-c711-40b7-8bd9-506fe0fefd6e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048584 Sep 27 10:02:08.347408 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 789/1422] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:08.359079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c394104b-a143-4109-97d4-170006be0314 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:08.360940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c394104b-a143-4109-97d4-170006be0314 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:08.373308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:08.379847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:08.380031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c394104b-a143-4109-97d4-170006be0314 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:08.380796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c394104b-a143-4109-97d4-170006be0314 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:08.381814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c394104b-a143-4109-97d4-170006be0314 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:08.384029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c394104b-a143-4109-97d4-170006be0314 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:08.384785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c394104b-a143-4109-97d4-170006be0314 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:08.385798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c394104b-a143-4109-97d4-170006be0314 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:08.394870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:08.397900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:08.399161 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-20fce169-e586-4b27-a7af-1d5598d3e1ac tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1604 microversion: 2.25 time: 0.645738 Sep 27 10:02:08.399833 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 634/1423] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:07 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1604 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:08.424386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5db09d71-8d67-4bd6-ad79-d3dc38ddb986 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f636371f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:08.734214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c394104b-a143-4109-97d4-170006be0314 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:08.748164 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c394104b-a143-4109-97d4-170006be0314 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1470 microversion: 2.1 time: 0.399152 Sep 27 10:02:08.749381 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 790/1424] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:08 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1470 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:08.756813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b77dd34b-5e8b-4ca3-9623-eb8a16ce1ebe tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:08.758096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b77dd34b-5e8b-4ca3-9623-eb8a16ce1ebe tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:08.767310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b77dd34b-5e8b-4ca3-9623-eb8a16ce1ebe tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:08.767888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b77dd34b-5e8b-4ca3-9623-eb8a16ce1ebe tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:08.768535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b77dd34b-5e8b-4ca3-9623-eb8a16ce1ebe tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:08.769301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b77dd34b-5e8b-4ca3-9623-eb8a16ce1ebe tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:08.769903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b77dd34b-5e8b-4ca3-9623-eb8a16ce1ebe tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:08.770511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b77dd34b-5e8b-4ca3-9623-eb8a16ce1ebe tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:08.911335 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b77dd34b-5e8b-4ca3-9623-eb8a16ce1ebe tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/os-interface" status: 200 len: 284 microversion: 2.25 time: 0.160159 Sep 27 10:02:08.912451 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 791/1425] 10.222.0.26 () {60 vars in 1316 bytes} [Tue Sep 27 10:02:08 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/os-interface => generated 284 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:02:08.954195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4bf1231a-6477-4a22-b312-7ecf2c363a4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:08.955761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4bf1231a-6477-4a22-b312-7ecf2c363a4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:08.981990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bf1231a-6477-4a22-b312-7ecf2c363a4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:08.982781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bf1231a-6477-4a22-b312-7ecf2c363a4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:08.983245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bf1231a-6477-4a22-b312-7ecf2c363a4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:08.984000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bf1231a-6477-4a22-b312-7ecf2c363a4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:08.984466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bf1231a-6477-4a22-b312-7ecf2c363a4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:08.984941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bf1231a-6477-4a22-b312-7ecf2c363a4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:09.443316 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5db09d71-8d67-4bd6-ad79-d3dc38ddb986 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 204 len: 0 microversion: 2.1 time: 1.042359 Sep 27 10:02:09.443881 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 635/1426] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:08 2022] DELETE /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 0 bytes in 1043 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:02:09.451717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:09.453791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:09.468474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:09.471034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:09.477483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:09.478587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:09.479226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:09.480021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:09.541979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=5 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:09.700086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4bf1231a-6477-4a22-b312-7ecf2c363a4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:09.710740 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4bf1231a-6477-4a22-b312-7ecf2c363a4a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1360 microversion: 2.1 time: 0.780223 Sep 27 10:02:09.712255 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 792/1427] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:08 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1360 bytes in 799 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:09.744048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:09.854919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:09.865551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:09.868273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:09.869904 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1cc9b0ce-758c-4ffd-8d40-18a3c8bc0a8d tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1359 microversion: 2.21 time: 0.424601 Sep 27 10:02:09.882468 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 636/1428] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:09 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1359 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:09.902306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cb7d5674-7d12-4a40-abd4-05d7dddde379 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:09.903343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cb7d5674-7d12-4a40-abd4-05d7dddde379 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:09.921355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb7d5674-7d12-4a40-abd4-05d7dddde379 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:09.922984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb7d5674-7d12-4a40-abd4-05d7dddde379 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:09.924287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb7d5674-7d12-4a40-abd4-05d7dddde379 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:09.926284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb7d5674-7d12-4a40-abd4-05d7dddde379 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:09.927253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb7d5674-7d12-4a40-abd4-05d7dddde379 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:09.929403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb7d5674-7d12-4a40-abd4-05d7dddde379 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:09.982018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cb7d5674-7d12-4a40-abd4-05d7dddde379 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:02:10.143225 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cb7d5674-7d12-4a40-abd4-05d7dddde379 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 204 len: 0 microversion: 2.1 time: 0.258037 Sep 27 10:02:10.144910 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 637/1429] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:02:09 2022] DELETE /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:02:10.173576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-15eb94b3-f9b4-4132-bf6d-22ac16fedd0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:10.175813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-15eb94b3-f9b4-4132-bf6d-22ac16fedd0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:10.186933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15eb94b3-f9b4-4132-bf6d-22ac16fedd0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:10.187549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15eb94b3-f9b4-4132-bf6d-22ac16fedd0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:10.188285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15eb94b3-f9b4-4132-bf6d-22ac16fedd0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:10.190131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15eb94b3-f9b4-4132-bf6d-22ac16fedd0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:10.191042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15eb94b3-f9b4-4132-bf6d-22ac16fedd0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:10.191676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15eb94b3-f9b4-4132-bf6d-22ac16fedd0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:10.258717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8ce74096-9a19-492c-ab04-819f7ab7e140 req-de89274a-1d7a-433f-8603-af17956966d9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e", "tag": "4ba20389-14eb-48c5-a482-6e64d2aef60d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:10.277248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8ce74096-9a19-492c-ab04-819f7ab7e140 req-de89274a-1d7a-433f-8603-af17956966d9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:10.280141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8ce74096-9a19-492c-ab04-819f7ab7e140 req-de89274a-1d7a-433f-8603-af17956966d9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:10.280928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8ce74096-9a19-492c-ab04-819f7ab7e140 req-de89274a-1d7a-433f-8603-af17956966d9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:10.298595 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8ce74096-9a19-492c-ab04-819f7ab7e140 req-de89274a-1d7a-433f-8603-af17956966d9 service nova] Creating event network-changed:4ba20389-14eb-48c5-a482-6e64d2aef60d for instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e on nch1-master-1 Sep 27 10:02:10.307458 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8ce74096-9a19-492c-ab04-819f7ab7e140 req-de89274a-1d7a-433f-8603-af17956966d9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.593602 Sep 27 10:02:10.309572 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 793/1430] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:10.427250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-1302216581", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "4568aca4-c33f-4379-b9d1-22f70d115851"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:10.520938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] validate_networks() for [('4568aca4-c33f-4379-b9d1-22f70d115851', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:02:10.696695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-15eb94b3-f9b4-4132-bf6d-22ac16fedd0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:10.707977 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-15eb94b3-f9b4-4132-bf6d-22ac16fedd0d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1470 microversion: 2.1 time: 0.561345 Sep 27 10:02:10.709541 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 638/1431] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:10 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1470 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:10.736777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6434a501-e1b2-43cb-b042-89e73511b2d5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:10.737860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6434a501-e1b2-43cb-b042-89e73511b2d5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:10.745621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6434a501-e1b2-43cb-b042-89e73511b2d5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:10.746194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6434a501-e1b2-43cb-b042-89e73511b2d5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:10.746689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6434a501-e1b2-43cb-b042-89e73511b2d5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:10.748146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6434a501-e1b2-43cb-b042-89e73511b2d5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:10.748714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6434a501-e1b2-43cb-b042-89e73511b2d5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:10.749286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6434a501-e1b2-43cb-b042-89e73511b2d5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:11.071488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=14 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:11.090071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6434a501-e1b2-43cb-b042-89e73511b2d5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:11.111917 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6434a501-e1b2-43cb-b042-89e73511b2d5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1360 microversion: 2.1 time: 0.378594 Sep 27 10:02:11.125877 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 639/1432] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:10 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1360 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:11.146790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:11.151449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:11.160597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:11.161445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:11.162141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:11.163575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:11.164421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:11.165401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:11.300091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:02:11.300684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:02:11.301453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:02:11.301982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:02:11.302708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:02:11.323927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:11.325365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:11.326603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:11.327551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:11.328503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:11.329139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:11.384158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Getting quotas for project e5a565f65b3849a99b74454791397ca5. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:02:11.386562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:11.390307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Getting quotas for user 6eb3ccfcd9c344e1a7ad55f8398ec87a and project e5a565f65b3849a99b74454791397ca5. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:02:11.405517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:02:11.408075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:02:11.410047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:11.418234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:11.421584 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-473110e1-782a-4832-bb9f-2bb321a50a34 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1359 microversion: 2.21 time: 0.292490 Sep 27 10:02:11.435173 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 640/1433] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:11 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1359 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:11.468750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-68f0cd94-7e7c-495a-a897-774acbc9dfdc req-7f61a121-a528-4d0c-b9e4-2ac3a67ca26b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fcd14928-8ef7-425b-89bb-2c714284e840", "tag": "b4c8bc77-41d3-49f6-84f2-59d40e203a22"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:11.481949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-68f0cd94-7e7c-495a-a897-774acbc9dfdc req-7f61a121-a528-4d0c-b9e4-2ac3a67ca26b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:11.482758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-68f0cd94-7e7c-495a-a897-774acbc9dfdc req-7f61a121-a528-4d0c-b9e4-2ac3a67ca26b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:11.483703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-68f0cd94-7e7c-495a-a897-774acbc9dfdc req-7f61a121-a528-4d0c-b9e4-2ac3a67ca26b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:11.496733 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-762497fc-fb83-4690-a454-40ea5fa96946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.074914 Sep 27 10:02:11.498072 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 794/1434] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:02:10 2022] POST /compute/v2.1/servers => generated 380 bytes in 1078 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:02:11.513365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-59b02660-ddd2-4333-bc13-cd7294fa29fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:11.521770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-59b02660-ddd2-4333-bc13-cd7294fa29fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:11.522452 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-68f0cd94-7e7c-495a-a897-774acbc9dfdc req-7f61a121-a528-4d0c-b9e4-2ac3a67ca26b service nova] Creating event network-changed:b4c8bc77-41d3-49f6-84f2-59d40e203a22 for instance fcd14928-8ef7-425b-89bb-2c714284e840 on nch2-master-1 Sep 27 10:02:11.549452 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-68f0cd94-7e7c-495a-a897-774acbc9dfdc req-7f61a121-a528-4d0c-b9e4-2ac3a67ca26b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.098902 Sep 27 10:02:11.553187 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 641/1435] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:11.660270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59b02660-ddd2-4333-bc13-cd7294fa29fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:11.671990 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-59b02660-ddd2-4333-bc13-cd7294fa29fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1208 microversion: 2.1 time: 0.163705 Sep 27 10:02:11.672749 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 795/1436] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:11 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1208 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:11.735064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1ba26c82-43a5-49ee-8b2a-3b9122289439 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:11.737377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1ba26c82-43a5-49ee-8b2a-3b9122289439 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:11.750923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba26c82-43a5-49ee-8b2a-3b9122289439 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:11.751981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba26c82-43a5-49ee-8b2a-3b9122289439 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:11.752592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba26c82-43a5-49ee-8b2a-3b9122289439 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:11.753574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba26c82-43a5-49ee-8b2a-3b9122289439 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:11.754210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba26c82-43a5-49ee-8b2a-3b9122289439 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:11.754830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba26c82-43a5-49ee-8b2a-3b9122289439 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:11.961342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1ba26c82-43a5-49ee-8b2a-3b9122289439 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:11.980384 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1ba26c82-43a5-49ee-8b2a-3b9122289439 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1470 microversion: 2.1 time: 0.250412 Sep 27 10:02:11.982509 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 642/1437] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:11 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1470 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:12.135770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dff3e2f4-9da1-41a4-9237-411502bdd918 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:12.136388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dff3e2f4-9da1-41a4-9237-411502bdd918 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:12.153168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff3e2f4-9da1-41a4-9237-411502bdd918 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.154774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff3e2f4-9da1-41a4-9237-411502bdd918 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.155240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff3e2f4-9da1-41a4-9237-411502bdd918 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.156300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff3e2f4-9da1-41a4-9237-411502bdd918 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.156936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff3e2f4-9da1-41a4-9237-411502bdd918 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.157920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff3e2f4-9da1-41a4-9237-411502bdd918 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.422446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dff3e2f4-9da1-41a4-9237-411502bdd918 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:12.437399 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dff3e2f4-9da1-41a4-9237-411502bdd918 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1360 microversion: 2.1 time: 0.303458 Sep 27 10:02:12.439870 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 796/1438] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:12 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1360 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:12.479459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:12.481392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:12.489961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.490868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.491504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.492565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.493557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.494227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.571853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a68d7715-87d0-4d54-b8ee-7104e243125f tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Action: 'update', calling method: .version_select at 0x7f7f636aec10>, body: {"status": "disabled", "disabled_reason": "maintenance"} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:12.576470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a68d7715-87d0-4d54-b8ee-7104e243125f tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.577756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a68d7715-87d0-4d54-b8ee-7104e243125f tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.578252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a68d7715-87d0-4d54-b8ee-7104e243125f tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.592611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a68d7715-87d0-4d54-b8ee-7104e243125f tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.593143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a68d7715-87d0-4d54-b8ee-7104e243125f tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.593592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a68d7715-87d0-4d54-b8ee-7104e243125f tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.606669 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a68d7715-87d0-4d54-b8ee-7104e243125f tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] HTTP exception thrown: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a68d7715-87d0-4d54-b8ee-7104e243125f tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Request method failure captured: Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/os-services/2c790eb0-6964-4273-a157-0367c336a444 HTTP/1.1 Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 56 Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsokUGrv86eDHyKbXvmNlSmWAXDoZniRVsDC3lHGZX4RDSR2sKPCAArNGxNIwDZ5jKoSvz2UHckbgdczdY3mIpRFGDMem-_Aw4WF0g8XPmZA3O9zWtFNFQUW5kWherVIs2gc8OEc-AvL3mupr9ZbLanRJAqzXsP0Njpxg8POIWDtfB8 Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.53 Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e65d33280d6e4c3289089cd018e5bba5 Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-762970070 Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 10:02:12.620821 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e65d33280d6e4c3289089cd018e5bba5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e65d33280d6e4c3289089cd018e5bba5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e65d33280d6e4c3289089cd018e5bba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e65d33280d6e4c3289089cd018e5bba5"}]}] Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-762970070 Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e65d33280d6e4c3289089cd018e5bba5 Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-762970070 Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 9d6be144314e42d4bb25f70ef6ed3a43 Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: {"status": "disabled", "disabled_reason": "maintenance"} Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:02:12.622548 ncdv-master-1 devstack@n-api.service[145969]: exception: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2c790eb0-6964-4273-a157-0367c336a444', 'body': {'status': 'disabled', 'disabled_reason': 'maintenance'}} Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6207, in service_get_by_id Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return _find_service_in_cell(context, service_id=service_id) Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6067, in _find_service_in_cell Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 436, in update Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi service = self.host_api.service_get_by_id(context, service_id) Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6209, in service_get_by_id Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ServiceNotFound(service_id=service_id) Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ServiceNotFound: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.629925 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 441, in update Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:12.631547 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:02:12.633145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:02:12.633145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.633145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.633145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a68d7715-87d0-4d54-b8ee-7104e243125f tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Returning 404 to user: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:12.633145 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a68d7715-87d0-4d54-b8ee-7104e243125f tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-services/2c790eb0-6964-4273-a157-0367c336a444" status: 404 len: 110 microversion: 2.53 time: 0.343538 Sep 27 10:02:12.633145 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 643/1439] 10.222.0.26 () {62 vars in 1309 bytes} [Tue Sep 27 10:02:12 2022] PUT /compute/v2.1/os-services/2c790eb0-6964-4273-a157-0367c336a444 => generated 110 bytes in 348 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:02:12.638518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23e61c96-f1f0-494f-a0b1-eee5adbc01c6 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Action: 'update', calling method: .version_select at 0x7f7f6377b4c0>, body: {"status": "disabled"} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:12.648964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23e61c96-f1f0-494f-a0b1-eee5adbc01c6 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.649570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23e61c96-f1f0-494f-a0b1-eee5adbc01c6 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.650355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23e61c96-f1f0-494f-a0b1-eee5adbc01c6 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.658644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23e61c96-f1f0-494f-a0b1-eee5adbc01c6 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.659205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23e61c96-f1f0-494f-a0b1-eee5adbc01c6 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.659842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23e61c96-f1f0-494f-a0b1-eee5adbc01c6 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.672099 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-23e61c96-f1f0-494f-a0b1-eee5adbc01c6 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] HTTP exception thrown: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23e61c96-f1f0-494f-a0b1-eee5adbc01c6 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Request method failure captured: Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/os-services/2c790eb0-6964-4273-a157-0367c336a444 HTTP/1.1 Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 22 Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsokUGrv86eDHyKbXvmNlSmWAXDoZniRVsDC3lHGZX4RDSR2sKPCAArNGxNIwDZ5jKoSvz2UHckbgdczdY3mIpRFGDMem-_Aw4WF0g8XPmZA3O9zWtFNFQUW5kWherVIs2gc8OEc-AvL3mupr9ZbLanRJAqzXsP0Njpxg8POIWDtfB8 Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.53 Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e65d33280d6e4c3289089cd018e5bba5 Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-762970070 Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 10:02:12.674134 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e65d33280d6e4c3289089cd018e5bba5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e65d33280d6e4c3289089cd018e5bba5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e65d33280d6e4c3289089cd018e5bba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e65d33280d6e4c3289089cd018e5bba5"}]}] Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-762970070 Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e65d33280d6e4c3289089cd018e5bba5 Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-762970070 Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 9d6be144314e42d4bb25f70ef6ed3a43 Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: {"status": "disabled"} Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: exception: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.675906 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2c790eb0-6964-4273-a157-0367c336a444', 'body': {'status': 'disabled'}} Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6207, in service_get_by_id Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return _find_service_in_cell(context, service_id=service_id) Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6067, in _find_service_in_cell Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 436, in update Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi service = self.host_api.service_get_by_id(context, service_id) Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6209, in service_get_by_id Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ServiceNotFound(service_id=service_id) Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ServiceNotFound: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.677052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 441, in update Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:02:12.678642 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:02:12.680126 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.680126 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.681683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23e61c96-f1f0-494f-a0b1-eee5adbc01c6 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Returning 404 to user: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:12.682725 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-23e61c96-f1f0-494f-a0b1-eee5adbc01c6 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-services/2c790eb0-6964-4273-a157-0367c336a444" status: 404 len: 110 microversion: 2.53 time: 0.052155 Sep 27 10:02:12.683820 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 644/1440] 10.222.0.26 () {62 vars in 1309 bytes} [Tue Sep 27 10:02:12 2022] PUT /compute/v2.1/os-services/2c790eb0-6964-4273-a157-0367c336a444 => generated 110 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:02:12.709614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e2886c6a-bf75-4aec-8fb2-07473eeae78e tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Action: 'update', calling method: .version_select at 0x7f7f636748b0>, body: {"status": "enabled"} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:12.716140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2886c6a-bf75-4aec-8fb2-07473eeae78e tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.718573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2886c6a-bf75-4aec-8fb2-07473eeae78e tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.719429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2886c6a-bf75-4aec-8fb2-07473eeae78e tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.737348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2886c6a-bf75-4aec-8fb2-07473eeae78e tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.739967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2886c6a-bf75-4aec-8fb2-07473eeae78e tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.743264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2886c6a-bf75-4aec-8fb2-07473eeae78e tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.760108 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-e2886c6a-bf75-4aec-8fb2-07473eeae78e tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] HTTP exception thrown: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e2886c6a-bf75-4aec-8fb2-07473eeae78e tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Request method failure captured: Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/os-services/2c790eb0-6964-4273-a157-0367c336a444 HTTP/1.1 Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 21 Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsokUGrv86eDHyKbXvmNlSmWAXDoZniRVsDC3lHGZX4RDSR2sKPCAArNGxNIwDZ5jKoSvz2UHckbgdczdY3mIpRFGDMem-_Aw4WF0g8XPmZA3O9zWtFNFQUW5kWherVIs2gc8OEc-AvL3mupr9ZbLanRJAqzXsP0Njpxg8POIWDtfB8 Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.53 Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e65d33280d6e4c3289089cd018e5bba5 Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-762970070 Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 10:02:12.766661 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e65d33280d6e4c3289089cd018e5bba5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e65d33280d6e4c3289089cd018e5bba5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e65d33280d6e4c3289089cd018e5bba5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e65d33280d6e4c3289089cd018e5bba5"}]}] Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-762970070 Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e65d33280d6e4c3289089cd018e5bba5 Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-762970070 Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 9d6be144314e42d4bb25f70ef6ed3a43 Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: {"status": "enabled"} Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: exception: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.768327 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2c790eb0-6964-4273-a157-0367c336a444', 'body': {'status': 'enabled'}} Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6207, in service_get_by_id Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return _find_service_in_cell(context, service_id=service_id) Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6067, in _find_service_in_cell Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.NotFound() Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.NotFound: Resource could not be found. Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 436, in update Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi service = self.host_api.service_get_by_id(context, service_id) Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6209, in service_get_by_id Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ServiceNotFound(service_id=service_id) Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ServiceNotFound: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.769442 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 441, in update Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:02:12.770998 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:02:12.772445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. Sep 27 10:02:12.772445 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:12.774533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e2886c6a-bf75-4aec-8fb2-07473eeae78e tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Returning 404 to user: Service 2c790eb0-6964-4273-a157-0367c336a444 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:12.775775 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e2886c6a-bf75-4aec-8fb2-07473eeae78e tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-services/2c790eb0-6964-4273-a157-0367c336a444" status: 404 len: 110 microversion: 2.53 time: 0.075770 Sep 27 10:02:12.776877 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 645/1441] 10.222.0.26 () {62 vars in 1309 bytes} [Tue Sep 27 10:02:12 2022] PUT /compute/v2.1/os-services/2c790eb0-6964-4273-a157-0367c336a444 => generated 110 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:02:12.783882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-35b5c1c2-bda5-4fb4-b15b-7784a4b20ac7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:12.791708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-35b5c1c2-bda5-4fb4-b15b-7784a4b20ac7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:12.816901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35b5c1c2-bda5-4fb4-b15b-7784a4b20ac7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.817781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35b5c1c2-bda5-4fb4-b15b-7784a4b20ac7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.818892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35b5c1c2-bda5-4fb4-b15b-7784a4b20ac7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.820117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35b5c1c2-bda5-4fb4-b15b-7784a4b20ac7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.820839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35b5c1c2-bda5-4fb4-b15b-7784a4b20ac7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.821506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35b5c1c2-bda5-4fb4-b15b-7784a4b20ac7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.927453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-35b5c1c2-bda5-4fb4-b15b-7784a4b20ac7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:12.940185 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-35b5c1c2-bda5-4fb4-b15b-7784a4b20ac7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1204 microversion: 2.1 time: 0.161406 Sep 27 10:02:12.941412 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 646/1442] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:12 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1204 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:12.946979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-063865d6-8312-4fb8-aae0-7218ec6bda49 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:12.951940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063865d6-8312-4fb8-aae0-7218ec6bda49 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.952702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063865d6-8312-4fb8-aae0-7218ec6bda49 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.953384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063865d6-8312-4fb8-aae0-7218ec6bda49 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:12.954764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063865d6-8312-4fb8-aae0-7218ec6bda49 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:12.955521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063865d6-8312-4fb8-aae0-7218ec6bda49 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:12.956269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063865d6-8312-4fb8-aae0-7218ec6bda49 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.006871 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-063865d6-8312-4fb8-aae0-7218ec6bda49 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.064103 Sep 27 10:02:13.008118 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 647/1443] 10.222.0.26 () {60 vars in 1178 bytes} [Tue Sep 27 10:02:12 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:13.015587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-13e13846-fecd-4d11-ba23-b7db3207d3f0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:13.016445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-13e13846-fecd-4d11-ba23-b7db3207d3f0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:13.024010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13e13846-fecd-4d11-ba23-b7db3207d3f0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.024384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13e13846-fecd-4d11-ba23-b7db3207d3f0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.024880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13e13846-fecd-4d11-ba23-b7db3207d3f0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.025630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13e13846-fecd-4d11-ba23-b7db3207d3f0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.026122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13e13846-fecd-4d11-ba23-b7db3207d3f0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.026555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13e13846-fecd-4d11-ba23-b7db3207d3f0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.123874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:13.138157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:13.142850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:13.149603 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-84b22307-d2eb-44c7-848f-59397dbfd74f tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1359 microversion: 2.21 time: 0.674625 Sep 27 10:02:13.150234 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 797/1444] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:12 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1359 bytes in 678 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:02:13.164746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bd028f5e-1dd6-4bb3-b738-e4aad0d901dc tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:13.174935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd028f5e-1dd6-4bb3-b738-e4aad0d901dc tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.176435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd028f5e-1dd6-4bb3-b738-e4aad0d901dc tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.177086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd028f5e-1dd6-4bb3-b738-e4aad0d901dc tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.181059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd028f5e-1dd6-4bb3-b738-e4aad0d901dc tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.181770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd028f5e-1dd6-4bb3-b738-e4aad0d901dc tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.182364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd028f5e-1dd6-4bb3-b738-e4aad0d901dc tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.242821 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bd028f5e-1dd6-4bb3-b738-e4aad0d901dc tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.090619 Sep 27 10:02:13.259355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-13e13846-fecd-4d11-ba23-b7db3207d3f0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:13.268135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:13.268838 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 798/1445] 10.222.0.26 () {60 vars in 1211 bytes} [Tue Sep 27 10:02:13 2022] GET /compute/v2.1/os-services?xxx=nova-compute => generated 1242 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:13.276298 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-13e13846-fecd-4d11-ba23-b7db3207d3f0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1470 microversion: 2.1 time: 0.266231 Sep 27 10:02:13.278116 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 648/1446] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:13 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1470 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:13.287142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-182024fc-7b12-4282-ba80-ca08e7b77752 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:13.292262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-182024fc-7b12-4282-ba80-ca08e7b77752 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.292909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-182024fc-7b12-4282-ba80-ca08e7b77752 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.293465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-182024fc-7b12-4282-ba80-ca08e7b77752 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.294444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-182024fc-7b12-4282-ba80-ca08e7b77752 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.294950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-182024fc-7b12-4282-ba80-ca08e7b77752 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.295590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-182024fc-7b12-4282-ba80-ca08e7b77752 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.335243 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-182024fc-7b12-4282-ba80-ca08e7b77752 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.053683 Sep 27 10:02:13.343056 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 649/1447] 10.222.0.26 () {60 vars in 1178 bytes} [Tue Sep 27 10:02:13 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:13.355983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7955e8b6-1f73-4613-89fb-2a5b6c59b977 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:13.362888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7955e8b6-1f73-4613-89fb-2a5b6c59b977 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.363752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7955e8b6-1f73-4613-89fb-2a5b6c59b977 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.364764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7955e8b6-1f73-4613-89fb-2a5b6c59b977 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.366178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7955e8b6-1f73-4613-89fb-2a5b6c59b977 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.366946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7955e8b6-1f73-4613-89fb-2a5b6c59b977 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.367979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7955e8b6-1f73-4613-89fb-2a5b6c59b977 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.421094 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7955e8b6-1f73-4613-89fb-2a5b6c59b977 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx" status: 200 len: 16 microversion: 2.53 time: 0.073761 Sep 27 10:02:13.429724 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 650/1448] 10.222.0.26 () {60 vars in 1237 bytes} [Tue Sep 27 10:02:13 2022] GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx => generated 16 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:13.429925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-3e46080e-6fcb-49f6-a970-9ccfb3099a64 req-059d7113-ae97-4c75-8627-ef8ab72d5ca1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1b526682-dfde-459f-bc61-2df38e0ec4b7", "name": "network-vif-unplugged", "status": "completed", "tag": "bdcd738d-10f6-4db4-ab57-e00ef79f23bb"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:13.445197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3e46080e-6fcb-49f6-a970-9ccfb3099a64 req-059d7113-ae97-4c75-8627-ef8ab72d5ca1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.445696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3e46080e-6fcb-49f6-a970-9ccfb3099a64 req-059d7113-ae97-4c75-8627-ef8ab72d5ca1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.446141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3e46080e-6fcb-49f6-a970-9ccfb3099a64 req-059d7113-ae97-4c75-8627-ef8ab72d5ca1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.477736 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-3e46080e-6fcb-49f6-a970-9ccfb3099a64 req-059d7113-ae97-4c75-8627-ef8ab72d5ca1 service nova] Creating event network-vif-unplugged:bdcd738d-10f6-4db4-ab57-e00ef79f23bb for instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 on nch1-master-1 Sep 27 10:02:13.494224 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-3e46080e-6fcb-49f6-a970-9ccfb3099a64 req-059d7113-ae97-4c75-8627-ef8ab72d5ca1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.067221 Sep 27 10:02:13.495485 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 651/1449] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:13 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:13.508749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6fec274e-bbff-4c27-b658-94276ae2f5ef tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:13.519201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fec274e-bbff-4c27-b658-94276ae2f5ef tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.519450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fec274e-bbff-4c27-b658-94276ae2f5ef tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.519908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fec274e-bbff-4c27-b658-94276ae2f5ef tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.520705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fec274e-bbff-4c27-b658-94276ae2f5ef tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.521178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fec274e-bbff-4c27-b658-94276ae2f5ef tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.521620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fec274e-bbff-4c27-b658-94276ae2f5ef tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.618013 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8f8c122d-36ca-4201-ba55-ec4d2af1dabc tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] 10.222.0.26 "DELETE /compute/v2.1/images/" status: 404 len: 112 microversion: - time: 0.347716 Sep 27 10:02:13.619245 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 799/1450] 10.222.0.26 () {58 vars in 1128 bytes} [Tue Sep 27 10:02:13 2022] DELETE /compute/v2.1/images/ => generated 112 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 27 10:02:13.630655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b4ec0e6a-6235-46f2-b359-e2fbdd20816a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:13.631294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b4ec0e6a-6235-46f2-b359-e2fbdd20816a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:13.636834 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6fec274e-bbff-4c27-b658-94276ae2f5ef tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1242 microversion: 2.53 time: 0.135105 Sep 27 10:02:13.636834 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 652/1451] 10.222.0.26 () {60 vars in 1178 bytes} [Tue Sep 27 10:02:13 2022] GET /compute/v2.1/os-services => generated 1242 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:13.664549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e08852f-88db-44c2-a642-a2b1f20530b8 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Calling method '.version_select at 0x7f7f636aec10>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:13.677303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4ec0e6a-6235-46f2-b359-e2fbdd20816a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.677811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4ec0e6a-6235-46f2-b359-e2fbdd20816a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.684550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4ec0e6a-6235-46f2-b359-e2fbdd20816a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.686043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4ec0e6a-6235-46f2-b359-e2fbdd20816a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.693539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4ec0e6a-6235-46f2-b359-e2fbdd20816a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.694763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4ec0e6a-6235-46f2-b359-e2fbdd20816a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.747017 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-4e08852f-88db-44c2-a642-a2b1f20530b8 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] HTTP exception thrown: Image not found. Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e08852f-88db-44c2-a642-a2b1f20530b8 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Request method failure captured: Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/images/-1 HTTP/1.1 Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsok4rhDw8BzVIb8TSGcgczO5-EV8oP3P2omqDX50cgYvlwncqQaD1kgpDg5Q_ov8WrKpa07VIRcwoJGYOCXXGxw6Pnz6Z7W0_mAqR55PHMpy71J0V9iAtwQta_N1P8xANqMP5Pqh174aROWFfK7IYYgGVV2rOdqGTI0upYo5ZA8rEw Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 644bbf0f519f4cd6a02e998859ec00e2 Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:02:13.751014 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_644bbf0f519f4cd6a02e998859ec00e2"}]}] Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 644bbf0f519f4cd6a02e998859ec00e2 Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesDeleteNegativeTestJSON-1950635109-project Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 36deeaadd647404e877c2b07ff9a4fbc Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109-project Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: exception: Image not found. Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '-1'} Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:13.752580 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image -1 to delete Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.753746 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image -1 could not be found. Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:13.755183 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:02:13.757700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.760156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e08852f-88db-44c2-a642-a2b1f20530b8 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Returning 404 to user: Image not found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:13.761378 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4e08852f-88db-44c2-a642-a2b1f20530b8 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] 10.222.0.26 "DELETE /compute/v2.1/images/-1" status: 404 len: 62 microversion: 2.1 time: 0.126726 Sep 27 10:02:13.762833 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 653/1452] 10.222.0.26 () {58 vars in 1134 bytes} [Tue Sep 27 10:02:13 2022] DELETE /compute/v2.1/images/-1 => generated 62 bytes in 128 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:02:13.776396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3cf2dcf3-0d91-4776-8214-927a461521c4 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:13.787430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cf2dcf3-0d91-4776-8214-927a461521c4 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.791999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cf2dcf3-0d91-4776-8214-927a461521c4 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.793239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cf2dcf3-0d91-4776-8214-927a461521c4 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.794519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cf2dcf3-0d91-4776-8214-927a461521c4 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:13.796426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cf2dcf3-0d91-4776-8214-927a461521c4 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:13.797144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cf2dcf3-0d91-4776-8214-927a461521c4 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:13.835829 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3cf2dcf3-0d91-4776-8214-927a461521c4 tempest-ServicesAdminNegativeV253TestJSON-762970070 tempest-ServicesAdminNegativeV253TestJSON-762970070-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler" status: 200 len: 16 microversion: 2.53 time: 0.071195 Sep 27 10:02:13.836890 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 654/1453] 10.222.0.26 () {60 vars in 1239 bytes} [Tue Sep 27 10:02:13 2022] GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler => generated 16 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:13.843463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-252f518b-6f41-43b5-9945-da1f554e7d30 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Calling method '.version_select at 0x7f7f6382cee0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:13.882386 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-252f518b-6f41-43b5-9945-da1f554e7d30 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] HTTP exception thrown: Image not found. Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-252f518b-6f41-43b5-9945-da1f554e7d30 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Request method failure captured: Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/images/daa1defe-2bf6-430c-a7ad-297d767d878j HTTP/1.1 Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsok4rhDw8BzVIb8TSGcgczO5-EV8oP3P2omqDX50cgYvlwncqQaD1kgpDg5Q_ov8WrKpa07VIRcwoJGYOCXXGxw6Pnz6Z7W0_mAqR55PHMpy71J0V9iAtwQta_N1P8xANqMP5Pqh174aROWFfK7IYYgGVV2rOdqGTI0upYo5ZA8rEw Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 644bbf0f519f4cd6a02e998859ec00e2 Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:02:13.884806 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_644bbf0f519f4cd6a02e998859ec00e2"}]}] Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 644bbf0f519f4cd6a02e998859ec00e2 Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesDeleteNegativeTestJSON-1950635109-project Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 36deeaadd647404e877c2b07ff9a4fbc Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109-project Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: exception: Image not found. Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'daa1defe-2bf6-430c-a7ad-297d767d878j'} Sep 27 10:02:13.886316 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image daa1defe-2bf6-430c-a7ad-297d767d878j to delete Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:13.887134 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image daa1defe-2bf6-430c-a7ad-297d767d878j could not be found. Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:13.889069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-252f518b-6f41-43b5-9945-da1f554e7d30 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Returning 404 to user: Image not found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:13.891919 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-252f518b-6f41-43b5-9945-da1f554e7d30 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] 10.222.0.26 "DELETE /compute/v2.1/images/daa1defe-2bf6-430c-a7ad-297d767d878j" status: 404 len: 62 microversion: 2.1 time: 0.048308 Sep 27 10:02:13.894617 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 655/1454] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:13 2022] DELETE /compute/v2.1/images/daa1defe-2bf6-430c-a7ad-297d767d878j => generated 62 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:02:14.110070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7ea2b4cc-af86-46ee-9a9f-0284496f493f tempest-ServicesAdminNegativeV253TestJSON-1667307348 tempest-ServicesAdminNegativeV253TestJSON-1667307348-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:14.117552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7ea2b4cc-af86-46ee-9a9f-0284496f493f tempest-ServicesAdminNegativeV253TestJSON-1667307348 tempest-ServicesAdminNegativeV253TestJSON-1667307348-project] Policy check for os_compute_api:os-services:list failed with credentials {'is_admin': False, 'user_id': '7d51e5e0cb8f4e8e84ce6e63fe212acc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9f3336f27cfd4866b6e527676bb6acb0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7ea2b4cc-af86-46ee-9a9f-0284496f493f tempest-ServicesAdminNegativeV253TestJSON-1667307348 tempest-ServicesAdminNegativeV253TestJSON-1667307348-project] Request method failure captured: Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-services HTTP/1.1 Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsoiH2-T3JYLMOvSSPowIBIMAsCzSmvEmtvrdtOhgEj4coXtHu4AbjfgUS-uew1_XZ58Sssz9OWM0q91Bv59SI5hUltiqCzoUQ9xuB3RFWM4pNc1qqw8Qg_LzFQiW0wq5kIuQBt0g2D_HX-6h3f0YcHYdP1tInr8zo-5C67sPSsDwto Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.53 Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 9f3336f27cfd4866b6e527676bb6acb0 Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServicesAdminNegativeV253TestJSON-1667307348 Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:02:14.124705 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9f3336f27cfd4866b6e527676bb6acb0"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9f3336f27cfd4866b6e527676bb6acb0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9f3336f27cfd4866b6e527676bb6acb0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9f3336f27cfd4866b6e527676bb6acb0"}]}] Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServicesAdminNegativeV253TestJSON-1667307348 Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 9f3336f27cfd4866b6e527676bb6acb0 Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServicesAdminNegativeV253TestJSON-1667307348 Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServicesAdminNegativeV253TestJSON-1667307348-project Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7d51e5e0cb8f4e8e84ce6e63fe212acc Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServicesAdminNegativeV253TestJSON-1667307348-project Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-services:list to be performed. Sep 27 10:02:14.126263 ncdv-master-1 devstack@n-api.service[145969]: action_args: {} Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 384, in index Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(services_policies.BASE_POLICY_NAME % 'list', target={}) Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 10:02:14.127170 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-services:list to be performed. Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-services:list to be performed. Sep 27 10:02:14.128380 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:14.129703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7ea2b4cc-af86-46ee-9a9f-0284496f493f tempest-ServicesAdminNegativeV253TestJSON-1667307348 tempest-ServicesAdminNegativeV253TestJSON-1667307348-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-services:list to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:14.129703 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7ea2b4cc-af86-46ee-9a9f-0284496f493f tempest-ServicesAdminNegativeV253TestJSON-1667307348 tempest-ServicesAdminNegativeV253TestJSON-1667307348-project] 10.222.0.26 "GET /compute/v2.1/os-services" status: 403 len: 112 microversion: 2.53 time: 0.228988 Sep 27 10:02:14.129703 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 656/1455] 10.222.0.26 () {60 vars in 1178 bytes} [Tue Sep 27 10:02:13 2022] GET /compute/v2.1/os-services => generated 112 bytes in 240 msecs (HTTP/1.1 403) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:02:14.146361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-782d73ec-37ae-44ed-92a3-c2a335d5051e tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Calling method '.version_select at 0x7f7f63674ee0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:14.164627 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-782d73ec-37ae-44ed-92a3-c2a335d5051e tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] HTTP exception thrown: Image not found. Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-782d73ec-37ae-44ed-92a3-c2a335d5051e tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Request method failure captured: Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/images/eb15c46c-a1ce-4c2d-b6b3-b86a124cbe711 HTTP/1.1 Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsok4rhDw8BzVIb8TSGcgczO5-EV8oP3P2omqDX50cgYvlwncqQaD1kgpDg5Q_ov8WrKpa07VIRcwoJGYOCXXGxw6Pnz6Z7W0_mAqR55PHMpy71J0V9iAtwQta_N1P8xANqMP5Pqh174aROWFfK7IYYgGVV2rOdqGTI0upYo5ZA8rEw Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 644bbf0f519f4cd6a02e998859ec00e2 Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:02:14.167575 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_644bbf0f519f4cd6a02e998859ec00e2"}]}] Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 644bbf0f519f4cd6a02e998859ec00e2 Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesDeleteNegativeTestJSON-1950635109-project Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 36deeaadd647404e877c2b07ff9a4fbc Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109-project Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: exception: Image not found. Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'eb15c46c-a1ce-4c2d-b6b3-b86a124cbe711'} Sep 27 10:02:14.169262 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image eb15c46c-a1ce-4c2d-b6b3-b86a124cbe711 to delete Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:14.170095 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image eb15c46c-a1ce-4c2d-b6b3-b86a124cbe711 could not be found. Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:14.171290 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:14.172825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-782d73ec-37ae-44ed-92a3-c2a335d5051e tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Returning 404 to user: Image not found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:14.175921 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-782d73ec-37ae-44ed-92a3-c2a335d5051e tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] 10.222.0.26 "DELETE /compute/v2.1/images/eb15c46c-a1ce-4c2d-b6b3-b86a124cbe711" status: 404 len: 62 microversion: 2.1 time: 0.039950 Sep 27 10:02:14.178313 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 657/1456] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:02:14 2022] DELETE /compute/v2.1/images/eb15c46c-a1ce-4c2d-b6b3-b86a124cbe711 => generated 62 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:02:14.187440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f35e51e3-84c9-4c9e-ad82-8fb6dd1027bb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:14.188250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f35e51e3-84c9-4c9e-ad82-8fb6dd1027bb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:14.206845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f35e51e3-84c9-4c9e-ad82-8fb6dd1027bb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:14.207236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f35e51e3-84c9-4c9e-ad82-8fb6dd1027bb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:14.207677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f35e51e3-84c9-4c9e-ad82-8fb6dd1027bb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:14.208405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f35e51e3-84c9-4c9e-ad82-8fb6dd1027bb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:14.208902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f35e51e3-84c9-4c9e-ad82-8fb6dd1027bb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:14.209332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f35e51e3-84c9-4c9e-ad82-8fb6dd1027bb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:14.316621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f35e51e3-84c9-4c9e-ad82-8fb6dd1027bb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:14.328383 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f35e51e3-84c9-4c9e-ad82-8fb6dd1027bb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1266 microversion: 2.1 time: 0.148088 Sep 27 10:02:14.329679 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 658/1457] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:14 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1266 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:14.335521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:14.336600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:14.343201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:14.344160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:14.344160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:14.345038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:14.345584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:14.346026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:14.954802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b4ec0e6a-6235-46f2-b359-e2fbdd20816a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:14.977782 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b4ec0e6a-6235-46f2-b359-e2fbdd20816a tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1360 microversion: 2.1 time: 1.357056 Sep 27 10:02:14.978960 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 800/1458] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:13 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1360 bytes in 1359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:14.988256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9a5e08c7-b32b-48dd-a69c-ef06a002b114 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Calling method '.version_select at 0x7f7f6c2ccc10>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:15.050275 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-9a5e08c7-b32b-48dd-a69c-ef06a002b114 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] HTTP exception thrown: Image not found. Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9a5e08c7-b32b-48dd-a69c-ef06a002b114 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Request method failure captured: Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/images/tempest-invalid-524781939 HTTP/1.1 Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsok4rhDw8BzVIb8TSGcgczO5-EV8oP3P2omqDX50cgYvlwncqQaD1kgpDg5Q_ov8WrKpa07VIRcwoJGYOCXXGxw6Pnz6Z7W0_mAqR55PHMpy71J0V9iAtwQta_N1P8xANqMP5Pqh174aROWFfK7IYYgGVV2rOdqGTI0upYo5ZA8rEw Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 644bbf0f519f4cd6a02e998859ec00e2 Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:02:15.054306 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_644bbf0f519f4cd6a02e998859ec00e2"}]}] Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 644bbf0f519f4cd6a02e998859ec00e2 Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesDeleteNegativeTestJSON-1950635109-project Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 36deeaadd647404e877c2b07ff9a4fbc Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109-project Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: exception: Image not found. Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'tempest-invalid-524781939'} Sep 27 10:02:15.055775 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image tempest-invalid-524781939 to delete Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:15.056586 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image tempest-invalid-524781939 could not be found. Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:15.066299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9a5e08c7-b32b-48dd-a69c-ef06a002b114 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Returning 404 to user: Image not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:15.067517 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9a5e08c7-b32b-48dd-a69c-ef06a002b114 tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] 10.222.0.26 "DELETE /compute/v2.1/images/tempest-invalid-524781939" status: 404 len: 62 microversion: 2.1 time: 0.075849 Sep 27 10:02:15.068448 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 801/1459] 10.222.0.26 () {58 vars in 1203 bytes} [Tue Sep 27 10:02:14 2022] DELETE /compute/v2.1/images/tempest-invalid-524781939 => generated 62 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:02:15.068448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4e46f433-ba1d-4b78-91f0-31102dfb4a0f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:15.068448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4e46f433-ba1d-4b78-91f0-31102dfb4a0f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:15.080998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e46f433-ba1d-4b78-91f0-31102dfb4a0f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:15.081887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e46f433-ba1d-4b78-91f0-31102dfb4a0f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:15.082394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e46f433-ba1d-4b78-91f0-31102dfb4a0f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:15.083166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e46f433-ba1d-4b78-91f0-31102dfb4a0f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:15.084285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e46f433-ba1d-4b78-91f0-31102dfb4a0f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:15.084804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e46f433-ba1d-4b78-91f0-31102dfb4a0f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:15.126384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:15.141711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:15.145168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:15.149352 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ee602d5a-60c1-4ad7-8be5-dc302bf92680 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1359 microversion: 2.21 time: 0.816819 Sep 27 10:02:15.150874 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 659/1460] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:14 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1359 bytes in 821 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:15.170412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-51941ea7-47b2-4c09-a11f-112f77a8af9b tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Calling method '.version_select at 0x7f7f636741f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:15.198138 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-51941ea7-47b2-4c09-a11f-112f77a8af9b tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] HTTP exception thrown: Image not found. Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-51941ea7-47b2-4c09-a11f-112f77a8af9b tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Request method failure captured: Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/images/696f134b-073d-449b-99f5-59632bbae3a0 HTTP/1.1 Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsok4rhDw8BzVIb8TSGcgczO5-EV8oP3P2omqDX50cgYvlwncqQaD1kgpDg5Q_ov8WrKpa07VIRcwoJGYOCXXGxw6Pnz6Z7W0_mAqR55PHMpy71J0V9iAtwQta_N1P8xANqMP5Pqh174aROWFfK7IYYgGVV2rOdqGTI0upYo5ZA8rEw Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 644bbf0f519f4cd6a02e998859ec00e2 Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:02:15.201168 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/644bbf0f519f4cd6a02e998859ec00e2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_644bbf0f519f4cd6a02e998859ec00e2"}]}] Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 644bbf0f519f4cd6a02e998859ec00e2 Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109 Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesDeleteNegativeTestJSON-1950635109-project Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 36deeaadd647404e877c2b07ff9a4fbc Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesDeleteNegativeTestJSON-1950635109-project Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: exception: Image not found. Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '696f134b-073d-449b-99f5-59632bbae3a0'} Sep 27 10:02:15.202980 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 696f134b-073d-449b-99f5-59632bbae3a0 to delete Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:15.203812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 696f134b-073d-449b-99f5-59632bbae3a0 could not be found. Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:15.205915 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:02:15.207464 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:15.212030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-51941ea7-47b2-4c09-a11f-112f77a8af9b tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] Returning 404 to user: Image not found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:15.213880 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-51941ea7-47b2-4c09-a11f-112f77a8af9b tempest-ImagesDeleteNegativeTestJSON-1950635109 tempest-ImagesDeleteNegativeTestJSON-1950635109-project] 10.222.0.26 "DELETE /compute/v2.1/images/696f134b-073d-449b-99f5-59632bbae3a0" status: 404 len: 62 microversion: 2.1 time: 0.061491 Sep 27 10:02:15.223655 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 660/1461] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:15 2022] DELETE /compute/v2.1/images/696f134b-073d-449b-99f5-59632bbae3a0 => generated 62 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:02:15.361475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fdfecda7-4d27-459b-a1c0-41588e533ac1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:15.364287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fdfecda7-4d27-459b-a1c0-41588e533ac1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:15.380338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdfecda7-4d27-459b-a1c0-41588e533ac1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:15.381359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdfecda7-4d27-459b-a1c0-41588e533ac1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:15.382661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdfecda7-4d27-459b-a1c0-41588e533ac1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:15.384123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdfecda7-4d27-459b-a1c0-41588e533ac1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:15.385199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdfecda7-4d27-459b-a1c0-41588e533ac1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:15.386076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdfecda7-4d27-459b-a1c0-41588e533ac1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:15.452165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4e46f433-ba1d-4b78-91f0-31102dfb4a0f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:15.465833 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4e46f433-ba1d-4b78-91f0-31102dfb4a0f tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1470 microversion: 2.1 time: 0.403971 Sep 27 10:02:15.481724 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 802/1462] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:15 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1470 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:15.644322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fdfecda7-4d27-459b-a1c0-41588e533ac1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:15.662684 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fdfecda7-4d27-459b-a1c0-41588e533ac1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1266 microversion: 2.1 time: 0.310005 Sep 27 10:02:15.678626 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 661/1463] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:15 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1266 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:16.044419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-89d2f6c2-d08d-4d27-afe8-5071bd6301b6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:16.052971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-89d2f6c2-d08d-4d27-afe8-5071bd6301b6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:16.079516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89d2f6c2-d08d-4d27-afe8-5071bd6301b6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:16.080027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89d2f6c2-d08d-4d27-afe8-5071bd6301b6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:16.081380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89d2f6c2-d08d-4d27-afe8-5071bd6301b6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:16.082388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89d2f6c2-d08d-4d27-afe8-5071bd6301b6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:16.083048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89d2f6c2-d08d-4d27-afe8-5071bd6301b6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:16.083737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89d2f6c2-d08d-4d27-afe8-5071bd6301b6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:16.159783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-487258b0-19ae-4ddb-85cd-9bc26d075942 req-f73231fd-9689-4faf-a26b-dc4df339a451 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1b526682-dfde-459f-bc61-2df38e0ec4b7", "name": "network-vif-deleted", "tag": "bdcd738d-10f6-4db4-ab57-e00ef79f23bb"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:16.179224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-487258b0-19ae-4ddb-85cd-9bc26d075942 req-f73231fd-9689-4faf-a26b-dc4df339a451 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:16.180027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-487258b0-19ae-4ddb-85cd-9bc26d075942 req-f73231fd-9689-4faf-a26b-dc4df339a451 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:16.180838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-487258b0-19ae-4ddb-85cd-9bc26d075942 req-f73231fd-9689-4faf-a26b-dc4df339a451 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:16.200621 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-487258b0-19ae-4ddb-85cd-9bc26d075942 req-f73231fd-9689-4faf-a26b-dc4df339a451 service nova] Creating event network-vif-deleted:bdcd738d-10f6-4db4-ab57-e00ef79f23bb for instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 on nch1-master-1 Sep 27 10:02:16.212167 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-487258b0-19ae-4ddb-85cd-9bc26d075942 req-f73231fd-9689-4faf-a26b-dc4df339a451 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051330 Sep 27 10:02:16.213258 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 662/1464] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:02:16.227084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:16.228259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:16.240103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:16.240890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:16.241513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:16.243289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:16.243811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:16.244285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:16.526221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:16.536622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:16.538141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:16.539346 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-91978b3c-32b6-4856-bc11-4afeb79f7305 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1359 microversion: 2.21 time: 0.323319 Sep 27 10:02:16.540536 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 663/1465] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:16 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1359 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:16.546534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d4e267a7-8cf6-4155-8d02-6f1951b00132 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:16.547494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d4e267a7-8cf6-4155-8d02-6f1951b00132 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:16.555438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4e267a7-8cf6-4155-8d02-6f1951b00132 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:16.556414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4e267a7-8cf6-4155-8d02-6f1951b00132 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:16.558526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4e267a7-8cf6-4155-8d02-6f1951b00132 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:16.561092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4e267a7-8cf6-4155-8d02-6f1951b00132 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:16.561906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4e267a7-8cf6-4155-8d02-6f1951b00132 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:16.562517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4e267a7-8cf6-4155-8d02-6f1951b00132 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:16.593832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-89d2f6c2-d08d-4d27-afe8-5071bd6301b6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:16.611723 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-89d2f6c2-d08d-4d27-afe8-5071bd6301b6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1360 microversion: 2.1 time: 0.610502 Sep 27 10:02:16.618379 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 803/1466] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:15 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1360 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:16.692405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25c8cd06-1344-4426-8743-0eae95a2c653 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:16.695829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-25c8cd06-1344-4426-8743-0eae95a2c653 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:16.714396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25c8cd06-1344-4426-8743-0eae95a2c653 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:16.714896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25c8cd06-1344-4426-8743-0eae95a2c653 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:16.715335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25c8cd06-1344-4426-8743-0eae95a2c653 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:16.722881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25c8cd06-1344-4426-8743-0eae95a2c653 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:16.723304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25c8cd06-1344-4426-8743-0eae95a2c653 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:16.730399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25c8cd06-1344-4426-8743-0eae95a2c653 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:16.905076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d4e267a7-8cf6-4155-8d02-6f1951b00132 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:16.921665 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d4e267a7-8cf6-4155-8d02-6f1951b00132 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1470 microversion: 2.1 time: 0.379736 Sep 27 10:02:16.922936 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 664/1467] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:16 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1470 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:17.222372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-25c8cd06-1344-4426-8743-0eae95a2c653 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:17.235243 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-25c8cd06-1344-4426-8743-0eae95a2c653 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1308 microversion: 2.1 time: 0.546683 Sep 27 10:02:17.249420 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 804/1468] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:16 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1308 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:17.286979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b8ce9397-b503-453e-9255-287d2274e841 req-2573bf09-026d-4b9e-acbb-b98e4177cbb7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "11a0d56b-3e1f-40e7-9e3c-30fca67df25e", "name": "network-vif-plugged", "status": "completed", "tag": "b3618e9e-e34a-4212-aa29-7747a00843a9"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:17.305665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b8ce9397-b503-453e-9255-287d2274e841 req-2573bf09-026d-4b9e-acbb-b98e4177cbb7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:17.307405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b8ce9397-b503-453e-9255-287d2274e841 req-2573bf09-026d-4b9e-acbb-b98e4177cbb7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:17.308848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b8ce9397-b503-453e-9255-287d2274e841 req-2573bf09-026d-4b9e-acbb-b98e4177cbb7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:17.334749 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b8ce9397-b503-453e-9255-287d2274e841 req-2573bf09-026d-4b9e-acbb-b98e4177cbb7 service nova] Creating event network-vif-plugged:b3618e9e-e34a-4212-aa29-7747a00843a9 for instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e on nch1-master-1 Sep 27 10:02:17.343327 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b8ce9397-b503-453e-9255-287d2274e841 req-2573bf09-026d-4b9e-acbb-b98e4177cbb7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066809 Sep 27 10:02:17.344830 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 665/1469] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:17.571336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:17.572261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:17.581842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:17.582560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:17.583321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:17.585102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:17.585906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:17.586591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:17.638541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5afae9da-54bc-4b46-8093-3059b68a7490 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:17.639607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5afae9da-54bc-4b46-8093-3059b68a7490 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:17.656600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5afae9da-54bc-4b46-8093-3059b68a7490 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:17.657165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5afae9da-54bc-4b46-8093-3059b68a7490 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:17.657617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5afae9da-54bc-4b46-8093-3059b68a7490 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:17.661594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5afae9da-54bc-4b46-8093-3059b68a7490 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:17.662481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5afae9da-54bc-4b46-8093-3059b68a7490 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:17.668862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5afae9da-54bc-4b46-8093-3059b68a7490 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:17.917844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:17.932569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:17.936892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:17.941720 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-66cfeec9-a056-4d52-9644-58dbc17b4543 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1359 microversion: 2.21 time: 0.375499 Sep 27 10:02:17.957454 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 805/1470] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:17 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1359 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:17.957454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5f0fbe8d-bbe3-4828-b360-08e9355cb498 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:17.957454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5f0fbe8d-bbe3-4828-b360-08e9355cb498 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:17.974787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f0fbe8d-bbe3-4828-b360-08e9355cb498 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:17.976813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f0fbe8d-bbe3-4828-b360-08e9355cb498 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:17.977545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f0fbe8d-bbe3-4828-b360-08e9355cb498 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:17.978657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f0fbe8d-bbe3-4828-b360-08e9355cb498 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:17.980456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f0fbe8d-bbe3-4828-b360-08e9355cb498 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:17.981133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f0fbe8d-bbe3-4828-b360-08e9355cb498 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:18.005915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5afae9da-54bc-4b46-8093-3059b68a7490 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:18.021788 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5afae9da-54bc-4b46-8093-3059b68a7490 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1360 microversion: 2.1 time: 0.386468 Sep 27 10:02:18.024554 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 666/1471] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:17 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1360 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:18.263712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-937b2864-fd2e-475e-87c1-08a94bf80e1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:18.264601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-937b2864-fd2e-475e-87c1-08a94bf80e1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:18.271387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-937b2864-fd2e-475e-87c1-08a94bf80e1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:18.272048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-937b2864-fd2e-475e-87c1-08a94bf80e1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:18.272505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-937b2864-fd2e-475e-87c1-08a94bf80e1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:18.273382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-937b2864-fd2e-475e-87c1-08a94bf80e1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:18.274183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-937b2864-fd2e-475e-87c1-08a94bf80e1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:18.274858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-937b2864-fd2e-475e-87c1-08a94bf80e1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:18.315643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f0fbe8d-bbe3-4828-b360-08e9355cb498 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:18.334181 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5f0fbe8d-bbe3-4828-b360-08e9355cb498 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.386237 Sep 27 10:02:18.338500 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 806/1472] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:17 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:18.524725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-937b2864-fd2e-475e-87c1-08a94bf80e1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:18.533239 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-937b2864-fd2e-475e-87c1-08a94bf80e1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1308 microversion: 2.1 time: 0.273503 Sep 27 10:02:18.549391 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 667/1473] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:18 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1308 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:18.591540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3a925f8d-8053-4342-ba58-18900b4fb53d req-6bae51fe-34b4-4e4e-a9b9-9b6a04719f80 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1", "tag": "cc9dca96-4fbe-402f-b886-7bb414b8545d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:18.625657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3a925f8d-8053-4342-ba58-18900b4fb53d req-6bae51fe-34b4-4e4e-a9b9-9b6a04719f80 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:18.626147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3a925f8d-8053-4342-ba58-18900b4fb53d req-6bae51fe-34b4-4e4e-a9b9-9b6a04719f80 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:18.626576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3a925f8d-8053-4342-ba58-18900b4fb53d req-6bae51fe-34b4-4e4e-a9b9-9b6a04719f80 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:18.654764 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3a925f8d-8053-4342-ba58-18900b4fb53d req-6bae51fe-34b4-4e4e-a9b9-9b6a04719f80 service nova] Creating event network-changed:cc9dca96-4fbe-402f-b886-7bb414b8545d for instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 on nch2-master-1 Sep 27 10:02:18.661278 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3a925f8d-8053-4342-ba58-18900b4fb53d req-6bae51fe-34b4-4e4e-a9b9-9b6a04719f80 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072698 Sep 27 10:02:18.663132 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 807/1474] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:18.970989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:18.971652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:18.979590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:18.980111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:18.980559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:18.981323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:18.981984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:18.982422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:19.073741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2bd3e2ec-17f1-4de7-bebc-4a2cf50e2ced tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:19.078474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2bd3e2ec-17f1-4de7-bebc-4a2cf50e2ced tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:19.093868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bd3e2ec-17f1-4de7-bebc-4a2cf50e2ced tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:19.094717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bd3e2ec-17f1-4de7-bebc-4a2cf50e2ced tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:19.095411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bd3e2ec-17f1-4de7-bebc-4a2cf50e2ced tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:19.096606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bd3e2ec-17f1-4de7-bebc-4a2cf50e2ced tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:19.097337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bd3e2ec-17f1-4de7-bebc-4a2cf50e2ced tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:19.098663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bd3e2ec-17f1-4de7-bebc-4a2cf50e2ced tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:19.224128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:19.234245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:19.236091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:19.237433 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-75f1fb01-a00d-413e-8449-94c2f91472bb tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1359 microversion: 2.21 time: 0.270466 Sep 27 10:02:19.238528 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 668/1475] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:18 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1359 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:19.377668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73e5a5c8-1f97-4762-9066-e948695e3286 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:19.378605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-73e5a5c8-1f97-4762-9066-e948695e3286 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:19.388395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73e5a5c8-1f97-4762-9066-e948695e3286 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:19.388973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73e5a5c8-1f97-4762-9066-e948695e3286 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:19.389472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73e5a5c8-1f97-4762-9066-e948695e3286 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:19.390233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73e5a5c8-1f97-4762-9066-e948695e3286 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:19.391429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73e5a5c8-1f97-4762-9066-e948695e3286 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:19.391877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73e5a5c8-1f97-4762-9066-e948695e3286 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:19.461370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2bd3e2ec-17f1-4de7-bebc-4a2cf50e2ced tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:19.474283 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2bd3e2ec-17f1-4de7-bebc-4a2cf50e2ced tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1360 microversion: 2.1 time: 0.426208 Sep 27 10:02:19.475559 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 808/1476] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:19 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1360 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:19.481715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-b6ea4802-9b37-434b-91f9-3a7b92f3f6c4 req-6df95b4e-7bbe-437e-91a9-92cbce38e007 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e", "name": "network-vif-plugged", "status": "completed", "tag": "4ba20389-14eb-48c5-a482-6e64d2aef60d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:19.499073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b6ea4802-9b37-434b-91f9-3a7b92f3f6c4 req-6df95b4e-7bbe-437e-91a9-92cbce38e007 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:19.499893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b6ea4802-9b37-434b-91f9-3a7b92f3f6c4 req-6df95b4e-7bbe-437e-91a9-92cbce38e007 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:19.500335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b6ea4802-9b37-434b-91f9-3a7b92f3f6c4 req-6df95b4e-7bbe-437e-91a9-92cbce38e007 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:19.516977 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b6ea4802-9b37-434b-91f9-3a7b92f3f6c4 req-6df95b4e-7bbe-437e-91a9-92cbce38e007 service nova] Creating event network-vif-plugged:4ba20389-14eb-48c5-a482-6e64d2aef60d for instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e on nch1-master-1 Sep 27 10:02:19.522357 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-b6ea4802-9b37-434b-91f9-3a7b92f3f6c4 req-6df95b4e-7bbe-437e-91a9-92cbce38e007 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045287 Sep 27 10:02:19.523243 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 809/1477] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:19.558935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-565d2c96-3b2a-40bd-a7f9-2df58b47a355 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:19.559673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-565d2c96-3b2a-40bd-a7f9-2df58b47a355 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:19.575835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-565d2c96-3b2a-40bd-a7f9-2df58b47a355 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:19.576689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-565d2c96-3b2a-40bd-a7f9-2df58b47a355 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:19.577328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-565d2c96-3b2a-40bd-a7f9-2df58b47a355 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:19.579387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-565d2c96-3b2a-40bd-a7f9-2df58b47a355 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:19.579387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-565d2c96-3b2a-40bd-a7f9-2df58b47a355 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:19.579387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-565d2c96-3b2a-40bd-a7f9-2df58b47a355 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:19.584235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:19.763763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-73e5a5c8-1f97-4762-9066-e948695e3286 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:19.772921 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-73e5a5c8-1f97-4762-9066-e948695e3286 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.400764 Sep 27 10:02:19.773709 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 669/1478] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:19 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:19.792993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-565d2c96-3b2a-40bd-a7f9-2df58b47a355 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:19.808569 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-565d2c96-3b2a-40bd-a7f9-2df58b47a355 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1308 microversion: 2.1 time: 0.253690 Sep 27 10:02:19.810288 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 810/1479] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:19 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1308 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:20.263394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:20.264640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:20.271285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:20.272122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:20.273506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:20.274614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:20.275476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:20.276223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:20.498575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-68119293-b085-4bfe-861e-c031589a17ad tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:20.499934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-68119293-b085-4bfe-861e-c031589a17ad tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:20.508888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68119293-b085-4bfe-861e-c031589a17ad tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:20.509714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68119293-b085-4bfe-861e-c031589a17ad tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:20.510415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68119293-b085-4bfe-861e-c031589a17ad tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:20.511407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68119293-b085-4bfe-861e-c031589a17ad tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:20.512047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68119293-b085-4bfe-861e-c031589a17ad tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:20.512709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68119293-b085-4bfe-861e-c031589a17ad tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:20.594099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:20.610299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:20.621907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:20.624494 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4a251023-dd88-47f3-812d-32a89e72d375 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1359 microversion: 2.21 time: 0.365377 Sep 27 10:02:20.626163 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 670/1480] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:20 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1359 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:20.801239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-66e71b9a-7e1f-45b5-b352-0a6d6e2c4ede tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:20.802801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-66e71b9a-7e1f-45b5-b352-0a6d6e2c4ede tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:20.810327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66e71b9a-7e1f-45b5-b352-0a6d6e2c4ede tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:20.811257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66e71b9a-7e1f-45b5-b352-0a6d6e2c4ede tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:20.811958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66e71b9a-7e1f-45b5-b352-0a6d6e2c4ede tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:20.813178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66e71b9a-7e1f-45b5-b352-0a6d6e2c4ede tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:20.813921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66e71b9a-7e1f-45b5-b352-0a6d6e2c4ede tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:20.814653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66e71b9a-7e1f-45b5-b352-0a6d6e2c4ede tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:21.108347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-68119293-b085-4bfe-861e-c031589a17ad tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:21.123342 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-68119293-b085-4bfe-861e-c031589a17ad tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1544 microversion: 2.1 time: 0.628281 Sep 27 10:02:21.144425 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 811/1481] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:20 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1544 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:21.154647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-80b77b12-f3b3-4cb5-bfbc-78cf2dc52472 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:21.155502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-80b77b12-f3b3-4cb5-bfbc-78cf2dc52472 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:21.164770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b77b12-f3b3-4cb5-bfbc-78cf2dc52472 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:21.166393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b77b12-f3b3-4cb5-bfbc-78cf2dc52472 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:21.167062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b77b12-f3b3-4cb5-bfbc-78cf2dc52472 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:21.168165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b77b12-f3b3-4cb5-bfbc-78cf2dc52472 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:21.169023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b77b12-f3b3-4cb5-bfbc-78cf2dc52472 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:21.170613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b77b12-f3b3-4cb5-bfbc-78cf2dc52472 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:21.217307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-66e71b9a-7e1f-45b5-b352-0a6d6e2c4ede tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:21.226922 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-66e71b9a-7e1f-45b5-b352-0a6d6e2c4ede tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.430871 Sep 27 10:02:21.244340 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 671/1482] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:20 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:21.256492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c1bec36f-cb07-41b5-b4a5-12dd9bb2f889 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:21.257135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c1bec36f-cb07-41b5-b4a5-12dd9bb2f889 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:21.264230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1bec36f-cb07-41b5-b4a5-12dd9bb2f889 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:21.264786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1bec36f-cb07-41b5-b4a5-12dd9bb2f889 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:21.265235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1bec36f-cb07-41b5-b4a5-12dd9bb2f889 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:21.265955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1bec36f-cb07-41b5-b4a5-12dd9bb2f889 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:21.266415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1bec36f-cb07-41b5-b4a5-12dd9bb2f889 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:21.266877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1bec36f-cb07-41b5-b4a5-12dd9bb2f889 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:21.384899 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c1bec36f-cb07-41b5-b4a5-12dd9bb2f889 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.138229 Sep 27 10:02:21.385914 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 672/1483] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:02:21 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/os-interface => generated 285 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:21.431997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-80b77b12-f3b3-4cb5-bfbc-78cf2dc52472 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:21.441855 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-80b77b12-f3b3-4cb5-bfbc-78cf2dc52472 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1308 microversion: 2.1 time: 0.294099 Sep 27 10:02:21.443232 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 812/1484] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:21 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1308 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:21.651028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:21.651800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:21.669212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:21.669989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:21.672421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:21.682598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:21.703458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:21.706666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.022s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:22.252854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62a66eee-0be0-4e22-81d0-3d984f81d6c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:22.253542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-62a66eee-0be0-4e22-81d0-3d984f81d6c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:22.265438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:22.275633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a66eee-0be0-4e22-81d0-3d984f81d6c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:22.280549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a66eee-0be0-4e22-81d0-3d984f81d6c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:22.287805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a66eee-0be0-4e22-81d0-3d984f81d6c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:22.289080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a66eee-0be0-4e22-81d0-3d984f81d6c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:22.290738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a66eee-0be0-4e22-81d0-3d984f81d6c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:22.291364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a66eee-0be0-4e22-81d0-3d984f81d6c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:22.436263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:22.449113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:22.452384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:22.456953 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4bc46f41-ab13-42da-bf13-fe0faa3cb915 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1552 microversion: 2.21 time: 0.810144 Sep 27 10:02:22.458297 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 673/1485] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:21 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1552 bytes in 812 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:22.468342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2b47eef6-f8b5-4d5f-b213-b05975b2736c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:22.470103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2b47eef6-f8b5-4d5f-b213-b05975b2736c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:22.480518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b47eef6-f8b5-4d5f-b213-b05975b2736c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:22.481545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b47eef6-f8b5-4d5f-b213-b05975b2736c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:22.482932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b47eef6-f8b5-4d5f-b213-b05975b2736c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:22.484213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b47eef6-f8b5-4d5f-b213-b05975b2736c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:22.485166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b47eef6-f8b5-4d5f-b213-b05975b2736c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:22.486112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b47eef6-f8b5-4d5f-b213-b05975b2736c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:22.774277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-62a66eee-0be0-4e22-81d0-3d984f81d6c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:22.777610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2b47eef6-f8b5-4d5f-b213-b05975b2736c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:22.786287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-62a66eee-0be0-4e22-81d0-3d984f81d6c5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.538272 Sep 27 10:02:22.788136 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2b47eef6-f8b5-4d5f-b213-b05975b2736c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1308 microversion: 2.1 time: 0.324705 Sep 27 10:02:22.788444 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 813/1486] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:22 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:22.793815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4876cc9b-5343-4585-bcd3-3dc08b3ad6c4 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:22.794476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4876cc9b-5343-4585-bcd3-3dc08b3ad6c4 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:22.811102 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 674/1487] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:22 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1308 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:22.827943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4876cc9b-5343-4585-bcd3-3dc08b3ad6c4 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:22.832193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4876cc9b-5343-4585-bcd3-3dc08b3ad6c4 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.028s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:22.833431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4876cc9b-5343-4585-bcd3-3dc08b3ad6c4 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:22.834642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4876cc9b-5343-4585-bcd3-3dc08b3ad6c4 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:22.837522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4876cc9b-5343-4585-bcd3-3dc08b3ad6c4 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:22.839090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4876cc9b-5343-4585-bcd3-3dc08b3ad6c4 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:22.899690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4876cc9b-5343-4585-bcd3-3dc08b3ad6c4 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:02:23.106602 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4876cc9b-5343-4585-bcd3-3dc08b3ad6c4 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "DELETE /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 204 len: 0 microversion: 2.21 time: 0.314456 Sep 27 10:02:23.107792 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 814/1488] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:02:22 2022] DELETE /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:02:23.120715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:23.121470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:23.146240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:23.147042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:23.147710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:23.148649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:23.149293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:23.149883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:23.187056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:23.639810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:23.653428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:23.656104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:23.658133 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-95d2fcaf-9f76-423c-a07c-ee147c0c80c9 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1558 microversion: 2.21 time: 0.545156 Sep 27 10:02:23.674639 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 675/1489] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:23 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1558 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:23.814970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-111d5a97-dfab-4dff-b715-a676bcd02c58 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:23.816285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-111d5a97-dfab-4dff-b715-a676bcd02c58 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:23.817023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:23.822014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2b3942dd-e784-432f-ab01-d37112899d25 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:23.822634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2b3942dd-e784-432f-ab01-d37112899d25 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:23.828080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=11 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:23.833318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-111d5a97-dfab-4dff-b715-a676bcd02c58 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:23.835539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b3942dd-e784-432f-ab01-d37112899d25 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:23.836235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b3942dd-e784-432f-ab01-d37112899d25 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:23.836444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b3942dd-e784-432f-ab01-d37112899d25 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:23.837325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-111d5a97-dfab-4dff-b715-a676bcd02c58 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:23.837832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-111d5a97-dfab-4dff-b715-a676bcd02c58 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:23.838598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-111d5a97-dfab-4dff-b715-a676bcd02c58 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:23.839083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-111d5a97-dfab-4dff-b715-a676bcd02c58 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:23.839518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-111d5a97-dfab-4dff-b715-a676bcd02c58 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:23.839705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b3942dd-e784-432f-ab01-d37112899d25 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:23.840346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b3942dd-e784-432f-ab01-d37112899d25 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:23.842074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2b3942dd-e784-432f-ab01-d37112899d25 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:24.335244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-111d5a97-dfab-4dff-b715-a676bcd02c58 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:24.345472 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-111d5a97-dfab-4dff-b715-a676bcd02c58 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1308 microversion: 2.1 time: 0.535182 Sep 27 10:02:24.363613 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 815/1490] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:23 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1308 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:24.417633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-88615e38-5c16-48fd-b829-b05dc28a5d55 req-65320724-0c56-43eb-af93-7d40a43180f4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "11a0d56b-3e1f-40e7-9e3c-30fca67df25e", "tag": "b3618e9e-e34a-4212-aa29-7747a00843a9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:24.442121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-88615e38-5c16-48fd-b829-b05dc28a5d55 req-65320724-0c56-43eb-af93-7d40a43180f4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:24.442616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-88615e38-5c16-48fd-b829-b05dc28a5d55 req-65320724-0c56-43eb-af93-7d40a43180f4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:24.443113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-88615e38-5c16-48fd-b829-b05dc28a5d55 req-65320724-0c56-43eb-af93-7d40a43180f4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:24.470419 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-88615e38-5c16-48fd-b829-b05dc28a5d55 req-65320724-0c56-43eb-af93-7d40a43180f4 service nova] Creating event network-changed:b3618e9e-e34a-4212-aa29-7747a00843a9 for instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e on nch1-master-1 Sep 27 10:02:24.479545 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-88615e38-5c16-48fd-b829-b05dc28a5d55 req-65320724-0c56-43eb-af93-7d40a43180f4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065748 Sep 27 10:02:24.480826 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 816/1491] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:02:24.688988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:24.690129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:24.704629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:24.705993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:24.708541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:24.709665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:24.712576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:24.713446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:24.802569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2b3942dd-e784-432f-ab01-d37112899d25 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:24.815942 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2b3942dd-e784-432f-ab01-d37112899d25 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.999539 Sep 27 10:02:24.816730 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 676/1492] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:23 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 1001 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:25.370475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c16a9e75-aee1-4bb1-a5cb-d8e5f302cf83 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:25.371097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c16a9e75-aee1-4bb1-a5cb-d8e5f302cf83 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:25.381906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c16a9e75-aee1-4bb1-a5cb-d8e5f302cf83 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:25.382477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c16a9e75-aee1-4bb1-a5cb-d8e5f302cf83 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:25.382930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c16a9e75-aee1-4bb1-a5cb-d8e5f302cf83 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:25.383664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c16a9e75-aee1-4bb1-a5cb-d8e5f302cf83 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:25.384126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c16a9e75-aee1-4bb1-a5cb-d8e5f302cf83 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:25.390095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c16a9e75-aee1-4bb1-a5cb-d8e5f302cf83 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:25.620195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:25.633263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:25.635054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:25.637354 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4b4de14-6663-4ff3-8a85-a39d624822cf tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1558 microversion: 2.21 time: 0.955448 Sep 27 10:02:25.642836 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 817/1493] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:24 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1558 bytes in 961 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:25.844102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c53e0e8-590e-4589-a5c5-df360307142e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:25.845398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2c53e0e8-590e-4589-a5c5-df360307142e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:25.862271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c53e0e8-590e-4589-a5c5-df360307142e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:25.863910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c53e0e8-590e-4589-a5c5-df360307142e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:25.865501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c53e0e8-590e-4589-a5c5-df360307142e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:25.867200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c53e0e8-590e-4589-a5c5-df360307142e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:25.871686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c53e0e8-590e-4589-a5c5-df360307142e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:25.872112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c53e0e8-590e-4589-a5c5-df360307142e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:26.113945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c16a9e75-aee1-4bb1-a5cb-d8e5f302cf83 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:26.127898 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c16a9e75-aee1-4bb1-a5cb-d8e5f302cf83 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1308 microversion: 2.1 time: 0.761127 Sep 27 10:02:26.129358 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 677/1494] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:25 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1308 bytes in 763 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:26.674653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:26.675293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:26.696166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:26.700839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:26.701368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:26.702141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:26.705864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:26.708077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:26.959644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2c53e0e8-590e-4589-a5c5-df360307142e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:26.973947 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c53e0e8-590e-4589-a5c5-df360307142e tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 1.137568 Sep 27 10:02:26.976062 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 818/1495] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:25 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 1140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:26.980445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f6f41a90-0296-4741-b8ca-f266e461aa54 req-94771056-d097-4771-b4d4-1ef103eb628a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e", "name": "network-vif-deleted", "tag": "4ba20389-14eb-48c5-a482-6e64d2aef60d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:27.016257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f6f41a90-0296-4741-b8ca-f266e461aa54 req-94771056-d097-4771-b4d4-1ef103eb628a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:27.017038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f6f41a90-0296-4741-b8ca-f266e461aa54 req-94771056-d097-4771-b4d4-1ef103eb628a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:27.017615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f6f41a90-0296-4741-b8ca-f266e461aa54 req-94771056-d097-4771-b4d4-1ef103eb628a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:27.031598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:27.046757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:27.050476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'b31b5932bb8e4edcadc6954c46c2c386', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2750135860944b03816344b2e7e58f30', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:27.054229 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0bcc1b5e-117a-4edd-b532-deeac09e7812 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 200 len: 1516 microversion: 2.21 time: 0.386518 Sep 27 10:02:27.055557 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f6f41a90-0296-4741-b8ca-f266e461aa54 req-94771056-d097-4771-b4d4-1ef103eb628a service nova] Creating event network-vif-deleted:4ba20389-14eb-48c5-a482-6e64d2aef60d for instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e on nch1-master-1 Sep 27 10:02:27.059550 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 678/1496] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:26 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 1516 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:02:27.062531 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f6f41a90-0296-4741-b8ca-f266e461aa54 req-94771056-d097-4771-b4d4-1ef103eb628a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085790 Sep 27 10:02:27.063740 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 819/1497] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:27.153013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5e65ab68-8177-469f-8fb4-096226c87b01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:27.153905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5e65ab68-8177-469f-8fb4-096226c87b01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:27.175127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e65ab68-8177-469f-8fb4-096226c87b01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:27.175819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e65ab68-8177-469f-8fb4-096226c87b01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:27.176916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e65ab68-8177-469f-8fb4-096226c87b01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:27.177833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e65ab68-8177-469f-8fb4-096226c87b01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:27.178451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e65ab68-8177-469f-8fb4-096226c87b01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:27.179109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e65ab68-8177-469f-8fb4-096226c87b01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:27.414795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5e65ab68-8177-469f-8fb4-096226c87b01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:27.430657 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5e65ab68-8177-469f-8fb4-096226c87b01 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1308 microversion: 2.1 time: 0.279991 Sep 27 10:02:27.431316 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 679/1498] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:27 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1308 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:28.002192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e9c187a7-887d-4522-94fb-b9fb1d4bcc86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:28.009266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e9c187a7-887d-4522-94fb-b9fb1d4bcc86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:28.021505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9c187a7-887d-4522-94fb-b9fb1d4bcc86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.022261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9c187a7-887d-4522-94fb-b9fb1d4bcc86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.022884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9c187a7-887d-4522-94fb-b9fb1d4bcc86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.023839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9c187a7-887d-4522-94fb-b9fb1d4bcc86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.024492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9c187a7-887d-4522-94fb-b9fb1d4bcc86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.025142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9c187a7-887d-4522-94fb-b9fb1d4bcc86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.086989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:28.088090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:28.126803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.139802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.140971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.142187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.142736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.143397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.171970 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] HTTP exception thrown: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Request method failure captured: Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e HTTP/1.1 Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsoTXLKh36rVOpInUKk6zZMdYjDASqi-Tqd8UQ2pCoM6g2vecYfovH1cndvIvcggzhJ82RkTYR-g0LJt_vSts7k6kgIZJglfmKUdrhrHD221FpclKpeyRgaE2LbkF1MxE_Ib8USpwd5LaEtGFO40QHp4CBgxpNWco9v_wAA2HyJb3CA Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.21 Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2750135860944b03816344b2e7e58f30 Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-InstanceActionsV221TestJSON-1398653790 Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:02:28.174550 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2750135860944b03816344b2e7e58f30"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2750135860944b03816344b2e7e58f30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2750135860944b03816344b2e7e58f30"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2750135860944b03816344b2e7e58f30"}]}] Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-InstanceActionsV221TestJSON-1398653790 Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2750135860944b03816344b2e7e58f30 Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-InstanceActionsV221TestJSON-1398653790 Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-InstanceActionsV221TestJSON-1398653790-project Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b31b5932bb8e4edcadc6954c46c2c386 Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-InstanceActionsV221TestJSON-1398653790-project Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.176110 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e'} Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:28.177336 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.178712 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.181546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Returning 404 to user: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:28.182792 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-25bb14da-406e-4cee-852b-ae6de440a43a tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 404 len: 111 microversion: 2.21 time: 0.102288 Sep 27 10:02:28.187591 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 680/1499] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:28 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 111 bytes in 108 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:02:28.450718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-86bed1be-07c0-45bb-b0c1-f01b8d3c26e0 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Action: 'create', calling method: .version_select at 0x7f7f6390e9d0>, body: {"keypair": {"name": "tempest-keypair-253434363", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOIGIddJ+rbh3jK1IwDwKbYizMRGDZDeFeEcY1fCSAcvY4OEzh6EyhQNF3+0xZfpYlyzpFlvh4tDwUxpPhRyWTgPil/V1zxLWiZna4Nxe8eW71MCBT9hcv9MdHKmPeDkOg=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:28.487390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-86bed1be-07c0-45bb-b0c1-f01b8d3c26e0 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Getting quotas for project 183cb066566c4510945d6eb0d41a2853. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:02:28.494721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:28.496299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-86bed1be-07c0-45bb-b0c1-f01b8d3c26e0 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Getting quotas for user 06dc3376ada04ffe922c9023f4182259 and project 183cb066566c4510945d6eb0d41a2853. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:02:28.555893 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-86bed1be-07c0-45bb-b0c1-f01b8d3c26e0 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.366413 Sep 27 10:02:28.556899 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 681/1500] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:02:28 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:28.562820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c25134dc-80cc-4d59-9a02-c43fe49d371e tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '.version_select at 0x7f7f637eb280>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:28.563729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c25134dc-80cc-4d59-9a02-c43fe49d371e tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:28.571975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c25134dc-80cc-4d59-9a02-c43fe49d371e tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.572734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c25134dc-80cc-4d59-9a02-c43fe49d371e tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.573376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c25134dc-80cc-4d59-9a02-c43fe49d371e tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.574326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c25134dc-80cc-4d59-9a02-c43fe49d371e tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.575110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c25134dc-80cc-4d59-9a02-c43fe49d371e tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.575744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c25134dc-80cc-4d59-9a02-c43fe49d371e tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.579624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e9c187a7-887d-4522-94fb-b9fb1d4bcc86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:28.615628 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e9c187a7-887d-4522-94fb-b9fb1d4bcc86 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.619783 Sep 27 10:02:28.625288 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 820/1501] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:27 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:28.638997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5d2293a-0dc2-4503-a99e-a4bf64ee9b57 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:28.639737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a5d2293a-0dc2-4503-a99e-a4bf64ee9b57 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:28.663581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5d2293a-0dc2-4503-a99e-a4bf64ee9b57 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.668019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5d2293a-0dc2-4503-a99e-a4bf64ee9b57 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.670634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5d2293a-0dc2-4503-a99e-a4bf64ee9b57 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.673396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5d2293a-0dc2-4503-a99e-a4bf64ee9b57 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.673889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5d2293a-0dc2-4503-a99e-a4bf64ee9b57 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.674328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5d2293a-0dc2-4503-a99e-a4bf64ee9b57 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.683827 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c25134dc-80cc-4d59-9a02-c43fe49d371e tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e/os-instance-actions" status: 200 len: 611 microversion: 2.21 time: 0.125190 Sep 27 10:02:28.693664 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 682/1502] 10.222.0.26 () {60 vars in 1337 bytes} [Tue Sep 27 10:02:28 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e/os-instance-actions => generated 611 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:02:28.707597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:28.709944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:28.749078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.750014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.750699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.751701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.752383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.753154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.776266 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] HTTP exception thrown: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Request method failure captured: Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e HTTP/1.1 Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsoTXLKh36rVOpInUKk6zZMdYjDASqi-Tqd8UQ2pCoM6g2vecYfovH1cndvIvcggzhJ82RkTYR-g0LJt_vSts7k6kgIZJglfmKUdrhrHD221FpclKpeyRgaE2LbkF1MxE_Ib8USpwd5LaEtGFO40QHp4CBgxpNWco9v_wAA2HyJb3CA Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.21 Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2750135860944b03816344b2e7e58f30 Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-InstanceActionsV221TestJSON-1398653790 Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:02:28.779996 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2750135860944b03816344b2e7e58f30"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2750135860944b03816344b2e7e58f30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2750135860944b03816344b2e7e58f30"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2750135860944b03816344b2e7e58f30"}]}] Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-InstanceActionsV221TestJSON-1398653790 Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2750135860944b03816344b2e7e58f30 Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-InstanceActionsV221TestJSON-1398653790 Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-InstanceActionsV221TestJSON-1398653790-project Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b31b5932bb8e4edcadc6954c46c2c386 Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-InstanceActionsV221TestJSON-1398653790-project Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.781842 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e'} Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:28.782747 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.784265 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.786289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Returning 404 to user: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:28.787057 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-647d1bc2-88a6-43e0-8538-edc5c726c1f7 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "DELETE /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 404 len: 111 microversion: 2.21 time: 0.091875 Sep 27 10:02:28.792962 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 683/1503] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:02:28 2022] DELETE /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 111 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:02:28.803842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:28.805468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] [instance: 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:28.817919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.818632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.819222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.820240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:28.821735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:28.822374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:28.890669 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] HTTP exception thrown: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Request method failure captured: Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e HTTP/1.1 Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsoTXLKh36rVOpInUKk6zZMdYjDASqi-Tqd8UQ2pCoM6g2vecYfovH1cndvIvcggzhJ82RkTYR-g0LJt_vSts7k6kgIZJglfmKUdrhrHD221FpclKpeyRgaE2LbkF1MxE_Ib8USpwd5LaEtGFO40QHp4CBgxpNWco9v_wAA2HyJb3CA Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.21 Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2750135860944b03816344b2e7e58f30 Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-InstanceActionsV221TestJSON-1398653790 Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:02:28.892369 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2750135860944b03816344b2e7e58f30"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2750135860944b03816344b2e7e58f30"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2750135860944b03816344b2e7e58f30"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2750135860944b03816344b2e7e58f30"}]}] Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-InstanceActionsV221TestJSON-1398653790 Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2750135860944b03816344b2e7e58f30 Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-InstanceActionsV221TestJSON-1398653790 Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-InstanceActionsV221TestJSON-1398653790-project Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b31b5932bb8e4edcadc6954c46c2c386 Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-InstanceActionsV221TestJSON-1398653790-project Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.894090 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e'} Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:02:28.894908 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. Sep 27 10:02:28.896291 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:02:28.897433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] Returning 404 to user: Instance 17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:02:28.897433 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f7bd003e-e8a8-4cc6-b598-68cc5d1586d8 tempest-InstanceActionsV221TestJSON-1398653790 tempest-InstanceActionsV221TestJSON-1398653790-project] 10.222.0.26 "GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e" status: 404 len: 111 microversion: 2.21 time: 0.099733 Sep 27 10:02:28.897433 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 684/1504] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:02:28 2022] GET /compute/v2.1/servers/17ab92e5-8ac7-4f8b-9a9a-c33a15f5218e => generated 111 bytes in 101 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:02:29.030730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FloatingIPsAssociationTestJSON-server-2032301310", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "79454fbf-4335-40b3-b1c5-c3f69544e188"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:29.108134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a5d2293a-0dc2-4503-a99e-a4bf64ee9b57 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:29.145344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] validate_networks() for [('79454fbf-4335-40b3-b1c5-c3f69544e188', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:02:29.146929 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a5d2293a-0dc2-4503-a99e-a4bf64ee9b57 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1308 microversion: 2.1 time: 0.519819 Sep 27 10:02:29.167634 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 821/1505] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:28 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1308 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:29.344587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f46ac483-61e3-4f86-8e1a-6f0751572fb7 req-ac1bcc9d-90eb-4352-a23a-2c4c9261cf96 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1", "name": "network-vif-plugged", "status": "completed", "tag": "cc9dca96-4fbe-402f-b886-7bb414b8545d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:29.370546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f46ac483-61e3-4f86-8e1a-6f0751572fb7 req-ac1bcc9d-90eb-4352-a23a-2c4c9261cf96 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:29.371948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f46ac483-61e3-4f86-8e1a-6f0751572fb7 req-ac1bcc9d-90eb-4352-a23a-2c4c9261cf96 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:29.372690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f46ac483-61e3-4f86-8e1a-6f0751572fb7 req-ac1bcc9d-90eb-4352-a23a-2c4c9261cf96 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:29.403282 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f46ac483-61e3-4f86-8e1a-6f0751572fb7 req-ac1bcc9d-90eb-4352-a23a-2c4c9261cf96 service nova] Creating event network-vif-plugged:cc9dca96-4fbe-402f-b886-7bb414b8545d for instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 on nch2-master-1 Sep 27 10:02:29.421770 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f46ac483-61e3-4f86-8e1a-6f0751572fb7 req-ac1bcc9d-90eb-4352-a23a-2c4c9261cf96 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077801 Sep 27 10:02:29.422759 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 822/1506] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:29.640352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-342b0afd-7476-4948-bada-543ea2ab3920 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:29.641489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-342b0afd-7476-4948-bada-543ea2ab3920 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:29.648918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-342b0afd-7476-4948-bada-543ea2ab3920 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:29.649740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-342b0afd-7476-4948-bada-543ea2ab3920 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:29.650178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-342b0afd-7476-4948-bada-543ea2ab3920 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:29.651337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-342b0afd-7476-4948-bada-543ea2ab3920 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:29.651945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-342b0afd-7476-4948-bada-543ea2ab3920 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:29.652384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-342b0afd-7476-4948-bada-543ea2ab3920 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:30.045032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-342b0afd-7476-4948-bada-543ea2ab3920 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:30.056478 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-342b0afd-7476-4948-bada-543ea2ab3920 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.420316 Sep 27 10:02:30.057733 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 823/1507] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:29 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:30.101453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:02:30.101958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:02:30.102632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:02:30.108240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:02:30.108977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:02:30.121298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:30.121793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:30.122476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:30.123522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:30.124150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:30.124812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:30.162033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Getting quotas for project de87d5984a9a4d4481ab4921133841d6. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:02:30.168628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Getting quotas for user c7468cc8bc7d4dde8e2a5eba1fa42fd8 and project de87d5984a9a4d4481ab4921133841d6. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:02:30.177739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4b2c6366-e7a0-42a8-9ad1-98a953b9ecb1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:30.179658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4b2c6366-e7a0-42a8-9ad1-98a953b9ecb1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:30.190271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:02:30.193307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2c6366-e7a0-42a8-9ad1-98a953b9ecb1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:30.198536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2c6366-e7a0-42a8-9ad1-98a953b9ecb1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:30.200115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:02:30.201907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2c6366-e7a0-42a8-9ad1-98a953b9ecb1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:30.203044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2c6366-e7a0-42a8-9ad1-98a953b9ecb1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:30.205302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2c6366-e7a0-42a8-9ad1-98a953b9ecb1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:30.205867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2c6366-e7a0-42a8-9ad1-98a953b9ecb1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:30.275969 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-59910464-d10a-422a-837a-3717ea7dee64 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.360819 Sep 27 10:02:30.277430 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 685/1508] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:02:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 1382 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:02:30.288766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-01d49f13-05a1-4892-8dec-47d215811d8a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:30.292847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-01d49f13-05a1-4892-8dec-47d215811d8a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:30.469798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4b2c6366-e7a0-42a8-9ad1-98a953b9ecb1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:30.483044 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4b2c6366-e7a0-42a8-9ad1-98a953b9ecb1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1308 microversion: 2.1 time: 0.311673 Sep 27 10:02:30.483658 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 824/1509] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:30 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1308 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:30.676182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-01d49f13-05a1-4892-8dec-47d215811d8a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:30.688702 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-01d49f13-05a1-4892-8dec-47d215811d8a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1224 microversion: 2.1 time: 0.404283 Sep 27 10:02:30.691659 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 686/1510] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:30 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1224 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:31.084365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c195f490-59c5-4bce-b461-61e7d04fc124 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:31.085232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c195f490-59c5-4bce-b461-61e7d04fc124 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:31.098468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195f490-59c5-4bce-b461-61e7d04fc124 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:31.098908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195f490-59c5-4bce-b461-61e7d04fc124 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:31.103185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195f490-59c5-4bce-b461-61e7d04fc124 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:31.105541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195f490-59c5-4bce-b461-61e7d04fc124 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:31.110455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195f490-59c5-4bce-b461-61e7d04fc124 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:31.110910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195f490-59c5-4bce-b461-61e7d04fc124 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:31.507150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c58b7156-773d-4519-832b-4af815cbe257 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:31.507837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c58b7156-773d-4519-832b-4af815cbe257 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:31.525038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c58b7156-773d-4519-832b-4af815cbe257 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:31.525556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c58b7156-773d-4519-832b-4af815cbe257 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:31.526006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c58b7156-773d-4519-832b-4af815cbe257 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:31.526754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c58b7156-773d-4519-832b-4af815cbe257 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:31.527307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c58b7156-773d-4519-832b-4af815cbe257 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:31.527790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c58b7156-773d-4519-832b-4af815cbe257 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:31.626316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c195f490-59c5-4bce-b461-61e7d04fc124 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:31.639957 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c195f490-59c5-4bce-b461-61e7d04fc124 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.559789 Sep 27 10:02:31.640947 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 825/1511] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:31 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:31.718041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-80d5b596-abc6-4386-b650-32c5fb5390f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:31.718956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-80d5b596-abc6-4386-b650-32c5fb5390f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:31.726991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80d5b596-abc6-4386-b650-32c5fb5390f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:31.727787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80d5b596-abc6-4386-b650-32c5fb5390f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:31.728462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80d5b596-abc6-4386-b650-32c5fb5390f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:31.729480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80d5b596-abc6-4386-b650-32c5fb5390f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:31.730098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80d5b596-abc6-4386-b650-32c5fb5390f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:31.730688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80d5b596-abc6-4386-b650-32c5fb5390f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:31.843238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-80d5b596-abc6-4386-b650-32c5fb5390f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:31.853309 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-80d5b596-abc6-4386-b650-32c5fb5390f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1276 microversion: 2.1 time: 0.139493 Sep 27 10:02:31.854054 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 826/1512] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:31 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1276 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:32.321474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c58b7156-773d-4519-832b-4af815cbe257 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:32.334708 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c58b7156-773d-4519-832b-4af815cbe257 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1308 microversion: 2.1 time: 0.830799 Sep 27 10:02:32.336064 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 687/1513] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:31 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1308 bytes in 834 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:32.667270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-376f5fa2-acd1-453b-a359-dbcbee62fe77 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:32.667969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-376f5fa2-acd1-453b-a359-dbcbee62fe77 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:32.681944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-376f5fa2-acd1-453b-a359-dbcbee62fe77 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:32.682753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-376f5fa2-acd1-453b-a359-dbcbee62fe77 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:32.683728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-376f5fa2-acd1-453b-a359-dbcbee62fe77 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:32.685987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-376f5fa2-acd1-453b-a359-dbcbee62fe77 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:32.688283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-376f5fa2-acd1-453b-a359-dbcbee62fe77 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:32.690134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-376f5fa2-acd1-453b-a359-dbcbee62fe77 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:32.700442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:32.879497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e05461f5-3a03-4777-b450-9eb97ecfadc0 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:32.880274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e05461f5-3a03-4777-b450-9eb97ecfadc0 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:32.904286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e05461f5-3a03-4777-b450-9eb97ecfadc0 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:32.904967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e05461f5-3a03-4777-b450-9eb97ecfadc0 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:32.905525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e05461f5-3a03-4777-b450-9eb97ecfadc0 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:32.909047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e05461f5-3a03-4777-b450-9eb97ecfadc0 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:32.909636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e05461f5-3a03-4777-b450-9eb97ecfadc0 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:32.910165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e05461f5-3a03-4777-b450-9eb97ecfadc0 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:33.029656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e05461f5-3a03-4777-b450-9eb97ecfadc0 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:33.039978 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e05461f5-3a03-4777-b450-9eb97ecfadc0 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1282 microversion: 2.1 time: 0.163925 Sep 27 10:02:33.041106 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 688/1514] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:32 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1282 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:33.360473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1caa887-0f39-45c4-a74b-127ec1615d15 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:33.361142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a1caa887-0f39-45c4-a74b-127ec1615d15 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:33.373157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1caa887-0f39-45c4-a74b-127ec1615d15 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:33.373700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1caa887-0f39-45c4-a74b-127ec1615d15 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:33.374161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1caa887-0f39-45c4-a74b-127ec1615d15 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:33.377957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1caa887-0f39-45c4-a74b-127ec1615d15 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:33.378701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1caa887-0f39-45c4-a74b-127ec1615d15 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:33.379313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1caa887-0f39-45c4-a74b-127ec1615d15 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:33.669936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-376f5fa2-acd1-453b-a359-dbcbee62fe77 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:33.687394 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-376f5fa2-acd1-453b-a359-dbcbee62fe77 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 1.026622 Sep 27 10:02:33.688331 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 827/1515] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:32 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 1029 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:33.736314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a1caa887-0f39-45c4-a74b-127ec1615d15 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:33.746777 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a1caa887-0f39-45c4-a74b-127ec1615d15 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1489 microversion: 2.1 time: 0.390495 Sep 27 10:02:33.751501 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 689/1516] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:33 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1489 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:33.778383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5f09a4a0-b33b-4d91-bf6a-4e67318e2fec tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:33.790529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f09a4a0-b33b-4d91-bf6a-4e67318e2fec tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:33.791507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f09a4a0-b33b-4d91-bf6a-4e67318e2fec tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:33.794174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f09a4a0-b33b-4d91-bf6a-4e67318e2fec tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:33.927295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5f09a4a0-b33b-4d91-bf6a-4e67318e2fec tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Going to try to stop instance {{(pid=145968) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 10:02:34.066929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a2448610-6a5a-4d8a-b35d-6b59176746dc tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:34.069107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a2448610-6a5a-4d8a-b35d-6b59176746dc tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:34.083263 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5f09a4a0-b33b-4d91-bf6a-4e67318e2fec tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "POST /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1/action" status: 202 len: 0 microversion: 2.1 time: 0.294657 Sep 27 10:02:34.083960 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 828/1517] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:02:33 2022] POST /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1/action => generated 0 bytes in 312 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:02:34.094428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2448610-6a5a-4d8a-b35d-6b59176746dc tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:34.095250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2448610-6a5a-4d8a-b35d-6b59176746dc tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:34.097700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2448610-6a5a-4d8a-b35d-6b59176746dc tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:34.106828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2448610-6a5a-4d8a-b35d-6b59176746dc tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:34.108954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9904e09b-3bdc-42b7-afdc-c3fab4b35406 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:34.109721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2448610-6a5a-4d8a-b35d-6b59176746dc tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:34.112414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2448610-6a5a-4d8a-b35d-6b59176746dc tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:34.117142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9904e09b-3bdc-42b7-afdc-c3fab4b35406 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:34.161246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9904e09b-3bdc-42b7-afdc-c3fab4b35406 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:34.167115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9904e09b-3bdc-42b7-afdc-c3fab4b35406 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:34.167829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9904e09b-3bdc-42b7-afdc-c3fab4b35406 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:34.171022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9904e09b-3bdc-42b7-afdc-c3fab4b35406 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:34.174711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9904e09b-3bdc-42b7-afdc-c3fab4b35406 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:34.175400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9904e09b-3bdc-42b7-afdc-c3fab4b35406 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:34.386599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a2448610-6a5a-4d8a-b35d-6b59176746dc tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:34.403728 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a2448610-6a5a-4d8a-b35d-6b59176746dc tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1282 microversion: 2.1 time: 0.339749 Sep 27 10:02:34.427345 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 690/1518] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:34 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1282 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:34.721824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-618153b1-d857-4633-b4c7-294b44821031 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:34.722581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-618153b1-d857-4633-b4c7-294b44821031 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:34.743329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9904e09b-3bdc-42b7-afdc-c3fab4b35406 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:34.749354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618153b1-d857-4633-b4c7-294b44821031 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:34.751907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618153b1-d857-4633-b4c7-294b44821031 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:34.753191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618153b1-d857-4633-b4c7-294b44821031 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:34.756460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618153b1-d857-4633-b4c7-294b44821031 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:34.758318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618153b1-d857-4633-b4c7-294b44821031 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:34.760495 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9904e09b-3bdc-42b7-afdc-c3fab4b35406 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.660551 Sep 27 10:02:34.760819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618153b1-d857-4633-b4c7-294b44821031 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:34.764027 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 829/1519] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:34 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 674 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:35.113590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-618153b1-d857-4633-b4c7-294b44821031 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:35.130816 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-618153b1-d857-4633-b4c7-294b44821031 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.416403 Sep 27 10:02:35.132391 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 691/1520] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:34 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:35.441946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-df66729b-e8e4-4fab-954c-f0190c13585b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:35.442718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-df66729b-e8e4-4fab-954c-f0190c13585b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:35.459381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:35.470156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df66729b-e8e4-4fab-954c-f0190c13585b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:35.473400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df66729b-e8e4-4fab-954c-f0190c13585b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:35.474533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df66729b-e8e4-4fab-954c-f0190c13585b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:35.477714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df66729b-e8e4-4fab-954c-f0190c13585b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:35.478341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df66729b-e8e4-4fab-954c-f0190c13585b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:35.480633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df66729b-e8e4-4fab-954c-f0190c13585b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:35.790001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8aa357be-2568-49e2-8974-c8ec252424b4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:35.791173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8aa357be-2568-49e2-8974-c8ec252424b4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:35.813159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8aa357be-2568-49e2-8974-c8ec252424b4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:35.813668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8aa357be-2568-49e2-8974-c8ec252424b4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:35.814116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8aa357be-2568-49e2-8974-c8ec252424b4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:35.825539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8aa357be-2568-49e2-8974-c8ec252424b4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:35.828993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8aa357be-2568-49e2-8974-c8ec252424b4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:35.829877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8aa357be-2568-49e2-8974-c8ec252424b4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:35.997881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-df66729b-e8e4-4fab-954c-f0190c13585b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:36.006686 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-df66729b-e8e4-4fab-954c-f0190c13585b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1324 microversion: 2.1 time: 0.569016 Sep 27 10:02:36.011029 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 830/1521] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:35 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1324 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:36.155360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-663f6c8b-0eec-441d-b718-5b8ac8b10657 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:36.156065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-663f6c8b-0eec-441d-b718-5b8ac8b10657 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:36.165970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-663f6c8b-0eec-441d-b718-5b8ac8b10657 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:36.166482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-663f6c8b-0eec-441d-b718-5b8ac8b10657 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:36.170752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-663f6c8b-0eec-441d-b718-5b8ac8b10657 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:36.171370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-663f6c8b-0eec-441d-b718-5b8ac8b10657 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:36.172046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-663f6c8b-0eec-441d-b718-5b8ac8b10657 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:36.181425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-663f6c8b-0eec-441d-b718-5b8ac8b10657 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:36.267695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8aa357be-2568-49e2-8974-c8ec252424b4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:36.278355 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8aa357be-2568-49e2-8974-c8ec252424b4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.494281 Sep 27 10:02:36.297610 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 692/1522] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:35 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:36.525815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-663f6c8b-0eec-441d-b718-5b8ac8b10657 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:36.536631 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-663f6c8b-0eec-441d-b718-5b8ac8b10657 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.386291 Sep 27 10:02:36.538075 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 831/1523] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:36 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:36.628265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-9e0fd0e3-188d-409d-938d-576fcdf8fdf3 req-b03b15f9-afe0-4c4a-94f7-0cbb2805cd5b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3a955d66-3892-4838-ad7d-cc9e431a03e1", "tag": "b4f78072-6743-4c1d-bb89-c5bb42c81dcf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:36.647247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9e0fd0e3-188d-409d-938d-576fcdf8fdf3 req-b03b15f9-afe0-4c4a-94f7-0cbb2805cd5b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:36.648341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9e0fd0e3-188d-409d-938d-576fcdf8fdf3 req-b03b15f9-afe0-4c4a-94f7-0cbb2805cd5b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:36.649219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9e0fd0e3-188d-409d-938d-576fcdf8fdf3 req-b03b15f9-afe0-4c4a-94f7-0cbb2805cd5b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:36.670891 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-9e0fd0e3-188d-409d-938d-576fcdf8fdf3 req-b03b15f9-afe0-4c4a-94f7-0cbb2805cd5b service nova] Creating event network-changed:b4f78072-6743-4c1d-bb89-c5bb42c81dcf for instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 on nch1-master-1 Sep 27 10:02:36.681806 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-9e0fd0e3-188d-409d-938d-576fcdf8fdf3 req-b03b15f9-afe0-4c4a-94f7-0cbb2805cd5b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057939 Sep 27 10:02:36.683154 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 693/1524] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:37.040983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-985d3f2a-fe21-4c27-af2f-32e07c041d28 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:37.042299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-985d3f2a-fe21-4c27-af2f-32e07c041d28 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:37.055232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d3f2a-fe21-4c27-af2f-32e07c041d28 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:37.056631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d3f2a-fe21-4c27-af2f-32e07c041d28 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:37.057098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d3f2a-fe21-4c27-af2f-32e07c041d28 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:37.057846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d3f2a-fe21-4c27-af2f-32e07c041d28 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:37.058303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d3f2a-fe21-4c27-af2f-32e07c041d28 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:37.059313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d3f2a-fe21-4c27-af2f-32e07c041d28 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:37.311755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5b36ceb7-af74-43d2-80ff-1e16e79dd946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:37.314009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5b36ceb7-af74-43d2-80ff-1e16e79dd946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:37.328349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b36ceb7-af74-43d2-80ff-1e16e79dd946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:37.329353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b36ceb7-af74-43d2-80ff-1e16e79dd946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:37.330394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b36ceb7-af74-43d2-80ff-1e16e79dd946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:37.331873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b36ceb7-af74-43d2-80ff-1e16e79dd946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:37.333187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b36ceb7-af74-43d2-80ff-1e16e79dd946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:37.334199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b36ceb7-af74-43d2-80ff-1e16e79dd946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:37.443892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-985d3f2a-fe21-4c27-af2f-32e07c041d28 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:37.455953 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-985d3f2a-fe21-4c27-af2f-32e07c041d28 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1324 microversion: 2.1 time: 0.422626 Sep 27 10:02:37.468682 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 832/1525] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:37 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1324 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:37.533533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b36ceb7-af74-43d2-80ff-1e16e79dd946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:37.544192 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5b36ceb7-af74-43d2-80ff-1e16e79dd946 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.241460 Sep 27 10:02:37.545489 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 694/1526] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:37 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:37.567384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1e4e6a62-7de7-4bca-828c-7868e3f5b0be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:37.568275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1e4e6a62-7de7-4bca-828c-7868e3f5b0be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:37.576317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e4e6a62-7de7-4bca-828c-7868e3f5b0be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:37.576845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e4e6a62-7de7-4bca-828c-7868e3f5b0be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:37.577283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e4e6a62-7de7-4bca-828c-7868e3f5b0be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:37.578009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e4e6a62-7de7-4bca-828c-7868e3f5b0be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:37.578464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e4e6a62-7de7-4bca-828c-7868e3f5b0be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:37.578885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e4e6a62-7de7-4bca-828c-7868e3f5b0be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:37.779574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1e4e6a62-7de7-4bca-828c-7868e3f5b0be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:37.797102 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1e4e6a62-7de7-4bca-828c-7868e3f5b0be tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.235373 Sep 27 10:02:37.812730 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 833/1527] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:37 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:38.479934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8e78b638-9c12-40a2-85e2-fc60fa83a93b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:38.480536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8e78b638-9c12-40a2-85e2-fc60fa83a93b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:38.498747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e78b638-9c12-40a2-85e2-fc60fa83a93b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:38.499940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e78b638-9c12-40a2-85e2-fc60fa83a93b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:38.500625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e78b638-9c12-40a2-85e2-fc60fa83a93b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:38.501616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e78b638-9c12-40a2-85e2-fc60fa83a93b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:38.502659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e78b638-9c12-40a2-85e2-fc60fa83a93b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:38.503749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e78b638-9c12-40a2-85e2-fc60fa83a93b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:38.576432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2591700d-2e4d-48c6-b524-0e952c580eac tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:38.578158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2591700d-2e4d-48c6-b524-0e952c580eac tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:38.587472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2591700d-2e4d-48c6-b524-0e952c580eac tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:38.588480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2591700d-2e4d-48c6-b524-0e952c580eac tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:38.589175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2591700d-2e4d-48c6-b524-0e952c580eac tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:38.590200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2591700d-2e4d-48c6-b524-0e952c580eac tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:38.590949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2591700d-2e4d-48c6-b524-0e952c580eac tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:38.591625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2591700d-2e4d-48c6-b524-0e952c580eac tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:38.704098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8e78b638-9c12-40a2-85e2-fc60fa83a93b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:38.713246 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8e78b638-9c12-40a2-85e2-fc60fa83a93b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1324 microversion: 2.1 time: 0.237452 Sep 27 10:02:38.714568 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 695/1528] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:38 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1324 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:38.823709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3630a8a6-bed0-4e20-9d33-774e7b8504a0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:38.825827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3630a8a6-bed0-4e20-9d33-774e7b8504a0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:38.832985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3630a8a6-bed0-4e20-9d33-774e7b8504a0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:38.833625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3630a8a6-bed0-4e20-9d33-774e7b8504a0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:38.834387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3630a8a6-bed0-4e20-9d33-774e7b8504a0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:38.850556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3630a8a6-bed0-4e20-9d33-774e7b8504a0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:38.851792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3630a8a6-bed0-4e20-9d33-774e7b8504a0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:38.852457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3630a8a6-bed0-4e20-9d33-774e7b8504a0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:38.928276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2591700d-2e4d-48c6-b524-0e952c580eac tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:38.937654 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2591700d-2e4d-48c6-b524-0e952c580eac tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.370448 Sep 27 10:02:38.948553 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 834/1529] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:38 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:39.184410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3630a8a6-bed0-4e20-9d33-774e7b8504a0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:39.198488 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3630a8a6-bed0-4e20-9d33-774e7b8504a0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1475 microversion: 2.1 time: 0.380780 Sep 27 10:02:39.199331 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 696/1530] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:38 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1475 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:39.735627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-28a12e54-1146-45df-83dd-b62b32e27785 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:39.737124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-28a12e54-1146-45df-83dd-b62b32e27785 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:39.745685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28a12e54-1146-45df-83dd-b62b32e27785 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:39.746399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28a12e54-1146-45df-83dd-b62b32e27785 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:39.747036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28a12e54-1146-45df-83dd-b62b32e27785 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:39.748176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28a12e54-1146-45df-83dd-b62b32e27785 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:39.750773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28a12e54-1146-45df-83dd-b62b32e27785 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:39.751400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28a12e54-1146-45df-83dd-b62b32e27785 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:40.017072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-28a12e54-1146-45df-83dd-b62b32e27785 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:40.029006 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-28a12e54-1146-45df-83dd-b62b32e27785 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1324 microversion: 2.1 time: 0.297088 Sep 27 10:02:40.030172 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 835/1531] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:39 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1324 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:40.034911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8cec14c3-6a7b-4058-b88b-e1cee7152d37 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:40.035682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8cec14c3-6a7b-4058-b88b-e1cee7152d37 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:40.057686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cec14c3-6a7b-4058-b88b-e1cee7152d37 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:40.058193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cec14c3-6a7b-4058-b88b-e1cee7152d37 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:40.058980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cec14c3-6a7b-4058-b88b-e1cee7152d37 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:40.060139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cec14c3-6a7b-4058-b88b-e1cee7152d37 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:40.060602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cec14c3-6a7b-4058-b88b-e1cee7152d37 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:40.062564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cec14c3-6a7b-4058-b88b-e1cee7152d37 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:40.073156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServerFiltersTestJSON-instance-1561922851", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "41cca2ff-7533-4e7c-bba1-999622fc1a68"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:40.147069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] validate_networks() for [('41cca2ff-7533-4e7c-bba1-999622fc1a68', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:02:40.151748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:40.409857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8cec14c3-6a7b-4058-b88b-e1cee7152d37 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:40.419041 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8cec14c3-6a7b-4058-b88b-e1cee7152d37 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.387740 Sep 27 10:02:40.419641 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 836/1532] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:40 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:40.435899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a7391b93-6079-4e92-9998-dad541137383 req-9f5b3fcb-1e19-4ea3-ac4b-bd0434d53481 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a2aa469-5c5e-4a03-b09e-8e6b2252d615", "name": "network-vif-unplugged", "status": "completed", "tag": "4c0e9dc5-844c-484c-99a2-79cf10f253fe"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:40.455843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a7391b93-6079-4e92-9998-dad541137383 req-9f5b3fcb-1e19-4ea3-ac4b-bd0434d53481 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:40.456340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a7391b93-6079-4e92-9998-dad541137383 req-9f5b3fcb-1e19-4ea3-ac4b-bd0434d53481 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:40.460795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a7391b93-6079-4e92-9998-dad541137383 req-9f5b3fcb-1e19-4ea3-ac4b-bd0434d53481 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:40.489520 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a7391b93-6079-4e92-9998-dad541137383 req-9f5b3fcb-1e19-4ea3-ac4b-bd0434d53481 service nova] Creating event network-vif-unplugged:4c0e9dc5-844c-484c-99a2-79cf10f253fe for instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 on nch1-master-1 Sep 27 10:02:40.500192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-a7391b93-6079-4e92-9998-dad541137383 req-9f5b3fcb-1e19-4ea3-ac4b-bd0434d53481 service nova] Instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:02:40.523070 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a7391b93-6079-4e92-9998-dad541137383 req-9f5b3fcb-1e19-4ea3-ac4b-bd0434d53481 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.094937 Sep 27 10:02:40.523811 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 837/1533] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:40 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:40.529340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a68623f-d459-4579-88ca-b22cd926ec90 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:40.530046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3a68623f-d459-4579-88ca-b22cd926ec90 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:40.536717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a68623f-d459-4579-88ca-b22cd926ec90 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:40.536938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a68623f-d459-4579-88ca-b22cd926ec90 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:40.537346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a68623f-d459-4579-88ca-b22cd926ec90 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:40.538065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a68623f-d459-4579-88ca-b22cd926ec90 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:40.538588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a68623f-d459-4579-88ca-b22cd926ec90 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:40.539015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a68623f-d459-4579-88ca-b22cd926ec90 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:40.665522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:02:40.666177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:02:40.666858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:02:40.667276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:02:40.667682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:02:40.678135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:40.678925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:40.679575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:40.680501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:40.681139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:40.681734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:40.719869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Getting quotas for project f48a592eb8fa4221a56eac1a50152c91. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:02:40.725376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Getting quotas for user ac98fc0d0296493999906e8e3607c7f3 and project f48a592eb8fa4221a56eac1a50152c91. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:02:40.746924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:02:40.749477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:02:40.819000 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-697ab865-9c58-4fc3-a95d-44b516462efa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.974364 Sep 27 10:02:40.820227 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 697/1534] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:02:39 2022] POST /compute/v2.1/servers => generated 380 bytes in 976 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:02:40.884423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServerFiltersTestJSON-instance-1198818639", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "41cca2ff-7533-4e7c-bba1-999622fc1a68"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:40.901972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3a68623f-d459-4579-88ca-b22cd926ec90 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:40.910504 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3a68623f-d459-4579-88ca-b22cd926ec90 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1474 microversion: 2.1 time: 0.385437 Sep 27 10:02:40.917261 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 838/1535] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:40 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1474 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:40.981816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] validate_networks() for [('41cca2ff-7533-4e7c-bba1-999622fc1a68', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:02:41.055468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-97997ee8-8aed-4eaf-9bd3-e8e4d5cefee4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:41.056361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-97997ee8-8aed-4eaf-9bd3-e8e4d5cefee4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:41.067655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97997ee8-8aed-4eaf-9bd3-e8e4d5cefee4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:41.074098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97997ee8-8aed-4eaf-9bd3-e8e4d5cefee4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:41.077192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97997ee8-8aed-4eaf-9bd3-e8e4d5cefee4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:41.079892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97997ee8-8aed-4eaf-9bd3-e8e4d5cefee4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:41.083771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97997ee8-8aed-4eaf-9bd3-e8e4d5cefee4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:41.084339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97997ee8-8aed-4eaf-9bd3-e8e4d5cefee4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:41.545953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-97997ee8-8aed-4eaf-9bd3-e8e4d5cefee4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:41.556903 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-97997ee8-8aed-4eaf-9bd3-e8e4d5cefee4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1324 microversion: 2.1 time: 0.505475 Sep 27 10:02:41.557620 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 839/1536] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:41 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1324 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:41.565846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-28d41172-ad74-4fd1-8e61-69486c576091 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:41.566774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-28d41172-ad74-4fd1-8e61-69486c576091 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:41.583226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28d41172-ad74-4fd1-8e61-69486c576091 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:41.583730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28d41172-ad74-4fd1-8e61-69486c576091 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:41.584179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28d41172-ad74-4fd1-8e61-69486c576091 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:41.584960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28d41172-ad74-4fd1-8e61-69486c576091 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:41.585463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28d41172-ad74-4fd1-8e61-69486c576091 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:41.585901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28d41172-ad74-4fd1-8e61-69486c576091 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:41.738054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:02:41.739232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:02:41.740988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:02:41.742032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:02:41.742725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:02:41.758479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:41.759389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:41.760554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:41.761832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:41.762546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:41.763603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:41.791258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:41.815236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Getting quotas for project f48a592eb8fa4221a56eac1a50152c91. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:02:41.822674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Getting quotas for user ac98fc0d0296493999906e8e3607c7f3 and project f48a592eb8fa4221a56eac1a50152c91. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:02:41.837401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:02:41.840327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:02:41.962003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-28d41172-ad74-4fd1-8e61-69486c576091 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:41.973524 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4541f48f-3703-42c4-a7d9-8a7e7b54291a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.094411 Sep 27 10:02:41.985369 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 698/1537] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:02:40 2022] POST /compute/v2.1/servers => generated 380 bytes in 1099 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:02:41.985369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f392e683-1521-4343-937e-02d554f93dcd req-035d2a6c-b2c1-4053-90f1-400fca4c4e16 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3a955d66-3892-4838-ad7d-cc9e431a03e1", "name": "network-vif-plugged", "status": "completed", "tag": "b4f78072-6743-4c1d-bb89-c5bb42c81dcf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:42.008567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f392e683-1521-4343-937e-02d554f93dcd req-035d2a6c-b2c1-4053-90f1-400fca4c4e16 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:42.009665 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-28d41172-ad74-4fd1-8e61-69486c576091 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.447814 Sep 27 10:02:42.012582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f392e683-1521-4343-937e-02d554f93dcd req-035d2a6c-b2c1-4053-90f1-400fca4c4e16 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:42.013090 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 840/1538] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:41 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:42.013564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f392e683-1521-4343-937e-02d554f93dcd req-035d2a6c-b2c1-4053-90f1-400fca4c4e16 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:42.025562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67c22645-5aab-4ebb-82f0-f1296ba31cc9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:42.029007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67c22645-5aab-4ebb-82f0-f1296ba31cc9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:42.047244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c22645-5aab-4ebb-82f0-f1296ba31cc9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:42.050700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c22645-5aab-4ebb-82f0-f1296ba31cc9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:42.051524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c22645-5aab-4ebb-82f0-f1296ba31cc9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:42.056130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c22645-5aab-4ebb-82f0-f1296ba31cc9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:42.060226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c22645-5aab-4ebb-82f0-f1296ba31cc9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:42.060896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c22645-5aab-4ebb-82f0-f1296ba31cc9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:42.081098 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f392e683-1521-4343-937e-02d554f93dcd req-035d2a6c-b2c1-4053-90f1-400fca4c4e16 service nova] Creating event network-vif-plugged:b4f78072-6743-4c1d-bb89-c5bb42c81dcf for instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 on nch1-master-1 Sep 27 10:02:42.094703 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f392e683-1521-4343-937e-02d554f93dcd req-035d2a6c-b2c1-4053-90f1-400fca4c4e16 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.116739 Sep 27 10:02:42.095658 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 699/1539] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:42.112474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServerFiltersTestJSON-instance-1001945891", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "451", "networks": [{"uuid": "41cca2ff-7533-4e7c-bba1-999622fc1a68"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:42.264231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] validate_networks() for [('41cca2ff-7533-4e7c-bba1-999622fc1a68', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:02:42.383994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-67c22645-5aab-4ebb-82f0-f1296ba31cc9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:42.394518 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67c22645-5aab-4ebb-82f0-f1296ba31cc9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1474 microversion: 2.1 time: 0.378868 Sep 27 10:02:42.395278 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 841/1540] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:42 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1474 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:42.547228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-62653dec-ed23-4ae4-9ffd-d2bd2b4918e3 req-48cbe886-8884-4156-9d7e-119ee3fc83d5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9a2aa469-5c5e-4a03-b09e-8e6b2252d615", "tag": "4c0e9dc5-844c-484c-99a2-79cf10f253fe"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:42.577280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-62653dec-ed23-4ae4-9ffd-d2bd2b4918e3 req-48cbe886-8884-4156-9d7e-119ee3fc83d5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:42.578737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-62653dec-ed23-4ae4-9ffd-d2bd2b4918e3 req-48cbe886-8884-4156-9d7e-119ee3fc83d5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:42.579395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-62653dec-ed23-4ae4-9ffd-d2bd2b4918e3 req-48cbe886-8884-4156-9d7e-119ee3fc83d5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:42.597305 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-62653dec-ed23-4ae4-9ffd-d2bd2b4918e3 req-48cbe886-8884-4156-9d7e-119ee3fc83d5 service nova] Creating event network-changed:4c0e9dc5-844c-484c-99a2-79cf10f253fe for instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 on nch1-master-1 Sep 27 10:02:42.616559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-62653dec-ed23-4ae4-9ffd-d2bd2b4918e3 req-48cbe886-8884-4156-9d7e-119ee3fc83d5 service nova] Instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:02:42.630463 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-62653dec-ed23-4ae4-9ffd-d2bd2b4918e3 req-48cbe886-8884-4156-9d7e-119ee3fc83d5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.091937 Sep 27 10:02:42.631156 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 842/1541] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:02:42.639862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f7cbc0f5-19c2-4861-b24f-a1c593769207 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:42.640907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f7cbc0f5-19c2-4861-b24f-a1c593769207 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:42.652234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7cbc0f5-19c2-4861-b24f-a1c593769207 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:42.653216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7cbc0f5-19c2-4861-b24f-a1c593769207 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:42.654023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7cbc0f5-19c2-4861-b24f-a1c593769207 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:42.655737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7cbc0f5-19c2-4861-b24f-a1c593769207 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:42.656736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7cbc0f5-19c2-4861-b24f-a1c593769207 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:42.657437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7cbc0f5-19c2-4861-b24f-a1c593769207 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:42.693446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:42.976805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f7cbc0f5-19c2-4861-b24f-a1c593769207 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:42.998347 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f7cbc0f5-19c2-4861-b24f-a1c593769207 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1324 microversion: 2.1 time: 0.364387 Sep 27 10:02:43.006895 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 843/1542] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:42 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1324 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:43.042502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-223588b2-12e0-4dcd-8e59-d02feea23ba5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:43.044426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-223588b2-12e0-4dcd-8e59-d02feea23ba5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:43.050972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223588b2-12e0-4dcd-8e59-d02feea23ba5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:43.051626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223588b2-12e0-4dcd-8e59-d02feea23ba5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:43.052057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223588b2-12e0-4dcd-8e59-d02feea23ba5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:43.053268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223588b2-12e0-4dcd-8e59-d02feea23ba5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:43.054218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223588b2-12e0-4dcd-8e59-d02feea23ba5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:43.055378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223588b2-12e0-4dcd-8e59-d02feea23ba5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:43.267858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:02:43.268429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-223588b2-12e0-4dcd-8e59-d02feea23ba5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:43.268551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:02:43.269239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:02:43.269668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:02:43.270078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:02:43.278636 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-223588b2-12e0-4dcd-8e59-d02feea23ba5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.246843 Sep 27 10:02:43.287670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:43.288619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:43.289238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:43.290133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:43.290831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:43.291423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:43.296580 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 844/1543] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:43 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:43.333539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Getting quotas for project f48a592eb8fa4221a56eac1a50152c91. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:02:43.340591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Getting quotas for user ac98fc0d0296493999906e8e3607c7f3 and project f48a592eb8fa4221a56eac1a50152c91. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:02:43.364082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:02:43.368783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:02:43.387817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=10 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:43.420741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c195bc2a-ef2d-4109-8d84-fa20617d59a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:43.421589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c195bc2a-ef2d-4109-8d84-fa20617d59a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:43.430703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195bc2a-ef2d-4109-8d84-fa20617d59a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:43.431182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195bc2a-ef2d-4109-8d84-fa20617d59a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:43.431620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195bc2a-ef2d-4109-8d84-fa20617d59a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:43.432329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195bc2a-ef2d-4109-8d84-fa20617d59a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:43.432807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195bc2a-ef2d-4109-8d84-fa20617d59a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:43.433421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c195bc2a-ef2d-4109-8d84-fa20617d59a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:43.441640 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a4205d77-77a0-477b-b1ac-38830e243433 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.341307 Sep 27 10:02:43.444570 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 700/1544] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:02:42 2022] POST /compute/v2.1/servers => generated 380 bytes in 1347 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:02:43.463258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0410c2fd-2e0f-4209-b4d8-7cc87e1ccbe5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:43.467850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0410c2fd-2e0f-4209-b4d8-7cc87e1ccbe5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:43.627661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0410c2fd-2e0f-4209-b4d8-7cc87e1ccbe5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:43.636319 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0410c2fd-2e0f-4209-b4d8-7cc87e1ccbe5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1221 microversion: 2.1 time: 0.185125 Sep 27 10:02:43.652758 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 701/1545] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:43 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1221 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:43.653296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c195bc2a-ef2d-4109-8d84-fa20617d59a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:43.661317 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c195bc2a-ef2d-4109-8d84-fa20617d59a9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1472 microversion: 2.1 time: 0.244887 Sep 27 10:02:43.662085 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 845/1546] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:43 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1472 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:44.023440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6d567c4b-808a-49aa-9920-631662881a69 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:44.024076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6d567c4b-808a-49aa-9920-631662881a69 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:44.045045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d567c4b-808a-49aa-9920-631662881a69 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:44.045554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d567c4b-808a-49aa-9920-631662881a69 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:44.058212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d567c4b-808a-49aa-9920-631662881a69 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:44.062631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d567c4b-808a-49aa-9920-631662881a69 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:44.065186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d567c4b-808a-49aa-9920-631662881a69 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:44.065616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d567c4b-808a-49aa-9920-631662881a69 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:44.281539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6d567c4b-808a-49aa-9920-631662881a69 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:44.291612 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6d567c4b-808a-49aa-9920-631662881a69 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1324 microversion: 2.1 time: 0.272321 Sep 27 10:02:44.292799 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 702/1547] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:44 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1324 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:44.308470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e93c749-b633-43ae-887e-26e35d1a73e9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:44.309530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e93c749-b633-43ae-887e-26e35d1a73e9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:44.319301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e93c749-b633-43ae-887e-26e35d1a73e9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:44.319696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e93c749-b633-43ae-887e-26e35d1a73e9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:44.320118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e93c749-b633-43ae-887e-26e35d1a73e9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:44.321193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e93c749-b633-43ae-887e-26e35d1a73e9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:44.321812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e93c749-b633-43ae-887e-26e35d1a73e9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:44.322396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e93c749-b633-43ae-887e-26e35d1a73e9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:44.639557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0e93c749-b633-43ae-887e-26e35d1a73e9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:44.651058 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e93c749-b633-43ae-887e-26e35d1a73e9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.349687 Sep 27 10:02:44.655397 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 846/1548] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:44 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:44.674786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-974759a5-6433-4b88-bd40-08e140b9edb5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:44.675845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-974759a5-6433-4b88-bd40-08e140b9edb5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:44.684963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974759a5-6433-4b88-bd40-08e140b9edb5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:44.685758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974759a5-6433-4b88-bd40-08e140b9edb5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:44.686380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974759a5-6433-4b88-bd40-08e140b9edb5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:44.687269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974759a5-6433-4b88-bd40-08e140b9edb5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:44.687886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974759a5-6433-4b88-bd40-08e140b9edb5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:44.688013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-08954bad-d607-4a7c-bcab-d582ce490143 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:44.688589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974759a5-6433-4b88-bd40-08e140b9edb5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:44.688951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-08954bad-d607-4a7c-bcab-d582ce490143 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:44.705937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08954bad-d607-4a7c-bcab-d582ce490143 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:44.706516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08954bad-d607-4a7c-bcab-d582ce490143 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:44.706977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08954bad-d607-4a7c-bcab-d582ce490143 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:44.712804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08954bad-d607-4a7c-bcab-d582ce490143 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:44.713329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08954bad-d607-4a7c-bcab-d582ce490143 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:44.713822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08954bad-d607-4a7c-bcab-d582ce490143 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:45.054159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-08954bad-d607-4a7c-bcab-d582ce490143 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:45.067672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-974759a5-6433-4b88-bd40-08e140b9edb5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:45.070831 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-08954bad-d607-4a7c-bcab-d582ce490143 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1469 microversion: 2.1 time: 0.388146 Sep 27 10:02:45.079613 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 847/1549] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:44 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1469 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:45.090002 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-974759a5-6433-4b88-bd40-08e140b9edb5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1217 microversion: 2.1 time: 0.419852 Sep 27 10:02:45.098938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8d0a1ce1-53be-41ae-b9ad-9a0dcb1509f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:45.099618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8d0a1ce1-53be-41ae-b9ad-9a0dcb1509f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:45.106427 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 703/1550] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:44 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1217 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:45.108154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0a1ce1-53be-41ae-b9ad-9a0dcb1509f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:45.109319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0a1ce1-53be-41ae-b9ad-9a0dcb1509f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:45.109319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0a1ce1-53be-41ae-b9ad-9a0dcb1509f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:45.110148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0a1ce1-53be-41ae-b9ad-9a0dcb1509f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:45.110697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0a1ce1-53be-41ae-b9ad-9a0dcb1509f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:45.111355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0a1ce1-53be-41ae-b9ad-9a0dcb1509f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:45.263939 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8d0a1ce1-53be-41ae-b9ad-9a0dcb1509f7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "POST /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615/action" status: 202 len: 0 microversion: 2.1 time: 0.169226 Sep 27 10:02:45.265294 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 848/1551] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:02:45 2022] POST /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615/action => generated 0 bytes in 172 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:02:45.275065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1ab0243f-50f7-4468-9d27-6f8b07f5db34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:45.276564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1ab0243f-50f7-4468-9d27-6f8b07f5db34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:45.283511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ab0243f-50f7-4468-9d27-6f8b07f5db34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:45.285259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ab0243f-50f7-4468-9d27-6f8b07f5db34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:45.285747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ab0243f-50f7-4468-9d27-6f8b07f5db34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:45.287324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ab0243f-50f7-4468-9d27-6f8b07f5db34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:45.288034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ab0243f-50f7-4468-9d27-6f8b07f5db34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:45.288562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ab0243f-50f7-4468-9d27-6f8b07f5db34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:45.321959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a167bdde-c8a1-461d-8ba9-9413c1d77432 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:45.323358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a167bdde-c8a1-461d-8ba9-9413c1d77432 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:45.336470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a167bdde-c8a1-461d-8ba9-9413c1d77432 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:45.337017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a167bdde-c8a1-461d-8ba9-9413c1d77432 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:45.337466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a167bdde-c8a1-461d-8ba9-9413c1d77432 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:45.338205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a167bdde-c8a1-461d-8ba9-9413c1d77432 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:45.338665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a167bdde-c8a1-461d-8ba9-9413c1d77432 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:45.339091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a167bdde-c8a1-461d-8ba9-9413c1d77432 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:45.699248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a167bdde-c8a1-461d-8ba9-9413c1d77432 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:45.701027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1ab0243f-50f7-4468-9d27-6f8b07f5db34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:45.708575 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a167bdde-c8a1-461d-8ba9-9413c1d77432 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1324 microversion: 2.1 time: 0.391888 Sep 27 10:02:45.712859 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1ab0243f-50f7-4468-9d27-6f8b07f5db34 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1468 microversion: 2.1 time: 0.442810 Sep 27 10:02:45.713105 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 849/1552] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:45 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1324 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:45.714357 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 704/1553] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:45 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1468 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:45.719532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1fabe5b9-8083-4d7a-8c55-73c1a5a40149 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:45.720551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1fabe5b9-8083-4d7a-8c55-73c1a5a40149 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:45.740349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fabe5b9-8083-4d7a-8c55-73c1a5a40149 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:45.741015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fabe5b9-8083-4d7a-8c55-73c1a5a40149 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:45.741626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fabe5b9-8083-4d7a-8c55-73c1a5a40149 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:45.742839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fabe5b9-8083-4d7a-8c55-73c1a5a40149 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:45.743520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fabe5b9-8083-4d7a-8c55-73c1a5a40149 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:45.744164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fabe5b9-8083-4d7a-8c55-73c1a5a40149 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:46.115421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6b494933-eae8-4fbb-9ec1-a20b92061d3d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:46.116478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6b494933-eae8-4fbb-9ec1-a20b92061d3d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:46.124550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b494933-eae8-4fbb-9ec1-a20b92061d3d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:46.125364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b494933-eae8-4fbb-9ec1-a20b92061d3d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:46.125991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b494933-eae8-4fbb-9ec1-a20b92061d3d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:46.127477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b494933-eae8-4fbb-9ec1-a20b92061d3d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:46.127948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b494933-eae8-4fbb-9ec1-a20b92061d3d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:46.128392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b494933-eae8-4fbb-9ec1-a20b92061d3d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:46.224864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1fabe5b9-8083-4d7a-8c55-73c1a5a40149 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:46.239828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6b494933-eae8-4fbb-9ec1-a20b92061d3d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:46.245137 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1fabe5b9-8083-4d7a-8c55-73c1a5a40149 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.530882 Sep 27 10:02:46.254094 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6b494933-eae8-4fbb-9ec1-a20b92061d3d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1273 microversion: 2.1 time: 0.139756 Sep 27 10:02:46.254638 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 850/1554] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:45 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:46.259664 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 705/1555] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:46 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1273 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:46.743173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e7a97b82-bda8-4b11-b97a-93a2f05aa823 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:46.746712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-40b11f75-174e-4f06-a8b1-ea90b1f78555 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:46.747460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-40b11f75-174e-4f06-a8b1-ea90b1f78555 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:46.750253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e7a97b82-bda8-4b11-b97a-93a2f05aa823 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:46.764696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40b11f75-174e-4f06-a8b1-ea90b1f78555 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:46.767487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40b11f75-174e-4f06-a8b1-ea90b1f78555 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:46.769002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a97b82-bda8-4b11-b97a-93a2f05aa823 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:46.771146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40b11f75-174e-4f06-a8b1-ea90b1f78555 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:46.771852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a97b82-bda8-4b11-b97a-93a2f05aa823 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:46.774215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a97b82-bda8-4b11-b97a-93a2f05aa823 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:46.775091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40b11f75-174e-4f06-a8b1-ea90b1f78555 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:46.775958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40b11f75-174e-4f06-a8b1-ea90b1f78555 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:46.776624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a97b82-bda8-4b11-b97a-93a2f05aa823 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:46.777521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a97b82-bda8-4b11-b97a-93a2f05aa823 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:46.778420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40b11f75-174e-4f06-a8b1-ea90b1f78555 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:46.779538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a97b82-bda8-4b11-b97a-93a2f05aa823 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:47.092045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e7a97b82-bda8-4b11-b97a-93a2f05aa823 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:47.103055 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e7a97b82-bda8-4b11-b97a-93a2f05aa823 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1519 microversion: 2.1 time: 0.369094 Sep 27 10:02:47.108470 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 851/1556] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:46 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1519 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:47.134826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-371d3276-89d7-4f00-8ac3-6b424c16f0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Action: 'create', calling method: .version_select at 0x7f7f63ae1310>, body: {"pool": null} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:47.185240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-40b11f75-174e-4f06-a8b1-ea90b1f78555 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:47.201075 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-40b11f75-174e-4f06-a8b1-ea90b1f78555 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1468 microversion: 2.1 time: 0.461109 Sep 27 10:02:47.204951 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 706/1557] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:46 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1468 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:47.275860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-67f8bf48-969e-4104-9580-362f429e6557 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:47.276651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-67f8bf48-969e-4104-9580-362f429e6557 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:47.291618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67f8bf48-969e-4104-9580-362f429e6557 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:47.292636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67f8bf48-969e-4104-9580-362f429e6557 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:47.293236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67f8bf48-969e-4104-9580-362f429e6557 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:47.294072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67f8bf48-969e-4104-9580-362f429e6557 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:47.294651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67f8bf48-969e-4104-9580-362f429e6557 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:47.300477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67f8bf48-969e-4104-9580-362f429e6557 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:47.752492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-67f8bf48-969e-4104-9580-362f429e6557 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:47.763027 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-67f8bf48-969e-4104-9580-362f429e6557 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.489319 Sep 27 10:02:47.766682 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 707/1558] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:47 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:47.772597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2423d26f-0d92-4e4e-a1b8-2fb869448a23 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:47.773257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2423d26f-0d92-4e4e-a1b8-2fb869448a23 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:47.790436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2423d26f-0d92-4e4e-a1b8-2fb869448a23 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:47.790938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2423d26f-0d92-4e4e-a1b8-2fb869448a23 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:47.791386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2423d26f-0d92-4e4e-a1b8-2fb869448a23 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:47.792114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2423d26f-0d92-4e4e-a1b8-2fb869448a23 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:47.792578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2423d26f-0d92-4e4e-a1b8-2fb869448a23 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:47.802009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2423d26f-0d92-4e4e-a1b8-2fb869448a23 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:47.902502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2423d26f-0d92-4e4e-a1b8-2fb869448a23 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:47.917268 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2423d26f-0d92-4e4e-a1b8-2fb869448a23 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1279 microversion: 2.1 time: 0.149298 Sep 27 10:02:47.934180 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 708/1559] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:47 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1279 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:47.942386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e68c70f3-987a-44bd-80d5-ee068a14d303 req-22429a3f-89ab-4633-b089-141341aba231 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a2aa469-5c5e-4a03-b09e-8e6b2252d615", "name": "network-vif-plugged", "status": "completed", "tag": "4c0e9dc5-844c-484c-99a2-79cf10f253fe"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:47.954053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e68c70f3-987a-44bd-80d5-ee068a14d303 req-22429a3f-89ab-4633-b089-141341aba231 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:47.954679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e68c70f3-987a-44bd-80d5-ee068a14d303 req-22429a3f-89ab-4633-b089-141341aba231 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:47.955616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e68c70f3-987a-44bd-80d5-ee068a14d303 req-22429a3f-89ab-4633-b089-141341aba231 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:47.973931 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e68c70f3-987a-44bd-80d5-ee068a14d303 req-22429a3f-89ab-4633-b089-141341aba231 service nova] Creating event network-vif-plugged:4c0e9dc5-844c-484c-99a2-79cf10f253fe for instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 on nch1-master-1 Sep 27 10:02:47.981593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-e68c70f3-987a-44bd-80d5-ee068a14d303 req-22429a3f-89ab-4633-b089-141341aba231 service nova] Instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:02:47.997034 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e68c70f3-987a-44bd-80d5-ee068a14d303 req-22429a3f-89ab-4633-b089-141341aba231 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060777 Sep 27 10:02:47.998395 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 709/1560] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:48.148497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-59cf923f-37af-475d-9e2c-daf7d0cca385 req-1856b2f7-900e-4681-be95-4b5693347dc8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "96553659-f775-4568-a1a1-c3a9237df534", "tag": "2a657e89-bfee-4434-85bf-ab2d711c1e68"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:48.166273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-59cf923f-37af-475d-9e2c-daf7d0cca385 req-1856b2f7-900e-4681-be95-4b5693347dc8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:48.166831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-59cf923f-37af-475d-9e2c-daf7d0cca385 req-1856b2f7-900e-4681-be95-4b5693347dc8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:48.167302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-59cf923f-37af-475d-9e2c-daf7d0cca385 req-1856b2f7-900e-4681-be95-4b5693347dc8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:48.204895 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-59cf923f-37af-475d-9e2c-daf7d0cca385 req-1856b2f7-900e-4681-be95-4b5693347dc8 service nova] Creating event network-changed:2a657e89-bfee-4434-85bf-ab2d711c1e68 for instance 96553659-f775-4568-a1a1-c3a9237df534 on nch2-master-1 Sep 27 10:02:48.215580 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-59cf923f-37af-475d-9e2c-daf7d0cca385 req-1856b2f7-900e-4681-be95-4b5693347dc8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070770 Sep 27 10:02:48.216540 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 710/1561] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:02:48.230660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4405f19c-3606-490a-a00e-0892659c3353 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:48.231865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4405f19c-3606-490a-a00e-0892659c3353 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:48.247663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4405f19c-3606-490a-a00e-0892659c3353 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:48.248532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4405f19c-3606-490a-a00e-0892659c3353 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:48.249469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4405f19c-3606-490a-a00e-0892659c3353 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:48.250732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4405f19c-3606-490a-a00e-0892659c3353 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:48.251399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4405f19c-3606-490a-a00e-0892659c3353 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:48.252224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4405f19c-3606-490a-a00e-0892659c3353 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:48.651478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4405f19c-3606-490a-a00e-0892659c3353 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:48.661294 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4405f19c-3606-490a-a00e-0892659c3353 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1468 microversion: 2.1 time: 0.437894 Sep 27 10:02:48.673105 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 711/1562] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:48 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1468 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:48.787521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6774e6ff-2f09-4446-8043-b5a4109bac4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:48.788311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6774e6ff-2f09-4446-8043-b5a4109bac4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:48.802882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6774e6ff-2f09-4446-8043-b5a4109bac4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:48.806529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6774e6ff-2f09-4446-8043-b5a4109bac4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:48.807847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6774e6ff-2f09-4446-8043-b5a4109bac4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:48.810151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6774e6ff-2f09-4446-8043-b5a4109bac4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:48.826740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6774e6ff-2f09-4446-8043-b5a4109bac4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:48.827209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6774e6ff-2f09-4446-8043-b5a4109bac4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:49.318231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6774e6ff-2f09-4446-8043-b5a4109bac4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:49.337621 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6774e6ff-2f09-4446-8043-b5a4109bac4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.551496 Sep 27 10:02:49.339897 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 712/1563] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:48 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:49.373951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-768ee37e-c651-4fb0-80c4-e106dfdfa013 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:49.381771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-768ee37e-c651-4fb0-80c4-e106dfdfa013 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:49.413928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-768ee37e-c651-4fb0-80c4-e106dfdfa013 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:49.414794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-768ee37e-c651-4fb0-80c4-e106dfdfa013 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:49.415460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-768ee37e-c651-4fb0-80c4-e106dfdfa013 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:49.417928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-768ee37e-c651-4fb0-80c4-e106dfdfa013 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:49.418792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-768ee37e-c651-4fb0-80c4-e106dfdfa013 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:49.421210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-768ee37e-c651-4fb0-80c4-e106dfdfa013 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:49.697657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-768ee37e-c651-4fb0-80c4-e106dfdfa013 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:49.710642 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-768ee37e-c651-4fb0-80c4-e106dfdfa013 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1321 microversion: 2.1 time: 0.368039 Sep 27 10:02:49.712698 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 713/1564] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:49 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1321 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:49.718220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b67e5f68-31b8-4af0-895e-a8f92b43981d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:49.719143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b67e5f68-31b8-4af0-895e-a8f92b43981d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:49.726709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b67e5f68-31b8-4af0-895e-a8f92b43981d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:49.728034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b67e5f68-31b8-4af0-895e-a8f92b43981d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:49.728765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b67e5f68-31b8-4af0-895e-a8f92b43981d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:49.729913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b67e5f68-31b8-4af0-895e-a8f92b43981d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:49.730743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b67e5f68-31b8-4af0-895e-a8f92b43981d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:49.731442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b67e5f68-31b8-4af0-895e-a8f92b43981d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:49.894329 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-371d3276-89d7-4f00-8ac3-6b424c16f0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 2.765678 Sep 27 10:02:49.895345 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 852/1565] 10.222.0.26 () {60 vars in 1170 bytes} [Tue Sep 27 10:02:47 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 2768 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:49.902226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-40e101ac-ef60-4a00-b654-874e72929ff2 req-e4ec9b79-5dad-4270-b2ee-546b470f6eb3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6d511fc6-80ab-458e-88a6-593374a3e01c", "tag": "3f178b4c-caa9-4399-84bb-8322b9ac1a75"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:49.928458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-40e101ac-ef60-4a00-b654-874e72929ff2 req-e4ec9b79-5dad-4270-b2ee-546b470f6eb3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:49.929156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-40e101ac-ef60-4a00-b654-874e72929ff2 req-e4ec9b79-5dad-4270-b2ee-546b470f6eb3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:49.929599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-40e101ac-ef60-4a00-b654-874e72929ff2 req-e4ec9b79-5dad-4270-b2ee-546b470f6eb3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:49.966389 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-40e101ac-ef60-4a00-b654-874e72929ff2 req-e4ec9b79-5dad-4270-b2ee-546b470f6eb3 service nova] Creating event network-changed:3f178b4c-caa9-4399-84bb-8322b9ac1a75 for instance 6d511fc6-80ab-458e-88a6-593374a3e01c on nch1-master-1 Sep 27 10:02:49.980641 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-40e101ac-ef60-4a00-b654-874e72929ff2 req-e4ec9b79-5dad-4270-b2ee-546b470f6eb3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083557 Sep 27 10:02:49.986352 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 853/1566] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:50.075862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-FloatingIPsAssociationTestJSON-server-1374002422", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "79454fbf-4335-40b3-b1c5-c3f69544e188"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:50.104484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b67e5f68-31b8-4af0-895e-a8f92b43981d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:50.115443 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b67e5f68-31b8-4af0-895e-a8f92b43981d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1468 microversion: 2.1 time: 0.401123 Sep 27 10:02:50.133359 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 714/1567] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:49 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1468 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:50.142735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] validate_networks() for [('79454fbf-4335-40b3-b1c5-c3f69544e188', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:02:50.300712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-41194959-bbbf-45dc-9d21-5352195cdf42 req-c4c6b68e-a394-495b-b012-4723bf4836a4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ba4b5b70-d9bf-43b6-a52e-73be94b31bb1", "tag": "14ebcc5e-9aae-4755-a891-78f55436d129"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:50.322982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-41194959-bbbf-45dc-9d21-5352195cdf42 req-c4c6b68e-a394-495b-b012-4723bf4836a4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:50.327941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-41194959-bbbf-45dc-9d21-5352195cdf42 req-c4c6b68e-a394-495b-b012-4723bf4836a4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:50.328713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-41194959-bbbf-45dc-9d21-5352195cdf42 req-c4c6b68e-a394-495b-b012-4723bf4836a4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:50.356069 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-41194959-bbbf-45dc-9d21-5352195cdf42 req-c4c6b68e-a394-495b-b012-4723bf4836a4 service nova] Creating event network-changed:14ebcc5e-9aae-4755-a891-78f55436d129 for instance ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 on nch1-master-1 Sep 27 10:02:50.377133 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-41194959-bbbf-45dc-9d21-5352195cdf42 req-c4c6b68e-a394-495b-b012-4723bf4836a4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085590 Sep 27 10:02:50.379734 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 715/1568] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:50.392083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6b018124-29c7-4140-a267-7879b46f39fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:50.403273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6b018124-29c7-4140-a267-7879b46f39fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:50.412437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b018124-29c7-4140-a267-7879b46f39fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:50.412956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b018124-29c7-4140-a267-7879b46f39fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:50.413530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b018124-29c7-4140-a267-7879b46f39fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:50.415063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b018124-29c7-4140-a267-7879b46f39fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:50.415535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b018124-29c7-4140-a267-7879b46f39fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:50.416414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b018124-29c7-4140-a267-7879b46f39fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:50.671248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6b018124-29c7-4140-a267-7879b46f39fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:50.681373 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6b018124-29c7-4140-a267-7879b46f39fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.299994 Sep 27 10:02:50.689431 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 716/1569] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:50 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:50.696545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c820eb30-9f33-435d-80f6-ef5202282a0d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "814c3373-89e6-4c1a-b1e0-8ddb0523d638"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:50.699633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c820eb30-9f33-435d-80f6-ef5202282a0d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:50.706949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c820eb30-9f33-435d-80f6-ef5202282a0d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:50.707441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c820eb30-9f33-435d-80f6-ef5202282a0d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:50.707876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c820eb30-9f33-435d-80f6-ef5202282a0d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:50.708591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c820eb30-9f33-435d-80f6-ef5202282a0d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:50.709089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c820eb30-9f33-435d-80f6-ef5202282a0d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:50.709684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c820eb30-9f33-435d-80f6-ef5202282a0d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:50.838450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:02:50.839072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:02:50.839620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:02:50.840229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:02:50.840752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:02:50.850298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:50.851289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:50.851799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:50.852789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:50.853245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:50.854151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:50.872647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Getting quotas for project de87d5984a9a4d4481ab4921133841d6. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:02:50.877809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Getting quotas for user c7468cc8bc7d4dde8e2a5eba1fa42fd8 and project de87d5984a9a4d4481ab4921133841d6. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:02:50.892389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:02:50.894897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:02:50.948381 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4c536a29-cd49-4f1e-984b-3a6ee1a13be7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.878017 Sep 27 10:02:50.949652 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 854/1570] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:02:50 2022] POST /compute/v2.1/servers => generated 380 bytes in 880 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:02:50.954550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c3e5285e-468d-48fc-8241-f462ce207c4b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:50.955158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c3e5285e-468d-48fc-8241-f462ce207c4b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:50.962832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3e5285e-468d-48fc-8241-f462ce207c4b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:50.964231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3e5285e-468d-48fc-8241-f462ce207c4b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:50.964692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3e5285e-468d-48fc-8241-f462ce207c4b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:50.965425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3e5285e-468d-48fc-8241-f462ce207c4b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:50.965889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3e5285e-468d-48fc-8241-f462ce207c4b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:50.966315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3e5285e-468d-48fc-8241-f462ce207c4b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:51.146089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c3e5285e-468d-48fc-8241-f462ce207c4b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:51.158195 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c3e5285e-468d-48fc-8241-f462ce207c4b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1321 microversion: 2.1 time: 0.207355 Sep 27 10:02:51.158781 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 855/1571] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:50 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1321 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:51.164166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5df6df46-276b-4ac5-9586-3977ef18f618 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:51.164938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5df6df46-276b-4ac5-9586-3977ef18f618 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:51.245549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5df6df46-276b-4ac5-9586-3977ef18f618 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:51.253323 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5df6df46-276b-4ac5-9586-3977ef18f618 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1224 microversion: 2.1 time: 0.093490 Sep 27 10:02:51.253984 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 856/1572] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:51 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1224 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:51.260050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-40c62797-e7ad-432d-9622-3a0718fa337c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:51.261005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-40c62797-e7ad-432d-9622-3a0718fa337c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:51.267481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40c62797-e7ad-432d-9622-3a0718fa337c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:51.268292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40c62797-e7ad-432d-9622-3a0718fa337c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:51.268932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40c62797-e7ad-432d-9622-3a0718fa337c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:51.269682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40c62797-e7ad-432d-9622-3a0718fa337c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:51.270132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40c62797-e7ad-432d-9622-3a0718fa337c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:51.270554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40c62797-e7ad-432d-9622-3a0718fa337c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:51.419622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-40c62797-e7ad-432d-9622-3a0718fa337c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:51.428115 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-40c62797-e7ad-432d-9622-3a0718fa337c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1468 microversion: 2.1 time: 0.172900 Sep 27 10:02:51.440682 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 857/1573] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:51 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1468 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:51.706869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fba45ab7-9565-4234-885c-5bdc7fa59bc8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:51.709613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fba45ab7-9565-4234-885c-5bdc7fa59bc8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:51.716604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba45ab7-9565-4234-885c-5bdc7fa59bc8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:51.717670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba45ab7-9565-4234-885c-5bdc7fa59bc8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:51.718356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba45ab7-9565-4234-885c-5bdc7fa59bc8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:51.719578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba45ab7-9565-4234-885c-5bdc7fa59bc8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:51.720134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba45ab7-9565-4234-885c-5bdc7fa59bc8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:51.720906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fba45ab7-9565-4234-885c-5bdc7fa59bc8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:51.973655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fba45ab7-9565-4234-885c-5bdc7fa59bc8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:51.982980 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fba45ab7-9565-4234-885c-5bdc7fa59bc8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.280271 Sep 27 10:02:51.983570 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 858/1574] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:51 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:52.182717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1f4fd28a-7a51-4366-9304-1b2c9c6179e5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:52.183342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1f4fd28a-7a51-4366-9304-1b2c9c6179e5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:52.193707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f4fd28a-7a51-4366-9304-1b2c9c6179e5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:52.194297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f4fd28a-7a51-4366-9304-1b2c9c6179e5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:52.194641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f4fd28a-7a51-4366-9304-1b2c9c6179e5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:52.195366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f4fd28a-7a51-4366-9304-1b2c9c6179e5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:52.195823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f4fd28a-7a51-4366-9304-1b2c9c6179e5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:52.197041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f4fd28a-7a51-4366-9304-1b2c9c6179e5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:52.437523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1f4fd28a-7a51-4366-9304-1b2c9c6179e5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:52.462248 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1f4fd28a-7a51-4366-9304-1b2c9c6179e5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1321 microversion: 2.1 time: 0.281889 Sep 27 10:02:52.464456 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 859/1575] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:52 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1321 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:52.474204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa4b81b9-533f-4d5b-8f17-f0f9d545ab70 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:52.480868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fa4b81b9-533f-4d5b-8f17-f0f9d545ab70 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:52.495038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4b81b9-533f-4d5b-8f17-f0f9d545ab70 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:52.496035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4b81b9-533f-4d5b-8f17-f0f9d545ab70 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:52.508713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4b81b9-533f-4d5b-8f17-f0f9d545ab70 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:52.510188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4b81b9-533f-4d5b-8f17-f0f9d545ab70 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:52.511708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4b81b9-533f-4d5b-8f17-f0f9d545ab70 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:52.513386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4b81b9-533f-4d5b-8f17-f0f9d545ab70 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:52.552408 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c820eb30-9f33-435d-80f6-ef5202282a0d tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "POST /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.860312 Sep 27 10:02:52.554663 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 717/1576] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:02:50 2022] POST /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-volume_attachments => generated 194 bytes in 1864 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:52.565724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-367609af-8760-43ca-8455-4da03bc0726a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:52.566702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-367609af-8760-43ca-8455-4da03bc0726a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:52.579773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-367609af-8760-43ca-8455-4da03bc0726a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:52.582215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-367609af-8760-43ca-8455-4da03bc0726a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:52.584775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-367609af-8760-43ca-8455-4da03bc0726a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:52.586328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-367609af-8760-43ca-8455-4da03bc0726a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:52.587782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-367609af-8760-43ca-8455-4da03bc0726a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:52.589717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-367609af-8760-43ca-8455-4da03bc0726a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:52.627931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:52.650471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fa4b81b9-533f-4d5b-8f17-f0f9d545ab70 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:52.667197 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fa4b81b9-533f-4d5b-8f17-f0f9d545ab70 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1276 microversion: 2.1 time: 0.199990 Sep 27 10:02:52.667925 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 860/1577] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:52 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1276 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:52.801990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-367609af-8760-43ca-8455-4da03bc0726a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:52.811137 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-367609af-8760-43ca-8455-4da03bc0726a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1468 microversion: 2.1 time: 0.253252 Sep 27 10:02:52.811736 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 718/1578] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:52 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1468 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:53.005878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d9a76c14-6f9b-4e4b-b3af-597934dc3545 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:53.007173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d9a76c14-6f9b-4e4b-b3af-597934dc3545 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:53.019719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a76c14-6f9b-4e4b-b3af-597934dc3545 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:53.020229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a76c14-6f9b-4e4b-b3af-597934dc3545 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:53.020684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a76c14-6f9b-4e4b-b3af-597934dc3545 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:53.021436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a76c14-6f9b-4e4b-b3af-597934dc3545 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:53.025273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a76c14-6f9b-4e4b-b3af-597934dc3545 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:53.028297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a76c14-6f9b-4e4b-b3af-597934dc3545 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:53.073455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:53.249702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-14a44e20-0f4a-4914-8dbe-9c141bb7dafd req-fe4209aa-d8b5-4abe-9500-74c496a37106 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a2aa469-5c5e-4a03-b09e-8e6b2252d615", "name": "network-vif-unplugged", "status": "completed", "tag": "4c0e9dc5-844c-484c-99a2-79cf10f253fe"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:53.280625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-14a44e20-0f4a-4914-8dbe-9c141bb7dafd req-fe4209aa-d8b5-4abe-9500-74c496a37106 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:53.281504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-14a44e20-0f4a-4914-8dbe-9c141bb7dafd req-fe4209aa-d8b5-4abe-9500-74c496a37106 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:53.281966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-14a44e20-0f4a-4914-8dbe-9c141bb7dafd req-fe4209aa-d8b5-4abe-9500-74c496a37106 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:53.298531 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-14a44e20-0f4a-4914-8dbe-9c141bb7dafd req-fe4209aa-d8b5-4abe-9500-74c496a37106 service nova] Creating event network-vif-unplugged:4c0e9dc5-844c-484c-99a2-79cf10f253fe for instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 on nch2-master-1 Sep 27 10:02:53.316336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-14a44e20-0f4a-4914-8dbe-9c141bb7dafd req-fe4209aa-d8b5-4abe-9500-74c496a37106 service nova] Instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:02:53.330527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:53.345009 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-14a44e20-0f4a-4914-8dbe-9c141bb7dafd req-fe4209aa-d8b5-4abe-9500-74c496a37106 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.099383 Sep 27 10:02:53.350019 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 719/1579] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:53 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:02:53.438168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d9a76c14-6f9b-4e4b-b3af-597934dc3545 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:53.447176 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d9a76c14-6f9b-4e4b-b3af-597934dc3545 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.445308 Sep 27 10:02:53.447769 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 861/1580] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:53 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:53.498139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5af3eb66-a2ee-4e05-bf65-9e114dfb7108 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:53.499173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5af3eb66-a2ee-4e05-bf65-9e114dfb7108 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:53.514831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5af3eb66-a2ee-4e05-bf65-9e114dfb7108 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:53.515380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5af3eb66-a2ee-4e05-bf65-9e114dfb7108 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:53.515793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5af3eb66-a2ee-4e05-bf65-9e114dfb7108 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:53.516539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5af3eb66-a2ee-4e05-bf65-9e114dfb7108 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:53.517043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5af3eb66-a2ee-4e05-bf65-9e114dfb7108 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:53.517478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5af3eb66-a2ee-4e05-bf65-9e114dfb7108 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:53.635661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=32 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:53.697982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-087865fd-8d55-4178-9716-928b91d7c0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:53.701841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-087865fd-8d55-4178-9716-928b91d7c0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:53.711962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087865fd-8d55-4178-9716-928b91d7c0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:53.712755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087865fd-8d55-4178-9716-928b91d7c0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:53.713395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087865fd-8d55-4178-9716-928b91d7c0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:53.714364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087865fd-8d55-4178-9716-928b91d7c0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:53.716573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087865fd-8d55-4178-9716-928b91d7c0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:53.717130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087865fd-8d55-4178-9716-928b91d7c0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:53.847260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-087865fd-8d55-4178-9716-928b91d7c0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:53.856634 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-087865fd-8d55-4178-9716-928b91d7c0ab tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1276 microversion: 2.1 time: 0.162568 Sep 27 10:02:53.858436 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 862/1581] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:53 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1276 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:53.863604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b1ef16b3-ecf6-4616-b654-7a934e4f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:53.864559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b1ef16b3-ecf6-4616-b654-7a934e4f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:53.871659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1ef16b3-ecf6-4616-b654-7a934e4f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:53.872562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1ef16b3-ecf6-4616-b654-7a934e4f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:53.873274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1ef16b3-ecf6-4616-b654-7a934e4f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:53.874246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1ef16b3-ecf6-4616-b654-7a934e4f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:53.874921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1ef16b3-ecf6-4616-b654-7a934e4f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:53.875694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1ef16b3-ecf6-4616-b654-7a934e4f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:54.194340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b1ef16b3-ecf6-4616-b654-7a934e4f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:54.210264 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b1ef16b3-ecf6-4616-b654-7a934e4f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1468 microversion: 2.1 time: 0.349412 Sep 27 10:02:54.222464 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 863/1582] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:53 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1468 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:54.325918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5af3eb66-a2ee-4e05-bf65-9e114dfb7108 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:54.335845 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5af3eb66-a2ee-4e05-bf65-9e114dfb7108 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1321 microversion: 2.1 time: 0.843127 Sep 27 10:02:54.340211 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 720/1583] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:53 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1321 bytes in 848 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:54.475209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9984ebb5-2076-416c-aa3c-90b7d1358929 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:54.476469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9984ebb5-2076-416c-aa3c-90b7d1358929 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:54.483678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9984ebb5-2076-416c-aa3c-90b7d1358929 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:54.484436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9984ebb5-2076-416c-aa3c-90b7d1358929 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:54.485149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9984ebb5-2076-416c-aa3c-90b7d1358929 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:54.486035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9984ebb5-2076-416c-aa3c-90b7d1358929 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:54.486657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9984ebb5-2076-416c-aa3c-90b7d1358929 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:54.487241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9984ebb5-2076-416c-aa3c-90b7d1358929 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:54.597321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f1088aa0-86bb-4465-89d8-6eea24c5abe9 req-79e5eb67-2d5d-42d1-a043-f98d1b0f94e9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9a2aa469-5c5e-4a03-b09e-8e6b2252d615", "tag": "4c0e9dc5-844c-484c-99a2-79cf10f253fe"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:54.616321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f1088aa0-86bb-4465-89d8-6eea24c5abe9 req-79e5eb67-2d5d-42d1-a043-f98d1b0f94e9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:54.622100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f1088aa0-86bb-4465-89d8-6eea24c5abe9 req-79e5eb67-2d5d-42d1-a043-f98d1b0f94e9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:54.622760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f1088aa0-86bb-4465-89d8-6eea24c5abe9 req-79e5eb67-2d5d-42d1-a043-f98d1b0f94e9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:54.639964 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f1088aa0-86bb-4465-89d8-6eea24c5abe9 req-79e5eb67-2d5d-42d1-a043-f98d1b0f94e9 service nova] Creating event network-changed:4c0e9dc5-844c-484c-99a2-79cf10f253fe for instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 on nch2-master-1 Sep 27 10:02:54.647752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-f1088aa0-86bb-4465-89d8-6eea24c5abe9 req-79e5eb67-2d5d-42d1-a043-f98d1b0f94e9 service nova] Instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:02:54.661343 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f1088aa0-86bb-4465-89d8-6eea24c5abe9 req-79e5eb67-2d5d-42d1-a043-f98d1b0f94e9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067774 Sep 27 10:02:54.662734 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 721/1584] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:54.885755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a7fb476c-b745-4351-978c-cfb76be6844c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:54.887018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a7fb476c-b745-4351-978c-cfb76be6844c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:54.895683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7fb476c-b745-4351-978c-cfb76be6844c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:54.896906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7fb476c-b745-4351-978c-cfb76be6844c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:54.897809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7fb476c-b745-4351-978c-cfb76be6844c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:54.899539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7fb476c-b745-4351-978c-cfb76be6844c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:54.900400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7fb476c-b745-4351-978c-cfb76be6844c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:54.901613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7fb476c-b745-4351-978c-cfb76be6844c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:54.930223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9984ebb5-2076-416c-aa3c-90b7d1358929 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:54.944834 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9984ebb5-2076-416c-aa3c-90b7d1358929 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.475773 Sep 27 10:02:54.945755 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 864/1585] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:54 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:55.081723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a7fb476c-b745-4351-978c-cfb76be6844c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:55.098365 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a7fb476c-b745-4351-978c-cfb76be6844c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1282 microversion: 2.1 time: 0.217387 Sep 27 10:02:55.099551 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 722/1586] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:54 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1282 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:55.240764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b6a051a7-0843-4214-9181-296c77a5e0a5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:55.243253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b6a051a7-0843-4214-9181-296c77a5e0a5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:55.260187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a051a7-0843-4214-9181-296c77a5e0a5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:55.260862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a051a7-0843-4214-9181-296c77a5e0a5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:55.261415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a051a7-0843-4214-9181-296c77a5e0a5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:55.262359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a051a7-0843-4214-9181-296c77a5e0a5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:55.265394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a051a7-0843-4214-9181-296c77a5e0a5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:55.265969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a051a7-0843-4214-9181-296c77a5e0a5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:55.365733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-149550a9-f328-407a-a0de-255b1bbd750f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:55.366346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-149550a9-f328-407a-a0de-255b1bbd750f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:55.383163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-149550a9-f328-407a-a0de-255b1bbd750f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:55.383697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-149550a9-f328-407a-a0de-255b1bbd750f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:55.384132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-149550a9-f328-407a-a0de-255b1bbd750f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:55.389283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-149550a9-f328-407a-a0de-255b1bbd750f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:55.390583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-149550a9-f328-407a-a0de-255b1bbd750f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:55.391517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-149550a9-f328-407a-a0de-255b1bbd750f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:55.847651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-149550a9-f328-407a-a0de-255b1bbd750f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:55.862783 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-149550a9-f328-407a-a0de-255b1bbd750f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1321 microversion: 2.1 time: 0.499791 Sep 27 10:02:55.876993 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 723/1587] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:55 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1321 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:55.968439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bad750c5-3236-45dc-a88d-d00472c17180 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:55.969416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bad750c5-3236-45dc-a88d-d00472c17180 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:55.984260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bad750c5-3236-45dc-a88d-d00472c17180 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:55.988223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bad750c5-3236-45dc-a88d-d00472c17180 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:55.989883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bad750c5-3236-45dc-a88d-d00472c17180 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:55.992056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bad750c5-3236-45dc-a88d-d00472c17180 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:55.993989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bad750c5-3236-45dc-a88d-d00472c17180 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:55.994660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bad750c5-3236-45dc-a88d-d00472c17180 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:56.367845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bad750c5-3236-45dc-a88d-d00472c17180 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:56.385540 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bad750c5-3236-45dc-a88d-d00472c17180 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.420744 Sep 27 10:02:56.398199 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 724/1588] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:55 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:02:56.412415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8de4f041-197d-48d1-b7b9-ac141027a9e8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:56.413258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8de4f041-197d-48d1-b7b9-ac141027a9e8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:56.422471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8de4f041-197d-48d1-b7b9-ac141027a9e8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:56.423251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8de4f041-197d-48d1-b7b9-ac141027a9e8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:56.423866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8de4f041-197d-48d1-b7b9-ac141027a9e8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:56.425514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8de4f041-197d-48d1-b7b9-ac141027a9e8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:56.426322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8de4f041-197d-48d1-b7b9-ac141027a9e8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:56.426746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8de4f041-197d-48d1-b7b9-ac141027a9e8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:56.437437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b6a051a7-0843-4214-9181-296c77a5e0a5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:56.477333 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b6a051a7-0843-4214-9181-296c77a5e0a5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1468 microversion: 2.1 time: 1.237078 Sep 27 10:02:56.479239 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 865/1589] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:55 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1468 bytes in 1243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:56.639204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8de4f041-197d-48d1-b7b9-ac141027a9e8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:56.663180 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8de4f041-197d-48d1-b7b9-ac141027a9e8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.256839 Sep 27 10:02:56.679919 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 725/1590] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:56 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:56.894805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a3b8f3cf-ee55-4b11-aa71-445440ecd4ba tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:56.896098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a3b8f3cf-ee55-4b11-aa71-445440ecd4ba tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:56.905295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3b8f3cf-ee55-4b11-aa71-445440ecd4ba tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:56.906125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3b8f3cf-ee55-4b11-aa71-445440ecd4ba tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:56.906798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3b8f3cf-ee55-4b11-aa71-445440ecd4ba tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:56.907747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3b8f3cf-ee55-4b11-aa71-445440ecd4ba tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:56.908418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3b8f3cf-ee55-4b11-aa71-445440ecd4ba tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:56.909250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3b8f3cf-ee55-4b11-aa71-445440ecd4ba tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:57.028386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1c1e0baf-f85f-4238-9728-b948cc4409cb req-7d056a57-d33e-4089-ac24-ceb5bd2fad85 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9997587c-6839-4459-bd4b-b4d4cd6fd09d", "tag": "c24245da-1f2e-40bc-b00b-6bdc116ec9a4"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:57.051558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1c1e0baf-f85f-4238-9728-b948cc4409cb req-7d056a57-d33e-4089-ac24-ceb5bd2fad85 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:57.052057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1c1e0baf-f85f-4238-9728-b948cc4409cb req-7d056a57-d33e-4089-ac24-ceb5bd2fad85 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:57.052500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1c1e0baf-f85f-4238-9728-b948cc4409cb req-7d056a57-d33e-4089-ac24-ceb5bd2fad85 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:57.085702 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1c1e0baf-f85f-4238-9728-b948cc4409cb req-7d056a57-d33e-4089-ac24-ceb5bd2fad85 service nova] Creating event network-changed:c24245da-1f2e-40bc-b00b-6bdc116ec9a4 for instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d on nch2-master-1 Sep 27 10:02:57.098766 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1c1e0baf-f85f-4238-9728-b948cc4409cb req-7d056a57-d33e-4089-ac24-ceb5bd2fad85 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.075103 Sep 27 10:02:57.102450 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 726/1591] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:02:57.129093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a3b8f3cf-ee55-4b11-aa71-445440ecd4ba tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:57.139289 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a3b8f3cf-ee55-4b11-aa71-445440ecd4ba tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1321 microversion: 2.1 time: 0.251223 Sep 27 10:02:57.139828 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 866/1592] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:56 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1321 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:57.413632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-593b8153-ee33-4cde-8f89-de2a79031d5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:57.414302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-593b8153-ee33-4cde-8f89-de2a79031d5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:57.422088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-593b8153-ee33-4cde-8f89-de2a79031d5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:57.422565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-593b8153-ee33-4cde-8f89-de2a79031d5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:57.423006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-593b8153-ee33-4cde-8f89-de2a79031d5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:57.423723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-593b8153-ee33-4cde-8f89-de2a79031d5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:57.424177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-593b8153-ee33-4cde-8f89-de2a79031d5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:57.424609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-593b8153-ee33-4cde-8f89-de2a79031d5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:57.501596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6c82f9e-56d6-457e-8ebd-009b6b067ecb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:57.502178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c6c82f9e-56d6-457e-8ebd-009b6b067ecb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:57.509261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6c82f9e-56d6-457e-8ebd-009b6b067ecb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:57.509760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6c82f9e-56d6-457e-8ebd-009b6b067ecb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:57.510180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6c82f9e-56d6-457e-8ebd-009b6b067ecb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:57.511104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6c82f9e-56d6-457e-8ebd-009b6b067ecb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:57.511571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6c82f9e-56d6-457e-8ebd-009b6b067ecb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:57.511977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6c82f9e-56d6-457e-8ebd-009b6b067ecb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:57.572039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-593b8153-ee33-4cde-8f89-de2a79031d5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:57.581983 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-593b8153-ee33-4cde-8f89-de2a79031d5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.172642 Sep 27 10:02:57.583060 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 727/1593] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:57 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:57.675885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c6c82f9e-56d6-457e-8ebd-009b6b067ecb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:57.684579 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c6c82f9e-56d6-457e-8ebd-009b6b067ecb tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1468 microversion: 2.1 time: 0.186601 Sep 27 10:02:57.685204 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 867/1594] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:57 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1468 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:57.691353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-647e7d02-ed8b-4904-944d-9bb65872eded tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:57.692048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-647e7d02-ed8b-4904-944d-9bb65872eded tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:57.698359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647e7d02-ed8b-4904-944d-9bb65872eded tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:57.698913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647e7d02-ed8b-4904-944d-9bb65872eded tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:57.699456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647e7d02-ed8b-4904-944d-9bb65872eded tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:57.700217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647e7d02-ed8b-4904-944d-9bb65872eded tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:57.700777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647e7d02-ed8b-4904-944d-9bb65872eded tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:57.701276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-647e7d02-ed8b-4904-944d-9bb65872eded tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:57.846797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-647e7d02-ed8b-4904-944d-9bb65872eded tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:57.855131 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-647e7d02-ed8b-4904-944d-9bb65872eded tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.167571 Sep 27 10:02:57.871386 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 728/1595] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:57 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:57.971471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5c6e2d9b-62b9-47ef-96d7-2b694909509e req-bed68011-f2df-4d03-8d20-220a776be670 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "96553659-f775-4568-a1a1-c3a9237df534", "name": "network-vif-plugged", "status": "completed", "tag": "2a657e89-bfee-4434-85bf-ab2d711c1e68"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:02:57.986222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5c6e2d9b-62b9-47ef-96d7-2b694909509e req-bed68011-f2df-4d03-8d20-220a776be670 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:57.988160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5c6e2d9b-62b9-47ef-96d7-2b694909509e req-bed68011-f2df-4d03-8d20-220a776be670 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:57.988160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5c6e2d9b-62b9-47ef-96d7-2b694909509e req-bed68011-f2df-4d03-8d20-220a776be670 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:58.001588 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5c6e2d9b-62b9-47ef-96d7-2b694909509e req-bed68011-f2df-4d03-8d20-220a776be670 service nova] Creating event network-vif-plugged:2a657e89-bfee-4434-85bf-ab2d711c1e68 for instance 96553659-f775-4568-a1a1-c3a9237df534 on nch2-master-1 Sep 27 10:02:58.006870 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5c6e2d9b-62b9-47ef-96d7-2b694909509e req-bed68011-f2df-4d03-8d20-220a776be670 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039547 Sep 27 10:02:58.007545 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 868/1596] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:02:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:58.162782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3bb2cb01-e86a-42ef-94ba-b6c297bdcabf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:58.164085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3bb2cb01-e86a-42ef-94ba-b6c297bdcabf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:58.170925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb2cb01-e86a-42ef-94ba-b6c297bdcabf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:58.171432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb2cb01-e86a-42ef-94ba-b6c297bdcabf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:58.171857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb2cb01-e86a-42ef-94ba-b6c297bdcabf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:58.172835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb2cb01-e86a-42ef-94ba-b6c297bdcabf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:58.173495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb2cb01-e86a-42ef-94ba-b6c297bdcabf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:58.173913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bb2cb01-e86a-42ef-94ba-b6c297bdcabf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:58.317310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3bb2cb01-e86a-42ef-94ba-b6c297bdcabf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:58.325810 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3bb2cb01-e86a-42ef-94ba-b6c297bdcabf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1321 microversion: 2.1 time: 0.167199 Sep 27 10:02:58.326429 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 729/1597] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:58 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1321 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:58.608519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-91182383-995f-4d1c-8129-33520ffdc3bc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:58.609301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-91182383-995f-4d1c-8129-33520ffdc3bc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:58.616488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91182383-995f-4d1c-8129-33520ffdc3bc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:58.617007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91182383-995f-4d1c-8129-33520ffdc3bc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:58.617462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91182383-995f-4d1c-8129-33520ffdc3bc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:58.618218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91182383-995f-4d1c-8129-33520ffdc3bc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:58.618693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91182383-995f-4d1c-8129-33520ffdc3bc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:58.619085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91182383-995f-4d1c-8129-33520ffdc3bc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:58.707561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-01e1d15d-7cff-43cc-8e46-8b5e21b099f8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:58.708172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-01e1d15d-7cff-43cc-8e46-8b5e21b099f8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:58.717183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01e1d15d-7cff-43cc-8e46-8b5e21b099f8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:58.718136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01e1d15d-7cff-43cc-8e46-8b5e21b099f8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:58.718136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01e1d15d-7cff-43cc-8e46-8b5e21b099f8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:58.718976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01e1d15d-7cff-43cc-8e46-8b5e21b099f8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:58.719732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01e1d15d-7cff-43cc-8e46-8b5e21b099f8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:58.720166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01e1d15d-7cff-43cc-8e46-8b5e21b099f8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:58.766812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-91182383-995f-4d1c-8129-33520ffdc3bc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:58.775170 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-91182383-995f-4d1c-8129-33520ffdc3bc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.171054 Sep 27 10:02:58.775965 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 869/1598] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:58 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:58.862426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-01e1d15d-7cff-43cc-8e46-8b5e21b099f8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:58.871225 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-01e1d15d-7cff-43cc-8e46-8b5e21b099f8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1468 microversion: 2.1 time: 0.167616 Sep 27 10:02:58.871981 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 730/1599] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:58 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1468 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:58.890838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ead860f0-4c0c-48f0-a76b-9d46535508b4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:58.891611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ead860f0-4c0c-48f0-a76b-9d46535508b4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:58.894945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-978e674c-9236-47b4-8220-9ce97dbfd54c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:58.895632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-978e674c-9236-47b4-8220-9ce97dbfd54c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:58.898055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ead860f0-4c0c-48f0-a76b-9d46535508b4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:58.898450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ead860f0-4c0c-48f0-a76b-9d46535508b4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:58.898887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ead860f0-4c0c-48f0-a76b-9d46535508b4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:58.900062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ead860f0-4c0c-48f0-a76b-9d46535508b4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:58.900800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ead860f0-4c0c-48f0-a76b-9d46535508b4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:58.901228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ead860f0-4c0c-48f0-a76b-9d46535508b4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:58.903581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-978e674c-9236-47b4-8220-9ce97dbfd54c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:58.904235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-978e674c-9236-47b4-8220-9ce97dbfd54c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:58.904697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-978e674c-9236-47b4-8220-9ce97dbfd54c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:58.905412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-978e674c-9236-47b4-8220-9ce97dbfd54c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:58.906045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-978e674c-9236-47b4-8220-9ce97dbfd54c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:58.906643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-978e674c-9236-47b4-8220-9ce97dbfd54c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:59.056989 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-978e674c-9236-47b4-8220-9ce97dbfd54c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1814 microversion: 2.1 time: 0.165682 Sep 27 10:02:59.057866 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 731/1600] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:58 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1814 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:59.073136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ead860f0-4c0c-48f0-a76b-9d46535508b4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:59.081935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-82144d10-4ce3-47dd-b13d-3f7ac7028581 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:59.082298 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ead860f0-4c0c-48f0-a76b-9d46535508b4 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.195372 Sep 27 10:02:59.083896 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 870/1601] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:58 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:59.099234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82144d10-4ce3-47dd-b13d-3f7ac7028581 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:59.099808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82144d10-4ce3-47dd-b13d-3f7ac7028581 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:59.100317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82144d10-4ce3-47dd-b13d-3f7ac7028581 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:59.104381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82144d10-4ce3-47dd-b13d-3f7ac7028581 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:59.105126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82144d10-4ce3-47dd-b13d-3f7ac7028581 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:59.105629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82144d10-4ce3-47dd-b13d-3f7ac7028581 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:59.133804 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-82144d10-4ce3-47dd-b13d-3f7ac7028581 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.055591 Sep 27 10:02:59.134822 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 732/1602] 10.222.0.26 () {58 vars in 1176 bytes} [Tue Sep 27 10:02:59 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:02:59.351852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-263095e2-b814-47ed-8bc5-592a9a073161 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:59.353295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-263095e2-b814-47ed-8bc5-592a9a073161 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:59.357450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:02:59.363336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-263095e2-b814-47ed-8bc5-592a9a073161 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:59.364618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-263095e2-b814-47ed-8bc5-592a9a073161 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:59.365703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-263095e2-b814-47ed-8bc5-592a9a073161 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:59.366813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-263095e2-b814-47ed-8bc5-592a9a073161 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:59.367313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-263095e2-b814-47ed-8bc5-592a9a073161 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:59.367882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-263095e2-b814-47ed-8bc5-592a9a073161 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:59.688230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-263095e2-b814-47ed-8bc5-592a9a073161 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:02:59.702020 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-263095e2-b814-47ed-8bc5-592a9a073161 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1321 microversion: 2.1 time: 0.357508 Sep 27 10:02:59.703277 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 871/1603] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:59 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1321 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:02:59.800976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95f362b1-0c35-4b45-b86c-816e9d676141 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:59.801608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-95f362b1-0c35-4b45-b86c-816e9d676141 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:59.812640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f362b1-0c35-4b45-b86c-816e9d676141 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:59.813858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f362b1-0c35-4b45-b86c-816e9d676141 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:59.814325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f362b1-0c35-4b45-b86c-816e9d676141 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:59.815155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f362b1-0c35-4b45-b86c-816e9d676141 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:59.816574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f362b1-0c35-4b45-b86c-816e9d676141 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:59.817088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f362b1-0c35-4b45-b86c-816e9d676141 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:59.896021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6d984e2e-9865-4c3b-b89e-4f76cc2d87fa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:02:59.897250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6d984e2e-9865-4c3b-b89e-4f76cc2d87fa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:02:59.905804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d984e2e-9865-4c3b-b89e-4f76cc2d87fa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:59.906432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d984e2e-9865-4c3b-b89e-4f76cc2d87fa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:59.906974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d984e2e-9865-4c3b-b89e-4f76cc2d87fa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:02:59.907824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d984e2e-9865-4c3b-b89e-4f76cc2d87fa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:02:59.908756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d984e2e-9865-4c3b-b89e-4f76cc2d87fa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:02:59.909772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d984e2e-9865-4c3b-b89e-4f76cc2d87fa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:00.123713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-95f362b1-0c35-4b45-b86c-816e9d676141 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:00.141051 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-95f362b1-0c35-4b45-b86c-816e9d676141 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1499 microversion: 2.1 time: 0.339347 Sep 27 10:03:00.142537 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 733/1604] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:59 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1499 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:00.147767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2d677f45-026b-46da-9ce3-39e558de2c24 req-feae441d-8d57-4c5d-a4ea-51ad5c89f720 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ba4b5b70-d9bf-43b6-a52e-73be94b31bb1", "name": "network-vif-plugged", "status": "completed", "tag": "14ebcc5e-9aae-4755-a891-78f55436d129"}, {"server_uuid": "6d511fc6-80ab-458e-88a6-593374a3e01c", "name": "network-vif-plugged", "status": "completed", "tag": "3f178b4c-caa9-4399-84bb-8322b9ac1a75"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:00.160645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2d677f45-026b-46da-9ce3-39e558de2c24 req-feae441d-8d57-4c5d-a4ea-51ad5c89f720 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:00.161487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2d677f45-026b-46da-9ce3-39e558de2c24 req-feae441d-8d57-4c5d-a4ea-51ad5c89f720 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:00.163377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2d677f45-026b-46da-9ce3-39e558de2c24 req-feae441d-8d57-4c5d-a4ea-51ad5c89f720 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:00.175614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6d984e2e-9865-4c3b-b89e-4f76cc2d87fa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:00.179777 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2d677f45-026b-46da-9ce3-39e558de2c24 req-feae441d-8d57-4c5d-a4ea-51ad5c89f720 service nova] Creating event network-vif-plugged:14ebcc5e-9aae-4755-a891-78f55436d129 for instance ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 on nch1-master-1 Sep 27 10:03:00.180615 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2d677f45-026b-46da-9ce3-39e558de2c24 req-feae441d-8d57-4c5d-a4ea-51ad5c89f720 service nova] Creating event network-vif-plugged:3f178b4c-caa9-4399-84bb-8322b9ac1a75 for instance 6d511fc6-80ab-458e-88a6-593374a3e01c on nch1-master-1 Sep 27 10:03:00.199835 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2d677f45-026b-46da-9ce3-39e558de2c24 req-feae441d-8d57-4c5d-a4ea-51ad5c89f720 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.055563 Sep 27 10:03:00.201163 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 734/1605] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:00 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:00.202995 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6d984e2e-9865-4c3b-b89e-4f76cc2d87fa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1468 microversion: 2.1 time: 0.307168 Sep 27 10:03:00.204540 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 872/1606] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:02:59 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1468 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:00.208675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a78813e5-cd63-41f7-9ef1-57262922cd1c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:00.209662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a78813e5-cd63-41f7-9ef1-57262922cd1c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:00.217011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a78813e5-cd63-41f7-9ef1-57262922cd1c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:00.217511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a78813e5-cd63-41f7-9ef1-57262922cd1c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:00.218004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a78813e5-cd63-41f7-9ef1-57262922cd1c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:00.218672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a78813e5-cd63-41f7-9ef1-57262922cd1c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:00.219235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a78813e5-cd63-41f7-9ef1-57262922cd1c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:00.219862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a78813e5-cd63-41f7-9ef1-57262922cd1c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:00.486234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a78813e5-cd63-41f7-9ef1-57262922cd1c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:00.500886 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a78813e5-cd63-41f7-9ef1-57262922cd1c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.296060 Sep 27 10:03:00.508903 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 735/1607] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:00 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:00.650262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a24345b5-fbe7-444a-af66-459933243e47 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7", "device": "/dev/xvdb"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:00.653780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a24345b5-fbe7-444a-af66-459933243e47 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:00.663136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a24345b5-fbe7-444a-af66-459933243e47 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:00.663722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a24345b5-fbe7-444a-af66-459933243e47 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:00.664227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a24345b5-fbe7-444a-af66-459933243e47 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:00.665314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a24345b5-fbe7-444a-af66-459933243e47 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:00.665848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a24345b5-fbe7-444a-af66-459933243e47 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:00.666342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a24345b5-fbe7-444a-af66-459933243e47 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:00.729026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4f65c34c-afd7-498f-aa67-b00b2d36cbe4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:00.730505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4f65c34c-afd7-498f-aa67-b00b2d36cbe4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:00.739016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f65c34c-afd7-498f-aa67-b00b2d36cbe4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:00.739989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f65c34c-afd7-498f-aa67-b00b2d36cbe4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:00.740593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f65c34c-afd7-498f-aa67-b00b2d36cbe4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:00.741763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f65c34c-afd7-498f-aa67-b00b2d36cbe4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:00.742369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f65c34c-afd7-498f-aa67-b00b2d36cbe4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:00.743118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f65c34c-afd7-498f-aa67-b00b2d36cbe4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:00.862800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:00.907844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4f65c34c-afd7-498f-aa67-b00b2d36cbe4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:00.917260 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4f65c34c-afd7-498f-aa67-b00b2d36cbe4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1321 microversion: 2.1 time: 0.192054 Sep 27 10:03:00.928708 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 736/1608] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:00 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1321 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:01.167663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8010aeac-6406-4565-ad15-60d639cfd13d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:01.168272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8010aeac-6406-4565-ad15-60d639cfd13d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:01.174548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8010aeac-6406-4565-ad15-60d639cfd13d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:01.175028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8010aeac-6406-4565-ad15-60d639cfd13d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:01.175458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8010aeac-6406-4565-ad15-60d639cfd13d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:01.176161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8010aeac-6406-4565-ad15-60d639cfd13d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:01.176607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8010aeac-6406-4565-ad15-60d639cfd13d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:01.177058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8010aeac-6406-4565-ad15-60d639cfd13d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:01.196747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:01.250590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:01.331363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8010aeac-6406-4565-ad15-60d639cfd13d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:01.341510 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8010aeac-6406-4565-ad15-60d639cfd13d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 200 len: 1476 microversion: 2.1 time: 0.177861 Sep 27 10:03:01.352723 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 737/1609] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:01 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 1476 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:01.352723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7e8d6d5e-fe55-4b41-b10c-3ccb1e9d03d5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:01.352723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7e8d6d5e-fe55-4b41-b10c-3ccb1e9d03d5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:01.354410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e8d6d5e-fe55-4b41-b10c-3ccb1e9d03d5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:01.355086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e8d6d5e-fe55-4b41-b10c-3ccb1e9d03d5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:01.355727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e8d6d5e-fe55-4b41-b10c-3ccb1e9d03d5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:01.356534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e8d6d5e-fe55-4b41-b10c-3ccb1e9d03d5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:01.357088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e8d6d5e-fe55-4b41-b10c-3ccb1e9d03d5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:01.357603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e8d6d5e-fe55-4b41-b10c-3ccb1e9d03d5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:01.499887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7e8d6d5e-fe55-4b41-b10c-3ccb1e9d03d5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:01.507871 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7e8d6d5e-fe55-4b41-b10c-3ccb1e9d03d5 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1461 microversion: 2.1 time: 0.164464 Sep 27 10:03:01.508600 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 738/1610] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:01 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1461 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:01.527737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a378c70-e4c5-4e71-b693-611c16f147eb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-1129916641"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:01.530367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a378c70-e4c5-4e71-b693-611c16f147eb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:01.536853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a378c70-e4c5-4e71-b693-611c16f147eb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:01.537341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a378c70-e4c5-4e71-b693-611c16f147eb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:01.537769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a378c70-e4c5-4e71-b693-611c16f147eb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:01.538477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a378c70-e4c5-4e71-b693-611c16f147eb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:01.538933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a378c70-e4c5-4e71-b693-611c16f147eb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:01.539357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a378c70-e4c5-4e71-b693-611c16f147eb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:01.772067 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a378c70-e4c5-4e71-b693-611c16f147eb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "POST /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1/action" status: 202 len: 0 microversion: 2.1 time: 0.249965 Sep 27 10:03:01.779434 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 739/1611] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:03:01 2022] POST /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1/action => generated 0 bytes in 264 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 10:03:01.779434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4744e328-bf2c-43ff-891f-329fe076261a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:01.779434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4744e328-bf2c-43ff-891f-329fe076261a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:01.786451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4744e328-bf2c-43ff-891f-329fe076261a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:01.786918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4744e328-bf2c-43ff-891f-329fe076261a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:01.787330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4744e328-bf2c-43ff-891f-329fe076261a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:01.788124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4744e328-bf2c-43ff-891f-329fe076261a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:01.788615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4744e328-bf2c-43ff-891f-329fe076261a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:01.789078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4744e328-bf2c-43ff-891f-329fe076261a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:01.955247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4744e328-bf2c-43ff-891f-329fe076261a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:01.965503 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4744e328-bf2c-43ff-891f-329fe076261a tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.191190 Sep 27 10:03:01.966293 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 740/1612] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:01 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:02.090775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fc0dc31a-83bd-42d1-9c6f-694638bc2544 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:02.091367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fc0dc31a-83bd-42d1-9c6f-694638bc2544 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:02.097770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc0dc31a-83bd-42d1-9c6f-694638bc2544 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.098258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc0dc31a-83bd-42d1-9c6f-694638bc2544 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.099377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc0dc31a-83bd-42d1-9c6f-694638bc2544 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.099624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc0dc31a-83bd-42d1-9c6f-694638bc2544 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.100210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc0dc31a-83bd-42d1-9c6f-694638bc2544 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.100517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc0dc31a-83bd-42d1-9c6f-694638bc2544 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.261576 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fc0dc31a-83bd-42d1-9c6f-694638bc2544 tempest-MigrationsAdminTest-755919817 tempest-MigrationsAdminTest-755919817-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1612 microversion: 2.1 time: 0.291565 Sep 27 10:03:02.263267 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 741/1613] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:01 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1612 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:02.269213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-87247ed1-aabb-4c4c-bf4d-318996c3155c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63607940>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:02.313318 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a24345b5-fbe7-444a-af66-459933243e47 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "POST /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.667066 Sep 27 10:03:02.313953 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 873/1614] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:03:00 2022] POST /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/os-volume_attachments => generated 194 bytes in 1669 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:03:02.319437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-041c94e9-5dbb-4f89-a36e-3c64a6b65678 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:02.320423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-041c94e9-5dbb-4f89-a36e-3c64a6b65678 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:02.328163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-041c94e9-5dbb-4f89-a36e-3c64a6b65678 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.328734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-041c94e9-5dbb-4f89-a36e-3c64a6b65678 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.329102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-041c94e9-5dbb-4f89-a36e-3c64a6b65678 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.329818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-041c94e9-5dbb-4f89-a36e-3c64a6b65678 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.330459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-041c94e9-5dbb-4f89-a36e-3c64a6b65678 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.330958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-041c94e9-5dbb-4f89-a36e-3c64a6b65678 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.339169 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-87247ed1-aabb-4c4c-bf4d-318996c3155c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.074757 Sep 27 10:03:02.339768 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 742/1615] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:02 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:02.354940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a81f8d87-8003-44d1-a6dc-9953d801197c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:02.355457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a81f8d87-8003-44d1-a6dc-9953d801197c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:02.373601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a81f8d87-8003-44d1-a6dc-9953d801197c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.374123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a81f8d87-8003-44d1-a6dc-9953d801197c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.375701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a81f8d87-8003-44d1-a6dc-9953d801197c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.376434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a81f8d87-8003-44d1-a6dc-9953d801197c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.377104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a81f8d87-8003-44d1-a6dc-9953d801197c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.377719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a81f8d87-8003-44d1-a6dc-9953d801197c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.416556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a81f8d87-8003-44d1-a6dc-9953d801197c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:03:02.503568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-041c94e9-5dbb-4f89-a36e-3c64a6b65678 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:02.514999 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-041c94e9-5dbb-4f89-a36e-3c64a6b65678 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1511 microversion: 2.1 time: 0.199599 Sep 27 10:03:02.515857 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 874/1616] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:02 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1511 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:02.531435 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a81f8d87-8003-44d1-a6dc-9953d801197c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 204 len: 0 microversion: 2.1 time: 0.190331 Sep 27 10:03:02.532382 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 743/1617] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:03:02 2022] DELETE /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:03:02.539767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8330afe0-0b35-470c-904a-6ba45747c496 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:02.541137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8330afe0-0b35-470c-904a-6ba45747c496 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:02.542235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0f51c958-3215-4c01-b008-713af6b72d87 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:02.543116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0f51c958-3215-4c01-b008-713af6b72d87 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:02.547950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8330afe0-0b35-470c-904a-6ba45747c496 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.548524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8330afe0-0b35-470c-904a-6ba45747c496 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.549060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8330afe0-0b35-470c-904a-6ba45747c496 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.549604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f51c958-3215-4c01-b008-713af6b72d87 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.550088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f51c958-3215-4c01-b008-713af6b72d87 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.550521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f51c958-3215-4c01-b008-713af6b72d87 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.550678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8330afe0-0b35-470c-904a-6ba45747c496 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.550999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8330afe0-0b35-470c-904a-6ba45747c496 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.551222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f51c958-3215-4c01-b008-713af6b72d87 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.552088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f51c958-3215-4c01-b008-713af6b72d87 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.559325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8330afe0-0b35-470c-904a-6ba45747c496 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.559505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f51c958-3215-4c01-b008-713af6b72d87 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.759570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0f51c958-3215-4c01-b008-713af6b72d87 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:02.769473 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0f51c958-3215-4c01-b008-713af6b72d87 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1467 microversion: 2.1 time: 0.232738 Sep 27 10:03:02.770698 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 744/1618] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:02 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1467 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:02.774978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8330afe0-0b35-470c-904a-6ba45747c496 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:02.784742 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8330afe0-0b35-470c-904a-6ba45747c496 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1512 microversion: 2.1 time: 0.248987 Sep 27 10:03:02.786350 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 875/1619] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:02 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1512 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:02.812245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-89153bc9-fb40-4fd9-a9c6-7212f93d080a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:02.812890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-89153bc9-fb40-4fd9-a9c6-7212f93d080a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:02.820816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:INSERT=3,nova_api:SELECT=96,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:02.823433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89153bc9-fb40-4fd9-a9c6-7212f93d080a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.824321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89153bc9-fb40-4fd9-a9c6-7212f93d080a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.825024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89153bc9-fb40-4fd9-a9c6-7212f93d080a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.826022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89153bc9-fb40-4fd9-a9c6-7212f93d080a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.826695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89153bc9-fb40-4fd9-a9c6-7212f93d080a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:02.827542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89153bc9-fb40-4fd9-a9c6-7212f93d080a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:02.989783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f847ce7-b786-4be7-b22b-539fd12a321e tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:02.990922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9f847ce7-b786-4be7-b22b-539fd12a321e tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:02.999101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f847ce7-b786-4be7-b22b-539fd12a321e tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:02.999707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f847ce7-b786-4be7-b22b-539fd12a321e tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:03.000238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f847ce7-b786-4be7-b22b-539fd12a321e tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:03.001082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f847ce7-b786-4be7-b22b-539fd12a321e tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:03.001903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f847ce7-b786-4be7-b22b-539fd12a321e tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:03.003073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f847ce7-b786-4be7-b22b-539fd12a321e tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:03.046485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-89153bc9-fb40-4fd9-a9c6-7212f93d080a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:03.061182 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-89153bc9-fb40-4fd9-a9c6-7212f93d080a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 200 len: 1321 microversion: 2.1 time: 0.253429 Sep 27 10:03:03.062734 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 745/1620] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:02 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 1321 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:03.193296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9f847ce7-b786-4be7-b22b-539fd12a321e tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:03.202499 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9f847ce7-b786-4be7-b22b-539fd12a321e tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.217707 Sep 27 10:03:03.206143 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 876/1621] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:02 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:03.364200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e111e349-0ba0-4945-a368-1876faaeb73f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f635c7550>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:03.439866 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e111e349-0ba0-4945-a368-1876faaeb73f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.088233 Sep 27 10:03:03.440574 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 746/1622] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:03 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:03.733441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2a33aa52-23d8-4029-9295-efa5a2099c4f tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:03.736762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2a33aa52-23d8-4029-9295-efa5a2099c4f tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:03.746057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a33aa52-23d8-4029-9295-efa5a2099c4f tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:03.746550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a33aa52-23d8-4029-9295-efa5a2099c4f tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:03.747198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a33aa52-23d8-4029-9295-efa5a2099c4f tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:03.748102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a33aa52-23d8-4029-9295-efa5a2099c4f tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:03.749013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a33aa52-23d8-4029-9295-efa5a2099c4f tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:03.749458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a33aa52-23d8-4029-9295-efa5a2099c4f tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:03.791757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:03.795744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3bf4525a-26a3-4faf-90df-358459d734ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:03.796370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3bf4525a-26a3-4faf-90df-358459d734ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:03.808843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bf4525a-26a3-4faf-90df-358459d734ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:03.809436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bf4525a-26a3-4faf-90df-358459d734ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:03.809850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bf4525a-26a3-4faf-90df-358459d734ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:03.810776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bf4525a-26a3-4faf-90df-358459d734ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:03.811118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bf4525a-26a3-4faf-90df-358459d734ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:03.811607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bf4525a-26a3-4faf-90df-358459d734ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:03.813477 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2a33aa52-23d8-4029-9295-efa5a2099c4f tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.084097 Sep 27 10:03:03.814291 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 877/1623] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:03:03 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-volume_attachments => generated 197 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:03.832815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-556562d6-3b59-4ba2-a673-f07543d4b96b tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:03.833725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-556562d6-3b59-4ba2-a673-f07543d4b96b tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:03.842878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556562d6-3b59-4ba2-a673-f07543d4b96b tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:03.843690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556562d6-3b59-4ba2-a673-f07543d4b96b tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:03.844947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556562d6-3b59-4ba2-a673-f07543d4b96b tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:03.845681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556562d6-3b59-4ba2-a673-f07543d4b96b tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:03.846910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556562d6-3b59-4ba2-a673-f07543d4b96b tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:03.847421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556562d6-3b59-4ba2-a673-f07543d4b96b tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:04.021807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-556562d6-3b59-4ba2-a673-f07543d4b96b tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lazy-loading 'services' on Instance uuid 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:04.103539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3bf4525a-26a3-4faf-90df-358459d734ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:04.139916 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3bf4525a-26a3-4faf-90df-358459d734ec tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1467 microversion: 2.1 time: 0.349853 Sep 27 10:03:04.140931 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 747/1624] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:03 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1467 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:04.151981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-0b00b83d-b24f-4d40-a038-09d6418fe4d4 req-60dbf253-5c6b-4987-80ef-3e0353e4c882 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a2aa469-5c5e-4a03-b09e-8e6b2252d615", "name": "network-vif-plugged", "status": "completed", "tag": "4c0e9dc5-844c-484c-99a2-79cf10f253fe"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:04.180016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0b00b83d-b24f-4d40-a038-09d6418fe4d4 req-60dbf253-5c6b-4987-80ef-3e0353e4c882 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:04.180644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0b00b83d-b24f-4d40-a038-09d6418fe4d4 req-60dbf253-5c6b-4987-80ef-3e0353e4c882 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:04.181281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0b00b83d-b24f-4d40-a038-09d6418fe4d4 req-60dbf253-5c6b-4987-80ef-3e0353e4c882 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:04.196741 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-0b00b83d-b24f-4d40-a038-09d6418fe4d4 req-60dbf253-5c6b-4987-80ef-3e0353e4c882 service nova] Creating event network-vif-plugged:4c0e9dc5-844c-484c-99a2-79cf10f253fe for instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 on nch2-master-1 Sep 27 10:03:04.204379 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-0b00b83d-b24f-4d40-a038-09d6418fe4d4 req-60dbf253-5c6b-4987-80ef-3e0353e4c882 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061397 Sep 27 10:03:04.205080 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 748/1625] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:04.213922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1cdc8dc9-29bf-4e0f-8007-c5ce206ef8e4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:04.215043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1cdc8dc9-29bf-4e0f-8007-c5ce206ef8e4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:04.216253 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-556562d6-3b59-4ba2-a673-f07543d4b96b tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-volume_attachments/814c3373-89e6-4c1a-b1e0-8ddb0523d638" status: 202 len: 0 microversion: 2.1 time: 0.388462 Sep 27 10:03:04.216989 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 878/1626] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:03:03 2022] DELETE /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-volume_attachments/814c3373-89e6-4c1a-b1e0-8ddb0523d638 => generated 0 bytes in 395 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:03:04.223292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cdc8dc9-29bf-4e0f-8007-c5ce206ef8e4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:04.224056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cdc8dc9-29bf-4e0f-8007-c5ce206ef8e4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:04.224479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cdc8dc9-29bf-4e0f-8007-c5ce206ef8e4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:04.225659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cdc8dc9-29bf-4e0f-8007-c5ce206ef8e4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:04.226280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cdc8dc9-29bf-4e0f-8007-c5ce206ef8e4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:04.227041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cdc8dc9-29bf-4e0f-8007-c5ce206ef8e4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:04.235202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7ed09be6-81cd-49fc-8dbc-924bb35c8137 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:04.235202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7ed09be6-81cd-49fc-8dbc-924bb35c8137 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:04.241993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ed09be6-81cd-49fc-8dbc-924bb35c8137 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:04.242480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ed09be6-81cd-49fc-8dbc-924bb35c8137 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:04.243079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ed09be6-81cd-49fc-8dbc-924bb35c8137 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:04.243891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ed09be6-81cd-49fc-8dbc-924bb35c8137 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:04.244347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ed09be6-81cd-49fc-8dbc-924bb35c8137 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:04.244923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ed09be6-81cd-49fc-8dbc-924bb35c8137 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:04.424829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1cdc8dc9-29bf-4e0f-8007-c5ce206ef8e4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:04.440785 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1cdc8dc9-29bf-4e0f-8007-c5ce206ef8e4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 200 len: 1321 microversion: 2.1 time: 0.233462 Sep 27 10:03:04.441424 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 749/1627] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:04 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 1321 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:04.450126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7ed09be6-81cd-49fc-8dbc-924bb35c8137 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:04.459402 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7ed09be6-81cd-49fc-8dbc-924bb35c8137 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.230591 Sep 27 10:03:04.465477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0687b838-8679-4375-86f9-372713f3af95 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f635d9f70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:04.469109 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 879/1628] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:04 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:04.522389 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0687b838-8679-4375-86f9-372713f3af95 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.061166 Sep 27 10:03:04.531204 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 750/1629] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:04 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:05.178769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e624a9e3-3340-41ba-86c6-bd183f0524e7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:05.181761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e624a9e3-3340-41ba-86c6-bd183f0524e7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:05.197071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e624a9e3-3340-41ba-86c6-bd183f0524e7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:05.198163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e624a9e3-3340-41ba-86c6-bd183f0524e7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:05.198742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e624a9e3-3340-41ba-86c6-bd183f0524e7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:05.199946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e624a9e3-3340-41ba-86c6-bd183f0524e7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:05.200417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e624a9e3-3340-41ba-86c6-bd183f0524e7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:05.200974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e624a9e3-3340-41ba-86c6-bd183f0524e7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:05.389722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e624a9e3-3340-41ba-86c6-bd183f0524e7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:05.405047 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e624a9e3-3340-41ba-86c6-bd183f0524e7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1467 microversion: 2.1 time: 0.236281 Sep 27 10:03:05.405047 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 880/1630] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:05 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1467 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:05.468559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9d528fc-cdd5-4a87-bcb1-e1c09e7cd207 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:05.469228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9d528fc-cdd5-4a87-bcb1-e1c09e7cd207 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:05.478914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9d528fc-cdd5-4a87-bcb1-e1c09e7cd207 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:05.479570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9d528fc-cdd5-4a87-bcb1-e1c09e7cd207 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:05.480024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9d528fc-cdd5-4a87-bcb1-e1c09e7cd207 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:05.481181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9d528fc-cdd5-4a87-bcb1-e1c09e7cd207 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:05.481792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9d528fc-cdd5-4a87-bcb1-e1c09e7cd207 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:05.482357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9d528fc-cdd5-4a87-bcb1-e1c09e7cd207 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:05.494171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a656a32e-1401-46b8-826c-1b8bfac03562 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:05.495105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a656a32e-1401-46b8-826c-1b8bfac03562 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:05.502421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a656a32e-1401-46b8-826c-1b8bfac03562 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:05.503344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a656a32e-1401-46b8-826c-1b8bfac03562 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:05.504051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a656a32e-1401-46b8-826c-1b8bfac03562 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:05.505193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a656a32e-1401-46b8-826c-1b8bfac03562 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:05.505762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a656a32e-1401-46b8-826c-1b8bfac03562 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:05.506475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a656a32e-1401-46b8-826c-1b8bfac03562 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:05.710687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a656a32e-1401-46b8-826c-1b8bfac03562 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:05.721128 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a656a32e-1401-46b8-826c-1b8bfac03562 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.230325 Sep 27 10:03:05.721833 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 881/1631] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:05 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:05.726983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d32e8be5-b5f2-4803-89a8-798d31df6065 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c173700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:05.738672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c9d528fc-cdd5-4a87-bcb1-e1c09e7cd207 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:05.753287 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9d528fc-cdd5-4a87-bcb1-e1c09e7cd207 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 200 len: 1321 microversion: 2.1 time: 0.287482 Sep 27 10:03:05.754275 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 751/1632] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:05 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 1321 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:05.786303 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d32e8be5-b5f2-4803-89a8-798d31df6065 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.063019 Sep 27 10:03:05.787130 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 882/1633] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:05 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:06.427913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3e56ab92-8042-489a-a668-4707b5f46b76 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:06.428748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e56ab92-8042-489a-a668-4707b5f46b76 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:06.443216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e56ab92-8042-489a-a668-4707b5f46b76 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:06.443989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e56ab92-8042-489a-a668-4707b5f46b76 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:06.444626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e56ab92-8042-489a-a668-4707b5f46b76 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:06.445648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e56ab92-8042-489a-a668-4707b5f46b76 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:06.446164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e56ab92-8042-489a-a668-4707b5f46b76 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:06.446968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e56ab92-8042-489a-a668-4707b5f46b76 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:06.739369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3e56ab92-8042-489a-a668-4707b5f46b76 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:06.752205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4e504e33-1f4e-4ad7-9417-c61daf205022 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:06.753564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4e504e33-1f4e-4ad7-9417-c61daf205022 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:06.762193 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3e56ab92-8042-489a-a668-4707b5f46b76 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1467 microversion: 2.1 time: 0.331779 Sep 27 10:03:06.763488 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 752/1634] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:06 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1467 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:06.769527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e504e33-1f4e-4ad7-9417-c61daf205022 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:06.770307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e504e33-1f4e-4ad7-9417-c61daf205022 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:06.771531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e504e33-1f4e-4ad7-9417-c61daf205022 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:06.772975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e504e33-1f4e-4ad7-9417-c61daf205022 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:06.774383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e504e33-1f4e-4ad7-9417-c61daf205022 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:06.775024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e504e33-1f4e-4ad7-9417-c61daf205022 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:06.783179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2e2b9edc-8f43-4d29-9194-e8eb3a2068a1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:06.785522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2e2b9edc-8f43-4d29-9194-e8eb3a2068a1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:06.800046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2b9edc-8f43-4d29-9194-e8eb3a2068a1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:06.801234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2b9edc-8f43-4d29-9194-e8eb3a2068a1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:06.801744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2b9edc-8f43-4d29-9194-e8eb3a2068a1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:06.803144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2b9edc-8f43-4d29-9194-e8eb3a2068a1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:06.803667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2b9edc-8f43-4d29-9194-e8eb3a2068a1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:06.804217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2b9edc-8f43-4d29-9194-e8eb3a2068a1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:06.974021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4e504e33-1f4e-4ad7-9417-c61daf205022 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:06.989128 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4e504e33-1f4e-4ad7-9417-c61daf205022 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.243096 Sep 27 10:03:06.989656 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 883/1635] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:06 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:06.995590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5726a0fb-99a9-43e0-bca2-fdbf0d3b4575 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c1733a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:07.025899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2e2b9edc-8f43-4d29-9194-e8eb3a2068a1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:07.038488 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2e2b9edc-8f43-4d29-9194-e8eb3a2068a1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 200 len: 1512 microversion: 2.1 time: 0.259069 Sep 27 10:03:07.051972 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 753/1636] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:06 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 1512 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:07.071600 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5726a0fb-99a9-43e0-bca2-fdbf0d3b4575 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.080719 Sep 27 10:03:07.072508 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 884/1637] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:06 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:07.078608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:07.102884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:07.106293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'flavor': '451', 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:07.145841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:07.146557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:07.147878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:07.150536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:07.151241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:07.152608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:07.159684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-b19dd0aa-efca-4405-a05b-7d724ebe16dc req-d7ba8b1d-9b6d-4b00-8ed7-20cee004344c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9a2aa469-5c5e-4a03-b09e-8e6b2252d615", "name": "network-vif-deleted", "tag": "4c0e9dc5-844c-484c-99a2-79cf10f253fe"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:07.183777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b19dd0aa-efca-4405-a05b-7d724ebe16dc req-d7ba8b1d-9b6d-4b00-8ed7-20cee004344c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:07.186033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b19dd0aa-efca-4405-a05b-7d724ebe16dc req-d7ba8b1d-9b6d-4b00-8ed7-20cee004344c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:07.186843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b19dd0aa-efca-4405-a05b-7d724ebe16dc req-d7ba8b1d-9b6d-4b00-8ed7-20cee004344c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:07.209854 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b19dd0aa-efca-4405-a05b-7d724ebe16dc req-d7ba8b1d-9b6d-4b00-8ed7-20cee004344c service nova] Creating event network-vif-deleted:4c0e9dc5-844c-484c-99a2-79cf10f253fe for instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 on nch2-master-1 Sep 27 10:03:07.218757 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-b19dd0aa-efca-4405-a05b-7d724ebe16dc req-d7ba8b1d-9b6d-4b00-8ed7-20cee004344c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063233 Sep 27 10:03:07.219595 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 885/1638] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:03:07.277430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:07.298290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:07.409475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:07.429912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:07.430991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:07.432216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:07.717777 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dd40128f-9947-4a9e-a9ca-49afcb008606 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/detail?flavor=451" status: 200 len: 1514 microversion: 2.1 time: 0.644140 Sep 27 10:03:07.733267 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 754/1639] 10.222.0.26 () {58 vars in 1167 bytes} [Tue Sep 27 10:03:07 2022] GET /compute/v2.1/servers/detail?flavor=451 => generated 1514 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:07.748828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:07.754951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:07.756626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'name': 'tempest-ListServerFiltersTestJSON-instance-1561922851', 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:07.763086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:07.763930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:07.764622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:07.765623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:07.766352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:07.767097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:07.791723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:07.793134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:07.806941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:07.807820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:07.809333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:07.810633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:07.811308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:07.812548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:07.861830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:07.862386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:07.862886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:07.863377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lazy-loading 'fault' on Instance uuid 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:07.942505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:08.004917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:08.028858 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-68cb0169-32a7-4664-b402-b6e940fff1bf tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1430 microversion: 2.1 time: 0.241110 Sep 27 10:03:08.029548 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 755/1640] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:07 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1430 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:08.038089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f33b472c-107c-4b51-b242-f05e2646fd89 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:08.038962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f33b472c-107c-4b51-b242-f05e2646fd89 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:08.050355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f33b472c-107c-4b51-b242-f05e2646fd89 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.058217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f33b472c-107c-4b51-b242-f05e2646fd89 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.058791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f33b472c-107c-4b51-b242-f05e2646fd89 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.060009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f33b472c-107c-4b51-b242-f05e2646fd89 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.060777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f33b472c-107c-4b51-b242-f05e2646fd89 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.061303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f33b472c-107c-4b51-b242-f05e2646fd89 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.064404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:08.082220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.082961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.083588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.252594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f33b472c-107c-4b51-b242-f05e2646fd89 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:08.262950 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-93327dd4-4409-49d2-b667-3915e57f4e21 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/detail?name=tempest-ListServerFiltersTestJSON-instance-1561922851" status: 200 len: 1515 microversion: 2.1 time: 0.518746 Sep 27 10:03:08.272760 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 886/1641] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:03:07 2022] GET /compute/v2.1/servers/detail?name=tempest-ListServerFiltersTestJSON-instance-1561922851 => generated 1515 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:08.272978 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f33b472c-107c-4b51-b242-f05e2646fd89 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.233989 Sep 27 10:03:08.272978 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 756/1642] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:08 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:08.272978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-49cebe71-e4a5-4bb5-b05b-5a385cd6ff73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f635f8790>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:08.284346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8efdf237-1272-4b10-a8e1-963eef94db97 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:08.285158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8efdf237-1272-4b10-a8e1-963eef94db97 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:08.297580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8efdf237-1272-4b10-a8e1-963eef94db97 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.298195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8efdf237-1272-4b10-a8e1-963eef94db97 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.298962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8efdf237-1272-4b10-a8e1-963eef94db97 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.299803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8efdf237-1272-4b10-a8e1-963eef94db97 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.300687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8efdf237-1272-4b10-a8e1-963eef94db97 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.301117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8efdf237-1272-4b10-a8e1-963eef94db97 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.333405 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-49cebe71-e4a5-4bb5-b05b-5a385cd6ff73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.066480 Sep 27 10:03:08.342265 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 757/1643] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:08 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:08.342265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:08.349206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:08.350405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'vm_state': ['active'], 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:08.359643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.359643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.360224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.360843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.361301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.361873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.459656 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8efdf237-1272-4b10-a8e1-963eef94db97 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1860 microversion: 2.1 time: 0.194847 Sep 27 10:03:08.459842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:08.460810 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 887/1644] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:08 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1860 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:08.486652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": true, "disk_over_commit": false}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:08.489220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:08.495519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.496008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.496509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.497313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.497770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.498193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.522833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lazy-loading 'flavor' on Instance uuid 11a0d56b-3e1f-40e7-9e3c-30fca67df25e {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:08.534730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Going to try to live migrate instance to nch2-master-1 {{(pid=145968) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Sep 27 10:03:08.546693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:08.554513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.554927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.555437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.710660 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-93d7f989-16e8-448b-8b87-0d48420b6008 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/detail?status=active" status: 200 len: 4518 microversion: 2.1 time: 0.371908 Sep 27 10:03:08.711704 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 758/1645] 10.222.0.26 () {58 vars in 1173 bytes} [Tue Sep 27 10:03:08 2022] GET /compute/v2.1/servers/detail?status=active => generated 4518 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:08.733651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b14e8ecc-2893-4cf0-961e-a67258c02c9a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:08.737323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b14e8ecc-2893-4cf0-961e-a67258c02c9a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:08.754804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b14e8ecc-2893-4cf0-961e-a67258c02c9a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.755298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b14e8ecc-2893-4cf0-961e-a67258c02c9a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.756118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b14e8ecc-2893-4cf0-961e-a67258c02c9a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.757752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b14e8ecc-2893-4cf0-961e-a67258c02c9a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.758696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b14e8ecc-2893-4cf0-961e-a67258c02c9a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.759668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b14e8ecc-2893-4cf0-961e-a67258c02c9a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.797083 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b14e8ecc-2893-4cf0-961e-a67258c02c9a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.068062 Sep 27 10:03:08.803966 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 759/1646] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:03:08 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-volume_attachments => generated 25 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:03:08.803966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:08.808398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:08.809415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.servers [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Removing options 'limit' from query {{(pid=145969) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1490}} Sep 27 10:03:08.810631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:08.816765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.817621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.818498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.819587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.820674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.821451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:08.941569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:08.950729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:08.951215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:08.951652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.109208 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f8941290-020f-49eb-9864-e2fe8a3ee20f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/detail?limit=1" status: 200 len: 1660 microversion: 2.1 time: 0.310088 Sep 27 10:03:09.110191 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 760/1647] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 10:03:08 2022] GET /compute/v2.1/servers/detail?limit=1 => generated 1660 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:09.116448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b4d8858-d1f7-4a8e-ba6b-c5055e1d20a9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:09.120532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0b4d8858-d1f7-4a8e-ba6b-c5055e1d20a9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:09.127812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4d8858-d1f7-4a8e-ba6b-c5055e1d20a9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.128593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4d8858-d1f7-4a8e-ba6b-c5055e1d20a9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.129213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4d8858-d1f7-4a8e-ba6b-c5055e1d20a9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.130183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4d8858-d1f7-4a8e-ba6b-c5055e1d20a9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.130635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4d8858-d1f7-4a8e-ba6b-c5055e1d20a9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.131089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4d8858-d1f7-4a8e-ba6b-c5055e1d20a9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.190095 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0b4d8858-d1f7-4a8e-ba6b-c5055e1d20a9 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.078586 Sep 27 10:03:09.190839 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 761/1648] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:03:09 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1/os-volume_attachments => generated 25 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:03:09.196311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:09.197233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:09.224815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.225849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.226513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.227837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.228513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.229208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.283795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.284260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.284748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.285177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lazy-loading 'fault' on Instance uuid 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:09.330646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:09.339167 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e58c324a-570b-44b9-98cc-141251781072 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1430 microversion: 2.1 time: 0.146982 Sep 27 10:03:09.339793 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 762/1649] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:09 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1430 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:09.344549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ace80111-a613-4673-a989-4489a50e14ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:09.349277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ace80111-a613-4673-a989-4489a50e14ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:09.350444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ace80111-a613-4673-a989-4489a50e14ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'vm_state': ['active'], 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:09.356001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace80111-a613-4673-a989-4489a50e14ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.356484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace80111-a613-4673-a989-4489a50e14ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.356947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace80111-a613-4673-a989-4489a50e14ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.357697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace80111-a613-4673-a989-4489a50e14ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.358150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace80111-a613-4673-a989-4489a50e14ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.358571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ace80111-a613-4673-a989-4489a50e14ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.448013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-ace80111-a613-4673-a989-4489a50e14ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:09.505985 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ace80111-a613-4673-a989-4489a50e14ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?status=active" status: 200 len: 1012 microversion: 2.1 time: 0.165127 Sep 27 10:03:09.506773 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 763/1650] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 10:03:09 2022] GET /compute/v2.1/servers?status=active => generated 1012 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:09.511229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1ddc54b0-4095-43ce-aa0a-a8985b7269da tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:09.511718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1ddc54b0-4095-43ce-aa0a-a8985b7269da tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:09.517890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ddc54b0-4095-43ce-aa0a-a8985b7269da tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.518380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ddc54b0-4095-43ce-aa0a-a8985b7269da tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.518814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ddc54b0-4095-43ce-aa0a-a8985b7269da tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.519508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ddc54b0-4095-43ce-aa0a-a8985b7269da tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.519957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ddc54b0-4095-43ce-aa0a-a8985b7269da tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.520372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ddc54b0-4095-43ce-aa0a-a8985b7269da tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.549155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1ddc54b0-4095-43ce-aa0a-a8985b7269da tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:03:09.636105 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1ddc54b0-4095-43ce-aa0a-a8985b7269da tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 204 len: 0 microversion: 2.1 time: 0.128164 Sep 27 10:03:09.636721 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 764/1651] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:03:09 2022] DELETE /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:03:09.641728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8dae36ae-eae7-4b0b-b1a8-42fea0b271c7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:09.642407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8dae36ae-eae7-4b0b-b1a8-42fea0b271c7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:09.648377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dae36ae-eae7-4b0b-b1a8-42fea0b271c7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.649052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dae36ae-eae7-4b0b-b1a8-42fea0b271c7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.650074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dae36ae-eae7-4b0b-b1a8-42fea0b271c7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.650783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dae36ae-eae7-4b0b-b1a8-42fea0b271c7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.651244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dae36ae-eae7-4b0b-b1a8-42fea0b271c7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.651672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dae36ae-eae7-4b0b-b1a8-42fea0b271c7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.794468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8dae36ae-eae7-4b0b-b1a8-42fea0b271c7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:09.808897 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8dae36ae-eae7-4b0b-b1a8-42fea0b271c7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.170826 Sep 27 10:03:09.809885 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 765/1652] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:09 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:09.815381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a6f170c4-da6e-4663-921d-f8909384aa5f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c08ea60>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:09.860094 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a6f170c4-da6e-4663-921d-f8909384aa5f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.048896 Sep 27 10:03:09.860604 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 766/1653] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:09 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:09.865106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:09.870062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:09.870505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.servers [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Removing options 'limit' from query {{(pid=145969) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1490}} Sep 27 10:03:09.871240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:09.877316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.877795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.878238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.878969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:09.879552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:09.879863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:09.907747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:09.937115 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d0ad99f6-6891-4c42-b6f0-9db77ced34ef tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?limit=100000" status: 200 len: 1012 microversion: 2.1 time: 0.075403 Sep 27 10:03:09.937696 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 767/1654] 10.222.0.26 () {58 vars in 1150 bytes} [Tue Sep 27 10:03:09 2022] GET /compute/v2.1/servers?limit=100000 => generated 1012 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:09.944696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestBootFromVolume-server-1126196652", "imageRef": "", "flavorRef": "452", "metadata": {"hello": "world"}, "accessIPv4": "1.1.1.1", "accessIPv6": "0000:0000:0000:0000:0000:babe:220.12.22.2", "adminPass": "***", "networks": [{"uuid": "406cedbc-40ad-438a-95d4-8282ddfd193a"}], "security_groups": [{"name": "tempest-securitygroup--1712832039"}], "key_name": "tempest-keypair-253434363", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=", "block_device_mapping_v2": [{"uuid": "ff80fef3-6188-4603-afd2-55aa0b605e0c", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}], "OS-DCF:diskConfig": "}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:10.061248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] validate_networks() for [('406cedbc-40ad-438a-95d4-8282ddfd193a', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:03:10.462477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Getting quotas for project 183cb066566c4510945d6eb0d41a2853. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:03:10.467561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Getting quotas for user 06dc3376ada04ffe922c9023f4182259 and project 183cb066566c4510945d6eb0d41a2853. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:03:10.481346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:03:10.481773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:03:10.482251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:03:10.482663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:03:10.483083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:03:10.492763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:10.493599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:10.494046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:10.494791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:10.495273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:10.495699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:10.519383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Getting quotas for project 183cb066566c4510945d6eb0d41a2853. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:03:10.524938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Getting quotas for user 06dc3376ada04ffe922c9023f4182259 and project 183cb066566c4510945d6eb0d41a2853. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:03:10.604284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:03:10.606585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='ff80fef3-6188-4603-afd2-55aa0b605e0c',volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:03:10.745598 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3d0fc42a-c477-4d73-a429-438158b15d86 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 407 microversion: 2.1 time: 0.806763 Sep 27 10:03:10.746330 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 768/1655] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:03:09 2022] POST /compute/v2.1/servers => generated 407 bytes in 808 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:03:10.750743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d3617c0c-adc9-4b64-a07c-04b8c206130c tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:10.751350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d3617c0c-adc9-4b64-a07c-04b8c206130c tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:10.758628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3617c0c-adc9-4b64-a07c-04b8c206130c tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:10.759156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3617c0c-adc9-4b64-a07c-04b8c206130c tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:10.759577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3617c0c-adc9-4b64-a07c-04b8c206130c tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:10.760280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3617c0c-adc9-4b64-a07c-04b8c206130c tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:10.760785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3617c0c-adc9-4b64-a07c-04b8c206130c tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:10.761210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3617c0c-adc9-4b64-a07c-04b8c206130c tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:10.898342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d3617c0c-adc9-4b64-a07c-04b8c206130c tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:10.907580 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d3617c0c-adc9-4b64-a07c-04b8c206130c tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1681 microversion: 2.1 time: 0.160315 Sep 27 10:03:10.908069 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 769/1656] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:10 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1681 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:10.912824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f3e73ed7-64ca-4d75-a087-d7561adb8f28 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:10.918160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f3e73ed7-64ca-4d75-a087-d7561adb8f28 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:10.918640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f3e73ed7-64ca-4d75-a087-d7561adb8f28 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:10.923815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e73ed7-64ca-4d75-a087-d7561adb8f28 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:10.924284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e73ed7-64ca-4d75-a087-d7561adb8f28 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:10.924752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e73ed7-64ca-4d75-a087-d7561adb8f28 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:10.925473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e73ed7-64ca-4d75-a087-d7561adb8f28 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:10.925918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e73ed7-64ca-4d75-a087-d7561adb8f28 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:10.926334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e73ed7-64ca-4d75-a087-d7561adb8f28 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:10.939751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:10.955320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-f3e73ed7-64ca-4d75-a087-d7561adb8f28 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:10.983821 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f3e73ed7-64ca-4d75-a087-d7561adb8f28 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 1012 microversion: 2.1 time: 0.074518 Sep 27 10:03:10.984227 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 770/1657] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:03:10 2022] GET /compute/v2.1/servers => generated 1012 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:10.988691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-df41827f-5f7a-4dba-be50-2904a2282cfe req-9e0ba145-0c98-47d1-bf9f-65f8213bc4c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1", "name": "network-vif-plugged", "status": "completed", "tag": "cc9dca96-4fbe-402f-b886-7bb414b8545d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:10.998761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-df41827f-5f7a-4dba-be50-2904a2282cfe req-9e0ba145-0c98-47d1-bf9f-65f8213bc4c2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:10.999091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-df41827f-5f7a-4dba-be50-2904a2282cfe req-9e0ba145-0c98-47d1-bf9f-65f8213bc4c2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:10.999518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-df41827f-5f7a-4dba-be50-2904a2282cfe req-9e0ba145-0c98-47d1-bf9f-65f8213bc4c2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.012536 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-df41827f-5f7a-4dba-be50-2904a2282cfe req-9e0ba145-0c98-47d1-bf9f-65f8213bc4c2 service nova] Creating event network-vif-plugged:cc9dca96-4fbe-402f-b886-7bb414b8545d for instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 on nch2-master-1 Sep 27 10:03:11.018307 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-df41827f-5f7a-4dba-be50-2904a2282cfe req-9e0ba145-0c98-47d1-bf9f-65f8213bc4c2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032890 Sep 27 10:03:11.019206 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 771/1658] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:11.024256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:11.025238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:11.031638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.032121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.032549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.033282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.033738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.034285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.066727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.067209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.067638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.068081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lazy-loading 'fault' on Instance uuid 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:11.110028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:11.118299 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-228b7ea3-dff5-4037-975d-12f2e6783b58 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1430 microversion: 2.1 time: 0.098237 Sep 27 10:03:11.118974 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 772/1659] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:11 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1430 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:11.124498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-57ebbe4c-d747-4c68-8355-a45943487c3c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:11.125048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-57ebbe4c-d747-4c68-8355-a45943487c3c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:11.184577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-57ebbe4c-d747-4c68-8355-a45943487c3c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:11.191899 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-57ebbe4c-d747-4c68-8355-a45943487c3c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1121 microversion: 2.1 time: 0.071840 Sep 27 10:03:11.192486 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 773/1660] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:11 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1121 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:11.197110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9c98539b-7fe6-4ef9-bd77-30af340a56a2 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:11.197725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9c98539b-7fe6-4ef9-bd77-30af340a56a2 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:11.203572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c98539b-7fe6-4ef9-bd77-30af340a56a2 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.204094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c98539b-7fe6-4ef9-bd77-30af340a56a2 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.204627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c98539b-7fe6-4ef9-bd77-30af340a56a2 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.205387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c98539b-7fe6-4ef9-bd77-30af340a56a2 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.205840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c98539b-7fe6-4ef9-bd77-30af340a56a2 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.206323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c98539b-7fe6-4ef9-bd77-30af340a56a2 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.349316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9c98539b-7fe6-4ef9-bd77-30af340a56a2 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:11.357359 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9c98539b-7fe6-4ef9-bd77-30af340a56a2 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.163763 Sep 27 10:03:11.357905 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 774/1661] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:11 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:11.362309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5ab4bc4b-6545-4ffb-a45a-4d7f93b761a7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f630bbc10>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:11.408221 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5ab4bc4b-6545-4ffb-a45a-4d7f93b761a7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.049265 Sep 27 10:03:11.409055 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 775/1662] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:11 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:11.413689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-edc7db38-180e-45ac-ae9c-17659564eda5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:11.418686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-edc7db38-180e-45ac-ae9c-17659564eda5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:11.419440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-edc7db38-180e-45ac-ae9c-17659564eda5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'flavor': '451', 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:11.428117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:11.433269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edc7db38-180e-45ac-ae9c-17659564eda5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.433749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edc7db38-180e-45ac-ae9c-17659564eda5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.434523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edc7db38-180e-45ac-ae9c-17659564eda5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.435259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edc7db38-180e-45ac-ae9c-17659564eda5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.435695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edc7db38-180e-45ac-ae9c-17659564eda5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.436301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edc7db38-180e-45ac-ae9c-17659564eda5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.525937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-edc7db38-180e-45ac-ae9c-17659564eda5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:11.589062 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-edc7db38-180e-45ac-ae9c-17659564eda5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?flavor=451" status: 200 len: 346 microversion: 2.1 time: 0.179159 Sep 27 10:03:11.589634 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 776/1663] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:03:11 2022] GET /compute/v2.1/servers?flavor=451 => generated 346 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:11.602285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ac40d4c1-0340-4ac2-a26f-9958ef9c40c3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:11.607197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ac40d4c1-0340-4ac2-a26f-9958ef9c40c3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:11.607674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.servers [None req-ac40d4c1-0340-4ac2-a26f-9958ef9c40c3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Removing options 'limit' from query {{(pid=145969) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1490}} Sep 27 10:03:11.608313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ac40d4c1-0340-4ac2-a26f-9958ef9c40c3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:11.613552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac40d4c1-0340-4ac2-a26f-9958ef9c40c3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.614025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac40d4c1-0340-4ac2-a26f-9958ef9c40c3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.614456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac40d4c1-0340-4ac2-a26f-9958ef9c40c3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.615185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac40d4c1-0340-4ac2-a26f-9958ef9c40c3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.615654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac40d4c1-0340-4ac2-a26f-9958ef9c40c3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.616065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac40d4c1-0340-4ac2-a26f-9958ef9c40c3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.641217 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ac40d4c1-0340-4ac2-a26f-9958ef9c40c3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?limit=1" status: 200 len: 485 microversion: 2.1 time: 0.042544 Sep 27 10:03:11.641801 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 777/1664] 10.222.0.26 () {58 vars in 1140 bytes} [Tue Sep 27 10:03:11 2022] GET /compute/v2.1/servers?limit=1 => generated 485 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:03:11.654402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4cc025ad-71d2-4414-9a8d-1932ee768db2 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:11.659677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4cc025ad-71d2-4414-9a8d-1932ee768db2 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:11.660342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4cc025ad-71d2-4414-9a8d-1932ee768db2 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'name': 'tempest-ListServerFiltersTestJSON-instance-1561922851', 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:11.668473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cc025ad-71d2-4414-9a8d-1932ee768db2 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.669006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cc025ad-71d2-4414-9a8d-1932ee768db2 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.669617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cc025ad-71d2-4414-9a8d-1932ee768db2 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.670434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cc025ad-71d2-4414-9a8d-1932ee768db2 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.671398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cc025ad-71d2-4414-9a8d-1932ee768db2 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.671966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cc025ad-71d2-4414-9a8d-1932ee768db2 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.773345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-4cc025ad-71d2-4414-9a8d-1932ee768db2 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:11.836628 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4cc025ad-71d2-4414-9a8d-1932ee768db2 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?name=tempest-ListServerFiltersTestJSON-instance-1561922851" status: 200 len: 346 microversion: 2.1 time: 0.185683 Sep 27 10:03:11.837819 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 778/1665] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:03:11 2022] GET /compute/v2.1/servers?name=tempest-ListServerFiltersTestJSON-instance-1561922851 => generated 346 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:03:11.850628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-390d2048-f3f8-49b3-a574-7c34401791e3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:11.857516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-390d2048-f3f8-49b3-a574-7c34401791e3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.858004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-390d2048-f3f8-49b3-a574-7c34401791e3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.858438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-390d2048-f3f8-49b3-a574-7c34401791e3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.922989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-390d2048-f3f8-49b3-a574-7c34401791e3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Going to try to stop instance {{(pid=145969) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 10:03:11.974709 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-390d2048-f3f8-49b3-a574-7c34401791e3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "POST /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c/action" status: 202 len: 0 microversion: 2.1 time: 0.127783 Sep 27 10:03:11.975298 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 779/1666] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:03:11 2022] POST /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c/action => generated 0 bytes in 129 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:03:11.981298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32994a02-c49e-494c-b33a-338cf61f8c34 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:11.981993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32994a02-c49e-494c-b33a-338cf61f8c34 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:11.988163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32994a02-c49e-494c-b33a-338cf61f8c34 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.988644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32994a02-c49e-494c-b33a-338cf61f8c34 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.989183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32994a02-c49e-494c-b33a-338cf61f8c34 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:11.989843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32994a02-c49e-494c-b33a-338cf61f8c34 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:11.990290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32994a02-c49e-494c-b33a-338cf61f8c34 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:11.990695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32994a02-c49e-494c-b33a-338cf61f8c34 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:12.136395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32994a02-c49e-494c-b33a-338cf61f8c34 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:12.150006 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32994a02-c49e-494c-b33a-338cf61f8c34 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1681 microversion: 2.1 time: 0.173454 Sep 27 10:03:12.150794 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 780/1667] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:11 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1681 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:12.172085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd5cd27a-81a7-4c6a-a05d-e4133e23399c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:12.172931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd5cd27a-81a7-4c6a-a05d-e4133e23399c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:12.179636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd5cd27a-81a7-4c6a-a05d-e4133e23399c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:12.180494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd5cd27a-81a7-4c6a-a05d-e4133e23399c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:12.181548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd5cd27a-81a7-4c6a-a05d-e4133e23399c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:12.182310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd5cd27a-81a7-4c6a-a05d-e4133e23399c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:12.182775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd5cd27a-81a7-4c6a-a05d-e4133e23399c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:12.183209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd5cd27a-81a7-4c6a-a05d-e4133e23399c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:12.514492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd5cd27a-81a7-4c6a-a05d-e4133e23399c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:12.524813 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd5cd27a-81a7-4c6a-a05d-e4133e23399c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.358344 Sep 27 10:03:12.525661 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 781/1668] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:12 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:12.533480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:12.534291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:12.546751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:12.547550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:12.548283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:12.549241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:12.549903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:12.550574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:12.594507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:12.595094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:12.595547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:12.596018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lazy-loading 'fault' on Instance uuid 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:12.674869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:12.684789 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8ae7b40d-239c-4c61-99d0-79bc29550f8c tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 200 len: 1430 microversion: 2.1 time: 0.157484 Sep 27 10:03:12.697908 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 782/1669] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:12 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 1430 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:12.708134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0366078e-6d1a-4271-aab3-0c362b945815 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:12.708900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0366078e-6d1a-4271-aab3-0c362b945815 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:12.719885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0366078e-6d1a-4271-aab3-0c362b945815 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:12.720527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0366078e-6d1a-4271-aab3-0c362b945815 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:12.721050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0366078e-6d1a-4271-aab3-0c362b945815 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:12.721833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0366078e-6d1a-4271-aab3-0c362b945815 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:12.722925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0366078e-6d1a-4271-aab3-0c362b945815 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:12.723418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0366078e-6d1a-4271-aab3-0c362b945815 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:12.913733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0366078e-6d1a-4271-aab3-0c362b945815 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:12.924651 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0366078e-6d1a-4271-aab3-0c362b945815 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1237 microversion: 2.1 time: 0.225230 Sep 27 10:03:12.925328 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 783/1670] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:12 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1237 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:12.943217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8339306c-8a63-43f5-a75b-f2eb4a7a48d1 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:12.944533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8339306c-8a63-43f5-a75b-f2eb4a7a48d1 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:12.952645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8339306c-8a63-43f5-a75b-f2eb4a7a48d1 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:12.953174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8339306c-8a63-43f5-a75b-f2eb4a7a48d1 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:12.953662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8339306c-8a63-43f5-a75b-f2eb4a7a48d1 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:12.954380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8339306c-8a63-43f5-a75b-f2eb4a7a48d1 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:12.954869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8339306c-8a63-43f5-a75b-f2eb4a7a48d1 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:12.955484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8339306c-8a63-43f5-a75b-f2eb4a7a48d1 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:13.579615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8339306c-8a63-43f5-a75b-f2eb4a7a48d1 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:13.589512 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8339306c-8a63-43f5-a75b-f2eb4a7a48d1 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.655062 Sep 27 10:03:13.590834 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 784/1671] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:12 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 665 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:13.595959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cbda9829-4821-488e-89df-91a5cfa480ca tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62efd5e0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:13.692379 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cbda9829-4821-488e-89df-91a5cfa480ca tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.100101 Sep 27 10:03:13.693365 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 785/1672] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:13 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:13.701090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ffa38142-a910-41f9-aa0b-4f77cff6f770 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:13.701908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ffa38142-a910-41f9-aa0b-4f77cff6f770 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:13.709907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffa38142-a910-41f9-aa0b-4f77cff6f770 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:13.710927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffa38142-a910-41f9-aa0b-4f77cff6f770 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:13.711621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffa38142-a910-41f9-aa0b-4f77cff6f770 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:13.712760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffa38142-a910-41f9-aa0b-4f77cff6f770 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:13.713393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffa38142-a910-41f9-aa0b-4f77cff6f770 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:13.713771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffa38142-a910-41f9-aa0b-4f77cff6f770 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:13.963547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ffa38142-a910-41f9-aa0b-4f77cff6f770 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:13.971723 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ffa38142-a910-41f9-aa0b-4f77cff6f770 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1681 microversion: 2.1 time: 0.274545 Sep 27 10:03:13.972917 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 786/1673] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:13 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1681 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:13.977779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3e6f30a6-442c-402b-bd41-1f2b7c15d679 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:13.978754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e6f30a6-442c-402b-bd41-1f2b7c15d679 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:13.984948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6f30a6-442c-402b-bd41-1f2b7c15d679 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:13.985516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6f30a6-442c-402b-bd41-1f2b7c15d679 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:13.985966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6f30a6-442c-402b-bd41-1f2b7c15d679 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:13.986685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6f30a6-442c-402b-bd41-1f2b7c15d679 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:13.987140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6f30a6-442c-402b-bd41-1f2b7c15d679 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:13.987755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6f30a6-442c-402b-bd41-1f2b7c15d679 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:14.365035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3e6f30a6-442c-402b-bd41-1f2b7c15d679 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:14.373619 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3e6f30a6-442c-402b-bd41-1f2b7c15d679 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.399473 Sep 27 10:03:14.374215 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 787/1674] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:13 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:14.380627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:14.381415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 9a2aa469-5c5e-4a03-b09e-8e6b2252d615] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:14.387807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:14.388290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:14.388834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:14.389741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:14.390200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:14.392684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:14.398173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:14.409298 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] HTTP exception thrown: Instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 could not be found. Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Request method failure captured: Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 HTTP/1.1 Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMskCLHf4tkqS3pSUi9t60wr9dJGDTGxCi0y5vV2-npAQZy9nMughBkyNmzpYFTIfDMDU6ida_Ku6aBfVqvfMnjWlj1lI-ZQCS98nDVxSoBfWwyvM3HHJuzPC-Gj1jfOzb_4Y8JWfbz7YAFvVvXC5-61s0N6Z0oAPVLuEC5rZAIWkjPo Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e79eae6310354ca58046e59dfac23009 Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:03:14.413556 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e79eae6310354ca58046e59dfac23009"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e79eae6310354ca58046e59dfac23009"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e79eae6310354ca58046e59dfac23009"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e79eae6310354ca58046e59dfac23009"}]}] Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e79eae6310354ca58046e59dfac23009 Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-MigrationsAdminTest-328318727-project Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 9544cdb0392f416197a586d99e8a3210 Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-MigrationsAdminTest-328318727-project Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 could not be found. Sep 27 10:03:14.415320 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '9a2aa469-5c5e-4a03-b09e-8e6b2252d615'} Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 could not be found. Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:14.416215 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 could not be found. Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 could not be found. Sep 27 10:03:14.417760 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:14.419074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Returning 404 to user: Instance 9a2aa469-5c5e-4a03-b09e-8e6b2252d615 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:14.419074 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-23ddb273-654b-416a-82ec-666fd92e6a35 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615" status: 404 len: 111 microversion: 2.1 time: 0.039729 Sep 27 10:03:14.419074 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 788/1675] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:14 2022] GET /compute/v2.1/servers/9a2aa469-5c5e-4a03-b09e-8e6b2252d615 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:03:14.423604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f63c9bb8-186a-4d94-9adf-fdce08b4e456 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:14.423876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f63c9bb8-186a-4d94-9adf-fdce08b4e456 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:14.431565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63c9bb8-186a-4d94-9adf-fdce08b4e456 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:14.432882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63c9bb8-186a-4d94-9adf-fdce08b4e456 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:14.433285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63c9bb8-186a-4d94-9adf-fdce08b4e456 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:14.434014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63c9bb8-186a-4d94-9adf-fdce08b4e456 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:14.434805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63c9bb8-186a-4d94-9adf-fdce08b4e456 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:14.435358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63c9bb8-186a-4d94-9adf-fdce08b4e456 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:14.568295 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0a8b44ce-0027-4bda-99e9-65d7383b4f01 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/action" status: 202 len: 0 microversion: 2.1 time: 6.085665 Sep 27 10:03:14.572424 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 888/1676] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:03:08 2022] POST /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/action => generated 0 bytes in 6091 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:03:14.577819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-51286111-a342-426d-b239-de013fc0133e req-ff72c40b-1706-4eaf-9539-a92c3af92ba8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb6339bd-6a0a-497c-8064-cfcd4030b6c2", "tag": "4a81581e-1fd6-48e4-b61d-caf266779586"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:14.589802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-51286111-a342-426d-b239-de013fc0133e req-ff72c40b-1706-4eaf-9539-a92c3af92ba8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:14.590638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-51286111-a342-426d-b239-de013fc0133e req-ff72c40b-1706-4eaf-9539-a92c3af92ba8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:14.591301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-51286111-a342-426d-b239-de013fc0133e req-ff72c40b-1706-4eaf-9539-a92c3af92ba8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:14.618821 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-51286111-a342-426d-b239-de013fc0133e req-ff72c40b-1706-4eaf-9539-a92c3af92ba8 service nova] Creating event network-changed:4a81581e-1fd6-48e4-b61d-caf266779586 for instance cb6339bd-6a0a-497c-8064-cfcd4030b6c2 on nch1-master-1 Sep 27 10:03:14.624896 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-51286111-a342-426d-b239-de013fc0133e req-ff72c40b-1706-4eaf-9539-a92c3af92ba8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050540 Sep 27 10:03:14.625685 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 889/1677] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:14.632344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33f28a6c-298b-4dbc-8759-5fc1650c5710 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:14.633467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33f28a6c-298b-4dbc-8759-5fc1650c5710 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:14.642189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f28a6c-298b-4dbc-8759-5fc1650c5710 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:14.643014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f28a6c-298b-4dbc-8759-5fc1650c5710 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:14.643453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f28a6c-298b-4dbc-8759-5fc1650c5710 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:14.644382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f28a6c-298b-4dbc-8759-5fc1650c5710 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:14.645182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f28a6c-298b-4dbc-8759-5fc1650c5710 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:14.646452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f28a6c-298b-4dbc-8759-5fc1650c5710 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:14.685457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33f28a6c-298b-4dbc-8759-5fc1650c5710 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:03:14.724630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f63c9bb8-186a-4d94-9adf-fdce08b4e456 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:14.737931 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f63c9bb8-186a-4d94-9adf-fdce08b4e456 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1293 microversion: 2.1 time: 0.317542 Sep 27 10:03:14.740417 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 789/1678] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:14 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1293 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:14.746951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc329cc7-b06c-4dfa-b03d-5383bb460107 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:14.754598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc329cc7-b06c-4dfa-b03d-5383bb460107 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:14.767017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc329cc7-b06c-4dfa-b03d-5383bb460107 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:14.769462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc329cc7-b06c-4dfa-b03d-5383bb460107 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:14.770011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc329cc7-b06c-4dfa-b03d-5383bb460107 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:14.770822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc329cc7-b06c-4dfa-b03d-5383bb460107 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:14.771337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc329cc7-b06c-4dfa-b03d-5383bb460107 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:14.772107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc329cc7-b06c-4dfa-b03d-5383bb460107 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:14.841360 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33f28a6c-298b-4dbc-8759-5fc1650c5710 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "DELETE /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 204 len: 0 microversion: 2.1 time: 0.213971 Sep 27 10:03:14.842183 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 890/1679] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:03:14 2022] DELETE /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 0 bytes in 216 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:03:14.849570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0f7227cc-36e5-43f0-a353-5353f1f18eae tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:14.850895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0f7227cc-36e5-43f0-a353-5353f1f18eae tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:14.860841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7227cc-36e5-43f0-a353-5353f1f18eae tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:14.861803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7227cc-36e5-43f0-a353-5353f1f18eae tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:14.862652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7227cc-36e5-43f0-a353-5353f1f18eae tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:14.863752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7227cc-36e5-43f0-a353-5353f1f18eae tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:14.864533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7227cc-36e5-43f0-a353-5353f1f18eae tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:14.865145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7227cc-36e5-43f0-a353-5353f1f18eae tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:15.096689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cc329cc7-b06c-4dfa-b03d-5383bb460107 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:15.107793 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cc329cc7-b06c-4dfa-b03d-5383bb460107 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.365173 Sep 27 10:03:15.108955 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 790/1680] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:14 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:15.114349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6c44929f-a723-4095-9c6c-d1dc934462b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62eaf9d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:15.143261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0f7227cc-36e5-43f0-a353-5353f1f18eae tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:15.155185 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0f7227cc-36e5-43f0-a353-5353f1f18eae tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.311316 Sep 27 10:03:15.157676 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 891/1681] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:14 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:15.162683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ecd73fef-5888-46c1-a811-b1fb4178cff7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:15.163508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ecd73fef-5888-46c1-a811-b1fb4178cff7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:15.172268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecd73fef-5888-46c1-a811-b1fb4178cff7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:15.173306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecd73fef-5888-46c1-a811-b1fb4178cff7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:15.173786 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6c44929f-a723-4095-9c6c-d1dc934462b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.063561 Sep 27 10:03:15.174244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecd73fef-5888-46c1-a811-b1fb4178cff7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:15.175452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecd73fef-5888-46c1-a811-b1fb4178cff7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:15.176224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecd73fef-5888-46c1-a811-b1fb4178cff7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:15.176970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecd73fef-5888-46c1-a811-b1fb4178cff7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:15.177572 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 791/1682] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:15 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:15.186226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d3441732-d8cd-4cce-a2e6-7dfb8d74646c req-fa40dd44-96e3-475b-a153-45e425537cc3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9997587c-6839-4459-bd4b-b4d4cd6fd09d", "name": "network-vif-plugged", "status": "completed", "tag": "c24245da-1f2e-40bc-b00b-6bdc116ec9a4"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:15.218774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d3441732-d8cd-4cce-a2e6-7dfb8d74646c req-fa40dd44-96e3-475b-a153-45e425537cc3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:15.219564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d3441732-d8cd-4cce-a2e6-7dfb8d74646c req-fa40dd44-96e3-475b-a153-45e425537cc3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:15.220329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d3441732-d8cd-4cce-a2e6-7dfb8d74646c req-fa40dd44-96e3-475b-a153-45e425537cc3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:15.221166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:15.238911 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d3441732-d8cd-4cce-a2e6-7dfb8d74646c req-fa40dd44-96e3-475b-a153-45e425537cc3 service nova] Creating event network-vif-plugged:c24245da-1f2e-40bc-b00b-6bdc116ec9a4 for instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d on nch2-master-1 Sep 27 10:03:15.247642 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d3441732-d8cd-4cce-a2e6-7dfb8d74646c req-fa40dd44-96e3-475b-a153-45e425537cc3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068145 Sep 27 10:03:15.251166 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 792/1683] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:15.257052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a25d090-951c-446e-b6aa-d9fbe3317a67 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:15.257805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a25d090-951c-446e-b6aa-d9fbe3317a67 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:15.266230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a25d090-951c-446e-b6aa-d9fbe3317a67 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:15.266738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a25d090-951c-446e-b6aa-d9fbe3317a67 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:15.267246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a25d090-951c-446e-b6aa-d9fbe3317a67 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:15.268630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a25d090-951c-446e-b6aa-d9fbe3317a67 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:15.269289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a25d090-951c-446e-b6aa-d9fbe3317a67 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:15.269730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a25d090-951c-446e-b6aa-d9fbe3317a67 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:15.445326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0a25d090-951c-446e-b6aa-d9fbe3317a67 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:15.456301 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a25d090-951c-446e-b6aa-d9fbe3317a67 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1615 microversion: 2.1 time: 0.203523 Sep 27 10:03:15.457222 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 793/1684] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:15 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1615 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:15.462913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cf0eef4a-57ca-4921-a584-c6bf4349404c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:15.464067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cf0eef4a-57ca-4921-a584-c6bf4349404c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:15.470522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf0eef4a-57ca-4921-a584-c6bf4349404c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:15.471463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf0eef4a-57ca-4921-a584-c6bf4349404c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:15.472128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf0eef4a-57ca-4921-a584-c6bf4349404c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:15.473086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf0eef4a-57ca-4921-a584-c6bf4349404c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:15.473748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf0eef4a-57ca-4921-a584-c6bf4349404c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:15.474369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf0eef4a-57ca-4921-a584-c6bf4349404c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:15.486470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ecd73fef-5888-46c1-a811-b1fb4178cff7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:15.501634 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ecd73fef-5888-46c1-a811-b1fb4178cff7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1535 microversion: 2.1 time: 0.342177 Sep 27 10:03:15.503177 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 892/1685] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:15 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1535 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:15.728241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cf0eef4a-57ca-4921-a584-c6bf4349404c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:15.738019 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cf0eef4a-57ca-4921-a584-c6bf4349404c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.279473 Sep 27 10:03:15.755177 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 794/1686] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:15 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:15.768183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d0f4edfb-9ecc-42ec-ad5f-fd1b6254e9a8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:15.769768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d0f4edfb-9ecc-42ec-ad5f-fd1b6254e9a8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:15.785537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f4edfb-9ecc-42ec-ad5f-fd1b6254e9a8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:15.786234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f4edfb-9ecc-42ec-ad5f-fd1b6254e9a8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:15.786770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f4edfb-9ecc-42ec-ad5f-fd1b6254e9a8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:15.787602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f4edfb-9ecc-42ec-ad5f-fd1b6254e9a8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:15.788152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f4edfb-9ecc-42ec-ad5f-fd1b6254e9a8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:15.788923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f4edfb-9ecc-42ec-ad5f-fd1b6254e9a8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:15.792188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-854719ac-b133-4d50-838d-377494e0d59e req-77ebc85f-42f4-486d-91b1-69d73a746b8c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0fa79578-f299-4e5d-8dea-8bd1ca1c31d1", "name": "network-vif-deleted", "tag": "ffaeeb25-b086-4ac5-b059-d12a6497adf1"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:15.803829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-854719ac-b133-4d50-838d-377494e0d59e req-77ebc85f-42f4-486d-91b1-69d73a746b8c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:15.804393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-854719ac-b133-4d50-838d-377494e0d59e req-77ebc85f-42f4-486d-91b1-69d73a746b8c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:15.805298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-854719ac-b133-4d50-838d-377494e0d59e req-77ebc85f-42f4-486d-91b1-69d73a746b8c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:15.818991 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-854719ac-b133-4d50-838d-377494e0d59e req-77ebc85f-42f4-486d-91b1-69d73a746b8c service nova] Creating event network-vif-deleted:ffaeeb25-b086-4ac5-b059-d12a6497adf1 for instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 on nch2-master-1 Sep 27 10:03:15.824707 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-854719ac-b133-4d50-838d-377494e0d59e req-77ebc85f-42f4-486d-91b1-69d73a746b8c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037205 Sep 27 10:03:15.826093 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 795/1687] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:03:15.940792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d0f4edfb-9ecc-42ec-ad5f-fd1b6254e9a8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:15.956749 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d0f4edfb-9ecc-42ec-ad5f-fd1b6254e9a8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1293 microversion: 2.1 time: 0.192906 Sep 27 10:03:15.969914 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 893/1688] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:15 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1293 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:16.138489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d2590a23-e7e5-44cb-b0ad-ce69297e1d40 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:16.139198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d2590a23-e7e5-44cb-b0ad-ce69297e1d40 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:16.147505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2590a23-e7e5-44cb-b0ad-ce69297e1d40 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.148492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2590a23-e7e5-44cb-b0ad-ce69297e1d40 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.149397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2590a23-e7e5-44cb-b0ad-ce69297e1d40 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:16.150588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2590a23-e7e5-44cb-b0ad-ce69297e1d40 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.151465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2590a23-e7e5-44cb-b0ad-ce69297e1d40 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.152437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2590a23-e7e5-44cb-b0ad-ce69297e1d40 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:16.183553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f2c9f247-76cb-45c9-83a4-a741877b8942 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:16.184506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f2c9f247-76cb-45c9-83a4-a741877b8942 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:16.190740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2c9f247-76cb-45c9-83a4-a741877b8942 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.191223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2c9f247-76cb-45c9-83a4-a741877b8942 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.191659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2c9f247-76cb-45c9-83a4-a741877b8942 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:16.192840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2c9f247-76cb-45c9-83a4-a741877b8942 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.193579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2c9f247-76cb-45c9-83a4-a741877b8942 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.194015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2c9f247-76cb-45c9-83a4-a741877b8942 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:16.301060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d2590a23-e7e5-44cb-b0ad-ce69297e1d40 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:16.311415 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d2590a23-e7e5-44cb-b0ad-ce69297e1d40 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.179265 Sep 27 10:03:16.312012 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 796/1689] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:16 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:16.326721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d1ddd1ef-a076-481a-86e5-91f95495cc4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62eaf430>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:16.338014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f2c9f247-76cb-45c9-83a4-a741877b8942 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:16.346235 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f2c9f247-76cb-45c9-83a4-a741877b8942 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.167150 Sep 27 10:03:16.346813 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 894/1690] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:16 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:16.373658 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d1ddd1ef-a076-481a-86e5-91f95495cc4a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.052580 Sep 27 10:03:16.374246 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 797/1691] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:16 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:16.484041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-74a8d5f6-f260-4b63-97e8-fe41a2f6669a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:16.484769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-74a8d5f6-f260-4b63-97e8-fe41a2f6669a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:16.492104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74a8d5f6-f260-4b63-97e8-fe41a2f6669a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.492609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74a8d5f6-f260-4b63-97e8-fe41a2f6669a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.493227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74a8d5f6-f260-4b63-97e8-fe41a2f6669a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:16.494018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74a8d5f6-f260-4b63-97e8-fe41a2f6669a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.494412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74a8d5f6-f260-4b63-97e8-fe41a2f6669a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.494833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74a8d5f6-f260-4b63-97e8-fe41a2f6669a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:16.527803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9e045719-ac97-4e40-86b9-e52458353fe0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:16.528706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9e045719-ac97-4e40-86b9-e52458353fe0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:16.536155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e045719-ac97-4e40-86b9-e52458353fe0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.536641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e045719-ac97-4e40-86b9-e52458353fe0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.537107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e045719-ac97-4e40-86b9-e52458353fe0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:16.537817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e045719-ac97-4e40-86b9-e52458353fe0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.538270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e045719-ac97-4e40-86b9-e52458353fe0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.538694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e045719-ac97-4e40-86b9-e52458353fe0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:16.618683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-74a8d5f6-f260-4b63-97e8-fe41a2f6669a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:16.631799 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-74a8d5f6-f260-4b63-97e8-fe41a2f6669a tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1327 microversion: 2.1 time: 0.149234 Sep 27 10:03:16.635258 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 895/1692] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:16 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1327 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:16.687400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9e045719-ac97-4e40-86b9-e52458353fe0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:16.695543 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9e045719-ac97-4e40-86b9-e52458353fe0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1535 microversion: 2.1 time: 0.173000 Sep 27 10:03:16.696713 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 798/1693] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:16 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1535 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:16.768693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75fd068f-9921-495d-b032-43cb91bb30aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:16.769334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-75fd068f-9921-495d-b032-43cb91bb30aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:16.776397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75fd068f-9921-495d-b032-43cb91bb30aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.776991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75fd068f-9921-495d-b032-43cb91bb30aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.777446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75fd068f-9921-495d-b032-43cb91bb30aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:16.778156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75fd068f-9921-495d-b032-43cb91bb30aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.778787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75fd068f-9921-495d-b032-43cb91bb30aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.779393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75fd068f-9921-495d-b032-43cb91bb30aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:16.982218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0c3b9968-83ac-4410-9257-1fc473533ef2 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:16.982840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0c3b9968-83ac-4410-9257-1fc473533ef2 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:16.994551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c3b9968-83ac-4410-9257-1fc473533ef2 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.995084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c3b9968-83ac-4410-9257-1fc473533ef2 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.995559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c3b9968-83ac-4410-9257-1fc473533ef2 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:16.996294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c3b9968-83ac-4410-9257-1fc473533ef2 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:16.996784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c3b9968-83ac-4410-9257-1fc473533ef2 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:16.997279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c3b9968-83ac-4410-9257-1fc473533ef2 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:17.063177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-75fd068f-9921-495d-b032-43cb91bb30aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:17.075172 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-75fd068f-9921-495d-b032-43cb91bb30aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.309441 Sep 27 10:03:17.077129 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 896/1694] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:16 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:17.279942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c3b9968-83ac-4410-9257-1fc473533ef2 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:17.291631 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0c3b9968-83ac-4410-9257-1fc473533ef2 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1293 microversion: 2.1 time: 0.313976 Sep 27 10:03:17.295398 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 799/1695] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:16 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1293 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:17.334981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed246edb-3296-4a31-862c-b650f665f177 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:17.335963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed246edb-3296-4a31-862c-b650f665f177 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:17.343168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed246edb-3296-4a31-862c-b650f665f177 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:17.343773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed246edb-3296-4a31-862c-b650f665f177 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:17.344319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed246edb-3296-4a31-862c-b650f665f177 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:17.345160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed246edb-3296-4a31-862c-b650f665f177 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:17.345785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed246edb-3296-4a31-862c-b650f665f177 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:17.346479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed246edb-3296-4a31-862c-b650f665f177 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:17.370829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-34412aa8-2626-4626-98fe-6cdb6ea10ccb tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:17.371721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-34412aa8-2626-4626-98fe-6cdb6ea10ccb tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:17.379286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34412aa8-2626-4626-98fe-6cdb6ea10ccb tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:17.379873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34412aa8-2626-4626-98fe-6cdb6ea10ccb tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:17.380319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34412aa8-2626-4626-98fe-6cdb6ea10ccb tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:17.381516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34412aa8-2626-4626-98fe-6cdb6ea10ccb tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:17.392398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34412aa8-2626-4626-98fe-6cdb6ea10ccb tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:17.396416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34412aa8-2626-4626-98fe-6cdb6ea10ccb tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:17.571441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ed246edb-3296-4a31-862c-b650f665f177 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:17.581773 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed246edb-3296-4a31-862c-b650f665f177 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.250529 Sep 27 10:03:17.591990 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 897/1696] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:17 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:17.602338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-84fcb477-c9e5-4655-bc74-dd3c91d784b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63108b80>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:17.695896 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-84fcb477-c9e5-4655-bc74-dd3c91d784b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.111831 Sep 27 10:03:17.729237 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 898/1697] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:17 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:17.737113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-34412aa8-2626-4626-98fe-6cdb6ea10ccb tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:17.746358 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-34412aa8-2626-4626-98fe-6cdb6ea10ccb tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1324 microversion: 2.1 time: 0.380380 Sep 27 10:03:17.749453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:17.752359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:17.764492 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 800/1698] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:17 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1324 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:17.770994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:17.772155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:17.772797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:17.773875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:17.774867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:17.775521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:17.788230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bb2a21ac-423d-47e5-8dcc-c703b10dc895 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:17.789076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bb2a21ac-423d-47e5-8dcc-c703b10dc895 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:17.804958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb2a21ac-423d-47e5-8dcc-c703b10dc895 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:17.805674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb2a21ac-423d-47e5-8dcc-c703b10dc895 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:17.806225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb2a21ac-423d-47e5-8dcc-c703b10dc895 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:17.807047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb2a21ac-423d-47e5-8dcc-c703b10dc895 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:17.811223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb2a21ac-423d-47e5-8dcc-c703b10dc895 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:17.811920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb2a21ac-423d-47e5-8dcc-c703b10dc895 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:17.816801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:17.817516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:17.818133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:17.818760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lazy-loading 'fault' on Instance uuid 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:17.853709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:17.870547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0d563feacff84b4c9927ec759fb365be', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5de10e92e2024a24880c7205697e5732', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:17.880335 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-876e426c-e251-49ad-8d87-2bc2b912088e tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 200 len: 1332 microversion: 2.1 time: 0.149449 Sep 27 10:03:17.896744 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 899/1699] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:17 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 1332 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:17.902375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-4f425acc-af6e-4635-b471-ee55a4bf7738 req-f675e184-78b0-4845-a1b3-3291e4f5629e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "69ea013c-5143-45ff-81a2-3ed97a8659cd", "name": "network-vif-deleted", "tag": "96ac21a6-013a-4a5e-9320-82ff90906b76"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:17.914236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4f425acc-af6e-4635-b471-ee55a4bf7738 req-f675e184-78b0-4845-a1b3-3291e4f5629e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:17.914784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4f425acc-af6e-4635-b471-ee55a4bf7738 req-f675e184-78b0-4845-a1b3-3291e4f5629e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:17.915197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4f425acc-af6e-4635-b471-ee55a4bf7738 req-f675e184-78b0-4845-a1b3-3291e4f5629e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:17.932080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bb2a21ac-423d-47e5-8dcc-c703b10dc895 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:17.934884 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-4f425acc-af6e-4635-b471-ee55a4bf7738 req-f675e184-78b0-4845-a1b3-3291e4f5629e service nova] Creating event network-vif-deleted:96ac21a6-013a-4a5e-9320-82ff90906b76 for instance 69ea013c-5143-45ff-81a2-3ed97a8659cd on nch1-master-1 Sep 27 10:03:17.941296 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-4f425acc-af6e-4635-b471-ee55a4bf7738 req-f675e184-78b0-4845-a1b3-3291e4f5629e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042671 Sep 27 10:03:17.942133 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 900/1700] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:17.946433 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bb2a21ac-423d-47e5-8dcc-c703b10dc895 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1493 microversion: 2.1 time: 0.180206 Sep 27 10:03:17.948706 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 801/1701] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:17 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1493 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:18.104120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-653e5534-42ec-4171-ac64-2b7d806b4a1f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:18.105916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-653e5534-42ec-4171-ac64-2b7d806b4a1f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:18.108906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:18.126435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-653e5534-42ec-4171-ac64-2b7d806b4a1f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:18.130475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-653e5534-42ec-4171-ac64-2b7d806b4a1f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:18.131250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-653e5534-42ec-4171-ac64-2b7d806b4a1f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:18.132497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-653e5534-42ec-4171-ac64-2b7d806b4a1f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:18.133036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-653e5534-42ec-4171-ac64-2b7d806b4a1f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:18.133579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-653e5534-42ec-4171-ac64-2b7d806b4a1f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:18.319683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3ba0d8ba-7c13-4aba-b9cc-1f1862dcca39 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:18.320466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3ba0d8ba-7c13-4aba-b9cc-1f1862dcca39 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:18.329511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ba0d8ba-7c13-4aba-b9cc-1f1862dcca39 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:18.330204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ba0d8ba-7c13-4aba-b9cc-1f1862dcca39 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:18.331096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ba0d8ba-7c13-4aba-b9cc-1f1862dcca39 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:18.332210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ba0d8ba-7c13-4aba-b9cc-1f1862dcca39 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:18.332931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ba0d8ba-7c13-4aba-b9cc-1f1862dcca39 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:18.333959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ba0d8ba-7c13-4aba-b9cc-1f1862dcca39 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:18.477698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-653e5534-42ec-4171-ac64-2b7d806b4a1f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:18.497308 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-653e5534-42ec-4171-ac64-2b7d806b4a1f tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.396879 Sep 27 10:03:18.499080 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 901/1702] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:18 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:18.563862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3ba0d8ba-7c13-4aba-b9cc-1f1862dcca39 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:18.575253 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3ba0d8ba-7c13-4aba-b9cc-1f1862dcca39 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1293 microversion: 2.1 time: 0.259576 Sep 27 10:03:18.576283 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 802/1703] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:18 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1293 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:18.610737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6b1a35e9-e983-4db1-acdf-9c29192229f8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:18.611353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6b1a35e9-e983-4db1-acdf-9c29192229f8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:18.617992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b1a35e9-e983-4db1-acdf-9c29192229f8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:18.618709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b1a35e9-e983-4db1-acdf-9c29192229f8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:18.619535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b1a35e9-e983-4db1-acdf-9c29192229f8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:18.620338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b1a35e9-e983-4db1-acdf-9c29192229f8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:18.620729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b1a35e9-e983-4db1-acdf-9c29192229f8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:18.621161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b1a35e9-e983-4db1-acdf-9c29192229f8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:18.714198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2c57985a-8284-4d8c-9040-211e4f7fc3ce tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62e44dc0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:18.789769 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2c57985a-8284-4d8c-9040-211e4f7fc3ce tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.079664 Sep 27 10:03:18.792401 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 803/1704] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:18 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:18.808452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2185c4f3-6b13-47c5-89bf-74fee0e558b3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:18.809710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2185c4f3-6b13-47c5-89bf-74fee0e558b3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:18.816887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2185c4f3-6b13-47c5-89bf-74fee0e558b3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:18.817959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2185c4f3-6b13-47c5-89bf-74fee0e558b3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:18.818658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2185c4f3-6b13-47c5-89bf-74fee0e558b3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:18.819719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2185c4f3-6b13-47c5-89bf-74fee0e558b3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:18.820352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2185c4f3-6b13-47c5-89bf-74fee0e558b3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:18.821256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2185c4f3-6b13-47c5-89bf-74fee0e558b3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:18.836749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6b1a35e9-e983-4db1-acdf-9c29192229f8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:18.845672 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6b1a35e9-e983-4db1-acdf-9c29192229f8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.239425 Sep 27 10:03:18.846668 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 902/1705] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:18 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:18.907481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:18.907481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:18.915824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:18.916324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:18.916790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:18.918223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:18.918223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:18.918470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:18.941842 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] HTTP exception thrown: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Request method failure captured: Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 HTTP/1.1 Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsmYpLsm53szjmblk2Yvj_AlZMtqf4zi5uq1kDrDyc4WgqVbC8kZ9h87mF5GakXccNV40GTZ0VS6TWLZO7omNXICnwcyIomdT06n7N_YYcv5pJaxiIsIuQJMDojIXF5_BZheo_t3Mb86hZUuBPmCaWAJp5dg7pXr-bVPfQNZFaylLco Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 5de10e92e2024a24880c7205697e5732 Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-884590422 Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:03:18.943711 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5de10e92e2024a24880c7205697e5732"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5de10e92e2024a24880c7205697e5732"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5de10e92e2024a24880c7205697e5732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5de10e92e2024a24880c7205697e5732"}]}] Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-884590422 Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 5de10e92e2024a24880c7205697e5732 Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-884590422 Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachSCSIVolumeTestJSON-884590422-project Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 0d563feacff84b4c9927ec759fb365be Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-884590422-project Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:18.945481 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '0fa79578-f299-4e5d-8dea-8bd1ca1c31d1'} Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:18.946625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:18.948405 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:18.949873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Returning 404 to user: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:18.949873 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a13796e1-820a-494c-8fd1-5909630da6b2 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 404 len: 111 microversion: 2.1 time: 0.045830 Sep 27 10:03:18.950057 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 903/1706] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:18 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:03:18.955329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0c7edc9c-6961-4fd0-957a-42b72c0d5e05 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '.version_select at 0x7f7f6c07f430>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:18.984421 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0c7edc9c-6961-4fd0-957a-42b72c0d5e05 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1899311251" status: 202 len: 0 microversion: 2.1 time: 0.033261 Sep 27 10:03:18.988923 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 904/1707] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:03:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1899311251 => generated 0 bytes in 38 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:03:18.994496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:18.995424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:19.005285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:19.006084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:19.006684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:19.007607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:19.008262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:19.008982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:19.024089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2185c4f3-6b13-47c5-89bf-74fee0e558b3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:19.033078 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2185c4f3-6b13-47c5-89bf-74fee0e558b3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 200 len: 1519 microversion: 2.1 time: 0.234097 Sep 27 10:03:19.050651 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 804/1708] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:18 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 1519 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:19.075600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.1.106"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:19.079036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:19.085645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:19.087315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:19.087958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:19.088465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lazy-loading 'fault' on Instance uuid 69ea013c-5143-45ff-81a2-3ed97a8659cd {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:19.095580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:19.096376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:19.098313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:19.099219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:19.103674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:19.104322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:19.133454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "refresh_cache-3a955d66-3892-4838-ad7d-cc9e431a03e1" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 10:03:19.134155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquired lock "refresh_cache-3a955d66-3892-4838-ad7d-cc9e431a03e1" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 10:03:19.168375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:19.177533 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fbba811f-91d7-484e-95e4-eba15a6a59aa tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 200 len: 1369 microversion: 2.1 time: 0.187109 Sep 27 10:03:19.184899 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 905/1709] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:18 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 1369 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:19.524059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c48d0431-c320-4dc5-bc49-c5646ed76999 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:19.526048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c48d0431-c320-4dc5-bc49-c5646ed76999 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:19.534751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d0431-c320-4dc5-bc49-c5646ed76999 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:19.535562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d0431-c320-4dc5-bc49-c5646ed76999 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:19.536143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d0431-c320-4dc5-bc49-c5646ed76999 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:19.536938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d0431-c320-4dc5-bc49-c5646ed76999 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:19.537486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d0431-c320-4dc5-bc49-c5646ed76999 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:19.537889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d0431-c320-4dc5-bc49-c5646ed76999 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:19.797689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c48d0431-c320-4dc5-bc49-c5646ed76999 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:19.819922 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c48d0431-c320-4dc5-bc49-c5646ed76999 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.300049 Sep 27 10:03:19.821164 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 906/1710] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:19 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:19.827921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d75e11e-6654-49d1-9110-5c69ce127d60 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:19.828841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5d75e11e-6654-49d1-9110-5c69ce127d60 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:19.837306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d75e11e-6654-49d1-9110-5c69ce127d60 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:19.838550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d75e11e-6654-49d1-9110-5c69ce127d60 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:19.840157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d75e11e-6654-49d1-9110-5c69ce127d60 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:19.841093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d75e11e-6654-49d1-9110-5c69ce127d60 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:19.841736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d75e11e-6654-49d1-9110-5c69ce127d60 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:19.842643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d75e11e-6654-49d1-9110-5c69ce127d60 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:20.018995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5d75e11e-6654-49d1-9110-5c69ce127d60 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:20.028100 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5d75e11e-6654-49d1-9110-5c69ce127d60 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1293 microversion: 2.1 time: 0.205443 Sep 27 10:03:20.044567 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 907/1711] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:19 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1293 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:20.055710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2a0561d4-2075-42b1-8387-82dd55178528 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c19aaf0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:20.111655 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2a0561d4-2075-42b1-8387-82dd55178528 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.065296 Sep 27 10:03:20.113550 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 908/1712] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:20 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:20.119437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f2933d2a-fa3d-45a4-a36c-f4b1ad1d2814 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:20.120301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f2933d2a-fa3d-45a4-a36c-f4b1ad1d2814 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:20.131057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2933d2a-fa3d-45a4-a36c-f4b1ad1d2814 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:20.132105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2933d2a-fa3d-45a4-a36c-f4b1ad1d2814 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:20.132549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2933d2a-fa3d-45a4-a36c-f4b1ad1d2814 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:20.134114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2933d2a-fa3d-45a4-a36c-f4b1ad1d2814 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:20.134604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2933d2a-fa3d-45a4-a36c-f4b1ad1d2814 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:20.135495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2933d2a-fa3d-45a4-a36c-f4b1ad1d2814 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:20.302383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f2933d2a-fa3d-45a4-a36c-f4b1ad1d2814 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:20.313957 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f2933d2a-fa3d-45a4-a36c-f4b1ad1d2814 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.199064 Sep 27 10:03:20.326306 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 909/1713] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:20 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:20.326306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:20.327107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 69ea013c-5143-45ff-81a2-3ed97a8659cd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:20.340599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:20.341122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:20.341696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:20.342792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:20.343253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:20.343676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:20.366264 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] HTTP exception thrown: Instance 69ea013c-5143-45ff-81a2-3ed97a8659cd could not be found. Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Request method failure captured: Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd HTTP/1.1 Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMskCLHf4tkqS3pSUi9t60wr9dJGDTGxCi0y5vV2-npAQZy9nMughBkyNmzpYFTIfDMDU6ida_Ku6aBfVqvfMnjWlj1lI-ZQCS98nDVxSoBfWwyvM3HHJuzPC-Gj1jfOzb_4Y8JWfbz7YAFvVvXC5-61s0N6Z0oAPVLuEC5rZAIWkjPo Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e79eae6310354ca58046e59dfac23009 Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:03:20.369786 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e79eae6310354ca58046e59dfac23009"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e79eae6310354ca58046e59dfac23009"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e79eae6310354ca58046e59dfac23009"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e79eae6310354ca58046e59dfac23009"}]}] Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e79eae6310354ca58046e59dfac23009 Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-MigrationsAdminTest-328318727-project Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 9544cdb0392f416197a586d99e8a3210 Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-MigrationsAdminTest-328318727-project Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 69ea013c-5143-45ff-81a2-3ed97a8659cd could not be found. Sep 27 10:03:20.371665 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '69ea013c-5143-45ff-81a2-3ed97a8659cd'} Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 69ea013c-5143-45ff-81a2-3ed97a8659cd could not be found. Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:20.372598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 69ea013c-5143-45ff-81a2-3ed97a8659cd could not be found. Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 69ea013c-5143-45ff-81a2-3ed97a8659cd could not be found. Sep 27 10:03:20.374213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:20.375609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Returning 404 to user: Instance 69ea013c-5143-45ff-81a2-3ed97a8659cd could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:20.375609 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bf31664e-7725-4373-a1a4-8c4d38d17c1a tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd" status: 404 len: 111 microversion: 2.1 time: 0.052030 Sep 27 10:03:20.375609 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 910/1714] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:20 2022] GET /compute/v2.1/servers/69ea013c-5143-45ff-81a2-3ed97a8659cd => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:03:20.381313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67337d3e-7226-4366-9466-174e5b4ad619 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '.version_select at 0x7f7f63108ee0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:20.409075 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67337d3e-7226-4366-9466-174e5b4ad619 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1595116500" status: 202 len: 0 microversion: 2.1 time: 0.032103 Sep 27 10:03:20.410996 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 911/1715] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:03:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1595116500 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:03:20.848320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5cbe5c2-9aba-45a1-94f9-81c7146d3810 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:20.858108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b5cbe5c2-9aba-45a1-94f9-81c7146d3810 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:20.884186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cbe5c2-9aba-45a1-94f9-81c7146d3810 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:20.884920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cbe5c2-9aba-45a1-94f9-81c7146d3810 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:20.886373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cbe5c2-9aba-45a1-94f9-81c7146d3810 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:20.888763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cbe5c2-9aba-45a1-94f9-81c7146d3810 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:20.891743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cbe5c2-9aba-45a1-94f9-81c7146d3810 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:20.892479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cbe5c2-9aba-45a1-94f9-81c7146d3810 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:21.188677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=18 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:21.191293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b5cbe5c2-9aba-45a1-94f9-81c7146d3810 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:21.222837 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b5cbe5c2-9aba-45a1-94f9-81c7146d3810 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.378863 Sep 27 10:03:21.243629 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 912/1716] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:20 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:21.270673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-197d399a-d73d-4384-95b2-3cf9609bb579 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:21.271232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-197d399a-d73d-4384-95b2-3cf9609bb579 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:21.306396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-197d399a-d73d-4384-95b2-3cf9609bb579 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:21.307372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-197d399a-d73d-4384-95b2-3cf9609bb579 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:21.308010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-197d399a-d73d-4384-95b2-3cf9609bb579 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:21.309855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-197d399a-d73d-4384-95b2-3cf9609bb579 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:21.310516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-197d399a-d73d-4384-95b2-3cf9609bb579 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:21.311139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-197d399a-d73d-4384-95b2-3cf9609bb579 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:21.585757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Building network info cache for instance {{(pid=145969) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Sep 27 10:03:21.702825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=22 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:21.771987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-197d399a-d73d-4384-95b2-3cf9609bb579 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:21.782410 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-197d399a-d73d-4384-95b2-3cf9609bb579 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1293 microversion: 2.1 time: 0.537381 Sep 27 10:03:21.783634 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 913/1717] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:21 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1293 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:21.788909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5d52764-c726-46e7-91a2-da672a8a817d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c07faf0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:21.864881 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a5d52764-c726-46e7-91a2-da672a8a817d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.079903 Sep 27 10:03:21.866238 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 914/1718] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:21 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:21.872438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f9f9b4ba-ec74-4014-99d5-6117dca9a69e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:21.873443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f9f9b4ba-ec74-4014-99d5-6117dca9a69e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:21.884238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9f9b4ba-ec74-4014-99d5-6117dca9a69e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:21.885507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9f9b4ba-ec74-4014-99d5-6117dca9a69e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:21.886206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9f9b4ba-ec74-4014-99d5-6117dca9a69e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:21.889207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9f9b4ba-ec74-4014-99d5-6117dca9a69e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:21.890669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9f9b4ba-ec74-4014-99d5-6117dca9a69e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:21.891370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9f9b4ba-ec74-4014-99d5-6117dca9a69e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:21.918039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:22.196616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f9f9b4ba-ec74-4014-99d5-6117dca9a69e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:22.224609 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f9f9b4ba-ec74-4014-99d5-6117dca9a69e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.356817 Sep 27 10:03:22.226304 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 915/1719] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:21 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:22.258894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ab5a2f14-9ad5-492b-96f8-6ea601e3d0f2 req-19587b76-56dc-4419-b178-6c454f448fff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3a955d66-3892-4838-ad7d-cc9e431a03e1", "tag": "b4f78072-6743-4c1d-bb89-c5bb42c81dcf"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:22.281121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ab5a2f14-9ad5-492b-96f8-6ea601e3d0f2 req-19587b76-56dc-4419-b178-6c454f448fff service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:22.282254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ab5a2f14-9ad5-492b-96f8-6ea601e3d0f2 req-19587b76-56dc-4419-b178-6c454f448fff service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:22.283189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ab5a2f14-9ad5-492b-96f8-6ea601e3d0f2 req-19587b76-56dc-4419-b178-6c454f448fff service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:22.298537 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ab5a2f14-9ad5-492b-96f8-6ea601e3d0f2 req-19587b76-56dc-4419-b178-6c454f448fff service nova] Creating event network-changed:b4f78072-6743-4c1d-bb89-c5bb42c81dcf for instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 on nch1-master-1 Sep 27 10:03:22.305686 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ab5a2f14-9ad5-492b-96f8-6ea601e3d0f2 req-19587b76-56dc-4419-b178-6c454f448fff service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056911 Sep 27 10:03:22.309019 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 916/1720] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:22.317521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-85255f8c-3684-4251-9264-23200a848da8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:22.318685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-85255f8c-3684-4251-9264-23200a848da8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:22.327872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=1,nova_api:DELETE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:22.331693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85255f8c-3684-4251-9264-23200a848da8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:22.332517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85255f8c-3684-4251-9264-23200a848da8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:22.333427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85255f8c-3684-4251-9264-23200a848da8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:22.334425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85255f8c-3684-4251-9264-23200a848da8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:22.335361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85255f8c-3684-4251-9264-23200a848da8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:22.336183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85255f8c-3684-4251-9264-23200a848da8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:22.378733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-85255f8c-3684-4251-9264-23200a848da8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:03:22.674155 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-85255f8c-3684-4251-9264-23200a848da8 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "DELETE /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 204 len: 0 microversion: 2.1 time: 0.360424 Sep 27 10:03:22.675352 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 917/1721] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:03:22 2022] DELETE /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:03:22.687272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33f9b3fa-922d-4783-924d-c8336ac77c08 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:22.692169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33f9b3fa-922d-4783-924d-c8336ac77c08 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:22.782731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f9b3fa-922d-4783-924d-c8336ac77c08 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:22.783243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f9b3fa-922d-4783-924d-c8336ac77c08 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:22.802197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f9b3fa-922d-4783-924d-c8336ac77c08 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:22.802197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f9b3fa-922d-4783-924d-c8336ac77c08 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:22.802197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f9b3fa-922d-4783-924d-c8336ac77c08 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:22.802197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33f9b3fa-922d-4783-924d-c8336ac77c08 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:22.938891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Updating instance_info_cache with network_info: [{"id": "b4f78072-6743-4c1d-bb89-c5bb42c81dcf", "address": "fa:16:3e:92:4f:b3", "network": {"id": "79454fbf-4335-40b3-b1c5-c3f69544e188", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-450320224-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.1.106", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "de87d5984a9a4d4481ab4921133841d6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapb4f78072-67", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=145969) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:03:22.959756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Releasing lock "refresh_cache-3a955d66-3892-4838-ad7d-cc9e431a03e1" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 10:03:22.960825 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f94ed476-65af-4ea0-8398-81a18c596881 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "POST /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1/action" status: 202 len: 0 microversion: 2.1 time: 3.897198 Sep 27 10:03:22.961982 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 805/1722] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:03:19 2022] POST /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1/action => generated 0 bytes in 3899 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Sep 27 10:03:22.988304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-974ba35e-b531-45f7-96a3-afd17e738dd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:22.992259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-974ba35e-b531-45f7-96a3-afd17e738dd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:23.018204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ba35e-b531-45f7-96a3-afd17e738dd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:23.020876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ba35e-b531-45f7-96a3-afd17e738dd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:23.021564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ba35e-b531-45f7-96a3-afd17e738dd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:23.022538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ba35e-b531-45f7-96a3-afd17e738dd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:23.026392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ba35e-b531-45f7-96a3-afd17e738dd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:23.027131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ba35e-b531-45f7-96a3-afd17e738dd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:23.099954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33f9b3fa-922d-4783-924d-c8336ac77c08 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:23.113489 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33f9b3fa-922d-4783-924d-c8336ac77c08 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.436713 Sep 27 10:03:23.126472 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 918/1723] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:22 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:23.144966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:23.145493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:23.154544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:23.155296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:23.155899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:23.156864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:23.157490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:23.158078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:23.176945 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] HTTP exception thrown: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Request method failure captured: Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 HTTP/1.1 Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsmYpLsm53szjmblk2Yvj_AlZMtqf4zi5uq1kDrDyc4WgqVbC8kZ9h87mF5GakXccNV40GTZ0VS6TWLZO7omNXICnwcyIomdT06n7N_YYcv5pJaxiIsIuQJMDojIXF5_BZheo_t3Mb86hZUuBPmCaWAJp5dg7pXr-bVPfQNZFaylLco Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 5de10e92e2024a24880c7205697e5732 Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-884590422 Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:03:23.178971 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5de10e92e2024a24880c7205697e5732"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5de10e92e2024a24880c7205697e5732"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5de10e92e2024a24880c7205697e5732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5de10e92e2024a24880c7205697e5732"}]}] Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-884590422 Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 5de10e92e2024a24880c7205697e5732 Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-884590422 Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachSCSIVolumeTestJSON-884590422-project Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 0d563feacff84b4c9927ec759fb365be Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-884590422-project Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:23.180810 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '0fa79578-f299-4e5d-8dea-8bd1ca1c31d1'} Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:23.181877 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:23.183534 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:23.187969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Returning 404 to user: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:23.189643 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d43e4420-aaa9-404d-a0d5-c56f156e1224 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 404 len: 111 microversion: 2.1 time: 0.061470 Sep 27 10:03:23.194170 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 919/1724] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:03:23 2022] DELETE /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:03:23.202799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad407c77-eee7-41ad-a699-47f9e4eda7b8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:23.203687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ad407c77-eee7-41ad-a699-47f9e4eda7b8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:23.211092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad407c77-eee7-41ad-a699-47f9e4eda7b8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:23.211936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad407c77-eee7-41ad-a699-47f9e4eda7b8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:23.212539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad407c77-eee7-41ad-a699-47f9e4eda7b8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:23.213559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad407c77-eee7-41ad-a699-47f9e4eda7b8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:23.215406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad407c77-eee7-41ad-a699-47f9e4eda7b8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:23.216036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad407c77-eee7-41ad-a699-47f9e4eda7b8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:23.237298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-974ba35e-b531-45f7-96a3-afd17e738dd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:23.257160 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-974ba35e-b531-45f7-96a3-afd17e738dd0 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1469 microversion: 2.1 time: 0.285851 Sep 27 10:03:23.263509 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 806/1725] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:22 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1469 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:23.271963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e527cc25-44c3-41e6-8a1a-951e3450219a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62e974c0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:23.353769 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e527cc25-44c3-41e6-8a1a-951e3450219a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.087793 Sep 27 10:03:23.354669 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 807/1726] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:23 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:23.361130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7549bdcb-8303-4f57-8bf7-75a55ff23dbe req-ac1a4ba7-351a-4e0c-adcb-94cb8dc944ec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "11a0d56b-3e1f-40e7-9e3c-30fca67df25e", "name": "network-vif-unplugged", "status": "completed", "tag": "b3618e9e-e34a-4212-aa29-7747a00843a9"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:23.377672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7549bdcb-8303-4f57-8bf7-75a55ff23dbe req-ac1a4ba7-351a-4e0c-adcb-94cb8dc944ec service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:23.378306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7549bdcb-8303-4f57-8bf7-75a55ff23dbe req-ac1a4ba7-351a-4e0c-adcb-94cb8dc944ec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:23.378838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7549bdcb-8303-4f57-8bf7-75a55ff23dbe req-ac1a4ba7-351a-4e0c-adcb-94cb8dc944ec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:23.399943 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7549bdcb-8303-4f57-8bf7-75a55ff23dbe req-ac1a4ba7-351a-4e0c-adcb-94cb8dc944ec service nova] Creating event network-vif-unplugged:b3618e9e-e34a-4212-aa29-7747a00843a9 for instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e on nch1-master-1 Sep 27 10:03:23.408835 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7549bdcb-8303-4f57-8bf7-75a55ff23dbe req-ac1a4ba7-351a-4e0c-adcb-94cb8dc944ec service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.051493 Sep 27 10:03:23.411138 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 808/1727] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:23 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:23.419288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.1.106"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:23.423226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:23.431761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:23.432415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:23.433041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:23.433874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:23.434780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:23.435793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:23.482941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "refresh_cache-9997587c-6839-4459-bd4b-b4d4cd6fd09d" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 10:03:23.483647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquired lock "refresh_cache-9997587c-6839-4459-bd4b-b4d4cd6fd09d" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 10:03:23.936253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ad407c77-eee7-41ad-a699-47f9e4eda7b8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:23.954484 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ad407c77-eee7-41ad-a699-47f9e4eda7b8 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1293 microversion: 2.1 time: 0.757560 Sep 27 10:03:23.956970 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 920/1728] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:23 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1293 bytes in 762 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:23.963572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:23.964676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] [instance: 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:23.984756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:23.986947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:23.988528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:23.990679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:23.992448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:23.993858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:24.030922 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] HTTP exception thrown: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Request method failure captured: Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 HTTP/1.1 Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsmYpLsm53szjmblk2Yvj_AlZMtqf4zi5uq1kDrDyc4WgqVbC8kZ9h87mF5GakXccNV40GTZ0VS6TWLZO7omNXICnwcyIomdT06n7N_YYcv5pJaxiIsIuQJMDojIXF5_BZheo_t3Mb86hZUuBPmCaWAJp5dg7pXr-bVPfQNZFaylLco Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 5de10e92e2024a24880c7205697e5732 Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachSCSIVolumeTestJSON-884590422 Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:03:24.032760 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5de10e92e2024a24880c7205697e5732"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5de10e92e2024a24880c7205697e5732"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5de10e92e2024a24880c7205697e5732"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5de10e92e2024a24880c7205697e5732"}]}] Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachSCSIVolumeTestJSON-884590422 Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 5de10e92e2024a24880c7205697e5732 Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachSCSIVolumeTestJSON-884590422 Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachSCSIVolumeTestJSON-884590422-project Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 0d563feacff84b4c9927ec759fb365be Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachSCSIVolumeTestJSON-884590422-project Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:24.034657 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '0fa79578-f299-4e5d-8dea-8bd1ca1c31d1'} Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:24.035675 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. Sep 27 10:03:24.038560 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:24.042266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] Returning 404 to user: Instance 0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:24.044876 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-81bc00b7-2e1a-49f9-ba94-8ab6c95cede7 tempest-AttachSCSIVolumeTestJSON-884590422 tempest-AttachSCSIVolumeTestJSON-884590422-project] 10.222.0.26 "GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1" status: 404 len: 111 microversion: 2.1 time: 0.085613 Sep 27 10:03:24.046923 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 921/1729] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:23 2022] GET /compute/v2.1/servers/0fa79578-f299-4e5d-8dea-8bd1ca1c31d1 => generated 111 bytes in 88 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:03:24.061227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d93f251e-6bc9-41ec-93f9-4ca026c3ae3e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:24.065075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d93f251e-6bc9-41ec-93f9-4ca026c3ae3e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:24.079897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93f251e-6bc9-41ec-93f9-4ca026c3ae3e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:24.086588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93f251e-6bc9-41ec-93f9-4ca026c3ae3e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:24.087517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93f251e-6bc9-41ec-93f9-4ca026c3ae3e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:24.089690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93f251e-6bc9-41ec-93f9-4ca026c3ae3e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:24.091657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93f251e-6bc9-41ec-93f9-4ca026c3ae3e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:24.092114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93f251e-6bc9-41ec-93f9-4ca026c3ae3e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:24.603080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d93f251e-6bc9-41ec-93f9-4ca026c3ae3e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:24.617564 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d93f251e-6bc9-41ec-93f9-4ca026c3ae3e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.565398 Sep 27 10:03:24.622328 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 922/1730] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:24 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:24.634948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5ebc4f4a-a81c-4a46-8f1b-29899ba96c44 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:24.636803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5ebc4f4a-a81c-4a46-8f1b-29899ba96c44 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:24.645056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ebc4f4a-a81c-4a46-8f1b-29899ba96c44 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:24.645687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ebc4f4a-a81c-4a46-8f1b-29899ba96c44 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:24.651007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ebc4f4a-a81c-4a46-8f1b-29899ba96c44 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:24.652201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ebc4f4a-a81c-4a46-8f1b-29899ba96c44 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:24.652868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ebc4f4a-a81c-4a46-8f1b-29899ba96c44 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:24.653428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ebc4f4a-a81c-4a46-8f1b-29899ba96c44 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:24.941734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5ebc4f4a-a81c-4a46-8f1b-29899ba96c44 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:24.953655 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5ebc4f4a-a81c-4a46-8f1b-29899ba96c44 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.329776 Sep 27 10:03:24.956887 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 923/1731] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:24 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:24.966008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-912deb20-4d17-4af9-a11c-7e1b9f9754a4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:24.974047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-912deb20-4d17-4af9-a11c-7e1b9f9754a4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:24.986124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-912deb20-4d17-4af9-a11c-7e1b9f9754a4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:24.986965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-912deb20-4d17-4af9-a11c-7e1b9f9754a4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:24.988307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-912deb20-4d17-4af9-a11c-7e1b9f9754a4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:24.991801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-912deb20-4d17-4af9-a11c-7e1b9f9754a4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:24.992716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-912deb20-4d17-4af9-a11c-7e1b9f9754a4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:24.993992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-912deb20-4d17-4af9-a11c-7e1b9f9754a4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:25.232844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-912deb20-4d17-4af9-a11c-7e1b9f9754a4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:25.246087 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-912deb20-4d17-4af9-a11c-7e1b9f9754a4 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1469 microversion: 2.1 time: 0.285248 Sep 27 10:03:25.247038 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 924/1732] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:24 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1469 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:25.257082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e81cecdc-50f7-4836-841e-1e397c63557b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f630f9820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:25.328604 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e81cecdc-50f7-4836-841e-1e397c63557b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.079656 Sep 27 10:03:25.329805 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 925/1733] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:25 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:25.338667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dccf4951-d844-4891-8299-5d60dd0c7486 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:25.344013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dccf4951-d844-4891-8299-5d60dd0c7486 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:25.352856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccf4951-d844-4891-8299-5d60dd0c7486 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:25.353367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccf4951-d844-4891-8299-5d60dd0c7486 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:25.353808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccf4951-d844-4891-8299-5d60dd0c7486 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:25.354723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccf4951-d844-4891-8299-5d60dd0c7486 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:25.355426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccf4951-d844-4891-8299-5d60dd0c7486 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:25.355905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccf4951-d844-4891-8299-5d60dd0c7486 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:25.622775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dccf4951-d844-4891-8299-5d60dd0c7486 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:25.638979 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dccf4951-d844-4891-8299-5d60dd0c7486 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1293 microversion: 2.1 time: 0.307871 Sep 27 10:03:25.641848 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 926/1734] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:25 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1293 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:25.652349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-54ec3fe9-d9fd-415d-b742-0a3ffad192ca req-5982a755-db44-4526-8328-85d8ca4db100 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb6339bd-6a0a-497c-8064-cfcd4030b6c2", "name": "network-vif-plugged", "status": "completed", "tag": "4a81581e-1fd6-48e4-b61d-caf266779586"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:25.683820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-54ec3fe9-d9fd-415d-b742-0a3ffad192ca req-5982a755-db44-4526-8328-85d8ca4db100 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:25.689770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-54ec3fe9-d9fd-415d-b742-0a3ffad192ca req-5982a755-db44-4526-8328-85d8ca4db100 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:25.690321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-54ec3fe9-d9fd-415d-b742-0a3ffad192ca req-5982a755-db44-4526-8328-85d8ca4db100 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:25.719971 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-54ec3fe9-d9fd-415d-b742-0a3ffad192ca req-5982a755-db44-4526-8328-85d8ca4db100 service nova] Creating event network-vif-plugged:4a81581e-1fd6-48e4-b61d-caf266779586 for instance cb6339bd-6a0a-497c-8064-cfcd4030b6c2 on nch1-master-1 Sep 27 10:03:25.734066 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-54ec3fe9-d9fd-415d-b742-0a3ffad192ca req-5982a755-db44-4526-8328-85d8ca4db100 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090836 Sep 27 10:03:25.735294 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 927/1735] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:25.747327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b1bb707-c011-478f-8008-db4cec55f4f5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:25.747854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1b1bb707-c011-478f-8008-db4cec55f4f5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:25.766981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b1bb707-c011-478f-8008-db4cec55f4f5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:25.767492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b1bb707-c011-478f-8008-db4cec55f4f5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:25.768522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b1bb707-c011-478f-8008-db4cec55f4f5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:25.769323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b1bb707-c011-478f-8008-db4cec55f4f5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:25.769843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b1bb707-c011-478f-8008-db4cec55f4f5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:25.770227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b1bb707-c011-478f-8008-db4cec55f4f5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:26.074732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1b1bb707-c011-478f-8008-db4cec55f4f5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:26.090254 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1b1bb707-c011-478f-8008-db4cec55f4f5 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.353728 Sep 27 10:03:26.090851 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 928/1736] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:25 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:26.113393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ce9d6df7-f5a0-4adb-8dfd-7155e7fd832a req-672a4b09-2a4e-4542-9823-c0fa57ccc586 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "11a0d56b-3e1f-40e7-9e3c-30fca67df25e", "tag": "b3618e9e-e34a-4212-aa29-7747a00843a9"}, {"server_uuid": "48f61d6b-3985-49a2-9eb9-dd250711cdf4", "name": "network-vif-deleted", "tag": "04d6d3fb-4795-41f4-a54d-61772b8a10a8"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:26.130884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ce9d6df7-f5a0-4adb-8dfd-7155e7fd832a req-672a4b09-2a4e-4542-9823-c0fa57ccc586 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:26.131873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ce9d6df7-f5a0-4adb-8dfd-7155e7fd832a req-672a4b09-2a4e-4542-9823-c0fa57ccc586 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:26.132701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ce9d6df7-f5a0-4adb-8dfd-7155e7fd832a req-672a4b09-2a4e-4542-9823-c0fa57ccc586 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:26.147027 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ce9d6df7-f5a0-4adb-8dfd-7155e7fd832a req-672a4b09-2a4e-4542-9823-c0fa57ccc586 service nova] Creating event network-changed:b3618e9e-e34a-4212-aa29-7747a00843a9 for instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e on nch1-master-1 Sep 27 10:03:26.147548 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ce9d6df7-f5a0-4adb-8dfd-7155e7fd832a req-672a4b09-2a4e-4542-9823-c0fa57ccc586 service nova] Creating event network-vif-deleted:04d6d3fb-4795-41f4-a54d-61772b8a10a8 for instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 on nch1-master-1 Sep 27 10:03:26.154348 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ce9d6df7-f5a0-4adb-8dfd-7155e7fd832a req-672a4b09-2a4e-4542-9823-c0fa57ccc586 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.047364 Sep 27 10:03:26.155306 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 929/1737] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:26 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:26.163594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41769fc4-e186-487c-af3d-397de9c60c5d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:26.164701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-41769fc4-e186-487c-af3d-397de9c60c5d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:26.172224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41769fc4-e186-487c-af3d-397de9c60c5d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:26.173011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41769fc4-e186-487c-af3d-397de9c60c5d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:26.173637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41769fc4-e186-487c-af3d-397de9c60c5d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:26.174550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41769fc4-e186-487c-af3d-397de9c60c5d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:26.175008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41769fc4-e186-487c-af3d-397de9c60c5d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:26.179692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41769fc4-e186-487c-af3d-397de9c60c5d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:26.368346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-41769fc4-e186-487c-af3d-397de9c60c5d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:26.382437 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-41769fc4-e186-487c-af3d-397de9c60c5d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.225751 Sep 27 10:03:26.398351 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 930/1738] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:26 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:26.405872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:26.406891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:26.414416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:26.415762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:26.416423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:26.418016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:26.418707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:26.419351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:26.540779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:26.543529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:26.544291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:26.544997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lazy-loading 'fault' on Instance uuid 48f61d6b-3985-49a2-9eb9-dd250711cdf4 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:26.660285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:26.673010 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f1f99b61-5029-4647-b870-68df75fa0729 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1303 microversion: 2.1 time: 0.273209 Sep 27 10:03:26.674254 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 931/1739] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:26 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1303 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:26.682248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fe9c874b-6db4-488a-b8fc-dd3a180edf0f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f631080d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:26.791079 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fe9c874b-6db4-488a-b8fc-dd3a180edf0f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.112378 Sep 27 10:03:26.799516 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 932/1740] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:26 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:26.801347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8616b9d-180a-4802-a7ee-b6a3033f6ac1 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:26.801975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f8616b9d-180a-4802-a7ee-b6a3033f6ac1 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:26.808799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8616b9d-180a-4802-a7ee-b6a3033f6ac1 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:26.809785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8616b9d-180a-4802-a7ee-b6a3033f6ac1 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:26.810225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8616b9d-180a-4802-a7ee-b6a3033f6ac1 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:26.810948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8616b9d-180a-4802-a7ee-b6a3033f6ac1 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:26.811483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8616b9d-180a-4802-a7ee-b6a3033f6ac1 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:26.812233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8616b9d-180a-4802-a7ee-b6a3033f6ac1 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:27.002602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f8616b9d-180a-4802-a7ee-b6a3033f6ac1 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:27.016830 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f8616b9d-180a-4802-a7ee-b6a3033f6ac1 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1293 microversion: 2.1 time: 0.223833 Sep 27 10:03:27.017895 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 933/1741] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:26 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1293 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:27.116739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62a68de6-7647-4c5d-bf10-b738ecd3284d tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:27.117768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-62a68de6-7647-4c5d-bf10-b738ecd3284d tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:27.130330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a68de6-7647-4c5d-bf10-b738ecd3284d tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:27.131242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a68de6-7647-4c5d-bf10-b738ecd3284d tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:27.132066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a68de6-7647-4c5d-bf10-b738ecd3284d tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:27.133053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a68de6-7647-4c5d-bf10-b738ecd3284d tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:27.134018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a68de6-7647-4c5d-bf10-b738ecd3284d tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:27.137475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62a68de6-7647-4c5d-bf10-b738ecd3284d tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:27.407874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-62a68de6-7647-4c5d-bf10-b738ecd3284d tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:27.416731 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-62a68de6-7647-4c5d-bf10-b738ecd3284d tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.304470 Sep 27 10:03:27.434229 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 934/1742] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:27 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:27.439854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0b225d9b-6639-428a-bffc-5b0fa7783a65 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:27.440807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0b225d9b-6639-428a-bffc-5b0fa7783a65 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:27.447430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b225d9b-6639-428a-bffc-5b0fa7783a65 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:27.448186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b225d9b-6639-428a-bffc-5b0fa7783a65 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:27.448865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b225d9b-6639-428a-bffc-5b0fa7783a65 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:27.449765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b225d9b-6639-428a-bffc-5b0fa7783a65 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:27.450394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b225d9b-6639-428a-bffc-5b0fa7783a65 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:27.450988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b225d9b-6639-428a-bffc-5b0fa7783a65 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:27.505175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:27.734939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0b225d9b-6639-428a-bffc-5b0fa7783a65 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:27.756014 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0b225d9b-6639-428a-bffc-5b0fa7783a65 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.319818 Sep 27 10:03:27.757387 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 935/1743] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:27 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:27.765015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:27.765794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:27.788556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:27.789364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:27.789750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:27.790600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:27.791108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:27.798293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:27.850217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:27.850593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:27.851037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:27.851503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lazy-loading 'fault' on Instance uuid 48f61d6b-3985-49a2-9eb9-dd250711cdf4 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:28.006402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:28.017823 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5fc584e6-6667-476d-a040-614cec1c1efc tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 200 len: 1303 microversion: 2.1 time: 0.259258 Sep 27 10:03:28.033702 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 936/1744] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:27 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 1303 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:28.042004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b44a64de-c669-4041-8703-86d04383cd61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f630c9820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:28.099310 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b44a64de-c669-4041-8703-86d04383cd61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.063869 Sep 27 10:03:28.100378 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 937/1745] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:28 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:28.106058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d71c488f-6c29-4842-b62c-66fa32184e02 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:28.106995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d71c488f-6c29-4842-b62c-66fa32184e02 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:28.114007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d71c488f-6c29-4842-b62c-66fa32184e02 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:28.114877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d71c488f-6c29-4842-b62c-66fa32184e02 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:28.115655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d71c488f-6c29-4842-b62c-66fa32184e02 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:28.116595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d71c488f-6c29-4842-b62c-66fa32184e02 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:28.117286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d71c488f-6c29-4842-b62c-66fa32184e02 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:28.117996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d71c488f-6c29-4842-b62c-66fa32184e02 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:28.328262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d71c488f-6c29-4842-b62c-66fa32184e02 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:28.338372 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d71c488f-6c29-4842-b62c-66fa32184e02 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1484 microversion: 2.1 time: 0.236122 Sep 27 10:03:28.349605 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 938/1746] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:28 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1484 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:28.382291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba6e7d1e-f488-4b43-acae-9447f33ef551 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:28.387298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba6e7d1e-f488-4b43-acae-9447f33ef551 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:28.401110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba6e7d1e-f488-4b43-acae-9447f33ef551 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:28.401850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba6e7d1e-f488-4b43-acae-9447f33ef551 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:28.403110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba6e7d1e-f488-4b43-acae-9447f33ef551 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:28.404836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba6e7d1e-f488-4b43-acae-9447f33ef551 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:28.405704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba6e7d1e-f488-4b43-acae-9447f33ef551 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:28.406346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba6e7d1e-f488-4b43-acae-9447f33ef551 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:28.579189 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba6e7d1e-f488-4b43-acae-9447f33ef551 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.215692 Sep 27 10:03:28.580252 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 939/1747] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:03:28 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2/os-interface => generated 285 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:28.588064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3323d6e4-e364-47b6-96ab-b1153ef7c8f2 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:28.589735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3323d6e4-e364-47b6-96ab-b1153ef7c8f2 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:28.655944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3323d6e4-e364-47b6-96ab-b1153ef7c8f2 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:28.657099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3323d6e4-e364-47b6-96ab-b1153ef7c8f2 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:28.657783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3323d6e4-e364-47b6-96ab-b1153ef7c8f2 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:28.658791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3323d6e4-e364-47b6-96ab-b1153ef7c8f2 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:28.659531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3323d6e4-e364-47b6-96ab-b1153ef7c8f2 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:28.660811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3323d6e4-e364-47b6-96ab-b1153ef7c8f2 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:28.934983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3323d6e4-e364-47b6-96ab-b1153ef7c8f2 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:28.947418 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3323d6e4-e364-47b6-96ab-b1153ef7c8f2 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.364700 Sep 27 10:03:28.948433 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 940/1748] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:28 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:28.954643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-43a5ff38-7d7b-49cf-889c-045ad6806de4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:28.955834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-43a5ff38-7d7b-49cf-889c-045ad6806de4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:28.966132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43a5ff38-7d7b-49cf-889c-045ad6806de4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:28.966706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43a5ff38-7d7b-49cf-889c-045ad6806de4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:28.972197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43a5ff38-7d7b-49cf-889c-045ad6806de4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:28.972197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43a5ff38-7d7b-49cf-889c-045ad6806de4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:28.972197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43a5ff38-7d7b-49cf-889c-045ad6806de4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:28.972197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43a5ff38-7d7b-49cf-889c-045ad6806de4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:29.243595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-43a5ff38-7d7b-49cf-889c-045ad6806de4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:29.265734 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-43a5ff38-7d7b-49cf-889c-045ad6806de4 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.315847 Sep 27 10:03:29.268394 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 941/1749] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:28 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:29.273665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:29.274862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 48f61d6b-3985-49a2-9eb9-dd250711cdf4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:29.281450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:29.282150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:29.282939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:29.283835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:29.284472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:29.285117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:29.304644 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] HTTP exception thrown: Instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 could not be found. Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Request method failure captured: Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 HTTP/1.1 Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMskCLHf4tkqS3pSUi9t60wr9dJGDTGxCi0y5vV2-npAQZy9nMughBkyNmzpYFTIfDMDU6ida_Ku6aBfVqvfMnjWlj1lI-ZQCS98nDVxSoBfWwyvM3HHJuzPC-Gj1jfOzb_4Y8JWfbz7YAFvVvXC5-61s0N6Z0oAPVLuEC5rZAIWkjPo Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e79eae6310354ca58046e59dfac23009 Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:03:29.307267 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e79eae6310354ca58046e59dfac23009"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e79eae6310354ca58046e59dfac23009"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e79eae6310354ca58046e59dfac23009"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e79eae6310354ca58046e59dfac23009"}]}] Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e79eae6310354ca58046e59dfac23009 Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-MigrationsAdminTest-328318727-project Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 9544cdb0392f416197a586d99e8a3210 Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-MigrationsAdminTest-328318727-project Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 could not be found. Sep 27 10:03:29.309181 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '48f61d6b-3985-49a2-9eb9-dd250711cdf4'} Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 could not be found. Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:29.310755 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 could not be found. Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 could not be found. Sep 27 10:03:29.312473 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:29.315523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Returning 404 to user: Instance 48f61d6b-3985-49a2-9eb9-dd250711cdf4 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:29.316707 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c163dd24-4ff1-4ab3-9955-266b37f432bd tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4" status: 404 len: 111 microversion: 2.1 time: 0.047054 Sep 27 10:03:29.317910 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 942/1750] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:29 2022] GET /compute/v2.1/servers/48f61d6b-3985-49a2-9eb9-dd250711cdf4 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:03:29.323690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0613bec-6372-4e21-8f94-9661d88d0faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f630c9160>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:29.385337 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f0613bec-6372-4e21-8f94-9661d88d0faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.065984 Sep 27 10:03:29.386834 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 943/1751] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:29 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:29.392770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-acc70bda-b11e-4b79-93f2-5199a7b3396b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:29.393594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-acc70bda-b11e-4b79-93f2-5199a7b3396b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:29.411933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc70bda-b11e-4b79-93f2-5199a7b3396b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:29.415660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc70bda-b11e-4b79-93f2-5199a7b3396b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:29.417157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc70bda-b11e-4b79-93f2-5199a7b3396b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:29.422018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc70bda-b11e-4b79-93f2-5199a7b3396b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:29.422869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc70bda-b11e-4b79-93f2-5199a7b3396b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:29.424118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acc70bda-b11e-4b79-93f2-5199a7b3396b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:29.482248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-acc70bda-b11e-4b79-93f2-5199a7b3396b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:03:29.618798 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-acc70bda-b11e-4b79-93f2-5199a7b3396b tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "DELETE /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 204 len: 0 microversion: 2.1 time: 0.229099 Sep 27 10:03:29.620064 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 944/1752] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:03:29 2022] DELETE /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:03:29.636133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d65f4f50-6559-4d2c-a8d3-194871f8ea45 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:29.637242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d65f4f50-6559-4d2c-a8d3-194871f8ea45 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:29.652831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d65f4f50-6559-4d2c-a8d3-194871f8ea45 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:29.653333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d65f4f50-6559-4d2c-a8d3-194871f8ea45 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:29.656083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d65f4f50-6559-4d2c-a8d3-194871f8ea45 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:29.656083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d65f4f50-6559-4d2c-a8d3-194871f8ea45 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:29.656363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d65f4f50-6559-4d2c-a8d3-194871f8ea45 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:29.656829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d65f4f50-6559-4d2c-a8d3-194871f8ea45 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:29.956741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d65f4f50-6559-4d2c-a8d3-194871f8ea45 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:29.973099 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d65f4f50-6559-4d2c-a8d3-194871f8ea45 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1469 microversion: 2.1 time: 0.346330 Sep 27 10:03:29.989565 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 945/1753] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:29 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1469 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:29.989565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c3a8122-a480-4268-8ca2-0b896691240f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:29.990197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2c3a8122-a480-4268-8ca2-0b896691240f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:30.005018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3a8122-a480-4268-8ca2-0b896691240f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:30.005785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3a8122-a480-4268-8ca2-0b896691240f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:30.006222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3a8122-a480-4268-8ca2-0b896691240f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:30.006946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3a8122-a480-4268-8ca2-0b896691240f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:30.007401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3a8122-a480-4268-8ca2-0b896691240f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:30.008292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3a8122-a480-4268-8ca2-0b896691240f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:30.205619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2c3a8122-a480-4268-8ca2-0b896691240f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:30.219608 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c3a8122-a480-4268-8ca2-0b896691240f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.238104 Sep 27 10:03:30.220220 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 946/1754] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:29 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:30.291765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f2e1b3b7-b480-4288-bead-3db5ed77184b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:30.292424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f2e1b3b7-b480-4288-bead-3db5ed77184b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:30.298893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2e1b3b7-b480-4288-bead-3db5ed77184b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:30.299521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2e1b3b7-b480-4288-bead-3db5ed77184b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:30.300029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2e1b3b7-b480-4288-bead-3db5ed77184b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:30.300884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2e1b3b7-b480-4288-bead-3db5ed77184b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:30.301407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2e1b3b7-b480-4288-bead-3db5ed77184b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:30.301908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2e1b3b7-b480-4288-bead-3db5ed77184b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:30.492221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f2e1b3b7-b480-4288-bead-3db5ed77184b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:30.505829 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f2e1b3b7-b480-4288-bead-3db5ed77184b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.218689 Sep 27 10:03:30.524850 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 947/1755] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:30 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:30.532852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6c9a515a-fe1b-4f3c-9607-851037c4bef2 req-44bd4b5e-dfe6-4df5-8f2d-b058ee0d7344 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3a955d66-3892-4838-ad7d-cc9e431a03e1", "tag": "b4f78072-6743-4c1d-bb89-c5bb42c81dcf"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:30.546260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6c9a515a-fe1b-4f3c-9607-851037c4bef2 req-44bd4b5e-dfe6-4df5-8f2d-b058ee0d7344 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:30.547022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6c9a515a-fe1b-4f3c-9607-851037c4bef2 req-44bd4b5e-dfe6-4df5-8f2d-b058ee0d7344 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:30.547630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6c9a515a-fe1b-4f3c-9607-851037c4bef2 req-44bd4b5e-dfe6-4df5-8f2d-b058ee0d7344 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:30.561927 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6c9a515a-fe1b-4f3c-9607-851037c4bef2 req-44bd4b5e-dfe6-4df5-8f2d-b058ee0d7344 service nova] Creating event network-changed:b4f78072-6743-4c1d-bb89-c5bb42c81dcf for instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 on nch1-master-1 Sep 27 10:03:30.568346 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6c9a515a-fe1b-4f3c-9607-851037c4bef2 req-44bd4b5e-dfe6-4df5-8f2d-b058ee0d7344 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039905 Sep 27 10:03:30.570322 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 948/1756] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:30.575782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90191719-614d-4462-92b8-86f5baf3e332 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f630eb160>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:30.634680 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.network.neutron [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] re-assign floating IP 172.35.1.106 from instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 Sep 27 10:03:30.637012 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-90191719-614d-4462-92b8-86f5baf3e332 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.065335 Sep 27 10:03:30.638304 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 949/1757] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:30 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:30.697564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Building network info cache for instance {{(pid=145969) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Sep 27 10:03:31.001482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d6dbfddc-eb38-46a9-bb40-1a3a1a6699c9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:31.002775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d6dbfddc-eb38-46a9-bb40-1a3a1a6699c9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:31.013024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6dbfddc-eb38-46a9-bb40-1a3a1a6699c9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:31.013910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6dbfddc-eb38-46a9-bb40-1a3a1a6699c9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:31.015890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6dbfddc-eb38-46a9-bb40-1a3a1a6699c9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:31.017093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6dbfddc-eb38-46a9-bb40-1a3a1a6699c9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:31.018202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6dbfddc-eb38-46a9-bb40-1a3a1a6699c9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:31.018792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6dbfddc-eb38-46a9-bb40-1a3a1a6699c9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:31.392353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d6dbfddc-eb38-46a9-bb40-1a3a1a6699c9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:31.401046 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d6dbfddc-eb38-46a9-bb40-1a3a1a6699c9 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1469 microversion: 2.1 time: 0.406774 Sep 27 10:03:31.417602 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 950/1758] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:30 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1469 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:31.425151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9293abcf-453e-414e-90a7-683330645e35 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:31.426135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9293abcf-453e-414e-90a7-683330645e35 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:31.433583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9293abcf-453e-414e-90a7-683330645e35 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:31.434358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9293abcf-453e-414e-90a7-683330645e35 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:31.435928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9293abcf-453e-414e-90a7-683330645e35 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:31.436869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9293abcf-453e-414e-90a7-683330645e35 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:31.438249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9293abcf-453e-414e-90a7-683330645e35 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:31.438867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9293abcf-453e-414e-90a7-683330645e35 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:32.062656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9293abcf-453e-414e-90a7-683330645e35 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:32.072612 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9293abcf-453e-414e-90a7-683330645e35 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.651826 Sep 27 10:03:32.076254 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 951/1759] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:31 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:32.084485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58ea9a93-2f2a-4ba8-83d6-a33e8c1503c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:32.087687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58ea9a93-2f2a-4ba8-83d6-a33e8c1503c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:32.101110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58ea9a93-2f2a-4ba8-83d6-a33e8c1503c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:32.106547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58ea9a93-2f2a-4ba8-83d6-a33e8c1503c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:32.107237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58ea9a93-2f2a-4ba8-83d6-a33e8c1503c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:32.108173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58ea9a93-2f2a-4ba8-83d6-a33e8c1503c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:32.108870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58ea9a93-2f2a-4ba8-83d6-a33e8c1503c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:32.109493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58ea9a93-2f2a-4ba8-83d6-a33e8c1503c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:32.458271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58ea9a93-2f2a-4ba8-83d6-a33e8c1503c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:32.468327 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58ea9a93-2f2a-4ba8-83d6-a33e8c1503c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.389342 Sep 27 10:03:32.482553 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 952/1760] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:32 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:32.482553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7fde8f7d-9030-4bef-9a0d-7f5b5227115c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f630b0dc0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:32.600894 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7fde8f7d-9030-4bef-9a0d-7f5b5227115c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.130146 Sep 27 10:03:32.601493 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 953/1761] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:32 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:32.631300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5d70d6bf-68f3-48e0-8b11-3c9fb65a84f0 req-0c948f6b-3ba2-43e7-8c0e-9c7ca636637e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "706e3141-43af-4a6b-8789-dea152d0312a", "name": "network-vif-deleted", "tag": "ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:32.674180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5d70d6bf-68f3-48e0-8b11-3c9fb65a84f0 req-0c948f6b-3ba2-43e7-8c0e-9c7ca636637e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:32.676341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5d70d6bf-68f3-48e0-8b11-3c9fb65a84f0 req-0c948f6b-3ba2-43e7-8c0e-9c7ca636637e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:32.677282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5d70d6bf-68f3-48e0-8b11-3c9fb65a84f0 req-0c948f6b-3ba2-43e7-8c0e-9c7ca636637e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:32.714952 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5d70d6bf-68f3-48e0-8b11-3c9fb65a84f0 req-0c948f6b-3ba2-43e7-8c0e-9c7ca636637e service nova] Creating event network-vif-deleted:ca84bbf8-b3ee-49a0-8d87-4fb739e12fc4 for instance 706e3141-43af-4a6b-8789-dea152d0312a on nch2-master-1 Sep 27 10:03:32.735266 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5d70d6bf-68f3-48e0-8b11-3c9fb65a84f0 req-0c948f6b-3ba2-43e7-8c0e-9c7ca636637e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120909 Sep 27 10:03:32.739510 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 954/1762] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:32.753320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:32.758120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:32.789539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:32.794211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:32.795341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:32.801963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:32.806652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:32.810102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:32.836202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Updating instance_info_cache with network_info: [{"id": "b4f78072-6743-4c1d-bb89-c5bb42c81dcf", "address": "fa:16:3e:92:4f:b3", "network": {"id": "79454fbf-4335-40b3-b1c5-c3f69544e188", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-450320224-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "de87d5984a9a4d4481ab4921133841d6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapb4f78072-67", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=145969) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:03:32.852618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:SAVEPOINT=2,nova_cell1:UPDATE=1,nova_cell1:RELEASE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:32.861650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Building network info cache for instance {{(pid=145969) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Sep 27 10:03:32.876175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:32.894726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:32.895726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:32.896464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lazy-loading 'fault' on Instance uuid 706e3141-43af-4a6b-8789-dea152d0312a {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:32.985152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9544cdb0392f416197a586d99e8a3210', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e79eae6310354ca58046e59dfac23009', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:32.993474 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af9f576c-3c3d-4e8b-9b00-7f215b9f6cc7 tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 200 len: 1303 microversion: 2.1 time: 0.251934 Sep 27 10:03:32.994220 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 955/1763] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:32 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 1303 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:33.017639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-03141773-4c8e-47fb-bde7-b0dbfd9036f2 req-7944a1ad-a838-45c6-81cf-d7c7c4f29bc5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9997587c-6839-4459-bd4b-b4d4cd6fd09d", "tag": "c24245da-1f2e-40bc-b00b-6bdc116ec9a4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:33.065476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-03141773-4c8e-47fb-bde7-b0dbfd9036f2 req-7944a1ad-a838-45c6-81cf-d7c7c4f29bc5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:33.069668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-03141773-4c8e-47fb-bde7-b0dbfd9036f2 req-7944a1ad-a838-45c6-81cf-d7c7c4f29bc5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:33.072005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-03141773-4c8e-47fb-bde7-b0dbfd9036f2 req-7944a1ad-a838-45c6-81cf-d7c7c4f29bc5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:33.091160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:33.123109 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-03141773-4c8e-47fb-bde7-b0dbfd9036f2 req-7944a1ad-a838-45c6-81cf-d7c7c4f29bc5 service nova] Creating event network-changed:c24245da-1f2e-40bc-b00b-6bdc116ec9a4 for instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d on nch2-master-1 Sep 27 10:03:33.137492 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-03141773-4c8e-47fb-bde7-b0dbfd9036f2 req-7944a1ad-a838-45c6-81cf-d7c7c4f29bc5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.141215 Sep 27 10:03:33.153915 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 956/1764] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:33.160583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d113a1b4-4464-487f-bd7f-2ef0ba849913 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:33.161573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d113a1b4-4464-487f-bd7f-2ef0ba849913 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:33.168620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d113a1b4-4464-487f-bd7f-2ef0ba849913 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:33.169150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d113a1b4-4464-487f-bd7f-2ef0ba849913 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:33.169595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d113a1b4-4464-487f-bd7f-2ef0ba849913 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:33.170316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d113a1b4-4464-487f-bd7f-2ef0ba849913 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:33.170780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d113a1b4-4464-487f-bd7f-2ef0ba849913 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:33.171208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d113a1b4-4464-487f-bd7f-2ef0ba849913 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:33.431411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=59 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:33.516090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d113a1b4-4464-487f-bd7f-2ef0ba849913 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:33.528109 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d113a1b4-4464-487f-bd7f-2ef0ba849913 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.372643 Sep 27 10:03:33.529143 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 957/1765] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:33 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:33.536136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-76937e67-6ea5-4a65-9dfd-647c44aee136 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:33.537244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-76937e67-6ea5-4a65-9dfd-647c44aee136 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:33.567999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76937e67-6ea5-4a65-9dfd-647c44aee136 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:33.569462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76937e67-6ea5-4a65-9dfd-647c44aee136 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:33.570745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76937e67-6ea5-4a65-9dfd-647c44aee136 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:33.572022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76937e67-6ea5-4a65-9dfd-647c44aee136 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:33.573551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76937e67-6ea5-4a65-9dfd-647c44aee136 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:33.574880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76937e67-6ea5-4a65-9dfd-647c44aee136 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:34.065175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Updating instance_info_cache with network_info: [{"id": "c24245da-1f2e-40bc-b00b-6bdc116ec9a4", "address": "fa:16:3e:94:03:5a", "network": {"id": "79454fbf-4335-40b3-b1c5-c3f69544e188", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-450320224-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.1.106", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "de87d5984a9a4d4481ab4921133841d6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapc24245da-1f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=145969) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:03:34.092486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Releasing lock "refresh_cache-9997587c-6839-4459-bd4b-b4d4cd6fd09d" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 10:03:34.093433 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e96d1beb-97fe-4dee-b6fa-1be15b25642f tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "POST /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d/action" status: 202 len: 0 microversion: 2.1 time: 10.680395 Sep 27 10:03:34.094050 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 809/1766] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:03:23 2022] POST /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d/action => generated 0 bytes in 10682 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Sep 27 10:03:34.105847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0c0d849f-9219-4c7c-afed-2545a456d55c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:34.106475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0c0d849f-9219-4c7c-afed-2545a456d55c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:34.138162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c0d849f-9219-4c7c-afed-2545a456d55c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:34.138661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c0d849f-9219-4c7c-afed-2545a456d55c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:34.139099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c0d849f-9219-4c7c-afed-2545a456d55c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:34.139825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c0d849f-9219-4c7c-afed-2545a456d55c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:34.140287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c0d849f-9219-4c7c-afed-2545a456d55c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:34.141450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c0d849f-9219-4c7c-afed-2545a456d55c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:34.333882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-76937e67-6ea5-4a65-9dfd-647c44aee136 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:34.346825 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-76937e67-6ea5-4a65-9dfd-647c44aee136 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1484 microversion: 2.1 time: 0.816147 Sep 27 10:03:34.350731 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 958/1767] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:33 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1484 bytes in 821 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:34.361756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-674a7d7e-db03-43c7-928b-fe28eb75ce95 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c0cf790>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:34.414675 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-674a7d7e-db03-43c7-928b-fe28eb75ce95 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.058985 Sep 27 10:03:34.427073 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 959/1768] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:34 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:34.432761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:34.433609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] [instance: 706e3141-43af-4a6b-8789-dea152d0312a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:34.442552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:34.443326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:34.443935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:34.444843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:34.445453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:34.446094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:34.448953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c0d849f-9219-4c7c-afed-2545a456d55c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:34.462418 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0c0d849f-9219-4c7c-afed-2545a456d55c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.359666 Sep 27 10:03:34.463111 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 810/1769] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:34 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:34.479035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "172.35.1.106"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:34.483403 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] HTTP exception thrown: Instance 706e3141-43af-4a6b-8789-dea152d0312a could not be found. Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Request method failure captured: Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a HTTP/1.1 Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMskCLHf4tkqS3pSUi9t60wr9dJGDTGxCi0y5vV2-npAQZy9nMughBkyNmzpYFTIfDMDU6ida_Ku6aBfVqvfMnjWlj1lI-ZQCS98nDVxSoBfWwyvM3HHJuzPC-Gj1jfOzb_4Y8JWfbz7YAFvVvXC5-61s0N6Z0oAPVLuEC5rZAIWkjPo Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e79eae6310354ca58046e59dfac23009 Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:03:34.493611 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e79eae6310354ca58046e59dfac23009"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e79eae6310354ca58046e59dfac23009"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e79eae6310354ca58046e59dfac23009"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e79eae6310354ca58046e59dfac23009"}]}] Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e79eae6310354ca58046e59dfac23009 Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-MigrationsAdminTest-328318727 Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-MigrationsAdminTest-328318727-project Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 9544cdb0392f416197a586d99e8a3210 Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-MigrationsAdminTest-328318727-project Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 706e3141-43af-4a6b-8789-dea152d0312a could not be found. Sep 27 10:03:34.495556 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '706e3141-43af-4a6b-8789-dea152d0312a'} Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 706e3141-43af-4a6b-8789-dea152d0312a could not be found. Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:34.496583 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 706e3141-43af-4a6b-8789-dea152d0312a could not be found. Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 706e3141-43af-4a6b-8789-dea152d0312a could not be found. Sep 27 10:03:34.498341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:34.500080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] Returning 404 to user: Instance 706e3141-43af-4a6b-8789-dea152d0312a could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:34.500080 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1090db9e-a721-46b7-89c6-2e44fe03ca1d tempest-MigrationsAdminTest-328318727 tempest-MigrationsAdminTest-328318727-project] 10.222.0.26 "GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a" status: 404 len: 111 microversion: 2.1 time: 0.070786 Sep 27 10:03:34.500080 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 960/1770] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:34 2022] GET /compute/v2.1/servers/706e3141-43af-4a6b-8789-dea152d0312a => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:03:34.551721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d412daea-71b7-4f34-b160-9f02b10e42ea tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:34.552751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d412daea-71b7-4f34-b160-9f02b10e42ea tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:34.566909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d412daea-71b7-4f34-b160-9f02b10e42ea tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:34.570347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d412daea-71b7-4f34-b160-9f02b10e42ea tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:34.570910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d412daea-71b7-4f34-b160-9f02b10e42ea tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:34.577104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d412daea-71b7-4f34-b160-9f02b10e42ea tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:34.580022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d412daea-71b7-4f34-b160-9f02b10e42ea tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:34.581746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d412daea-71b7-4f34-b160-9f02b10e42ea tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:34.734174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:34.753084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:34.754389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:34.755240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:34.758940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:34.762880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:34.764012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:35.338099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d412daea-71b7-4f34-b160-9f02b10e42ea tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:35.347432 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d412daea-71b7-4f34-b160-9f02b10e42ea tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.799576 Sep 27 10:03:35.353334 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 961/1771] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:34 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 806 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:35.358407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7cc48cf6-55d4-4220-96cd-6a72da002e05 req-178f4836-68a8-464c-bffc-ac026859afe8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb6339bd-6a0a-497c-8064-cfcd4030b6c2", "tag": "4a81581e-1fd6-48e4-b61d-caf266779586"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:35.389369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7cc48cf6-55d4-4220-96cd-6a72da002e05 req-178f4836-68a8-464c-bffc-ac026859afe8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:35.390438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7cc48cf6-55d4-4220-96cd-6a72da002e05 req-178f4836-68a8-464c-bffc-ac026859afe8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:35.391074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7cc48cf6-55d4-4220-96cd-6a72da002e05 req-178f4836-68a8-464c-bffc-ac026859afe8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:35.422063 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7cc48cf6-55d4-4220-96cd-6a72da002e05 req-178f4836-68a8-464c-bffc-ac026859afe8 service nova] Creating event network-changed:4a81581e-1fd6-48e4-b61d-caf266779586 for instance cb6339bd-6a0a-497c-8064-cfcd4030b6c2 on nch1-master-1 Sep 27 10:03:35.436691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] HTTP exception thrown: Floating IP 172.35.1.106 is not associated with instance 3a955d66-3892-4838-ad7d-cc9e431a03e1. Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Request method failure captured: Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1/action HTTP/1.1 Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 49 Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMso0VFJjEU5GfoTJmvmRWSj3r_ZSEES5ggmqM4nZlWEK900hPKDrqtEKyMz2MCicCfNf7rpaIQw-u6nipnwKAax2_dGjlwUHftSTX1xbyTI-AkFXK6miTzPu2A2Cr1HWm1A6ftJPMy0819xaVtNSoCE0LZNTLHxKFqqwB_JIN9zzIG4 Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: de87d5984a9a4d4481ab4921133841d6 Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:03:35.438659 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_de87d5984a9a4d4481ab4921133841d6"}]}] Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: de87d5984a9a4d4481ab4921133841d6 Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FloatingIPsAssociationTestJSON-924459803-project Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: c7468cc8bc7d4dde8e2a5eba1fa42fd8 Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-924459803-project Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: {"removeFloatingIp": {"address": "172.35.1.106"}} Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:03:35.440220 ncdv-master-1 devstack@n-api.service[145969]: exception: Floating IP 172.35.1.106 is not associated with instance 3a955d66-3892-4838-ad7d-cc9e431a03e1. Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3a955d66-3892-4838-ad7d-cc9e431a03e1', 'body': {'removeFloatingIp': {'address': '172.35.1.106'}}} Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/floating_ips.py", line 300, in _remove_floating_ip Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Floating IP 172.35.1.106 is not associated with instance 3a955d66-3892-4838-ad7d-cc9e431a03e1. Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:35.441394 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:35.443059 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:35.443059 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:35.443059 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:35.443059 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:35.443059 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Floating IP 172.35.1.106 is not associated with instance 3a955d66-3892-4838-ad7d-cc9e431a03e1. Sep 27 10:03:35.443059 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:35.443059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Returning 409 to user: Floating IP 172.35.1.106 is not associated with instance 3a955d66-3892-4838-ad7d-cc9e431a03e1. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:35.443059 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f887c86e-704a-4304-8857-beff88ce4a35 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "POST /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1/action" status: 409 len: 146 microversion: 2.1 time: 0.968875 Sep 27 10:03:35.443993 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 811/1772] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:03:34 2022] POST /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1/action => generated 146 bytes in 973 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:03:35.444948 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7cc48cf6-55d4-4220-96cd-6a72da002e05 req-178f4836-68a8-464c-bffc-ac026859afe8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084222 Sep 27 10:03:35.448128 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 962/1773] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:35.454243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9829d18f-be2d-4133-8c2e-584b816f988c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c08e550>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:35.456945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "172.35.1.106"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:35.542419 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9829d18f-be2d-4133-8c2e-584b816f988c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.097314 Sep 27 10:03:35.543014 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 812/1774] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:35 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:35.553408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-820d511e-ba96-4c05-9293-c7cf1e62b04e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:35.554075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-820d511e-ba96-4c05-9293-c7cf1e62b04e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:35.576588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-820d511e-ba96-4c05-9293-c7cf1e62b04e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:35.577241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-820d511e-ba96-4c05-9293-c7cf1e62b04e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:35.577712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-820d511e-ba96-4c05-9293-c7cf1e62b04e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:35.578492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-820d511e-ba96-4c05-9293-c7cf1e62b04e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:35.580050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-820d511e-ba96-4c05-9293-c7cf1e62b04e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:35.581156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-820d511e-ba96-4c05-9293-c7cf1e62b04e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:36.019015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:36.051973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:36.053581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:36.055205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:36.064382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:36.066985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:36.069857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:36.489637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-820d511e-ba96-4c05-9293-c7cf1e62b04e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:36.511746 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-820d511e-ba96-4c05-9293-c7cf1e62b04e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1522 microversion: 2.1 time: 0.967281 Sep 27 10:03:36.519891 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 813/1775] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:35 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1522 bytes in 976 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:36.544431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-344f53ef-6cef-44e3-abf8-430d5d13e438 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:36.547972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-344f53ef-6cef-44e3-abf8-430d5d13e438 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:36.566117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-344f53ef-6cef-44e3-abf8-430d5d13e438 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:36.568151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-344f53ef-6cef-44e3-abf8-430d5d13e438 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:36.569468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-344f53ef-6cef-44e3-abf8-430d5d13e438 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:36.570361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-344f53ef-6cef-44e3-abf8-430d5d13e438 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:36.572120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-344f53ef-6cef-44e3-abf8-430d5d13e438 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:36.574069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-344f53ef-6cef-44e3-abf8-430d5d13e438 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:36.829957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-344f53ef-6cef-44e3-abf8-430d5d13e438 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid fcd14928-8ef7-425b-89bb-2c714284e840 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:36.848015 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-344f53ef-6cef-44e3-abf8-430d5d13e438 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 2340 microversion: 2.25 time: 0.325804 Sep 27 10:03:36.862090 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 814/1776] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:03:36 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 2340 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:03:36.873222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-045f02c5-0803-498f-aa01-8889ddd510fe tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:36.873885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-045f02c5-0803-498f-aa01-8889ddd510fe tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:36.879852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045f02c5-0803-498f-aa01-8889ddd510fe tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:36.880398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045f02c5-0803-498f-aa01-8889ddd510fe tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:36.880961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045f02c5-0803-498f-aa01-8889ddd510fe tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:36.881861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045f02c5-0803-498f-aa01-8889ddd510fe tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:36.882483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045f02c5-0803-498f-aa01-8889ddd510fe tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:36.883066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045f02c5-0803-498f-aa01-8889ddd510fe tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:36.971701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "refresh_cache-9997587c-6839-4459-bd4b-b4d4cd6fd09d" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 10:03:36.972351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquired lock "refresh_cache-9997587c-6839-4459-bd4b-b4d4cd6fd09d" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 10:03:37.104845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-045f02c5-0803-498f-aa01-8889ddd510fe tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:37.116110 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-045f02c5-0803-498f-aa01-8889ddd510fe tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.252544 Sep 27 10:03:37.116934 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 815/1777] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:36 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:37.122076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-daa4fd68-9c7b-432d-8943-a887a9437c78 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62e5e040>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:37.181579 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-daa4fd68-9c7b-432d-8943-a887a9437c78 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1412 microversion: 2.1 time: 0.063106 Sep 27 10:03:37.182821 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 816/1778] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:37 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1412 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:37.189067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73fee5cf-9378-488c-98b4-6996fb1b8cb8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:37.193144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73fee5cf-9378-488c-98b4-6996fb1b8cb8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:37.193907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73fee5cf-9378-488c-98b4-6996fb1b8cb8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:37.194571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73fee5cf-9378-488c-98b4-6996fb1b8cb8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:37.195547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73fee5cf-9378-488c-98b4-6996fb1b8cb8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:37.196308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73fee5cf-9378-488c-98b4-6996fb1b8cb8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:37.197065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73fee5cf-9378-488c-98b4-6996fb1b8cb8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:37.242505 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-73fee5cf-9378-488c-98b4-6996fb1b8cb8 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 424 microversion: 2.25 time: 0.058398 Sep 27 10:03:37.243788 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 817/1779] 10.222.0.26 () {60 vars in 1217 bytes} [Tue Sep 27 10:03:37 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 424 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:03:37.546650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dfeebbcb-41a9-49c0-a3ba-72f5867042a6 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:37.548013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dfeebbcb-41a9-49c0-a3ba-72f5867042a6 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:37.556432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfeebbcb-41a9-49c0-a3ba-72f5867042a6 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:37.557338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfeebbcb-41a9-49c0-a3ba-72f5867042a6 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:37.557338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfeebbcb-41a9-49c0-a3ba-72f5867042a6 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:37.558069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfeebbcb-41a9-49c0-a3ba-72f5867042a6 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:37.558509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfeebbcb-41a9-49c0-a3ba-72f5867042a6 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:37.559103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfeebbcb-41a9-49c0-a3ba-72f5867042a6 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:37.732904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dfeebbcb-41a9-49c0-a3ba-72f5867042a6 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:37.741385 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dfeebbcb-41a9-49c0-a3ba-72f5867042a6 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1499 microversion: 2.1 time: 0.199356 Sep 27 10:03:37.741976 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 818/1780] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:37 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1499 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:37.765069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd37638c-f234-4b57-b827-88f1bea23315 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:37.770143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd37638c-f234-4b57-b827-88f1bea23315 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:37.770966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd37638c-f234-4b57-b827-88f1bea23315 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'vm_state': ['stopped'], 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:37.777472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd37638c-f234-4b57-b827-88f1bea23315 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:37.777939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd37638c-f234-4b57-b827-88f1bea23315 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:37.778620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd37638c-f234-4b57-b827-88f1bea23315 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:37.779458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd37638c-f234-4b57-b827-88f1bea23315 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:37.779932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd37638c-f234-4b57-b827-88f1bea23315 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:37.780347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd37638c-f234-4b57-b827-88f1bea23315 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:37.817152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-fd37638c-f234-4b57-b827-88f1bea23315 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:37.850880 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd37638c-f234-4b57-b827-88f1bea23315 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?status=shutoff" status: 200 len: 346 microversion: 2.1 time: 0.089783 Sep 27 10:03:37.852376 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 819/1781] 10.222.0.26 () {58 vars in 1154 bytes} [Tue Sep 27 10:03:37 2022] GET /compute/v2.1/servers?status=shutoff => generated 346 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:03:37.864253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b2bcf5a-c3e3-4e0b-8529-e00f2fa19c51 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:37.871362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b2bcf5a-c3e3-4e0b-8529-e00f2fa19c51 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:37.871979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b2bcf5a-c3e3-4e0b-8529-e00f2fa19c51 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:37.872281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b2bcf5a-c3e3-4e0b-8529-e00f2fa19c51 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:37.898032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3b2bcf5a-c3e3-4e0b-8529-e00f2fa19c51 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Going to try to start instance {{(pid=145969) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 27 10:03:37.947595 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b2bcf5a-c3e3-4e0b-8529-e00f2fa19c51 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "POST /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c/action" status: 202 len: 0 microversion: 2.1 time: 0.087362 Sep 27 10:03:37.951875 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 820/1782] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:03:37 2022] POST /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c/action => generated 0 bytes in 93 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:03:37.959344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e78e09b5-8d8f-4655-9a09-c0b91fe90c83 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:37.960698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e78e09b5-8d8f-4655-9a09-c0b91fe90c83 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:37.968542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e78e09b5-8d8f-4655-9a09-c0b91fe90c83 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:37.969081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e78e09b5-8d8f-4655-9a09-c0b91fe90c83 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:37.969560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e78e09b5-8d8f-4655-9a09-c0b91fe90c83 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:37.970388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e78e09b5-8d8f-4655-9a09-c0b91fe90c83 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:37.970847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e78e09b5-8d8f-4655-9a09-c0b91fe90c83 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:37.971476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e78e09b5-8d8f-4655-9a09-c0b91fe90c83 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:38.202848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e78e09b5-8d8f-4655-9a09-c0b91fe90c83 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:38.211737 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e78e09b5-8d8f-4655-9a09-c0b91fe90c83 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1508 microversion: 2.1 time: 0.258214 Sep 27 10:03:38.212332 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 821/1783] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:37 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1508 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:38.217997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-72876f02-aaa5-4f3e-9c80-b701739cff92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:38.218825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-72876f02-aaa5-4f3e-9c80-b701739cff92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:38.225847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72876f02-aaa5-4f3e-9c80-b701739cff92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:38.226569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72876f02-aaa5-4f3e-9c80-b701739cff92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:38.227421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72876f02-aaa5-4f3e-9c80-b701739cff92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:38.228417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72876f02-aaa5-4f3e-9c80-b701739cff92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:38.228913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72876f02-aaa5-4f3e-9c80-b701739cff92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:38.229377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72876f02-aaa5-4f3e-9c80-b701739cff92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:38.492492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-72876f02-aaa5-4f3e-9c80-b701739cff92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:38.503674 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-72876f02-aaa5-4f3e-9c80-b701739cff92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.290050 Sep 27 10:03:38.504824 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 822/1784] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:38 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:38.511603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f07e2ea-77fe-424f-acf6-669defd2b3bf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62efdd30>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:38.588274 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f07e2ea-77fe-424f-acf6-669defd2b3bf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1421 microversion: 2.1 time: 0.079949 Sep 27 10:03:38.589786 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 823/1785] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:38 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1421 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:38.609215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-18229500-1f4b-430c-a3b1-f3eedbbf6e5d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62eefd30>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:38.681087 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-18229500-1f4b-430c-a3b1-f3eedbbf6e5d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 200 len: 1421 microversion: 2.1 time: 0.078862 Sep 27 10:03:38.683481 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 824/1786] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:03:38 2022] GET /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 1421 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:38.701892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5c92eef3-4fae-4129-bec2-1ab6d692adc7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62efd5e0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:39.300484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Building network info cache for instance {{(pid=145968) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Sep 27 10:03:39.699657 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5c92eef3-4fae-4129-bec2-1ab6d692adc7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 204 len: 0 microversion: 2.1 time: 1.005144 Sep 27 10:03:39.700314 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 825/1787] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:38 2022] DELETE /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 0 bytes in 1007 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:03:39.705867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3791482b-9549-44ab-8cae-118f99dd9b3e", "device": "/dev/xvdb"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:39.710264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:39.725970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:39.729360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:39.730377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:39.732078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:39.732826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:39.733459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:40.202539 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c38b61be-2228-4f45-94c8-26ec33e8c7c9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "POST /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/os-volume_attachments" status: 200 len: 194 microversion: 2.25 time: 0.500010 Sep 27 10:03:40.203866 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 826/1788] 10.222.0.26 () {62 vars in 1364 bytes} [Tue Sep 27 10:03:39 2022] POST /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/os-volume_attachments => generated 194 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:03:40.210610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-169ca2de-a51a-4443-a3a8-015d12f686aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:40.211642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-169ca2de-a51a-4443-a3a8-015d12f686aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:40.221300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-169ca2de-a51a-4443-a3a8-015d12f686aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:40.222270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-169ca2de-a51a-4443-a3a8-015d12f686aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:40.223305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-169ca2de-a51a-4443-a3a8-015d12f686aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:40.224368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-169ca2de-a51a-4443-a3a8-015d12f686aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:40.225033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-169ca2de-a51a-4443-a3a8-015d12f686aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:40.225814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-169ca2de-a51a-4443-a3a8-015d12f686aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:40.463461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-169ca2de-a51a-4443-a3a8-015d12f686aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:40.473955 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-169ca2de-a51a-4443-a3a8-015d12f686aa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1512 microversion: 2.1 time: 0.268567 Sep 27 10:03:40.475583 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 827/1789] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:40 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1512 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:40.505637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-0f1c2294-fd99-44d7-a996-4045b6c8be6a req-af2adb15-3eac-4359-9532-2cd3ea0a3c28 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9997587c-6839-4459-bd4b-b4d4cd6fd09d", "tag": "c24245da-1f2e-40bc-b00b-6bdc116ec9a4"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:40.543238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0f1c2294-fd99-44d7-a996-4045b6c8be6a req-af2adb15-3eac-4359-9532-2cd3ea0a3c28 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:40.543938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0f1c2294-fd99-44d7-a996-4045b6c8be6a req-af2adb15-3eac-4359-9532-2cd3ea0a3c28 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:40.544452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0f1c2294-fd99-44d7-a996-4045b6c8be6a req-af2adb15-3eac-4359-9532-2cd3ea0a3c28 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:40.565181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:RELEASE=2,nova_cell1:SELECT=91,nova_cell1:SAVEPOINT=1,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:40.571040 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-0f1c2294-fd99-44d7-a996-4045b6c8be6a req-af2adb15-3eac-4359-9532-2cd3ea0a3c28 service nova] Creating event network-changed:c24245da-1f2e-40bc-b00b-6bdc116ec9a4 for instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d on nch2-master-1 Sep 27 10:03:40.576813 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-0f1c2294-fd99-44d7-a996-4045b6c8be6a req-af2adb15-3eac-4359-9532-2cd3ea0a3c28 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.099721 Sep 27 10:03:40.578084 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 828/1790] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:40.584358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-83cd7106-640d-4d7d-a853-02e8b72774dd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:40.585370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-83cd7106-640d-4d7d-a853-02e8b72774dd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:40.599101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83cd7106-640d-4d7d-a853-02e8b72774dd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:40.600092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83cd7106-640d-4d7d-a853-02e8b72774dd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:40.601318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83cd7106-640d-4d7d-a853-02e8b72774dd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:40.602479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83cd7106-640d-4d7d-a853-02e8b72774dd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:40.603405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83cd7106-640d-4d7d-a853-02e8b72774dd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:40.604012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83cd7106-640d-4d7d-a853-02e8b72774dd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:40.931161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-83cd7106-640d-4d7d-a853-02e8b72774dd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:40.945624 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-83cd7106-640d-4d7d-a853-02e8b72774dd tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1719 microversion: 2.1 time: 0.365993 Sep 27 10:03:40.946626 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 829/1791] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:40 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1719 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:40.955311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-03b64b07-408d-4816-ace4-d4ab8e4714b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:40.955822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-03b64b07-408d-4816-ace4-d4ab8e4714b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:40.972272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03b64b07-408d-4816-ace4-d4ab8e4714b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:40.977345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03b64b07-408d-4816-ace4-d4ab8e4714b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:40.978680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03b64b07-408d-4816-ace4-d4ab8e4714b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:40.978680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03b64b07-408d-4816-ace4-d4ab8e4714b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:40.979356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03b64b07-408d-4816-ace4-d4ab8e4714b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:40.980082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03b64b07-408d-4816-ace4-d4ab8e4714b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:41.040108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-03b64b07-408d-4816-ace4-d4ab8e4714b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:03:41.186382 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-03b64b07-408d-4816-ace4-d4ab8e4714b0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 204 len: 0 microversion: 2.1 time: 0.234438 Sep 27 10:03:41.188131 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 830/1792] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:03:40 2022] DELETE /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:03:41.207449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-925a67ea-2886-4208-a776-ae39262366fd tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:41.215533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-925a67ea-2886-4208-a776-ae39262366fd tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:41.216264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.servers [None req-925a67ea-2886-4208-a776-ae39262366fd tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Removing options 'limit' from query {{(pid=145969) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1490}} Sep 27 10:03:41.217277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-925a67ea-2886-4208-a776-ae39262366fd tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:41.218633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-925a67ea-2886-4208-a776-ae39262366fd tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:41.219391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-925a67ea-2886-4208-a776-ae39262366fd tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:41.219996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-925a67ea-2886-4208-a776-ae39262366fd tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:41.220967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-925a67ea-2886-4208-a776-ae39262366fd tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:41.221634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-925a67ea-2886-4208-a776-ae39262366fd tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:41.222243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-925a67ea-2886-4208-a776-ae39262366fd tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:41.224275 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-925a67ea-2886-4208-a776-ae39262366fd tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?limit=0" status: 200 len: 15 microversion: 2.1 time: 0.034673 Sep 27 10:03:41.225502 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 831/1793] 10.222.0.26 () {58 vars in 1140 bytes} [Tue Sep 27 10:03:41 2022] GET /compute/v2.1/servers?limit=0 => generated 15 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:03:41.341365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Updating instance_info_cache with network_info: [{"id": "c24245da-1f2e-40bc-b00b-6bdc116ec9a4", "address": "fa:16:3e:94:03:5a", "network": {"id": "79454fbf-4335-40b3-b1c5-c3f69544e188", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-450320224-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "de87d5984a9a4d4481ab4921133841d6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapc24245da-1f", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=145968) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:03:41.368704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Releasing lock "refresh_cache-9997587c-6839-4459-bd4b-b4d4cd6fd09d" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 10:03:41.370196 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9db33191-ad0a-4596-a413-3f82e38683f3 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "POST /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d/action" status: 202 len: 0 microversion: 2.1 time: 5.919418 Sep 27 10:03:41.374715 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 963/1794] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:03:35 2022] POST /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d/action => generated 0 bytes in 5926 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Sep 27 10:03:41.398145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5b76af38-fca9-4c71-a478-428018d9e7cf req-a3dbd136-0195-4a8e-a479-8f09870262b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1", "name": "network-vif-unplugged", "status": "completed", "tag": "cc9dca96-4fbe-402f-b886-7bb414b8545d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:41.420472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5b76af38-fca9-4c71-a478-428018d9e7cf req-a3dbd136-0195-4a8e-a479-8f09870262b1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:41.423153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5b76af38-fca9-4c71-a478-428018d9e7cf req-a3dbd136-0195-4a8e-a479-8f09870262b1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:41.424555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5b76af38-fca9-4c71-a478-428018d9e7cf req-a3dbd136-0195-4a8e-a479-8f09870262b1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:41.452054 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5b76af38-fca9-4c71-a478-428018d9e7cf req-a3dbd136-0195-4a8e-a479-8f09870262b1 service nova] Creating event network-vif-unplugged:cc9dca96-4fbe-402f-b886-7bb414b8545d for instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 on nch2-master-1 Sep 27 10:03:41.463707 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5b76af38-fca9-4c71-a478-428018d9e7cf req-a3dbd136-0195-4a8e-a479-8f09870262b1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.085379 Sep 27 10:03:41.465383 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 964/1795] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:41 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:41.473020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-247ee15a-ae1e-40d0-9f4f-5c0865ce69b8 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:41.474297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-247ee15a-ae1e-40d0-9f4f-5c0865ce69b8 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:41.487816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:41.494997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-247ee15a-ae1e-40d0-9f4f-5c0865ce69b8 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:41.497590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-247ee15a-ae1e-40d0-9f4f-5c0865ce69b8 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:41.498434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-247ee15a-ae1e-40d0-9f4f-5c0865ce69b8 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:41.500546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-247ee15a-ae1e-40d0-9f4f-5c0865ce69b8 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:41.501287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-247ee15a-ae1e-40d0-9f4f-5c0865ce69b8 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:41.502439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-247ee15a-ae1e-40d0-9f4f-5c0865ce69b8 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:41.537256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TenantUsagesTestJSON-server-1858225271", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:41.634404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:03:41.707888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-247ee15a-ae1e-40d0-9f4f-5c0865ce69b8 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:41.717435 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-247ee15a-ae1e-40d0-9f4f-5c0865ce69b8 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1512 microversion: 2.1 time: 0.249215 Sep 27 10:03:41.733854 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 965/1796] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:41 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1512 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:41.741329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eea479ec-fbef-477e-8445-29ace046a47c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:41.742189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eea479ec-fbef-477e-8445-29ace046a47c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:41.750678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eea479ec-fbef-477e-8445-29ace046a47c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:41.751464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eea479ec-fbef-477e-8445-29ace046a47c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:41.752136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eea479ec-fbef-477e-8445-29ace046a47c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:41.753536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eea479ec-fbef-477e-8445-29ace046a47c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:41.754519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eea479ec-fbef-477e-8445-29ace046a47c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:41.755208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eea479ec-fbef-477e-8445-29ace046a47c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:41.787110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eea479ec-fbef-477e-8445-29ace046a47c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:03:41.890859 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eea479ec-fbef-477e-8445-29ace046a47c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 204 len: 0 microversion: 2.1 time: 0.154319 Sep 27 10:03:41.893025 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 966/1797] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:03:41 2022] DELETE /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:03:41.904826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-474945064", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "4568aca4-c33f-4379-b9d1-22f70d115851"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:41.969383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] validate_networks() for [('4568aca4-c33f-4379-b9d1-22f70d115851', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:03:42.047805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:03:42.048392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:03:42.048726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:03:42.049139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:03:42.049551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:03:42.064016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.064499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.065168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:42.066078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.066692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.067120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:42.087251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Getting quotas for project 8b87f09e1bf645a4b71d8b31ee512ec0. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:03:42.092853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Getting quotas for user b7892230632b490c8f110206461a87c6 and project 8b87f09e1bf645a4b71d8b31ee512ec0. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:03:42.106822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:03:42.109676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:03:42.179035 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ef825c53-5608-46e2-a88f-8fbe6a020005 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.951918 Sep 27 10:03:42.180523 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 832/1798] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:03:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 954 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:03:42.197381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4ef061d7-0e98-42ea-be41-a67a48daa73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:42.198321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4ef061d7-0e98-42ea-be41-a67a48daa73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:42.207420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ef061d7-0e98-42ea-be41-a67a48daa73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.208497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ef061d7-0e98-42ea-be41-a67a48daa73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.209184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ef061d7-0e98-42ea-be41-a67a48daa73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:42.210119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ef061d7-0e98-42ea-be41-a67a48daa73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.210732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ef061d7-0e98-42ea-be41-a67a48daa73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.213654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ef061d7-0e98-42ea-be41-a67a48daa73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:42.432573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4ef061d7-0e98-42ea-be41-a67a48daa73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:42.446863 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4ef061d7-0e98-42ea-be41-a67a48daa73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 200 len: 1512 microversion: 2.1 time: 0.264554 Sep 27 10:03:42.448859 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 833/1799] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:42 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 1512 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:42.454847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.1.106"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:42.458206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:42.466821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.467895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.468621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:42.469674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.470242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.471093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:42.507729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:03:42.508297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:03:42.509421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:03:42.509955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:03:42.510305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:03:42.531939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "refresh_cache-3a955d66-3892-4838-ad7d-cc9e431a03e1" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 10:03:42.532422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquired lock "refresh_cache-3a955d66-3892-4838-ad7d-cc9e431a03e1" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 10:03:42.535439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.536006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.536454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:42.539700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.540194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.549260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:42.577210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Getting quotas for project e5a565f65b3849a99b74454791397ca5. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:03:42.582366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Getting quotas for user 6eb3ccfcd9c344e1a7ad55f8398ec87a and project e5a565f65b3849a99b74454791397ca5. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:03:42.596643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:03:42.599070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:03:42.662098 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-01ed0682-400e-4609-bf3f-f07a7017a89d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.767515 Sep 27 10:03:42.663184 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 967/1800] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:03:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 769 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:03:42.674222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ac7f3603-af61-4429-8b08-724c8e04068c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:42.675062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ac7f3603-af61-4429-8b08-724c8e04068c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:42.683094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac7f3603-af61-4429-8b08-724c8e04068c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.683955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac7f3603-af61-4429-8b08-724c8e04068c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.684607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac7f3603-af61-4429-8b08-724c8e04068c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:42.686246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac7f3603-af61-4429-8b08-724c8e04068c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.686902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac7f3603-af61-4429-8b08-724c8e04068c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.688160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac7f3603-af61-4429-8b08-724c8e04068c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:42.904625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ac7f3603-af61-4429-8b08-724c8e04068c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:42.918281 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ac7f3603-af61-4429-8b08-724c8e04068c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1709 microversion: 2.1 time: 0.252619 Sep 27 10:03:42.919335 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 968/1801] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:42 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1709 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:42.930497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3905489-bd21-4e5c-a98e-aa957c8e96d9 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:42.930497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d3905489-bd21-4e5c-a98e-aa957c8e96d9 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:42.935840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3905489-bd21-4e5c-a98e-aa957c8e96d9 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.936340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3905489-bd21-4e5c-a98e-aa957c8e96d9 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.936864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3905489-bd21-4e5c-a98e-aa957c8e96d9 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:42.938234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3905489-bd21-4e5c-a98e-aa957c8e96d9 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:42.938690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3905489-bd21-4e5c-a98e-aa957c8e96d9 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:42.939162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3905489-bd21-4e5c-a98e-aa957c8e96d9 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:43.018457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d3905489-bd21-4e5c-a98e-aa957c8e96d9 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:43.028254 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d3905489-bd21-4e5c-a98e-aa957c8e96d9 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1218 microversion: 2.1 time: 0.107454 Sep 27 10:03:43.029436 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 969/1802] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:42 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1218 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:43.034731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-81403972-de86-4994-91b8-84d50c2236c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:43.035341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-81403972-de86-4994-91b8-84d50c2236c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:43.042266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81403972-de86-4994-91b8-84d50c2236c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:43.043160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81403972-de86-4994-91b8-84d50c2236c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:43.043498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81403972-de86-4994-91b8-84d50c2236c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:43.044201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81403972-de86-4994-91b8-84d50c2236c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:43.044951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81403972-de86-4994-91b8-84d50c2236c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:43.045532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81403972-de86-4994-91b8-84d50c2236c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:43.338136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-81403972-de86-4994-91b8-84d50c2236c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:43.346984 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-81403972-de86-4994-91b8-84d50c2236c9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1511 microversion: 2.1 time: 0.316169 Sep 27 10:03:43.347572 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 970/1803] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:43 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1511 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:43.353475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-68948118-72d2-45fe-9bf7-a7c748bb05e7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:43.354094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-68948118-72d2-45fe-9bf7-a7c748bb05e7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:43.360274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68948118-72d2-45fe-9bf7-a7c748bb05e7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:43.361113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68948118-72d2-45fe-9bf7-a7c748bb05e7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:43.367358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68948118-72d2-45fe-9bf7-a7c748bb05e7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:43.368523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68948118-72d2-45fe-9bf7-a7c748bb05e7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:43.371878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68948118-72d2-45fe-9bf7-a7c748bb05e7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:43.372335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68948118-72d2-45fe-9bf7-a7c748bb05e7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:43.460394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-68948118-72d2-45fe-9bf7-a7c748bb05e7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:43.468954 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-68948118-72d2-45fe-9bf7-a7c748bb05e7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1211 microversion: 2.1 time: 0.120059 Sep 27 10:03:43.469860 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 971/1804] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:43 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1211 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:43.474588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c999f644-466b-4f3f-8880-49fe33ad651c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '.version_select at 0x7f7f6308b550>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:43.479222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c999f644-466b-4f3f-8880-49fe33ad651c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:43.479966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c999f644-466b-4f3f-8880-49fe33ad651c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:43.480589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c999f644-466b-4f3f-8880-49fe33ad651c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:43.496444 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c999f644-466b-4f3f-8880-49fe33ad651c tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/os-migrations" status: 200 len: 1934 microversion: 2.1 time: 0.025313 Sep 27 10:03:43.497121 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 972/1805] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 10:03:43 2022] GET /compute/v2.1/os-migrations => generated 1934 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:43.505956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-46df39fc-d914-482a-9028-1b1f6d49294e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:43.515700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-46df39fc-d914-482a-9028-1b1f6d49294e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:43.517268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-46df39fc-d914-482a-9028-1b1f6d49294e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'ip': '172.20.0.14', 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:43.733460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46df39fc-d914-482a-9028-1b1f6d49294e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:43.733978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46df39fc-d914-482a-9028-1b1f6d49294e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:43.734527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46df39fc-d914-482a-9028-1b1f6d49294e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:43.736000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46df39fc-d914-482a-9028-1b1f6d49294e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:43.741463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46df39fc-d914-482a-9028-1b1f6d49294e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:43.742118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46df39fc-d914-482a-9028-1b1f6d49294e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:43.854288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-46df39fc-d914-482a-9028-1b1f6d49294e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:43.915107 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-46df39fc-d914-482a-9028-1b1f6d49294e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?ip=172.20.0.14" status: 200 len: 346 microversion: 2.1 time: 0.416830 Sep 27 10:03:43.915710 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 973/1806] 10.222.0.26 () {58 vars in 1154 bytes} [Tue Sep 27 10:03:43 2022] GET /compute/v2.1/servers?ip=172.20.0.14 => generated 346 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:43.938200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8b70ccff-7ca5-495b-bb8f-57103e8872ce tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:43.938849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8b70ccff-7ca5-495b-bb8f-57103e8872ce tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:43.957848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b70ccff-7ca5-495b-bb8f-57103e8872ce tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:43.958677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b70ccff-7ca5-495b-bb8f-57103e8872ce tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:43.960157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b70ccff-7ca5-495b-bb8f-57103e8872ce tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:43.961453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b70ccff-7ca5-495b-bb8f-57103e8872ce tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:43.962208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b70ccff-7ca5-495b-bb8f-57103e8872ce tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:43.962808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b70ccff-7ca5-495b-bb8f-57103e8872ce tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:44.113893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Building network info cache for instance {{(pid=145969) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Sep 27 10:03:44.276913 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8b70ccff-7ca5-495b-bb8f-57103e8872ce tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1860 microversion: 2.1 time: 0.350521 Sep 27 10:03:44.278109 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 974/1807] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:43 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1860 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:44.285551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e8c65571-e7d8-4f15-bdb2-d94bf08d26b3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:44.286464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e8c65571-e7d8-4f15-bdb2-d94bf08d26b3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:44.311787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8c65571-e7d8-4f15-bdb2-d94bf08d26b3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:44.315119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8c65571-e7d8-4f15-bdb2-d94bf08d26b3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:44.315685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8c65571-e7d8-4f15-bdb2-d94bf08d26b3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:44.317437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8c65571-e7d8-4f15-bdb2-d94bf08d26b3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:44.318856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8c65571-e7d8-4f15-bdb2-d94bf08d26b3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:44.319455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8c65571-e7d8-4f15-bdb2-d94bf08d26b3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:44.336472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:SAVEPOINT=1,nova_cell1:UPDATE=2,nova_cell1:RELEASE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:44.784142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e8c65571-e7d8-4f15-bdb2-d94bf08d26b3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:44.803598 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e8c65571-e7d8-4f15-bdb2-d94bf08d26b3 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1512 microversion: 2.1 time: 0.524032 Sep 27 10:03:44.807175 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 975/1808] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:44 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1512 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:44.820211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90f14d3f-95f5-4806-9c6f-b0d88dcbce80 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:44.821935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-90f14d3f-95f5-4806-9c6f-b0d88dcbce80 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:44.830701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90f14d3f-95f5-4806-9c6f-b0d88dcbce80 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:44.831974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90f14d3f-95f5-4806-9c6f-b0d88dcbce80 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:44.834667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90f14d3f-95f5-4806-9c6f-b0d88dcbce80 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:44.838254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90f14d3f-95f5-4806-9c6f-b0d88dcbce80 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:44.840035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90f14d3f-95f5-4806-9c6f-b0d88dcbce80 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:44.841362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90f14d3f-95f5-4806-9c6f-b0d88dcbce80 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:44.953338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-90f14d3f-95f5-4806-9c6f-b0d88dcbce80 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:44.971287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-90f14d3f-95f5-4806-9c6f-b0d88dcbce80 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1266 microversion: 2.1 time: 0.159784 Sep 27 10:03:44.973218 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 976/1809] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:44 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:44.983805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-95138c8b-c705-42af-a953-981203608899 req-6e511862-1ad8-4549-ac1d-425c6db44ba9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3a955d66-3892-4838-ad7d-cc9e431a03e1", "tag": "b4f78072-6743-4c1d-bb89-c5bb42c81dcf"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:44.995719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-95138c8b-c705-42af-a953-981203608899 req-6e511862-1ad8-4549-ac1d-425c6db44ba9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:44.997663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-95138c8b-c705-42af-a953-981203608899 req-6e511862-1ad8-4549-ac1d-425c6db44ba9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:44.998440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-95138c8b-c705-42af-a953-981203608899 req-6e511862-1ad8-4549-ac1d-425c6db44ba9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:45.018666 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-95138c8b-c705-42af-a953-981203608899 req-6e511862-1ad8-4549-ac1d-425c6db44ba9 service nova] Creating event network-changed:b4f78072-6743-4c1d-bb89-c5bb42c81dcf for instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 on nch1-master-1 Sep 27 10:03:45.031521 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-95138c8b-c705-42af-a953-981203608899 req-6e511862-1ad8-4549-ac1d-425c6db44ba9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053622 Sep 27 10:03:45.032620 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 977/1810] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:45.039184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e5247824-1f7d-4def-9ee8-f8d9041e73f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:45.040828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e5247824-1f7d-4def-9ee8-f8d9041e73f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:45.050568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5247824-1f7d-4def-9ee8-f8d9041e73f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:45.051071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5247824-1f7d-4def-9ee8-f8d9041e73f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:45.051638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5247824-1f7d-4def-9ee8-f8d9041e73f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:45.052445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5247824-1f7d-4def-9ee8-f8d9041e73f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:45.053405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5247824-1f7d-4def-9ee8-f8d9041e73f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:45.053557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5247824-1f7d-4def-9ee8-f8d9041e73f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:45.393208 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e5247824-1f7d-4def-9ee8-f8d9041e73f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1860 microversion: 2.1 time: 0.359280 Sep 27 10:03:45.393802 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 978/1811] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:45 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1860 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:45.406364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ccfe9f13-2a1b-47cc-a125-78682395d383 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:45.406986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ccfe9f13-2a1b-47cc-a125-78682395d383 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:45.419328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe9f13-2a1b-47cc-a125-78682395d383 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:45.419788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe9f13-2a1b-47cc-a125-78682395d383 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:45.420230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe9f13-2a1b-47cc-a125-78682395d383 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:45.421008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe9f13-2a1b-47cc-a125-78682395d383 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:45.421467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe9f13-2a1b-47cc-a125-78682395d383 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:45.421896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe9f13-2a1b-47cc-a125-78682395d383 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:45.660639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ccfe9f13-2a1b-47cc-a125-78682395d383 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:45.673928 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ccfe9f13-2a1b-47cc-a125-78682395d383 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1203 microversion: 2.1 time: 0.274855 Sep 27 10:03:45.675640 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 979/1812] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:45 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1203 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:45.680954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bed7e215-1f31-4eeb-a6db-6b8c2d8c0042 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:45.690934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bed7e215-1f31-4eeb-a6db-6b8c2d8c0042 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:45.706450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bed7e215-1f31-4eeb-a6db-6b8c2d8c0042 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:45.707248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bed7e215-1f31-4eeb-a6db-6b8c2d8c0042 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:45.708090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bed7e215-1f31-4eeb-a6db-6b8c2d8c0042 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:45.708483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bed7e215-1f31-4eeb-a6db-6b8c2d8c0042 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:45.710677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bed7e215-1f31-4eeb-a6db-6b8c2d8c0042 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:45.712866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bed7e215-1f31-4eeb-a6db-6b8c2d8c0042 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:45.979422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bed7e215-1f31-4eeb-a6db-6b8c2d8c0042 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:45.991171 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bed7e215-1f31-4eeb-a6db-6b8c2d8c0042 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 200 len: 1512 microversion: 2.1 time: 0.312688 Sep 27 10:03:46.003233 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 980/1813] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:45 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 1512 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:46.019980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7659be88-4874-4490-ad0d-1a5f65acbfa9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:46.021285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7659be88-4874-4490-ad0d-1a5f65acbfa9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:46.048050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7659be88-4874-4490-ad0d-1a5f65acbfa9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:46.048797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7659be88-4874-4490-ad0d-1a5f65acbfa9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:46.049259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7659be88-4874-4490-ad0d-1a5f65acbfa9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:46.050203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7659be88-4874-4490-ad0d-1a5f65acbfa9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:46.050828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7659be88-4874-4490-ad0d-1a5f65acbfa9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:46.051469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7659be88-4874-4490-ad0d-1a5f65acbfa9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:46.214464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-7659be88-4874-4490-ad0d-1a5f65acbfa9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lazy-loading 'services' on Instance uuid 11a0d56b-3e1f-40e7-9e3c-30fca67df25e {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:46.332352 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7659be88-4874-4490-ad0d-1a5f65acbfa9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "DELETE /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/os-volume_attachments/ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7" status: 202 len: 0 microversion: 2.1 time: 0.326117 Sep 27 10:03:46.339093 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 981/1814] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:03:46 2022] DELETE /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/os-volume_attachments/ad5d0d8f-1bf9-4511-b11c-4c08c59b55b7 => generated 0 bytes in 335 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:03:46.350746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d165c21-e6a7-42cd-ac08-a100bdf2a56c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:46.352967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d165c21-e6a7-42cd-ac08-a100bdf2a56c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:46.371775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d165c21-e6a7-42cd-ac08-a100bdf2a56c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:46.372801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d165c21-e6a7-42cd-ac08-a100bdf2a56c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:46.373556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d165c21-e6a7-42cd-ac08-a100bdf2a56c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:46.374589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d165c21-e6a7-42cd-ac08-a100bdf2a56c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:46.375279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d165c21-e6a7-42cd-ac08-a100bdf2a56c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:46.375906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d165c21-e6a7-42cd-ac08-a100bdf2a56c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:46.470162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7d165c21-e6a7-42cd-ac08-a100bdf2a56c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:46.488055 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d165c21-e6a7-42cd-ac08-a100bdf2a56c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1274 microversion: 2.1 time: 0.146564 Sep 27 10:03:46.491500 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 982/1815] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:46 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1274 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:46.506745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-98d2d839-d979-4a3c-872c-9a97421570fa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:46.509568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-98d2d839-d979-4a3c-872c-9a97421570fa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:46.523815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d2d839-d979-4a3c-872c-9a97421570fa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:46.525389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d2d839-d979-4a3c-872c-9a97421570fa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:46.526164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d2d839-d979-4a3c-872c-9a97421570fa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:46.532284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d2d839-d979-4a3c-872c-9a97421570fa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:46.534007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d2d839-d979-4a3c-872c-9a97421570fa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:46.535077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d2d839-d979-4a3c-872c-9a97421570fa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:46.740310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-98d2d839-d979-4a3c-872c-9a97421570fa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:46.751462 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-98d2d839-d979-4a3c-872c-9a97421570fa tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1511 microversion: 2.1 time: 0.258576 Sep 27 10:03:46.752823 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 983/1816] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:46 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1511 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:46.758979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3d9c9e8-e6e6-49d9-b933-8f415a085048 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:46.759603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e3d9c9e8-e6e6-49d9-b933-8f415a085048 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:46.766956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3d9c9e8-e6e6-49d9-b933-8f415a085048 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:46.767691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3d9c9e8-e6e6-49d9-b933-8f415a085048 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:46.768823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3d9c9e8-e6e6-49d9-b933-8f415a085048 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:46.770048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3d9c9e8-e6e6-49d9-b933-8f415a085048 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:46.770944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3d9c9e8-e6e6-49d9-b933-8f415a085048 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:46.771798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3d9c9e8-e6e6-49d9-b933-8f415a085048 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:46.973475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-e3d9c9e8-e6e6-49d9-b933-8f415a085048 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid fcd14928-8ef7-425b-89bb-2c714284e840 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:46.999006 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e3d9c9e8-e6e6-49d9-b933-8f415a085048 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 2418 microversion: 2.25 time: 0.244805 Sep 27 10:03:47.005216 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 984/1817] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:03:46 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 2418 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:03:47.012044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92f356dc-85d2-4a6a-99ae-9fb9cfd4c819 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:47.015297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-92f356dc-85d2-4a6a-99ae-9fb9cfd4c819 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:47.022686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Updating instance_info_cache with network_info: [{"id": "b4f78072-6743-4c1d-bb89-c5bb42c81dcf", "address": "fa:16:3e:92:4f:b3", "network": {"id": "79454fbf-4335-40b3-b1c5-c3f69544e188", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-450320224-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.1.106", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "de87d5984a9a4d4481ab4921133841d6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapb4f78072-67", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=145969) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:03:47.028813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92f356dc-85d2-4a6a-99ae-9fb9cfd4c819 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:47.029483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92f356dc-85d2-4a6a-99ae-9fb9cfd4c819 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:47.033697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92f356dc-85d2-4a6a-99ae-9fb9cfd4c819 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:47.036329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92f356dc-85d2-4a6a-99ae-9fb9cfd4c819 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:47.038077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92f356dc-85d2-4a6a-99ae-9fb9cfd4c819 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:47.038968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92f356dc-85d2-4a6a-99ae-9fb9cfd4c819 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:47.050918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Releasing lock "refresh_cache-3a955d66-3892-4838-ad7d-cc9e431a03e1" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 10:03:47.052301 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5bb5ff4-e81a-4486-912e-3157d15a7960 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "POST /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1/action" status: 202 len: 0 microversion: 2.1 time: 4.602244 Sep 27 10:03:47.053574 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 834/1818] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:03:42 2022] POST /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1/action => generated 0 bytes in 4604 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Sep 27 10:03:47.076565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aec95fbe-3d9e-473e-b5c9-d58ab2e8a2b1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:47.078340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aec95fbe-3d9e-473e-b5c9-d58ab2e8a2b1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:47.091493 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-92f356dc-85d2-4a6a-99ae-9fb9cfd4c819 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.084534 Sep 27 10:03:47.095412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aec95fbe-3d9e-473e-b5c9-d58ab2e8a2b1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:47.096693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aec95fbe-3d9e-473e-b5c9-d58ab2e8a2b1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:47.097482 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 985/1819] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:03:47 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:47.097766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aec95fbe-3d9e-473e-b5c9-d58ab2e8a2b1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:47.099640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aec95fbe-3d9e-473e-b5c9-d58ab2e8a2b1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:47.100478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aec95fbe-3d9e-473e-b5c9-d58ab2e8a2b1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:47.101564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aec95fbe-3d9e-473e-b5c9-d58ab2e8a2b1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:47.103566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:47.110283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:47.111112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:47.117956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:47.119059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:47.119569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:47.121116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:47.121751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:47.122198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:47.222371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aec95fbe-3d9e-473e-b5c9-d58ab2e8a2b1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:47.237326 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aec95fbe-3d9e-473e-b5c9-d58ab2e8a2b1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1259 microversion: 2.1 time: 0.179472 Sep 27 10:03:47.253674 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 835/1820] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:47 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1259 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:47.266920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c19f787-4624-4839-8912-fb63c53cbcf0 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch1-master-1", "block_migration": "auto"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:47.271245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c19f787-4624-4839-8912-fb63c53cbcf0 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:47.285455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c19f787-4624-4839-8912-fb63c53cbcf0 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:47.286321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c19f787-4624-4839-8912-fb63c53cbcf0 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:47.287201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c19f787-4624-4839-8912-fb63c53cbcf0 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:47.288591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c19f787-4624-4839-8912-fb63c53cbcf0 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:47.289266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c19f787-4624-4839-8912-fb63c53cbcf0 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:47.290489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c19f787-4624-4839-8912-fb63c53cbcf0 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:47.326376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:47.356772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-4c19f787-4624-4839-8912-fb63c53cbcf0 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'flavor' on Instance uuid fcd14928-8ef7-425b-89bb-2c714284e840 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:03:47.399314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c19f787-4624-4839-8912-fb63c53cbcf0 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Going to try to live migrate instance to nch1-master-1 {{(pid=145969) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Sep 27 10:03:47.506974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:47.522102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:47.522831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:47.523441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:47.735923 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e1a48761-1334-4b95-9458-390a0547641c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/detail" status: 200 len: 4518 microversion: 2.1 time: 0.637031 Sep 27 10:03:47.737282 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 986/1821] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:03:47 2022] GET /compute/v2.1/servers/detail => generated 4518 bytes in 639 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:47.768870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5c23801-d501-46e5-ad15-cea1087e7bc7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '.version_select at 0x7f7f630a6940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:48.182166 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b5c23801-d501-46e5-ad15-cea1087e7bc7 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/os-floating-ips/4d3b605f-75fd-4c63-bc0e-0cb327fdc5e1" status: 200 len: 184 microversion: 2.1 time: 0.443014 Sep 27 10:03:48.183206 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 987/1822] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:03:47 2022] GET /compute/v2.1/os-floating-ips/4d3b605f-75fd-4c63-bc0e-0cb327fdc5e1 => generated 184 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:48.189745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-fe42f54e-ab2c-44b5-a62f-48cef00af380 req-1d2a127c-63a3-4702-9238-909eee1c2332 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1", "name": "network-vif-deleted", "tag": "cc9dca96-4fbe-402f-b886-7bb414b8545d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:48.218083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-fe42f54e-ab2c-44b5-a62f-48cef00af380 req-1d2a127c-63a3-4702-9238-909eee1c2332 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:48.218961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-fe42f54e-ab2c-44b5-a62f-48cef00af380 req-1d2a127c-63a3-4702-9238-909eee1c2332 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:48.220476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-fe42f54e-ab2c-44b5-a62f-48cef00af380 req-1d2a127c-63a3-4702-9238-909eee1c2332 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:48.235096 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-fe42f54e-ab2c-44b5-a62f-48cef00af380 req-1d2a127c-63a3-4702-9238-909eee1c2332 service nova] Creating event network-vif-deleted:cc9dca96-4fbe-402f-b886-7bb414b8545d for instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 on nch2-master-1 Sep 27 10:03:48.244441 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-fe42f54e-ab2c-44b5-a62f-48cef00af380 req-1d2a127c-63a3-4702-9238-909eee1c2332 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057714 Sep 27 10:03:48.246001 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 988/1823] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:48.259500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e84a97c-069d-437e-9531-e79484e46886 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:48.263162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2e84a97c-069d-437e-9531-e79484e46886 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:48.277006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e84a97c-069d-437e-9531-e79484e46886 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:48.279113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e84a97c-069d-437e-9531-e79484e46886 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:48.279832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e84a97c-069d-437e-9531-e79484e46886 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:48.281915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e84a97c-069d-437e-9531-e79484e46886 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:48.283185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e84a97c-069d-437e-9531-e79484e46886 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:48.283857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e84a97c-069d-437e-9531-e79484e46886 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:48.375157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2e84a97c-069d-437e-9531-e79484e46886 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:48.383536 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2e84a97c-069d-437e-9531-e79484e46886 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1272 microversion: 2.1 time: 0.135393 Sep 27 10:03:48.384468 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 989/1824] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:48 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1272 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:48.389746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-50d0d342-a036-4c53-9e32-f9343b919772 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:48.394969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-50d0d342-a036-4c53-9e32-f9343b919772 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:48.395746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-50d0d342-a036-4c53-9e32-f9343b919772 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'ip': '172.20.0.', 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:48.474826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50d0d342-a036-4c53-9e32-f9343b919772 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:48.475216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50d0d342-a036-4c53-9e32-f9343b919772 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:48.475863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50d0d342-a036-4c53-9e32-f9343b919772 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:48.476953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50d0d342-a036-4c53-9e32-f9343b919772 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:48.477549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50d0d342-a036-4c53-9e32-f9343b919772 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:48.478019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50d0d342-a036-4c53-9e32-f9343b919772 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:48.514300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-50d0d342-a036-4c53-9e32-f9343b919772 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:48.555961 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-50d0d342-a036-4c53-9e32-f9343b919772 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?ip=172.20.0." status: 200 len: 1012 microversion: 2.1 time: 0.170037 Sep 27 10:03:48.557253 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 990/1825] 10.222.0.26 () {58 vars in 1150 bytes} [Tue Sep 27 10:03:48 2022] GET /compute/v2.1/servers?ip=172.20.0. => generated 1012 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:48.563108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e196600b-c433-44b6-8676-d38c3dfb6a17 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:48.565924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e196600b-c433-44b6-8676-d38c3dfb6a17 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:48.575898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e196600b-c433-44b6-8676-d38c3dfb6a17 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:48.576903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e196600b-c433-44b6-8676-d38c3dfb6a17 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:48.577486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e196600b-c433-44b6-8676-d38c3dfb6a17 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:48.578574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e196600b-c433-44b6-8676-d38c3dfb6a17 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:48.579150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e196600b-c433-44b6-8676-d38c3dfb6a17 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:48.580014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e196600b-c433-44b6-8676-d38c3dfb6a17 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:48.622384 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e196600b-c433-44b6-8676-d38c3dfb6a17 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.063601 Sep 27 10:03:48.622990 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 991/1826] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:03:48 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e/os-volume_attachments => generated 25 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:03:48.628218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "172.35.1.106"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:48.793131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:48.801298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:48.802495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:48.803416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:48.804425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:48.805141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:48.806414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:48.812436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:49.229123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "refresh_cache-3a955d66-3892-4838-ad7d-cc9e431a03e1" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 10:03:49.230468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquired lock "refresh_cache-3a955d66-3892-4838-ad7d-cc9e431a03e1" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 10:03:50.081329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Building network info cache for instance {{(pid=145968) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Sep 27 10:03:50.764803 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c19f787-4624-4839-8912-fb63c53cbcf0 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/action" status: 202 len: 0 microversion: 2.25 time: 3.502932 Sep 27 10:03:50.767618 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 836/1827] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:03:47 2022] POST /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/action => generated 0 bytes in 3513 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:03:50.776440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4f1c10b3-bab1-401e-88c4-41c70ce6cb75 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:50.777941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4f1c10b3-bab1-401e-88c4-41c70ce6cb75 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:50.792624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f1c10b3-bab1-401e-88c4-41c70ce6cb75 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:50.793213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f1c10b3-bab1-401e-88c4-41c70ce6cb75 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:50.793654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f1c10b3-bab1-401e-88c4-41c70ce6cb75 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:50.794595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f1c10b3-bab1-401e-88c4-41c70ce6cb75 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:50.795289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f1c10b3-bab1-401e-88c4-41c70ce6cb75 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:50.795879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f1c10b3-bab1-401e-88c4-41c70ce6cb75 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:51.196080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Updating instance_info_cache with network_info: [{"id": "b4f78072-6743-4c1d-bb89-c5bb42c81dcf", "address": "fa:16:3e:92:4f:b3", "network": {"id": "79454fbf-4335-40b3-b1c5-c3f69544e188", "bridge": null, "label": "tempest-FloatingIPsAssociationTestJSON-450320224-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "172.20.0.2"}}], "meta": {"injected": false, "tenant_id": "de87d5984a9a4d4481ab4921133841d6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tapb4f78072-67", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=145968) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:03:51.222871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Releasing lock "refresh_cache-3a955d66-3892-4838-ad7d-cc9e431a03e1" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 10:03:51.224818 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4ec9c2cb-9510-4e64-af98-89812972595d tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "POST /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1/action" status: 202 len: 0 microversion: 2.1 time: 2.599636 Sep 27 10:03:51.233514 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 992/1828] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:03:48 2022] POST /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1/action => generated 0 bytes in 2609 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Sep 27 10:03:51.408426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eda05ca0-fdb5-4619-80e8-f9f90b2a49da tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:51.429528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4f1c10b3-bab1-401e-88c4-41c70ce6cb75 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:51.433439 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eda05ca0-fdb5-4619-80e8-f9f90b2a49da tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.192814 Sep 27 10:03:51.442280 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4f1c10b3-bab1-401e-88c4-41c70ce6cb75 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.672414 Sep 27 10:03:51.446271 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 993/1829] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:03:51 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:51.451568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cfd54e0f-fc03-4946-abc8-bbc616f80ca9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:51.457525 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 837/1830] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:50 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:51.465808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cfd54e0f-fc03-4946-abc8-bbc616f80ca9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:51.466841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2a14a30b-393a-446e-be60-eeae8eab88f0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:51.468002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2a14a30b-393a-446e-be60-eeae8eab88f0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:51.469141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cfd54e0f-fc03-4946-abc8-bbc616f80ca9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'name': '^.*\\-instance\\-[0-9]+$', 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:51.476300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a14a30b-393a-446e-be60-eeae8eab88f0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:51.477084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a14a30b-393a-446e-be60-eeae8eab88f0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:51.477893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a14a30b-393a-446e-be60-eeae8eab88f0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:51.478792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a14a30b-393a-446e-be60-eeae8eab88f0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:51.479415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a14a30b-393a-446e-be60-eeae8eab88f0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:51.480006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a14a30b-393a-446e-be60-eeae8eab88f0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:51.483818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd54e0f-fc03-4946-abc8-bbc616f80ca9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:51.484331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd54e0f-fc03-4946-abc8-bbc616f80ca9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:51.484861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd54e0f-fc03-4946-abc8-bbc616f80ca9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:51.485511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd54e0f-fc03-4946-abc8-bbc616f80ca9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:51.485961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd54e0f-fc03-4946-abc8-bbc616f80ca9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:51.486376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd54e0f-fc03-4946-abc8-bbc616f80ca9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:51.633200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-cfd54e0f-fc03-4946-abc8-bbc616f80ca9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:51.704163 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cfd54e0f-fc03-4946-abc8-bbc616f80ca9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-%5B0-9%5D%2B%24" status: 200 len: 1012 microversion: 2.1 time: 0.256294 Sep 27 10:03:51.705328 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 994/1831] 10.222.0.26 () {58 vars in 1212 bytes} [Tue Sep 27 10:03:51 2022] GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-%5B0-9%5D%2B%24 => generated 1012 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:51.714316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7fb70c6e-7e16-4b1b-ae99-13ef5da92201 req-9f252400-e076-4c02-8807-e530905f0c5d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3a955d66-3892-4838-ad7d-cc9e431a03e1", "tag": "b4f78072-6743-4c1d-bb89-c5bb42c81dcf"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:51.728806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7fb70c6e-7e16-4b1b-ae99-13ef5da92201 req-9f252400-e076-4c02-8807-e530905f0c5d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:51.729844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7fb70c6e-7e16-4b1b-ae99-13ef5da92201 req-9f252400-e076-4c02-8807-e530905f0c5d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:51.730819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7fb70c6e-7e16-4b1b-ae99-13ef5da92201 req-9f252400-e076-4c02-8807-e530905f0c5d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:51.743185 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7fb70c6e-7e16-4b1b-ae99-13ef5da92201 req-9f252400-e076-4c02-8807-e530905f0c5d service nova] Creating event network-changed:b4f78072-6743-4c1d-bb89-c5bb42c81dcf for instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 on nch1-master-1 Sep 27 10:03:51.750446 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7fb70c6e-7e16-4b1b-ae99-13ef5da92201 req-9f252400-e076-4c02-8807-e530905f0c5d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039799 Sep 27 10:03:51.751138 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 995/1832] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:51.761929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0db79881-df85-4548-be7a-71c0c8f206d8 req-85ee6387-35f2-4eb0-bad1-5fd30772ad0f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9997587c-6839-4459-bd4b-b4d4cd6fd09d", "name": "network-vif-deleted", "tag": "c24245da-1f2e-40bc-b00b-6bdc116ec9a4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:51.772922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0db79881-df85-4548-be7a-71c0c8f206d8 req-85ee6387-35f2-4eb0-bad1-5fd30772ad0f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:51.774149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0db79881-df85-4548-be7a-71c0c8f206d8 req-85ee6387-35f2-4eb0-bad1-5fd30772ad0f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:51.774149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0db79881-df85-4548-be7a-71c0c8f206d8 req-85ee6387-35f2-4eb0-bad1-5fd30772ad0f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:51.788104 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0db79881-df85-4548-be7a-71c0c8f206d8 req-85ee6387-35f2-4eb0-bad1-5fd30772ad0f service nova] Creating event network-vif-deleted:c24245da-1f2e-40bc-b00b-6bdc116ec9a4 for instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d on nch2-master-1 Sep 27 10:03:51.795478 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0db79881-df85-4548-be7a-71c0c8f206d8 req-85ee6387-35f2-4eb0-bad1-5fd30772ad0f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042367 Sep 27 10:03:51.796412 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 996/1833] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:51.802413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:51.803517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:51.813731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:51.814279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:51.814720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:51.815458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:51.815918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:51.816412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:51.995984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2a14a30b-393a-446e-be60-eeae8eab88f0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:52.016403 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2a14a30b-393a-446e-be60-eeae8eab88f0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1314 microversion: 2.1 time: 0.556654 Sep 27 10:03:52.017835 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 838/1834] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:51 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1314 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:52.030266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:52.031501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:52.044243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:52.045266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:52.045980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:52.046988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:52.047648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:52.048330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:52.072221 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] HTTP exception thrown: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Request method failure captured: Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d HTTP/1.1 Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMso0VFJjEU5GfoTJmvmRWSj3r_ZSEES5ggmqM4nZlWEK900hPKDrqtEKyMz2MCicCfNf7rpaIQw-u6nipnwKAax2_dGjlwUHftSTX1xbyTI-AkFXK6miTzPu2A2Cr1HWm1A6ftJPMy0819xaVtNSoCE0LZNTLHxKFqqwB_JIN9zzIG4 Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: de87d5984a9a4d4481ab4921133841d6 Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:03:52.074340 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_de87d5984a9a4d4481ab4921133841d6"}]}] Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: de87d5984a9a4d4481ab4921133841d6 Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FloatingIPsAssociationTestJSON-924459803-project Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: c7468cc8bc7d4dde8e2a5eba1fa42fd8 Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-924459803-project Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. Sep 27 10:03:52.076253 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '9997587c-6839-4459-bd4b-b4d4cd6fd09d'} Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:52.078270 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. Sep 27 10:03:52.081679 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:52.087013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Returning 404 to user: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:52.088898 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8056588c-9b31-4c50-a848-f633e7146fc6 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 404 len: 111 microversion: 2.1 time: 0.069580 Sep 27 10:03:52.090344 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 839/1835] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:03:52 2022] DELETE /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 111 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:03:52.098079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b949c123-055e-4aea-af87-7598d70bd7fd tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:52.120139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:52.121415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:52.130327 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b949c123-055e-4aea-af87-7598d70bd7fd tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors" status: 200 len: 2594 microversion: 2.1 time: 0.038468 Sep 27 10:03:52.141501 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 840/1836] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:03:52 2022] GET /compute/v2.1/flavors => generated 2594 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:52.150360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveMigrationTest-server-1987100358", "imageRef": "", "flavorRef": "452", "networks": [{"uuid": "66dfc7b2-4ec7-40ad-b073-f4df786a999e"}], "block_device_mapping_v2": [{"uuid": "51c64884-725f-4278-8118-f50cc398b48a", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:52.258268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] validate_networks() for [('66dfc7b2-4ec7-40ad-b073-f4df786a999e', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:03:52.394472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:52.411248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:52.413679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:52.417018 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9dd202fe-b4a6-41d3-9733-3dcaa6d86600 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.618749 Sep 27 10:03:52.418037 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 997/1837] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:03:51 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:03:52.423192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86f6adca-9d38-4f13-a5ce-54636217647d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:52.431826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-86f6adca-9d38-4f13-a5ce-54636217647d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:52.432968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86f6adca-9d38-4f13-a5ce-54636217647d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'name': '^.*\\-instance\\-.*$', 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:52.440318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f6adca-9d38-4f13-a5ce-54636217647d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:52.441167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f6adca-9d38-4f13-a5ce-54636217647d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:52.441675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f6adca-9d38-4f13-a5ce-54636217647d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:52.442444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f6adca-9d38-4f13-a5ce-54636217647d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:52.443033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f6adca-9d38-4f13-a5ce-54636217647d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:52.443708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f6adca-9d38-4f13-a5ce-54636217647d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:52.563217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-86f6adca-9d38-4f13-a5ce-54636217647d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:52.651790 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86f6adca-9d38-4f13-a5ce-54636217647d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-.%2A%24" status: 200 len: 1012 microversion: 2.1 time: 0.232122 Sep 27 10:03:52.653188 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 998/1838] 10.222.0.26 () {58 vars in 1196 bytes} [Tue Sep 27 10:03:52 2022] GET /compute/v2.1/servers?name=%5E.%2A%5C-instance%5C-.%2A%24 => generated 1012 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:52.658988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:52.660374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 9997587c-6839-4459-bd4b-b4d4cd6fd09d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:52.694285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:52.695163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:52.697163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:52.698351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:52.698907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:52.699477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:52.714761 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] HTTP exception thrown: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Request method failure captured: Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d HTTP/1.1 Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMso0VFJjEU5GfoTJmvmRWSj3r_ZSEES5ggmqM4nZlWEK900hPKDrqtEKyMz2MCicCfNf7rpaIQw-u6nipnwKAax2_dGjlwUHftSTX1xbyTI-AkFXK6miTzPu2A2Cr1HWm1A6ftJPMy0819xaVtNSoCE0LZNTLHxKFqqwB_JIN9zzIG4 Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: de87d5984a9a4d4481ab4921133841d6 Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:03:52.717266 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_de87d5984a9a4d4481ab4921133841d6"}]}] Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: de87d5984a9a4d4481ab4921133841d6 Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-FloatingIPsAssociationTestJSON-924459803-project Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: c7468cc8bc7d4dde8e2a5eba1fa42fd8 Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-924459803-project Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. Sep 27 10:03:52.719084 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '9997587c-6839-4459-bd4b-b4d4cd6fd09d'} Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:52.725362 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. Sep 27 10:03:52.728007 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:52.729513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Returning 404 to user: Instance 9997587c-6839-4459-bd4b-b4d4cd6fd09d could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:52.729513 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cd25fb46-7bb4-4207-aaa9-c5a784bed280 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d" status: 404 len: 111 microversion: 2.1 time: 0.064382 Sep 27 10:03:52.729513 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 999/1839] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:52 2022] GET /compute/v2.1/servers/9997587c-6839-4459-bd4b-b4d4cd6fd09d => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:03:52.729513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0fae549a-d191-477e-a2b5-46f6a9301d82 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:52.735105 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0fae549a-d191-477e-a2b5-46f6a9301d82 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.014132 Sep 27 10:03:52.736547 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1000/1840] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:03:52 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 16 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:52.741301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-51ebeddc-ad77-4140-9036-57a63f62d4af tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:52.741927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-51ebeddc-ad77-4140-9036-57a63f62d4af tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:52.758986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51ebeddc-ad77-4140-9036-57a63f62d4af tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:52.759605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51ebeddc-ad77-4140-9036-57a63f62d4af tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:52.760096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51ebeddc-ad77-4140-9036-57a63f62d4af tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:52.761015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51ebeddc-ad77-4140-9036-57a63f62d4af tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:52.761648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51ebeddc-ad77-4140-9036-57a63f62d4af tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:52.762353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51ebeddc-ad77-4140-9036-57a63f62d4af tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:52.952038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-51ebeddc-ad77-4140-9036-57a63f62d4af tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:52.960789 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-51ebeddc-ad77-4140-9036-57a63f62d4af tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.223146 Sep 27 10:03:52.962052 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1001/1841] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:52 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:52.966478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e090c2dc-ea40-4b0d-a81a-3b2a618d4cf9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:52.971402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e090c2dc-ea40-4b0d-a81a-3b2a618d4cf9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:52.972066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e090c2dc-ea40-4b0d-a81a-3b2a618d4cf9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'name': '1561922851', 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:52.977442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e090c2dc-ea40-4b0d-a81a-3b2a618d4cf9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:52.977966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e090c2dc-ea40-4b0d-a81a-3b2a618d4cf9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:52.978337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e090c2dc-ea40-4b0d-a81a-3b2a618d4cf9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:52.979069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e090c2dc-ea40-4b0d-a81a-3b2a618d4cf9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:52.979521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e090c2dc-ea40-4b0d-a81a-3b2a618d4cf9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:52.979941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e090c2dc-ea40-4b0d-a81a-3b2a618d4cf9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:53.079976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-e090c2dc-ea40-4b0d-a81a-3b2a618d4cf9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:53.149963 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e090c2dc-ea40-4b0d-a81a-3b2a618d4cf9 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?name=1561922851" status: 200 len: 346 microversion: 2.1 time: 0.186803 Sep 27 10:03:53.158298 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1002/1842] 10.222.0.26 () {58 vars in 1156 bytes} [Tue Sep 27 10:03:52 2022] GET /compute/v2.1/servers?name=1561922851 => generated 346 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:53.158298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-54daabc0-9f90-43af-afe4-9206fe638d95 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '.version_select at 0x7f7f6301d0d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:53.733967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:03:53.735005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:03:53.735804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:03:53.736529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:03:53.737116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:03:53.748478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:53.749561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:53.750234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:53.750966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:53.752134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:53.752758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:53.777234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Getting quotas for project ec0980b5824b425fbad9b8554cf5c42c. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:03:53.783039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Getting quotas for user dce01d3212934161a38be853b4cd93ae and project ec0980b5824b425fbad9b8554cf5c42c. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:03:53.867409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:03:53.870184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='51c64884-725f-4278-8118-f50cc398b48a',volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:03:54.000416 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-54daabc0-9f90-43af-afe4-9206fe638d95 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/4d3b605f-75fd-4c63-bc0e-0cb327fdc5e1" status: 202 len: 0 microversion: 2.1 time: 0.847576 Sep 27 10:03:54.001079 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1003/1843] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:03:53 2022] DELETE /compute/v2.1/os-floating-ips/4d3b605f-75fd-4c63-bc0e-0cb327fdc5e1 => generated 0 bytes in 849 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:03:54.006396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ce56838d-b5fa-4d13-9a84-3b5403430e8e tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:54.016814 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ce56838d-b5fa-4d13-9a84-3b5403430e8e tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.014662 Sep 27 10:03:54.017760 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1004/1844] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:03:54 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 16 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:54.022400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e9594405-606e-4a8d-bb62-ffdfdd967611 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:54.023011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e9594405-606e-4a8d-bb62-ffdfdd967611 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:54.029658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9594405-606e-4a8d-bb62-ffdfdd967611 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.030139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9594405-606e-4a8d-bb62-ffdfdd967611 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.030591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9594405-606e-4a8d-bb62-ffdfdd967611 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.031287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9594405-606e-4a8d-bb62-ffdfdd967611 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.031848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9594405-606e-4a8d-bb62-ffdfdd967611 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.032283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9594405-606e-4a8d-bb62-ffdfdd967611 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.035198 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a021032-1ebe-44ba-ad81-6c463a8d4c6f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.890857 Sep 27 10:03:54.036184 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 841/1845] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:03:52 2022] POST /compute/v2.1/servers => generated 380 bytes in 1894 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:03:54.041803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-881edd07-978d-460b-b3dd-fd2dac25161e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:54.048385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-881edd07-978d-460b-b3dd-fd2dac25161e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:54.049735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-881edd07-978d-460b-b3dd-fd2dac25161e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'name': '-instance', 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:54.068388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881edd07-978d-460b-b3dd-fd2dac25161e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.069279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881edd07-978d-460b-b3dd-fd2dac25161e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.070519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881edd07-978d-460b-b3dd-fd2dac25161e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.073582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881edd07-978d-460b-b3dd-fd2dac25161e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.074138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881edd07-978d-460b-b3dd-fd2dac25161e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.075005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881edd07-978d-460b-b3dd-fd2dac25161e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.194219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-881edd07-978d-460b-b3dd-fd2dac25161e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:54.243635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e9594405-606e-4a8d-bb62-ffdfdd967611 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:54.258988 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e9594405-606e-4a8d-bb62-ffdfdd967611 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1314 microversion: 2.1 time: 0.240021 Sep 27 10:03:54.260009 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1005/1846] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:54 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1314 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:54.265877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:54.266823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:54.275050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.275664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.276193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.277044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.277614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.278392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.293612 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-881edd07-978d-460b-b3dd-fd2dac25161e tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?name=-instance" status: 200 len: 1012 microversion: 2.1 time: 0.255997 Sep 27 10:03:54.294248 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 842/1847] 10.222.0.26 () {58 vars in 1154 bytes} [Tue Sep 27 10:03:54 2022] GET /compute/v2.1/servers?name=-instance => generated 1012 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:54.300035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-dcfbba19-6640-4bcd-807c-380a5a8af0ed req-6593669f-eb80-4c60-b310-3f0742e984f8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "14bb2db7-2b1d-46d4-8800-26b149825654", "tag": "dc496928-7dd4-48f7-a375-66707f33ecf8"}, {"name": "network-changed", "server_uuid": "152a5758-d14b-4ca9-917e-f7017c6d99b2", "tag": "61c12b2f-7268-4252-a482-600cb57542ca"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:54.323626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dcfbba19-6640-4bcd-807c-380a5a8af0ed req-6593669f-eb80-4c60-b310-3f0742e984f8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.323826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dcfbba19-6640-4bcd-807c-380a5a8af0ed req-6593669f-eb80-4c60-b310-3f0742e984f8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.329560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dcfbba19-6640-4bcd-807c-380a5a8af0ed req-6593669f-eb80-4c60-b310-3f0742e984f8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.353821 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-dcfbba19-6640-4bcd-807c-380a5a8af0ed req-6593669f-eb80-4c60-b310-3f0742e984f8 service nova] Creating event network-changed:dc496928-7dd4-48f7-a375-66707f33ecf8 for instance 14bb2db7-2b1d-46d4-8800-26b149825654 on nch2-master-1 Sep 27 10:03:54.354414 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-dcfbba19-6640-4bcd-807c-380a5a8af0ed req-6593669f-eb80-4c60-b310-3f0742e984f8 service nova] Creating event network-changed:61c12b2f-7268-4252-a482-600cb57542ca for instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 on nch2-master-1 Sep 27 10:03:54.362835 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-dcfbba19-6640-4bcd-807c-380a5a8af0ed req-6593669f-eb80-4c60-b310-3f0742e984f8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.066469 Sep 27 10:03:54.363467 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 843/1848] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:54 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:54.376359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4043c081-ad70-4f14-b807-11914dcc5e8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:54.377169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4043c081-ad70-4f14-b807-11914dcc5e8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:54.385549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043c081-ad70-4f14-b807-11914dcc5e8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.386048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043c081-ad70-4f14-b807-11914dcc5e8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.386489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043c081-ad70-4f14-b807-11914dcc5e8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.387255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043c081-ad70-4f14-b807-11914dcc5e8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.387716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043c081-ad70-4f14-b807-11914dcc5e8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.388332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4043c081-ad70-4f14-b807-11914dcc5e8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.492219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:54.504596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:54.506670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:54.508288 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba66bd32-8e5f-4c0d-a762-92ed8100f9ce tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.247111 Sep 27 10:03:54.509212 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1006/1849] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:03:54 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:03:54.521299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5a52ee0a-c95e-4bc3-a1e7-8897c050a5c8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:54.523168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5a52ee0a-c95e-4bc3-a1e7-8897c050a5c8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:54.532295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a52ee0a-c95e-4bc3-a1e7-8897c050a5c8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.533101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a52ee0a-c95e-4bc3-a1e7-8897c050a5c8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.533630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a52ee0a-c95e-4bc3-a1e7-8897c050a5c8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.534490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a52ee0a-c95e-4bc3-a1e7-8897c050a5c8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.534972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a52ee0a-c95e-4bc3-a1e7-8897c050a5c8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.535552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a52ee0a-c95e-4bc3-a1e7-8897c050a5c8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.573766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5a52ee0a-c95e-4bc3-a1e7-8897c050a5c8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:03:54.663676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:54.670965 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5a52ee0a-c95e-4bc3-a1e7-8897c050a5c8 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 204 len: 0 microversion: 2.1 time: 0.155565 Sep 27 10:03:54.672139 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1007/1850] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:03:54 2022] DELETE /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:03:54.678204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0a58fb6-f275-47c1-b5d1-2708b259f599 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:54.708178 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f0a58fb6-f275-47c1-b5d1-2708b259f599 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail?minDisk=2" status: 200 len: 3209 microversion: 2.1 time: 0.034545 Sep 27 10:03:54.708835 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1008/1851] 10.222.0.26 () {58 vars in 1165 bytes} [Tue Sep 27 10:03:54 2022] GET /compute/v2.1/flavors/detail?minDisk=2 => generated 3209 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:54.713653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82cc030f-cdaa-4c20-ad36-3b16ea5eb8c0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:54.714409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-82cc030f-cdaa-4c20-ad36-3b16ea5eb8c0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:54.721611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82cc030f-cdaa-4c20-ad36-3b16ea5eb8c0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.722105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82cc030f-cdaa-4c20-ad36-3b16ea5eb8c0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.722551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82cc030f-cdaa-4c20-ad36-3b16ea5eb8c0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.723270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82cc030f-cdaa-4c20-ad36-3b16ea5eb8c0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.723737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82cc030f-cdaa-4c20-ad36-3b16ea5eb8c0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.724201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82cc030f-cdaa-4c20-ad36-3b16ea5eb8c0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.798283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-82cc030f-cdaa-4c20-ad36-3b16ea5eb8c0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:54.808121 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-82cc030f-cdaa-4c20-ad36-3b16ea5eb8c0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1103 microversion: 2.1 time: 0.098088 Sep 27 10:03:54.812710 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1009/1852] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:54 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1103 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:54.814923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f4bd0cfb-69c8-4d7d-b7f3-63e87b93c594 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:54.820811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f4bd0cfb-69c8-4d7d-b7f3-63e87b93c594 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:54.822891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4043c081-ad70-4f14-b807-11914dcc5e8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:54.823471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f4bd0cfb-69c8-4d7d-b7f3-63e87b93c594 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Searching by: {'name': 't-ListServerFiltersTestJSON-instance-156192285', 'deleted': False, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:03:54.829451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bd0cfb-69c8-4d7d-b7f3-63e87b93c594 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.829935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bd0cfb-69c8-4d7d-b7f3-63e87b93c594 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.830598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bd0cfb-69c8-4d7d-b7f3-63e87b93c594 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.831339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bd0cfb-69c8-4d7d-b7f3-63e87b93c594 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.831900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bd0cfb-69c8-4d7d-b7f3-63e87b93c594 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.832760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bd0cfb-69c8-4d7d-b7f3-63e87b93c594 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.832933 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4043c081-ad70-4f14-b807-11914dcc5e8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.466594 Sep 27 10:03:54.833538 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 844/1853] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:54 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:54.838719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-879ab7e3-a938-4e9e-b3f1-71a65b5e0291 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:54.839423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-879ab7e3-a938-4e9e-b3f1-71a65b5e0291 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:54.846137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-879ab7e3-a938-4e9e-b3f1-71a65b5e0291 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.846644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-879ab7e3-a938-4e9e-b3f1-71a65b5e0291 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.847365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-879ab7e3-a938-4e9e-b3f1-71a65b5e0291 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.848077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-879ab7e3-a938-4e9e-b3f1-71a65b5e0291 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:54.848621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-879ab7e3-a938-4e9e-b3f1-71a65b5e0291 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:54.849718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-879ab7e3-a938-4e9e-b3f1-71a65b5e0291 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:54.868594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=4,nova_cell1:INSERT=2,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:54.942072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-f4bd0cfb-69c8-4d7d-b7f3-63e87b93c594 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:03:55.005443 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f4bd0cfb-69c8-4d7d-b7f3-63e87b93c594 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers?name=t-ListServerFiltersTestJSON-instance-156192285" status: 200 len: 346 microversion: 2.1 time: 0.195005 Sep 27 10:03:55.007333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-879ab7e3-a938-4e9e-b3f1-71a65b5e0291 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:55.014277 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1010/1854] 10.222.0.26 () {58 vars in 1228 bytes} [Tue Sep 27 10:03:54 2022] GET /compute/v2.1/servers?name=t-ListServerFiltersTestJSON-instance-156192285 => generated 346 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:55.017255 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-879ab7e3-a938-4e9e-b3f1-71a65b5e0291 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1525 microversion: 2.1 time: 0.182312 Sep 27 10:03:55.018042 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 845/1855] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:54 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1525 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:55.019502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a6b823fe-88ea-4a6b-b17e-dd2694e4a673 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.022994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6b3f9461-5696-4da6-bb59-381e7accec29 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.023560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6b3f9461-5696-4da6-bb59-381e7accec29 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:55.026865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=1,nova_api:INSERT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:55.031252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b3f9461-5696-4da6-bb59-381e7accec29 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.031850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b3f9461-5696-4da6-bb59-381e7accec29 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.032351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b3f9461-5696-4da6-bb59-381e7accec29 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.033140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b3f9461-5696-4da6-bb59-381e7accec29 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.033597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b3f9461-5696-4da6-bb59-381e7accec29 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.034028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b3f9461-5696-4da6-bb59-381e7accec29 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.035420 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a6b823fe-88ea-4a6b-b17e-dd2694e4a673 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.020010 Sep 27 10:03:55.036020 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1011/1856] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:55.051063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9f52c79-8ce6-4daa-9397-0bc7366925e7 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.078222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6b3f9461-5696-4da6-bb59-381e7accec29 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:03:55.079968 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9f52c79-8ce6-4daa-9397-0bc7366925e7 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail?minRam=1025" status: 200 len: 2145 microversion: 2.1 time: 0.034445 Sep 27 10:03:55.080577 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1012/1857] 10.222.0.26 () {58 vars in 1169 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors/detail?minRam=1025 => generated 2145 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:55.098581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e2a763f0-b862-4fa6-b66d-5015fa66ad8e tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.113094 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e2a763f0-b862-4fa6-b66d-5015fa66ad8e tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail?limit=1" status: 200 len: 476 microversion: 2.1 time: 0.018748 Sep 27 10:03:55.114319 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1013/1858] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors/detail?limit=1 => generated 476 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:03:55.130423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41fb4c23-803e-4ecd-b707-d1647384f286 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.138978 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-41fb4c23-803e-4ecd-b707-d1647384f286 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.013643 Sep 27 10:03:55.139545 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1014/1859] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:03:55.152505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0838684e-5d14-4c89-883f-68c321a2e028 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.179341 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6b3f9461-5696-4da6-bb59-381e7accec29 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "DELETE /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 204 len: 0 microversion: 2.1 time: 0.159545 Sep 27 10:03:55.180208 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 846/1860] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:03:55 2022] DELETE /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:03:55.190014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9c1ea245-63f0-49dc-acac-b0b9a12d39f1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.190826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9c1ea245-63f0-49dc-acac-b0b9a12d39f1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:55.195257 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0838684e-5d14-4c89-883f-68c321a2e028 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail?marker=452" status: 200 len: 2500 microversion: 2.1 time: 0.047036 Sep 27 10:03:55.196829 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1015/1861] 10.222.0.26 () {58 vars in 1167 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors/detail?marker=452 => generated 2500 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:55.209118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1ea245-63f0-49dc-acac-b0b9a12d39f1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.209609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1ea245-63f0-49dc-acac-b0b9a12d39f1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.210046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1ea245-63f0-49dc-acac-b0b9a12d39f1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.210811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1ea245-63f0-49dc-acac-b0b9a12d39f1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.211310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1ea245-63f0-49dc-acac-b0b9a12d39f1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.211819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c1ea245-63f0-49dc-acac-b0b9a12d39f1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.213623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0b929192-eade-49b6-a583-4835cd484446 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.222183 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0b929192-eade-49b6-a583-4835cd484446 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.013401 Sep 27 10:03:55.222848 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1016/1862] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:55.236307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5cabb0ac-5a3f-4106-b59c-38ed86909d06 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.256059 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5cabb0ac-5a3f-4106-b59c-38ed86909d06 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors?minDisk=2" status: 200 len: 1663 microversion: 2.1 time: 0.023514 Sep 27 10:03:55.256885 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1017/1863] 10.222.0.26 () {58 vars in 1144 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors?minDisk=2 => generated 1663 bytes in 25 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:55.272640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9a62ae22-b7d3-479a-93aa-b1e69df11d62 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.281643 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9a62ae22-b7d3-479a-93aa-b1e69df11d62 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.012915 Sep 27 10:03:55.282322 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1018/1864] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:55.288971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-772ddc5c-e4ee-411f-b1e3-a606fd85ede1 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.288971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-772ddc5c-e4ee-411f-b1e3-a606fd85ede1 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:55.296840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-772ddc5c-e4ee-411f-b1e3-a606fd85ede1 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.297340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-772ddc5c-e4ee-411f-b1e3-a606fd85ede1 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.297867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-772ddc5c-e4ee-411f-b1e3-a606fd85ede1 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.298549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-772ddc5c-e4ee-411f-b1e3-a606fd85ede1 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.299043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-772ddc5c-e4ee-411f-b1e3-a606fd85ede1 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.299474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-772ddc5c-e4ee-411f-b1e3-a606fd85ede1 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.356722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9c1ea245-63f0-49dc-acac-b0b9a12d39f1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:55.366054 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9c1ea245-63f0-49dc-acac-b0b9a12d39f1 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1517 microversion: 2.1 time: 0.181079 Sep 27 10:03:55.368731 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 847/1865] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1517 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:55.372150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e79079c8-8a50-4b9a-ad0e-c618270327d4 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.398358 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e79079c8-8a50-4b9a-ad0e-c618270327d4 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors?minRam=1025" status: 200 len: 1111 microversion: 2.1 time: 0.030384 Sep 27 10:03:55.399256 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 848/1866] 10.222.0.26 () {58 vars in 1148 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors?minRam=1025 => generated 1111 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:55.414030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-96bbbe80-4338-4b95-a03a-9a7e99ce2898 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.428925 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-96bbbe80-4338-4b95-a03a-9a7e99ce2898 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors?limit=1" status: 200 len: 299 microversion: 2.1 time: 0.018762 Sep 27 10:03:55.429796 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 849/1867] 10.222.0.26 () {58 vars in 1140 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors?limit=1 => generated 299 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:55.443926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-82f795ef-d4e5-4164-a4f9-38f2591bb217 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.453996 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-82f795ef-d4e5-4164-a4f9-38f2591bb217 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.014061 Sep 27 10:03:55.454804 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 850/1868] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:03:55.466077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-772ddc5c-e4ee-411f-b1e3-a606fd85ede1 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:55.467916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f12457fb-0de0-4180-aa5b-3786e1fed723 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.469423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:SAVEPOINT=1,nova_cell1:UPDATE=2,nova_cell1:RELEASE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:55.474547 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-772ddc5c-e4ee-411f-b1e3-a606fd85ede1 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1314 microversion: 2.1 time: 0.191065 Sep 27 10:03:55.475123 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1019/1869] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1314 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:55.514188 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f12457fb-0de0-4180-aa5b-3786e1fed723 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors?marker=452" status: 200 len: 1300 microversion: 2.1 time: 0.050502 Sep 27 10:03:55.514669 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 851/1870] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors?marker=452 => generated 1300 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:55.529460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a8789f3c-b9b0-417e-a8f0-390f03e43640 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.536651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.537371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:55.544991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.545523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.546218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.547183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.547649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.548160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.552322 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a8789f3c-b9b0-417e-a8f0-390f03e43640 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.026758 Sep 27 10:03:55.554310 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1020/1871] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:55.576516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0eb63806-6599-4d69-bc6b-4727597f67a7 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.585284 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0eb63806-6599-4d69-bc6b-4727597f67a7 tempest-FlavorsV2TestJSON-1941280693 tempest-FlavorsV2TestJSON-1941280693-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.013377 Sep 27 10:03:55.585930 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1021/1872] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:55.716449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:55.728205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:55.729708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:55.730981 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5f220448-5e74-4dac-b2c6-337b9c46d6bd tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.199130 Sep 27 10:03:55.731557 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 852/1873] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:03:55.833320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-744d0b5b-acb9-4b30-a660-ae68772e8a07 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.834362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-744d0b5b-acb9-4b30-a660-ae68772e8a07 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:55.845814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-744d0b5b-acb9-4b30-a660-ae68772e8a07 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.847714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-744d0b5b-acb9-4b30-a660-ae68772e8a07 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.847714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-744d0b5b-acb9-4b30-a660-ae68772e8a07 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.848117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-744d0b5b-acb9-4b30-a660-ae68772e8a07 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.848984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-744d0b5b-acb9-4b30-a660-ae68772e8a07 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.849386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-744d0b5b-acb9-4b30-a660-ae68772e8a07 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.859821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5fecab7e-e892-4b47-b3c0-8409a67c905a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:55.861562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5fecab7e-e892-4b47-b3c0-8409a67c905a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:55.869459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fecab7e-e892-4b47-b3c0-8409a67c905a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.870535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fecab7e-e892-4b47-b3c0-8409a67c905a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.871042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fecab7e-e892-4b47-b3c0-8409a67c905a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.872012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fecab7e-e892-4b47-b3c0-8409a67c905a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:55.872505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fecab7e-e892-4b47-b3c0-8409a67c905a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:55.873131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fecab7e-e892-4b47-b3c0-8409a67c905a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:55.932773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-744d0b5b-acb9-4b30-a660-ae68772e8a07 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:55.943975 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-744d0b5b-acb9-4b30-a660-ae68772e8a07 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1103 microversion: 2.1 time: 0.115490 Sep 27 10:03:55.953027 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1022/1874] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1103 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:56.045141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e9c39e5-8134-4fa8-8f9a-18004a568a6b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:56.046049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2e9c39e5-8134-4fa8-8f9a-18004a568a6b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:56.054257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e9c39e5-8134-4fa8-8f9a-18004a568a6b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:56.054849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e9c39e5-8134-4fa8-8f9a-18004a568a6b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:56.056135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e9c39e5-8134-4fa8-8f9a-18004a568a6b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:56.057017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e9c39e5-8134-4fa8-8f9a-18004a568a6b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:56.057667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e9c39e5-8134-4fa8-8f9a-18004a568a6b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:56.058170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e9c39e5-8134-4fa8-8f9a-18004a568a6b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:56.233539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2e9c39e5-8134-4fa8-8f9a-18004a568a6b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:56.242939 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2e9c39e5-8134-4fa8-8f9a-18004a568a6b tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1525 microversion: 2.1 time: 0.204445 Sep 27 10:03:56.244050 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1023/1875] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:56 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1525 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:56.293510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fecab7e-e892-4b47-b3c0-8409a67c905a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:56.303977 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5fecab7e-e892-4b47-b3c0-8409a67c905a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.448955 Sep 27 10:03:56.304592 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 853/1876] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:55 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:56.393936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5cd5a20-ebd8-47f7-b09c-54cdc3e4909d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:56.394594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b5cd5a20-ebd8-47f7-b09c-54cdc3e4909d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:56.402085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cd5a20-ebd8-47f7-b09c-54cdc3e4909d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:56.402952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cd5a20-ebd8-47f7-b09c-54cdc3e4909d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:56.403567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cd5a20-ebd8-47f7-b09c-54cdc3e4909d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:56.404303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cd5a20-ebd8-47f7-b09c-54cdc3e4909d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:56.404803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cd5a20-ebd8-47f7-b09c-54cdc3e4909d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:56.405700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5cd5a20-ebd8-47f7-b09c-54cdc3e4909d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:56.501062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c201d88d-07cb-40a0-b5e6-3414739d9ca4 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:56.502109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c201d88d-07cb-40a0-b5e6-3414739d9ca4 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:56.511816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c201d88d-07cb-40a0-b5e6-3414739d9ca4 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:56.512425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c201d88d-07cb-40a0-b5e6-3414739d9ca4 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:56.513407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c201d88d-07cb-40a0-b5e6-3414739d9ca4 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:56.514470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c201d88d-07cb-40a0-b5e6-3414739d9ca4 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:56.515181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c201d88d-07cb-40a0-b5e6-3414739d9ca4 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:56.515991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c201d88d-07cb-40a0-b5e6-3414739d9ca4 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:56.575486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b5cd5a20-ebd8-47f7-b09c-54cdc3e4909d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:56.585644 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b5cd5a20-ebd8-47f7-b09c-54cdc3e4909d tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1517 microversion: 2.1 time: 0.197237 Sep 27 10:03:56.586560 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1024/1877] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:56 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1517 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:56.757425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:56.758429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:56.766182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:56.766763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:56.767474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:56.768479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:56.769266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:56.769628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:56.869729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c201d88d-07cb-40a0-b5e6-3414739d9ca4 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:56.878005 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c201d88d-07cb-40a0-b5e6-3414739d9ca4 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1314 microversion: 2.1 time: 0.382936 Sep 27 10:03:56.878835 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 854/1878] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:56 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1314 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:56.958409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:56.972987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:56.973880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4f88b04c-73f6-47f4-9b46-a39440648f4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:56.974422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4f88b04c-73f6-47f4-9b46-a39440648f4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:56.976173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:56.978183 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e2a72630-db7c-44b6-a8fd-1cb629db802f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.224540 Sep 27 10:03:56.979502 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1025/1879] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:03:56 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:03:56.990434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f88b04c-73f6-47f4-9b46-a39440648f4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:56.990930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f88b04c-73f6-47f4-9b46-a39440648f4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:56.991361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f88b04c-73f6-47f4-9b46-a39440648f4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:56.992832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f88b04c-73f6-47f4-9b46-a39440648f4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:56.993459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f88b04c-73f6-47f4-9b46-a39440648f4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:56.994277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f88b04c-73f6-47f4-9b46-a39440648f4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:57.083713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4f88b04c-73f6-47f4-9b46-a39440648f4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:57.092342 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4f88b04c-73f6-47f4-9b46-a39440648f4e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1103 microversion: 2.1 time: 0.125322 Sep 27 10:03:57.092965 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 855/1880] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:56 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1103 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:57.271039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4c299e5a-91fc-4c7f-a195-761dc22c327c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:57.272193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4c299e5a-91fc-4c7f-a195-761dc22c327c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:57.280630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c299e5a-91fc-4c7f-a195-761dc22c327c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:57.281208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c299e5a-91fc-4c7f-a195-761dc22c327c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:57.281810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c299e5a-91fc-4c7f-a195-761dc22c327c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:57.282530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c299e5a-91fc-4c7f-a195-761dc22c327c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:57.283061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c299e5a-91fc-4c7f-a195-761dc22c327c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:57.283596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c299e5a-91fc-4c7f-a195-761dc22c327c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:57.327936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1e362acc-9d2e-459d-9a29-9e4f94bd097b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:57.328961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1e362acc-9d2e-459d-9a29-9e4f94bd097b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:57.337559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e362acc-9d2e-459d-9a29-9e4f94bd097b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:57.337971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e362acc-9d2e-459d-9a29-9e4f94bd097b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:57.338410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e362acc-9d2e-459d-9a29-9e4f94bd097b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:57.339129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e362acc-9d2e-459d-9a29-9e4f94bd097b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:57.339582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e362acc-9d2e-459d-9a29-9e4f94bd097b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:57.340005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e362acc-9d2e-459d-9a29-9e4f94bd097b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:57.484309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4c299e5a-91fc-4c7f-a195-761dc22c327c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:57.495418 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4c299e5a-91fc-4c7f-a195-761dc22c327c tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1525 microversion: 2.1 time: 0.228326 Sep 27 10:03:57.496212 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1026/1881] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:57 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1525 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:57.567581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1e362acc-9d2e-459d-9a29-9e4f94bd097b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:57.587097 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1e362acc-9d2e-459d-9a29-9e4f94bd097b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.262722 Sep 27 10:03:57.590057 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 856/1882] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:57 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:57.612063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dccd63b7-ca08-4243-b23d-981f0ede2bc0 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:57.612832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dccd63b7-ca08-4243-b23d-981f0ede2bc0 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:57.615208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:03:57.624302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccd63b7-ca08-4243-b23d-981f0ede2bc0 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:57.625141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccd63b7-ca08-4243-b23d-981f0ede2bc0 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:57.625816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccd63b7-ca08-4243-b23d-981f0ede2bc0 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:57.626563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccd63b7-ca08-4243-b23d-981f0ede2bc0 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:57.627511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccd63b7-ca08-4243-b23d-981f0ede2bc0 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:57.628144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccd63b7-ca08-4243-b23d-981f0ede2bc0 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:57.802123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dccd63b7-ca08-4243-b23d-981f0ede2bc0 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:57.811180 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dccd63b7-ca08-4243-b23d-981f0ede2bc0 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1517 microversion: 2.1 time: 0.203951 Sep 27 10:03:57.812042 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1027/1883] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:57 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1517 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:57.905890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2b2551dc-9b56-4dd8-8e9b-d97e6ab12044 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:57.907049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2b2551dc-9b56-4dd8-8e9b-d97e6ab12044 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:57.914030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa76a745-cab0-46a8-8f1b-5a0d57110eab tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Calling method '.version_select at 0x7f7f62e2f550>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:57.916422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2551dc-9b56-4dd8-8e9b-d97e6ab12044 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:57.917165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2551dc-9b56-4dd8-8e9b-d97e6ab12044 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:57.917917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2551dc-9b56-4dd8-8e9b-d97e6ab12044 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:57.918845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2551dc-9b56-4dd8-8e9b-d97e6ab12044 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:57.919476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2551dc-9b56-4dd8-8e9b-d97e6ab12044 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:57.920086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2551dc-9b56-4dd8-8e9b-d97e6ab12044 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:57.940110 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-aa76a745-cab0-46a8-8f1b-5a0d57110eab tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] HTTP exception thrown: Image not found. Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa76a745-cab0-46a8-8f1b-5a0d57110eab tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Request method failure captured: Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/images/a7fc0a3f-fb0e-4409-84c9-6da9f4d5e6ab/metadata/os_distro HTTP/1.1 Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsqNqVWsJd3bcnIJ5UMQ0u6omIPVr113A3D2vGcajBOylTgWB4icdQaKLm5LB-DvZRgG1VUt4TcXfJbsiZnqvAbqruwq5ysLJIn8nNxCSF6lqaxQ8eE_B5tAWA7hNkgkW1S7gT_hflgRJZZV4CV1NjvhHgmeAvB6bY-7x7U-VF3vsKw Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:03:57.957823 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c568082c4b344fbdb179d91ffab1c5bb"}]}] Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4fbf4eb1ec34c7d89f30efe61c0959a Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: exception: Image not found. Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'image_id': 'a7fc0a3f-fb0e-4409-84c9-6da9f4d5e6ab', 'id': 'os_distro'} Sep 27 10:03:57.959767 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:57.961586 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID a7fc0a3f-fb0e-4409-84c9-6da9f4d5e6ab Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:57.963115 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image a7fc0a3f-fb0e-4409-84c9-6da9f4d5e6ab could not be found. Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:03:57.965036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 125, in delete Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:57.972994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa76a745-cab0-46a8-8f1b-5a0d57110eab tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Returning 404 to user: Image not found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:57.974398 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aa76a745-cab0-46a8-8f1b-5a0d57110eab tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] 10.222.0.26 "DELETE /compute/v2.1/images/a7fc0a3f-fb0e-4409-84c9-6da9f4d5e6ab/metadata/os_distro" status: 404 len: 62 microversion: 2.1 time: 0.191920 Sep 27 10:03:57.974398 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 857/1884] 10.222.0.26 () {58 vars in 1293 bytes} [Tue Sep 27 10:03:57 2022] DELETE /compute/v2.1/images/a7fc0a3f-fb0e-4409-84c9-6da9f4d5e6ab/metadata/os_distro => generated 62 bytes in 198 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Sep 27 10:03:57.982451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6b7a9516-141b-4820-a190-3af2918f434b tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Calling method '.version_select at 0x7f7f62ec4d30>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:58.013160 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-6b7a9516-141b-4820-a190-3af2918f434b tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] HTTP exception thrown: Image not found. Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6b7a9516-141b-4820-a190-3af2918f434b tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Request method failure captured: Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/images/d30c29c6-fefd-4f21-a821-9b3ad04caca2/metadata/os_version HTTP/1.1 Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsqNqVWsJd3bcnIJ5UMQ0u6omIPVr113A3D2vGcajBOylTgWB4icdQaKLm5LB-DvZRgG1VUt4TcXfJbsiZnqvAbqruwq5ysLJIn8nNxCSF6lqaxQ8eE_B5tAWA7hNkgkW1S7gT_hflgRJZZV4CV1NjvhHgmeAvB6bY-7x7U-VF3vsKw Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:03:58.015493 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c568082c4b344fbdb179d91ffab1c5bb"}]}] Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4fbf4eb1ec34c7d89f30efe61c0959a Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: exception: Image not found. Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'image_id': 'd30c29c6-fefd-4f21-a821-9b3ad04caca2', 'id': 'os_version'} Sep 27 10:03:58.017391 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:58.018456 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID d30c29c6-fefd-4f21-a821-9b3ad04caca2 Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:58.021069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image d30c29c6-fefd-4f21-a821-9b3ad04caca2 could not be found. Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:03:58.023118 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 58, in show Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi metadata = self._get_image(context, image_id)['properties'] Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.025834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6b7a9516-141b-4820-a190-3af2918f434b tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Returning 404 to user: Image not found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:58.027492 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6b7a9516-141b-4820-a190-3af2918f434b tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] 10.222.0.26 "GET /compute/v2.1/images/d30c29c6-fefd-4f21-a821-9b3ad04caca2/metadata/os_version" status: 404 len: 62 microversion: 2.1 time: 0.045167 Sep 27 10:03:58.027492 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 858/1885] 10.222.0.26 () {58 vars in 1293 bytes} [Tue Sep 27 10:03:57 2022] GET /compute/v2.1/images/d30c29c6-fefd-4f21-a821-9b3ad04caca2/metadata/os_version => generated 62 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Sep 27 10:03:58.027724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:58.028358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:58.035963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:58.035963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:58.036269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:58.036821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:58.037367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:58.037820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:58.094483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2b2551dc-9b56-4dd8-8e9b-d97e6ab12044 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:58.116302 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2b2551dc-9b56-4dd8-8e9b-d97e6ab12044 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1314 microversion: 2.1 time: 0.214636 Sep 27 10:03:58.117711 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1028/1886] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:57 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1314 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:58.124005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9873f30-be3a-404d-8822-55d36cc51ec3 tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Calling method '.version_select at 0x7f7f62ff7670>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:58.156917 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b9873f30-be3a-404d-8822-55d36cc51ec3 tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] HTTP exception thrown: Image not found. Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9873f30-be3a-404d-8822-55d36cc51ec3 tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Request method failure captured: Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/images/3ee777a7-cebd-4a9a-97ee-0b30f8f93d47/metadata HTTP/1.1 Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsqNqVWsJd3bcnIJ5UMQ0u6omIPVr113A3D2vGcajBOylTgWB4icdQaKLm5LB-DvZRgG1VUt4TcXfJbsiZnqvAbqruwq5ysLJIn8nNxCSF6lqaxQ8eE_B5tAWA7hNkgkW1S7gT_hflgRJZZV4CV1NjvhHgmeAvB6bY-7x7U-VF3vsKw Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:03:58.161281 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c568082c4b344fbdb179d91ffab1c5bb"}]}] Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b4fbf4eb1ec34c7d89f30efe61c0959a Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: exception: Image not found. Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'image_id': '3ee777a7-cebd-4a9a-97ee-0b30f8f93d47'} Sep 27 10:03:58.163433 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:58.164814 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 3ee777a7-cebd-4a9a-97ee-0b30f8f93d47 Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:58.166633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 3ee777a7-cebd-4a9a-97ee-0b30f8f93d47 could not be found. Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:03:58.168615 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 51, in index Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi metadata = self._get_image(context, image_id)['properties'] Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.170293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9873f30-be3a-404d-8822-55d36cc51ec3 tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Returning 404 to user: Image not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:58.171758 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b9873f30-be3a-404d-8822-55d36cc51ec3 tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] 10.222.0.26 "GET /compute/v2.1/images/3ee777a7-cebd-4a9a-97ee-0b30f8f93d47/metadata" status: 404 len: 62 microversion: 2.1 time: 0.046000 Sep 27 10:03:58.177713 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1029/1887] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:03:58 2022] GET /compute/v2.1/images/3ee777a7-cebd-4a9a-97ee-0b30f8f93d47/metadata => generated 62 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:03:58.184021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5daa80ea-8a05-4476-a395-4a12e5b57fb0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:58.184476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5daa80ea-8a05-4476-a395-4a12e5b57fb0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:58.193840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5daa80ea-8a05-4476-a395-4a12e5b57fb0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:58.194393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5daa80ea-8a05-4476-a395-4a12e5b57fb0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:58.194817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5daa80ea-8a05-4476-a395-4a12e5b57fb0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:58.196045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5daa80ea-8a05-4476-a395-4a12e5b57fb0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:58.196209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5daa80ea-8a05-4476-a395-4a12e5b57fb0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:58.196894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5daa80ea-8a05-4476-a395-4a12e5b57fb0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:58.307440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5daa80ea-8a05-4476-a395-4a12e5b57fb0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:58.319609 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5daa80ea-8a05-4476-a395-4a12e5b57fb0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1151 microversion: 2.1 time: 0.140616 Sep 27 10:03:58.320329 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1030/1888] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:58 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1151 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:58.337907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf6e993a-b7c3-4428-aff3-aeb13e94c0d7 tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Action: 'update_all', calling method: .version_select at 0x7f7f6c17de50>, body: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:58.366183 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-bf6e993a-b7c3-4428-aff3-aeb13e94c0d7 tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] HTTP exception thrown: Image not found. Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf6e993a-b7c3-4428-aff3-aeb13e94c0d7 tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Request method failure captured: Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: request: PUT /compute/v2.1/images/f89556e6-6e0c-42e3-92a3-e5399bc15735/metadata HTTP/1.1 Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 57 Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsqNqVWsJd3bcnIJ5UMQ0u6omIPVr113A3D2vGcajBOylTgWB4icdQaKLm5LB-DvZRgG1VUt4TcXfJbsiZnqvAbqruwq5ysLJIn8nNxCSF6lqaxQ8eE_B5tAWA7hNkgkW1S7gT_hflgRJZZV4CV1NjvhHgmeAvB6bY-7x7U-VF3vsKw Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:03:58.368898 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c568082c4b344fbdb179d91ffab1c5bb"}]}] Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b4fbf4eb1ec34c7d89f30efe61c0959a Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:03:58.370821 ncdv-master-1 devstack@n-api.service[145968]: exception: Image not found. Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'image_id': 'f89556e6-6e0c-42e3-92a3-e5399bc15735', 'body': {'metadata': {'os_distro': 'alt1', 'os_version': 'alt2'}}} Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:58.371892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID f89556e6-6e0c-42e3-92a3-e5399bc15735 Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:58.373478 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image f89556e6-6e0c-42e3-92a3-e5399bc15735 could not be found. Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:58.375188 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 109, in update_all Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:03:58.376771 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.378487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf6e993a-b7c3-4428-aff3-aeb13e94c0d7 tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Returning 404 to user: Image not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:58.378601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:58.378721 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bf6e993a-b7c3-4428-aff3-aeb13e94c0d7 tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] 10.222.0.26 "PUT /compute/v2.1/images/f89556e6-6e0c-42e3-92a3-e5399bc15735/metadata" status: 404 len: 62 microversion: 2.1 time: 0.044317 Sep 27 10:03:58.379916 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1031/1889] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:03:58 2022] PUT /compute/v2.1/images/f89556e6-6e0c-42e3-92a3-e5399bc15735/metadata => generated 62 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:03:58.385280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:58.387337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:58.389452 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-39850637-dab8-472f-b015-eecb5637d069 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.368533 Sep 27 10:03:58.390166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3cfbd8ea-61b8-4e13-ae55-c0c61b2f93cc tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Action: 'update', calling method: .version_select at 0x7f7f630b0040>, body: {"meta": {"os_distro": "alt"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:58.397264 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 859/1890] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:03:58 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:03:58.426103 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-3cfbd8ea-61b8-4e13-ae55-c0c61b2f93cc tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] HTTP exception thrown: Image not found. Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3cfbd8ea-61b8-4e13-ae55-c0c61b2f93cc tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Request method failure captured: Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: request: PUT /compute/v2.1/images/05d6d363-eaff-4de8-9e0e-5f66eac06dbd/metadata/os_distro HTTP/1.1 Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 30 Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsqNqVWsJd3bcnIJ5UMQ0u6omIPVr113A3D2vGcajBOylTgWB4icdQaKLm5LB-DvZRgG1VUt4TcXfJbsiZnqvAbqruwq5ysLJIn8nNxCSF6lqaxQ8eE_B5tAWA7hNkgkW1S7gT_hflgRJZZV4CV1NjvhHgmeAvB6bY-7x7U-VF3vsKw Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:03:58.428798 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c568082c4b344fbdb179d91ffab1c5bb"}]}] Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b4fbf4eb1ec34c7d89f30efe61c0959a Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: {"meta": {"os_distro": "alt"}} Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: exception: Image not found. Sep 27 10:03:58.431298 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'image_id': '05d6d363-eaff-4de8-9e0e-5f66eac06dbd', 'id': 'os_distro', 'body': {'meta': {'os_distro': 'alt'}}} Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:58.432787 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 05d6d363-eaff-4de8-9e0e-5f66eac06dbd Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:58.436371 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 05d6d363-eaff-4de8-9e0e-5f66eac06dbd could not be found. Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:58.439234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 93, in update Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:03:58.440809 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.443564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3cfbd8ea-61b8-4e13-ae55-c0c61b2f93cc tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Returning 404 to user: Image not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:58.443564 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3cfbd8ea-61b8-4e13-ae55-c0c61b2f93cc tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] 10.222.0.26 "PUT /compute/v2.1/images/05d6d363-eaff-4de8-9e0e-5f66eac06dbd/metadata/os_distro" status: 404 len: 62 microversion: 2.1 time: 0.044444 Sep 27 10:03:58.443564 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1032/1891] 10.222.0.26 () {60 vars in 1310 bytes} [Tue Sep 27 10:03:58 2022] PUT /compute/v2.1/images/05d6d363-eaff-4de8-9e0e-5f66eac06dbd/metadata/os_distro => generated 62 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:03:58.450931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-27d8caf3-4b3c-48e7-bd6f-cb4d306f97cf tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Action: 'create', calling method: .version_select at 0x7f7f62e2fca0>, body: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:58.483907 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-27d8caf3-4b3c-48e7-bd6f-cb4d306f97cf tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] HTTP exception thrown: Image not found. Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-27d8caf3-4b3c-48e7-bd6f-cb4d306f97cf tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Request method failure captured: Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/images/ea820479-9da0-4c07-a23b-df8b8a5d6c74/metadata HTTP/1.1 Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 57 Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsqNqVWsJd3bcnIJ5UMQ0u6omIPVr113A3D2vGcajBOylTgWB4icdQaKLm5LB-DvZRgG1VUt4TcXfJbsiZnqvAbqruwq5ysLJIn8nNxCSF6lqaxQ8eE_B5tAWA7hNkgkW1S7gT_hflgRJZZV4CV1NjvhHgmeAvB6bY-7x7U-VF3vsKw Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:03:58.488334 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c568082c4b344fbdb179d91ffab1c5bb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c568082c4b344fbdb179d91ffab1c5bb"}]}] Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: c568082c4b344fbdb179d91ffab1c5bb Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986 Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4fbf4eb1ec34c7d89f30efe61c0959a Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesMetadataNegativeTestJSON-2121051986-project Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: {"metadata": {"os_distro": "alt1", "os_version": "alt2"}} Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: exception: Image not found. Sep 27 10:03:58.490943 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'image_id': 'ea820479-9da0-4c07-a23b-df8b8a5d6c74', 'body': {'metadata': {'os_distro': 'alt1', 'os_version': 'alt2'}}} Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:58.492305 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID ea820479-9da0-4c07-a23b-df8b8a5d6c74 Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 39, in _get_image Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.image_api.get(context, image_id) Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:03:58.495884 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image ea820479-9da0-4c07-a23b-df8b8a5d6c74 could not be found. Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:03:58.498471 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 69, in create Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._get_image(context, image_id) Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/image_metadata.py", line 44, in _get_image Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:03:58.500879 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:03:58.504603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-27d8caf3-4b3c-48e7-bd6f-cb4d306f97cf tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] Returning 404 to user: Image not found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:03:58.506012 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-27d8caf3-4b3c-48e7-bd6f-cb4d306f97cf tempest-ImagesMetadataNegativeTestJSON-2121051986 tempest-ImagesMetadataNegativeTestJSON-2121051986-project] 10.222.0.26 "POST /compute/v2.1/images/ea820479-9da0-4c07-a23b-df8b8a5d6c74/metadata" status: 404 len: 62 microversion: 2.1 time: 0.064568 Sep 27 10:03:58.509439 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 860/1892] 10.222.0.26 () {60 vars in 1281 bytes} [Tue Sep 27 10:03:58 2022] POST /compute/v2.1/images/ea820479-9da0-4c07-a23b-df8b8a5d6c74/metadata => generated 62 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:03:58.520858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-03a0c9e2-5357-416f-8b50-d3e36ccdec29 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:58.521835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-03a0c9e2-5357-416f-8b50-d3e36ccdec29 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:58.529583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a0c9e2-5357-416f-8b50-d3e36ccdec29 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:58.530339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a0c9e2-5357-416f-8b50-d3e36ccdec29 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:58.531466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a0c9e2-5357-416f-8b50-d3e36ccdec29 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:58.533158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a0c9e2-5357-416f-8b50-d3e36ccdec29 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:58.534223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a0c9e2-5357-416f-8b50-d3e36ccdec29 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:58.534930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a0c9e2-5357-416f-8b50-d3e36ccdec29 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:58.619702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-81edf50f-ed6f-4095-8394-541484733faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:58.620316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-81edf50f-ed6f-4095-8394-541484733faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:58.629057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81edf50f-ed6f-4095-8394-541484733faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:58.629953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81edf50f-ed6f-4095-8394-541484733faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:58.630637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81edf50f-ed6f-4095-8394-541484733faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:58.632022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81edf50f-ed6f-4095-8394-541484733faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:58.632843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81edf50f-ed6f-4095-8394-541484733faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:58.634766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81edf50f-ed6f-4095-8394-541484733faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:58.710340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03a0c9e2-5357-416f-8b50-d3e36ccdec29 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:58.723202 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-03a0c9e2-5357-416f-8b50-d3e36ccdec29 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1483 microversion: 2.1 time: 0.207515 Sep 27 10:03:58.724409 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1033/1893] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:58 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1483 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:58.835821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d63497a-e9cb-46c8-b4f1-70cbc841479a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:58.836528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2d63497a-e9cb-46c8-b4f1-70cbc841479a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:58.845195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d63497a-e9cb-46c8-b4f1-70cbc841479a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:58.847985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d63497a-e9cb-46c8-b4f1-70cbc841479a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:58.848888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d63497a-e9cb-46c8-b4f1-70cbc841479a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:58.850750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d63497a-e9cb-46c8-b4f1-70cbc841479a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:58.851958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d63497a-e9cb-46c8-b4f1-70cbc841479a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:58.852478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d63497a-e9cb-46c8-b4f1-70cbc841479a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:58.955770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-81edf50f-ed6f-4095-8394-541484733faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:58.970437 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-81edf50f-ed6f-4095-8394-541484733faf tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.357582 Sep 27 10:03:58.973017 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 861/1894] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:58 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:58.978492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-8e048ae7-f165-42cb-b416-03a1c067f24d req-c815065a-87ea-44ce-909d-96f9886a107e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3a955d66-3892-4838-ad7d-cc9e431a03e1", "name": "network-vif-deleted", "tag": "b4f78072-6743-4c1d-bb89-c5bb42c81dcf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:59.012783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8e048ae7-f165-42cb-b416-03a1c067f24d req-c815065a-87ea-44ce-909d-96f9886a107e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:59.013306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8e048ae7-f165-42cb-b416-03a1c067f24d req-c815065a-87ea-44ce-909d-96f9886a107e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:59.013769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8e048ae7-f165-42cb-b416-03a1c067f24d req-c815065a-87ea-44ce-909d-96f9886a107e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:59.047710 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8e048ae7-f165-42cb-b416-03a1c067f24d req-c815065a-87ea-44ce-909d-96f9886a107e service nova] Creating event network-vif-deleted:b4f78072-6743-4c1d-bb89-c5bb42c81dcf for instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 on nch1-master-1 Sep 27 10:03:59.062581 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-8e048ae7-f165-42cb-b416-03a1c067f24d req-c815065a-87ea-44ce-909d-96f9886a107e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085469 Sep 27 10:03:59.068924 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 862/1895] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:59.148135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d75f2d49-cf3b-4f72-b204-eb42d9a57f94 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:59.149412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d75f2d49-cf3b-4f72-b204-eb42d9a57f94 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:59.165382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d75f2d49-cf3b-4f72-b204-eb42d9a57f94 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:59.166421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d75f2d49-cf3b-4f72-b204-eb42d9a57f94 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:59.167330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d75f2d49-cf3b-4f72-b204-eb42d9a57f94 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:59.168990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d75f2d49-cf3b-4f72-b204-eb42d9a57f94 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:59.169710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d75f2d49-cf3b-4f72-b204-eb42d9a57f94 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:59.170542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d75f2d49-cf3b-4f72-b204-eb42d9a57f94 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:59.231876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2d63497a-e9cb-46c8-b4f1-70cbc841479a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:59.245042 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d63497a-e9cb-46c8-b4f1-70cbc841479a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1475 microversion: 2.1 time: 0.414072 Sep 27 10:03:59.246758 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1034/1896] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:58 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1475 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:03:59.269401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-bf2a4667-1b16-42c9-96e0-4cd60b9a1bb2 req-1dd52965-de64-432f-acf2-ed6960747876 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ba4b5b70-d9bf-43b6-a52e-73be94b31bb1", "name": "network-vif-deleted", "tag": "14ebcc5e-9aae-4755-a891-78f55436d129"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:03:59.331607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bf2a4667-1b16-42c9-96e0-4cd60b9a1bb2 req-1dd52965-de64-432f-acf2-ed6960747876 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:59.336177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bf2a4667-1b16-42c9-96e0-4cd60b9a1bb2 req-1dd52965-de64-432f-acf2-ed6960747876 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:59.337996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bf2a4667-1b16-42c9-96e0-4cd60b9a1bb2 req-1dd52965-de64-432f-acf2-ed6960747876 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:59.378837 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-bf2a4667-1b16-42c9-96e0-4cd60b9a1bb2 req-1dd52965-de64-432f-acf2-ed6960747876 service nova] Creating event network-vif-deleted:14ebcc5e-9aae-4755-a891-78f55436d129 for instance ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 on nch1-master-1 Sep 27 10:03:59.397795 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-bf2a4667-1b16-42c9-96e0-4cd60b9a1bb2 req-1dd52965-de64-432f-acf2-ed6960747876 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.148059 Sep 27 10:03:59.401119 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1035/1897] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:03:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:03:59.409026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4ef628a3-588d-4c6f-815b-78f336d52b2f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:59.410040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4ef628a3-588d-4c6f-815b-78f336d52b2f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:59.478341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ef628a3-588d-4c6f-815b-78f336d52b2f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:59.481535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ef628a3-588d-4c6f-815b-78f336d52b2f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:59.484087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ef628a3-588d-4c6f-815b-78f336d52b2f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:59.489101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ef628a3-588d-4c6f-815b-78f336d52b2f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:59.491390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ef628a3-588d-4c6f-815b-78f336d52b2f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:59.495391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ef628a3-588d-4c6f-815b-78f336d52b2f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:59.781717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4ef628a3-588d-4c6f-815b-78f336d52b2f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:59.817706 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4ef628a3-588d-4c6f-815b-78f336d52b2f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1169 microversion: 2.1 time: 0.414568 Sep 27 10:03:59.818314 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1036/1898] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:59 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1169 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:59.849988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:59.853035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:59.866400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:59.867087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:59.867843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:59.869731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:59.872042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:59.872892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:59.894395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d75f2d49-cf3b-4f72-b204-eb42d9a57f94 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:03:59.906205 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d75f2d49-cf3b-4f72-b204-eb42d9a57f94 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1314 microversion: 2.1 time: 0.764225 Sep 27 10:03:59.906728 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 863/1899] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:59 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1314 bytes in 766 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:03:59.913929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:03:59.914991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:03:59.921782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:59.922551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:59.923169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:59.924087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:59.924748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:59.925412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:59.985075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:03:59.985751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:03:59.989749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:03:59.990405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lazy-loading 'fault' on Instance uuid 3a955d66-3892-4838-ad7d-cc9e431a03e1 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:00.158586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:00.172808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:00.177691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:00.182406 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6d5ad245-8021-4196-b646-4a499f42b5bc tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.344990 Sep 27 10:04:00.183520 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1037/1900] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:03:59 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:00.194888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c7468cc8bc7d4dde8e2a5eba1fa42fd8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'de87d5984a9a4d4481ab4921133841d6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:00.196713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-2c84bbc7-63a4-4941-9101-f2dddbf5a3c0 req-6b676f97-995b-4a84-9e75-fdbc8aaf9794 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "14bb2db7-2b1d-46d4-8800-26b149825654", "name": "network-vif-plugged", "status": "completed", "tag": "dc496928-7dd4-48f7-a375-66707f33ecf8"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:00.215946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2c84bbc7-63a4-4941-9101-f2dddbf5a3c0 req-6b676f97-995b-4a84-9e75-fdbc8aaf9794 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:00.216767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2c84bbc7-63a4-4941-9101-f2dddbf5a3c0 req-6b676f97-995b-4a84-9e75-fdbc8aaf9794 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:00.217652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2c84bbc7-63a4-4941-9101-f2dddbf5a3c0 req-6b676f97-995b-4a84-9e75-fdbc8aaf9794 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:00.221785 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5f410fd4-2d34-4c35-bce4-766b0650ece5 tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 200 len: 1314 microversion: 2.1 time: 0.308898 Sep 27 10:04:00.225251 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 864/1901] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:03:59 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 1314 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:00.245963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d52c4913-dbad-400c-ad2d-4ebfe79a38c7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:00.246878 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-2c84bbc7-63a4-4941-9101-f2dddbf5a3c0 req-6b676f97-995b-4a84-9e75-fdbc8aaf9794 service nova] Creating event network-vif-plugged:dc496928-7dd4-48f7-a375-66707f33ecf8 for instance 14bb2db7-2b1d-46d4-8800-26b149825654 on nch2-master-1 Sep 27 10:04:00.253126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d52c4913-dbad-400c-ad2d-4ebfe79a38c7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:00.256287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-2c84bbc7-63a4-4941-9101-f2dddbf5a3c0 req-6b676f97-995b-4a84-9e75-fdbc8aaf9794 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065785 Sep 27 10:04:00.258467 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1038/1902] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:00.274297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d52c4913-dbad-400c-ad2d-4ebfe79a38c7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:00.276427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:00.278376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d52c4913-dbad-400c-ad2d-4ebfe79a38c7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:00.278620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:00.280558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d52c4913-dbad-400c-ad2d-4ebfe79a38c7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:00.282864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d52c4913-dbad-400c-ad2d-4ebfe79a38c7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:00.285318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d52c4913-dbad-400c-ad2d-4ebfe79a38c7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:00.286302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d52c4913-dbad-400c-ad2d-4ebfe79a38c7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:00.304024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:00.313826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:00.317009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:00.318805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:00.321783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:00.322589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:00.434063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:00.436346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:00.436971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:00.437653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lazy-loading 'fault' on Instance uuid ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:00.527404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:00.556042 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-132299ac-686b-47ab-84c4-66ced1274498 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 200 len: 1311 microversion: 2.1 time: 0.289503 Sep 27 10:04:00.557116 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1039/1903] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:00 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 1311 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:00.846113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9eedc80-fe69-4572-933b-d02cec060131 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:00.850018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c9eedc80-fe69-4572-933b-d02cec060131 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:00.855264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9eedc80-fe69-4572-933b-d02cec060131 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:00.856483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9eedc80-fe69-4572-933b-d02cec060131 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:00.857248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9eedc80-fe69-4572-933b-d02cec060131 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:00.859204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9eedc80-fe69-4572-933b-d02cec060131 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:00.859800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9eedc80-fe69-4572-933b-d02cec060131 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:00.860347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9eedc80-fe69-4572-933b-d02cec060131 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:00.923512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d52c4913-dbad-400c-ad2d-4ebfe79a38c7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:00.936699 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d52c4913-dbad-400c-ad2d-4ebfe79a38c7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.706196 Sep 27 10:04:00.939521 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 865/1904] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:00 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:00.951071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-452df115-4565-4b9d-9d06-2461e0071c4d tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:00.952272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-452df115-4565-4b9d-9d06-2461e0071c4d tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:00.966814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-452df115-4565-4b9d-9d06-2461e0071c4d tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:00.968171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-452df115-4565-4b9d-9d06-2461e0071c4d tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:00.970042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-452df115-4565-4b9d-9d06-2461e0071c4d tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:00.971082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-452df115-4565-4b9d-9d06-2461e0071c4d tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:00.975981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-452df115-4565-4b9d-9d06-2461e0071c4d tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:00.977192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-452df115-4565-4b9d-9d06-2461e0071c4d tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:01.023849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9eedc80-fe69-4572-933b-d02cec060131 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:01.042825 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9eedc80-fe69-4572-933b-d02cec060131 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1169 microversion: 2.1 time: 0.202730 Sep 27 10:04:01.051599 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1040/1905] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:00 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1169 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:01.220822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:01.224318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:01.253697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:01.254196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:01.259698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:01.260472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:01.261006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:01.261422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:01.496409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-452df115-4565-4b9d-9d06-2461e0071c4d tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:01.516926 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-452df115-4565-4b9d-9d06-2461e0071c4d tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1314 microversion: 2.1 time: 0.572291 Sep 27 10:04:01.517530 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 866/1906] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:00 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1314 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:01.541384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:01.548843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:01.550282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] [instance: 3a955d66-3892-4838-ad7d-cc9e431a03e1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:01.556145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:01.560984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:01.562329 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e7e83d0e-efe0-43ae-bdb0-e30a8a491857 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.348000 Sep 27 10:04:01.565852 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1041/1907] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:01 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:04:01.567016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:01.567691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:01.569759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:01.570986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:01.573892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-cae16bcb-1523-4f31-a6c4-c0f34ed83740 req-a6ba3569-d430-49dd-ae6f-6f784e6ac884 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fcd14928-8ef7-425b-89bb-2c714284e840", "name": "network-vif-unplugged", "status": "completed", "tag": "b4c8bc77-41d3-49f6-84f2-59d40e203a22"}, {"name": "network-changed", "server_uuid": "fcd14928-8ef7-425b-89bb-2c714284e840", "tag": "b4c8bc77-41d3-49f6-84f2-59d40e203a22"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:01.575468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:01.576125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:01.593477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cae16bcb-1523-4f31-a6c4-c0f34ed83740 req-a6ba3569-d430-49dd-ae6f-6f784e6ac884 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:01.594194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cae16bcb-1523-4f31-a6c4-c0f34ed83740 req-a6ba3569-d430-49dd-ae6f-6f784e6ac884 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:01.594913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cae16bcb-1523-4f31-a6c4-c0f34ed83740 req-a6ba3569-d430-49dd-ae6f-6f784e6ac884 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:01.628451 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-cae16bcb-1523-4f31-a6c4-c0f34ed83740 req-a6ba3569-d430-49dd-ae6f-6f784e6ac884 service nova] Creating event network-vif-unplugged:b4c8bc77-41d3-49f6-84f2-59d40e203a22 for instance fcd14928-8ef7-425b-89bb-2c714284e840 on nch2-master-1 Sep 27 10:04:01.629135 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-cae16bcb-1523-4f31-a6c4-c0f34ed83740 req-a6ba3569-d430-49dd-ae6f-6f784e6ac884 service nova] Creating event network-changed:b4c8bc77-41d3-49f6-84f2-59d40e203a22 for instance fcd14928-8ef7-425b-89bb-2c714284e840 on nch2-master-1 Sep 27 10:04:01.635389 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] HTTP exception thrown: Instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 could not be found. Sep 27 10:04:01.639738 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-cae16bcb-1523-4f31-a6c4-c0f34ed83740 req-a6ba3569-d430-49dd-ae6f-6f784e6ac884 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 352 microversion: 2.1 time: 0.071560 Sep 27 10:04:01.640764 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1042/1908] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:01 2022] POST /compute/v2.1/os-server-external-events => generated 352 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Request method failure captured: Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 HTTP/1.1 Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMso0VFJjEU5GfoTJmvmRWSj3r_ZSEES5ggmqM4nZlWEK900hPKDrqtEKyMz2MCicCfNf7rpaIQw-u6nipnwKAax2_dGjlwUHftSTX1xbyTI-AkFXK6miTzPu2A2Cr1HWm1A6ftJPMy0819xaVtNSoCE0LZNTLHxKFqqwB_JIN9zzIG4 Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: de87d5984a9a4d4481ab4921133841d6 Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:04:01.644138 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/de87d5984a9a4d4481ab4921133841d6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_de87d5984a9a4d4481ab4921133841d6"}]}] Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: de87d5984a9a4d4481ab4921133841d6 Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-FloatingIPsAssociationTestJSON-924459803 Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-FloatingIPsAssociationTestJSON-924459803-project Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: c7468cc8bc7d4dde8e2a5eba1fa42fd8 Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-FloatingIPsAssociationTestJSON-924459803-project Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 could not be found. Sep 27 10:04:01.646090 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3a955d66-3892-4838-ad7d-cc9e431a03e1'} Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 could not be found. Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:04:01.647253 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 could not be found. Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 could not be found. Sep 27 10:04:01.649403 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:04:01.651016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:01.651016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: ba4b5b70-d9bf-43b6-a52e-73be94b31bb1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:01.651723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] Returning 404 to user: Instance 3a955d66-3892-4838-ad7d-cc9e431a03e1 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:04:01.655593 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-08769d46-77b6-4911-9d5c-1658e15770ea tempest-FloatingIPsAssociationTestJSON-924459803 tempest-FloatingIPsAssociationTestJSON-924459803-project] 10.222.0.26 "GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1" status: 404 len: 111 microversion: 2.1 time: 0.115606 Sep 27 10:04:01.657280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:01.657779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:01.658215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:01.659072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:01.659576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:01.660054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:01.662362 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 867/1909] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:01 2022] GET /compute/v2.1/servers/3a955d66-3892-4838-ad7d-cc9e431a03e1 => generated 111 bytes in 144 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:01.682223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:01.684668 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] HTTP exception thrown: Instance ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 could not be found. Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Request method failure captured: Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 HTTP/1.1 Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMso_T5-k00r-jNMmt-sE3AO4w-bk9w_R-TIb97BkNgtX430OKum_e_1Ajxl5FN5vu42mVBl2t9CzMeiQzlqPlORJKPijU1tf0C6z8at9kYV7c7xulwZegzv6clP9NU3qX8u7EF1IsTY4NHcT_P_xJX9RRB_V2Pz10WEXih8yHjsEtCw Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f48a592eb8fa4221a56eac1a50152c91 Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ListServerFiltersTestJSON-809664979 Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:04:01.686986 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f48a592eb8fa4221a56eac1a50152c91"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f48a592eb8fa4221a56eac1a50152c91"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f48a592eb8fa4221a56eac1a50152c91"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f48a592eb8fa4221a56eac1a50152c91"}]}] Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ListServerFiltersTestJSON-809664979 Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f48a592eb8fa4221a56eac1a50152c91 Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ListServerFiltersTestJSON-809664979 Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ListServerFiltersTestJSON-809664979-project Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: ac98fc0d0296493999906e8e3607c7f3 Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ListServerFiltersTestJSON-809664979-project Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 could not be found. Sep 27 10:04:01.688803 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'ba4b5b70-d9bf-43b6-a52e-73be94b31bb1'} Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 could not be found. Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:04:01.689924 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 could not be found. Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 could not be found. Sep 27 10:04:01.691751 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:01.693442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Returning 404 to user: Instance ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:04:01.693442 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f8ec32b6-3252-4ea3-8e20-f7b61a1ecd7b tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1" status: 404 len: 111 microversion: 2.1 time: 0.047162 Sep 27 10:04:01.694397 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1043/1910] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:01 2022] GET /compute/v2.1/servers/ba4b5b70-d9bf-43b6-a52e-73be94b31bb1 => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:01.707822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-62cdbcd3-045a-4f0b-be48-32d858753d27 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:01.709355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-62cdbcd3-045a-4f0b-be48-32d858753d27 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:01.721046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62cdbcd3-045a-4f0b-be48-32d858753d27 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:01.728473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62cdbcd3-045a-4f0b-be48-32d858753d27 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:01.728473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62cdbcd3-045a-4f0b-be48-32d858753d27 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:01.728473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62cdbcd3-045a-4f0b-be48-32d858753d27 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:01.728473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62cdbcd3-045a-4f0b-be48-32d858753d27 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:01.729968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62cdbcd3-045a-4f0b-be48-32d858753d27 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:01.800461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:01.860075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-62cdbcd3-045a-4f0b-be48-32d858753d27 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:04:01.967621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6de512ee-26e0-4eca-8fc8-cd3105c0df67 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:01.968304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6de512ee-26e0-4eca-8fc8-cd3105c0df67 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:01.981007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6de512ee-26e0-4eca-8fc8-cd3105c0df67 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:01.982405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6de512ee-26e0-4eca-8fc8-cd3105c0df67 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:01.983296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6de512ee-26e0-4eca-8fc8-cd3105c0df67 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:01.984450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6de512ee-26e0-4eca-8fc8-cd3105c0df67 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:01.985315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6de512ee-26e0-4eca-8fc8-cd3105c0df67 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:01.986161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6de512ee-26e0-4eca-8fc8-cd3105c0df67 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:02.128196 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-62cdbcd3-045a-4f0b-be48-32d858753d27 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "DELETE /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 204 len: 0 microversion: 2.1 time: 0.431654 Sep 27 10:04:02.129703 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 868/1911] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:04:01 2022] DELETE /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:02.139161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-86275948-dd8d-408f-9078-a3ac8a818b91 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:02.140645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-86275948-dd8d-408f-9078-a3ac8a818b91 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:02.151021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-86275948-dd8d-408f-9078-a3ac8a818b91 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:02.151964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-86275948-dd8d-408f-9078-a3ac8a818b91 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:02.153331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-86275948-dd8d-408f-9078-a3ac8a818b91 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:02.154306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-86275948-dd8d-408f-9078-a3ac8a818b91 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:02.155190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-86275948-dd8d-408f-9078-a3ac8a818b91 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:02.155904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-86275948-dd8d-408f-9078-a3ac8a818b91 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:02.358861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6de512ee-26e0-4eca-8fc8-cd3105c0df67 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:02.368894 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6de512ee-26e0-4eca-8fc8-cd3105c0df67 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.410091 Sep 27 10:04:02.374037 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1044/1912] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:01 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:02.388212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d15fc5b4-fa0e-4e45-b182-13b16702e73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:02.389211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d15fc5b4-fa0e-4e45-b182-13b16702e73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:02.404044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d15fc5b4-fa0e-4e45-b182-13b16702e73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:02.404793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d15fc5b4-fa0e-4e45-b182-13b16702e73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:02.405438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d15fc5b4-fa0e-4e45-b182-13b16702e73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:02.411870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d15fc5b4-fa0e-4e45-b182-13b16702e73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:02.412826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d15fc5b4-fa0e-4e45-b182-13b16702e73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:02.413516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d15fc5b4-fa0e-4e45-b182-13b16702e73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:02.540510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-86275948-dd8d-408f-9078-a3ac8a818b91 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:02.549297 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-86275948-dd8d-408f-9078-a3ac8a818b91 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1211 microversion: 2.1 time: 0.416821 Sep 27 10:04:02.551851 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 869/1913] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:02 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1211 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:02.571979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5a5d176-3712-4f11-aa41-7c5128b52a9c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:02.572978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5a5d176-3712-4f11-aa41-7c5128b52a9c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:02.581094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5a5d176-3712-4f11-aa41-7c5128b52a9c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:02.581677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5a5d176-3712-4f11-aa41-7c5128b52a9c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:02.582186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5a5d176-3712-4f11-aa41-7c5128b52a9c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:02.583000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5a5d176-3712-4f11-aa41-7c5128b52a9c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:02.583542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5a5d176-3712-4f11-aa41-7c5128b52a9c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:02.584044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5a5d176-3712-4f11-aa41-7c5128b52a9c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:02.732480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d15fc5b4-fa0e-4e45-b182-13b16702e73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:02.745259 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d15fc5b4-fa0e-4e45-b182-13b16702e73a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 200 len: 1518 microversion: 2.1 time: 0.369760 Sep 27 10:04:02.762486 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1045/1914] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:02 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 1518 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:02.767235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:02.768292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:02.776197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:02.777129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:02.777749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:02.779019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:02.779644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:02.780405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:02.877285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5a5d176-3712-4f11-aa41-7c5128b52a9c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:02.888046 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5a5d176-3712-4f11-aa41-7c5128b52a9c tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1314 microversion: 2.1 time: 0.321645 Sep 27 10:04:02.888932 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 870/1915] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:02 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1314 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:03.062750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerTestJSON-server-2080223881", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:03.074328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:03.086297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:03.105326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:03.106583 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58e13d35-756f-46f0-8e1a-9303cd21e8e2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.348175 Sep 27 10:04:03.109055 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1046/1916] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:02 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:03.145535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:04:03.400704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-657d1eec-7cee-4b44-8d4f-24d431b758b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:03.401640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-657d1eec-7cee-4b44-8d4f-24d431b758b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:03.409882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-657d1eec-7cee-4b44-8d4f-24d431b758b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:03.410372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-657d1eec-7cee-4b44-8d4f-24d431b758b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:03.410813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-657d1eec-7cee-4b44-8d4f-24d431b758b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:03.411730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-657d1eec-7cee-4b44-8d4f-24d431b758b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:03.412198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-657d1eec-7cee-4b44-8d4f-24d431b758b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:03.412629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-657d1eec-7cee-4b44-8d4f-24d431b758b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:03.640522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-657d1eec-7cee-4b44-8d4f-24d431b758b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:03.652638 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-657d1eec-7cee-4b44-8d4f-24d431b758b3 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.256003 Sep 27 10:04:03.654000 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1047/1917] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:03 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:03.661063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-888e6c7c-a14d-4957-ab47-5579c4e0569f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:03.662172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-888e6c7c-a14d-4957-ab47-5579c4e0569f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:03.675053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-888e6c7c-a14d-4957-ab47-5579c4e0569f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:03.675734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-888e6c7c-a14d-4957-ab47-5579c4e0569f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:03.676225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-888e6c7c-a14d-4957-ab47-5579c4e0569f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:03.677043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-888e6c7c-a14d-4957-ab47-5579c4e0569f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:03.677521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-888e6c7c-a14d-4957-ab47-5579c4e0569f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:03.677947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-888e6c7c-a14d-4957-ab47-5579c4e0569f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:03.792035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:03.933301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-888e6c7c-a14d-4957-ab47-5579c4e0569f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:03.936163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:04:03.938850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:04:03.940053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:04:03.940724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:04:03.941875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:04:03.943307 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-888e6c7c-a14d-4957-ab47-5579c4e0569f tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1199 microversion: 2.1 time: 0.287586 Sep 27 10:04:03.958772 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1048/1918] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:03 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1199 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:03.965825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf625a4a-8cc7-473c-89d0-7db07cb76297 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:03.966676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bf625a4a-8cc7-473c-89d0-7db07cb76297 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:03.972134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:03.975092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:03.976422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf625a4a-8cc7-473c-89d0-7db07cb76297 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:03.977733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf625a4a-8cc7-473c-89d0-7db07cb76297 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:03.978388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf625a4a-8cc7-473c-89d0-7db07cb76297 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:03.979110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf625a4a-8cc7-473c-89d0-7db07cb76297 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:03.979568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:03.979963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf625a4a-8cc7-473c-89d0-7db07cb76297 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:03.980552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf625a4a-8cc7-473c-89d0-7db07cb76297 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:03.981646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:03.984363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:03.985260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:04.026700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Getting quotas for project 58c33b046dc7494c970f26eb5900b3f3. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:04.032360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Getting quotas for user a10600a9277a436396fc70971dd27893 and project 58c33b046dc7494c970f26eb5900b3f3. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:04.068575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:04:04.093464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:04:04.170603 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1c4c6aa0-be7a-4d96-854a-fcaeb7a9b260 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.279151 Sep 27 10:04:04.171563 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 871/1919] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:04:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 1282 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:04:04.188243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-47dd244c-96fe-4d4e-ad38-230bec3a57aa tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:04.189063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-47dd244c-96fe-4d4e-ad38-230bec3a57aa tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:04.204215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47dd244c-96fe-4d4e-ad38-230bec3a57aa tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:04.205020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47dd244c-96fe-4d4e-ad38-230bec3a57aa tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:04.205549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47dd244c-96fe-4d4e-ad38-230bec3a57aa tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:04.206285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47dd244c-96fe-4d4e-ad38-230bec3a57aa tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:04.206748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47dd244c-96fe-4d4e-ad38-230bec3a57aa tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:04.207237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47dd244c-96fe-4d4e-ad38-230bec3a57aa tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:04.726449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-47dd244c-96fe-4d4e-ad38-230bec3a57aa tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:04.737789 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-47dd244c-96fe-4d4e-ad38-230bec3a57aa tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1464 microversion: 2.1 time: 0.565163 Sep 27 10:04:04.739934 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 872/1920] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:04 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1464 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:04.747248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-99b9f43b-1a28-49d9-8326-81ecdde1d562 req-138d4457-0dce-4518-bd36-bddc4392f509 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bd544723-d1ca-4f7e-a231-ba0b8351464d", "tag": "f2d7b45b-7823-4ee9-a9a3-1e4c5cda1e27"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:04.759826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-99b9f43b-1a28-49d9-8326-81ecdde1d562 req-138d4457-0dce-4518-bd36-bddc4392f509 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:04.760604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-99b9f43b-1a28-49d9-8326-81ecdde1d562 req-138d4457-0dce-4518-bd36-bddc4392f509 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:04.761161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-99b9f43b-1a28-49d9-8326-81ecdde1d562 req-138d4457-0dce-4518-bd36-bddc4392f509 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:04.765748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bf625a4a-8cc7-473c-89d0-7db07cb76297 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:04.778808 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bf625a4a-8cc7-473c-89d0-7db07cb76297 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 200 len: 1518 microversion: 2.1 time: 0.817730 Sep 27 10:04:04.783442 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-99b9f43b-1a28-49d9-8326-81ecdde1d562 req-138d4457-0dce-4518-bd36-bddc4392f509 service nova] Creating event network-changed:f2d7b45b-7823-4ee9-a9a3-1e4c5cda1e27 for instance bd544723-d1ca-4f7e-a231-ba0b8351464d on nch1-master-1 Sep 27 10:04:04.789296 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-99b9f43b-1a28-49d9-8326-81ecdde1d562 req-138d4457-0dce-4518-bd36-bddc4392f509 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047400 Sep 27 10:04:04.791128 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 873/1921] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:04.799501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:04.801231 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1049/1922] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:03 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 1518 bytes in 841 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:04.801548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:04.809164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13170d16-056d-402c-855d-849b253f6cc5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:04.809831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:04.811005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-13170d16-056d-402c-855d-849b253f6cc5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:04.811237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:04.812383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:04.813481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:04.814170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:04.814920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:04.873261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=16 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:04.892580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-13170d16-056d-402c-855d-849b253f6cc5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:04.902182 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13170d16-056d-402c-855d-849b253f6cc5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1217 microversion: 2.1 time: 0.099061 Sep 27 10:04:04.903676 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1050/1923] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:04 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1217 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:04.910857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-282dd3c4-f66d-45bf-a137-0244918da48c req-b379ee03-89d7-4b9b-9ff0-d12699699989 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "152a5758-d14b-4ca9-917e-f7017c6d99b2", "name": "network-vif-plugged", "status": "completed", "tag": "61c12b2f-7268-4252-a482-600cb57542ca"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:04.945959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-282dd3c4-f66d-45bf-a137-0244918da48c req-b379ee03-89d7-4b9b-9ff0-d12699699989 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:04.946842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-282dd3c4-f66d-45bf-a137-0244918da48c req-b379ee03-89d7-4b9b-9ff0-d12699699989 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:04.950377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-282dd3c4-f66d-45bf-a137-0244918da48c req-b379ee03-89d7-4b9b-9ff0-d12699699989 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:04.974022 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-282dd3c4-f66d-45bf-a137-0244918da48c req-b379ee03-89d7-4b9b-9ff0-d12699699989 service nova] Creating event network-vif-plugged:61c12b2f-7268-4252-a482-600cb57542ca for instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 on nch2-master-1 Sep 27 10:04:04.981889 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-282dd3c4-f66d-45bf-a137-0244918da48c req-b379ee03-89d7-4b9b-9ff0-d12699699989 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075945 Sep 27 10:04:04.982690 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1051/1924] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:04.992259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a649f5dc-1178-4ffa-aaca-a55bf7ee8a7b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:04.993676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a649f5dc-1178-4ffa-aaca-a55bf7ee8a7b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:05.006227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a649f5dc-1178-4ffa-aaca-a55bf7ee8a7b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.006800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a649f5dc-1178-4ffa-aaca-a55bf7ee8a7b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.007248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a649f5dc-1178-4ffa-aaca-a55bf7ee8a7b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.009694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a649f5dc-1178-4ffa-aaca-a55bf7ee8a7b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.010167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a649f5dc-1178-4ffa-aaca-a55bf7ee8a7b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.021577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a649f5dc-1178-4ffa-aaca-a55bf7ee8a7b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.359681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:05.371602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:05.376217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:05.378132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a649f5dc-1178-4ffa-aaca-a55bf7ee8a7b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:05.380235 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1fd6635e-a966-4280-8a9c-04dbbfc7a5e8 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.586032 Sep 27 10:04:05.396961 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a649f5dc-1178-4ffa-aaca-a55bf7ee8a7b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.399843 Sep 27 10:04:05.396961 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1052/1925] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:04 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:05.398112 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 874/1926] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:04 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:05.405745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-297f8175-6e79-4002-b9fb-36ac20636119 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:05.407160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-297f8175-6e79-4002-b9fb-36ac20636119 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:05.414201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-297f8175-6e79-4002-b9fb-36ac20636119 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.414950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-297f8175-6e79-4002-b9fb-36ac20636119 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.415586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-297f8175-6e79-4002-b9fb-36ac20636119 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.416512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-297f8175-6e79-4002-b9fb-36ac20636119 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.417175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-297f8175-6e79-4002-b9fb-36ac20636119 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.417773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-297f8175-6e79-4002-b9fb-36ac20636119 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.570112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b32855a7-7d2b-4c5f-9eed-c5d7925744f9 tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Calling method '.version_select at 0x7f7f6c29b3a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:05.593303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b32855a7-7d2b-4c5f-9eed-c5d7925744f9 tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.606078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b32855a7-7d2b-4c5f-9eed-c5d7925744f9 tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.610749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b32855a7-7d2b-4c5f-9eed-c5d7925744f9 tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.638962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b32855a7-7d2b-4c5f-9eed-c5d7925744f9 tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.639852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b32855a7-7d2b-4c5f-9eed-c5d7925744f9 tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.640471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b32855a7-7d2b-4c5f-9eed-c5d7925744f9 tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.692938 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b32855a7-7d2b-4c5f-9eed-c5d7925744f9 tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] 10.222.0.26 "GET /compute/v2.1/os-simple-tenant-usage/8b87f09e1bf645a4b71d8b31ee512ec0?start=2022-09-26T10%3A04%3A04.755995&end=2022-09-28T10%3A04%3A04.755995" status: 200 len: 695 microversion: 2.1 time: 0.303113 Sep 27 10:04:05.693995 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1053/1927] 10.222.0.26 () {58 vars in 1412 bytes} [Tue Sep 27 10:04:05 2022] GET /compute/v2.1/os-simple-tenant-usage/8b87f09e1bf645a4b71d8b31ee512ec0?start=2022-09-26T10%3A04%3A04.755995&end=2022-09-28T10%3A04%3A04.755995 => generated 695 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:05.718271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5beae3d4-56f3-4dc4-8c48-c58e962be472 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '.version_select at 0x7f7f6c178310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:05.732844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5beae3d4-56f3-4dc4-8c48-c58e962be472 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.733480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5beae3d4-56f3-4dc4-8c48-c58e962be472 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.734363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5beae3d4-56f3-4dc4-8c48-c58e962be472 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.745323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-297f8175-6e79-4002-b9fb-36ac20636119 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:05.748468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5beae3d4-56f3-4dc4-8c48-c58e962be472 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.749192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5beae3d4-56f3-4dc4-8c48-c58e962be472 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.749830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5beae3d4-56f3-4dc4-8c48-c58e962be472 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.759033 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-297f8175-6e79-4002-b9fb-36ac20636119 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1199 microversion: 2.1 time: 0.358614 Sep 27 10:04:05.759710 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 875/1928] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:05 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1199 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:05.779990 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5beae3d4-56f3-4dc4-8c48-c58e962be472 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/os-simple-tenant-usage/8b87f09e1bf645a4b71d8b31ee512ec0?start=2022-09-26T10%3A04%3A04.755995&end=2022-09-28T10%3A04%3A04.755995" status: 200 len: 695 microversion: 2.1 time: 0.069084 Sep 27 10:04:05.792477 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1054/1929] 10.222.0.26 () {58 vars in 1412 bytes} [Tue Sep 27 10:04:05 2022] GET /compute/v2.1/os-simple-tenant-usage/8b87f09e1bf645a4b71d8b31ee512ec0?start=2022-09-26T10%3A04%3A04.755995&end=2022-09-28T10%3A04%3A04.755995 => generated 695 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:04:05.797131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e800f085-2e51-4f40-a7f0-afbfc826b1cd tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Calling method '.version_select at 0x7f7f62eb0b80>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:05.808941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58c72fcf-f2a8-4b87-b357-9a2495979383 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:05.809859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58c72fcf-f2a8-4b87-b357-9a2495979383 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:05.814906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e800f085-2e51-4f40-a7f0-afbfc826b1cd tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.816629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e800f085-2e51-4f40-a7f0-afbfc826b1cd tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.817279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e800f085-2e51-4f40-a7f0-afbfc826b1cd tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.817959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c72fcf-f2a8-4b87-b357-9a2495979383 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.818788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c72fcf-f2a8-4b87-b357-9a2495979383 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.819365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c72fcf-f2a8-4b87-b357-9a2495979383 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.820195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c72fcf-f2a8-4b87-b357-9a2495979383 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.820891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c72fcf-f2a8-4b87-b357-9a2495979383 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.821583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58c72fcf-f2a8-4b87-b357-9a2495979383 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.853928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e800f085-2e51-4f40-a7f0-afbfc826b1cd tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.857990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e800f085-2e51-4f40-a7f0-afbfc826b1cd tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.858535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e800f085-2e51-4f40-a7f0-afbfc826b1cd tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.968424 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e800f085-2e51-4f40-a7f0-afbfc826b1cd tempest-TenantUsagesTestJSON-1774494104 tempest-TenantUsagesTestJSON-1774494104-project-admin] 10.222.0.26 "GET /compute/v2.1/os-simple-tenant-usage?start=2022-09-26T10%3A04%3A04.755995&end=2022-09-28T10%3A04%3A04.755995&detailed=1" status: 200 len: 22477 microversion: 2.1 time: 0.175118 Sep 27 10:04:05.968752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58c72fcf-f2a8-4b87-b357-9a2495979383 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:05.969542 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 876/1930] 10.222.0.26 () {58 vars in 1335 bytes} [Tue Sep 27 10:04:05 2022] GET /compute/v2.1/os-simple-tenant-usage?start=2022-09-26T10%3A04%3A04.755995&end=2022-09-28T10%3A04%3A04.755995&detailed=1 => generated 22477 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 10:04:05.975220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d07341d5-3abe-4eab-a4f9-8a3f1b5bdde9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:05.975833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d07341d5-3abe-4eab-a4f9-8a3f1b5bdde9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:05.983142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d07341d5-3abe-4eab-a4f9-8a3f1b5bdde9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.983845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d07341d5-3abe-4eab-a4f9-8a3f1b5bdde9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.984285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d07341d5-3abe-4eab-a4f9-8a3f1b5bdde9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.985044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d07341d5-3abe-4eab-a4f9-8a3f1b5bdde9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:05.985495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d07341d5-3abe-4eab-a4f9-8a3f1b5bdde9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:05.985924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d07341d5-3abe-4eab-a4f9-8a3f1b5bdde9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:05.994244 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58c72fcf-f2a8-4b87-b357-9a2495979383 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 200 len: 1476 microversion: 2.1 time: 0.190574 Sep 27 10:04:06.010225 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1055/1931] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:05 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 1476 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:06.018715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-036353f3-7427-4cb1-b141-cd67be48481f tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:06.019637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-036353f3-7427-4cb1-b141-cd67be48481f tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:06.024056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:06.028529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-036353f3-7427-4cb1-b141-cd67be48481f tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:06.029249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-036353f3-7427-4cb1-b141-cd67be48481f tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:06.029794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-036353f3-7427-4cb1-b141-cd67be48481f tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:06.030834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-036353f3-7427-4cb1-b141-cd67be48481f tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:06.031530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-036353f3-7427-4cb1-b141-cd67be48481f tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:06.032082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-036353f3-7427-4cb1-b141-cd67be48481f tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:06.068230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d07341d5-3abe-4eab-a4f9-8a3f1b5bdde9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:06.075287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-036353f3-7427-4cb1-b141-cd67be48481f tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:04:06.090668 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d07341d5-3abe-4eab-a4f9-8a3f1b5bdde9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1269 microversion: 2.1 time: 0.119578 Sep 27 10:04:06.091410 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 877/1932] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:05 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1269 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:06.213134 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-036353f3-7427-4cb1-b141-cd67be48481f tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "DELETE /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 204 len: 0 microversion: 2.1 time: 0.201072 Sep 27 10:04:06.214010 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1056/1933] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:04:06 2022] DELETE /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:06.222057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5d7674fb-b96b-4d89-9318-6affa82348a0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:06.223118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5d7674fb-b96b-4d89-9318-6affa82348a0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:06.231140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d7674fb-b96b-4d89-9318-6affa82348a0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:06.232651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d7674fb-b96b-4d89-9318-6affa82348a0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:06.233318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d7674fb-b96b-4d89-9318-6affa82348a0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:06.234047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d7674fb-b96b-4d89-9318-6affa82348a0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:06.234503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d7674fb-b96b-4d89-9318-6affa82348a0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:06.234929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d7674fb-b96b-4d89-9318-6affa82348a0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:06.412411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4fa3e3d4-a7e7-45ca-9e27-b71fa28b965b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:06.413422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4fa3e3d4-a7e7-45ca-9e27-b71fa28b965b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:06.430349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fa3e3d4-a7e7-45ca-9e27-b71fa28b965b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:06.430349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fa3e3d4-a7e7-45ca-9e27-b71fa28b965b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:06.430960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fa3e3d4-a7e7-45ca-9e27-b71fa28b965b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:06.432143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fa3e3d4-a7e7-45ca-9e27-b71fa28b965b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:06.434116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fa3e3d4-a7e7-45ca-9e27-b71fa28b965b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:06.434871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fa3e3d4-a7e7-45ca-9e27-b71fa28b965b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:06.457566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5d7674fb-b96b-4d89-9318-6affa82348a0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:06.466922 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5d7674fb-b96b-4d89-9318-6affa82348a0 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1470 microversion: 2.1 time: 0.248766 Sep 27 10:04:06.467518 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 878/1934] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:06 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1470 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:06.492480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:06.498127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:06.510214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:06.520857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:06.521626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:06.522566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:06.523190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:06.523848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:06.918793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:06.937192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:06.944009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4fa3e3d4-a7e7-45ca-9e27-b71fa28b965b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:06.944363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:06.945928 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fba218f5-90fc-4d22-8c2e-7ae7149d6c8f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.457512 Sep 27 10:04:06.947693 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 879/1935] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:06 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:06.956502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-785ac3f1-39d4-4b96-9a57-23bf669834a7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:06.957466 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4fa3e3d4-a7e7-45ca-9e27-b71fa28b965b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.549409 Sep 27 10:04:06.957993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-785ac3f1-39d4-4b96-9a57-23bf669834a7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:06.958129 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1057/1936] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:06 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:06.976517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-785ac3f1-39d4-4b96-9a57-23bf669834a7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:06.977226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-785ac3f1-39d4-4b96-9a57-23bf669834a7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:06.978856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-785ac3f1-39d4-4b96-9a57-23bf669834a7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:06.980527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-785ac3f1-39d4-4b96-9a57-23bf669834a7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:06.981113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-785ac3f1-39d4-4b96-9a57-23bf669834a7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:06.982009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-785ac3f1-39d4-4b96-9a57-23bf669834a7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:06.982156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-9f148e01-eb13-48a5-9da8-3444f00e478d req-cb37c3ea-4cd5-4360-9f5e-e49aa3e497db service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "96553659-f775-4568-a1a1-c3a9237df534", "name": "network-vif-deleted", "tag": "2a657e89-bfee-4434-85bf-ab2d711c1e68"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:07.011790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f148e01-eb13-48a5-9da8-3444f00e478d req-cb37c3ea-4cd5-4360-9f5e-e49aa3e497db service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:07.012693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f148e01-eb13-48a5-9da8-3444f00e478d req-cb37c3ea-4cd5-4360-9f5e-e49aa3e497db service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:07.013432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f148e01-eb13-48a5-9da8-3444f00e478d req-cb37c3ea-4cd5-4360-9f5e-e49aa3e497db service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:07.034436 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-9f148e01-eb13-48a5-9da8-3444f00e478d req-cb37c3ea-4cd5-4360-9f5e-e49aa3e497db service nova] Creating event network-vif-deleted:2a657e89-bfee-4434-85bf-ab2d711c1e68 for instance 96553659-f775-4568-a1a1-c3a9237df534 on nch2-master-1 Sep 27 10:04:07.040140 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-9f148e01-eb13-48a5-9da8-3444f00e478d req-cb37c3ea-4cd5-4360-9f5e-e49aa3e497db service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071896 Sep 27 10:04:07.041179 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1058/1937] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:07.047498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:07.048767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:07.055978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:07.066327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:07.066930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:07.067896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:07.068561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:07.069257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:07.140556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:07.141256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:07.141800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:07.142205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lazy-loading 'fault' on Instance uuid 96553659-f775-4568-a1a1-c3a9237df534 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:07.216877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:07.231139 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cadd517c-422e-4f66-aa34-ced955209a63 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 200 len: 1311 microversion: 2.1 time: 0.188365 Sep 27 10:04:07.231863 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1059/1938] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:07 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 1311 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:07.236843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b396af7-76be-46db-811e-e3ae2abbc6b3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:07.245122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7b396af7-76be-46db-811e-e3ae2abbc6b3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:07.261141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b396af7-76be-46db-811e-e3ae2abbc6b3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:07.261755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b396af7-76be-46db-811e-e3ae2abbc6b3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:07.262293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b396af7-76be-46db-811e-e3ae2abbc6b3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:07.270053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b396af7-76be-46db-811e-e3ae2abbc6b3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:07.270924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b396af7-76be-46db-811e-e3ae2abbc6b3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:07.271934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b396af7-76be-46db-811e-e3ae2abbc6b3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:07.361231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-785ac3f1-39d4-4b96-9a57-23bf669834a7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:07.370899 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-785ac3f1-39d4-4b96-9a57-23bf669834a7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1199 microversion: 2.1 time: 0.421548 Sep 27 10:04:07.378091 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 880/1939] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:06 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1199 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:07.466583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7b396af7-76be-46db-811e-e3ae2abbc6b3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:07.482566 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7b396af7-76be-46db-811e-e3ae2abbc6b3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1275 microversion: 2.1 time: 0.249286 Sep 27 10:04:07.483713 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1060/1940] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:07 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1275 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:07.494240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ff4f45cc-f288-44c1-8e6f-a5b7411c4041 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:07.495198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ff4f45cc-f288-44c1-8e6f-a5b7411c4041 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:07.501783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4f45cc-f288-44c1-8e6f-a5b7411c4041 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:07.502540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4f45cc-f288-44c1-8e6f-a5b7411c4041 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:07.503135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4f45cc-f288-44c1-8e6f-a5b7411c4041 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:07.504196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4f45cc-f288-44c1-8e6f-a5b7411c4041 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:07.504833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4f45cc-f288-44c1-8e6f-a5b7411c4041 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:07.505448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4f45cc-f288-44c1-8e6f-a5b7411c4041 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:07.670096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ff4f45cc-f288-44c1-8e6f-a5b7411c4041 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:07.679812 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ff4f45cc-f288-44c1-8e6f-a5b7411c4041 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1470 microversion: 2.1 time: 0.189722 Sep 27 10:04:07.681484 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 881/1941] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:07 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1470 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:07.978283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:07.979529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:07.983090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fdadf6b8-4348-4414-82d2-e17d1a828144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:07.983959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fdadf6b8-4348-4414-82d2-e17d1a828144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:07.995604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:07.998465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.000996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.002300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdadf6b8-4348-4414-82d2-e17d1a828144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.002807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdadf6b8-4348-4414-82d2-e17d1a828144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.003255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdadf6b8-4348-4414-82d2-e17d1a828144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.003697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.003988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdadf6b8-4348-4414-82d2-e17d1a828144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.004458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdadf6b8-4348-4414-82d2-e17d1a828144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.004932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdadf6b8-4348-4414-82d2-e17d1a828144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.005741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.007197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.224889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fdadf6b8-4348-4414-82d2-e17d1a828144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:08.234882 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fdadf6b8-4348-4414-82d2-e17d1a828144 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1307 microversion: 2.1 time: 0.255942 Sep 27 10:04:08.237320 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 882/1942] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:07 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1307 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:08.268289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:08.269067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:08.278036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.278978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.279917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.281364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.282005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.282616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.324709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.325305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.325848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.326383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lazy-loading 'fault' on Instance uuid 96553659-f775-4568-a1a1-c3a9237df534 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:08.375728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:08.385522 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f2bc8057-0de5-46a9-97b5-8eff94b87dcf tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 200 len: 1311 microversion: 2.1 time: 0.123041 Sep 27 10:04:08.386160 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 883/1943] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:08 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 1311 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:08.406069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:08.417412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:08.419374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:08.420881 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-016d09a7-98ca-4668-983a-b9245d49216a tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.451149 Sep 27 10:04:08.421348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc526c0c-26e2-4260-bfc5-60e060e50554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:08.422208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc526c0c-26e2-4260-bfc5-60e060e50554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:08.425080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:08.431611 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1061/1944] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:07 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:08.431811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc526c0c-26e2-4260-bfc5-60e060e50554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.432616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc526c0c-26e2-4260-bfc5-60e060e50554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.433424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc526c0c-26e2-4260-bfc5-60e060e50554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.434592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc526c0c-26e2-4260-bfc5-60e060e50554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.435748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc526c0c-26e2-4260-bfc5-60e060e50554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.436189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc526c0c-26e2-4260-bfc5-60e060e50554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.511757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-456f1f2f-2b0d-4689-a806-34a36ea67834 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:08.513558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-456f1f2f-2b0d-4689-a806-34a36ea67834 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:08.526074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-456f1f2f-2b0d-4689-a806-34a36ea67834 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.526650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-456f1f2f-2b0d-4689-a806-34a36ea67834 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.530491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-456f1f2f-2b0d-4689-a806-34a36ea67834 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.531289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-456f1f2f-2b0d-4689-a806-34a36ea67834 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.533387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-456f1f2f-2b0d-4689-a806-34a36ea67834 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.535703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-456f1f2f-2b0d-4689-a806-34a36ea67834 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.541816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:08.694020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-456f1f2f-2b0d-4689-a806-34a36ea67834 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:08.711909 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-456f1f2f-2b0d-4689-a806-34a36ea67834 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1275 microversion: 2.1 time: 0.205212 Sep 27 10:04:08.714632 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1062/1945] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:08 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1275 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:08.725013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bb3b3bed-eb89-41e3-819f-073431fbb3ab tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:08.727923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bb3b3bed-eb89-41e3-819f-073431fbb3ab tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:08.736281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3b3bed-eb89-41e3-819f-073431fbb3ab tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.737706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3b3bed-eb89-41e3-819f-073431fbb3ab tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.738495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3b3bed-eb89-41e3-819f-073431fbb3ab tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.739444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3b3bed-eb89-41e3-819f-073431fbb3ab tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:08.739992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3b3bed-eb89-41e3-819f-073431fbb3ab tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:08.740530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3b3bed-eb89-41e3-819f-073431fbb3ab tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:08.993791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cc526c0c-26e2-4260-bfc5-60e060e50554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:09.006718 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cc526c0c-26e2-4260-bfc5-60e060e50554 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1199 microversion: 2.1 time: 0.604476 Sep 27 10:04:09.008769 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 884/1946] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:08 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1199 bytes in 607 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:09.154573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb3b3bed-eb89-41e3-819f-073431fbb3ab tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:09.179638 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bb3b3bed-eb89-41e3-819f-073431fbb3ab tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1470 microversion: 2.1 time: 0.462973 Sep 27 10:04:09.200726 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1063/1947] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:08 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1470 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:09.259978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b8b8d0c4-39bf-437e-9a27-1fe42ba24c27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:09.260558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b8b8d0c4-39bf-437e-9a27-1fe42ba24c27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:09.272347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8b8d0c4-39bf-437e-9a27-1fe42ba24c27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:09.281030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8b8d0c4-39bf-437e-9a27-1fe42ba24c27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:09.281790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8b8d0c4-39bf-437e-9a27-1fe42ba24c27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:09.293034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8b8d0c4-39bf-437e-9a27-1fe42ba24c27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:09.294167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8b8d0c4-39bf-437e-9a27-1fe42ba24c27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:09.294343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8b8d0c4-39bf-437e-9a27-1fe42ba24c27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:09.297402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:09.414881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:09.415646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 96553659-f775-4568-a1a1-c3a9237df534] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:09.443748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:09.445502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:09.446978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:09.448565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:09.449346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:09.450049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:09.475787 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] HTTP exception thrown: Instance 96553659-f775-4568-a1a1-c3a9237df534 could not be found. Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Request method failure captured: Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 HTTP/1.1 Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMso_T5-k00r-jNMmt-sE3AO4w-bk9w_R-TIb97BkNgtX430OKum_e_1Ajxl5FN5vu42mVBl2t9CzMeiQzlqPlORJKPijU1tf0C6z8at9kYV7c7xulwZegzv6clP9NU3qX8u7EF1IsTY4NHcT_P_xJX9RRB_V2Pz10WEXih8yHjsEtCw Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f48a592eb8fa4221a56eac1a50152c91 Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ListServerFiltersTestJSON-809664979 Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:04:09.479915 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f48a592eb8fa4221a56eac1a50152c91"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f48a592eb8fa4221a56eac1a50152c91"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f48a592eb8fa4221a56eac1a50152c91"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f48a592eb8fa4221a56eac1a50152c91"}]}] Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ListServerFiltersTestJSON-809664979 Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f48a592eb8fa4221a56eac1a50152c91 Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ListServerFiltersTestJSON-809664979 Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ListServerFiltersTestJSON-809664979-project Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: ac98fc0d0296493999906e8e3607c7f3 Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ListServerFiltersTestJSON-809664979-project Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 96553659-f775-4568-a1a1-c3a9237df534 could not be found. Sep 27 10:04:09.482181 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '96553659-f775-4568-a1a1-c3a9237df534'} Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 96553659-f775-4568-a1a1-c3a9237df534 could not be found. Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:04:09.483032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 96553659-f775-4568-a1a1-c3a9237df534 could not be found. Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 96553659-f775-4568-a1a1-c3a9237df534 could not be found. Sep 27 10:04:09.484501 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:09.491318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Returning 404 to user: Instance 96553659-f775-4568-a1a1-c3a9237df534 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:04:09.494012 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d875ae2-f777-4872-8733-18e049801390 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534" status: 404 len: 111 microversion: 2.1 time: 0.085830 Sep 27 10:04:09.499985 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1064/1948] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:09 2022] GET /compute/v2.1/servers/96553659-f775-4568-a1a1-c3a9237df534 => generated 111 bytes in 93 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:09.505216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:09.506004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:09.517046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:09.518776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:09.520106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:09.524325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:09.525323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:09.526072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:09.827401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:09.839916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:09.843797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:09.851565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-15f6ca08-64a9-4ba9-af30-c055acbb0a28 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.350129 Sep 27 10:04:09.864909 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1065/1949] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:09 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:09.876803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c75d3671-8ed4-4a15-90a1-d8494bb12f0c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:09.878096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c75d3671-8ed4-4a15-90a1-d8494bb12f0c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:09.885835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c75d3671-8ed4-4a15-90a1-d8494bb12f0c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:09.886660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c75d3671-8ed4-4a15-90a1-d8494bb12f0c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:09.887334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c75d3671-8ed4-4a15-90a1-d8494bb12f0c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:09.888288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c75d3671-8ed4-4a15-90a1-d8494bb12f0c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:09.888416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b8b8d0c4-39bf-437e-9a27-1fe42ba24c27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:09.889161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c75d3671-8ed4-4a15-90a1-d8494bb12f0c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:09.889850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c75d3671-8ed4-4a15-90a1-d8494bb12f0c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:09.902484 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b8b8d0c4-39bf-437e-9a27-1fe42ba24c27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1487 microversion: 2.1 time: 0.647015 Sep 27 10:04:09.912695 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 885/1950] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:09 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1487 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:09.913398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0472c5a4-12f1-4bf4-8128-e59fea7bb7d0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:09.914239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0472c5a4-12f1-4bf4-8128-e59fea7bb7d0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:09.927546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0472c5a4-12f1-4bf4-8128-e59fea7bb7d0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:09.941857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0472c5a4-12f1-4bf4-8128-e59fea7bb7d0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:09.942439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c75d3671-8ed4-4a15-90a1-d8494bb12f0c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:04:09.942786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0472c5a4-12f1-4bf4-8128-e59fea7bb7d0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:09.943748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0472c5a4-12f1-4bf4-8128-e59fea7bb7d0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:09.944409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0472c5a4-12f1-4bf4-8128-e59fea7bb7d0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:09.945130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0472c5a4-12f1-4bf4-8128-e59fea7bb7d0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:10.118872 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c75d3671-8ed4-4a15-90a1-d8494bb12f0c tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "DELETE /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 204 len: 0 microversion: 2.1 time: 0.249563 Sep 27 10:04:10.121472 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1066/1951] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:04:09 2022] DELETE /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:10.129147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3333bc16-e8f0-4b83-b081-436d5dcb3682 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:10.130547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3333bc16-e8f0-4b83-b081-436d5dcb3682 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:10.140589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3333bc16-e8f0-4b83-b081-436d5dcb3682 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:10.143504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3333bc16-e8f0-4b83-b081-436d5dcb3682 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:10.144596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3333bc16-e8f0-4b83-b081-436d5dcb3682 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:10.146351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3333bc16-e8f0-4b83-b081-436d5dcb3682 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:10.147248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3333bc16-e8f0-4b83-b081-436d5dcb3682 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:10.148242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3333bc16-e8f0-4b83-b081-436d5dcb3682 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:10.203245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-3333bc16-e8f0-4b83-b081-436d5dcb3682 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lazy-loading 'flavor' on Instance uuid 152a5758-d14b-4ca9-917e-f7017c6d99b2 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:10.281915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0472c5a4-12f1-4bf4-8128-e59fea7bb7d0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:10.295613 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0472c5a4-12f1-4bf4-8128-e59fea7bb7d0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1317 microversion: 2.1 time: 0.391137 Sep 27 10:04:10.300710 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 886/1952] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:09 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1317 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:10.312200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd67487f-bf22-4bb5-9a60-ce0027107cb6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:10.313180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cd67487f-bf22-4bb5-9a60-ce0027107cb6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:10.324086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd67487f-bf22-4bb5-9a60-ce0027107cb6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:10.325539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd67487f-bf22-4bb5-9a60-ce0027107cb6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:10.327306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd67487f-bf22-4bb5-9a60-ce0027107cb6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:10.330490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd67487f-bf22-4bb5-9a60-ce0027107cb6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:10.332575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd67487f-bf22-4bb5-9a60-ce0027107cb6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:10.333161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd67487f-bf22-4bb5-9a60-ce0027107cb6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:10.416909 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3333bc16-e8f0-4b83-b081-436d5dcb3682 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "POST /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2/action" status: 202 len: 0 microversion: 2.1 time: 0.277139 Sep 27 10:04:10.420690 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1067/1953] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:04:10 2022] POST /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2/action => generated 0 bytes in 298 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:04:10.444809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf33f540-01c2-45b7-a81d-00839c667e6a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:10.445786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bf33f540-01c2-45b7-a81d-00839c667e6a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:10.466637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf33f540-01c2-45b7-a81d-00839c667e6a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:10.467149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf33f540-01c2-45b7-a81d-00839c667e6a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:10.467605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf33f540-01c2-45b7-a81d-00839c667e6a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:10.468397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf33f540-01c2-45b7-a81d-00839c667e6a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:10.468895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf33f540-01c2-45b7-a81d-00839c667e6a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:10.469326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf33f540-01c2-45b7-a81d-00839c667e6a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:10.745974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bf33f540-01c2-45b7-a81d-00839c667e6a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:10.755078 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bf33f540-01c2-45b7-a81d-00839c667e6a tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1518 microversion: 2.1 time: 0.330090 Sep 27 10:04:10.760763 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1068/1954] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:10 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1518 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:10.776949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:10.779689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:10.792581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:10.793628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:10.794095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:10.794823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:10.795999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:10.796693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:10.873301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:10.875172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:10.875825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:10.876809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lazy-loading 'fault' on Instance uuid 14bb2db7-2b1d-46d4-8800-26b149825654 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:10.956306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7892230632b490c8f110206461a87c6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8b87f09e1bf645a4b71d8b31ee512ec0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:10.972014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cd67487f-bf22-4bb5-9a60-ce0027107cb6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:10.974498 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-182d8a62-4da4-4ac6-888a-db3b7d06ce09 tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 200 len: 1304 microversion: 2.1 time: 0.212138 Sep 27 10:04:10.976854 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1069/1955] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:10 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 1304 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:10.990536 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd67487f-bf22-4bb5-9a60-ce0027107cb6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1199 microversion: 2.1 time: 0.687664 Sep 27 10:04:10.991621 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 887/1956] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:10 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1199 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:10.995020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4ddfc486-1c5f-4f69-8ea6-bc5135bf5e86 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:10.995658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4ddfc486-1c5f-4f69-8ea6-bc5135bf5e86 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:11.008172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-996e1a18-8a18-4329-b939-76d495e40aec req-1dfed269-6dee-4024-bf2b-47265a92c455 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "14bb2db7-2b1d-46d4-8800-26b149825654", "name": "network-vif-deleted", "tag": "dc496928-7dd4-48f7-a375-66707f33ecf8"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:11.009655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ddfc486-1c5f-4f69-8ea6-bc5135bf5e86 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:11.010353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ddfc486-1c5f-4f69-8ea6-bc5135bf5e86 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:11.011174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ddfc486-1c5f-4f69-8ea6-bc5135bf5e86 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:11.012203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ddfc486-1c5f-4f69-8ea6-bc5135bf5e86 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:11.019124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ddfc486-1c5f-4f69-8ea6-bc5135bf5e86 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:11.019994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ddfc486-1c5f-4f69-8ea6-bc5135bf5e86 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:11.029709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-996e1a18-8a18-4329-b939-76d495e40aec req-1dfed269-6dee-4024-bf2b-47265a92c455 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:11.030249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-996e1a18-8a18-4329-b939-76d495e40aec req-1dfed269-6dee-4024-bf2b-47265a92c455 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:11.030706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-996e1a18-8a18-4329-b939-76d495e40aec req-1dfed269-6dee-4024-bf2b-47265a92c455 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:11.066457 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-996e1a18-8a18-4329-b939-76d495e40aec req-1dfed269-6dee-4024-bf2b-47265a92c455 service nova] Creating event network-vif-deleted:dc496928-7dd4-48f7-a375-66707f33ecf8 for instance 14bb2db7-2b1d-46d4-8800-26b149825654 on nch2-master-1 Sep 27 10:04:11.074396 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-996e1a18-8a18-4329-b939-76d495e40aec req-1dfed269-6dee-4024-bf2b-47265a92c455 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081183 Sep 27 10:04:11.080820 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 888/1957] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:11.086644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-17d4f655-1a66-4e28-b9f6-798cdeb5f282 req-1354944a-7da6-4490-88b1-3cb51db5ac02 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3b563b3f-326c-41ed-9185-2a6d112c871d", "tag": "917af05f-ebc3-433e-a57d-d09e51b4d3c7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:11.117908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-17d4f655-1a66-4e28-b9f6-798cdeb5f282 req-1354944a-7da6-4490-88b1-3cb51db5ac02 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:11.118407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-17d4f655-1a66-4e28-b9f6-798cdeb5f282 req-1354944a-7da6-4490-88b1-3cb51db5ac02 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:11.119134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-17d4f655-1a66-4e28-b9f6-798cdeb5f282 req-1354944a-7da6-4490-88b1-3cb51db5ac02 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:11.138320 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-17d4f655-1a66-4e28-b9f6-798cdeb5f282 req-1354944a-7da6-4490-88b1-3cb51db5ac02 service nova] Creating event network-changed:917af05f-ebc3-433e-a57d-d09e51b4d3c7 for instance 3b563b3f-326c-41ed-9185-2a6d112c871d on nch1-master-1 Sep 27 10:04:11.144226 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-17d4f655-1a66-4e28-b9f6-798cdeb5f282 req-1354944a-7da6-4490-88b1-3cb51db5ac02 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061803 Sep 27 10:04:11.144889 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 889/1958] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:11.151412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:11.152130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:11.158778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:11.159253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:11.159700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:11.160647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:11.161320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:11.161740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:11.262154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4ddfc486-1c5f-4f69-8ea6-bc5135bf5e86 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:11.278368 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4ddfc486-1c5f-4f69-8ea6-bc5135bf5e86 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1495 microversion: 2.1 time: 0.300232 Sep 27 10:04:11.279433 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1070/1959] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:10 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1495 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:11.322335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30dfc99a-dbd9-48f7-b3b1-50ebf15d667c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:11.322956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-30dfc99a-dbd9-48f7-b3b1-50ebf15d667c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:11.330260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30dfc99a-dbd9-48f7-b3b1-50ebf15d667c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:11.331041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30dfc99a-dbd9-48f7-b3b1-50ebf15d667c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:11.332237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30dfc99a-dbd9-48f7-b3b1-50ebf15d667c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:11.333180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30dfc99a-dbd9-48f7-b3b1-50ebf15d667c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:11.334083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30dfc99a-dbd9-48f7-b3b1-50ebf15d667c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:11.334542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30dfc99a-dbd9-48f7-b3b1-50ebf15d667c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:11.370409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:11.380833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:11.383988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:11.385533 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7b25d1f1-3ed1-4878-bb04-0e6f23882305 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.239429 Sep 27 10:04:11.386609 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 890/1960] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:11 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:11.547965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-30dfc99a-dbd9-48f7-b3b1-50ebf15d667c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:11.556887 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-30dfc99a-dbd9-48f7-b3b1-50ebf15d667c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1317 microversion: 2.1 time: 0.238928 Sep 27 10:04:11.572293 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1071/1961] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:11 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1317 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:11.664390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-c85b6d4f-dd24-4aea-b243-c1e6456ff479 req-2258c1d3-feeb-4f02-a6bc-b5b3677b5602 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bd544723-d1ca-4f7e-a231-ba0b8351464d", "name": "network-vif-plugged", "status": "completed", "tag": "f2d7b45b-7823-4ee9-a9a3-1e4c5cda1e27"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:11.677748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c85b6d4f-dd24-4aea-b243-c1e6456ff479 req-2258c1d3-feeb-4f02-a6bc-b5b3677b5602 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:11.678605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c85b6d4f-dd24-4aea-b243-c1e6456ff479 req-2258c1d3-feeb-4f02-a6bc-b5b3677b5602 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:11.679108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c85b6d4f-dd24-4aea-b243-c1e6456ff479 req-2258c1d3-feeb-4f02-a6bc-b5b3677b5602 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:11.694955 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c85b6d4f-dd24-4aea-b243-c1e6456ff479 req-2258c1d3-feeb-4f02-a6bc-b5b3677b5602 service nova] Creating event network-vif-plugged:f2d7b45b-7823-4ee9-a9a3-1e4c5cda1e27 for instance bd544723-d1ca-4f7e-a231-ba0b8351464d on nch1-master-1 Sep 27 10:04:11.701804 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-c85b6d4f-dd24-4aea-b243-c1e6456ff479 req-2258c1d3-feeb-4f02-a6bc-b5b3677b5602 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045733 Sep 27 10:04:11.702963 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 891/1962] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:11.786441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-32279570-04b1-4132-82c1-40a4f3d36971 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:11.787143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-32279570-04b1-4132-82c1-40a4f3d36971 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:11.793898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32279570-04b1-4132-82c1-40a4f3d36971 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:11.794523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32279570-04b1-4132-82c1-40a4f3d36971 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:11.795002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32279570-04b1-4132-82c1-40a4f3d36971 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:11.795851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32279570-04b1-4132-82c1-40a4f3d36971 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:11.796484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32279570-04b1-4132-82c1-40a4f3d36971 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:11.796935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32279570-04b1-4132-82c1-40a4f3d36971 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.001153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:12.002158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] [instance: 14bb2db7-2b1d-46d4-8800-26b149825654] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:12.011586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.013379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.013379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.014820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.015964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.016411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.035164 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] HTTP exception thrown: Instance 14bb2db7-2b1d-46d4-8800-26b149825654 could not be found. Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Request method failure captured: Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 HTTP/1.1 Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsp76FJxiey0DALy4EJKOF_YfAMHEfcVJ08E2-636PDmmEZBsLcN0Z48ijMXXi3agYno0ZPL-TkRcWyt4OAWiBtyNO97BSkXGE0I4kREwNzlNdM1Mk7a-wH7MpZauD4JusWpMCxyIcH28tky4h75LPOk4OCW--IH9yZ5hGXR3Y_lkn4 Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8b87f09e1bf645a4b71d8b31ee512ec0 Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TenantUsagesTestJSON-153198515 Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:04:12.037051 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8b87f09e1bf645a4b71d8b31ee512ec0"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8b87f09e1bf645a4b71d8b31ee512ec0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8b87f09e1bf645a4b71d8b31ee512ec0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8b87f09e1bf645a4b71d8b31ee512ec0"}]}] Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TenantUsagesTestJSON-153198515 Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8b87f09e1bf645a4b71d8b31ee512ec0 Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TenantUsagesTestJSON-153198515 Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TenantUsagesTestJSON-153198515-project Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b7892230632b490c8f110206461a87c6 Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TenantUsagesTestJSON-153198515-project Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 14bb2db7-2b1d-46d4-8800-26b149825654 could not be found. Sep 27 10:04:12.038845 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '14bb2db7-2b1d-46d4-8800-26b149825654'} Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 14bb2db7-2b1d-46d4-8800-26b149825654 could not be found. Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:04:12.039995 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 14bb2db7-2b1d-46d4-8800-26b149825654 could not be found. Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 14bb2db7-2b1d-46d4-8800-26b149825654 could not be found. Sep 27 10:04:12.041662 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:04:12.044380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] Returning 404 to user: Instance 14bb2db7-2b1d-46d4-8800-26b149825654 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:04:12.044380 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c5510c93-f27c-4dfa-aab9-d992e4595a5a tempest-TenantUsagesTestJSON-153198515 tempest-TenantUsagesTestJSON-153198515-project] 10.222.0.26 "GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654" status: 404 len: 111 microversion: 2.1 time: 0.042871 Sep 27 10:04:12.044380 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 892/1963] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:11 2022] GET /compute/v2.1/servers/14bb2db7-2b1d-46d4-8800-26b149825654 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:12.049808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-15a3ba0c-6b88-4efa-b2ba-65add8e702f1 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:12.050428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-15a3ba0c-6b88-4efa-b2ba-65add8e702f1 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:12.059705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15a3ba0c-6b88-4efa-b2ba-65add8e702f1 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.060201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15a3ba0c-6b88-4efa-b2ba-65add8e702f1 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.061117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15a3ba0c-6b88-4efa-b2ba-65add8e702f1 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.061838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15a3ba0c-6b88-4efa-b2ba-65add8e702f1 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.062706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15a3ba0c-6b88-4efa-b2ba-65add8e702f1 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.063285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15a3ba0c-6b88-4efa-b2ba-65add8e702f1 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.172124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-32279570-04b1-4132-82c1-40a4f3d36971 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:12.180869 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-32279570-04b1-4132-82c1-40a4f3d36971 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1518 microversion: 2.1 time: 0.398539 Sep 27 10:04:12.181454 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1072/1964] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:11 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1518 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:12.304252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90eeb6a7-f6b2-46fa-8829-a3a5f84a1ffc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:12.305835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-90eeb6a7-f6b2-46fa-8829-a3a5f84a1ffc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:12.318230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90eeb6a7-f6b2-46fa-8829-a3a5f84a1ffc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.319932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90eeb6a7-f6b2-46fa-8829-a3a5f84a1ffc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.321730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90eeb6a7-f6b2-46fa-8829-a3a5f84a1ffc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.323257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90eeb6a7-f6b2-46fa-8829-a3a5f84a1ffc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.323970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90eeb6a7-f6b2-46fa-8829-a3a5f84a1ffc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.324619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90eeb6a7-f6b2-46fa-8829-a3a5f84a1ffc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.488967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-15a3ba0c-6b88-4efa-b2ba-65add8e702f1 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:12.505680 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-15a3ba0c-6b88-4efa-b2ba-65add8e702f1 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1199 microversion: 2.1 time: 0.460301 Sep 27 10:04:12.506289 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 893/1965] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:12 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1199 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:12.511207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:12.512044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:12.518470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.519155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.520348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.521439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.522050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.522715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.704821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-90eeb6a7-f6b2-46fa-8829-a3a5f84a1ffc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:12.717695 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-90eeb6a7-f6b2-46fa-8829-a3a5f84a1ffc tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 200 len: 1478 microversion: 2.1 time: 0.415989 Sep 27 10:04:12.720365 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1073/1966] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:12 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 1478 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:12.745660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92548ef5-0bf6-49ba-9f7f-cc6e8049bec5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:12.749202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-92548ef5-0bf6-49ba-9f7f-cc6e8049bec5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:12.771062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92548ef5-0bf6-49ba-9f7f-cc6e8049bec5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.771719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92548ef5-0bf6-49ba-9f7f-cc6e8049bec5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.775911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92548ef5-0bf6-49ba-9f7f-cc6e8049bec5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.777274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92548ef5-0bf6-49ba-9f7f-cc6e8049bec5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.777943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92548ef5-0bf6-49ba-9f7f-cc6e8049bec5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.778730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92548ef5-0bf6-49ba-9f7f-cc6e8049bec5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.864111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:12.877837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:12.880115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:12.881785 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-71796d25-0b01-4c8c-95a4-46238f465d8c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.374427 Sep 27 10:04:12.883022 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 894/1967] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:12 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:12.909096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-89eefcc8-dc00-42f3-84f4-0c4c5b19161a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-934916470"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:12.912692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-89eefcc8-dc00-42f3-84f4-0c4c5b19161a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:12.921453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eefcc8-dc00-42f3-84f4-0c4c5b19161a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.922495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eefcc8-dc00-42f3-84f4-0c4c5b19161a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.923259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eefcc8-dc00-42f3-84f4-0c4c5b19161a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:12.924469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eefcc8-dc00-42f3-84f4-0c4c5b19161a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:12.925419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eefcc8-dc00-42f3-84f4-0c4c5b19161a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:12.926067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89eefcc8-dc00-42f3-84f4-0c4c5b19161a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:13.344787 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-89eefcc8-dc00-42f3-84f4-0c4c5b19161a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "POST /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2/action" status: 202 len: 0 microversion: 2.1 time: 0.460072 Sep 27 10:04:13.345748 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 895/1968] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:04:12 2022] POST /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2/action => generated 0 bytes in 462 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 10:04:13.357266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5332cf38-5ffa-4c52-b70a-b6aa81632a49 req-e6807cc8-7065-40d2-b373-9adbd3e1ba25 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6d511fc6-80ab-458e-88a6-593374a3e01c", "name": "network-vif-deleted", "tag": "3f178b4c-caa9-4399-84bb-8322b9ac1a75"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:13.373418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5332cf38-5ffa-4c52-b70a-b6aa81632a49 req-e6807cc8-7065-40d2-b373-9adbd3e1ba25 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:13.374577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5332cf38-5ffa-4c52-b70a-b6aa81632a49 req-e6807cc8-7065-40d2-b373-9adbd3e1ba25 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:13.379863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5332cf38-5ffa-4c52-b70a-b6aa81632a49 req-e6807cc8-7065-40d2-b373-9adbd3e1ba25 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:13.398080 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5332cf38-5ffa-4c52-b70a-b6aa81632a49 req-e6807cc8-7065-40d2-b373-9adbd3e1ba25 service nova] Creating event network-vif-deleted:3f178b4c-caa9-4399-84bb-8322b9ac1a75 for instance 6d511fc6-80ab-458e-88a6-593374a3e01c on nch1-master-1 Sep 27 10:04:13.407028 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5332cf38-5ffa-4c52-b70a-b6aa81632a49 req-e6807cc8-7065-40d2-b373-9adbd3e1ba25 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059071 Sep 27 10:04:13.408306 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 896/1969] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:13.415388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:13.416689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:13.425612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:13.426613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:13.427286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:13.428458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:13.429413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:13.430633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:13.488268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:13.489804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:13.491213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:13.493281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lazy-loading 'fault' on Instance uuid 6d511fc6-80ab-458e-88a6-593374a3e01c {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:13.580183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-92548ef5-0bf6-49ba-9f7f-cc6e8049bec5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:13.599506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:13.608686 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-92548ef5-0bf6-49ba-9f7f-cc6e8049bec5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1317 microversion: 2.1 time: 0.859999 Sep 27 10:04:13.629235 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1074/1970] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:12 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1317 bytes in 908 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:13.642487 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cef55718-498b-4724-a4fa-ca140f4913f5 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1311 microversion: 2.1 time: 0.231588 Sep 27 10:04:13.644492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4da603ba-4c9f-4184-a9f4-e35d7b43a5dd tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6c2b4af0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:13.645432 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 897/1971] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:13 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1311 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:13.665643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2f810340-6b4c-41bf-a59e-78a6069f99f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f635df550>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:13.745341 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4da603ba-4c9f-4184-a9f4-e35d7b43a5dd tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/25e2dfe0-5b9d-4793-b9db-ea5bb05e74cf" status: 200 len: 662 microversion: 2.1 time: 0.111892 Sep 27 10:04:13.745948 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1075/1972] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:13 2022] GET /compute/v2.1/images/25e2dfe0-5b9d-4793-b9db-ea5bb05e74cf => generated 662 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:13.759484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7c8d4164-925c-45f4-a747-9f240abfd835 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:13.761652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7c8d4164-925c-45f4-a747-9f240abfd835 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:13.772902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c8d4164-925c-45f4-a747-9f240abfd835 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:13.773464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c8d4164-925c-45f4-a747-9f240abfd835 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:13.773949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c8d4164-925c-45f4-a747-9f240abfd835 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:13.774960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c8d4164-925c-45f4-a747-9f240abfd835 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:13.775616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c8d4164-925c-45f4-a747-9f240abfd835 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:13.776257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c8d4164-925c-45f4-a747-9f240abfd835 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:13.778537 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2f810340-6b4c-41bf-a59e-78a6069f99f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 200 len: 1437 microversion: 2.1 time: 0.131659 Sep 27 10:04:13.779540 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 898/1973] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:13 2022] GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 1437 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:13.786650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6105f15e-6e0c-4681-b633-fb1f56dc4f3d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62efd4c0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:13.848805 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6105f15e-6e0c-4681-b633-fb1f56dc4f3d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/25e2dfe0-5b9d-4793-b9db-ea5bb05e74cf" status: 200 len: 662 microversion: 2.1 time: 0.066531 Sep 27 10:04:13.849502 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 899/1974] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:13 2022] GET /compute/v2.1/images/25e2dfe0-5b9d-4793-b9db-ea5bb05e74cf => generated 662 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:13.912183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:13.913803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:13.925200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:13.926009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:13.926633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:13.927754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:13.928582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:13.929399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.042360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:14.107501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7c8d4164-925c-45f4-a747-9f240abfd835 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:14.115336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:14.124139 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7c8d4164-925c-45f4-a747-9f240abfd835 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1382 microversion: 2.1 time: 0.376973 Sep 27 10:04:14.125898 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1076/1975] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:13 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1382 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:14.153841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3e4ff01b-6273-4a66-be0b-3b52913f184a tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:14.155182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3e4ff01b-6273-4a66-be0b-3b52913f184a tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:14.164450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:14.165572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4ff01b-6273-4a66-be0b-3b52913f184a tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.166480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4ff01b-6273-4a66-be0b-3b52913f184a tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.167455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4ff01b-6273-4a66-be0b-3b52913f184a tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.168811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4ff01b-6273-4a66-be0b-3b52913f184a tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.169901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4ff01b-6273-4a66-be0b-3b52913f184a tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.171086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4ff01b-6273-4a66-be0b-3b52913f184a tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.182269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:14.183868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:14.187385 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5297bfa8-1b10-48a0-9153-fd60dca98969 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.282148 Sep 27 10:04:14.190251 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 900/1976] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:13 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:14.401981 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3e4ff01b-6273-4a66-be0b-3b52913f184a tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1533 microversion: 2.1 time: 0.253546 Sep 27 10:04:14.402925 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1077/1977] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:14 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1533 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:14.426650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce7d9bfe-5b84-4dcc-8944-246b708cf423 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:14.427339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ce7d9bfe-5b84-4dcc-8944-246b708cf423 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:14.442989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce7d9bfe-5b84-4dcc-8944-246b708cf423 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.443917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce7d9bfe-5b84-4dcc-8944-246b708cf423 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.443917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce7d9bfe-5b84-4dcc-8944-246b708cf423 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.444553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce7d9bfe-5b84-4dcc-8944-246b708cf423 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.445135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce7d9bfe-5b84-4dcc-8944-246b708cf423 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.445848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce7d9bfe-5b84-4dcc-8944-246b708cf423 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.623090 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ce7d9bfe-5b84-4dcc-8944-246b708cf423 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1533 microversion: 2.1 time: 0.200402 Sep 27 10:04:14.624139 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 901/1978] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:14 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1533 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:14.646480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-540bd2b4-59f3-45a5-89b2-0a61caa1cba5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:14.648212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-540bd2b4-59f3-45a5-89b2-0a61caa1cba5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:14.649247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1463267c-052e-4854-9b6e-1940d0a12aac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:14.653933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1463267c-052e-4854-9b6e-1940d0a12aac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.654747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1463267c-052e-4854-9b6e-1940d0a12aac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.655579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1463267c-052e-4854-9b6e-1940d0a12aac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.657424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1463267c-052e-4854-9b6e-1940d0a12aac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.657973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1463267c-052e-4854-9b6e-1940d0a12aac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.658423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1463267c-052e-4854-9b6e-1940d0a12aac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.663439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-540bd2b4-59f3-45a5-89b2-0a61caa1cba5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.665642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-540bd2b4-59f3-45a5-89b2-0a61caa1cba5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.666785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-540bd2b4-59f3-45a5-89b2-0a61caa1cba5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.667470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-540bd2b4-59f3-45a5-89b2-0a61caa1cba5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.668241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-540bd2b4-59f3-45a5-89b2-0a61caa1cba5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.669222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-540bd2b4-59f3-45a5-89b2-0a61caa1cba5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.690363 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1463267c-052e-4854-9b6e-1940d0a12aac tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.047255 Sep 27 10:04:14.691009 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 902/1979] 10.222.0.26 () {58 vars in 1176 bytes} [Tue Sep 27 10:04:14 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:04:14.695955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:14.696590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:14.704177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.704859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.705301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.706482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.707108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.707736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.755827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.756329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.757501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.759642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lazy-loading 'fault' on Instance uuid 6d511fc6-80ab-458e-88a6-593374a3e01c {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:14.854636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ac98fc0d0296493999906e8e3607c7f3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f48a592eb8fa4221a56eac1a50152c91', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:14.865966 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c8d3129-5055-4ab2-9f34-12ce4bd30c79 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 200 len: 1311 microversion: 2.1 time: 0.173826 Sep 27 10:04:14.867626 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 903/1980] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:14 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 1311 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:14.874428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": false, "disk_over_commit": false}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:14.877076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:14.884020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.885117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.885735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.887099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:14.887794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:14.888465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:14.897198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:14.927884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-540bd2b4-59f3-45a5-89b2-0a61caa1cba5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:14.943633 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-540bd2b4-59f3-45a5-89b2-0a61caa1cba5 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1317 microversion: 2.1 time: 0.307642 Sep 27 10:04:14.944722 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1078/1981] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:14 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1317 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:14.945028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lazy-loading 'flavor' on Instance uuid bd544723-d1ca-4f7e-a231-ba0b8351464d {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:14.951137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-05bf2f8d-81a9-4473-921e-e1e314761fba tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f638305e0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:14.964711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Going to try to live migrate instance to nch2-master-1 {{(pid=145969) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Sep 27 10:04:15.003786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:15.036598 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-05bf2f8d-81a9-4473-921e-e1e314761fba tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 200 len: 1437 microversion: 2.1 time: 0.090077 Sep 27 10:04:15.037303 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1079/1982] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:14 2022] GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 1437 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:15.229713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:15.230787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:15.239172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:15.239856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:15.240548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:15.241511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:15.242021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:15.243093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:15.509958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:15.532002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:15.533518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:15.541799 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e804af44-4005-450f-a936-67badd4b0753 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1810 microversion: 2.25 time: 0.318026 Sep 27 10:04:15.543322 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1080/1983] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:15 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1810 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:15.896587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb618f2e-f865-481c-9b51-c3d827c9b0e1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f63830160>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:15.921707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:15.973070 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cb618f2e-f865-481c-9b51-c3d827c9b0e1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/fa809ef6-dd19-4a85-b3be-62e8c2304c98" status: 200 len: 661 microversion: 2.1 time: 0.083157 Sep 27 10:04:15.974638 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1081/1984] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:15 2022] GET /compute/v2.1/images/fa809ef6-dd19-4a85-b3be-62e8c2304c98 => generated 661 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:15.991282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:15.992207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] [instance: 6d511fc6-80ab-458e-88a6-593374a3e01c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:15.999596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:16.000445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:16.001319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:16.002210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:16.002943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:16.003641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:16.021665 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] HTTP exception thrown: Instance 6d511fc6-80ab-458e-88a6-593374a3e01c could not be found. Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Request method failure captured: Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c HTTP/1.1 Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMso_T5-k00r-jNMmt-sE3AO4w-bk9w_R-TIb97BkNgtX430OKum_e_1Ajxl5FN5vu42mVBl2t9CzMeiQzlqPlORJKPijU1tf0C6z8at9kYV7c7xulwZegzv6clP9NU3qX8u7EF1IsTY4NHcT_P_xJX9RRB_V2Pz10WEXih8yHjsEtCw Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f48a592eb8fa4221a56eac1a50152c91 Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ListServerFiltersTestJSON-809664979 Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:04:16.023774 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f48a592eb8fa4221a56eac1a50152c91"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f48a592eb8fa4221a56eac1a50152c91"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f48a592eb8fa4221a56eac1a50152c91"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f48a592eb8fa4221a56eac1a50152c91"}]}] Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ListServerFiltersTestJSON-809664979 Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f48a592eb8fa4221a56eac1a50152c91 Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ListServerFiltersTestJSON-809664979 Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ListServerFiltersTestJSON-809664979-project Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: ac98fc0d0296493999906e8e3607c7f3 Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ListServerFiltersTestJSON-809664979-project Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 6d511fc6-80ab-458e-88a6-593374a3e01c could not be found. Sep 27 10:04:16.025678 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '6d511fc6-80ab-458e-88a6-593374a3e01c'} Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6d511fc6-80ab-458e-88a6-593374a3e01c could not be found. Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:04:16.026795 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6d511fc6-80ab-458e-88a6-593374a3e01c could not be found. Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6d511fc6-80ab-458e-88a6-593374a3e01c could not be found. Sep 27 10:04:16.028818 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:16.030632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] Returning 404 to user: Instance 6d511fc6-80ab-458e-88a6-593374a3e01c could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:04:16.033680 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1db58290-0866-4903-b953-1ce0a0931c39 tempest-ListServerFiltersTestJSON-809664979 tempest-ListServerFiltersTestJSON-809664979-project] 10.222.0.26 "GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c" status: 404 len: 111 microversion: 2.1 time: 0.046296 Sep 27 10:04:16.034617 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1082/1985] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:15 2022] GET /compute/v2.1/servers/6d511fc6-80ab-458e-88a6-593374a3e01c => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:16.051472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f00c42f4-c756-4bdf-8db8-3f24c5a94be4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:16.052592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f00c42f4-c756-4bdf-8db8-3f24c5a94be4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:16.061219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00c42f4-c756-4bdf-8db8-3f24c5a94be4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:16.062473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00c42f4-c756-4bdf-8db8-3f24c5a94be4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:16.063144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00c42f4-c756-4bdf-8db8-3f24c5a94be4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:16.064279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00c42f4-c756-4bdf-8db8-3f24c5a94be4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:16.065213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00c42f4-c756-4bdf-8db8-3f24c5a94be4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:16.066252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00c42f4-c756-4bdf-8db8-3f24c5a94be4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:16.547954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f00c42f4-c756-4bdf-8db8-3f24c5a94be4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:16.558167 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f00c42f4-c756-4bdf-8db8-3f24c5a94be4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1317 microversion: 2.1 time: 0.520190 Sep 27 10:04:16.558775 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1083/1986] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:16 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1317 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:16.574839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b61d2f9-2fc8-4eb4-889b-3eb8ae026d71 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6c2b4af0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:16.653991 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7b61d2f9-2fc8-4eb4-889b-3eb8ae026d71 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/fa809ef6-dd19-4a85-b3be-62e8c2304c98" status: 200 len: 661 microversion: 2.1 time: 0.094017 Sep 27 10:04:16.654594 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1084/1987] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:16 2022] GET /compute/v2.1/images/fa809ef6-dd19-4a85-b3be-62e8c2304c98 => generated 661 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:16.678836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-437f9421-3c17-448b-8fca-ab6dc36aa2f5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63815ca0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:16.929803 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-437f9421-3c17-448b-8fca-ab6dc36aa2f5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 200 len: 1437 microversion: 2.1 time: 0.273311 Sep 27 10:04:16.931457 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1085/1988] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:16 2022] GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 1437 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:16.944278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:16.946234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:16.965385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:16.966902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:16.969128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:16.970384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:16.971267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:16.972001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:17.271991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:17.286208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:17.288073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:17.292922 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d455a8c3-4e15-44fa-934d-fe38f1951999 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1800 microversion: 2.25 time: 0.357692 Sep 27 10:04:17.296172 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1086/1989] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:16 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1800 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:17.313464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fb9b3946-cdf4-4356-9b87-ed8e81faa92c tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Calling method '.version_select at 0x7f7f63830820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:17.388153 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fb9b3946-cdf4-4356-9b87-ed8e81faa92c tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee" status: 200 len: 637 microversion: 2.1 time: 0.089992 Sep 27 10:04:17.390378 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1087/1990] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:17 2022] GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee => generated 637 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:17.395639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bd601fd1-be98-4ef6-ace0-5fe6d60a060d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '.version_select at 0x7f7f6308baf0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:17.401051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd601fd1-be98-4ef6-ace0-5fe6d60a060d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:17.402052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd601fd1-be98-4ef6-ace0-5fe6d60a060d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:17.402804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd601fd1-be98-4ef6-ace0-5fe6d60a060d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:17.419061 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bd601fd1-be98-4ef6-ace0-5fe6d60a060d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/os-migrations" status: 200 len: 3157 microversion: 2.25 time: 0.026631 Sep 27 10:04:17.419984 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1088/1991] 10.222.0.26 () {60 vars in 1184 bytes} [Tue Sep 27 10:04:17 2022] GET /compute/v2.1/os-migrations => generated 3157 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:17.484797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-955cad8b-81cc-436d-b37b-49a7923d8367 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Action: 'update_all', calling method: .version_select at 0x7f7f6c22d3a0>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:17.583637 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5bcb3fca-9ff8-48bd-b17b-fb3c22fb86c1 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d/action" status: 202 len: 0 microversion: 2.1 time: 2.713913 Sep 27 10:04:17.584362 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 904/1992] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:04:14 2022] POST /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d/action => generated 0 bytes in 2715 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:04:17.592684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c80ed01f-b9c9-4358-a6f6-68d8acdee57d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:17.593905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c80ed01f-b9c9-4358-a6f6-68d8acdee57d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:17.602931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-955cad8b-81cc-436d-b37b-49a7923d8367 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:17.618266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80ed01f-b9c9-4358-a6f6-68d8acdee57d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:17.619076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80ed01f-b9c9-4358-a6f6-68d8acdee57d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:17.619827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80ed01f-b9c9-4358-a6f6-68d8acdee57d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:17.620850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80ed01f-b9c9-4358-a6f6-68d8acdee57d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:17.621517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80ed01f-b9c9-4358-a6f6-68d8acdee57d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:17.622410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80ed01f-b9c9-4358-a6f6-68d8acdee57d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:17.625171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-955cad8b-81cc-436d-b37b-49a7923d8367 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for user bff991c97d5a44969f435b4af0d71045 and project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:17.837746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-c80ed01f-b9c9-4358-a6f6-68d8acdee57d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid fcd14928-8ef7-425b-89bb-2c714284e840 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:17.860901 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c80ed01f-b9c9-4358-a6f6-68d8acdee57d tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 2418 microversion: 2.25 time: 0.274830 Sep 27 10:04:17.870819 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 905/1993] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:17 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 2418 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:17.877730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-baafb4a8-a220-4267-9bdf-c7be2bf7e023 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:17.879100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-baafb4a8-a220-4267-9bdf-c7be2bf7e023 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:17.890954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baafb4a8-a220-4267-9bdf-c7be2bf7e023 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:17.891753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baafb4a8-a220-4267-9bdf-c7be2bf7e023 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:17.892354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baafb4a8-a220-4267-9bdf-c7be2bf7e023 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:17.893782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baafb4a8-a220-4267-9bdf-c7be2bf7e023 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:17.894737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baafb4a8-a220-4267-9bdf-c7be2bf7e023 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:17.895438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baafb4a8-a220-4267-9bdf-c7be2bf7e023 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:18.206294 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-955cad8b-81cc-436d-b37b-49a7923d8367 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 61 microversion: 2.1 time: 0.784735 Sep 27 10:04:18.212916 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1089/1994] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:04:17 2022] PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 61 bytes in 792 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:18.219323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0386f429-d812-4d50-904d-fac6c66493db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:18.221352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0386f429-d812-4d50-904d-fac6c66493db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:18.229078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-baafb4a8-a220-4267-9bdf-c7be2bf7e023 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:18.247882 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-baafb4a8-a220-4267-9bdf-c7be2bf7e023 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1317 microversion: 2.1 time: 0.374514 Sep 27 10:04:18.248504 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 906/1995] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:17 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1317 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:18.267181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0386f429-d812-4d50-904d-fac6c66493db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:18.271756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0386f429-d812-4d50-904d-fac6c66493db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.027s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:18.273203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0386f429-d812-4d50-904d-fac6c66493db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:18.275072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f2e53eb1-e561-4412-a6da-c35cb3857116 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:18.277073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f2e53eb1-e561-4412-a6da-c35cb3857116 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:18.283212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0386f429-d812-4d50-904d-fac6c66493db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:18.293762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0386f429-d812-4d50-904d-fac6c66493db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:18.295593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0386f429-d812-4d50-904d-fac6c66493db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:18.305052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e53eb1-e561-4412-a6da-c35cb3857116 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:18.311775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e53eb1-e561-4412-a6da-c35cb3857116 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:18.318901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e53eb1-e561-4412-a6da-c35cb3857116 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:18.320230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e53eb1-e561-4412-a6da-c35cb3857116 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:18.321427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e53eb1-e561-4412-a6da-c35cb3857116 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:18.324181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2e53eb1-e561-4412-a6da-c35cb3857116 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:18.625890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0386f429-d812-4d50-904d-fac6c66493db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:18.644575 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0386f429-d812-4d50-904d-fac6c66493db tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1392 microversion: 2.1 time: 0.429441 Sep 27 10:04:18.660690 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1090/1996] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:18 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1392 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:18.660690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c4a5d802-5062-494e-ada5-65a1c3754765 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f680525e0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:18.752345 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c4a5d802-5062-494e-ada5-65a1c3754765 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 200 len: 1437 microversion: 2.1 time: 0.103799 Sep 27 10:04:18.753226 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1091/1997] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:18 2022] GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 1437 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:18.765271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c3b8cb22-8e58-42c1-ac42-fd717a1ca293 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Calling method '.version_select at 0x7f7f6c1734c0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:18.822707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-f2e53eb1-e561-4412-a6da-c35cb3857116 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid fcd14928-8ef7-425b-89bb-2c714284e840 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:18.850233 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f2e53eb1-e561-4412-a6da-c35cb3857116 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 2418 microversion: 2.25 time: 0.595681 Sep 27 10:04:18.851435 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 907/1998] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:18 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 2418 bytes in 598 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:18.857516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d1b9b761-9556-4098-95a0-f9ed39538bdf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f635c71f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:18.946707 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d1b9b761-9556-4098-95a0-f9ed39538bdf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/b57e736e-9630-498b-b438-5b87c52b4e9a" status: 200 len: 661 microversion: 2.1 time: 0.093369 Sep 27 10:04:18.949165 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 908/1999] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:18 2022] GET /compute/v2.1/images/b57e736e-9630-498b-b438-5b87c52b4e9a => generated 661 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:18.965112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-684f0a70-86d0-4e94-b8bc-73f76105f67a tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6c2228b0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:19.041663 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-684f0a70-86d0-4e94-b8bc-73f76105f67a tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/b57e736e-9630-498b-b438-5b87c52b4e9a" status: 200 len: 661 microversion: 2.1 time: 0.081637 Sep 27 10:04:19.044679 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 909/2000] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:18 2022] GET /compute/v2.1/images/b57e736e-9630-498b-b438-5b87c52b4e9a => generated 661 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:19.062639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-177a58a9-4520-4085-a36c-7f644f3ff2de tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:19.063161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-177a58a9-4520-4085-a36c-7f644f3ff2de tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:19.075662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-177a58a9-4520-4085-a36c-7f644f3ff2de tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:19.079555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-177a58a9-4520-4085-a36c-7f644f3ff2de tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:19.080672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-177a58a9-4520-4085-a36c-7f644f3ff2de tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:19.081677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-177a58a9-4520-4085-a36c-7f644f3ff2de tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:19.084421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-177a58a9-4520-4085-a36c-7f644f3ff2de tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:19.087183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-177a58a9-4520-4085-a36c-7f644f3ff2de tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:19.254080 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c3b8cb22-8e58-42c1-ac42-fd717a1ca293 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "DELETE /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata/os_version" status: 204 len: 0 microversion: 2.1 time: 0.492833 Sep 27 10:04:19.258401 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1092/2001] 10.222.0.26 () {58 vars in 1296 bytes} [Tue Sep 27 10:04:18 2022] DELETE /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata/os_version => generated 0 bytes in 499 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:19.266827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1f649762-23a4-46f9-8063-47d9b074de5e tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Calling method '.version_select at 0x7f7f6c2bb940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:19.344716 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1f649762-23a4-46f9-8063-47d9b074de5e tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 37 microversion: 2.1 time: 0.083952 Sep 27 10:04:19.347116 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1093/2002] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:04:19 2022] GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 37 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 27 10:04:19.373846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7041af77-96f9-44af-b8af-8e1ca6ef9870 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:19.374746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7041af77-96f9-44af-b8af-8e1ca6ef9870 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:19.382640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7041af77-96f9-44af-b8af-8e1ca6ef9870 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:19.383202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7041af77-96f9-44af-b8af-8e1ca6ef9870 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:19.385046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7041af77-96f9-44af-b8af-8e1ca6ef9870 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:19.385408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7041af77-96f9-44af-b8af-8e1ca6ef9870 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:19.386062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7041af77-96f9-44af-b8af-8e1ca6ef9870 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:19.386486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7041af77-96f9-44af-b8af-8e1ca6ef9870 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:19.437273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-177a58a9-4520-4085-a36c-7f644f3ff2de tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lazy-loading 'services' on Instance uuid fcd14928-8ef7-425b-89bb-2c714284e840 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:19.636909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7041af77-96f9-44af-b8af-8e1ca6ef9870 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:19.653357 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7041af77-96f9-44af-b8af-8e1ca6ef9870 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1317 microversion: 2.1 time: 0.300459 Sep 27 10:04:19.655959 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1094/2003] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:19 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1317 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:19.673188 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-177a58a9-4520-4085-a36c-7f644f3ff2de tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "DELETE /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/os-volume_attachments/3791482b-9549-44ab-8cae-118f99dd9b3e" status: 202 len: 0 microversion: 2.25 time: 0.626337 Sep 27 10:04:19.674697 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 910/2004] 10.222.0.26 () {60 vars in 1457 bytes} [Tue Sep 27 10:04:19 2022] DELETE /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/os-volume_attachments/3791482b-9549-44ab-8cae-118f99dd9b3e => generated 0 bytes in 629 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:19.696318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-44356b72-d949-4a2c-97ca-b361428408c9 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Action: 'update_all', calling method: .version_select at 0x7f7f6c184ca0>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:19.703971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListImageFiltersTestJSON-server-920774636", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:19.803138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-44356b72-d949-4a2c-97ca-b361428408c9 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:19.835472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-44356b72-d949-4a2c-97ca-b361428408c9 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for user bff991c97d5a44969f435b4af0d71045 and project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:19.878687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:04:20.395689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:04:20.396543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:04:20.397148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:04:20.397723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:04:20.398158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:04:20.420982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:20.421887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:20.423979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:20.425470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:20.427498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:20.429588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:20.488184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Getting quotas for project b932820432be493b8a85520f58f49606. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:20.497488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Getting quotas for user 032de99964be4424ba019adf2eedf680 and project b932820432be493b8a85520f58f49606. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:20.509892 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-44356b72-d949-4a2c-97ca-b361428408c9 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 61 microversion: 2.1 time: 0.827188 Sep 27 10:04:20.512336 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1095/2005] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:04:19 2022] PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 61 bytes in 839 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:20.518638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67624651-7b13-4394-9a22-de129f759dd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:20.521162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:04:20.521466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67624651-7b13-4394-9a22-de129f759dd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:20.525415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:04:20.539488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67624651-7b13-4394-9a22-de129f759dd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:20.543234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67624651-7b13-4394-9a22-de129f759dd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:20.545033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67624651-7b13-4394-9a22-de129f759dd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:20.546691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67624651-7b13-4394-9a22-de129f759dd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:20.548981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67624651-7b13-4394-9a22-de129f759dd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:20.552478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67624651-7b13-4394-9a22-de129f759dd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:20.597938 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4726465e-59c8-4d83-b591-ac6ff1cb945d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.910084 Sep 27 10:04:20.598830 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 911/2006] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:04:19 2022] POST /compute/v2.1/servers => generated 380 bytes in 919 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:04:20.621266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9fddf4d4-8ca7-4437-aaf7-fbe01a49671c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:20.628267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9fddf4d4-8ca7-4437-aaf7-fbe01a49671c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:20.647153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fddf4d4-8ca7-4437-aaf7-fbe01a49671c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:20.651202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fddf4d4-8ca7-4437-aaf7-fbe01a49671c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:20.653103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fddf4d4-8ca7-4437-aaf7-fbe01a49671c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:20.655816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fddf4d4-8ca7-4437-aaf7-fbe01a49671c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:20.656319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fddf4d4-8ca7-4437-aaf7-fbe01a49671c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:20.656798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9fddf4d4-8ca7-4437-aaf7-fbe01a49671c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:20.763330 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9fddf4d4-8ca7-4437-aaf7-fbe01a49671c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/os-volume_attachments" status: 200 len: 197 microversion: 2.25 time: 0.159012 Sep 27 10:04:20.765199 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 912/2007] 10.222.0.26 () {60 vars in 1343 bytes} [Tue Sep 27 10:04:20 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/os-volume_attachments => generated 197 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:04:20.783652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f283b656-917e-486f-bd27-c53ea3bde82e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62e5e790>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:20.889217 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f283b656-917e-486f-bd27-c53ea3bde82e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 200 len: 1437 microversion: 2.1 time: 0.121085 Sep 27 10:04:20.893177 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 913/2008] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:20 2022] GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 1437 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:20.913083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7be62b22-61c6-415b-a452-cf05f8d30089 req-3c9afbc8-65bf-4573-94ad-0fac4d4c58d2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b563b3f-326c-41ed-9185-2a6d112c871d", "name": "network-vif-plugged", "status": "completed", "tag": "917af05f-ebc3-433e-a57d-d09e51b4d3c7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:20.940771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7be62b22-61c6-415b-a452-cf05f8d30089 req-3c9afbc8-65bf-4573-94ad-0fac4d4c58d2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:20.944026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7be62b22-61c6-415b-a452-cf05f8d30089 req-3c9afbc8-65bf-4573-94ad-0fac4d4c58d2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:20.944650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7be62b22-61c6-415b-a452-cf05f8d30089 req-3c9afbc8-65bf-4573-94ad-0fac4d4c58d2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:20.980386 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7be62b22-61c6-415b-a452-cf05f8d30089 req-3c9afbc8-65bf-4573-94ad-0fac4d4c58d2 service nova] Creating event network-vif-plugged:917af05f-ebc3-433e-a57d-d09e51b4d3c7 for instance 3b563b3f-326c-41ed-9185-2a6d112c871d on nch1-master-1 Sep 27 10:04:20.983670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-67624651-7b13-4394-9a22-de129f759dd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:20.995700 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7be62b22-61c6-415b-a452-cf05f8d30089 req-3c9afbc8-65bf-4573-94ad-0fac4d4c58d2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097060 Sep 27 10:04:20.998272 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 914/2009] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:21.000292 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67624651-7b13-4394-9a22-de129f759dd6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1392 microversion: 2.1 time: 0.486625 Sep 27 10:04:21.004897 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1096/2010] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:20 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1392 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:21.022866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3de74ffc-8c96-49c8-a79a-8c300478d327 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Calling method '.version_select at 0x7f7f635c7f70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:21.023052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-71f9e71f-3875-421c-823d-5e6f89edbfcb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:21.023052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-71f9e71f-3875-421c-823d-5e6f89edbfcb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:21.036819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f9e71f-3875-421c-823d-5e6f89edbfcb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:21.037538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f9e71f-3875-421c-823d-5e6f89edbfcb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:21.040839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f9e71f-3875-421c-823d-5e6f89edbfcb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:21.043016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f9e71f-3875-421c-823d-5e6f89edbfcb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:21.043343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f9e71f-3875-421c-823d-5e6f89edbfcb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:21.043813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f9e71f-3875-421c-823d-5e6f89edbfcb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:21.107028 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3de74ffc-8c96-49c8-a79a-8c300478d327 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata/os_distro" status: 200 len: 33 microversion: 2.1 time: 0.101859 Sep 27 10:04:21.107801 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 915/2011] 10.222.0.26 () {58 vars in 1290 bytes} [Tue Sep 27 10:04:21 2022] GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata/os_distro => generated 33 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:21.125192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListImageFiltersTestJSON-server-14878888", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:21.260865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:04:21.380609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-71f9e71f-3875-421c-823d-5e6f89edbfcb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:21.398763 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-71f9e71f-3875-421c-823d-5e6f89edbfcb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1317 microversion: 2.1 time: 0.391280 Sep 27 10:04:21.401186 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1097/2012] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:21 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1317 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:21.409912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-02fde1f1-75e6-4c05-be5c-8787c7eb3b74 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Action: 'update_all', calling method: .version_select at 0x7f7f680525e0>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:21.532161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-02fde1f1-75e6-4c05-be5c-8787c7eb3b74 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:21.558933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-02fde1f1-75e6-4c05-be5c-8787c7eb3b74 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for user bff991c97d5a44969f435b4af0d71045 and project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:21.843665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:04:21.844334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:04:21.844904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:04:21.845727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:04:21.846442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:04:21.882495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:21.885472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:21.887525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:21.890702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:21.891289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:21.893504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:21.940814 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-02fde1f1-75e6-4c05-be5c-8787c7eb3b74 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 61 microversion: 2.1 time: 0.537832 Sep 27 10:04:21.941422 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1098/2013] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:04:21 2022] PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 61 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:21.952455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e5056880-f4c4-4518-a6ae-9896f1eacd64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:21.957987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Getting quotas for project b932820432be493b8a85520f58f49606. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:21.958663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e5056880-f4c4-4518-a6ae-9896f1eacd64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:21.967838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:21.968812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Getting quotas for user 032de99964be4424ba019adf2eedf680 and project b932820432be493b8a85520f58f49606. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:21.973336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5056880-f4c4-4518-a6ae-9896f1eacd64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:21.974089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5056880-f4c4-4518-a6ae-9896f1eacd64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:21.974717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5056880-f4c4-4518-a6ae-9896f1eacd64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:21.975650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5056880-f4c4-4518-a6ae-9896f1eacd64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:21.976466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5056880-f4c4-4518-a6ae-9896f1eacd64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:21.977246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5056880-f4c4-4518-a6ae-9896f1eacd64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:21.986660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:04:21.991226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:04:22.025120 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e5056880-f4c4-4518-a6ae-9896f1eacd64 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/os-volume_attachments" status: 200 len: 25 microversion: 2.25 time: 0.082275 Sep 27 10:04:22.031954 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1099/2014] 10.222.0.26 () {60 vars in 1343 bytes} [Tue Sep 27 10:04:21 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840/os-volume_attachments => generated 25 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:22.070119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c7b8d088-3fbf-4be6-893a-a6366c964212 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63ae1700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:22.099950 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1f3e8030-d51f-4fec-9b13-47296ae15ab1 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.983192 Sep 27 10:04:22.103755 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 916/2015] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:04:21 2022] POST /compute/v2.1/servers => generated 380 bytes in 995 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:04:22.142610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c3591859-aa9b-4246-b944-aa0c7a72b16b tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Calling method '.version_select at 0x7f7f62e63f70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:22.186287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c7b8d088-3fbf-4be6-893a-a6366c964212 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 200 len: 1437 microversion: 2.1 time: 0.158575 Sep 27 10:04:22.194791 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1100/2016] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:22 2022] GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 1437 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:22.201309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dfe0db8d-b8b8-4d18-9ccd-da861a54acf7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:22.202202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dfe0db8d-b8b8-4d18-9ccd-da861a54acf7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:22.208635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfe0db8d-b8b8-4d18-9ccd-da861a54acf7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:22.209351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfe0db8d-b8b8-4d18-9ccd-da861a54acf7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:22.209957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfe0db8d-b8b8-4d18-9ccd-da861a54acf7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:22.210863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfe0db8d-b8b8-4d18-9ccd-da861a54acf7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:22.211569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfe0db8d-b8b8-4d18-9ccd-da861a54acf7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:22.212152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfe0db8d-b8b8-4d18-9ccd-da861a54acf7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:22.278357 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c3591859-aa9b-4246-b944-aa0c7a72b16b tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 61 microversion: 2.1 time: 0.168985 Sep 27 10:04:22.284071 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 917/2017] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:04:22 2022] GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 61 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:22.328451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-478a9eee-d137-48d5-b52b-fc75c6208f57 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:22.333482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-478a9eee-d137-48d5-b52b-fc75c6208f57 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:22.512909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dfe0db8d-b8b8-4d18-9ccd-da861a54acf7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:22.533710 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dfe0db8d-b8b8-4d18-9ccd-da861a54acf7 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1392 microversion: 2.1 time: 0.337327 Sep 27 10:04:22.537455 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1101/2018] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:22 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1392 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:22.552834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-478a9eee-d137-48d5-b52b-fc75c6208f57 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:22.568088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92406261-cc17-4d10-a0b9-638fdca013d5 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Action: 'update_all', calling method: .version_select at 0x7f7f63b2d0d0>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:22.585583 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-478a9eee-d137-48d5-b52b-fc75c6208f57 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1216 microversion: 2.1 time: 0.295787 Sep 27 10:04:22.589636 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 918/2019] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:22 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1216 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:22.613572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a9db1020-1509-4139-a17f-03d6d5270deb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:22.623879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a9db1020-1509-4139-a17f-03d6d5270deb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:22.686436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9db1020-1509-4139-a17f-03d6d5270deb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:22.689459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9db1020-1509-4139-a17f-03d6d5270deb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:22.692802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9db1020-1509-4139-a17f-03d6d5270deb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:22.696890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9db1020-1509-4139-a17f-03d6d5270deb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:22.702197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9db1020-1509-4139-a17f-03d6d5270deb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:22.705476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9db1020-1509-4139-a17f-03d6d5270deb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:22.836878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-92406261-cc17-4d10-a0b9-638fdca013d5 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:22.854027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-92406261-cc17-4d10-a0b9-638fdca013d5 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for user bff991c97d5a44969f435b4af0d71045 and project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:23.200158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a9db1020-1509-4139-a17f-03d6d5270deb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:23.211781 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a9db1020-1509-4139-a17f-03d6d5270deb tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1467 microversion: 2.1 time: 0.619804 Sep 27 10:04:23.213144 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 919/2020] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:22 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1467 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:23.234736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ccdcb2c3-b959-4d2e-a1f2-6c96f899bdcd req-0b65299d-2ec3-42ec-a486-68dc949e3342 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "152a5758-d14b-4ca9-917e-f7017c6d99b2", "name": "network-vif-plugged", "status": "completed", "tag": "61c12b2f-7268-4252-a482-600cb57542ca"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:23.249140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ccdcb2c3-b959-4d2e-a1f2-6c96f899bdcd req-0b65299d-2ec3-42ec-a486-68dc949e3342 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:23.251971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ccdcb2c3-b959-4d2e-a1f2-6c96f899bdcd req-0b65299d-2ec3-42ec-a486-68dc949e3342 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:23.253594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ccdcb2c3-b959-4d2e-a1f2-6c96f899bdcd req-0b65299d-2ec3-42ec-a486-68dc949e3342 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:23.335730 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ccdcb2c3-b959-4d2e-a1f2-6c96f899bdcd req-0b65299d-2ec3-42ec-a486-68dc949e3342 service nova] Creating event network-vif-plugged:61c12b2f-7268-4252-a482-600cb57542ca for instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 on nch2-master-1 Sep 27 10:04:23.348775 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ccdcb2c3-b959-4d2e-a1f2-6c96f899bdcd req-0b65299d-2ec3-42ec-a486-68dc949e3342 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.133603 Sep 27 10:04:23.350591 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 920/2021] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:23.381854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-863759da-5a2f-4358-b82b-b9a20f0bb42d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62e97280>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:23.604444 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-863759da-5a2f-4358-b82b-b9a20f0bb42d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 200 len: 1437 microversion: 2.1 time: 0.249242 Sep 27 10:04:23.617594 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 921/2022] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:23 2022] GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 1437 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:23.634849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dd75a320-e6b9-4179-9742-d322ca22be32 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-image-976310778", "metadata": {"image_type": "test"}}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:23.650821 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-92406261-cc17-4d10-a0b9-638fdca013d5 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 61 microversion: 2.1 time: 1.090311 Sep 27 10:04:23.652902 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1102/2023] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:04:22 2022] PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 61 bytes in 1093 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:23.662793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-61beb0ee-2cd9-4e1c-b980-384fdcedbd69 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:23.663490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dd75a320-e6b9-4179-9742-d322ca22be32 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:23.666522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-61beb0ee-2cd9-4e1c-b980-384fdcedbd69 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:23.690208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd75a320-e6b9-4179-9742-d322ca22be32 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:23.691932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61beb0ee-2cd9-4e1c-b980-384fdcedbd69 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:23.693169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61beb0ee-2cd9-4e1c-b980-384fdcedbd69 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:23.693662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd75a320-e6b9-4179-9742-d322ca22be32 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:23.696209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd75a320-e6b9-4179-9742-d322ca22be32 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:23.696543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61beb0ee-2cd9-4e1c-b980-384fdcedbd69 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:23.697767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd75a320-e6b9-4179-9742-d322ca22be32 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:23.699867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61beb0ee-2cd9-4e1c-b980-384fdcedbd69 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:23.702103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd75a320-e6b9-4179-9742-d322ca22be32 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:23.703020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61beb0ee-2cd9-4e1c-b980-384fdcedbd69 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:23.703788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd75a320-e6b9-4179-9742-d322ca22be32 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:23.705085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61beb0ee-2cd9-4e1c-b980-384fdcedbd69 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:23.834685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:23.867502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-dd75a320-e6b9-4179-9742-d322ca22be32 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Getting quotas for project 58c33b046dc7494c970f26eb5900b3f3. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:23.885624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-dd75a320-e6b9-4179-9742-d322ca22be32 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Getting quotas for user a10600a9277a436396fc70971dd27893 and project 58c33b046dc7494c970f26eb5900b3f3. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:24.349334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-61beb0ee-2cd9-4e1c-b980-384fdcedbd69 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:24.360132 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-61beb0ee-2cd9-4e1c-b980-384fdcedbd69 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1392 microversion: 2.1 time: 0.705577 Sep 27 10:04:24.363155 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1103/2024] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:23 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1392 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:24.381740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e771422-b698-4dd0-8197-156a080c1417 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:24.387965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8e771422-b698-4dd0-8197-156a080c1417 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:24.413122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e771422-b698-4dd0-8197-156a080c1417 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:24.414412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e771422-b698-4dd0-8197-156a080c1417 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:24.415569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e771422-b698-4dd0-8197-156a080c1417 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:24.417018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e771422-b698-4dd0-8197-156a080c1417 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:24.418520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e771422-b698-4dd0-8197-156a080c1417 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:24.420364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e771422-b698-4dd0-8197-156a080c1417 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:24.468640 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dd75a320-e6b9-4179-9742-d322ca22be32 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "POST /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d/action" status: 202 len: 0 microversion: 2.1 time: 0.847064 Sep 27 10:04:24.475398 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 922/2025] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:04:23 2022] POST /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d/action => generated 0 bytes in 853 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 10:04:24.483781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5ae1f886-3b43-442f-ab0b-096d2898613b tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Action: 'update_all', calling method: .version_select at 0x7f7f635f2040>, body: {"metadata": {"os_version": "value2", "architecture": "value3"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:24.710402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8e771422-b698-4dd0-8197-156a080c1417 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:24.742857 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e771422-b698-4dd0-8197-156a080c1417 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1212 microversion: 2.1 time: 0.377955 Sep 27 10:04:24.748795 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1104/2026] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:24 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1212 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:24.768228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a139b6c-a12c-457b-9960-fb9cbbf92d9d tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63ac3310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:24.832571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5ae1f886-3b43-442f-ab0b-096d2898613b tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:24.890757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5ae1f886-3b43-442f-ab0b-096d2898613b tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for user bff991c97d5a44969f435b4af0d71045 and project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:25.074294 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a139b6c-a12c-457b-9960-fb9cbbf92d9d tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 200 len: 1421 microversion: 2.1 time: 0.319748 Sep 27 10:04:25.094555 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1105/2027] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:24 2022] GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 1421 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:25.125895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2aa093a6-1ad3-4b54-820b-c6012c27039b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c2d9310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:25.435265 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2aa093a6-1ad3-4b54-820b-c6012c27039b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 200 len: 1437 microversion: 2.1 time: 0.335143 Sep 27 10:04:25.445109 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1106/2028] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:25 2022] GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 1437 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:25.469895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-40d22c08-9813-4110-899f-93be23903528 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:25.470838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-40d22c08-9813-4110-899f-93be23903528 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:25.522920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40d22c08-9813-4110-899f-93be23903528 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:25.528813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40d22c08-9813-4110-899f-93be23903528 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:25.532321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40d22c08-9813-4110-899f-93be23903528 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:25.534234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40d22c08-9813-4110-899f-93be23903528 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:25.536955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40d22c08-9813-4110-899f-93be23903528 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:25.539844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40d22c08-9813-4110-899f-93be23903528 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:26.223062 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5ae1f886-3b43-442f-ab0b-096d2898613b tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 64 microversion: 2.1 time: 1.748429 Sep 27 10:04:26.223844 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 923/2029] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:04:24 2022] PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 64 bytes in 1752 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:26.234977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8b86a64d-6df7-48a6-8d37-1f7861b6524f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:26.246283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8b86a64d-6df7-48a6-8d37-1f7861b6524f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:26.280559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b86a64d-6df7-48a6-8d37-1f7861b6524f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:26.282698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b86a64d-6df7-48a6-8d37-1f7861b6524f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:26.283694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b86a64d-6df7-48a6-8d37-1f7861b6524f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:26.286248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b86a64d-6df7-48a6-8d37-1f7861b6524f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:26.287157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b86a64d-6df7-48a6-8d37-1f7861b6524f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:26.292125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b86a64d-6df7-48a6-8d37-1f7861b6524f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:26.450643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-40d22c08-9813-4110-899f-93be23903528 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:26.470191 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-40d22c08-9813-4110-899f-93be23903528 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1392 microversion: 2.1 time: 1.020087 Sep 27 10:04:26.472595 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1107/2030] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:25 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1392 bytes in 1027 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:26.483724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a6afd476-9227-4bb4-abba-b94aa0594f5d tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f6c2b48b0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:26.606304 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a6afd476-9227-4bb4-abba-b94aa0594f5d tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 200 len: 1421 microversion: 2.1 time: 0.125591 Sep 27 10:04:26.607648 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1108/2031] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:26 2022] GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 1421 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:26.644081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e779be1d-fbbd-45b5-b6c9-07149528e617 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Calling method '.version_select at 0x7f7f63dac670>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:26.760378 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e779be1d-fbbd-45b5-b6c9-07149528e617 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 64 microversion: 2.1 time: 0.135303 Sep 27 10:04:26.768225 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1109/2032] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:04:26 2022] GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 64 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:26.776911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ea01b111-47f4-4c76-80d5-e0e4c349e37b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63f36ca0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:26.929669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8b86a64d-6df7-48a6-8d37-1f7861b6524f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:26.930612 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ea01b111-47f4-4c76-80d5-e0e4c349e37b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 200 len: 1437 microversion: 2.1 time: 0.158525 Sep 27 10:04:26.932649 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1110/2033] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:26 2022] GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 1437 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:26.948042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f5bfb619-79c1-4c85-83c1-5c070cd74d61 req-fe780132-6de9-484e-8831-60f0ed976cbc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bd544723-d1ca-4f7e-a231-ba0b8351464d", "name": "network-vif-unplugged", "status": "completed", "tag": "f2d7b45b-7823-4ee9-a9a3-1e4c5cda1e27"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:26.959444 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8b86a64d-6df7-48a6-8d37-1f7861b6524f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1276 microversion: 2.1 time: 0.733394 Sep 27 10:04:26.960056 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 924/2034] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:26 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1276 bytes in 735 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:26.983454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ebeecca2-5c36-49f8-8300-0bbc5ce62b86 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Action: 'update_all', calling method: .version_select at 0x7f7f639bb670>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:27.001208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f5bfb619-79c1-4c85-83c1-5c070cd74d61 req-fe780132-6de9-484e-8831-60f0ed976cbc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:27.003043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f5bfb619-79c1-4c85-83c1-5c070cd74d61 req-fe780132-6de9-484e-8831-60f0ed976cbc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:27.003887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f5bfb619-79c1-4c85-83c1-5c070cd74d61 req-fe780132-6de9-484e-8831-60f0ed976cbc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:27.054076 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f5bfb619-79c1-4c85-83c1-5c070cd74d61 req-fe780132-6de9-484e-8831-60f0ed976cbc service nova] Creating event network-vif-unplugged:f2d7b45b-7823-4ee9-a9a3-1e4c5cda1e27 for instance bd544723-d1ca-4f7e-a231-ba0b8351464d on nch1-master-1 Sep 27 10:04:27.069249 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f5bfb619-79c1-4c85-83c1-5c070cd74d61 req-fe780132-6de9-484e-8831-60f0ed976cbc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.130897 Sep 27 10:04:27.074128 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1111/2035] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:26 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:27.092829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ebeecca2-5c36-49f8-8300-0bbc5ce62b86 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:27.109855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ebeecca2-5c36-49f8-8300-0bbc5ce62b86 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for user bff991c97d5a44969f435b4af0d71045 and project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:27.509717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c038dbdc-de61-4632-9a25-f18570fe6854 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:27.511790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c038dbdc-de61-4632-9a25-f18570fe6854 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:27.526930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c038dbdc-de61-4632-9a25-f18570fe6854 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:27.528130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c038dbdc-de61-4632-9a25-f18570fe6854 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:27.528765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c038dbdc-de61-4632-9a25-f18570fe6854 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:27.529866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c038dbdc-de61-4632-9a25-f18570fe6854 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:27.530512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c038dbdc-de61-4632-9a25-f18570fe6854 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:27.531177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c038dbdc-de61-4632-9a25-f18570fe6854 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:28.012147 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ebeecca2-5c36-49f8-8300-0bbc5ce62b86 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 61 microversion: 2.1 time: 1.050725 Sep 27 10:04:28.013267 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 925/2036] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:04:26 2022] PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 61 bytes in 1052 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:28.367183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-LiveAutoBlockMigrationV225Test-server-84872688", "imageRef": "", "flavorRef": "452", "networks": [{"uuid": "43a35817-b5c9-4e15-8250-6b84b0c2b25f"}], "block_device_mapping_v2": [{"uuid": "9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:28.509170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] validate_networks() for [('43a35817-b5c9-4e15-8250-6b84b0c2b25f', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:04:28.712166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c038dbdc-de61-4632-9a25-f18570fe6854 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:28.722610 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c038dbdc-de61-4632-9a25-f18570fe6854 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1392 microversion: 2.1 time: 1.225260 Sep 27 10:04:28.724114 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1112/2037] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:27 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1392 bytes in 1229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:28.733857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-77954d70-fcd9-40c8-8c02-e56089645f5a tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63830820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:28.887325 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-77954d70-fcd9-40c8-8c02-e56089645f5a tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 200 len: 1421 microversion: 2.1 time: 0.157405 Sep 27 10:04:28.889581 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1113/2038] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:28 2022] GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 1421 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:28.903977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e4ec0ec3-e71e-469d-b853-9bc300458fc9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6392b8b0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:30.062518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:04:30.063783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:04:30.064455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:04:30.065147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:04:30.065746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:04:30.083092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:30.093516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:30.094338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:30.095486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:30.096153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:30.096840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:30.121463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=15 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:30.147545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Getting quotas for project 2369f5ee467643948f618bbdee837521. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:30.164973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Getting quotas for user 29d2f7b0e8274c769521d13930be9825 and project 2369f5ee467643948f618bbdee837521. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:30.284126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:04:30.286557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='9f3d9ce1-0df8-4932-8943-e8e22e9ae4f5',volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:04:30.543576 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dbc98aea-3eb7-4fb8-af2c-cae136fcaa10 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.25 time: 2.529138 Sep 27 10:04:30.552175 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 926/2039] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:04:28 2022] POST /compute/v2.1/servers => generated 380 bytes in 2538 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:04:30.568434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b855de20-0a57-47da-9f4b-058668b716d6 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:30.569096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b855de20-0a57-47da-9f4b-058668b716d6 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:30.595666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b855de20-0a57-47da-9f4b-058668b716d6 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:30.604624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b855de20-0a57-47da-9f4b-058668b716d6 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:30.605564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b855de20-0a57-47da-9f4b-058668b716d6 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:30.608854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b855de20-0a57-47da-9f4b-058668b716d6 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:30.609556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b855de20-0a57-47da-9f4b-058668b716d6 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:30.612892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b855de20-0a57-47da-9f4b-058668b716d6 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:30.887406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b855de20-0a57-47da-9f4b-058668b716d6 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:30.896341 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b855de20-0a57-47da-9f4b-058668b716d6 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1274 microversion: 2.1 time: 0.334574 Sep 27 10:04:30.897035 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 927/2040] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:30 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1274 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:30.915341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7d8e95d8-282f-496e-ac4f-ea5699352ad7 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Action: 'update', calling method: .version_select at 0x7f7f639abd30>, body: {"meta": {"os_version": "alt"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:32.937698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=16 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:37.524906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=20 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:38.717119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:40.137946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:40.185524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=28 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:40.573543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:40.891939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=86,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:43.443711 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e4ec0ec3-e71e-469d-b853-9bc300458fc9 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 200 len: 1446 microversion: 2.1 time: 14.552325 Sep 27 10:04:43.453984 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1114/2041] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:28 2022] GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 1446 bytes in 14563 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:43.459374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f391d8b2-d0f8-472f-a912-5f0c8b54017c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:43.460819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f391d8b2-d0f8-472f-a912-5f0c8b54017c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:43.462609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7d8e95d8-282f-496e-ac4f-ea5699352ad7 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:43.468521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7d8e95d8-282f-496e-ac4f-ea5699352ad7 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for user bff991c97d5a44969f435b4af0d71045 and project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:43.468854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f391d8b2-d0f8-472f-a912-5f0c8b54017c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:43.469557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f391d8b2-d0f8-472f-a912-5f0c8b54017c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:43.470006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f391d8b2-d0f8-472f-a912-5f0c8b54017c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:43.470726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f391d8b2-d0f8-472f-a912-5f0c8b54017c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:43.471374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f391d8b2-d0f8-472f-a912-5f0c8b54017c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:43.471806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f391d8b2-d0f8-472f-a912-5f0c8b54017c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:43.651804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f391d8b2-d0f8-472f-a912-5f0c8b54017c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:43.668124 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f391d8b2-d0f8-472f-a912-5f0c8b54017c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1382 microversion: 2.1 time: 0.212929 Sep 27 10:04:43.668744 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1115/2042] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:43 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1382 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:43.678348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-2e2082d5-1a36-43ee-8142-a87b511186dd req-2a7de439-bd1e-4636-8d32-f1b3f0201f14 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bd544723-d1ca-4f7e-a231-ba0b8351464d", "tag": "f2d7b45b-7823-4ee9-a9a3-1e4c5cda1e27"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:43.708028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2e2082d5-1a36-43ee-8142-a87b511186dd req-2a7de439-bd1e-4636-8d32-f1b3f0201f14 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:43.708841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2e2082d5-1a36-43ee-8142-a87b511186dd req-2a7de439-bd1e-4636-8d32-f1b3f0201f14 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:43.709448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2e2082d5-1a36-43ee-8142-a87b511186dd req-2a7de439-bd1e-4636-8d32-f1b3f0201f14 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:43.725251 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-2e2082d5-1a36-43ee-8142-a87b511186dd req-2a7de439-bd1e-4636-8d32-f1b3f0201f14 service nova] Creating event network-changed:f2d7b45b-7823-4ee9-a9a3-1e4c5cda1e27 for instance bd544723-d1ca-4f7e-a231-ba0b8351464d on nch2-master-1 Sep 27 10:04:43.732867 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-2e2082d5-1a36-43ee-8142-a87b511186dd req-2a7de439-bd1e-4636-8d32-f1b3f0201f14 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057633 Sep 27 10:04:43.736082 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1116/2043] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:43.741856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-37c5ced1-78b6-4fb5-9ef3-23f04ae3ad6f tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f637dc790>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:43.877206 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7d8e95d8-282f-496e-ac4f-ea5699352ad7 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata/os_version" status: 200 len: 31 microversion: 2.1 time: 12.973832 Sep 27 10:04:43.877842 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 928/2044] 10.222.0.26 () {60 vars in 1313 bytes} [Tue Sep 27 10:04:30 2022] PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata/os_version => generated 31 bytes in 12976 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:43.894610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:43.894971 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-37c5ced1-78b6-4fb5-9ef3-23f04ae3ad6f tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 200 len: 1421 microversion: 2.1 time: 0.158739 Sep 27 10:04:43.895740 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1117/2045] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:43 2022] GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 1421 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:43.902783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:43.910729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d96c6e1d-2277-43d2-9c66-6ccb65a2efbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:43.913551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:43.914064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:43.915283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d96c6e1d-2277-43d2-9c66-6ccb65a2efbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:43.915755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:43.916569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:43.917391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:43.918900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:43.935219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d96c6e1d-2277-43d2-9c66-6ccb65a2efbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:43.935667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d96c6e1d-2277-43d2-9c66-6ccb65a2efbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:43.937245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d96c6e1d-2277-43d2-9c66-6ccb65a2efbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:43.938156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d96c6e1d-2277-43d2-9c66-6ccb65a2efbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:43.938790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d96c6e1d-2277-43d2-9c66-6ccb65a2efbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:43.939534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d96c6e1d-2277-43d2-9c66-6ccb65a2efbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:44.108064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:44.130494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d96c6e1d-2277-43d2-9c66-6ccb65a2efbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:44.131311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:44.133329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:44.135022 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a1ddc21-ecf8-43a3-bfbf-d275e7998931 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1279 microversion: 2.25 time: 0.255628 Sep 27 10:04:44.136298 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 929/2046] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:43 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1279 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:44.143355 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d96c6e1d-2277-43d2-9c66-6ccb65a2efbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1316 microversion: 2.1 time: 0.241236 Sep 27 10:04:44.143548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-72f37349-a5fe-4091-bfa8-a130adaa2ccf req-ddbfcce3-950c-4673-aa4b-950d32cd0c03 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b563b3f-326c-41ed-9185-2a6d112c871d", "name": "network-vif-plugged", "status": "completed", "tag": "917af05f-ebc3-433e-a57d-d09e51b4d3c7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:44.150804 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1118/2047] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:43 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1316 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:44.155006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-72f37349-a5fe-4091-bfa8-a130adaa2ccf req-ddbfcce3-950c-4673-aa4b-950d32cd0c03 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:44.155737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-72f37349-a5fe-4091-bfa8-a130adaa2ccf req-ddbfcce3-950c-4673-aa4b-950d32cd0c03 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:44.156345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-72f37349-a5fe-4091-bfa8-a130adaa2ccf req-ddbfcce3-950c-4673-aa4b-950d32cd0c03 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:44.195013 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-72f37349-a5fe-4091-bfa8-a130adaa2ccf req-ddbfcce3-950c-4673-aa4b-950d32cd0c03 service nova] Creating event network-vif-plugged:917af05f-ebc3-433e-a57d-d09e51b4d3c7 for instance 3b563b3f-326c-41ed-9185-2a6d112c871d on nch1-master-1 Sep 27 10:04:44.205300 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-72f37349-a5fe-4091-bfa8-a130adaa2ccf req-ddbfcce3-950c-4673-aa4b-950d32cd0c03 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067405 Sep 27 10:04:44.207417 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 930/2048] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:44.215853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d0f0fa01-bdf4-48bf-bc07-711350375347 req-d19928e7-5bd7-4115-98b9-0bb61e0bb4ec service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b0aa6e63-d0ba-47c2-8f6b-572f168104ee", "tag": "f97afb19-a003-40a4-a6f6-2158ddcc5448"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:44.227047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d0f0fa01-bdf4-48bf-bc07-711350375347 req-d19928e7-5bd7-4115-98b9-0bb61e0bb4ec service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:44.227885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d0f0fa01-bdf4-48bf-bc07-711350375347 req-d19928e7-5bd7-4115-98b9-0bb61e0bb4ec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:44.229443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d0f0fa01-bdf4-48bf-bc07-711350375347 req-d19928e7-5bd7-4115-98b9-0bb61e0bb4ec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:44.245991 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d0f0fa01-bdf4-48bf-bc07-711350375347 req-d19928e7-5bd7-4115-98b9-0bb61e0bb4ec service nova] Creating event network-changed:f97afb19-a003-40a4-a6f6-2158ddcc5448 for instance b0aa6e63-d0ba-47c2-8f6b-572f168104ee on nch1-master-1 Sep 27 10:04:44.253594 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d0f0fa01-bdf4-48bf-bc07-711350375347 req-d19928e7-5bd7-4115-98b9-0bb61e0bb4ec service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042901 Sep 27 10:04:44.254564 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 931/2049] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:44.260486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f85a5e9b-ab65-458c-9ffe-fcb20caa2637 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:44.262971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerAddressesTestJSON-server-1858371224", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "a2e3a09c-b364-4505-8abd-56c9c062ef28"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:44.266927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f85a5e9b-ab65-458c-9ffe-fcb20caa2637 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:44.267993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f85a5e9b-ab65-458c-9ffe-fcb20caa2637 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Searching by: {'deleted': False, 'project_id': '183cb066566c4510945d6eb0d41a2853'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:04:44.274206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f85a5e9b-ab65-458c-9ffe-fcb20caa2637 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:44.274982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f85a5e9b-ab65-458c-9ffe-fcb20caa2637 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:44.275932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f85a5e9b-ab65-458c-9ffe-fcb20caa2637 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:44.277059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f85a5e9b-ab65-458c-9ffe-fcb20caa2637 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:44.278425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f85a5e9b-ab65-458c-9ffe-fcb20caa2637 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:44.279084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f85a5e9b-ab65-458c-9ffe-fcb20caa2637 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:44.334381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-f85a5e9b-ab65-458c-9ffe-fcb20caa2637 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:04:44.392128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] validate_networks() for [('a2e3a09c-b364-4505-8abd-56c9c062ef28', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:04:44.404321 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f85a5e9b-ab65-458c-9ffe-fcb20caa2637 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 344 microversion: 2.1 time: 0.148122 Sep 27 10:04:44.404947 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 932/2050] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:04:44 2022] GET /compute/v2.1/servers => generated 344 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:44.418484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8753276d-8412-4281-a8d0-4cb095c43f06 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6361c5e0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:44.490864 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8753276d-8412-4281-a8d0-4cb095c43f06 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 200 len: 1446 microversion: 2.1 time: 0.083208 Sep 27 10:04:44.492471 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 933/2051] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:44 2022] GET /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 1446 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:44.498637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9550ab0e-996e-4898-aa14-2c84d29a4e40 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '.version_select at 0x7f7f6c222430>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:44.503166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9550ab0e-996e-4898-aa14-2c84d29a4e40 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:44.504032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9550ab0e-996e-4898-aa14-2c84d29a4e40 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:44.504681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9550ab0e-996e-4898-aa14-2c84d29a4e40 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:44.523008 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9550ab0e-996e-4898-aa14-2c84d29a4e40 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/os-migrations" status: 200 len: 2704 microversion: 2.1 time: 0.028794 Sep 27 10:04:44.524394 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 934/2052] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 10:04:44 2022] GET /compute/v2.1/os-migrations => generated 2704 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:44.531581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80587d94-84c0-402e-a40e-04af7e4dde22 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Calling method '.version_select at 0x7f7f638fb1f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:44.614674 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-80587d94-84c0-402e-a40e-04af7e4dde22 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 58 microversion: 2.1 time: 0.087966 Sep 27 10:04:44.615590 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 935/2053] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:04:44 2022] GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 58 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:44.629717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:44.637246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:44.638233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Searching by: {'deleted': False, 'project_id': '183cb066566c4510945d6eb0d41a2853'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:04:44.648248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:44.651815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:44.653412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:44.654737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:44.655282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:44.655814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:44.691636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:04:44.820011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:44.831785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:44.832946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:44.834583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:45.044135 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f3d3a7d4-c39c-4c13-8a93-3e6fe8116854 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/detail" status: 200 len: 1605 microversion: 2.1 time: 0.420287 Sep 27 10:04:45.045036 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 936/2054] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:04:44 2022] GET /compute/v2.1/servers/detail => generated 1605 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:45.072472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a7098b3-2c05-40d5-ae25-5f3abebea7fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63629a60>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:45.252090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:04:45.252978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:04:45.253379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:04:45.254323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:04:45.254883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:04:45.269747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:45.271695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:45.272502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:45.273683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:45.274443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:45.275162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:45.300228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Getting quotas for project 3f33c209ffd1480e94175241ccc90414. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:45.307449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Getting quotas for user 297a11c331fb473291750ced9c843a6e and project 3f33c209ffd1480e94175241ccc90414. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:45.325844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:04:45.329429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:04:45.469605 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-988b5124-4d65-4b9c-b73f-31041747801c tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.317302 Sep 27 10:04:45.470442 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1119/2055] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:04:44 2022] POST /compute/v2.1/servers => generated 380 bytes in 1319 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:04:45.483649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d7a19deb-f6b9-4a3f-9c88-80db21b763f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:45.496220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d7a19deb-f6b9-4a3f-9c88-80db21b763f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:45.534793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7a19deb-f6b9-4a3f-9c88-80db21b763f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:45.535693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7a19deb-f6b9-4a3f-9c88-80db21b763f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:45.536570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7a19deb-f6b9-4a3f-9c88-80db21b763f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:45.540721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7a19deb-f6b9-4a3f-9c88-80db21b763f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:45.542200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7a19deb-f6b9-4a3f-9c88-80db21b763f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:45.543187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7a19deb-f6b9-4a3f-9c88-80db21b763f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:46.055894 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d7a19deb-f6b9-4a3f-9c88-80db21b763f3 tempest-LiveMigrationTest-1179050027 tempest-LiveMigrationTest-1179050027-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1533 microversion: 2.1 time: 0.580227 Sep 27 10:04:46.059153 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1120/2056] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:45 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1533 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:46.110223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dab9c7c6-90f8-4d81-a297-0d40c3e055b7 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Action: 'update_all', calling method: .version_select at 0x7f7f6819e550>, body: {"metadata": {"os_version": "value1", "os_distro": "value2"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:46.248436 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a7098b3-2c05-40d5-ae25-5f3abebea7fb tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 204 len: 0 microversion: 2.1 time: 1.200881 Sep 27 10:04:46.250502 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 937/2057] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:45 2022] DELETE /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 0 bytes in 1205 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:46.256164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a13bc39d-b1a3-413e-8e11-2a52437967c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f635f2040>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:46.375460 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a13bc39d-b1a3-413e-8e11-2a52437967c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 200 len: 1421 microversion: 2.1 time: 0.123528 Sep 27 10:04:46.376513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-dab9c7c6-90f8-4d81-a297-0d40c3e055b7 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:46.385570 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 938/2058] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:46 2022] GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 1421 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:46.385570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-309a9236-2469-461a-842e-9b4e7163733b tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:46.395680 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-309a9236-2469-461a-842e-9b4e7163733b tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.016755 Sep 27 10:04:46.396619 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 939/2059] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:04:46 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:46.397732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-dab9c7c6-90f8-4d81-a297-0d40c3e055b7 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for user bff991c97d5a44969f435b4af0d71045 and project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:46.402068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:46.402804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:46.411737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:46.412264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:46.412817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:46.413948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:46.414700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:46.419347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:46.816978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:46.831626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:46.844315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:46.851395 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f4e0ae29-c63f-492e-bfda-603d43401f9f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1279 microversion: 2.25 time: 0.451454 Sep 27 10:04:46.855320 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 940/2060] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:46 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1279 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:46.875297 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dab9c7c6-90f8-4d81-a297-0d40c3e055b7 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 61 microversion: 2.1 time: 0.814797 Sep 27 10:04:46.876541 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1121/2061] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:04:46 2022] PUT /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 61 bytes in 817 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:46.884756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fdc6177e-5733-493b-8892-c7f7a35499da tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:46.887393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f6e6a034-7316-4b11-9dce-80b886d27d65 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:46.888714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fdc6177e-5733-493b-8892-c7f7a35499da tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:46.890250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f6e6a034-7316-4b11-9dce-80b886d27d65 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:46.899704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdc6177e-5733-493b-8892-c7f7a35499da tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:46.901725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdc6177e-5733-493b-8892-c7f7a35499da tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:46.902627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdc6177e-5733-493b-8892-c7f7a35499da tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:46.906818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdc6177e-5733-493b-8892-c7f7a35499da tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:46.907361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdc6177e-5733-493b-8892-c7f7a35499da tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:46.907907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdc6177e-5733-493b-8892-c7f7a35499da tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:46.924997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e6a034-7316-4b11-9dce-80b886d27d65 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:46.926913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e6a034-7316-4b11-9dce-80b886d27d65 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:46.928498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e6a034-7316-4b11-9dce-80b886d27d65 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:46.930499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e6a034-7316-4b11-9dce-80b886d27d65 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:46.932348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e6a034-7316-4b11-9dce-80b886d27d65 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:46.934134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e6a034-7316-4b11-9dce-80b886d27d65 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:47.018442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fdc6177e-5733-493b-8892-c7f7a35499da tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:47.046128 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fdc6177e-5733-493b-8892-c7f7a35499da tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1221 microversion: 2.1 time: 0.167668 Sep 27 10:04:47.049540 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1122/2062] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:46 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1221 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:47.056193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-79af496e-2b4c-4b19-b572-89144e42a8ad req-ca7ea333-9247-4011-832f-c2f758c46521 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "961407e4-8e66-4f1c-88bd-6d2b4795c11d", "tag": "52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f"}, {"name": "network-changed", "server_uuid": "55ead6af-a0d9-4d6f-b742-4a3fd4a418ad", "tag": "a33d85f6-8d2f-4192-8ab2-06ef5d149185"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:47.131118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-79af496e-2b4c-4b19-b572-89144e42a8ad req-ca7ea333-9247-4011-832f-c2f758c46521 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:47.135534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-79af496e-2b4c-4b19-b572-89144e42a8ad req-ca7ea333-9247-4011-832f-c2f758c46521 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:47.135776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-79af496e-2b4c-4b19-b572-89144e42a8ad req-ca7ea333-9247-4011-832f-c2f758c46521 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:47.208685 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-79af496e-2b4c-4b19-b572-89144e42a8ad req-ca7ea333-9247-4011-832f-c2f758c46521 service nova] Creating event network-changed:52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f for instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d on nch2-master-1 Sep 27 10:04:47.212779 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-79af496e-2b4c-4b19-b572-89144e42a8ad req-ca7ea333-9247-4011-832f-c2f758c46521 service nova] Creating event network-changed:a33d85f6-8d2f-4192-8ab2-06ef5d149185 for instance 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad on nch2-master-1 Sep 27 10:04:47.230212 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-79af496e-2b4c-4b19-b572-89144e42a8ad req-ca7ea333-9247-4011-832f-c2f758c46521 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.175887 Sep 27 10:04:47.233465 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1123/2063] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:47 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:47.249003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8d401e23-4139-4ea6-ab72-486bddca97b0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:47.253077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8d401e23-4139-4ea6-ab72-486bddca97b0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:47.268315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d401e23-4139-4ea6-ab72-486bddca97b0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:47.268861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d401e23-4139-4ea6-ab72-486bddca97b0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:47.272208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d401e23-4139-4ea6-ab72-486bddca97b0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:47.275546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d401e23-4139-4ea6-ab72-486bddca97b0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:47.277457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d401e23-4139-4ea6-ab72-486bddca97b0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:47.278972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d401e23-4139-4ea6-ab72-486bddca97b0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:47.343421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8d401e23-4139-4ea6-ab72-486bddca97b0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:04:47.499510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6e6a034-7316-4b11-9dce-80b886d27d65 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:47.514140 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8d401e23-4139-4ea6-ab72-486bddca97b0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "DELETE /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 204 len: 0 microversion: 2.1 time: 0.277920 Sep 27 10:04:47.515520 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1124/2064] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:04:47 2022] DELETE /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:47.516040 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f6e6a034-7316-4b11-9dce-80b886d27d65 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1316 microversion: 2.1 time: 0.659272 Sep 27 10:04:47.516640 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 941/2065] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:46 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1316 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:47.529312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-807c39b5-973e-4b21-87c7-4db56c5c7a1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:47.529312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-807c39b5-973e-4b21-87c7-4db56c5c7a1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:47.537423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-807c39b5-973e-4b21-87c7-4db56c5c7a1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:47.538505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-807c39b5-973e-4b21-87c7-4db56c5c7a1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:47.540185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-807c39b5-973e-4b21-87c7-4db56c5c7a1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:47.541268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-807c39b5-973e-4b21-87c7-4db56c5c7a1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:47.542124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-807c39b5-973e-4b21-87c7-4db56c5c7a1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:47.543674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-807c39b5-973e-4b21-87c7-4db56c5c7a1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:47.562629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e7c6d32c-de4a-4a51-8520-9576de0a74dd req-ebd8450e-3dd9-4f1c-958d-fa94ec93a366 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0aa6e63-d0ba-47c2-8f6b-572f168104ee", "name": "network-vif-plugged", "status": "completed", "tag": "f97afb19-a003-40a4-a6f6-2158ddcc5448"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:47.583337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-807c39b5-973e-4b21-87c7-4db56c5c7a1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:04:47.589784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e7c6d32c-de4a-4a51-8520-9576de0a74dd req-ebd8450e-3dd9-4f1c-958d-fa94ec93a366 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:47.590474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e7c6d32c-de4a-4a51-8520-9576de0a74dd req-ebd8450e-3dd9-4f1c-958d-fa94ec93a366 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:47.591174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e7c6d32c-de4a-4a51-8520-9576de0a74dd req-ebd8450e-3dd9-4f1c-958d-fa94ec93a366 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:47.642342 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e7c6d32c-de4a-4a51-8520-9576de0a74dd req-ebd8450e-3dd9-4f1c-958d-fa94ec93a366 service nova] Creating event network-vif-plugged:f97afb19-a003-40a4-a6f6-2158ddcc5448 for instance b0aa6e63-d0ba-47c2-8f6b-572f168104ee on nch1-master-1 Sep 27 10:04:47.654107 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e7c6d32c-de4a-4a51-8520-9576de0a74dd req-ebd8450e-3dd9-4f1c-958d-fa94ec93a366 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.135487 Sep 27 10:04:47.655110 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1125/2066] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:47.661095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:47.661926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:47.670505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:47.671812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:47.671812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:47.672600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:47.674300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:47.675020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:47.721866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:04:47.747171 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-807c39b5-973e-4b21-87c7-4db56c5c7a1a tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 204 len: 0 microversion: 2.1 time: 0.229175 Sep 27 10:04:47.748959 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 942/2067] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:04:47 2022] DELETE /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:47.758433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-55c03eb1-d34a-41cd-8337-771de7f41edf tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Action: 'create', calling method: .version_select at 0x7f7f635d48b0>, body: {"metadata": {"os_version": "alt1", "architecture": "value3"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:47.848263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-55c03eb1-d34a-41cd-8337-771de7f41edf tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:47.855671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-55c03eb1-d34a-41cd-8337-771de7f41edf tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Getting quotas for user bff991c97d5a44969f435b4af0d71045 and project 63978eadb2a84ba7bada10574661830d. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:47.979082 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-12102dde-7aaf-495b-beb1-8d8a2c66fe1a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "DELETE /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 204 len: 0 microversion: 2.1 time: 0.322250 Sep 27 10:04:47.979833 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1126/2068] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:04:47 2022] DELETE /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:47.993766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c089df10-f163-4219-a46a-10497d90c273 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f6c2d9310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:48.094069 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c089df10-f163-4219-a46a-10497d90c273 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 200 len: 1421 microversion: 2.1 time: 0.112281 Sep 27 10:04:48.094735 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1127/2069] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:47 2022] GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 1421 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:48.254511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-63124cf1-2d06-4043-bbf7-cc8070253062 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:48.255270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-63124cf1-2d06-4043-bbf7-cc8070253062 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:48.266920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63124cf1-2d06-4043-bbf7-cc8070253062 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:48.267726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63124cf1-2d06-4043-bbf7-cc8070253062 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:48.268335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63124cf1-2d06-4043-bbf7-cc8070253062 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:48.269292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63124cf1-2d06-4043-bbf7-cc8070253062 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:48.269924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63124cf1-2d06-4043-bbf7-cc8070253062 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:48.270514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63124cf1-2d06-4043-bbf7-cc8070253062 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:48.347958 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-55c03eb1-d34a-41cd-8337-771de7f41edf tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "POST /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 85 microversion: 2.1 time: 0.597421 Sep 27 10:04:48.348553 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 943/2070] 10.222.0.26 () {60 vars in 1281 bytes} [Tue Sep 27 10:04:47 2022] POST /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 85 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:48.353795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:48.354546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:48.361220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:48.361947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:48.362550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:48.364326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:48.364719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:48.365147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:48.429551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-63124cf1-2d06-4043-bbf7-cc8070253062 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:48.442547 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-63124cf1-2d06-4043-bbf7-cc8070253062 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1388 microversion: 2.1 time: 0.346049 Sep 27 10:04:48.443732 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1128/2071] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:48 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1388 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:48.450156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d99abe89-8e28-4cae-aa5a-daf27689352a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:48.451261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d99abe89-8e28-4cae-aa5a-daf27689352a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:48.458928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99abe89-8e28-4cae-aa5a-daf27689352a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:48.459673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99abe89-8e28-4cae-aa5a-daf27689352a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:48.460315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99abe89-8e28-4cae-aa5a-daf27689352a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:48.461672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99abe89-8e28-4cae-aa5a-daf27689352a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:48.462258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99abe89-8e28-4cae-aa5a-daf27689352a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:48.462941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d99abe89-8e28-4cae-aa5a-daf27689352a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:48.561749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:48.574675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:48.576497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:48.577877 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-48177f78-d3aa-4344-a9af-4909c7781615 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1279 microversion: 2.25 time: 0.228312 Sep 27 10:04:48.578586 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 944/2072] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:48 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1279 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:48.584367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesTestJSON-server-852111209", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "4568aca4-c33f-4379-b9d1-22f70d115851"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:48.639805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d99abe89-8e28-4cae-aa5a-daf27689352a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:48.649589 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d99abe89-8e28-4cae-aa5a-daf27689352a tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1608 microversion: 2.1 time: 0.203895 Sep 27 10:04:48.665747 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1129/2073] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:48 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1608 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:48.665747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a40b4cf1-a7c5-4a04-bb55-1017c24531a8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:48.665747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a40b4cf1-a7c5-4a04-bb55-1017c24531a8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:48.667745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40b4cf1-a7c5-4a04-bb55-1017c24531a8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:48.668250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40b4cf1-a7c5-4a04-bb55-1017c24531a8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:48.668795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40b4cf1-a7c5-4a04-bb55-1017c24531a8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:48.669430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40b4cf1-a7c5-4a04-bb55-1017c24531a8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:48.669887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40b4cf1-a7c5-4a04-bb55-1017c24531a8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:48.670308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a40b4cf1-a7c5-4a04-bb55-1017c24531a8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:48.672371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] validate_networks() for [('4568aca4-c33f-4379-b9d1-22f70d115851', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:04:48.740723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a40b4cf1-a7c5-4a04-bb55-1017c24531a8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:48.749246 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a40b4cf1-a7c5-4a04-bb55-1017c24531a8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1269 microversion: 2.1 time: 0.097907 Sep 27 10:04:48.760439 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1130/2074] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:48 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1269 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:48.760439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-59b6c3f5-62f2-4470-9192-b346bb312b94 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] Calling method '.version_select at 0x7f7f6c2d9af0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:48.800256 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-59b6c3f5-62f2-4470-9192-b346bb312b94 tempest-ImagesMetadataTestJSON-730315380 tempest-ImagesMetadataTestJSON-730315380-project] 10.222.0.26 "GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata" status: 200 len: 85 microversion: 2.1 time: 0.049333 Sep 27 10:04:48.807373 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1131/2075] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:04:48 2022] GET /compute/v2.1/images/bf54b126-4bf8-423a-8e0a-32fa6ad0b4ee/metadata => generated 85 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:04:48.807373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-03b2642e-f133-4cd2-ae85-f470d3f07c92 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:48.808613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-03b2642e-f133-4cd2-ae85-f470d3f07c92 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:48.814565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03b2642e-f133-4cd2-ae85-f470d3f07c92 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:48.815271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03b2642e-f133-4cd2-ae85-f470d3f07c92 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:48.815785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03b2642e-f133-4cd2-ae85-f470d3f07c92 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:48.816539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03b2642e-f133-4cd2-ae85-f470d3f07c92 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:48.817068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03b2642e-f133-4cd2-ae85-f470d3f07c92 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:48.817503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03b2642e-f133-4cd2-ae85-f470d3f07c92 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:48.983687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03b2642e-f133-4cd2-ae85-f470d3f07c92 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:48.993798 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-03b2642e-f133-4cd2-ae85-f470d3f07c92 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1316 microversion: 2.1 time: 0.191337 Sep 27 10:04:48.996692 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1132/2076] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:48 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1316 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:49.112261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e833027-ab87-40f2-b1bb-213941013807 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63830700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:49.282269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:04:49.282706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:04:49.283190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:04:49.283775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:04:49.284526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:04:49.306307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:49.307021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:49.307475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:49.308230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:49.308395 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7e833027-ab87-40f2-b1bb-213941013807 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 200 len: 1421 microversion: 2.1 time: 0.200433 Sep 27 10:04:49.308715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:49.309157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:49.309495 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1133/2077] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:49 2022] GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 1421 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:49.341067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Getting quotas for project e5a565f65b3849a99b74454791397ca5. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:04:49.346564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Getting quotas for user 6eb3ccfcd9c344e1a7ad55f8398ec87a and project e5a565f65b3849a99b74454791397ca5. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:04:49.362306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:04:49.365866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:04:49.429088 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e10f6fe5-12f0-437b-8579-9a31b47fcbe1 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.848953 Sep 27 10:04:49.430505 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 945/2078] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:04:48 2022] POST /compute/v2.1/servers => generated 380 bytes in 851 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:04:49.443380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6a18a16b-bc5f-4455-99b9-1f38b62720ce tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:49.444315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6a18a16b-bc5f-4455-99b9-1f38b62720ce tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:49.467678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-44afb339-97ca-489e-8411-fdd21c9655a9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:49.468634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-44afb339-97ca-489e-8411-fdd21c9655a9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:49.484395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44afb339-97ca-489e-8411-fdd21c9655a9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:49.487018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44afb339-97ca-489e-8411-fdd21c9655a9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:49.487710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44afb339-97ca-489e-8411-fdd21c9655a9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:49.488755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44afb339-97ca-489e-8411-fdd21c9655a9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:49.489386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44afb339-97ca-489e-8411-fdd21c9655a9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:49.491304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44afb339-97ca-489e-8411-fdd21c9655a9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:49.567877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6a18a16b-bc5f-4455-99b9-1f38b62720ce tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:49.577378 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6a18a16b-bc5f-4455-99b9-1f38b62720ce tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1207 microversion: 2.1 time: 0.139348 Sep 27 10:04:49.584172 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1134/2079] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:49 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1207 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:49.613744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:49.614546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:49.630615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:49.631129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:49.631588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:49.632355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:49.632840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:49.633365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:49.881393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-44afb339-97ca-489e-8411-fdd21c9655a9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:49.889624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:49.891160 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-44afb339-97ca-489e-8411-fdd21c9655a9 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1388 microversion: 2.1 time: 0.427502 Sep 27 10:04:49.892276 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 946/2080] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:49 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1388 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:49.898343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-431eac61-2297-4324-b7f8-dd7eb64a8a44 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:49.899266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-431eac61-2297-4324-b7f8-dd7eb64a8a44 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:49.900023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:49.902532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:49.903660 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d14014a0-fa3a-4973-a36e-28d952be6dd5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1279 microversion: 2.25 time: 0.294889 Sep 27 10:04:49.906425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-431eac61-2297-4324-b7f8-dd7eb64a8a44 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:49.907249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-431eac61-2297-4324-b7f8-dd7eb64a8a44 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:49.907917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-431eac61-2297-4324-b7f8-dd7eb64a8a44 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:49.908919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-431eac61-2297-4324-b7f8-dd7eb64a8a44 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:49.909606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-431eac61-2297-4324-b7f8-dd7eb64a8a44 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:49.910256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-431eac61-2297-4324-b7f8-dd7eb64a8a44 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:49.917309 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1135/2081] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:49 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1279 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:04:49.928849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8474c83b-35cb-4269-a76f-2d6e90c07786 req-1d049110-3ae7-4197-9537-e8ee73d34d05 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "961407e4-8e66-4f1c-88bd-6d2b4795c11d", "name": "network-vif-plugged", "status": "completed", "tag": "52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:49.940953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8474c83b-35cb-4269-a76f-2d6e90c07786 req-1d049110-3ae7-4197-9537-e8ee73d34d05 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:49.941750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8474c83b-35cb-4269-a76f-2d6e90c07786 req-1d049110-3ae7-4197-9537-e8ee73d34d05 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:49.942391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8474c83b-35cb-4269-a76f-2d6e90c07786 req-1d049110-3ae7-4197-9537-e8ee73d34d05 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:49.975331 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8474c83b-35cb-4269-a76f-2d6e90c07786 req-1d049110-3ae7-4197-9537-e8ee73d34d05 service nova] Creating event network-vif-plugged:52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f for instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d on nch2-master-1 Sep 27 10:04:49.988995 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8474c83b-35cb-4269-a76f-2d6e90c07786 req-1d049110-3ae7-4197-9537-e8ee73d34d05 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062277 Sep 27 10:04:49.996111 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1136/2082] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:50.025439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3c02a3e-d592-4fe0-8378-df8804975817 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:50.026927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3c02a3e-d592-4fe0-8378-df8804975817 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:50.045990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3c02a3e-d592-4fe0-8378-df8804975817 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:50.046908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3c02a3e-d592-4fe0-8378-df8804975817 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:50.048577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3c02a3e-d592-4fe0-8378-df8804975817 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:50.049689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3c02a3e-d592-4fe0-8378-df8804975817 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:50.050755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3c02a3e-d592-4fe0-8378-df8804975817 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:50.051938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3c02a3e-d592-4fe0-8378-df8804975817 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:50.081653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:50.175580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b3c02a3e-d592-4fe0-8378-df8804975817 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:50.184358 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3c02a3e-d592-4fe0-8378-df8804975817 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1269 microversion: 2.1 time: 0.184766 Sep 27 10:04:50.189308 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1137/2083] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:49 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1269 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:50.198563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-369cab08-655f-486f-bc26-37b9289a3fbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:50.199638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-369cab08-655f-486f-bc26-37b9289a3fbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:50.226519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-369cab08-655f-486f-bc26-37b9289a3fbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:50.228395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-369cab08-655f-486f-bc26-37b9289a3fbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:50.229388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-369cab08-655f-486f-bc26-37b9289a3fbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:50.230837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-369cab08-655f-486f-bc26-37b9289a3fbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:50.231316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-369cab08-655f-486f-bc26-37b9289a3fbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:50.231872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-369cab08-655f-486f-bc26-37b9289a3fbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:50.485699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-369cab08-655f-486f-bc26-37b9289a3fbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:50.493950 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-369cab08-655f-486f-bc26-37b9289a3fbe tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1316 microversion: 2.1 time: 0.300434 Sep 27 10:04:50.494531 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1138/2084] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:50 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1316 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:50.500008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-66f6e823-df25-4393-aeb1-a9d6f7957bdf tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f637dc280>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:50.562248 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-66f6e823-df25-4393-aeb1-a9d6f7957bdf tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 200 len: 1430 microversion: 2.1 time: 0.065351 Sep 27 10:04:50.562942 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1139/2085] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:50 2022] GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 1430 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:50.577603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bfe26b4b-07c4-4590-a5c5-9085f33eb55b tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63aa3820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:50.601601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-431eac61-2297-4324-b7f8-dd7eb64a8a44 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:50.616382 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-431eac61-2297-4324-b7f8-dd7eb64a8a44 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1608 microversion: 2.1 time: 0.722032 Sep 27 10:04:50.619161 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 947/2086] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:49 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1608 bytes in 726 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:50.626998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3c976f5a-d3f1-4338-a498-ce880cc9a50b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:50.627995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3c976f5a-d3f1-4338-a498-ce880cc9a50b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:50.659483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c976f5a-d3f1-4338-a498-ce880cc9a50b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:50.660001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c976f5a-d3f1-4338-a498-ce880cc9a50b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:50.660447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c976f5a-d3f1-4338-a498-ce880cc9a50b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:50.661982 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bfe26b4b-07c4-4590-a5c5-9085f33eb55b tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 200 len: 1430 microversion: 2.1 time: 0.088343 Sep 27 10:04:50.662917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c976f5a-d3f1-4338-a498-ce880cc9a50b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:50.663398 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1140/2087] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:50 2022] GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 1430 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:50.665767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c976f5a-d3f1-4338-a498-ce880cc9a50b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:50.666258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c976f5a-d3f1-4338-a498-ce880cc9a50b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:50.678889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2f5912cf-5d1d-40fd-847c-69bb46d64460 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:50.680999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2f5912cf-5d1d-40fd-847c-69bb46d64460 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:50.690583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f5912cf-5d1d-40fd-847c-69bb46d64460 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:50.691486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f5912cf-5d1d-40fd-847c-69bb46d64460 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:50.692539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f5912cf-5d1d-40fd-847c-69bb46d64460 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:50.694459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f5912cf-5d1d-40fd-847c-69bb46d64460 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:50.695736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f5912cf-5d1d-40fd-847c-69bb46d64460 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:50.697256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f5912cf-5d1d-40fd-847c-69bb46d64460 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:50.837403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c976f5a-d3f1-4338-a498-ce880cc9a50b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:50.852860 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3c976f5a-d3f1-4338-a498-ce880cc9a50b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1211 microversion: 2.1 time: 0.231275 Sep 27 10:04:50.876627 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 948/2088] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:50 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1211 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:50.919422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-83d3535d-885d-44cd-b6ca-204b2faecdf0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:50.920942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-83d3535d-885d-44cd-b6ca-204b2faecdf0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:50.933304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83d3535d-885d-44cd-b6ca-204b2faecdf0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:50.934181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83d3535d-885d-44cd-b6ca-204b2faecdf0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:50.934858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83d3535d-885d-44cd-b6ca-204b2faecdf0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:50.935918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83d3535d-885d-44cd-b6ca-204b2faecdf0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:50.936991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83d3535d-885d-44cd-b6ca-204b2faecdf0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:50.937644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83d3535d-885d-44cd-b6ca-204b2faecdf0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:51.367181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2f5912cf-5d1d-40fd-847c-69bb46d64460 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:51.381768 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2f5912cf-5d1d-40fd-847c-69bb46d64460 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1467 microversion: 2.1 time: 0.708329 Sep 27 10:04:51.383775 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1141/2089] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:50 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1467 bytes in 711 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:51.411098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:51.416220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:51.435584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:51.436879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:51.437738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:51.443234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:51.443916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:51.444464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:51.719678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-83d3535d-885d-44cd-b6ca-204b2faecdf0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:51.731922 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-83d3535d-885d-44cd-b6ca-204b2faecdf0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1388 microversion: 2.1 time: 0.817847 Sep 27 10:04:51.736043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:51.750269 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 949/2090] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:50 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1388 bytes in 837 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:04:51.756428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a90a354e-0139-4305-b7d1-723f14138d5f tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:51.757362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a90a354e-0139-4305-b7d1-723f14138d5f tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:51.760036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:51.764307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:51.765827 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4ed394c6-d67c-4a58-81ec-b83c692e3603 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1474 microversion: 2.25 time: 0.365029 Sep 27 10:04:51.768149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a90a354e-0139-4305-b7d1-723f14138d5f tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:51.768313 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1142/2091] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:51 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1474 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:51.770040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a90a354e-0139-4305-b7d1-723f14138d5f tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:51.770881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a90a354e-0139-4305-b7d1-723f14138d5f tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:51.772784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a90a354e-0139-4305-b7d1-723f14138d5f tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:51.773420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a90a354e-0139-4305-b7d1-723f14138d5f tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:51.775073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a90a354e-0139-4305-b7d1-723f14138d5f tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:51.778852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c8948a6-8bb2-4454-96f4-d78081a9c0e2 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f6c2bb8b0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:51.854775 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c8948a6-8bb2-4454-96f4-d78081a9c0e2 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6" status: 200 len: 784 microversion: 2.1 time: 0.083992 Sep 27 10:04:51.855991 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1143/2092] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:51 2022] GET /compute/v2.1/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6 => generated 784 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:51.871240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3697afca-c427-4f39-b072-b34ccff338f7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:51.872006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3697afca-c427-4f39-b072-b34ccff338f7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:51.893276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3697afca-c427-4f39-b072-b34ccff338f7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:51.893877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3697afca-c427-4f39-b072-b34ccff338f7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:51.894341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3697afca-c427-4f39-b072-b34ccff338f7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:51.895111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3697afca-c427-4f39-b072-b34ccff338f7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:51.895576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3697afca-c427-4f39-b072-b34ccff338f7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:51.897677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3697afca-c427-4f39-b072-b34ccff338f7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:51.940047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a90a354e-0139-4305-b7d1-723f14138d5f tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:51.955435 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a90a354e-0139-4305-b7d1-723f14138d5f tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1269 microversion: 2.1 time: 0.197663 Sep 27 10:04:51.956421 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 950/2093] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:51 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1269 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:51.969373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b873fdf2-0703-4184-9676-bc505e2d067c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:51.982075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b873fdf2-0703-4184-9676-bc505e2d067c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:51.989460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b873fdf2-0703-4184-9676-bc505e2d067c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:51.989959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b873fdf2-0703-4184-9676-bc505e2d067c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:51.990473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b873fdf2-0703-4184-9676-bc505e2d067c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:51.991334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b873fdf2-0703-4184-9676-bc505e2d067c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:51.991587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b873fdf2-0703-4184-9676-bc505e2d067c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:52.001606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b873fdf2-0703-4184-9676-bc505e2d067c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:52.059814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:52.404904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b873fdf2-0703-4184-9676-bc505e2d067c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:52.419121 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b873fdf2-0703-4184-9676-bc505e2d067c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1608 microversion: 2.1 time: 0.453434 Sep 27 10:04:52.421680 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 951/2094] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:51 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1608 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:52.431404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-08777712-7e44-4008-a8b7-a756a00a0542 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:52.433622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-08777712-7e44-4008-a8b7-a756a00a0542 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:52.450355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08777712-7e44-4008-a8b7-a756a00a0542 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:52.453546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08777712-7e44-4008-a8b7-a756a00a0542 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:52.454052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08777712-7e44-4008-a8b7-a756a00a0542 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:52.457010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08777712-7e44-4008-a8b7-a756a00a0542 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:52.457689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08777712-7e44-4008-a8b7-a756a00a0542 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:52.459005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08777712-7e44-4008-a8b7-a756a00a0542 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:52.502840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3697afca-c427-4f39-b072-b34ccff338f7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:52.524881 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3697afca-c427-4f39-b072-b34ccff338f7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1316 microversion: 2.1 time: 0.667461 Sep 27 10:04:52.525482 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1144/2095] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:51 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1316 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:52.538568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b141f79c-08f2-428f-9e86-54e9e6a6cf69 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:52.564962 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b141f79c-08f2-428f-9e86-54e9e6a6cf69 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.036338 Sep 27 10:04:52.565297 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1145/2096] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:04:52 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:52.596639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7248e02c-a9c3-4cd9-b091-710107aa52f7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:52.605938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7248e02c-a9c3-4cd9-b091-710107aa52f7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:52.650884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7248e02c-a9c3-4cd9-b091-710107aa52f7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:52.651682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7248e02c-a9c3-4cd9-b091-710107aa52f7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:52.652335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7248e02c-a9c3-4cd9-b091-710107aa52f7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:52.653306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7248e02c-a9c3-4cd9-b091-710107aa52f7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:52.653938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7248e02c-a9c3-4cd9-b091-710107aa52f7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:52.654536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7248e02c-a9c3-4cd9-b091-710107aa52f7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:52.790295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7248e02c-a9c3-4cd9-b091-710107aa52f7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:52.800868 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7248e02c-a9c3-4cd9-b091-710107aa52f7 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1265 microversion: 2.1 time: 0.216692 Sep 27 10:04:52.813262 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1146/2097] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:52 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1265 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:52.818223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7551e97b-5478-4d7f-b948-528a264fea73 req-c42a3c36-501e-407b-b67a-b54fe13d9daa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "55ead6af-a0d9-4d6f-b742-4a3fd4a418ad", "name": "network-vif-plugged", "status": "completed", "tag": "a33d85f6-8d2f-4192-8ab2-06ef5d149185"}, {"server_uuid": "152a5758-d14b-4ca9-917e-f7017c6d99b2", "name": "network-vif-unplugged", "status": "completed", "tag": "61c12b2f-7268-4252-a482-600cb57542ca"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:52.838919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7551e97b-5478-4d7f-b948-528a264fea73 req-c42a3c36-501e-407b-b67a-b54fe13d9daa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:52.839411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7551e97b-5478-4d7f-b948-528a264fea73 req-c42a3c36-501e-407b-b67a-b54fe13d9daa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:52.839847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7551e97b-5478-4d7f-b948-528a264fea73 req-c42a3c36-501e-407b-b67a-b54fe13d9daa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:52.864966 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7551e97b-5478-4d7f-b948-528a264fea73 req-c42a3c36-501e-407b-b67a-b54fe13d9daa service nova] Creating event network-vif-plugged:a33d85f6-8d2f-4192-8ab2-06ef5d149185 for instance 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad on nch2-master-1 Sep 27 10:04:52.867175 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7551e97b-5478-4d7f-b948-528a264fea73 req-c42a3c36-501e-407b-b67a-b54fe13d9daa service nova] Creating event network-vif-unplugged:61c12b2f-7268-4252-a482-600cb57542ca for instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 on nch2-master-1 Sep 27 10:04:52.879636 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7551e97b-5478-4d7f-b948-528a264fea73 req-c42a3c36-501e-407b-b67a-b54fe13d9daa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 356 microversion: 2.1 time: 0.064868 Sep 27 10:04:52.880467 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1147/2098] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:52 2022] POST /compute/v2.1/os-server-external-events => generated 356 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:52.888040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-487fd308-3e60-490f-8000-ee6403981b13 req-08cba245-b485-47ed-86ba-75065040cc1a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "152a5758-d14b-4ca9-917e-f7017c6d99b2", "name": "network-vif-deleted", "tag": "61c12b2f-7268-4252-a482-600cb57542ca"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:52.922595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-487fd308-3e60-490f-8000-ee6403981b13 req-08cba245-b485-47ed-86ba-75065040cc1a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:52.923111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-487fd308-3e60-490f-8000-ee6403981b13 req-08cba245-b485-47ed-86ba-75065040cc1a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:52.930849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-487fd308-3e60-490f-8000-ee6403981b13 req-08cba245-b485-47ed-86ba-75065040cc1a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:52.961151 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-487fd308-3e60-490f-8000-ee6403981b13 req-08cba245-b485-47ed-86ba-75065040cc1a service nova] Creating event network-vif-deleted:61c12b2f-7268-4252-a482-600cb57542ca for instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 on nch2-master-1 Sep 27 10:04:52.970831 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-487fd308-3e60-490f-8000-ee6403981b13 req-08cba245-b485-47ed-86ba-75065040cc1a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.088692 Sep 27 10:04:52.971989 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1148/2099] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:52.978973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6aa67b2e-d9d2-4e24-bb94-c3aa06ff2d0a tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63aa3940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:53.018458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-08777712-7e44-4008-a8b7-a756a00a0542 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid b0aa6e63-d0ba-47c2-8f6b-572f168104ee {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:53.086306 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-08777712-7e44-4008-a8b7-a756a00a0542 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1965 microversion: 2.25 time: 0.658631 Sep 27 10:04:53.092023 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 952/2100] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:52 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1965 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:53.111357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:53.112777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:53.133791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:53.134373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:53.134819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:53.135587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:53.136003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:53.136428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:53.268797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:53.270774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:53.271222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:53.271690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lazy-loading 'fault' on Instance uuid bd544723-d1ca-4f7e-a231-ba0b8351464d {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:53.631294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:53.649145 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1e4d442c-ed8b-44fb-9379-6bc783315811 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 200 len: 1189 microversion: 2.1 time: 0.546801 Sep 27 10:04:53.664833 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 953/2101] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:53 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 1189 bytes in 572 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:53.670321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd4d82c2-5698-4022-9284-cf5c3b9a3840 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:53.681964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd4d82c2-5698-4022-9284-cf5c3b9a3840 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:53.702574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4d82c2-5698-4022-9284-cf5c3b9a3840 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:53.703130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4d82c2-5698-4022-9284-cf5c3b9a3840 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:53.703584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4d82c2-5698-4022-9284-cf5c3b9a3840 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:53.704370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4d82c2-5698-4022-9284-cf5c3b9a3840 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:53.707176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4d82c2-5698-4022-9284-cf5c3b9a3840 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:53.707667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4d82c2-5698-4022-9284-cf5c3b9a3840 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:54.056138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd4d82c2-5698-4022-9284-cf5c3b9a3840 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:54.073733 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd4d82c2-5698-4022-9284-cf5c3b9a3840 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1275 microversion: 2.1 time: 0.407047 Sep 27 10:04:54.074712 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 954/2102] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:53 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1275 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:54.084792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f53aca3c-d7e6-4237-a145-af0664d27490 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:54.085454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f53aca3c-d7e6-4237-a145-af0664d27490 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:54.108635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53aca3c-d7e6-4237-a145-af0664d27490 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:54.109536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53aca3c-d7e6-4237-a145-af0664d27490 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:54.110182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53aca3c-d7e6-4237-a145-af0664d27490 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:54.111910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53aca3c-d7e6-4237-a145-af0664d27490 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:54.112569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53aca3c-d7e6-4237-a145-af0664d27490 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:54.113230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53aca3c-d7e6-4237-a145-af0664d27490 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:54.668492 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6aa67b2e-d9d2-4e24-bb94-c3aa06ff2d0a tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "DELETE /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 204 len: 0 microversion: 2.1 time: 1.694945 Sep 27 10:04:54.669224 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1149/2103] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:52 2022] DELETE /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 0 bytes in 1697 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:54.674248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:54.675023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:54.693455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:54.693971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:54.694472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:54.695131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:54.695806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:54.696062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:54.716096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-f53aca3c-d7e6-4237-a145-af0664d27490 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid b0aa6e63-d0ba-47c2-8f6b-572f168104ee {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:54.737379 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f53aca3c-d7e6-4237-a145-af0664d27490 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1965 microversion: 2.25 time: 0.660984 Sep 27 10:04:54.738381 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 955/2104] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:54 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1965 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:54.765096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7b69290d-cc6c-4403-a19a-0b0b3f79c6b4 req-01918acd-7641-427c-904c-df74c74a4878 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb6339bd-6a0a-497c-8064-cfcd4030b6c2", "name": "network-vif-deleted", "tag": "4a81581e-1fd6-48e4-b61d-caf266779586"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:54.778891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7b69290d-cc6c-4403-a19a-0b0b3f79c6b4 req-01918acd-7641-427c-904c-df74c74a4878 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:54.779882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7b69290d-cc6c-4403-a19a-0b0b3f79c6b4 req-01918acd-7641-427c-904c-df74c74a4878 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:54.780599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7b69290d-cc6c-4403-a19a-0b0b3f79c6b4 req-01918acd-7641-427c-904c-df74c74a4878 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:54.781848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:54.782596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:54.785267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:54.785793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lazy-loading 'fault' on Instance uuid cb6339bd-6a0a-497c-8064-cfcd4030b6c2 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:54.796932 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7b69290d-cc6c-4403-a19a-0b0b3f79c6b4 req-01918acd-7641-427c-904c-df74c74a4878 service nova] Creating event network-vif-deleted:4a81581e-1fd6-48e4-b61d-caf266779586 for instance cb6339bd-6a0a-497c-8064-cfcd4030b6c2 on nch1-master-1 Sep 27 10:04:54.803335 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7b69290d-cc6c-4403-a19a-0b0b3f79c6b4 req-01918acd-7641-427c-904c-df74c74a4878 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048100 Sep 27 10:04:54.805624 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 956/2105] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:54.811901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ff4cf0cc-c550-4789-8e96-4acbe88a43d0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:54.812931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ff4cf0cc-c550-4789-8e96-4acbe88a43d0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:54.822659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4cf0cc-c550-4789-8e96-4acbe88a43d0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:54.823559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4cf0cc-c550-4789-8e96-4acbe88a43d0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:54.824190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4cf0cc-c550-4789-8e96-4acbe88a43d0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:54.826063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4cf0cc-c550-4789-8e96-4acbe88a43d0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:54.826820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4cf0cc-c550-4789-8e96-4acbe88a43d0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:54.827458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff4cf0cc-c550-4789-8e96-4acbe88a43d0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:54.893429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '06dc3376ada04ffe922c9023f4182259', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '183cb066566c4510945d6eb0d41a2853', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:54.906911 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-91e2b6df-0164-4dcf-a9e0-f86e18279826 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 200 len: 1257 microversion: 2.1 time: 0.236607 Sep 27 10:04:54.907848 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1150/2106] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:54 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 1257 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:54.913918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9cd60f09-2936-41a3-a84b-26c88cbc4b61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:54.914924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9cd60f09-2936-41a3-a84b-26c88cbc4b61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:54.921678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd60f09-2936-41a3-a84b-26c88cbc4b61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:54.922900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd60f09-2936-41a3-a84b-26c88cbc4b61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:54.923490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd60f09-2936-41a3-a84b-26c88cbc4b61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:54.924772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd60f09-2936-41a3-a84b-26c88cbc4b61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:54.925421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd60f09-2936-41a3-a84b-26c88cbc4b61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:54.926039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd60f09-2936-41a3-a84b-26c88cbc4b61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.286198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:55.297919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9cd60f09-2936-41a3-a84b-26c88cbc4b61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:55.309644 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9cd60f09-2936-41a3-a84b-26c88cbc4b61 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1307 microversion: 2.1 time: 0.398948 Sep 27 10:04:55.316922 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1151/2107] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:54 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1307 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:55.323275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-28fc47dc-64a4-4434-b71d-8e5ba919de4d tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63b2d040>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:55.348265 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-28fc47dc-64a4-4434-b71d-8e5ba919de4d tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] HTTP exception thrown: Image not found. Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-28fc47dc-64a4-4434-b71d-8e5ba919de4d tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Request method failure captured: Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 HTTP/1.1 Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsqS8wTqwJCKAaZfBPF_2GNu90piB3rTxDlvrWvyfpVn6Ph3kb3VLfMM5uXumXMZCvvsRDaPbrg7UKxhCmUYWyv6FeFeUaguQxNpFTL2dK6jI7chvqj_DBGJ05vqT2g2d8KV_lycaLPzFP-wAXgdsdjSmN55CqgqVC-YkSijA2Ko5UA Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 58c33b046dc7494c970f26eb5900b3f3 Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesOneServerTestJSON-353655151 Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:04:55.351307 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/58c33b046dc7494c970f26eb5900b3f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/58c33b046dc7494c970f26eb5900b3f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/58c33b046dc7494c970f26eb5900b3f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_58c33b046dc7494c970f26eb5900b3f3"}]}] Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesOneServerTestJSON-353655151 Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 58c33b046dc7494c970f26eb5900b3f3 Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesOneServerTestJSON-353655151 Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesOneServerTestJSON-353655151-project Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a10600a9277a436396fc70971dd27893 Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesOneServerTestJSON-353655151-project Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: exception: Image not found. Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '7067aeff-ca0d-43a5-a172-93e103d961e7'} Sep 27 10:04:55.353522 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:04:55.354736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID 7067aeff-ca0d-43a5-a172-93e103d961e7 Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:04:55.356339 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 7067aeff-ca0d-43a5-a172-93e103d961e7 could not be found. Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:04:55.359096 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:04:55.361173 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.364131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-28fc47dc-64a4-4434-b71d-8e5ba919de4d tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Returning 404 to user: Image not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:04:55.365409 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-28fc47dc-64a4-4434-b71d-8e5ba919de4d tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 404 len: 62 microversion: 2.1 time: 0.047060 Sep 27 10:04:55.371656 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1152/2108] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:55 2022] GET /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 62 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:04:55.376774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:55.377407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: bd544723-d1ca-4f7e-a231-ba0b8351464d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:55.390789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.391279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.391894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.392851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.393488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.398215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.419118 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] HTTP exception thrown: Instance bd544723-d1ca-4f7e-a231-ba0b8351464d could not be found. Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Request method failure captured: Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d HTTP/1.1 Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsl2ww1DCmrYlsvxa1ad0I1ZocI8nPKJGQ863hbj2ddYMyd4rN82uIGtKsNLlGkxpy9k3eCbNYNIEH6L36dtBDP6h0PGa_WIvNJEnlD4pYTb78k24vaHnHkvI4YxqwLlRHU0jrpIrk9bJl_QJobcfTL2k7WBj1Ty6gn63cG3JKTY8h8 Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: ec0980b5824b425fbad9b8554cf5c42c Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-LiveMigrationTest-1517090534 Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:04:55.421260 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ec0980b5824b425fbad9b8554cf5c42c"}]}] Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-LiveMigrationTest-1517090534 Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: ec0980b5824b425fbad9b8554cf5c42c Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-LiveMigrationTest-1517090534 Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-LiveMigrationTest-1517090534-project Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: dce01d3212934161a38be853b4cd93ae Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-LiveMigrationTest-1517090534-project Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance bd544723-d1ca-4f7e-a231-ba0b8351464d could not be found. Sep 27 10:04:55.422957 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'bd544723-d1ca-4f7e-a231-ba0b8351464d'} Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bd544723-d1ca-4f7e-a231-ba0b8351464d could not be found. Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:04:55.423891 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bd544723-d1ca-4f7e-a231-ba0b8351464d could not be found. Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bd544723-d1ca-4f7e-a231-ba0b8351464d could not be found. Sep 27 10:04:55.426740 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:55.429917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Returning 404 to user: Instance bd544723-d1ca-4f7e-a231-ba0b8351464d could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:04:55.429917 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b4b1ddda-a997-4f92-84b2-0acfe099c17c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d" status: 404 len: 111 microversion: 2.1 time: 0.055501 Sep 27 10:04:55.429917 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1153/2109] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:55 2022] GET /compute/v2.1/servers/bd544723-d1ca-4f7e-a231-ba0b8351464d => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:55.430574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d1cdb7d3-57a3-41aa-84fc-57fce5de9915 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:55.431370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ff4cf0cc-c550-4789-8e96-4acbe88a43d0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:55.439697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cdb7d3-57a3-41aa-84fc-57fce5de9915 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.440525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cdb7d3-57a3-41aa-84fc-57fce5de9915 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.441302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cdb7d3-57a3-41aa-84fc-57fce5de9915 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.442398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cdb7d3-57a3-41aa-84fc-57fce5de9915 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.443442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cdb7d3-57a3-41aa-84fc-57fce5de9915 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.444090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cdb7d3-57a3-41aa-84fc-57fce5de9915 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.455719 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ff4cf0cc-c550-4789-8e96-4acbe88a43d0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1316 microversion: 2.1 time: 0.648183 Sep 27 10:04:55.457754 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 957/2110] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:54 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1316 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:55.469159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:55.475181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-abfb0732-d962-496b-a4e9-4d4826d06f3a req-f3c6eab8-39c8-4c15-b6b2-237412e31d94 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b563b3f-326c-41ed-9185-2a6d112c871d", "name": "network-vif-unplugged", "status": "completed", "tag": "917af05f-ebc3-433e-a57d-d09e51b4d3c7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:55.495441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abfb0732-d962-496b-a4e9-4d4826d06f3a req-f3c6eab8-39c8-4c15-b6b2-237412e31d94 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.496062 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d1cdb7d3-57a3-41aa-84fc-57fce5de9915 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 424 microversion: 2.25 time: 0.067186 Sep 27 10:04:55.497855 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1154/2111] 10.222.0.26 () {60 vars in 1217 bytes} [Tue Sep 27 10:04:55 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 424 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:04:55.498704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abfb0732-d962-496b-a4e9-4d4826d06f3a req-f3c6eab8-39c8-4c15-b6b2-237412e31d94 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.501691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abfb0732-d962-496b-a4e9-4d4826d06f3a req-f3c6eab8-39c8-4c15-b6b2-237412e31d94 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.514234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-4dcc7344-a729-4253-bbc7-cc71629586bd req-cabc63df-a557-4ccc-8be2-a041453bcb62 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bd544723-d1ca-4f7e-a231-ba0b8351464d", "name": "network-vif-deleted", "tag": "f2d7b45b-7823-4ee9-a9a3-1e4c5cda1e27"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:55.532146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4dcc7344-a729-4253-bbc7-cc71629586bd req-cabc63df-a557-4ccc-8be2-a041453bcb62 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.532414 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-abfb0732-d962-496b-a4e9-4d4826d06f3a req-f3c6eab8-39c8-4c15-b6b2-237412e31d94 service nova] Creating event network-vif-unplugged:917af05f-ebc3-433e-a57d-d09e51b4d3c7 for instance 3b563b3f-326c-41ed-9185-2a6d112c871d on nch1-master-1 Sep 27 10:04:55.533141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4dcc7344-a729-4253-bbc7-cc71629586bd req-cabc63df-a557-4ccc-8be2-a041453bcb62 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.533738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4dcc7344-a729-4253-bbc7-cc71629586bd req-cabc63df-a557-4ccc-8be2-a041453bcb62 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.546879 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-abfb0732-d962-496b-a4e9-4d4826d06f3a req-f3c6eab8-39c8-4c15-b6b2-237412e31d94 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.086612 Sep 27 10:04:55.548748 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 958/2112] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:55 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:55.551198 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-4dcc7344-a729-4253-bbc7-cc71629586bd req-cabc63df-a557-4ccc-8be2-a041453bcb62 service nova] Creating event network-vif-deleted:f2d7b45b-7823-4ee9-a9a3-1e4c5cda1e27 for instance bd544723-d1ca-4f7e-a231-ba0b8351464d on nch2-master-1 Sep 27 10:04:55.563244 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-4dcc7344-a729-4253-bbc7-cc71629586bd req-cabc63df-a557-4ccc-8be2-a041453bcb62 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062510 Sep 27 10:04:55.565266 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1155/2113] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:55.568533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-42b29bed-d7cb-4b12-8344-2d1fdd5e591e tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:55.573161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-42b29bed-d7cb-4b12-8344-2d1fdd5e591e tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:55.580704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f33199e-d525-46ef-b99e-d6396af949e0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-\u20a1-428961575"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:55.593093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42b29bed-d7cb-4b12-8344-2d1fdd5e591e tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.593863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9f33199e-d525-46ef-b99e-d6396af949e0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:55.595290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42b29bed-d7cb-4b12-8344-2d1fdd5e591e tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.598387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42b29bed-d7cb-4b12-8344-2d1fdd5e591e tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.599573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42b29bed-d7cb-4b12-8344-2d1fdd5e591e tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.600093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42b29bed-d7cb-4b12-8344-2d1fdd5e591e tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.600617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42b29bed-d7cb-4b12-8344-2d1fdd5e591e tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.610928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f33199e-d525-46ef-b99e-d6396af949e0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.611427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f33199e-d525-46ef-b99e-d6396af949e0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.611864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f33199e-d525-46ef-b99e-d6396af949e0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.612577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f33199e-d525-46ef-b99e-d6396af949e0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.621220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f33199e-d525-46ef-b99e-d6396af949e0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.621759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f33199e-d525-46ef-b99e-d6396af949e0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.870581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-42b29bed-d7cb-4b12-8344-2d1fdd5e591e tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:55.884993 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-42b29bed-d7cb-4b12-8344-2d1fdd5e591e tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1275 microversion: 2.1 time: 0.329694 Sep 27 10:04:55.885983 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 959/2114] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:55 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1275 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:55.891054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-329f59b7-93ec-4914-bc7d-a3d827dcd126 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": "nch2-master-1", "block_migration": "auto"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:55.902012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-329f59b7-93ec-4914-bc7d-a3d827dcd126 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:55.916833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-329f59b7-93ec-4914-bc7d-a3d827dcd126 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.917646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-329f59b7-93ec-4914-bc7d-a3d827dcd126 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.918262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-329f59b7-93ec-4914-bc7d-a3d827dcd126 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:55.920737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-329f59b7-93ec-4914-bc7d-a3d827dcd126 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:55.927014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-329f59b7-93ec-4914-bc7d-a3d827dcd126 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:55.927735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-329f59b7-93ec-4914-bc7d-a3d827dcd126 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:56.004385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-329f59b7-93ec-4914-bc7d-a3d827dcd126 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'flavor' on Instance uuid b0aa6e63-d0ba-47c2-8f6b-572f168104ee {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:04:56.031080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-329f59b7-93ec-4914-bc7d-a3d827dcd126 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Going to try to live migrate instance to nch2-master-1 {{(pid=145969) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Sep 27 10:04:56.158221 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9f33199e-d525-46ef-b99e-d6396af949e0 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "POST /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d/action" status: 202 len: 0 microversion: 2.1 time: 0.590183 Sep 27 10:04:56.158891 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1156/2115] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:04:55 2022] POST /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d/action => generated 0 bytes in 593 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 10:04:56.170325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:56.171032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] [instance: cb6339bd-6a0a-497c-8064-cfcd4030b6c2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:56.203820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:56.204624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:56.205248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:56.206150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:56.207565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:56.208200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:56.236146 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] HTTP exception thrown: Instance cb6339bd-6a0a-497c-8064-cfcd4030b6c2 could not be found. Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Request method failure captured: Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 HTTP/1.1 Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsozSoNYWd434HizgzOXgJ9ZtKd0aAwjKil-p8xESml-_X7O9OHSCel4Gah9gYCqj5Ad8SL4I-U1hZ0kuZt24SRkJLeaFzm_y3IAyGf6pqE4TUsjjD4zE8rbuca9x04gy0XyaB3w6dKXX745GRwlyqE0W_kHE0KoaF97XplerNaPAm8 Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 183cb066566c4510945d6eb0d41a2853 Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestBootFromVolume-1362999175 Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:04:56.246900 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/183cb066566c4510945d6eb0d41a2853"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/183cb066566c4510945d6eb0d41a2853"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/183cb066566c4510945d6eb0d41a2853"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_183cb066566c4510945d6eb0d41a2853"}]}] Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestBootFromVolume-1362999175 Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 183cb066566c4510945d6eb0d41a2853 Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestBootFromVolume-1362999175 Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestBootFromVolume-1362999175-project Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 06dc3376ada04ffe922c9023f4182259 Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestBootFromVolume-1362999175-project Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance cb6339bd-6a0a-497c-8064-cfcd4030b6c2 could not be found. Sep 27 10:04:56.248819 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'cb6339bd-6a0a-497c-8064-cfcd4030b6c2'} Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb6339bd-6a0a-497c-8064-cfcd4030b6c2 could not be found. Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:04:56.249947 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb6339bd-6a0a-497c-8064-cfcd4030b6c2 could not be found. Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb6339bd-6a0a-497c-8064-cfcd4030b6c2 could not be found. Sep 27 10:04:56.251919 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:04:56.254785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Returning 404 to user: Instance cb6339bd-6a0a-497c-8064-cfcd4030b6c2 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:04:56.255902 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2748f8a9-04b1-4a82-9619-1aba92c5018c tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2" status: 404 len: 111 microversion: 2.1 time: 0.095915 Sep 27 10:04:56.256697 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1157/2116] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:56 2022] GET /compute/v2.1/servers/cb6339bd-6a0a-497c-8064-cfcd4030b6c2 => generated 111 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:04:56.267982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2296c7d0-6b20-4515-8b67-5628771b12dd tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63c28280>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:56.373739 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2296c7d0-6b20-4515-8b67-5628771b12dd tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1426 microversion: 2.1 time: 0.111602 Sep 27 10:04:56.385582 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1158/2117] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:56 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1426 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:56.403032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-37278c51-998b-4f46-92ab-778bf7627781 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] Calling method '.version_select at 0x7f7f63797ca0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:56.446239 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-37278c51-998b-4f46-92ab-778bf7627781 tempest-ServersTestBootFromVolume-1362999175 tempest-ServersTestBootFromVolume-1362999175-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-253434363" status: 202 len: 0 microversion: 2.1 time: 0.052485 Sep 27 10:04:56.448920 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1159/2118] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:04:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-253434363 => generated 0 bytes in 63 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:04:56.459347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69dc3e9c-fdad-472d-ae19-33640c06b295 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:56.460272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69dc3e9c-fdad-472d-ae19-33640c06b295 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:56.469556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69dc3e9c-fdad-472d-ae19-33640c06b295 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:56.470512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69dc3e9c-fdad-472d-ae19-33640c06b295 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:56.471166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69dc3e9c-fdad-472d-ae19-33640c06b295 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:56.472244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69dc3e9c-fdad-472d-ae19-33640c06b295 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:56.483011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69dc3e9c-fdad-472d-ae19-33640c06b295 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:56.486312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69dc3e9c-fdad-472d-ae19-33640c06b295 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:56.995482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69dc3e9c-fdad-472d-ae19-33640c06b295 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:57.021995 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69dc3e9c-fdad-472d-ae19-33640c06b295 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1307 microversion: 2.1 time: 0.569560 Sep 27 10:04:57.030607 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1160/2119] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:56 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1307 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:57.045238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b2d2e1e7-e579-4b7e-af9d-61d936aa6990 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:57.046064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b2d2e1e7-e579-4b7e-af9d-61d936aa6990 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:57.065966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2d2e1e7-e579-4b7e-af9d-61d936aa6990 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:57.066471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2d2e1e7-e579-4b7e-af9d-61d936aa6990 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:57.067240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2d2e1e7-e579-4b7e-af9d-61d936aa6990 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:57.068272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2d2e1e7-e579-4b7e-af9d-61d936aa6990 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:57.069189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2d2e1e7-e579-4b7e-af9d-61d936aa6990 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:57.069923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2d2e1e7-e579-4b7e-af9d-61d936aa6990 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:57.443109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b2d2e1e7-e579-4b7e-af9d-61d936aa6990 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:57.461416 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b2d2e1e7-e579-4b7e-af9d-61d936aa6990 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1465 microversion: 2.1 time: 0.433766 Sep 27 10:04:57.463009 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1161/2120] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:57 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1465 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:57.471917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6054828e-10fd-41c5-b042-67fc28981041 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:57.475297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6054828e-10fd-41c5-b042-67fc28981041 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:57.484392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6054828e-10fd-41c5-b042-67fc28981041 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:57.485183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6054828e-10fd-41c5-b042-67fc28981041 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:57.486905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6054828e-10fd-41c5-b042-67fc28981041 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:57.487962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6054828e-10fd-41c5-b042-67fc28981041 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:57.488613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6054828e-10fd-41c5-b042-67fc28981041 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:57.489241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6054828e-10fd-41c5-b042-67fc28981041 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:57.531430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6054828e-10fd-41c5-b042-67fc28981041 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:04:57.627680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:57.692720 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6054828e-10fd-41c5-b042-67fc28981041 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "DELETE /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 204 len: 0 microversion: 2.1 time: 0.228012 Sep 27 10:04:57.694324 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1162/2121] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:04:57 2022] DELETE /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:57.702545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-921bb24b-5595-44e9-9825-084251044742 req-f9915ca8-0c79-41d1-ab2f-588422911210 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "705d2d84-afe0-4760-bb57-66126418dacb", "tag": "3306435b-7a82-4c08-bb65-5f863338089e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:57.719469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-921bb24b-5595-44e9-9825-084251044742 req-f9915ca8-0c79-41d1-ab2f-588422911210 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:57.720322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-921bb24b-5595-44e9-9825-084251044742 req-f9915ca8-0c79-41d1-ab2f-588422911210 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:57.721359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-921bb24b-5595-44e9-9825-084251044742 req-f9915ca8-0c79-41d1-ab2f-588422911210 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:57.745913 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-921bb24b-5595-44e9-9825-084251044742 req-f9915ca8-0c79-41d1-ab2f-588422911210 service nova] Creating event network-changed:3306435b-7a82-4c08-bb65-5f863338089e for instance 705d2d84-afe0-4760-bb57-66126418dacb on nch1-master-1 Sep 27 10:04:57.752459 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-921bb24b-5595-44e9-9825-084251044742 req-f9915ca8-0c79-41d1-ab2f-588422911210 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056412 Sep 27 10:04:57.753222 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1163/2122] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:04:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:04:57.759927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-93dad49f-f5db-43f9-8248-aadc0b9eacf6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:57.761028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-93dad49f-f5db-43f9-8248-aadc0b9eacf6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:57.770061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93dad49f-f5db-43f9-8248-aadc0b9eacf6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:57.771744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93dad49f-f5db-43f9-8248-aadc0b9eacf6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:57.772242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93dad49f-f5db-43f9-8248-aadc0b9eacf6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:57.773994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93dad49f-f5db-43f9-8248-aadc0b9eacf6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:57.774868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93dad49f-f5db-43f9-8248-aadc0b9eacf6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:57.775428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93dad49f-f5db-43f9-8248-aadc0b9eacf6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:58.127304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-93dad49f-f5db-43f9-8248-aadc0b9eacf6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:58.139686 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-93dad49f-f5db-43f9-8248-aadc0b9eacf6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1317 microversion: 2.1 time: 0.384135 Sep 27 10:04:58.141757 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1164/2123] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:57 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1317 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:58.151645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d96b67b-4102-49df-8322-a482fe4d7aec tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63aa3940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:58.191409 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-329f59b7-93ec-4914-bc7d-a3d827dcd126 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee/action" status: 202 len: 0 microversion: 2.25 time: 2.303571 Sep 27 10:04:58.192187 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 960/2124] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:04:55 2022] POST /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee/action => generated 0 bytes in 2305 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:04:58.197934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-accd4695-1026-4b29-a39f-409d00febce0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:58.200029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-accd4695-1026-4b29-a39f-409d00febce0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:58.231133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accd4695-1026-4b29-a39f-409d00febce0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:58.232001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accd4695-1026-4b29-a39f-409d00febce0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:58.232465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accd4695-1026-4b29-a39f-409d00febce0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:58.241230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accd4695-1026-4b29-a39f-409d00febce0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:58.241230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accd4695-1026-4b29-a39f-409d00febce0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:58.241230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accd4695-1026-4b29-a39f-409d00febce0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:58.250908 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5d96b67b-4102-49df-8322-a482fe4d7aec tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1426 microversion: 2.1 time: 0.107179 Sep 27 10:04:58.261613 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1165/2125] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:04:58 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1426 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:58.268761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2290dacc-3f14-43a8-a85b-a39b4fc1dbd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:58.269405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2290dacc-3f14-43a8-a85b-a39b4fc1dbd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:58.284198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2290dacc-3f14-43a8-a85b-a39b4fc1dbd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:58.284758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2290dacc-3f14-43a8-a85b-a39b4fc1dbd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:58.285397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2290dacc-3f14-43a8-a85b-a39b4fc1dbd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:58.286536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2290dacc-3f14-43a8-a85b-a39b4fc1dbd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:58.287413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2290dacc-3f14-43a8-a85b-a39b4fc1dbd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:58.293333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2290dacc-3f14-43a8-a85b-a39b4fc1dbd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:59.007566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-accd4695-1026-4b29-a39f-409d00febce0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:59.017028 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-accd4695-1026-4b29-a39f-409d00febce0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d" status: 200 len: 1467 microversion: 2.1 time: 0.823619 Sep 27 10:04:59.017744 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 961/2126] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:58 2022] GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d => generated 1467 bytes in 825 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:59.023650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cea2857d-456e-4cd0-b437-de0188de08f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:59.024585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cea2857d-456e-4cd0-b437-de0188de08f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:59.039929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cea2857d-456e-4cd0-b437-de0188de08f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:59.040811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cea2857d-456e-4cd0-b437-de0188de08f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:59.041447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cea2857d-456e-4cd0-b437-de0188de08f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:59.047136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cea2857d-456e-4cd0-b437-de0188de08f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:59.047651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cea2857d-456e-4cd0-b437-de0188de08f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:59.048102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cea2857d-456e-4cd0-b437-de0188de08f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:59.092236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2290dacc-3f14-43a8-a85b-a39b4fc1dbd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:59.101023 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2290dacc-3f14-43a8-a85b-a39b4fc1dbd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1669 microversion: 2.1 time: 0.838019 Sep 27 10:04:59.109415 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1166/2127] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:58 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1669 bytes in 847 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:04:59.114802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:59.115651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:59.125105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:59.125855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:59.126462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:59.127403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:59.127861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:59.128285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:59.329334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:59.357506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:59.530892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:59.539672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:59.541843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:04:59.543257 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-566ef98d-9ac9-4ea8-a7a0-6ac1fdac4fc4 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1484 microversion: 2.25 time: 0.432574 Sep 27 10:04:59.544363 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1167/2128] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:04:59 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1484 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:04:59.550111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4badb643-fc21-4b4a-b73d-85be51ddf229 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ListImageFiltersTestJSON-image-381114768"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:04:59.552964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4badb643-fc21-4b4a-b73d-85be51ddf229 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:59.567951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4badb643-fc21-4b4a-b73d-85be51ddf229 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:59.568517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4badb643-fc21-4b4a-b73d-85be51ddf229 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:59.569014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4badb643-fc21-4b4a-b73d-85be51ddf229 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:59.572477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4badb643-fc21-4b4a-b73d-85be51ddf229 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:59.572477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4badb643-fc21-4b4a-b73d-85be51ddf229 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:59.572921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4badb643-fc21-4b4a-b73d-85be51ddf229 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:59.573440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=12 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:04:59.918909 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4badb643-fc21-4b4a-b73d-85be51ddf229 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "POST /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d/action" status: 202 len: 0 microversion: 2.1 time: 0.372446 Sep 27 10:04:59.928551 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1168/2129] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:04:59 2022] POST /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d/action => generated 0 bytes in 383 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 10:04:59.943036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-77739e2e-77ca-4350-8e95-d316b8bc547b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:04:59.944116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-77739e2e-77ca-4350-8e95-d316b8bc547b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:04:59.976527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77739e2e-77ca-4350-8e95-d316b8bc547b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:59.981448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77739e2e-77ca-4350-8e95-d316b8bc547b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:59.983288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77739e2e-77ca-4350-8e95-d316b8bc547b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:04:59.986330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77739e2e-77ca-4350-8e95-d316b8bc547b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:04:59.987013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77739e2e-77ca-4350-8e95-d316b8bc547b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:04:59.987735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77739e2e-77ca-4350-8e95-d316b8bc547b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:00.132526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cea2857d-456e-4cd0-b437-de0188de08f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:00.150451 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cea2857d-456e-4cd0-b437-de0188de08f8 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1307 microversion: 2.1 time: 1.128047 Sep 27 10:05:00.152675 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 962/2130] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:59 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1307 bytes in 1133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:00.160556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c4c02688-d75c-428b-9d96-c60186483594 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f6377b700>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:00.238671 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c4c02688-d75c-428b-9d96-c60186483594 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1426 microversion: 2.1 time: 0.084663 Sep 27 10:05:00.241400 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 963/2131] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:00 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1426 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:00.249242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-bb46f50b-ae42-4719-9994-50227871b83a req-2ee0b7b6-d9d3-41c7-b74e-30fe38c80bf7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "219e4ceb-d811-4e6f-913f-c51410efcee6", "tag": "b2aabccc-ddd4-4740-aef9-c80e07db2d17"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:00.259816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:00.267022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bb46f50b-ae42-4719-9994-50227871b83a req-2ee0b7b6-d9d3-41c7-b74e-30fe38c80bf7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:00.269947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bb46f50b-ae42-4719-9994-50227871b83a req-2ee0b7b6-d9d3-41c7-b74e-30fe38c80bf7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:00.269947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bb46f50b-ae42-4719-9994-50227871b83a req-2ee0b7b6-d9d3-41c7-b74e-30fe38c80bf7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:00.287967 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-bb46f50b-ae42-4719-9994-50227871b83a req-2ee0b7b6-d9d3-41c7-b74e-30fe38c80bf7 service nova] Creating event network-changed:b2aabccc-ddd4-4740-aef9-c80e07db2d17 for instance 219e4ceb-d811-4e6f-913f-c51410efcee6 on nch2-master-1 Sep 27 10:05:00.313958 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-bb46f50b-ae42-4719-9994-50227871b83a req-2ee0b7b6-d9d3-41c7-b74e-30fe38c80bf7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068488 Sep 27 10:05:00.315648 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 964/2132] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:00.325904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2dfe04e8-36b7-45db-b973-4138fa54ad11 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f637f31f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:00.402930 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2dfe04e8-36b7-45db-b973-4138fa54ad11 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 0.085548 Sep 27 10:05:00.420841 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 965/2133] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:00 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:00.426316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e885387-f31e-4fd2-aa77-391f9e824646 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:00.427304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6e885387-f31e-4fd2-aa77-391f9e824646 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:00.442612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e885387-f31e-4fd2-aa77-391f9e824646 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:00.443366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e885387-f31e-4fd2-aa77-391f9e824646 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:00.444260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e885387-f31e-4fd2-aa77-391f9e824646 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:00.449703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e885387-f31e-4fd2-aa77-391f9e824646 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:00.450325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e885387-f31e-4fd2-aa77-391f9e824646 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:00.450981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e885387-f31e-4fd2-aa77-391f9e824646 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:00.528966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-77739e2e-77ca-4350-8e95-d316b8bc547b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:00.538060 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-77739e2e-77ca-4350-8e95-d316b8bc547b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1317 microversion: 2.1 time: 0.607770 Sep 27 10:05:00.545800 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1169/2134] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:04:59 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1317 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:00.572289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:00.573413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:00.581042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:00.582675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:00.583441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:00.585882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:00.588206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:00.591431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:00.884959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:00.896702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:00.898261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:00.899545 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b49377cb-7a1b-45ce-9c2c-e0a19d8d1c04 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1484 microversion: 2.25 time: 0.332521 Sep 27 10:05:00.900452 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1170/2135] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:00 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1484 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:05:01.141266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6e885387-f31e-4fd2-aa77-391f9e824646 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:01.176548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cd527447-d849-4d7f-9b65-1e23a61ce1d5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:01.177528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cd527447-d849-4d7f-9b65-1e23a61ce1d5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:01.182403 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6e885387-f31e-4fd2-aa77-391f9e824646 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1669 microversion: 2.1 time: 0.759914 Sep 27 10:05:01.188449 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 966/2136] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:00 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1669 bytes in 767 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:01.190556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd527447-d849-4d7f-9b65-1e23a61ce1d5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:01.192066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd527447-d849-4d7f-9b65-1e23a61ce1d5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:01.193180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd527447-d849-4d7f-9b65-1e23a61ce1d5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:01.200156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd527447-d849-4d7f-9b65-1e23a61ce1d5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:01.202151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd527447-d849-4d7f-9b65-1e23a61ce1d5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:01.203523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd527447-d849-4d7f-9b65-1e23a61ce1d5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:01.256759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-936837c3-9fe4-4923-a029-b65b0bd62c52 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f635d9ee0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:01.343310 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-936837c3-9fe4-4923-a029-b65b0bd62c52 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1426 microversion: 2.1 time: 0.091084 Sep 27 10:05:01.349916 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 967/2137] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:01 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1426 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:01.420193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fbbbe940-8995-4808-accf-6a6fb2ce1ae4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6361c820>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:01.493754 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fbbbe940-8995-4808-accf-6a6fb2ce1ae4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 0.078032 Sep 27 10:05:01.495637 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 968/2138] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:01 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:01.567534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-33bd6718-4ca4-4275-b213-a8a18c1e9e20 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:01.568858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-33bd6718-4ca4-4275-b213-a8a18c1e9e20 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:01.577240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bd6718-4ca4-4275-b213-a8a18c1e9e20 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:01.578154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bd6718-4ca4-4275-b213-a8a18c1e9e20 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:01.578796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bd6718-4ca4-4275-b213-a8a18c1e9e20 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:01.579743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bd6718-4ca4-4275-b213-a8a18c1e9e20 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:01.580405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bd6718-4ca4-4275-b213-a8a18c1e9e20 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:01.581075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bd6718-4ca4-4275-b213-a8a18c1e9e20 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:01.697284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cd527447-d849-4d7f-9b65-1e23a61ce1d5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:01.720305 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cd527447-d849-4d7f-9b65-1e23a61ce1d5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1307 microversion: 2.1 time: 0.549157 Sep 27 10:05:01.728285 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1171/2139] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:01 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1307 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:01.932842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:01.936986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:01.959060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:01.962436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:01.964166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:01.967101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:01.968357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:01.969669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:02.288809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:02.301404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:02.301702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-33bd6718-4ca4-4275-b213-a8a18c1e9e20 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:02.304162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:02.307627 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c47a8c6a-f33e-482b-8b47-be8ae6bc446c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1484 microversion: 2.25 time: 0.383428 Sep 27 10:05:02.317740 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-33bd6718-4ca4-4275-b213-a8a18c1e9e20 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1317 microversion: 2.1 time: 0.754839 Sep 27 10:05:02.329921 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1172/2140] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:01 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1484 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:02.333406 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 969/2141] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:01 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1317 bytes in 771 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:02.341390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed198589-5734-4efb-beb3-1c700cc7953e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:02.342013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed198589-5734-4efb-beb3-1c700cc7953e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:02.366103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ad889d25-9db8-4ad8-94ba-06e142b19d51 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63899e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:02.373169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed198589-5734-4efb-beb3-1c700cc7953e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:02.373936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed198589-5734-4efb-beb3-1c700cc7953e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:02.374542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed198589-5734-4efb-beb3-1c700cc7953e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:02.377544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed198589-5734-4efb-beb3-1c700cc7953e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:02.378174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed198589-5734-4efb-beb3-1c700cc7953e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:02.378796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed198589-5734-4efb-beb3-1c700cc7953e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:02.456034 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ad889d25-9db8-4ad8-94ba-06e142b19d51 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1426 microversion: 2.1 time: 0.093796 Sep 27 10:05:02.457545 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 970/2142] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:02 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1426 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:02.511809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e6df4697-780f-4288-a05c-1562716e0de9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6361c790>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:02.592058 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e6df4697-780f-4288-a05c-1562716e0de9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 0.084958 Sep 27 10:05:02.594610 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 971/2143] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:02 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:02.676868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ed198589-5734-4efb-beb3-1c700cc7953e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:02.689672 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed198589-5734-4efb-beb3-1c700cc7953e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1669 microversion: 2.1 time: 0.358450 Sep 27 10:05:02.690616 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1173/2144] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:02 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1669 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:02.751327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b99a788-531f-445d-982e-1007e6539ca5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:02.752031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3b99a788-531f-445d-982e-1007e6539ca5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:02.762862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b99a788-531f-445d-982e-1007e6539ca5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:02.765913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b99a788-531f-445d-982e-1007e6539ca5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:02.767018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b99a788-531f-445d-982e-1007e6539ca5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:02.769560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b99a788-531f-445d-982e-1007e6539ca5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:02.770834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b99a788-531f-445d-982e-1007e6539ca5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:02.772358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b99a788-531f-445d-982e-1007e6539ca5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:03.313481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5efae6ad-48a1-40e8-8eb9-aa651de961b6 req-c45bedf0-686c-4599-b541-38f29e72e36e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "11a0d56b-3e1f-40e7-9e3c-30fca67df25e", "name": "network-vif-deleted", "tag": "b3618e9e-e34a-4212-aa29-7747a00843a9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:03.345385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5efae6ad-48a1-40e8-8eb9-aa651de961b6 req-c45bedf0-686c-4599-b541-38f29e72e36e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:03.370125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5efae6ad-48a1-40e8-8eb9-aa651de961b6 req-c45bedf0-686c-4599-b541-38f29e72e36e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:03.370803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5efae6ad-48a1-40e8-8eb9-aa651de961b6 req-c45bedf0-686c-4599-b541-38f29e72e36e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.025s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:03.398654 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5efae6ad-48a1-40e8-8eb9-aa651de961b6 req-c45bedf0-686c-4599-b541-38f29e72e36e service nova] Creating event network-vif-deleted:b3618e9e-e34a-4212-aa29-7747a00843a9 for instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e on nch2-master-1 Sep 27 10:05:03.406975 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5efae6ad-48a1-40e8-8eb9-aa651de961b6 req-c45bedf0-686c-4599-b541-38f29e72e36e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097710 Sep 27 10:05:03.409247 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1174/2145] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:03.420339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:03.426162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:03.451306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:03.452100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:03.456083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:03.459706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:03.462329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:03.473566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:03.476208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3b99a788-531f-445d-982e-1007e6539ca5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:03.495639 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b99a788-531f-445d-982e-1007e6539ca5 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1307 microversion: 2.1 time: 0.748801 Sep 27 10:05:03.496260 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 972/2146] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:02 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1307 bytes in 750 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:03.532029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bef9b2ba-54bb-427f-8248-86dfa633339d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:03.533143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bef9b2ba-54bb-427f-8248-86dfa633339d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:03.557726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bef9b2ba-54bb-427f-8248-86dfa633339d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:03.562416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bef9b2ba-54bb-427f-8248-86dfa633339d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:03.563173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bef9b2ba-54bb-427f-8248-86dfa633339d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:03.566018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bef9b2ba-54bb-427f-8248-86dfa633339d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:03.571338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bef9b2ba-54bb-427f-8248-86dfa633339d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:03.572128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bef9b2ba-54bb-427f-8248-86dfa633339d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:03.589994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:04.101287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:04.124046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:04.126913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:04.128438 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a979424-9a82-4fd4-8032-ac9ea384f1e3 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1484 microversion: 2.25 time: 0.717759 Sep 27 10:05:04.131464 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1175/2147] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:03 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1484 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:04.145677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cbd97077-c0b9-4d91-adf8-d17c1bdae4f3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63727940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:04.282085 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cbd97077-c0b9-4d91-adf8-d17c1bdae4f3 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1426 microversion: 2.1 time: 0.142905 Sep 27 10:05:04.291606 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1176/2148] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:04 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1426 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:04.332775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fd97476e-0ffc-4c46-9b54-ad0ff2979ea5 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f63ac3c10>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:04.445673 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fd97476e-0ffc-4c46-9b54-ad0ff2979ea5 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 0.147795 Sep 27 10:05:04.453848 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1177/2149] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:04 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:04.471260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5a994b95-7a0b-4b29-a52b-480352adf2a4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:04.473102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5a994b95-7a0b-4b29-a52b-480352adf2a4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:04.488463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a994b95-7a0b-4b29-a52b-480352adf2a4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:04.490591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a994b95-7a0b-4b29-a52b-480352adf2a4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:04.491463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a994b95-7a0b-4b29-a52b-480352adf2a4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:04.494671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a994b95-7a0b-4b29-a52b-480352adf2a4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:04.495678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a994b95-7a0b-4b29-a52b-480352adf2a4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:04.496368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5a994b95-7a0b-4b29-a52b-480352adf2a4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:04.670691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5a994b95-7a0b-4b29-a52b-480352adf2a4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:04.721051 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5a994b95-7a0b-4b29-a52b-480352adf2a4 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1601 microversion: 2.1 time: 0.263651 Sep 27 10:05:04.725390 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1178/2150] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:04 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1601 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:04.736485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b051355a-021d-41fa-895c-38ba3ae123e4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:04.748113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b051355a-021d-41fa-895c-38ba3ae123e4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:04.772465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b051355a-021d-41fa-895c-38ba3ae123e4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:04.777537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b051355a-021d-41fa-895c-38ba3ae123e4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:04.779641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b051355a-021d-41fa-895c-38ba3ae123e4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:04.780096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b051355a-021d-41fa-895c-38ba3ae123e4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:04.780781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b051355a-021d-41fa-895c-38ba3ae123e4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:04.781923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b051355a-021d-41fa-895c-38ba3ae123e4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:04.854343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bef9b2ba-54bb-427f-8248-86dfa633339d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:04.877033 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bef9b2ba-54bb-427f-8248-86dfa633339d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1317 microversion: 2.1 time: 1.359885 Sep 27 10:05:04.880941 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 973/2151] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:03 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1317 bytes in 1365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:05.164003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:05.168528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:05.208116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:05.211114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:05.213994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:05.217969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:05.229833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:05.230513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:05.426272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b051355a-021d-41fa-895c-38ba3ae123e4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:05.437007 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b051355a-021d-41fa-895c-38ba3ae123e4 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1307 microversion: 2.1 time: 0.710010 Sep 27 10:05:05.446071 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1179/2152] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:04 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1307 bytes in 720 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:05.451226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa224d2a-82ad-420e-b33d-f9771848f2d7 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f6c184f70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:05.458441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:05.538265 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fa224d2a-82ad-420e-b33d-f9771848f2d7 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1426 microversion: 2.1 time: 0.090782 Sep 27 10:05:05.539487 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1180/2153] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:05 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1426 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:05.549778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7557bc19-5c0a-4eb7-8488-fe18320e3d9d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f637275e0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:05.649303 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7557bc19-5c0a-4eb7-8488-fe18320e3d9d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 0.108114 Sep 27 10:05:05.650327 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1181/2154] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:05 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:05.745248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:05.749715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:05.771263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:05.773412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:05.774961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:05.777426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:05.778285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:05.781205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:05.877540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:05.881343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:05.881989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:05.882781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lazy-loading 'fault' on Instance uuid 11a0d56b-3e1f-40e7-9e3c-30fca67df25e {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:05:06.007639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:06.020887 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f49b61c4-e740-4823-bdea-05f2182822b3 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 200 len: 1324 microversion: 2.1 time: 0.279667 Sep 27 10:05:06.026200 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1182/2155] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:05 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 1324 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:06.041044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1769dc8e-e2e9-4bab-a90e-7a91059f7217 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:06.044283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1769dc8e-e2e9-4bab-a90e-7a91059f7217 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:06.063006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1769dc8e-e2e9-4bab-a90e-7a91059f7217 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:06.063512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1769dc8e-e2e9-4bab-a90e-7a91059f7217 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:06.069362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1769dc8e-e2e9-4bab-a90e-7a91059f7217 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:06.074902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1769dc8e-e2e9-4bab-a90e-7a91059f7217 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:06.075640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1769dc8e-e2e9-4bab-a90e-7a91059f7217 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:06.076285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1769dc8e-e2e9-4bab-a90e-7a91059f7217 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:06.396587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1769dc8e-e2e9-4bab-a90e-7a91059f7217 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:06.412626 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1769dc8e-e2e9-4bab-a90e-7a91059f7217 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1317 microversion: 2.1 time: 0.374868 Sep 27 10:05:06.414985 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1183/2156] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:06 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1317 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:06.426231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-792bd055-c937-4f45-ab00-daa0f2107b2c req-412f0645-1005-4b76-a3ef-0425a845b8bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "705d2d84-afe0-4760-bb57-66126418dacb", "name": "network-vif-plugged", "status": "completed", "tag": "3306435b-7a82-4c08-bb65-5f863338089e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:06.449958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-792bd055-c937-4f45-ab00-daa0f2107b2c req-412f0645-1005-4b76-a3ef-0425a845b8bc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:06.452494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-792bd055-c937-4f45-ab00-daa0f2107b2c req-412f0645-1005-4b76-a3ef-0425a845b8bc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:06.453458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-792bd055-c937-4f45-ab00-daa0f2107b2c req-412f0645-1005-4b76-a3ef-0425a845b8bc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:06.488099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:06.495111 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-792bd055-c937-4f45-ab00-daa0f2107b2c req-412f0645-1005-4b76-a3ef-0425a845b8bc service nova] Creating event network-vif-plugged:3306435b-7a82-4c08-bb65-5f863338089e for instance 705d2d84-afe0-4760-bb57-66126418dacb on nch1-master-1 Sep 27 10:05:06.507412 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-792bd055-c937-4f45-ab00-daa0f2107b2c req-412f0645-1005-4b76-a3ef-0425a845b8bc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089375 Sep 27 10:05:06.511832 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1184/2157] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:06.523889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e2011271-07ab-442a-b7de-75db54d36e73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:06.527662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e2011271-07ab-442a-b7de-75db54d36e73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:06.552628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2011271-07ab-442a-b7de-75db54d36e73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:06.553161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2011271-07ab-442a-b7de-75db54d36e73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:06.553604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2011271-07ab-442a-b7de-75db54d36e73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:06.554342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2011271-07ab-442a-b7de-75db54d36e73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:06.554855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2011271-07ab-442a-b7de-75db54d36e73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:06.555295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2011271-07ab-442a-b7de-75db54d36e73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:06.953971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2011271-07ab-442a-b7de-75db54d36e73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:06.969624 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e2011271-07ab-442a-b7de-75db54d36e73 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1307 microversion: 2.1 time: 0.454362 Sep 27 10:05:06.989026 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1185/2158] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:06 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1307 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:06.999198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4e08127-1f47-49ed-97bb-58b86ecc89be tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f6c1734c0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:07.047240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:07.065642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:07.071125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:07.073684 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-efbb8c6f-292c-43c3-8197-da561889e357 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1484 microversion: 2.25 time: 1.919261 Sep 27 10:05:07.077793 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 974/2159] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:05 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1484 bytes in 1924 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:05:07.091030 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4e08127-1f47-49ed-97bb-58b86ecc89be tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1426 microversion: 2.1 time: 0.100443 Sep 27 10:05:07.091624 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1186/2160] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:06 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1426 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:07.099229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-99dd9731-f09a-4437-9531-601f11cf6086 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6c222280>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:07.121020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:07.122003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 11a0d56b-3e1f-40e7-9e3c-30fca67df25e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:07.131190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:07.138185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:07.138895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:07.139801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:07.142027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:07.142623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:07.171528 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] HTTP exception thrown: Instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e could not be found. Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Request method failure captured: Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e HTTP/1.1 Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsl2ww1DCmrYlsvxa1ad0I1ZocI8nPKJGQ863hbj2ddYMyd4rN82uIGtKsNLlGkxpy9k3eCbNYNIEH6L36dtBDP6h0PGa_WIvNJEnlD4pYTb78k24vaHnHkvI4YxqwLlRHU0jrpIrk9bJl_QJobcfTL2k7WBj1Ty6gn63cG3JKTY8h8 Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: ec0980b5824b425fbad9b8554cf5c42c Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-LiveMigrationTest-1517090534 Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:07.174363 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ec0980b5824b425fbad9b8554cf5c42c"}]}] Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-LiveMigrationTest-1517090534 Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: ec0980b5824b425fbad9b8554cf5c42c Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-LiveMigrationTest-1517090534 Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-LiveMigrationTest-1517090534-project Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: dce01d3212934161a38be853b4cd93ae Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-LiveMigrationTest-1517090534-project Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e could not be found. Sep 27 10:05:07.175560 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '11a0d56b-3e1f-40e7-9e3c-30fca67df25e'} Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e could not be found. Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:07.176234 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e could not be found. Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e could not be found. Sep 27 10:05:07.177460 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:07.179623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Returning 404 to user: Instance 11a0d56b-3e1f-40e7-9e3c-30fca67df25e could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:07.180845 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9f563788-4542-4dc3-a162-03bf8f3eca92 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e" status: 404 len: 111 microversion: 2.1 time: 0.073434 Sep 27 10:05:07.181979 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1187/2161] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:07 2022] GET /compute/v2.1/servers/11a0d56b-3e1f-40e7-9e3c-30fca67df25e => generated 111 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:07.195117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9b907881-4f16-4c6c-87b2-fcfc6bf8e574 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '.version_select at 0x7f7f63ac3c10>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:07.240766 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9b907881-4f16-4c6c-87b2-fcfc6bf8e574 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1620198310" status: 202 len: 0 microversion: 2.1 time: 0.053859 Sep 27 10:05:07.246570 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1188/2162] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:05:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1620198310 => generated 0 bytes in 61 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:05:07.258181 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-99dd9731-f09a-4437-9531-601f11cf6086 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 0.178027 Sep 27 10:05:07.260577 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 975/2163] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:07 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:07.445311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13885b8f-05a3-4eac-af3b-b701828468c8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:07.446547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-13885b8f-05a3-4eac-af3b-b701828468c8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:07.455917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13885b8f-05a3-4eac-af3b-b701828468c8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:07.457405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13885b8f-05a3-4eac-af3b-b701828468c8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:07.458528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13885b8f-05a3-4eac-af3b-b701828468c8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:07.459524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13885b8f-05a3-4eac-af3b-b701828468c8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:07.460208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13885b8f-05a3-4eac-af3b-b701828468c8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:07.461124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13885b8f-05a3-4eac-af3b-b701828468c8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:08.000622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-84db4ca6-83d6-45d8-902c-57170d907e4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:08.001762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-84db4ca6-83d6-45d8-902c-57170d907e4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:08.054944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84db4ca6-83d6-45d8-902c-57170d907e4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:08.059357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84db4ca6-83d6-45d8-902c-57170d907e4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:08.062635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84db4ca6-83d6-45d8-902c-57170d907e4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:08.063576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84db4ca6-83d6-45d8-902c-57170d907e4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:08.064222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84db4ca6-83d6-45d8-902c-57170d907e4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:08.065158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84db4ca6-83d6-45d8-902c-57170d907e4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:08.512990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-13885b8f-05a3-4eac-af3b-b701828468c8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:08.522240 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13885b8f-05a3-4eac-af3b-b701828468c8 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1317 microversion: 2.1 time: 1.084298 Sep 27 10:05:08.526927 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1189/2164] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:07 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1317 bytes in 1090 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:08.540873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-77300c67-d28d-4697-b474-df303b917b51 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63715b80>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:08.661642 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-77300c67-d28d-4697-b474-df303b917b51 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1426 microversion: 2.1 time: 0.133324 Sep 27 10:05:08.662092 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1190/2165] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:08 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1426 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:08.677525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:08.678149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:08.686153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:08.701501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:08.702361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:08.703854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:08.704484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:08.705372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:08.762509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-84db4ca6-83d6-45d8-902c-57170d907e4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:08.798924 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-84db4ca6-83d6-45d8-902c-57170d907e4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1307 microversion: 2.1 time: 0.806338 Sep 27 10:05:08.801232 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 976/2166] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:07 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1307 bytes in 810 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:08.812575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f2278632-b98e-4926-ade0-01f19cfb1591 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f639631f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:08.900955 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f2278632-b98e-4926-ade0-01f19cfb1591 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 0.098130 Sep 27 10:05:08.902166 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 977/2167] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:08 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:08.967362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-88965426-03ed-41ce-9894-9a466fa962bf req-6817bedf-3c71-4916-afb4-45e5bda3c67a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b563b3f-326c-41ed-9185-2a6d112c871d", "name": "network-vif-plugged", "status": "completed", "tag": "917af05f-ebc3-433e-a57d-d09e51b4d3c7"}, {"server_uuid": "961407e4-8e66-4f1c-88bd-6d2b4795c11d", "name": "network-vif-plugged", "status": "completed", "tag": "52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:08.999777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-88965426-03ed-41ce-9894-9a466fa962bf req-6817bedf-3c71-4916-afb4-45e5bda3c67a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:09.002973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-88965426-03ed-41ce-9894-9a466fa962bf req-6817bedf-3c71-4916-afb4-45e5bda3c67a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:09.003799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-88965426-03ed-41ce-9894-9a466fa962bf req-6817bedf-3c71-4916-afb4-45e5bda3c67a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:09.045502 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-88965426-03ed-41ce-9894-9a466fa962bf req-6817bedf-3c71-4916-afb4-45e5bda3c67a service nova] Creating event network-vif-plugged:917af05f-ebc3-433e-a57d-d09e51b4d3c7 for instance 3b563b3f-326c-41ed-9185-2a6d112c871d on nch1-master-1 Sep 27 10:05:09.046041 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-88965426-03ed-41ce-9894-9a466fa962bf req-6817bedf-3c71-4916-afb4-45e5bda3c67a service nova] Creating event network-vif-plugged:52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f for instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d on nch2-master-1 Sep 27 10:05:09.065918 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-88965426-03ed-41ce-9894-9a466fa962bf req-6817bedf-3c71-4916-afb4-45e5bda3c67a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.103421 Sep 27 10:05:09.078432 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 978/2168] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:08 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:09.164150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:09.180455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:09.185888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:09.190399 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-46ee881b-262f-436e-9fe7-752245b1938f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1484 microversion: 2.25 time: 0.525146 Sep 27 10:05:09.202408 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1191/2169] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:08 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1484 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:09.552152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2782cc70-be7d-4f74-a2de-afa3ce3bf262 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:09.552917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2782cc70-be7d-4f74-a2de-afa3ce3bf262 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:09.567132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2782cc70-be7d-4f74-a2de-afa3ce3bf262 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:09.567629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2782cc70-be7d-4f74-a2de-afa3ce3bf262 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:09.573666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2782cc70-be7d-4f74-a2de-afa3ce3bf262 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:09.574598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2782cc70-be7d-4f74-a2de-afa3ce3bf262 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:09.576612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2782cc70-be7d-4f74-a2de-afa3ce3bf262 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:09.577276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2782cc70-be7d-4f74-a2de-afa3ce3bf262 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:09.679694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06c61af7-2454-41e3-a9b3-710eda2e0af7 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63ac3310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:09.782901 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-06c61af7-2454-41e3-a9b3-710eda2e0af7 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1426 microversion: 2.1 time: 0.108991 Sep 27 10:05:09.783503 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1192/2170] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:09 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1426 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:09.824697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e4e324b6-9431-404a-9cf9-cbf9e50dfa8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:09.825679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e4e324b6-9431-404a-9cf9-cbf9e50dfa8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:09.839754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4e324b6-9431-404a-9cf9-cbf9e50dfa8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:09.848814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4e324b6-9431-404a-9cf9-cbf9e50dfa8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:09.849308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4e324b6-9431-404a-9cf9-cbf9e50dfa8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:09.850101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4e324b6-9431-404a-9cf9-cbf9e50dfa8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:09.850580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4e324b6-9431-404a-9cf9-cbf9e50dfa8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:09.851021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4e324b6-9431-404a-9cf9-cbf9e50dfa8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:10.057531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2782cc70-be7d-4f74-a2de-afa3ce3bf262 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:10.071084 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2782cc70-be7d-4f74-a2de-afa3ce3bf262 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1317 microversion: 2.1 time: 0.521042 Sep 27 10:05:10.072450 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 979/2171] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:09 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1317 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:10.082236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-190909c8-e67a-4850-a316-b7203d124436 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:10.083861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-190909c8-e67a-4850-a316-b7203d124436 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:10.098626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-190909c8-e67a-4850-a316-b7203d124436 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:10.099508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-190909c8-e67a-4850-a316-b7203d124436 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:10.103046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-190909c8-e67a-4850-a316-b7203d124436 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:10.104149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-190909c8-e67a-4850-a316-b7203d124436 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:10.104923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-190909c8-e67a-4850-a316-b7203d124436 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:10.107491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-190909c8-e67a-4850-a316-b7203d124436 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:10.164938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-190909c8-e67a-4850-a316-b7203d124436 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:05:10.254596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e4e324b6-9431-404a-9cf9-cbf9e50dfa8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:10.271127 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e4e324b6-9431-404a-9cf9-cbf9e50dfa8c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1488 microversion: 2.1 time: 0.449035 Sep 27 10:05:10.272239 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1193/2172] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:09 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1488 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:10.295378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c61cf03-9b75-491f-b47a-3866084f9895 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6c2cc940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:10.372270 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-190909c8-e67a-4850-a316-b7203d124436 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 204 len: 0 microversion: 2.1 time: 0.296667 Sep 27 10:05:10.373125 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 980/2173] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:05:10 2022] DELETE /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:10.385975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:10.386606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:10.403952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:10.404305 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c61cf03-9b75-491f-b47a-3866084f9895 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 0.129435 Sep 27 10:05:10.404843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:10.405445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:10.405694 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1194/2174] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:10 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:10.406509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:10.407095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:10.414201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:10.453235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-525bd804-cf01-4921-980d-9f2202538c8f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-test-snap-2095895036"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:10.487557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-525bd804-cf01-4921-980d-9f2202538c8f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:10.501571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-525bd804-cf01-4921-980d-9f2202538c8f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:10.502073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-525bd804-cf01-4921-980d-9f2202538c8f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:10.502514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-525bd804-cf01-4921-980d-9f2202538c8f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:10.503234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-525bd804-cf01-4921-980d-9f2202538c8f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:10.503692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-525bd804-cf01-4921-980d-9f2202538c8f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:10.504120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-525bd804-cf01-4921-980d-9f2202538c8f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:10.906399 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-525bd804-cf01-4921-980d-9f2202538c8f tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "POST /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb/action" status: 202 len: 0 microversion: 2.1 time: 0.498065 Sep 27 10:05:10.919251 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1195/2175] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:05:10 2022] POST /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb/action => generated 0 bytes in 513 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 10:05:10.927946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7feb4b47-8eb7-4178-ac34-87ecfcc43eae tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:10.929104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7feb4b47-8eb7-4178-ac34-87ecfcc43eae tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:10.937371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7feb4b47-8eb7-4178-ac34-87ecfcc43eae tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:10.938119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7feb4b47-8eb7-4178-ac34-87ecfcc43eae tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:10.938748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7feb4b47-8eb7-4178-ac34-87ecfcc43eae tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:10.939648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7feb4b47-8eb7-4178-ac34-87ecfcc43eae tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:10.940251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7feb4b47-8eb7-4178-ac34-87ecfcc43eae tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:10.940715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7feb4b47-8eb7-4178-ac34-87ecfcc43eae tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:11.170943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:11.195095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:11.209830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:11.215108 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d873d55a-1d48-4db4-ac7b-f84999c32a3c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1484 microversion: 2.25 time: 0.836976 Sep 27 10:05:11.216120 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 981/2176] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:10 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1484 bytes in 840 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:11.231732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a96b1735-2456-45dd-a9aa-7e7359ef9974 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f636740d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:11.343173 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a96b1735-2456-45dd-a9aa-7e7359ef9974 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1426 microversion: 2.1 time: 0.125399 Sep 27 10:05:11.361135 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 982/2177] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:11 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1426 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:11.372186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-469c42ba-3c0a-4e60-98cd-b131c33f1639 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63760700>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:11.384756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7feb4b47-8eb7-4178-ac34-87ecfcc43eae tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:11.417061 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7feb4b47-8eb7-4178-ac34-87ecfcc43eae tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 200 len: 1485 microversion: 2.1 time: 0.491940 Sep 27 10:05:11.427897 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1196/2178] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:10 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 1485 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:11.461758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d09b69ce-6079-4225-a972-2460ecfde4a5 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:11.462482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d09b69ce-6079-4225-a972-2460ecfde4a5 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:11.489633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d09b69ce-6079-4225-a972-2460ecfde4a5 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:11.490176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d09b69ce-6079-4225-a972-2460ecfde4a5 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:11.490609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d09b69ce-6079-4225-a972-2460ecfde4a5 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:11.493798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d09b69ce-6079-4225-a972-2460ecfde4a5 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:11.494904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d09b69ce-6079-4225-a972-2460ecfde4a5 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:11.495420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d09b69ce-6079-4225-a972-2460ecfde4a5 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:11.516478 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-469c42ba-3c0a-4e60-98cd-b131c33f1639 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e" status: 200 len: 1412 microversion: 2.1 time: 0.152698 Sep 27 10:05:11.517424 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 983/2179] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:11 2022] GET /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e => generated 1412 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:11.526380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b6f464e0-174b-44bd-a067-5b76192f4d28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f63674ca0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:11.616864 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b6f464e0-174b-44bd-a067-5b76192f4d28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 0.097073 Sep 27 10:05:11.617600 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 984/2180] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:11 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:11.869135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d09b69ce-6079-4225-a972-2460ecfde4a5 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:11.881774 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d09b69ce-6079-4225-a972-2460ecfde4a5 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1317 microversion: 2.1 time: 0.446623 Sep 27 10:05:11.884724 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1197/2181] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:11 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1317 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:11.893393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a6dd3909-6b54-475c-a270-d10bb20169da tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f638f2ee0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:11.963135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea020c72-4d10-4561-b56e-9d8f8406261d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'create', calling method: .version_select at 0x7f7f637f3670>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1644292222", "description": "tempest-description-1440815208"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:12.036142 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a6dd3909-6b54-475c-a270-d10bb20169da tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e" status: 200 len: 1412 microversion: 2.1 time: 0.149255 Sep 27 10:05:12.041685 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1198/2182] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:11 2022] GET /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e => generated 1412 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:12.060386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8fe37924-a510-4a66-a7d3-b77e367d3525 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62be2a60>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:13.405928 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea020c72-4d10-4561-b56e-9d8f8406261d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 244 microversion: 2.1 time: 1.781375 Sep 27 10:05:13.407289 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 985/2183] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:05:11 2022] POST /compute/v2.1/os-security-groups => generated 244 bytes in 1789 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:13.415243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5c07629a-9b90-4110-8edd-60f9a6eb48a4 req-46e65020-1e4b-4f44-966f-a75254eef9de service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "219e4ceb-d811-4e6f-913f-c51410efcee6", "name": "network-vif-plugged", "status": "completed", "tag": "b2aabccc-ddd4-4740-aef9-c80e07db2d17"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:13.438348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5c07629a-9b90-4110-8edd-60f9a6eb48a4 req-46e65020-1e4b-4f44-966f-a75254eef9de service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:13.439176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5c07629a-9b90-4110-8edd-60f9a6eb48a4 req-46e65020-1e4b-4f44-966f-a75254eef9de service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:13.439854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5c07629a-9b90-4110-8edd-60f9a6eb48a4 req-46e65020-1e4b-4f44-966f-a75254eef9de service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:13.482111 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5c07629a-9b90-4110-8edd-60f9a6eb48a4 req-46e65020-1e4b-4f44-966f-a75254eef9de service nova] Creating event network-vif-plugged:b2aabccc-ddd4-4740-aef9-c80e07db2d17 for instance 219e4ceb-d811-4e6f-913f-c51410efcee6 on nch2-master-1 Sep 27 10:05:13.494721 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5c07629a-9b90-4110-8edd-60f9a6eb48a4 req-46e65020-1e4b-4f44-966f-a75254eef9de service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085650 Sep 27 10:05:13.497076 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 986/2184] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:13.513201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:13.519734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:13.583457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:13.583973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:13.585777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:13.594747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:13.613858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:13.625268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:14.463970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:14.489515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:14.501371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:14.505360 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4ae40af4-8e82-4517-ade8-d74db525ea8e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1484 microversion: 2.25 time: 1.005246 Sep 27 10:05:14.508924 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 987/2185] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:13 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1484 bytes in 1011 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:14.522658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b035a9bc-18d1-4c75-aa74-50a88f8dd51c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63637e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:21.473152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=75,nova_api:DELETE=2,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:21.874087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=50,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:23.579944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=29,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:24.469987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:26.222487 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8fe37924-a510-4a66-a7d3-b77e367d3525 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e" status: 204 len: 0 microversion: 2.1 time: 14.169548 Sep 27 10:05:26.223614 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1199/2186] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:12 2022] DELETE /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e => generated 0 bytes in 14172 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:26.229044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:26.229933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: e8ca3e24-d297-4435-abe8-a70d128f4bb4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:26.251945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:26.252783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:26.253307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:26.254429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:26.255311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:26.256149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:26.288379 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] HTTP exception thrown: Instance e8ca3e24-d297-4435-abe8-a70d128f4bb4 could not be found. Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Request method failure captured: Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 HTTP/1.1 Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsl2ww1DCmrYlsvxa1ad0I1ZocI8nPKJGQ863hbj2ddYMyd4rN82uIGtKsNLlGkxpy9k3eCbNYNIEH6L36dtBDP6h0PGa_WIvNJEnlD4pYTb78k24vaHnHkvI4YxqwLlRHU0jrpIrk9bJl_QJobcfTL2k7WBj1Ty6gn63cG3JKTY8h8 Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: ec0980b5824b425fbad9b8554cf5c42c Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-LiveMigrationTest-1517090534 Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:26.295486 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ec0980b5824b425fbad9b8554cf5c42c"}]}] Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-LiveMigrationTest-1517090534 Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: ec0980b5824b425fbad9b8554cf5c42c Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-LiveMigrationTest-1517090534 Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-LiveMigrationTest-1517090534-project Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: dce01d3212934161a38be853b4cd93ae Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-LiveMigrationTest-1517090534-project Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance e8ca3e24-d297-4435-abe8-a70d128f4bb4 could not be found. Sep 27 10:05:26.297029 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'e8ca3e24-d297-4435-abe8-a70d128f4bb4'} Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e8ca3e24-d297-4435-abe8-a70d128f4bb4 could not be found. Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:26.297941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e8ca3e24-d297-4435-abe8-a70d128f4bb4 could not be found. Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e8ca3e24-d297-4435-abe8-a70d128f4bb4 could not be found. Sep 27 10:05:26.299479 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:26.302759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Returning 404 to user: Instance e8ca3e24-d297-4435-abe8-a70d128f4bb4 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:26.310629 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f0b74fc-e702-4c3b-b416-779cc375942c tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4" status: 404 len: 111 microversion: 2.1 time: 0.085547 Sep 27 10:05:26.311497 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1200/2187] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:26 2022] GET /compute/v2.1/servers/e8ca3e24-d297-4435-abe8-a70d128f4bb4 => generated 111 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:26.316439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d21ee37-5084-4596-b576-f99040a4189c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62be2310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:26.328403 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b035a9bc-18d1-4c75-aa74-50a88f8dd51c tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1435 microversion: 2.1 time: 11.817827 Sep 27 10:05:26.339123 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 988/2188] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:14 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1435 bytes in 11819 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:26.342561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e46d4a61-12da-46c4-817c-1464b5b2b8c9 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:26.343217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e46d4a61-12da-46c4-817c-1464b5b2b8c9 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:26.349804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e46d4a61-12da-46c4-817c-1464b5b2b8c9 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:26.352503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e46d4a61-12da-46c4-817c-1464b5b2b8c9 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:26.352503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e46d4a61-12da-46c4-817c-1464b5b2b8c9 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:26.352503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e46d4a61-12da-46c4-817c-1464b5b2b8c9 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:26.352503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e46d4a61-12da-46c4-817c-1464b5b2b8c9 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:26.352992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e46d4a61-12da-46c4-817c-1464b5b2b8c9 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:26.398701 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d21ee37-5084-4596-b576-f99040a4189c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 0.086038 Sep 27 10:05:26.407042 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1201/2189] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:26 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:26.412039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8aca587-139b-48bf-a688-1535e27f3c60 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'create', calling method: .version_select at 0x7f7f63727f70>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-376389634", "description": "tempest-description-1004749600"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:26.688670 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f8aca587-139b-48bf-a688-1535e27f3c60 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 243 microversion: 2.1 time: 0.280186 Sep 27 10:05:26.689726 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1202/2190] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:05:26 2022] POST /compute/v2.1/os-security-groups => generated 243 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:26.694951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a77e70de-a4cc-4921-96c4-f023b2fd9579 req-7f47eaaa-41dd-4a3c-81fe-59781b77b8d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0aa6e63-d0ba-47c2-8f6b-572f168104ee", "name": "network-vif-unplugged", "status": "completed", "tag": "f97afb19-a003-40a4-a6f6-2158ddcc5448"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:26.721957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a77e70de-a4cc-4921-96c4-f023b2fd9579 req-7f47eaaa-41dd-4a3c-81fe-59781b77b8d5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:26.725396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a77e70de-a4cc-4921-96c4-f023b2fd9579 req-7f47eaaa-41dd-4a3c-81fe-59781b77b8d5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:26.727057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a77e70de-a4cc-4921-96c4-f023b2fd9579 req-7f47eaaa-41dd-4a3c-81fe-59781b77b8d5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:26.755683 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a77e70de-a4cc-4921-96c4-f023b2fd9579 req-7f47eaaa-41dd-4a3c-81fe-59781b77b8d5 service nova] Creating event network-vif-unplugged:f97afb19-a003-40a4-a6f6-2158ddcc5448 for instance b0aa6e63-d0ba-47c2-8f6b-572f168104ee on nch2-master-1 Sep 27 10:05:26.763726 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a77e70de-a4cc-4921-96c4-f023b2fd9579 req-7f47eaaa-41dd-4a3c-81fe-59781b77b8d5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.071876 Sep 27 10:05:26.766977 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1203/2191] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:26 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:26.783866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:26.783866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:26.796116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:26.796696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:26.800840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:26.801879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:26.802618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:26.803206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:27.020246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:27.042670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:27.044696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:27.046222 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a13f9918-dc58-4900-8567-804bee9c7f68 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1474 microversion: 2.25 time: 0.277711 Sep 27 10:05:27.063482 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1204/2192] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:26 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1474 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:27.078897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c63eebb6-75ef-4911-976f-168b130b229a req-87588210-9848-465d-b8d3-31263b185853 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "705d2d84-afe0-4760-bb57-66126418dacb", "name": "network-vif-plugged", "status": "completed", "tag": "3306435b-7a82-4c08-bb65-5f863338089e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:27.093638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c63eebb6-75ef-4911-976f-168b130b229a req-87588210-9848-465d-b8d3-31263b185853 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:27.094568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c63eebb6-75ef-4911-976f-168b130b229a req-87588210-9848-465d-b8d3-31263b185853 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:27.095307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c63eebb6-75ef-4911-976f-168b130b229a req-87588210-9848-465d-b8d3-31263b185853 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:27.113777 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c63eebb6-75ef-4911-976f-168b130b229a req-87588210-9848-465d-b8d3-31263b185853 service nova] Creating event network-vif-plugged:3306435b-7a82-4c08-bb65-5f863338089e for instance 705d2d84-afe0-4760-bb57-66126418dacb on nch1-master-1 Sep 27 10:05:27.123321 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c63eebb6-75ef-4911-976f-168b130b229a req-87588210-9848-465d-b8d3-31263b185853 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058088 Sep 27 10:05:27.124379 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1205/2193] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:27.128181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e46d4a61-12da-46c4-817c-1464b5b2b8c9 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:27.152173 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e46d4a61-12da-46c4-817c-1464b5b2b8c9 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1505 microversion: 2.1 time: 0.821346 Sep 27 10:05:27.165969 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 989/2194] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:26 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1505 bytes in 836 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:27.179230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0cb04172-ed95-45bd-b3ab-9663126f6192 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f6c222e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:27.208030 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-0cb04172-ed95-45bd-b3ab-9663126f6192 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Image not found. Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0cb04172-ed95-45bd-b3ab-9663126f6192 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e HTTP/1.1 Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:05:27.211908 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: exception: Image not found. Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'a05f4bb5-4b9e-4225-95ff-aaf45774503e'} Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:27.213241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID a05f4bb5-4b9e-4225-95ff-aaf45774503e Sep 27 10:05:27.213958 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:05:27.214939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image a05f4bb5-4b9e-4225-95ff-aaf45774503e could not be found. Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:27.217209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:05:27.218528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:27.221998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0cb04172-ed95-45bd-b3ab-9663126f6192 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Image not found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:27.223918 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0cb04172-ed95-45bd-b3ab-9663126f6192 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e" status: 404 len: 62 microversion: 2.1 time: 0.055908 Sep 27 10:05:27.225797 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 990/2195] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:27 2022] GET /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e => generated 62 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:05:27.231994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f50bc7c5-8477-40ae-b342-de3b7f3281e6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:27.232764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f50bc7c5-8477-40ae-b342-de3b7f3281e6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:27.244718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f50bc7c5-8477-40ae-b342-de3b7f3281e6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:27.246345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f50bc7c5-8477-40ae-b342-de3b7f3281e6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:27.247383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f50bc7c5-8477-40ae-b342-de3b7f3281e6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:27.249825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f50bc7c5-8477-40ae-b342-de3b7f3281e6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:27.252514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f50bc7c5-8477-40ae-b342-de3b7f3281e6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:27.253181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f50bc7c5-8477-40ae-b342-de3b7f3281e6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:27.285113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1000990118", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "0117c46b-39f2-4cf5-9f8a-7710125bdf55"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:27.295073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f50bc7c5-8477-40ae-b342-de3b7f3281e6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:05:27.358273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] validate_networks() for [('0117c46b-39f2-4cf5-9f8a-7710125bdf55', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:05:27.471682 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f50bc7c5-8477-40ae-b342-de3b7f3281e6 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "DELETE /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 204 len: 0 microversion: 2.1 time: 0.243818 Sep 27 10:05:27.476756 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 991/2196] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:05:27 2022] DELETE /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:27.484212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3ea7f6c0-f78e-40b7-8eca-03c70ba1fcdc tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f6377b0d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:27.572821 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3ea7f6c0-f78e-40b7-8eca-03c70ba1fcdc tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 200 len: 1435 microversion: 2.1 time: 0.093502 Sep 27 10:05:27.582529 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 992/2197] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:27 2022] GET /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 1435 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:27.582529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-SecurityGroupsTestJSON-server-289231629", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "63be3c83-0a96-45db-bf25-b57bb15795b7"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:27.675666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] validate_networks() for [('63be3c83-0a96-45db-bf25-b57bb15795b7', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:05:28.025143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:05:28.025575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:05:28.026094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:05:28.026590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:05:28.027358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:05:28.051454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.051954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.052382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.053436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.054486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.055380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.094024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:28.112597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for user 7f7c5adbac4f48b6bc141e0f888f2496 and project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:28.138687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:05:28.142170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:05:28.165883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:05:28.167001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:05:28.167631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:05:28.168194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:05:28.168608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:05:28.185514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.186298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.188712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.197356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.198803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.199519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.251324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Getting quotas for project 513655e40ccd4152a2e343e5415ce347. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:28.255016 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-78138d46-6c9a-4f9c-9cbb-d5166600fed0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.129009 Sep 27 10:05:28.255687 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1206/2198] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:05:27 2022] POST /compute/v2.1/servers => generated 380 bytes in 1131 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:05:28.260382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-45e97bff-7d05-4a3c-8b80-4c2025896c50 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '.version_select at 0x7f7f6c2d9af0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:28.263581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Getting quotas for user 866c1a2a2ca8452eb0a66cbe9692ab75 and project 513655e40ccd4152a2e343e5415ce347. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:28.263850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e97bff-7d05-4a3c-8b80-4c2025896c50 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.277793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e97bff-7d05-4a3c-8b80-4c2025896c50 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.277793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e97bff-7d05-4a3c-8b80-4c2025896c50 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.298588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:05:28.302941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:05:28.309291 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-45e97bff-7d05-4a3c-8b80-4c2025896c50 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/os-migrations" status: 200 len: 2118 microversion: 2.25 time: 0.052493 Sep 27 10:05:28.309972 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1207/2199] 10.222.0.26 () {60 vars in 1184 bytes} [Tue Sep 27 10:05:28 2022] GET /compute/v2.1/os-migrations => generated 2118 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:28.327898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-27aca784-fde3-4d4d-b095-eb38f8b5d8e6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:28.341917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-27aca784-fde3-4d4d-b095-eb38f8b5d8e6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:28.350743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27aca784-fde3-4d4d-b095-eb38f8b5d8e6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.351577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27aca784-fde3-4d4d-b095-eb38f8b5d8e6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.352521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27aca784-fde3-4d4d-b095-eb38f8b5d8e6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.353676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27aca784-fde3-4d4d-b095-eb38f8b5d8e6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.355154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27aca784-fde3-4d4d-b095-eb38f8b5d8e6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.355812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27aca784-fde3-4d4d-b095-eb38f8b5d8e6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.407018 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-27aca784-fde3-4d4d-b095-eb38f8b5d8e6 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6/ips" status: 200 len: 108 microversion: 2.1 time: 0.082552 Sep 27 10:05:28.409061 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1208/2200] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:05:28 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6/ips => generated 108 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:28.421995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6b52cc8d-2f3d-4215-a78b-fcf9ae2ccc27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:28.422834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6b52cc8d-2f3d-4215-a78b-fcf9ae2ccc27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:28.427474 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5b90f33f-97e8-40be-9aab-d90808f740f2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.852436 Sep 27 10:05:28.434246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b52cc8d-2f3d-4215-a78b-fcf9ae2ccc27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.437054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b52cc8d-2f3d-4215-a78b-fcf9ae2ccc27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.437169 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 993/2201] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:05:27 2022] POST /compute/v2.1/servers => generated 380 bytes in 862 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:05:28.440743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b52cc8d-2f3d-4215-a78b-fcf9ae2ccc27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.442229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b52cc8d-2f3d-4215-a78b-fcf9ae2ccc27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.443043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b52cc8d-2f3d-4215-a78b-fcf9ae2ccc27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.443705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b52cc8d-2f3d-4215-a78b-fcf9ae2ccc27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.447012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e1334da7-2536-4604-ba58-e87071ab5746 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f637f3d30>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:28.550694 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e1334da7-2536-4604-ba58-e87071ab5746 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 0.112329 Sep 27 10:05:28.562957 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 994/2202] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:28 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:28.570739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f52da94-dabb-49f5-9d21-0cb5fc443fc0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:28.578321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f52da94-dabb-49f5-9d21-0cb5fc443fc0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:28.641731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f52da94-dabb-49f5-9d21-0cb5fc443fc0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.643413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f52da94-dabb-49f5-9d21-0cb5fc443fc0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.644239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f52da94-dabb-49f5-9d21-0cb5fc443fc0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.655258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f52da94-dabb-49f5-9d21-0cb5fc443fc0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.658189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f52da94-dabb-49f5-9d21-0cb5fc443fc0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.658747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f52da94-dabb-49f5-9d21-0cb5fc443fc0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.928360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6b52cc8d-2f3d-4215-a78b-fcf9ae2ccc27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:28.955178 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6b52cc8d-2f3d-4215-a78b-fcf9ae2ccc27 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1488 microversion: 2.1 time: 0.542626 Sep 27 10:05:28.958626 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1209/2203] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:28 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1488 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:28.968963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-185e7c71-8a6c-4d76-a34f-877ae46450c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:28.970102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-185e7c71-8a6c-4d76-a34f-877ae46450c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:28.981692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-185e7c71-8a6c-4d76-a34f-877ae46450c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.988819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-185e7c71-8a6c-4d76-a34f-877ae46450c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.996788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-185e7c71-8a6c-4d76-a34f-877ae46450c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:28.997953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-185e7c71-8a6c-4d76-a34f-877ae46450c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:28.998911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-185e7c71-8a6c-4d76-a34f-877ae46450c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:28.999542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-185e7c71-8a6c-4d76-a34f-877ae46450c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:29.389749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-185e7c71-8a6c-4d76-a34f-877ae46450c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:29.406929 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-185e7c71-8a6c-4d76-a34f-877ae46450c4 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1467 microversion: 2.1 time: 0.446903 Sep 27 10:05:29.408149 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1210/2204] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:28 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1467 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:29.416326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8973bedb-6ad0-4ed0-bc88-01ea05de7b82 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:29.417660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8973bedb-6ad0-4ed0-bc88-01ea05de7b82 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:29.435593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8973bedb-6ad0-4ed0-bc88-01ea05de7b82 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:29.436459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8973bedb-6ad0-4ed0-bc88-01ea05de7b82 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:29.437850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8973bedb-6ad0-4ed0-bc88-01ea05de7b82 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:29.438846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8973bedb-6ad0-4ed0-bc88-01ea05de7b82 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:29.439768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8973bedb-6ad0-4ed0-bc88-01ea05de7b82 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:29.440333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8973bedb-6ad0-4ed0-bc88-01ea05de7b82 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:29.521157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8973bedb-6ad0-4ed0-bc88-01ea05de7b82 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:29.534363 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8973bedb-6ad0-4ed0-bc88-01ea05de7b82 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1219 microversion: 2.1 time: 0.123085 Sep 27 10:05:29.535562 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1211/2205] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:29 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1219 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:29.541822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-db5fe23f-993a-45fc-9200-3996ca688386 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:29.543026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-db5fe23f-993a-45fc-9200-3996ca688386 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:29.553182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db5fe23f-993a-45fc-9200-3996ca688386 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:29.554293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db5fe23f-993a-45fc-9200-3996ca688386 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:29.555544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db5fe23f-993a-45fc-9200-3996ca688386 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:29.557055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db5fe23f-993a-45fc-9200-3996ca688386 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:29.558135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db5fe23f-993a-45fc-9200-3996ca688386 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:29.558723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db5fe23f-993a-45fc-9200-3996ca688386 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:29.691849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8f52da94-dabb-49f5-9d21-0cb5fc443fc0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:29.707211 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f52da94-dabb-49f5-9d21-0cb5fc443fc0 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1499 microversion: 2.1 time: 1.136938 Sep 27 10:05:29.736496 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 995/2206] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:28 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1499 bytes in 1169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:29.751279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e1ab7ac-7e3d-4721-a80c-76fb1d7b1c5b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:29.751894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4e1ab7ac-7e3d-4721-a80c-76fb1d7b1c5b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:29.783569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e1ab7ac-7e3d-4721-a80c-76fb1d7b1c5b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:29.786676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e1ab7ac-7e3d-4721-a80c-76fb1d7b1c5b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:29.787594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e1ab7ac-7e3d-4721-a80c-76fb1d7b1c5b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:29.791598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e1ab7ac-7e3d-4721-a80c-76fb1d7b1c5b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:29.792977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e1ab7ac-7e3d-4721-a80c-76fb1d7b1c5b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:29.817051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e1ab7ac-7e3d-4721-a80c-76fb1d7b1c5b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:29.965565 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4e1ab7ac-7e3d-4721-a80c-76fb1d7b1c5b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6/ips" status: 200 len: 108 microversion: 2.1 time: 0.227529 Sep 27 10:05:29.966569 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 996/2207] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:05:29 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6/ips => generated 108 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:29.983390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e66c0a12-26ce-41ef-99ac-237357ccbfc7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:29.985969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-db5fe23f-993a-45fc-9200-3996ca688386 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] Lazy-loading 'services' on Instance uuid b0aa6e63-d0ba-47c2-8f6b-572f168104ee {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:05:29.988288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e66c0a12-26ce-41ef-99ac-237357ccbfc7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:30.019452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66c0a12-26ce-41ef-99ac-237357ccbfc7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:30.019661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66c0a12-26ce-41ef-99ac-237357ccbfc7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:30.019971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66c0a12-26ce-41ef-99ac-237357ccbfc7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:30.022938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66c0a12-26ce-41ef-99ac-237357ccbfc7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:30.024272 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-db5fe23f-993a-45fc-9200-3996ca688386 tempest-LiveAutoBlockMigrationV225Test-1224109597 tempest-LiveAutoBlockMigrationV225Test-1224109597-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1965 microversion: 2.25 time: 0.486771 Sep 27 10:05:30.025437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66c0a12-26ce-41ef-99ac-237357ccbfc7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:30.025873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e66c0a12-26ce-41ef-99ac-237357ccbfc7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:30.028861 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1212/2208] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:29 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1965 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:30.045546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aea86fea-4b33-4b2c-a0b7-e82cf898d792 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:30.055652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aea86fea-4b33-4b2c-a0b7-e82cf898d792 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:30.066232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea86fea-4b33-4b2c-a0b7-e82cf898d792 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:30.075764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea86fea-4b33-4b2c-a0b7-e82cf898d792 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:30.077273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea86fea-4b33-4b2c-a0b7-e82cf898d792 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:30.078584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea86fea-4b33-4b2c-a0b7-e82cf898d792 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:30.083392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea86fea-4b33-4b2c-a0b7-e82cf898d792 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:30.084048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea86fea-4b33-4b2c-a0b7-e82cf898d792 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:30.157423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aea86fea-4b33-4b2c-a0b7-e82cf898d792 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:05:30.199400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e66c0a12-26ce-41ef-99ac-237357ccbfc7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:30.225178 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e66c0a12-26ce-41ef-99ac-237357ccbfc7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1211 microversion: 2.1 time: 0.241791 Sep 27 10:05:30.249679 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 997/2209] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:29 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1211 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:30.266092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-8b925e5f-b8e3-4787-b4fb-58a644467c23 req-f56587e4-8021-4b36-8bec-b87ed81f3070 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e8ca3e24-d297-4435-abe8-a70d128f4bb4", "name": "network-vif-deleted", "tag": "769d6c7f-2742-4b0f-b84e-a0dcd3b8b472"}, {"name": "network-changed", "server_uuid": "b0aa6e63-d0ba-47c2-8f6b-572f168104ee", "tag": "f97afb19-a003-40a4-a6f6-2158ddcc5448"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:30.296498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8b925e5f-b8e3-4787-b4fb-58a644467c23 req-f56587e4-8021-4b36-8bec-b87ed81f3070 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:30.298503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8b925e5f-b8e3-4787-b4fb-58a644467c23 req-f56587e4-8021-4b36-8bec-b87ed81f3070 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:30.299250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8b925e5f-b8e3-4787-b4fb-58a644467c23 req-f56587e4-8021-4b36-8bec-b87ed81f3070 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:30.322516 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8b925e5f-b8e3-4787-b4fb-58a644467c23 req-f56587e4-8021-4b36-8bec-b87ed81f3070 service nova] Creating event network-vif-deleted:769d6c7f-2742-4b0f-b84e-a0dcd3b8b472 for instance e8ca3e24-d297-4435-abe8-a70d128f4bb4 on nch2-master-1 Sep 27 10:05:30.323020 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8b925e5f-b8e3-4787-b4fb-58a644467c23 req-f56587e4-8021-4b36-8bec-b87ed81f3070 service nova] Creating event network-changed:f97afb19-a003-40a4-a6f6-2158ddcc5448 for instance b0aa6e63-d0ba-47c2-8f6b-572f168104ee on nch2-master-1 Sep 27 10:05:30.350521 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-8b925e5f-b8e3-4787-b4fb-58a644467c23 req-f56587e4-8021-4b36-8bec-b87ed81f3070 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.096824 Sep 27 10:05:30.351634 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 998/2210] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:30 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:30.360353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-74dad08e-1b00-48ed-9d86-be1542a76cf8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f6360a4c0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:30.473809 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aea86fea-4b33-4b2c-a0b7-e82cf898d792 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 204 len: 0 microversion: 2.1 time: 0.442118 Sep 27 10:05:30.474414 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1213/2211] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:05:30 2022] DELETE /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 0 bytes in 443 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:30.491206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef4abcad-dd7b-4605-98ca-62debaea9afb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6c173790>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:31.639321 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-74dad08e-1b00-48ed-9d86-be1542a76cf8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "DELETE /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f" status: 204 len: 0 microversion: 2.1 time: 1.286328 Sep 27 10:05:31.639946 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 999/2212] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:30 2022] DELETE /compute/v2.1/images/ecea37fd-790b-4c26-8ff4-4a45a881011f => generated 0 bytes in 1288 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:31.645921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e23229ae-57d7-4679-a4f8-df663c8a7834 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:31.646641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e23229ae-57d7-4679-a4f8-df663c8a7834 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:31.663578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e23229ae-57d7-4679-a4f8-df663c8a7834 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.664631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e23229ae-57d7-4679-a4f8-df663c8a7834 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.665099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e23229ae-57d7-4679-a4f8-df663c8a7834 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.665815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e23229ae-57d7-4679-a4f8-df663c8a7834 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.666267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e23229ae-57d7-4679-a4f8-df663c8a7834 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.666840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e23229ae-57d7-4679-a4f8-df663c8a7834 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.700755 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef4abcad-dd7b-4605-98ca-62debaea9afb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1452 microversion: 2.1 time: 1.225068 Sep 27 10:05:31.706330 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1214/2213] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:30 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1452 bytes in 1231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:31.714129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:31.714843 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e23229ae-57d7-4679-a4f8-df663c8a7834 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6/ips/tempest-ServerAddressesTestJSON-396018772-network" status: 200 len: 93 microversion: 2.1 time: 0.073884 Sep 27 10:05:31.722891 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1000/2214] 10.222.0.26 () {58 vars in 1398 bytes} [Tue Sep 27 10:05:31 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6/ips/tempest-ServerAddressesTestJSON-396018772-network => generated 93 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:05:31.723201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:31.732546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b9f3856a-1526-437e-ac81-939ab4c2b592 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f636d7f70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:31.736462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.736989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.738363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.739122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.739585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.740183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.761542 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-b9f3856a-1526-437e-ac81-939ab4c2b592 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Image not found. Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b9f3856a-1526-437e-ac81-939ab4c2b592 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e HTTP/1.1 Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:05:31.763628 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: exception: Image not found. Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'a05f4bb5-4b9e-4225-95ff-aaf45774503e'} Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:31.764978 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image a05f4bb5-4b9e-4225-95ff-aaf45774503e to delete Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:31.765830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image a05f4bb5-4b9e-4225-95ff-aaf45774503e could not be found. Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:31.766996 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:05:31.768214 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:31.769768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b9f3856a-1526-437e-ac81-939ab4c2b592 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Image not found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:31.770819 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b9f3856a-1526-437e-ac81-939ab4c2b592 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e" status: 404 len: 62 microversion: 2.1 time: 0.043892 Sep 27 10:05:31.771817 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1001/2215] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:31 2022] DELETE /compute/v2.1/images/a05f4bb5-4b9e-4225-95ff-aaf45774503e => generated 62 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:05:31.776969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:05:31.779495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e7579adc-a60a-4f4d-aef5-8e87f0d01675 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:31.781674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e7579adc-a60a-4f4d-aef5-8e87f0d01675 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:31.791233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7579adc-a60a-4f4d-aef5-8e87f0d01675 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.791934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7579adc-a60a-4f4d-aef5-8e87f0d01675 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.792513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7579adc-a60a-4f4d-aef5-8e87f0d01675 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.793589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7579adc-a60a-4f4d-aef5-8e87f0d01675 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.794532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7579adc-a60a-4f4d-aef5-8e87f0d01675 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.795169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7579adc-a60a-4f4d-aef5-8e87f0d01675 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.882195 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d557dfb3-ea20-4d6a-9da0-8118bd48f32f tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "DELETE /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 204 len: 0 microversion: 2.25 time: 0.173886 Sep 27 10:05:31.884628 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1215/2216] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:05:31 2022] DELETE /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 0 bytes in 177 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:05:31.891135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-da4143aa-3ebe-4f7d-9fe0-41363b7a8ccf req-73f40274-90aa-4728-a182-51158c8b3455 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "705d2d84-afe0-4760-bb57-66126418dacb", "name": "network-vif-unplugged", "status": "completed", "tag": "3306435b-7a82-4c08-bb65-5f863338089e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:31.905703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-da4143aa-3ebe-4f7d-9fe0-41363b7a8ccf req-73f40274-90aa-4728-a182-51158c8b3455 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.906551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-da4143aa-3ebe-4f7d-9fe0-41363b7a8ccf req-73f40274-90aa-4728-a182-51158c8b3455 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.907152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-da4143aa-3ebe-4f7d-9fe0-41363b7a8ccf req-73f40274-90aa-4728-a182-51158c8b3455 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.924425 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-da4143aa-3ebe-4f7d-9fe0-41363b7a8ccf req-73f40274-90aa-4728-a182-51158c8b3455 service nova] Creating event network-vif-unplugged:3306435b-7a82-4c08-bb65-5f863338089e for instance 705d2d84-afe0-4760-bb57-66126418dacb on nch1-master-1 Sep 27 10:05:31.924568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e7579adc-a60a-4f4d-aef5-8e87f0d01675 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:31.930400 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-da4143aa-3ebe-4f7d-9fe0-41363b7a8ccf req-73f40274-90aa-4728-a182-51158c8b3455 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.043962 Sep 27 10:05:31.931432 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1216/2217] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:31 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:31.938641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:31.939143 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e7579adc-a60a-4f4d-aef5-8e87f0d01675 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1273 microversion: 2.1 time: 0.165882 Sep 27 10:05:31.939795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:31.940859 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1002/2218] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:31 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1273 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:31.947673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.948409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.949193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.950046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.950639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.951197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.954304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6c3ab877-5710-44f1-9aa2-e0de62cd871e tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:31.955125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6c3ab877-5710-44f1-9aa2-e0de62cd871e tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:31.968717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3ab877-5710-44f1-9aa2-e0de62cd871e tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.971190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3ab877-5710-44f1-9aa2-e0de62cd871e tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.971785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3ab877-5710-44f1-9aa2-e0de62cd871e tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.974465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3ab877-5710-44f1-9aa2-e0de62cd871e tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.974957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3ab877-5710-44f1-9aa2-e0de62cd871e tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.975613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c3ab877-5710-44f1-9aa2-e0de62cd871e tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.994384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:31.995252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:31.995831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:31.996476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lazy-loading 'fault' on Instance uuid 16cd150f-c67a-4a9e-9654-ba6bb9e1517a {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:05:32.053547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:32.066340 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c719bf73-228e-4aae-97ff-dfe6013a4091 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1300 microversion: 2.1 time: 0.133494 Sep 27 10:05:32.073414 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1217/2219] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:31 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1300 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:32.086526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d7a10aa-f6fc-475d-9feb-a5ef5e88efa9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '.version_select at 0x7f7f63aa3430>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:32.121402 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-4d7a10aa-f6fc-475d-9feb-a5ef5e88efa9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] HTTP exception thrown: Image not found. Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d7a10aa-f6fc-475d-9feb-a5ef5e88efa9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Request method failure captured: Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 HTTP/1.1 Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsqS8wTqwJCKAaZfBPF_2GNu90piB3rTxDlvrWvyfpVn6Ph3kb3VLfMM5uXumXMZCvvsRDaPbrg7UKxhCmUYWyv6FeFeUaguQxNpFTL2dK6jI7chvqj_DBGJ05vqT2g2d8KV_lycaLPzFP-wAXgdsdjSmN55CqgqVC-YkSijA2Ko5UA Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 58c33b046dc7494c970f26eb5900b3f3 Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesOneServerTestJSON-353655151 Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:32.123599 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/58c33b046dc7494c970f26eb5900b3f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/58c33b046dc7494c970f26eb5900b3f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/58c33b046dc7494c970f26eb5900b3f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_58c33b046dc7494c970f26eb5900b3f3"}]}] Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesOneServerTestJSON-353655151 Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 58c33b046dc7494c970f26eb5900b3f3 Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesOneServerTestJSON-353655151 Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesOneServerTestJSON-353655151-project Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a10600a9277a436396fc70971dd27893 Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesOneServerTestJSON-353655151-project Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: exception: Image not found. Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '7067aeff-ca0d-43a5-a172-93e103d961e7'} Sep 27 10:05:32.124902 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 7067aeff-ca0d-43a5-a172-93e103d961e7 to delete Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:32.125587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 7067aeff-ca0d-43a5-a172-93e103d961e7 could not be found. Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:32.126509 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:05:32.127587 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:32.130361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d7a10aa-f6fc-475d-9feb-a5ef5e88efa9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Returning 404 to user: Image not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:32.131790 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4d7a10aa-f6fc-475d-9feb-a5ef5e88efa9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "DELETE /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7" status: 404 len: 62 microversion: 2.1 time: 0.057039 Sep 27 10:05:32.133070 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1218/2220] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:32 2022] DELETE /compute/v2.1/images/7067aeff-ca0d-43a5-a172-93e103d961e7 => generated 62 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:05:32.139485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4cb5256-3c4b-4dfc-acd2-9568203d5265 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:32.140537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4cb5256-3c4b-4dfc-acd2-9568203d5265 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:32.148333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4cb5256-3c4b-4dfc-acd2-9568203d5265 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.150389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4cb5256-3c4b-4dfc-acd2-9568203d5265 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.151059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4cb5256-3c4b-4dfc-acd2-9568203d5265 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:32.152050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4cb5256-3c4b-4dfc-acd2-9568203d5265 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.153085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4cb5256-3c4b-4dfc-acd2-9568203d5265 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.153726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4cb5256-3c4b-4dfc-acd2-9568203d5265 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:32.165921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6c3ab877-5710-44f1-9aa2-e0de62cd871e tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:32.183196 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6c3ab877-5710-44f1-9aa2-e0de62cd871e tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1267 microversion: 2.1 time: 0.241061 Sep 27 10:05:32.186069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4cb5256-3c4b-4dfc-acd2-9568203d5265 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:05:32.209839 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1003/2221] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:31 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1267 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:32.220731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a8a7bf0f-6cc1-433f-b25c-541e16f5cf5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:32.221243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a8a7bf0f-6cc1-433f-b25c-541e16f5cf5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:32.235151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:32.244080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8a7bf0f-6cc1-433f-b25c-541e16f5cf5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.247525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8a7bf0f-6cc1-433f-b25c-541e16f5cf5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.250862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8a7bf0f-6cc1-433f-b25c-541e16f5cf5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:32.251880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8a7bf0f-6cc1-433f-b25c-541e16f5cf5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.252563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8a7bf0f-6cc1-433f-b25c-541e16f5cf5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.253093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8a7bf0f-6cc1-433f-b25c-541e16f5cf5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:32.309151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a8a7bf0f-6cc1-433f-b25c-541e16f5cf5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:05:32.348071 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4cb5256-3c4b-4dfc-acd2-9568203d5265 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "DELETE /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 204 len: 0 microversion: 2.1 time: 0.212762 Sep 27 10:05:32.352693 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1219/2222] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:05:32 2022] DELETE /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:32.359153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:32.363732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:32.379295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.379962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.380491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:32.385389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.386080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.386633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:32.451613 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.compute.api [None req-a8a7bf0f-6cc1-433f-b25c-541e16f5cf5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Instance is already in deleting state, ignoring this request Sep 27 10:05:32.454909 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a8a7bf0f-6cc1-433f-b25c-541e16f5cf5c tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 204 len: 0 microversion: 2.1 time: 0.243787 Sep 27 10:05:32.456890 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1004/2223] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:05:32 2022] DELETE /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 0 bytes in 246 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:32.473976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73a56b1c-9e60-4e09-9f55-6217a82ceb09 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:32.474522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-73a56b1c-9e60-4e09-9f55-6217a82ceb09 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:32.490393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73a56b1c-9e60-4e09-9f55-6217a82ceb09 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.490907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73a56b1c-9e60-4e09-9f55-6217a82ceb09 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.491358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73a56b1c-9e60-4e09-9f55-6217a82ceb09 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:32.492134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73a56b1c-9e60-4e09-9f55-6217a82ceb09 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.492954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73a56b1c-9e60-4e09-9f55-6217a82ceb09 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.493993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73a56b1c-9e60-4e09-9f55-6217a82ceb09 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:32.526938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-73a56b1c-9e60-4e09-9f55-6217a82ceb09 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:05:32.648243 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-73a56b1c-9e60-4e09-9f55-6217a82ceb09 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 204 len: 0 microversion: 2.1 time: 0.188320 Sep 27 10:05:32.649819 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1005/2224] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:05:32 2022] DELETE /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:32.659981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0728c4b8-d4f3-494f-ad63-93e6871f5098 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:32.661105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0728c4b8-d4f3-494f-ad63-93e6871f5098 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:32.668287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0728c4b8-d4f3-494f-ad63-93e6871f5098 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.669121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0728c4b8-d4f3-494f-ad63-93e6871f5098 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.669742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0728c4b8-d4f3-494f-ad63-93e6871f5098 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:32.670753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0728c4b8-d4f3-494f-ad63-93e6871f5098 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.671363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0728c4b8-d4f3-494f-ad63-93e6871f5098 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.671947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0728c4b8-d4f3-494f-ad63-93e6871f5098 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:32.873154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0728c4b8-d4f3-494f-ad63-93e6871f5098 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:32.883161 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0728c4b8-d4f3-494f-ad63-93e6871f5098 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1511 microversion: 2.1 time: 0.227470 Sep 27 10:05:32.883977 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1006/2225] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:32 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1511 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:32.902787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9d3ef59f-c431-496f-8b9e-9ff671a51455 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:32.903553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9d3ef59f-c431-496f-8b9e-9ff671a51455 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:32.911503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d3ef59f-c431-496f-8b9e-9ff671a51455 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.911788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d3ef59f-c431-496f-8b9e-9ff671a51455 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.912070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d3ef59f-c431-496f-8b9e-9ff671a51455 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:32.913115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d3ef59f-c431-496f-8b9e-9ff671a51455 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:32.913616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d3ef59f-c431-496f-8b9e-9ff671a51455 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:32.914032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d3ef59f-c431-496f-8b9e-9ff671a51455 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.075655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:33.092445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:33.093927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:33.095306 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef95f427-4b7d-4517-a4fe-daec9a1387e9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1480 microversion: 2.25 time: 0.741347 Sep 27 10:05:33.096550 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1220/2226] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:32 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1480 bytes in 743 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:33.102306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-801d6d77-2b45-4f1d-bc67-e32a920793ca req-ea5e9e8c-571c-42ba-be49-e21262a8f21c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "16cd150f-c67a-4a9e-9654-ba6bb9e1517a", "name": "network-vif-deleted", "tag": "fa2d1e07-1a2a-40d0-baa6-738c999a9e3a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:33.127380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-801d6d77-2b45-4f1d-bc67-e32a920793ca req-ea5e9e8c-571c-42ba-be49-e21262a8f21c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.128282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-801d6d77-2b45-4f1d-bc67-e32a920793ca req-ea5e9e8c-571c-42ba-be49-e21262a8f21c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.129228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9d3ef59f-c431-496f-8b9e-9ff671a51455 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:33.129374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-801d6d77-2b45-4f1d-bc67-e32a920793ca req-ea5e9e8c-571c-42ba-be49-e21262a8f21c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.134408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:33.141202 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9d3ef59f-c431-496f-8b9e-9ff671a51455 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1494 microversion: 2.1 time: 0.242576 Sep 27 10:05:33.149965 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-801d6d77-2b45-4f1d-bc67-e32a920793ca req-ea5e9e8c-571c-42ba-be49-e21262a8f21c service nova] Creating event network-vif-deleted:fa2d1e07-1a2a-40d0-baa6-738c999a9e3a for instance 16cd150f-c67a-4a9e-9654-ba6bb9e1517a on nch2-master-1 Sep 27 10:05:33.158381 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-801d6d77-2b45-4f1d-bc67-e32a920793ca req-ea5e9e8c-571c-42ba-be49-e21262a8f21c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059699 Sep 27 10:05:33.160728 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1221/2227] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:33.161861 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1007/2228] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:32 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1494 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:33.167614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fe8a299b-8f4c-4b64-b6fc-d13b3cd6f7f8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:33.168292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7a3ba8c9-f88c-4c30-be3a-97212de707fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6366dee0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:33.173020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fe8a299b-8f4c-4b64-b6fc-d13b3cd6f7f8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:33.180860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8a299b-8f4c-4b64-b6fc-d13b3cd6f7f8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.181639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8a299b-8f4c-4b64-b6fc-d13b3cd6f7f8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.182144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8a299b-8f4c-4b64-b6fc-d13b3cd6f7f8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.182877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8a299b-8f4c-4b64-b6fc-d13b3cd6f7f8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.183351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8a299b-8f4c-4b64-b6fc-d13b3cd6f7f8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.183774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8a299b-8f4c-4b64-b6fc-d13b3cd6f7f8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.230787 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7a3ba8c9-f88c-4c30-be3a-97212de707fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1461 microversion: 2.1 time: 0.066889 Sep 27 10:05:33.231496 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1008/2229] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:33 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1461 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:33.239891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-85900a6f-0a7e-47c8-822e-6c51d25a0661 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:33.240845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-85900a6f-0a7e-47c8-822e-6c51d25a0661 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:33.247580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85900a6f-0a7e-47c8-822e-6c51d25a0661 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.248068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85900a6f-0a7e-47c8-822e-6c51d25a0661 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.248498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85900a6f-0a7e-47c8-822e-6c51d25a0661 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.249211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85900a6f-0a7e-47c8-822e-6c51d25a0661 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.249665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85900a6f-0a7e-47c8-822e-6c51d25a0661 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.250089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85900a6f-0a7e-47c8-822e-6c51d25a0661 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.403567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-85900a6f-0a7e-47c8-822e-6c51d25a0661 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:33.417962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fe8a299b-8f4c-4b64-b6fc-d13b3cd6f7f8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:33.432000 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fe8a299b-8f4c-4b64-b6fc-d13b3cd6f7f8 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1473 microversion: 2.1 time: 0.269655 Sep 27 10:05:33.435925 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-85900a6f-0a7e-47c8-822e-6c51d25a0661 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 0.203070 Sep 27 10:05:33.436634 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1009/2230] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:33 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:33.450985 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1222/2231] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:33 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1473 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:33.458053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ca7d7fa3-1072-42b1-badb-16a71d93c83b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:33.460636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:33.462608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ca7d7fa3-1072-42b1-badb-16a71d93c83b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:33.463147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:33.473698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.474724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.475382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.476328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.477204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.477907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.479546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca7d7fa3-1072-42b1-badb-16a71d93c83b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.480549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca7d7fa3-1072-42b1-badb-16a71d93c83b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.481852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca7d7fa3-1072-42b1-badb-16a71d93c83b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.484762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca7d7fa3-1072-42b1-badb-16a71d93c83b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.485463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca7d7fa3-1072-42b1-badb-16a71d93c83b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.486075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca7d7fa3-1072-42b1-badb-16a71d93c83b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.517116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.517794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.518556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.519093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lazy-loading 'fault' on Instance uuid 16cd150f-c67a-4a9e-9654-ba6bb9e1517a {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:05:33.556475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ca7d7fa3-1072-42b1-badb-16a71d93c83b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:33.565599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:33.568447 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ca7d7fa3-1072-42b1-badb-16a71d93c83b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1275 microversion: 2.1 time: 0.115799 Sep 27 10:05:33.569258 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1223/2232] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:33 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1275 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:33.574213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d9c82a76-ef74-46e0-aef4-e2cfedae9ad9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6c29b1f0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:33.584643 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-73f8f7eb-46bc-4454-8cfa-07e859f4ac7e tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1300 microversion: 2.1 time: 0.134537 Sep 27 10:05:33.585445 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1010/2233] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:33 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1300 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:33.590465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-cfc37d00-94c3-4321-85b0-8b45bdae7778 req-04950309-2fa7-45ea-aed6-544355daff74 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "705d2d84-afe0-4760-bb57-66126418dacb", "name": "network-vif-deleted", "tag": "3306435b-7a82-4c08-bb65-5f863338089e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:33.601213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cfc37d00-94c3-4321-85b0-8b45bdae7778 req-04950309-2fa7-45ea-aed6-544355daff74 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.601727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cfc37d00-94c3-4321-85b0-8b45bdae7778 req-04950309-2fa7-45ea-aed6-544355daff74 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.602165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cfc37d00-94c3-4321-85b0-8b45bdae7778 req-04950309-2fa7-45ea-aed6-544355daff74 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.615277 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-cfc37d00-94c3-4321-85b0-8b45bdae7778 req-04950309-2fa7-45ea-aed6-544355daff74 service nova] Creating event network-vif-deleted:3306435b-7a82-4c08-bb65-5f863338089e for instance 705d2d84-afe0-4760-bb57-66126418dacb on nch1-master-1 Sep 27 10:05:33.621217 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-cfc37d00-94c3-4321-85b0-8b45bdae7778 req-04950309-2fa7-45ea-aed6-544355daff74 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034722 Sep 27 10:05:33.622113 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1011/2234] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:33.638540 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d9c82a76-ef74-46e0-aef4-e2cfedae9ad9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 200 len: 1461 microversion: 2.1 time: 0.068044 Sep 27 10:05:33.639125 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1224/2235] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:33 2022] GET /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 1461 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:33.653550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dbfb27ea-2b76-43ef-a653-65a13460490d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:33.654285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dbfb27ea-2b76-43ef-a653-65a13460490d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:33.660665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfb27ea-2b76-43ef-a653-65a13460490d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.661164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfb27ea-2b76-43ef-a653-65a13460490d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.661631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfb27ea-2b76-43ef-a653-65a13460490d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.662329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfb27ea-2b76-43ef-a653-65a13460490d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.662779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfb27ea-2b76-43ef-a653-65a13460490d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.663228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfb27ea-2b76-43ef-a653-65a13460490d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.828088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dbfb27ea-2b76-43ef-a653-65a13460490d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:33.839084 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dbfb27ea-2b76-43ef-a653-65a13460490d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d" status: 200 len: 1480 microversion: 2.1 time: 0.189627 Sep 27 10:05:33.839954 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1012/2236] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:33 2022] GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d => generated 1480 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:33.910412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-985d58cb-1cf1-4cf7-9692-1e0de24ded59 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:33.910954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-985d58cb-1cf1-4cf7-9692-1e0de24ded59 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:33.917549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d58cb-1cf1-4cf7-9692-1e0de24ded59 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.918273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d58cb-1cf1-4cf7-9692-1e0de24ded59 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.918720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d58cb-1cf1-4cf7-9692-1e0de24ded59 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.919448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d58cb-1cf1-4cf7-9692-1e0de24ded59 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.919908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d58cb-1cf1-4cf7-9692-1e0de24ded59 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.920339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985d58cb-1cf1-4cf7-9692-1e0de24ded59 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.946102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-95bc2de7-3834-436d-92e6-17d93bcdfa90 req-e14a013e-71b4-4113-9485-e1113d722237 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b563b3f-326c-41ed-9185-2a6d112c871d", "name": "network-vif-unplugged", "status": "completed", "tag": "917af05f-ebc3-433e-a57d-d09e51b4d3c7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:33.958083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-95bc2de7-3834-436d-92e6-17d93bcdfa90 req-e14a013e-71b4-4113-9485-e1113d722237 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:33.958778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-95bc2de7-3834-436d-92e6-17d93bcdfa90 req-e14a013e-71b4-4113-9485-e1113d722237 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:33.959429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-95bc2de7-3834-436d-92e6-17d93bcdfa90 req-e14a013e-71b4-4113-9485-e1113d722237 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:33.985207 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-95bc2de7-3834-436d-92e6-17d93bcdfa90 req-e14a013e-71b4-4113-9485-e1113d722237 service nova] Creating event network-vif-unplugged:917af05f-ebc3-433e-a57d-d09e51b4d3c7 for instance 3b563b3f-326c-41ed-9185-2a6d112c871d on nch1-master-1 Sep 27 10:05:33.997480 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-95bc2de7-3834-436d-92e6-17d93bcdfa90 req-e14a013e-71b4-4113-9485-e1113d722237 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.055181 Sep 27 10:05:34.002352 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1013/2237] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:33 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:34.073329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-985d58cb-1cf1-4cf7-9692-1e0de24ded59 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:34.081662 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-985d58cb-1cf1-4cf7-9692-1e0de24ded59 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1511 microversion: 2.1 time: 0.176241 Sep 27 10:05:34.082912 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1225/2238] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:33 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1511 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:34.128168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:34.129034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:34.135487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:34.135972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:34.136411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:34.137145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:34.138163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:34.138163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:34.168906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed1cfa01-3a9b-4cda-ae76-d29887701612 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:34.169587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed1cfa01-3a9b-4cda-ae76-d29887701612 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:34.175806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed1cfa01-3a9b-4cda-ae76-d29887701612 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:34.176289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed1cfa01-3a9b-4cda-ae76-d29887701612 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:34.176767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed1cfa01-3a9b-4cda-ae76-d29887701612 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:34.177470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed1cfa01-3a9b-4cda-ae76-d29887701612 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:34.177920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed1cfa01-3a9b-4cda-ae76-d29887701612 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:34.178338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed1cfa01-3a9b-4cda-ae76-d29887701612 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:34.263041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ed1cfa01-3a9b-4cda-ae76-d29887701612 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6eb3ccfcd9c344e1a7ad55f8398ec87a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e5a565f65b3849a99b74454791397ca5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:34.271533 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed1cfa01-3a9b-4cda-ae76-d29887701612 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 200 len: 1452 microversion: 2.1 time: 0.106788 Sep 27 10:05:34.272117 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1226/2239] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:34 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 1452 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:34.454953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:34.469647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:34.473761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:34.474574 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-159fce02-5730-4f8e-acf1-958ce4b95d62 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1480 microversion: 2.25 time: 0.350479 Sep 27 10:05:34.476787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82299532-0912-4e1c-8edf-186ad3a34523 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:34.478652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-82299532-0912-4e1c-8edf-186ad3a34523 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:34.489535 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1014/2240] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:34 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1480 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:34.491045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82299532-0912-4e1c-8edf-186ad3a34523 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:34.492124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82299532-0912-4e1c-8edf-186ad3a34523 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:34.493839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82299532-0912-4e1c-8edf-186ad3a34523 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:34.495399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82299532-0912-4e1c-8edf-186ad3a34523 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:34.496398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82299532-0912-4e1c-8edf-186ad3a34523 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:34.497449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82299532-0912-4e1c-8edf-186ad3a34523 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:34.522387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9e459d36-c24e-4138-9296-e542eb8d0424 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:34.523930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9e459d36-c24e-4138-9296-e542eb8d0424 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:34.534910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e459d36-c24e-4138-9296-e542eb8d0424 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:34.535844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e459d36-c24e-4138-9296-e542eb8d0424 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:34.536767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e459d36-c24e-4138-9296-e542eb8d0424 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:34.538015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e459d36-c24e-4138-9296-e542eb8d0424 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:34.539008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e459d36-c24e-4138-9296-e542eb8d0424 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:34.539919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e459d36-c24e-4138-9296-e542eb8d0424 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:34.915686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-82299532-0912-4e1c-8edf-186ad3a34523 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:34.927348 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-82299532-0912-4e1c-8edf-186ad3a34523 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 0.457350 Sep 27 10:05:34.943231 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1227/2241] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:34 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:34.956493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-44856430-0813-4fe5-8c92-65517032e5b5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:34.957586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-44856430-0813-4fe5-8c92-65517032e5b5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:34.967517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44856430-0813-4fe5-8c92-65517032e5b5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:34.968182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44856430-0813-4fe5-8c92-65517032e5b5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:34.968799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44856430-0813-4fe5-8c92-65517032e5b5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:34.970449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44856430-0813-4fe5-8c92-65517032e5b5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:34.971065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44856430-0813-4fe5-8c92-65517032e5b5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:34.971698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44856430-0813-4fe5-8c92-65517032e5b5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.075542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9e459d36-c24e-4138-9296-e542eb8d0424 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:35.118689 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9e459d36-c24e-4138-9296-e542eb8d0424 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1473 microversion: 2.1 time: 0.625208 Sep 27 10:05:35.128695 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1015/2242] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:34 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1473 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:35.128898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-44856430-0813-4fe5-8c92-65517032e5b5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:35.129372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:35.129372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:35.138072 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-44856430-0813-4fe5-8c92-65517032e5b5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1273 microversion: 2.1 time: 0.193367 Sep 27 10:05:35.138930 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1228/2243] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:34 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1273 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:35.146628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.147304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.148139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.149575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.150110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.150658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.167921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4f3eaee1-3778-4495-b16e-1e525e7635d9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:35.168616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4f3eaee1-3778-4495-b16e-1e525e7635d9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:35.182116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f3eaee1-3778-4495-b16e-1e525e7635d9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.186570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f3eaee1-3778-4495-b16e-1e525e7635d9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.187412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f3eaee1-3778-4495-b16e-1e525e7635d9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.190076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f3eaee1-3778-4495-b16e-1e525e7635d9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.193444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f3eaee1-3778-4495-b16e-1e525e7635d9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.194722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f3eaee1-3778-4495-b16e-1e525e7635d9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.246031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.246651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.247139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.247616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lazy-loading 'fault' on Instance uuid 16cd150f-c67a-4a9e-9654-ba6bb9e1517a {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:05:35.377093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dce01d3212934161a38be853b4cd93ae', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ec0980b5824b425fbad9b8554cf5c42c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:35.387602 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a0a7dc2b-9bcb-45ad-aa80-4ec91cfa9331 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 200 len: 1300 microversion: 2.1 time: 0.266499 Sep 27 10:05:35.389026 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1016/2244] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:35 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 1300 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:35.396431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-042ba89a-1c07-41e4-aa72-7db3946ec48d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:35.397617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-042ba89a-1c07-41e4-aa72-7db3946ec48d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:35.409723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-042ba89a-1c07-41e4-aa72-7db3946ec48d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.410248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-042ba89a-1c07-41e4-aa72-7db3946ec48d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.410972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-042ba89a-1c07-41e4-aa72-7db3946ec48d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.411892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-042ba89a-1c07-41e4-aa72-7db3946ec48d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.412359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-042ba89a-1c07-41e4-aa72-7db3946ec48d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.412866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-042ba89a-1c07-41e4-aa72-7db3946ec48d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.635584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4f3eaee1-3778-4495-b16e-1e525e7635d9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:35.645234 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4f3eaee1-3778-4495-b16e-1e525e7635d9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d" status: 200 len: 1480 microversion: 2.1 time: 0.485671 Sep 27 10:05:35.645978 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1229/2245] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:35 2022] GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d => generated 1480 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:35.669323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:35.670366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 705d2d84-afe0-4760-bb57-66126418dacb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:35.677478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.678316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.678988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.680139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.680866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.681513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.700557 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Instance 705d2d84-afe0-4760-bb57-66126418dacb could not be found. Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb HTTP/1.1 Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:35.703390 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 705d2d84-afe0-4760-bb57-66126418dacb could not be found. Sep 27 10:05:35.704578 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '705d2d84-afe0-4760-bb57-66126418dacb'} Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 705d2d84-afe0-4760-bb57-66126418dacb could not be found. Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:35.705246 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 705d2d84-afe0-4760-bb57-66126418dacb could not be found. Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 705d2d84-afe0-4760-bb57-66126418dacb could not be found. Sep 27 10:05:35.706256 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.707167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Instance 705d2d84-afe0-4760-bb57-66126418dacb could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:35.708696 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c2e8b1fe-23a2-4fdb-8bc8-81f11f217c04 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb" status: 404 len: 111 microversion: 2.1 time: 0.043218 Sep 27 10:05:35.709081 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1230/2246] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:35 2022] GET /compute/v2.1/servers/705d2d84-afe0-4760-bb57-66126418dacb => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:35.718161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:35.718795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:35.726547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.727562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.728163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.729431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.730084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.730656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.810729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-042ba89a-1c07-41e4-aa72-7db3946ec48d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:35.830807 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-042ba89a-1c07-41e4-aa72-7db3946ec48d tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1511 microversion: 2.1 time: 0.439661 Sep 27 10:05:35.831798 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1017/2247] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:35 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1511 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:35.839983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7977753c-a0f4-4834-af72-9b9a4ac4ac36 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63641430>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:35.869107 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-7977753c-a0f4-4834-af72-9b9a4ac4ac36 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Image not found. Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7977753c-a0f4-4834-af72-9b9a4ac4ac36 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 HTTP/1.1 Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:05:35.871151 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: exception: Image not found. Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '4d7ede6e-2068-452e-baca-99dc4c31cf30'} Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:35.872302 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 4d7ede6e-2068-452e-baca-99dc4c31cf30 to delete Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.873085 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 4d7ede6e-2068-452e-baca-99dc4c31cf30 could not be found. Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:35.874180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7977753c-a0f4-4834-af72-9b9a4ac4ac36 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Image not found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:35.875164 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7977753c-a0f4-4834-af72-9b9a4ac4ac36 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30" status: 404 len: 62 microversion: 2.1 time: 0.039636 Sep 27 10:05:35.876674 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1018/2248] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:35 2022] DELETE /compute/v2.1/images/4d7ede6e-2068-452e-baca-99dc4c31cf30 => generated 62 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:05:35.881465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:35.881964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:35.890536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.891410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.892027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.893004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.893791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.894792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.919425 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 HTTP/1.1 Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:05:35.921122 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. Sep 27 10:05:35.922450 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '152a5758-d14b-4ca9-917e-f7017c6d99b2'} Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:35.923221 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. Sep 27 10:05:35.924514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.925789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:35.925789 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-867be0f6-555e-416e-bfe0-f31e4db30e1b tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 404 len: 111 microversion: 2.1 time: 0.045056 Sep 27 10:05:35.928081 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1019/2249] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:05:35 2022] DELETE /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:35.934271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:35.934904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 152a5758-d14b-4ca9-917e-f7017c6d99b2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:35.941655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.942144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.942734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.943580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.944185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.944793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.962612 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 HTTP/1.1 Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:05:35.964225 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. Sep 27 10:05:35.965748 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '152a5758-d14b-4ca9-917e-f7017c6d99b2'} Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:35.966507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. Sep 27 10:05:35.967957 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:35.969285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Instance 152a5758-d14b-4ca9-917e-f7017c6d99b2 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:35.969285 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a56121d-de04-409a-8468-ef5d3f55733d tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2" status: 404 len: 111 microversion: 2.1 time: 0.037973 Sep 27 10:05:35.969285 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1020/2250] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:35 2022] GET /compute/v2.1/servers/152a5758-d14b-4ca9-917e-f7017c6d99b2 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:35.974474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c2e36243-58b6-4742-9829-a7dc12aaa1ed tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:35.975300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c2e36243-58b6-4742-9829-a7dc12aaa1ed tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:35.983377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e36243-58b6-4742-9829-a7dc12aaa1ed tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.984105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e36243-58b6-4742-9829-a7dc12aaa1ed tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.984573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e36243-58b6-4742-9829-a7dc12aaa1ed tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:35.985310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e36243-58b6-4742-9829-a7dc12aaa1ed tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:35.985839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e36243-58b6-4742-9829-a7dc12aaa1ed tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:35.986274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e36243-58b6-4742-9829-a7dc12aaa1ed tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:36.126668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:36.142773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:36.144321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:36.145638 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9f04d54f-9f4a-4e92-b7bd-6b3a99aae971 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 200 len: 1480 microversion: 2.25 time: 0.435505 Sep 27 10:05:36.146578 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1231/2251] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:35 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 1480 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:36.152258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23c944b6-eda0-4728-813b-164e129a4d4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f62b8dd30>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:36.178287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-23c944b6-eda0-4728-813b-164e129a4d4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Image not found. Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23c944b6-eda0-4728-813b-164e129a4d4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 HTTP/1.1 Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:36.181967 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: exception: Image not found. Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '743efb1b-d45c-4063-8b10-3b39c9cf86b3'} Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:36.183853 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 743efb1b-d45c-4063-8b10-3b39c9cf86b3 to delete Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.185306 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 743efb1b-d45c-4063-8b10-3b39c9cf86b3 could not be found. Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:36.186428 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23c944b6-eda0-4728-813b-164e129a4d4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Image not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:36.187799 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-23c944b6-eda0-4728-813b-164e129a4d4e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3" status: 404 len: 62 microversion: 2.1 time: 0.036092 Sep 27 10:05:36.188564 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1232/2252] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:36 2022] DELETE /compute/v2.1/images/743efb1b-d45c-4063-8b10-3b39c9cf86b3 => generated 62 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:05:36.193998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8669bc7d-46f5-4c82-b7ec-6f23178d524c req-816e2591-632a-4cdf-94f2-3676abe4d642 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "35973885-926e-4fa3-80a3-0df40511ebd2", "tag": "8ac1d5d0-a175-4922-8f13-aacbb8c5d6b0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:36.207383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8669bc7d-46f5-4c82-b7ec-6f23178d524c req-816e2591-632a-4cdf-94f2-3676abe4d642 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:36.208047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8669bc7d-46f5-4c82-b7ec-6f23178d524c req-816e2591-632a-4cdf-94f2-3676abe4d642 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:36.218848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8669bc7d-46f5-4c82-b7ec-6f23178d524c req-816e2591-632a-4cdf-94f2-3676abe4d642 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:36.233060 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8669bc7d-46f5-4c82-b7ec-6f23178d524c req-816e2591-632a-4cdf-94f2-3676abe4d642 service nova] Creating event network-changed:8ac1d5d0-a175-4922-8f13-aacbb8c5d6b0 for instance 35973885-926e-4fa3-80a3-0df40511ebd2 on nch2-master-1 Sep 27 10:05:36.259295 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8669bc7d-46f5-4c82-b7ec-6f23178d524c req-816e2591-632a-4cdf-94f2-3676abe4d642 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069186 Sep 27 10:05:36.265056 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1233/2253] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:36.270764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-085cc92c-2612-4821-b2a8-1643beee6934 req-0f9bcbb6-a719-4fb8-a676-fdddfdbe7ba4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0aa6e63-d0ba-47c2-8f6b-572f168104ee", "name": "network-vif-deleted", "tag": "f97afb19-a003-40a4-a6f6-2158ddcc5448"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:36.298712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-085cc92c-2612-4821-b2a8-1643beee6934 req-0f9bcbb6-a719-4fb8-a676-fdddfdbe7ba4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:36.304170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-085cc92c-2612-4821-b2a8-1643beee6934 req-0f9bcbb6-a719-4fb8-a676-fdddfdbe7ba4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:36.304747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-085cc92c-2612-4821-b2a8-1643beee6934 req-0f9bcbb6-a719-4fb8-a676-fdddfdbe7ba4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:36.330523 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-085cc92c-2612-4821-b2a8-1643beee6934 req-0f9bcbb6-a719-4fb8-a676-fdddfdbe7ba4 service nova] Creating event network-vif-deleted:f97afb19-a003-40a4-a6f6-2158ddcc5448 for instance b0aa6e63-d0ba-47c2-8f6b-572f168104ee on nch2-master-1 Sep 27 10:05:36.347664 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-085cc92c-2612-4821-b2a8-1643beee6934 req-0f9bcbb6-a719-4fb8-a676-fdddfdbe7ba4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071210 Sep 27 10:05:36.357706 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1234/2254] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:36.366182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:36.369778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:36.380071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:36.380554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:36.381021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:36.381743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:36.382198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:36.382783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:36.433862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:36.434706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:36.435309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:36.435984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lazy-loading 'fault' on Instance uuid 3b563b3f-326c-41ed-9185-2a6d112c871d {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:05:36.527017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a10600a9277a436396fc70971dd27893', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '58c33b046dc7494c970f26eb5900b3f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:36.559546 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7dca3f8c-428c-4c5b-a047-a8d7d78e6c61 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 200 len: 1307 microversion: 2.1 time: 0.197243 Sep 27 10:05:36.561418 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1235/2255] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:36 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 1307 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:36.570431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1cbe036f-7343-47b8-a6f8-89c94b7cff49 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:36.571221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1cbe036f-7343-47b8-a6f8-89c94b7cff49 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:36.579664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cbe036f-7343-47b8-a6f8-89c94b7cff49 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:36.580407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cbe036f-7343-47b8-a6f8-89c94b7cff49 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:36.581036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cbe036f-7343-47b8-a6f8-89c94b7cff49 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:36.582193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cbe036f-7343-47b8-a6f8-89c94b7cff49 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:36.583562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cbe036f-7343-47b8-a6f8-89c94b7cff49 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:36.584509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cbe036f-7343-47b8-a6f8-89c94b7cff49 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:36.843436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1cbe036f-7343-47b8-a6f8-89c94b7cff49 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:36.861928 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1cbe036f-7343-47b8-a6f8-89c94b7cff49 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.298076 Sep 27 10:05:36.864800 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1236/2256] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:36 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:36.871815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:36.872742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:36.896585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:36.907184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:36.907825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:36.909610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:36.915632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:36.916285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:36.936125 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 HTTP/1.1 Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:36.938786 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. Sep 27 10:05:36.940080 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1'} Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:36.940868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. Sep 27 10:05:36.942130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:36.944623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:36.945789 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7f0af62d-2e81-44a9-80e3-6de8eaa91df6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 404 len: 111 microversion: 2.1 time: 0.079417 Sep 27 10:05:36.947077 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1237/2257] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:05:36 2022] DELETE /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:36.952528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:36.953411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] [instance: 16cd150f-c67a-4a9e-9654-ba6bb9e1517a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:36.959718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=3,nova_api:UPDATE=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:36.961586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:36.963725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:36.964945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:36.965898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:36.966381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:36.966791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.001339 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] HTTP exception thrown: Instance 16cd150f-c67a-4a9e-9654-ba6bb9e1517a could not be found. Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Request method failure captured: Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a HTTP/1.1 Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsl2ww1DCmrYlsvxa1ad0I1ZocI8nPKJGQ863hbj2ddYMyd4rN82uIGtKsNLlGkxpy9k3eCbNYNIEH6L36dtBDP6h0PGa_WIvNJEnlD4pYTb78k24vaHnHkvI4YxqwLlRHU0jrpIrk9bJl_QJobcfTL2k7WBj1Ty6gn63cG3JKTY8h8 Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: ec0980b5824b425fbad9b8554cf5c42c Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-LiveMigrationTest-1517090534 Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:37.007032 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ec0980b5824b425fbad9b8554cf5c42c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ec0980b5824b425fbad9b8554cf5c42c"}]}] Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-LiveMigrationTest-1517090534 Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: ec0980b5824b425fbad9b8554cf5c42c Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-LiveMigrationTest-1517090534 Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-LiveMigrationTest-1517090534-project Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: dce01d3212934161a38be853b4cd93ae Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-LiveMigrationTest-1517090534-project Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 16cd150f-c67a-4a9e-9654-ba6bb9e1517a could not be found. Sep 27 10:05:37.008495 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '16cd150f-c67a-4a9e-9654-ba6bb9e1517a'} Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 16cd150f-c67a-4a9e-9654-ba6bb9e1517a could not be found. Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:37.009355 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 16cd150f-c67a-4a9e-9654-ba6bb9e1517a could not be found. Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 16cd150f-c67a-4a9e-9654-ba6bb9e1517a could not be found. Sep 27 10:05:37.010571 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.011765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] Returning 404 to user: Instance 16cd150f-c67a-4a9e-9654-ba6bb9e1517a could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:37.011765 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-21fb55d3-c868-4dd2-b491-e5bb264e6fd8 tempest-LiveMigrationTest-1517090534 tempest-LiveMigrationTest-1517090534-project] 10.222.0.26 "GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a" status: 404 len: 111 microversion: 2.1 time: 0.061390 Sep 27 10:05:37.012920 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1238/2258] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:36 2022] GET /compute/v2.1/servers/16cd150f-c67a-4a9e-9654-ba6bb9e1517a => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:37.032297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa088d4f-2149-41bd-8249-7a6a865eba28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:37.033441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fa088d4f-2149-41bd-8249-7a6a865eba28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:37.044900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa088d4f-2149-41bd-8249-7a6a865eba28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.047769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa088d4f-2149-41bd-8249-7a6a865eba28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.048227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa088d4f-2149-41bd-8249-7a6a865eba28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.049032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa088d4f-2149-41bd-8249-7a6a865eba28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.049722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa088d4f-2149-41bd-8249-7a6a865eba28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.050513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa088d4f-2149-41bd-8249-7a6a865eba28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.124516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c2e36243-58b6-4742-9829-a7dc12aaa1ed tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:37.136742 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c2e36243-58b6-4742-9829-a7dc12aaa1ed tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 1.165401 Sep 27 10:05:37.137389 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1021/2259] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:35 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 1171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:37.142775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca77b4d1-247f-4d8f-b8db-32a157a1dae3 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:37.143735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ca77b4d1-247f-4d8f-b8db-32a157a1dae3 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:37.151099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca77b4d1-247f-4d8f-b8db-32a157a1dae3 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.151906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca77b4d1-247f-4d8f-b8db-32a157a1dae3 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.152678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca77b4d1-247f-4d8f-b8db-32a157a1dae3 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.153613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca77b4d1-247f-4d8f-b8db-32a157a1dae3 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.154396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca77b4d1-247f-4d8f-b8db-32a157a1dae3 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.154991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca77b4d1-247f-4d8f-b8db-32a157a1dae3 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.353119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ca77b4d1-247f-4d8f-b8db-32a157a1dae3 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:37.374698 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ca77b4d1-247f-4d8f-b8db-32a157a1dae3 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1469 microversion: 2.1 time: 0.236109 Sep 27 10:05:37.393689 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1022/2260] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:37 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1469 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:37.400345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:37.401173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:37.419286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.420621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.421315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.422254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.423217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.427544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fa088d4f-2149-41bd-8249-7a6a865eba28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:37.432914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.439410 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fa088d4f-2149-41bd-8249-7a6a865eba28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d" status: 200 len: 1467 microversion: 2.1 time: 0.425300 Sep 27 10:05:37.457166 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1239/2261] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:37 2022] GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d => generated 1467 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:37.471412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:37.472055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: b0aa6e63-d0ba-47c2-8f6b-572f168104ee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:37.484544 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. Sep 27 10:05:37.487349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.487900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.488330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 HTTP/1.1 Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:05:37.489218 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. Sep 27 10:05:37.490606 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1'} Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:37.491359 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. Sep 27 10:05:37.492807 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.494554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.494554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.494554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.496790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Instance 9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:37.498203 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7aef8c7d-52d6-4913-a72b-45edf59295c0 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1" status: 404 len: 111 microversion: 2.1 time: 0.102536 Sep 27 10:05:37.498783 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1023/2262] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:37 2022] GET /compute/v2.1/servers/9e5fc0fa-8f9f-42b3-92a0-c70da75ea8b1 => generated 111 bytes in 104 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:37.507754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6f1ea7d0-0e2c-4616-a6e1-b23b6ed8e407 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ListImageFiltersTestJSON-image-1901027848"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:37.510283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6f1ea7d0-0e2c-4616-a6e1-b23b6ed8e407 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:37.525094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f1ea7d0-0e2c-4616-a6e1-b23b6ed8e407 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.525983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f1ea7d0-0e2c-4616-a6e1-b23b6ed8e407 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.526624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f1ea7d0-0e2c-4616-a6e1-b23b6ed8e407 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.527875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f1ea7d0-0e2c-4616-a6e1-b23b6ed8e407 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.528921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f1ea7d0-0e2c-4616-a6e1-b23b6ed8e407 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.529514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f1ea7d0-0e2c-4616-a6e1-b23b6ed8e407 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.538222 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] HTTP exception thrown: Instance b0aa6e63-d0ba-47c2-8f6b-572f168104ee could not be found. Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Request method failure captured: Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee HTTP/1.1 Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsls2FdIOhvD4ZxN7OIKQmiUCELV6z27WbYJyCg5zc33cF3I2IG0F16O0kd5J_CobjZKV0lxKRF-IbyT-BYgGU09wQILwgGZLa230rmDdLmWzYS6zGfOygm-RvpfT-sJE6lwtE6xYYRfLbLdFYc5W5dGOjgOEW5aqauEbGKNWvjw5os Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.25 Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 2369f5ee467643948f618bbdee837521 Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:37.539497 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2369f5ee467643948f618bbdee837521"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2369f5ee467643948f618bbdee837521"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2369f5ee467643948f618bbdee837521"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2369f5ee467643948f618bbdee837521"}]}] Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 2369f5ee467643948f618bbdee837521 Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-LiveAutoBlockMigrationV225Test-1036261617-project Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 29d2f7b0e8274c769521d13930be9825 Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617-project Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance b0aa6e63-d0ba-47c2-8f6b-572f168104ee could not be found. Sep 27 10:05:37.540882 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'b0aa6e63-d0ba-47c2-8f6b-572f168104ee'} Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b0aa6e63-d0ba-47c2-8f6b-572f168104ee could not be found. Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:37.541648 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b0aa6e63-d0ba-47c2-8f6b-572f168104ee could not be found. Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b0aa6e63-d0ba-47c2-8f6b-572f168104ee could not be found. Sep 27 10:05:37.542733 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.543822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Returning 404 to user: Instance b0aa6e63-d0ba-47c2-8f6b-572f168104ee could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:37.543822 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1b588d7a-06f6-4898-b5c4-058d6ef9ae05 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee" status: 404 len: 111 microversion: 2.25 time: 0.082516 Sep 27 10:05:37.545320 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1240/2263] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:37 2022] GET /compute/v2.1/servers/b0aa6e63-d0ba-47c2-8f6b-572f168104ee => generated 111 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:05:37.560258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c841149-5983-4c72-bca6-d71cbbc063e6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '.version_select at 0x7f7f63839ee0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:37.599533 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-2c841149-5983-4c72-bca6-d71cbbc063e6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Image not found. Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c841149-5983-4c72-bca6-d71cbbc063e6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa HTTP/1.1 Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:37.605047 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: exception: Image not found. Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '21b1f887-d1bd-49ec-a937-3eed4a3c87aa'} Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:37.606424 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 21b1f887-d1bd-49ec-a937-3eed4a3c87aa to delete Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.607187 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 21b1f887-d1bd-49ec-a937-3eed4a3c87aa could not be found. Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:37.608345 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c841149-5983-4c72-bca6-d71cbbc063e6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Image not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c841149-5983-4c72-bca6-d71cbbc063e6 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa" status: 404 len: 62 microversion: 2.1 time: 0.059587 Sep 27 10:05:37.609417 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1241/2264] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:37 2022] DELETE /compute/v2.1/images/21b1f887-d1bd-49ec-a937-3eed4a3c87aa => generated 62 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:05:37.628304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:37.631655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] [instance: 3b563b3f-326c-41ed-9185-2a6d112c871d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:37.662615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.664609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.666692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.668704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.670449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.670967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.740847 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] HTTP exception thrown: Instance 3b563b3f-326c-41ed-9185-2a6d112c871d could not be found. Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Request method failure captured: Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d HTTP/1.1 Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsqS8wTqwJCKAaZfBPF_2GNu90piB3rTxDlvrWvyfpVn6Ph3kb3VLfMM5uXumXMZCvvsRDaPbrg7UKxhCmUYWyv6FeFeUaguQxNpFTL2dK6jI7chvqj_DBGJ05vqT2g2d8KV_lycaLPzFP-wAXgdsdjSmN55CqgqVC-YkSijA2Ko5UA Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 58c33b046dc7494c970f26eb5900b3f3 Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesOneServerTestJSON-353655151 Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:37.744705 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/58c33b046dc7494c970f26eb5900b3f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/58c33b046dc7494c970f26eb5900b3f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/58c33b046dc7494c970f26eb5900b3f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_58c33b046dc7494c970f26eb5900b3f3"}]}] Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesOneServerTestJSON-353655151 Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 58c33b046dc7494c970f26eb5900b3f3 Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesOneServerTestJSON-353655151 Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesOneServerTestJSON-353655151-project Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a10600a9277a436396fc70971dd27893 Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesOneServerTestJSON-353655151-project Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 3b563b3f-326c-41ed-9185-2a6d112c871d could not be found. Sep 27 10:05:37.746115 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3b563b3f-326c-41ed-9185-2a6d112c871d'} Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3b563b3f-326c-41ed-9185-2a6d112c871d could not be found. Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:37.746864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3b563b3f-326c-41ed-9185-2a6d112c871d could not be found. Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3b563b3f-326c-41ed-9185-2a6d112c871d could not be found. Sep 27 10:05:37.748068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.749561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] Returning 404 to user: Instance 3b563b3f-326c-41ed-9185-2a6d112c871d could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:37.749561 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4d3d34d7-c757-4500-bfbd-13ff6eca54a9 tempest-ImagesOneServerTestJSON-353655151 tempest-ImagesOneServerTestJSON-353655151-project] 10.222.0.26 "GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d" status: 404 len: 111 microversion: 2.1 time: 0.131149 Sep 27 10:05:37.749561 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1242/2265] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:37 2022] GET /compute/v2.1/servers/3b563b3f-326c-41ed-9185-2a6d112c871d => generated 111 bytes in 141 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:37.763683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:37.764196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:37.797209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.797721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.798166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.801151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.801636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.802078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.849501 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 HTTP/1.1 Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:37.850313 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. Sep 27 10:05:37.851618 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '1b526682-dfde-459f-bc61-2df38e0ec4b7'} Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:37.852309 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. Sep 27 10:05:37.853521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.855215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:37.856289 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a5530e07-ce30-44bb-9159-4380468ad71e tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 404 len: 111 microversion: 2.1 time: 0.103737 Sep 27 10:05:37.857700 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1243/2266] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:05:37 2022] DELETE /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 111 bytes in 106 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:37.867550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:37.870229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] [instance: 1b526682-dfde-459f-bc61-2df38e0ec4b7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:37.893928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.894635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.895763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.899465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:37.904093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:37.905028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:37.980385 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] HTTP exception thrown: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Request method failure captured: Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 HTTP/1.1 Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsnepiH6Gkvp1hQ5tv7jASF8zWyT3QZPbxQ797jkQdNGj1ccrAoq4Qicw3LaansAy_bFzCZWJO6jI801y6eKV-awPPOf8fZv8gxM2Cl0KTzwiJLyZ9Yk04WqxHwxx-PcGDrn0FUoyUpxXU2Yu93bPP2Z9Ca_hwps21-PjhTJBS7Liqk Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:37.985792 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e5a565f65b3849a99b74454791397ca5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e5a565f65b3849a99b74454791397ca5"}]}] Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e5a565f65b3849a99b74454791397ca5 Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesTestJSON-105456677 Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6eb3ccfcd9c344e1a7ad55f8398ec87a Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesTestJSON-105456677-project Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. Sep 27 10:05:37.987245 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '1b526682-dfde-459f-bc61-2df38e0ec4b7'} Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:37.987974 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. Sep 27 10:05:37.989269 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:37.990449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] Returning 404 to user: Instance 1b526682-dfde-459f-bc61-2df38e0ec4b7 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:37.992474 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86f35dfb-690e-4712-9192-ffc4485f2998 tempest-ImagesTestJSON-105456677 tempest-ImagesTestJSON-105456677-project] 10.222.0.26 "GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7" status: 404 len: 111 microversion: 2.1 time: 0.128834 Sep 27 10:05:37.995191 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1244/2267] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:37 2022] GET /compute/v2.1/servers/1b526682-dfde-459f-bc61-2df38e0ec4b7 => generated 111 bytes in 134 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:38.005459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f5cb15b-6f92-431f-862c-a755868a0819 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:38.006682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f5cb15b-6f92-431f-862c-a755868a0819 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:38.030338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5cb15b-6f92-431f-862c-a755868a0819 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:38.031723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5cb15b-6f92-431f-862c-a755868a0819 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:38.033235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5cb15b-6f92-431f-862c-a755868a0819 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:38.034636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5cb15b-6f92-431f-862c-a755868a0819 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:38.034812 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6f1ea7d0-0e2c-4616-a6e1-b23b6ed8e407 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "POST /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad/action" status: 202 len: 0 microversion: 2.1 time: 0.533510 Sep 27 10:05:38.035904 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1024/2268] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:05:37 2022] POST /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad/action => generated 0 bytes in 537 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 10:05:38.036440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5cb15b-6f92-431f-862c-a755868a0819 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:38.039689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5cb15b-6f92-431f-862c-a755868a0819 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:38.048092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7139c1a1-04cb-4e69-a268-f4e9bc8ad7e6 req-06562e7d-1252-499b-97dd-dd27dde186ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "961407e4-8e66-4f1c-88bd-6d2b4795c11d", "name": "network-vif-unplugged", "status": "completed", "tag": "52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:38.050357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:38.072878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7139c1a1-04cb-4e69-a268-f4e9bc8ad7e6 req-06562e7d-1252-499b-97dd-dd27dde186ce service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:38.073368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7139c1a1-04cb-4e69-a268-f4e9bc8ad7e6 req-06562e7d-1252-499b-97dd-dd27dde186ce service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:38.073808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7139c1a1-04cb-4e69-a268-f4e9bc8ad7e6 req-06562e7d-1252-499b-97dd-dd27dde186ce service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:38.081377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:38.110787 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7139c1a1-04cb-4e69-a268-f4e9bc8ad7e6 req-06562e7d-1252-499b-97dd-dd27dde186ce service nova] Creating event network-vif-unplugged:52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f for instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d on nch2-master-1 Sep 27 10:05:38.122993 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7139c1a1-04cb-4e69-a268-f4e9bc8ad7e6 req-06562e7d-1252-499b-97dd-dd27dde186ce service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.084044 Sep 27 10:05:38.123919 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1025/2269] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:38 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:38.125703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:38.134132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-56e7088c-78b0-49a7-b4a7-dc8a5c0e854f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f635f2e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:38.188513 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-56e7088c-78b0-49a7-b4a7-dc8a5c0e854f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.063354 Sep 27 10:05:38.189693 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1026/2270] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:38 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:38.194552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bce97b0f-05ac-45c9-b5e4-2b96722fa244 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:38.195390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bce97b0f-05ac-45c9-b5e4-2b96722fa244 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:38.201930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce97b0f-05ac-45c9-b5e4-2b96722fa244 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:38.202429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce97b0f-05ac-45c9-b5e4-2b96722fa244 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:38.202869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce97b0f-05ac-45c9-b5e4-2b96722fa244 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:38.203763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce97b0f-05ac-45c9-b5e4-2b96722fa244 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:38.204372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce97b0f-05ac-45c9-b5e4-2b96722fa244 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:38.204968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce97b0f-05ac-45c9-b5e4-2b96722fa244 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:38.223919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:38.286622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:38.326585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6f5cb15b-6f92-431f-862c-a755868a0819 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:38.340136 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f5cb15b-6f92-431f-862c-a755868a0819 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.339362 Sep 27 10:05:38.341495 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1245/2271] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:37 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:38.375427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3857ca80-774c-4e7e-8068-b1f53fd96dfe req-4f9afa72-6c31-42c4-a1d6-cc06a7808a7d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b563b3f-326c-41ed-9185-2a6d112c871d", "name": "network-vif-deleted", "tag": "917af05f-ebc3-433e-a57d-d09e51b4d3c7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:38.392892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3857ca80-774c-4e7e-8068-b1f53fd96dfe req-4f9afa72-6c31-42c4-a1d6-cc06a7808a7d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:38.393579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3857ca80-774c-4e7e-8068-b1f53fd96dfe req-4f9afa72-6c31-42c4-a1d6-cc06a7808a7d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:38.395273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3857ca80-774c-4e7e-8068-b1f53fd96dfe req-4f9afa72-6c31-42c4-a1d6-cc06a7808a7d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:38.411843 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3857ca80-774c-4e7e-8068-b1f53fd96dfe req-4f9afa72-6c31-42c4-a1d6-cc06a7808a7d service nova] Creating event network-vif-deleted:917af05f-ebc3-433e-a57d-d09e51b4d3c7 for instance 3b563b3f-326c-41ed-9185-2a6d112c871d on nch1-master-1 Sep 27 10:05:38.418972 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3857ca80-774c-4e7e-8068-b1f53fd96dfe req-4f9afa72-6c31-42c4-a1d6-cc06a7808a7d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049398 Sep 27 10:05:38.420238 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1246/2272] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:38.426338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:38.427351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:38.437766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:38.439237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:38.439975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:38.441560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:38.442249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:38.442893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:38.490281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:38.490834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:38.491281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:38.492475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lazy-loading 'fault' on Instance uuid 219e4ceb-d811-4e6f-913f-c51410efcee6 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:05:38.517193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bce97b0f-05ac-45c9-b5e4-2b96722fa244 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:38.529623 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bce97b0f-05ac-45c9-b5e4-2b96722fa244 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 0.338532 Sep 27 10:05:38.530195 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1027/2273] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:38 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:38.579099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:38.588643 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69d983d5-4539-44f9-b379-f60b7776e41b tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1307 microversion: 2.1 time: 0.166816 Sep 27 10:05:38.590485 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1247/2274] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:38 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1307 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:38.719543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-8e65a468-04ef-46cc-9cb6-13b7bc217ded req-4baff1c8-2f48-4f29-ba38-e2b6ffe63d94 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "219e4ceb-d811-4e6f-913f-c51410efcee6", "name": "network-vif-deleted", "tag": "b2aabccc-ddd4-4740-aef9-c80e07db2d17"}, {"name": "network-changed", "server_uuid": "e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191", "tag": "16a06bd9-639b-4f11-8e0c-4ca53b9b60d5"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:38.734178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8e65a468-04ef-46cc-9cb6-13b7bc217ded req-4baff1c8-2f48-4f29-ba38-e2b6ffe63d94 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:38.735069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8e65a468-04ef-46cc-9cb6-13b7bc217ded req-4baff1c8-2f48-4f29-ba38-e2b6ffe63d94 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:38.735612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8e65a468-04ef-46cc-9cb6-13b7bc217ded req-4baff1c8-2f48-4f29-ba38-e2b6ffe63d94 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:38.794595 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8e65a468-04ef-46cc-9cb6-13b7bc217ded req-4baff1c8-2f48-4f29-ba38-e2b6ffe63d94 service nova] Creating event network-vif-deleted:b2aabccc-ddd4-4740-aef9-c80e07db2d17 for instance 219e4ceb-d811-4e6f-913f-c51410efcee6 on nch2-master-1 Sep 27 10:05:38.796149 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8e65a468-04ef-46cc-9cb6-13b7bc217ded req-4baff1c8-2f48-4f29-ba38-e2b6ffe63d94 service nova] Creating event network-changed:16a06bd9-639b-4f11-8e0c-4ca53b9b60d5 for instance e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 on nch2-master-1 Sep 27 10:05:38.814883 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-8e65a468-04ef-46cc-9cb6-13b7bc217ded req-4baff1c8-2f48-4f29-ba38-e2b6ffe63d94 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.098910 Sep 27 10:05:38.818171 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1028/2275] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:38 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:39.205044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-daf5dbda-0afc-44f6-84a8-4d114ee3f2ac tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62b8de50>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:39.274810 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-daf5dbda-0afc-44f6-84a8-4d114ee3f2ac tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.073718 Sep 27 10:05:39.281386 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1248/2276] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:39 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:39.372623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5088a216-1613-4b14-b91a-44f20561b566 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:39.373712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5088a216-1613-4b14-b91a-44f20561b566 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:39.386339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5088a216-1613-4b14-b91a-44f20561b566 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:39.386867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5088a216-1613-4b14-b91a-44f20561b566 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:39.387325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5088a216-1613-4b14-b91a-44f20561b566 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:39.388075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5088a216-1613-4b14-b91a-44f20561b566 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:39.388543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5088a216-1613-4b14-b91a-44f20561b566 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:39.389018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5088a216-1613-4b14-b91a-44f20561b566 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:39.560350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-839b49d9-2e5d-4317-b525-01c03e1b9937 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:39.562919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-839b49d9-2e5d-4317-b525-01c03e1b9937 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:39.575850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839b49d9-2e5d-4317-b525-01c03e1b9937 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:39.577576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839b49d9-2e5d-4317-b525-01c03e1b9937 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:39.578093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839b49d9-2e5d-4317-b525-01c03e1b9937 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:39.578920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839b49d9-2e5d-4317-b525-01c03e1b9937 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:39.581187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839b49d9-2e5d-4317-b525-01c03e1b9937 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:39.581712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839b49d9-2e5d-4317-b525-01c03e1b9937 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:39.853070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-839b49d9-2e5d-4317-b525-01c03e1b9937 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:39.863507 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-839b49d9-2e5d-4317-b525-01c03e1b9937 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 0.313233 Sep 27 10:05:39.864773 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1249/2277] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:39 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:39.871726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:39.872679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:39.879875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:39.880816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:39.881678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:39.882608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:39.883351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:39.883979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:39.912213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5088a216-1613-4b14-b91a-44f20561b566 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:39.928787 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5088a216-1613-4b14-b91a-44f20561b566 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.557073 Sep 27 10:05:39.929996 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1029/2278] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:39 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:39.937648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:39.938406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:39.939020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:39.943633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a74b687-ff59-45f6-8ccc-6e33b7f61d34 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:39.944424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a74b687-ff59-45f6-8ccc-6e33b7f61d34 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:39.948361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lazy-loading 'fault' on Instance uuid 219e4ceb-d811-4e6f-913f-c51410efcee6 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:05:39.952971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a74b687-ff59-45f6-8ccc-6e33b7f61d34 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:39.953483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a74b687-ff59-45f6-8ccc-6e33b7f61d34 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:39.954080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a74b687-ff59-45f6-8ccc-6e33b7f61d34 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:39.954954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a74b687-ff59-45f6-8ccc-6e33b7f61d34 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:39.955552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a74b687-ff59-45f6-8ccc-6e33b7f61d34 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:39.956120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a74b687-ff59-45f6-8ccc-6e33b7f61d34 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:40.000109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:40.007089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a74b687-ff59-45f6-8ccc-6e33b7f61d34 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:05:40.012363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '297a11c331fb473291750ced9c843a6e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3f33c209ffd1480e94175241ccc90414', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:40.020887 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-918a4780-2a0d-4b3a-bb8e-7bb513c04d8a tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 200 len: 1307 microversion: 2.1 time: 0.154718 Sep 27 10:05:40.029058 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1250/2279] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:39 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 1307 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:40.097070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:40.125700 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a74b687-ff59-45f6-8ccc-6e33b7f61d34 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "DELETE /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 204 len: 0 microversion: 2.25 time: 0.193956 Sep 27 10:05:40.128137 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1030/2280] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:05:39 2022] DELETE /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:05:40.142327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:40.143826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:40.159639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:40.160138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:40.161050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:40.162152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:40.164266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:40.164967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:40.207605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-974c6370-f69e-491b-bf58-269315f0e3c5 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Action: 'create', calling method: .version_select at 0x7f7f62b93e50>, body: {"keypair": {"name": "key_/.\\@:", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJdTnZaJoqhEy0N1KR2idg7kxE+VqUbezuGwIHuo77D4RKg/lQ4hoInUNCBaoJsOXoBi0k2LHmLFeIXvw0nTiSf+3NiZUGEWYRYbGKQbt/xoVRtrrLwffRwsjrJYpF/rpA=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-974c6370-f69e-491b-bf58-269315f0e3c5 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Request method failure captured: Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 257 Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrz26zv0qwrqHOGoFZxbH9Ep3vN62LYjjKCLxLkQqXJtRRp4k7CEWGqYmo8g5_equ4762Edl1kclX0iPkt3X5mjzI2MmgKOaaB0qoKoVFuO-y91hYddJHGTRx-O3pl82saYOZjiFPTAxYMgQmG8u56CBDHkBYAh15Nc5QRqdm3lZWg Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:40.215460 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:40.216801 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_daeb3231e3b54f95b8fe082ef62c330e"}]}] Sep 27 10:05:40.216801 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.216801 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.216801 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.216801 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.216801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:40.216801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:40.216801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.216801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.216801 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:05:40.216801 ncdv-master-1 devstack@n-api.service[145968]: {"keypair": {"name": "key_/.\\@:", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJdTnZaJoqhEy0N1KR2idg7kxE+VqUbezuGwIHuo77D4RKg/lQ4hoInUNCBaoJsOXoBi0k2LHmLFeIXvw0nTiSf+3NiZUGEWYRYbGKQbt/xoVRtrrLwffRwsjrJYpF/rpA=="}} Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: exception: An invalid 'name' value was provided. The name must be: Only expected characters: [_- 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'keypair': {'name': 'key_/.\\@:', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJdTnZaJoqhEy0N1KR2idg7kxE+VqUbezuGwIHuo77D4RKg/lQ4hoInUNCBaoJsOXoBi0k2LHmLFeIXvw0nTiSf+3NiZUGEWYRYbGKQbt/xoVRtrrLwffRwsjrJYpF/rpA=='}}} Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 263, in check Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = func(instance) Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 161, in _validate_keypair_name_20 Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return _validate_keypair_name(keypair_name, safe_chars) Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 175, in _validate_keypair_name Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InvalidName(reason=reason) Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InvalidName: An invalid 'name' value was provided. The name must be: Only expected characters: [_- 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.217451 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'key_/.\\@:' is not a 'keypair_name_20' Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'format' in schema['properties']['keypair']['properties']['name']['allOf'][1]: Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'format': 'keypair_name_20', Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'maxLength': 255, Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'minLength': 1, Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'type': 'string'} Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['keypair']['name']: Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'key_/.\\@:' Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:40.218450 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: An invalid 'name' value was provided. The name must be: Only expected characters: [_- 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:05:40.222332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:05:40.224504 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: An invalid 'name' value was provided. The name must be: Only expected characters: [_- 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] Sep 27 10:05:40.224504 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.225916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-974c6370-f69e-491b-bf58-269315f0e3c5 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Returning 400 to user: An invalid 'name' value was provided. The name must be: Only expected characters: [_- 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:40.227049 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-974c6370-f69e-491b-bf58-269315f0e3c5 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 400 len: 193 microversion: 2.1 time: 0.196718 Sep 27 10:05:40.232382 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1251/2281] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:05:40 2022] POST /compute/v2.1/os-keypairs => generated 193 bytes in 202 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:05:40.239269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99ebdf26-6c34-440c-9c5d-2cc46cbc81a2 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Action: 'create', calling method: .version_select at 0x7f7f6c2bbca0>, body: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-2124819525", "public_key": "ssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA openstack@ubuntu"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:40.264104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-99ebdf26-6c34-440c-9c5d-2cc46cbc81a2 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Getting quotas for project daeb3231e3b54f95b8fe082ef62c330e. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:40.271059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-99ebdf26-6c34-440c-9c5d-2cc46cbc81a2 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Getting quotas for user a27051677d2e492eb0ebe1287c69a9d5 and project daeb3231e3b54f95b8fe082ef62c330e. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:40.294907 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-99ebdf26-6c34-440c-9c5d-2cc46cbc81a2 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] HTTP exception thrown: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99ebdf26-6c34-440c-9c5d-2cc46cbc81a2 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Request method failure captured: Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 2167 Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrz26zv0qwrqHOGoFZxbH9Ep3vN62LYjjKCLxLkQqXJtRRp4k7CEWGqYmo8g5_equ4762Edl1kclX0iPkt3X5mjzI2MmgKOaaB0qoKoVFuO-y91hYddJHGTRx-O3pl82saYOZjiFPTAxYMgQmG8u56CBDHkBYAh15Nc5QRqdm3lZWg Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_daeb3231e3b54f95b8fe082ef62c330e"}]}] Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-2124819525", "public_key": "ssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA openstack@ubuntu"}} Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:05:40.317839 ncdv-master-1 devstack@n-api.service[145968]: exception: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'keypair': {'name': 'tempest-KeyPairsNegativeTestJSON-keypair-2124819525', 'public_key': 'ssh-rsa AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA openstack@ubuntu'}}} Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 71, in generate_fingerprint Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi serialization.load_ssh_public_key( Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 666, in load_ssh_public_key Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise ValueError("Invalid key format") Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ValueError: Invalid key format Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 117, in _create Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6774, in import_key_pair Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi fingerprint = self._generate_fingerprint(public_key, key_type) Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6847, in _generate_fingerprint Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return crypto.generate_fingerprint(public_key) Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 77, in generate_fingerprint Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InvalidKeypair( Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InvalidKeypair: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:40.319773 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 103, in create Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._create(req, body) Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 130, in _create Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=exc.format_message()) Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.322759 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.325200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99ebdf26-6c34-440c-9c5d-2cc46cbc81a2 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Returning 400 to user: Keypair data is invalid: failed to generate fingerprint {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:40.325200 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-99ebdf26-6c34-440c-9c5d-2cc46cbc81a2 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 400 len: 99 microversion: 2.1 time: 0.091134 Sep 27 10:05:40.326963 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1252/2282] 10.222.0.26 () {60 vars in 1160 bytes} [Tue Sep 27 10:05:40 2022] POST /compute/v2.1/os-keypairs => generated 99 bytes in 93 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:40.332267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b38815b5-08cc-43bd-9a5d-5c55c1cf5915 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f637150d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:40.383440 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b38815b5-08cc-43bd-9a5d-5c55c1cf5915 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.055081 Sep 27 10:05:40.392695 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1253/2283] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:40 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:40.407263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef6cef5d-b4bd-46f8-a877-d384235222e3 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Action: 'create', calling method: .version_select at 0x7f7f62b96700>, body: {"keypair": {"name": "tempest-keypair-910348920", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEmScQ7P7qC9Ha7m8iKRo8pFNrsVNWlNwvAAjKdJx0TBN9E1SfeLKtn42nv19eWLctOr0j2ghKv0Au9XkZNYK5+ccdnH7pYiRsoUV5knzbB+Z6+gPgk6HRM0vdC3kHlPxA=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:40.428540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-ef6cef5d-b4bd-46f8-a877-d384235222e3 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Getting quotas for project daeb3231e3b54f95b8fe082ef62c330e. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:40.434375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-ef6cef5d-b4bd-46f8-a877-d384235222e3 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Getting quotas for user a27051677d2e492eb0ebe1287c69a9d5 and project daeb3231e3b54f95b8fe082ef62c330e. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:40.473466 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef6cef5d-b4bd-46f8-a877-d384235222e3 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.087440 Sep 27 10:05:40.474121 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1254/2284] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:05:40 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:40.493630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a402b23c-5c56-46ff-aa1e-27fa076d177d tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Action: 'create', calling method: .version_select at 0x7f7f62b960d0>, body: {"keypair": {"name": "tempest-keypair-910348920", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHG+ZGpdvnj7Zc5qRdppegWAeGfrJvxJPqvuWZfV+aotkiiNogz8fbl2Aqf1op5JR8nJgM5T/t3VtAywKBqz02E4hcxmwdFXK5I5ze+PTKNWR/Cl2f4vZhiPQ8eyND0zDg=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:40.528044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-a402b23c-5c56-46ff-aa1e-27fa076d177d tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Getting quotas for project daeb3231e3b54f95b8fe082ef62c330e. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:40.537072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-a402b23c-5c56-46ff-aa1e-27fa076d177d tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Getting quotas for user a27051677d2e492eb0ebe1287c69a9d5 and project daeb3231e3b54f95b8fe082ef62c330e. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:40.572225 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a402b23c-5c56-46ff-aa1e-27fa076d177d tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] HTTP exception thrown: Key pair 'tempest-keypair-910348920' already exists. Sep 27 10:05:40.614445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a402b23c-5c56-46ff-aa1e-27fa076d177d tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Request method failure captured: Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 272 Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrz26zv0qwrqHOGoFZxbH9Ep3vN62LYjjKCLxLkQqXJtRRp4k7CEWGqYmo8g5_equ4762Edl1kclX0iPkt3X5mjzI2MmgKOaaB0qoKoVFuO-y91hYddJHGTRx-O3pl82saYOZjiFPTAxYMgQmG8u56CBDHkBYAh15Nc5QRqdm3lZWg Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_daeb3231e3b54f95b8fe082ef62c330e"}]}] Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: {"keypair": {"name": "tempest-keypair-910348920", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHG+ZGpdvnj7Zc5qRdppegWAeGfrJvxJPqvuWZfV+aotkiiNogz8fbl2Aqf1op5JR8nJgM5T/t3VtAywKBqz02E4hcxmwdFXK5I5ze+PTKNWR/Cl2f4vZhiPQ8eyND0zDg=="}} Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: exception: Key pair 'tempest-keypair-910348920' already exists. Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'keypair': {'name': 'tempest-keypair-910348920', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHG+ZGpdvnj7Zc5qRdppegWAeGfrJvxJPqvuWZfV+aotkiiNogz8fbl2Aqf1op5JR8nJgM5T/t3VtAywKBqz02E4hcxmwdFXK5I5ze+PTKNWR/Cl2f4vZhiPQ8eyND0zDg=='}}} Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = self._query(query) Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi conn.query(q) Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result.read() Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi pymysql.err.IntegrityError: (1062, "Duplicate entry 'a27051677d2e492eb0ebe1287c69a9d5-tempest-keypair-910348920' for key 'key_pairs.uniq_key_pairs0user_id0name'") Sep 27 10:05:40.620121 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 69, in _create_in_db Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi kp.save(context.session) Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/models.py", line 38, in save Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi session.flush() Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._flush(objects) Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi transaction.rollback(_capture_exception=True) Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi compat.raise_( Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi flush_context.execute() Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi rec.execute(self) Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi util.preloaded.orm_persistence.save_obj( Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _emit_insert_statements( Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = connection._execute_20( Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return meth(self, args_10style, kwargs_10style, execution_options) Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return connection._execute_clauseelement( Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = self._execute_context( Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_dbapi_exception( Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.dialect.do_execute( Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi cursor.execute(statement, parameters) Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = self._query(query) Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi conn.query(q) Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result.read() Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi first_packet = self.connection._read_packet() Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi packet.raise_for_error() Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi err.raise_mysql_exception(self._data) Sep 27 10:05:40.624204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise errorclass(errno, errval) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'a27051677d2e492eb0ebe1287c69a9d5-tempest-keypair-910348920' for key 'key_pairs.uniq_key_pairs0user_id0name'") Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [SQL: INSERT INTO key_pairs (created_at, updated_at, name, user_id, fingerprint, public_key, type) VALUES (%(created_at)s, %(updated_at)s, %(name)s, %(user_id)s, %(fingerprint)s, %(public_key)s, %(type)s)] Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [parameters: {'created_at': datetime.datetime(2022, 9, 27, 10, 5, 40, 564469), 'updated_at': None, 'name': 'tempest-keypair-910348920', 'user_id': 'a27051677d2e492eb0ebe1287c69a9d5', 'fingerprint': '6a:f0:dd:91:03:45:e8:bf:7b:00:a4:4b:84:c1:44:2a', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHG+ZGpdvnj7Zc5qRdppegWAeGfrJvxJPqvuWZfV+aotkiiNogz8fbl2Aqf1op5JR8nJgM5T/t3VtAywKBqz02E4hcxmwdFXK5I5ze+PTKNWR/Cl2f4vZhiPQ8eyND0zDg==', 'type': 'ssh'}] Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 117, in _create Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6777, in import_key_pair Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi keypair.create() Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 226, in wrapper Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(self, *args, **kwargs) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 158, in create Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._create() Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 162, in _create Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_keypair = self._create_in_db(self._context, updates) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 134, in _create_in_db Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return _create_in_db(context, values) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 71, in _create_in_db Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.KeyPairExists(key_name=values['name']) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.KeyPairExists: Key pair 'tempest-keypair-910348920' already exists. Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 103, in create Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._create(req, body) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 132, in _create Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=exc.format_message()) Sep 27 10:05:40.634270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Key pair 'tempest-keypair-910348920' already exists. Sep 27 10:05:40.641537 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.641537 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.641537 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.641537 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.641537 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.641537 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.641537 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:40.641537 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:40.641537 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Key pair 'tempest-keypair-910348920' already exists. Sep 27 10:05:40.641537 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.642782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:40.642782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:40.642782 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-91803260-ac60-421c-ae36-63218ea5a5f6 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1728 microversion: 2.25 time: 0.493379 Sep 27 10:05:40.642782 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1031/2285] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:40 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1728 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:40.643392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a402b23c-5c56-46ff-aa1e-27fa076d177d tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Returning 409 to user: Key pair 'tempest-keypair-910348920' already exists. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:40.643392 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a402b23c-5c56-46ff-aa1e-27fa076d177d tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 409 len: 104 microversion: 2.1 time: 0.148716 Sep 27 10:05:40.643392 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1255/2286] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:05:40 2022] POST /compute/v2.1/os-keypairs => generated 104 bytes in 151 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:05:40.646805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-69066ec1-1075-4c34-9dde-8a656cc69bab tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Calling method '.version_select at 0x7f7f63760430>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:40.682664 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-69066ec1-1075-4c34-9dde-8a656cc69bab tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-910348920" status: 202 len: 0 microversion: 2.1 time: 0.039837 Sep 27 10:05:40.683162 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1032/2287] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:05:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-910348920 => generated 0 bytes in 41 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:05:40.697351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ab3ad22f-8f2e-44a0-bebf-f49fc522024d tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Action: 'create', calling method: .version_select at 0x7f7f62be3f70>, body: {"keypair": {"name": "", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE4Z20odcWPHVmMYC1X22ESVwuSs0/4RCwGquAPJH0GMrIRsLNwEqCg2V2Gi29io4pSZrlU20MIzdbLGcLYHOM8UX1a3qnOz3jqOzPOlF2fesDzDfWoTL+85aXT1mGaU/A=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ab3ad22f-8f2e-44a0-bebf-f49fc522024d tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Request method failure captured: Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 247 Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrz26zv0qwrqHOGoFZxbH9Ep3vN62LYjjKCLxLkQqXJtRRp4k7CEWGqYmo8g5_equ4762Edl1kclX0iPkt3X5mjzI2MmgKOaaB0qoKoVFuO-y91hYddJHGTRx-O3pl82saYOZjiFPTAxYMgQmG8u56CBDHkBYAh15Nc5QRqdm3lZWg Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_daeb3231e3b54f95b8fe082ef62c330e"}]}] Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: {"keypair": {"name": "", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE4Z20odcWPHVmMYC1X22ESVwuSs0/4RCwGquAPJH0GMrIRsLNwEqCg2V2Gi29io4pSZrlU20MIzdbLGcLYHOM8UX1a3qnOz3jqOzPOlF2fesDzDfWoTL+85aXT1mGaU/A=="}} Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute name. Value: . '' is too short Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'keypair': {'name': '', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE4Z20odcWPHVmMYC1X22ESVwuSs0/4RCwGquAPJH0GMrIRsLNwEqCg2V2Gi29io4pSZrlU20MIzdbLGcLYHOM8UX1a3qnOz3jqOzPOlF2fesDzDfWoTL+85aXT1mGaU/A=='}}} Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['keypair']['properties']['name']['allOf'][0]: Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['keypair']: Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi '' Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:05:40.701189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ab3ad22f-8f2e-44a0-bebf-f49fc522024d tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:40.706067 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ab3ad22f-8f2e-44a0-bebf-f49fc522024d tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 400 len: 108 microversion: 2.1 time: 0.010796 Sep 27 10:05:40.708575 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1256/2288] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:05:40 2022] POST /compute/v2.1/os-keypairs => generated 108 bytes in 16 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:05:40.716485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-164e0e02-afc8-4574-b3c4-8d3a51e9ec58 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Action: 'create', calling method: .version_select at 0x7f7f63641e50>, body: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-202187596", "public_key": " "}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:40.738379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-164e0e02-afc8-4574-b3c4-8d3a51e9ec58 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Getting quotas for project daeb3231e3b54f95b8fe082ef62c330e. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:40.743867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-164e0e02-afc8-4574-b3c4-8d3a51e9ec58 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Getting quotas for user a27051677d2e492eb0ebe1287c69a9d5 and project daeb3231e3b54f95b8fe082ef62c330e. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:40.762484 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-164e0e02-afc8-4574-b3c4-8d3a51e9ec58 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] HTTP exception thrown: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-164e0e02-afc8-4574-b3c4-8d3a51e9ec58 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Request method failure captured: Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 94 Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsrz26zv0qwrqHOGoFZxbH9Ep3vN62LYjjKCLxLkQqXJtRRp4k7CEWGqYmo8g5_equ4762Edl1kclX0iPkt3X5mjzI2MmgKOaaB0qoKoVFuO-y91hYddJHGTRx-O3pl82saYOZjiFPTAxYMgQmG8u56CBDHkBYAh15Nc5QRqdm3lZWg Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_daeb3231e3b54f95b8fe082ef62c330e"}]}] Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-202187596", "public_key": " "}} Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: exception: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'keypair': {'name': 'tempest-KeyPairsNegativeTestJSON-keypair-202187596', 'public_key': ' '}}} Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 71, in generate_fingerprint Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi serialization.load_ssh_public_key( Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 650, in load_ssh_public_key Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise ValueError("Invalid line format") Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ValueError: Invalid line format Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 117, in _create Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6774, in import_key_pair Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi fingerprint = self._generate_fingerprint(public_key, key_type) Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6847, in _generate_fingerprint Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return crypto.generate_fingerprint(public_key) Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 77, in generate_fingerprint Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InvalidKeypair( Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InvalidKeypair: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.765786 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 103, in create Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._create(req, body) Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 130, in _create Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=exc.format_message()) Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.771507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-164e0e02-afc8-4574-b3c4-8d3a51e9ec58 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Returning 400 to user: Keypair data is invalid: failed to generate fingerprint {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:40.775602 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-164e0e02-afc8-4574-b3c4-8d3a51e9ec58 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 400 len: 99 microversion: 2.1 time: 0.061203 Sep 27 10:05:40.775602 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1033/2289] 10.222.0.26 () {60 vars in 1158 bytes} [Tue Sep 27 10:05:40 2022] POST /compute/v2.1/os-keypairs => generated 99 bytes in 62 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:40.787060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13f31600-9ae6-4d0c-86ff-e26815e8b9f4 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Action: 'create', calling method: .version_select at 0x7f7f62b96ee0>, body: {"keypair": {"name": "keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFT1ihUcy9dAIPkAgaImWBsZodIgKUqkvDJVfBo/Wu7YsOOvVdHHgBS1obbeXoF3G22uT5Cu0+ENwVCwPuptxZFv5lw8m9ozPlzbCK35cVCcpPQzpkH3e7GwZlyMTQaZFQ=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13f31600-9ae6-4d0c-86ff-e26815e8b9f4 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Request method failure captured: Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 507 Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrz26zv0qwrqHOGoFZxbH9Ep3vN62LYjjKCLxLkQqXJtRRp4k7CEWGqYmo8g5_equ4762Edl1kclX0iPkt3X5mjzI2MmgKOaaB0qoKoVFuO-y91hYddJHGTRx-O3pl82saYOZjiFPTAxYMgQmG8u56CBDHkBYAh15Nc5QRqdm3lZWg Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_daeb3231e3b54f95b8fe082ef62c330e"}]}] Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: {"keypair": {"name": "keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFT1ihUcy9dAIPkAgaImWBsZodIgKUqkvDJVfBo/Wu7YsOOvVdHHgBS1obbeXoF3G22uT5Cu0+ENwVCwPuptxZFv5lw8m9ozPlzbCK35cVCcpPQzpkH3e7GwZlyMTQaZFQ=="}} Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'keypair': {'name': 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000', 'public_key': 'ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFT1ihUcy9dAIPkAgaImWBsZodIgKUqkvDJVfBo/Wu7YsOOvVdHHgBS1obbeXoF3G22uT5Cu0+ENwVCwPuptxZFv5lw8m9ozPlzbCK35cVCcpPQzpkH3e7GwZlyMTQaZFQ=='}}} Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['keypair']['properties']['name']['allOf'][0]: Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['keypair']: Sep 27 10:05:40.793647 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13f31600-9ae6-4d0c-86ff-e26815e8b9f4 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Returning 400 to user: Invalid input for field/attribute name. Value: keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000. 'keypair-000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000' is too long {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13f31600-9ae6-4d0c-86ff-e26815e8b9f4 tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 400 len: 627 microversion: 2.1 time: 0.013097 Sep 27 10:05:40.796387 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1257/2290] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:05:40 2022] POST /compute/v2.1/os-keypairs => generated 627 bytes in 15 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:05:40.807163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-607ed30c-747b-4685-86a1-37838d4f78ab tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Action: 'create', calling method: .version_select at 0x7f7f6366d0d0>, body: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-1904364168", "public_key": "ssh-rsa JUNK nova@ubuntu"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:40.826816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-607ed30c-747b-4685-86a1-37838d4f78ab tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Getting quotas for project daeb3231e3b54f95b8fe082ef62c330e. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:40.833388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-607ed30c-747b-4685-86a1-37838d4f78ab tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Getting quotas for user a27051677d2e492eb0ebe1287c69a9d5 and project daeb3231e3b54f95b8fe082ef62c330e. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:40.851376 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-607ed30c-747b-4685-86a1-37838d4f78ab tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] HTTP exception thrown: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-607ed30c-747b-4685-86a1-37838d4f78ab tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Request method failure captured: Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/os-keypairs HTTP/1.1 Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 118 Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsrz26zv0qwrqHOGoFZxbH9Ep3vN62LYjjKCLxLkQqXJtRRp4k7CEWGqYmo8g5_equ4762Edl1kclX0iPkt3X5mjzI2MmgKOaaB0qoKoVFuO-y91hYddJHGTRx-O3pl82saYOZjiFPTAxYMgQmG8u56CBDHkBYAh15Nc5QRqdm3lZWg Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_daeb3231e3b54f95b8fe082ef62c330e"}]}] Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: {"keypair": {"name": "tempest-KeyPairsNegativeTestJSON-keypair-1904364168", "public_key": "ssh-rsa JUNK nova@ubuntu"}} Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: exception: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'keypair': {'name': 'tempest-KeyPairsNegativeTestJSON-keypair-1904364168', 'public_key': 'ssh-rsa JUNK nova@ubuntu'}}} Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 71, in generate_fingerprint Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi serialization.load_ssh_public_key( Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 664, in load_ssh_public_key Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi inner_key_type, data = _get_sshstr(data) Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 134, in _get_sshstr Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi n, data = _get_u32(data) Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cryptography/hazmat/primitives/serialization/ssh.py", line 121, in _get_u32 Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise ValueError("Invalid data") Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ValueError: Invalid data Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 117, in _create Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi keypair = self.api.import_key_pair( Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6774, in import_key_pair Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi fingerprint = self._generate_fingerprint(public_key, key_type) Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6847, in _generate_fingerprint Sep 27 10:05:40.853868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return crypto.generate_fingerprint(public_key) Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/crypto.py", line 77, in generate_fingerprint Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InvalidKeypair( Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InvalidKeypair: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 103, in create Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._create(req, body) Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 130, in _create Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPBadRequest(explanation=exc.format_message()) Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair data is invalid: failed to generate fingerprint Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-607ed30c-747b-4685-86a1-37838d4f78ab tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Returning 400 to user: Keypair data is invalid: failed to generate fingerprint {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:40.857390 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-607ed30c-747b-4685-86a1-37838d4f78ab tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 400 len: 99 microversion: 2.1 time: 0.053535 Sep 27 10:05:40.860827 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1034/2291] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:05:40 2022] POST /compute/v2.1/os-keypairs => generated 99 bytes in 60 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 10:05:40.869515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b7cbcc80-fc5a-4604-831d-691c591aca8e tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Calling method '.version_select at 0x7f7f69129700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:40.882091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b7cbcc80-fc5a-4604-831d-691c591aca8e tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] HTTP exception thrown: Keypair tempest-keypair-non-existent-2144753534 not found for user a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b7cbcc80-fc5a-4604-831d-691c591aca8e tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Request method failure captured: Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-non-existent-2144753534 HTTP/1.1 Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrz26zv0qwrqHOGoFZxbH9Ep3vN62LYjjKCLxLkQqXJtRRp4k7CEWGqYmo8g5_equ4762Edl1kclX0iPkt3X5mjzI2MmgKOaaB0qoKoVFuO-y91hYddJHGTRx-O3pl82saYOZjiFPTAxYMgQmG8u56CBDHkBYAh15Nc5QRqdm3lZWg Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/daeb3231e3b54f95b8fe082ef62c330e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_daeb3231e3b54f95b8fe082ef62c330e"}]}] Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: daeb3231e3b54f95b8fe082ef62c330e Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-KeyPairsNegativeTestJSON-1481576023 Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-KeyPairsNegativeTestJSON-1481576023-project Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: exception: Keypair tempest-keypair-non-existent-2144753534 not found for user a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'tempest-keypair-non-existent-2144753534'} Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-non-existent-2144753534 not found for user a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.884053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-non-existent-2144753534 not found for user a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-non-existent-2144753534 not found for user a27051677d2e492eb0ebe1287c69a9d5 Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b7cbcc80-fc5a-4604-831d-691c591aca8e tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] Returning 404 to user: Keypair tempest-keypair-non-existent-2144753534 not found for user a27051677d2e492eb0ebe1287c69a9d5 {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:40.887800 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b7cbcc80-fc5a-4604-831d-691c591aca8e tempest-KeyPairsNegativeTestJSON-1481576023 tempest-KeyPairsNegativeTestJSON-1481576023-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-non-existent-2144753534" status: 404 len: 145 microversion: 2.1 time: 0.022993 Sep 27 10:05:40.893976 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1258/2292] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:05:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-non-existent-2144753534 => generated 145 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:40.899204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7c54bdeb-cc22-4951-970a-097d78bfce48 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:40.901058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7c54bdeb-cc22-4951-970a-097d78bfce48 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:40.912547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c54bdeb-cc22-4951-970a-097d78bfce48 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:40.913464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c54bdeb-cc22-4951-970a-097d78bfce48 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:40.915493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c54bdeb-cc22-4951-970a-097d78bfce48 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:40.916436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c54bdeb-cc22-4951-970a-097d78bfce48 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:40.917083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c54bdeb-cc22-4951-970a-097d78bfce48 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:40.918621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c54bdeb-cc22-4951-970a-097d78bfce48 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:40.955455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6d30f51d-ae5b-4eb9-8d91-2595dfd9eae8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:40.956107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6d30f51d-ae5b-4eb9-8d91-2595dfd9eae8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:40.968454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d30f51d-ae5b-4eb9-8d91-2595dfd9eae8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:40.968995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d30f51d-ae5b-4eb9-8d91-2595dfd9eae8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:40.971833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d30f51d-ae5b-4eb9-8d91-2595dfd9eae8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:40.972812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d30f51d-ae5b-4eb9-8d91-2595dfd9eae8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:40.973311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d30f51d-ae5b-4eb9-8d91-2595dfd9eae8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:40.973860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d30f51d-ae5b-4eb9-8d91-2595dfd9eae8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:41.325885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6d30f51d-ae5b-4eb9-8d91-2595dfd9eae8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:41.350983 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6d30f51d-ae5b-4eb9-8d91-2595dfd9eae8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.399500 Sep 27 10:05:41.355028 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1259/2293] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:40 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:41.364214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:41.365153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] [instance: 219e4ceb-d811-4e6f-913f-c51410efcee6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:41.384406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:41.385198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:41.385815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:41.386803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:41.387768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:41.388501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:41.413929 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] HTTP exception thrown: Instance 219e4ceb-d811-4e6f-913f-c51410efcee6 could not be found. Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Request method failure captured: Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 HTTP/1.1 Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsqxodNJy6n1VE9D34Va7wv0Zd2JjDvgxjia6ye538ZZsPnizcF3Xqteez0D9Wf7OOcEPGHKSHHEAGaV0bt6l4NESjiChjKXDIS-bvgjfS2WDBDr5M7lz6sh8DS5Zisk2eNSTqUWak68uJBesPZb_Na0IQCcTvfIwA6EGD0uZhMX5Po Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 3f33c209ffd1480e94175241ccc90414 Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerAddressesTestJSON-2066038785 Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3f33c209ffd1480e94175241ccc90414"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3f33c209ffd1480e94175241ccc90414"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3f33c209ffd1480e94175241ccc90414"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3f33c209ffd1480e94175241ccc90414"}]}] Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerAddressesTestJSON-2066038785 Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 3f33c209ffd1480e94175241ccc90414 Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerAddressesTestJSON-2066038785 Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerAddressesTestJSON-2066038785-project Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 297a11c331fb473291750ced9c843a6e Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerAddressesTestJSON-2066038785-project Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 219e4ceb-d811-4e6f-913f-c51410efcee6 could not be found. Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '219e4ceb-d811-4e6f-913f-c51410efcee6'} Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 219e4ceb-d811-4e6f-913f-c51410efcee6 could not be found. Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 219e4ceb-d811-4e6f-913f-c51410efcee6 could not be found. Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:41.417010 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:41.420342 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:41.420342 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:41.420342 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:41.420342 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:41.420342 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:41.420342 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 219e4ceb-d811-4e6f-913f-c51410efcee6 could not be found. Sep 27 10:05:41.420342 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:41.422238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] Returning 404 to user: Instance 219e4ceb-d811-4e6f-913f-c51410efcee6 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:41.423276 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a646ebf9-c23f-46f9-a461-d5738b5e2501 tempest-ServerAddressesTestJSON-2066038785 tempest-ServerAddressesTestJSON-2066038785-project] 10.222.0.26 "GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6" status: 404 len: 111 microversion: 2.1 time: 0.067258 Sep 27 10:05:41.427697 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1260/2294] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:41 2022] GET /compute/v2.1/servers/219e4ceb-d811-4e6f-913f-c51410efcee6 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:05:41.433991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-faa07e66-3407-4620-87ec-f669014124e8 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62b93160>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:41.533080 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-faa07e66-3407-4620-87ec-f669014124e8 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.103969 Sep 27 10:05:41.541489 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1261/2295] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:41 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:41.652414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:41.653799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:41.661927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:41.662664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:41.663438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:41.665282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:41.666063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:41.666940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:41.752683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:41.773739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:41.778680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:41.780983 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86eb8d0d-d180-480e-8e2d-42f0e4b54e9e tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1661 microversion: 2.25 time: 0.133353 Sep 27 10:05:41.800972 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1262/2296] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:41 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1661 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:41.990009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7c54bdeb-cc22-4951-970a-097d78bfce48 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:41.999140 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7c54bdeb-cc22-4951-970a-097d78bfce48 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 1.107851 Sep 27 10:05:41.999740 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1035/2297] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:40 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 1109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:42.374982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a1e7f568-bb58-4412-91f9-73930d5e085d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:42.375607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a1e7f568-bb58-4412-91f9-73930d5e085d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:42.392606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1e7f568-bb58-4412-91f9-73930d5e085d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:42.392966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1e7f568-bb58-4412-91f9-73930d5e085d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:42.393753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1e7f568-bb58-4412-91f9-73930d5e085d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:42.394619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1e7f568-bb58-4412-91f9-73930d5e085d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:42.395222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1e7f568-bb58-4412-91f9-73930d5e085d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:42.395793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1e7f568-bb58-4412-91f9-73930d5e085d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:42.408620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b04006ac-1bab-4b3c-9653-403066614f04 req-7671af1a-fc06-4bda-9449-1bbec7353804 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fcd14928-8ef7-425b-89bb-2c714284e840", "name": "network-vif-deleted", "tag": "b4c8bc77-41d3-49f6-84f2-59d40e203a22"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:42.439574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b04006ac-1bab-4b3c-9653-403066614f04 req-7671af1a-fc06-4bda-9449-1bbec7353804 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:42.440399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b04006ac-1bab-4b3c-9653-403066614f04 req-7671af1a-fc06-4bda-9449-1bbec7353804 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:42.441117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b04006ac-1bab-4b3c-9653-403066614f04 req-7671af1a-fc06-4bda-9449-1bbec7353804 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:42.486777 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b04006ac-1bab-4b3c-9653-403066614f04 req-7671af1a-fc06-4bda-9449-1bbec7353804 service nova] Creating event network-vif-deleted:b4c8bc77-41d3-49f6-84f2-59d40e203a22 for instance fcd14928-8ef7-425b-89bb-2c714284e840 on nch1-master-1 Sep 27 10:05:42.495644 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b04006ac-1bab-4b3c-9653-403066614f04 req-7671af1a-fc06-4bda-9449-1bbec7353804 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090618 Sep 27 10:05:42.499504 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1036/2298] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:42.551633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6868b754-538e-4c2f-bc67-58c0f6288ece tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6377b280>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:42.633062 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6868b754-538e-4c2f-bc67-58c0f6288ece tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.085891 Sep 27 10:05:42.639841 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1037/2299] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:42 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:42.811806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:42.813161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:42.837096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a1e7f568-bb58-4412-91f9-73930d5e085d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:42.838667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:42.839405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:42.840437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:42.842360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:42.845253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:42.847981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:42.851237 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a1e7f568-bb58-4412-91f9-73930d5e085d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.480146 Sep 27 10:05:42.856718 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1263/2300] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:42 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:42.927429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:42.927725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:42.928374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:42.929805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lazy-loading 'fault' on Instance uuid fcd14928-8ef7-425b-89bb-2c714284e840 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:05:43.028016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30bd2fde-8767-4c1b-af12-bb75c4152ecf tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:43.028986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-30bd2fde-8767-4c1b-af12-bb75c4152ecf tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:43.046366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bd2fde-8767-4c1b-af12-bb75c4152ecf tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:43.047061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bd2fde-8767-4c1b-af12-bb75c4152ecf tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:43.047524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bd2fde-8767-4c1b-af12-bb75c4152ecf tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:43.053097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bd2fde-8767-4c1b-af12-bb75c4152ecf tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:43.054297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bd2fde-8767-4c1b-af12-bb75c4152ecf tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:43.055009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bd2fde-8767-4c1b-af12-bb75c4152ecf tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:43.229921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:43.247368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:43.249011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:43.250522 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-509d8b13-c2fb-4e8e-8305-18766e1854ca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 200 len: 1374 microversion: 2.25 time: 0.443141 Sep 27 10:05:43.255488 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1038/2301] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:42 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 1374 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:43.364374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-30bd2fde-8767-4c1b-af12-bb75c4152ecf tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:43.379396 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-30bd2fde-8767-4c1b-af12-bb75c4152ecf tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 0.358045 Sep 27 10:05:43.393018 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1264/2302] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:43 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:43.649082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-31fe8fc2-fbc9-454c-8fcd-8ad7f8de4c7d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f63760ee0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:43.758520 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-31fe8fc2-fbc9-454c-8fcd-8ad7f8de4c7d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.112846 Sep 27 10:05:43.769883 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1039/2303] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:43 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:43.881568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-77138d98-4547-4005-8069-33e6e6e6be54 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:43.883488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-77138d98-4547-4005-8069-33e6e6e6be54 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:43.894479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77138d98-4547-4005-8069-33e6e6e6be54 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:43.899578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77138d98-4547-4005-8069-33e6e6e6be54 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:43.900319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77138d98-4547-4005-8069-33e6e6e6be54 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:43.902389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77138d98-4547-4005-8069-33e6e6e6be54 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:43.910476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77138d98-4547-4005-8069-33e6e6e6be54 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:43.916253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77138d98-4547-4005-8069-33e6e6e6be54 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:44.203618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-77138d98-4547-4005-8069-33e6e6e6be54 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:44.216389 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-77138d98-4547-4005-8069-33e6e6e6be54 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.339460 Sep 27 10:05:44.217374 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1265/2304] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:43 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:44.277676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-87ac750f-c43f-4d04-a480-cea0e97fd914 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Action: 'create', calling method: .version_select at 0x7f7f62bcd1f0>, body: {"keypair": {"name": "tempest-keypair-698556521", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEBVUoTZ+/GeALhWm+eaLzp2V8cVRo6LhGrhEzmaIi1E3wahqXAF3jlj69V2eXRHuZfCEQ0GD9nGilZ/0ViLkzekWAdb39TX+6n0w5Mn/x8C3Q8wKBZLY5AknY+8Ky2Drg=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:44.287587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:44.288629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: fcd14928-8ef7-425b-89bb-2c714284e840] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:44.300652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-87ac750f-c43f-4d04-a480-cea0e97fd914 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:44.303928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:44.304425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:44.306837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:44.308239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-87ac750f-c43f-4d04-a480-cea0e97fd914 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for user 0f4dad886747497094caf63cb33f56d6 and project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:44.310389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:44.311359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:44.311838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:44.350428 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] HTTP exception thrown: Instance fcd14928-8ef7-425b-89bb-2c714284e840 could not be found. Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Request method failure captured: Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 HTTP/1.1 Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsls2FdIOhvD4ZxN7OIKQmiUCELV6z27WbYJyCg5zc33cF3I2IG0F16O0kd5J_CobjZKV0lxKRF-IbyT-BYgGU09wQILwgGZLa230rmDdLmWzYS6zGfOygm-RvpfT-sJE6lwtE6xYYRfLbLdFYc5W5dGOjgOEW5aqauEbGKNWvjw5os Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.25 Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 2369f5ee467643948f618bbdee837521 Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2369f5ee467643948f618bbdee837521"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2369f5ee467643948f618bbdee837521"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2369f5ee467643948f618bbdee837521"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2369f5ee467643948f618bbdee837521"}]}] Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 2369f5ee467643948f618bbdee837521 Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-LiveAutoBlockMigrationV225Test-1036261617-project Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 29d2f7b0e8274c769521d13930be9825 Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617-project Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance fcd14928-8ef7-425b-89bb-2c714284e840 could not be found. Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'fcd14928-8ef7-425b-89bb-2c714284e840'} Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fcd14928-8ef7-425b-89bb-2c714284e840 could not be found. Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fcd14928-8ef7-425b-89bb-2c714284e840 could not be found. Sep 27 10:05:44.355153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:44.359081 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:44.359081 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:44.359081 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:44.359081 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:44.359081 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:44.359081 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:44.359081 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:44.359081 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fcd14928-8ef7-425b-89bb-2c714284e840 could not be found. Sep 27 10:05:44.359081 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:05:44.359565 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-87ac750f-c43f-4d04-a480-cea0e97fd914 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.2 time: 0.319765 Sep 27 10:05:44.359565 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1040/2305] 10.222.0.26 () {62 vars in 1199 bytes} [Tue Sep 27 10:05:44 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 326 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Sep 27 10:05:44.360609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Returning 404 to user: Instance fcd14928-8ef7-425b-89bb-2c714284e840 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:44.364576 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b54944d7-d472-4853-bcbd-7935f4d66e0b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840" status: 404 len: 111 microversion: 2.25 time: 0.083148 Sep 27 10:05:44.367349 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1266/2306] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:44 2022] GET /compute/v2.1/servers/fcd14928-8ef7-425b-89bb-2c714284e840 => generated 111 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:05:44.377418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f758cd25-83fe-4d70-b9a9-1de7125738c5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '.version_select at 0x7f7f6297f1f0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:44.384395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5f6608ce-7c05-49d2-ad5d-95c2ad7aa3c2 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f62bcd4c0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:44.398292 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5f6608ce-7c05-49d2-ad5d-95c2ad7aa3c2 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "GET /compute/v2.1/os-keypairs/tempest-keypair-698556521" status: 200 len: 512 microversion: 2.2 time: 0.025130 Sep 27 10:05:44.407250 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1041/2307] 10.222.0.26 () {60 vars in 1255 bytes} [Tue Sep 27 10:05:44 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-698556521 => generated 512 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:44.418585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3a6d6389-54ef-468c-ad5f-cae89889d6b2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:44.419421 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f758cd25-83fe-4d70-b9a9-1de7125738c5 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-270589461" status: 204 len: 0 microversion: 2.25 time: 0.047340 Sep 27 10:05:44.420205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3a6d6389-54ef-468c-ad5f-cae89889d6b2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:44.420591 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1267/2308] 10.222.0.26 () {60 vars in 1259 bytes} [Tue Sep 27 10:05:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-270589461 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Sep 27 10:05:44.427852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-40bb31fe-d525-45eb-ad23-579ee7547bef tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f62b8d940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:44.430688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a6d6389-54ef-468c-ad5f-cae89889d6b2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:44.433174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a6d6389-54ef-468c-ad5f-cae89889d6b2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:44.433889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a6d6389-54ef-468c-ad5f-cae89889d6b2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:44.434848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a6d6389-54ef-468c-ad5f-cae89889d6b2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:44.435531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a6d6389-54ef-468c-ad5f-cae89889d6b2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:44.436213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a6d6389-54ef-468c-ad5f-cae89889d6b2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:44.483745 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-40bb31fe-d525-45eb-ad23-579ee7547bef tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-698556521" status: 204 len: 0 microversion: 2.2 time: 0.061925 Sep 27 10:05:44.485305 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1268/2309] 10.222.0.26 () {60 vars in 1258 bytes} [Tue Sep 27 10:05:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-698556521 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:44.512102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6da6d218-9a3c-432f-9be5-1e4a79cb0c52 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Action: 'create', calling method: .version_select at 0x7f7f6297f8b0>, body: {"keypair": {"name": "tempest-keypair-19247740", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJSgv+25jik5mUz14lpZrPk9vBapFXDf+n+rM8A49P7+d3g+ZB9NhfnEIIhcirdElBS34mq1cF1kkT9As89fQbjsM/OjyvvLt0fbs9T+PWAXBMfWmtKa/AW6/YLxs7PldQ=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:44.545364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6da6d218-9a3c-432f-9be5-1e4a79cb0c52 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:44.558021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6da6d218-9a3c-432f-9be5-1e4a79cb0c52 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for user 0f4dad886747497094caf63cb33f56d6 and project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:44.597943 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6da6d218-9a3c-432f-9be5-1e4a79cb0c52 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 399 microversion: 2.2 time: 0.101573 Sep 27 10:05:44.600272 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1269/2310] 10.222.0.26 () {62 vars in 1199 bytes} [Tue Sep 27 10:05:44 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 108 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Sep 27 10:05:44.619127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99b17b89-79a4-40f6-98f3-32528119f925 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f6298e700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:44.689059 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-99b17b89-79a4-40f6-98f3-32528119f925 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-19247740" status: 204 len: 0 microversion: 2.2 time: 0.078198 Sep 27 10:05:44.690516 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1270/2311] 10.222.0.26 () {60 vars in 1255 bytes} [Tue Sep 27 10:05:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-19247740 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:44.700066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-09755101-4ba8-4c15-90a3-0f25fabad756 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Action: 'create', calling method: .version_select at 0x7f7f62992dc0>, body: {"keypair": {"name": "tempest-keypair-1073561101", "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsNe3/1ILNCqFyfYWDeTKLD6jEXC2OQHLmietMWW+/vdaZq7KZEwO0jhglaFjU1mpqq4Gz5RX156sCTNM9vRbwKAxfsdF9laBYVsex3m3Wmui3uYrKyumsoJn2g9GNnG1PI1mrVjZ61i0GY3khna+wzlTpCCmy5HNlrmbj3XLqBUpipTOXmsnr4sChzC53KCd8LXuwc1i/CZPvF+3XipvAgFSE53pCtLOeB1kYMOBaiUPLQTWXR3JpckqFIQwhIH0zoHlJvZE8hh90XcPojYN56tI0OlrGqojbediJYD0rUsJu4weZpbn8vilb3JuDY+jwssnSA8wzBx3A/8y9Pp1B nova@ubuntu"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:44.739888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=3,nova_api:DELETE=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:44.753737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-09755101-4ba8-4c15-90a3-0f25fabad756 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:44.760259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-09755101-4ba8-4c15-90a3-0f25fabad756 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for user 0f4dad886747497094caf63cb33f56d6 and project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:44.810101 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-09755101-4ba8-4c15-90a3-0f25fabad756 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 589 microversion: 2.2 time: 0.114290 Sep 27 10:05:44.812603 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1271/2312] 10.222.0.26 () {62 vars in 1199 bytes} [Tue Sep 27 10:05:44 2022] POST /compute/v2.1/os-keypairs => generated 589 bytes in 117 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Sep 27 10:05:44.819733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0f431275-8d98-44e3-abf6-6af8f928fb42 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6298f280>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:44.886745 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0f431275-8d98-44e3-abf6-6af8f928fb42 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.072562 Sep 27 10:05:44.888009 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1272/2313] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:44 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:44.896363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fae23820-9187-4674-a537-0a9eb6c113b9 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f629929d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:44.946703 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fae23820-9187-4674-a537-0a9eb6c113b9 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1073561101" status: 204 len: 0 microversion: 2.2 time: 0.056842 Sep 27 10:05:44.947336 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1273/2314] 10.222.0.26 () {60 vars in 1261 bytes} [Tue Sep 27 10:05:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1073561101 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:44.957806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c523ed76-3153-4055-9c78-d369b674ef2d req-4bb74400-5ce0-4590-a21b-a3e5de436dfc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35973885-926e-4fa3-80a3-0df40511ebd2", "name": "network-vif-plugged", "status": "completed", "tag": "8ac1d5d0-a175-4922-8f13-aacbb8c5d6b0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:44.970086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c523ed76-3153-4055-9c78-d369b674ef2d req-4bb74400-5ce0-4590-a21b-a3e5de436dfc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:44.970840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c523ed76-3153-4055-9c78-d369b674ef2d req-4bb74400-5ce0-4590-a21b-a3e5de436dfc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:44.971496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c523ed76-3153-4055-9c78-d369b674ef2d req-4bb74400-5ce0-4590-a21b-a3e5de436dfc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:45.002163 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c523ed76-3153-4055-9c78-d369b674ef2d req-4bb74400-5ce0-4590-a21b-a3e5de436dfc service nova] Creating event network-vif-plugged:8ac1d5d0-a175-4922-8f13-aacbb8c5d6b0 for instance 35973885-926e-4fa3-80a3-0df40511ebd2 on nch2-master-1 Sep 27 10:05:45.010802 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c523ed76-3153-4055-9c78-d369b674ef2d req-4bb74400-5ce0-4590-a21b-a3e5de436dfc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061990 Sep 27 10:05:45.011686 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1274/2315] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:45.018266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-365dae53-e5ff-426b-852c-ff9475e9a8cf tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Action: 'create', calling method: .version_select at 0x7f7f6297f4c0>, body: {"keypair": {"name": "tempest-KeyPairsV22TestJSON-keypair-1105064478", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCpklPLRBZL95CNJlwpZ6ZHpW8Np66YwQuXt+QAIw0F5wCyzoinehrbfobkhd2r84zkI0trl8brREXpgVEnl+Z+rHyeaqK/bNO2dnALedd0eT+hc9bGbUQUjfjoNGQK7sw=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:45.023802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3a6d6389-54ef-468c-ad5f-cae89889d6b2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:45.045852 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3a6d6389-54ef-468c-ad5f-cae89889d6b2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 0.633668 Sep 27 10:05:45.046575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-365dae53-e5ff-426b-852c-ff9475e9a8cf tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:45.046782 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1042/2316] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:44 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 639 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:45.080826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-365dae53-e5ff-426b-852c-ff9475e9a8cf tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for user 0f4dad886747497094caf63cb33f56d6 and project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:45.125771 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-365dae53-e5ff-426b-852c-ff9475e9a8cf tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 421 microversion: 2.2 time: 0.112541 Sep 27 10:05:45.127192 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1275/2317] 10.222.0.26 () {62 vars in 1199 bytes} [Tue Sep 27 10:05:45 2022] POST /compute/v2.1/os-keypairs => generated 421 bytes in 115 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Sep 27 10:05:45.142442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5619d158-8f09-4069-b230-44feb55e74ca tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Action: 'create', calling method: .version_select at 0x7f7f6377b280>, body: {"keypair": {"name": "tempest-KeyPairsV22TestJSON-keypair-917118208", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGx0Uj7qbddhGv3iwXgLqDnv22Hgvsy9AiTxLGFw6mOnknrxh6rjIojgdrQ2Q24OuKA3Vz8OzZ81xfgay3tWNrvvtOYQ2WsGgsu1QqxjI/eAPEcZtJNn32g2Rl6Xjm4vlA=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:45.161659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5619d158-8f09-4069-b230-44feb55e74ca tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:45.168528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5619d158-8f09-4069-b230-44feb55e74ca tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for user 0f4dad886747497094caf63cb33f56d6 and project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:45.199270 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5619d158-8f09-4069-b230-44feb55e74ca tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 420 microversion: 2.2 time: 0.060174 Sep 27 10:05:45.200808 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1043/2318] 10.222.0.26 () {62 vars in 1199 bytes} [Tue Sep 27 10:05:45 2022] POST /compute/v2.1/os-keypairs => generated 420 bytes in 63 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Sep 27 10:05:45.218749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3bba372e-84f3-4473-9ee3-03809ceeb758 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Action: 'create', calling method: .version_select at 0x7f7f62992820>, body: {"keypair": {"name": "tempest-KeyPairsV22TestJSON-keypair-2085881304", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ14GhPAHZHTttxw9XCC/KhOlwOdu2yEQC5hqCqtymcZxHOnyPODbP+BpcmLWnwModsCSCxADAlaIcY6hRas3Mb8yRdMBj5y+diGOQJ97biyCfgeznLY0rMeCJAyJ9ryeg=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:45.241621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d0bd1054-5f61-4cde-b717-8587bfd73a95 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:45.241829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3bba372e-84f3-4473-9ee3-03809ceeb758 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:45.242788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d0bd1054-5f61-4cde-b717-8587bfd73a95 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:45.249834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3bba372e-84f3-4473-9ee3-03809ceeb758 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for user 0f4dad886747497094caf63cb33f56d6 and project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:45.252981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0bd1054-5f61-4cde-b717-8587bfd73a95 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:45.253722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0bd1054-5f61-4cde-b717-8587bfd73a95 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:45.254948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0bd1054-5f61-4cde-b717-8587bfd73a95 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:45.256043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0bd1054-5f61-4cde-b717-8587bfd73a95 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:45.257324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0bd1054-5f61-4cde-b717-8587bfd73a95 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:45.258044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0bd1054-5f61-4cde-b717-8587bfd73a95 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:45.299098 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3bba372e-84f3-4473-9ee3-03809ceeb758 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 421 microversion: 2.2 time: 0.086458 Sep 27 10:05:45.300613 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1276/2319] 10.222.0.26 () {62 vars in 1199 bytes} [Tue Sep 27 10:05:45 2022] POST /compute/v2.1/os-keypairs => generated 421 bytes in 90 msecs (HTTP/1.1 201) 9 headers in 362 bytes (1 switches on core 0) Sep 27 10:05:45.311236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d7245601-36ac-4462-8ebf-65d8b6ba65c6 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f6298f310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:45.316256 ncdv-master-1 devstack@n-api.service[145968]: WARNING oslo_db.sqlalchemy.utils [None req-d7245601-36ac-4462-8ebf-65d8b6ba65c6 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Unique keys not in sort_keys. The sorting order may be unstable. Sep 27 10:05:45.326349 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d7245601-36ac-4462-8ebf-65d8b6ba65c6 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "GET /compute/v2.1/os-keypairs" status: 200 len: 1141 microversion: 2.2 time: 0.018604 Sep 27 10:05:45.326696 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1277/2320] 10.222.0.26 () {60 vars in 1177 bytes} [Tue Sep 27 10:05:45 2022] GET /compute/v2.1/os-keypairs => generated 1141 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:45.355748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82c0b8c1-7f91-4ee6-95e6-89380e7508fa tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f62be3e50>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:45.383120 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-82c0b8c1-7f91-4ee6-95e6-89380e7508fa tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-2085881304" status: 204 len: 0 microversion: 2.2 time: 0.034003 Sep 27 10:05:45.385283 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1278/2321] 10.222.0.26 () {60 vars in 1321 bytes} [Tue Sep 27 10:05:45 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-2085881304 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:45.393667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-400997fa-3631-4a38-84a6-c01307a4a43d tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f62b965e0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:45.426410 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-400997fa-3631-4a38-84a6-c01307a4a43d tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-917118208" status: 204 len: 0 microversion: 2.2 time: 0.036768 Sep 27 10:05:45.427322 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1279/2322] 10.222.0.26 () {60 vars in 1318 bytes} [Tue Sep 27 10:05:45 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-917118208 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:45.437012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8274c22-6b22-49e5-a0f1-e05ff66ed3d6 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f62b933a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:45.466632 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c8274c22-6b22-49e5-a0f1-e05ff66ed3d6 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-1105064478" status: 204 len: 0 microversion: 2.2 time: 0.033434 Sep 27 10:05:45.469439 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1280/2323] 10.222.0.26 () {60 vars in 1321 bytes} [Tue Sep 27 10:05:45 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV22TestJSON-keypair-1105064478 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:45.480139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7399f1ba-48fd-4f9b-bb69-b172b6afab5f tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Action: 'create', calling method: .version_select at 0x7f7f62bb1040>, body: {"keypair": {"name": "tempest-keypair-1205122436", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFWX8UfC3GcYfIM9b1B5nnY+ICcPmnFXsRwpDN5JEvy1Pza9gYLnncBf2D49eclfaYXTeLRnn9So/+2EaXcXFPKWB/i54234a6AzyeVfy6zlxWbAh1fiHprijUr53TpxJQ=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:45.518041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-7399f1ba-48fd-4f9b-bb69-b172b6afab5f tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:45.524401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-7399f1ba-48fd-4f9b-bb69-b172b6afab5f tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for user 0f4dad886747497094caf63cb33f56d6 and project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:45.555361 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7399f1ba-48fd-4f9b-bb69-b172b6afab5f tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.2 time: 0.078670 Sep 27 10:05:45.555361 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1281/2324] 10.222.0.26 () {62 vars in 1199 bytes} [Tue Sep 27 10:05:45 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 80 msecs (HTTP/1.1 201) 9 headers in 362 bytes (2 switches on core 0) Sep 27 10:05:45.570936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b45ffb6-4dbd-43dd-ba5d-7f7a6152994d tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f6298f700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:45.593108 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1b45ffb6-4dbd-43dd-ba5d-7f7a6152994d tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "GET /compute/v2.1/os-keypairs/tempest-keypair-1205122436" status: 200 len: 513 microversion: 2.2 time: 0.027034 Sep 27 10:05:45.595395 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1282/2325] 10.222.0.26 () {60 vars in 1258 bytes} [Tue Sep 27 10:05:45 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-1205122436 => generated 513 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:05:45.613730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1525ff29-35c7-4437-b6c9-3141924bce66 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f6298faf0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:45.633839 ncdv-master-1 devstack@n-api.service[145968]: WARNING oslo_db.sqlalchemy.utils [None req-1525ff29-35c7-4437-b6c9-3141924bce66 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Unique keys not in sort_keys. The sorting order may be unstable. Sep 27 10:05:45.646709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0bd1054-5f61-4cde-b717-8587bfd73a95 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:45.671553 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d0bd1054-5f61-4cde-b717-8587bfd73a95 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.435331 Sep 27 10:05:45.672422 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1044/2326] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:45 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:45.673387 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1525ff29-35c7-4437-b6c9-3141924bce66 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "GET /compute/v2.1/os-keypairs" status: 200 len: 370 microversion: 2.2 time: 0.071291 Sep 27 10:05:45.674428 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1283/2327] 10.222.0.26 () {60 vars in 1177 bytes} [Tue Sep 27 10:05:45 2022] GET /compute/v2.1/os-keypairs => generated 370 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:45.707431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d1af44bb-2a11-4f85-a65f-41e85a88cebb tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f62bcda60>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:45.763007 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d1af44bb-2a11-4f85-a65f-41e85a88cebb tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1205122436" status: 204 len: 0 microversion: 2.2 time: 0.068359 Sep 27 10:05:45.763852 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1045/2328] 10.222.0.26 () {60 vars in 1261 bytes} [Tue Sep 27 10:05:45 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1205122436 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:45.774034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5effa631-aa70-4208-bfd0-554153443e07 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Action: 'create', calling method: .version_select at 0x7f7f62992430>, body: {"keypair": {"name": "tempest-keypair-1670333196", "type": "x509"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:45.811129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5effa631-aa70-4208-bfd0-554153443e07 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:45.817839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5effa631-aa70-4208-bfd0-554153443e07 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for user 0f4dad886747497094caf63cb33f56d6 and project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:45.840430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.processutils [None req-5effa631-aa70-4208-bfd0-554153443e07 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Running cmd (subprocess): openssl req -x509 -nodes -days 3650 -config /tmp/tmpfm61flvo/temp.conf -newkey rsa:2048 -outform PEM -keyout /tmp/tmpfm61flvo/temp.key -subj /CN=0f4dad886747497094caf63cb33f56d6 -extensions v3_req_client {{(pid=145968) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:05:45.904204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-68188268-36de-4ab0-b324-66388226b7b7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62e445e0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:45.982471 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-68188268-36de-4ab0-b324-66388226b7b7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.083289 Sep 27 10:05:45.983699 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1046/2329] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:45 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:46.076858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-389a7fd5-d158-46d6-89cb-b630aee09dc1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:46.078163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-389a7fd5-d158-46d6-89cb-b630aee09dc1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:46.087554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-389a7fd5-d158-46d6-89cb-b630aee09dc1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:46.088054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-389a7fd5-d158-46d6-89cb-b630aee09dc1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:46.088501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-389a7fd5-d158-46d6-89cb-b630aee09dc1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:46.093685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-389a7fd5-d158-46d6-89cb-b630aee09dc1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:46.094182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-389a7fd5-d158-46d6-89cb-b630aee09dc1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:46.097634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-389a7fd5-d158-46d6-89cb-b630aee09dc1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:46.192629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.processutils [None req-5effa631-aa70-4208-bfd0-554153443e07 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] CMD "openssl req -x509 -nodes -days 3650 -config /tmp/tmpfm61flvo/temp.conf -newkey rsa:2048 -outform PEM -keyout /tmp/tmpfm61flvo/temp.key -subj /CN=0f4dad886747497094caf63cb33f56d6 -extensions v3_req_client" returned: 0 in 0.352s {{(pid=145968) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:05:46.194900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.processutils [None req-5effa631-aa70-4208-bfd0-554153443e07 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Running cmd (subprocess): openssl pkcs12 -export -inkey /tmp/tmpfm61flvo/temp.key -password pass: {{(pid=145968) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} Sep 27 10:05:46.225053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.processutils [None req-5effa631-aa70-4208-bfd0-554153443e07 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] CMD "openssl pkcs12 -export -inkey /tmp/tmpfm61flvo/temp.key -password pass:" returned: 0 in 0.029s {{(pid=145968) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} Sep 27 10:05:46.283776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5effa631-aa70-4208-bfd0-554153443e07 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:05:46.289120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5effa631-aa70-4208-bfd0-554153443e07 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Getting quotas for user 0f4dad886747497094caf63cb33f56d6 and project 092557b4387f406e851d6b09a9076193. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:05:46.305876 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5effa631-aa70-4208-bfd0-554153443e07 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 4693 microversion: 2.2 time: 0.535810 Sep 27 10:05:46.306847 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1284/2330] 10.222.0.26 () {62 vars in 1198 bytes} [Tue Sep 27 10:05:45 2022] POST /compute/v2.1/os-keypairs => generated 4693 bytes in 537 msecs (HTTP/1.1 201) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:46.314105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ee2b1078-0e2e-4d60-b5c7-e292d0869dca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:46.314854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ee2b1078-0e2e-4d60-b5c7-e292d0869dca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:46.323404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee2b1078-0e2e-4d60-b5c7-e292d0869dca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:46.324368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee2b1078-0e2e-4d60-b5c7-e292d0869dca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:46.324774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee2b1078-0e2e-4d60-b5c7-e292d0869dca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:46.325860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee2b1078-0e2e-4d60-b5c7-e292d0869dca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:46.326304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee2b1078-0e2e-4d60-b5c7-e292d0869dca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:46.326994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee2b1078-0e2e-4d60-b5c7-e292d0869dca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:46.359748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ee2b1078-0e2e-4d60-b5c7-e292d0869dca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:05:46.397992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-389a7fd5-d158-46d6-89cb-b630aee09dc1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:46.406481 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-389a7fd5-d158-46d6-89cb-b630aee09dc1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 0.338798 Sep 27 10:05:46.407090 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1047/2331] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:46 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:46.430130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-18ca5a5c-ade4-45b1-b844-d5b53c75e988 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f62e44af0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:46.440089 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-18ca5a5c-ade4-45b1-b844-d5b53c75e988 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "GET /compute/v2.1/os-keypairs/tempest-keypair-1670333196" status: 200 len: 1534 microversion: 2.2 time: 0.014012 Sep 27 10:05:46.441330 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1048/2332] 10.222.0.26 () {60 vars in 1258 bytes} [Tue Sep 27 10:05:46 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-1670333196 => generated 1534 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:46.455318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6d141a0b-542c-4940-b573-d3d59a049ce5 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f63760f70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:46.462956 ncdv-master-1 devstack@n-api.service[145969]: WARNING oslo_db.sqlalchemy.utils [None req-6d141a0b-542c-4940-b573-d3d59a049ce5 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Unique keys not in sort_keys. The sorting order may be unstable. Sep 27 10:05:46.476459 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6d141a0b-542c-4940-b573-d3d59a049ce5 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "GET /compute/v2.1/os-keypairs" status: 200 len: 1391 microversion: 2.2 time: 0.026046 Sep 27 10:05:46.479738 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1049/2333] 10.222.0.26 () {60 vars in 1177 bytes} [Tue Sep 27 10:05:46 2022] GET /compute/v2.1/os-keypairs => generated 1391 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:46.483606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=5,nova_api:DELETE=4,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:46.491803 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ee2b1078-0e2e-4d60-b5c7-e292d0869dca tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 204 len: 0 microversion: 2.25 time: 0.183235 Sep 27 10:05:46.495958 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1285/2334] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:05:46 2022] DELETE /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:05:46.501380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3defdbe7-aac7-4e96-a9b0-56a192351e22 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] Calling method '.version_select at 0x7f7f62def8b0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:46.503660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:46.505053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:46.516646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:46.517545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:46.518196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:46.519207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:46.519985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:46.520584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:46.544139 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3defdbe7-aac7-4e96-a9b0-56a192351e22 tempest-KeyPairsV22TestJSON-328296400 tempest-KeyPairsV22TestJSON-328296400-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1670333196" status: 204 len: 0 microversion: 2.2 time: 0.057257 Sep 27 10:05:46.549927 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1050/2335] 10.222.0.26 () {60 vars in 1261 bytes} [Tue Sep 27 10:05:46 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1670333196 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:05:46.712251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ab7e48e-4a5a-4f73-bed4-8e0b075c36fd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:46.713849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ab7e48e-4a5a-4f73-bed4-8e0b075c36fd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:46.723780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ab7e48e-4a5a-4f73-bed4-8e0b075c36fd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:46.724581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ab7e48e-4a5a-4f73-bed4-8e0b075c36fd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:46.725731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ab7e48e-4a5a-4f73-bed4-8e0b075c36fd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:46.728004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ab7e48e-4a5a-4f73-bed4-8e0b075c36fd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:46.728649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ab7e48e-4a5a-4f73-bed4-8e0b075c36fd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:46.729319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ab7e48e-4a5a-4f73-bed4-8e0b075c36fd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:47.046781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:47.058157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:47.063630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:47.066146 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-469da4e6-da66-412e-8da7-f89e69921e70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1548 microversion: 2.25 time: 0.567995 Sep 27 10:05:47.079751 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1286/2336] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:46 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1548 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:47.089361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d7da875f-54a9-4468-97d0-410b5f3c3a45 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6297f820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:47.157448 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d7da875f-54a9-4468-97d0-410b5f3c3a45 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.084692 Sep 27 10:05:47.158333 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1287/2337] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:47 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:47.369207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2ab7e48e-4a5a-4f73-bed4-8e0b075c36fd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:47.378078 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ab7e48e-4a5a-4f73-bed4-8e0b075c36fd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.678677 Sep 27 10:05:47.378676 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1051/2338] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:46 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 680 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:47.434136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8d7e81b-fd47-44e6-a84c-f62e8a5eb105 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:47.434890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c8d7e81b-fd47-44e6-a84c-f62e8a5eb105 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:47.449529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8d7e81b-fd47-44e6-a84c-f62e8a5eb105 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:47.450073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8d7e81b-fd47-44e6-a84c-f62e8a5eb105 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:47.450963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8d7e81b-fd47-44e6-a84c-f62e8a5eb105 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:47.451702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8d7e81b-fd47-44e6-a84c-f62e8a5eb105 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:47.452181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8d7e81b-fd47-44e6-a84c-f62e8a5eb105 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:47.452613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8d7e81b-fd47-44e6-a84c-f62e8a5eb105 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:48.065510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c8d7e81b-fd47-44e6-a84c-f62e8a5eb105 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:48.074666 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c8d7e81b-fd47-44e6-a84c-f62e8a5eb105 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 0.645782 Sep 27 10:05:48.086122 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1288/2339] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:47 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:48.098306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:48.099495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:48.116556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:48.117092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:48.117742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:48.118635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:48.119411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:48.120134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:48.173373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-192f6676-46f3-44ff-8db2-6f184fdd3c20 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6298f820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:48.243364 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-192f6676-46f3-44ff-8db2-6f184fdd3c20 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.074083 Sep 27 10:05:48.244201 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1289/2340] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:48 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:48.403885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9a164ad9-5ede-4c03-bc76-6487097d7e6a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:48.404980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9a164ad9-5ede-4c03-bc76-6487097d7e6a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:48.416841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a164ad9-5ede-4c03-bc76-6487097d7e6a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:48.420196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a164ad9-5ede-4c03-bc76-6487097d7e6a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:48.421006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a164ad9-5ede-4c03-bc76-6487097d7e6a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:48.425599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a164ad9-5ede-4c03-bc76-6487097d7e6a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:48.426353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a164ad9-5ede-4c03-bc76-6487097d7e6a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:48.426948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a164ad9-5ede-4c03-bc76-6487097d7e6a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:48.578072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:48.590927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:48.596948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:48.599919 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-40fedba0-e768-45e9-9802-bda888cdc372 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1548 microversion: 2.25 time: 0.507596 Sep 27 10:05:48.603240 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1052/2341] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:48 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1548 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:48.839597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7253a186-06fc-4d7e-95f8-2e0171c74c82 req-49c52c89-7cf6-4acd-bb8d-d3a2ab2427c1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "55ead6af-a0d9-4d6f-b742-4a3fd4a418ad", "name": "network-vif-plugged", "status": "completed", "tag": "a33d85f6-8d2f-4192-8ab2-06ef5d149185"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:48.862939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7253a186-06fc-4d7e-95f8-2e0171c74c82 req-49c52c89-7cf6-4acd-bb8d-d3a2ab2427c1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:48.863860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7253a186-06fc-4d7e-95f8-2e0171c74c82 req-49c52c89-7cf6-4acd-bb8d-d3a2ab2427c1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:48.864296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7253a186-06fc-4d7e-95f8-2e0171c74c82 req-49c52c89-7cf6-4acd-bb8d-d3a2ab2427c1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:48.879863 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7253a186-06fc-4d7e-95f8-2e0171c74c82 req-49c52c89-7cf6-4acd-bb8d-d3a2ab2427c1 service nova] Creating event network-vif-plugged:a33d85f6-8d2f-4192-8ab2-06ef5d149185 for instance 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad on nch2-master-1 Sep 27 10:05:48.889782 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7253a186-06fc-4d7e-95f8-2e0171c74c82 req-49c52c89-7cf6-4acd-bb8d-d3a2ab2427c1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055818 Sep 27 10:05:48.891753 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1053/2342] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:49.043068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-43597514-b3c6-4b13-bd99-b81137decfa7 req-dfa7a0ab-b46b-496b-9ea2-7c96979c34f9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9edcd7b8-f01b-4529-b9c1-3729e79ec760", "name": "network-vif-deleted", "tag": "5a182f65-e926-4e9f-97dc-7f90cd3a6dc9"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:49.076825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-43597514-b3c6-4b13-bd99-b81137decfa7 req-dfa7a0ab-b46b-496b-9ea2-7c96979c34f9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:49.077766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-43597514-b3c6-4b13-bd99-b81137decfa7 req-dfa7a0ab-b46b-496b-9ea2-7c96979c34f9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:49.078360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-43597514-b3c6-4b13-bd99-b81137decfa7 req-dfa7a0ab-b46b-496b-9ea2-7c96979c34f9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:49.104851 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-43597514-b3c6-4b13-bd99-b81137decfa7 req-dfa7a0ab-b46b-496b-9ea2-7c96979c34f9 service nova] Creating event network-vif-deleted:5a182f65-e926-4e9f-97dc-7f90cd3a6dc9 for instance 9edcd7b8-f01b-4529-b9c1-3729e79ec760 on nch1-master-1 Sep 27 10:05:49.116994 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-43597514-b3c6-4b13-bd99-b81137decfa7 req-dfa7a0ab-b46b-496b-9ea2-7c96979c34f9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093122 Sep 27 10:05:49.117953 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1054/2343] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:05:49.125541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b30a20d5-b474-4075-b333-4ebf87eb12ac tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:49.126552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b30a20d5-b474-4075-b333-4ebf87eb12ac tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:49.126692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9a164ad9-5ede-4c03-bc76-6487097d7e6a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:49.143868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b30a20d5-b474-4075-b333-4ebf87eb12ac tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:49.144415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b30a20d5-b474-4075-b333-4ebf87eb12ac tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:49.144800 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9a164ad9-5ede-4c03-bc76-6487097d7e6a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.746204 Sep 27 10:05:49.145132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b30a20d5-b474-4075-b333-4ebf87eb12ac tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:49.146019 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1290/2344] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:48 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 748 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:49.149242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b30a20d5-b474-4075-b333-4ebf87eb12ac tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:49.149930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b30a20d5-b474-4075-b333-4ebf87eb12ac tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:49.150527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b30a20d5-b474-4075-b333-4ebf87eb12ac tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:49.260007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1df7d566-819b-4d2e-a9df-32064543821a tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6c29bc10>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:49.317670 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1df7d566-819b-4d2e-a9df-32064543821a tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.062030 Sep 27 10:05:49.318269 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1291/2345] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:49 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:49.468993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b30a20d5-b474-4075-b333-4ebf87eb12ac tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:49.482175 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b30a20d5-b474-4075-b333-4ebf87eb12ac tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 0.362628 Sep 27 10:05:49.482834 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1055/2346] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:49 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:49.633220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:49.633764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:49.650267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:49.660549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:49.661196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:49.662430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:49.663151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:49.663866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:49.673226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:49.749830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:49.750474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:49.750993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:49.751724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lazy-loading 'fault' on Instance uuid 9edcd7b8-f01b-4529-b9c1-3729e79ec760 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:05:49.926506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:49.938263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:49.939715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:49.941186 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9e7c475a-61d9-4246-8be9-5fbf1548a5d9 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 200 len: 1352 microversion: 2.25 time: 0.313022 Sep 27 10:05:49.942136 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1292/2347] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:49 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 1352 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:50.170272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e9a8e7a-7ab1-442b-9d16-0e84e72e14aa tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:50.171540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6e9a8e7a-7ab1-442b-9d16-0e84e72e14aa tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:50.179113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9a8e7a-7ab1-442b-9d16-0e84e72e14aa tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:50.180182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9a8e7a-7ab1-442b-9d16-0e84e72e14aa tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:50.180916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9a8e7a-7ab1-442b-9d16-0e84e72e14aa tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:50.181972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9a8e7a-7ab1-442b-9d16-0e84e72e14aa tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:50.182600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9a8e7a-7ab1-442b-9d16-0e84e72e14aa tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:50.183088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9a8e7a-7ab1-442b-9d16-0e84e72e14aa tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:50.338142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d8119cf3-4e70-4eb1-a3e2-dc8c60a8513c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62b8dc10>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:50.391596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6e9a8e7a-7ab1-442b-9d16-0e84e72e14aa tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:50.406568 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d8119cf3-4e70-4eb1-a3e2-dc8c60a8513c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.075207 Sep 27 10:05:50.417588 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1293/2348] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:50 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:50.421584 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6e9a8e7a-7ab1-442b-9d16-0e84e72e14aa tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.255455 Sep 27 10:05:50.442796 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1056/2349] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:50 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:50.513183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fc06d7ca-4dc5-43e9-b23b-8a875fab6d51 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:50.514932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fc06d7ca-4dc5-43e9-b23b-8a875fab6d51 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:50.531184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc06d7ca-4dc5-43e9-b23b-8a875fab6d51 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:50.532912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc06d7ca-4dc5-43e9-b23b-8a875fab6d51 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:50.534339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc06d7ca-4dc5-43e9-b23b-8a875fab6d51 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:50.535325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc06d7ca-4dc5-43e9-b23b-8a875fab6d51 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:50.535908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc06d7ca-4dc5-43e9-b23b-8a875fab6d51 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:50.536434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc06d7ca-4dc5-43e9-b23b-8a875fab6d51 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:50.817315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc06d7ca-4dc5-43e9-b23b-8a875fab6d51 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:50.828421 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fc06d7ca-4dc5-43e9-b23b-8a875fab6d51 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1315 microversion: 2.1 time: 0.320637 Sep 27 10:05:50.829862 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1294/2350] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:50 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1315 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:50.971534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:50.972479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: 9edcd7b8-f01b-4529-b9c1-3729e79ec760] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:50.984854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:50.985529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:50.986149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:50.987844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:50.989264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:50.989913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] HTTP exception thrown: Instance 9edcd7b8-f01b-4529-b9c1-3729e79ec760 could not be found. Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Request method failure captured: Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 HTTP/1.1 Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsls2FdIOhvD4ZxN7OIKQmiUCELV6z27WbYJyCg5zc33cF3I2IG0F16O0kd5J_CobjZKV0lxKRF-IbyT-BYgGU09wQILwgGZLa230rmDdLmWzYS6zGfOygm-RvpfT-sJE6lwtE6xYYRfLbLdFYc5W5dGOjgOEW5aqauEbGKNWvjw5os Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.25 Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2369f5ee467643948f618bbdee837521 Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2369f5ee467643948f618bbdee837521"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2369f5ee467643948f618bbdee837521"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2369f5ee467643948f618bbdee837521"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2369f5ee467643948f618bbdee837521"}]}] Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2369f5ee467643948f618bbdee837521 Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-LiveAutoBlockMigrationV225Test-1036261617-project Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 29d2f7b0e8274c769521d13930be9825 Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617-project Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 9edcd7b8-f01b-4529-b9c1-3729e79ec760 could not be found. Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '9edcd7b8-f01b-4529-b9c1-3729e79ec760'} Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9edcd7b8-f01b-4529-b9c1-3729e79ec760 could not be found. Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:51.032921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9edcd7b8-f01b-4529-b9c1-3729e79ec760 could not be found. Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9edcd7b8-f01b-4529-b9c1-3729e79ec760 could not be found. Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Returning 404 to user: Instance 9edcd7b8-f01b-4529-b9c1-3729e79ec760 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-623f7212-400f-4f77-9d1b-0bc6c958bea2 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760" status: 404 len: 111 microversion: 2.25 time: 0.065844 Sep 27 10:05:51.037023 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1057/2351] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:50 2022] GET /compute/v2.1/servers/9edcd7b8-f01b-4529-b9c1-3729e79ec760 => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:05:51.072002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f15ccc89-969f-4f25-a756-8fb6f120db81 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:51.072602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f15ccc89-969f-4f25-a756-8fb6f120db81 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:51.079095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f15ccc89-969f-4f25-a756-8fb6f120db81 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:51.079582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f15ccc89-969f-4f25-a756-8fb6f120db81 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:51.080019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f15ccc89-969f-4f25-a756-8fb6f120db81 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:51.091152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f15ccc89-969f-4f25-a756-8fb6f120db81 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:51.091824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f15ccc89-969f-4f25-a756-8fb6f120db81 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:51.092313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f15ccc89-969f-4f25-a756-8fb6f120db81 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:51.135126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f15ccc89-969f-4f25-a756-8fb6f120db81 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:05:51.253307 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f15ccc89-969f-4f25-a756-8fb6f120db81 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 204 len: 0 microversion: 2.25 time: 0.204869 Sep 27 10:05:51.254056 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1295/2352] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:05:51 2022] DELETE /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:05:51.263451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:51.264170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:51.279442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:51.279966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:51.280504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:51.281942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:51.282716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:51.283301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:51.424760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-57d91ad9-459d-442d-83a4-95be0b230589 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6297fd30>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:51.521622 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-57d91ad9-459d-442d-83a4-95be0b230589 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.101078 Sep 27 10:05:51.527508 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1296/2353] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:51 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:51.549069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa940076-e56b-49ab-978d-24ee0b8b3f37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:51.552690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fa940076-e56b-49ab-978d-24ee0b8b3f37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:51.575199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa940076-e56b-49ab-978d-24ee0b8b3f37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:51.578015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa940076-e56b-49ab-978d-24ee0b8b3f37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:51.580351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa940076-e56b-49ab-978d-24ee0b8b3f37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:51.585557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa940076-e56b-49ab-978d-24ee0b8b3f37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:51.588080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa940076-e56b-49ab-978d-24ee0b8b3f37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:51.594694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa940076-e56b-49ab-978d-24ee0b8b3f37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:51.670882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:51.683887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:51.687738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:51.692613 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2dc91332-27d8-4388-81e3-a8aa036ecb1b tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1563 microversion: 2.25 time: 0.429912 Sep 27 10:05:51.694659 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1058/2354] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:51 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1563 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:51.854754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-85a86c70-3ec6-4741-a4a8-2a8596d2256b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:51.855974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-85a86c70-3ec6-4741-a4a8-2a8596d2256b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:51.863713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=1,nova_api:DELETE=3,nova_api:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:51.877274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85a86c70-3ec6-4741-a4a8-2a8596d2256b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:51.881477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85a86c70-3ec6-4741-a4a8-2a8596d2256b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:51.882161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85a86c70-3ec6-4741-a4a8-2a8596d2256b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:51.884608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85a86c70-3ec6-4741-a4a8-2a8596d2256b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:51.887615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85a86c70-3ec6-4741-a4a8-2a8596d2256b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:51.888229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85a86c70-3ec6-4741-a4a8-2a8596d2256b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:52.196041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-85a86c70-3ec6-4741-a4a8-2a8596d2256b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:52.200595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fa940076-e56b-49ab-978d-24ee0b8b3f37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:52.212080 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-85a86c70-3ec6-4741-a4a8-2a8596d2256b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1502 microversion: 2.1 time: 0.362477 Sep 27 10:05:52.212521 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1059/2355] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:51 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1502 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:52.218689 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fa940076-e56b-49ab-978d-24ee0b8b3f37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.689735 Sep 27 10:05:52.239091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aad5896e-44e4-47db-ac8d-ae807e92c0a6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:52.239594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aad5896e-44e4-47db-ac8d-ae807e92c0a6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:52.240033 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1297/2356] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:51 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:52.252282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aad5896e-44e4-47db-ac8d-ae807e92c0a6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:52.252797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aad5896e-44e4-47db-ac8d-ae807e92c0a6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:52.253405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aad5896e-44e4-47db-ac8d-ae807e92c0a6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:52.268316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aad5896e-44e4-47db-ac8d-ae807e92c0a6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:52.268983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aad5896e-44e4-47db-ac8d-ae807e92c0a6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:52.276561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aad5896e-44e4-47db-ac8d-ae807e92c0a6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:52.321572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:52.330927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aad5896e-44e4-47db-ac8d-ae807e92c0a6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:05:52.495809 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aad5896e-44e4-47db-ac8d-ae807e92c0a6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 204 len: 0 microversion: 2.1 time: 0.259294 Sep 27 10:05:52.501991 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1060/2357] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:05:52 2022] DELETE /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:05:52.510969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-21d0a3bc-e0de-416f-972f-11de151aa4cb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:52.512880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-21d0a3bc-e0de-416f-972f-11de151aa4cb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:52.541713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-65471716-eecf-41e4-8e8e-c3e1ffd779b3 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62e44ee0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:52.544621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21d0a3bc-e0de-416f-972f-11de151aa4cb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:52.547787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21d0a3bc-e0de-416f-972f-11de151aa4cb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:52.555835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21d0a3bc-e0de-416f-972f-11de151aa4cb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:52.561743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21d0a3bc-e0de-416f-972f-11de151aa4cb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:52.563141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21d0a3bc-e0de-416f-972f-11de151aa4cb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:52.566138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21d0a3bc-e0de-416f-972f-11de151aa4cb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:52.648778 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-65471716-eecf-41e4-8e8e-c3e1ffd779b3 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.113988 Sep 27 10:05:52.649372 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1061/2358] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:52 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:52.721783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:52.723070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:52.755841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:52.758644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:52.762235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:52.765211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:52.766876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:52.767327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:52.914687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21d0a3bc-e0de-416f-972f-11de151aa4cb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:52.929912 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-21d0a3bc-e0de-416f-972f-11de151aa4cb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1508 microversion: 2.1 time: 0.426419 Sep 27 10:05:52.930600 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1298/2359] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:52 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1508 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:52.936222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-13be9b33-20eb-44f4-bdde-c13a800f1136 req-db27e899-e342-4ac8-bd65-52bdd4fb88d6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f1ff4a24-1251-4827-955f-159c665a1d57", "name": "network-vif-deleted", "tag": "9c2668a6-7fb1-48ea-8f26-e6c77b4e01f9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:52.948108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-13be9b33-20eb-44f4-bdde-c13a800f1136 req-db27e899-e342-4ac8-bd65-52bdd4fb88d6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:52.948876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-13be9b33-20eb-44f4-bdde-c13a800f1136 req-db27e899-e342-4ac8-bd65-52bdd4fb88d6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:52.949641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-13be9b33-20eb-44f4-bdde-c13a800f1136 req-db27e899-e342-4ac8-bd65-52bdd4fb88d6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:52.967784 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-13be9b33-20eb-44f4-bdde-c13a800f1136 req-db27e899-e342-4ac8-bd65-52bdd4fb88d6 service nova] Creating event network-vif-deleted:9c2668a6-7fb1-48ea-8f26-e6c77b4e01f9 for instance f1ff4a24-1251-4827-955f-159c665a1d57 on nch1-master-1 Sep 27 10:05:52.972628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:52.978404 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-13be9b33-20eb-44f4-bdde-c13a800f1136 req-db27e899-e342-4ac8-bd65-52bdd4fb88d6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046363 Sep 27 10:05:52.979027 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1299/2360] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:52.988181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:52.990070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '29d2f7b0e8274c769521d13930be9825', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2369f5ee467643948f618bbdee837521', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:52.991285 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-04420d12-a347-404a-8a2c-efd40bd8749c tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 200 len: 1521 microversion: 2.25 time: 0.274379 Sep 27 10:05:52.997407 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1062/2361] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:52 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 1521 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:05:53.168878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-049f2da3-a5e6-439a-a627-2ec9bb264ccd req-c079db5f-9c3f-4c8f-80ea-c8893684cd6f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191", "name": "network-vif-plugged", "status": "completed", "tag": "16a06bd9-639b-4f11-8e0c-4ca53b9b60d5"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:53.180059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-049f2da3-a5e6-439a-a627-2ec9bb264ccd req-c079db5f-9c3f-4c8f-80ea-c8893684cd6f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:53.180767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-049f2da3-a5e6-439a-a627-2ec9bb264ccd req-c079db5f-9c3f-4c8f-80ea-c8893684cd6f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:53.181248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-049f2da3-a5e6-439a-a627-2ec9bb264ccd req-c079db5f-9c3f-4c8f-80ea-c8893684cd6f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:53.194039 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-049f2da3-a5e6-439a-a627-2ec9bb264ccd req-c079db5f-9c3f-4c8f-80ea-c8893684cd6f service nova] Creating event network-vif-plugged:16a06bd9-639b-4f11-8e0c-4ca53b9b60d5 for instance e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 on nch2-master-1 Sep 27 10:05:53.205164 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-049f2da3-a5e6-439a-a627-2ec9bb264ccd req-c079db5f-9c3f-4c8f-80ea-c8893684cd6f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041439 Sep 27 10:05:53.206454 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1300/2362] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:53.247675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-22648934-a8a1-4097-a594-35d2e682dbf1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:53.248314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-22648934-a8a1-4097-a594-35d2e682dbf1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:53.255412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22648934-a8a1-4097-a594-35d2e682dbf1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:53.255941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22648934-a8a1-4097-a594-35d2e682dbf1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:53.256378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22648934-a8a1-4097-a594-35d2e682dbf1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:53.257140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22648934-a8a1-4097-a594-35d2e682dbf1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:53.257605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22648934-a8a1-4097-a594-35d2e682dbf1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:53.258106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22648934-a8a1-4097-a594-35d2e682dbf1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:53.667468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67b91ed8-a789-403f-a9b6-8e402abff271 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6297fe50>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:53.748026 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67b91ed8-a789-403f-a9b6-8e402abff271 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.085717 Sep 27 10:05:53.753995 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1301/2363] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:53 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:53.960541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-816f52c7-8907-4dc2-9794-13b14be50a38 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:53.963098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-816f52c7-8907-4dc2-9794-13b14be50a38 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:53.991976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816f52c7-8907-4dc2-9794-13b14be50a38 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:53.993232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816f52c7-8907-4dc2-9794-13b14be50a38 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:53.993690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816f52c7-8907-4dc2-9794-13b14be50a38 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:54.004287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816f52c7-8907-4dc2-9794-13b14be50a38 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:54.004627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816f52c7-8907-4dc2-9794-13b14be50a38 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:54.005097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816f52c7-8907-4dc2-9794-13b14be50a38 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:54.128906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-22648934-a8a1-4097-a594-35d2e682dbf1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:54.143350 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-22648934-a8a1-4097-a594-35d2e682dbf1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.899858 Sep 27 10:05:54.159794 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1063/2364] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:53 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 916 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:54.173250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:54.173874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] [instance: f1ff4a24-1251-4827-955f-159c665a1d57] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:54.200282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:54.200893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:54.201359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:54.202259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:54.202854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:54.203711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:54.279476 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] HTTP exception thrown: Instance f1ff4a24-1251-4827-955f-159c665a1d57 could not be found. Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Request method failure captured: Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 HTTP/1.1 Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsls2FdIOhvD4ZxN7OIKQmiUCELV6z27WbYJyCg5zc33cF3I2IG0F16O0kd5J_CobjZKV0lxKRF-IbyT-BYgGU09wQILwgGZLa230rmDdLmWzYS6zGfOygm-RvpfT-sJE6lwtE6xYYRfLbLdFYc5W5dGOjgOEW5aqauEbGKNWvjw5os Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.25 Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2369f5ee467643948f618bbdee837521 Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2369f5ee467643948f618bbdee837521"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2369f5ee467643948f618bbdee837521"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2369f5ee467643948f618bbdee837521"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2369f5ee467643948f618bbdee837521"}]}] Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2369f5ee467643948f618bbdee837521 Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617 Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-LiveAutoBlockMigrationV225Test-1036261617-project Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 29d2f7b0e8274c769521d13930be9825 Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-LiveAutoBlockMigrationV225Test-1036261617-project Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance f1ff4a24-1251-4827-955f-159c665a1d57 could not be found. Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'f1ff4a24-1251-4827-955f-159c665a1d57'} Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f1ff4a24-1251-4827-955f-159c665a1d57 could not be found. Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f1ff4a24-1251-4827-955f-159c665a1d57 could not be found. Sep 27 10:05:54.284713 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f1ff4a24-1251-4827-955f-159c665a1d57 could not be found. Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] Returning 404 to user: Instance f1ff4a24-1251-4827-955f-159c665a1d57 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b3d4b1dd-0911-4abc-9ebb-390b3f8edd70 tempest-LiveAutoBlockMigrationV225Test-1036261617 tempest-LiveAutoBlockMigrationV225Test-1036261617-project] 10.222.0.26 "GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57" status: 404 len: 111 microversion: 2.25 time: 0.125369 Sep 27 10:05:54.288481 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1064/2365] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:05:54 2022] GET /compute/v2.1/servers/f1ff4a24-1251-4827-955f-159c665a1d57 => generated 111 bytes in 127 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:05:54.683117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-816f52c7-8907-4dc2-9794-13b14be50a38 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:54.699343 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-816f52c7-8907-4dc2-9794-13b14be50a38 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1508 microversion: 2.1 time: 0.745832 Sep 27 10:05:54.701466 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1302/2366] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:53 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1508 bytes in 750 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:54.770506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5311a7d-37ed-43fe-b34f-06615fdfa9fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62defee0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:54.882663 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c5311a7d-37ed-43fe-b34f-06615fdfa9fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.116173 Sep 27 10:05:54.887518 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1065/2367] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:54 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:55.175779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2ef5126f-5a38-424f-b19f-00bd3b20fa75 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:55.176582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2ef5126f-5a38-424f-b19f-00bd3b20fa75 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:55.207694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef5126f-5a38-424f-b19f-00bd3b20fa75 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:55.208222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef5126f-5a38-424f-b19f-00bd3b20fa75 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:55.208687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef5126f-5a38-424f-b19f-00bd3b20fa75 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:55.221850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef5126f-5a38-424f-b19f-00bd3b20fa75 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:55.222444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef5126f-5a38-424f-b19f-00bd3b20fa75 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:55.224060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef5126f-5a38-424f-b19f-00bd3b20fa75 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:55.663556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2ef5126f-5a38-424f-b19f-00bd3b20fa75 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:55.683080 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2ef5126f-5a38-424f-b19f-00bd3b20fa75 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1315 microversion: 2.1 time: 0.513156 Sep 27 10:05:55.698404 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1303/2368] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:55 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1315 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:55.727912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:55.734808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5571b1e-6274-4dfb-b6de-a4659db043cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:55.735984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5571b1e-6274-4dfb-b6de-a4659db043cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:55.745758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5571b1e-6274-4dfb-b6de-a4659db043cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:55.746755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5571b1e-6274-4dfb-b6de-a4659db043cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:55.747498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5571b1e-6274-4dfb-b6de-a4659db043cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:55.748495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5571b1e-6274-4dfb-b6de-a4659db043cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:55.749386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5571b1e-6274-4dfb-b6de-a4659db043cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:55.750048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5571b1e-6274-4dfb-b6de-a4659db043cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:55.903325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6614fb78-b01e-4d60-bec1-e75fed580023 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62b93af0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:55.989622 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6614fb78-b01e-4d60-bec1-e75fed580023 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.090206 Sep 27 10:05:55.990650 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1304/2369] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:55 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:56.663778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5571b1e-6274-4dfb-b6de-a4659db043cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:56.675896 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5571b1e-6274-4dfb-b6de-a4659db043cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1508 microversion: 2.1 time: 0.949631 Sep 27 10:05:56.696790 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1066/2370] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:55 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1508 bytes in 971 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:56.709867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=40 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:05:56.717098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c3f73ab-6c83-45bd-bff5-63fb6603957d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:56.718708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2c3f73ab-6c83-45bd-bff5-63fb6603957d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:56.727734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3f73ab-6c83-45bd-bff5-63fb6603957d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:56.728626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3f73ab-6c83-45bd-bff5-63fb6603957d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:56.729309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3f73ab-6c83-45bd-bff5-63fb6603957d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:56.730465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3f73ab-6c83-45bd-bff5-63fb6603957d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:56.731153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3f73ab-6c83-45bd-bff5-63fb6603957d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:56.731797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3f73ab-6c83-45bd-bff5-63fb6603957d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:57.006740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-183e32ed-c2aa-41b7-9373-780b735d6409 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f637f3430>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:57.077856 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-183e32ed-c2aa-41b7-9373-780b735d6409 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.075800 Sep 27 10:05:57.093370 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1067/2371] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:57 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:57.492472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-09ede24f-12c0-41d3-b2a8-18baaa4e9fbb req-0e90ef13-712a-458d-a393-548c1084b383 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "35973885-926e-4fa3-80a3-0df40511ebd2", "name": "network-vif-deleted", "tag": "8ac1d5d0-a175-4922-8f13-aacbb8c5d6b0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:57.508372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-09ede24f-12c0-41d3-b2a8-18baaa4e9fbb req-0e90ef13-712a-458d-a393-548c1084b383 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:57.508885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-09ede24f-12c0-41d3-b2a8-18baaa4e9fbb req-0e90ef13-712a-458d-a393-548c1084b383 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:57.509319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-09ede24f-12c0-41d3-b2a8-18baaa4e9fbb req-0e90ef13-712a-458d-a393-548c1084b383 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:57.551938 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-09ede24f-12c0-41d3-b2a8-18baaa4e9fbb req-0e90ef13-712a-458d-a393-548c1084b383 service nova] Creating event network-vif-deleted:8ac1d5d0-a175-4922-8f13-aacbb8c5d6b0 for instance 35973885-926e-4fa3-80a3-0df40511ebd2 on nch2-master-1 Sep 27 10:05:57.565929 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-09ede24f-12c0-41d3-b2a8-18baaa4e9fbb req-0e90ef13-712a-458d-a393-548c1084b383 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077249 Sep 27 10:05:57.566861 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1068/2372] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:05:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:05:57.714020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:57.715100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:57.746787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:57.749252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:57.754072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:57.755189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:57.761313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:57.763444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:57.894089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:57.897384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:57.898078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:57.899348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lazy-loading 'fault' on Instance uuid 35973885-926e-4fa3-80a3-0df40511ebd2 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:05:57.910174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2c3f73ab-6c83-45bd-bff5-63fb6603957d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:57.928467 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c3f73ab-6c83-45bd-bff5-63fb6603957d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1503 microversion: 2.1 time: 1.220540 Sep 27 10:05:57.937713 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1305/2373] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:56 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1503 bytes in 1231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:57.969177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-47523505-ca50-48cd-abbc-8b0b4b7b9628 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1644292222"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:05:57.978167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-47523505-ca50-48cd-abbc-8b0b4b7b9628 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:58.011762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47523505-ca50-48cd-abbc-8b0b4b7b9628 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:58.013283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47523505-ca50-48cd-abbc-8b0b4b7b9628 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:58.013888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47523505-ca50-48cd-abbc-8b0b4b7b9628 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:58.015813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47523505-ca50-48cd-abbc-8b0b4b7b9628 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:58.018208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47523505-ca50-48cd-abbc-8b0b4b7b9628 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:58.021814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47523505-ca50-48cd-abbc-8b0b4b7b9628 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:58.675953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:05:58.678847 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.network.security_group_api [None req-47523505-ca50-48cd-abbc-8b0b4b7b9628 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Adding security group 5a9156d9-5a31-4358-a05a-d0676691bcb7 to port 16a06bd9-639b-4f11-8e0c-4ca53b9b60d5 Sep 27 10:05:58.699548 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9c7da6b3-e366-456d-a1c1-902cefd50365 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 200 len: 1305 microversion: 2.1 time: 0.995910 Sep 27 10:05:58.702480 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1069/2374] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:57 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 1305 bytes in 1003 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:05:58.717837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6511b0d2-a807-4b42-ac0b-bfff4b4f2955 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6377b9d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:58.831260 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6511b0d2-a807-4b42-ac0b-bfff4b4f2955 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.125630 Sep 27 10:05:58.832154 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1070/2375] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:58 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:05:59.726194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:05:59.729625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:05:59.792016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:59.797485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:59.803017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:59.807625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:05:59.826101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:05:59.827013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:05:59.878219 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 HTTP/1.1 Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '35973885-926e-4fa3-80a3-0df40511ebd2'} Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:05:59.881014 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:59.884914 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:05:59.884914 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:05:59.884914 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:05:59.884914 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:05:59.884914 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:05:59.884914 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:05:59.884914 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:05:59.892814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:05:59.901294 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0fbea77a-8a5e-488e-8050-8e7b97efd526 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 404 len: 111 microversion: 2.1 time: 0.178925 Sep 27 10:05:59.908462 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1071/2376] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:05:59 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 111 bytes in 187 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:05:59.916361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e971b5ee-8c3c-489b-ba98-1445f7cc8d17 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f637f39d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:00.052926 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e971b5ee-8c3c-489b-ba98-1445f7cc8d17 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.142164 Sep 27 10:06:00.054193 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1072/2377] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:05:59 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:00.283106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1861606860", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "0117c46b-39f2-4cf5-9f8a-7710125bdf55"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:00.476118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] validate_networks() for [('0117c46b-39f2-4cf5-9f8a-7710125bdf55', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:06:00.643555 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-47523505-ca50-48cd-abbc-8b0b4b7b9628 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191/action" status: 202 len: 0 microversion: 2.1 time: 2.679263 Sep 27 10:06:00.647501 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1306/2378] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:05:57 2022] POST /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191/action => generated 0 bytes in 2681 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:06:00.653940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-52c630db-c527-4088-834b-6de9533804c7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-376389634"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:00.654524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-52c630db-c527-4088-834b-6de9533804c7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:00.676737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52c630db-c527-4088-834b-6de9533804c7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:00.677246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52c630db-c527-4088-834b-6de9533804c7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:00.677698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52c630db-c527-4088-834b-6de9533804c7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:00.678430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52c630db-c527-4088-834b-6de9533804c7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:00.678894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52c630db-c527-4088-834b-6de9533804c7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:00.679326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52c630db-c527-4088-834b-6de9533804c7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:01.016155 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.network.security_group_api [None req-52c630db-c527-4088-834b-6de9533804c7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Adding security group abc5efc2-4989-4a63-9cad-dc80766b95b8 to port 16a06bd9-639b-4f11-8e0c-4ca53b9b60d5 Sep 27 10:06:03.193503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:06:03.194503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:06:03.195879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:06:03.196491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:06:03.197090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:06:03.240038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:03.240988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:03.243098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:03.245792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:03.246444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:03.247512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:03.325245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:06:03.341687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for user 7f7c5adbac4f48b6bc141e0f888f2496 and project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:06:03.390657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:06:03.394929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:06:03.547328 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0f6acb07-78a2-4c50-9b7d-1c491cde6a1b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.268941 Sep 27 10:06:03.548581 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1073/2379] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:06:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 3271 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:06:03.553797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2788b791-e776-4c09-93bf-19d328c24c79 req-e8fda682-2ca3-4074-b7cd-fce78333b372 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191", "tag": "16a06bd9-639b-4f11-8e0c-4ca53b9b60d5"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:03.580323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2788b791-e776-4c09-93bf-19d328c24c79 req-e8fda682-2ca3-4074-b7cd-fce78333b372 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:03.581140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2788b791-e776-4c09-93bf-19d328c24c79 req-e8fda682-2ca3-4074-b7cd-fce78333b372 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:03.581737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2788b791-e776-4c09-93bf-19d328c24c79 req-e8fda682-2ca3-4074-b7cd-fce78333b372 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:03.617820 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2788b791-e776-4c09-93bf-19d328c24c79 req-e8fda682-2ca3-4074-b7cd-fce78333b372 service nova] Creating event network-changed:16a06bd9-639b-4f11-8e0c-4ca53b9b60d5 for instance e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 on nch2-master-1 Sep 27 10:06:03.632515 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2788b791-e776-4c09-93bf-19d328c24c79 req-e8fda682-2ca3-4074-b7cd-fce78333b372 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082448 Sep 27 10:06:03.636939 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1074/2380] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:03.647970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6eb31b1c-c82f-4093-912a-68af90973ee0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62defc10>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:03.841030 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6eb31b1c-c82f-4093-912a-68af90973ee0 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.201286 Sep 27 10:06:03.841644 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1075/2381] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:03 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:04.210027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-UnshelveToHostMultiNodesTest-server-997699116", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": "none"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:04.299571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:06:04.301074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:06:04.301775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:06:04.302457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:06:04.303778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:06:04.322729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:04.323779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:04.324558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:04.325753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:04.326523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:04.327163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:04.375813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Getting quotas for project 7fb866af50c14a0393c03c1463330238. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:06:04.389611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Getting quotas for user f7bf3532492e43719ce03e29f9740b5f and project 7fb866af50c14a0393c03c1463330238. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:06:04.417665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:06:04.420566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:06:04.531694 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3fe9a2e1-5290-4395-ba63-9da80a218791 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.91 time: 0.678281 Sep 27 10:06:04.539200 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1076/2382] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:06:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 697 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:06:04.581338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bef7ebef-c1b7-4179-b3ed-19152c17e986 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:04.582276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bef7ebef-c1b7-4179-b3ed-19152c17e986 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:04.810045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bef7ebef-c1b7-4179-b3ed-19152c17e986 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:04.836507 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bef7ebef-c1b7-4179-b3ed-19152c17e986 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1215 microversion: 2.1 time: 0.281668 Sep 27 10:06:04.837824 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1077/2383] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:04 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1215 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:04.844427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5d387b50-69ff-497d-9774-a02d4d9b1c57 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:04.847459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5d387b50-69ff-497d-9774-a02d4d9b1c57 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:05.077786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5d387b50-69ff-497d-9774-a02d4d9b1c57 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:05.098199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5d387b50-69ff-497d-9774-a02d4d9b1c57 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:05.101046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5d387b50-69ff-497d-9774-a02d4d9b1c57 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:05.141019 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5d387b50-69ff-497d-9774-a02d4d9b1c57 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1452 microversion: 2.91 time: 0.301409 Sep 27 10:06:05.159772 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1078/2384] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:04 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1452 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:05.183165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b2826436-ff3e-4526-85e7-76848296952e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62b8b040>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:05.288274 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b2826436-ff3e-4526-85e7-76848296952e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.115510 Sep 27 10:06:05.290745 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1079/2385] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:05 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:05.871555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-46ec8cb4-2990-4140-8bf1-d8abda340c09 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:05.872224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-46ec8cb4-2990-4140-8bf1-d8abda340c09 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:05.920604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ec8cb4-2990-4140-8bf1-d8abda340c09 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:05.923030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ec8cb4-2990-4140-8bf1-d8abda340c09 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:05.940834 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-52c630db-c527-4088-834b-6de9533804c7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191/action" status: 202 len: 0 microversion: 2.1 time: 5.291915 Sep 27 10:06:05.941759 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1307/2386] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:06:00 2022] POST /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191/action => generated 0 bytes in 5293 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:06:05.950829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b656b826-1d58-42c5-8441-91b59ceb0ab3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:05.951331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b656b826-1d58-42c5-8441-91b59ceb0ab3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:05.953151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ec8cb4-2990-4140-8bf1-d8abda340c09 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:05.958232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ec8cb4-2990-4140-8bf1-d8abda340c09 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:05.958820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ec8cb4-2990-4140-8bf1-d8abda340c09 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:05.959280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46ec8cb4-2990-4140-8bf1-d8abda340c09 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:05.989309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b656b826-1d58-42c5-8441-91b59ceb0ab3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:05.989726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b656b826-1d58-42c5-8441-91b59ceb0ab3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:05.990169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b656b826-1d58-42c5-8441-91b59ceb0ab3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:06.002942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b656b826-1d58-42c5-8441-91b59ceb0ab3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:06.003559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b656b826-1d58-42c5-8441-91b59ceb0ab3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:06.004011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b656b826-1d58-42c5-8441-91b59ceb0ab3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:06.470437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-46ec8cb4-2990-4140-8bf1-d8abda340c09 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:06.480463 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-46ec8cb4-2990-4140-8bf1-d8abda340c09 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1267 microversion: 2.1 time: 0.616068 Sep 27 10:06:06.508686 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1080/2387] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:05 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1267 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:06.529905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:06.530868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:06.553138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:06.554030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:06.555562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:06.558991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:06.559792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:06.560449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:07.225713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:07.239846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:07.245366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:07.262096 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1df86ba8-406d-47f6-889f-e1767e5012d0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1504 microversion: 2.91 time: 0.749749 Sep 27 10:06:07.270449 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1081/2388] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:06 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1504 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:07.275810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8464b96e-34f1-450d-870e-eece3fd791f9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62bd14c0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:07.376247 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8464b96e-34f1-450d-870e-eece3fd791f9 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.104589 Sep 27 10:06:07.377431 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1082/2389] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:07 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:07.383630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-28c9b4ba-ede4-4ea2-b728-51d12543c6df req-278f12b0-d6af-4997-9f67-6b620ecdccd9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191", "tag": "16a06bd9-639b-4f11-8e0c-4ca53b9b60d5"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:07.398585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-28c9b4ba-ede4-4ea2-b728-51d12543c6df req-278f12b0-d6af-4997-9f67-6b620ecdccd9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:07.399366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-28c9b4ba-ede4-4ea2-b728-51d12543c6df req-278f12b0-d6af-4997-9f67-6b620ecdccd9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:07.400736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-28c9b4ba-ede4-4ea2-b728-51d12543c6df req-278f12b0-d6af-4997-9f67-6b620ecdccd9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:07.422667 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-28c9b4ba-ede4-4ea2-b728-51d12543c6df req-278f12b0-d6af-4997-9f67-6b620ecdccd9 service nova] Creating event network-changed:16a06bd9-639b-4f11-8e0c-4ca53b9b60d5 for instance e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 on nch2-master-1 Sep 27 10:06:07.448146 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-28c9b4ba-ede4-4ea2-b728-51d12543c6df req-278f12b0-d6af-4997-9f67-6b620ecdccd9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069166 Sep 27 10:06:07.449848 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1083/2390] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:07.518839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b130b158-0e62-4a4e-bfd0-be9ebfd74950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:07.520249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b130b158-0e62-4a4e-bfd0-be9ebfd74950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:07.533154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b130b158-0e62-4a4e-bfd0-be9ebfd74950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:07.534567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b130b158-0e62-4a4e-bfd0-be9ebfd74950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:07.535248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b130b158-0e62-4a4e-bfd0-be9ebfd74950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:07.536701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b130b158-0e62-4a4e-bfd0-be9ebfd74950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:07.538049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b130b158-0e62-4a4e-bfd0-be9ebfd74950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:07.543833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b130b158-0e62-4a4e-bfd0-be9ebfd74950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:07.677269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b130b158-0e62-4a4e-bfd0-be9ebfd74950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:07.717469 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b130b158-0e62-4a4e-bfd0-be9ebfd74950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1273 microversion: 2.1 time: 0.201597 Sep 27 10:06:07.758708 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1084/2391] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:07 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1273 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:08.299843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:08.301422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:08.317994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:08.319711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:08.321544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:08.323511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:08.325231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:08.325886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:08.453568 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b656b826-1d58-42c5-8441-91b59ceb0ab3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191/os-security-groups" status: 200 len: 1158 microversion: 2.1 time: 2.507064 Sep 27 10:06:08.455701 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1308/2392] 10.222.0.26 () {58 vars in 1293 bytes} [Tue Sep 27 10:06:05 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191/os-security-groups => generated 1158 bytes in 2510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:06:08.460716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e9ab3ee-8986-4828-b600-98d88df0eb23 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6295fdc0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:08.527815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:08.542633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:08.579273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:08.580116 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7e9ab3ee-8986-4828-b600-98d88df0eb23 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.123061 Sep 27 10:06:08.591386 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1309/2393] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:08 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:08.593122 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-61d67d93-f2fc-4cc3-9704-f2b58f3ff07f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1510 microversion: 2.91 time: 0.295014 Sep 27 10:06:08.596633 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1085/2394] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:08 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1510 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:06:08.652214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f6dd112e-b01b-4d34-a834-e846c85b4d2b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'create', calling method: .version_select at 0x7f7f6295f430>, body: {"security_group": {"name": " tempest-securitygroup -1908801638 ", "description": "tempest-description-1367515395"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:08.747745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5e41c811-0fd7-4a24-ba7e-506fbf350e61 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:08.748627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5e41c811-0fd7-4a24-ba7e-506fbf350e61 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:08.762312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e41c811-0fd7-4a24-ba7e-506fbf350e61 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:08.766459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e41c811-0fd7-4a24-ba7e-506fbf350e61 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:08.767044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e41c811-0fd7-4a24-ba7e-506fbf350e61 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:08.769140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e41c811-0fd7-4a24-ba7e-506fbf350e61 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:08.772440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e41c811-0fd7-4a24-ba7e-506fbf350e61 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:08.773062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e41c811-0fd7-4a24-ba7e-506fbf350e61 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:09.064398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5e41c811-0fd7-4a24-ba7e-506fbf350e61 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:09.083119 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5e41c811-0fd7-4a24-ba7e-506fbf350e61 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1273 microversion: 2.1 time: 0.339389 Sep 27 10:06:09.102907 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1086/2395] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:08 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1273 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:09.602552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f0796cb7-efaa-42c0-bb82-6418ef6e545b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f63641e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:09.706627 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f0796cb7-efaa-42c0-bb82-6418ef6e545b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.112043 Sep 27 10:06:09.711889 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1087/2396] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:09 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:09.721245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:09.723931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:09.742365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:09.743163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:09.743714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:09.744450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:09.744950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:09.745383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:09.857799 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f6dd112e-b01b-4d34-a834-e846c85b4d2b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 224 microversion: 2.1 time: 1.264981 Sep 27 10:06:09.859020 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1310/2397] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:06:08 2022] POST /compute/v2.1/os-security-groups => generated 224 bytes in 1267 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:09.875901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d9f1b13-1789-44bf-94b8-df8b6b2b3dae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62bb13a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:09.932877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:09.942344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:09.943777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:09.964280 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f6a9d406-98c4-48bd-b8fb-9d5cf6616a7b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1531 microversion: 2.91 time: 0.250947 Sep 27 10:06:09.965359 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1088/2398] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:09 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1531 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:10.057769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9a3cf024-f025-4bdf-afa5-c198b267575b tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:10.065756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9a3cf024-f025-4bdf-afa5-c198b267575b tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:10.075345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a3cf024-f025-4bdf-afa5-c198b267575b tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:10.077116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a3cf024-f025-4bdf-afa5-c198b267575b tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:10.078435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a3cf024-f025-4bdf-afa5-c198b267575b tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:10.079417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a3cf024-f025-4bdf-afa5-c198b267575b tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:10.080038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a3cf024-f025-4bdf-afa5-c198b267575b tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:10.080646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a3cf024-f025-4bdf-afa5-c198b267575b tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:10.259866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-9a3cf024-f025-4bdf-afa5-c198b267575b tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lazy-loading 'services' on Instance uuid 42709739-135c-4adb-aa6c-4fc216a5d618 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:06:10.314624 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9a3cf024-f025-4bdf-afa5-c198b267575b tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1936 microversion: 2.91 time: 0.322239 Sep 27 10:06:10.317271 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1089/2399] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:09 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1936 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:06:10.329900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6047b4f9-eb73-4739-8f1d-b8038a2a23f3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:10.330899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6047b4f9-eb73-4739-8f1d-b8038a2a23f3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:10.344933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6047b4f9-eb73-4739-8f1d-b8038a2a23f3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:10.347935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6047b4f9-eb73-4739-8f1d-b8038a2a23f3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:10.350145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6047b4f9-eb73-4739-8f1d-b8038a2a23f3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:10.354649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6047b4f9-eb73-4739-8f1d-b8038a2a23f3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:10.355366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6047b4f9-eb73-4739-8f1d-b8038a2a23f3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:10.356038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6047b4f9-eb73-4739-8f1d-b8038a2a23f3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:10.394854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:10.515278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6047b4f9-eb73-4739-8f1d-b8038a2a23f3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:10.541474 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6047b4f9-eb73-4739-8f1d-b8038a2a23f3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1273 microversion: 2.1 time: 0.222977 Sep 27 10:06:10.542094 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1090/2400] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:10 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1273 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:10.565879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7156f9e8-176a-4b11-a6f7-29deade78767 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:10.566836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7156f9e8-176a-4b11-a6f7-29deade78767 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:10.584211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7156f9e8-176a-4b11-a6f7-29deade78767 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:10.586761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7156f9e8-176a-4b11-a6f7-29deade78767 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:10.589124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7156f9e8-176a-4b11-a6f7-29deade78767 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:10.590088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7156f9e8-176a-4b11-a6f7-29deade78767 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:10.591338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7156f9e8-176a-4b11-a6f7-29deade78767 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:10.591946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7156f9e8-176a-4b11-a6f7-29deade78767 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:10.714074 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5d9f1b13-1789-44bf-94b8-df8b6b2b3dae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups/5e9c7520-4b04-421c-973d-54c7ff2bbd9e" status: 200 len: 224 microversion: 2.1 time: 0.842741 Sep 27 10:06:10.725395 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1311/2401] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:06:09 2022] GET /compute/v2.1/os-security-groups/5e9c7520-4b04-421c-973d-54c7ff2bbd9e => generated 224 bytes in 855 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:10.746136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67ac3014-ede7-4607-b416-ce26342de21d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6295fdc0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:10.853603 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67ac3014-ede7-4607-b416-ce26342de21d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.126950 Sep 27 10:06:10.854499 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1312/2402] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:10 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:10.871318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e240212-e857-458a-8ad7-9d42109df651 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62bb1040>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:10.891770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-7156f9e8-176a-4b11-a6f7-29deade78767 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lazy-loading 'services' on Instance uuid 42709739-135c-4adb-aa6c-4fc216a5d618 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:06:10.979998 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7156f9e8-176a-4b11-a6f7-29deade78767 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1936 microversion: 2.91 time: 0.434839 Sep 27 10:06:10.981541 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1091/2403] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:10 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1936 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:11.022700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4a886732-aaed-476f-a026-b3c6b1923447 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:11.040396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a886732-aaed-476f-a026-b3c6b1923447 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:11.040956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a886732-aaed-476f-a026-b3c6b1923447 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:11.041835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a886732-aaed-476f-a026-b3c6b1923447 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:11.042720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a886732-aaed-476f-a026-b3c6b1923447 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:11.044692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a886732-aaed-476f-a026-b3c6b1923447 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:11.045154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a886732-aaed-476f-a026-b3c6b1923447 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:11.073675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=1,nova_api:INSERT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:11.107174 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4a886732-aaed-476f-a026-b3c6b1923447 tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 498 microversion: 2.91 time: 0.096346 Sep 27 10:06:11.111454 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1092/2404] 10.222.0.26 () {60 vars in 1217 bytes} [Tue Sep 27 10:06:11 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 498 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:06:11.129735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cb07c996-6d33-4967-9bdc-1395e2be5384 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:11.130133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cb07c996-6d33-4967-9bdc-1395e2be5384 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:11.142950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb07c996-6d33-4967-9bdc-1395e2be5384 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:11.145658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb07c996-6d33-4967-9bdc-1395e2be5384 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:11.146171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb07c996-6d33-4967-9bdc-1395e2be5384 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:11.147048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb07c996-6d33-4967-9bdc-1395e2be5384 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:11.147411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb07c996-6d33-4967-9bdc-1395e2be5384 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:11.147843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb07c996-6d33-4967-9bdc-1395e2be5384 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:11.177771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-cb07c996-6d33-4967-9bdc-1395e2be5384 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lazy-loading 'flavor' on Instance uuid 42709739-135c-4adb-aa6c-4fc216a5d618 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:06:11.688066 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cb07c996-6d33-4967-9bdc-1395e2be5384 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "POST /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618/action" status: 202 len: 0 microversion: 2.91 time: 0.549362 Sep 27 10:06:11.689641 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1093/2405] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:06:11 2022] POST /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618/action => generated 0 bytes in 574 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:06:11.706937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3938e0e0-6a4b-43df-b7bd-21f1012a666a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:11.710353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3938e0e0-6a4b-43df-b7bd-21f1012a666a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:11.738515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3938e0e0-6a4b-43df-b7bd-21f1012a666a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:11.739060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3938e0e0-6a4b-43df-b7bd-21f1012a666a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:11.739511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3938e0e0-6a4b-43df-b7bd-21f1012a666a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:11.741917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3938e0e0-6a4b-43df-b7bd-21f1012a666a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:11.742942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3938e0e0-6a4b-43df-b7bd-21f1012a666a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:11.743412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3938e0e0-6a4b-43df-b7bd-21f1012a666a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:13.235480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3938e0e0-6a4b-43df-b7bd-21f1012a666a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:13.249581 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3938e0e0-6a4b-43df-b7bd-21f1012a666a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1315 microversion: 2.1 time: 1.558535 Sep 27 10:06:13.252594 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1094/2406] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:11 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1315 bytes in 1562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:13.270521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:13.280942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:13.301956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:13.302462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:13.302900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:13.303630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:13.304086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:13.321280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:13.418800 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e240212-e857-458a-8ad7-9d42109df651 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/5e9c7520-4b04-421c-973d-54c7ff2bbd9e" status: 202 len: 0 microversion: 2.1 time: 2.553579 Sep 27 10:06:13.420146 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1313/2407] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:06:10 2022] DELETE /compute/v2.1/os-security-groups/5e9c7520-4b04-421c-973d-54c7ff2bbd9e => generated 0 bytes in 2556 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:06:13.430406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86d39c77-6647-4db7-9a3a-e649b158cf76 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f629ad9d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:13.514839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:13.531498 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86d39c77-6647-4db7-9a3a-e649b158cf76 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.106521 Sep 27 10:06:13.532082 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1314/2408] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:13 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:13.541681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:13.545927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:13.567243 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7fa5043e-ddbd-48b8-b46b-ef1cd18ce5ba tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.312077 Sep 27 10:06:13.569049 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1095/2409] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:13 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:13.583447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba768e5c-5c0d-495e-afa0-9db4eb00aa69 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62b8b550>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:13.665736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerNegativeTestJSON-server-67188678", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "3757a0b4-fb97-4218-a71c-9743174aeb68"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:13.798038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] validate_networks() for [('3757a0b4-fb97-4218-a71c-9743174aeb68', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:06:13.810013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG neutronclient.v2_0.client [None req-ba768e5c-5c0d-495e-afa0-9db4eb00aa69 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist", "detail": ""}} {{(pid=145969) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:06:13.810639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.security_group_api [None req-ba768e5c-5c0d-495e-afa0-9db4eb00aa69 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Neutron security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e not found {{(pid=145969) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:06:13.811177 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-ba768e5c-5c0d-495e-afa0-9db4eb00aa69 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:06:13.811177 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-3be858e6-781f-4e4a-9821-bacc26e4badb'] Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba768e5c-5c0d-495e-afa0-9db4eb00aa69 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-security-groups/5e9c7520-4b04-421c-973d-54c7ff2bbd9e HTTP/1.1 Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: exception: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-3be858e6-781f-4e4a-9821-bacc26e4badb'] Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '5e9c7520-4b04-421c-973d-54c7ff2bbd9e'} Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:06:13.814877 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3be858e6-781f-4e4a-9821-bacc26e4badb'] Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3be858e6-781f-4e4a-9821-bacc26e4badb'] Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3be858e6-781f-4e4a-9821-bacc26e4badb'] Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3be858e6-781f-4e4a-9821-bacc26e4badb'] Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba768e5c-5c0d-495e-afa0-9db4eb00aa69 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-3be858e6-781f-4e4a-9821-bacc26e4badb'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ba768e5c-5c0d-495e-afa0-9db4eb00aa69 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups/5e9c7520-4b04-421c-973d-54c7ff2bbd9e" status: 404 len: 194 microversion: 2.1 time: 0.244891 Sep 27 10:06:13.818752 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1096/2410] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:06:13 2022] GET /compute/v2.1/os-security-groups/5e9c7520-4b04-421c-973d-54c7ff2bbd9e => generated 194 bytes in 247 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:06:13.833974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-889d8a61-83fd-4442-a508-9804f58f94b7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'create', calling method: .version_select at 0x7f7f62b9c280>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1810224219", "description": "tempest-description-558977440"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:14.542738 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-889d8a61-83fd-4442-a508-9804f58f94b7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 243 microversion: 2.1 time: 0.719163 Sep 27 10:06:14.544458 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1097/2411] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:06:13 2022] POST /compute/v2.1/os-security-groups => generated 243 bytes in 721 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:14.556762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-19cc902f-f5d1-4c80-8106-9c50f47d4303 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:14.557676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-19cc902f-f5d1-4c80-8106-9c50f47d4303 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:14.582747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19cc902f-f5d1-4c80-8106-9c50f47d4303 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:14.587568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19cc902f-f5d1-4c80-8106-9c50f47d4303 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:14.589069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19cc902f-f5d1-4c80-8106-9c50f47d4303 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:14.591103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19cc902f-f5d1-4c80-8106-9c50f47d4303 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:14.592687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19cc902f-f5d1-4c80-8106-9c50f47d4303 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:14.596403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19cc902f-f5d1-4c80-8106-9c50f47d4303 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:15.091598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:15.183195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-19cc902f-f5d1-4c80-8106-9c50f47d4303 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:15.209961 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-19cc902f-f5d1-4c80-8106-9c50f47d4303 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1315 microversion: 2.1 time: 0.664184 Sep 27 10:06:15.228047 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1098/2412] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:14 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1315 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:15.253064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa0d31a5-37ca-428b-9baa-afc6f7573cda tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62ba2af0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:15.360610 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aa0d31a5-37ca-428b-9baa-afc6f7573cda tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.125917 Sep 27 10:06:15.365299 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1099/2413] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:15 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:15.399408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-191ff5eb-1ede-46f3-bfda-f3be84005045 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'create', calling method: .version_select at 0x7f7f62b8b0d0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1135651632", "description": "tempest-description-1421382738"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:16.076824 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-191ff5eb-1ede-46f3-bfda-f3be84005045 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 244 microversion: 2.1 time: 0.710252 Sep 27 10:06:16.078644 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1100/2414] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:06:15 2022] POST /compute/v2.1/os-security-groups => generated 244 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:16.090719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:16.091953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:16.093971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=72,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:16.108764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:16.128463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:16.129159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:16.130908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:16.131574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:16.132205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:16.355521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:16.386321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:16.393376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:16.410148 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cca8dbe8-957e-455b-9f3c-e0a8a7c1847f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.329906 Sep 27 10:06:16.411521 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1101/2415] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:16 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:16.451714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b26b3517-654a-4849-8dd3-df52bcf39c4a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'create', calling method: .version_select at 0x7f7f62ba2e50>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-2036552537", "description": "tempest-description-775321768"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:16.525367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:06:16.525463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:06:16.525945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:06:16.526556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:06:16.527111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:06:16.570643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:16.578380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:16.579537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:16.580828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:16.581598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:16.589588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:16.667378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=44,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:16.686811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Getting quotas for project f8130333e2c04a3f960dc6465e07cc46. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:06:16.709641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Getting quotas for user 68d9297ca0884c2fa7f5283f722451ba and project f8130333e2c04a3f960dc6465e07cc46. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:06:16.757571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:06:16.768914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:06:16.908098 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6e5a47f8-15eb-49d4-ab22-670ff92f49ed tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.366895 Sep 27 10:06:16.908871 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1315/2416] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:06:13 2022] POST /compute/v2.1/servers => generated 380 bytes in 3369 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:06:16.931092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a412d34c-8070-4e36-80b0-ef1999f09ddd req-cf4ffc9c-124c-4c56-92b4-b4f48bc750e8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4e215381-8752-4c41-aae8-d3b564df3dee", "tag": "a3f8b8ee-e42c-499e-a75f-6895f1ce45bd"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:16.958368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a412d34c-8070-4e36-80b0-ef1999f09ddd req-cf4ffc9c-124c-4c56-92b4-b4f48bc750e8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:16.963366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a412d34c-8070-4e36-80b0-ef1999f09ddd req-cf4ffc9c-124c-4c56-92b4-b4f48bc750e8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:16.964158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a412d34c-8070-4e36-80b0-ef1999f09ddd req-cf4ffc9c-124c-4c56-92b4-b4f48bc750e8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:17.027782 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a412d34c-8070-4e36-80b0-ef1999f09ddd req-cf4ffc9c-124c-4c56-92b4-b4f48bc750e8 service nova] Creating event network-changed:a3f8b8ee-e42c-499e-a75f-6895f1ce45bd for instance 4e215381-8752-4c41-aae8-d3b564df3dee on nch1-master-1 Sep 27 10:06:17.079069 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a412d34c-8070-4e36-80b0-ef1999f09ddd req-cf4ffc9c-124c-4c56-92b4-b4f48bc750e8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.162215 Sep 27 10:06:17.079965 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1316/2417] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:17.086341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-16cf4837-c120-4ce4-a92e-50e0c2cde730 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:17.116380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-16cf4837-c120-4ce4-a92e-50e0c2cde730 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:17.136371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cf4837-c120-4ce4-a92e-50e0c2cde730 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:17.137312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cf4837-c120-4ce4-a92e-50e0c2cde730 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:17.137912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cf4837-c120-4ce4-a92e-50e0c2cde730 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:17.138809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cf4837-c120-4ce4-a92e-50e0c2cde730 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:17.139513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cf4837-c120-4ce4-a92e-50e0c2cde730 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:17.140297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cf4837-c120-4ce4-a92e-50e0c2cde730 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:17.921370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-16cf4837-c120-4ce4-a92e-50e0c2cde730 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:17.933496 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-16cf4837-c120-4ce4-a92e-50e0c2cde730 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1315 microversion: 2.1 time: 0.851421 Sep 27 10:06:17.953852 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1317/2418] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:17 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1315 bytes in 872 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:17.960272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e5786da-b6d1-436e-b874-1da96db2adab tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62be3e50>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:18.096963 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e5786da-b6d1-436e-b874-1da96db2adab tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.141536 Sep 27 10:06:18.119990 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1318/2419] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:17 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:18.133431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53e6cb3c-aa84-4dec-a2d1-b7d0468fb531 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:18.135181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53e6cb3c-aa84-4dec-a2d1-b7d0468fb531 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:18.236580 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b26b3517-654a-4849-8dd3-df52bcf39c4a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 243 microversion: 2.1 time: 1.823548 Sep 27 10:06:18.237772 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1102/2420] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:06:16 2022] POST /compute/v2.1/os-security-groups => generated 243 bytes in 1825 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:18.242832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:18.243460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:18.289094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:18.289623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:18.300392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:18.301253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:18.301748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:18.302197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:18.518335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:18.583087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:18.589107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:18.631167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-53e6cb3c-aa84-4dec-a2d1-b7d0468fb531 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:18.673889 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fad3c4da-66c4-407d-acfd-75bb976aa09c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.431866 Sep 27 10:06:18.674770 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1103/2421] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:18 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:18.697700 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53e6cb3c-aa84-4dec-a2d1-b7d0468fb531 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1223 microversion: 2.1 time: 0.575791 Sep 27 10:06:18.706326 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1319/2422] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:18 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1223 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:18.729471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f9f8311f-2b1e-4974-abde-b3c0a004dcb2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62b8b4c0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:18.982613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b52f70f2-5eb4-434f-af74-b96a3133917e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:18.984519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b52f70f2-5eb4-434f-af74-b96a3133917e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:18.997690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b52f70f2-5eb4-434f-af74-b96a3133917e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:18.998965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b52f70f2-5eb4-434f-af74-b96a3133917e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:18.999599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b52f70f2-5eb4-434f-af74-b96a3133917e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:19.000864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b52f70f2-5eb4-434f-af74-b96a3133917e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:19.001829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b52f70f2-5eb4-434f-af74-b96a3133917e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:19.003787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b52f70f2-5eb4-434f-af74-b96a3133917e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:19.371157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b52f70f2-5eb4-434f-af74-b96a3133917e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:19.385377 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b52f70f2-5eb4-434f-af74-b96a3133917e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1315 microversion: 2.1 time: 0.408188 Sep 27 10:06:19.387991 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1320/2423] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:18 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1315 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:06:19.396455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5ed82084-221a-4df4-8fe2-d45ba42aca18 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6296a9d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:19.501541 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5ed82084-221a-4df4-8fe2-d45ba42aca18 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.108645 Sep 27 10:06:19.504736 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1321/2424] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:19 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:19.716125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:19.717061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:19.729672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:19.733428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:19.733981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:19.734836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:19.736119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:19.738217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:20.042305 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f9f8311f-2b1e-4974-abde-b3c0a004dcb2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups" status: 200 len: 1834 microversion: 2.1 time: 1.333947 Sep 27 10:06:20.042940 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1104/2425] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 10:06:18 2022] GET /compute/v2.1/os-security-groups => generated 1834 bytes in 1344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:20.048119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a6bedd93-fbeb-434f-9bfe-439dfe82b3b9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:20.048993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a6bedd93-fbeb-434f-9bfe-439dfe82b3b9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:20.064699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bedd93-fbeb-434f-9bfe-439dfe82b3b9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:20.066954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bedd93-fbeb-434f-9bfe-439dfe82b3b9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:20.067736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bedd93-fbeb-434f-9bfe-439dfe82b3b9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:20.070035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bedd93-fbeb-434f-9bfe-439dfe82b3b9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:20.071507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bedd93-fbeb-434f-9bfe-439dfe82b3b9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:20.074849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bedd93-fbeb-434f-9bfe-439dfe82b3b9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:20.118391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:20.130270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:20.136104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:20.170429 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-95fc4c53-b8c6-4ecd-83a1-835f5d2a01ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.459202 Sep 27 10:06:20.173330 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1322/2426] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:19 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:20.189424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a16c058f-fd77-4d87-b02c-1308234d39a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f6296a790>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:20.238255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a6bedd93-fbeb-434f-9bfe-439dfe82b3b9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:20.246214 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a6bedd93-fbeb-434f-9bfe-439dfe82b3b9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1275 microversion: 2.1 time: 0.201580 Sep 27 10:06:20.264922 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1105/2427] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:20 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1275 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:20.380266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestMultiNic-server-1504980907", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "617932e5-9268-4324-b220-c276322bb4f8"}, {"uuid": "836fb6ee-83bc-44fe-b3d2-01473c38b718"}, {"uuid": "617932e5-9268-4324-b220-c276322bb4f8"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:20.519782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] validate_networks() for [('617932e5-9268-4324-b220-c276322bb4f8', None, None, None, None, None), ('836fb6ee-83bc-44fe-b3d2-01473c38b718', None, None, None, None, None), ('617932e5-9268-4324-b220-c276322bb4f8', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:06:21.058243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:22.321575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:06:22.336224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:06:22.337267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:06:22.337837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:06:22.338439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:06:22.366898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:22.388060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:22.388826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:22.390415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:22.391249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:22.391923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:22.420409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Getting quotas for project fedcdffc12b94b489ce433d5caa5e4be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:06:22.427152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Getting quotas for user 430de2723f234fac8f09068195d8ce2c and project fedcdffc12b94b489ce433d5caa5e4be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:06:22.443126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:06:22.446543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:06:22.522918 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-310b334a-c508-41ec-90a6-0afb0c07c546 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.146588 Sep 27 10:06:22.525029 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1106/2428] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:06:20 2022] POST /compute/v2.1/servers => generated 380 bytes in 2149 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:06:22.530672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c42239b5-e95f-4aad-8c6a-1678ce59a989 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:22.531740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c42239b5-e95f-4aad-8c6a-1678ce59a989 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:22.540151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c42239b5-e95f-4aad-8c6a-1678ce59a989 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:22.541038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c42239b5-e95f-4aad-8c6a-1678ce59a989 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:22.541780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c42239b5-e95f-4aad-8c6a-1678ce59a989 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:22.543049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c42239b5-e95f-4aad-8c6a-1678ce59a989 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:22.543669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c42239b5-e95f-4aad-8c6a-1678ce59a989 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:22.544250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c42239b5-e95f-4aad-8c6a-1678ce59a989 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:22.844245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c42239b5-e95f-4aad-8c6a-1678ce59a989 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:22.855076 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c42239b5-e95f-4aad-8c6a-1678ce59a989 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1315 microversion: 2.1 time: 0.328502 Sep 27 10:06:22.861043 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1107/2429] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:22 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1315 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:22.881713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1adbb757-3560-4e38-9c9b-d4575bd473f5 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62b8bd30>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:23.003202 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1adbb757-3560-4e38-9c9b-d4575bd473f5 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.124802 Sep 27 10:06:23.004136 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1108/2430] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:22 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:23.156093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-866740943", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:23.321235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] validate_networks() for [('fabc7738-7394-4084-bbfe-24c62c6029b4', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:06:23.540287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a16c058f-fd77-4d87-b02c-1308234d39a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/88d2fe43-c785-472b-822f-124530320c7a" status: 202 len: 0 microversion: 2.1 time: 3.365300 Sep 27 10:06:23.540987 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1323/2431] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:06:20 2022] DELETE /compute/v2.1/os-security-groups/88d2fe43-c785-472b-822f-124530320c7a => generated 0 bytes in 3367 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:06:23.550559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:23.552422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:23.577103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:23.581091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:23.581931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:23.582981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:23.584996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:23.590274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:24.336790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:06:24.337317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:06:24.337876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:06:24.338287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:06:24.338778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:06:24.357596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:24.361554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:24.362441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:24.363839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:24.366891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:24.367652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:24.450028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Getting quotas for project 027612746ecc44f4ab8e7af121ec27a1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:06:24.478908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Getting quotas for user 92b0186cd92744dabd001b86560b5c90 and project 027612746ecc44f4ab8e7af121ec27a1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:06:24.519828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:06:24.527304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:06:24.535835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:24.553812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:24.557966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:24.586134 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d77a0774-b412-4eff-9e39-9b0134f26e6d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 1.036545 Sep 27 10:06:24.589111 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1324/2432] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:23 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 1047 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:24.594915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ca5b95c8-217b-4dac-a549-40d673f731db tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:24.595726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ca5b95c8-217b-4dac-a549-40d673f731db tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:24.637771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca5b95c8-217b-4dac-a549-40d673f731db tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:24.641741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca5b95c8-217b-4dac-a549-40d673f731db tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:24.642661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca5b95c8-217b-4dac-a549-40d673f731db tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:24.644325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca5b95c8-217b-4dac-a549-40d673f731db tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:24.646903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca5b95c8-217b-4dac-a549-40d673f731db tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:24.647657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca5b95c8-217b-4dac-a549-40d673f731db tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:24.691475 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4d25228a-8bbf-40d3-9c31-29e9b1f1c653 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.686373 Sep 27 10:06:24.701968 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1109/2433] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:06:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 1696 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:06:24.711595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-88f14ca3-8c42-41ee-a897-d8f9d3544c6f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:24.719232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-88f14ca3-8c42-41ee-a897-d8f9d3544c6f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:24.747382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88f14ca3-8c42-41ee-a897-d8f9d3544c6f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:24.747885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88f14ca3-8c42-41ee-a897-d8f9d3544c6f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:24.748322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88f14ca3-8c42-41ee-a897-d8f9d3544c6f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:24.750901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88f14ca3-8c42-41ee-a897-d8f9d3544c6f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:24.751394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88f14ca3-8c42-41ee-a897-d8f9d3544c6f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:24.751833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88f14ca3-8c42-41ee-a897-d8f9d3544c6f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:24.851537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-88f14ca3-8c42-41ee-a897-d8f9d3544c6f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:24.861071 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-88f14ca3-8c42-41ee-a897-d8f9d3544c6f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1265 microversion: 2.1 time: 0.157653 Sep 27 10:06:24.868910 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1110/2434] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:24 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1265 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:24.868910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e292ff9d-c8d8-47ac-9935-8d854a911747 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62b8c310>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:24.893900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ca5b95c8-217b-4dac-a549-40d673f731db tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:24.913624 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ca5b95c8-217b-4dac-a549-40d673f731db tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1281 microversion: 2.1 time: 0.322867 Sep 27 10:06:24.916517 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1325/2435] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:24 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1281 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:24.919539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG neutronclient.v2_0.client [None req-e292ff9d-c8d8-47ac-9935-8d854a911747 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist", "detail": ""}} {{(pid=145969) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:06:24.920673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.security_group_api [None req-e292ff9d-c8d8-47ac-9935-8d854a911747 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Neutron security group 88d2fe43-c785-472b-822f-124530320c7a not found {{(pid=145969) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:06:24.921661 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-e292ff9d-c8d8-47ac-9935-8d854a911747 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:06:24.921661 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-02ebf40c-b4c5-4538-a862-86c7863273f6'] Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e292ff9d-c8d8-47ac-9935-8d854a911747 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-security-groups/88d2fe43-c785-472b-822f-124530320c7a HTTP/1.1 Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: exception: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-02ebf40c-b4c5-4538-a862-86c7863273f6'] Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '88d2fe43-c785-472b-822f-124530320c7a'} Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:06:24.924065 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-02ebf40c-b4c5-4538-a862-86c7863273f6'] Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-02ebf40c-b4c5-4538-a862-86c7863273f6'] Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-02ebf40c-b4c5-4538-a862-86c7863273f6'] Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-02ebf40c-b4c5-4538-a862-86c7863273f6'] Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e292ff9d-c8d8-47ac-9935-8d854a911747 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-02ebf40c-b4c5-4538-a862-86c7863273f6'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:06:24.927492 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e292ff9d-c8d8-47ac-9935-8d854a911747 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups/88d2fe43-c785-472b-822f-124530320c7a" status: 404 len: 194 microversion: 2.1 time: 0.064039 Sep 27 10:06:24.931583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-38be0ed5-dc66-4875-867e-0dcc3bfda3db tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:24.934888 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1111/2436] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:06:24 2022] GET /compute/v2.1/os-security-groups/88d2fe43-c785-472b-822f-124530320c7a => generated 194 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:06:24.936612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-38be0ed5-dc66-4875-867e-0dcc3bfda3db tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:24.940198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dd278600-bb17-4b2f-ae46-2386f983478a tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f637f3430>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:24.950449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38be0ed5-dc66-4875-867e-0dcc3bfda3db tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:24.950961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38be0ed5-dc66-4875-867e-0dcc3bfda3db tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:24.951406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38be0ed5-dc66-4875-867e-0dcc3bfda3db tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:24.957486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38be0ed5-dc66-4875-867e-0dcc3bfda3db tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:24.958015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38be0ed5-dc66-4875-867e-0dcc3bfda3db tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:24.958468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38be0ed5-dc66-4875-867e-0dcc3bfda3db tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:25.085218 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dd278600-bb17-4b2f-ae46-2386f983478a tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1453 microversion: 2.1 time: 0.146372 Sep 27 10:06:25.090697 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1112/2437] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:24 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1453 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:25.119849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-565cecad-a05a-4fd5-95de-953ad3eb5ab6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:25.121024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-565cecad-a05a-4fd5-95de-953ad3eb5ab6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:25.240055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-565cecad-a05a-4fd5-95de-953ad3eb5ab6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:25.270044 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-565cecad-a05a-4fd5-95de-953ad3eb5ab6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1216 microversion: 2.1 time: 0.174904 Sep 27 10:06:25.273068 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1113/2438] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:25 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1216 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:25.281506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f8780fc-a8ba-4f86-b7ba-74519ba020b3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62b8b3a0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:26.074564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-38be0ed5-dc66-4875-867e-0dcc3bfda3db tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:26.088735 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-38be0ed5-dc66-4875-867e-0dcc3bfda3db tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1315 microversion: 2.1 time: 1.170063 Sep 27 10:06:26.090335 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1326/2439] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:24 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1315 bytes in 1173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:26.098864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:26.099966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:26.110398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:26.112789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:26.113418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:26.114355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:26.114978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:26.115558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:26.298265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:26.329292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:26.330747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:26.417466 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-24153f3f-5ceb-4c5f-83fb-2d725844f202 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.325380 Sep 27 10:06:26.422029 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1327/2440] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:26 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:26.465548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f73f77f-4a78-4aa9-a031-13d70e1d1263 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:26.466258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f73f77f-4a78-4aa9-a031-13d70e1d1263 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:26.510934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f73f77f-4a78-4aa9-a031-13d70e1d1263 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:26.511454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f73f77f-4a78-4aa9-a031-13d70e1d1263 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:26.511902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f73f77f-4a78-4aa9-a031-13d70e1d1263 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:26.519433 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f8780fc-a8ba-4f86-b7ba-74519ba020b3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/8f50a8f9-7234-4cf4-9730-39b3803a1274" status: 202 len: 0 microversion: 2.1 time: 1.244175 Sep 27 10:06:26.520281 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1114/2441] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:06:25 2022] DELETE /compute/v2.1/os-security-groups/8f50a8f9-7234-4cf4-9730-39b3803a1274 => generated 0 bytes in 1246 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:06:26.523027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f73f77f-4a78-4aa9-a031-13d70e1d1263 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:26.526273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f73f77f-4a78-4aa9-a031-13d70e1d1263 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:26.526739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f73f77f-4a78-4aa9-a031-13d70e1d1263 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:26.537780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5c88240-a712-4158-80dd-56dead540959 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:26.538826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5c88240-a712-4158-80dd-56dead540959 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:26.575085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5c88240-a712-4158-80dd-56dead540959 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:26.577130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5c88240-a712-4158-80dd-56dead540959 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:26.578825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5c88240-a712-4158-80dd-56dead540959 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:26.580709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5c88240-a712-4158-80dd-56dead540959 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:26.581917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5c88240-a712-4158-80dd-56dead540959 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:26.583330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5c88240-a712-4158-80dd-56dead540959 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:26.670374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:26.742193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6f73f77f-4a78-4aa9-a031-13d70e1d1263 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:26.751930 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f73f77f-4a78-4aa9-a031-13d70e1d1263 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1271 microversion: 2.1 time: 0.322356 Sep 27 10:06:26.757242 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1328/2442] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:26 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1271 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:26.769681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86898a60-91f0-4217-a4c9-6336adf5fa06 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f636d0430>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:26.878281 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86898a60-91f0-4217-a4c9-6336adf5fa06 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1462 microversion: 2.1 time: 0.117975 Sep 27 10:06:26.878942 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1329/2443] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:26 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1462 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:26.894337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a01917da-19e4-4609-acee-8b2df51ec231 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:26.894978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a01917da-19e4-4609-acee-8b2df51ec231 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:26.949947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a01917da-19e4-4609-acee-8b2df51ec231 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:26.950467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a01917da-19e4-4609-acee-8b2df51ec231 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:26.950923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a01917da-19e4-4609-acee-8b2df51ec231 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:26.955160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a01917da-19e4-4609-acee-8b2df51ec231 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:26.955690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a01917da-19e4-4609-acee-8b2df51ec231 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:26.962902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a01917da-19e4-4609-acee-8b2df51ec231 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:27.166094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a01917da-19e4-4609-acee-8b2df51ec231 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:27.176852 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a01917da-19e4-4609-acee-8b2df51ec231 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1268 microversion: 2.1 time: 0.290254 Sep 27 10:06:27.177363 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1330/2444] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:26 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1268 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:27.190717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3924809-ccae-4db0-b050-fed19fb62607 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62953430>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:27.196254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5c88240-a712-4158-80dd-56dead540959 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:27.204773 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5c88240-a712-4158-80dd-56dead540959 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1323 microversion: 2.1 time: 0.681535 Sep 27 10:06:27.209495 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1115/2445] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:26 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1323 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:27.232716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1731a22d-a71a-43a2-8070-8935dbe12592 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62e44e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:27.294231 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1731a22d-a71a-43a2-8070-8935dbe12592 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 200 len: 1462 microversion: 2.1 time: 0.071353 Sep 27 10:06:27.296089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-e3924809-ccae-4db0-b050-fed19fb62607 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:06:27.297562 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1116/2446] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:27 2022] GET /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 1462 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:27.297703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.security_group_api [None req-e3924809-ccae-4db0-b050-fed19fb62607 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Neutron security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 not found {{(pid=145968) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:06:27.298187 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-e3924809-ccae-4db0-b050-fed19fb62607 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:06:27.298187 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-faa9b508-f145-4856-b82f-6e2f1cfd74c4'] Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3924809-ccae-4db0-b050-fed19fb62607 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-security-groups/8f50a8f9-7234-4cf4-9730-39b3803a1274 HTTP/1.1 Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: exception: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-faa9b508-f145-4856-b82f-6e2f1cfd74c4'] Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '8f50a8f9-7234-4cf4-9730-39b3803a1274'} Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:06:27.300699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-faa9b508-f145-4856-b82f-6e2f1cfd74c4'] Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-faa9b508-f145-4856-b82f-6e2f1cfd74c4'] Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-faa9b508-f145-4856-b82f-6e2f1cfd74c4'] Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-faa9b508-f145-4856-b82f-6e2f1cfd74c4'] Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3924809-ccae-4db0-b050-fed19fb62607 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-faa9b508-f145-4856-b82f-6e2f1cfd74c4'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:06:27.304100 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e3924809-ccae-4db0-b050-fed19fb62607 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups/8f50a8f9-7234-4cf4-9730-39b3803a1274" status: 404 len: 194 microversion: 2.1 time: 0.123890 Sep 27 10:06:27.308553 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1331/2447] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:06:27 2022] GET /compute/v2.1/os-security-groups/8f50a8f9-7234-4cf4-9730-39b3803a1274 => generated 194 bytes in 130 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:06:27.310431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-31006be5-219b-4d99-b463-cf976e6eb8f0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:27.311279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-31006be5-219b-4d99-b463-cf976e6eb8f0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:27.318326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31006be5-219b-4d99-b463-cf976e6eb8f0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:27.319155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31006be5-219b-4d99-b463-cf976e6eb8f0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:27.319805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31006be5-219b-4d99-b463-cf976e6eb8f0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:27.320364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e2599dd1-9f05-437a-84b7-5a70b7e61f6b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:27.321144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31006be5-219b-4d99-b463-cf976e6eb8f0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:27.321932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31006be5-219b-4d99-b463-cf976e6eb8f0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:27.322588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31006be5-219b-4d99-b463-cf976e6eb8f0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:27.323340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e2599dd1-9f05-437a-84b7-5a70b7e61f6b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:27.341047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2599dd1-9f05-437a-84b7-5a70b7e61f6b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:27.346091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2599dd1-9f05-437a-84b7-5a70b7e61f6b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:27.346931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2599dd1-9f05-437a-84b7-5a70b7e61f6b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:27.347902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2599dd1-9f05-437a-84b7-5a70b7e61f6b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:27.348522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2599dd1-9f05-437a-84b7-5a70b7e61f6b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:27.349159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2599dd1-9f05-437a-84b7-5a70b7e61f6b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:27.640288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-31006be5-219b-4d99-b463-cf976e6eb8f0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:27.651256 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-31006be5-219b-4d99-b463-cf976e6eb8f0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1315 microversion: 2.1 time: 0.352065 Sep 27 10:06:27.654564 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1117/2448] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:27 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1315 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:27.664569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-09218a32-296d-4c55-9826-351bd8dd735a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62e44e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:27.705891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2599dd1-9f05-437a-84b7-5a70b7e61f6b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:27.728748 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e2599dd1-9f05-437a-84b7-5a70b7e61f6b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1465 microversion: 2.1 time: 0.418171 Sep 27 10:06:27.730310 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1332/2449] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:27 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1465 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:27.743307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:27.744829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:27.768917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:27.769427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:27.769869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:27.770825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:27.771442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:27.772030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:27.915090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:27.927636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:27.933476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:27.947599 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4dd40aef-cf23-4ecb-a55a-da1990e0d7b3 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.215989 Sep 27 10:06:27.948228 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1333/2450] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:27 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:27.974420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c8004563-7bb8-4de5-acb0-281a14cd6683 req-b9f1c7cc-f61a-4d97-aafa-822b211e4fcc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4e215381-8752-4c41-aae8-d3b564df3dee", "name": "network-vif-plugged", "status": "completed", "tag": "a3f8b8ee-e42c-499e-a75f-6895f1ce45bd"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:27.985725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c8004563-7bb8-4de5-acb0-281a14cd6683 req-b9f1c7cc-f61a-4d97-aafa-822b211e4fcc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:27.986404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c8004563-7bb8-4de5-acb0-281a14cd6683 req-b9f1c7cc-f61a-4d97-aafa-822b211e4fcc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:27.987028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c8004563-7bb8-4de5-acb0-281a14cd6683 req-b9f1c7cc-f61a-4d97-aafa-822b211e4fcc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:28.000267 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c8004563-7bb8-4de5-acb0-281a14cd6683 req-b9f1c7cc-f61a-4d97-aafa-822b211e4fcc service nova] Creating event network-vif-plugged:a3f8b8ee-e42c-499e-a75f-6895f1ce45bd for instance 4e215381-8752-4c41-aae8-d3b564df3dee on nch1-master-1 Sep 27 10:06:28.007035 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c8004563-7bb8-4de5-acb0-281a14cd6683 req-b9f1c7cc-f61a-4d97-aafa-822b211e4fcc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040804 Sep 27 10:06:28.013117 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1334/2451] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:28.021669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9e28792-19e7-48e4-bc1e-93505e8377fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ListImageFiltersTestJSON-image-1715315505"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:28.025181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c9e28792-19e7-48e4-bc1e-93505e8377fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:28.032467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9e28792-19e7-48e4-bc1e-93505e8377fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:28.033215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9e28792-19e7-48e4-bc1e-93505e8377fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:28.033809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9e28792-19e7-48e4-bc1e-93505e8377fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:28.034712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9e28792-19e7-48e4-bc1e-93505e8377fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:28.035319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9e28792-19e7-48e4-bc1e-93505e8377fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:28.036045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9e28792-19e7-48e4-bc1e-93505e8377fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:28.405983 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9e28792-19e7-48e4-bc1e-93505e8377fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "POST /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d/action" status: 202 len: 0 microversion: 2.1 time: 0.389021 Sep 27 10:06:28.409185 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1335/2452] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:06:28 2022] POST /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d/action => generated 0 bytes in 395 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 10:06:28.426233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4157fd2b-8f71-46d5-bfb2-04dc7730929d tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:28.427192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4157fd2b-8f71-46d5-bfb2-04dc7730929d tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:28.453863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4157fd2b-8f71-46d5-bfb2-04dc7730929d tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:28.454364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4157fd2b-8f71-46d5-bfb2-04dc7730929d tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:28.456831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4157fd2b-8f71-46d5-bfb2-04dc7730929d tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:28.459009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4157fd2b-8f71-46d5-bfb2-04dc7730929d tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:28.460389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4157fd2b-8f71-46d5-bfb2-04dc7730929d tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:28.461921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4157fd2b-8f71-46d5-bfb2-04dc7730929d tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:28.640430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:28.789509 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-09218a32-296d-4c55-9826-351bd8dd735a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/327e2132-3353-4d5f-8082-a6489c6018b2" status: 202 len: 0 microversion: 2.1 time: 1.131845 Sep 27 10:06:28.794055 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1118/2453] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:06:27 2022] DELETE /compute/v2.1/os-security-groups/327e2132-3353-4d5f-8082-a6489c6018b2 => generated 0 bytes in 1138 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:06:28.801519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6be03891-9d1c-4128-98b4-e50cee7783b3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:28.802462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6be03891-9d1c-4128-98b4-e50cee7783b3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:28.810066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6be03891-9d1c-4128-98b4-e50cee7783b3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:28.810907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6be03891-9d1c-4128-98b4-e50cee7783b3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:28.811564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6be03891-9d1c-4128-98b4-e50cee7783b3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:28.812623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6be03891-9d1c-4128-98b4-e50cee7783b3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:28.813948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6be03891-9d1c-4128-98b4-e50cee7783b3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:28.815231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6be03891-9d1c-4128-98b4-e50cee7783b3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:28.915227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6be03891-9d1c-4128-98b4-e50cee7783b3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:28.925717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:28.937661 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6be03891-9d1c-4128-98b4-e50cee7783b3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1274 microversion: 2.1 time: 0.139179 Sep 27 10:06:28.938862 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1119/2454] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:28 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1274 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:28.949373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7a8bc4c7-7fd8-492c-942e-4f4a6227c893 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:28.950276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7a8bc4c7-7fd8-492c-942e-4f4a6227c893 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:28.960545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a8bc4c7-7fd8-492c-942e-4f4a6227c893 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:28.961385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a8bc4c7-7fd8-492c-942e-4f4a6227c893 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:28.962116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a8bc4c7-7fd8-492c-942e-4f4a6227c893 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:28.963746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a8bc4c7-7fd8-492c-942e-4f4a6227c893 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:28.964366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a8bc4c7-7fd8-492c-942e-4f4a6227c893 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:28.966061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a8bc4c7-7fd8-492c-942e-4f4a6227c893 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:29.038140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4157fd2b-8f71-46d5-bfb2-04dc7730929d tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:29.050371 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4157fd2b-8f71-46d5-bfb2-04dc7730929d tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1313 microversion: 2.1 time: 0.639512 Sep 27 10:06:29.057832 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1336/2455] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:28 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1313 bytes in 648 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:29.063416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b208b7e4-fc4c-4538-bf18-51c4cb0dcea8 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6295f700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:29.167437 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b208b7e4-fc4c-4538-bf18-51c4cb0dcea8 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.108204 Sep 27 10:06:29.168934 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1337/2456] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:29 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:29.181324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-36275c53-de20-40f2-8d0c-6127fd2c1f80 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:29.185101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-36275c53-de20-40f2-8d0c-6127fd2c1f80 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:29.194361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36275c53-de20-40f2-8d0c-6127fd2c1f80 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:29.195178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36275c53-de20-40f2-8d0c-6127fd2c1f80 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:29.195810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36275c53-de20-40f2-8d0c-6127fd2c1f80 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:29.208146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36275c53-de20-40f2-8d0c-6127fd2c1f80 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:29.208950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36275c53-de20-40f2-8d0c-6127fd2c1f80 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:29.210392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36275c53-de20-40f2-8d0c-6127fd2c1f80 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:29.581685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a8bc4c7-7fd8-492c-942e-4f4a6227c893 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:29.591384 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7a8bc4c7-7fd8-492c-942e-4f4a6227c893 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1323 microversion: 2.1 time: 0.650749 Sep 27 10:06:29.595914 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1120/2457] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:28 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1323 bytes in 656 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:29.610461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f01b800-eb8d-482d-bde2-7cf1d308d730 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f637f1b80>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:29.639810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-36275c53-de20-40f2-8d0c-6127fd2c1f80 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:29.655711 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-36275c53-de20-40f2-8d0c-6127fd2c1f80 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1315 microversion: 2.1 time: 0.485434 Sep 27 10:06:29.657131 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1338/2458] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:29 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1315 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:29.664537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:29.665877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:29.677018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:29.677949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:29.678995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:29.680759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:29.681520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:29.682483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:29.694654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:29.702006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG neutronclient.v2_0.client [None req-8f01b800-eb8d-482d-bde2-7cf1d308d730 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist", "detail": ""}} {{(pid=145969) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:06:29.702795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.security_group_api [None req-8f01b800-eb8d-482d-bde2-7cf1d308d730 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Neutron security group 327e2132-3353-4d5f-8082-a6489c6018b2 not found {{(pid=145969) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:06:29.703552 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-8f01b800-eb8d-482d-bde2-7cf1d308d730 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:06:29.703552 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-3598456a-08b1-4e61-ba5b-6a89b34d7f72'] Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f01b800-eb8d-482d-bde2-7cf1d308d730 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-security-groups/327e2132-3353-4d5f-8082-a6489c6018b2 HTTP/1.1 Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: exception: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-3598456a-08b1-4e61-ba5b-6a89b34d7f72'] Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '327e2132-3353-4d5f-8082-a6489c6018b2'} Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:06:29.707638 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3598456a-08b1-4e61-ba5b-6a89b34d7f72'] Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3598456a-08b1-4e61-ba5b-6a89b34d7f72'] Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3598456a-08b1-4e61-ba5b-6a89b34d7f72'] Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3598456a-08b1-4e61-ba5b-6a89b34d7f72'] Sep 27 10:06:29.711706 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:29.718561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f01b800-eb8d-482d-bde2-7cf1d308d730 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:06:29.718561 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-3598456a-08b1-4e61-ba5b-6a89b34d7f72'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:06:29.719990 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f01b800-eb8d-482d-bde2-7cf1d308d730 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups/327e2132-3353-4d5f-8082-a6489c6018b2" status: 404 len: 194 microversion: 2.1 time: 0.122844 Sep 27 10:06:29.720789 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1121/2459] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:06:29 2022] GET /compute/v2.1/os-security-groups/327e2132-3353-4d5f-8082-a6489c6018b2 => generated 194 bytes in 124 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:06:29.732176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-0e598898-00a6-465a-9a79-c5f3d25d13da req-c8725789-1c44-43ed-a104-ba8c105ff439 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0abee65c-58ee-4b9c-ac6c-65631519b216", "tag": "447ad707-a60a-4017-9b0d-e87acd64bee0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:29.758396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0e598898-00a6-465a-9a79-c5f3d25d13da req-c8725789-1c44-43ed-a104-ba8c105ff439 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:29.762043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0e598898-00a6-465a-9a79-c5f3d25d13da req-c8725789-1c44-43ed-a104-ba8c105ff439 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:29.762892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0e598898-00a6-465a-9a79-c5f3d25d13da req-c8725789-1c44-43ed-a104-ba8c105ff439 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:29.809445 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-0e598898-00a6-465a-9a79-c5f3d25d13da req-c8725789-1c44-43ed-a104-ba8c105ff439 service nova] Creating event network-changed:447ad707-a60a-4017-9b0d-e87acd64bee0 for instance 0abee65c-58ee-4b9c-ac6c-65631519b216 on nch1-master-1 Sep 27 10:06:29.830638 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-0e598898-00a6-465a-9a79-c5f3d25d13da req-c8725789-1c44-43ed-a104-ba8c105ff439 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.107697 Sep 27 10:06:29.831788 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1122/2460] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:29.839452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-71a056eb-81f8-4749-b626-9f0acb92fb37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f637f3670>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:30.123142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:30.142118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:30.148409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:30.171604 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cfd59de8-8f33-4332-97c2-c27a33b6b777 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.513029 Sep 27 10:06:30.189298 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1339/2461] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:29 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:30.214023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cf00024e-25cd-4a81-8969-4f0faedf30b7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:30.214729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cf00024e-25cd-4a81-8969-4f0faedf30b7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:30.233466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf00024e-25cd-4a81-8969-4f0faedf30b7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:30.234638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf00024e-25cd-4a81-8969-4f0faedf30b7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:30.235398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf00024e-25cd-4a81-8969-4f0faedf30b7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:30.236399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf00024e-25cd-4a81-8969-4f0faedf30b7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:30.237132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf00024e-25cd-4a81-8969-4f0faedf30b7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:30.238607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf00024e-25cd-4a81-8969-4f0faedf30b7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:30.562482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cf00024e-25cd-4a81-8969-4f0faedf30b7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:30.573184 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cf00024e-25cd-4a81-8969-4f0faedf30b7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1316 microversion: 2.1 time: 0.381620 Sep 27 10:06:30.574367 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1340/2462] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:30 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1316 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:30.579070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-36f7413f-5e84-4c42-ad55-04973afad92d req-6d7cba6f-8ad9-435b-805b-cac0e40e88c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "55ead6af-a0d9-4d6f-b742-4a3fd4a418ad", "name": "network-vif-unplugged", "status": "completed", "tag": "a33d85f6-8d2f-4192-8ab2-06ef5d149185"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:30.596573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-36f7413f-5e84-4c42-ad55-04973afad92d req-6d7cba6f-8ad9-435b-805b-cac0e40e88c3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:30.597263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-36f7413f-5e84-4c42-ad55-04973afad92d req-6d7cba6f-8ad9-435b-805b-cac0e40e88c3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:30.597838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-36f7413f-5e84-4c42-ad55-04973afad92d req-6d7cba6f-8ad9-435b-805b-cac0e40e88c3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:30.611143 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-36f7413f-5e84-4c42-ad55-04973afad92d req-6d7cba6f-8ad9-435b-805b-cac0e40e88c3 service nova] Creating event network-vif-unplugged:a33d85f6-8d2f-4192-8ab2-06ef5d149185 for instance 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad on nch2-master-1 Sep 27 10:06:30.617435 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-36f7413f-5e84-4c42-ad55-04973afad92d req-6d7cba6f-8ad9-435b-805b-cac0e40e88c3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.040941 Sep 27 10:06:30.619170 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1341/2463] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:30 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:30.629889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b577eb81-ffea-4bf8-85a6-727cef743a51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:30.630619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b577eb81-ffea-4bf8-85a6-727cef743a51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:30.642507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b577eb81-ffea-4bf8-85a6-727cef743a51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:30.643099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b577eb81-ffea-4bf8-85a6-727cef743a51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:30.643417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b577eb81-ffea-4bf8-85a6-727cef743a51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:30.644593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b577eb81-ffea-4bf8-85a6-727cef743a51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:30.645367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b577eb81-ffea-4bf8-85a6-727cef743a51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:30.645940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b577eb81-ffea-4bf8-85a6-727cef743a51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:30.755749 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-71a056eb-81f8-4749-b626-9f0acb92fb37 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups" status: 200 len: 1158 microversion: 2.1 time: 0.922544 Sep 27 10:06:30.757062 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1123/2464] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 10:06:29 2022] GET /compute/v2.1/os-security-groups => generated 1158 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:30.773139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-18100e72-bc78-4409-846f-ba3d7dcd8443 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62b40310>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:30.847857 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-18100e72-bc78-4409-846f-ba3d7dcd8443 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.085755 Sep 27 10:06:30.849473 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1124/2465] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:30 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:30.855774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec3a97b2-94a8-40c9-bbbc-a81266c940b7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:30.858257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ec3a97b2-94a8-40c9-bbbc-a81266c940b7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:30.867554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec3a97b2-94a8-40c9-bbbc-a81266c940b7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:30.868878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec3a97b2-94a8-40c9-bbbc-a81266c940b7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:30.869482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec3a97b2-94a8-40c9-bbbc-a81266c940b7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:30.871128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec3a97b2-94a8-40c9-bbbc-a81266c940b7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:30.872741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec3a97b2-94a8-40c9-bbbc-a81266c940b7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:30.873379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec3a97b2-94a8-40c9-bbbc-a81266c940b7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:30.934967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b577eb81-ffea-4bf8-85a6-727cef743a51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:30.947984 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b577eb81-ffea-4bf8-85a6-727cef743a51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1334 microversion: 2.1 time: 0.327368 Sep 27 10:06:30.948603 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1342/2466] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:30 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1334 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:30.968610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-db829dc9-d684-4fd5-be7e-f50a7924b819 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:30.969254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-db829dc9-d684-4fd5-be7e-f50a7924b819 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:30.977490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db829dc9-d684-4fd5-be7e-f50a7924b819 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:30.978079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db829dc9-d684-4fd5-be7e-f50a7924b819 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:30.978516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db829dc9-d684-4fd5-be7e-f50a7924b819 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:30.984574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db829dc9-d684-4fd5-be7e-f50a7924b819 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:30.985107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db829dc9-d684-4fd5-be7e-f50a7924b819 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:30.985558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db829dc9-d684-4fd5-be7e-f50a7924b819 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:31.180223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-db829dc9-d684-4fd5-be7e-f50a7924b819 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:31.189692 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-db829dc9-d684-4fd5-be7e-f50a7924b819 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1502 microversion: 2.1 time: 0.225000 Sep 27 10:06:31.190210 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1343/2467] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:30 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1502 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:31.195626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ff9751aa-1f35-4790-b5c5-e78a17c235f5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'create', calling method: .version_select at 0x7f7f62953940>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-369040765", "description": "tempest-description-1305567190"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:31.408371 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ff9751aa-1f35-4790-b5c5-e78a17c235f5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 243 microversion: 2.1 time: 0.216849 Sep 27 10:06:31.409029 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1344/2468] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:06:31 2022] POST /compute/v2.1/os-security-groups => generated 243 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:31.415037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:31.415913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:31.423031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:31.423754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:31.424337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:31.425236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:31.425845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:31.426634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:31.478953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ec3a97b2-94a8-40c9-bbbc-a81266c940b7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:31.487519 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ec3a97b2-94a8-40c9-bbbc-a81266c940b7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1323 microversion: 2.1 time: 0.636839 Sep 27 10:06:31.488440 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1125/2469] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:30 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1323 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:31.500829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a19ab39b-eb7a-4d88-ae78-5da67d6f3e9a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'create', calling method: .version_select at 0x7f7f6366d1f0>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-400049632", "description": "tempest-description-552161283"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:31.509668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:31.520199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:31.521865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:31.527911 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bf409162-5bf4-4862-b711-9cc78510db34 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.117595 Sep 27 10:06:31.528514 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1345/2470] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:31 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:31.626549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f4684b90-43df-4121-b9c8-d981729e25b1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:31.628524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f4684b90-43df-4121-b9c8-d981729e25b1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:31.636915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4684b90-43df-4121-b9c8-d981729e25b1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:31.637462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4684b90-43df-4121-b9c8-d981729e25b1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:31.638064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4684b90-43df-4121-b9c8-d981729e25b1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:31.639280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4684b90-43df-4121-b9c8-d981729e25b1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:31.639732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4684b90-43df-4121-b9c8-d981729e25b1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:31.640159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4684b90-43df-4121-b9c8-d981729e25b1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:31.884457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f4684b90-43df-4121-b9c8-d981729e25b1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:31.893577 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f4684b90-43df-4121-b9c8-d981729e25b1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1316 microversion: 2.1 time: 0.295972 Sep 27 10:06:31.902878 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1346/2471] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:31 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1316 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:31.909165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f9884a30-516d-4495-a3f7-9ebb5c3e1702 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f629ad310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:31.989580 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f9884a30-516d-4495-a3f7-9ebb5c3e1702 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.085708 Sep 27 10:06:31.998443 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1347/2472] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:31 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:31.998443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7893fb1b-eea8-4d45-974e-4e01d2ffa668 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:31.998443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7893fb1b-eea8-4d45-974e-4e01d2ffa668 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:32.004049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7893fb1b-eea8-4d45-974e-4e01d2ffa668 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:32.004049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7893fb1b-eea8-4d45-974e-4e01d2ffa668 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:32.004251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7893fb1b-eea8-4d45-974e-4e01d2ffa668 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:32.005417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7893fb1b-eea8-4d45-974e-4e01d2ffa668 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:32.005528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7893fb1b-eea8-4d45-974e-4e01d2ffa668 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:32.005987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7893fb1b-eea8-4d45-974e-4e01d2ffa668 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:32.230947 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a19ab39b-eb7a-4d88-ae78-5da67d6f3e9a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 242 microversion: 2.1 time: 0.741231 Sep 27 10:06:32.245804 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1126/2473] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:06:31 2022] POST /compute/v2.1/os-security-groups => generated 242 bytes in 743 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:32.421555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-SecurityGroupsTestJSON-server-1498541489", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "63be3c83-0a96-45db-bf25-b57bb15795b7"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:32.529222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] validate_networks() for [('63be3c83-0a96-45db-bf25-b57bb15795b7', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:06:32.626844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7893fb1b-eea8-4d45-974e-4e01d2ffa668 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:32.644291 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7893fb1b-eea8-4d45-974e-4e01d2ffa668 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1334 microversion: 2.1 time: 0.651460 Sep 27 10:06:32.645652 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1348/2474] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:31 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1334 bytes in 654 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:32.658204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1ab35df6-0d75-4939-accf-cc86659397ca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:32.659324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1ab35df6-0d75-4939-accf-cc86659397ca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:32.672582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ab35df6-0d75-4939-accf-cc86659397ca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:32.673070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ab35df6-0d75-4939-accf-cc86659397ca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:32.690801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ab35df6-0d75-4939-accf-cc86659397ca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:32.696906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ab35df6-0d75-4939-accf-cc86659397ca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:32.697663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ab35df6-0d75-4939-accf-cc86659397ca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:32.698673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ab35df6-0d75-4939-accf-cc86659397ca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:33.039126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1ab35df6-0d75-4939-accf-cc86659397ca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:33.051379 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1ab35df6-0d75-4939-accf-cc86659397ca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1323 microversion: 2.1 time: 0.404340 Sep 27 10:06:33.066802 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1349/2475] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:32 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1323 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:33.077784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:33.078903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:33.091196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:33.091417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:33.091983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:33.092857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:33.093321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:33.093747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:33.482832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:33.502911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:33.507664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:33.517147 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e0a9b7c8-c104-4dc7-b61c-68dceece6902 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.448989 Sep 27 10:06:33.518291 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1350/2476] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:33 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:33.525823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3f763a50-6adb-4ae4-8f2a-2599d21fe850"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:33.528885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:33.535224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:33.536007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:33.537200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:33.537980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:33.538644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:33.539308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:33.894011 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4f79ee62-89b3-4ac1-9349-54c9263b7b01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.373638 Sep 27 10:06:33.895260 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1351/2477] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:06:33 2022] POST /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee/os-volume_attachments => generated 194 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:33.902762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fe8b086c-5812-40bc-9f7e-981c73281875 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:33.903719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fe8b086c-5812-40bc-9f7e-981c73281875 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:33.912921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8b086c-5812-40bc-9f7e-981c73281875 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:33.914216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8b086c-5812-40bc-9f7e-981c73281875 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:33.915376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8b086c-5812-40bc-9f7e-981c73281875 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:33.916521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8b086c-5812-40bc-9f7e-981c73281875 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:33.917286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8b086c-5812-40bc-9f7e-981c73281875 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:33.918430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe8b086c-5812-40bc-9f7e-981c73281875 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:34.122368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fe8b086c-5812-40bc-9f7e-981c73281875 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:34.146198 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fe8b086c-5812-40bc-9f7e-981c73281875 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1316 microversion: 2.1 time: 0.249248 Sep 27 10:06:34.147035 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1352/2478] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:33 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1316 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:34.157470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e852e7b-1585-4bdd-a7fd-40f21a03d35c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62953820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:34.208994 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9e852e7b-1585-4bdd-a7fd-40f21a03d35c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.061053 Sep 27 10:06:34.210058 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1353/2479] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:34 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:34.215079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-dc2d50cb-961b-4346-988d-55777a9dd652 req-63811e60-abee-45ee-afbe-ed5afbde69aa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3327ff95-5c05-4167-a73b-5fd55f7bfabd", "tag": "d571b78e-f1b6-4353-b09e-cac4cd18cde7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:34.226187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-dc2d50cb-961b-4346-988d-55777a9dd652 req-63811e60-abee-45ee-afbe-ed5afbde69aa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:34.227172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-dc2d50cb-961b-4346-988d-55777a9dd652 req-63811e60-abee-45ee-afbe-ed5afbde69aa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:34.227953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-dc2d50cb-961b-4346-988d-55777a9dd652 req-63811e60-abee-45ee-afbe-ed5afbde69aa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:34.241091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-dc2d50cb-961b-4346-988d-55777a9dd652 req-63811e60-abee-45ee-afbe-ed5afbde69aa service nova] Creating event network-changed:d571b78e-f1b6-4353-b09e-cac4cd18cde7 for instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd on nch2-master-1 Sep 27 10:06:34.247777 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-dc2d50cb-961b-4346-988d-55777a9dd652 req-63811e60-abee-45ee-afbe-ed5afbde69aa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036175 Sep 27 10:06:34.249112 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1354/2480] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:34.254565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-458a0285-4356-4310-9732-bcc04fa2d826 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:34.255011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:06:34.255457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:06:34.255906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:06:34.256232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-458a0285-4356-4310-9732-bcc04fa2d826 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:34.256369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:06:34.256744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:06:34.264529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-458a0285-4356-4310-9732-bcc04fa2d826 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:34.265293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-458a0285-4356-4310-9732-bcc04fa2d826 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:34.265960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-458a0285-4356-4310-9732-bcc04fa2d826 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:34.266944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-458a0285-4356-4310-9732-bcc04fa2d826 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:34.267616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-458a0285-4356-4310-9732-bcc04fa2d826 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:34.268267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-458a0285-4356-4310-9732-bcc04fa2d826 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:34.274587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:34.275794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:34.280813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:34.281707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:34.282198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:34.282636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:34.338120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Getting quotas for project 513655e40ccd4152a2e343e5415ce347. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:06:34.344678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Getting quotas for user 866c1a2a2ca8452eb0a66cbe9692ab75 and project 513655e40ccd4152a2e343e5415ce347. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:06:34.360179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:06:34.363178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:06:34.463357 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e1068fdd-3563-423c-871b-c6ca49c5af12 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.047617 Sep 27 10:06:34.464596 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1127/2481] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:06:32 2022] POST /compute/v2.1/servers => generated 380 bytes in 2050 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:06:34.470017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-809c8d80-8244-415b-a041-0de4c6d50757 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:34.473457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-809c8d80-8244-415b-a041-0de4c6d50757 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:34.483519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-809c8d80-8244-415b-a041-0de4c6d50757 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:34.484582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-809c8d80-8244-415b-a041-0de4c6d50757 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:34.485209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-809c8d80-8244-415b-a041-0de4c6d50757 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:34.486100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-809c8d80-8244-415b-a041-0de4c6d50757 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:34.487032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-809c8d80-8244-415b-a041-0de4c6d50757 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:34.487607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-809c8d80-8244-415b-a041-0de4c6d50757 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:34.566638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-458a0285-4356-4310-9732-bcc04fa2d826 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:34.583699 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-458a0285-4356-4310-9732-bcc04fa2d826 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.333155 Sep 27 10:06:34.584306 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1355/2482] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:34 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:34.592543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e8866251-7c04-4400-b6dc-4c451d33646d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:34.594676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e8866251-7c04-4400-b6dc-4c451d33646d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:34.685689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e8866251-7c04-4400-b6dc-4c451d33646d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:34.695888 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e8866251-7c04-4400-b6dc-4c451d33646d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1216 microversion: 2.1 time: 0.109292 Sep 27 10:06:34.696724 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1356/2483] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:34 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1216 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:34.718870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:34.720073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:34.729101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:34.730041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:34.730693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:34.731631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:34.732284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:34.732949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:34.837502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:34.848231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:34.849759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:34.856684 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8c8bf597-764c-49d8-9c7a-fa50e261c8c4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.146002 Sep 27 10:06:34.857335 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1357/2484] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:34 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:35.171094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7c7c887f-05d7-493c-92df-f7b5ad9cffef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:35.171840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7c7c887f-05d7-493c-92df-f7b5ad9cffef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:35.187760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c7c887f-05d7-493c-92df-f7b5ad9cffef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:35.191666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c7c887f-05d7-493c-92df-f7b5ad9cffef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:35.192175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c7c887f-05d7-493c-92df-f7b5ad9cffef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:35.193041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c7c887f-05d7-493c-92df-f7b5ad9cffef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:35.193603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c7c887f-05d7-493c-92df-f7b5ad9cffef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:35.194065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c7c887f-05d7-493c-92df-f7b5ad9cffef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:35.325298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-809c8d80-8244-415b-a041-0de4c6d50757 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:35.341529 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-809c8d80-8244-415b-a041-0de4c6d50757 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1323 microversion: 2.1 time: 0.873162 Sep 27 10:06:35.356695 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1128/2485] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:34 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1323 bytes in 879 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:35.357890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4646c163-3b2e-4061-a470-85b3b405b0ee tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6361cca0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:35.452450 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4646c163-3b2e-4061-a470-85b3b405b0ee tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.106636 Sep 27 10:06:35.453088 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1129/2486] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:35 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:35.533671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7c7c887f-05d7-493c-92df-f7b5ad9cffef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:35.545109 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7c7c887f-05d7-493c-92df-f7b5ad9cffef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1316 microversion: 2.1 time: 0.377000 Sep 27 10:06:35.559698 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1358/2487] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:35 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1316 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:35.615588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95c2f908-cbbb-4c9e-b1ce-5d879d55c964 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:35.616232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-95c2f908-cbbb-4c9e-b1ce-5d879d55c964 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:35.629208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95c2f908-cbbb-4c9e-b1ce-5d879d55c964 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:35.631805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95c2f908-cbbb-4c9e-b1ce-5d879d55c964 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:35.632221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95c2f908-cbbb-4c9e-b1ce-5d879d55c964 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:35.638895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95c2f908-cbbb-4c9e-b1ce-5d879d55c964 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:35.642522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95c2f908-cbbb-4c9e-b1ce-5d879d55c964 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:35.643876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95c2f908-cbbb-4c9e-b1ce-5d879d55c964 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:35.724432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cc2501db-ff9d-4d2e-b2f2-e028f3602f7b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:35.725579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cc2501db-ff9d-4d2e-b2f2-e028f3602f7b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:35.732041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:35.734121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc2501db-ff9d-4d2e-b2f2-e028f3602f7b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:35.735120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc2501db-ff9d-4d2e-b2f2-e028f3602f7b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:35.735753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc2501db-ff9d-4d2e-b2f2-e028f3602f7b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:35.736792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc2501db-ff9d-4d2e-b2f2-e028f3602f7b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:35.737495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc2501db-ff9d-4d2e-b2f2-e028f3602f7b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:35.738081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc2501db-ff9d-4d2e-b2f2-e028f3602f7b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:35.883603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cc2501db-ff9d-4d2e-b2f2-e028f3602f7b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:35.888210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:35.899411 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cc2501db-ff9d-4d2e-b2f2-e028f3602f7b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1268 microversion: 2.1 time: 0.180390 Sep 27 10:06:35.900400 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1359/2488] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:35 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1268 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:35.927593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:35.928844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:35.940583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:35.941192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:35.942710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:35.942710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:35.943114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:35.943539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:36.005684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-95c2f908-cbbb-4c9e-b1ce-5d879d55c964 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:36.016517 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-95c2f908-cbbb-4c9e-b1ce-5d879d55c964 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.409323 Sep 27 10:06:36.017699 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1130/2489] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:35 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:36.071514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:36.081748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:36.085198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:36.093000 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4aa0b6a7-72b4-4574-830a-7f782f192a42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.191254 Sep 27 10:06:36.094086 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1360/2490] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:35 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:36.315106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-58cc3bf3-5a04-44dc-83cd-c4bb8727ba6e req-64a285d3-b2f8-4d99-a0cf-e320e5da425b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5b5bcf59-770f-40f1-83b0-0738578e921c", "tag": "f3e142bd-656b-42c1-b4a6-376501c219f9"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:36.325658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-58cc3bf3-5a04-44dc-83cd-c4bb8727ba6e req-64a285d3-b2f8-4d99-a0cf-e320e5da425b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:36.326247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-58cc3bf3-5a04-44dc-83cd-c4bb8727ba6e req-64a285d3-b2f8-4d99-a0cf-e320e5da425b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:36.326556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-58cc3bf3-5a04-44dc-83cd-c4bb8727ba6e req-64a285d3-b2f8-4d99-a0cf-e320e5da425b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:36.344501 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-58cc3bf3-5a04-44dc-83cd-c4bb8727ba6e req-64a285d3-b2f8-4d99-a0cf-e320e5da425b service nova] Creating event network-changed:f3e142bd-656b-42c1-b4a6-376501c219f9 for instance 5b5bcf59-770f-40f1-83b0-0738578e921c on nch2-master-1 Sep 27 10:06:36.350127 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-58cc3bf3-5a04-44dc-83cd-c4bb8727ba6e req-64a285d3-b2f8-4d99-a0cf-e320e5da425b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038992 Sep 27 10:06:36.351558 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1131/2491] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:36.371569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cce5dd25-4273-44a7-8c26-5ff0efc0dac1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:36.372807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cce5dd25-4273-44a7-8c26-5ff0efc0dac1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:36.380826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cce5dd25-4273-44a7-8c26-5ff0efc0dac1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:36.381323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cce5dd25-4273-44a7-8c26-5ff0efc0dac1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:36.382230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cce5dd25-4273-44a7-8c26-5ff0efc0dac1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:36.382681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cce5dd25-4273-44a7-8c26-5ff0efc0dac1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:36.383142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cce5dd25-4273-44a7-8c26-5ff0efc0dac1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:36.383569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cce5dd25-4273-44a7-8c26-5ff0efc0dac1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:36.524777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a17c7780-00eb-4467-a9de-ce549340076e tempest-QuotaClassesAdminTestJSON-1942772469 tempest-QuotaClassesAdminTestJSON-1942772469-project-admin] Calling method '.version_select at 0x7f7f62b8c700>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:36.537497 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a17c7780-00eb-4467-a9de-ce549340076e tempest-QuotaClassesAdminTestJSON-1942772469 tempest-QuotaClassesAdminTestJSON-1942772469-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 303 microversion: 2.1 time: 0.170473 Sep 27 10:06:36.546926 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1132/2492] 10.222.0.26 () {58 vars in 1185 bytes} [Tue Sep 27 10:06:36 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 303 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:36.552500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca60a956-c219-46da-baf7-151eb00a3f8d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f637f3e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:36.603986 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ca60a956-c219-46da-baf7-151eb00a3f8d tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.055403 Sep 27 10:06:36.604699 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1133/2493] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:36 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:36.610465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-497b7095-4cce-49cf-80fb-6f03fe0442a6 tempest-QuotaClassesAdminTestJSON-1942772469 tempest-QuotaClassesAdminTestJSON-1942772469-project-admin] Action: 'update', calling method: .version_select at 0x7f7f62b5ad30>, body: {"quota_class_set": {"cores": 120, "fixed_ips": 99, "floating_ips": 99, "injected_file_content_bytes": 10340, "injected_file_path_bytes": 355, "injected_files": 105, "instances": 110, "key_pairs": 200, "metadata_items": 228, "ram": 51300, "security_group_rules": 99, "security_groups": 99}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:36.621778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=9,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:36.639199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cce5dd25-4273-44a7-8c26-5ff0efc0dac1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:36.650605 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cce5dd25-4273-44a7-8c26-5ff0efc0dac1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1323 microversion: 2.1 time: 0.284290 Sep 27 10:06:36.665006 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1361/2494] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:36 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1323 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:36.665006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0b729ec6-1b49-4328-9658-716e0aa69dab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:36.665006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0b729ec6-1b49-4328-9658-716e0aa69dab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:36.665006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b729ec6-1b49-4328-9658-716e0aa69dab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:36.665006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b729ec6-1b49-4328-9658-716e0aa69dab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:36.670758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b729ec6-1b49-4328-9658-716e0aa69dab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:36.670758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b729ec6-1b49-4328-9658-716e0aa69dab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:36.670758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b729ec6-1b49-4328-9658-716e0aa69dab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:36.670758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b729ec6-1b49-4328-9658-716e0aa69dab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:36.895733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0b729ec6-1b49-4328-9658-716e0aa69dab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:36.912944 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-497b7095-4cce-49cf-80fb-6f03fe0442a6 tempest-QuotaClassesAdminTestJSON-1942772469 tempest-QuotaClassesAdminTestJSON-1942772469-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 290 microversion: 2.1 time: 0.306794 Sep 27 10:06:36.913943 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1134/2495] 10.222.0.26 () {60 vars in 1206 bytes} [Tue Sep 27 10:06:36 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 290 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:36.918426 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0b729ec6-1b49-4328-9658-716e0aa69dab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1316 microversion: 2.1 time: 0.264931 Sep 27 10:06:36.927840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cbb87784-95f4-4ceb-85bd-3549c54484e3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:36.928794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cbb87784-95f4-4ceb-85bd-3549c54484e3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:36.936169 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1362/2496] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:36 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1316 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:36.937633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb87784-95f4-4ceb-85bd-3549c54484e3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:36.938725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb87784-95f4-4ceb-85bd-3549c54484e3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:36.939422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb87784-95f4-4ceb-85bd-3549c54484e3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:36.940298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb87784-95f4-4ceb-85bd-3549c54484e3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:36.940940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb87784-95f4-4ceb-85bd-3549c54484e3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:36.941682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb87784-95f4-4ceb-85bd-3549c54484e3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:36.943011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-218bc81e-092c-4693-bf3b-0d624ba224fc tempest-QuotaClassesAdminTestJSON-1942772469 tempest-QuotaClassesAdminTestJSON-1942772469-project-admin] Calling method '.version_select at 0x7f7f6296a940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:36.955631 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-218bc81e-092c-4693-bf3b-0d624ba224fc tempest-QuotaClassesAdminTestJSON-1942772469 tempest-QuotaClassesAdminTestJSON-1942772469-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-class-sets/default" status: 200 len: 307 microversion: 2.1 time: 0.018219 Sep 27 10:06:36.956718 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1363/2497] 10.222.0.26 () {58 vars in 1185 bytes} [Tue Sep 27 10:06:36 2022] GET /compute/v2.1/os-quota-class-sets/default => generated 307 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:36.961412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-9f332123-f545-42af-8049-06041a60a3c9 req-4350e294-8945-427b-a155-139c4abd1f49 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0abee65c-58ee-4b9c-ac6c-65631519b216", "name": "network-vif-plugged", "status": "completed", "tag": "447ad707-a60a-4017-9b0d-e87acd64bee0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:36.972181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f332123-f545-42af-8049-06041a60a3c9 req-4350e294-8945-427b-a155-139c4abd1f49 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:36.973022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f332123-f545-42af-8049-06041a60a3c9 req-4350e294-8945-427b-a155-139c4abd1f49 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:36.974075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f332123-f545-42af-8049-06041a60a3c9 req-4350e294-8945-427b-a155-139c4abd1f49 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:36.989641 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-9f332123-f545-42af-8049-06041a60a3c9 req-4350e294-8945-427b-a155-139c4abd1f49 service nova] Creating event network-vif-plugged:447ad707-a60a-4017-9b0d-e87acd64bee0 for instance 0abee65c-58ee-4b9c-ac6c-65631519b216 on nch1-master-1 Sep 27 10:06:36.995455 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-9f332123-f545-42af-8049-06041a60a3c9 req-4350e294-8945-427b-a155-139c4abd1f49 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037508 Sep 27 10:06:36.996461 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1364/2498] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:37.002999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ea49e2aa-6661-476d-868f-b5ea71fff006 tempest-QuotaClassesAdminTestJSON-1942772469 tempest-QuotaClassesAdminTestJSON-1942772469-project-admin] Action: 'update', calling method: .version_select at 0x7f7f62b8db80>, body: {"quota_class_set": {"cores": 20, "fixed_ips": -1, "floating_ips": -1, "injected_file_content_bytes": 10240, "injected_file_path_bytes": 255, "injected_files": 5, "instances": 10, "key_pairs": 100, "metadata_items": 128, "ram": 51200, "security_group_rules": -1, "security_groups": -1}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:37.022834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cbb87784-95f4-4ceb-85bd-3549c54484e3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:37.032770 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cbb87784-95f4-4ceb-85bd-3549c54484e3 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1274 microversion: 2.1 time: 0.110238 Sep 27 10:06:37.045217 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1135/2499] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:36 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1274 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:37.052366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca7c6a2f-cc9b-4443-a862-8cad4bea32fb tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:37.052963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ca7c6a2f-cc9b-4443-a862-8cad4bea32fb tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:37.061445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca7c6a2f-cc9b-4443-a862-8cad4bea32fb tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:37.062074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca7c6a2f-cc9b-4443-a862-8cad4bea32fb tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:37.062611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca7c6a2f-cc9b-4443-a862-8cad4bea32fb tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:37.063864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca7c6a2f-cc9b-4443-a862-8cad4bea32fb tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:37.064447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca7c6a2f-cc9b-4443-a862-8cad4bea32fb tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:37.064982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca7c6a2f-cc9b-4443-a862-8cad4bea32fb tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:37.147224 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ea49e2aa-6661-476d-868f-b5ea71fff006 tempest-QuotaClassesAdminTestJSON-1942772469 tempest-QuotaClassesAdminTestJSON-1942772469-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-class-sets/default" status: 200 len: 286 microversion: 2.1 time: 0.149116 Sep 27 10:06:37.155653 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1365/2500] 10.222.0.26 () {60 vars in 1206 bytes} [Tue Sep 27 10:06:36 2022] PUT /compute/v2.1/os-quota-class-sets/default => generated 286 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:37.155653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:37.155653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:37.162206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:37.162761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:37.163369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:37.164126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:37.165266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:37.165867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:37.246902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ca7c6a2f-cc9b-4443-a862-8cad4bea32fb tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:37.256595 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ca7c6a2f-cc9b-4443-a862-8cad4bea32fb tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.210113 Sep 27 10:06:37.268712 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1136/2501] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:37 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:37.286419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:37.298752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:37.300202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:37.308220 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a287043f-0695-423c-9fce-f262222b4499 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.159058 Sep 27 10:06:37.315752 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1366/2502] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:37 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:37.622239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fa17acf2-cd70-45d9-9143-fc168c851641 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62b404c0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:37.678716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9a8804c3-cd19-4357-9ba3-d139ce97b6fe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:37.682307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9a8804c3-cd19-4357-9ba3-d139ce97b6fe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:37.692840 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fa17acf2-cd70-45d9-9143-fc168c851641 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.071290 Sep 27 10:06:37.695229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a8804c3-cd19-4357-9ba3-d139ce97b6fe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:37.695953 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1137/2503] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:37 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:37.696407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a8804c3-cd19-4357-9ba3-d139ce97b6fe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:37.697068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a8804c3-cd19-4357-9ba3-d139ce97b6fe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:37.697982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a8804c3-cd19-4357-9ba3-d139ce97b6fe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:37.698583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a8804c3-cd19-4357-9ba3-d139ce97b6fe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:37.704108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a8804c3-cd19-4357-9ba3-d139ce97b6fe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:37.944223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fa344346-8030-412e-95ce-1c701f18e496 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:37.945999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa344346-8030-412e-95ce-1c701f18e496 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:37.956450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa344346-8030-412e-95ce-1c701f18e496 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:37.957270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa344346-8030-412e-95ce-1c701f18e496 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:37.957997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa344346-8030-412e-95ce-1c701f18e496 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:37.958926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa344346-8030-412e-95ce-1c701f18e496 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:37.960074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa344346-8030-412e-95ce-1c701f18e496 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:37.960721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa344346-8030-412e-95ce-1c701f18e496 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.028809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9a8804c3-cd19-4357-9ba3-d139ce97b6fe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:38.039554 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9a8804c3-cd19-4357-9ba3-d139ce97b6fe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1323 microversion: 2.1 time: 0.364856 Sep 27 10:06:38.040246 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1367/2504] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:37 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1323 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:38.068198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5cf8fc60-6031-49f4-96d5-8951be2bd279 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:38.069112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5cf8fc60-6031-49f4-96d5-8951be2bd279 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:38.075301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cf8fc60-6031-49f4-96d5-8951be2bd279 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:38.075796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cf8fc60-6031-49f4-96d5-8951be2bd279 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:38.076232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cf8fc60-6031-49f4-96d5-8951be2bd279 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.076968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cf8fc60-6031-49f4-96d5-8951be2bd279 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:38.077422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cf8fc60-6031-49f4-96d5-8951be2bd279 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:38.077847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cf8fc60-6031-49f4-96d5-8951be2bd279 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.156330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fa344346-8030-412e-95ce-1c701f18e496 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:38.164555 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fa344346-8030-412e-95ce-1c701f18e496 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1316 microversion: 2.1 time: 0.224981 Sep 27 10:06:38.165233 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1138/2505] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:37 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1316 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:38.281830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d50df6e0-1009-422d-a8d1-7679dca2117b tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:38.282809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d50df6e0-1009-422d-a8d1-7679dca2117b tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:38.290534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50df6e0-1009-422d-a8d1-7679dca2117b tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:38.291064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50df6e0-1009-422d-a8d1-7679dca2117b tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:38.291658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50df6e0-1009-422d-a8d1-7679dca2117b tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.292443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50df6e0-1009-422d-a8d1-7679dca2117b tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:38.292953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50df6e0-1009-422d-a8d1-7679dca2117b tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:38.293635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50df6e0-1009-422d-a8d1-7679dca2117b tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.350600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5cf8fc60-6031-49f4-96d5-8951be2bd279 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:38.361488 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5cf8fc60-6031-49f4-96d5-8951be2bd279 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1316 microversion: 2.1 time: 0.299475 Sep 27 10:06:38.376105 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1368/2506] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:38 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1316 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:38.384057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:38.384793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:38.394344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:38.394889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:38.395473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.396473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:38.396967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:38.397433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.552986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:38.562361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:38.564131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:38.580880 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-55f20624-8165-4694-bc16-dd4ad6ec0a3a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1558 microversion: 2.91 time: 0.203494 Sep 27 10:06:38.581495 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1369/2507] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:38 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1558 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:38.631983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-100e5a37-bc4d-408f-b772-b4d671985258 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:38.632180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-100e5a37-bc4d-408f-b772-b4d671985258 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:38.644193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d50df6e0-1009-422d-a8d1-7679dca2117b tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:38.644722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-100e5a37-bc4d-408f-b772-b4d671985258 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:38.645193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-100e5a37-bc4d-408f-b772-b4d671985258 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:38.645555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-100e5a37-bc4d-408f-b772-b4d671985258 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.646285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-100e5a37-bc4d-408f-b772-b4d671985258 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:38.646767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-100e5a37-bc4d-408f-b772-b4d671985258 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:38.647176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-100e5a37-bc4d-408f-b772-b4d671985258 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.657488 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d50df6e0-1009-422d-a8d1-7679dca2117b tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.379554 Sep 27 10:06:38.658814 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1139/2508] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:38 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:06:38.691951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-100e5a37-bc4d-408f-b772-b4d671985258 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:06:38.709727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b808e10c-7b48-4b0a-85ed-6c5541ae5c04 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62b40430>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:38.781819 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b808e10c-7b48-4b0a-85ed-6c5541ae5c04 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.075770 Sep 27 10:06:38.784190 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1140/2509] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:38 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:38.838508 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-100e5a37-bc4d-408f-b772-b4d671985258 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 204 len: 0 microversion: 2.1 time: 0.210964 Sep 27 10:06:38.840930 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1370/2510] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:06:38 2022] DELETE /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 0 bytes in 213 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:06:38.848010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a11d82f-1e41-41a9-acde-6580e8c21728 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:38.848634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a11d82f-1e41-41a9-acde-6580e8c21728 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:38.857508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a11d82f-1e41-41a9-acde-6580e8c21728 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:38.858004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a11d82f-1e41-41a9-acde-6580e8c21728 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:38.862119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a11d82f-1e41-41a9-acde-6580e8c21728 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.864742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a11d82f-1e41-41a9-acde-6580e8c21728 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:38.865506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a11d82f-1e41-41a9-acde-6580e8c21728 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:38.866006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a11d82f-1e41-41a9-acde-6580e8c21728 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.917952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-2e00b086-0231-4f7d-ad83-7709119b3fb0 req-58d4995e-5001-4b7c-9f6e-384d78aec935 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ac5c4cdc-90a8-48c7-84f3-0053b32834f3", "tag": "4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:38.935182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2e00b086-0231-4f7d-ad83-7709119b3fb0 req-58d4995e-5001-4b7c-9f6e-384d78aec935 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:38.935928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2e00b086-0231-4f7d-ad83-7709119b3fb0 req-58d4995e-5001-4b7c-9f6e-384d78aec935 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:38.936449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2e00b086-0231-4f7d-ad83-7709119b3fb0 req-58d4995e-5001-4b7c-9f6e-384d78aec935 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:38.953904 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-2e00b086-0231-4f7d-ad83-7709119b3fb0 req-58d4995e-5001-4b7c-9f6e-384d78aec935 service nova] Creating event network-changed:4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801 for instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 on nch1-master-1 Sep 27 10:06:38.960524 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-2e00b086-0231-4f7d-ad83-7709119b3fb0 req-58d4995e-5001-4b7c-9f6e-384d78aec935 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046871 Sep 27 10:06:38.961041 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1371/2511] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:39.012946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5b01327e-7cfd-4eed-a733-02f839814f04 req-d1105ac8-4d13-4b34-bf1b-3fcba08b3f90 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "961407e4-8e66-4f1c-88bd-6d2b4795c11d", "name": "network-vif-plugged", "status": "completed", "tag": "52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:39.023547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5b01327e-7cfd-4eed-a733-02f839814f04 req-d1105ac8-4d13-4b34-bf1b-3fcba08b3f90 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:39.024126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5b01327e-7cfd-4eed-a733-02f839814f04 req-d1105ac8-4d13-4b34-bf1b-3fcba08b3f90 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:39.024789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5b01327e-7cfd-4eed-a733-02f839814f04 req-d1105ac8-4d13-4b34-bf1b-3fcba08b3f90 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:39.036645 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5b01327e-7cfd-4eed-a733-02f839814f04 req-d1105ac8-4d13-4b34-bf1b-3fcba08b3f90 service nova] Creating event network-vif-plugged:52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f for instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d on nch2-master-1 Sep 27 10:06:39.043523 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5b01327e-7cfd-4eed-a733-02f839814f04 req-d1105ac8-4d13-4b34-bf1b-3fcba08b3f90 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034082 Sep 27 10:06:39.048099 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1372/2512] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:39.064352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-108444f1-2524-4131-a8b4-e4f86710639e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:39.064771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-108444f1-2524-4131-a8b4-e4f86710639e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:39.071383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-108444f1-2524-4131-a8b4-e4f86710639e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:39.071882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-108444f1-2524-4131-a8b4-e4f86710639e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:39.072325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-108444f1-2524-4131-a8b4-e4f86710639e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:39.073985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-108444f1-2524-4131-a8b4-e4f86710639e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:39.073985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-108444f1-2524-4131-a8b4-e4f86710639e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:39.073985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-108444f1-2524-4131-a8b4-e4f86710639e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:39.256527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-108444f1-2524-4131-a8b4-e4f86710639e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:39.265207 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-108444f1-2524-4131-a8b4-e4f86710639e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1323 microversion: 2.1 time: 0.206323 Sep 27 10:06:39.265762 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1373/2513] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:39 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1323 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:06:39.276201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5fb5c91d-e956-45b8-baed-111e5cca3919 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:39.277018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5fb5c91d-e956-45b8-baed-111e5cca3919 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:39.283506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb5c91d-e956-45b8-baed-111e5cca3919 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:39.283992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb5c91d-e956-45b8-baed-111e5cca3919 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:39.284429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb5c91d-e956-45b8-baed-111e5cca3919 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:39.285166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb5c91d-e956-45b8-baed-111e5cca3919 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:39.285627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb5c91d-e956-45b8-baed-111e5cca3919 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:39.286054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb5c91d-e956-45b8-baed-111e5cca3919 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:39.433811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5fb5c91d-e956-45b8-baed-111e5cca3919 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:39.442660 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5fb5c91d-e956-45b8-baed-111e5cca3919 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1316 microversion: 2.1 time: 0.175568 Sep 27 10:06:39.443391 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1374/2514] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:39 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1316 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:39.448372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-787f3599-a113-4caf-867f-682883f7ad9b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:39.449010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-787f3599-a113-4caf-867f-682883f7ad9b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:39.455597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787f3599-a113-4caf-867f-682883f7ad9b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:39.456086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787f3599-a113-4caf-867f-682883f7ad9b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:39.456519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787f3599-a113-4caf-867f-682883f7ad9b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:39.457269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787f3599-a113-4caf-867f-682883f7ad9b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:39.457719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787f3599-a113-4caf-867f-682883f7ad9b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:39.458136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787f3599-a113-4caf-867f-682883f7ad9b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:39.604201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-787f3599-a113-4caf-867f-682883f7ad9b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:39.613914 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-787f3599-a113-4caf-867f-682883f7ad9b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1316 microversion: 2.1 time: 0.169395 Sep 27 10:06:39.614517 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1375/2515] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:39 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1316 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:39.619591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:39.620365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:39.626988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:39.627488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:39.627923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:39.628639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:39.629244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:39.629669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:39.702530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:39.713839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:39.715679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:39.722052 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e2e71967-a761-4769-af8f-f207aaf7898b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1558 microversion: 2.91 time: 0.106529 Sep 27 10:06:39.722636 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1376/2516] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:39 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1558 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:39.727332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90303683-9e6c-4b61-b326-a04f5ade8769 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:39.728195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-90303683-9e6c-4b61-b326-a04f5ade8769 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:39.734539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90303683-9e6c-4b61-b326-a04f5ade8769 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:39.735226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90303683-9e6c-4b61-b326-a04f5ade8769 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:39.735805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90303683-9e6c-4b61-b326-a04f5ade8769 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:39.736641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90303683-9e6c-4b61-b326-a04f5ade8769 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:39.737265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90303683-9e6c-4b61-b326-a04f5ade8769 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:39.738053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90303683-9e6c-4b61-b326-a04f5ade8769 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:39.775347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a11d82f-1e41-41a9-acde-6580e8c21728 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:39.784189 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a11d82f-1e41-41a9-acde-6580e8c21728 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1554 microversion: 2.1 time: 0.940094 Sep 27 10:06:39.785191 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1141/2517] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:38 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1554 bytes in 942 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:39.806396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7c8399b5-4549-4245-ad34-51be0a4da466 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f637f3160>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:39.856440 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7c8399b5-4549-4245-ad34-51be0a4da466 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.060923 Sep 27 10:06:39.857308 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1142/2518] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:39 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:39.901932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-90303683-9e6c-4b61-b326-a04f5ade8769 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:39.910583 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-90303683-9e6c-4b61-b326-a04f5ade8769 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.186902 Sep 27 10:06:39.911270 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1377/2519] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:39 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:40.289106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e82acb0-2fed-40bf-a56c-f3d5219ab5a9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:40.290146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6e82acb0-2fed-40bf-a56c-f3d5219ab5a9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:40.305213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e82acb0-2fed-40bf-a56c-f3d5219ab5a9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:40.306635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e82acb0-2fed-40bf-a56c-f3d5219ab5a9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:40.307270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e82acb0-2fed-40bf-a56c-f3d5219ab5a9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:40.308209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e82acb0-2fed-40bf-a56c-f3d5219ab5a9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:40.308687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e82acb0-2fed-40bf-a56c-f3d5219ab5a9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:40.309118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e82acb0-2fed-40bf-a56c-f3d5219ab5a9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:40.467450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9af3b0cd-cbbe-4c55-b757-04b78670706b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:40.468567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9af3b0cd-cbbe-4c55-b757-04b78670706b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:40.475692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9af3b0cd-cbbe-4c55-b757-04b78670706b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:40.476186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9af3b0cd-cbbe-4c55-b757-04b78670706b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:40.476773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9af3b0cd-cbbe-4c55-b757-04b78670706b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:40.477666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9af3b0cd-cbbe-4c55-b757-04b78670706b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:40.478294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9af3b0cd-cbbe-4c55-b757-04b78670706b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:40.478716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9af3b0cd-cbbe-4c55-b757-04b78670706b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:40.554141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6e82acb0-2fed-40bf-a56c-f3d5219ab5a9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:40.572633 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6e82acb0-2fed-40bf-a56c-f3d5219ab5a9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1323 microversion: 2.1 time: 0.287991 Sep 27 10:06:40.579413 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1143/2520] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:40 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1323 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:40.638289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc0277b8-676a-415a-b284-ee424be786c9 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:40.639262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc0277b8-676a-415a-b284-ee424be786c9 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:40.656793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0277b8-676a-415a-b284-ee424be786c9 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:40.658640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0277b8-676a-415a-b284-ee424be786c9 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:40.659485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0277b8-676a-415a-b284-ee424be786c9 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:40.661465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0277b8-676a-415a-b284-ee424be786c9 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:40.662776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0277b8-676a-415a-b284-ee424be786c9 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:40.663450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0277b8-676a-415a-b284-ee424be786c9 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:40.706570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9af3b0cd-cbbe-4c55-b757-04b78670706b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:40.710348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:40.719242 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9af3b0cd-cbbe-4c55-b757-04b78670706b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1316 microversion: 2.1 time: 0.255593 Sep 27 10:06:40.719848 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1378/2521] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:40 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1316 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:06:40.725348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-cd91a47d-6acc-4a8e-a0bf-db3b34170809 req-94c1d590-b23b-479f-a412-eefa1b20b557 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5b5bcf59-770f-40f1-83b0-0738578e921c", "tag": "cd5a9e3b-1cef-4cbd-a955-7fc284d5c0d4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:40.745493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cd91a47d-6acc-4a8e-a0bf-db3b34170809 req-94c1d590-b23b-479f-a412-eefa1b20b557 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:40.746263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cd91a47d-6acc-4a8e-a0bf-db3b34170809 req-94c1d590-b23b-479f-a412-eefa1b20b557 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:40.746873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cd91a47d-6acc-4a8e-a0bf-db3b34170809 req-94c1d590-b23b-479f-a412-eefa1b20b557 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:40.762492 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-cd91a47d-6acc-4a8e-a0bf-db3b34170809 req-94c1d590-b23b-479f-a412-eefa1b20b557 service nova] Creating event network-changed:cd5a9e3b-1cef-4cbd-a955-7fc284d5c0d4 for instance 5b5bcf59-770f-40f1-83b0-0738578e921c on nch2-master-1 Sep 27 10:06:40.770275 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-cd91a47d-6acc-4a8e-a0bf-db3b34170809 req-94c1d590-b23b-479f-a412-eefa1b20b557 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048430 Sep 27 10:06:40.771002 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1379/2522] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:40.778179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:40.780327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:40.788762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:40.789852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:40.790831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:40.792493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:40.794908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:40.796452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:40.939645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:40.949550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:40.951059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:40.956944 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1300d9f1-f6d1-466b-9ff1-903ea6d093a9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1558 microversion: 2.91 time: 0.183881 Sep 27 10:06:40.957587 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1380/2523] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:40 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1558 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:40.969473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-786cbe43-3ee5-4787-81ba-e7b6132f341a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:40.969473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-786cbe43-3ee5-4787-81ba-e7b6132f341a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:40.979802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786cbe43-3ee5-4787-81ba-e7b6132f341a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:40.979802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786cbe43-3ee5-4787-81ba-e7b6132f341a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:40.979802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786cbe43-3ee5-4787-81ba-e7b6132f341a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:40.979802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786cbe43-3ee5-4787-81ba-e7b6132f341a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:40.979802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786cbe43-3ee5-4787-81ba-e7b6132f341a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:40.979802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786cbe43-3ee5-4787-81ba-e7b6132f341a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:41.159754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cc0277b8-676a-415a-b284-ee424be786c9 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:41.177532 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cc0277b8-676a-415a-b284-ee424be786c9 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1316 microversion: 2.1 time: 0.544726 Sep 27 10:06:41.177984 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1144/2524] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:40 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1316 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:41.182867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-57a3c082-559d-4f0f-9e3b-f9c2cdadc16f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62b029d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:41.248150 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-57a3c082-559d-4f0f-9e3b-f9c2cdadc16f tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.069065 Sep 27 10:06:41.256025 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1145/2525] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:41 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:41.262714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b10d2db8-8632-443f-8d48-7aedcf45ff53 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:41.264511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b10d2db8-8632-443f-8d48-7aedcf45ff53 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:41.282193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b10d2db8-8632-443f-8d48-7aedcf45ff53 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:41.282735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b10d2db8-8632-443f-8d48-7aedcf45ff53 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:41.286733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b10d2db8-8632-443f-8d48-7aedcf45ff53 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:41.287723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b10d2db8-8632-443f-8d48-7aedcf45ff53 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:41.290705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b10d2db8-8632-443f-8d48-7aedcf45ff53 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:41.291632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b10d2db8-8632-443f-8d48-7aedcf45ff53 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:41.502012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-786cbe43-3ee5-4787-81ba-e7b6132f341a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:41.514679 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-786cbe43-3ee5-4787-81ba-e7b6132f341a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1554 microversion: 2.1 time: 0.555838 Sep 27 10:06:41.516565 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1381/2526] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:40 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1554 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:41.598337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8aca219d-1925-4049-bd55-313db6ed56d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:41.599633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8aca219d-1925-4049-bd55-313db6ed56d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:41.615428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8aca219d-1925-4049-bd55-313db6ed56d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:41.616323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8aca219d-1925-4049-bd55-313db6ed56d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:41.617129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8aca219d-1925-4049-bd55-313db6ed56d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:41.618433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8aca219d-1925-4049-bd55-313db6ed56d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:41.622194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8aca219d-1925-4049-bd55-313db6ed56d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:41.622796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8aca219d-1925-4049-bd55-313db6ed56d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:41.754408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b10d2db8-8632-443f-8d48-7aedcf45ff53 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:41.781244 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b10d2db8-8632-443f-8d48-7aedcf45ff53 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.528762 Sep 27 10:06:41.791384 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1146/2527] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:41 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:41.811704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-12000f68-184b-4deb-8569-c0230502fea2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:41.814114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-12000f68-184b-4deb-8569-c0230502fea2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:41.847112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12000f68-184b-4deb-8569-c0230502fea2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:41.848055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12000f68-184b-4deb-8569-c0230502fea2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:41.853992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12000f68-184b-4deb-8569-c0230502fea2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:41.860579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12000f68-184b-4deb-8569-c0230502fea2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:41.863138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12000f68-184b-4deb-8569-c0230502fea2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:41.864201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12000f68-184b-4deb-8569-c0230502fea2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:42.016571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8aca219d-1925-4049-bd55-313db6ed56d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:42.033102 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8aca219d-1925-4049-bd55-313db6ed56d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1519 microversion: 2.1 time: 0.431519 Sep 27 10:06:42.034387 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1382/2528] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:41 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1519 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:06:42.039553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:42.040568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:42.056697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:42.057481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:42.058087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:42.068952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:42.069682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:42.070403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:42.098949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-12000f68-184b-4deb-8569-c0230502fea2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:42.116414 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-12000f68-184b-4deb-8569-c0230502fea2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1316 microversion: 2.1 time: 0.319379 Sep 27 10:06:42.118982 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1147/2529] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:41 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1316 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:42.135985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d898081d-702d-40ea-b684-b72920fcd0d8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:42.139334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d898081d-702d-40ea-b684-b72920fcd0d8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:42.153452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d898081d-702d-40ea-b684-b72920fcd0d8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:42.155486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d898081d-702d-40ea-b684-b72920fcd0d8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:42.156402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d898081d-702d-40ea-b684-b72920fcd0d8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:42.157970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d898081d-702d-40ea-b684-b72920fcd0d8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:42.160338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d898081d-702d-40ea-b684-b72920fcd0d8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:42.161597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d898081d-702d-40ea-b684-b72920fcd0d8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:42.207769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:42.217227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:42.222749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:42.239350 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-68e25856-5e6b-4c1a-9e3c-7addf0737e27 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.203683 Sep 27 10:06:42.240296 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1383/2530] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:42 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:42.260585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-16ca656b-e2fb-4551-a62d-4e46ae2cb8cd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:42.261942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-16ca656b-e2fb-4551-a62d-4e46ae2cb8cd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:42.285255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ca656b-e2fb-4551-a62d-4e46ae2cb8cd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:42.285255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ca656b-e2fb-4551-a62d-4e46ae2cb8cd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:42.285255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ca656b-e2fb-4551-a62d-4e46ae2cb8cd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:42.285255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ca656b-e2fb-4551-a62d-4e46ae2cb8cd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:42.285255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ca656b-e2fb-4551-a62d-4e46ae2cb8cd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:42.285255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ca656b-e2fb-4551-a62d-4e46ae2cb8cd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:42.584851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-16ca656b-e2fb-4551-a62d-4e46ae2cb8cd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:42.600198 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-16ca656b-e2fb-4551-a62d-4e46ae2cb8cd tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1316 microversion: 2.1 time: 0.358403 Sep 27 10:06:42.601148 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1384/2531] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:42 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1316 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:42.625545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-35a84d0b-13bb-457c-ae73-de8fd574ab44 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f628ca700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:42.681717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d898081d-702d-40ea-b684-b72920fcd0d8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:42.694254 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d898081d-702d-40ea-b684-b72920fcd0d8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1519 microversion: 2.1 time: 0.573270 Sep 27 10:06:42.698591 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1148/2532] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:42 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1519 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:42.706792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e7bb3949-c7ed-4af6-9cc5-1c51b8b8a8ff req-39455f1b-5e1a-4203-9c4b-79c903c2bd0f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4e215381-8752-4c41-aae8-d3b564df3dee", "name": "network-vif-deleted", "tag": "a3f8b8ee-e42c-499e-a75f-6895f1ce45bd"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:42.743968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e7bb3949-c7ed-4af6-9cc5-1c51b8b8a8ff req-39455f1b-5e1a-4203-9c4b-79c903c2bd0f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:42.744805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e7bb3949-c7ed-4af6-9cc5-1c51b8b8a8ff req-39455f1b-5e1a-4203-9c4b-79c903c2bd0f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:42.745581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e7bb3949-c7ed-4af6-9cc5-1c51b8b8a8ff req-39455f1b-5e1a-4203-9c4b-79c903c2bd0f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:42.765837 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-35a84d0b-13bb-457c-ae73-de8fd574ab44 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.161639 Sep 27 10:06:42.766451 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1385/2533] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:42 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:42.782134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eb2e1095-6957-4d2c-aeb9-8a935b454fa6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:42.782773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eb2e1095-6957-4d2c-aeb9-8a935b454fa6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:42.811100 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e7bb3949-c7ed-4af6-9cc5-1c51b8b8a8ff req-39455f1b-5e1a-4203-9c4b-79c903c2bd0f service nova] Creating event network-vif-deleted:a3f8b8ee-e42c-499e-a75f-6895f1ce45bd for instance 4e215381-8752-4c41-aae8-d3b564df3dee on nch1-master-1 Sep 27 10:06:42.822829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2e1095-6957-4d2c-aeb9-8a935b454fa6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:42.823498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2e1095-6957-4d2c-aeb9-8a935b454fa6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:42.824035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2e1095-6957-4d2c-aeb9-8a935b454fa6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:42.824691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e7bb3949-c7ed-4af6-9cc5-1c51b8b8a8ff req-39455f1b-5e1a-4203-9c4b-79c903c2bd0f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.124587 Sep 27 10:06:42.825024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2e1095-6957-4d2c-aeb9-8a935b454fa6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:42.825575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2e1095-6957-4d2c-aeb9-8a935b454fa6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:42.826002 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1149/2534] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:42.826590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2e1095-6957-4d2c-aeb9-8a935b454fa6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:42.832865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9e3869ce-f790-4524-8bff-01674f1d34fc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1780544210", "metadata": {"": ""}}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9e3869ce-f790-4524-8bff-01674f1d34fc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action HTTP/1.1 Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 107 Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1780544210", "metadata": {"": ""}}} Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '0abee65c-58ee-4b9c-ac6c-65631519b216', 'body': {'createImage': {'name': 'tempest-ImagesOneServerNegativeTestJSON-image-1780544210', 'metadata': {'': ''}}}} Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['createImage']['properties']['metadata']: Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'object'} Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['createImage']: Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'': ''} Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:06:42.847506 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:06:42.852120 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:42.854179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9e3869ce-f790-4524-8bff-01674f1d34fc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'': ''}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:06:42.855054 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9e3869ce-f790-4524-8bff-01674f1d34fc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action" status: 400 len: 170 microversion: 2.1 time: 0.027564 Sep 27 10:06:42.855645 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1150/2535] 10.222.0.26 () {60 vars in 1279 bytes} [Tue Sep 27 10:06:42 2022] POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action => generated 170 bytes in 29 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:06:42.865763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73043431-f8ce-4bec-8349-3b2626288ce5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:42.866560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-73043431-f8ce-4bec-8349-3b2626288ce5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:42.888853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73043431-f8ce-4bec-8349-3b2626288ce5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:42.889521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73043431-f8ce-4bec-8349-3b2626288ce5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:42.889969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73043431-f8ce-4bec-8349-3b2626288ce5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:42.890768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73043431-f8ce-4bec-8349-3b2626288ce5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:42.891216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73043431-f8ce-4bec-8349-3b2626288ce5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:42.891782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73043431-f8ce-4bec-8349-3b2626288ce5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:43.032700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-eb2e1095-6957-4d2c-aeb9-8a935b454fa6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:43.045127 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eb2e1095-6957-4d2c-aeb9-8a935b454fa6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 200 len: 1512 microversion: 2.1 time: 0.277442 Sep 27 10:06:43.067438 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1386/2536] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:42 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 1512 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:43.078596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fc1daa74-e122-4c84-986c-d000283d3113 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:43.079601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fc1daa74-e122-4c84-986c-d000283d3113 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:43.121023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1daa74-e122-4c84-986c-d000283d3113 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:43.125304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1daa74-e122-4c84-986c-d000283d3113 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:43.135074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1daa74-e122-4c84-986c-d000283d3113 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:43.139707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1daa74-e122-4c84-986c-d000283d3113 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:43.142114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1daa74-e122-4c84-986c-d000283d3113 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:43.145166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1daa74-e122-4c84-986c-d000283d3113 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:43.468567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc1daa74-e122-4c84-986c-d000283d3113 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:43.478366 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fc1daa74-e122-4c84-986c-d000283d3113 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1519 microversion: 2.1 time: 0.408320 Sep 27 10:06:43.479558 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1387/2537] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:43 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1519 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:43.488619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d3304344-330e-4cb3-bca6-72f4e281722c req-5621f9c9-8b66-4974-a2be-02ef9eba7b07 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5b5bcf59-770f-40f1-83b0-0738578e921c", "tag": "2acf9231-7703-4c8c-83e8-58ba465f8803"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:43.500867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-73043431-f8ce-4bec-8349-3b2626288ce5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:43.516001 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-73043431-f8ce-4bec-8349-3b2626288ce5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.658274 Sep 27 10:06:43.530769 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1151/2538] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:42 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 673 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:43.547466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d3304344-330e-4cb3-bca6-72f4e281722c req-5621f9c9-8b66-4974-a2be-02ef9eba7b07 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:43.548568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d3304344-330e-4cb3-bca6-72f4e281722c req-5621f9c9-8b66-4974-a2be-02ef9eba7b07 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:43.548769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0f7bac17-291d-4569-9021-c5d0d01cdc11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:43.550216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0f7bac17-291d-4569-9021-c5d0d01cdc11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:43.550840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d3304344-330e-4cb3-bca6-72f4e281722c req-5621f9c9-8b66-4974-a2be-02ef9eba7b07 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:43.571127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f7bac17-291d-4569-9021-c5d0d01cdc11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:43.571876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f7bac17-291d-4569-9021-c5d0d01cdc11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:43.572493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f7bac17-291d-4569-9021-c5d0d01cdc11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:43.574535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f7bac17-291d-4569-9021-c5d0d01cdc11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:43.577414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f7bac17-291d-4569-9021-c5d0d01cdc11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:43.578062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f7bac17-291d-4569-9021-c5d0d01cdc11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:43.582733 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d3304344-330e-4cb3-bca6-72f4e281722c req-5621f9c9-8b66-4974-a2be-02ef9eba7b07 service nova] Creating event network-changed:2acf9231-7703-4c8c-83e8-58ba465f8803 for instance 5b5bcf59-770f-40f1-83b0-0738578e921c on nch2-master-1 Sep 27 10:06:43.590292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:43.597040 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d3304344-330e-4cb3-bca6-72f4e281722c req-5621f9c9-8b66-4974-a2be-02ef9eba7b07 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.116040 Sep 27 10:06:43.614126 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1388/2539] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:43.845168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-296ce148-df8b-4b63-bd60-e772d1497432 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Calling method '.version_select at 0x7f7f628ca1f0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:43.919284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0f7bac17-291d-4569-9021-c5d0d01cdc11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:43.921984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=87,nova_api:UPDATE=13 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:43.929735 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0f7bac17-291d-4569-9021-c5d0d01cdc11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1316 microversion: 2.1 time: 0.397124 Sep 27 10:06:43.933742 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1152/2540] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:43 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1316 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:43.946882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:43.949004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:43.960916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:43.962217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:43.962874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:43.963768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:43.964384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:43.965589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:43.975929 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-296ce148-df8b-4b63-bd60-e772d1497432 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/86cb911098764324864c44a5d4ea9fcb" status: 200 len: 371 microversion: 2.1 time: 0.360409 Sep 27 10:06:43.976750 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1389/2541] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:06:43 2022] GET /compute/v2.1/os-quota-sets/86cb911098764324864c44a5d4ea9fcb => generated 371 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:43.993629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-79feac6c-210b-491c-9b93-5dc2c9f69c43 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:43.994396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-79feac6c-210b-491c-9b93-5dc2c9f69c43 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:44.025769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79feac6c-210b-491c-9b93-5dc2c9f69c43 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:44.026292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79feac6c-210b-491c-9b93-5dc2c9f69c43 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:44.026744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79feac6c-210b-491c-9b93-5dc2c9f69c43 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:44.027492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79feac6c-210b-491c-9b93-5dc2c9f69c43 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:44.033039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79feac6c-210b-491c-9b93-5dc2c9f69c43 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:44.034153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79feac6c-210b-491c-9b93-5dc2c9f69c43 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:44.318036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:44.334477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:44.358895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:44.362268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:44.371155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-79feac6c-210b-491c-9b93-5dc2c9f69c43 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:44.381460 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2d8523c9-fab2-4966-ba12-42c28c43c744 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.445910 Sep 27 10:06:44.397082 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1153/2542] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:43 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:44.398562 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-79feac6c-210b-491c-9b93-5dc2c9f69c43 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1519 microversion: 2.1 time: 0.410240 Sep 27 10:06:44.405719 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1390/2543] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:43 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1519 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:44.409120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0be07ae9-2354-42eb-99b8-aae3998502a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:44.409976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0be07ae9-2354-42eb-99b8-aae3998502a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:44.422713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-94b71525-c89e-49d2-b77d-d695f937841e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f629701f0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:44.426181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0be07ae9-2354-42eb-99b8-aae3998502a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:44.427306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0be07ae9-2354-42eb-99b8-aae3998502a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:44.427985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0be07ae9-2354-42eb-99b8-aae3998502a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:44.429054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0be07ae9-2354-42eb-99b8-aae3998502a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:44.437757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0be07ae9-2354-42eb-99b8-aae3998502a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:44.438943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0be07ae9-2354-42eb-99b8-aae3998502a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:44.808301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0be07ae9-2354-42eb-99b8-aae3998502a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:44.824070 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0be07ae9-2354-42eb-99b8-aae3998502a8 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1316 microversion: 2.1 time: 0.421566 Sep 27 10:06:44.825880 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1154/2544] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:44 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1316 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:44.845316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ba1f7a09-2ae6-4975-a844-63a939a4f597 req-73f181d7-987e-4382-91d6-f4311763da66 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3327ff95-5c05-4167-a73b-5fd55f7bfabd", "name": "network-vif-plugged", "status": "completed", "tag": "d571b78e-f1b6-4353-b09e-cac4cd18cde7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:44.860247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ba1f7a09-2ae6-4975-a844-63a939a4f597 req-73f181d7-987e-4382-91d6-f4311763da66 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:44.861110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ba1f7a09-2ae6-4975-a844-63a939a4f597 req-73f181d7-987e-4382-91d6-f4311763da66 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:44.861709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ba1f7a09-2ae6-4975-a844-63a939a4f597 req-73f181d7-987e-4382-91d6-f4311763da66 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:44.877246 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ba1f7a09-2ae6-4975-a844-63a939a4f597 req-73f181d7-987e-4382-91d6-f4311763da66 service nova] Creating event network-vif-plugged:d571b78e-f1b6-4353-b09e-cac4cd18cde7 for instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd on nch2-master-1 Sep 27 10:06:44.885989 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ba1f7a09-2ae6-4975-a844-63a939a4f597 req-73f181d7-987e-4382-91d6-f4311763da66 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058513 Sep 27 10:06:44.886785 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1155/2545] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:44.893080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f0d9bee6-e3f1-4439-8118-880b3506aaa6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Action: 'update', calling method: .version_select at 0x7f7f63760940>, body: {"quota_set": {"ram": "5120"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:45.155651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=46,nova_cell0:UPDATE=7 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:45.157383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0d9bee6-e3f1-4439-8118-880b3506aaa6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:45.157896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0d9bee6-e3f1-4439-8118-880b3506aaa6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:45.158373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0d9bee6-e3f1-4439-8118-880b3506aaa6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:45.159105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0d9bee6-e3f1-4439-8118-880b3506aaa6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:45.161027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0d9bee6-e3f1-4439-8118-880b3506aaa6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:45.161768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0d9bee6-e3f1-4439-8118-880b3506aaa6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:45.293742 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f0d9bee6-e3f1-4439-8118-880b3506aaa6 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/86cb911098764324864c44a5d4ea9fcb" status: 200 len: 328 microversion: 2.1 time: 0.404806 Sep 27 10:06:45.294928 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1156/2546] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 10:06:44 2022] PUT /compute/v2.1/os-quota-sets/86cb911098764324864c44a5d4ea9fcb => generated 328 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:45.302891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:45.305495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:45.316890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:45.317544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:45.318058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:45.318860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:45.319390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:45.319887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:45.345535 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee HTTP/1.1 Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '4e215381-8752-4c41-aae8-d3b564df3dee'} Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:45.347482 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.352385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:45.352385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:45.352385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:45.352385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:06:45.352385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:06:45.352385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:06:45.352385 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.352385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:06:45.357253 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e4984acd-6848-43cc-ae38-dde424e3a15d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 404 len: 111 microversion: 2.1 time: 0.060200 Sep 27 10:06:45.359240 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1157/2547] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:45 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:06:45.389231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a6879f8-827d-4243-805c-a68950abcedc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-9286881", "metadata": {"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a6879f8-827d-4243-805c-a68950abcedc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action HTTP/1.1 Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 616 Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-9286881", "metadata": {"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}}} Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '0abee65c-58ee-4b9c-ac6c-65631519b216', 'body': {'createImage': {'name': 'tempest-ImagesOneServerNegativeTestJSON-image-9286881', 'metadata': {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}}}} Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:06:45.413372 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['createImage']['properties']['metadata']: Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'object'} Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['createImage']: Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'} Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:06:45.416062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:45.421106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a6879f8-827d-4243-805c-a68950abcedc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:06:45.431679 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a6879f8-827d-4243-805c-a68950abcedc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action" status: 400 len: 938 microversion: 2.1 time: 0.054489 Sep 27 10:06:45.435494 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1158/2548] 10.222.0.26 () {60 vars in 1279 bytes} [Tue Sep 27 10:06:45 2022] POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action => generated 938 bytes in 73 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:06:45.482137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3950056e-0a70-485f-a783-103228438a20 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:45.487435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3950056e-0a70-485f-a783-103228438a20 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:45.537625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3950056e-0a70-485f-a783-103228438a20 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:45.538138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3950056e-0a70-485f-a783-103228438a20 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:45.538587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3950056e-0a70-485f-a783-103228438a20 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:45.545544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3950056e-0a70-485f-a783-103228438a20 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:45.546057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3950056e-0a70-485f-a783-103228438a20 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:45.546513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3950056e-0a70-485f-a783-103228438a20 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:45.874707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3950056e-0a70-485f-a783-103228438a20 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:45.883958 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3950056e-0a70-485f-a783-103228438a20 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.447733 Sep 27 10:06:45.884992 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1159/2549] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:45 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:45.890114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8265724-e728-49f9-abda-cdb444556e2d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:45.890961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f8265724-e728-49f9-abda-cdb444556e2d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:45.898056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8265724-e728-49f9-abda-cdb444556e2d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:45.898710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8265724-e728-49f9-abda-cdb444556e2d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:45.899246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8265724-e728-49f9-abda-cdb444556e2d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:45.900452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8265724-e728-49f9-abda-cdb444556e2d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:45.901027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8265724-e728-49f9-abda-cdb444556e2d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:45.901533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8265724-e728-49f9-abda-cdb444556e2d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:46.166956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f8265724-e728-49f9-abda-cdb444556e2d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:46.175945 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f8265724-e728-49f9-abda-cdb444556e2d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1316 microversion: 2.1 time: 0.289788 Sep 27 10:06:46.193599 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1160/2550] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:45 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1316 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:46.199293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5bfe5a1c-2c47-409c-b95f-46cadbd6c05e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:46.204937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=81,nova_api:UPDATE=11,nova_api:INSERT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:46.226618 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5bfe5a1c-2c47-409c-b95f-46cadbd6c05e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-quota-sets/86cb911098764324864c44a5d4ea9fcb" status: 202 len: 0 microversion: 2.1 time: 0.031213 Sep 27 10:06:46.228238 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1161/2551] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:06:46 2022] DELETE /compute/v2.1/os-quota-sets/86cb911098764324864c44a5d4ea9fcb => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:06:46.243541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:46.243541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:46.262858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:46.263346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:46.263788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:46.265901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:46.266503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:46.266941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:46.371334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:46.380272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:46.389812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:46.391645 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-49680b64-ef47-44ce-82cc-43d7cd4e9d9c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.162277 Sep 27 10:06:46.392563 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1162/2552] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:46 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:46.397573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-89af0947-4979-4d13-8a97-ef92756e0d51 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:46.398253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-89af0947-4979-4d13-8a97-ef92756e0d51 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:46.411941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89af0947-4979-4d13-8a97-ef92756e0d51 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:46.412566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89af0947-4979-4d13-8a97-ef92756e0d51 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:46.413123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89af0947-4979-4d13-8a97-ef92756e0d51 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:46.414091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89af0947-4979-4d13-8a97-ef92756e0d51 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:46.420268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89af0947-4979-4d13-8a97-ef92756e0d51 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:46.420268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-89af0947-4979-4d13-8a97-ef92756e0d51 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:46.915172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-89af0947-4979-4d13-8a97-ef92756e0d51 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:46.927589 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-89af0947-4979-4d13-8a97-ef92756e0d51 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1519 microversion: 2.1 time: 0.533834 Sep 27 10:06:46.928200 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1163/2553] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:46 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1519 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:46.963113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:46.965793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:46.984486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:46.988434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:46.991134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:46.993350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:47.007390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:47.008410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:47.034843 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee/os-volume_attachments HTTP/1.1 Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'server_id': '4e215381-8752-4c41-aae8-d3b564df3dee'} Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:06:47.046717 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:06:47.050886 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:06:47.050886 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:47.050886 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:47.050886 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:47.050886 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:47.050886 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:47.050886 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:47.050886 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:06:47.050886 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:06:47.050886 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:06:47.050886 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:47.053579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:06:47.054850 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0791e158-b66d-4f49-85a0-1c42392a093d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.103441 Sep 27 10:06:47.056239 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1164/2554] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:06:46 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee/os-volume_attachments => generated 111 bytes in 127 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:06:47.090695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd126d50-3565-4bcf-b43f-e4d3fec9344a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:47.092219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd126d50-3565-4bcf-b43f-e4d3fec9344a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:47.106466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd126d50-3565-4bcf-b43f-e4d3fec9344a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:47.107487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd126d50-3565-4bcf-b43f-e4d3fec9344a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:47.108571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd126d50-3565-4bcf-b43f-e4d3fec9344a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:47.111451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd126d50-3565-4bcf-b43f-e4d3fec9344a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:47.113471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd126d50-3565-4bcf-b43f-e4d3fec9344a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:47.116971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd126d50-3565-4bcf-b43f-e4d3fec9344a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:47.430016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd126d50-3565-4bcf-b43f-e4d3fec9344a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:47.442074 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd126d50-3565-4bcf-b43f-e4d3fec9344a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1316 microversion: 2.1 time: 0.383847 Sep 27 10:06:47.442322 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1165/2555] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:47 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1316 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:47.452891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b25c7931-4d70-4257-8444-4267d22f7ac2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Calling method '.version_select at 0x7f7f62b8b9d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:47.533255 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b25c7931-4d70-4257-8444-4267d22f7ac2 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/86cb911098764324864c44a5d4ea9fcb" status: 200 len: 371 microversion: 2.1 time: 0.089808 Sep 27 10:06:47.534592 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1166/2556] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:06:47 2022] GET /compute/v2.1/os-quota-sets/86cb911098764324864c44a5d4ea9fcb => generated 371 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:47.540079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a3aa663-b1e2-45c7-8a8c-4eec81411131 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:47.541126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a3aa663-b1e2-45c7-8a8c-4eec81411131 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:47.564067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3aa663-b1e2-45c7-8a8c-4eec81411131 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:47.565684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3aa663-b1e2-45c7-8a8c-4eec81411131 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:47.571352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3aa663-b1e2-45c7-8a8c-4eec81411131 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:47.572965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3aa663-b1e2-45c7-8a8c-4eec81411131 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:47.574502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3aa663-b1e2-45c7-8a8c-4eec81411131 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:47.576881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3aa663-b1e2-45c7-8a8c-4eec81411131 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:47.943587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a3aa663-b1e2-45c7-8a8c-4eec81411131 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:47.952147 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a3aa663-b1e2-45c7-8a8c-4eec81411131 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.415903 Sep 27 10:06:47.953503 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1167/2557] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:47 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:47.958698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5144b300-5226-4d44-9471-2e900094140d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:47.959663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5144b300-5226-4d44-9471-2e900094140d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:47.973150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5144b300-5226-4d44-9471-2e900094140d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:47.974073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5144b300-5226-4d44-9471-2e900094140d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:47.974751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5144b300-5226-4d44-9471-2e900094140d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:47.975828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5144b300-5226-4d44-9471-2e900094140d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:47.977024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5144b300-5226-4d44-9471-2e900094140d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:47.977727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5144b300-5226-4d44-9471-2e900094140d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:48.216270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5144b300-5226-4d44-9471-2e900094140d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:48.229628 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5144b300-5226-4d44-9471-2e900094140d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1519 microversion: 2.1 time: 0.274820 Sep 27 10:06:48.231720 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1168/2558] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:47 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1519 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:48.244227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b2baa0a4-d518-4786-bb67-366016acfa47 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:48.245573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b2baa0a4-d518-4786-bb67-366016acfa47 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:48.256768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2baa0a4-d518-4786-bb67-366016acfa47 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:48.258566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2baa0a4-d518-4786-bb67-366016acfa47 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:48.259950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2baa0a4-d518-4786-bb67-366016acfa47 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:48.260963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2baa0a4-d518-4786-bb67-366016acfa47 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:48.261679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2baa0a4-d518-4786-bb67-366016acfa47 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:48.262322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2baa0a4-d518-4786-bb67-366016acfa47 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:48.489140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b2baa0a4-d518-4786-bb67-366016acfa47 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:48.500241 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b2baa0a4-d518-4786-bb67-366016acfa47 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.267181 Sep 27 10:06:48.501802 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1169/2559] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:48 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:48.520621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-53e283e3-eea8-4cc1-83dd-a43cb889f848 req-03ab3bd4-bfb1-4c20-8c9d-721bc2099844 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ac5c4cdc-90a8-48c7-84f3-0053b32834f3", "name": "network-vif-plugged", "status": "completed", "tag": "4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:48.566028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-53e283e3-eea8-4cc1-83dd-a43cb889f848 req-03ab3bd4-bfb1-4c20-8c9d-721bc2099844 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:48.573456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-53e283e3-eea8-4cc1-83dd-a43cb889f848 req-03ab3bd4-bfb1-4c20-8c9d-721bc2099844 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:48.574300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-53e283e3-eea8-4cc1-83dd-a43cb889f848 req-03ab3bd4-bfb1-4c20-8c9d-721bc2099844 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:48.598905 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-53e283e3-eea8-4cc1-83dd-a43cb889f848 req-03ab3bd4-bfb1-4c20-8c9d-721bc2099844 service nova] Creating event network-vif-plugged:4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801 for instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 on nch1-master-1 Sep 27 10:06:48.610566 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-53e283e3-eea8-4cc1-83dd-a43cb889f848 req-03ab3bd4-bfb1-4c20-8c9d-721bc2099844 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.107109 Sep 27 10:06:48.611247 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1170/2560] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:48.617025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1937771479", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "0117c46b-39f2-4cf5-9f8a-7710125bdf55"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:53.955018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:54.015910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=5 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:54.376727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=52 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:54.865270 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-94b71525-c89e-49d2-b77d-d695f937841e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 10.462582 Sep 27 10:06:54.871250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] validate_networks() for [('0117c46b-39f2-4cf5-9f8a-7710125bdf55', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:06:54.874648 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1391/2561] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:44 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 10474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:54.880563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:54.881194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:54.888243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:54.888802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:54.889251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:54.890060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:54.890698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:54.891135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:54.980929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:54.993427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:54.995257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:55.001375 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ca014bfe-1237-4311-906b-fb10fe2aac75 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.125380 Sep 27 10:06:55.002286 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1392/2562] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:54 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:55.007506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e13ff894-2643-4811-aa36-57be423d0246 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Calling method '.version_select at 0x7f7f628cf310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:55.186006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:55.333142 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e13ff894-2643-4811-aa36-57be423d0246 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/d59d3d57f7ee4b08a2fb841333c7212e/defaults" status: 200 len: 371 microversion: 2.1 time: 0.329083 Sep 27 10:06:55.334157 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1393/2563] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:06:55 2022] GET /compute/v2.1/os-quota-sets/d59d3d57f7ee4b08a2fb841333c7212e/defaults => generated 371 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:55.356424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c717ef7b-0048-4c31-8dd6-3b1c5c304ac2 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Action: 'action', calling method: >, body: {"createImage": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c717ef7b-0048-4c31-8dd6-3b1c5c304ac2 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action HTTP/1.1 Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 285 Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: {"createImage": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '0abee65c-58ee-4b9c-ac6c-65631519b216', 'body': {'createImage': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'}}} Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['createImage']['properties']['name']: Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['createImage']: Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:55.361394 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:06:55.366403 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:55.370678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c717ef7b-0048-4c31-8dd6-3b1c5c304ac2 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:06:55.371986 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c717ef7b-0048-4c31-8dd6-3b1c5c304ac2 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action" status: 400 len: 619 microversion: 2.1 time: 0.025052 Sep 27 10:06:55.372779 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1394/2564] 10.222.0.26 () {60 vars in 1279 bytes} [Tue Sep 27 10:06:55 2022] POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action => generated 619 bytes in 27 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:06:55.382896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-79e228b0-9b13-4e78-8591-f2cc4605555f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:55.383847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-79e228b0-9b13-4e78-8591-f2cc4605555f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:55.394334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79e228b0-9b13-4e78-8591-f2cc4605555f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:55.395065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79e228b0-9b13-4e78-8591-f2cc4605555f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:55.395697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79e228b0-9b13-4e78-8591-f2cc4605555f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:55.396599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79e228b0-9b13-4e78-8591-f2cc4605555f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:55.397330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79e228b0-9b13-4e78-8591-f2cc4605555f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:55.398331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79e228b0-9b13-4e78-8591-f2cc4605555f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:55.495834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:06:55.496265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:06:55.496812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:06:55.497233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:06:55.497701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:06:55.517026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:55.517554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:55.518003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:55.519289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:55.519850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:55.520806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:55.546697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:06:55.559111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for user 7f7c5adbac4f48b6bc141e0f888f2496 and project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:06:55.575017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:06:55.578132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:06:55.646929 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4bb29660-ffd0-4fbc-9db3-e1977433619b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 7.034495 Sep 27 10:06:55.648320 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1171/2565] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:06:48 2022] POST /compute/v2.1/servers => generated 380 bytes in 7036 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:06:55.654469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-179783951", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:55.762008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] validate_networks() for [('fabc7738-7394-4084-bbfe-24c62c6029b4', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:06:56.150267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-79e228b0-9b13-4e78-8591-f2cc4605555f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:56.165803 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-79e228b0-9b13-4e78-8591-f2cc4605555f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1504 microversion: 2.1 time: 0.787951 Sep 27 10:06:56.168978 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1395/2566] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:55 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1504 bytes in 792 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:56.190162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c4c292cf-eba1-4fda-b6a8-52197b035807 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:56.195398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c4c292cf-eba1-4fda-b6a8-52197b035807 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:56.208182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4c292cf-eba1-4fda-b6a8-52197b035807 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.208999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4c292cf-eba1-4fda-b6a8-52197b035807 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.209679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4c292cf-eba1-4fda-b6a8-52197b035807 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:56.210620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4c292cf-eba1-4fda-b6a8-52197b035807 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.211534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4c292cf-eba1-4fda-b6a8-52197b035807 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.212211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4c292cf-eba1-4fda-b6a8-52197b035807 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:56.422663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:06:56.423295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:06:56.423830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:06:56.424370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:06:56.425000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:06:56.437217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.438115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.439170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:56.440434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.441149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.441815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:56.448295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c4c292cf-eba1-4fda-b6a8-52197b035807 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:56.459108 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c4c292cf-eba1-4fda-b6a8-52197b035807 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.287525 Sep 27 10:06:56.460151 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1396/2567] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:56 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:56.470635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Getting quotas for project 027612746ecc44f4ab8e7af121ec27a1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:06:56.475614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3adf181b-7db8-4954-824a-e4d98a90c682 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:56.476248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3adf181b-7db8-4954-824a-e4d98a90c682 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:56.479715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Getting quotas for user 92b0186cd92744dabd001b86560b5c90 and project 027612746ecc44f4ab8e7af121ec27a1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:06:56.484380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3adf181b-7db8-4954-824a-e4d98a90c682 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.485013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3adf181b-7db8-4954-824a-e4d98a90c682 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.485743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3adf181b-7db8-4954-824a-e4d98a90c682 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:56.486935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3adf181b-7db8-4954-824a-e4d98a90c682 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.487523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3adf181b-7db8-4954-824a-e4d98a90c682 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.488049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3adf181b-7db8-4954-824a-e4d98a90c682 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:56.521070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:06:56.526234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:06:56.616433 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-de4f7d55-3927-479d-b546-3b199fc78a93 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.959831 Sep 27 10:06:56.619139 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1172/2568] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:06:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 969 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:06:56.629923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-958df003-8195-4c99-be9a-99fd9c9301de tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Action: 'update', calling method: .version_select at 0x7f7f62acf5e0>, body: {"quota_set": {"ram": "5120"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:56.711355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3adf181b-7db8-4954-824a-e4d98a90c682 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:56.739391 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3adf181b-7db8-4954-824a-e4d98a90c682 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1519 microversion: 2.1 time: 0.265254 Sep 27 10:06:56.750609 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1397/2569] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:56 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1519 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:56.777206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dea2550e-9b6c-492c-b821-50dc07676847 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:56.777206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dea2550e-9b6c-492c-b821-50dc07676847 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:56.795514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dea2550e-9b6c-492c-b821-50dc07676847 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.797150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dea2550e-9b6c-492c-b821-50dc07676847 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.797583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dea2550e-9b6c-492c-b821-50dc07676847 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:56.798337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dea2550e-9b6c-492c-b821-50dc07676847 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.798794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dea2550e-9b6c-492c-b821-50dc07676847 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.799196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dea2550e-9b6c-492c-b821-50dc07676847 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:56.838332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-958df003-8195-4c99-be9a-99fd9c9301de tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.840651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-958df003-8195-4c99-be9a-99fd9c9301de tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.841179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-958df003-8195-4c99-be9a-99fd9c9301de tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:56.841946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-958df003-8195-4c99-be9a-99fd9c9301de tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.842420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-958df003-8195-4c99-be9a-99fd9c9301de tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.843111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-958df003-8195-4c99-be9a-99fd9c9301de tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:56.889678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dea2550e-9b6c-492c-b821-50dc07676847 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:56.899611 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dea2550e-9b6c-492c-b821-50dc07676847 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0" status: 200 len: 1219 microversion: 2.1 time: 0.147917 Sep 27 10:06:56.900240 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1398/2570] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:56 2022] GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 => generated 1219 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:56.908814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0517a2df-e1f1-409a-8deb-0aed8b6ffd04 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62916ee0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:56.927128 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-958df003-8195-4c99-be9a-99fd9c9301de tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/56ad5c69df974e3f9230cccf025c7f66" status: 200 len: 328 microversion: 2.1 time: 0.305937 Sep 27 10:06:56.928098 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1173/2571] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 10:06:56 2022] PUT /compute/v2.1/os-quota-sets/56ad5c69df974e3f9230cccf025c7f66 => generated 328 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:56.933297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:56.935651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:56.943471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.944122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.944572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:56.945435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:56.945897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:56.946334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:57.006985 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0517a2df-e1f1-409a-8deb-0aed8b6ffd04 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1453 microversion: 2.1 time: 0.105610 Sep 27 10:06:57.009002 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1399/2572] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:56 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1453 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:57.016843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-21a1435d-c7c8-4289-9f7e-d39152cdb974 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-369040765"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:57.017393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-21a1435d-c7c8-4289-9f7e-d39152cdb974 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:57.023965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a1435d-c7c8-4289-9f7e-d39152cdb974 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:57.024790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a1435d-c7c8-4289-9f7e-d39152cdb974 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:57.025424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a1435d-c7c8-4289-9f7e-d39152cdb974 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:57.026353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a1435d-c7c8-4289-9f7e-d39152cdb974 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:57.027004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a1435d-c7c8-4289-9f7e-d39152cdb974 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:57.027699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a1435d-c7c8-4289-9f7e-d39152cdb974 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:57.041523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:57.051654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:57.053517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:57.059378 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-53f91269-04e9-4eb2-907f-3d53e516e7ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1535 microversion: 2.91 time: 0.130136 Sep 27 10:06:57.060601 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1174/2573] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:56 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1535 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:57.066341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-49e60bfa-d3fe-4666-a019-54b46731d867 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:57.067380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-49e60bfa-d3fe-4666-a019-54b46731d867 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:57.153024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-49e60bfa-d3fe-4666-a019-54b46731d867 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:57.161059 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-49e60bfa-d3fe-4666-a019-54b46731d867 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1216 microversion: 2.1 time: 0.099117 Sep 27 10:06:57.164510 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1175/2574] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:57 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1216 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:57.174145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-03608bdb-19ea-4a85-a74f-b02e9c1e96b8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:57.174999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-03608bdb-19ea-4a85-a74f-b02e9c1e96b8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:57.183996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03608bdb-19ea-4a85-a74f-b02e9c1e96b8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:57.184901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03608bdb-19ea-4a85-a74f-b02e9c1e96b8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:57.185779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03608bdb-19ea-4a85-a74f-b02e9c1e96b8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:57.186827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03608bdb-19ea-4a85-a74f-b02e9c1e96b8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:57.188053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03608bdb-19ea-4a85-a74f-b02e9c1e96b8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:57.189345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03608bdb-19ea-4a85-a74f-b02e9c1e96b8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:57.246694 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.network.security_group_api [None req-21a1435d-c7c8-4289-9f7e-d39152cdb974 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Adding security group 6244f0a6-16ad-4669-ba27-a50476f4d398 to port 4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801 Sep 27 10:06:57.397242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-03608bdb-19ea-4a85-a74f-b02e9c1e96b8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:57.409983 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-03608bdb-19ea-4a85-a74f-b02e9c1e96b8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1519 microversion: 2.1 time: 0.243423 Sep 27 10:06:57.411408 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1176/2575] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:57 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1519 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:57.418346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:57.419316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:57.434785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:57.436274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:57.438989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:57.440211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:57.441147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:57.442419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:57.482254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:06:57.509943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:57.510715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:57.511303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:57.552002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lazy-loading 'flavor' on Instance uuid 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:06:57.562410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:57.584309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lazy-loading 'system_metadata' on Instance uuid 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:06:57.733326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lazy-loading 'metadata' on Instance uuid 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:06:57.932495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:57.934372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:57.935156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:57.958839 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-21a1435d-c7c8-4289-9f7e-d39152cdb974 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3/action" status: 202 len: 0 microversion: 2.1 time: 0.948663 Sep 27 10:06:57.959774 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1400/2576] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:06:57 2022] POST /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3/action => generated 0 bytes in 950 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:06:57.965286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6ff63ce-38a4-4234-875c-d2ee9f51d24d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Calling method '.version_select at 0x7f7f6296aee0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:58.107723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ff63ce-38a4-4234-875c-d2ee9f51d24d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.108242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ff63ce-38a4-4234-875c-d2ee9f51d24d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.108923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ff63ce-38a4-4234-875c-d2ee9f51d24d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.110008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ff63ce-38a4-4234-875c-d2ee9f51d24d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.110647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ff63ce-38a4-4234-875c-d2ee9f51d24d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.111283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ff63ce-38a4-4234-875c-d2ee9f51d24d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.146211 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c0524d49-c15a-43cd-b35d-609db0735b9d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0" status: 204 len: 0 microversion: 2.1 time: 0.732698 Sep 27 10:06:58.149082 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1177/2577] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:06:57 2022] DELETE /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:06:58.156774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4b1e8a11-e1d7-467e-b4f6-08c2169021a6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-448215871"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:58.159965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4b1e8a11-e1d7-467e-b4f6-08c2169021a6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:58.163201 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c6ff63ce-38a4-4234-875c-d2ee9f51d24d tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/56ad5c69df974e3f9230cccf025c7f66/detail" status: 200 len: 916 microversion: 2.1 time: 0.201809 Sep 27 10:06:58.165503 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1401/2578] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:06:57 2022] GET /compute/v2.1/os-quota-sets/56ad5c69df974e3f9230cccf025c7f66/detail => generated 916 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:58.170679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0d1926f4-0a3f-42cf-8b4c-4a07950c9623 req-c020ca0c-f4cb-458e-9166-7546eacbcc2d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b5bcf59-770f-40f1-83b0-0738578e921c", "name": "network-vif-plugged", "status": "completed", "tag": "2acf9231-7703-4c8c-83e8-58ba465f8803"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:58.182165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0d1926f4-0a3f-42cf-8b4c-4a07950c9623 req-c020ca0c-f4cb-458e-9166-7546eacbcc2d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.185595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0d1926f4-0a3f-42cf-8b4c-4a07950c9623 req-c020ca0c-f4cb-458e-9166-7546eacbcc2d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.186416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0d1926f4-0a3f-42cf-8b4c-4a07950c9623 req-c020ca0c-f4cb-458e-9166-7546eacbcc2d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.205331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b1e8a11-e1d7-467e-b4f6-08c2169021a6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.205814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b1e8a11-e1d7-467e-b4f6-08c2169021a6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.206252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b1e8a11-e1d7-467e-b4f6-08c2169021a6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.211015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b1e8a11-e1d7-467e-b4f6-08c2169021a6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.211691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b1e8a11-e1d7-467e-b4f6-08c2169021a6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.212732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b1e8a11-e1d7-467e-b4f6-08c2169021a6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.219170 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0d1926f4-0a3f-42cf-8b4c-4a07950c9623 req-c020ca0c-f4cb-458e-9166-7546eacbcc2d service nova] Creating event network-vif-plugged:2acf9231-7703-4c8c-83e8-58ba465f8803 for instance 5b5bcf59-770f-40f1-83b0-0738578e921c on nch2-master-1 Sep 27 10:06:58.233750 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0d1926f4-0a3f-42cf-8b4c-4a07950c9623 req-c020ca0c-f4cb-458e-9166-7546eacbcc2d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064738 Sep 27 10:06:58.235386 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1402/2579] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:58.271528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fad73ba7-fb04-4124-ab06-4c422b1ce32e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:58.273794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fad73ba7-fb04-4124-ab06-4c422b1ce32e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:58.292233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fad73ba7-fb04-4124-ab06-4c422b1ce32e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.292794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fad73ba7-fb04-4124-ab06-4c422b1ce32e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.293238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fad73ba7-fb04-4124-ab06-4c422b1ce32e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.296839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fad73ba7-fb04-4124-ab06-4c422b1ce32e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.297323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fad73ba7-fb04-4124-ab06-4c422b1ce32e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.297759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fad73ba7-fb04-4124-ab06-4c422b1ce32e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.590915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fad73ba7-fb04-4124-ab06-4c422b1ce32e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:58.604402 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fad73ba7-fb04-4124-ab06-4c422b1ce32e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.367714 Sep 27 10:06:58.617707 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1403/2580] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:58 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:58.628822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9c4a9118-6e0e-4b18-b00a-15e96581f20f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f628e25e0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:58.725815 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4b1e8a11-e1d7-467e-b4f6-08c2169021a6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action" status: 202 len: 0 microversion: 2.1 time: 0.574870 Sep 27 10:06:58.726665 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1178/2581] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:06:58 2022] POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action => generated 0 bytes in 577 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 10:06:58.845675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-77a06f6d-fcbf-4a93-9668-44930d997ab0 req-4e84dfe7-c500-454b-8111-3506481c7d02 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ac5c4cdc-90a8-48c7-84f3-0053b32834f3", "tag": "4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:58.856617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-77a06f6d-fcbf-4a93-9668-44930d997ab0 req-4e84dfe7-c500-454b-8111-3506481c7d02 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.857158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-77a06f6d-fcbf-4a93-9668-44930d997ab0 req-4e84dfe7-c500-454b-8111-3506481c7d02 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.857592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-77a06f6d-fcbf-4a93-9668-44930d997ab0 req-4e84dfe7-c500-454b-8111-3506481c7d02 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.859354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-9c4a9118-6e0e-4b18-b00a-15e96581f20f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupInUse", "message": "Security Group 6244f0a6-16ad-4669-ba27-a50476f4d398 in use.", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:06:58.860161 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-9c4a9118-6e0e-4b18-b00a-15e96581f20f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security Group 6244f0a6-16ad-4669-ba27-a50476f4d398 in use. Sep 27 10:06:58.860161 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-bce0b2f9-bb68-4733-bf13-e97450176fc9'] Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9c4a9118-6e0e-4b18-b00a-15e96581f20f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-security-groups/6244f0a6-16ad-4669-ba27-a50476f4d398 HTTP/1.1 Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: exception: Security Group 6244f0a6-16ad-4669-ba27-a50476f4d398 in use. Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-bce0b2f9-bb68-4733-bf13-e97450176fc9'] Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '6244f0a6-16ad-4669-ba27-a50476f4d398'} Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 359, in destroy Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutron.delete_security_group(security_group['id']) Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1073, in delete_security_group Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.delete(self.security_group_path % (security_group)) Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 352, in delete Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("DELETE", action, body=body, Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:06:58.862631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Security Group 6244f0a6-16ad-4669-ba27-a50476f4d398 in use. Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-bce0b2f9-bb68-4733-bf13-e97450176fc9'] Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 179, in delete Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_group_api.destroy(context, security_group) Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 364, in destroy Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.Invalid(str(e)) Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Security Group 6244f0a6-16ad-4669-ba27-a50476f4d398 in use. Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-bce0b2f9-bb68-4733-bf13-e97450176fc9'] Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 183, in delete Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Security Group 6244f0a6-16ad-4669-ba27-a50476f4d398 in use. Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-bce0b2f9-bb68-4733-bf13-e97450176fc9'] Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security Group 6244f0a6-16ad-4669-ba27-a50476f4d398 in use. Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-bce0b2f9-bb68-4733-bf13-e97450176fc9'] Sep 27 10:06:58.869358 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:06:58.871964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9c4a9118-6e0e-4b18-b00a-15e96581f20f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 400 to user: Security Group 6244f0a6-16ad-4669-ba27-a50476f4d398 in use. Sep 27 10:06:58.871964 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-bce0b2f9-bb68-4733-bf13-e97450176fc9'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:06:58.871964 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9c4a9118-6e0e-4b18-b00a-15e96581f20f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/6244f0a6-16ad-4669-ba27-a50476f4d398" status: 400 len: 185 microversion: 2.1 time: 0.248302 Sep 27 10:06:58.872638 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1404/2582] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:06:58 2022] DELETE /compute/v2.1/os-security-groups/6244f0a6-16ad-4669-ba27-a50476f4d398 => generated 185 bytes in 254 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:06:58.877639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8d97c1f5-87f2-4287-b183-2b74b979b623 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62920d30>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:58.878471 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-77a06f6d-fcbf-4a93-9668-44930d997ab0 req-4e84dfe7-c500-454b-8111-3506481c7d02 service nova] Creating event network-changed:4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801 for instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 on nch1-master-1 Sep 27 10:06:58.883977 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-77a06f6d-fcbf-4a93-9668-44930d997ab0 req-4e84dfe7-c500-454b-8111-3506481c7d02 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.156099 Sep 27 10:06:58.885080 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1179/2583] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:06:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:58.891124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:58.892177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:58.895513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:DELETE=2,nova_api:SELECT=90,nova_api:INSERT=7,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:06:58.901130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.901781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.902311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.903130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.903699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.904205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.931070 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8d97c1f5-87f2-4287-b183-2b74b979b623 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1462 microversion: 2.1 time: 0.056949 Sep 27 10:06:58.932107 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1405/2584] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:58 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1462 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:58.938200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad3eda59-0058-4203-9695-b0e6cd1bc4f9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:58.939237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ad3eda59-0058-4203-9695-b0e6cd1bc4f9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:58.948872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad3eda59-0058-4203-9695-b0e6cd1bc4f9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.950045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad3eda59-0058-4203-9695-b0e6cd1bc4f9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.950787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad3eda59-0058-4203-9695-b0e6cd1bc4f9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.951594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad3eda59-0058-4203-9695-b0e6cd1bc4f9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:58.952145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad3eda59-0058-4203-9695-b0e6cd1bc4f9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:58.952647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad3eda59-0058-4203-9695-b0e6cd1bc4f9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:58.999292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:59.009062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:59.010541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:59.017252 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fe3bfa30-7537-46bc-b157-eebfba31ac9e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1538 microversion: 2.91 time: 0.130926 Sep 27 10:06:59.018061 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1180/2585] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:06:58 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1538 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:06:59.022745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-808ad8bd-cbac-406c-beb0-fec147212b79 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:59.026932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ad3eda59-0058-4203-9695-b0e6cd1bc4f9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:59.029718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-808ad8bd-cbac-406c-beb0-fec147212b79 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:59.036384 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ad3eda59-0058-4203-9695-b0e6cd1bc4f9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0" status: 200 len: 1273 microversion: 2.1 time: 0.102109 Sep 27 10:06:59.037302 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1406/2586] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:58 2022] GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 => generated 1273 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:59.042768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808ad8bd-cbac-406c-beb0-fec147212b79 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.043189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Action: 'update', calling method: .version_select at 0x7f7f6298f0d0>, body: {"quota_set": {"ram": "2048"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:59.043312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808ad8bd-cbac-406c-beb0-fec147212b79 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.043703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808ad8bd-cbac-406c-beb0-fec147212b79 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.044444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808ad8bd-cbac-406c-beb0-fec147212b79 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.045129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808ad8bd-cbac-406c-beb0-fec147212b79 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.045718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808ad8bd-cbac-406c-beb0-fec147212b79 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.115008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-808ad8bd-cbac-406c-beb0-fec147212b79 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:06:59.124332 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-808ad8bd-cbac-406c-beb0-fec147212b79 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1268 microversion: 2.1 time: 0.105369 Sep 27 10:06:59.136688 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1181/2587] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:59 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1268 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:59.149517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-708596859"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:59.149856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.152024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:59.157998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.158515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.159293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.159756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.160341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.161218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.161985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.163555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.163689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.164338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.164976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.217241 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] HTTP exception thrown: Cannot 'createImage' instance 0abee65c-58ee-4b9c-ac6c-65631519b216 while it is in task_state image_snapshot Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action HTTP/1.1 Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 84 Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-708596859"}} Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: exception: Cannot 'createImage' instance 0abee65c-58ee-4b9c-ac6c-65631519b216 while it is in task_state image_snapshot Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '0abee65c-58ee-4b9c-ac6c-65631519b216', 'body': {'createImage': {'name': 'tempest-ImagesOneServerNegativeTestJSON-image-708596859'}}} Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1349, in _action_create_image Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi image = self.compute_api.snapshot(context, Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 155, in inner Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 in task_state image_snapshot. Cannot snapshot while the instance is in this state. Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1354, in _action_create_image Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:06:59.220036 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'createImage' instance 0abee65c-58ee-4b9c-ac6c-65631519b216 while it is in task_state image_snapshot Sep 27 10:06:59.224360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:59.224360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:06:59.224360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:59.224360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:06:59.224360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:06:59.224360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:06:59.224360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:06:59.224360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:06:59.224360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'createImage' instance 0abee65c-58ee-4b9c-ac6c-65631519b216 while it is in task_state image_snapshot Sep 27 10:06:59.224360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:06:59.228101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 409 to user: Cannot 'createImage' instance 0abee65c-58ee-4b9c-ac6c-65631519b216 while it is in task_state image_snapshot {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:06:59.231793 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1b63f5b9-072a-4440-b175-3ca89327e962 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action" status: 409 len: 159 microversion: 2.1 time: 0.098846 Sep 27 10:06:59.232724 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1182/2588] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:06:59 2022] POST /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216/action => generated 159 bytes in 103 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:06:59.248192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9426aecb-5391-48ca-8734-d8a4d195c04d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:59.254677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9426aecb-5391-48ca-8734-d8a4d195c04d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:59.262086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.262852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.264167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.265227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.265995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.266587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.272617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9426aecb-5391-48ca-8734-d8a4d195c04d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.274450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9426aecb-5391-48ca-8734-d8a4d195c04d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.276274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9426aecb-5391-48ca-8734-d8a4d195c04d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.278772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9426aecb-5391-48ca-8734-d8a4d195c04d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.280549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9426aecb-5391-48ca-8734-d8a4d195c04d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.281715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9426aecb-5391-48ca-8734-d8a4d195c04d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.404459 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4dda6c3-f6b4-4db4-91b9-544278866bcc tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/56ad5c69df974e3f9230cccf025c7f66?user_id=8d2ea4e9f97e4bf3bc76ecc51cba3182" status: 200 len: 328 microversion: 2.1 time: 0.365868 Sep 27 10:06:59.408251 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1407/2589] 10.222.0.26 () {60 vars in 1343 bytes} [Tue Sep 27 10:06:59 2022] PUT /compute/v2.1/os-quota-sets/56ad5c69df974e3f9230cccf025c7f66?user_id=8d2ea4e9f97e4bf3bc76ecc51cba3182 => generated 328 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:59.421179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d49f3f6-2079-482b-892a-9403b7cee8fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f628ca0d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:59.487290 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9426aecb-5391-48ca-8734-d8a4d195c04d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3/action" status: 202 len: 0 microversion: 2.1 time: 0.252451 Sep 27 10:06:59.488779 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1183/2590] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:06:59 2022] POST /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3/action => generated 0 bytes in 255 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:06:59.502843 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d49f3f6-2079-482b-892a-9403b7cee8fb tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 200 len: 1462 microversion: 2.1 time: 0.093194 Sep 27 10:06:59.505923 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1408/2591] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:06:59 2022] GET /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 1462 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:06:59.515473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c348ab81-e1f7-4ee3-8959-3f203afe5df3 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '.version_select at 0x7f7f62920a60>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:59.518942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f6e93854-ce67-4b5f-9546-bd4c63e0b94c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Action: 'action', calling method: >, body: {"unshelve": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:06:59.529441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f6e93854-ce67-4b5f-9546-bd4c63e0b94c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:59.542466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e93854-ce67-4b5f-9546-bd4c63e0b94c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.542969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e93854-ce67-4b5f-9546-bd4c63e0b94c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.543419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e93854-ce67-4b5f-9546-bd4c63e0b94c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.544131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e93854-ce67-4b5f-9546-bd4c63e0b94c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.544591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e93854-ce67-4b5f-9546-bd4c63e0b94c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.549285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6e93854-ce67-4b5f-9546-bd4c63e0b94c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.718121 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c348ab81-e1f7-4ee3-8959-3f203afe5df3 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "DELETE /compute/v2.1/images/82527f1b-6d7a-4517-bf5e-959689e1b5d0" status: 204 len: 0 microversion: 2.1 time: 0.209203 Sep 27 10:06:59.718773 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1409/2592] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:59 2022] DELETE /compute/v2.1/images/82527f1b-6d7a-4517-bf5e-959689e1b5d0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:06:59.723553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d5dbed98-cee7-4645-b716-b5880ec7ddf5 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Calling method '.version_select at 0x7f7f6298f310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:59.767517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f6e93854-ce67-4b5f-9546-bd4c63e0b94c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "nch1-master-1". {{(pid=145969) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 27 10:06:59.816595 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d5dbed98-cee7-4645-b716-b5880ec7ddf5 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/56ad5c69df974e3f9230cccf025c7f66?user_id=8d2ea4e9f97e4bf3bc76ecc51cba3182" status: 200 len: 370 microversion: 2.1 time: 0.096517 Sep 27 10:06:59.817648 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1410/2593] 10.222.0.26 () {58 vars in 1323 bytes} [Tue Sep 27 10:06:59 2022] GET /compute/v2.1/os-quota-sets/56ad5c69df974e3f9230cccf025c7f66?user_id=8d2ea4e9f97e4bf3bc76ecc51cba3182 => generated 370 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:06:59.822254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-934eaeed-bbc5-42e9-b275-f0af8499e84d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:59.822888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-934eaeed-bbc5-42e9-b275-f0af8499e84d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:59.833378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-934eaeed-bbc5-42e9-b275-f0af8499e84d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.833909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-934eaeed-bbc5-42e9-b275-f0af8499e84d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.834351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-934eaeed-bbc5-42e9-b275-f0af8499e84d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.835148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-934eaeed-bbc5-42e9-b275-f0af8499e84d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.835614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-934eaeed-bbc5-42e9-b275-f0af8499e84d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.836265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-934eaeed-bbc5-42e9-b275-f0af8499e84d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.859627 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f6e93854-ce67-4b5f-9546-bd4c63e0b94c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618/action" status: 202 len: 0 microversion: 2.91 time: 0.369167 Sep 27 10:06:59.863664 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1184/2594] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:06:59 2022] POST /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618/action => generated 0 bytes in 371 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:06:59.868277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2f6c961a-4429-4e61-8db3-1b247034234b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:06:59.868965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2f6c961a-4429-4e61-8db3-1b247034234b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:06:59.875779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6c961a-4429-4e61-8db3-1b247034234b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.876256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6c961a-4429-4e61-8db3-1b247034234b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.876813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6c961a-4429-4e61-8db3-1b247034234b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:06:59.877521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6c961a-4429-4e61-8db3-1b247034234b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:06:59.877978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6c961a-4429-4e61-8db3-1b247034234b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:06:59.878402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6c961a-4429-4e61-8db3-1b247034234b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.137511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2f6c961a-4429-4e61-8db3-1b247034234b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:00.153940 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2f6c961a-4429-4e61-8db3-1b247034234b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d" status: 200 len: 1467 microversion: 2.1 time: 0.292176 Sep 27 10:07:00.154546 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1185/2595] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:59 2022] GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d => generated 1467 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:00.159301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-97210b5a-dc6a-4de6-8368-3396c7c36e56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:00.159923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-97210b5a-dc6a-4de6-8368-3396c7c36e56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:00.198203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97210b5a-dc6a-4de6-8368-3396c7c36e56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:00.198874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97210b5a-dc6a-4de6-8368-3396c7c36e56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:00.199491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97210b5a-dc6a-4de6-8368-3396c7c36e56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.200715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97210b5a-dc6a-4de6-8368-3396c7c36e56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:00.201327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97210b5a-dc6a-4de6-8368-3396c7c36e56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:00.201942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97210b5a-dc6a-4de6-8368-3396c7c36e56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.289804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-934eaeed-bbc5-42e9-b275-f0af8499e84d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:00.299614 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-934eaeed-bbc5-42e9-b275-f0af8499e84d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1574 microversion: 2.1 time: 0.481067 Sep 27 10:07:00.300750 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1411/2596] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:06:59 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1574 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:00.317133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e78c168-860b-4b84-ac3e-b53b663421e1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:00.318027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7e78c168-860b-4b84-ac3e-b53b663421e1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:00.328012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e78c168-860b-4b84-ac3e-b53b663421e1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:00.328815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e78c168-860b-4b84-ac3e-b53b663421e1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:00.330377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e78c168-860b-4b84-ac3e-b53b663421e1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.333313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e78c168-860b-4b84-ac3e-b53b663421e1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:00.334311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e78c168-860b-4b84-ac3e-b53b663421e1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:00.338665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e78c168-860b-4b84-ac3e-b53b663421e1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.499514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-97210b5a-dc6a-4de6-8368-3396c7c36e56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:00.507434 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-97210b5a-dc6a-4de6-8368-3396c7c36e56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.351847 Sep 27 10:07:00.508379 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1186/2597] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:00 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:00.514725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:00.515488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:00.523232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:00.523948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:00.524806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.525539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:00.526002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:00.526432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.531494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=5,nova_cell1:INSERT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:00.619716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:00.628411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:00.629909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:00.635666 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-122a9cc9-bab5-4b29-9ec2-f150d2c52714 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1486 microversion: 2.91 time: 0.125114 Sep 27 10:07:00.636512 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1187/2598] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:00 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1486 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:00.641271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e9458833-0f2a-45eb-9a70-e2dfde7a0e86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:00.641887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e9458833-0f2a-45eb-9a70-e2dfde7a0e86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:00.648533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9458833-0f2a-45eb-9a70-e2dfde7a0e86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:00.649463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9458833-0f2a-45eb-9a70-e2dfde7a0e86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:00.650108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9458833-0f2a-45eb-9a70-e2dfde7a0e86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.651027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9458833-0f2a-45eb-9a70-e2dfde7a0e86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:00.651672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9458833-0f2a-45eb-9a70-e2dfde7a0e86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:00.652269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9458833-0f2a-45eb-9a70-e2dfde7a0e86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.721110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7e78c168-860b-4b84-ac3e-b53b663421e1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:00.736034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e9458833-0f2a-45eb-9a70-e2dfde7a0e86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:00.743613 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7e78c168-860b-4b84-ac3e-b53b663421e1 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1531 microversion: 2.1 time: 0.441252 Sep 27 10:07:00.767865 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1412/2599] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:00 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1531 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:00.772082 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e9458833-0f2a-45eb-9a70-e2dfde7a0e86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0" status: 200 len: 1217 microversion: 2.1 time: 0.126876 Sep 27 10:07:00.782863 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1188/2600] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:00 2022] GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 => generated 1217 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:00.789192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0de1966e-107d-4283-897b-a6b7d14a1cfa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:00.792214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0de1966e-107d-4283-897b-a6b7d14a1cfa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:00.797456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1b56e580-7af3-4e56-8020-d7df90c420c7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62ade310>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:00.809605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0de1966e-107d-4283-897b-a6b7d14a1cfa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:00.810296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0de1966e-107d-4283-897b-a6b7d14a1cfa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:00.810864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0de1966e-107d-4283-897b-a6b7d14a1cfa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.812998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0de1966e-107d-4283-897b-a6b7d14a1cfa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:00.815083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0de1966e-107d-4283-897b-a6b7d14a1cfa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:00.816365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0de1966e-107d-4283-897b-a6b7d14a1cfa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.949405 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1b56e580-7af3-4e56-8020-d7df90c420c7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images?changes-since=2022-09-27T10%3A04%3A17Z" status: 200 len: 1902 microversion: 2.1 time: 0.163269 Sep 27 10:07:00.950497 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1189/2601] 10.222.0.26 () {58 vars in 1199 bytes} [Tue Sep 27 10:07:00 2022] GET /compute/v2.1/images?changes-since=2022-09-27T10%3A04%3A17Z => generated 1902 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:00.956069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a7a174c6-00a1-4bdc-bf97-0f488af81c2a req-4894454b-7585-4f66-b0a0-ca6fa7304c1a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b5bcf59-770f-40f1-83b0-0738578e921c", "name": "network-vif-plugged", "status": "completed", "tag": "cd5a9e3b-1cef-4cbd-a955-7fc284d5c0d4"}, {"server_uuid": "5b5bcf59-770f-40f1-83b0-0738578e921c", "name": "network-vif-plugged", "status": "completed", "tag": "f3e142bd-656b-42c1-b4a6-376501c219f9"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:00.970484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a7a174c6-00a1-4bdc-bf97-0f488af81c2a req-4894454b-7585-4f66-b0a0-ca6fa7304c1a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:00.970975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a7a174c6-00a1-4bdc-bf97-0f488af81c2a req-4894454b-7585-4f66-b0a0-ca6fa7304c1a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:00.971586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a7a174c6-00a1-4bdc-bf97-0f488af81c2a req-4894454b-7585-4f66-b0a0-ca6fa7304c1a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:00.984128 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a7a174c6-00a1-4bdc-bf97-0f488af81c2a req-4894454b-7585-4f66-b0a0-ca6fa7304c1a service nova] Creating event network-vif-plugged:cd5a9e3b-1cef-4cbd-a955-7fc284d5c0d4 for instance 5b5bcf59-770f-40f1-83b0-0738578e921c on nch2-master-1 Sep 27 10:07:00.984903 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a7a174c6-00a1-4bdc-bf97-0f488af81c2a req-4894454b-7585-4f66-b0a0-ca6fa7304c1a service nova] Creating event network-vif-plugged:f3e142bd-656b-42c1-b4a6-376501c219f9 for instance 5b5bcf59-770f-40f1-83b0-0738578e921c on nch2-master-1 Sep 27 10:07:00.989490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0de1966e-107d-4283-897b-a6b7d14a1cfa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:00.998244 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0de1966e-107d-4283-897b-a6b7d14a1cfa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1274 microversion: 2.1 time: 0.229074 Sep 27 10:07:01.002853 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a7a174c6-00a1-4bdc-bf97-0f488af81c2a req-4894454b-7585-4f66-b0a0-ca6fa7304c1a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.051177 Sep 27 10:07:01.003852 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1190/2603] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:00 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:01.003970 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1413/2603] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:00 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1274 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:01.010101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-92f516d5-4ad3-47b2-8c0c-5cb6b24de413 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Calling method '.version_select at 0x7f7f62adee50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:01.020519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f015bb0-7717-4e8d-8cb7-9ede71254749 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f629164c0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:01.148917 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f015bb0-7717-4e8d-8cb7-9ede71254749 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images?name=tempest-ListImageFiltersTestJSON-image-1258822588" status: 200 len: 485 microversion: 2.1 time: 0.143703 Sep 27 10:07:01.158001 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1414/2604] 10.222.0.26 () {58 vars in 1231 bytes} [Tue Sep 27 10:07:01 2022] GET /compute/v2.1/images?name=tempest-ListImageFiltersTestJSON-image-1258822588 => generated 485 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:01.163687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a256fb45-959e-46ff-9371-127f71ba2c89 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f628ca430>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:01.279913 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a256fb45-959e-46ff-9371-127f71ba2c89 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images?server=961407e4-8e66-4f1c-88bd-6d2b4795c11d" status: 200 len: 957 microversion: 2.1 time: 0.120397 Sep 27 10:07:01.286225 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1415/2605] 10.222.0.26 () {58 vars in 1209 bytes} [Tue Sep 27 10:07:01 2022] GET /compute/v2.1/images?server=961407e4-8e66-4f1c-88bd-6d2b4795c11d => generated 957 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:01.298510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0d8059b-f316-4e2d-8d36-bc92f9288efc tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f629363a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:01.363391 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-92f516d5-4ad3-47b2-8c0c-5cb6b24de413 tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/d59d3d57f7ee4b08a2fb841333c7212e/defaults" status: 200 len: 371 microversion: 2.1 time: 0.357789 Sep 27 10:07:01.365175 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1191/2606] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:07:01 2022] GET /compute/v2.1/os-quota-sets/d59d3d57f7ee4b08a2fb841333c7212e/defaults => generated 371 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:01.367781 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b0d8059b-f316-4e2d-8d36-bc92f9288efc tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.26%2Fcompute%2Fv2.1%2Fservers%2F55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 485 microversion: 2.1 time: 0.074285 Sep 27 10:07:01.368601 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1416/2607] 10.222.0.26 () {58 vars in 1317 bytes} [Tue Sep 27 10:07:01 2022] GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.26%2Fcompute%2Fv2.1%2Fservers%2F55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 485 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:01.371037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-744bb869-c87f-4de7-9ee8-9a3726c44138 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:01.373391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-744bb869-c87f-4de7-9ee8-9a3726c44138 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:01.379537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-806dd541-8116-4c74-a14d-c34c8f2f137e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Action: 'update', calling method: .version_select at 0x7f7f62936670>, body: {"quota_set": {"force": true, "metadata_items": 256, "ram": 10240, "key_pairs": 200, "instances": 20, "server_groups": 20, "server_group_members": 20, "cores": 2, "fixed_ips": 10, "floating_ips": 20, "security_group_rules": 20, "security_groups": 20, "injected_file_content_bytes": 20480, "injected_file_path_bytes": 512, "injected_files": 10}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:01.387594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744bb869-c87f-4de7-9ee8-9a3726c44138 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:01.400393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744bb869-c87f-4de7-9ee8-9a3726c44138 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:01.400997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744bb869-c87f-4de7-9ee8-9a3726c44138 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:01.403274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744bb869-c87f-4de7-9ee8-9a3726c44138 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:01.403750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744bb869-c87f-4de7-9ee8-9a3726c44138 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:01.404184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744bb869-c87f-4de7-9ee8-9a3726c44138 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:01.473626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-806dd541-8116-4c74-a14d-c34c8f2f137e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:01.474340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-806dd541-8116-4c74-a14d-c34c8f2f137e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:01.475165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-806dd541-8116-4c74-a14d-c34c8f2f137e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:01.476159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-806dd541-8116-4c74-a14d-c34c8f2f137e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:01.477086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-806dd541-8116-4c74-a14d-c34c8f2f137e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:01.477768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-806dd541-8116-4c74-a14d-c34c8f2f137e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:01.658991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-744bb869-c87f-4de7-9ee8-9a3726c44138 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:01.668770 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-744bb869-c87f-4de7-9ee8-9a3726c44138 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1574 microversion: 2.1 time: 0.302185 Sep 27 10:07:01.669554 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1192/2608] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:01 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1574 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:01.674711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ef2519c4-b804-4142-a64d-84f002582eee tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62b08dc0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:01.743149 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ef2519c4-b804-4142-a64d-84f002582eee tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.26%2Fcompute%2Fservers%2F55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 485 microversion: 2.1 time: 0.071646 Sep 27 10:07:01.748866 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1193/2609] 10.222.0.26 () {58 vars in 1303 bytes} [Tue Sep 27 10:07:01 2022] GET /compute/v2.1/images?server=http%3A%2F%2F10.222.0.26%2Fcompute%2Fservers%2F55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 485 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:01.766928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-810998c6-24cf-4142-a692-3b4f1aee082f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:01.767654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-810998c6-24cf-4142-a692-3b4f1aee082f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:01.777582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-810998c6-24cf-4142-a692-3b4f1aee082f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:01.778369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-810998c6-24cf-4142-a692-3b4f1aee082f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:01.778820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-810998c6-24cf-4142-a692-3b4f1aee082f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:01.779598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-810998c6-24cf-4142-a692-3b4f1aee082f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:01.780054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-810998c6-24cf-4142-a692-3b4f1aee082f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:01.782467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-810998c6-24cf-4142-a692-3b4f1aee082f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:01.818124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=15 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:01.843823 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-806dd541-8116-4c74-a14d-c34c8f2f137e tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/d59d3d57f7ee4b08a2fb841333c7212e" status: 200 len: 329 microversion: 2.1 time: 0.469067 Sep 27 10:07:01.859104 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1417/2610] 10.222.0.26 () {60 vars in 1263 bytes} [Tue Sep 27 10:07:01 2022] PUT /compute/v2.1/os-quota-sets/d59d3d57f7ee4b08a2fb841333c7212e => generated 329 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:01.859104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:01.859827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:01.870124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:01.872482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:01.873406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:01.874245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:01.874889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:01.875882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:01.995218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:02.004507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:02.006574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:02.012648 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0ab573a3-44f9-4436-b335-d5d8b8c8ba80 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.157858 Sep 27 10:07:02.013268 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1418/2611] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:01 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:02.028964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba59f6ed-b5b9-46a3-8cdf-cd722e030756 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62936820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:02.093420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-810998c6-24cf-4142-a692-3b4f1aee082f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:02.103111 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-810998c6-24cf-4142-a692-3b4f1aee082f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1355 microversion: 2.1 time: 0.357590 Sep 27 10:07:02.104313 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1194/2612] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:01 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1355 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:02.109921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8192922f-1123-472d-a399-1fc686eecafe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:02.110467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8192922f-1123-472d-a399-1fc686eecafe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:02.117037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8192922f-1123-472d-a399-1fc686eecafe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:02.117525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8192922f-1123-472d-a399-1fc686eecafe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:02.118162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8192922f-1123-472d-a399-1fc686eecafe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:02.119272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8192922f-1123-472d-a399-1fc686eecafe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:02.119869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8192922f-1123-472d-a399-1fc686eecafe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:02.120448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8192922f-1123-472d-a399-1fc686eecafe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:02.188408 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba59f6ed-b5b9-46a3-8cdf-cd722e030756 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images?status=ACTIVE" status: 200 len: 3280 microversion: 2.1 time: 0.174025 Sep 27 10:07:02.189683 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1419/2613] 10.222.0.26 () {58 vars in 1149 bytes} [Tue Sep 27 10:07:02 2022] GET /compute/v2.1/images?status=ACTIVE => generated 3280 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:02.205325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba4098c7-288d-4d9f-9eba-e467536bd41a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:02.206504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba4098c7-288d-4d9f-9eba-e467536bd41a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:02.218307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba4098c7-288d-4d9f-9eba-e467536bd41a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:02.219112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba4098c7-288d-4d9f-9eba-e467536bd41a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:02.219752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba4098c7-288d-4d9f-9eba-e467536bd41a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:02.220954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba4098c7-288d-4d9f-9eba-e467536bd41a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:02.221574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba4098c7-288d-4d9f-9eba-e467536bd41a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:02.222385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba4098c7-288d-4d9f-9eba-e467536bd41a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:02.288681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8192922f-1123-472d-a399-1fc686eecafe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:02.300196 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8192922f-1123-472d-a399-1fc686eecafe tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1537 microversion: 2.1 time: 0.194042 Sep 27 10:07:02.301257 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1195/2614] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:02 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1537 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:02.317124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-598f7c84-5b8c-419c-bf2f-5ffd1edb08fd tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Action: 'update', calling method: .version_select at 0x7f7f629d80d0>, body: {"quota_set": {"cores": 20, "fixed_ips": -1, "floating_ips": -1, "injected_file_content_bytes": 10240, "injected_file_path_bytes": 255, "injected_files": 5, "instances": 10, "key_pairs": 100, "metadata_items": 128, "ram": 51200, "security_group_rules": -1, "security_groups": -1, "server_group_members": 10, "server_groups": 10}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:02.365913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba4098c7-288d-4d9f-9eba-e467536bd41a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:02.380777 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba4098c7-288d-4d9f-9eba-e467536bd41a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0" status: 200 len: 1219 microversion: 2.1 time: 0.179032 Sep 27 10:07:02.398902 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1420/2615] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:02 2022] GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 => generated 1219 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:02.404187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598f7c84-5b8c-419c-bf2f-5ffd1edb08fd tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:02.405794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598f7c84-5b8c-419c-bf2f-5ffd1edb08fd tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:02.409711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598f7c84-5b8c-419c-bf2f-5ffd1edb08fd tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:02.411374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598f7c84-5b8c-419c-bf2f-5ffd1edb08fd tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:02.413403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598f7c84-5b8c-419c-bf2f-5ffd1edb08fd tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:02.415137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6486b0c-4267-4dff-ad8d-326b68bcb883 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:02.415601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598f7c84-5b8c-419c-bf2f-5ffd1edb08fd tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:02.415973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c6486b0c-4267-4dff-ad8d-326b68bcb883 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:02.422408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6486b0c-4267-4dff-ad8d-326b68bcb883 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:02.422913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6486b0c-4267-4dff-ad8d-326b68bcb883 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:02.423363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6486b0c-4267-4dff-ad8d-326b68bcb883 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:02.424112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6486b0c-4267-4dff-ad8d-326b68bcb883 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:02.424554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6486b0c-4267-4dff-ad8d-326b68bcb883 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:02.425030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6486b0c-4267-4dff-ad8d-326b68bcb883 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:02.732446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c6486b0c-4267-4dff-ad8d-326b68bcb883 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:02.748239 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c6486b0c-4267-4dff-ad8d-326b68bcb883 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1316 microversion: 2.1 time: 0.347306 Sep 27 10:07:02.754507 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1421/2616] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:02 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1316 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:02.765697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da0ee124-ff7e-47b1-9e66-c9a4d4e22ed3 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f628e2040>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:02.898223 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da0ee124-ff7e-47b1-9e66-c9a4d4e22ed3 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images?type=snapshot" status: 200 len: 1430 microversion: 2.1 time: 0.135584 Sep 27 10:07:02.900252 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1422/2617] 10.222.0.26 () {58 vars in 1149 bytes} [Tue Sep 27 10:07:02 2022] GET /compute/v2.1/images?type=snapshot => generated 1430 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:02.914779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9fd48233-3a84-476c-bacb-2f013018c727 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:02.916083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9fd48233-3a84-476c-bacb-2f013018c727 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:02.927674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fd48233-3a84-476c-bacb-2f013018c727 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:02.929190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fd48233-3a84-476c-bacb-2f013018c727 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:02.930466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fd48233-3a84-476c-bacb-2f013018c727 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:02.932202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fd48233-3a84-476c-bacb-2f013018c727 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:02.933500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fd48233-3a84-476c-bacb-2f013018c727 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:02.934356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fd48233-3a84-476c-bacb-2f013018c727 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:02.966471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=71,nova_api:UPDATE=15,nova_api:INSERT=14 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:03.003277 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-598f7c84-5b8c-419c-bf2f-5ffd1edb08fd tempest-QuotasAdminTestJSON-190338447 tempest-QuotasAdminTestJSON-190338447-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/d59d3d57f7ee4b08a2fb841333c7212e" status: 200 len: 329 microversion: 2.1 time: 0.693943 Sep 27 10:07:03.004127 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1196/2618] 10.222.0.26 () {60 vars in 1263 bytes} [Tue Sep 27 10:07:02 2022] PUT /compute/v2.1/os-quota-sets/d59d3d57f7ee4b08a2fb841333c7212e => generated 329 bytes in 702 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:03.019226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0163e66e-4f6a-4081-875d-af87904c5d36 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62adeaf0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:03.145578 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0163e66e-4f6a-4081-875d-af87904c5d36 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images?limit=1" status: 200 len: 622 microversion: 2.1 time: 0.131330 Sep 27 10:07:03.146845 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1197/2619] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:07:03 2022] GET /compute/v2.1/images?limit=1 => generated 622 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:03.156043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:03.157301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:03.168528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:03.169897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:03.170803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:03.172031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:03.172970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:03.173755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:03.287322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:03.303815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:03.309106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:03.322485 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2f92c9b2-f733-478f-9c34-515fca61b432 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.173907 Sep 27 10:07:03.323895 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1198/2620] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:03 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:03.333524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c196f591-6352-4870-814e-dfcd95a410e7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:03.335782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c196f591-6352-4870-814e-dfcd95a410e7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:03.361638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c196f591-6352-4870-814e-dfcd95a410e7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:03.365092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c196f591-6352-4870-814e-dfcd95a410e7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:03.367898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c196f591-6352-4870-814e-dfcd95a410e7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:03.370357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c196f591-6352-4870-814e-dfcd95a410e7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:03.371794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c196f591-6352-4870-814e-dfcd95a410e7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:03.373535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c196f591-6352-4870-814e-dfcd95a410e7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:03.714928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9fd48233-3a84-476c-bacb-2f013018c727 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:03.726643 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9fd48233-3a84-476c-bacb-2f013018c727 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1572 microversion: 2.1 time: 0.824737 Sep 27 10:07:03.727706 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1423/2621] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:02 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1572 bytes in 827 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:03.746883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06e54088-81c8-4f15-8f79-d1f9584dfaef tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f628e25e0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:03.856551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c196f591-6352-4870-814e-dfcd95a410e7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:03.875544 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c196f591-6352-4870-814e-dfcd95a410e7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1803 microversion: 2.1 time: 0.542011 Sep 27 10:07:03.900501 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1199/2622] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:03 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1803 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:03.928186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-36bcd803-1e92-486b-9568-fbbd260d2287 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:03.928921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-36bcd803-1e92-486b-9568-fbbd260d2287 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:03.966389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36bcd803-1e92-486b-9568-fbbd260d2287 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:03.967691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36bcd803-1e92-486b-9568-fbbd260d2287 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:03.971146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36bcd803-1e92-486b-9568-fbbd260d2287 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:03.975454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36bcd803-1e92-486b-9568-fbbd260d2287 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:03.975981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36bcd803-1e92-486b-9568-fbbd260d2287 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:03.976506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36bcd803-1e92-486b-9568-fbbd260d2287 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:04.119136 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-06e54088-81c8-4f15-8f79-d1f9584dfaef tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/detail?changes-since=2022-09-27T10%3A04%3A11Z" status: 200 len: 6327 microversion: 2.1 time: 0.387455 Sep 27 10:07:04.120636 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1424/2623] 10.222.0.26 () {58 vars in 1220 bytes} [Tue Sep 27 10:07:03 2022] GET /compute/v2.1/images/detail?changes-since=2022-09-27T10%3A04%3A11Z => generated 6327 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:04.172239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69f9425c-e74d-4d1a-afae-e8ca3bfd9328 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:04.172937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69f9425c-e74d-4d1a-afae-e8ca3bfd9328 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:04.200871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69f9425c-e74d-4d1a-afae-e8ca3bfd9328 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:04.201360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69f9425c-e74d-4d1a-afae-e8ca3bfd9328 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:04.201888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69f9425c-e74d-4d1a-afae-e8ca3bfd9328 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:04.202619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69f9425c-e74d-4d1a-afae-e8ca3bfd9328 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:04.203088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69f9425c-e74d-4d1a-afae-e8ca3bfd9328 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:04.203628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69f9425c-e74d-4d1a-afae-e8ca3bfd9328 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:04.314282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:04.413324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69f9425c-e74d-4d1a-afae-e8ca3bfd9328 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:04.425118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-36bcd803-1e92-486b-9568-fbbd260d2287 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:04.426569 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69f9425c-e74d-4d1a-afae-e8ca3bfd9328 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0" status: 200 len: 1219 microversion: 2.1 time: 0.288996 Sep 27 10:07:04.435071 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-36bcd803-1e92-486b-9568-fbbd260d2287 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1532 microversion: 2.1 time: 0.532145 Sep 27 10:07:04.447117 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1425/2624] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:04 2022] GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 => generated 1219 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:04.453743 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1200/2625] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:03 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1532 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:04.460949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-563e98c5-d7e5-41d9-8004-b31d96cbb83a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:04.462245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ab94856a-0538-4855-a340-ef716d7297d4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-400049632"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:04.462738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-563e98c5-d7e5-41d9-8004-b31d96cbb83a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:04.464022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ab94856a-0538-4855-a340-ef716d7297d4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:04.470064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-563e98c5-d7e5-41d9-8004-b31d96cbb83a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:04.471483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-563e98c5-d7e5-41d9-8004-b31d96cbb83a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:04.472136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-563e98c5-d7e5-41d9-8004-b31d96cbb83a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:04.473089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-563e98c5-d7e5-41d9-8004-b31d96cbb83a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:04.473722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-563e98c5-d7e5-41d9-8004-b31d96cbb83a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:04.474346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-563e98c5-d7e5-41d9-8004-b31d96cbb83a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:04.479922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab94856a-0538-4855-a340-ef716d7297d4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:04.480138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab94856a-0538-4855-a340-ef716d7297d4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:04.480627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab94856a-0538-4855-a340-ef716d7297d4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:04.481512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab94856a-0538-4855-a340-ef716d7297d4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:04.481961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab94856a-0538-4855-a340-ef716d7297d4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:04.482394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ab94856a-0538-4855-a340-ef716d7297d4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:04.800596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-563e98c5-d7e5-41d9-8004-b31d96cbb83a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:04.814246 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.network.security_group_api [None req-ab94856a-0538-4855-a340-ef716d7297d4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Adding security group 24e4adb8-847d-4647-a320-2ac81698f21f to port 4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801 Sep 27 10:07:04.819021 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-563e98c5-d7e5-41d9-8004-b31d96cbb83a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1316 microversion: 2.1 time: 0.362166 Sep 27 10:07:04.819729 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1201/2626] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:04 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1316 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:04.842790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58c2e1a9-180a-4676-96eb-f4499248a7e5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:04.849901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-58c2e1a9-180a-4676-96eb-f4499248a7e5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:04.875744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c2e1a9-180a-4676-96eb-f4499248a7e5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:04.880271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c2e1a9-180a-4676-96eb-f4499248a7e5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:04.881092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c2e1a9-180a-4676-96eb-f4499248a7e5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:04.881575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c2e1a9-180a-4676-96eb-f4499248a7e5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:04.888713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c2e1a9-180a-4676-96eb-f4499248a7e5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:04.893698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58c2e1a9-180a-4676-96eb-f4499248a7e5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:04.963458 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-58c2e1a9-180a-4676-96eb-f4499248a7e5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c/ips" status: 200 len: 216 microversion: 2.1 time: 0.132137 Sep 27 10:07:04.964289 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1202/2627] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:07:04 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c/ips => generated 216 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:05.255514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b8ad3657-6b55-466f-97e4-75053ba69324 req-f36c089a-fdd2-407f-9946-ce9e1e5863b9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "961407e4-8e66-4f1c-88bd-6d2b4795c11d", "name": "network-vif-unplugged", "status": "completed", "tag": "52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:05.282087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b8ad3657-6b55-466f-97e4-75053ba69324 req-f36c089a-fdd2-407f-9946-ce9e1e5863b9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:05.283517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b8ad3657-6b55-466f-97e4-75053ba69324 req-f36c089a-fdd2-407f-9946-ce9e1e5863b9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:05.284146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b8ad3657-6b55-466f-97e4-75053ba69324 req-f36c089a-fdd2-407f-9946-ce9e1e5863b9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:05.306776 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b8ad3657-6b55-466f-97e4-75053ba69324 req-f36c089a-fdd2-407f-9946-ce9e1e5863b9 service nova] Creating event network-vif-unplugged:52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f for instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d on nch2-master-1 Sep 27 10:07:05.326961 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b8ad3657-6b55-466f-97e4-75053ba69324 req-f36c089a-fdd2-407f-9946-ce9e1e5863b9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.359315 Sep 27 10:07:05.328163 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1203/2628] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:04 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:05.338354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ab0fe02a-23d1-4c4d-98fa-ce733a37fbad req-cee65490-fcdd-4714-91a3-8584d7c54e7d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5", "tag": "205dffe7-0942-4b05-a466-8456d8d8713f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:05.369500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ab0fe02a-23d1-4c4d-98fa-ce733a37fbad req-cee65490-fcdd-4714-91a3-8584d7c54e7d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:05.369987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ab0fe02a-23d1-4c4d-98fa-ce733a37fbad req-cee65490-fcdd-4714-91a3-8584d7c54e7d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:05.370612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ab0fe02a-23d1-4c4d-98fa-ce733a37fbad req-cee65490-fcdd-4714-91a3-8584d7c54e7d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:05.388624 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ab0fe02a-23d1-4c4d-98fa-ce733a37fbad req-cee65490-fcdd-4714-91a3-8584d7c54e7d service nova] Creating event network-changed:205dffe7-0942-4b05-a466-8456d8d8713f for instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 on nch1-master-1 Sep 27 10:07:05.397386 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ab0fe02a-23d1-4c4d-98fa-ce733a37fbad req-cee65490-fcdd-4714-91a3-8584d7c54e7d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067553 Sep 27 10:07:05.398277 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1204/2629] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:05.404428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-357cf6e7-ab66-42c8-b607-302b70838873 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f629edaf0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:05.484396 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-357cf6e7-ab66-42c8-b607-302b70838873 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/detail?name=tempest-ListImageFiltersTestJSON-image-1258822588" status: 200 len: 665 microversion: 2.1 time: 0.084694 Sep 27 10:07:05.497282 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1205/2630] 10.222.0.26 () {58 vars in 1252 bytes} [Tue Sep 27 10:07:05 2022] GET /compute/v2.1/images/detail?name=tempest-ListImageFiltersTestJSON-image-1258822588 => generated 665 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:05.504402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:05.505429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:05.515854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:05.517030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:05.518044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:05.519612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:05.520527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:05.521428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:05.651709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:05.688534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:05.696590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:05.712751 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bb28c97b-03d1-4ddc-a516-1f62164ea886 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.213593 Sep 27 10:07:05.737216 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1206/2631] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:05 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:05.768339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-61d9cc4f-4337-48d6-bc35-4db251d39fef tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:05.769376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-61d9cc4f-4337-48d6-bc35-4db251d39fef tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:05.783423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d9cc4f-4337-48d6-bc35-4db251d39fef tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:05.784291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d9cc4f-4337-48d6-bc35-4db251d39fef tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:05.785165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d9cc4f-4337-48d6-bc35-4db251d39fef tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:05.786423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d9cc4f-4337-48d6-bc35-4db251d39fef tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:05.787159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d9cc4f-4337-48d6-bc35-4db251d39fef tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:05.788013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d9cc4f-4337-48d6-bc35-4db251d39fef tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:05.828628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-61d9cc4f-4337-48d6-bc35-4db251d39fef tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:07:06.002877 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-61d9cc4f-4337-48d6-bc35-4db251d39fef tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "DELETE /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 204 len: 0 microversion: 2.1 time: 0.263556 Sep 27 10:07:06.003721 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1207/2632] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:07:05 2022] DELETE /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:06.018393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:06.019178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:06.033759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:06.034262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:06.034710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:06.035443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:06.035910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:06.036338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:06.062292 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 HTTP/1.1 Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1c7bdeda-4ac1-4db1-a521-41316a9cf5c0'} Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:06.066454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:06.071069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:06.071069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:06.071069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:06.071069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:06.071069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:06.071069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:07:06.071069 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:06.071069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:06.071069 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0b3844a4-ac3d-450c-8f29-d581e60c61bd tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0" status: 404 len: 111 microversion: 2.1 time: 0.062724 Sep 27 10:07:06.073078 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1208/2633] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:06 2022] GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:06.082292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-07754c91-c3b6-4f14-9efa-41a718bf72b6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:06.082969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-07754c91-c3b6-4f14-9efa-41a718bf72b6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:06.095567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07754c91-c3b6-4f14-9efa-41a718bf72b6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:06.096068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07754c91-c3b6-4f14-9efa-41a718bf72b6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:06.109033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07754c91-c3b6-4f14-9efa-41a718bf72b6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:06.109033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07754c91-c3b6-4f14-9efa-41a718bf72b6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:06.109033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07754c91-c3b6-4f14-9efa-41a718bf72b6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:06.109033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07754c91-c3b6-4f14-9efa-41a718bf72b6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:06.288544 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ab94856a-0538-4855-a340-ef716d7297d4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3/action" status: 202 len: 0 microversion: 2.1 time: 1.839995 Sep 27 10:07:06.296120 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1426/2634] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:07:04 2022] POST /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3/action => generated 0 bytes in 1848 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:07:06.303706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5b27ce47-8223-473c-afcf-1d022a9bd8e8 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f6296a940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:06.416335 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5b27ce47-8223-473c-afcf-1d022a9bd8e8 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.26%2Fcompute%2Fv2.1%2Fservers%2F55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1465 microversion: 2.1 time: 0.118634 Sep 27 10:07:06.419150 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1427/2635] 10.222.0.26 () {58 vars in 1338 bytes} [Tue Sep 27 10:07:06 2022] GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.26%2Fcompute%2Fv2.1%2Fservers%2F55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1465 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:06.433786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5cbd3540-4691-4aa1-9f19-aaf45359dac0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:06.436282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5cbd3540-4691-4aa1-9f19-aaf45359dac0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:06.458072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cbd3540-4691-4aa1-9f19-aaf45359dac0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:06.458840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cbd3540-4691-4aa1-9f19-aaf45359dac0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:06.461226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cbd3540-4691-4aa1-9f19-aaf45359dac0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:06.463306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cbd3540-4691-4aa1-9f19-aaf45359dac0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:06.465457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cbd3540-4691-4aa1-9f19-aaf45359dac0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:06.466615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cbd3540-4691-4aa1-9f19-aaf45359dac0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:06.476218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-07754c91-c3b6-4f14-9efa-41a718bf72b6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:06.489890 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-07754c91-c3b6-4f14-9efa-41a718bf72b6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1519 microversion: 2.1 time: 0.415597 Sep 27 10:07:06.490522 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1209/2636] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:06 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1519 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:06.510091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7131ab8-927c-4af9-b3e3-0df1f352067c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:06.511004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7131ab8-927c-4af9-b3e3-0df1f352067c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:06.517979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7131ab8-927c-4af9-b3e3-0df1f352067c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:06.523155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7131ab8-927c-4af9-b3e3-0df1f352067c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:06.524107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7131ab8-927c-4af9-b3e3-0df1f352067c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:06.525130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7131ab8-927c-4af9-b3e3-0df1f352067c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:06.525817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7131ab8-927c-4af9-b3e3-0df1f352067c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:06.527424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7131ab8-927c-4af9-b3e3-0df1f352067c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:06.864076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c7131ab8-927c-4af9-b3e3-0df1f352067c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:06.875287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:06.898201 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c7131ab8-927c-4af9-b3e3-0df1f352067c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1809 microversion: 2.1 time: 0.405195 Sep 27 10:07:06.903172 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1210/2637] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:06 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1809 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:06.925408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1126070510", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "0117c46b-39f2-4cf5-9f8a-7710125bdf55"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:06.969986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5cbd3540-4691-4aa1-9f19-aaf45359dac0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:06.994522 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5cbd3540-4691-4aa1-9f19-aaf45359dac0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1316 microversion: 2.1 time: 0.573913 Sep 27 10:07:07.004136 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1428/2638] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:06 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1316 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:07.020385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58fc16f1-7ebe-4870-87ad-1787b8940a5d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62992c10>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:07.083839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] validate_networks() for [('0117c46b-39f2-4cf5-9f8a-7710125bdf55', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:07:07.519213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-58fc16f1-7ebe-4870-87ad-1787b8940a5d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupInUse", "message": "Security Group 24e4adb8-847d-4647-a320-2ac81698f21f in use.", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:07:07.520039 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-58fc16f1-7ebe-4870-87ad-1787b8940a5d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security Group 24e4adb8-847d-4647-a320-2ac81698f21f in use. Sep 27 10:07:07.520039 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-f2ad36e2-5e9b-4fde-8079-b0acb96fb7fa'] Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58fc16f1-7ebe-4870-87ad-1787b8940a5d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-security-groups/24e4adb8-847d-4647-a320-2ac81698f21f HTTP/1.1 Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: exception: Security Group 24e4adb8-847d-4647-a320-2ac81698f21f in use. Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-f2ad36e2-5e9b-4fde-8079-b0acb96fb7fa'] Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '24e4adb8-847d-4647-a320-2ac81698f21f'} Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 359, in destroy Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutron.delete_security_group(security_group['id']) Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1073, in delete_security_group Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.delete(self.security_group_path % (security_group)) Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 352, in delete Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("DELETE", action, body=body, Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:07:07.521993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Security Group 24e4adb8-847d-4647-a320-2ac81698f21f in use. Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f2ad36e2-5e9b-4fde-8079-b0acb96fb7fa'] Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 179, in delete Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_group_api.destroy(context, security_group) Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 364, in destroy Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.Invalid(str(e)) Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Security Group 24e4adb8-847d-4647-a320-2ac81698f21f in use. Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f2ad36e2-5e9b-4fde-8079-b0acb96fb7fa'] Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 183, in delete Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Security Group 24e4adb8-847d-4647-a320-2ac81698f21f in use. Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f2ad36e2-5e9b-4fde-8079-b0acb96fb7fa'] Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security Group 24e4adb8-847d-4647-a320-2ac81698f21f in use. Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f2ad36e2-5e9b-4fde-8079-b0acb96fb7fa'] Sep 27 10:07:07.530160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:07.538110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58fc16f1-7ebe-4870-87ad-1787b8940a5d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 400 to user: Security Group 24e4adb8-847d-4647-a320-2ac81698f21f in use. Sep 27 10:07:07.538110 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-f2ad36e2-5e9b-4fde-8079-b0acb96fb7fa'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:07.539960 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58fc16f1-7ebe-4870-87ad-1787b8940a5d tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/24e4adb8-847d-4647-a320-2ac81698f21f" status: 400 len: 185 microversion: 2.1 time: 0.533395 Sep 27 10:07:07.544531 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1429/2639] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:07 2022] DELETE /compute/v2.1/os-security-groups/24e4adb8-847d-4647-a320-2ac81698f21f => generated 185 bytes in 540 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:07:07.550854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9536f6c0-b8fa-475d-a093-bb5a2ebbad68 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f629201f0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:07.690909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:07:07.692583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:07:07.693613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:07:07.694334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:07:07.695420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:07:07.709430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:07.710030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:07.710534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:07.711485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:07.712328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:07.712778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:07.735860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:07:07.742663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for user 7f7c5adbac4f48b6bc141e0f888f2496 and project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:07:07.762928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:07:07.766255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:07:07.780992 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9536f6c0-b8fa-475d-a093-bb5a2ebbad68 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.26%2Fcompute%2Fservers%2F55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1465 microversion: 2.1 time: 0.234048 Sep 27 10:07:07.781755 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1430/2640] 10.222.0.26 () {58 vars in 1324 bytes} [Tue Sep 27 10:07:07 2022] GET /compute/v2.1/images/detail?server=http%3A%2F%2F10.222.0.26%2Fcompute%2Fservers%2F55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1465 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:07.787221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f48364e-854e-4861-b676-3287f4395f22 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:07.787730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f48364e-854e-4861-b676-3287f4395f22 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:07.814084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f48364e-854e-4861-b676-3287f4395f22 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:07.815648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f48364e-854e-4861-b676-3287f4395f22 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:07.817265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f48364e-854e-4861-b676-3287f4395f22 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:07.821519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f48364e-854e-4861-b676-3287f4395f22 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:07.823895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f48364e-854e-4861-b676-3287f4395f22 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:07.824605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f48364e-854e-4861-b676-3287f4395f22 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:07.855887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f48364e-854e-4861-b676-3287f4395f22 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:07:07.903613 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a1c4389-5466-4b92-8bbd-66cf47063f9f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.998405 Sep 27 10:07:07.910521 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1211/2641] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 1000 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:07:07.910521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:07.910521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:07.918500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:07.919203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:07.919865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:07.921439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:07.922075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:07.922616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:07.970670 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f48364e-854e-4861-b676-3287f4395f22 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 204 len: 0 microversion: 2.1 time: 0.187024 Sep 27 10:07:07.979026 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1431/2642] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:07:07 2022] DELETE /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:07.985439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f670fd91-3f8f-4a72-8ddf-b0bb5840e5a9 req-2c45be9d-995b-4e64-a410-7611bb4101b3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0abee65c-58ee-4b9c-ac6c-65631519b216", "name": "network-vif-plugged", "status": "completed", "tag": "447ad707-a60a-4017-9b0d-e87acd64bee0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:07.997589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f670fd91-3f8f-4a72-8ddf-b0bb5840e5a9 req-2c45be9d-995b-4e64-a410-7611bb4101b3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:07.998141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f670fd91-3f8f-4a72-8ddf-b0bb5840e5a9 req-2c45be9d-995b-4e64-a410-7611bb4101b3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:07.999408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f670fd91-3f8f-4a72-8ddf-b0bb5840e5a9 req-2c45be9d-995b-4e64-a410-7611bb4101b3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.013737 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f670fd91-3f8f-4a72-8ddf-b0bb5840e5a9 req-2c45be9d-995b-4e64-a410-7611bb4101b3 service nova] Creating event network-vif-plugged:447ad707-a60a-4017-9b0d-e87acd64bee0 for instance 0abee65c-58ee-4b9c-ac6c-65631519b216 on nch1-master-1 Sep 27 10:07:08.019401 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f670fd91-3f8f-4a72-8ddf-b0bb5840e5a9 req-2c45be9d-995b-4e64-a410-7611bb4101b3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038565 Sep 27 10:07:08.023487 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1432/2643] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:08.028825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-16d502e6-a27e-4282-89e1-e57204ea192c tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:08.030906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-16d502e6-a27e-4282-89e1-e57204ea192c tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:08.036641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:08.037656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16d502e6-a27e-4282-89e1-e57204ea192c tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:08.041427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16d502e6-a27e-4282-89e1-e57204ea192c tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:08.042252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16d502e6-a27e-4282-89e1-e57204ea192c tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.043262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16d502e6-a27e-4282-89e1-e57204ea192c tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:08.043885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16d502e6-a27e-4282-89e1-e57204ea192c tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:08.044462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16d502e6-a27e-4282-89e1-e57204ea192c tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.050640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:08.052283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:08.058554 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8ab7c07c-118e-4577-bff8-e7a43d887f6a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.153206 Sep 27 10:07:08.073291 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1212/2644] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:07 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:08.081252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-16d502e6-a27e-4282-89e1-e57204ea192c tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:07:08.087351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-97b88c1b-9e9f-43be-b4d4-7ad790ad0031 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f629e9af0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:08.238287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-16d502e6-a27e-4282-89e1-e57204ea192c tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 204 len: 0 microversion: 2.1 time: 0.212768 Sep 27 10:07:08.240026 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1433/2645] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:07:08 2022] DELETE /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:08.247919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ab7c62b6-a02a-45c6-a511-dc5d535f134d req-f7c2aa0a-974f-4708-8d43-f0fe9bfdf926 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ac5c4cdc-90a8-48c7-84f3-0053b32834f3", "tag": "4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:08.259327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ab7c62b6-a02a-45c6-a511-dc5d535f134d req-f7c2aa0a-974f-4708-8d43-f0fe9bfdf926 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:08.259448 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-97b88c1b-9e9f-43be-b4d4-7ad790ad0031 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/detail?status=ACTIVE" status: 200 len: 7102 microversion: 2.1 time: 0.184624 Sep 27 10:07:08.259885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ab7c62b6-a02a-45c6-a511-dc5d535f134d req-f7c2aa0a-974f-4708-8d43-f0fe9bfdf926 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:08.260381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ab7c62b6-a02a-45c6-a511-dc5d535f134d req-f7c2aa0a-974f-4708-8d43-f0fe9bfdf926 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.270967 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1213/2646] 10.222.0.26 () {58 vars in 1170 bytes} [Tue Sep 27 10:07:08 2022] GET /compute/v2.1/images/detail?status=ACTIVE => generated 7102 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:08.284143 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ab7c62b6-a02a-45c6-a511-dc5d535f134d req-f7c2aa0a-974f-4708-8d43-f0fe9bfdf926 service nova] Creating event network-changed:4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801 for instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 on nch1-master-1 Sep 27 10:07:08.287722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4bcd7e6b-6932-4ff9-86e2-bc254b1f9b02 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:08.288600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4bcd7e6b-6932-4ff9-86e2-bc254b1f9b02 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:08.290558 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ab7c62b6-a02a-45c6-a511-dc5d535f134d req-f7c2aa0a-974f-4708-8d43-f0fe9bfdf926 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048251 Sep 27 10:07:08.296076 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1434/2647] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:08.304352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f31e937c-6ef4-4ac5-8f0b-15b0de432a92 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:08.305633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f31e937c-6ef4-4ac5-8f0b-15b0de432a92 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:08.321104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f31e937c-6ef4-4ac5-8f0b-15b0de432a92 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:08.321104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f31e937c-6ef4-4ac5-8f0b-15b0de432a92 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:08.321443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f31e937c-6ef4-4ac5-8f0b-15b0de432a92 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.322178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f31e937c-6ef4-4ac5-8f0b-15b0de432a92 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:08.322646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f31e937c-6ef4-4ac5-8f0b-15b0de432a92 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:08.323080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f31e937c-6ef4-4ac5-8f0b-15b0de432a92 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.381743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4bcd7e6b-6932-4ff9-86e2-bc254b1f9b02 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:08.392826 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4bcd7e6b-6932-4ff9-86e2-bc254b1f9b02 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1215 microversion: 2.1 time: 0.118402 Sep 27 10:07:08.396315 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1214/2648] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:08 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1215 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:08.402561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dcef3c17-34b7-410e-9207-96c1ea1145cc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:08.403182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dcef3c17-34b7-410e-9207-96c1ea1145cc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:08.410252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcef3c17-34b7-410e-9207-96c1ea1145cc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:08.410748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcef3c17-34b7-410e-9207-96c1ea1145cc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:08.411188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcef3c17-34b7-410e-9207-96c1ea1145cc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.411987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcef3c17-34b7-410e-9207-96c1ea1145cc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:08.412443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcef3c17-34b7-410e-9207-96c1ea1145cc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:08.412909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcef3c17-34b7-410e-9207-96c1ea1145cc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.555521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f31e937c-6ef4-4ac5-8f0b-15b0de432a92 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:08.567328 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f31e937c-6ef4-4ac5-8f0b-15b0de432a92 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1809 microversion: 2.1 time: 0.269092 Sep 27 10:07:08.569305 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1435/2649] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:08 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1809 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:08.576784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b8722f06-dc54-42ab-a7e4-6a0b659e6b81 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:08.577788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b8722f06-dc54-42ab-a7e4-6a0b659e6b81 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:08.585458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8722f06-dc54-42ab-a7e4-6a0b659e6b81 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:08.586580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8722f06-dc54-42ab-a7e4-6a0b659e6b81 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:08.587354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8722f06-dc54-42ab-a7e4-6a0b659e6b81 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.588295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8722f06-dc54-42ab-a7e4-6a0b659e6b81 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:08.588988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8722f06-dc54-42ab-a7e4-6a0b659e6b81 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:08.589614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8722f06-dc54-42ab-a7e4-6a0b659e6b81 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.741829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dcef3c17-34b7-410e-9207-96c1ea1145cc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:08.766605 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dcef3c17-34b7-410e-9207-96c1ea1145cc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1525 microversion: 2.1 time: 0.367815 Sep 27 10:07:08.786161 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1215/2650] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:08 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1525 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:08.809966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f4b518e9-02c4-4fd4-929a-28ba44be4b51 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:08.810999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f4b518e9-02c4-4fd4-929a-28ba44be4b51 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:08.832406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b518e9-02c4-4fd4-929a-28ba44be4b51 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:08.833156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b518e9-02c4-4fd4-929a-28ba44be4b51 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:08.833793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b518e9-02c4-4fd4-929a-28ba44be4b51 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.834678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b518e9-02c4-4fd4-929a-28ba44be4b51 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:08.835295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b518e9-02c4-4fd4-929a-28ba44be4b51 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:08.836003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b518e9-02c4-4fd4-929a-28ba44be4b51 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:08.861472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b8722f06-dc54-42ab-a7e4-6a0b659e6b81 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:08.888560 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b8722f06-dc54-42ab-a7e4-6a0b659e6b81 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1316 microversion: 2.1 time: 0.316708 Sep 27 10:07:08.894778 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1436/2651] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:08 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1316 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:08.908082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2dda4477-0a65-4fa8-9f9c-beccdb4fccb2 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f628e33a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:09.071170 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2dda4477-0a65-4fa8-9f9c-beccdb4fccb2 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/detail?type=snapshot" status: 200 len: 4370 microversion: 2.1 time: 0.175031 Sep 27 10:07:09.072593 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1437/2652] 10.222.0.26 () {58 vars in 1170 bytes} [Tue Sep 27 10:07:08 2022] GET /compute/v2.1/images/detail?type=snapshot => generated 4370 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:09.093574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:09.095450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:09.106667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:09.108919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:09.109616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:09.110727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:09.111502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:09.112262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:09.209321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:09.225072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:09.227298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:09.236159 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bc4bbbcd-e6e3-48dc-92c9-971c5b1c8402 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.146868 Sep 27 10:07:09.237103 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1438/2653] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:09 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:09.254416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c5f5f9a7-4842-4295-a114-56f2d7acad28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f628ec3a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:09.345245 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c5f5f9a7-4842-4295-a114-56f2d7acad28 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6" status: 200 len: 784 microversion: 2.1 time: 0.102880 Sep 27 10:07:09.347167 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1439/2654] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:07:09 2022] GET /compute/v2.1/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6 => generated 784 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:09.365524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-4076bec8-14f8-4ae4-b66b-f9f0b78bd72b req-f665d8cf-1cd3-407e-8010-5a83391b9a71 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b5bcf59-770f-40f1-83b0-0738578e921c", "name": "network-vif-deleted", "tag": "cd5a9e3b-1cef-4cbd-a955-7fc284d5c0d4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:09.394835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4076bec8-14f8-4ae4-b66b-f9f0b78bd72b req-f665d8cf-1cd3-407e-8010-5a83391b9a71 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:09.395521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4076bec8-14f8-4ae4-b66b-f9f0b78bd72b req-f665d8cf-1cd3-407e-8010-5a83391b9a71 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:09.395968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4076bec8-14f8-4ae4-b66b-f9f0b78bd72b req-f665d8cf-1cd3-407e-8010-5a83391b9a71 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:09.441852 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-4076bec8-14f8-4ae4-b66b-f9f0b78bd72b req-f665d8cf-1cd3-407e-8010-5a83391b9a71 service nova] Creating event network-vif-deleted:cd5a9e3b-1cef-4cbd-a955-7fc284d5c0d4 for instance 5b5bcf59-770f-40f1-83b0-0738578e921c on nch2-master-1 Sep 27 10:07:09.450425 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-4076bec8-14f8-4ae4-b66b-f9f0b78bd72b req-f665d8cf-1cd3-407e-8010-5a83391b9a71 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087615 Sep 27 10:07:09.451456 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1440/2655] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:09.458582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d1d0bf99-24b0-4a46-9ff5-2863425fce58 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f628ca9d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:09.568489 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d1d0bf99-24b0-4a46-9ff5-2863425fce58 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/images/detail?limit=1" status: 200 len: 1609 microversion: 2.1 time: 0.115562 Sep 27 10:07:09.570220 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1441/2656] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 10:07:09 2022] GET /compute/v2.1/images/detail?limit=1 => generated 1609 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:09.587978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a89d5e7a-461c-41dd-8779-c9e8ed40c434 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:09.589226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a89d5e7a-461c-41dd-8779-c9e8ed40c434 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:09.597246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d5e7a-461c-41dd-8779-c9e8ed40c434 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:09.598014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d5e7a-461c-41dd-8779-c9e8ed40c434 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:09.598753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d5e7a-461c-41dd-8779-c9e8ed40c434 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:09.607936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f4b518e9-02c4-4fd4-929a-28ba44be4b51 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:09.612994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d5e7a-461c-41dd-8779-c9e8ed40c434 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:09.613552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d5e7a-461c-41dd-8779-c9e8ed40c434 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:09.614005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d5e7a-461c-41dd-8779-c9e8ed40c434 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:09.623039 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f4b518e9-02c4-4fd4-929a-28ba44be4b51 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1646 microversion: 2.1 time: 0.831254 Sep 27 10:07:09.624360 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1216/2657] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:08 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1646 bytes in 837 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:09.637337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-86b400e3-b1aa-458a-b449-19fdf799e53c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f629e4b80>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:09.701981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a89d5e7a-461c-41dd-8779-c9e8ed40c434 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:09.714204 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a89d5e7a-461c-41dd-8779-c9e8ed40c434 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1267 microversion: 2.1 time: 0.142147 Sep 27 10:07:09.729212 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1442/2658] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:09 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1267 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:09.735369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6c536de0-aaaa-4790-b3bc-c745296e0ea6 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:09.736005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6c536de0-aaaa-4790-b3bc-c745296e0ea6 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:09.743257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c536de0-aaaa-4790-b3bc-c745296e0ea6 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:09.743692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c536de0-aaaa-4790-b3bc-c745296e0ea6 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:09.744142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c536de0-aaaa-4790-b3bc-c745296e0ea6 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:09.744920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c536de0-aaaa-4790-b3bc-c745296e0ea6 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:09.745977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c536de0-aaaa-4790-b3bc-c745296e0ea6 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:09.746091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c536de0-aaaa-4790-b3bc-c745296e0ea6 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:09.905358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6c536de0-aaaa-4790-b3bc-c745296e0ea6 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:09.915960 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6c536de0-aaaa-4790-b3bc-c745296e0ea6 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1630 microversion: 2.1 time: 0.185592 Sep 27 10:07:09.916777 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1443/2659] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:09 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1630 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:09.939788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e01f737-c0bf-4991-8717-37b8d5347b2e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:09.940322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9e01f737-c0bf-4991-8717-37b8d5347b2e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:09.946974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e01f737-c0bf-4991-8717-37b8d5347b2e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:09.947620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e01f737-c0bf-4991-8717-37b8d5347b2e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:09.948397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e01f737-c0bf-4991-8717-37b8d5347b2e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:09.949571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e01f737-c0bf-4991-8717-37b8d5347b2e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:09.950086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e01f737-c0bf-4991-8717-37b8d5347b2e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:09.950527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e01f737-c0bf-4991-8717-37b8d5347b2e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:10.184517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9e01f737-c0bf-4991-8717-37b8d5347b2e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:10.193711 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9e01f737-c0bf-4991-8717-37b8d5347b2e tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1525 microversion: 2.1 time: 0.264352 Sep 27 10:07:10.194668 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1444/2660] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:09 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1525 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:10.209131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25a89281-07ee-447d-9afa-5539af246e7c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:10.209767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-25a89281-07ee-447d-9afa-5539af246e7c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:10.223166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25a89281-07ee-447d-9afa-5539af246e7c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:10.230832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25a89281-07ee-447d-9afa-5539af246e7c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:10.230832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25a89281-07ee-447d-9afa-5539af246e7c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:10.230832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25a89281-07ee-447d-9afa-5539af246e7c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:10.231187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25a89281-07ee-447d-9afa-5539af246e7c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:10.231841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25a89281-07ee-447d-9afa-5539af246e7c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:10.473838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-25a89281-07ee-447d-9afa-5539af246e7c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:10.484713 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-25a89281-07ee-447d-9afa-5539af246e7c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1316 microversion: 2.1 time: 0.288594 Sep 27 10:07:10.486421 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1445/2661] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:10 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1316 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:10.492696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0a6c10c1-82ef-4aa9-8ecc-81e497a14d62 req-dfa6cf0c-4adf-4301-9297-a56b2b56e139 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0abee65c-58ee-4b9c-ac6c-65631519b216", "name": "network-vif-unplugged", "status": "completed", "tag": "447ad707-a60a-4017-9b0d-e87acd64bee0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:10.531282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0a6c10c1-82ef-4aa9-8ecc-81e497a14d62 req-dfa6cf0c-4adf-4301-9297-a56b2b56e139 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:10.531888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0a6c10c1-82ef-4aa9-8ecc-81e497a14d62 req-dfa6cf0c-4adf-4301-9297-a56b2b56e139 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:10.532191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0a6c10c1-82ef-4aa9-8ecc-81e497a14d62 req-dfa6cf0c-4adf-4301-9297-a56b2b56e139 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:10.548184 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0a6c10c1-82ef-4aa9-8ecc-81e497a14d62 req-dfa6cf0c-4adf-4301-9297-a56b2b56e139 service nova] Creating event network-vif-unplugged:447ad707-a60a-4017-9b0d-e87acd64bee0 for instance 0abee65c-58ee-4b9c-ac6c-65631519b216 on nch1-master-1 Sep 27 10:07:10.564036 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0a6c10c1-82ef-4aa9-8ecc-81e497a14d62 req-dfa6cf0c-4adf-4301-9297-a56b2b56e139 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.074639 Sep 27 10:07:10.569116 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1446/2662] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:10 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:10.576221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:10.577195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:10.586714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:10.587666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:10.588379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:10.591156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:10.591941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:10.592748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:10.626979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:10.693892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:10.705709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:10.708301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:10.718820 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-78c6a1ab-6263-4cc0-b8b3-833e996e38c1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.146985 Sep 27 10:07:10.720092 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1447/2663] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:10 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:10.730333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6c171dc4-f138-43fc-80a4-14d59f2b6af2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:10.731184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6c171dc4-f138-43fc-80a4-14d59f2b6af2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:10.742151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c171dc4-f138-43fc-80a4-14d59f2b6af2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:10.743034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c171dc4-f138-43fc-80a4-14d59f2b6af2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:10.743655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c171dc4-f138-43fc-80a4-14d59f2b6af2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:10.744611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c171dc4-f138-43fc-80a4-14d59f2b6af2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:10.745497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c171dc4-f138-43fc-80a4-14d59f2b6af2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:10.749740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c171dc4-f138-43fc-80a4-14d59f2b6af2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:10.875715 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-86b400e3-b1aa-458a-b449-19fdf799e53c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "DELETE /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7" status: 204 len: 0 microversion: 2.1 time: 1.249547 Sep 27 10:07:10.876295 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1217/2664] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:09 2022] DELETE /compute/v2.1/images/66ebf0ca-16ea-4f22-9664-f9cd188002c7 => generated 0 bytes in 1251 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:10.886427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ded04f74-3c63-433f-93f7-62b07e2cfdf9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:10.887476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ded04f74-3c63-433f-93f7-62b07e2cfdf9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:10.925824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ded04f74-3c63-433f-93f7-62b07e2cfdf9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:10.927788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ded04f74-3c63-433f-93f7-62b07e2cfdf9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:10.932465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ded04f74-3c63-433f-93f7-62b07e2cfdf9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:10.934842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ded04f74-3c63-433f-93f7-62b07e2cfdf9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:10.941133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ded04f74-3c63-433f-93f7-62b07e2cfdf9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:10.943027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ded04f74-3c63-433f-93f7-62b07e2cfdf9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:11.134713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ded04f74-3c63-433f-93f7-62b07e2cfdf9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:11.151705 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ded04f74-3c63-433f-93f7-62b07e2cfdf9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1273 microversion: 2.1 time: 0.273635 Sep 27 10:07:11.157403 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1218/2665] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:10 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1273 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:11.175049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6c171dc4-f138-43fc-80a4-14d59f2b6af2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:11.183429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f1efdfc-b241-4a08-bf68-427738fdd23c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f62ad3430>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:11.191585 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6c171dc4-f138-43fc-80a4-14d59f2b6af2 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1646 microversion: 2.1 time: 0.469466 Sep 27 10:07:11.192864 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1448/2666] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:10 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1646 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:11.201907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a1bc2c7-cd24-4a4d-b815-2a4f8fc56209 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:11.202876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3a1bc2c7-cd24-4a4d-b815-2a4f8fc56209 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:11.211609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1bc2c7-cd24-4a4d-b815-2a4f8fc56209 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:11.213344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1bc2c7-cd24-4a4d-b815-2a4f8fc56209 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:11.214091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1bc2c7-cd24-4a4d-b815-2a4f8fc56209 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:11.215103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1bc2c7-cd24-4a4d-b815-2a4f8fc56209 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:11.215756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1bc2c7-cd24-4a4d-b815-2a4f8fc56209 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:11.216773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1bc2c7-cd24-4a4d-b815-2a4f8fc56209 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:11.441069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3a1bc2c7-cd24-4a4d-b815-2a4f8fc56209 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:11.451979 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3a1bc2c7-cd24-4a4d-b815-2a4f8fc56209 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 200 len: 1609 microversion: 2.1 time: 0.253062 Sep 27 10:07:11.453029 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1449/2667] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:11 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 1609 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:11.496626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=9 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:11.677076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ec8c57cc-0fa7-4897-9777-418d97a9e1e4 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Action: 'update', calling method: .version_select at 0x7f7f62920ee0>, body: {"quota_set": {"ram": "5120"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:11.787503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8c57cc-0fa7-4897-9777-418d97a9e1e4 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:11.788007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8c57cc-0fa7-4897-9777-418d97a9e1e4 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:11.789165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8c57cc-0fa7-4897-9777-418d97a9e1e4 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:11.790831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8c57cc-0fa7-4897-9777-418d97a9e1e4 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:11.791812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8c57cc-0fa7-4897-9777-418d97a9e1e4 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:11.792481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8c57cc-0fa7-4897-9777-418d97a9e1e4 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:11.890431 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ec8c57cc-0fa7-4897-9777-418d97a9e1e4 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/91e709a461b944f4a9118e979d016f7b" status: 200 len: 148 microversion: 2.57 time: 0.431646 Sep 27 10:07:11.891645 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1450/2668] 10.222.0.26 () {62 vars in 1303 bytes} [Tue Sep 27 10:07:11 2022] PUT /compute/v2.1/os-quota-sets/91e709a461b944f4a9118e979d016f7b => generated 148 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:07:11.906842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da99d5b6-61cf-419a-a3c6-00d83928b40b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:11.907550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-da99d5b6-61cf-419a-a3c6-00d83928b40b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:11.916746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da99d5b6-61cf-419a-a3c6-00d83928b40b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:11.917272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da99d5b6-61cf-419a-a3c6-00d83928b40b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:11.917753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da99d5b6-61cf-419a-a3c6-00d83928b40b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:11.918646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da99d5b6-61cf-419a-a3c6-00d83928b40b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:11.919137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da99d5b6-61cf-419a-a3c6-00d83928b40b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:11.919972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da99d5b6-61cf-419a-a3c6-00d83928b40b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:12.186631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-da99d5b6-61cf-419a-a3c6-00d83928b40b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:12.201116 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da99d5b6-61cf-419a-a3c6-00d83928b40b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1525 microversion: 2.1 time: 0.305923 Sep 27 10:07:12.201116 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1451/2669] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:11 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1525 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:12.224406 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f1efdfc-b241-4a08-bf68-427738fdd23c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "DELETE /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0" status: 204 len: 0 microversion: 2.1 time: 1.065146 Sep 27 10:07:12.226069 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1219/2670] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:11 2022] DELETE /compute/v2.1/images/c97600cb-d1c9-40ad-9413-c03315fd26b0 => generated 0 bytes in 1068 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:12.232503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5c7bcbff-2bc0-447c-a845-0c4c7e5495b2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:12.233571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5c7bcbff-2bc0-447c-a845-0c4c7e5495b2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:12.240717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c7bcbff-2bc0-447c-a845-0c4c7e5495b2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:12.241106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c7bcbff-2bc0-447c-a845-0c4c7e5495b2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:12.241544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c7bcbff-2bc0-447c-a845-0c4c7e5495b2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:12.242496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c7bcbff-2bc0-447c-a845-0c4c7e5495b2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:12.242990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c7bcbff-2bc0-447c-a845-0c4c7e5495b2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:12.243467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c7bcbff-2bc0-447c-a845-0c4c7e5495b2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:12.329575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-66d97309-e171-448c-a0f2-0b38530f74a0 req-10521512-5129-474a-b4d2-2fa0ed609032 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b5bcf59-770f-40f1-83b0-0738578e921c", "name": "network-vif-deleted", "tag": "f3e142bd-656b-42c1-b4a6-376501c219f9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:12.342527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-66d97309-e171-448c-a0f2-0b38530f74a0 req-10521512-5129-474a-b4d2-2fa0ed609032 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:12.347932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-66d97309-e171-448c-a0f2-0b38530f74a0 req-10521512-5129-474a-b4d2-2fa0ed609032 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:12.352437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-66d97309-e171-448c-a0f2-0b38530f74a0 req-10521512-5129-474a-b4d2-2fa0ed609032 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:12.366368 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-66d97309-e171-448c-a0f2-0b38530f74a0 req-10521512-5129-474a-b4d2-2fa0ed609032 service nova] Creating event network-vif-deleted:f3e142bd-656b-42c1-b4a6-376501c219f9 for instance 5b5bcf59-770f-40f1-83b0-0738578e921c on nch2-master-1 Sep 27 10:07:12.373145 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-66d97309-e171-448c-a0f2-0b38530f74a0 req-10521512-5129-474a-b4d2-2fa0ed609032 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.171960 Sep 27 10:07:12.373895 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1452/2671] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:12.379868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:12.380966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:12.390899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:12.391696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:12.392391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:12.393396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:12.394081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:12.394731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:12.512193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:12.524039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:12.526829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:12.534069 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-308b81a6-ca8b-4cbd-9eb3-69a99e950593 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.158875 Sep 27 10:07:12.535370 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1453/2672] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:12 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:12.541896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a6a0cb33-1073-4cc9-9e03-3e4901f27e2a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Calling method '.version_select at 0x7f7f628e3940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:12.668057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6a0cb33-1073-4cc9-9e03-3e4901f27e2a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:12.668854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6a0cb33-1073-4cc9-9e03-3e4901f27e2a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:12.669471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6a0cb33-1073-4cc9-9e03-3e4901f27e2a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:12.670522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6a0cb33-1073-4cc9-9e03-3e4901f27e2a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:12.671136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6a0cb33-1073-4cc9-9e03-3e4901f27e2a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:12.671773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6a0cb33-1073-4cc9-9e03-3e4901f27e2a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:12.698819 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a6a0cb33-1073-4cc9-9e03-3e4901f27e2a tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/91e709a461b944f4a9118e979d016f7b/detail" status: 200 len: 463 microversion: 2.57 time: 0.161345 Sep 27 10:07:12.703038 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1454/2673] 10.222.0.26 () {60 vars in 1304 bytes} [Tue Sep 27 10:07:12 2022] GET /compute/v2.1/os-quota-sets/91e709a461b944f4a9118e979d016f7b/detail => generated 463 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:07:12.717158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4482fa9b-60ac-40d7-bd91-32d9e5ac33ca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:12.722163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4482fa9b-60ac-40d7-bd91-32d9e5ac33ca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:12.732082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4482fa9b-60ac-40d7-bd91-32d9e5ac33ca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:12.733429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4482fa9b-60ac-40d7-bd91-32d9e5ac33ca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:12.734379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4482fa9b-60ac-40d7-bd91-32d9e5ac33ca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:12.735352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4482fa9b-60ac-40d7-bd91-32d9e5ac33ca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:12.736034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4482fa9b-60ac-40d7-bd91-32d9e5ac33ca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:12.736704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4482fa9b-60ac-40d7-bd91-32d9e5ac33ca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:12.756309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5c7bcbff-2bc0-447c-a845-0c4c7e5495b2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:12.780541 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5c7bcbff-2bc0-447c-a845-0c4c7e5495b2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1316 microversion: 2.1 time: 0.549428 Sep 27 10:07:12.781859 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1220/2674] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:12 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1316 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:12.796296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eb754924-cde6-4432-b506-7b69bc99ecae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:12.797428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eb754924-cde6-4432-b506-7b69bc99ecae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:12.817181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb754924-cde6-4432-b506-7b69bc99ecae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:12.818213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb754924-cde6-4432-b506-7b69bc99ecae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:12.818895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb754924-cde6-4432-b506-7b69bc99ecae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:12.819857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb754924-cde6-4432-b506-7b69bc99ecae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:12.820738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb754924-cde6-4432-b506-7b69bc99ecae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:12.822306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb754924-cde6-4432-b506-7b69bc99ecae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:13.401387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4482fa9b-60ac-40d7-bd91-32d9e5ac33ca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:13.412842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eb754924-cde6-4432-b506-7b69bc99ecae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:13.413550 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4482fa9b-60ac-40d7-bd91-32d9e5ac33ca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1315 microversion: 2.1 time: 0.708109 Sep 27 10:07:13.416083 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1455/2675] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:12 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1315 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:13.437993 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eb754924-cde6-4432-b506-7b69bc99ecae tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1646 microversion: 2.1 time: 0.654674 Sep 27 10:07:13.439777 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1221/2676] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:12 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1646 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:13.441645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b86a6042-df2b-41a8-a25c-ca87c49c7151 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f628ec430>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:13.448072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:13.451958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:13.469076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:13.469855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:13.471001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:13.471897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:13.481835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:13.482849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:13.501041 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] HTTP exception thrown: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Request method failure captured: Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c HTTP/1.1 Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMssUrCgLigC9SH81tPyx_Z2smIvqriOlzOxLjwou3UdxxR5VrdtDv5ZLspr7gwjUKnJ9eO9sY6_VK0lEb_or5QoOte1wEO6cw3psqOXVHSx1G3EHl2Qng7IOMpa9960M9YvpOEI8way30BSGJK5IxbZEPCLKI85uv_-gSO6hLkR0C4A Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fedcdffc12b94b489ce433d5caa5e4be"}]}] Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersTestMultiNic-242412447 Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersTestMultiNic-242412447-project Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 430de2723f234fac8f09068195d8ce2c Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersTestMultiNic-242412447-project Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '5b5bcf59-770f-40f1-83b0-0738578e921c'} Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:13.503352 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:13.508194 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:13.508194 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:13.508194 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:13.508194 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:13.508194 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:13.508194 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:07:13.508194 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:13.510445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Returning 404 to user: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:13.512159 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8600ddda-3a08-4808-a4c8-9ac97ed01031 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 404 len: 111 microversion: 2.1 time: 0.070811 Sep 27 10:07:13.513477 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1222/2677] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:13 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 111 bytes in 73 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:13.520467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Action: 'update', calling method: .version_select at 0x7f7f629ef940>, body: {"quota_set": {"ram": "2048"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:13.755780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:13.756275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:13.756737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:13.757790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:13.758753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:13.771913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:13.919416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:13.919835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:13.920286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:13.921071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:13.921534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:13.922247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:14.078441 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c044fda-c2a7-4a6b-a268-b34051257fbd tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-quota-sets/91e709a461b944f4a9118e979d016f7b?user_id=0f1ba9f0940a46b59b18d87049d9eea9" status: 200 len: 148 microversion: 2.57 time: 0.563139 Sep 27 10:07:14.081632 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1223/2678] 10.222.0.26 () {62 vars in 1384 bytes} [Tue Sep 27 10:07:13 2022] PUT /compute/v2.1/os-quota-sets/91e709a461b944f4a9118e979d016f7b?user_id=0f1ba9f0940a46b59b18d87049d9eea9 => generated 148 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:07:14.083898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd925202-616a-4d39-9f67-17f192af4fbb tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:14.084527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cd925202-616a-4d39-9f67-17f192af4fbb tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:14.102977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd925202-616a-4d39-9f67-17f192af4fbb tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:14.103505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd925202-616a-4d39-9f67-17f192af4fbb tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:14.103989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd925202-616a-4d39-9f67-17f192af4fbb tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:14.104786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd925202-616a-4d39-9f67-17f192af4fbb tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:14.105278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd925202-616a-4d39-9f67-17f192af4fbb tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:14.105929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd925202-616a-4d39-9f67-17f192af4fbb tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:14.667168 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b86a6042-df2b-41a8-a25c-ca87c49c7151 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "DELETE /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5" status: 204 len: 0 microversion: 2.1 time: 1.232548 Sep 27 10:07:14.668908 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1456/2679] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:13 2022] DELETE /compute/v2.1/images/2d3ce7d8-60a3-4b7b-a142-c7c6240632b5 => generated 0 bytes in 1252 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:14.696953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-457b7b5d-8844-4c1e-b695-2fe670748f73 req-ab60dbfb-acec-4aef-a755-9ae37d56ffbe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5", "name": "network-vif-plugged", "status": "completed", "tag": "205dffe7-0942-4b05-a466-8456d8d8713f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:14.714279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cd925202-616a-4d39-9f67-17f192af4fbb tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:14.733912 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd925202-616a-4d39-9f67-17f192af4fbb tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1525 microversion: 2.1 time: 0.653681 Sep 27 10:07:14.746262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-457b7b5d-8844-4c1e-b695-2fe670748f73 req-ab60dbfb-acec-4aef-a755-9ae37d56ffbe service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:14.748878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-457b7b5d-8844-4c1e-b695-2fe670748f73 req-ab60dbfb-acec-4aef-a755-9ae37d56ffbe service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:14.751226 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1224/2680] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:14 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1525 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:14.752435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-457b7b5d-8844-4c1e-b695-2fe670748f73 req-ab60dbfb-acec-4aef-a755-9ae37d56ffbe service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:14.766243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:14.767513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:14.773821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:14.789454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:14.792847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:14.793344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:14.796302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:14.798364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:14.799325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:14.804029 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-457b7b5d-8844-4c1e-b695-2fe670748f73 req-ab60dbfb-acec-4aef-a755-9ae37d56ffbe service nova] Creating event network-vif-plugged:205dffe7-0942-4b05-a466-8456d8d8713f for instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 on nch1-master-1 Sep 27 10:07:14.845696 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-457b7b5d-8844-4c1e-b695-2fe670748f73 req-ab60dbfb-acec-4aef-a755-9ae37d56ffbe service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.155276 Sep 27 10:07:14.852269 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1457/2681] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:14.866998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-116a0ac5-be9f-4c56-b52c-e6a2a559bf8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:14.875229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-116a0ac5-be9f-4c56-b52c-e6a2a559bf8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:14.916976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:14.926182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-116a0ac5-be9f-4c56-b52c-e6a2a559bf8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:14.927856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-116a0ac5-be9f-4c56-b52c-e6a2a559bf8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:14.929149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-116a0ac5-be9f-4c56-b52c-e6a2a559bf8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:14.934744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-116a0ac5-be9f-4c56-b52c-e6a2a559bf8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:14.936854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-116a0ac5-be9f-4c56-b52c-e6a2a559bf8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:14.940722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-116a0ac5-be9f-4c56-b52c-e6a2a559bf8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:15.383116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:15.398177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:15.403159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:15.419261 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a686ebbe-1c1c-42f0-8b93-08ed9322a4fe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.666394 Sep 27 10:07:15.420111 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1225/2682] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:14 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 668 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:15.434482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6017eb0a-2bfc-4810-a087-f1fc812f7c05 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] Calling method '.version_select at 0x7f7f629850d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:15.614903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=86 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:15.671932 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6017eb0a-2bfc-4810-a087-f1fc812f7c05 tempest-QuotasAdminTestV257-2258371 tempest-QuotasAdminTestV257-2258371-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/91e709a461b944f4a9118e979d016f7b?user_id=0f1ba9f0940a46b59b18d87049d9eea9" status: 200 len: 190 microversion: 2.57 time: 0.250464 Sep 27 10:07:15.690535 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1226/2683] 10.222.0.26 () {60 vars in 1364 bytes} [Tue Sep 27 10:07:15 2022] GET /compute/v2.1/os-quota-sets/91e709a461b944f4a9118e979d016f7b?user_id=0f1ba9f0940a46b59b18d87049d9eea9 => generated 190 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:07:15.695817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-503848a3-c4e7-466d-a66f-f9f44e92c5b9 req-ecbbc113-712d-4ea3-b5cf-1b54c4b57fb3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5b5bcf59-770f-40f1-83b0-0738578e921c", "name": "network-vif-deleted", "tag": "2acf9231-7703-4c8c-83e8-58ba465f8803"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:15.757827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-503848a3-c4e7-466d-a66f-f9f44e92c5b9 req-ecbbc113-712d-4ea3-b5cf-1b54c4b57fb3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:15.766549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-503848a3-c4e7-466d-a66f-f9f44e92c5b9 req-ecbbc113-712d-4ea3-b5cf-1b54c4b57fb3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:15.770054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-503848a3-c4e7-466d-a66f-f9f44e92c5b9 req-ecbbc113-712d-4ea3-b5cf-1b54c4b57fb3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:15.856894 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-503848a3-c4e7-466d-a66f-f9f44e92c5b9 req-ecbbc113-712d-4ea3-b5cf-1b54c4b57fb3 service nova] Creating event network-vif-deleted:2acf9231-7703-4c8c-83e8-58ba465f8803 for instance 5b5bcf59-770f-40f1-83b0-0738578e921c on nch2-master-1 Sep 27 10:07:15.868276 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-503848a3-c4e7-466d-a66f-f9f44e92c5b9 req-ecbbc113-712d-4ea3-b5cf-1b54c4b57fb3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.173385 Sep 27 10:07:15.870048 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1227/2684] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:15.890259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-01bbb23e-a944-4124-80d5-f4754ec7ed1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:15.893398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-01bbb23e-a944-4124-80d5-f4754ec7ed1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:15.919476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01bbb23e-a944-4124-80d5-f4754ec7ed1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:15.921236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01bbb23e-a944-4124-80d5-f4754ec7ed1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:15.925966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01bbb23e-a944-4124-80d5-f4754ec7ed1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:15.930141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01bbb23e-a944-4124-80d5-f4754ec7ed1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:15.930909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01bbb23e-a944-4124-80d5-f4754ec7ed1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:15.931495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01bbb23e-a944-4124-80d5-f4754ec7ed1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:16.132714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-116a0ac5-be9f-4c56-b52c-e6a2a559bf8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:16.159140 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-116a0ac5-be9f-4c56-b52c-e6a2a559bf8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1316 microversion: 2.1 time: 1.300783 Sep 27 10:07:16.178301 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1458/2685] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:14 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1316 bytes in 1322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:16.190697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8c368b95-ff65-437f-be12-ec26f1a83685 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:16.194189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8c368b95-ff65-437f-be12-ec26f1a83685 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:16.211512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c368b95-ff65-437f-be12-ec26f1a83685 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:16.216121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c368b95-ff65-437f-be12-ec26f1a83685 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:16.217072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c368b95-ff65-437f-be12-ec26f1a83685 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:16.218087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c368b95-ff65-437f-be12-ec26f1a83685 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:16.221212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c368b95-ff65-437f-be12-ec26f1a83685 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:16.221900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c368b95-ff65-437f-be12-ec26f1a83685 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:16.627302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-01bbb23e-a944-4124-80d5-f4754ec7ed1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:16.642104 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-01bbb23e-a944-4124-80d5-f4754ec7ed1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1315 microversion: 2.1 time: 0.766310 Sep 27 10:07:16.643345 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1228/2686] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:15 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1315 bytes in 772 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:16.654562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80791ed6-ea1e-411f-94d8-cb8000e8b93c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:16.655875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-80791ed6-ea1e-411f-94d8-cb8000e8b93c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:16.675796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80791ed6-ea1e-411f-94d8-cb8000e8b93c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:16.676689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80791ed6-ea1e-411f-94d8-cb8000e8b93c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:16.677142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80791ed6-ea1e-411f-94d8-cb8000e8b93c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:16.678070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80791ed6-ea1e-411f-94d8-cb8000e8b93c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:16.678675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80791ed6-ea1e-411f-94d8-cb8000e8b93c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:16.680258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80791ed6-ea1e-411f-94d8-cb8000e8b93c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:16.741809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-80791ed6-ea1e-411f-94d8-cb8000e8b93c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:07:16.938109 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-80791ed6-ea1e-411f-94d8-cb8000e8b93c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "DELETE /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 204 len: 0 microversion: 2.1 time: 0.293123 Sep 27 10:07:16.940801 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1229/2687] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:07:16 2022] DELETE /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 0 bytes in 297 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:16.958938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:16.961196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:16.975040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:16.978712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:16.979372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:16.981696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:16.982323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:16.982919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:17.085297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:17.085899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:17.086329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:17.086798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lazy-loading 'fault' on Instance uuid 0abee65c-58ee-4b9c-ac6c-65631519b216 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:07:17.289910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:17.307158 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-899cd9c9-a29f-4a39-abf9-f7e1d18a1185 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1313 microversion: 2.1 time: 0.363888 Sep 27 10:07:17.315933 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1230/2688] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:16 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1313 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:17.327597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:17.329188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:17.342824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:17.345402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:17.346164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:17.347339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:17.356205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:17.366127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:17.504280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:17.523150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:17.529822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:17.536438 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-41b2a551-2d57-4728-9c1c-a256bc726700 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.215611 Sep 27 10:07:17.537776 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1231/2689] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:17 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:17.582784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4f5a0199-f0c1-44af-a7ea-3083d089dee4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:17.584478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4f5a0199-f0c1-44af-a7ea-3083d089dee4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:17.596239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f5a0199-f0c1-44af-a7ea-3083d089dee4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:17.597337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f5a0199-f0c1-44af-a7ea-3083d089dee4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:17.597877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f5a0199-f0c1-44af-a7ea-3083d089dee4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:17.598760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f5a0199-f0c1-44af-a7ea-3083d089dee4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:17.599514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f5a0199-f0c1-44af-a7ea-3083d089dee4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:17.600133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f5a0199-f0c1-44af-a7ea-3083d089dee4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:17.644097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:17.655253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c368b95-ff65-437f-be12-ec26f1a83685 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:17.669383 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8c368b95-ff65-437f-be12-ec26f1a83685 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1646 microversion: 2.1 time: 1.489362 Sep 27 10:07:17.687099 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1459/2690] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:16 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1646 bytes in 1508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:17.700063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f5cc90ea-5a28-4841-b055-bdb3c89732c2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:17.701281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f5cc90ea-5a28-4841-b055-bdb3c89732c2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:17.711873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5cc90ea-5a28-4841-b055-bdb3c89732c2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:17.712803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5cc90ea-5a28-4841-b055-bdb3c89732c2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:17.713536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5cc90ea-5a28-4841-b055-bdb3c89732c2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:17.714433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5cc90ea-5a28-4841-b055-bdb3c89732c2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:17.715747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5cc90ea-5a28-4841-b055-bdb3c89732c2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:17.716336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5cc90ea-5a28-4841-b055-bdb3c89732c2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:17.831687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4f5a0199-f0c1-44af-a7ea-3083d089dee4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:17.845072 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4f5a0199-f0c1-44af-a7ea-3083d089dee4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1471 microversion: 2.1 time: 0.305714 Sep 27 10:07:17.846052 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1232/2691] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:17 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1471 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:17.863579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7bfa5d9b-6c45-41f2-affe-c16ec0ef5769 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:17.865597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7bfa5d9b-6c45-41f2-affe-c16ec0ef5769 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:17.924248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfa5d9b-6c45-41f2-affe-c16ec0ef5769 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:17.931214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfa5d9b-6c45-41f2-affe-c16ec0ef5769 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:17.931902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfa5d9b-6c45-41f2-affe-c16ec0ef5769 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:17.932717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfa5d9b-6c45-41f2-affe-c16ec0ef5769 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:17.937082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfa5d9b-6c45-41f2-affe-c16ec0ef5769 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:17.937674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfa5d9b-6c45-41f2-affe-c16ec0ef5769 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:17.939861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f5cc90ea-5a28-4841-b055-bdb3c89732c2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:17.951638 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f5cc90ea-5a28-4841-b055-bdb3c89732c2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1316 microversion: 2.1 time: 0.259044 Sep 27 10:07:17.976114 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1460/2692] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:17 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1316 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:18.037475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-b4d12067-6984-4452-a5fa-f9f946d07ff8 req-0ba22b0c-f45d-4188-a5d8-83f735927ca8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "918832ca-5183-4a17-b0ed-01fc0c661f3e", "tag": "6266abbd-ab22-410b-b499-c771bc142abf"}, {"server_uuid": "0abee65c-58ee-4b9c-ac6c-65631519b216", "name": "network-vif-deleted", "tag": "447ad707-a60a-4017-9b0d-e87acd64bee0"}, {"server_uuid": "ac5c4cdc-90a8-48c7-84f3-0053b32834f3", "name": "network-vif-deleted", "tag": "4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:18.052463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b4d12067-6984-4452-a5fa-f9f946d07ff8 req-0ba22b0c-f45d-4188-a5d8-83f735927ca8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:18.053011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b4d12067-6984-4452-a5fa-f9f946d07ff8 req-0ba22b0c-f45d-4188-a5d8-83f735927ca8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:18.053493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b4d12067-6984-4452-a5fa-f9f946d07ff8 req-0ba22b0c-f45d-4188-a5d8-83f735927ca8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:18.073586 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b4d12067-6984-4452-a5fa-f9f946d07ff8 req-0ba22b0c-f45d-4188-a5d8-83f735927ca8 service nova] Creating event network-changed:6266abbd-ab22-410b-b499-c771bc142abf for instance 918832ca-5183-4a17-b0ed-01fc0c661f3e on nch2-master-1 Sep 27 10:07:18.074100 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b4d12067-6984-4452-a5fa-f9f946d07ff8 req-0ba22b0c-f45d-4188-a5d8-83f735927ca8 service nova] Creating event network-vif-deleted:447ad707-a60a-4017-9b0d-e87acd64bee0 for instance 0abee65c-58ee-4b9c-ac6c-65631519b216 on nch1-master-1 Sep 27 10:07:18.074789 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b4d12067-6984-4452-a5fa-f9f946d07ff8 req-0ba22b0c-f45d-4188-a5d8-83f735927ca8 service nova] Creating event network-vif-deleted:4e1f7ba1-2665-48a9-b3e1-7fd09c8a9801 for instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 on nch1-master-1 Sep 27 10:07:18.099168 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-b4d12067-6984-4452-a5fa-f9f946d07ff8 req-0ba22b0c-f45d-4188-a5d8-83f735927ca8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 521 microversion: 2.1 time: 0.066097 Sep 27 10:07:18.100099 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1461/2693] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:18 2022] POST /compute/v2.1/os-server-external-events => generated 521 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:07:18.360424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:18.363010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:18.372836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:18.374527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:18.375269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:18.376515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:18.377418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:18.378205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:18.424905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7bfa5d9b-6c45-41f2-affe-c16ec0ef5769 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:18.456517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:18.458278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:18.458554 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7bfa5d9b-6c45-41f2-affe-c16ec0ef5769 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1315 microversion: 2.1 time: 0.610477 Sep 27 10:07:18.459497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:18.460852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lazy-loading 'fault' on Instance uuid 0abee65c-58ee-4b9c-ac6c-65631519b216 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:07:18.468675 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1233/2694] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:17 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1315 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:18.549478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:18.579346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:18.580488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:18.582376 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b01fbad2-f58a-451e-ba54-78522874b9c6 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 200 len: 1313 microversion: 2.1 time: 0.228699 Sep 27 10:07:18.589032 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1462/2695] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:18 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 1313 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:18.599098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:18.616808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:18.617573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:18.618870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:18.619712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:18.620472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:18.703390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:18.703785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:18.719544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:18.721531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:18.722473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:18.723758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:18.725191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:18.726329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:18.783074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:18.801911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:18.803566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:18.805360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:18.807980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:18.808688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:18.809340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lazy-loading 'fault' on Instance uuid ac5c4cdc-90a8-48c7-84f3-0053b32834f3 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:07:18.847593 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7d7bb634-06f5-4e22-a92a-c6a909aeac28 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.279543 Sep 27 10:07:18.854121 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1234/2696] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:18 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:18.878530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-131955ae-7b2f-4d77-a6de-04feae7fa4af tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:18.879157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-131955ae-7b2f-4d77-a6de-04feae7fa4af tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:18.893989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131955ae-7b2f-4d77-a6de-04feae7fa4af tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:18.894781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131955ae-7b2f-4d77-a6de-04feae7fa4af tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:18.895514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131955ae-7b2f-4d77-a6de-04feae7fa4af tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:18.896429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131955ae-7b2f-4d77-a6de-04feae7fa4af tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:18.897099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131955ae-7b2f-4d77-a6de-04feae7fa4af tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:18.899505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131955ae-7b2f-4d77-a6de-04feae7fa4af tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:18.910200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:18.921932 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a2246d9e-331a-45f1-853d-05aa79373393 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1306 microversion: 2.1 time: 0.230826 Sep 27 10:07:18.937909 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1463/2697] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:18 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1306 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:18.988113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-978fb49b-e188-4208-8d93-9aea9972ea0c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:18.991379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-978fb49b-e188-4208-8d93-9aea9972ea0c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:19.002980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978fb49b-e188-4208-8d93-9aea9972ea0c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.003868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978fb49b-e188-4208-8d93-9aea9972ea0c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.004610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978fb49b-e188-4208-8d93-9aea9972ea0c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:19.005951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978fb49b-e188-4208-8d93-9aea9972ea0c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.006705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978fb49b-e188-4208-8d93-9aea9972ea0c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.007315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978fb49b-e188-4208-8d93-9aea9972ea0c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:19.383114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-978fb49b-e188-4208-8d93-9aea9972ea0c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:19.396071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-131955ae-7b2f-4d77-a6de-04feae7fa4af tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:19.408149 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-131955ae-7b2f-4d77-a6de-04feae7fa4af tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1471 microversion: 2.1 time: 0.532778 Sep 27 10:07:19.410800 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1235/2698] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:18 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1471 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:07:19.439376 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-978fb49b-e188-4208-8d93-9aea9972ea0c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1506 microversion: 2.1 time: 0.457347 Sep 27 10:07:19.458331 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1464/2699] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:18 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1506 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:19.470711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25cd4c08-1dea-49b7-ae8a-7c1307ed25d9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:19.471463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-25cd4c08-1dea-49b7-ae8a-7c1307ed25d9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:19.480288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25cd4c08-1dea-49b7-ae8a-7c1307ed25d9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.481000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25cd4c08-1dea-49b7-ae8a-7c1307ed25d9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.481875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25cd4c08-1dea-49b7-ae8a-7c1307ed25d9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:19.482786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25cd4c08-1dea-49b7-ae8a-7c1307ed25d9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.483495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25cd4c08-1dea-49b7-ae8a-7c1307ed25d9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.484135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25cd4c08-1dea-49b7-ae8a-7c1307ed25d9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:19.490395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-692d5f19-336f-4b69-8092-f4dd4c417972 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:19.491020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-692d5f19-336f-4b69-8092-f4dd4c417972 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:19.511286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-692d5f19-336f-4b69-8092-f4dd4c417972 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.512499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-692d5f19-336f-4b69-8092-f4dd4c417972 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.513686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-692d5f19-336f-4b69-8092-f4dd4c417972 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:19.515083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-692d5f19-336f-4b69-8092-f4dd4c417972 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.516047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-692d5f19-336f-4b69-8092-f4dd4c417972 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.516958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-692d5f19-336f-4b69-8092-f4dd4c417972 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:19.518488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-25cd4c08-1dea-49b7-ae8a-7c1307ed25d9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:07:19.660229 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-25cd4c08-1dea-49b7-ae8a-7c1307ed25d9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "DELETE /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 204 len: 0 microversion: 2.1 time: 0.191658 Sep 27 10:07:19.664055 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1236/2700] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:07:19 2022] DELETE /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:07:19.668251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:19.669010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:19.676128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.676711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.677081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:19.677811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.678273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.678735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:19.695275 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] HTTP exception thrown: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 HTTP/1.1 Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '0abee65c-58ee-4b9c-ac6c-65631519b216'} Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:19.697866 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:19.703511 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:19.703511 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:19.703511 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:19.703511 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:19.703511 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:19.703511 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:19.703511 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:07:19.703511 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:19.703511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 404 to user: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:19.703511 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9ca5bf5a-04a8-475b-880f-7aac8590ac7a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 404 len: 111 microversion: 2.1 time: 0.037018 Sep 27 10:07:19.703511 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1237/2701] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:19 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:19.705127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ffd3416c-8d43-414b-9edd-16fe8473f610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:19.705942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ffd3416c-8d43-414b-9edd-16fe8473f610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:19.712724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffd3416c-8d43-414b-9edd-16fe8473f610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.713208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffd3416c-8d43-414b-9edd-16fe8473f610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.713651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffd3416c-8d43-414b-9edd-16fe8473f610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:19.714388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffd3416c-8d43-414b-9edd-16fe8473f610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.714846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffd3416c-8d43-414b-9edd-16fe8473f610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.715419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffd3416c-8d43-414b-9edd-16fe8473f610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:19.720497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-692d5f19-336f-4b69-8092-f4dd4c417972 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:19.729673 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-692d5f19-336f-4b69-8092-f4dd4c417972 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1315 microversion: 2.1 time: 0.243847 Sep 27 10:07:19.730262 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1465/2702] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:19 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1315 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:19.749708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6114a5a8-0a48-4d31-bcc7-3c82434d9a33 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Action: 'create', calling method: .version_select at 0x7f7f628e2ee0>, body: {"keypair": {"name": "tempest-keypair-1310573978", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDNpEYa23UFTSru6m9+Lev4YjytnGvQRmWYTCQwEXLOQogJFrLs/SQaZhMDZuTdqrlHeHpWW/o2+G0cvhuELjW0/77wfOX0LM7xLVQP94AgbagorCMHNRLMZjTLwA8AhKg=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:19.778815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6114a5a8-0a48-4d31-bcc7-3c82434d9a33 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Getting quotas for project f8130333e2c04a3f960dc6465e07cc46. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:07:19.786769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6114a5a8-0a48-4d31-bcc7-3c82434d9a33 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Getting quotas for user 68d9297ca0884c2fa7f5283f722451ba and project f8130333e2c04a3f960dc6465e07cc46. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:07:19.823397 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6114a5a8-0a48-4d31-bcc7-3c82434d9a33 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.079328 Sep 27 10:07:19.830890 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1466/2703] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:07:19 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:19.890451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:19.890703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:19.898412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.899181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.899982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:19.900762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:19.901496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:19.902192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:20.003851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:20.020643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:20.022725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:20.030685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ffd3416c-8d43-414b-9edd-16fe8473f610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:20.043020 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ffd3416c-8d43-414b-9edd-16fe8473f610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1512 microversion: 2.1 time: 0.341655 Sep 27 10:07:20.044614 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1238/2704] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:19 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1512 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:20.045194 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-afe929f8-0ae5-40da-babe-61640c534082 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.158368 Sep 27 10:07:20.048788 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1467/2705] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:19 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:07:20.062877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:20.063503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:20.079895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:20.085007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:20.085470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:20.086301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:20.089008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:20.090588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:20.153701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:20.154104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:20.154786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:20.155563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lazy-loading 'fault' on Instance uuid ac5c4cdc-90a8-48c7-84f3-0053b32834f3 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:07:20.186060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-4b157d8b-97d6-49fa-bf37-545aba6b50d2 req-cda8b5a1-e8b0-4984-b834-86f92958bb4e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "55ead6af-a0d9-4d6f-b742-4a3fd4a418ad", "name": "network-vif-deleted", "tag": "a33d85f6-8d2f-4192-8ab2-06ef5d149185"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:20.203916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4b157d8b-97d6-49fa-bf37-545aba6b50d2 req-cda8b5a1-e8b0-4984-b834-86f92958bb4e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:20.204452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4b157d8b-97d6-49fa-bf37-545aba6b50d2 req-cda8b5a1-e8b0-4984-b834-86f92958bb4e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:20.205106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4b157d8b-97d6-49fa-bf37-545aba6b50d2 req-cda8b5a1-e8b0-4984-b834-86f92958bb4e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:20.218071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:20.219866 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-4b157d8b-97d6-49fa-bf37-545aba6b50d2 req-cda8b5a1-e8b0-4984-b834-86f92958bb4e service nova] Creating event network-vif-deleted:a33d85f6-8d2f-4192-8ab2-06ef5d149185 for instance 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad on nch2-master-1 Sep 27 10:07:20.225952 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-4b157d8b-97d6-49fa-bf37-545aba6b50d2 req-cda8b5a1-e8b0-4984-b834-86f92958bb4e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044373 Sep 27 10:07:20.227146 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1468/2706] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:07:20.228248 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-31480a2b-a0a3-4f91-94f4-00c45bd6c707 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1306 microversion: 2.1 time: 0.177624 Sep 27 10:07:20.228896 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1239/2707] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:20 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1306 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:20.439760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dfb4b074-2eab-4069-9e8f-9bdbe2aa200c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:20.440471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dfb4b074-2eab-4069-9e8f-9bdbe2aa200c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:20.447407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfb4b074-2eab-4069-9e8f-9bdbe2aa200c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:20.448037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfb4b074-2eab-4069-9e8f-9bdbe2aa200c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:20.448474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfb4b074-2eab-4069-9e8f-9bdbe2aa200c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:20.449460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfb4b074-2eab-4069-9e8f-9bdbe2aa200c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:20.449759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfb4b074-2eab-4069-9e8f-9bdbe2aa200c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:20.450190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dfb4b074-2eab-4069-9e8f-9bdbe2aa200c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:20.474717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:20.545213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dfb4b074-2eab-4069-9e8f-9bdbe2aa200c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:20.556481 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dfb4b074-2eab-4069-9e8f-9bdbe2aa200c tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1429 microversion: 2.1 time: 0.121618 Sep 27 10:07:20.557772 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1469/2708] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:20 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1429 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:20.755230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b44f2bd6-e4e5-47dd-bdcc-7a632fdc754e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:20.756583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b44f2bd6-e4e5-47dd-bdcc-7a632fdc754e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:20.766316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b44f2bd6-e4e5-47dd-bdcc-7a632fdc754e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:20.767022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b44f2bd6-e4e5-47dd-bdcc-7a632fdc754e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:20.769056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b44f2bd6-e4e5-47dd-bdcc-7a632fdc754e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:20.770181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b44f2bd6-e4e5-47dd-bdcc-7a632fdc754e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:20.770808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b44f2bd6-e4e5-47dd-bdcc-7a632fdc754e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:20.771419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b44f2bd6-e4e5-47dd-bdcc-7a632fdc754e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:20.996094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b44f2bd6-e4e5-47dd-bdcc-7a632fdc754e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:21.016438 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b44f2bd6-e4e5-47dd-bdcc-7a632fdc754e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1315 microversion: 2.1 time: 0.266104 Sep 27 10:07:21.017052 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1240/2709] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:20 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1315 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:21.073519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f628f46-9c17-486b-a156-2cd3e6cadaa6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:21.074792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8f628f46-9c17-486b-a156-2cd3e6cadaa6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:21.078471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:21.079343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:21.086669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f628f46-9c17-486b-a156-2cd3e6cadaa6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:21.088429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f628f46-9c17-486b-a156-2cd3e6cadaa6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:21.088868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:21.089754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:21.090598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f628f46-9c17-486b-a156-2cd3e6cadaa6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:21.091182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:21.091359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f628f46-9c17-486b-a156-2cd3e6cadaa6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:21.092005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f628f46-9c17-486b-a156-2cd3e6cadaa6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:21.092526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:21.093329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:21.094065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:21.094468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f628f46-9c17-486b-a156-2cd3e6cadaa6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:21.218936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:21.231039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:21.232508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:21.249261 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9718fd1a-c08a-4884-bc6e-8c2b2f3abe4f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.176983 Sep 27 10:07:21.249854 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1241/2710] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:21 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:21.262407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:21.265740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:21.279490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:21.280019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:21.280436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:21.282488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:21.282961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:21.283395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:21.371567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:21.372088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:21.372543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:21.373053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lazy-loading 'fault' on Instance uuid ac5c4cdc-90a8-48c7-84f3-0053b32834f3 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:07:21.587909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:21.598199 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2c6a3103-da7d-497b-831c-4f51135e7dd4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 200 len: 1306 microversion: 2.1 time: 0.343575 Sep 27 10:07:21.617972 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1242/2711] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:21 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 1306 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:21.626193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f53af48f-cfad-4eb9-9e97-40e6406acf7b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:21.627080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f53af48f-cfad-4eb9-9e97-40e6406acf7b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:21.638845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53af48f-cfad-4eb9-9e97-40e6406acf7b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:21.639570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53af48f-cfad-4eb9-9e97-40e6406acf7b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:21.640196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53af48f-cfad-4eb9-9e97-40e6406acf7b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:21.641147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53af48f-cfad-4eb9-9e97-40e6406acf7b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:21.641771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53af48f-cfad-4eb9-9e97-40e6406acf7b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:21.643285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f53af48f-cfad-4eb9-9e97-40e6406acf7b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:21.889365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f53af48f-cfad-4eb9-9e97-40e6406acf7b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:21.903384 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f53af48f-cfad-4eb9-9e97-40e6406acf7b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 200 len: 1429 microversion: 2.1 time: 0.283634 Sep 27 10:07:21.904030 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1243/2712] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:21 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 1429 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:21.939508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8f628f46-9c17-486b-a156-2cd3e6cadaa6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:21.949154 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8f628f46-9c17-486b-a156-2cd3e6cadaa6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1512 microversion: 2.1 time: 0.884726 Sep 27 10:07:21.949746 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1470/2713] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:21 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1512 bytes in 886 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:22.039376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-59bc505c-42a0-4f4e-b227-685a15274cf3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:22.040008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-59bc505c-42a0-4f4e-b227-685a15274cf3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:22.055352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59bc505c-42a0-4f4e-b227-685a15274cf3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:22.055352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59bc505c-42a0-4f4e-b227-685a15274cf3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:22.055352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59bc505c-42a0-4f4e-b227-685a15274cf3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:22.059207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59bc505c-42a0-4f4e-b227-685a15274cf3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:22.059713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59bc505c-42a0-4f4e-b227-685a15274cf3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:22.065393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59bc505c-42a0-4f4e-b227-685a15274cf3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:22.283178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:22.283847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:22.308832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:22.310508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:22.313549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:22.319180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:22.319812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:22.321527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:22.503502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:22.517352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:22.522138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:22.537819 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d77f276-9ed4-4d03-a91c-444c689f8bca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.259038 Sep 27 10:07:22.541935 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1471/2714] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:22 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:22.640905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:22.641990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:22.654224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:22.659233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:22.659890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:22.660871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:22.662289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:22.665325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:22.707028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:22.711179 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 HTTP/1.1 Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'ac5c4cdc-90a8-48c7-84f3-0053b32834f3'} Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:22.718870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:22.723633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:22.723633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:22.723633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:22.723633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:22.723633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:22.723633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:22.723633 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:22.729882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:22.731128 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-07a63a02-27f9-49eb-b9df-0a13d1b870b1 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 404 len: 111 microversion: 2.1 time: 0.095392 Sep 27 10:07:22.736881 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1472/2715] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:22 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 111 bytes in 101 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:07:23.077091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminTestJSON-server-953324831", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:23.183875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:07:23.297724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-59bc505c-42a0-4f4e-b227-685a15274cf3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:23.311774 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-59bc505c-42a0-4f4e-b227-685a15274cf3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1315 microversion: 2.1 time: 1.275919 Sep 27 10:07:23.314506 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1244/2716] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:22 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1315 bytes in 1279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:23.323133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-48fd4025-1ea0-4ff2-a856-d814eea78c80 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f629851f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:23.894503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:07:23.894990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:07:23.895388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:07:23.895804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:07:23.896209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:07:23.916894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:23.917618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:23.918212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:23.919147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:23.921244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:23.923871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:23.963066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=69 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:23.963394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=15 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:23.970680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Getting quotas for project 52efcec2c8a14640a8c3df1dc576bbf1. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:07:23.988302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Getting quotas for user b4e0f20e52074142a3b525bafbb30c97 and project 52efcec2c8a14640a8c3df1dc576bbf1. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:07:24.003379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:07:24.006094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:07:24.073854 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7595676b-f6db-4fc8-a7b8-1895adce67d6 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.332879 Sep 27 10:07:24.086210 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1473/2717] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 1348 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:07:24.092644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:24.093334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:24.103981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.105710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.106337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.107106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.108128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.108865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.130978 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] HTTP exception thrown: Instance 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad could not be found. Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Request method failure captured: Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad HTTP/1.1 Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsqbF49kTqg2OfPo0tbTuW_gqKOl2lw_jczFixJFmB1obsvcOkG_4u8HNqlSejQ_ffVI6ktTjnJ-Fcr0PMzqRmeXTIfZxjrlCjFOiOSvKBOGKrOZCgWNxKLTQLQ1nVeG6Y9l3m4nmiBpwq2AAG1aJ3o0OwY_VhJ249dqL-HF-F6wC4o Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b932820432be493b8a85520f58f49606 Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ListImageFiltersTestJSON-361314172 Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b932820432be493b8a85520f58f49606"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b932820432be493b8a85520f58f49606"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b932820432be493b8a85520f58f49606"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b932820432be493b8a85520f58f49606"}]}] Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ListImageFiltersTestJSON-361314172 Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b932820432be493b8a85520f58f49606 Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ListImageFiltersTestJSON-361314172 Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ListImageFiltersTestJSON-361314172-project Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 032de99964be4424ba019adf2eedf680 Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ListImageFiltersTestJSON-361314172-project Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad could not be found. Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '55ead6af-a0d9-4d6f-b742-4a3fd4a418ad'} Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad could not be found. Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad could not be found. Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:24.134349 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:24.141164 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:24.141164 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:24.141164 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:24.141164 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:24.141164 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:24.141164 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad could not be found. Sep 27 10:07:24.141164 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:24.143836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Returning 404 to user: Instance 55ead6af-a0d9-4d6f-b742-4a3fd4a418ad could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:24.145931 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-46ba980b-9f70-470a-8698-108e6a7bf586 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad" status: 404 len: 111 microversion: 2.1 time: 0.057815 Sep 27 10:07:24.147179 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1474/2718] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:24 2022] GET /compute/v2.1/servers/55ead6af-a0d9-4d6f-b742-4a3fd4a418ad => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:24.170557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:24.173435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:24.238665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.243764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.245323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.250483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.251114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.257051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.322937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.323623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.324190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.325188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lazy-loading 'fault' on Instance uuid 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:07:24.407246 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-48fd4025-1ea0-4ff2-a856-d814eea78c80 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/6244f0a6-16ad-4669-ba27-a50476f4d398" status: 202 len: 0 microversion: 2.1 time: 1.091282 Sep 27 10:07:24.411871 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1245/2719] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:23 2022] DELETE /compute/v2.1/os-security-groups/6244f0a6-16ad-4669-ba27-a50476f4d398 => generated 0 bytes in 1096 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:07:24.417620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-364c2ced-af6b-40d8-a81e-5abe3b88381b req-2de05df0-0a50-4827-9cb6-724764f9b137 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5", "name": "network-vif-deleted", "tag": "205dffe7-0942-4b05-a466-8456d8d8713f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:24.423836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:24.443961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-364c2ced-af6b-40d8-a81e-5abe3b88381b req-2de05df0-0a50-4827-9cb6-724764f9b137 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.444264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-364c2ced-af6b-40d8-a81e-5abe3b88381b req-2de05df0-0a50-4827-9cb6-724764f9b137 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.444735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-364c2ced-af6b-40d8-a81e-5abe3b88381b req-2de05df0-0a50-4827-9cb6-724764f9b137 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.447894 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9e4a22f9-5440-488c-b0f8-42b521e529c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 200 len: 1306 microversion: 2.1 time: 0.289722 Sep 27 10:07:24.451717 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1475/2720] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:24 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 1306 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:24.467022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:24.473676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:24.475851 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-364c2ced-af6b-40d8-a81e-5abe3b88381b req-2de05df0-0a50-4827-9cb6-724764f9b137 service nova] Creating event network-vif-deleted:205dffe7-0942-4b05-a466-8456d8d8713f for instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 on nch1-master-1 Sep 27 10:07:24.483556 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-364c2ced-af6b-40d8-a81e-5abe3b88381b req-2de05df0-0a50-4827-9cb6-724764f9b137 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070041 Sep 27 10:07:24.489956 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1246/2721] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:24.497805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-02912411-c530-4efe-a9dd-7c5548239aaf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:24.499634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-02912411-c530-4efe-a9dd-7c5548239aaf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:24.507745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02912411-c530-4efe-a9dd-7c5548239aaf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.509309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02912411-c530-4efe-a9dd-7c5548239aaf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.510131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02912411-c530-4efe-a9dd-7c5548239aaf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.510588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.511127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.511580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.511816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02912411-c530-4efe-a9dd-7c5548239aaf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.512768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02912411-c530-4efe-a9dd-7c5548239aaf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.513638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.513993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02912411-c530-4efe-a9dd-7c5548239aaf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.514969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.515727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.561208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-02912411-c530-4efe-a9dd-7c5548239aaf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:07:24.668913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:24.680534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:24.680890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:24.682767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:24.689396 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-17653ca5-9568-4bfb-9375-9f3880073602 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1531 microversion: 2.91 time: 0.233933 Sep 27 10:07:24.690745 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1476/2722] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:24 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1531 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:24.697472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminTestJSON-server-1152528096", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:24.759716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:07:24.766377 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-02912411-c530-4efe-a9dd-7c5548239aaf tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "DELETE /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d" status: 204 len: 0 microversion: 2.1 time: 0.273315 Sep 27 10:07:24.767184 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1247/2723] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:07:24 2022] DELETE /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:24.774091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4918ae0b-fd06-4cb0-a538-d0d094466011 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:24.775173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4918ae0b-fd06-4cb0-a538-d0d094466011 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:24.781746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4918ae0b-fd06-4cb0-a538-d0d094466011 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.782376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4918ae0b-fd06-4cb0-a538-d0d094466011 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.782912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4918ae0b-fd06-4cb0-a538-d0d094466011 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.784002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4918ae0b-fd06-4cb0-a538-d0d094466011 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:24.785450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4918ae0b-fd06-4cb0-a538-d0d094466011 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:24.785957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4918ae0b-fd06-4cb0-a538-d0d094466011 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:24.974438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4918ae0b-fd06-4cb0-a538-d0d094466011 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:24.984856 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4918ae0b-fd06-4cb0-a538-d0d094466011 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1315 microversion: 2.1 time: 0.216083 Sep 27 10:07:24.986064 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1248/2724] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:24 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1315 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:24.992853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0dc2716-d45c-4d54-85fa-992e4ca393c5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62b08280>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:25.054177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:07:25.055473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:07:25.056318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:07:25.057624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:07:25.058146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:07:25.069146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:25.070012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:25.070682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:25.071671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:25.072619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:25.073304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:25.096271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Getting quotas for project 52efcec2c8a14640a8c3df1dc576bbf1. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:07:25.104285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Getting quotas for user b4e0f20e52074142a3b525bafbb30c97 and project 52efcec2c8a14640a8c3df1dc576bbf1. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:07:25.123349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:07:25.127465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:07:25.185362 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4cebcc9-a02e-47a0-b6e2-7e212f10e4b0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.492955 Sep 27 10:07:25.186616 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1477/2725] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:24 2022] POST /compute/v2.1/servers => generated 380 bytes in 495 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:07:25.193333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerNegativeTestJSON-server-588049221", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:25.272500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] validate_networks() for [('3757a0b4-fb97-4218-a71c-9743174aeb68', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:07:25.398869 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0dc2716-d45c-4d54-85fa-992e4ca393c5 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/24e4adb8-847d-4647-a320-2ac81698f21f" status: 202 len: 0 microversion: 2.1 time: 0.411159 Sep 27 10:07:25.400152 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1249/2726] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:24 2022] DELETE /compute/v2.1/os-security-groups/24e4adb8-847d-4647-a320-2ac81698f21f => generated 0 bytes in 413 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:07:25.405968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestMultiNic-server-1270413010", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "b54a8474-eeb5-4363-b0de-487e478d0b0a"}, {"uuid": "44a2a214-cff0-4e04-807a-089fae0587fd"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:25.508764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] validate_networks() for [('b54a8474-eeb5-4363-b0de-487e478d0b0a', None, None, None, None, None), ('44a2a214-cff0-4e04-807a-089fae0587fd', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:07:25.652279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:25.986732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:07:25.987162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:07:25.987654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:07:25.988046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:07:25.988461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:07:25.998873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:25.999367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:25.999800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.000792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.001304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.001791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.021584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Getting quotas for project f8130333e2c04a3f960dc6465e07cc46. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:07:26.037919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Getting quotas for user 68d9297ca0884c2fa7f5283f722451ba and project f8130333e2c04a3f960dc6465e07cc46. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:07:26.051056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:26.056015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:07:26.059052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:07:26.119843 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f6dd87f1-4ecd-4962-a89f-857d5b05a01b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 0.931265 Sep 27 10:07:26.126474 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1478/2727] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:25 2022] POST /compute/v2.1/servers => generated 383 bytes in 933 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:07:26.126474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d74fbe5a-eaf2-462c-9395-062208fb9d7d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:26.127026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d74fbe5a-eaf2-462c-9395-062208fb9d7d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:26.135542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74fbe5a-eaf2-462c-9395-062208fb9d7d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.136115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74fbe5a-eaf2-462c-9395-062208fb9d7d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.136625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74fbe5a-eaf2-462c-9395-062208fb9d7d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.137448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74fbe5a-eaf2-462c-9395-062208fb9d7d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.137954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74fbe5a-eaf2-462c-9395-062208fb9d7d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.138376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d74fbe5a-eaf2-462c-9395-062208fb9d7d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.318632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-d74fbe5a-eaf2-462c-9395-062208fb9d7d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lazy-loading 'services' on Instance uuid 42709739-135c-4adb-aa6c-4fc216a5d618 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:07:26.405025 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d74fbe5a-eaf2-462c-9395-062208fb9d7d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1936 microversion: 2.91 time: 0.280434 Sep 27 10:07:26.405681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1479/2728] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:26 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1936 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:26.435969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a20673ac-56bc-4c36-a191-e079a8227f4e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:26.437095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a20673ac-56bc-4c36-a191-e079a8227f4e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:26.453802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a20673ac-56bc-4c36-a191-e079a8227f4e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.454571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a20673ac-56bc-4c36-a191-e079a8227f4e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.455174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a20673ac-56bc-4c36-a191-e079a8227f4e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.457342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a20673ac-56bc-4c36-a191-e079a8227f4e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.462171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a20673ac-56bc-4c36-a191-e079a8227f4e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.463030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a20673ac-56bc-4c36-a191-e079a8227f4e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.534045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:07:26.534423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:07:26.534762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:07:26.535363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:07:26.535811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:07:26.547149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.547650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.548091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.548882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.549341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.549924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.575020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Getting quotas for project fedcdffc12b94b489ce433d5caa5e4be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:07:26.583985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Getting quotas for user 430de2723f234fac8f09068195d8ce2c and project fedcdffc12b94b489ce433d5caa5e4be. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:07:26.624274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:07:26.629844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:07:26.693521 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5ae146c8-0dbe-4ae5-909b-2d364fbe15f3 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.291982 Sep 27 10:07:26.699751 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1250/2729] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:25 2022] POST /compute/v2.1/servers => generated 380 bytes in 1299 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:07:26.711442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-78e53afb-3200-48a6-87bc-42fc1390820e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:26.712177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-78e53afb-3200-48a6-87bc-42fc1390820e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:26.719976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78e53afb-3200-48a6-87bc-42fc1390820e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.720507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78e53afb-3200-48a6-87bc-42fc1390820e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.721157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78e53afb-3200-48a6-87bc-42fc1390820e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.721893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78e53afb-3200-48a6-87bc-42fc1390820e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.722544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78e53afb-3200-48a6-87bc-42fc1390820e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.722910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78e53afb-3200-48a6-87bc-42fc1390820e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.807703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a20673ac-56bc-4c36-a191-e079a8227f4e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:26.808006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-78e53afb-3200-48a6-87bc-42fc1390820e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:26.816692 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a20673ac-56bc-4c36-a191-e079a8227f4e tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d" status: 200 len: 1473 microversion: 2.1 time: 0.386696 Sep 27 10:07:26.817355 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-78e53afb-3200-48a6-87bc-42fc1390820e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1266 microversion: 2.1 time: 0.116379 Sep 27 10:07:26.818693 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1251/2730] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:26 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1266 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:26.824367 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1480/2731] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:26 2022] GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d => generated 1473 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:26.832151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:26.832827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:26.838029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f7f189d1-4a11-47bc-9ca4-600dd4caf2bc tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'create', calling method: .version_select at 0x7f7f62985670>, body: {"security_group": {"name": "tempest-SecurityGroupsTestJSON-securitygroup-1881249096", "description": "tempest-description-444404803"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:26.862525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.864597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.865223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.866403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.867008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.867752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.888358 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 HTTP/1.1 Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5'} Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:26.890413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:26.894546 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:26.894546 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:26.894546 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:26.894546 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:26.894546 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:26.894546 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:07:26.894546 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:26.894546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:26.894546 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-be5d504a-b119-43f1-bda8-88bdae9db460 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 404 len: 111 microversion: 2.1 time: 0.067332 Sep 27 10:07:26.896820 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1481/2732] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:26 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:26.910859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6257ea63-c72b-4da8-bbc8-6fd9c2246ba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:26.912238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6257ea63-c72b-4da8-bbc8-6fd9c2246ba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:26.921171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6257ea63-c72b-4da8-bbc8-6fd9c2246ba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.923429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6257ea63-c72b-4da8-bbc8-6fd9c2246ba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.924298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6257ea63-c72b-4da8-bbc8-6fd9c2246ba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:26.927186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6257ea63-c72b-4da8-bbc8-6fd9c2246ba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:26.928446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6257ea63-c72b-4da8-bbc8-6fd9c2246ba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:26.930086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6257ea63-c72b-4da8-bbc8-6fd9c2246ba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:27.395382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6257ea63-c72b-4da8-bbc8-6fd9c2246ba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:27.406849 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6257ea63-c72b-4da8-bbc8-6fd9c2246ba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1315 microversion: 2.1 time: 0.508477 Sep 27 10:07:27.409948 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1482/2733] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:26 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1315 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:27.428542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0316b3d9-642b-4cea-9c81-00f5e2a8777f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:27.429279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0316b3d9-642b-4cea-9c81-00f5e2a8777f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:27.442194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0316b3d9-642b-4cea-9c81-00f5e2a8777f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:27.445190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0316b3d9-642b-4cea-9c81-00f5e2a8777f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:27.446759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0316b3d9-642b-4cea-9c81-00f5e2a8777f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:27.449486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0316b3d9-642b-4cea-9c81-00f5e2a8777f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:27.451059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0316b3d9-642b-4cea-9c81-00f5e2a8777f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:27.460333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0316b3d9-642b-4cea-9c81-00f5e2a8777f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:27.581307 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f7f189d1-4a11-47bc-9ca4-600dd4caf2bc tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 243 microversion: 2.1 time: 0.761149 Sep 27 10:07:27.582053 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1252/2734] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:07:26 2022] POST /compute/v2.1/os-security-groups => generated 243 bytes in 763 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:27.588190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0bc4a910-aaa8-4eac-aa75-9e32745c89d9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:27.588837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0bc4a910-aaa8-4eac-aa75-9e32745c89d9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:27.601116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=3,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:27.616907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bc4a910-aaa8-4eac-aa75-9e32745c89d9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:27.618212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bc4a910-aaa8-4eac-aa75-9e32745c89d9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:27.619071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bc4a910-aaa8-4eac-aa75-9e32745c89d9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:27.619936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bc4a910-aaa8-4eac-aa75-9e32745c89d9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:27.620447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bc4a910-aaa8-4eac-aa75-9e32745c89d9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:27.621252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bc4a910-aaa8-4eac-aa75-9e32745c89d9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:27.652300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0316b3d9-642b-4cea-9c81-00f5e2a8777f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:27.668775 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0316b3d9-642b-4cea-9c81-00f5e2a8777f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1276 microversion: 2.1 time: 0.257358 Sep 27 10:07:27.689881 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1483/2735] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:27 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1276 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:27.703617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e6711c14-4bfd-4360-86bf-7ce0490cc8b0 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:27.704784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e6711c14-4bfd-4360-86bf-7ce0490cc8b0 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:27.728273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-0bc4a910-aaa8-4eac-aa75-9e32745c89d9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lazy-loading 'flavor' on Instance uuid 42709739-135c-4adb-aa6c-4fc216a5d618 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:07:27.732104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6711c14-4bfd-4360-86bf-7ce0490cc8b0 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:27.733031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6711c14-4bfd-4360-86bf-7ce0490cc8b0 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:27.735661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6711c14-4bfd-4360-86bf-7ce0490cc8b0 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:27.736625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6711c14-4bfd-4360-86bf-7ce0490cc8b0 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:27.738191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6711c14-4bfd-4360-86bf-7ce0490cc8b0 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:27.739018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6711c14-4bfd-4360-86bf-7ce0490cc8b0 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:27.878349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e6711c14-4bfd-4360-86bf-7ce0490cc8b0 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:27.891085 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e6711c14-4bfd-4360-86bf-7ce0490cc8b0 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1217 microversion: 2.1 time: 0.199096 Sep 27 10:07:27.892644 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1484/2736] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:27 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1217 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:27.901608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7397cbb5-26ef-460f-92a8-16472c37e124 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:27.902603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7397cbb5-26ef-460f-92a8-16472c37e124 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:27.920575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7397cbb5-26ef-460f-92a8-16472c37e124 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:27.921464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7397cbb5-26ef-460f-92a8-16472c37e124 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:27.922130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7397cbb5-26ef-460f-92a8-16472c37e124 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:27.923029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7397cbb5-26ef-460f-92a8-16472c37e124 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:27.923706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7397cbb5-26ef-460f-92a8-16472c37e124 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:27.924378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7397cbb5-26ef-460f-92a8-16472c37e124 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:28.217085 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0bc4a910-aaa8-4eac-aa75-9e32745c89d9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "POST /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618/action" status: 202 len: 0 microversion: 2.91 time: 0.633410 Sep 27 10:07:28.217964 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1253/2737] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:07:27 2022] POST /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618/action => generated 0 bytes in 635 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:28.249879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a14e0b20-bc62-46ef-9c0f-a6c7cf8792c4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Action: 'update', calling method: .version_select at 0x7f7f629bcaf0>, body: {"security_group": {"name": "tempest-sg-hth-1126232817", "description": "tempest-description-hth-248809301"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:28.485651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7397cbb5-26ef-460f-92a8-16472c37e124 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:28.499992 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7397cbb5-26ef-460f-92a8-16472c37e124 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.605487 Sep 27 10:07:28.500953 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1485/2738] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:27 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 607 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:28.519985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-012184f1-c6cd-4f2e-bd0f-5e551b236c3e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:28.529353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-012184f1-c6cd-4f2e-bd0f-5e551b236c3e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:28.556260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012184f1-c6cd-4f2e-bd0f-5e551b236c3e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:28.556927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012184f1-c6cd-4f2e-bd0f-5e551b236c3e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:28.557378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012184f1-c6cd-4f2e-bd0f-5e551b236c3e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:28.558299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012184f1-c6cd-4f2e-bd0f-5e551b236c3e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:28.559642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012184f1-c6cd-4f2e-bd0f-5e551b236c3e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:28.560246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012184f1-c6cd-4f2e-bd0f-5e551b236c3e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:28.755042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-012184f1-c6cd-4f2e-bd0f-5e551b236c3e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:28.771340 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-012184f1-c6cd-4f2e-bd0f-5e551b236c3e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1272 microversion: 2.1 time: 0.267751 Sep 27 10:07:28.786432 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1486/2739] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:28 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1272 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:28.797563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-78bfe514-113c-4455-a337-728178fb3a9b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:28.799472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-78bfe514-113c-4455-a337-728178fb3a9b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:28.810491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78bfe514-113c-4455-a337-728178fb3a9b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:28.811261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78bfe514-113c-4455-a337-728178fb3a9b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:28.811973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78bfe514-113c-4455-a337-728178fb3a9b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:28.813312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78bfe514-113c-4455-a337-728178fb3a9b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:28.813960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78bfe514-113c-4455-a337-728178fb3a9b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:28.814985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78bfe514-113c-4455-a337-728178fb3a9b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:28.910530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-78bfe514-113c-4455-a337-728178fb3a9b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:28.921773 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-78bfe514-113c-4455-a337-728178fb3a9b tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d" status: 200 len: 1431 microversion: 2.1 time: 0.133683 Sep 27 10:07:28.922632 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1487/2740] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:28 2022] GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d => generated 1431 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:28.942324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:28.954064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:28.980249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:28.980788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:28.981241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:28.988928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:28.991477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:28.994807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.096804 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a14e0b20-bc62-46ef-9c0f-a6c7cf8792c4 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "PUT /compute/v2.1/os-security-groups/20c89bc9-bde0-408a-83b1-ba108d0f9c22" status: 200 len: 217 microversion: 2.1 time: 0.861613 Sep 27 10:07:29.098583 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1254/2741] 10.222.0.26 () {60 vars in 1290 bytes} [Tue Sep 27 10:07:28 2022] PUT /compute/v2.1/os-security-groups/20c89bc9-bde0-408a-83b1-ba108d0f9c22 => generated 217 bytes in 864 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:29.105020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e63cd8ae-c469-48c2-913b-8b037dbfa920 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:29.105972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e63cd8ae-c469-48c2-913b-8b037dbfa920 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:29.119218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e63cd8ae-c469-48c2-913b-8b037dbfa920 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.119775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e63cd8ae-c469-48c2-913b-8b037dbfa920 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.120342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e63cd8ae-c469-48c2-913b-8b037dbfa920 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.121109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e63cd8ae-c469-48c2-913b-8b037dbfa920 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.122720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e63cd8ae-c469-48c2-913b-8b037dbfa920 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.123754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e63cd8ae-c469-48c2-913b-8b037dbfa920 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.284296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:29.307391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:29.309457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:29.330424 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-32c3f8a4-3aad-4e85-b017-bf593e73a2c8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.403347 Sep 27 10:07:29.332184 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1488/2742] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:28 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:29.342930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3b1d82fa-5b49-4b46-b64f-b65b79471c08 req-f2c35986-748b-41c6-8f81-6caa8a17e3b8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "918832ca-5183-4a17-b0ed-01fc0c661f3e", "name": "network-vif-plugged", "status": "completed", "tag": "6266abbd-ab22-410b-b499-c771bc142abf"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:29.368333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3b1d82fa-5b49-4b46-b64f-b65b79471c08 req-f2c35986-748b-41c6-8f81-6caa8a17e3b8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.371220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3b1d82fa-5b49-4b46-b64f-b65b79471c08 req-f2c35986-748b-41c6-8f81-6caa8a17e3b8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.373569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3b1d82fa-5b49-4b46-b64f-b65b79471c08 req-f2c35986-748b-41c6-8f81-6caa8a17e3b8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.406564 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3b1d82fa-5b49-4b46-b64f-b65b79471c08 req-f2c35986-748b-41c6-8f81-6caa8a17e3b8 service nova] Creating event network-vif-plugged:6266abbd-ab22-410b-b499-c771bc142abf for instance 918832ca-5183-4a17-b0ed-01fc0c661f3e on nch2-master-1 Sep 27 10:07:29.430681 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3b1d82fa-5b49-4b46-b64f-b65b79471c08 req-f2c35986-748b-41c6-8f81-6caa8a17e3b8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083257 Sep 27 10:07:29.432775 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1489/2743] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:29.445513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-466b1d89-f054-48a9-bfd4-bb6a630d5ea8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:29.446856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-466b1d89-f054-48a9-bfd4-bb6a630d5ea8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:29.474835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466b1d89-f054-48a9-bfd4-bb6a630d5ea8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.475384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466b1d89-f054-48a9-bfd4-bb6a630d5ea8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.476057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466b1d89-f054-48a9-bfd4-bb6a630d5ea8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.477582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466b1d89-f054-48a9-bfd4-bb6a630d5ea8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.478074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466b1d89-f054-48a9-bfd4-bb6a630d5ea8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.478520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-466b1d89-f054-48a9-bfd4-bb6a630d5ea8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.507821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:29.592468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-466b1d89-f054-48a9-bfd4-bb6a630d5ea8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:29.601732 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-466b1d89-f054-48a9-bfd4-bb6a630d5ea8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1282 microversion: 2.1 time: 0.164253 Sep 27 10:07:29.602760 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1490/2744] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:29 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1282 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:29.609189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1016d580-c14d-4ea2-a13f-44c4823ba707 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": 12345, "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1016d580-c14d-4ea2-a13f-44c4823ba707 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 163 Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": 12345, "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 12345, 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 12345 is not of type 'string' Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['server']['properties']['name']: Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 12345 Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:07:29.613298 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1016d580-c14d-4ea2-a13f-44c4823ba707 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Invalid input for field/attribute name. Value: 12345. 12345 is not of type 'string' {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:29.617445 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1016d580-c14d-4ea2-a13f-44c4823ba707 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 127 microversion: 2.1 time: 0.010726 Sep 27 10:07:29.620631 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1491/2745] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:29 2022] POST /compute/v2.1/servers => generated 127 bytes in 17 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:07:29.626569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9fdd7568-b78e-4fd4-8b0c-3d9f17c985da tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:29.627287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9fdd7568-b78e-4fd4-8b0c-3d9f17c985da tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:29.635768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fdd7568-b78e-4fd4-8b0c-3d9f17c985da tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.636437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fdd7568-b78e-4fd4-8b0c-3d9f17c985da tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.636941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fdd7568-b78e-4fd4-8b0c-3d9f17c985da tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.637750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fdd7568-b78e-4fd4-8b0c-3d9f17c985da tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.638466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fdd7568-b78e-4fd4-8b0c-3d9f17c985da tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.638989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fdd7568-b78e-4fd4-8b0c-3d9f17c985da tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.706773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9fdd7568-b78e-4fd4-8b0c-3d9f17c985da tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:29.718560 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9fdd7568-b78e-4fd4-8b0c-3d9f17c985da tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1271 microversion: 2.1 time: 0.095962 Sep 27 10:07:29.720649 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1492/2746] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:29 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1271 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:29.727781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c6ab441-6cc7-4e1c-b136-f4e1c3830748 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62889f70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:29.754952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e63cd8ae-c469-48c2-913b-8b037dbfa920 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:29.768780 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e63cd8ae-c469-48c2-913b-8b037dbfa920 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1315 microversion: 2.1 time: 0.669027 Sep 27 10:07:29.769389 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1255/2747] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:29 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1315 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:29.774364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7b58e56b-d875-4da5-bb2d-dd939cb3ef31 req-10123b21-99f8-4b55-b807-5de51ee5a694 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "961407e4-8e66-4f1c-88bd-6d2b4795c11d", "name": "network-vif-deleted", "tag": "52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:29.790236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7b58e56b-d875-4da5-bb2d-dd939cb3ef31 req-10123b21-99f8-4b55-b807-5de51ee5a694 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.791024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7b58e56b-d875-4da5-bb2d-dd939cb3ef31 req-10123b21-99f8-4b55-b807-5de51ee5a694 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.791687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7b58e56b-d875-4da5-bb2d-dd939cb3ef31 req-10123b21-99f8-4b55-b807-5de51ee5a694 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.813129 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7b58e56b-d875-4da5-bb2d-dd939cb3ef31 req-10123b21-99f8-4b55-b807-5de51ee5a694 service nova] Creating event network-vif-deleted:52ed5cf5-8e0a-46e5-b7f9-31b30afa1c6f for instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d on nch2-master-1 Sep 27 10:07:29.825404 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7b58e56b-d875-4da5-bb2d-dd939cb3ef31 req-10123b21-99f8-4b55-b807-5de51ee5a694 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054519 Sep 27 10:07:29.826350 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1256/2748] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:29.833530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a0e91132-c1a6-4576-96ca-35dbdffa2563 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:29.834612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a0e91132-c1a6-4576-96ca-35dbdffa2563 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:29.842673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0e91132-c1a6-4576-96ca-35dbdffa2563 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.843448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0e91132-c1a6-4576-96ca-35dbdffa2563 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.844074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0e91132-c1a6-4576-96ca-35dbdffa2563 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.844997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0e91132-c1a6-4576-96ca-35dbdffa2563 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.846279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0e91132-c1a6-4576-96ca-35dbdffa2563 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.846982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0e91132-c1a6-4576-96ca-35dbdffa2563 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.884367 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c6ab441-6cc7-4e1c-b136-f4e1c3830748 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups/20c89bc9-bde0-408a-83b1-ba108d0f9c22" status: 200 len: 217 microversion: 2.1 time: 0.161735 Sep 27 10:07:29.894678 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1493/2749] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:07:29 2022] GET /compute/v2.1/os-security-groups/20c89bc9-bde0-408a-83b1-ba108d0f9c22 => generated 217 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:29.899868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-816d19fc-c82e-4a05-8d87-e9160091eebd tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:29.900526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-816d19fc-c82e-4a05-8d87-e9160091eebd tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:29.907291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816d19fc-c82e-4a05-8d87-e9160091eebd tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.907801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816d19fc-c82e-4a05-8d87-e9160091eebd tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.908242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816d19fc-c82e-4a05-8d87-e9160091eebd tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:29.908988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816d19fc-c82e-4a05-8d87-e9160091eebd tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:29.909453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816d19fc-c82e-4a05-8d87-e9160091eebd tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:29.909886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-816d19fc-c82e-4a05-8d87-e9160091eebd tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.165381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-816d19fc-c82e-4a05-8d87-e9160091eebd tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:30.175702 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-816d19fc-c82e-4a05-8d87-e9160091eebd tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1314 microversion: 2.1 time: 0.279768 Sep 27 10:07:30.176912 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1494/2750] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:29 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1314 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:30.193240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad263503-3c72-490c-8808-10bf4b93c8c6 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f628c98b0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:30.263462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a0e91132-c1a6-4576-96ca-35dbdffa2563 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:30.274408 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a0e91132-c1a6-4576-96ca-35dbdffa2563 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.446213 Sep 27 10:07:30.275457 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1257/2751] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:29 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:30.281777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:30.283266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:30.291655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.293302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.293974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.295005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.295939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.296701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.337399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.338222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.338862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.339324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lazy-loading 'fault' on Instance uuid 961407e4-8e66-4f1c-88bd-6d2b4795c11d {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:07:30.450265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '032de99964be4424ba019adf2eedf680', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b932820432be493b8a85520f58f49606', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:30.459848 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-063f9bc1-7385-468d-a6a8-c4a075420711 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d" status: 200 len: 1307 microversion: 2.1 time: 0.182770 Sep 27 10:07:30.460510 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1258/2752] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:30 2022] GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d => generated 1307 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:30.465868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ee15bea3-c420-42ef-873f-b8bacd6ffb70 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:30.466551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ee15bea3-c420-42ef-873f-b8bacd6ffb70 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:30.473205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee15bea3-c420-42ef-873f-b8bacd6ffb70 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.474015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee15bea3-c420-42ef-873f-b8bacd6ffb70 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.474661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee15bea3-c420-42ef-873f-b8bacd6ffb70 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.475578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee15bea3-c420-42ef-873f-b8bacd6ffb70 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.476316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee15bea3-c420-42ef-873f-b8bacd6ffb70 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.476985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee15bea3-c420-42ef-873f-b8bacd6ffb70 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.559711 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ad263503-3c72-490c-8808-10bf4b93c8c6 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/20c89bc9-bde0-408a-83b1-ba108d0f9c22" status: 202 len: 0 microversion: 2.1 time: 0.380897 Sep 27 10:07:30.560966 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1495/2753] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:30 2022] DELETE /compute/v2.1/os-security-groups/20c89bc9-bde0-408a-83b1-ba108d0f9c22 => generated 0 bytes in 383 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:07:30.571205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:30.572273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:30.595761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.598642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.601063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.602166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.602719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.603071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.751753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:30.764043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:30.765875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:30.772497 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-182fdac4-01e9-42b9-9989-07c820b7e28e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.209773 Sep 27 10:07:30.785141 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1496/2754] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:30 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:30.785141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:30.785141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:30.790816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.792152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.792813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.796047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.796758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.797372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.813392 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 HTTP/1.1 Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'ac5c4cdc-90a8-48c7-84f3-0053b32834f3'} Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:30.815506 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9b1930b6-a532-4f8b-b886-73b3919f1394 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 404 len: 111 microversion: 2.1 time: 0.042455 Sep 27 10:07:30.820172 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1497/2755] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:07:30 2022] DELETE /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:30.823982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-22b8d53d-ea24-4b9b-9e67-3af890f26e1d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:30.824747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-22b8d53d-ea24-4b9b-9e67-3af890f26e1d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:30.831304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22b8d53d-ea24-4b9b-9e67-3af890f26e1d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.831795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22b8d53d-ea24-4b9b-9e67-3af890f26e1d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.832235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22b8d53d-ea24-4b9b-9e67-3af890f26e1d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.833503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22b8d53d-ea24-4b9b-9e67-3af890f26e1d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.833503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22b8d53d-ea24-4b9b-9e67-3af890f26e1d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.833937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22b8d53d-ea24-4b9b-9e67-3af890f26e1d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.918513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-22b8d53d-ea24-4b9b-9e67-3af890f26e1d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:30.927512 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-22b8d53d-ea24-4b9b-9e67-3af890f26e1d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1282 microversion: 2.1 time: 0.107979 Sep 27 10:07:30.928099 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1498/2756] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:30 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1282 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:30.933121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ba30689f-6f22-4b8e-8c19-5895b72d8c73 req-e66d8907-1e68-4d30-bd03-7a2fc3c68358 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b902a024-8aa7-4843-8680-41d3e189d3af", "tag": "df367b62-0029-488e-9897-fac934fc397e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:30.945219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ba30689f-6f22-4b8e-8c19-5895b72d8c73 req-e66d8907-1e68-4d30-bd03-7a2fc3c68358 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.945946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ba30689f-6f22-4b8e-8c19-5895b72d8c73 req-e66d8907-1e68-4d30-bd03-7a2fc3c68358 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.946646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ba30689f-6f22-4b8e-8c19-5895b72d8c73 req-e66d8907-1e68-4d30-bd03-7a2fc3c68358 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.958713 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ba30689f-6f22-4b8e-8c19-5895b72d8c73 req-e66d8907-1e68-4d30-bd03-7a2fc3c68358 service nova] Creating event network-changed:df367b62-0029-488e-9897-fac934fc397e for instance b902a024-8aa7-4843-8680-41d3e189d3af on nch1-master-1 Sep 27 10:07:30.963896 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ba30689f-6f22-4b8e-8c19-5895b72d8c73 req-e66d8907-1e68-4d30-bd03-7a2fc3c68358 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034247 Sep 27 10:07:30.974879 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1499/2757] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:30.981640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-517cb6f4-2460-42c5-80fb-eeb4aecc514e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:30.982977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-517cb6f4-2460-42c5-80fb-eeb4aecc514e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:30.994521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-517cb6f4-2460-42c5-80fb-eeb4aecc514e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.995554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-517cb6f4-2460-42c5-80fb-eeb4aecc514e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.996237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-517cb6f4-2460-42c5-80fb-eeb4aecc514e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:30.998269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-517cb6f4-2460-42c5-80fb-eeb4aecc514e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:30.999013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-517cb6f4-2460-42c5-80fb-eeb4aecc514e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:30.999710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-517cb6f4-2460-42c5-80fb-eeb4aecc514e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:31.091959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-517cb6f4-2460-42c5-80fb-eeb4aecc514e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:31.101244 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-517cb6f4-2460-42c5-80fb-eeb4aecc514e tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1271 microversion: 2.1 time: 0.124901 Sep 27 10:07:31.111675 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1500/2758] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:30 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1271 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:31.116784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e027bf83-f668-4087-81ae-512f6c6ce093 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:31.118181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e027bf83-f668-4087-81ae-512f6c6ce093 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:31.125374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e027bf83-f668-4087-81ae-512f6c6ce093 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:31.125881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e027bf83-f668-4087-81ae-512f6c6ce093 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:31.126305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e027bf83-f668-4087-81ae-512f6c6ce093 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:31.127026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e027bf83-f668-4087-81ae-512f6c6ce093 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:31.128167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e027bf83-f668-4087-81ae-512f6c6ce093 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:31.128632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e027bf83-f668-4087-81ae-512f6c6ce093 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:31.266881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ee15bea3-c420-42ef-873f-b8bacd6ffb70 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:31.280425 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ee15bea3-c420-42ef-873f-b8bacd6ffb70 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.818171 Sep 27 10:07:31.307127 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1259/2759] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:30 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 833 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:31.312947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:31.314456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: ac5c4cdc-90a8-48c7-84f3-0053b32834f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:31.329098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e027bf83-f668-4087-81ae-512f6c6ce093 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:31.346081 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e027bf83-f668-4087-81ae-512f6c6ce093 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1315 microversion: 2.1 time: 0.242389 Sep 27 10:07:31.349693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:31.350765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:31.351767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:31.360784 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1501/2760] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:31 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1315 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:31.365457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:31.365457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:31.365457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:31.366545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9972f606-5916-4908-87f7-da6f2436fb19 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:31.372327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9972f606-5916-4908-87f7-da6f2436fb19 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:31.387616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9972f606-5916-4908-87f7-da6f2436fb19 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:31.390121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9972f606-5916-4908-87f7-da6f2436fb19 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:31.393808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9972f606-5916-4908-87f7-da6f2436fb19 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:31.394691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9972f606-5916-4908-87f7-da6f2436fb19 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:31.395189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9972f606-5916-4908-87f7-da6f2436fb19 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:31.395647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9972f606-5916-4908-87f7-da6f2436fb19 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:31.399463 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 HTTP/1.1 Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'ac5c4cdc-90a8-48c7-84f3-0053b32834f3'} Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.401678 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.407720 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.407720 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:31.407720 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:31.407720 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:31.407720 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:31.407720 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. Sep 27 10:07:31.407720 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.411821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Instance ac5c4cdc-90a8-48c7-84f3-0053b32834f3 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:31.412980 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5ad12f65-4cbf-4563-a0bf-8a279865c88b tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3" status: 404 len: 111 microversion: 2.1 time: 0.104478 Sep 27 10:07:31.414121 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1260/2761] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:31 2022] GET /compute/v2.1/servers/ac5c4cdc-90a8-48c7-84f3-0053b32834f3 => generated 111 bytes in 106 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:31.430096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-22ab5487-3ac0-4de7-a749-d4b26106d766 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f629ed700>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:31.571938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG neutronclient.v2_0.client [None req-22ab5487-3ac0-4de7-a749-d4b26106d766 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 24e4adb8-847d-4647-a320-2ac81698f21f does not exist", "detail": ""}} {{(pid=145969) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:07:31.572730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.security_group_api [None req-22ab5487-3ac0-4de7-a749-d4b26106d766 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Neutron security group 24e4adb8-847d-4647-a320-2ac81698f21f not found {{(pid=145969) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:07:31.573530 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-22ab5487-3ac0-4de7-a749-d4b26106d766 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security group 24e4adb8-847d-4647-a320-2ac81698f21f does not exist Sep 27 10:07:31.573530 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-d81d9155-0e85-4403-90bb-fb60d7929765'] Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-22ab5487-3ac0-4de7-a749-d4b26106d766 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/os-security-groups/24e4adb8-847d-4647-a320-2ac81698f21f HTTP/1.1 Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: exception: Security group 24e4adb8-847d-4647-a320-2ac81698f21f does not exist Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-d81d9155-0e85-4403-90bb-fb60d7929765'] Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '24e4adb8-847d-4647-a320-2ac81698f21f'} Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:07:31.575828 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 24e4adb8-847d-4647-a320-2ac81698f21f does not exist Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d81d9155-0e85-4403-90bb-fb60d7929765'] Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 24e4adb8-847d-4647-a320-2ac81698f21f does not exist Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d81d9155-0e85-4403-90bb-fb60d7929765'] Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 24e4adb8-847d-4647-a320-2ac81698f21f does not exist Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d81d9155-0e85-4403-90bb-fb60d7929765'] Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 24e4adb8-847d-4647-a320-2ac81698f21f does not exist Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-d81d9155-0e85-4403-90bb-fb60d7929765'] Sep 27 10:07:31.580812 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.584990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-22ab5487-3ac0-4de7-a749-d4b26106d766 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Security group 24e4adb8-847d-4647-a320-2ac81698f21f does not exist Sep 27 10:07:31.584990 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-d81d9155-0e85-4403-90bb-fb60d7929765'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:31.586243 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-22ab5487-3ac0-4de7-a749-d4b26106d766 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/24e4adb8-847d-4647-a320-2ac81698f21f" status: 404 len: 194 microversion: 2.1 time: 0.164424 Sep 27 10:07:31.588170 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1261/2762] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:31 2022] DELETE /compute/v2.1/os-security-groups/24e4adb8-847d-4647-a320-2ac81698f21f => generated 194 bytes in 170 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:31.613091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:31.613729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] [instance: 961407e4-8e66-4f1c-88bd-6d2b4795c11d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:31.637187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:31.642044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:31.645055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:31.649185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:31.649912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:31.653246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:31.679345 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] HTTP exception thrown: Instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d could not be found. Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Request method failure captured: Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d HTTP/1.1 Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsqbF49kTqg2OfPo0tbTuW_gqKOl2lw_jczFixJFmB1obsvcOkG_4u8HNqlSejQ_ffVI6ktTjnJ-Fcr0PMzqRmeXTIfZxjrlCjFOiOSvKBOGKrOZCgWNxKLTQLQ1nVeG6Y9l3m4nmiBpwq2AAG1aJ3o0OwY_VhJ249dqL-HF-F6wC4o Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: b932820432be493b8a85520f58f49606 Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ListImageFiltersTestJSON-361314172 Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b932820432be493b8a85520f58f49606"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b932820432be493b8a85520f58f49606"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b932820432be493b8a85520f58f49606"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b932820432be493b8a85520f58f49606"}]}] Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ListImageFiltersTestJSON-361314172 Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: b932820432be493b8a85520f58f49606 Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ListImageFiltersTestJSON-361314172 Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ListImageFiltersTestJSON-361314172-project Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 032de99964be4424ba019adf2eedf680 Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ListImageFiltersTestJSON-361314172-project Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d could not be found. Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '961407e4-8e66-4f1c-88bd-6d2b4795c11d'} Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d could not be found. Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d could not be found. Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.682999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.688830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.688830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:31.688830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:31.688830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:31.688830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:31.688830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d could not be found. Sep 27 10:07:31.688830 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.688830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Returning 404 to user: Instance 961407e4-8e66-4f1c-88bd-6d2b4795c11d could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:31.688830 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-90a6af93-1d17-4613-b8ee-d5823473cb10 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d" status: 404 len: 111 microversion: 2.1 time: 0.098396 Sep 27 10:07:31.690167 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1262/2763] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:31 2022] GET /compute/v2.1/servers/961407e4-8e66-4f1c-88bd-6d2b4795c11d => generated 111 bytes in 101 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:31.697640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-261ac389-dabd-47cd-ad5e-a50719cc1f8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62ad3f70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:31.711656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9972f606-5916-4908-87f7-da6f2436fb19 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:31.721772 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9972f606-5916-4908-87f7-da6f2436fb19 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1314 microversion: 2.1 time: 0.366138 Sep 27 10:07:31.723227 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1502/2764] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:31 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1314 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:31.729586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6acf52b7-fb8c-4f43-a791-5585d309cbb4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f628b6430>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:31.757118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG neutronclient.v2_0.client [None req-261ac389-dabd-47cd-ad5e-a50719cc1f8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 6244f0a6-16ad-4669-ba27-a50476f4d398 does not exist", "detail": ""}} {{(pid=145969) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:07:31.760746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.security_group_api [None req-261ac389-dabd-47cd-ad5e-a50719cc1f8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Neutron security group 6244f0a6-16ad-4669-ba27-a50476f4d398 not found {{(pid=145969) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:07:31.762280 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-261ac389-dabd-47cd-ad5e-a50719cc1f8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security group 6244f0a6-16ad-4669-ba27-a50476f4d398 does not exist Sep 27 10:07:31.762280 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-aef52c27-e99a-49f7-80a7-eaf9654ff15d'] Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-261ac389-dabd-47cd-ad5e-a50719cc1f8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/os-security-groups/6244f0a6-16ad-4669-ba27-a50476f4d398 HTTP/1.1 Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: exception: Security group 6244f0a6-16ad-4669-ba27-a50476f4d398 does not exist Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-aef52c27-e99a-49f7-80a7-eaf9654ff15d'] Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '6244f0a6-16ad-4669-ba27-a50476f4d398'} Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:07:31.765414 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 6244f0a6-16ad-4669-ba27-a50476f4d398 does not exist Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-aef52c27-e99a-49f7-80a7-eaf9654ff15d'] Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 6244f0a6-16ad-4669-ba27-a50476f4d398 does not exist Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-aef52c27-e99a-49f7-80a7-eaf9654ff15d'] Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 6244f0a6-16ad-4669-ba27-a50476f4d398 does not exist Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-aef52c27-e99a-49f7-80a7-eaf9654ff15d'] Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 6244f0a6-16ad-4669-ba27-a50476f4d398 does not exist Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-aef52c27-e99a-49f7-80a7-eaf9654ff15d'] Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-261ac389-dabd-47cd-ad5e-a50719cc1f8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Security group 6244f0a6-16ad-4669-ba27-a50476f4d398 does not exist Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-aef52c27-e99a-49f7-80a7-eaf9654ff15d'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:31.771661 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-261ac389-dabd-47cd-ad5e-a50719cc1f8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/6244f0a6-16ad-4669-ba27-a50476f4d398" status: 404 len: 194 microversion: 2.1 time: 0.075979 Sep 27 10:07:31.778367 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1263/2765] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:31 2022] DELETE /compute/v2.1/os-security-groups/6244f0a6-16ad-4669-ba27-a50476f4d398 => generated 194 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:31.783739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a52f60cb-8827-4df8-ad56-1f83bedfe5e7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f629a9310>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:31.865132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG neutronclient.v2_0.client [None req-a52f60cb-8827-4df8-ad56-1f83bedfe5e7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist", "detail": ""}} {{(pid=145969) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:07:31.865724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.security_group_api [None req-a52f60cb-8827-4df8-ad56-1f83bedfe5e7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Neutron security group 327e2132-3353-4d5f-8082-a6489c6018b2 not found {{(pid=145969) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:07:31.866515 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a52f60cb-8827-4df8-ad56-1f83bedfe5e7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:07:31.866515 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-45b46dbd-2e4d-4242-86b9-4092d4f7807a'] Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a52f60cb-8827-4df8-ad56-1f83bedfe5e7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/os-security-groups/327e2132-3353-4d5f-8082-a6489c6018b2 HTTP/1.1 Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: exception: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-45b46dbd-2e4d-4242-86b9-4092d4f7807a'] Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '327e2132-3353-4d5f-8082-a6489c6018b2'} Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:07:31.869752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-45b46dbd-2e4d-4242-86b9-4092d4f7807a'] Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-45b46dbd-2e4d-4242-86b9-4092d4f7807a'] Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-45b46dbd-2e4d-4242-86b9-4092d4f7807a'] Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-45b46dbd-2e4d-4242-86b9-4092d4f7807a'] Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a52f60cb-8827-4df8-ad56-1f83bedfe5e7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Security group 327e2132-3353-4d5f-8082-a6489c6018b2 does not exist Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-45b46dbd-2e4d-4242-86b9-4092d4f7807a'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a52f60cb-8827-4df8-ad56-1f83bedfe5e7 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/327e2132-3353-4d5f-8082-a6489c6018b2" status: 404 len: 194 microversion: 2.1 time: 0.097106 Sep 27 10:07:31.873434 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1264/2766] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:31 2022] DELETE /compute/v2.1/os-security-groups/327e2132-3353-4d5f-8082-a6489c6018b2 => generated 194 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:31.885081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:31.885754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:31.895685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:31.896922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:31.897644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:31.899393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:31.899855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:31.900292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:32.075299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:32.090939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:32.094992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:32.113165 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7e5b86a2-f5fb-487c-9d85-42b233196a6c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.239450 Sep 27 10:07:32.113774 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1265/2767] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:31 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:32.128583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a27b65d0-177e-412f-a8f1-ad25923ee5ed tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62ad3f70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:32.198170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG neutronclient.v2_0.client [None req-a27b65d0-177e-412f-a8f1-ad25923ee5ed tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist", "detail": ""}} {{(pid=145969) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:07:32.198778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.security_group_api [None req-a27b65d0-177e-412f-a8f1-ad25923ee5ed tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Neutron security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 not found {{(pid=145969) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:07:32.199516 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a27b65d0-177e-412f-a8f1-ad25923ee5ed tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:07:32.199516 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-911536cb-24ee-4e29-823c-2a1b8f3202ed'] Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a27b65d0-177e-412f-a8f1-ad25923ee5ed tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/os-security-groups/8f50a8f9-7234-4cf4-9730-39b3803a1274 HTTP/1.1 Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: exception: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-911536cb-24ee-4e29-823c-2a1b8f3202ed'] Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '8f50a8f9-7234-4cf4-9730-39b3803a1274'} Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:07:32.201749 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-911536cb-24ee-4e29-823c-2a1b8f3202ed'] Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-911536cb-24ee-4e29-823c-2a1b8f3202ed'] Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-911536cb-24ee-4e29-823c-2a1b8f3202ed'] Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-911536cb-24ee-4e29-823c-2a1b8f3202ed'] Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a27b65d0-177e-412f-a8f1-ad25923ee5ed tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Security group 8f50a8f9-7234-4cf4-9730-39b3803a1274 does not exist Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-911536cb-24ee-4e29-823c-2a1b8f3202ed'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:32.207723 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a27b65d0-177e-412f-a8f1-ad25923ee5ed tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/8f50a8f9-7234-4cf4-9730-39b3803a1274" status: 404 len: 194 microversion: 2.1 time: 0.085159 Sep 27 10:07:32.219956 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1266/2768] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:32 2022] DELETE /compute/v2.1/os-security-groups/8f50a8f9-7234-4cf4-9730-39b3803a1274 => generated 194 bytes in 102 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:32.238982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-30b3cf74-e862-4cce-983b-38bac3f4e570 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:32.239607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-30b3cf74-e862-4cce-983b-38bac3f4e570 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:32.252096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30b3cf74-e862-4cce-983b-38bac3f4e570 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:32.252846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30b3cf74-e862-4cce-983b-38bac3f4e570 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:32.253608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30b3cf74-e862-4cce-983b-38bac3f4e570 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:32.254508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30b3cf74-e862-4cce-983b-38bac3f4e570 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:32.255138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30b3cf74-e862-4cce-983b-38bac3f4e570 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:32.255580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30b3cf74-e862-4cce-983b-38bac3f4e570 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:32.346218 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6acf52b7-fb8c-4f43-a791-5585d309cbb4 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "DELETE /compute/v2.1/images/b57e736e-9630-498b-b438-5b87c52b4e9a" status: 204 len: 0 microversion: 2.1 time: 0.621501 Sep 27 10:07:32.347025 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1503/2769] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:31 2022] DELETE /compute/v2.1/images/b57e736e-9630-498b-b438-5b87c52b4e9a => generated 0 bytes in 623 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:32.354568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fc948f99-47bb-4954-ae0a-5feabccd3fc5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:32.355358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fc948f99-47bb-4954-ae0a-5feabccd3fc5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:32.363127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc948f99-47bb-4954-ae0a-5feabccd3fc5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:32.363673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc948f99-47bb-4954-ae0a-5feabccd3fc5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:32.364161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc948f99-47bb-4954-ae0a-5feabccd3fc5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:32.365211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc948f99-47bb-4954-ae0a-5feabccd3fc5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:32.365823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc948f99-47bb-4954-ae0a-5feabccd3fc5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:32.366509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc948f99-47bb-4954-ae0a-5feabccd3fc5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:32.592231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc948f99-47bb-4954-ae0a-5feabccd3fc5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:32.603478 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fc948f99-47bb-4954-ae0a-5feabccd3fc5 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1313 microversion: 2.1 time: 0.253702 Sep 27 10:07:32.604462 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1504/2770] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:32 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1313 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:32.627054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9278eed6-0c90-45f6-8b3c-94b685694a74 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f628e35e0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:32.658920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-9278eed6-0c90-45f6-8b3c-94b685694a74 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:07:32.659902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.security_group_api [None req-9278eed6-0c90-45f6-8b3c-94b685694a74 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Neutron security group 88d2fe43-c785-472b-822f-124530320c7a not found {{(pid=145968) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:07:32.660015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-30b3cf74-e862-4cce-983b-38bac3f4e570 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:32.660560 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-9278eed6-0c90-45f6-8b3c-94b685694a74 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:07:32.660560 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-c3601d1c-1072-4678-a510-dd653a7b288e'] Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9278eed6-0c90-45f6-8b3c-94b685694a74 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-security-groups/88d2fe43-c785-472b-822f-124530320c7a HTTP/1.1 Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: exception: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-c3601d1c-1072-4678-a510-dd653a7b288e'] Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '88d2fe43-c785-472b-822f-124530320c7a'} Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:07:32.663701 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c3601d1c-1072-4678-a510-dd653a7b288e'] Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c3601d1c-1072-4678-a510-dd653a7b288e'] Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c3601d1c-1072-4678-a510-dd653a7b288e'] Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-c3601d1c-1072-4678-a510-dd653a7b288e'] Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9278eed6-0c90-45f6-8b3c-94b685694a74 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Security group 88d2fe43-c785-472b-822f-124530320c7a does not exist Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-c3601d1c-1072-4678-a510-dd653a7b288e'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9278eed6-0c90-45f6-8b3c-94b685694a74 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/88d2fe43-c785-472b-822f-124530320c7a" status: 404 len: 194 microversion: 2.1 time: 0.041720 Sep 27 10:07:32.667432 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1505/2771] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:32 2022] DELETE /compute/v2.1/os-security-groups/88d2fe43-c785-472b-822f-124530320c7a => generated 194 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:32.679587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-74dbc836-44fc-4469-a7a7-b0dc469aa2d7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f628b60d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:32.682401 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-30b3cf74-e862-4cce-983b-38bac3f4e570 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1324 microversion: 2.1 time: 0.461014 Sep 27 10:07:32.683185 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1267/2772] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:32 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1324 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:32.687941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-70a4d4be-d985-4260-8b1a-f404bf617e44 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:32.688798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-70a4d4be-d985-4260-8b1a-f404bf617e44 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:32.701308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4d4be-d985-4260-8b1a-f404bf617e44 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:32.702306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4d4be-d985-4260-8b1a-f404bf617e44 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:32.702961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4d4be-d985-4260-8b1a-f404bf617e44 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:32.703869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4d4be-d985-4260-8b1a-f404bf617e44 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:32.704483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4d4be-d985-4260-8b1a-f404bf617e44 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:32.704948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4d4be-d985-4260-8b1a-f404bf617e44 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.010046 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-74dbc836-44fc-4469-a7a7-b0dc469aa2d7 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "DELETE /compute/v2.1/images/fa809ef6-dd19-4a85-b3be-62e8c2304c98" status: 204 len: 0 microversion: 2.1 time: 0.341950 Sep 27 10:07:33.013781 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1506/2773] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:32 2022] DELETE /compute/v2.1/images/fa809ef6-dd19-4a85-b3be-62e8c2304c98 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:33.015952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5e3d173-b8a3-4173-86e8-32f5241328bc tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f62936b80>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:33.057330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-a5e3d173-b8a3-4173-86e8-32f5241328bc tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:07:33.058328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.security_group_api [None req-a5e3d173-b8a3-4173-86e8-32f5241328bc tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Neutron security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e not found {{(pid=145968) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:07:33.059259 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a5e3d173-b8a3-4173-86e8-32f5241328bc tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:07:33.059259 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-3be2fed7-f50d-4fb1-87ec-36fb0cfcfbe1'] Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5e3d173-b8a3-4173-86e8-32f5241328bc tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-security-groups/5e9c7520-4b04-421c-973d-54c7ff2bbd9e HTTP/1.1 Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: exception: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-3be2fed7-f50d-4fb1-87ec-36fb0cfcfbe1'] Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '5e9c7520-4b04-421c-973d-54c7ff2bbd9e'} Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:07:33.062160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3be2fed7-f50d-4fb1-87ec-36fb0cfcfbe1'] Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3be2fed7-f50d-4fb1-87ec-36fb0cfcfbe1'] Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3be2fed7-f50d-4fb1-87ec-36fb0cfcfbe1'] Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-3be2fed7-f50d-4fb1-87ec-36fb0cfcfbe1'] Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5e3d173-b8a3-4173-86e8-32f5241328bc tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Security group 5e9c7520-4b04-421c-973d-54c7ff2bbd9e does not exist Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-3be2fed7-f50d-4fb1-87ec-36fb0cfcfbe1'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:33.067578 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a5e3d173-b8a3-4173-86e8-32f5241328bc tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/5e9c7520-4b04-421c-973d-54c7ff2bbd9e" status: 404 len: 194 microversion: 2.1 time: 0.055600 Sep 27 10:07:33.075624 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1507/2774] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:33 2022] DELETE /compute/v2.1/os-security-groups/5e9c7520-4b04-421c-973d-54c7ff2bbd9e => generated 194 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:33.075624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d5554203-7fc0-421d-92e6-1784832b244e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:33.075624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d5554203-7fc0-421d-92e6-1784832b244e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:33.084495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5554203-7fc0-421d-92e6-1784832b244e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.084962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5554203-7fc0-421d-92e6-1784832b244e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.085984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5554203-7fc0-421d-92e6-1784832b244e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.087605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5554203-7fc0-421d-92e6-1784832b244e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.088076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5554203-7fc0-421d-92e6-1784832b244e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.088537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5554203-7fc0-421d-92e6-1784832b244e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.159548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-70a4d4be-d985-4260-8b1a-f404bf617e44 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:33.170533 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-70a4d4be-d985-4260-8b1a-f404bf617e44 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1502 microversion: 2.1 time: 0.486113 Sep 27 10:07:33.171438 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1268/2775] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:32 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1502 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:33.177275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b7ee2202-fdda-4266-a0a2-9daed0808f20 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] Calling method '.version_select at 0x7f7f629ef040>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:33.261440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d5554203-7fc0-421d-92e6-1784832b244e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:33.270670 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d5554203-7fc0-421d-92e6-1784832b244e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1314 microversion: 2.1 time: 0.200745 Sep 27 10:07:33.271824 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1508/2776] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:33 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1314 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:33.278180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d1a4d416-cba1-4459-885d-851c63b8ee55 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:33.279079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d1a4d416-cba1-4459-885d-851c63b8ee55 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:33.291218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a4d416-cba1-4459-885d-851c63b8ee55 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.292196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a4d416-cba1-4459-885d-851c63b8ee55 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.292940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a4d416-cba1-4459-885d-851c63b8ee55 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.294451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a4d416-cba1-4459-885d-851c63b8ee55 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.294541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a4d416-cba1-4459-885d-851c63b8ee55 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.294908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a4d416-cba1-4459-885d-851c63b8ee55 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.326139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d1a4d416-cba1-4459-885d-851c63b8ee55 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:07:33.465135 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d1a4d416-cba1-4459-885d-851c63b8ee55 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 204 len: 0 microversion: 2.1 time: 0.191714 Sep 27 10:07:33.466396 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1509/2777] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:07:33 2022] DELETE /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:33.472213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-88ad29eb-9970-4106-a988-dbf52f52d156 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1167072180", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"boot_index": "0", "uuid": "6696cf01-1e97-4497-ade5-48bf358e8e2d", "source_type": "volume", "destination_type": "volume", "delete_on_termination": false}], "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:33.483291 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b7ee2202-fdda-4266-a0a2-9daed0808f20 tempest-ListImageFiltersTestJSON-361314172 tempest-ListImageFiltersTestJSON-361314172-project] 10.222.0.26 "DELETE /compute/v2.1/images/25e2dfe0-5b9d-4793-b9db-ea5bb05e74cf" status: 204 len: 0 microversion: 2.1 time: 0.310308 Sep 27 10:07:33.484125 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1269/2778] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:33 2022] DELETE /compute/v2.1/images/25e2dfe0-5b9d-4793-b9db-ea5bb05e74cf => generated 0 bytes in 312 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:33.493486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:33.495006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:33.504054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.505033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.505750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.507659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.509623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.511295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.557910 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-88ad29eb-9970-4106-a988-dbf52f52d156 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Block Device 6696cf01-1e97-4497-ade5-48bf358e8e2d is not bootable. Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-88ad29eb-9970-4106-a988-dbf52f52d156 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 358 Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1167072180", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"boot_index": "0", "uuid": "6696cf01-1e97-4497-ade5-48bf358e8e2d", "source_type": "volume", "destination_type": "volume", "delete_on_termination": false}], "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: exception: Block Device 6696cf01-1e97-4497-ade5-48bf358e8e2d is not bootable. Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1167072180', 'imageRef': '', 'flavorRef': '452', 'block_device_mapping_v2': [{'boot_index': 0, 'source_type': 'volume', 'destination_type': 'volume', 'delete_on_termination': False, 'volume_id': '6696cf01-1e97-4497-ade5-48bf358e8e2d'}], 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._create_instance( Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1672, in _create_instance Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi boot_meta = block_device.get_bdm_image_metadata( Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/block_device.py", line 664, in get_bdm_image_metadata Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InvalidBDMVolumeNotBootable(id=volume_id) Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InvalidBDMVolumeNotBootable: Block Device 6696cf01-1e97-4497-ade5-48bf358e8e2d is not bootable. Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:33.576774 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Block Device 6696cf01-1e97-4497-ade5-48bf358e8e2d is not bootable. Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Block Device 6696cf01-1e97-4497-ade5-48bf358e8e2d is not bootable. Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-88ad29eb-9970-4106-a988-dbf52f52d156 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Block Device 6696cf01-1e97-4497-ade5-48bf358e8e2d is not bootable. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-88ad29eb-9970-4106-a988-dbf52f52d156 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 110 microversion: 2.1 time: 0.110897 Sep 27 10:07:33.580448 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1510/2779] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:33 2022] POST /compute/v2.1/servers => generated 110 bytes in 112 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:07:33.588830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae03d1cd-fe8d-49dc-b514-83e2463c116a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:33.589591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ae03d1cd-fe8d-49dc-b514-83e2463c116a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:33.597676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae03d1cd-fe8d-49dc-b514-83e2463c116a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.598177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae03d1cd-fe8d-49dc-b514-83e2463c116a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.598625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae03d1cd-fe8d-49dc-b514-83e2463c116a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.599348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae03d1cd-fe8d-49dc-b514-83e2463c116a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.599812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae03d1cd-fe8d-49dc-b514-83e2463c116a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.600431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae03d1cd-fe8d-49dc-b514-83e2463c116a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.638973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:33.647576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:33.649112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:33.654998 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f6b574a8-671c-41ac-8eed-a9f1ccfc52a0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.169610 Sep 27 10:07:33.656204 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1270/2780] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:33 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:33.661015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1ab901e4-7a78-4dbc-b016-58be1ec27e5d req-e7f504cb-202d-47e6-9158-550150466009 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d9bf371a-9052-4e46-b718-75e61fa47db3", "tag": "3bdfdad8-1835-4508-a3f9-69be7eeab18d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:33.685051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1ab901e4-7a78-4dbc-b016-58be1ec27e5d req-e7f504cb-202d-47e6-9158-550150466009 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.685527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1ab901e4-7a78-4dbc-b016-58be1ec27e5d req-e7f504cb-202d-47e6-9158-550150466009 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.686223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1ab901e4-7a78-4dbc-b016-58be1ec27e5d req-e7f504cb-202d-47e6-9158-550150466009 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.701734 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1ab901e4-7a78-4dbc-b016-58be1ec27e5d req-e7f504cb-202d-47e6-9158-550150466009 service nova] Creating event network-changed:3bdfdad8-1835-4508-a3f9-69be7eeab18d for instance d9bf371a-9052-4e46-b718-75e61fa47db3 on nch1-master-1 Sep 27 10:07:33.707915 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1ab901e4-7a78-4dbc-b016-58be1ec27e5d req-e7f504cb-202d-47e6-9158-550150466009 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050248 Sep 27 10:07:33.709216 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1271/2781] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:33.716637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a47a0cf1-447d-4eb6-b506-ab1503575b84 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:33.717523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a47a0cf1-447d-4eb6-b506-ab1503575b84 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:33.720028 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ae03d1cd-fe8d-49dc-b514-83e2463c116a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e/action" status: 202 len: 0 microversion: 2.1 time: 0.139157 Sep 27 10:07:33.724140 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1511/2782] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:07:33 2022] POST /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e/action => generated 0 bytes in 144 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:07:33.724935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a47a0cf1-447d-4eb6-b506-ab1503575b84 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.725698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a47a0cf1-447d-4eb6-b506-ab1503575b84 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.726799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a47a0cf1-447d-4eb6-b506-ab1503575b84 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.727692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a47a0cf1-447d-4eb6-b506-ab1503575b84 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.728754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a47a0cf1-447d-4eb6-b506-ab1503575b84 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.729342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a47a0cf1-447d-4eb6-b506-ab1503575b84 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.733271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2aa2aec4-7c2d-4e9d-af95-d5cb3acd575d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:33.734206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2aa2aec4-7c2d-4e9d-af95-d5cb3acd575d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:33.744449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa2aec4-7c2d-4e9d-af95-d5cb3acd575d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.744971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa2aec4-7c2d-4e9d-af95-d5cb3acd575d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.745487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa2aec4-7c2d-4e9d-af95-d5cb3acd575d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.746208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa2aec4-7c2d-4e9d-af95-d5cb3acd575d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.746741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa2aec4-7c2d-4e9d-af95-d5cb3acd575d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.748391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa2aec4-7c2d-4e9d-af95-d5cb3acd575d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.917186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2aa2aec4-7c2d-4e9d-af95-d5cb3acd575d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:33.928160 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2aa2aec4-7c2d-4e9d-af95-d5cb3acd575d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.201062 Sep 27 10:07:33.941585 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1512/2783] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:33 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:33.941585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0833f310-abfb-4c4f-ab94-2a87f2a72c51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:33.941585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0833f310-abfb-4c4f-ab94-2a87f2a72c51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:33.944692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0833f310-abfb-4c4f-ab94-2a87f2a72c51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.944692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0833f310-abfb-4c4f-ab94-2a87f2a72c51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.944692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0833f310-abfb-4c4f-ab94-2a87f2a72c51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.949197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0833f310-abfb-4c4f-ab94-2a87f2a72c51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:33.949197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0833f310-abfb-4c4f-ab94-2a87f2a72c51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:33.949197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0833f310-abfb-4c4f-ab94-2a87f2a72c51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:33.972902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:34.185839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0833f310-abfb-4c4f-ab94-2a87f2a72c51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:34.210285 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0833f310-abfb-4c4f-ab94-2a87f2a72c51 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.279662 Sep 27 10:07:34.211760 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1513/2784] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:33 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:34.219029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-46e6f3a5-52ae-4db9-86f8-dc562eb20772 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:34.219869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-46e6f3a5-52ae-4db9-86f8-dc562eb20772 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:34.228560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46e6f3a5-52ae-4db9-86f8-dc562eb20772 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:34.229884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46e6f3a5-52ae-4db9-86f8-dc562eb20772 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:34.230644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46e6f3a5-52ae-4db9-86f8-dc562eb20772 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:34.231869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46e6f3a5-52ae-4db9-86f8-dc562eb20772 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:34.232488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46e6f3a5-52ae-4db9-86f8-dc562eb20772 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:34.233501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46e6f3a5-52ae-4db9-86f8-dc562eb20772 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:34.300409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a47a0cf1-447d-4eb6-b506-ab1503575b84 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:34.310130 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a47a0cf1-447d-4eb6-b506-ab1503575b84 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1646 microversion: 2.1 time: 0.599450 Sep 27 10:07:34.311291 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1272/2785] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:33 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1646 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:34.322179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7bd26cd9-90b7-48b4-84dd-cdf252ee9534 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:34.326452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7bd26cd9-90b7-48b4-84dd-cdf252ee9534 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:34.337110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bd26cd9-90b7-48b4-84dd-cdf252ee9534 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:34.340385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bd26cd9-90b7-48b4-84dd-cdf252ee9534 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:34.341091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bd26cd9-90b7-48b4-84dd-cdf252ee9534 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:34.342036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bd26cd9-90b7-48b4-84dd-cdf252ee9534 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:34.342671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bd26cd9-90b7-48b4-84dd-cdf252ee9534 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:34.343256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bd26cd9-90b7-48b4-84dd-cdf252ee9534 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:34.349553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:34.449441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-46e6f3a5-52ae-4db9-86f8-dc562eb20772 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:34.459292 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-46e6f3a5-52ae-4db9-86f8-dc562eb20772 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1324 microversion: 2.1 time: 0.246180 Sep 27 10:07:34.462676 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1514/2786] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:34 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1324 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:34.478057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e2dc963-b479-4afc-b691-fea8dc55b56e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:34.478682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8e2dc963-b479-4afc-b691-fea8dc55b56e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:34.486947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e2dc963-b479-4afc-b691-fea8dc55b56e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:34.487437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e2dc963-b479-4afc-b691-fea8dc55b56e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:34.487871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e2dc963-b479-4afc-b691-fea8dc55b56e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:34.488609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e2dc963-b479-4afc-b691-fea8dc55b56e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:34.489097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e2dc963-b479-4afc-b691-fea8dc55b56e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:34.489520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e2dc963-b479-4afc-b691-fea8dc55b56e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:34.639201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7bd26cd9-90b7-48b4-84dd-cdf252ee9534 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:34.652121 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7bd26cd9-90b7-48b4-84dd-cdf252ee9534 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1507 microversion: 2.1 time: 0.339136 Sep 27 10:07:34.655296 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1273/2787] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:34 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1507 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:34.664763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-84d6e655-5180-4af3-8bd0-65e3069c6264 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:34.665522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-84d6e655-5180-4af3-8bd0-65e3069c6264 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:34.672102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84d6e655-5180-4af3-8bd0-65e3069c6264 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:34.672895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84d6e655-5180-4af3-8bd0-65e3069c6264 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:34.673491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84d6e655-5180-4af3-8bd0-65e3069c6264 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:34.674488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84d6e655-5180-4af3-8bd0-65e3069c6264 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:34.675759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84d6e655-5180-4af3-8bd0-65e3069c6264 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:34.676231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84d6e655-5180-4af3-8bd0-65e3069c6264 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:34.716565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8e2dc963-b479-4afc-b691-fea8dc55b56e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:34.729748 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e2dc963-b479-4afc-b691-fea8dc55b56e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.265839 Sep 27 10:07:34.730701 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1515/2788] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:34 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:34.736054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:34.737286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:34.746431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:34.747927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:34.749312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:34.750372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:34.752560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:34.753957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:35.066975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-84d6e655-5180-4af3-8bd0-65e3069c6264 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:35.078783 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-84d6e655-5180-4af3-8bd0-65e3069c6264 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1314 microversion: 2.1 time: 0.422322 Sep 27 10:07:35.097298 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1274/2789] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:34 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1314 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:35.245521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9487f241-1fd9-4961-af67-b5c5f022f458 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:35.245521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9487f241-1fd9-4961-af67-b5c5f022f458 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:35.265601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9487f241-1fd9-4961-af67-b5c5f022f458 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:35.265601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9487f241-1fd9-4961-af67-b5c5f022f458 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:35.265930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9487f241-1fd9-4961-af67-b5c5f022f458 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:35.274791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9487f241-1fd9-4961-af67-b5c5f022f458 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:35.275305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9487f241-1fd9-4961-af67-b5c5f022f458 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:35.279351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9487f241-1fd9-4961-af67-b5c5f022f458 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:35.391306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:35.402961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:35.404759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:35.410879 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4f9f73ea-58f9-4763-91dc-75456fb3b73d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.678608 Sep 27 10:07:35.426019 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1516/2790] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:34 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 694 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:35.446554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0a5609ef-9446-40af-8499-c7d05012cc8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:35.449606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0a5609ef-9446-40af-8499-c7d05012cc8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:35.471336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a5609ef-9446-40af-8499-c7d05012cc8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:35.471849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a5609ef-9446-40af-8499-c7d05012cc8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:35.473551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a5609ef-9446-40af-8499-c7d05012cc8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:35.473780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a5609ef-9446-40af-8499-c7d05012cc8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:35.474592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a5609ef-9446-40af-8499-c7d05012cc8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:35.476094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a5609ef-9446-40af-8499-c7d05012cc8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:35.726912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0a5609ef-9446-40af-8499-c7d05012cc8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:35.738238 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0a5609ef-9446-40af-8499-c7d05012cc8a tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1646 microversion: 2.1 time: 0.310713 Sep 27 10:07:35.745592 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1517/2791] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:35 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1646 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:35.759167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-98858af5-7907-453e-bace-bf53c026e634 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:35.762968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-98858af5-7907-453e-bace-bf53c026e634 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:35.777799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98858af5-7907-453e-bace-bf53c026e634 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:35.778312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98858af5-7907-453e-bace-bf53c026e634 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:35.778776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98858af5-7907-453e-bace-bf53c026e634 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:35.779521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98858af5-7907-453e-bace-bf53c026e634 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:35.784863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98858af5-7907-453e-bace-bf53c026e634 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:35.786142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98858af5-7907-453e-bace-bf53c026e634 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.013399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:36.026420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-98858af5-7907-453e-bace-bf53c026e634 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:36.035382 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-98858af5-7907-453e-bace-bf53c026e634 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1324 microversion: 2.1 time: 0.288131 Sep 27 10:07:36.040894 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1518/2792] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:35 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1324 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:36.044849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e5110c3e-e853-49bd-9bdc-680ae51bc55a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:36.045620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e5110c3e-e853-49bd-9bdc-680ae51bc55a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:36.047730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=15 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:36.054585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5110c3e-e853-49bd-9bdc-680ae51bc55a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.055130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5110c3e-e853-49bd-9bdc-680ae51bc55a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.055873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5110c3e-e853-49bd-9bdc-680ae51bc55a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.057002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5110c3e-e853-49bd-9bdc-680ae51bc55a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.057606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5110c3e-e853-49bd-9bdc-680ae51bc55a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.058072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5110c3e-e853-49bd-9bdc-680ae51bc55a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.299328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e5110c3e-e853-49bd-9bdc-680ae51bc55a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:36.305924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9487f241-1fd9-4961-af67-b5c5f022f458 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:36.311103 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e5110c3e-e853-49bd-9bdc-680ae51bc55a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1487 microversion: 2.1 time: 0.273411 Sep 27 10:07:36.318355 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9487f241-1fd9-4961-af67-b5c5f022f458 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 1.080319 Sep 27 10:07:36.320678 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1275/2793] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:35 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 1083 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:36.325822 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1519/2794] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:36 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1487 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:36.326392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-6bfb669b-57d9-4067-8505-4c3deef8bf87 req-69d97da3-b8a5-4bb8-863e-6ea2f9a7b11c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa", "tag": "6431263b-bb42-4039-8d74-79f58ae6f0b2"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:36.337337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-65ce6e59-f981-42fc-831f-260e414d02d0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:36.340773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-65ce6e59-f981-42fc-831f-260e414d02d0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:36.343664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6bfb669b-57d9-4067-8505-4c3deef8bf87 req-69d97da3-b8a5-4bb8-863e-6ea2f9a7b11c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.344586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6bfb669b-57d9-4067-8505-4c3deef8bf87 req-69d97da3-b8a5-4bb8-863e-6ea2f9a7b11c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.345276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6bfb669b-57d9-4067-8505-4c3deef8bf87 req-69d97da3-b8a5-4bb8-863e-6ea2f9a7b11c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.348850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65ce6e59-f981-42fc-831f-260e414d02d0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.352478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65ce6e59-f981-42fc-831f-260e414d02d0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.353617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65ce6e59-f981-42fc-831f-260e414d02d0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.354397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65ce6e59-f981-42fc-831f-260e414d02d0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.354875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65ce6e59-f981-42fc-831f-260e414d02d0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.355572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65ce6e59-f981-42fc-831f-260e414d02d0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.412481 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-6bfb669b-57d9-4067-8505-4c3deef8bf87 req-69d97da3-b8a5-4bb8-863e-6ea2f9a7b11c service nova] Creating event network-changed:6431263b-bb42-4039-8d74-79f58ae6f0b2 for instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa on nch1-master-1 Sep 27 10:07:36.425528 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-6bfb669b-57d9-4067-8505-4c3deef8bf87 req-69d97da3-b8a5-4bb8-863e-6ea2f9a7b11c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.101159 Sep 27 10:07:36.427146 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1276/2795] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:36.443961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-051b57ec-bbd7-4b8d-8593-c721a4797e40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:36.446123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-051b57ec-bbd7-4b8d-8593-c721a4797e40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:36.462073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b57ec-bbd7-4b8d-8593-c721a4797e40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.462964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b57ec-bbd7-4b8d-8593-c721a4797e40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.463622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b57ec-bbd7-4b8d-8593-c721a4797e40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.469694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b57ec-bbd7-4b8d-8593-c721a4797e40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.470559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b57ec-bbd7-4b8d-8593-c721a4797e40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.471564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b57ec-bbd7-4b8d-8593-c721a4797e40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.504328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-051b57ec-bbd7-4b8d-8593-c721a4797e40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:07:36.575610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:36.576731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-65ce6e59-f981-42fc-831f-260e414d02d0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:36.590397 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-65ce6e59-f981-42fc-831f-260e414d02d0 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1314 microversion: 2.1 time: 0.263367 Sep 27 10:07:36.590993 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1520/2796] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:36 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1314 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:36.594647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:36.628599 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-051b57ec-bbd7-4b8d-8593-c721a4797e40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 204 len: 0 microversion: 2.1 time: 0.195741 Sep 27 10:07:36.631570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:36.632061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:36.634568 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1277/2797] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:07:36 2022] DELETE /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:36.641134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe8fad47-bceb-4c8c-98ef-6f4df89efbaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1511953531", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "block_device_mapping_v2": [{"source_type": "image", "delete_on_termination": true, "boot_index": 0, "uuid": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "destination_type": "local"}, {"source_type": "volume", "uuid": "40ba1110-077d-49a8-9222-bed471d90f62", "destination_type": "invalid"}], "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:36.646470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe8fad47-bceb-4c8c-98ef-6f4df89efbaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 495 Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1511953531", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "block_device_mapping_v2": [{"source_type": "image", "delete_on_termination": true, "boot_index": 0, "uuid": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "destination_type": "local"}, {"source_type": "volume", "uuid": "40ba1110-077d-49a8-9222-bed471d90f62", "destination_type": "invalid"}], "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1511953531', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'block_device_mapping_v2': [{'source_type': 'image', 'delete_on_termination': True, 'boot_index': 0, 'uuid': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'destination_type': 'local'}, {'source_type': 'volume', 'uuid': '40ba1110-077d-49a8-9222-bed471d90f62', 'destination_type': 'invalid'}], 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'invalid' is not one of ['local', 'volume'] Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'enum' in schema['properties']['server']['properties']['block_device_mapping_v2']['items']['properties']['destination_type']: Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'enum': ['local', 'volume'], 'type': 'string'} Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['server']['block_device_mapping_v2'][1]: Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'invalid' Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:36.646736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] Sep 27 10:07:36.650224 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:36.651481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.651567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe8fad47-bceb-4c8c-98ef-6f4df89efbaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Invalid input for field/attribute destination_type. Value: invalid. 'invalid' is not one of ['local', 'volume'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:36.651652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.651652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.651652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.651856 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fe8fad47-bceb-4c8c-98ef-6f4df89efbaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 155 microversion: 2.1 time: 0.013504 Sep 27 10:07:36.651856 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1278/2798] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:36 2022] POST /compute/v2.1/servers => generated 155 bytes in 15 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:07:36.653854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.657247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7c49ba11-1c8e-4fba-a8a3-89e46fcb0569 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:36.658269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7c49ba11-1c8e-4fba-a8a3-89e46fcb0569 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:36.667172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c49ba11-1c8e-4fba-a8a3-89e46fcb0569 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.667737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c49ba11-1c8e-4fba-a8a3-89e46fcb0569 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.668402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c49ba11-1c8e-4fba-a8a3-89e46fcb0569 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.669168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c49ba11-1c8e-4fba-a8a3-89e46fcb0569 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.669635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c49ba11-1c8e-4fba-a8a3-89e46fcb0569 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.670069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c49ba11-1c8e-4fba-a8a3-89e46fcb0569 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.743386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:36.756140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:36.758609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:36.765232 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-12b8b7cb-81b2-4efb-a0b6-9f1467c3ffd8 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.157586 Sep 27 10:07:36.767092 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1521/2799] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:36 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:36.771928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dab52134-b4ec-4a75-a111-279e8a23c45a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:36.772623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dab52134-b4ec-4a75-a111-279e8a23c45a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:36.779095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab52134-b4ec-4a75-a111-279e8a23c45a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.779831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab52134-b4ec-4a75-a111-279e8a23c45a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.780448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab52134-b4ec-4a75-a111-279e8a23c45a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.781409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab52134-b4ec-4a75-a111-279e8a23c45a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:36.782282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab52134-b4ec-4a75-a111-279e8a23c45a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:36.782892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab52134-b4ec-4a75-a111-279e8a23c45a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:36.991608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dab52134-b4ec-4a75-a111-279e8a23c45a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:37.006438 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dab52134-b4ec-4a75-a111-279e8a23c45a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.238063 Sep 27 10:07:37.007794 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1522/2800] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:36 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:37.012832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af4ead18-e6b4-47b3-be9b-64a0ec31ea93 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:37.013743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af4ead18-e6b4-47b3-be9b-64a0ec31ea93 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:37.020018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4ead18-e6b4-47b3-be9b-64a0ec31ea93 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:37.020862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4ead18-e6b4-47b3-be9b-64a0ec31ea93 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:37.021549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4ead18-e6b4-47b3-be9b-64a0ec31ea93 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:37.022758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4ead18-e6b4-47b3-be9b-64a0ec31ea93 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:37.023483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4ead18-e6b4-47b3-be9b-64a0ec31ea93 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:37.024136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af4ead18-e6b4-47b3-be9b-64a0ec31ea93 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:37.177670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7c49ba11-1c8e-4fba-a8a3-89e46fcb0569 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:37.189817 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7c49ba11-1c8e-4fba-a8a3-89e46fcb0569 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1493 microversion: 2.1 time: 0.535683 Sep 27 10:07:37.192979 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1279/2801] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:36 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1493 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:37.221935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-947c7759-dd67-44f7-b8da-974f0b2c5ebb tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:37.224337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-947c7759-dd67-44f7-b8da-974f0b2c5ebb tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:37.237066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-947c7759-dd67-44f7-b8da-974f0b2c5ebb tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:37.237943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-947c7759-dd67-44f7-b8da-974f0b2c5ebb tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:37.238515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-947c7759-dd67-44f7-b8da-974f0b2c5ebb tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:37.239537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-947c7759-dd67-44f7-b8da-974f0b2c5ebb tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:37.241528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-947c7759-dd67-44f7-b8da-974f0b2c5ebb tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:37.242499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-947c7759-dd67-44f7-b8da-974f0b2c5ebb tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:37.322797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af4ead18-e6b4-47b3-be9b-64a0ec31ea93 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:37.334796 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af4ead18-e6b4-47b3-be9b-64a0ec31ea93 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1646 microversion: 2.1 time: 0.325577 Sep 27 10:07:37.337244 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1523/2802] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:37 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1646 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:37.345401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7643c9b9-1f6c-4a3a-be4d-cc8899644ac0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:37.346411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7643c9b9-1f6c-4a3a-be4d-cc8899644ac0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:37.365500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7643c9b9-1f6c-4a3a-be4d-cc8899644ac0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:37.366153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7643c9b9-1f6c-4a3a-be4d-cc8899644ac0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:37.366698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7643c9b9-1f6c-4a3a-be4d-cc8899644ac0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:37.367753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7643c9b9-1f6c-4a3a-be4d-cc8899644ac0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:37.368443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7643c9b9-1f6c-4a3a-be4d-cc8899644ac0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:37.368945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7643c9b9-1f6c-4a3a-be4d-cc8899644ac0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:37.944328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-947c7759-dd67-44f7-b8da-974f0b2c5ebb tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:37.953698 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-947c7759-dd67-44f7-b8da-974f0b2c5ebb tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.748087 Sep 27 10:07:37.954332 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1280/2803] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:37 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:37.971038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1aba20a1-c28d-4a51-b525-105bfab7236c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:37.971788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1aba20a1-c28d-4a51-b525-105bfab7236c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:37.988227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1aba20a1-c28d-4a51-b525-105bfab7236c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:37.988760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1aba20a1-c28d-4a51-b525-105bfab7236c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:37.989200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1aba20a1-c28d-4a51-b525-105bfab7236c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:37.989926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1aba20a1-c28d-4a51-b525-105bfab7236c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:37.990381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1aba20a1-c28d-4a51-b525-105bfab7236c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:37.993590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1aba20a1-c28d-4a51-b525-105bfab7236c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.010653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7643c9b9-1f6c-4a3a-be4d-cc8899644ac0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:38.027747 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7643c9b9-1f6c-4a3a-be4d-cc8899644ac0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1324 microversion: 2.1 time: 0.688630 Sep 27 10:07:38.036747 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1524/2804] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:37 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1324 bytes in 691 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:38.047492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0c50a4f8-a7a4-4343-bfeb-31fff9b1402a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:38.047492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0c50a4f8-a7a4-4343-bfeb-31fff9b1402a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:38.047492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:38.050503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c50a4f8-a7a4-4343-bfeb-31fff9b1402a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.051334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c50a4f8-a7a4-4343-bfeb-31fff9b1402a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.051942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c50a4f8-a7a4-4343-bfeb-31fff9b1402a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.053802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c50a4f8-a7a4-4343-bfeb-31fff9b1402a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.054427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c50a4f8-a7a4-4343-bfeb-31fff9b1402a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.055158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c50a4f8-a7a4-4343-bfeb-31fff9b1402a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.293890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1aba20a1-c28d-4a51-b525-105bfab7236c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:38.306689 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1aba20a1-c28d-4a51-b525-105bfab7236c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.351209 Sep 27 10:07:38.307309 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1281/2805] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:37 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:38.312407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-81f959a5-5bc0-4b01-bbe5-0eb43083e4cd req-bbf0bcc3-19a2-430b-8f26-c6963d545cde service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b902a024-8aa7-4843-8680-41d3e189d3af", "name": "network-vif-plugged", "status": "completed", "tag": "df367b62-0029-488e-9897-fac934fc397e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:38.339904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-81f959a5-5bc0-4b01-bbe5-0eb43083e4cd req-bbf0bcc3-19a2-430b-8f26-c6963d545cde service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.340400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-81f959a5-5bc0-4b01-bbe5-0eb43083e4cd req-bbf0bcc3-19a2-430b-8f26-c6963d545cde service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.340887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-81f959a5-5bc0-4b01-bbe5-0eb43083e4cd req-bbf0bcc3-19a2-430b-8f26-c6963d545cde service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.370198 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-81f959a5-5bc0-4b01-bbe5-0eb43083e4cd req-bbf0bcc3-19a2-430b-8f26-c6963d545cde service nova] Creating event network-vif-plugged:df367b62-0029-488e-9897-fac934fc397e for instance b902a024-8aa7-4843-8680-41d3e189d3af on nch1-master-1 Sep 27 10:07:38.384617 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-81f959a5-5bc0-4b01-bbe5-0eb43083e4cd req-bbf0bcc3-19a2-430b-8f26-c6963d545cde service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075968 Sep 27 10:07:38.386040 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1282/2806] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:38.391488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:38.400461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:38.407780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.408287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.410884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.411703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.412216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.412634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.436905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0c50a4f8-a7a4-4343-bfeb-31fff9b1402a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:38.462050 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0c50a4f8-a7a4-4343-bfeb-31fff9b1402a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1314 microversion: 2.1 time: 0.431576 Sep 27 10:07:38.462990 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1525/2807] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:38 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1314 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:38.467809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f1668e6-c550-41c7-af6c-760967fbeb84 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1244628575", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"a": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}, "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f1668e6-c550-41c7-af6c-760967fbeb84 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 492 Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1244628575", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"a": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"}, "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1244628575', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'metadata': {'a': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'}, 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['server']['properties']['metadata']['patternProperties']['^[a-zA-Z0-9-_:. ]{1,255}$']: Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'maxLength': 255, 'type': 'string'} Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:38.472470 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['server']['metadata']: Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f1668e6-c550-41c7-af6c-760967fbeb84 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Invalid input for field/attribute a. Value: bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb. 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb' is too long {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:38.475566 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8f1668e6-c550-41c7-af6c-760967fbeb84 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 624 microversion: 2.1 time: 0.009962 Sep 27 10:07:38.478352 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1526/2808] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:38 2022] POST /compute/v2.1/servers => generated 624 bytes in 14 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:07:38.482979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8fd6eb5c-5a64-4c9c-a56c-c210cc5689bb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:38.483870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8fd6eb5c-5a64-4c9c-a56c-c210cc5689bb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:38.496541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fd6eb5c-5a64-4c9c-a56c-c210cc5689bb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.497341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fd6eb5c-5a64-4c9c-a56c-c210cc5689bb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.497950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fd6eb5c-5a64-4c9c-a56c-c210cc5689bb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.498844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fd6eb5c-5a64-4c9c-a56c-c210cc5689bb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.499469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fd6eb5c-5a64-4c9c-a56c-c210cc5689bb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.500092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fd6eb5c-5a64-4c9c-a56c-c210cc5689bb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.669782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:38.691550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:38.694295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:38.702219 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-30c8c39a-aec8-43d6-913c-185e75f47f20 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.314680 Sep 27 10:07:38.716851 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1283/2809] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:38 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:38.722602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:38.723369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:38.739214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.739981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.740601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.741619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.742328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.742968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.811834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.812826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.812826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.813261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lazy-loading 'fault' on Instance uuid e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:07:38.882115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8fd6eb5c-5a64-4c9c-a56c-c210cc5689bb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:38.898091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8fd6eb5c-5a64-4c9c-a56c-c210cc5689bb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1493 microversion: 2.1 time: 0.418789 Sep 27 10:07:38.899193 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1527/2810] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:38 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1493 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:38.904322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b2db5c93-f866-4e68-840d-63e4a9f86d63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:38.905250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b2db5c93-f866-4e68-840d-63e4a9f86d63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:38.923301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '866c1a2a2ca8452eb0a66cbe9692ab75', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '513655e40ccd4152a2e343e5415ce347', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:38.927623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2db5c93-f866-4e68-840d-63e4a9f86d63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.928428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2db5c93-f866-4e68-840d-63e4a9f86d63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.929242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2db5c93-f866-4e68-840d-63e4a9f86d63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.930215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2db5c93-f866-4e68-840d-63e4a9f86d63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:38.931013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2db5c93-f866-4e68-840d-63e4a9f86d63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:38.931632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2db5c93-f866-4e68-840d-63e4a9f86d63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:38.938471 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-63b436b5-26fa-4aa6-ae58-1a047cd339ff tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 200 len: 1305 microversion: 2.1 time: 0.220126 Sep 27 10:07:38.939080 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1284/2811] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:38 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 1305 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:38.973034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a51e056e-3b7c-45e5-ab0f-f89bbeac9e0c req-6f6a6d97-1075-427c-848c-de4433394e13 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191", "name": "network-vif-deleted", "tag": "16a06bd9-639b-4f11-8e0c-4ca53b9b60d5"}, {"name": "network-changed", "server_uuid": "768e537b-c8ab-418f-9523-ae18bd5a2259", "tag": "e7426e33-6ec8-40d9-a1de-120d41f1f5cc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:39.010228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a51e056e-3b7c-45e5-ab0f-f89bbeac9e0c req-6f6a6d97-1075-427c-848c-de4433394e13 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:39.011022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a51e056e-3b7c-45e5-ab0f-f89bbeac9e0c req-6f6a6d97-1075-427c-848c-de4433394e13 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:39.011463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a51e056e-3b7c-45e5-ab0f-f89bbeac9e0c req-6f6a6d97-1075-427c-848c-de4433394e13 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:39.031224 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a51e056e-3b7c-45e5-ab0f-f89bbeac9e0c req-6f6a6d97-1075-427c-848c-de4433394e13 service nova] Creating event network-vif-deleted:16a06bd9-639b-4f11-8e0c-4ca53b9b60d5 for instance e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 on nch2-master-1 Sep 27 10:07:39.032077 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a51e056e-3b7c-45e5-ab0f-f89bbeac9e0c req-6f6a6d97-1075-427c-848c-de4433394e13 service nova] Creating event network-changed:e7426e33-6ec8-40d9-a1de-120d41f1f5cc for instance 768e537b-c8ab-418f-9523-ae18bd5a2259 on nch2-master-1 Sep 27 10:07:39.047013 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a51e056e-3b7c-45e5-ab0f-f89bbeac9e0c req-6f6a6d97-1075-427c-848c-de4433394e13 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.104972 Sep 27 10:07:39.047665 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1285/2812] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:38 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:39.057463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-22452097-6a88-4fd5-81f6-900ade8989ab tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:39.058193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-22452097-6a88-4fd5-81f6-900ade8989ab tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:39.065355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22452097-6a88-4fd5-81f6-900ade8989ab tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:39.066021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22452097-6a88-4fd5-81f6-900ade8989ab tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:39.066467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22452097-6a88-4fd5-81f6-900ade8989ab tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:39.067510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22452097-6a88-4fd5-81f6-900ade8989ab tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:39.067979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22452097-6a88-4fd5-81f6-900ade8989ab tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:39.068409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-22452097-6a88-4fd5-81f6-900ade8989ab tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:39.432394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b2db5c93-f866-4e68-840d-63e4a9f86d63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:39.442962 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b2db5c93-f866-4e68-840d-63e4a9f86d63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.542359 Sep 27 10:07:39.443596 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1528/2813] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:38 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:39.450035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53f31423-6a43-464b-b4fc-ecf6326a5abd tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:39.450936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53f31423-6a43-464b-b4fc-ecf6326a5abd tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:39.463929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f31423-6a43-464b-b4fc-ecf6326a5abd tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:39.465142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f31423-6a43-464b-b4fc-ecf6326a5abd tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:39.466607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f31423-6a43-464b-b4fc-ecf6326a5abd tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:39.468084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f31423-6a43-464b-b4fc-ecf6326a5abd tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:39.469063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f31423-6a43-464b-b4fc-ecf6326a5abd tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:39.469907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f31423-6a43-464b-b4fc-ecf6326a5abd tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:39.782297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-53f31423-6a43-464b-b4fc-ecf6326a5abd tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:39.793368 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53f31423-6a43-464b-b4fc-ecf6326a5abd tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.346814 Sep 27 10:07:39.795016 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1529/2814] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:39 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:39.801975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53b613fb-5a65-4044-8f33-7a658fa2ee30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:39.802999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53b613fb-5a65-4044-8f33-7a658fa2ee30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:39.811162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53b613fb-5a65-4044-8f33-7a658fa2ee30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:39.812057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53b613fb-5a65-4044-8f33-7a658fa2ee30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:39.812768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53b613fb-5a65-4044-8f33-7a658fa2ee30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:39.823306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53b613fb-5a65-4044-8f33-7a658fa2ee30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:39.824312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53b613fb-5a65-4044-8f33-7a658fa2ee30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:39.824819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53b613fb-5a65-4044-8f33-7a658fa2ee30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:40.105253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-53b613fb-5a65-4044-8f33-7a658fa2ee30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:40.114791 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53b613fb-5a65-4044-8f33-7a658fa2ee30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.317980 Sep 27 10:07:40.115503 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1530/2815] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:39 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:40.121220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b1005a62-eaf5-4ff7-8fd0-ebbbc4e88188 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:40.122218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b1005a62-eaf5-4ff7-8fd0-ebbbc4e88188 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:40.142382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1005a62-eaf5-4ff7-8fd0-ebbbc4e88188 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:40.142883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1005a62-eaf5-4ff7-8fd0-ebbbc4e88188 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:40.143320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1005a62-eaf5-4ff7-8fd0-ebbbc4e88188 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:40.144033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1005a62-eaf5-4ff7-8fd0-ebbbc4e88188 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:40.144491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1005a62-eaf5-4ff7-8fd0-ebbbc4e88188 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:40.144948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1005a62-eaf5-4ff7-8fd0-ebbbc4e88188 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:40.150134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:40.246063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-22452097-6a88-4fd5-81f6-900ade8989ab tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:40.254107 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-22452097-6a88-4fd5-81f6-900ade8989ab tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1324 microversion: 2.1 time: 1.202731 Sep 27 10:07:40.255218 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1286/2816] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:39 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1324 bytes in 1204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:40.260187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:40.261034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:40.273833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:40.275115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:40.275973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:40.276974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:40.277689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:40.278353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:40.452390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b1005a62-eaf5-4ff7-8fd0-ebbbc4e88188 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:40.460805 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b1005a62-eaf5-4ff7-8fd0-ebbbc4e88188 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1314 microversion: 2.1 time: 0.343620 Sep 27 10:07:40.461575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:40.464992 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1531/2817] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:40 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1314 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:40.473131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-59ca3de6-0cfa-4fbe-b863-a46db24000aa tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:40.473765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-59ca3de6-0cfa-4fbe-b863-a46db24000aa tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:40.484939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:40.486948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:40.494349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59ca3de6-0cfa-4fbe-b863-a46db24000aa tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:40.495118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59ca3de6-0cfa-4fbe-b863-a46db24000aa tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:40.495904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59ca3de6-0cfa-4fbe-b863-a46db24000aa tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:40.496875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59ca3de6-0cfa-4fbe-b863-a46db24000aa tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:40.497344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59ca3de6-0cfa-4fbe-b863-a46db24000aa tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:40.497783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59ca3de6-0cfa-4fbe-b863-a46db24000aa tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:40.499131 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7c0c12c8-3c8e-441e-a5b6-9535c801dad2 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.242618 Sep 27 10:07:40.504986 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1287/2818] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:40 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:40.516771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:40.517546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] [instance: e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:40.531144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:40.531807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:40.532346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:40.533188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:40.533786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:40.543935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:40.566169 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] HTTP exception thrown: Instance e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 could not be found. Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Request method failure captured: Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 HTTP/1.1 Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsrXTckeXO0dA6YBRnQbSBftS_sZqz0YAGekER-Bg2xJKYcbFblXBzlBhCSv_oj7-wTEKdfZOmxpBfYwKrvLyOatBBh5bmS8cUmrAFfxrnGZOuLYMK7YyqYcmLNI-bA6qh6_Z-KxW6SWe9YH7PTgA1XEhv1pj9joy0a9mqFIBFkC3Ew Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/513655e40ccd4152a2e343e5415ce347"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_513655e40ccd4152a2e343e5415ce347"}]}] Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 513655e40ccd4152a2e343e5415ce347 Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-SecurityGroupsTestJSON-281219426 Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 866c1a2a2ca8452eb0a66cbe9692ab75 Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-SecurityGroupsTestJSON-281219426-project Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 could not be found. Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191'} Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 could not be found. Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 could not be found. Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:40.573801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.578905 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:40.578905 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:40.578905 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:40.578905 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:40.578905 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:40.578905 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 could not be found. Sep 27 10:07:40.578905 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.578905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Returning 404 to user: Instance e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:40.580892 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-14a08e1b-8861-44c3-8dad-71266919a422 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191" status: 404 len: 111 microversion: 2.1 time: 0.069197 Sep 27 10:07:40.585295 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1288/2819] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:40 2022] GET /compute/v2.1/servers/e71b4f0a-7aa0-40f9-88bc-4c5f92b6e191 => generated 111 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:40.596955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5016ad69-5ca0-4108-8d97-38695b299bd0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5016ad69-5ca0-4108-8d97-38695b299bd0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 416 Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['server']['properties']['name']: Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.612793 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:40.616005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5016ad69-5ca0-4108-8d97-38695b299bd0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:40.620572 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5016ad69-5ca0-4108-8d97-38695b299bd0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 619 microversion: 2.1 time: 0.024545 Sep 27 10:07:40.624580 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1289/2820] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:40 2022] POST /compute/v2.1/servers => generated 619 bytes in 32 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:07:40.639102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a363e4a8-b5ca-4c62-a826-b7326df6c4ad req-c51929c0-a2c6-4771-9986-97d83288b2ad service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9bf371a-9052-4e46-b718-75e61fa47db3", "name": "network-vif-plugged", "status": "completed", "tag": "3bdfdad8-1835-4508-a3f9-69be7eeab18d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:40.678718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a363e4a8-b5ca-4c62-a826-b7326df6c4ad req-c51929c0-a2c6-4771-9986-97d83288b2ad service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:40.679372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a363e4a8-b5ca-4c62-a826-b7326df6c4ad req-c51929c0-a2c6-4771-9986-97d83288b2ad service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:40.679913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a363e4a8-b5ca-4c62-a826-b7326df6c4ad req-c51929c0-a2c6-4771-9986-97d83288b2ad service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:40.692972 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a363e4a8-b5ca-4c62-a826-b7326df6c4ad req-c51929c0-a2c6-4771-9986-97d83288b2ad service nova] Creating event network-vif-plugged:3bdfdad8-1835-4508-a3f9-69be7eeab18d for instance d9bf371a-9052-4e46-b718-75e61fa47db3 on nch1-master-1 Sep 27 10:07:40.699046 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a363e4a8-b5ca-4c62-a826-b7326df6c4ad req-c51929c0-a2c6-4771-9986-97d83288b2ad service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064508 Sep 27 10:07:40.700050 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1290/2821] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:40.707340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0e93154e-5825-4f4e-9bc4-710d0f778d36 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f629ea820>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:40.920571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59ca3de6-0cfa-4fbe-b863-a46db24000aa tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:40.932449 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-59ca3de6-0cfa-4fbe-b863-a46db24000aa tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 200 len: 1493 microversion: 2.1 time: 0.466274 Sep 27 10:07:40.933424 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1532/2822] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:40 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 1493 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:40.966672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae4240df-d0b9-4a9b-b81a-e2425dc894c7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:40.968639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ae4240df-d0b9-4a9b-b81a-e2425dc894c7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:40.996276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae4240df-d0b9-4a9b-b81a-e2425dc894c7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:40.998611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae4240df-d0b9-4a9b-b81a-e2425dc894c7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:41.001438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae4240df-d0b9-4a9b-b81a-e2425dc894c7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:41.006248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae4240df-d0b9-4a9b-b81a-e2425dc894c7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:41.007295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae4240df-d0b9-4a9b-b81a-e2425dc894c7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:41.008172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae4240df-d0b9-4a9b-b81a-e2425dc894c7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:41.495600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ae4240df-d0b9-4a9b-b81a-e2425dc894c7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:41.513716 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ae4240df-d0b9-4a9b-b81a-e2425dc894c7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.557764 Sep 27 10:07:41.514685 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1533/2823] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:40 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:41.553254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cfe16fb2-034b-4775-9467-cd71d38d2853 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:41.557559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cfe16fb2-034b-4775-9467-cd71d38d2853 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:41.579607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfe16fb2-034b-4775-9467-cd71d38d2853 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:41.580968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfe16fb2-034b-4775-9467-cd71d38d2853 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:41.581653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfe16fb2-034b-4775-9467-cd71d38d2853 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:41.582836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfe16fb2-034b-4775-9467-cd71d38d2853 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:41.587860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfe16fb2-034b-4775-9467-cd71d38d2853 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:41.588352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfe16fb2-034b-4775-9467-cd71d38d2853 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:41.901576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cfe16fb2-034b-4775-9467-cd71d38d2853 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:41.916723 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cfe16fb2-034b-4775-9467-cd71d38d2853 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.380841 Sep 27 10:07:41.923318 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1534/2824] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:41 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:41.929996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-242f8d81-e85f-4e98-af2c-7275ba3fcbe9 req-12245be5-564f-4cfb-8984-36274373b6a8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "768e537b-c8ab-418f-9523-ae18bd5a2259", "tag": "f6f547a5-211e-44e6-9a66-e9913a99365d"}, {"server_uuid": "918832ca-5183-4a17-b0ed-01fc0c661f3e", "name": "network-vif-deleted", "tag": "6266abbd-ab22-410b-b499-c771bc142abf"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:41.959707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-242f8d81-e85f-4e98-af2c-7275ba3fcbe9 req-12245be5-564f-4cfb-8984-36274373b6a8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:41.960589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-242f8d81-e85f-4e98-af2c-7275ba3fcbe9 req-12245be5-564f-4cfb-8984-36274373b6a8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:41.961367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-242f8d81-e85f-4e98-af2c-7275ba3fcbe9 req-12245be5-564f-4cfb-8984-36274373b6a8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:41.999656 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-242f8d81-e85f-4e98-af2c-7275ba3fcbe9 req-12245be5-564f-4cfb-8984-36274373b6a8 service nova] Creating event network-changed:f6f547a5-211e-44e6-9a66-e9913a99365d for instance 768e537b-c8ab-418f-9523-ae18bd5a2259 on nch2-master-1 Sep 27 10:07:42.000487 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-242f8d81-e85f-4e98-af2c-7275ba3fcbe9 req-12245be5-564f-4cfb-8984-36274373b6a8 service nova] Creating event network-vif-deleted:6266abbd-ab22-410b-b499-c771bc142abf for instance 918832ca-5183-4a17-b0ed-01fc0c661f3e on nch2-master-1 Sep 27 10:07:42.016452 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-242f8d81-e85f-4e98-af2c-7275ba3fcbe9 req-12245be5-564f-4cfb-8984-36274373b6a8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.089492 Sep 27 10:07:42.017457 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1535/2825] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:41 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:42.031051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c66b2a99-776a-494b-913f-331fdfecb09a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:42.032016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c66b2a99-776a-494b-913f-331fdfecb09a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:42.051404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c66b2a99-776a-494b-913f-331fdfecb09a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:42.054880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c66b2a99-776a-494b-913f-331fdfecb09a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:42.055875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c66b2a99-776a-494b-913f-331fdfecb09a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:42.056919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c66b2a99-776a-494b-913f-331fdfecb09a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:42.059607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c66b2a99-776a-494b-913f-331fdfecb09a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:42.062013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c66b2a99-776a-494b-913f-331fdfecb09a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:42.475189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c66b2a99-776a-494b-913f-331fdfecb09a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:42.496433 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c66b2a99-776a-494b-913f-331fdfecb09a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1324 microversion: 2.1 time: 0.477481 Sep 27 10:07:42.498849 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1536/2826] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:42 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1324 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:42.517115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f06287c2-abad-4c37-95b4-055a1235f39a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:42.517757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f06287c2-abad-4c37-95b4-055a1235f39a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:42.532524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06287c2-abad-4c37-95b4-055a1235f39a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:42.533072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06287c2-abad-4c37-95b4-055a1235f39a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:42.533523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06287c2-abad-4c37-95b4-055a1235f39a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:42.534648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06287c2-abad-4c37-95b4-055a1235f39a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:42.535315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06287c2-abad-4c37-95b4-055a1235f39a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:42.535941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06287c2-abad-4c37-95b4-055a1235f39a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:42.598726 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0e93154e-5825-4f4e-9bc4-710d0f778d36 tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/abc5efc2-4989-4a63-9cad-dc80766b95b8" status: 202 len: 0 microversion: 2.1 time: 1.897136 Sep 27 10:07:42.603352 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1291/2827] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:40 2022] DELETE /compute/v2.1/os-security-groups/abc5efc2-4989-4a63-9cad-dc80766b95b8 => generated 0 bytes in 1902 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:07:42.615305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:42.617217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:42.630861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:42.635971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:42.636993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:42.638003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:42.641380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:42.642169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:42.770979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:42.784055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:42.787810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:42.805046 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-afd6d748-0463-415e-9227-a7893bd44a7a tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.199963 Sep 27 10:07:42.806680 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1292/2828] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:42 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:42.818916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-db426f2b-afce-4b2f-b6f7-aecdfbba045f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:42.819731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-db426f2b-afce-4b2f-b6f7-aecdfbba045f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:42.837673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db426f2b-afce-4b2f-b6f7-aecdfbba045f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:42.838579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db426f2b-afce-4b2f-b6f7-aecdfbba045f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:42.841799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db426f2b-afce-4b2f-b6f7-aecdfbba045f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:42.842773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db426f2b-afce-4b2f-b6f7-aecdfbba045f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:42.843545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db426f2b-afce-4b2f-b6f7-aecdfbba045f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:42.844166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db426f2b-afce-4b2f-b6f7-aecdfbba045f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:43.191114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f06287c2-abad-4c37-95b4-055a1235f39a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:43.202436 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f06287c2-abad-4c37-95b4-055a1235f39a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1314 microversion: 2.1 time: 0.702386 Sep 27 10:07:43.203058 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1537/2829] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:42 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1314 bytes in 704 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:43.209386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-db426f2b-afce-4b2f-b6f7-aecdfbba045f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:43.223859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:43.225223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:43.231115 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-db426f2b-afce-4b2f-b6f7-aecdfbba045f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.423026 Sep 27 10:07:43.234500 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1293/2830] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:42 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:43.241526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ff590356-fde4-472e-9c71-f668391ab31f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] Calling method '.version_select at 0x7f7f629b33a0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:43.253092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:43.259398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:43.268268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:43.280537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:43.281240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:43.281729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:43.313373 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e HTTP/1.1 Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '918832ca-5183-4a17-b0ed-01fc0c661f3e'} Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:43.315793 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:43.320932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:43.320932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:43.320932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:43.320932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:43.320932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:43.320932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:07:43.320932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:43.322533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:43.324179 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-02eb3889-6436-49f0-97dd-86a0a2e13f7c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 404 len: 111 microversion: 2.1 time: 0.120099 Sep 27 10:07:43.325240 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1538/2831] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:43 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 111 bytes in 122 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:43.336540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bde84604-5201-457d-8d60-bee0b57882a8 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:43.337575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bde84604-5201-457d-8d60-bee0b57882a8 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:43.345524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bde84604-5201-457d-8d60-bee0b57882a8 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:43.346308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bde84604-5201-457d-8d60-bee0b57882a8 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:43.347761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bde84604-5201-457d-8d60-bee0b57882a8 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:43.348687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bde84604-5201-457d-8d60-bee0b57882a8 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:43.349362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bde84604-5201-457d-8d60-bee0b57882a8 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:43.350067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bde84604-5201-457d-8d60-bee0b57882a8 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:43.605362 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ff590356-fde4-472e-9c71-f668391ab31f tempest-SecurityGroupsTestJSON-281219426 tempest-SecurityGroupsTestJSON-281219426-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/5a9156d9-5a31-4358-a05a-d0676691bcb7" status: 202 len: 0 microversion: 2.1 time: 0.369085 Sep 27 10:07:43.606024 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1294/2832] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:43 2022] DELETE /compute/v2.1/os-security-groups/5a9156d9-5a31-4358-a05a-d0676691bcb7 => generated 0 bytes in 371 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:07:43.612025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9695e74e-1e84-49bc-8a75-94606289fb11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-73408850", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": -1, "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:43.656316 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-9695e74e-1e84-49bc-8a75-94606289fb11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Flavor -1 could not be found. Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9695e74e-1e84-49bc-8a75-94606289fb11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 204 Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-73408850", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": -1, "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: exception: Flavor -1 could not be found. Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-73408850', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': -1, 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 778, in create Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id( Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor -1 could not be found. Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:43.667752 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor -1 could not be found. Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor -1 could not be found. Sep 27 10:07:43.671762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:07:43.682066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9695e74e-1e84-49bc-8a75-94606289fb11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Flavor -1 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:43.683750 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9695e74e-1e84-49bc-8a75-94606289fb11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 73 microversion: 2.1 time: 0.076382 Sep 27 10:07:43.688969 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1295/2833] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:43 2022] POST /compute/v2.1/servers => generated 73 bytes in 82 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:43.706739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3275d5a8-0fa8-4af2-a4f4-f5b375d56b94 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:43.708474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3275d5a8-0fa8-4af2-a4f4-f5b375d56b94 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:43.723196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3275d5a8-0fa8-4af2-a4f4-f5b375d56b94 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:43.723741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3275d5a8-0fa8-4af2-a4f4-f5b375d56b94 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:43.724175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3275d5a8-0fa8-4af2-a4f4-f5b375d56b94 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:43.725105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3275d5a8-0fa8-4af2-a4f4-f5b375d56b94 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:43.725757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3275d5a8-0fa8-4af2-a4f4-f5b375d56b94 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:43.726788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3275d5a8-0fa8-4af2-a4f4-f5b375d56b94 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:43.733879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:43.795092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bde84604-5201-457d-8d60-bee0b57882a8 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:43.809062 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bde84604-5201-457d-8d60-bee0b57882a8 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.482136 Sep 27 10:07:43.823428 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1539/2834] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:43 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:43.832167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-211141820", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "0117c46b-39f2-4cf5-9f8a-7710125bdf55"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:43.935998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] validate_networks() for [('0117c46b-39f2-4cf5-9f8a-7710125bdf55', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:07:44.051211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3275d5a8-0fa8-4af2-a4f4-f5b375d56b94 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:44.063438 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3275d5a8-0fa8-4af2-a4f4-f5b375d56b94 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1324 microversion: 2.1 time: 0.372555 Sep 27 10:07:44.064781 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1296/2835] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:43 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1324 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:44.071496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-888bcfff-3ccf-4f7b-89a0-009d39ddf54a req-9d3178eb-137a-408a-9ea9-60f1fc7839fb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa", "name": "network-vif-plugged", "status": "completed", "tag": "6431263b-bb42-4039-8d74-79f58ae6f0b2"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:44.088323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-888bcfff-3ccf-4f7b-89a0-009d39ddf54a req-9d3178eb-137a-408a-9ea9-60f1fc7839fb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:44.089512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-888bcfff-3ccf-4f7b-89a0-009d39ddf54a req-9d3178eb-137a-408a-9ea9-60f1fc7839fb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:44.091128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-888bcfff-3ccf-4f7b-89a0-009d39ddf54a req-9d3178eb-137a-408a-9ea9-60f1fc7839fb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:44.111274 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-888bcfff-3ccf-4f7b-89a0-009d39ddf54a req-9d3178eb-137a-408a-9ea9-60f1fc7839fb service nova] Creating event network-vif-plugged:6431263b-bb42-4039-8d74-79f58ae6f0b2 for instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa on nch1-master-1 Sep 27 10:07:44.118325 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-888bcfff-3ccf-4f7b-89a0-009d39ddf54a req-9d3178eb-137a-408a-9ea9-60f1fc7839fb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052236 Sep 27 10:07:44.119319 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1297/2836] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:44.129330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-797a00f0-6ae5-4bad-be08-54e9983a7f8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:44.130983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-797a00f0-6ae5-4bad-be08-54e9983a7f8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:44.140395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-797a00f0-6ae5-4bad-be08-54e9983a7f8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:44.141314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-797a00f0-6ae5-4bad-be08-54e9983a7f8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:44.142102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-797a00f0-6ae5-4bad-be08-54e9983a7f8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:44.143040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-797a00f0-6ae5-4bad-be08-54e9983a7f8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:44.143712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-797a00f0-6ae5-4bad-be08-54e9983a7f8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:44.144615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-797a00f0-6ae5-4bad-be08-54e9983a7f8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:44.495272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-797a00f0-6ae5-4bad-be08-54e9983a7f8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:44.511764 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-797a00f0-6ae5-4bad-be08-54e9983a7f8a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.390975 Sep 27 10:07:44.517781 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1298/2837] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:44 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:44.517781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:44.517781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:44.547125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:44.547730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:44.548963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:44.549785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:44.550884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:44.551588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:44.577440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:07:44.577871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:07:44.578358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:07:44.578779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:07:44.579204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:07:44.591633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:44.592130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:44.592577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:44.593356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:44.593838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:44.594274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:44.620197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:07:44.625679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for user 7f7c5adbac4f48b6bc141e0f888f2496 and project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:07:44.638949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:07:44.641354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:07:44.666654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:44.678386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:44.680340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:44.687688 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8580b9d6-f92e-4ca6-9df8-e747c762c40b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.173438 Sep 27 10:07:44.689248 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1299/2838] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:44 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:44.694616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7cf06196-c7ed-4d8c-923d-baf9c97aae9a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:44.695650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7cf06196-c7ed-4d8c-923d-baf9c97aae9a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:44.704435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cf06196-c7ed-4d8c-923d-baf9c97aae9a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:44.705190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cf06196-c7ed-4d8c-923d-baf9c97aae9a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:44.705645 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c7dbf769-cc13-416a-bdcd-5adb129a170d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.880778 Sep 27 10:07:44.706076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cf06196-c7ed-4d8c-923d-baf9c97aae9a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:44.706823 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1540/2839] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 883 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:07:44.707249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cf06196-c7ed-4d8c-923d-baf9c97aae9a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:44.708263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cf06196-c7ed-4d8c-923d-baf9c97aae9a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:44.708943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cf06196-c7ed-4d8c-923d-baf9c97aae9a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:44.902551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2fa58d94-5740-4d7f-996b-a58111ac62ad tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f628fe9d0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1108643468", "description": "tempest-description-147177845"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:45.003078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7cf06196-c7ed-4d8c-923d-baf9c97aae9a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:45.019449 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7cf06196-c7ed-4d8c-923d-baf9c97aae9a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1464 microversion: 2.1 time: 0.328588 Sep 27 10:07:45.022811 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1300/2840] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:44 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1464 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:45.037213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e5174a2-485e-4917-a145-85cf991978c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:45.041719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4e5174a2-485e-4917-a145-85cf991978c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:45.050372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e5174a2-485e-4917-a145-85cf991978c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:45.051163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e5174a2-485e-4917-a145-85cf991978c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:45.052635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e5174a2-485e-4917-a145-85cf991978c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:45.053595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e5174a2-485e-4917-a145-85cf991978c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:45.054222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e5174a2-485e-4917-a145-85cf991978c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:45.054909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e5174a2-485e-4917-a145-85cf991978c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:45.198493 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2fa58d94-5740-4d7f-996b-a58111ac62ad tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 247 microversion: 2.1 time: 0.490420 Sep 27 10:07:45.203390 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1541/2841] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:07:44 2022] POST /compute/v2.1/os-security-groups => generated 247 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:45.209448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b6c20653-86a3-4b80-bfe5-0737bc99ff01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:45.210361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b6c20653-86a3-4b80-bfe5-0737bc99ff01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:45.274856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b6c20653-86a3-4b80-bfe5-0737bc99ff01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:45.285392 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b6c20653-86a3-4b80-bfe5-0737bc99ff01 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1214 microversion: 2.1 time: 0.080249 Sep 27 10:07:45.286153 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1542/2842] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:45 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1214 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:45.309022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e7a4ed55-3fc7-4fc0-8d19-db9c44251d56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:45.311034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e7a4ed55-3fc7-4fc0-8d19-db9c44251d56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:45.319582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a4ed55-3fc7-4fc0-8d19-db9c44251d56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:45.320143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a4ed55-3fc7-4fc0-8d19-db9c44251d56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:45.321930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a4ed55-3fc7-4fc0-8d19-db9c44251d56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:45.323008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a4ed55-3fc7-4fc0-8d19-db9c44251d56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:45.323162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a4ed55-3fc7-4fc0-8d19-db9c44251d56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:45.323569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7a4ed55-3fc7-4fc0-8d19-db9c44251d56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:45.467299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4e5174a2-485e-4917-a145-85cf991978c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:45.476404 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4e5174a2-485e-4917-a145-85cf991978c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.452215 Sep 27 10:07:45.481950 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1301/2843] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:45 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:45.487026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c34694c3-11ae-4c9e-83ad-eaf70566dc3b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:45.493000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c34694c3-11ae-4c9e-83ad-eaf70566dc3b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:45.501285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e7a4ed55-3fc7-4fc0-8d19-db9c44251d56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:45.509801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:45.512037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c34694c3-11ae-4c9e-83ad-eaf70566dc3b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:45.512934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c34694c3-11ae-4c9e-83ad-eaf70566dc3b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:45.513437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c34694c3-11ae-4c9e-83ad-eaf70566dc3b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:45.519338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c34694c3-11ae-4c9e-83ad-eaf70566dc3b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:45.520024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c34694c3-11ae-4c9e-83ad-eaf70566dc3b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:45.520405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c34694c3-11ae-4c9e-83ad-eaf70566dc3b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:45.533386 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e7a4ed55-3fc7-4fc0-8d19-db9c44251d56 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.228195 Sep 27 10:07:45.534115 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1543/2844] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:45 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:45.547176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-64504af1-3280-44d4-8468-155651a72b3c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:45.548109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-64504af1-3280-44d4-8468-155651a72b3c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:45.555979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64504af1-3280-44d4-8468-155651a72b3c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:45.556952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64504af1-3280-44d4-8468-155651a72b3c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:45.557730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64504af1-3280-44d4-8468-155651a72b3c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:45.559413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64504af1-3280-44d4-8468-155651a72b3c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:45.560072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64504af1-3280-44d4-8468-155651a72b3c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:45.560878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64504af1-3280-44d4-8468-155651a72b3c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:45.769256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-64504af1-3280-44d4-8468-155651a72b3c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:45.778874 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-64504af1-3280-44d4-8468-155651a72b3c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1324 microversion: 2.1 time: 0.241737 Sep 27 10:07:45.779931 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1544/2845] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:45 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1324 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:45.785207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f753097e-20a4-4036-a3b6-9ebbe5e9777f tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f62bb1040>, body: {"security_group_rule": {"parent_group_id": "d26bfbb8-a573-4bf2-bd76-5d96aabdbdb1", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:46.042350 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f753097e-20a4-4036-a3b6-9ebbe5e9777f tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 232 microversion: 2.1 time: 0.261113 Sep 27 10:07:46.053668 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1545/2846] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:07:45 2022] POST /compute/v2.1/os-security-group-rules => generated 232 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:46.053668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c52e2ec2-ba1b-4054-89f2-68f1f3b15780 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-568689318", "imageRef": -1, "flavorRef": "452", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c52e2ec2-ba1b-4054-89f2-68f1f3b15780 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 172 Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-568689318", "imageRef": -1, "flavorRef": "452", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-568689318', 'imageRef': -1, 'flavorRef': '452', 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: -1 is not valid under any of the given schemas Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'oneOf' in schema['properties']['server']['properties']['imageRef']: Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'oneOf': [{'format': 'uuid', 'type': 'string'}, Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'maxLength': 0, 'type': 'string'}]} Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi -1 Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:07:46.057351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:46.062554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c52e2ec2-ba1b-4054-89f2-68f1f3b15780 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Invalid input for field/attribute imageRef. Value: -1. -1 is not valid under any of the given schemas {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:46.078077 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c52e2ec2-ba1b-4054-89f2-68f1f3b15780 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 145 microversion: 2.1 time: 0.016829 Sep 27 10:07:46.082206 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1546/2847] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:46 2022] POST /compute/v2.1/servers => generated 145 bytes in 38 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:07:46.087549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:46.088296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:46.105214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:46.105214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:46.105676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:46.107223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:46.107769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:46.108304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:46.181562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:46.198886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:46.199567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:46.206587 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c58b0e7f-b74f-41ad-badc-2c11f9d74824 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.123335 Sep 27 10:07:46.207511 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1547/2848] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:46 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:46.215356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99c93cf6-fbcb-45e8-8e32-e8b8d44e0a0c tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f63c28c10>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1910845586", "description": "tempest-description-1968575958"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:46.300321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c34694c3-11ae-4c9e-83ad-eaf70566dc3b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:46.310880 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c34694c3-11ae-4c9e-83ad-eaf70566dc3b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1462 microversion: 2.1 time: 0.827581 Sep 27 10:07:46.313495 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1302/2849] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:45 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1462 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:46.325875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eef8ee8d-0975-442e-8c29-d13ef9ac2589 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:46.326948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eef8ee8d-0975-442e-8c29-d13ef9ac2589 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:46.345641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eef8ee8d-0975-442e-8c29-d13ef9ac2589 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:46.346207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eef8ee8d-0975-442e-8c29-d13ef9ac2589 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:46.346658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eef8ee8d-0975-442e-8c29-d13ef9ac2589 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:46.347557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eef8ee8d-0975-442e-8c29-d13ef9ac2589 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:46.347851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eef8ee8d-0975-442e-8c29-d13ef9ac2589 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:46.348282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eef8ee8d-0975-442e-8c29-d13ef9ac2589 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:46.468928 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-99c93cf6-fbcb-45e8-8e32-e8b8d44e0a0c tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 248 microversion: 2.1 time: 0.260251 Sep 27 10:07:46.478289 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1548/2850] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:07:46 2022] POST /compute/v2.1/os-security-groups => generated 248 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:46.484116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30bb3714-63e4-4219-937e-d7b29662312b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:46.485004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-30bb3714-63e4-4219-937e-d7b29662312b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:46.492321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bb3714-63e4-4219-937e-d7b29662312b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:46.492863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bb3714-63e4-4219-937e-d7b29662312b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:46.493310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bb3714-63e4-4219-937e-d7b29662312b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:46.494028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bb3714-63e4-4219-937e-d7b29662312b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:46.494489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bb3714-63e4-4219-937e-d7b29662312b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:46.494915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30bb3714-63e4-4219-937e-d7b29662312b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:46.592243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-30bb3714-63e4-4219-937e-d7b29662312b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:46.604245 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-30bb3714-63e4-4219-937e-d7b29662312b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1266 microversion: 2.1 time: 0.124590 Sep 27 10:07:46.618251 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1549/2851] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:46 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1266 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:46.629778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7fabbbdd-7ab8-4694-a3d9-49e6495d4a4a tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f681e91f0>, body: {"security_group_rule": {"parent_group_id": "52056c00-1675-467c-aee9-0ee850a67899", "ip_protocol": "tcp", "from_port": 22, "to_port": 22, "cidr": "10.2.3.124/24"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:46.818074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eef8ee8d-0975-442e-8c29-d13ef9ac2589 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:46.840794 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eef8ee8d-0975-442e-8c29-d13ef9ac2589 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.521680 Sep 27 10:07:46.841794 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1303/2852] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:46 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:46.861418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminTestJSON-server-1855283807", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}, "os:scheduler_hints": {"same_host": "b902a024-8aa7-4843-8680-41d3e189d3af"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:46.882122 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7fabbbdd-7ab8-4694-a3d9-49e6495d4a4a tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 236 microversion: 2.1 time: 0.260926 Sep 27 10:07:46.883403 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1550/2853] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:07:46 2022] POST /compute/v2.1/os-security-group-rules => generated 236 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:46.888639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-002b54c7-6c42-4943-a5dc-a814518f2103 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:46.893197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-002b54c7-6c42-4943-a5dc-a814518f2103 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:46.900115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-002b54c7-6c42-4943-a5dc-a814518f2103 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:46.901174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-002b54c7-6c42-4943-a5dc-a814518f2103 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:46.902945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-002b54c7-6c42-4943-a5dc-a814518f2103 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:46.904112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-002b54c7-6c42-4943-a5dc-a814518f2103 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:46.905070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-002b54c7-6c42-4943-a5dc-a814518f2103 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:46.905754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-002b54c7-6c42-4943-a5dc-a814518f2103 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:46.945786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:07:47.081327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-002b54c7-6c42-4943-a5dc-a814518f2103 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:47.089969 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-002b54c7-6c42-4943-a5dc-a814518f2103 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.205218 Sep 27 10:07:47.100694 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1551/2854] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:46 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:47.100694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f7bfacd6-8d47-4427-8c4e-8b69771a31d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:47.100694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f7bfacd6-8d47-4427-8c4e-8b69771a31d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:47.102794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7bfacd6-8d47-4427-8c4e-8b69771a31d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:47.102993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7bfacd6-8d47-4427-8c4e-8b69771a31d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:47.108211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7bfacd6-8d47-4427-8c4e-8b69771a31d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:47.108211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7bfacd6-8d47-4427-8c4e-8b69771a31d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:47.108211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7bfacd6-8d47-4427-8c4e-8b69771a31d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:47.108211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7bfacd6-8d47-4427-8c4e-8b69771a31d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:47.279374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f7bfacd6-8d47-4427-8c4e-8b69771a31d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:47.287720 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f7bfacd6-8d47-4427-8c4e-8b69771a31d9 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1520 microversion: 2.1 time: 0.195997 Sep 27 10:07:47.295739 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1552/2855] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:47 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1520 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:47.295739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fdfbeaaa-f048-48a6-a6d6-6a4eebcd00af tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:47.295739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fdfbeaaa-f048-48a6-a6d6-6a4eebcd00af tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:47.301886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdfbeaaa-f048-48a6-a6d6-6a4eebcd00af tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:47.303263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdfbeaaa-f048-48a6-a6d6-6a4eebcd00af tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:47.314209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdfbeaaa-f048-48a6-a6d6-6a4eebcd00af tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:47.316583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdfbeaaa-f048-48a6-a6d6-6a4eebcd00af tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:47.317341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdfbeaaa-f048-48a6-a6d6-6a4eebcd00af tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:47.317987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdfbeaaa-f048-48a6-a6d6-6a4eebcd00af tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:47.532749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fdfbeaaa-f048-48a6-a6d6-6a4eebcd00af tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:47.541565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fdfbeaaa-f048-48a6-a6d6-6a4eebcd00af tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.251894 Sep 27 10:07:47.542161 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1553/2856] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:47 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:47.551598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0f4676d2-be96-4b8e-bd4c-787b57528395 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f62889940>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-949045513", "description": "tempest-description-871967423"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:47.867554 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0f4676d2-be96-4b8e-bd4c-787b57528395 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 246 microversion: 2.1 time: 0.321541 Sep 27 10:07:47.868312 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1554/2857] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:07:47 2022] POST /compute/v2.1/os-security-groups => generated 246 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:47.873455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:47.874078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:47.888874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:47.889387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:47.890529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:47.891464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:47.892081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:47.893235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:47.946699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:07:47.947700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:07:47.948342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:07:47.948966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:07:47.949559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:07:47.964774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:47.970222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:47.971059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:47.972094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:47.972743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:47.977856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:48.023392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Getting quotas for project 52efcec2c8a14640a8c3df1dc576bbf1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:07:48.026389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:48.034900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Getting quotas for user b4e0f20e52074142a3b525bafbb30c97 and project 52efcec2c8a14640a8c3df1dc576bbf1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:07:48.037300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:48.038987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:48.046031 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-21ca3d9c-7e76-4601-99e4-249cf01fedd9 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.176624 Sep 27 10:07:48.046764 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1555/2858] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:47 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:48.075761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d48ce4f4-5544-4a0e-9130-18c7c6ad7c61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:48.076462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d48ce4f4-5544-4a0e-9130-18c7c6ad7c61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:48.082753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:07:48.084255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d48ce4f4-5544-4a0e-9130-18c7c6ad7c61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:48.085247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:07:48.085629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d48ce4f4-5544-4a0e-9130-18c7c6ad7c61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:48.086492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d48ce4f4-5544-4a0e-9130-18c7c6ad7c61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:48.087446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d48ce4f4-5544-4a0e-9130-18c7c6ad7c61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:48.088075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d48ce4f4-5544-4a0e-9130-18c7c6ad7c61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:48.088724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d48ce4f4-5544-4a0e-9130-18c7c6ad7c61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:48.215739 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ef4152da-5d7e-495c-80f8-a125eefde80f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.372718 Sep 27 10:07:48.218674 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1304/2859] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 1377 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:07:48.230605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ec33552-841a-45e2-ae5c-1a5c5fd0c908 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:48.234859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ec33552-841a-45e2-ae5c-1a5c5fd0c908 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:48.263721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec33552-841a-45e2-ae5c-1a5c5fd0c908 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:48.277979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec33552-841a-45e2-ae5c-1a5c5fd0c908 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:48.278749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec33552-841a-45e2-ae5c-1a5c5fd0c908 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:48.279698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec33552-841a-45e2-ae5c-1a5c5fd0c908 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:48.285339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec33552-841a-45e2-ae5c-1a5c5fd0c908 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:48.286022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec33552-841a-45e2-ae5c-1a5c5fd0c908 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:48.422280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d48ce4f4-5544-4a0e-9130-18c7c6ad7c61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:48.424859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:48.450496 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d48ce4f4-5544-4a0e-9130-18c7c6ad7c61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1520 microversion: 2.1 time: 0.387276 Sep 27 10:07:48.451320 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1556/2860] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:48 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1520 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:48.472440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9a2cd04-7390-4d07-bf1e-143664403e52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-75237470", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"fixed_ip": "10.0.1.1", "uuid": "a-b-c-d-e-f-g-h-i-j"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:48.483497 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b9a2cd04-7390-4d07-bf1e-143664403e52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9a2cd04-7390-4d07-bf1e-143664403e52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 214 Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-75237470", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"fixed_ip": "10.0.1.1", "uuid": "a-b-c-d-e-f-g-h-i-j"}]}} Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: exception: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-75237470', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'networks': [{'fixed_ip': '10.0.1.1', 'uuid': 'a-b-c-d-e-f-g-h-i-j'}]}}} Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 773, in create Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._process_networks_for_create( Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 547, in _process_networks_for_create Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi requested_networks = self._get_requested_networks( Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 447, in _get_requested_networks Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._validate_network_id( Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 396, in _validate_network_id Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:48.490836 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:48.495329 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:48.495329 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:48.495329 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:48.495329 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:48.495329 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:48.495329 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:48.495329 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) Sep 27 10:07:48.495329 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:48.499445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9a2cd04-7390-4d07-bf1e-143664403e52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Bad networks format: network uuid is not in proper format (a-b-c-d-e-f-g-h-i-j) {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:48.501704 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b9a2cd04-7390-4d07-bf1e-143664403e52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 123 microversion: 2.1 time: 0.049433 Sep 27 10:07:48.504828 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1557/2861] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:48 2022] POST /compute/v2.1/servers => generated 123 bytes in 53 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:07:48.516313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-83271a03-13af-473c-af8a-976ea88f480e tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f62b8d5e0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1166095837", "description": "tempest-description-816136986"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:48.595556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2ec33552-841a-45e2-ae5c-1a5c5fd0c908 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:48.613784 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ec33552-841a-45e2-ae5c-1a5c5fd0c908 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1272 microversion: 2.1 time: 0.385471 Sep 27 10:07:48.630215 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1305/2862] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:48 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1272 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:48.639674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-756aaaa0-fba2-4df0-9bc8-5ab446fbb669 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:48.642691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-756aaaa0-fba2-4df0-9bc8-5ab446fbb669 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:48.659001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-756aaaa0-fba2-4df0-9bc8-5ab446fbb669 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:48.660390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-756aaaa0-fba2-4df0-9bc8-5ab446fbb669 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:48.662023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-756aaaa0-fba2-4df0-9bc8-5ab446fbb669 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:48.665138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-756aaaa0-fba2-4df0-9bc8-5ab446fbb669 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:48.667181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-756aaaa0-fba2-4df0-9bc8-5ab446fbb669 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:48.667868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-756aaaa0-fba2-4df0-9bc8-5ab446fbb669 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:48.834980 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-83271a03-13af-473c-af8a-976ea88f480e tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 247 microversion: 2.1 time: 0.328308 Sep 27 10:07:48.851810 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1558/2863] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:07:48 2022] POST /compute/v2.1/os-security-groups => generated 247 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:48.863362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53b94956-25c1-42a6-97b6-44abba9f1efc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:48.864005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53b94956-25c1-42a6-97b6-44abba9f1efc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:48.929043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-53b94956-25c1-42a6-97b6-44abba9f1efc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:48.937845 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53b94956-25c1-42a6-97b6-44abba9f1efc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1214 microversion: 2.1 time: 0.084398 Sep 27 10:07:48.938681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1559/2864] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:48 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1214 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:48.944435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d9e7e4da-c441-478a-9156-38175dd610c8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-ImagesOneServerNegativeTestJSON-image-1983499002"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:48.947144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d9e7e4da-c441-478a-9156-38175dd610c8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:48.954334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e7e4da-c441-478a-9156-38175dd610c8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:48.955078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e7e4da-c441-478a-9156-38175dd610c8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:48.955850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e7e4da-c441-478a-9156-38175dd610c8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:48.956802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e7e4da-c441-478a-9156-38175dd610c8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:48.957458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e7e4da-c441-478a-9156-38175dd610c8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:48.957942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e7e4da-c441-478a-9156-38175dd610c8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:49.049353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-756aaaa0-fba2-4df0-9bc8-5ab446fbb669 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:49.059452 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-756aaaa0-fba2-4df0-9bc8-5ab446fbb669 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.428138 Sep 27 10:07:49.079419 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1306/2865] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:48 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:49.092840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0ea3de59-9a85-42f3-9ac2-f0acba7769d8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:49.094376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0ea3de59-9a85-42f3-9ac2-f0acba7769d8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:49.110069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ea3de59-9a85-42f3-9ac2-f0acba7769d8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:49.111278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ea3de59-9a85-42f3-9ac2-f0acba7769d8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:49.116683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ea3de59-9a85-42f3-9ac2-f0acba7769d8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:49.119956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ea3de59-9a85-42f3-9ac2-f0acba7769d8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:49.120559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ea3de59-9a85-42f3-9ac2-f0acba7769d8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:49.121109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ea3de59-9a85-42f3-9ac2-f0acba7769d8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:49.423383 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d9e7e4da-c441-478a-9156-38175dd610c8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "POST /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa/action" status: 202 len: 0 microversion: 2.1 time: 0.483137 Sep 27 10:07:49.424477 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1560/2866] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:07:48 2022] POST /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa/action => generated 0 bytes in 485 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 10:07:49.432758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4a216c7-c6e7-48ca-a96f-4a557c9ff707 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f62889040>, body: {"security_group_rule": {"parent_group_id": "5920ee72-7918-4382-95c4-dfd3ff0c2510", "ip_protocol": "tcp", "from_port": 22, "to_port": 22, "group_id": "e1f2726d-87ed-4b6c-937b-4051572aad8c"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:49.838483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0ea3de59-9a85-42f3-9ac2-f0acba7769d8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:49.851074 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0ea3de59-9a85-42f3-9ac2-f0acba7769d8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.770197 Sep 27 10:07:49.861698 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1307/2867] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:49 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 782 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:49.882595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:49.883424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:49.894507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:49.895350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:49.896074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:49.897301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:49.898230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:49.898936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:49.967679 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4a216c7-c6e7-48ca-a96f-4a557c9ff707 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 331 microversion: 2.1 time: 0.541797 Sep 27 10:07:49.968301 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1561/2868] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:07:49 2022] POST /compute/v2.1/os-security-group-rules => generated 331 bytes in 543 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:49.974041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef1ed602-9e67-47fd-b875-44442f1b53da tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '.version_select at 0x7f7f63727040>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:50.121790 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef1ed602-9e67-47fd-b875-44442f1b53da tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "DELETE /compute/v2.1/images/c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb" status: 204 len: 0 microversion: 2.1 time: 0.151744 Sep 27 10:07:50.122365 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1562/2869] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:49 2022] DELETE /compute/v2.1/images/c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:50.128124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4277e45b-6232-435f-96b5-0b80f2c51e50 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:50.129169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4277e45b-6232-435f-96b5-0b80f2c51e50 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:50.139775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4277e45b-6232-435f-96b5-0b80f2c51e50 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:50.140243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4277e45b-6232-435f-96b5-0b80f2c51e50 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:50.140708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4277e45b-6232-435f-96b5-0b80f2c51e50 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:50.141439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4277e45b-6232-435f-96b5-0b80f2c51e50 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:50.141891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4277e45b-6232-435f-96b5-0b80f2c51e50 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:50.142318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4277e45b-6232-435f-96b5-0b80f2c51e50 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:50.294341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4277e45b-6232-435f-96b5-0b80f2c51e50 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:50.305529 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4277e45b-6232-435f-96b5-0b80f2c51e50 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1314 microversion: 2.1 time: 0.181387 Sep 27 10:07:50.307496 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1563/2870] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:50 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1314 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:50.313600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7ca82412-b651-47d4-9e1a-7683b5ff1570 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:50.315391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7ca82412-b651-47d4-9e1a-7683b5ff1570 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:50.323172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ca82412-b651-47d4-9e1a-7683b5ff1570 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:50.323850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ca82412-b651-47d4-9e1a-7683b5ff1570 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:50.324530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ca82412-b651-47d4-9e1a-7683b5ff1570 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:50.325379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ca82412-b651-47d4-9e1a-7683b5ff1570 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:50.326125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ca82412-b651-47d4-9e1a-7683b5ff1570 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:50.326653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ca82412-b651-47d4-9e1a-7683b5ff1570 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:50.414837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:50.426156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:50.432396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:50.438540 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3d82baca-f972-4edc-b9c3-3f4cb8033a7e tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1558 microversion: 2.91 time: 0.574052 Sep 27 10:07:50.439315 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1308/2871] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:49 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1558 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:50.444595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cf4a2f44-40d4-4ad7-b118-c035966fa377 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:50.445291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cf4a2f44-40d4-4ad7-b118-c035966fa377 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:50.453067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf4a2f44-40d4-4ad7-b118-c035966fa377 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:50.453720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf4a2f44-40d4-4ad7-b118-c035966fa377 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:50.454314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf4a2f44-40d4-4ad7-b118-c035966fa377 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:50.455320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf4a2f44-40d4-4ad7-b118-c035966fa377 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:50.455948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf4a2f44-40d4-4ad7-b118-c035966fa377 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:50.456519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf4a2f44-40d4-4ad7-b118-c035966fa377 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:50.512034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7ca82412-b651-47d4-9e1a-7683b5ff1570 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:50.522541 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7ca82412-b651-47d4-9e1a-7683b5ff1570 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.213501 Sep 27 10:07:50.523432 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1564/2872] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:50 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:50.528540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-707392cc-35ea-4eb0-b081-801fe273cb80 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f62819af0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-1143700286", "description": "tempest-description-2114476435"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:50.622716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cf4a2f44-40d4-4ad7-b118-c035966fa377 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:50.640493 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cf4a2f44-40d4-4ad7-b118-c035966fa377 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1274 microversion: 2.1 time: 0.194521 Sep 27 10:07:50.642931 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1309/2873] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:50 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1274 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:50.658484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-69d1be6a-a7a6-40a7-883e-1c6983dfeb6c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:50.659571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-69d1be6a-a7a6-40a7-883e-1c6983dfeb6c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:50.667946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d1be6a-a7a6-40a7-883e-1c6983dfeb6c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:50.668610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d1be6a-a7a6-40a7-883e-1c6983dfeb6c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:50.669423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d1be6a-a7a6-40a7-883e-1c6983dfeb6c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:50.671230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d1be6a-a7a6-40a7-883e-1c6983dfeb6c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:50.671858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d1be6a-a7a6-40a7-883e-1c6983dfeb6c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:50.672444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d1be6a-a7a6-40a7-883e-1c6983dfeb6c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:50.878378 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-707392cc-35ea-4eb0-b081-801fe273cb80 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 248 microversion: 2.1 time: 0.353472 Sep 27 10:07:50.879701 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1565/2874] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:07:50 2022] POST /compute/v2.1/os-security-groups => generated 248 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:50.885723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3c6679e2-91a2-4c93-8652-d8bda6334423 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '.version_select at 0x7f7f636c4af0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:50.915019 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-3c6679e2-91a2-4c93-8652-d8bda6334423 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] HTTP exception thrown: Image not found. Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3c6679e2-91a2-4c93-8652-d8bda6334423 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/images/c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb HTTP/1.1 Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: exception: Image not found. Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb'} Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: No image found with ID c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 86, in show Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._image_api.get(context, id) Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1205, in get Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return session.show(context, image_id, Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 287, in show Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _reraise_translated_image_exception(image_id) Sep 27 10:07:50.917352 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1031, in _reraise_translated_image_exception Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise new_exc.with_traceback(exc_trace) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 285, in show Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi image = self._client.call(context, 2, 'get', args=(image_id,)) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 197, in get Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get(image_id) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 190, in _get Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.get(url, headers=header) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 395, in get Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'GET', **kwargs) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb could not be found. Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 89, in show Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3c6679e2-91a2-4c93-8652-d8bda6334423 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 404 to user: Image not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3c6679e2-91a2-4c93-8652-d8bda6334423 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/images/c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb" status: 404 len: 62 microversion: 2.1 time: 0.037926 Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1566/2875] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:07:50 2022] GET /compute/v2.1/images/c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb => generated 62 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:07:50.926440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f44c121-5a0f-48e0-adb3-3f74ffc296da tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1109712703", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "key_name": "tempest-key-922082667", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:50.986047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-6f44c121-5a0f-48e0-adb3-3f74ffc296da tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] validate_networks() for [('fabc7738-7394-4084-bbfe-24c62c6029b4', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:07:51.071948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-69d1be6a-a7a6-40a7-883e-1c6983dfeb6c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:51.081744 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-69d1be6a-a7a6-40a7-883e-1c6983dfeb6c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.436540 Sep 27 10:07:51.093441 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1310/2876] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:50 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:51.106690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2d2a9df9-5c8f-461a-b6f9-d0ba9615440e tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f62977ca0>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-167880976", "description": "tempest-description-93974249"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:51.368523 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-6f44c121-5a0f-48e0-adb3-3f74ffc296da tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Invalid key_name provided. Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f44c121-5a0f-48e0-adb3-3f74ffc296da tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 246 Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1109712703", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "key_name": "tempest-key-922082667", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid key_name provided. Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1109712703', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'key_name': 'tempest-key-922082667', 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._create_instance( Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1682, in _create_instance Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ) = self._validate_and_build_base_options( Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1089, in _validate_and_build_base_options Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi key_pair = objects.KeyPair.get_by_name(context, Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-key-922082667 not found for user 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:51.373042 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 807, in create Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid key_name provided. Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid key_name provided. Sep 27 10:07:51.378661 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:51.381571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f44c121-5a0f-48e0-adb3-3f74ffc296da tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Invalid key_name provided. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:51.382686 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f44c121-5a0f-48e0-adb3-3f74ffc296da tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 70 microversion: 2.1 time: 0.462016 Sep 27 10:07:51.387872 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1567/2877] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:50 2022] POST /compute/v2.1/servers => generated 70 bytes in 468 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 10:07:51.403031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8ffd3fe4-96a1-4630-8636-b9e169c3f22b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:51.407093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8ffd3fe4-96a1-4630-8636-b9e169c3f22b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:51.419382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ffd3fe4-96a1-4630-8636-b9e169c3f22b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:51.422342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ffd3fe4-96a1-4630-8636-b9e169c3f22b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:51.423686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ffd3fe4-96a1-4630-8636-b9e169c3f22b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:51.426789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ffd3fe4-96a1-4630-8636-b9e169c3f22b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:51.427296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ffd3fe4-96a1-4630-8636-b9e169c3f22b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:51.427932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ffd3fe4-96a1-4630-8636-b9e169c3f22b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:51.677218 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2d2a9df9-5c8f-461a-b6f9-d0ba9615440e tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 245 microversion: 2.1 time: 0.582612 Sep 27 10:07:51.688208 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1311/2878] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:07:51 2022] POST /compute/v2.1/os-security-groups => generated 245 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:51.694227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-96b89d6d-15ec-4ed0-a513-0dcddc4bd7e4 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:51.697415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-96b89d6d-15ec-4ed0-a513-0dcddc4bd7e4 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:51.710842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96b89d6d-15ec-4ed0-a513-0dcddc4bd7e4 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:51.714890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96b89d6d-15ec-4ed0-a513-0dcddc4bd7e4 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:51.718020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96b89d6d-15ec-4ed0-a513-0dcddc4bd7e4 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:51.719357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96b89d6d-15ec-4ed0-a513-0dcddc4bd7e4 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:51.719973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96b89d6d-15ec-4ed0-a513-0dcddc4bd7e4 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:51.720518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96b89d6d-15ec-4ed0-a513-0dcddc4bd7e4 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:51.749978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8ffd3fe4-96a1-4630-8636-b9e169c3f22b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:51.767804 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8ffd3fe4-96a1-4630-8636-b9e169c3f22b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1538 microversion: 2.1 time: 0.378345 Sep 27 10:07:51.768633 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1568/2879] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:51 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1538 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:51.799181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4c9cf4b8-1468-4ecf-9cc4-24066eace908 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:51.799849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4c9cf4b8-1468-4ecf-9cc4-24066eace908 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:51.818305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9cf4b8-1468-4ecf-9cc4-24066eace908 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:51.820466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9cf4b8-1468-4ecf-9cc4-24066eace908 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:51.827196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9cf4b8-1468-4ecf-9cc4-24066eace908 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:51.829941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9cf4b8-1468-4ecf-9cc4-24066eace908 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:51.830769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9cf4b8-1468-4ecf-9cc4-24066eace908 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:51.832629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c9cf4b8-1468-4ecf-9cc4-24066eace908 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:52.004388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-96b89d6d-15ec-4ed0-a513-0dcddc4bd7e4 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:52.019967 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-96b89d6d-15ec-4ed0-a513-0dcddc4bd7e4 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1314 microversion: 2.1 time: 0.327064 Sep 27 10:07:52.021730 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1312/2880] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:51 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1314 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:52.037494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:52.038994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:52.048534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:52.055177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:52.055913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:52.056869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:52.059528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:52.060173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:52.181200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4c9cf4b8-1468-4ecf-9cc4-24066eace908 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:52.194045 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4c9cf4b8-1468-4ecf-9cc4-24066eace908 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.412871 Sep 27 10:07:52.195066 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1569/2881] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:51 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:52.201071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-356fd2c0-1cce-4416-9d15-e348c1f6e708 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:52.208606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-356fd2c0-1cce-4416-9d15-e348c1f6e708 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:52.224969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-356fd2c0-1cce-4416-9d15-e348c1f6e708 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:52.226927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-356fd2c0-1cce-4416-9d15-e348c1f6e708 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:52.228633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-356fd2c0-1cce-4416-9d15-e348c1f6e708 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:52.231683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-356fd2c0-1cce-4416-9d15-e348c1f6e708 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:52.233370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-356fd2c0-1cce-4416-9d15-e348c1f6e708 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:52.237291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-356fd2c0-1cce-4416-9d15-e348c1f6e708 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:52.762482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-356fd2c0-1cce-4416-9d15-e348c1f6e708 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:52.771283 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-356fd2c0-1cce-4416-9d15-e348c1f6e708 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1314 microversion: 2.1 time: 0.574960 Sep 27 10:07:52.776006 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1570/2882] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:52 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1314 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:52.788463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9da4f22d-059d-4138-8957-5d676abe3ada tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f638f2dc0>, body: {"security_group_rule": {"parent_group_id": "7e9448f1-b28d-4c62-90d3-b73a1f196cc7", "ip_protocol": "tcp", "from_port": 22, "to_port": 22, "group_id": "42ad5834-b3e8-4d03-b36a-217fe275f5eb"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:53.136824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:53.156565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:53.164928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:53.178750 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-05dd9bac-82ce-49b6-8c7e-5fbd5b0d5f26 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 1.152337 Sep 27 10:07:53.193466 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1313/2883] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:52 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 1168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:53.207475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d6306910-7bbe-4166-9648-f07e3bcfe9a1 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:53.208101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d6306910-7bbe-4166-9648-f07e3bcfe9a1 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:53.215174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6306910-7bbe-4166-9648-f07e3bcfe9a1 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:53.215749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6306910-7bbe-4166-9648-f07e3bcfe9a1 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:53.216191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6306910-7bbe-4166-9648-f07e3bcfe9a1 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:53.217028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6306910-7bbe-4166-9648-f07e3bcfe9a1 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:53.217522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6306910-7bbe-4166-9648-f07e3bcfe9a1 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:53.217945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6306910-7bbe-4166-9648-f07e3bcfe9a1 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:53.260960 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9da4f22d-059d-4138-8957-5d676abe3ada tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 330 microversion: 2.1 time: 0.478940 Sep 27 10:07:53.269945 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1571/2884] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:07:52 2022] POST /compute/v2.1/os-security-group-rules => generated 330 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:53.275899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6dafc69f-6b34-4d96-8d3f-a047e63d9f5f req-d5aafba7-17ca-470d-829f-8a448be2b505 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "768e537b-c8ab-418f-9523-ae18bd5a2259", "name": "network-vif-plugged", "status": "completed", "tag": "f6f547a5-211e-44e6-9a66-e9913a99365d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:53.302008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6dafc69f-6b34-4d96-8d3f-a047e63d9f5f req-d5aafba7-17ca-470d-829f-8a448be2b505 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:53.302907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6dafc69f-6b34-4d96-8d3f-a047e63d9f5f req-d5aafba7-17ca-470d-829f-8a448be2b505 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:53.303528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6dafc69f-6b34-4d96-8d3f-a047e63d9f5f req-d5aafba7-17ca-470d-829f-8a448be2b505 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:53.332383 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6dafc69f-6b34-4d96-8d3f-a047e63d9f5f req-d5aafba7-17ca-470d-829f-8a448be2b505 service nova] Creating event network-vif-plugged:f6f547a5-211e-44e6-9a66-e9913a99365d for instance 768e537b-c8ab-418f-9523-ae18bd5a2259 on nch2-master-1 Sep 27 10:07:53.341515 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6dafc69f-6b34-4d96-8d3f-a047e63d9f5f req-d5aafba7-17ca-470d-829f-8a448be2b505 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069637 Sep 27 10:07:53.342850 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1572/2885] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:53.356775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d27df588-22a1-4393-b0a0-2614aac575d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:53.358954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d27df588-22a1-4393-b0a0-2614aac575d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:53.376911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d27df588-22a1-4393-b0a0-2614aac575d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:53.378022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d27df588-22a1-4393-b0a0-2614aac575d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:53.378881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d27df588-22a1-4393-b0a0-2614aac575d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:53.379798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d27df588-22a1-4393-b0a0-2614aac575d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:53.380484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d27df588-22a1-4393-b0a0-2614aac575d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:53.381115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d27df588-22a1-4393-b0a0-2614aac575d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:53.679971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d27df588-22a1-4393-b0a0-2614aac575d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:53.700067 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d27df588-22a1-4393-b0a0-2614aac575d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.351485 Sep 27 10:07:53.702188 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1573/2886] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:53 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:53.716441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d6306910-7bbe-4166-9648-f07e3bcfe9a1 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:53.719978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-020adb2f-748e-43c9-bfb3-67d92f1c9027 req-87ec9867-8734-478d-be50-26b18e73e78d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "08babbce-373e-4cf5-a138-066b6eeb02c3", "tag": "2ca9e80c-8a06-414a-81cf-a330f210be36"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:53.742304 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d6306910-7bbe-4166-9648-f07e3bcfe9a1 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.541243 Sep 27 10:07:53.750585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-020adb2f-748e-43c9-bfb3-67d92f1c9027 req-87ec9867-8734-478d-be50-26b18e73e78d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:53.756274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-020adb2f-748e-43c9-bfb3-67d92f1c9027 req-87ec9867-8734-478d-be50-26b18e73e78d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:53.757526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-020adb2f-748e-43c9-bfb3-67d92f1c9027 req-87ec9867-8734-478d-be50-26b18e73e78d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:53.763882 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1314/2887] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:53 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:53.782811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c8d9fdc3-7cf7-489b-a9e1-d68513d8c100 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:53.785998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c8d9fdc3-7cf7-489b-a9e1-d68513d8c100 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:53.809701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8d9fdc3-7cf7-489b-a9e1-d68513d8c100 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:53.812523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8d9fdc3-7cf7-489b-a9e1-d68513d8c100 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:53.816850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8d9fdc3-7cf7-489b-a9e1-d68513d8c100 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:53.819663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8d9fdc3-7cf7-489b-a9e1-d68513d8c100 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:53.821522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8d9fdc3-7cf7-489b-a9e1-d68513d8c100 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:53.822217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8d9fdc3-7cf7-489b-a9e1-d68513d8c100 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:53.841590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:53.880888 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-020adb2f-748e-43c9-bfb3-67d92f1c9027 req-87ec9867-8734-478d-be50-26b18e73e78d service nova] Creating event network-changed:2ca9e80c-8a06-414a-81cf-a330f210be36 for instance 08babbce-373e-4cf5-a138-066b6eeb02c3 on nch2-master-1 Sep 27 10:07:53.944421 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-020adb2f-748e-43c9-bfb3-67d92f1c9027 req-87ec9867-8734-478d-be50-26b18e73e78d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.240670 Sep 27 10:07:53.947681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1574/2888] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:53.956521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba61ef47-03a2-49c0-b465-f4a123e2324d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:53.958771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba61ef47-03a2-49c0-b465-f4a123e2324d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:53.969606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba61ef47-03a2-49c0-b465-f4a123e2324d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:53.970405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba61ef47-03a2-49c0-b465-f4a123e2324d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:53.971695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba61ef47-03a2-49c0-b465-f4a123e2324d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:53.972605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba61ef47-03a2-49c0-b465-f4a123e2324d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:53.973390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba61ef47-03a2-49c0-b465-f4a123e2324d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:53.974621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba61ef47-03a2-49c0-b465-f4a123e2324d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:54.224781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba61ef47-03a2-49c0-b465-f4a123e2324d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:54.235656 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba61ef47-03a2-49c0-b465-f4a123e2324d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1314 microversion: 2.1 time: 0.286222 Sep 27 10:07:54.236906 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1575/2889] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:53 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1314 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:54.244899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-81bbdd17-2ec6-4fa4-a287-c98c7e887100 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Calling method '.version_select at 0x7f7f63aa3280>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:54.303307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c8d9fdc3-7cf7-489b-a9e1-d68513d8c100 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:54.324542 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c8d9fdc3-7cf7-489b-a9e1-d68513d8c100 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1538 microversion: 2.1 time: 0.558270 Sep 27 10:07:54.329298 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1315/2890] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:53 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1538 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:54.355630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-54c8beea-ccda-4385-b8d4-dfade9a17e2a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:54.362058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-54c8beea-ccda-4385-b8d4-dfade9a17e2a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:54.374319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54c8beea-ccda-4385-b8d4-dfade9a17e2a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:54.376557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54c8beea-ccda-4385-b8d4-dfade9a17e2a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:54.377266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54c8beea-ccda-4385-b8d4-dfade9a17e2a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:54.378695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54c8beea-ccda-4385-b8d4-dfade9a17e2a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:54.382772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54c8beea-ccda-4385-b8d4-dfade9a17e2a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:54.386947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54c8beea-ccda-4385-b8d4-dfade9a17e2a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:54.585182 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-81bbdd17-2ec6-4fa4-a287-c98c7e887100 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/42ad5834-b3e8-4d03-b36a-217fe275f5eb" status: 202 len: 0 microversion: 2.1 time: 0.346618 Sep 27 10:07:54.587112 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1576/2891] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:07:54 2022] DELETE /compute/v2.1/os-security-groups/42ad5834-b3e8-4d03-b36a-217fe275f5eb => generated 0 bytes in 349 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:07:54.597374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e2418ef-ed51-4192-b1df-c15c8102fa14 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-500342840", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "security_groups": [{"name": "does_not_exist"}], "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:54.688060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:54.844305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.security_group_api [None req-0e2418ef-ed51-4192-b1df-c15c8102fa14 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Neutron security group does_not_exist not found {{(pid=145968) validate_name /opt/stack/nova/nova/network/security_group_api.py:74}} Sep 27 10:07:54.845255 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-0e2418ef-ed51-4192-b1df-c15c8102fa14 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Unable to find security_group with name or id 'does_not_exist' Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e2418ef-ed51-4192-b1df-c15c8102fa14 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 257 Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-500342840", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "security_groups": [{"name": "does_not_exist"}], "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: exception: Unable to find security_group with name or id 'does_not_exist' Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-500342840', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'security_groups': [{'name': 'does_not_exist'}], 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 476, in find_resource Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.find_resource_by_id(resource, name_or_id, Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 440, in find_resource_by_id Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound(message=not_found_message) Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with id 'does_not_exist' Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 480, in find_resource Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._find_resource_by_name( Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 469, in _find_resource_by_name Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound(message=not_found_message) Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with name 'does_not_exist' Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 68, in validate_name Sep 27 10:07:54.857092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return neutronv20.find_resourceid_by_name_or_id( Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 56, in find_resourceid_by_name_or_id Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return find_resource_by_name_or_id(client, resource, name_or_id, Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/neutron/v2_0/__init__.py", line 49, in find_resource_by_name_or_id Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return client.find_resource(resource, name_or_id, project_id, Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 488, in find_resource Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exceptions.NotFound( Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Unable to find security_group with name or id 'does_not_exist' Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._create_instance( Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1682, in _create_instance Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ) = self._validate_and_build_base_options( Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1074, in _validate_and_build_base_options Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_groups = self._check_requested_secgroups( Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 521, in _check_requested_secgroups Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi secgroup_uuid = security_group_api.validate_name(context, secgroup) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 75, in validate_name Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Unable to find security_group with name or id 'does_not_exist' Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 866, in create Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=error.format_message()) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Unable to find security_group with name or id 'does_not_exist' Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Unable to find security_group with name or id 'does_not_exist' Sep 27 10:07:54.861811 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:54.867394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e2418ef-ed51-4192-b1df-c15c8102fa14 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Unable to find security_group with name or id 'does_not_exist' {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:54.868844 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e2418ef-ed51-4192-b1df-c15c8102fa14 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 106 microversion: 2.1 time: 0.280200 Sep 27 10:07:54.870432 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1577/2892] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:07:54 2022] POST /compute/v2.1/servers => generated 106 bytes in 282 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:07:54.877745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:54.878369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:54.883537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-54c8beea-ccda-4385-b8d4-dfade9a17e2a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:54.889454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:54.890987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:54.891483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:54.892460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:54.893773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:54.894814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:54.895465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:54.904259 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-54c8beea-ccda-4385-b8d4-dfade9a17e2a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1314 microversion: 2.1 time: 0.551920 Sep 27 10:07:54.910755 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1316/2893] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:54 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1314 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:54.932824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-de4d7c2d-ef11-4c5a-a627-f59a49bbab3d tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Calling method '.version_select at 0x7f7f629d81f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:55.014865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:55.026809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:55.028897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:55.035698 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a9f2fb43-1eb0-410e-b67b-db87421d58b5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.163778 Sep 27 10:07:55.037367 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1578/2894] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:54 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:55.043918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5ce05cdd-cf0b-4ed0-b98c-684962a87157 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:55.044909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5ce05cdd-cf0b-4ed0-b98c-684962a87157 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:55.056617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ce05cdd-cf0b-4ed0-b98c-684962a87157 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:55.057337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ce05cdd-cf0b-4ed0-b98c-684962a87157 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:55.057967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ce05cdd-cf0b-4ed0-b98c-684962a87157 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:55.060453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ce05cdd-cf0b-4ed0-b98c-684962a87157 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:55.061178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ce05cdd-cf0b-4ed0-b98c-684962a87157 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:55.061933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ce05cdd-cf0b-4ed0-b98c-684962a87157 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:55.084986 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-de4d7c2d-ef11-4c5a-a627-f59a49bbab3d tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups/7e9448f1-b28d-4c62-90d3-b73a1f196cc7" status: 200 len: 248 microversion: 2.1 time: 0.172976 Sep 27 10:07:55.086398 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1317/2895] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:07:54 2022] GET /compute/v2.1/os-security-groups/7e9448f1-b28d-4c62-90d3-b73a1f196cc7 => generated 248 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:55.092639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cba75627-5660-4343-bd16-57fa89b2c5a2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:55.093505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cba75627-5660-4343-bd16-57fa89b2c5a2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:55.118267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba75627-5660-4343-bd16-57fa89b2c5a2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:55.118907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba75627-5660-4343-bd16-57fa89b2c5a2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:55.119686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba75627-5660-4343-bd16-57fa89b2c5a2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:55.121539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba75627-5660-4343-bd16-57fa89b2c5a2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:55.123192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba75627-5660-4343-bd16-57fa89b2c5a2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:55.124639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cba75627-5660-4343-bd16-57fa89b2c5a2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:55.362726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5ce05cdd-cf0b-4ed0-b98c-684962a87157 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:55.375013 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5ce05cdd-cf0b-4ed0-b98c-684962a87157 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.336127 Sep 27 10:07:55.388960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cba75627-5660-4343-bd16-57fa89b2c5a2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:55.392823 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1579/2896] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:55 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:55.405226 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cba75627-5660-4343-bd16-57fa89b2c5a2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.316549 Sep 27 10:07:55.406429 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1318/2897] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:55 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:55.416203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c571b766-b786-4ea1-9a1e-60617a749035 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:55.417127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c571b766-b786-4ea1-9a1e-60617a749035 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:55.418919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed7b243b-74a3-41de-9b01-4dad5faa9c18 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f62936c10>, body: {"security_group": {"name": "tempest-SecurityGroupRulesTestJSON-securitygroup-607243849", "description": "tempest-description-2009148287"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:55.435573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c571b766-b786-4ea1-9a1e-60617a749035 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:55.436510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c571b766-b786-4ea1-9a1e-60617a749035 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:55.437731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c571b766-b786-4ea1-9a1e-60617a749035 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:55.438768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c571b766-b786-4ea1-9a1e-60617a749035 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:55.440621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c571b766-b786-4ea1-9a1e-60617a749035 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:55.441330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c571b766-b786-4ea1-9a1e-60617a749035 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:55.737960 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed7b243b-74a3-41de-9b01-4dad5faa9c18 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 247 microversion: 2.1 time: 0.340822 Sep 27 10:07:55.738603 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1580/2898] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:07:55 2022] POST /compute/v2.1/os-security-groups => generated 247 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:55.755526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6682e9ae-cd97-4555-90a4-a6ec3864a871 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:55.756489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6682e9ae-cd97-4555-90a4-a6ec3864a871 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:55.775229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6682e9ae-cd97-4555-90a4-a6ec3864a871 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:55.777049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6682e9ae-cd97-4555-90a4-a6ec3864a871 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:55.777690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6682e9ae-cd97-4555-90a4-a6ec3864a871 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:55.781441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6682e9ae-cd97-4555-90a4-a6ec3864a871 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:55.783415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6682e9ae-cd97-4555-90a4-a6ec3864a871 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:55.783694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c571b766-b786-4ea1-9a1e-60617a749035 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:55.786298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6682e9ae-cd97-4555-90a4-a6ec3864a871 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:55.798401 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c571b766-b786-4ea1-9a1e-60617a749035 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1314 microversion: 2.1 time: 0.390380 Sep 27 10:07:55.798990 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1319/2899] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:55 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1314 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:55.804110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-df2190b5-dc44-4f14-a6d7-be308df769ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:55.810015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-df2190b5-dc44-4f14-a6d7-be308df769ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:55.821853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df2190b5-dc44-4f14-a6d7-be308df769ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:55.823239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df2190b5-dc44-4f14-a6d7-be308df769ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:55.824341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df2190b5-dc44-4f14-a6d7-be308df769ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:55.825370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df2190b5-dc44-4f14-a6d7-be308df769ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:55.825995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df2190b5-dc44-4f14-a6d7-be308df769ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:55.826763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df2190b5-dc44-4f14-a6d7-be308df769ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.056188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-df2190b5-dc44-4f14-a6d7-be308df769ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:56.061715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6682e9ae-cd97-4555-90a4-a6ec3864a871 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:56.071777 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6682e9ae-cd97-4555-90a4-a6ec3864a871 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1520 microversion: 2.1 time: 0.321174 Sep 27 10:07:56.072952 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-df2190b5-dc44-4f14-a6d7-be308df769ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.270855 Sep 27 10:07:56.073807 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1320/2900] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:55 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:56.080885 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1581/2901] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:55 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1520 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:56.097422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b955a685-976a-495d-a7ee-05b6f667c895 req-fee83618-92b8-4bc0-924b-037b88a03377 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "768e537b-c8ab-418f-9523-ae18bd5a2259", "name": "network-vif-plugged", "status": "completed", "tag": "e7426e33-6ec8-40d9-a1de-120d41f1f5cc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:56.103822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f31279f8-5bab-4099-9745-ad674ccc9b87 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f628fed30>, body: {"security_group_rule": {"parent_group_id": "dbb5a6a7-e5d0-4bbd-8e12-4ad804f888cf", "ip_protocol": "tcp", "from_port": 22, "to_port": 22}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:56.131241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b955a685-976a-495d-a7ee-05b6f667c895 req-fee83618-92b8-4bc0-924b-037b88a03377 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.132546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b955a685-976a-495d-a7ee-05b6f667c895 req-fee83618-92b8-4bc0-924b-037b88a03377 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.134902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b955a685-976a-495d-a7ee-05b6f667c895 req-fee83618-92b8-4bc0-924b-037b88a03377 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.164003 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b955a685-976a-495d-a7ee-05b6f667c895 req-fee83618-92b8-4bc0-924b-037b88a03377 service nova] Creating event network-vif-plugged:e7426e33-6ec8-40d9-a1de-120d41f1f5cc for instance 768e537b-c8ab-418f-9523-ae18bd5a2259 on nch2-master-1 Sep 27 10:07:56.177253 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b955a685-976a-495d-a7ee-05b6f667c895 req-fee83618-92b8-4bc0-924b-037b88a03377 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100007 Sep 27 10:07:56.178484 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1321/2902] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:56.193847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da6b711c-2293-4683-a311-bf37fe20ae0b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:56.194698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-da6b711c-2293-4683-a311-bf37fe20ae0b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:56.209401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da6b711c-2293-4683-a311-bf37fe20ae0b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.210152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da6b711c-2293-4683-a311-bf37fe20ae0b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.210749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da6b711c-2293-4683-a311-bf37fe20ae0b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.211747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da6b711c-2293-4683-a311-bf37fe20ae0b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.213869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da6b711c-2293-4683-a311-bf37fe20ae0b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.214555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da6b711c-2293-4683-a311-bf37fe20ae0b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.479365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da6b711c-2293-4683-a311-bf37fe20ae0b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:56.490917 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-da6b711c-2293-4683-a311-bf37fe20ae0b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1314 microversion: 2.1 time: 0.306513 Sep 27 10:07:56.493091 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1322/2903] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:56 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1314 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:56.498617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:56.499174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:56.517053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.519699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.520394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.523322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.524596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.525239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.628982 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f31279f8-5bab-4099-9745-ad674ccc9b87 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 232 microversion: 2.1 time: 0.545779 Sep 27 10:07:56.629603 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1582/2904] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:07:56 2022] POST /compute/v2.1/os-security-group-rules => generated 232 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:56.634428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-062e20a3-2f90-4cb0-97cf-3b715ee7aa75 req-8c67997e-d75a-4999-ba17-0447e3d62a7a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bfc0bc5d-a883-4255-9869-8aaba51eaae2", "tag": "8d8cb0aa-16a7-4483-8c75-b88ae1eeb6d4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:56.645280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-062e20a3-2f90-4cb0-97cf-3b715ee7aa75 req-8c67997e-d75a-4999-ba17-0447e3d62a7a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.645767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-062e20a3-2f90-4cb0-97cf-3b715ee7aa75 req-8c67997e-d75a-4999-ba17-0447e3d62a7a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.646398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-062e20a3-2f90-4cb0-97cf-3b715ee7aa75 req-8c67997e-d75a-4999-ba17-0447e3d62a7a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.659958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:56.661137 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-062e20a3-2f90-4cb0-97cf-3b715ee7aa75 req-8c67997e-d75a-4999-ba17-0447e3d62a7a service nova] Creating event network-changed:8d8cb0aa-16a7-4483-8c75-b88ae1eeb6d4 for instance bfc0bc5d-a883-4255-9869-8aaba51eaae2 on nch1-master-1 Sep 27 10:07:56.672062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:56.674211 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-062e20a3-2f90-4cb0-97cf-3b715ee7aa75 req-8c67997e-d75a-4999-ba17-0447e3d62a7a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043196 Sep 27 10:07:56.679521 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1583/2905] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:56.686542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:56.695825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-711dbff1-38e4-47c4-b6d1-5cd3d6992a6f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:56.697445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-711dbff1-38e4-47c4-b6d1-5cd3d6992a6f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: a3046900-befa-4954-9aef-4ab042adf800] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:56.703829 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c92980d3-d38b-4a2d-a434-44dc77fd959f tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.209166 Sep 27 10:07:56.704760 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1323/2906] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:56 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:56.706707 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-711dbff1-38e4-47c4-b6d1-5cd3d6992a6f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance a3046900-befa-4954-9aef-4ab042adf800 could not be found. Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-711dbff1-38e4-47c4-b6d1-5cd3d6992a6f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/a3046900-befa-4954-9aef-4ab042adf800 HTTP/1.1 Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance a3046900-befa-4954-9aef-4ab042adf800 could not be found. Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a3046900-befa-4954-9aef-4ab042adf800'} Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a3046900-befa-4954-9aef-4ab042adf800 could not be found. Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a3046900-befa-4954-9aef-4ab042adf800 could not be found. Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:56.708402 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:56.713849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:56.713849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:56.713849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:56.713849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:56.713849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:56.713849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:56.713849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a3046900-befa-4954-9aef-4ab042adf800 could not be found. Sep 27 10:07:56.713849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:56.713849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-711dbff1-38e4-47c4-b6d1-5cd3d6992a6f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance a3046900-befa-4954-9aef-4ab042adf800 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:56.715043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0f03b994-5c9d-4d61-afb2-2e36d38a5049 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:56.715253 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-711dbff1-38e4-47c4-b6d1-5cd3d6992a6f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a3046900-befa-4954-9aef-4ab042adf800" status: 404 len: 111 microversion: 2.1 time: 0.029349 Sep 27 10:07:56.715253 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1584/2907] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:07:56 2022] DELETE /compute/v2.1/servers/a3046900-befa-4954-9aef-4ab042adf800 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:07:56.715535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0f03b994-5c9d-4d61-afb2-2e36d38a5049 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:56.716955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-017763aa-50a0-4e9c-a21c-cb568e3adb28 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:56.717750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-017763aa-50a0-4e9c-a21c-cb568e3adb28 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:56.725804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f03b994-5c9d-4d61-afb2-2e36d38a5049 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.726347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f03b994-5c9d-4d61-afb2-2e36d38a5049 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.726795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f03b994-5c9d-4d61-afb2-2e36d38a5049 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.727518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f03b994-5c9d-4d61-afb2-2e36d38a5049 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.727664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-017763aa-50a0-4e9c-a21c-cb568e3adb28 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.727972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f03b994-5c9d-4d61-afb2-2e36d38a5049 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.728084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-017763aa-50a0-4e9c-a21c-cb568e3adb28 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.728399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f03b994-5c9d-4d61-afb2-2e36d38a5049 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.728501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-017763aa-50a0-4e9c-a21c-cb568e3adb28 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.729473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-017763aa-50a0-4e9c-a21c-cb568e3adb28 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.730126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-017763aa-50a0-4e9c-a21c-cb568e3adb28 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.730710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-017763aa-50a0-4e9c-a21c-cb568e3adb28 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.779459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0f03b994-5c9d-4d61-afb2-2e36d38a5049 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:07:56.942582 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0f03b994-5c9d-4d61-afb2-2e36d38a5049 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 204 len: 0 microversion: 2.1 time: 0.231522 Sep 27 10:07:56.945701 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1324/2908] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:07:56 2022] DELETE /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 0 bytes in 240 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:07:56.950139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-017763aa-50a0-4e9c-a21c-cb568e3adb28 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:56.953268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ba9384c-ee5a-443e-a7e2-4e4da5117971 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Action: 'create', calling method: .version_select at 0x7f7f629d8dc0>, body: {"security_group_rule": {"parent_group_id": "dbb5a6a7-e5d0-4bbd-8e12-4ad804f888cf", "ip_protocol": "icmp", "from_port": -1, "to_port": -1}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:56.960676 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-017763aa-50a0-4e9c-a21c-cb568e3adb28 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.248192 Sep 27 10:07:56.962586 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1585/2909] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:56 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:56.968072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-51d16b35-2fe2-4cd0-bc49-3b2544fe7040 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:56.968735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-51d16b35-2fe2-4cd0-bc49-3b2544fe7040 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:56.978556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51d16b35-2fe2-4cd0-bc49-3b2544fe7040 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.979331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51d16b35-2fe2-4cd0-bc49-3b2544fe7040 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.979986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51d16b35-2fe2-4cd0-bc49-3b2544fe7040 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:56.980904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51d16b35-2fe2-4cd0-bc49-3b2544fe7040 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:56.981378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51d16b35-2fe2-4cd0-bc49-3b2544fe7040 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:56.981911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51d16b35-2fe2-4cd0-bc49-3b2544fe7040 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:57.157438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-51d16b35-2fe2-4cd0-bc49-3b2544fe7040 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:57.172318 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-51d16b35-2fe2-4cd0-bc49-3b2544fe7040 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.208364 Sep 27 10:07:57.173996 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1586/2910] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:56 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:57.184558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9df6eb3-8700-47e5-89af-f29857fe070b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:57.186050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c9df6eb3-8700-47e5-89af-f29857fe070b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:57.195510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9df6eb3-8700-47e5-89af-f29857fe070b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:57.196081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9df6eb3-8700-47e5-89af-f29857fe070b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:57.196542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9df6eb3-8700-47e5-89af-f29857fe070b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:57.197480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9df6eb3-8700-47e5-89af-f29857fe070b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:57.197987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9df6eb3-8700-47e5-89af-f29857fe070b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:57.198602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9df6eb3-8700-47e5-89af-f29857fe070b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:57.387116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9df6eb3-8700-47e5-89af-f29857fe070b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:57.400702 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9df6eb3-8700-47e5-89af-f29857fe070b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1314 microversion: 2.1 time: 0.217763 Sep 27 10:07:57.418242 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1587/2911] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:57 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1314 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:57.425273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8af377a5-f152-4fc1-b4f4-c224bf6507e7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:57.426222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8af377a5-f152-4fc1-b4f4-c224bf6507e7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:57.435986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8af377a5-f152-4fc1-b4f4-c224bf6507e7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:57.436751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8af377a5-f152-4fc1-b4f4-c224bf6507e7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:57.437025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8af377a5-f152-4fc1-b4f4-c224bf6507e7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:57.437869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8af377a5-f152-4fc1-b4f4-c224bf6507e7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:57.438799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8af377a5-f152-4fc1-b4f4-c224bf6507e7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:57.439292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8af377a5-f152-4fc1-b4f4-c224bf6507e7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:57.445514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:57.506876 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8ba9384c-ee5a-443e-a7e2-4e4da5117971 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "POST /compute/v2.1/os-security-group-rules" status: 200 len: 233 microversion: 2.1 time: 0.559635 Sep 27 10:07:57.509856 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1325/2912] 10.222.0.26 () {60 vars in 1195 bytes} [Tue Sep 27 10:07:56 2022] POST /compute/v2.1/os-security-group-rules => generated 233 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:57.515238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-83edea32-ee99-4aac-a13c-c5a6a039412f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:57.516348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-83edea32-ee99-4aac-a13c-c5a6a039412f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:57.542088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83edea32-ee99-4aac-a13c-c5a6a039412f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:57.542594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83edea32-ee99-4aac-a13c-c5a6a039412f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:57.558888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83edea32-ee99-4aac-a13c-c5a6a039412f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:57.558888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83edea32-ee99-4aac-a13c-c5a6a039412f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:57.558888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83edea32-ee99-4aac-a13c-c5a6a039412f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:57.558888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83edea32-ee99-4aac-a13c-c5a6a039412f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:57.891793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-83edea32-ee99-4aac-a13c-c5a6a039412f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:57.905848 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-83edea32-ee99-4aac-a13c-c5a6a039412f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.394530 Sep 27 10:07:57.908388 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1326/2913] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:57 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:57.926188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f1d347f2-5b04-4e12-acdd-a636bb675b28 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:57.927144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f1d347f2-5b04-4e12-acdd-a636bb675b28 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:57.947592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d347f2-5b04-4e12-acdd-a636bb675b28 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:57.948097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d347f2-5b04-4e12-acdd-a636bb675b28 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:57.948810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d347f2-5b04-4e12-acdd-a636bb675b28 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:57.956907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d347f2-5b04-4e12-acdd-a636bb675b28 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:57.957956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d347f2-5b04-4e12-acdd-a636bb675b28 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:57.958603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d347f2-5b04-4e12-acdd-a636bb675b28 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:58.007914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:58.068255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:58.069147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8af377a5-f152-4fc1-b4f4-c224bf6507e7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:58.083148 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8af377a5-f152-4fc1-b4f4-c224bf6507e7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1526 microversion: 2.1 time: 0.663278 Sep 27 10:07:58.097491 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1588/2914] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:57 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1526 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:58.102978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-afadcb7a-b41e-47e8-a710-8d55cfad695e tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Calling method '.version_select at 0x7f7f62bb14c0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:58.182279 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-afadcb7a-b41e-47e8-a710-8d55cfad695e tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups/dbb5a6a7-e5d0-4bbd-8e12-4ad804f888cf" status: 200 len: 664 microversion: 2.1 time: 0.083346 Sep 27 10:07:58.182969 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1589/2915] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:07:58 2022] GET /compute/v2.1/os-security-groups/dbb5a6a7-e5d0-4bbd-8e12-4ad804f888cf => generated 664 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:58.187896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:58.188530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:58.207124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:58.207639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:58.208088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:58.210788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:58.211304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:58.212270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:58.319037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:58.334128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:58.335693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:58.354698 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d838a73f-d7a8-4cb4-ae41-fc2db7d35d63 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.170089 Sep 27 10:07:58.354802 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1590/2916] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:07:58 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:07:58.364735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-197528fa-070c-44a4-99ee-f6e5151daf83 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:58.365203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-197528fa-070c-44a4-99ee-f6e5151daf83 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Failed to fetch instance by id 9223372036854775808 {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2981}} Sep 27 10:07:58.365724 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-197528fa-070c-44a4-99ee-f6e5151daf83 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 9223372036854775808 could not be found. Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-197528fa-070c-44a4-99ee-f6e5151daf83 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/9223372036854775808 HTTP/1.1 Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 9223372036854775808 could not be found. Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '9223372036854775808'} Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9223372036854775808 could not be found. Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9223372036854775808 could not be found. Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:07:58.367418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:07:58.371545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:07:58.371545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:07:58.371545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:07:58.371545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9223372036854775808 could not be found. Sep 27 10:07:58.371545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:07:58.371545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-197528fa-070c-44a4-99ee-f6e5151daf83 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 9223372036854775808 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:07:58.371545 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-197528fa-070c-44a4-99ee-f6e5151daf83 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9223372036854775808" status: 404 len: 94 microversion: 2.1 time: 0.013134 Sep 27 10:07:58.372543 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1591/2917] 10.222.0.26 () {58 vars in 1188 bytes} [Tue Sep 27 10:07:58 2022] DELETE /compute/v2.1/servers/9223372036854775808 => generated 94 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:07:58.378007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-803d1324-2146-4df7-8344-7ca58c55fcdc tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:58.378664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-803d1324-2146-4df7-8344-7ca58c55fcdc tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:58.388439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803d1324-2146-4df7-8344-7ca58c55fcdc tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:58.388882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803d1324-2146-4df7-8344-7ca58c55fcdc tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:58.389354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803d1324-2146-4df7-8344-7ca58c55fcdc tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:58.390104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803d1324-2146-4df7-8344-7ca58c55fcdc tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:58.399073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803d1324-2146-4df7-8344-7ca58c55fcdc tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:58.399553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803d1324-2146-4df7-8344-7ca58c55fcdc tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:58.652349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-803d1324-2146-4df7-8344-7ca58c55fcdc tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:58.662948 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-803d1324-2146-4df7-8344-7ca58c55fcdc tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1334 microversion: 2.1 time: 0.289250 Sep 27 10:07:58.663924 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1592/2918] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:58 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1334 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:58.691483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-18c19be3-286d-4310-84c9-c3e509bdfeca tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Calling method '.version_select at 0x7f7f63715700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:58.934241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=10 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:07:59.135916 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-18c19be3-286d-4310-84c9-c3e509bdfeca tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-group-rules/5fd34a33-24a1-4055-9962-2ed8b72b7d4d" status: 202 len: 0 microversion: 2.1 time: 0.469771 Sep 27 10:07:59.137357 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1593/2919] 10.222.0.26 () {58 vars in 1287 bytes} [Tue Sep 27 10:07:58 2022] DELETE /compute/v2.1/os-security-group-rules/5fd34a33-24a1-4055-9962-2ed8b72b7d4d => generated 0 bytes in 472 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:07:59.144743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3c2136f5-3d6b-41da-a36e-6fadbed8eca7 req-fe8302c4-af30-4961-9196-4cfec02fa382 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa", "name": "network-vif-plugged", "status": "completed", "tag": "6431263b-bb42-4039-8d74-79f58ae6f0b2"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:07:59.176946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3c2136f5-3d6b-41da-a36e-6fadbed8eca7 req-fe8302c4-af30-4961-9196-4cfec02fa382 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:59.179591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3c2136f5-3d6b-41da-a36e-6fadbed8eca7 req-fe8302c4-af30-4961-9196-4cfec02fa382 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:59.179591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3c2136f5-3d6b-41da-a36e-6fadbed8eca7 req-fe8302c4-af30-4961-9196-4cfec02fa382 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:59.192795 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3c2136f5-3d6b-41da-a36e-6fadbed8eca7 req-fe8302c4-af30-4961-9196-4cfec02fa382 service nova] Creating event network-vif-plugged:6431263b-bb42-4039-8d74-79f58ae6f0b2 for instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa on nch1-master-1 Sep 27 10:07:59.202755 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3c2136f5-3d6b-41da-a36e-6fadbed8eca7 req-fe8302c4-af30-4961-9196-4cfec02fa382 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063437 Sep 27 10:07:59.204764 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1594/2920] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:07:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:07:59.213158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-43b77125-daea-467b-8e7c-0e50ecd2e67a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:59.213866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-43b77125-daea-467b-8e7c-0e50ecd2e67a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:59.229046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43b77125-daea-467b-8e7c-0e50ecd2e67a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:59.229526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43b77125-daea-467b-8e7c-0e50ecd2e67a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:59.229964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43b77125-daea-467b-8e7c-0e50ecd2e67a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:59.230684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43b77125-daea-467b-8e7c-0e50ecd2e67a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:59.231139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43b77125-daea-467b-8e7c-0e50ecd2e67a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:59.231568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43b77125-daea-467b-8e7c-0e50ecd2e67a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:59.477754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-43b77125-daea-467b-8e7c-0e50ecd2e67a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:59.487120 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-43b77125-daea-467b-8e7c-0e50ecd2e67a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.280977 Sep 27 10:07:59.502049 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1595/2921] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:59 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:59.519841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-20cc26f6-660b-475d-b154-638c30a2212d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:59.520465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-20cc26f6-660b-475d-b154-638c30a2212d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:59.529061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20cc26f6-660b-475d-b154-638c30a2212d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:59.529564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20cc26f6-660b-475d-b154-638c30a2212d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:59.530004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20cc26f6-660b-475d-b154-638c30a2212d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:59.530728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20cc26f6-660b-475d-b154-638c30a2212d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:59.531184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20cc26f6-660b-475d-b154-638c30a2212d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:59.531610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20cc26f6-660b-475d-b154-638c30a2212d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:59.765508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-20cc26f6-660b-475d-b154-638c30a2212d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:07:59.774906 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-20cc26f6-660b-475d-b154-638c30a2212d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1314 microversion: 2.1 time: 0.271589 Sep 27 10:07:59.776131 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1596/2922] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:59 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1314 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:07:59.781861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2114a39c-98e8-436a-9988-d4900c477ff0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:07:59.782628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2114a39c-98e8-436a-9988-d4900c477ff0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:07:59.789652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2114a39c-98e8-436a-9988-d4900c477ff0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:59.790166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2114a39c-98e8-436a-9988-d4900c477ff0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:59.790610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2114a39c-98e8-436a-9988-d4900c477ff0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:07:59.791338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2114a39c-98e8-436a-9988-d4900c477ff0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:07:59.791925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2114a39c-98e8-436a-9988-d4900c477ff0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:07:59.792356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2114a39c-98e8-436a-9988-d4900c477ff0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:00.004015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2114a39c-98e8-436a-9988-d4900c477ff0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:00.022890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f1d347f2-5b04-4e12-acdd-a636bb675b28 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:00.035007 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f1d347f2-5b04-4e12-acdd-a636bb675b28 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1314 microversion: 2.1 time: 2.124325 Sep 27 10:08:00.036232 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2114a39c-98e8-436a-9988-d4900c477ff0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1526 microversion: 2.1 time: 0.258854 Sep 27 10:08:00.050229 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1597/2923] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:59 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1526 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:00.050743 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1327/2924] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:07:57 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1314 bytes in 2142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:00.062973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4800c2fc-97d1-4ef0-8cfe-b95fa1175696 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Calling method '.version_select at 0x7f7f629e4f70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:00.067310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:00.070627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:00.083570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:00.084488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:00.085241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:00.086315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:00.086948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:00.087818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:00.215500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:00.228122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:00.239280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:00.246479 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2b539501-9c2f-4223-9055-fb6306018819 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.188063 Sep 27 10:08:00.260695 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1598/2925] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:00 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:00.260695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-39455f19-eae4-4167-8606-8de725f38f0f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:00.262029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-39455f19-eae4-4167-8606-8de725f38f0f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:00.282049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39455f19-eae4-4167-8606-8de725f38f0f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:00.282546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39455f19-eae4-4167-8606-8de725f38f0f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:00.283004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39455f19-eae4-4167-8606-8de725f38f0f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:00.283706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39455f19-eae4-4167-8606-8de725f38f0f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:00.284168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39455f19-eae4-4167-8606-8de725f38f0f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:00.284596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39455f19-eae4-4167-8606-8de725f38f0f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:00.493095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-39455f19-eae4-4167-8606-8de725f38f0f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:00.505793 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-39455f19-eae4-4167-8606-8de725f38f0f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.253319 Sep 27 10:08:00.514852 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1599/2926] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:00 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:00.527016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-20848b0e-62b0-4d2b-bceb-e2beda680e9c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:00.527668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-20848b0e-62b0-4d2b-bceb-e2beda680e9c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:00.543171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20848b0e-62b0-4d2b-bceb-e2beda680e9c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:00.543676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20848b0e-62b0-4d2b-bceb-e2beda680e9c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:00.544122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20848b0e-62b0-4d2b-bceb-e2beda680e9c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:00.544885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20848b0e-62b0-4d2b-bceb-e2beda680e9c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:00.545353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20848b0e-62b0-4d2b-bceb-e2beda680e9c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:00.545952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20848b0e-62b0-4d2b-bceb-e2beda680e9c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:00.781521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-20848b0e-62b0-4d2b-bceb-e2beda680e9c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:00.793087 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-20848b0e-62b0-4d2b-bceb-e2beda680e9c tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1670 microversion: 2.1 time: 0.276096 Sep 27 10:08:00.794064 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1600/2927] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:00 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1670 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:00.811214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9590b649-3269-4f63-9973-e7f600925f56 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:00.811910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9590b649-3269-4f63-9973-e7f600925f56 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Failed to fetch instance by id -1 {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2981}} Sep 27 10:08:00.812481 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-9590b649-3269-4f63-9973-e7f600925f56 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance -1 could not be found. Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9590b649-3269-4f63-9973-e7f600925f56 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/-1 HTTP/1.1 Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance -1 could not be found. Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '-1'} Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance -1 could not be found. Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance -1 could not be found. Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:00.816436 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:00.821404 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:00.821404 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:00.821404 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance -1 could not be found. Sep 27 10:08:00.821404 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:00.821404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9590b649-3269-4f63-9973-e7f600925f56 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance -1 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:00.821404 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9590b649-3269-4f63-9973-e7f600925f56 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "DELETE /compute/v2.1/servers/-1" status: 404 len: 77 microversion: 2.1 time: 0.022871 Sep 27 10:08:00.822478 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1601/2928] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:08:00 2022] DELETE /compute/v2.1/servers/-1 => generated 77 bytes in 28 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:08:00.833952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e1b3dca2-8a22-42f2-bfd3-60d574657fa3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:00.834590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e1b3dca2-8a22-42f2-bfd3-60d574657fa3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:00.854028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1b3dca2-8a22-42f2-bfd3-60d574657fa3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:00.854435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1b3dca2-8a22-42f2-bfd3-60d574657fa3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:00.854894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1b3dca2-8a22-42f2-bfd3-60d574657fa3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:00.857208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1b3dca2-8a22-42f2-bfd3-60d574657fa3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:00.857692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1b3dca2-8a22-42f2-bfd3-60d574657fa3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:00.858137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1b3dca2-8a22-42f2-bfd3-60d574657fa3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:01.007504 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4800c2fc-97d1-4ef0-8cfe-b95fa1175696 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/dbb5a6a7-e5d0-4bbd-8e12-4ad804f888cf" status: 202 len: 0 microversion: 2.1 time: 0.955326 Sep 27 10:08:01.011998 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1328/2929] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:08:00 2022] DELETE /compute/v2.1/os-security-groups/dbb5a6a7-e5d0-4bbd-8e12-4ad804f888cf => generated 0 bytes in 960 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:01.017695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e9d1e65-894b-4712-b30c-63a30547d873 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:01.018403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6e9d1e65-894b-4712-b30c-63a30547d873 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:01.042193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9d1e65-894b-4712-b30c-63a30547d873 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:01.042843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9d1e65-894b-4712-b30c-63a30547d873 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:01.043632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9d1e65-894b-4712-b30c-63a30547d873 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:01.044382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9d1e65-894b-4712-b30c-63a30547d873 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:01.044920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9d1e65-894b-4712-b30c-63a30547d873 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:01.045358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e9d1e65-894b-4712-b30c-63a30547d873 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:01.092582 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6e9d1e65-894b-4712-b30c-63a30547d873 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259/ips" status: 200 len: 180 microversion: 2.1 time: 0.079133 Sep 27 10:08:01.093223 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1329/2930] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:08:01 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259/ips => generated 180 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:01.108286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3221a234-fdbf-4758-bb90-237d4f2b9ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:01.108966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3221a234-fdbf-4758-bb90-237d4f2b9ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:01.142067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3221a234-fdbf-4758-bb90-237d4f2b9ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:01.143679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3221a234-fdbf-4758-bb90-237d4f2b9ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:01.144250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3221a234-fdbf-4758-bb90-237d4f2b9ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:01.147193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3221a234-fdbf-4758-bb90-237d4f2b9ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:01.148922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3221a234-fdbf-4758-bb90-237d4f2b9ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:01.150835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3221a234-fdbf-4758-bb90-237d4f2b9ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:01.178743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e1b3dca2-8a22-42f2-bfd3-60d574657fa3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:01.207475 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e1b3dca2-8a22-42f2-bfd3-60d574657fa3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1314 microversion: 2.1 time: 0.380158 Sep 27 10:08:01.209996 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1602/2931] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:00 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1314 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:01.218392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a3c10e96-d6e8-4789-b229-4ff2a1426b47 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Calling method '.version_select at 0x7f7f62be3040>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:01.261020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-a3c10e96-d6e8-4789-b229-4ff2a1426b47 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 42ad5834-b3e8-4d03-b36a-217fe275f5eb does not exist", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:08:01.261635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.security_group_api [None req-a3c10e96-d6e8-4789-b229-4ff2a1426b47 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Neutron security group 42ad5834-b3e8-4d03-b36a-217fe275f5eb not found {{(pid=145968) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:08:01.262165 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a3c10e96-d6e8-4789-b229-4ff2a1426b47 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] HTTP exception thrown: Security group 42ad5834-b3e8-4d03-b36a-217fe275f5eb does not exist Sep 27 10:08:01.262165 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-f68165a0-d830-4a91-9ac4-47b4bcf2c371'] Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a3c10e96-d6e8-4789-b229-4ff2a1426b47 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Request method failure captured: Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-security-groups/42ad5834-b3e8-4d03-b36a-217fe275f5eb HTTP/1.1 Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMstwLWN5QxYUOb4KmOxiQb-WfTGECKeJNLSHGLgH-mnwTUcobCC7HnxSMuXOlYUlNz_nZqsjs4Nm3Fz8RoAdEe-CPWIHh1fKu0w_8hQ6JNP5jH9I52o50NuRIK9TOnAo_LWpjUFTszdALv5MYerOzqhUstAuuTqFT5M_yLtrsVe6o7Y Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 462859c44a93480aba0df9344b789348 Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupRulesTestJSON-1927993946 Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/462859c44a93480aba0df9344b789348"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/462859c44a93480aba0df9344b789348"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/462859c44a93480aba0df9344b789348"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_462859c44a93480aba0df9344b789348"}]}] Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupRulesTestJSON-1927993946 Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 462859c44a93480aba0df9344b789348 Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupRulesTestJSON-1927993946 Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupRulesTestJSON-1927993946-project Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 906b7b3d2366422cbc5da1f3080680f6 Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupRulesTestJSON-1927993946-project Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: exception: Security group 42ad5834-b3e8-4d03-b36a-217fe275f5eb does not exist Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-f68165a0-d830-4a91-9ac4-47b4bcf2c371'] Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '42ad5834-b3e8-4d03-b36a-217fe275f5eb'} Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:08:01.264037 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 42ad5834-b3e8-4d03-b36a-217fe275f5eb does not exist Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f68165a0-d830-4a91-9ac4-47b4bcf2c371'] Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 42ad5834-b3e8-4d03-b36a-217fe275f5eb does not exist Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f68165a0-d830-4a91-9ac4-47b4bcf2c371'] Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 42ad5834-b3e8-4d03-b36a-217fe275f5eb does not exist Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f68165a0-d830-4a91-9ac4-47b4bcf2c371'] Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 42ad5834-b3e8-4d03-b36a-217fe275f5eb does not exist Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-f68165a0-d830-4a91-9ac4-47b4bcf2c371'] Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a3c10e96-d6e8-4789-b229-4ff2a1426b47 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Returning 404 to user: Security group 42ad5834-b3e8-4d03-b36a-217fe275f5eb does not exist Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-f68165a0-d830-4a91-9ac4-47b4bcf2c371'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a3c10e96-d6e8-4789-b229-4ff2a1426b47 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/42ad5834-b3e8-4d03-b36a-217fe275f5eb" status: 404 len: 194 microversion: 2.1 time: 0.052292 Sep 27 10:08:01.267973 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1603/2932] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:08:01 2022] DELETE /compute/v2.1/os-security-groups/42ad5834-b3e8-4d03-b36a-217fe275f5eb => generated 194 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:01.279996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f68eab4a-8315-4b60-988a-cb00f591c2e2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:01.285944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f68eab4a-8315-4b60-988a-cb00f591c2e2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:01.306604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f68eab4a-8315-4b60-988a-cb00f591c2e2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:01.307127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f68eab4a-8315-4b60-988a-cb00f591c2e2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:01.307580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f68eab4a-8315-4b60-988a-cb00f591c2e2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:01.308321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f68eab4a-8315-4b60-988a-cb00f591c2e2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:01.308814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f68eab4a-8315-4b60-988a-cb00f591c2e2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:01.309245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f68eab4a-8315-4b60-988a-cb00f591c2e2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:01.588963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f68eab4a-8315-4b60-988a-cb00f591c2e2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:01.601281 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f68eab4a-8315-4b60-988a-cb00f591c2e2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1314 microversion: 2.1 time: 0.329392 Sep 27 10:08:01.604904 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1604/2933] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:01 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1314 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:01.628078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2a881a09-1729-4c1a-bf77-e3f79d5be3a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:01.630119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2a881a09-1729-4c1a-bf77-e3f79d5be3a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:01.641636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a881a09-1729-4c1a-bf77-e3f79d5be3a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:01.642395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a881a09-1729-4c1a-bf77-e3f79d5be3a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:01.642995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a881a09-1729-4c1a-bf77-e3f79d5be3a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:01.644541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a881a09-1729-4c1a-bf77-e3f79d5be3a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:01.645187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a881a09-1729-4c1a-bf77-e3f79d5be3a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:01.645770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a881a09-1729-4c1a-bf77-e3f79d5be3a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:01.749192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3221a234-fdbf-4758-bb90-237d4f2b9ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:01.763929 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3221a234-fdbf-4758-bb90-237d4f2b9ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.669400 Sep 27 10:08:01.768984 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1330/2934] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:01 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 675 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:01.774642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-338d46ae-64bc-44d4-8071-82f49b628f62 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:01.785585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-338d46ae-64bc-44d4-8071-82f49b628f62 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:01.793740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-338d46ae-64bc-44d4-8071-82f49b628f62 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:01.799501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-338d46ae-64bc-44d4-8071-82f49b628f62 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:01.800317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-338d46ae-64bc-44d4-8071-82f49b628f62 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:01.801951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-338d46ae-64bc-44d4-8071-82f49b628f62 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:01.803103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-338d46ae-64bc-44d4-8071-82f49b628f62 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:01.804133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-338d46ae-64bc-44d4-8071-82f49b628f62 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:01.863367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-338d46ae-64bc-44d4-8071-82f49b628f62 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:08:02.017768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2a881a09-1729-4c1a-bf77-e3f79d5be3a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:02.038394 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2a881a09-1729-4c1a-bf77-e3f79d5be3a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 200 len: 1526 microversion: 2.1 time: 0.416496 Sep 27 10:08:02.040790 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1605/2935] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:01 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 1526 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:02.062180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9b8fb2e6-d117-400a-b08a-6a9d15cc065a tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Calling method '.version_select at 0x7f7f63730f70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:02.066016 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-338d46ae-64bc-44d4-8071-82f49b628f62 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "DELETE /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 204 len: 0 microversion: 2.1 time: 0.294168 Sep 27 10:08:02.067530 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1331/2936] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:08:01 2022] DELETE /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:08:02.074719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:02.075818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:02.083729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:02.084484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:02.085733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:02.086545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:02.087313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:02.087725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:02.119382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:02.255579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:02.270594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:02.274821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:02.287845 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eb5a01c2-caa4-4bee-a538-587e02591073 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.217939 Sep 27 10:08:02.304946 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1332/2937] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:02 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:02.320595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-02d94215-3112-4bed-9078-ddbd9771d847 req-f279f373-22e3-4b97-b16c-b7c3d88c07b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa", "name": "network-vif-unplugged", "status": "completed", "tag": "6431263b-bb42-4039-8d74-79f58ae6f0b2"}, {"server_uuid": "08babbce-373e-4cf5-a138-066b6eeb02c3", "name": "network-vif-plugged", "status": "completed", "tag": "2ca9e80c-8a06-414a-81cf-a330f210be36"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:02.357425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-02d94215-3112-4bed-9078-ddbd9771d847 req-f279f373-22e3-4b97-b16c-b7c3d88c07b6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:02.357924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-02d94215-3112-4bed-9078-ddbd9771d847 req-f279f373-22e3-4b97-b16c-b7c3d88c07b6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:02.358366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-02d94215-3112-4bed-9078-ddbd9771d847 req-f279f373-22e3-4b97-b16c-b7c3d88c07b6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:02.404399 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-02d94215-3112-4bed-9078-ddbd9771d847 req-f279f373-22e3-4b97-b16c-b7c3d88c07b6 service nova] Creating event network-vif-unplugged:6431263b-bb42-4039-8d74-79f58ae6f0b2 for instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa on nch1-master-1 Sep 27 10:08:02.404947 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-02d94215-3112-4bed-9078-ddbd9771d847 req-f279f373-22e3-4b97-b16c-b7c3d88c07b6 service nova] Creating event network-vif-plugged:2ca9e80c-8a06-414a-81cf-a330f210be36 for instance 08babbce-373e-4cf5-a138-066b6eeb02c3 on nch2-master-1 Sep 27 10:08:02.430842 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-02d94215-3112-4bed-9078-ddbd9771d847 req-f279f373-22e3-4b97-b16c-b7c3d88c07b6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 356 microversion: 2.1 time: 0.125669 Sep 27 10:08:02.431479 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1333/2938] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:02 2022] POST /compute/v2.1/os-server-external-events => generated 356 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:02.449984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-62128901-d6a7-4dd2-b00b-5c910e76efd2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:02.450977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-62128901-d6a7-4dd2-b00b-5c910e76efd2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:02.471582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62128901-d6a7-4dd2-b00b-5c910e76efd2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:02.472355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62128901-d6a7-4dd2-b00b-5c910e76efd2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:02.473067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62128901-d6a7-4dd2-b00b-5c910e76efd2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:02.474024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62128901-d6a7-4dd2-b00b-5c910e76efd2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:02.474667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62128901-d6a7-4dd2-b00b-5c910e76efd2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:02.475265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62128901-d6a7-4dd2-b00b-5c910e76efd2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:02.601833 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9b8fb2e6-d117-400a-b08a-6a9d15cc065a tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/7e9448f1-b28d-4c62-90d3-b73a1f196cc7" status: 202 len: 0 microversion: 2.1 time: 0.558691 Sep 27 10:08:02.602494 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1606/2939] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:08:02 2022] DELETE /compute/v2.1/os-security-groups/7e9448f1-b28d-4c62-90d3-b73a1f196cc7 => generated 0 bytes in 561 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:02.611074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f268753e-7b42-4ef0-9497-d67994c148f7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:02.612979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f268753e-7b42-4ef0-9497-d67994c148f7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:02.628916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f268753e-7b42-4ef0-9497-d67994c148f7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:02.629661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f268753e-7b42-4ef0-9497-d67994c148f7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:02.631376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f268753e-7b42-4ef0-9497-d67994c148f7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:02.632481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f268753e-7b42-4ef0-9497-d67994c148f7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:02.634490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f268753e-7b42-4ef0-9497-d67994c148f7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:02.635963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f268753e-7b42-4ef0-9497-d67994c148f7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:02.872494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f268753e-7b42-4ef0-9497-d67994c148f7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:02.885891 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f268753e-7b42-4ef0-9497-d67994c148f7 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1676 microversion: 2.1 time: 0.282111 Sep 27 10:08:02.887554 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1607/2940] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:02 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1676 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:02.894818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-64b84705-bd86-464a-91e1-ec6668e5a1f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:02.895880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-64b84705-bd86-464a-91e1-ec6668e5a1f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:02.904647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b84705-bd86-464a-91e1-ec6668e5a1f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:02.905573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b84705-bd86-464a-91e1-ec6668e5a1f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:02.906167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b84705-bd86-464a-91e1-ec6668e5a1f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:02.907032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b84705-bd86-464a-91e1-ec6668e5a1f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:02.907693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b84705-bd86-464a-91e1-ec6668e5a1f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:02.908507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b84705-bd86-464a-91e1-ec6668e5a1f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:02.991910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-62128901-d6a7-4dd2-b00b-5c910e76efd2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:03.008502 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-62128901-d6a7-4dd2-b00b-5c910e76efd2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.570760 Sep 27 10:08:03.011508 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1334/2941] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:02 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:03.021029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e0787cc1-f129-4a48-a36f-f39061631033 req-a565796c-ae9f-49e1-9a4c-241f6d935433 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa", "name": "network-vif-deleted", "tag": "6431263b-bb42-4039-8d74-79f58ae6f0b2"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:03.049473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e0787cc1-f129-4a48-a36f-f39061631033 req-a565796c-ae9f-49e1-9a4c-241f6d935433 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:03.049972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e0787cc1-f129-4a48-a36f-f39061631033 req-a565796c-ae9f-49e1-9a4c-241f6d935433 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:03.050419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e0787cc1-f129-4a48-a36f-f39061631033 req-a565796c-ae9f-49e1-9a4c-241f6d935433 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:03.068535 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e0787cc1-f129-4a48-a36f-f39061631033 req-a565796c-ae9f-49e1-9a4c-241f6d935433 service nova] Creating event network-vif-deleted:6431263b-bb42-4039-8d74-79f58ae6f0b2 for instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa on nch1-master-1 Sep 27 10:08:03.085063 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e0787cc1-f129-4a48-a36f-f39061631033 req-a565796c-ae9f-49e1-9a4c-241f6d935433 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070700 Sep 27 10:08:03.086198 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1335/2942] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:03.091823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c818f625-dc8a-4c8b-abbe-1727af62fb7d tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Calling method '.version_select at 0x7f7f62ac3ca0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:03.181662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-64b84705-bd86-464a-91e1-ec6668e5a1f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:03.214987 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-64b84705-bd86-464a-91e1-ec6668e5a1f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1314 microversion: 2.1 time: 0.325956 Sep 27 10:08:03.218664 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1608/2943] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:02 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1314 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:03.229459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-daf5f59b-ae3a-46b2-ac58-f6e8870f04de tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:03.230350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-daf5f59b-ae3a-46b2-ac58-f6e8870f04de tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:03.244003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daf5f59b-ae3a-46b2-ac58-f6e8870f04de tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:03.250139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daf5f59b-ae3a-46b2-ac58-f6e8870f04de tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:03.250850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daf5f59b-ae3a-46b2-ac58-f6e8870f04de tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:03.251922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daf5f59b-ae3a-46b2-ac58-f6e8870f04de tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:03.252533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daf5f59b-ae3a-46b2-ac58-f6e8870f04de tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:03.253137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daf5f59b-ae3a-46b2-ac58-f6e8870f04de tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:03.539111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-daf5f59b-ae3a-46b2-ac58-f6e8870f04de tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:03.548084 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-daf5f59b-ae3a-46b2-ac58-f6e8870f04de tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1314 microversion: 2.1 time: 0.328074 Sep 27 10:08:03.560711 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1609/2944] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:03 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1314 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:03.560711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67371f10-5a4f-4445-a649-2012d3ef6f1a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"forceDelete": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:03.560711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67371f10-5a4f-4445-a649-2012d3ef6f1a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3e55a3ce-7f43-4971-82d8-bf1a63291f7a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:03.566008 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-67371f10-5a4f-4445-a649-2012d3ef6f1a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 3e55a3ce-7f43-4971-82d8-bf1a63291f7a could not be found. Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67371f10-5a4f-4445-a649-2012d3ef6f1a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/3e55a3ce-7f43-4971-82d8-bf1a63291f7a/action HTTP/1.1 Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 19 Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: {"forceDelete": {}} Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 3e55a3ce-7f43-4971-82d8-bf1a63291f7a could not be found. Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3e55a3ce-7f43-4971-82d8-bf1a63291f7a', 'body': {'forceDelete': {}}} Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3e55a3ce-7f43-4971-82d8-bf1a63291f7a could not be found. Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 55, in _force_delete Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3e55a3ce-7f43-4971-82d8-bf1a63291f7a could not be found. Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:03.586098 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:03.593485 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:03.593485 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:03.593485 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3e55a3ce-7f43-4971-82d8-bf1a63291f7a could not be found. Sep 27 10:08:03.593485 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:03.596217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67371f10-5a4f-4445-a649-2012d3ef6f1a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 3e55a3ce-7f43-4971-82d8-bf1a63291f7a could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:03.597456 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67371f10-5a4f-4445-a649-2012d3ef6f1a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3e55a3ce-7f43-4971-82d8-bf1a63291f7a/action" status: 404 len: 111 microversion: 2.1 time: 0.046931 Sep 27 10:08:03.598278 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1610/2945] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:03 2022] POST /compute/v2.1/servers/3e55a3ce-7f43-4971-82d8-bf1a63291f7a/action => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:03.620945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:03.621891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:03.633993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:03.645420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:03.645990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:03.646848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:03.647561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:03.648130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:03.664565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] HTTP exception thrown: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa HTTP/1.1 Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa'} Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:03.666196 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:03.670285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:03.670285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:03.670285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:03.670285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:03.670285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:03.670285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:03.670285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:03.670285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:03.670285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 404 to user: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:03.670285 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-309e7169-9bbf-4a1c-9606-5cf31b842947 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 404 len: 111 microversion: 2.1 time: 0.051068 Sep 27 10:08:03.670285 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1611/2946] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:03 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:03.673135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:03.673773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:03.680853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:03.681606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:03.682195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:03.683139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:03.683765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:03.684196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:03.768415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:03.779790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:03.786846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:03.793809 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e744e512-7aab-424e-ae7e-40259afa49ca tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.124482 Sep 27 10:08:03.794961 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1612/2947] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:03 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:03.800684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e269379c-eedd-4308-b42c-d5b7e395ba5e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:03.801301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e269379c-eedd-4308-b42c-d5b7e395ba5e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:03.827873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e269379c-eedd-4308-b42c-d5b7e395ba5e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:03.828702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e269379c-eedd-4308-b42c-d5b7e395ba5e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:03.829379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e269379c-eedd-4308-b42c-d5b7e395ba5e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:03.830325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e269379c-eedd-4308-b42c-d5b7e395ba5e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:03.831495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e269379c-eedd-4308-b42c-d5b7e395ba5e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:03.832357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e269379c-eedd-4308-b42c-d5b7e395ba5e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:03.842447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:04.052583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e269379c-eedd-4308-b42c-d5b7e395ba5e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:04.061754 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e269379c-eedd-4308-b42c-d5b7e395ba5e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.265543 Sep 27 10:08:04.062969 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1613/2948] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:03 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:04.070037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ImagesOneServerNegativeTestJSON-server-90038689", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:04.138911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] validate_networks() for [('3757a0b4-fb97-4218-a71c-9743174aeb68', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:08:04.385518 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c818f625-dc8a-4c8b-abbe-1727af62fb7d tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/e1f2726d-87ed-4b6c-937b-4051572aad8c" status: 202 len: 0 microversion: 2.1 time: 1.297331 Sep 27 10:08:04.390048 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1336/2949] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:08:03 2022] DELETE /compute/v2.1/os-security-groups/e1f2726d-87ed-4b6c-937b-4051572aad8c => generated 0 bytes in 1303 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:04.397314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7466b56b-9acf-47e1-bb38-85bcb151aaca tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:04.398865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7466b56b-9acf-47e1-bb38-85bcb151aaca tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:04.410224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7466b56b-9acf-47e1-bb38-85bcb151aaca tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:04.411018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7466b56b-9acf-47e1-bb38-85bcb151aaca tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:04.411655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7466b56b-9acf-47e1-bb38-85bcb151aaca tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:04.413844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7466b56b-9acf-47e1-bb38-85bcb151aaca tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:04.414812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7466b56b-9acf-47e1-bb38-85bcb151aaca tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:04.415602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7466b56b-9acf-47e1-bb38-85bcb151aaca tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:04.740040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:08:04.740614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:08:04.741036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:08:04.741460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:08:04.741909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:08:04.764719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:04.765218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:04.765657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:04.766592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:04.767202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:04.767869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7466b56b-9acf-47e1-bb38-85bcb151aaca tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:04.768120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:04.776935 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7466b56b-9acf-47e1-bb38-85bcb151aaca tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1676 microversion: 2.1 time: 0.385459 Sep 27 10:08:04.781701 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1337/2950] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:04 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1676 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:04.792754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc601224-1333-4db1-aa92-58685ecd6f94 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:04.793662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc601224-1333-4db1-aa92-58685ecd6f94 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:04.803751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc601224-1333-4db1-aa92-58685ecd6f94 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:04.811472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc601224-1333-4db1-aa92-58685ecd6f94 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:04.811943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc601224-1333-4db1-aa92-58685ecd6f94 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:04.812710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc601224-1333-4db1-aa92-58685ecd6f94 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:04.813352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc601224-1333-4db1-aa92-58685ecd6f94 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:04.813510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Getting quotas for project f8130333e2c04a3f960dc6465e07cc46. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:08:04.813786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc601224-1333-4db1-aa92-58685ecd6f94 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:04.831214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Getting quotas for user 68d9297ca0884c2fa7f5283f722451ba and project f8130333e2c04a3f960dc6465e07cc46. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:08:04.862231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:08:04.865002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:08:05.046128 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f76def21-9992-463e-8932-34e19c0374b4 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 0.981846 Sep 27 10:08:05.059962 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1614/2951] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:08:04 2022] POST /compute/v2.1/servers => generated 383 bytes in 990 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:08:05.075440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ca3d424a-4b4c-499f-b2fc-18d3ff9afaf7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:05.097703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ca3d424a-4b4c-499f-b2fc-18d3ff9afaf7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:05.106624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca3d424a-4b4c-499f-b2fc-18d3ff9afaf7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:05.110030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca3d424a-4b4c-499f-b2fc-18d3ff9afaf7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:05.115061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca3d424a-4b4c-499f-b2fc-18d3ff9afaf7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:05.115838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca3d424a-4b4c-499f-b2fc-18d3ff9afaf7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:05.116314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca3d424a-4b4c-499f-b2fc-18d3ff9afaf7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:05.116843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca3d424a-4b4c-499f-b2fc-18d3ff9afaf7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:05.229978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cc601224-1333-4db1-aa92-58685ecd6f94 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:05.239568 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cc601224-1333-4db1-aa92-58685ecd6f94 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.456562 Sep 27 10:08:05.240750 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1338/2952] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:04 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:05.246597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1e6f742f-8b12-4a51-8930-0c7983c0b693 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Calling method '.version_select at 0x7f7f629d80d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:05.588788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ca3d424a-4b4c-499f-b2fc-18d3ff9afaf7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:05.598732 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ca3d424a-4b4c-499f-b2fc-18d3ff9afaf7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1314 microversion: 2.1 time: 0.537479 Sep 27 10:08:05.602791 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1615/2953] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:05 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1314 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:05.611180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c41a3b92-b34a-4b4e-9959-ef7fb4891316 req-b3461b48-7873-4eea-91ff-639199838227 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bfc0bc5d-a883-4255-9869-8aaba51eaae2", "name": "network-vif-plugged", "status": "completed", "tag": "8d8cb0aa-16a7-4483-8c75-b88ae1eeb6d4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:05.638459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c41a3b92-b34a-4b4e-9959-ef7fb4891316 req-b3461b48-7873-4eea-91ff-639199838227 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:05.640942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c41a3b92-b34a-4b4e-9959-ef7fb4891316 req-b3461b48-7873-4eea-91ff-639199838227 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:05.642323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c41a3b92-b34a-4b4e-9959-ef7fb4891316 req-b3461b48-7873-4eea-91ff-639199838227 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:05.674635 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c41a3b92-b34a-4b4e-9959-ef7fb4891316 req-b3461b48-7873-4eea-91ff-639199838227 service nova] Creating event network-vif-plugged:8d8cb0aa-16a7-4483-8c75-b88ae1eeb6d4 for instance bfc0bc5d-a883-4255-9869-8aaba51eaae2 on nch1-master-1 Sep 27 10:08:05.687561 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c41a3b92-b34a-4b4e-9959-ef7fb4891316 req-b3461b48-7873-4eea-91ff-639199838227 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082954 Sep 27 10:08:05.688694 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1616/2954] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:05.699779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fd06f998-dd62-44f2-b47e-ffb295915823 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:05.705803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fd06f998-dd62-44f2-b47e-ffb295915823 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:05.722723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd06f998-dd62-44f2-b47e-ffb295915823 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:05.723483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd06f998-dd62-44f2-b47e-ffb295915823 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:05.724317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd06f998-dd62-44f2-b47e-ffb295915823 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:05.725213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd06f998-dd62-44f2-b47e-ffb295915823 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:05.725820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd06f998-dd62-44f2-b47e-ffb295915823 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:05.726426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd06f998-dd62-44f2-b47e-ffb295915823 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:05.989369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fd06f998-dd62-44f2-b47e-ffb295915823 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:06.001857 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fd06f998-dd62-44f2-b47e-ffb295915823 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1314 microversion: 2.1 time: 0.311766 Sep 27 10:08:06.002769 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1617/2955] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:05 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1314 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:06.020641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:06.021287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:06.029010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:06.029746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:06.030344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:06.031263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:06.031877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:06.032461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:06.167320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:06.179358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:06.183942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:06.198081 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8205aa83-7ffb-4d27-9cdf-f79dfd9b115d tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.193977 Sep 27 10:08:06.198698 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1618/2956] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:06 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:06.210912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e2c70402-2da8-4e3c-96b7-dbfc168c1e0b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:06.212116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e2c70402-2da8-4e3c-96b7-dbfc168c1e0b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:06.224286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c70402-2da8-4e3c-96b7-dbfc168c1e0b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:06.226913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c70402-2da8-4e3c-96b7-dbfc168c1e0b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:06.229661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c70402-2da8-4e3c-96b7-dbfc168c1e0b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:06.230745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c70402-2da8-4e3c-96b7-dbfc168c1e0b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:06.234601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c70402-2da8-4e3c-96b7-dbfc168c1e0b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:06.235263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c70402-2da8-4e3c-96b7-dbfc168c1e0b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:06.245143 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1e6f742f-8b12-4a51-8930-0c7983c0b693 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/5920ee72-7918-4382-95c4-dfd3ff0c2510" status: 202 len: 0 microversion: 2.1 time: 1.003098 Sep 27 10:08:06.246584 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1339/2957] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:08:05 2022] DELETE /compute/v2.1/os-security-groups/5920ee72-7918-4382-95c4-dfd3ff0c2510 => generated 0 bytes in 1005 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:06.251351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-76e76cf5-9b88-4320-ae44-3b27fdb4e4d8 req-24da04f8-1fa7-4bee-b366-a3319ae6d1f8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "768e537b-c8ab-418f-9523-ae18bd5a2259", "name": "network-vif-deleted", "tag": "e7426e33-6ec8-40d9-a1de-120d41f1f5cc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:06.285482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-76e76cf5-9b88-4320-ae44-3b27fdb4e4d8 req-24da04f8-1fa7-4bee-b366-a3319ae6d1f8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:06.285970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-76e76cf5-9b88-4320-ae44-3b27fdb4e4d8 req-24da04f8-1fa7-4bee-b366-a3319ae6d1f8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:06.289662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-76e76cf5-9b88-4320-ae44-3b27fdb4e4d8 req-24da04f8-1fa7-4bee-b366-a3319ae6d1f8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:06.324332 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-76e76cf5-9b88-4320-ae44-3b27fdb4e4d8 req-24da04f8-1fa7-4bee-b366-a3319ae6d1f8 service nova] Creating event network-vif-deleted:e7426e33-6ec8-40d9-a1de-120d41f1f5cc for instance 768e537b-c8ab-418f-9523-ae18bd5a2259 on nch2-master-1 Sep 27 10:08:06.330249 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-76e76cf5-9b88-4320-ae44-3b27fdb4e4d8 req-24da04f8-1fa7-4bee-b366-a3319ae6d1f8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082300 Sep 27 10:08:06.331428 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1340/2958] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:06.338711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce9f0888-e511-406e-9f68-3234096ab287 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {"length": 10}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:06.341311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ce9f0888-e511-406e-9f68-3234096ab287 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 0f5929f8-afdd-4b76-af86-5895c7dc34f9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:06.347495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:06.349702 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-ce9f0888-e511-406e-9f68-3234096ab287 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 0f5929f8-afdd-4b76-af86-5895c7dc34f9 could not be found. Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce9f0888-e511-406e-9f68-3234096ab287 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/0f5929f8-afdd-4b76-af86-5895c7dc34f9/action HTTP/1.1 Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 39 Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: {"os-getConsoleOutput": {"length": 10}} Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 0f5929f8-afdd-4b76-af86-5895c7dc34f9 could not be found. Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '0f5929f8-afdd-4b76-af86-5895c7dc34f9', 'body': {'os-getConsoleOutput': {'length': 10}}} Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0f5929f8-afdd-4b76-af86-5895c7dc34f9 could not be found. Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/console_output.py", line 41, in get_console_output Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0f5929f8-afdd-4b76-af86-5895c7dc34f9 could not be found. Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:06.357813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:06.362921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:06.362921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:06.362921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:06.362921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:06.362921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0f5929f8-afdd-4b76-af86-5895c7dc34f9 could not be found. Sep 27 10:08:06.362921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:06.362921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce9f0888-e511-406e-9f68-3234096ab287 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 0f5929f8-afdd-4b76-af86-5895c7dc34f9 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:06.362921 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ce9f0888-e511-406e-9f68-3234096ab287 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/0f5929f8-afdd-4b76-af86-5895c7dc34f9/action" status: 404 len: 111 microversion: 2.1 time: 0.026800 Sep 27 10:08:06.362921 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1341/2959] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:06 2022] POST /compute/v2.1/servers/0f5929f8-afdd-4b76-af86-5895c7dc34f9/action => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:06.368976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-557944e6-c3a9-46b3-9636-0c5b8a66390a tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:06.370034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-557944e6-c3a9-46b3-9636-0c5b8a66390a tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:06.380447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557944e6-c3a9-46b3-9636-0c5b8a66390a tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:06.381072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557944e6-c3a9-46b3-9636-0c5b8a66390a tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:06.381521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557944e6-c3a9-46b3-9636-0c5b8a66390a tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:06.383383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557944e6-c3a9-46b3-9636-0c5b8a66390a tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:06.383877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557944e6-c3a9-46b3-9636-0c5b8a66390a tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:06.384309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557944e6-c3a9-46b3-9636-0c5b8a66390a tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:06.454286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2c70402-2da8-4e3c-96b7-dbfc168c1e0b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:06.464739 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e2c70402-2da8-4e3c-96b7-dbfc168c1e0b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1227 microversion: 2.1 time: 0.257770 Sep 27 10:08:06.468869 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1619/2960] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:06 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1227 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:06.496423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-79e698b7-3785-487b-b00d-c0d3acbcef8e tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Calling method '.version_select at 0x7f7f63ac3e50>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:06.619947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-557944e6-c3a9-46b3-9636-0c5b8a66390a tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '430de2723f234fac8f09068195d8ce2c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fedcdffc12b94b489ce433d5caa5e4be', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:06.639453 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-557944e6-c3a9-46b3-9636-0c5b8a66390a tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 200 len: 1497 microversion: 2.1 time: 0.274674 Sep 27 10:08:06.640051 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1342/2961] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:06 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 1497 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:06.653752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-24d3b8e3-aa65-4858-9b4e-ac4f805b0dd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:06.657417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-24d3b8e3-aa65-4858-9b4e-ac4f805b0dd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:06.668374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24d3b8e3-aa65-4858-9b4e-ac4f805b0dd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:06.675858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24d3b8e3-aa65-4858-9b4e-ac4f805b0dd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:06.676686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24d3b8e3-aa65-4858-9b4e-ac4f805b0dd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:06.677650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24d3b8e3-aa65-4858-9b4e-ac4f805b0dd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:06.678340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24d3b8e3-aa65-4858-9b4e-ac4f805b0dd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:06.678928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24d3b8e3-aa65-4858-9b4e-ac4f805b0dd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:07.128976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-24d3b8e3-aa65-4858-9b4e-ac4f805b0dd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:07.138485 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-24d3b8e3-aa65-4858-9b4e-ac4f805b0dd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.492896 Sep 27 10:08:07.140717 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1343/2962] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:06 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:07.150455 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-79e698b7-3785-487b-b00d-c0d3acbcef8e tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/52056c00-1675-467c-aee9-0ee850a67899" status: 202 len: 0 microversion: 2.1 time: 0.667972 Sep 27 10:08:07.151915 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1620/2963] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:08:06 2022] DELETE /compute/v2.1/os-security-groups/52056c00-1675-467c-aee9-0ee850a67899 => generated 0 bytes in 670 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:07.156795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c32eda65-9b8d-4353-bf9c-8f47f764c33f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:07.157606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c32eda65-9b8d-4353-bf9c-8f47f764c33f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:07.164244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2bd25702-ef35-45dd-922d-afba1598512b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:07.175043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c32eda65-9b8d-4353-bf9c-8f47f764c33f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:07.175532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c32eda65-9b8d-4353-bf9c-8f47f764c33f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:07.175964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c32eda65-9b8d-4353-bf9c-8f47f764c33f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:07.176814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c32eda65-9b8d-4353-bf9c-8f47f764c33f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:07.177183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c32eda65-9b8d-4353-bf9c-8f47f764c33f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:07.177612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c32eda65-9b8d-4353-bf9c-8f47f764c33f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:07.181124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2bd25702-ef35-45dd-922d-afba1598512b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:07.194681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bd25702-ef35-45dd-922d-afba1598512b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:07.196806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bd25702-ef35-45dd-922d-afba1598512b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:07.197482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bd25702-ef35-45dd-922d-afba1598512b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:07.198479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bd25702-ef35-45dd-922d-afba1598512b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:07.199372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bd25702-ef35-45dd-922d-afba1598512b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:07.199948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bd25702-ef35-45dd-922d-afba1598512b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:07.644871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2bd25702-ef35-45dd-922d-afba1598512b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:07.658953 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2bd25702-ef35-45dd-922d-afba1598512b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1501 microversion: 2.1 time: 0.516927 Sep 27 10:08:07.666636 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1344/2964] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:07 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1501 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:07.690616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dfd1ebe3-de71-4b18-a1b0-aa13f5efdee8 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] Calling method '.version_select at 0x7f7f62b02dc0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:07.786213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c32eda65-9b8d-4353-bf9c-8f47f764c33f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:07.796947 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c32eda65-9b8d-4353-bf9c-8f47f764c33f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1314 microversion: 2.1 time: 0.643691 Sep 27 10:08:07.819804 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1621/2965] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:07 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1314 bytes in 667 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:07.836094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-917db750-fa81-4937-acd6-9f76f822ef10 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:07.836781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-917db750-fa81-4937-acd6-9f76f822ef10 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:07.856062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-917db750-fa81-4937-acd6-9f76f822ef10 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:07.856581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-917db750-fa81-4937-acd6-9f76f822ef10 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:07.858064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-917db750-fa81-4937-acd6-9f76f822ef10 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:07.859003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-917db750-fa81-4937-acd6-9f76f822ef10 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:07.861071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-917db750-fa81-4937-acd6-9f76f822ef10 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:07.862818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-917db750-fa81-4937-acd6-9f76f822ef10 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:08.143817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-917db750-fa81-4937-acd6-9f76f822ef10 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:08.172798 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-917db750-fa81-4937-acd6-9f76f822ef10 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.350540 Sep 27 10:08:08.178587 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1622/2966] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:07 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:08.207590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:08.211328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:08.226358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:08.227370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:08.228024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:08.229212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:08.230112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:08.233406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:08.325491 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dfd1ebe3-de71-4b18-a1b0-aa13f5efdee8 tempest-SecurityGroupRulesTestJSON-1927993946 tempest-SecurityGroupRulesTestJSON-1927993946-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/d26bfbb8-a573-4bf2-bd76-5d96aabdbdb1" status: 202 len: 0 microversion: 2.1 time: 0.657331 Sep 27 10:08:08.328033 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1345/2967] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:08:07 2022] DELETE /compute/v2.1/os-security-groups/d26bfbb8-a573-4bf2-bd76-5d96aabdbdb1 => generated 0 bytes in 661 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:08.334225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-757788e3-dc3d-4aeb-8857-59a86767bc3f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:08.335052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-757788e3-dc3d-4aeb-8857-59a86767bc3f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:08.344157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-757788e3-dc3d-4aeb-8857-59a86767bc3f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:08.347214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-757788e3-dc3d-4aeb-8857-59a86767bc3f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:08.347869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-757788e3-dc3d-4aeb-8857-59a86767bc3f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:08.349660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-757788e3-dc3d-4aeb-8857-59a86767bc3f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:08.352270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-757788e3-dc3d-4aeb-8857-59a86767bc3f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:08.363531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-757788e3-dc3d-4aeb-8857-59a86767bc3f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:08.514257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:08.542231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:08.546175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:08.558887 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3ca99ef2-1aa8-47f9-9bb4-1bc8ee9098bc tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.368696 Sep 27 10:08:08.559541 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1623/2968] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:08 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:08.579754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:08.580403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:08.613299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:08.615026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:08.619341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:08.619341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:08.623602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:08.624260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:08.624800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:08.670503 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] HTTP exception thrown: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Request method failure captured: Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 HTTP/1.1 Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssUrCgLigC9SH81tPyx_Z2smIvqriOlzOxLjwou3UdxxR5VrdtDv5ZLspr7gwjUKnJ9eO9sY6_VK0lEb_or5QoOte1wEO6cw3psqOXVHSx1G3EHl2Qng7IOMpa9960M9YvpOEI8way30BSGJK5IxbZEPCLKI85uv_-gSO6hLkR0C4A Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fedcdffc12b94b489ce433d5caa5e4be"}]}] Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestMultiNic-242412447-project Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 430de2723f234fac8f09068195d8ce2c Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestMultiNic-242412447-project Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '768e537b-c8ab-418f-9523-ae18bd5a2259'} Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:08.672244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:08.676137 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:08.676137 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:08.676137 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:08.676137 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:08.676137 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:08.676137 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:08.676137 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:08.676137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Returning 404 to user: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:08.676876 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fd470c7d-b602-4ca0-8548-5e4a92e0b6e2 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 404 len: 111 microversion: 2.1 time: 0.114394 Sep 27 10:08:08.677735 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1624/2969] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:08 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 111 bytes in 117 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:08.689129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-757788e3-dc3d-4aeb-8857-59a86767bc3f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:08.701730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13a921a2-bbbc-4c06-b6be-9f6a47f25dbe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:08.702288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-13a921a2-bbbc-4c06-b6be-9f6a47f25dbe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:08.708040 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-757788e3-dc3d-4aeb-8857-59a86767bc3f tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1281 microversion: 2.1 time: 0.378400 Sep 27 10:08:08.708636 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1346/2970] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:08 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1281 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:08.728243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c62ee966-7f8e-4b18-838a-19c7dd63ff4a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:08.730802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13a921a2-bbbc-4c06-b6be-9f6a47f25dbe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:08.731305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13a921a2-bbbc-4c06-b6be-9f6a47f25dbe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:08.731749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13a921a2-bbbc-4c06-b6be-9f6a47f25dbe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:08.732559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c62ee966-7f8e-4b18-838a-19c7dd63ff4a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3b3e5657-7c59-4e73-9cc1-8578aa84dc42] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:08.735851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13a921a2-bbbc-4c06-b6be-9f6a47f25dbe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:08.736376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13a921a2-bbbc-4c06-b6be-9f6a47f25dbe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:08.736892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13a921a2-bbbc-4c06-b6be-9f6a47f25dbe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:08.750033 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-c62ee966-7f8e-4b18-838a-19c7dd63ff4a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 3b3e5657-7c59-4e73-9cc1-8578aa84dc42 could not be found. Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c62ee966-7f8e-4b18-838a-19c7dd63ff4a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/3b3e5657-7c59-4e73-9cc1-8578aa84dc42 HTTP/1.1 Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 3b3e5657-7c59-4e73-9cc1-8578aa84dc42 could not be found. Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3b3e5657-7c59-4e73-9cc1-8578aa84dc42'} Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3b3e5657-7c59-4e73-9cc1-8578aa84dc42 could not be found. Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3b3e5657-7c59-4e73-9cc1-8578aa84dc42 could not be found. Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:08.756377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:08.760767 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:08.760767 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:08.760767 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:08.760767 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3b3e5657-7c59-4e73-9cc1-8578aa84dc42 could not be found. Sep 27 10:08:08.760767 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:08.760767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c62ee966-7f8e-4b18-838a-19c7dd63ff4a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 3b3e5657-7c59-4e73-9cc1-8578aa84dc42 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:08.760767 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c62ee966-7f8e-4b18-838a-19c7dd63ff4a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3b3e5657-7c59-4e73-9cc1-8578aa84dc42" status: 404 len: 111 microversion: 2.1 time: 0.046901 Sep 27 10:08:08.767226 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1347/2971] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:08 2022] GET /compute/v2.1/servers/3b3e5657-7c59-4e73-9cc1-8578aa84dc42 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:08.783590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-79e2bc04-ca33-4591-bd49-992a26b6361a req-cdb0624e-a1b1-44fb-bce2-d2980f24694c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "768e537b-c8ab-418f-9523-ae18bd5a2259", "name": "network-vif-deleted", "tag": "f6f547a5-211e-44e6-9a66-e9913a99365d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:08.825471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-79e2bc04-ca33-4591-bd49-992a26b6361a req-cdb0624e-a1b1-44fb-bce2-d2980f24694c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:08.825963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-79e2bc04-ca33-4591-bd49-992a26b6361a req-cdb0624e-a1b1-44fb-bce2-d2980f24694c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:08.826399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-79e2bc04-ca33-4591-bd49-992a26b6361a req-cdb0624e-a1b1-44fb-bce2-d2980f24694c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:08.827630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-13a921a2-bbbc-4c06-b6be-9f6a47f25dbe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lazy-loading 'flavor' on Instance uuid 08babbce-373e-4cf5-a138-066b6eeb02c3 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:08:08.867842 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-79e2bc04-ca33-4591-bd49-992a26b6361a req-cdb0624e-a1b1-44fb-bce2-d2980f24694c service nova] Creating event network-vif-deleted:f6f547a5-211e-44e6-9a66-e9913a99365d for instance 768e537b-c8ab-418f-9523-ae18bd5a2259 on nch2-master-1 Sep 27 10:08:08.878098 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-79e2bc04-ca33-4591-bd49-992a26b6361a req-cdb0624e-a1b1-44fb-bce2-d2980f24694c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.108659 Sep 27 10:08:08.882129 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1348/2972] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:08.896015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-24b351b2-ccb1-4d15-a65d-930fc6e195dc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:08.899125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-24b351b2-ccb1-4d15-a65d-930fc6e195dc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:08.914952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24b351b2-ccb1-4d15-a65d-930fc6e195dc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:08.915695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24b351b2-ccb1-4d15-a65d-930fc6e195dc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:08.917024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24b351b2-ccb1-4d15-a65d-930fc6e195dc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:08.922629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24b351b2-ccb1-4d15-a65d-930fc6e195dc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:08.923628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24b351b2-ccb1-4d15-a65d-930fc6e195dc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:08.924266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24b351b2-ccb1-4d15-a65d-930fc6e195dc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:09.308237 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13a921a2-bbbc-4c06-b6be-9f6a47f25dbe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3/action" status: 202 len: 0 microversion: 2.1 time: 0.628783 Sep 27 10:08:09.319391 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1625/2973] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:08 2022] POST /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3/action => generated 0 bytes in 641 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:09.337301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0c6efef5-e1e4-4417-95df-8fa1c165e99f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:09.342492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0c6efef5-e1e4-4417-95df-8fa1c165e99f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:09.366593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c6efef5-e1e4-4417-95df-8fa1c165e99f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:09.368627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c6efef5-e1e4-4417-95df-8fa1c165e99f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:09.369268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c6efef5-e1e4-4417-95df-8fa1c165e99f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:09.372304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c6efef5-e1e4-4417-95df-8fa1c165e99f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:09.374306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c6efef5-e1e4-4417-95df-8fa1c165e99f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:09.375083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c6efef5-e1e4-4417-95df-8fa1c165e99f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:09.762217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-24b351b2-ccb1-4d15-a65d-930fc6e195dc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:09.771687 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-24b351b2-ccb1-4d15-a65d-930fc6e195dc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.884970 Sep 27 10:08:09.777477 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1349/2974] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:08 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 895 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:09.791557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5fcc18b1-9a69-4c4e-a4bd-019472b6e245 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:09.803088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5fcc18b1-9a69-4c4e-a4bd-019472b6e245 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:09.823285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fcc18b1-9a69-4c4e-a4bd-019472b6e245 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:09.823681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fcc18b1-9a69-4c4e-a4bd-019472b6e245 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:09.824127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fcc18b1-9a69-4c4e-a4bd-019472b6e245 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:09.841604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fcc18b1-9a69-4c4e-a4bd-019472b6e245 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:09.842130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fcc18b1-9a69-4c4e-a4bd-019472b6e245 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:09.842587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fcc18b1-9a69-4c4e-a4bd-019472b6e245 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:10.031091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0c6efef5-e1e4-4417-95df-8fa1c165e99f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:10.051046 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0c6efef5-e1e4-4417-95df-8fa1c165e99f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1463 microversion: 2.1 time: 0.729608 Sep 27 10:08:10.051976 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1626/2975] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:09 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1463 bytes in 732 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:10.066482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:10.067114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:10.079966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:10.080472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:10.080962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:10.081676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:10.082146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:10.082579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:10.200243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:10.221074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:10.233129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:10.248311 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ae00d00d-e3aa-46a6-b7a2-6d00663d277c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.195095 Sep 27 10:08:10.264435 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1627/2976] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:10 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:10.276692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c1b2a64d-80da-4cba-bed4-79f678827dca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:10.277655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c1b2a64d-80da-4cba-bed4-79f678827dca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:10.284785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b2a64d-80da-4cba-bed4-79f678827dca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:10.285475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b2a64d-80da-4cba-bed4-79f678827dca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:10.287166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b2a64d-80da-4cba-bed4-79f678827dca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:10.289824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b2a64d-80da-4cba-bed4-79f678827dca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:10.292709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b2a64d-80da-4cba-bed4-79f678827dca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:10.293775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b2a64d-80da-4cba-bed4-79f678827dca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:10.445096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fcc18b1-9a69-4c4e-a4bd-019472b6e245 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:10.467613 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5fcc18b1-9a69-4c4e-a4bd-019472b6e245 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.686983 Sep 27 10:08:10.468987 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1350/2977] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:09 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 689 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:10.473976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7797470b-32da-457a-acf4-898e10399e04 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:10.474793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7797470b-32da-457a-acf4-898e10399e04 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:10.490402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7797470b-32da-457a-acf4-898e10399e04 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:10.491148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7797470b-32da-457a-acf4-898e10399e04 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:10.491833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7797470b-32da-457a-acf4-898e10399e04 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:10.492891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7797470b-32da-457a-acf4-898e10399e04 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:10.493529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7797470b-32da-457a-acf4-898e10399e04 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:10.494151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7797470b-32da-457a-acf4-898e10399e04 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:10.593787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c1b2a64d-80da-4cba-bed4-79f678827dca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:10.626411 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c1b2a64d-80da-4cba-bed4-79f678827dca tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1323 microversion: 2.1 time: 0.360409 Sep 27 10:08:10.651639 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1628/2978] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:10 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1323 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:11.286771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b35213c-50cc-4b8c-af7a-4521651dbca1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:11.301627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1b35213c-50cc-4b8c-af7a-4521651dbca1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Searching by: {'deleted': False, 'project_id': '89a2927af7e8412fa3fd454fbd375459'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:08:11.349016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b35213c-50cc-4b8c-af7a-4521651dbca1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:11.350750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b35213c-50cc-4b8c-af7a-4521651dbca1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:11.351342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b35213c-50cc-4b8c-af7a-4521651dbca1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:11.356401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b35213c-50cc-4b8c-af7a-4521651dbca1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:11.360524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b35213c-50cc-4b8c-af7a-4521651dbca1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:11.361827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b35213c-50cc-4b8c-af7a-4521651dbca1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:11.420782 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1b35213c-50cc-4b8c-af7a-4521651dbca1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/detail" status: 200 len: 15 microversion: 2.1 time: 0.766021 Sep 27 10:08:11.438609 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1629/2979] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:08:10 2022] GET /compute/v2.1/servers/detail => generated 15 bytes in 786 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:08:11.571290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5a9734b3-55db-446c-85ce-697ae45191a4 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Calling method '.version_select at 0x7f7f638394c0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:11.623496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-5a9734b3-55db-446c-85ce-697ae45191a4 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 93b22d94-0870-4936-a36c-3dbb3ae94a60 does not exist", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:08:11.625192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.security_group_api [None req-5a9734b3-55db-446c-85ce-697ae45191a4 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Neutron security group 93b22d94-0870-4936-a36c-3dbb3ae94a60 not found {{(pid=145968) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:08:11.625975 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-5a9734b3-55db-446c-85ce-697ae45191a4 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] HTTP exception thrown: Security group 93b22d94-0870-4936-a36c-3dbb3ae94a60 does not exist Sep 27 10:08:11.625975 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-1ac883b6-57d4-45f1-aec9-76419c36e50d'] Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5a9734b3-55db-446c-85ce-697ae45191a4 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Request method failure captured: Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-security-groups/93b22d94-0870-4936-a36c-3dbb3ae94a60 HTTP/1.1 Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsuKmYjJh_je3_RR_huRXtV99O_s3rea_dZIVorhO35HEK2mxR1anBOh8YhH_uTaJb_yniCYSZ-rhXGWFntduQ_DyVl7003KpqFlH_JGChYysB_F-RWJ6JypN6VPz7x5vZ_h0jFm6foUniKZtQ4GFm2scTf33OJPE0u8M5i6l1OSlq0 Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 7957b9f761414b969a3681f9d9a2945a Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7957b9f761414b969a3681f9d9a2945a"}]}] Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 7957b9f761414b969a3681f9d9a2945a Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupsNegativeTestJSON-80378142-project Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b4056bf8843c453a8f6ca05daef0e096 Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-80378142-project Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: exception: Security group 93b22d94-0870-4936-a36c-3dbb3ae94a60 does not exist Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-1ac883b6-57d4-45f1-aec9-76419c36e50d'] Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '93b22d94-0870-4936-a36c-3dbb3ae94a60'} Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:08:11.628532 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 93b22d94-0870-4936-a36c-3dbb3ae94a60 does not exist Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1ac883b6-57d4-45f1-aec9-76419c36e50d'] Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 178, in delete Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 93b22d94-0870-4936-a36c-3dbb3ae94a60 does not exist Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1ac883b6-57d4-45f1-aec9-76419c36e50d'] Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 181, in delete Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 93b22d94-0870-4936-a36c-3dbb3ae94a60 does not exist Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1ac883b6-57d4-45f1-aec9-76419c36e50d'] Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 93b22d94-0870-4936-a36c-3dbb3ae94a60 does not exist Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-1ac883b6-57d4-45f1-aec9-76419c36e50d'] Sep 27 10:08:11.632393 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:11.639181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5a9734b3-55db-446c-85ce-697ae45191a4 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Returning 404 to user: Security group 93b22d94-0870-4936-a36c-3dbb3ae94a60 does not exist Sep 27 10:08:11.639181 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-1ac883b6-57d4-45f1-aec9-76419c36e50d'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:11.640418 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5a9734b3-55db-446c-85ce-697ae45191a4 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/93b22d94-0870-4936-a36c-3dbb3ae94a60" status: 404 len: 194 microversion: 2.1 time: 0.192676 Sep 27 10:08:11.648855 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1630/2980] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:08:11 2022] DELETE /compute/v2.1/os-security-groups/93b22d94-0870-4936-a36c-3dbb3ae94a60 => generated 194 bytes in 208 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:11.654087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:11.662270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:11.682883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:11.683553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:11.684089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:11.684969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:11.685516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:11.686973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:11.858955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:11.874244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:11.876729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:11.889100 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-429b1b1d-a51c-400f-8bc5-7793adac17da tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1553 microversion: 2.91 time: 0.238748 Sep 27 10:08:11.891954 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1631/2981] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:11 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1553 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:11.919183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:11.936616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Searching by: {'deleted': False} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:08:11.943294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:11.944446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:11.945180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:11.954565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:11.955615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:11.956258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:12.069305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7797470b-32da-457a-acf4-898e10399e04 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:12.086550 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7797470b-32da-457a-acf4-898e10399e04 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 1.616139 Sep 27 10:08:12.092169 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1351/2982] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:10 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 1622 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:12.123686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-47f0d8bf-1a35-4e19-b3de-7fb50e336ee1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:12.124676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-47f0d8bf-1a35-4e19-b3de-7fb50e336ee1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:12.133040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47f0d8bf-1a35-4e19-b3de-7fb50e336ee1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:12.133993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47f0d8bf-1a35-4e19-b3de-7fb50e336ee1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:12.134637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47f0d8bf-1a35-4e19-b3de-7fb50e336ee1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:12.135945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47f0d8bf-1a35-4e19-b3de-7fb50e336ee1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:12.136618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47f0d8bf-1a35-4e19-b3de-7fb50e336ee1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:12.137481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47f0d8bf-1a35-4e19-b3de-7fb50e336ee1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:12.224239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:12.265173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Listed batch of 7 results from cell out of 1000 limit. Returned 7 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:08:12.385387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:12.386202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:12.386856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:12.691234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-47f0d8bf-1a35-4e19-b3de-7fb50e336ee1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:12.704454 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-47f0d8bf-1a35-4e19-b3de-7fb50e336ee1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.596804 Sep 27 10:08:12.706314 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1352/2983] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:12 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:12.723418 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f1b2c880-0464-437c-b99d-bd7fcc6313ea tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/" status: 404 len: 112 microversion: - time: 0.015625 Sep 27 10:08:12.724018 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1353/2984] 10.222.0.26 () {58 vars in 1164 bytes} [Tue Sep 27 10:08:12 2022] DELETE /compute/v2.1/os-security-groups/ => generated 112 bytes in 17 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 27 10:08:12.731777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce8b21fd-156b-4c81-bf5c-39ad2c815e3a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:12.737587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ce8b21fd-156b-4c81-bf5c-39ad2c815e3a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:12.753439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce8b21fd-156b-4c81-bf5c-39ad2c815e3a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:12.754212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce8b21fd-156b-4c81-bf5c-39ad2c815e3a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:12.754861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce8b21fd-156b-4c81-bf5c-39ad2c815e3a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:12.755814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce8b21fd-156b-4c81-bf5c-39ad2c815e3a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:12.756434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce8b21fd-156b-4c81-bf5c-39ad2c815e3a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:12.757055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce8b21fd-156b-4c81-bf5c-39ad2c815e3a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:13.247426 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e1bc4707-a51f-47e2-b724-1b2752913961 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/detail?all_tenants=" status: 200 len: 11046 microversion: 2.1 time: 1.344153 Sep 27 10:08:13.249328 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1632/2985] 10.222.0.26 () {58 vars in 1171 bytes} [Tue Sep 27 10:08:11 2022] GET /compute/v2.1/servers/detail?all_tenants= => generated 11046 bytes in 1347 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:08:13.254941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67c31fd5-1ea7-4dd0-835a-a527fc52258a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1693813773", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "accessIPv4": "1.1.1.1.1.1", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67c31fd5-1ea7-4dd0-835a-a527fc52258a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 238 Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1693813773", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "accessIPv4": "1.1.1.1.1.1", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1693813773', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'accessIPv4': '1.1.1.1.1.1', 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 263, in check Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = func(instance) Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/_format.py", line 253, in is_ipv4 Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return bool(ipaddress.IPv4Address(instance)) Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/ipaddress.py", line 1257, in __init__ Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._ip = self._ip_int_from_string(addr_str) Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/ipaddress.py", line 1144, in _ip_int_from_string Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise AddressValueError("Expected 4 octets in %r" % ip_str) Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ipaddress.AddressValueError: Expected 4 octets in '1.1.1.1.1.1' Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '1.1.1.1.1.1' is not a 'ipv4' Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'format' in schema['properties']['server']['properties']['accessIPv4']: Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'format': 'ipv4', 'type': 'string'} Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi '1.1.1.1.1.1' Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:13.285437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' Sep 27 10:08:13.289491 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:13.292994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67c31fd5-1ea7-4dd0-835a-a527fc52258a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Invalid input for field/attribute accessIPv4. Value: 1.1.1.1.1.1. '1.1.1.1.1.1' is not a 'ipv4' {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:13.294742 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67c31fd5-1ea7-4dd0-835a-a527fc52258a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 139 microversion: 2.1 time: 0.043558 Sep 27 10:08:13.295828 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1633/2986] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:08:13 2022] POST /compute/v2.1/servers => generated 139 bytes in 46 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:08:13.301754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3266b4dc-db8b-4132-af1b-949fd5dedf70 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Calling method '.version_select at 0x7f7f628190d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:13.400426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ce8b21fd-156b-4c81-bf5c-39ad2c815e3a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:13.448483 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ce8b21fd-156b-4c81-bf5c-39ad2c815e3a tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1323 microversion: 2.1 time: 0.722932 Sep 27 10:08:13.452887 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1354/2987] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:12 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1323 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:13.458488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:13.459279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:13.468213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:13.469544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:13.470350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:13.471872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:13.472761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:13.473420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:13.609131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:13.630785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:13.632258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:13.638179 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9dc63c19-fe62-480f-9912-44939a251d8b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1535 microversion: 2.91 time: 0.183890 Sep 27 10:08:13.639450 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1355/2988] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:13 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1535 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:13.644613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ced5f8a0-53f8-42c6-a09b-b5805a3fe44f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:13.652088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ced5f8a0-53f8-42c6-a09b-b5805a3fe44f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Searching by: {'deleted': False, 'project_id': '89a2927af7e8412fa3fd454fbd375459'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:08:13.658824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ced5f8a0-53f8-42c6-a09b-b5805a3fe44f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:13.659656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ced5f8a0-53f8-42c6-a09b-b5805a3fe44f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:13.660315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ced5f8a0-53f8-42c6-a09b-b5805a3fe44f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:13.662192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ced5f8a0-53f8-42c6-a09b-b5805a3fe44f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:13.662982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ced5f8a0-53f8-42c6-a09b-b5805a3fe44f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:13.663706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ced5f8a0-53f8-42c6-a09b-b5805a3fe44f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:13.701175 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ced5f8a0-53f8-42c6-a09b-b5805a3fe44f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/detail?tenant_id=52efcec2c8a14640a8c3df1dc576bbf1" status: 200 len: 15 microversion: 2.1 time: 0.058351 Sep 27 10:08:13.716688 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1356/2989] 10.222.0.26 () {58 vars in 1231 bytes} [Tue Sep 27 10:08:13 2022] GET /compute/v2.1/servers/detail?tenant_id=52efcec2c8a14640a8c3df1dc576bbf1 => generated 15 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:08:13.719052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c791d23b-97a5-43df-bf07-3d366824aa0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:13.722036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c791d23b-97a5-43df-bf07-3d366824aa0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:13.749309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c791d23b-97a5-43df-bf07-3d366824aa0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:13.751606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c791d23b-97a5-43df-bf07-3d366824aa0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:13.752841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c791d23b-97a5-43df-bf07-3d366824aa0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:13.754484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c791d23b-97a5-43df-bf07-3d366824aa0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:13.755158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c791d23b-97a5-43df-bf07-3d366824aa0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:13.755830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c791d23b-97a5-43df-bf07-3d366824aa0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:13.944937 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3266b4dc-db8b-4132-af1b-949fd5dedf70 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups" status: 200 len: 707 microversion: 2.1 time: 0.647774 Sep 27 10:08:13.946316 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1634/2990] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 10:08:13 2022] GET /compute/v2.1/os-security-groups => generated 707 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:13.951838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-2bed052f-7243-41e4-89d0-46bed48b6819 req-5e61d71c-2b95-4189-860f-167d79b2340d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1b7c9d0a-630a-4c9a-8c25-56f06e0d851c", "tag": "4fd1690d-93ce-4d30-a796-27712c3d8637"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:13.979128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2bed052f-7243-41e4-89d0-46bed48b6819 req-5e61d71c-2b95-4189-860f-167d79b2340d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:13.979872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2bed052f-7243-41e4-89d0-46bed48b6819 req-5e61d71c-2b95-4189-860f-167d79b2340d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:13.980335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2bed052f-7243-41e4-89d0-46bed48b6819 req-5e61d71c-2b95-4189-860f-167d79b2340d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.010783 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-2bed052f-7243-41e4-89d0-46bed48b6819 req-5e61d71c-2b95-4189-860f-167d79b2340d service nova] Creating event network-changed:4fd1690d-93ce-4d30-a796-27712c3d8637 for instance 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c on nch2-master-1 Sep 27 10:08:14.021986 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-2bed052f-7243-41e4-89d0-46bed48b6819 req-5e61d71c-2b95-4189-860f-167d79b2340d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.073934 Sep 27 10:08:14.022836 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1635/2991] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:14.029734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f3a86617-120c-4a29-9c6c-a3d1ef8f065c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:14.030473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f3a86617-120c-4a29-9c6c-a3d1ef8f065c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:14.039913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3a86617-120c-4a29-9c6c-a3d1ef8f065c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.040398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3a86617-120c-4a29-9c6c-a3d1ef8f065c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.041128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3a86617-120c-4a29-9c6c-a3d1ef8f065c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.042078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3a86617-120c-4a29-9c6c-a3d1ef8f065c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.042706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3a86617-120c-4a29-9c6c-a3d1ef8f065c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.043502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3a86617-120c-4a29-9c6c-a3d1ef8f065c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.242629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c791d23b-97a5-43df-bf07-3d366824aa0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:14.254775 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c791d23b-97a5-43df-bf07-3d366824aa0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.550641 Sep 27 10:08:14.259204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f3a86617-120c-4a29-9c6c-a3d1ef8f065c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:14.265149 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1357/2992] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:13 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:14.269521 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f3a86617-120c-4a29-9c6c-a3d1ef8f065c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.244325 Sep 27 10:08:14.272696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:14.279092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Searching by: {'tenant_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'deleted': False} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:08:14.284351 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1636/2993] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:14 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:14.287676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.288449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.289991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.291231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.292011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.292754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.295099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ffaf05b8-dc78-47fe-a826-ca54d8b98349 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Calling method '.version_select at 0x7f7f6c2b4550>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:14.327925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:08:14.333032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:14.379973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.380752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.381387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.468456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-ffaf05b8-dc78-47fe-a826-ca54d8b98349 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Error message: {"NeutronError": {"type": "SecurityGroupCannotRemoveDefault", "message": "Insufficient rights for removing default security group.", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:08:14.469077 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-ffaf05b8-dc78-47fe-a826-ca54d8b98349 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] HTTP exception thrown: Insufficient rights for removing default security group. Sep 27 10:08:14.469077 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-ee7171af-1548-47a3-9246-69209a274d01'] Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ffaf05b8-dc78-47fe-a826-ca54d8b98349 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Request method failure captured: Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-security-groups/3a1f0e64-b260-460b-997e-36dc149c062b HTTP/1.1 Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsuKmYjJh_je3_RR_huRXtV99O_s3rea_dZIVorhO35HEK2mxR1anBOh8YhH_uTaJb_yniCYSZ-rhXGWFntduQ_DyVl7003KpqFlH_JGChYysB_F-RWJ6JypN6VPz7x5vZ_h0jFm6foUniKZtQ4GFm2scTf33OJPE0u8M5i6l1OSlq0 Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 7957b9f761414b969a3681f9d9a2945a Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7957b9f761414b969a3681f9d9a2945a"}]}] Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 7957b9f761414b969a3681f9d9a2945a Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupsNegativeTestJSON-80378142-project Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b4056bf8843c453a8f6ca05daef0e096 Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-80378142-project Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: exception: Insufficient rights for removing default security group. Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-ee7171af-1548-47a3-9246-69209a274d01'] Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3a1f0e64-b260-460b-997e-36dc149c062b'} Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 359, in destroy Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutron.delete_security_group(security_group['id']) Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1073, in delete_security_group Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.delete(self.security_group_path % (security_group)) Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 352, in delete Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("DELETE", action, body=body, Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:08:14.471679 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.Conflict: Insufficient rights for removing default security group. Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ee7171af-1548-47a3-9246-69209a274d01'] Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 179, in delete Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_group_api.destroy(context, security_group) Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 364, in destroy Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.Invalid(str(e)) Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.Invalid: Insufficient rights for removing default security group. Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ee7171af-1548-47a3-9246-69209a274d01'] Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 183, in delete Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=exp.format_message()) Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Insufficient rights for removing default security group. Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ee7171af-1548-47a3-9246-69209a274d01'] Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Insufficient rights for removing default security group. Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-ee7171af-1548-47a3-9246-69209a274d01'] Sep 27 10:08:14.475598 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.478807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ffaf05b8-dc78-47fe-a826-ca54d8b98349 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Returning 400 to user: Insufficient rights for removing default security group. Sep 27 10:08:14.478807 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-ee7171af-1548-47a3-9246-69209a274d01'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:14.479736 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ffaf05b8-dc78-47fe-a826-ca54d8b98349 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/3a1f0e64-b260-460b-997e-36dc149c062b" status: 400 len: 182 microversion: 2.1 time: 0.194356 Sep 27 10:08:14.480527 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1637/2994] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:08:14 2022] DELETE /compute/v2.1/os-security-groups/3a1f0e64-b260-460b-997e-36dc149c062b => generated 182 bytes in 195 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:08:14.485423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:14.486675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:14.495554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.496053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.496492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.497246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.497709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.498169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.516129 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] HTTP exception thrown: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Request method failure captured: Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 HTTP/1.1 Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssUrCgLigC9SH81tPyx_Z2smIvqriOlzOxLjwou3UdxxR5VrdtDv5ZLspr7gwjUKnJ9eO9sY6_VK0lEb_or5QoOte1wEO6cw3psqOXVHSx1G3EHl2Qng7IOMpa9960M9YvpOEI8way30BSGJK5IxbZEPCLKI85uv_-gSO6hLkR0C4A Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fedcdffc12b94b489ce433d5caa5e4be"}]}] Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestMultiNic-242412447-project Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 430de2723f234fac8f09068195d8ce2c Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestMultiNic-242412447-project Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '768e537b-c8ab-418f-9523-ae18bd5a2259'} Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:14.517953 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:14.521791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:14.521791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.521791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:14.521791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.521791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.521791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:14.521791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:14.521791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:14.521791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:14.521791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:14.521791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.524168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Returning 404 to user: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:14.525407 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5e8e6407-2598-4c93-b3a8-cccd7616a691 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "DELETE /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 404 len: 111 microversion: 2.1 time: 0.044107 Sep 27 10:08:14.526463 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1638/2995] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:08:14 2022] DELETE /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:14.536622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eb5f7ddd-5421-4780-b6c6-3399146a0a9b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:14.537391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eb5f7ddd-5421-4780-b6c6-3399146a0a9b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:14.543891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb5f7ddd-5421-4780-b6c6-3399146a0a9b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.544422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb5f7ddd-5421-4780-b6c6-3399146a0a9b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.544867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb5f7ddd-5421-4780-b6c6-3399146a0a9b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.545595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb5f7ddd-5421-4780-b6c6-3399146a0a9b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.546058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb5f7ddd-5421-4780-b6c6-3399146a0a9b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.546489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb5f7ddd-5421-4780-b6c6-3399146a0a9b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.693843 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c2a1405f-e389-4b88-b145-ee25c143751f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=52efcec2c8a14640a8c3df1dc576bbf1" status: 200 len: 4825 microversion: 2.1 time: 0.425749 Sep 27 10:08:14.700859 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1358/2996] 10.222.0.26 () {58 vars in 1257 bytes} [Tue Sep 27 10:08:14 2022] GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=52efcec2c8a14640a8c3df1dc576bbf1 => generated 4825 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:14.700859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9e544549-160f-484f-9076-d71a9dbd4d62 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:14.700859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9e544549-160f-484f-9076-d71a9dbd4d62 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:14.716779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-eb5f7ddd-5421-4780-b6c6-3399146a0a9b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:14.723358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e544549-160f-484f-9076-d71a9dbd4d62 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.723925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e544549-160f-484f-9076-d71a9dbd4d62 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.725662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e544549-160f-484f-9076-d71a9dbd4d62 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.727234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e544549-160f-484f-9076-d71a9dbd4d62 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.727958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e544549-160f-484f-9076-d71a9dbd4d62 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.728599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e544549-160f-484f-9076-d71a9dbd4d62 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.733332 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eb5f7ddd-5421-4780-b6c6-3399146a0a9b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.205498 Sep 27 10:08:14.736263 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1639/2997] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:14 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:14.742640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-950edaad-aa90-4709-8480-2af6a826237c tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Calling method '.version_select at 0x7f7f6c2b4dc0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:14.779689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG neutronclient.v2_0.client [None req-950edaad-aa90-4709-8480-2af6a826237c tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 35574e79-0047-46d1-84ed-2f20d07de90d does not exist", "detail": ""}} {{(pid=145968) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:08:14.781404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.security_group_api [None req-950edaad-aa90-4709-8480-2af6a826237c tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Neutron security group 35574e79-0047-46d1-84ed-2f20d07de90d not found {{(pid=145968) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:08:14.781947 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-950edaad-aa90-4709-8480-2af6a826237c tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] HTTP exception thrown: Security group 35574e79-0047-46d1-84ed-2f20d07de90d does not exist Sep 27 10:08:14.781947 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-cbefff74-e124-4ef8-aba7-2a30f9ad241c'] Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-950edaad-aa90-4709-8480-2af6a826237c tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Request method failure captured: Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/os-security-groups/35574e79-0047-46d1-84ed-2f20d07de90d HTTP/1.1 Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsuKmYjJh_je3_RR_huRXtV99O_s3rea_dZIVorhO35HEK2mxR1anBOh8YhH_uTaJb_yniCYSZ-rhXGWFntduQ_DyVl7003KpqFlH_JGChYysB_F-RWJ6JypN6VPz7x5vZ_h0jFm6foUniKZtQ4GFm2scTf33OJPE0u8M5i6l1OSlq0 Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 7957b9f761414b969a3681f9d9a2945a Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7957b9f761414b969a3681f9d9a2945a"}]}] Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 7957b9f761414b969a3681f9d9a2945a Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-SecurityGroupsNegativeTestJSON-80378142-project Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b4056bf8843c453a8f6ca05daef0e096 Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-80378142-project Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: exception: Security group 35574e79-0047-46d1-84ed-2f20d07de90d does not exist Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-cbefff74-e124-4ef8-aba7-2a30f9ad241c'] Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '35574e79-0047-46d1-84ed-2f20d07de90d'} Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:08:14.783812 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 35574e79-0047-46d1-84ed-2f20d07de90d does not exist Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-cbefff74-e124-4ef8-aba7-2a30f9ad241c'] Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 158, in show Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 35574e79-0047-46d1-84ed-2f20d07de90d does not exist Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-cbefff74-e124-4ef8-aba7-2a30f9ad241c'] Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 160, in show Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 35574e79-0047-46d1-84ed-2f20d07de90d does not exist Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-cbefff74-e124-4ef8-aba7-2a30f9ad241c'] Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 35574e79-0047-46d1-84ed-2f20d07de90d does not exist Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-cbefff74-e124-4ef8-aba7-2a30f9ad241c'] Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-950edaad-aa90-4709-8480-2af6a826237c tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Returning 404 to user: Security group 35574e79-0047-46d1-84ed-2f20d07de90d does not exist Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: Neutron server returns request_ids: ['req-cbefff74-e124-4ef8-aba7-2a30f9ad241c'] {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-950edaad-aa90-4709-8480-2af6a826237c tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] 10.222.0.26 "GET /compute/v2.1/os-security-groups/35574e79-0047-46d1-84ed-2f20d07de90d" status: 404 len: 194 microversion: 2.1 time: 0.047871 Sep 27 10:08:14.788064 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1640/2998] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:08:14 2022] GET /compute/v2.1/os-security-groups/35574e79-0047-46d1-84ed-2f20d07de90d => generated 194 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:14.791519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:14.791740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 768e537b-c8ab-418f-9523-ae18bd5a2259] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:14.800180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.800928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.801537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.802588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.803300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.803876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.822393 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] HTTP exception thrown: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Request method failure captured: Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 HTTP/1.1 Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssUrCgLigC9SH81tPyx_Z2smIvqriOlzOxLjwou3UdxxR5VrdtDv5ZLspr7gwjUKnJ9eO9sY6_VK0lEb_or5QoOte1wEO6cw3psqOXVHSx1G3EHl2Qng7IOMpa9960M9YvpOEI8way30BSGJK5IxbZEPCLKI85uv_-gSO6hLkR0C4A Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fedcdffc12b94b489ce433d5caa5e4be"}]}] Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestMultiNic-242412447-project Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 430de2723f234fac8f09068195d8ce2c Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestMultiNic-242412447-project Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '768e537b-c8ab-418f-9523-ae18bd5a2259'} Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:14.824055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.828305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:14.828305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:14.828305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:14.828305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:14.828305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:14.828305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. Sep 27 10:08:14.828305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:14.828305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Returning 404 to user: Instance 768e537b-c8ab-418f-9523-ae18bd5a2259 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:14.828305 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e8d7941b-0ec7-4c76-8c5b-86d3698f1ba9 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259" status: 404 len: 111 microversion: 2.1 time: 0.038206 Sep 27 10:08:14.828305 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1641/2999] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:14 2022] GET /compute/v2.1/servers/768e537b-c8ab-418f-9523-ae18bd5a2259 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:14.839697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:14.840048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:14.848013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.848515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.849173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.849899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.850406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.851081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.936084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:14.946816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:14.948330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:14.954020 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8d78920a-6694-49aa-a782-214b410dda82 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1535 microversion: 2.91 time: 0.124612 Sep 27 10:08:14.954579 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1642/3000] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:14 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1535 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:14.959085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eaf5de22-2a62-4a72-a89d-64c33c9f1629 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:14.966471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eaf5de22-2a62-4a72-a89d-64c33c9f1629 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Searching by: {'tenant_id': '89a2927af7e8412fa3fd454fbd375459', 'deleted': False} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:08:14.981537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf5de22-2a62-4a72-a89d-64c33c9f1629 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.981678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9e544549-160f-484f-9076-d71a9dbd4d62 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:14.981902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf5de22-2a62-4a72-a89d-64c33c9f1629 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.982284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf5de22-2a62-4a72-a89d-64c33c9f1629 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.983051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf5de22-2a62-4a72-a89d-64c33c9f1629 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:14.983595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf5de22-2a62-4a72-a89d-64c33c9f1629 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:14.983919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf5de22-2a62-4a72-a89d-64c33c9f1629 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:14.989688 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9e544549-160f-484f-9076-d71a9dbd4d62 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1323 microversion: 2.1 time: 0.293699 Sep 27 10:08:14.990600 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1359/3001] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:14 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1323 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:15.011904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c24d2f89-6760-4b3f-bba4-6efa20134b60 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Action: 'update', calling method: .version_select at 0x7f7f62acfca0>, body: {"security_group": {"name": "tempest-sg-2098926692", "description": "tempest-description-2109662267"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:15.034774 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eaf5de22-2a62-4a72-a89d-64c33c9f1629 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=89a2927af7e8412fa3fd454fbd375459" status: 200 len: 15 microversion: 2.1 time: 0.078858 Sep 27 10:08:15.035372 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1643/3002] 10.222.0.26 () {58 vars in 1257 bytes} [Tue Sep 27 10:08:14 2022] GET /compute/v2.1/servers/detail?all_tenants=&tenant_id=89a2927af7e8412fa3fd454fbd375459 => generated 15 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:08:15.044961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a767eca7-be75-450d-9806-3e8c024a7261 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1953455734", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "accessIPv6": "notvalid", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:15.049789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG neutronclient.v2_0.client [None req-c24d2f89-6760-4b3f-bba4-6efa20134b60 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Error message: {"NeutronError": {"type": "SecurityGroupNotFound", "message": "Security group 2f629249-08ec-4b60-b9df-9974e4f0118b does not exist", "detail": ""}} {{(pid=145969) _handle_fault_response /usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py:262}} Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a767eca7-be75-450d-9806-3e8c024a7261 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 235 Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1953455734", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "accessIPv6": "notvalid", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1953455734', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'accessIPv6': 'notvalid', 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 263, in check Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = func(instance) Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/_format.py", line 260, in is_ipv6 Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi address = ipaddress.IPv6Address(instance) Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/ipaddress.py", line 1844, in __init__ Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._ip = self._ip_int_from_string(addr_str) Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3.8/ipaddress.py", line 1581, in _ip_int_from_string Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise AddressValueError(msg) Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi ipaddress.AddressValueError: At least 3 parts expected in 'notvalid' Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi The above exception was the direct cause of the following exception: Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'notvalid' is not a 'ipv6' Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'format' in schema['properties']['server']['properties']['accessIPv6']: Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'format': 'ipv6', 'type': 'string'} Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'notvalid' Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.050514 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a767eca7-be75-450d-9806-3e8c024a7261 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Invalid input for field/attribute accessIPv6. Value: notvalid. 'notvalid' is not a 'ipv6' {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a767eca7-be75-450d-9806-3e8c024a7261 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 133 microversion: 2.1 time: 0.015375 Sep 27 10:08:15.059045 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1644/3003] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:08:15 2022] POST /compute/v2.1/servers => generated 133 bytes in 17 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:08:15.061837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.security_group_api [None req-c24d2f89-6760-4b3f-bba4-6efa20134b60 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Neutron security group 2f629249-08ec-4b60-b9df-9974e4f0118b not found {{(pid=145969) get /opt/stack/nova/nova/network/security_group_api.py:313}} Sep 27 10:08:15.062613 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-c24d2f89-6760-4b3f-bba4-6efa20134b60 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] HTTP exception thrown: Security group 2f629249-08ec-4b60-b9df-9974e4f0118b does not exist Sep 27 10:08:15.062613 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-28800fa1-08f9-4d1a-915d-74e7b76f40b0'] Sep 27 10:08:15.063362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:15.063846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c24d2f89-6760-4b3f-bba4-6efa20134b60 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Request method failure captured: Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/os-security-groups/2f629249-08ec-4b60-b9df-9974e4f0118b HTTP/1.1 Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 102 Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsuKmYjJh_je3_RR_huRXtV99O_s3rea_dZIVorhO35HEK2mxR1anBOh8YhH_uTaJb_yniCYSZ-rhXGWFntduQ_DyVl7003KpqFlH_JGChYysB_F-RWJ6JypN6VPz7x5vZ_h0jFm6foUniKZtQ4GFm2scTf33OJPE0u8M5i6l1OSlq0 Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 7957b9f761414b969a3681f9d9a2945a Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7957b9f761414b969a3681f9d9a2945a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7957b9f761414b969a3681f9d9a2945a"}]}] Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 7957b9f761414b969a3681f9d9a2945a Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-SecurityGroupsNegativeTestJSON-80378142 Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-SecurityGroupsNegativeTestJSON-80378142-project Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4056bf8843c453a8f6ca05daef0e096 Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-SecurityGroupsNegativeTestJSON-80378142-project Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: {"security_group": {"name": "tempest-sg-2098926692", "description": "tempest-description-2109662267"}} Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: exception: Security group 2f629249-08ec-4b60-b9df-9974e4f0118b does not exist Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-28800fa1-08f9-4d1a-915d-74e7b76f40b0'] Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2f629249-08ec-4b60-b9df-9974e4f0118b', 'body': {'security_group': {'name': 'tempest-sg-2098926692', 'description': 'tempest-description-2109662267'}}} Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 309, in get Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi group = neutron.show_security_group(id).get('security_group') Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 1068, in show_security_group Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.get(self.security_group_path % (security_group), Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 356, in get Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.retry_request("GET", action, body=body, Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 333, in retry_request Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.do_request(method, action, body=body, Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 297, in do_request Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._handle_fault_response(status_code, replybody, resp) Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/neutron.py", line 196, in wrapper Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi ret = obj(*args, **kwargs) Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 272, in _handle_fault_response Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi exception_handler_v20(status_code, error_body) Sep 27 10:08:15.066868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/neutronclient/v2_0/client.py", line 90, in exception_handler_v20 Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise client_exc(message=error_message, Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi neutronclient.common.exceptions.NotFound: Security group 2f629249-08ec-4b60-b9df-9974e4f0118b does not exist Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-28800fa1-08f9-4d1a-915d-74e7b76f40b0'] Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 246, in update Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi security_group = security_group_api.get(context, id) Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/network/security_group_api.py", line 314, in get Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.SecurityGroupNotFound(str(e)) Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.SecurityGroupNotFound: Security group 2f629249-08ec-4b60-b9df-9974e4f0118b does not exist Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-28800fa1-08f9-4d1a-915d-74e7b76f40b0'] Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/security_groups.py", line 248, in update Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=exp.format_message()) Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Security group 2f629249-08ec-4b60-b9df-9974e4f0118b does not exist Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-28800fa1-08f9-4d1a-915d-74e7b76f40b0'] Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Security group 2f629249-08ec-4b60-b9df-9974e4f0118b does not exist Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Neutron server returns request_ids: ['req-28800fa1-08f9-4d1a-915d-74e7b76f40b0'] Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c24d2f89-6760-4b3f-bba4-6efa20134b60 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] Returning 404 to user: Security group 2f629249-08ec-4b60-b9df-9974e4f0118b does not exist Sep 27 10:08:15.071484 ncdv-master-1 devstack@n-api.service[145969]: Neutron server returns request_ids: ['req-28800fa1-08f9-4d1a-915d-74e7b76f40b0'] {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:15.076231 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c24d2f89-6760-4b3f-bba4-6efa20134b60 tempest-SecurityGroupsNegativeTestJSON-80378142 tempest-SecurityGroupsNegativeTestJSON-80378142-project] 10.222.0.26 "PUT /compute/v2.1/os-security-groups/2f629249-08ec-4b60-b9df-9974e4f0118b" status: 404 len: 194 microversion: 2.1 time: 0.082048 Sep 27 10:08:15.076231 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1360/3004] 10.222.0.26 () {60 vars in 1290 bytes} [Tue Sep 27 10:08:14 2022] PUT /compute/v2.1/os-security-groups/2f629249-08ec-4b60-b9df-9974e4f0118b => generated 194 bytes in 85 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:15.089484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.090279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.090425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.096954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.098034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.098649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.098760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aded66ad-f69c-46ed-8359-8d1f888fa977 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:15.099389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aded66ad-f69c-46ed-8359-8d1f888fa977 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:15.171095 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] HTTP exception thrown: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:08:15.171962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aded66ad-f69c-46ed-8359-8d1f888fa977 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.174440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aded66ad-f69c-46ed-8359-8d1f888fa977 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.178582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aded66ad-f69c-46ed-8359-8d1f888fa977 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Request method failure captured: Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c HTTP/1.1 Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssUrCgLigC9SH81tPyx_Z2smIvqriOlzOxLjwou3UdxxR5VrdtDv5ZLspr7gwjUKnJ9eO9sY6_VK0lEb_or5QoOte1wEO6cw3psqOXVHSx1G3EHl2Qng7IOMpa9960M9YvpOEI8way30BSGJK5IxbZEPCLKI85uv_-gSO6hLkR0C4A Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fedcdffc12b94b489ce433d5caa5e4be"}]}] Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestMultiNic-242412447-project Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 430de2723f234fac8f09068195d8ce2c Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestMultiNic-242412447-project Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '5b5bcf59-770f-40f1-83b0-0738578e921c'} Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:15.185642 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:15.190849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:08:15.190849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.190849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:15.190849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.190849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.190849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:15.190849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:15.190849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:15.190849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:15.190849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:08:15.190849 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.191870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aded66ad-f69c-46ed-8359-8d1f888fa977 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.193043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Returning 404 to user: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:15.193200 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-600a789c-2608-450e-903c-51ca88f15b8f tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "DELETE /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 404 len: 111 microversion: 2.1 time: 0.139230 Sep 27 10:08:15.193983 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1645/3005] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:08:15 2022] DELETE /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 111 bytes in 140 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:15.204115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aded66ad-f69c-46ed-8359-8d1f888fa977 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.024s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.208386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-96eb823a-bf31-43b7-9246-9189fb0a3e5f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:15.209364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aded66ad-f69c-46ed-8359-8d1f888fa977 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.261633 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-96eb823a-bf31-43b7-9246-9189fb0a3e5f tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/detail?status=invalid_status" status: 200 len: 15 microversion: 2.1 time: 0.066446 Sep 27 10:08:15.262336 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1646/3006] 10.222.0.26 () {58 vars in 1189 bytes} [Tue Sep 27 10:08:15 2022] GET /compute/v2.1/servers/detail?status=invalid_status => generated 15 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:08:15.270223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:15.271156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] [instance: 5b5bcf59-770f-40f1-83b0-0738578e921c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:15.283510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.284490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.286920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.290151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.291122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.291879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.312927 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] HTTP exception thrown: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Request method failure captured: Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c HTTP/1.1 Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssUrCgLigC9SH81tPyx_Z2smIvqriOlzOxLjwou3UdxxR5VrdtDv5ZLspr7gwjUKnJ9eO9sY6_VK0lEb_or5QoOte1wEO6cw3psqOXVHSx1G3EHl2Qng7IOMpa9960M9YvpOEI8way30BSGJK5IxbZEPCLKI85uv_-gSO6hLkR0C4A Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/fedcdffc12b94b489ce433d5caa5e4be"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_fedcdffc12b94b489ce433d5caa5e4be"}]}] Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: fedcdffc12b94b489ce433d5caa5e4be Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestMultiNic-242412447 Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestMultiNic-242412447-project Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 430de2723f234fac8f09068195d8ce2c Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestMultiNic-242412447-project Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '5b5bcf59-770f-40f1-83b0-0738578e921c'} Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:15.315879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.323203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:15.323203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:15.323203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:15.323203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:15.323203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:15.323203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. Sep 27 10:08:15.323203 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:15.323203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] Returning 404 to user: Instance 5b5bcf59-770f-40f1-83b0-0738578e921c could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:15.323203 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b5f54dd1-5cb3-463c-855c-04fac30e0464 tempest-ServersTestMultiNic-242412447 tempest-ServersTestMultiNic-242412447-project] 10.222.0.26 "GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c" status: 404 len: 111 microversion: 2.1 time: 0.054268 Sep 27 10:08:15.323203 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1647/3007] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:15 2022] GET /compute/v2.1/servers/5b5bcf59-770f-40f1-83b0-0738578e921c => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:15.335059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f3f064bf-4bdc-480a-b969-1859c7c95a42 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:15.337923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f3f064bf-4bdc-480a-b969-1859c7c95a42 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:15.350272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3f064bf-4bdc-480a-b969-1859c7c95a42 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.350666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3f064bf-4bdc-480a-b969-1859c7c95a42 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.351158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3f064bf-4bdc-480a-b969-1859c7c95a42 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.351934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3f064bf-4bdc-480a-b969-1859c7c95a42 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.352358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3f064bf-4bdc-480a-b969-1859c7c95a42 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.352797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3f064bf-4bdc-480a-b969-1859c7c95a42 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.466646 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f3f064bf-4bdc-480a-b969-1859c7c95a42 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action" status: 202 len: 0 microversion: 2.1 time: 0.142728 Sep 27 10:08:15.472281 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1648/3008] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:15 2022] POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action => generated 0 bytes in 153 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:15.480433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1306dad4-015b-4c9f-9c05-1ba6c51d4ee7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:15.482278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1306dad4-015b-4c9f-9c05-1ba6c51d4ee7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:15.489480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1306dad4-015b-4c9f-9c05-1ba6c51d4ee7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.490182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1306dad4-015b-4c9f-9c05-1ba6c51d4ee7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.490905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1306dad4-015b-4c9f-9c05-1ba6c51d4ee7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.492977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1306dad4-015b-4c9f-9c05-1ba6c51d4ee7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.493752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1306dad4-015b-4c9f-9c05-1ba6c51d4ee7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.494374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1306dad4-015b-4c9f-9c05-1ba6c51d4ee7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.634904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aded66ad-f69c-46ed-8359-8d1f888fa977 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:15.643116 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aded66ad-f69c-46ed-8359-8d1f888fa977 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.565626 Sep 27 10:08:15.643926 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1361/3009] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:15 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:15.649710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bf6151a7-03f9-484d-abcc-e5c0701925b4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:15.655020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bf6151a7-03f9-484d-abcc-e5c0701925b4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:15.655942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bf6151a7-03f9-484d-abcc-e5c0701925b4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Searching by: {'vm_state': ['error'], 'deleted': False, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:08:15.663089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf6151a7-03f9-484d-abcc-e5c0701925b4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.663844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf6151a7-03f9-484d-abcc-e5c0701925b4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.664463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf6151a7-03f9-484d-abcc-e5c0701925b4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.665706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf6151a7-03f9-484d-abcc-e5c0701925b4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.666333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf6151a7-03f9-484d-abcc-e5c0701925b4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.667062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf6151a7-03f9-484d-abcc-e5c0701925b4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.697337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-bf6151a7-03f9-484d-abcc-e5c0701925b4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:08:15.751507 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bf6151a7-03f9-484d-abcc-e5c0701925b4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers?status=error" status: 200 len: 338 microversion: 2.1 time: 0.106271 Sep 27 10:08:15.755297 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1362/3010] 10.222.0.26 () {58 vars in 1150 bytes} [Tue Sep 27 10:08:15 2022] GET /compute/v2.1/servers?status=error => generated 338 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:15.767470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1d470745-2c30-452a-8840-1d2b78bbc88e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:15.768133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1d470745-2c30-452a-8840-1d2b78bbc88e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:15.783071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d470745-2c30-452a-8840-1d2b78bbc88e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.783575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d470745-2c30-452a-8840-1d2b78bbc88e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.786587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d470745-2c30-452a-8840-1d2b78bbc88e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.787513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d470745-2c30-452a-8840-1d2b78bbc88e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.788104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d470745-2c30-452a-8840-1d2b78bbc88e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.788868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d470745-2c30-452a-8840-1d2b78bbc88e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.858601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1306dad4-015b-4c9f-9c05-1ba6c51d4ee7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:15.880149 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1306dad4-015b-4c9f-9c05-1ba6c51d4ee7 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.406238 Sep 27 10:08:15.886237 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1649/3011] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:15 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:15.896027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99dde217-7faa-4f61-9225-f53630c991e1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "active"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:15.899235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-99dde217-7faa-4f61-9225-f53630c991e1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:15.915674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dde217-7faa-4f61-9225-f53630c991e1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.917946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dde217-7faa-4f61-9225-f53630c991e1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.919757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dde217-7faa-4f61-9225-f53630c991e1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:15.921293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dde217-7faa-4f61-9225-f53630c991e1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:15.923293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dde217-7faa-4f61-9225-f53630c991e1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:15.924020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dde217-7faa-4f61-9225-f53630c991e1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.059036 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-99dde217-7faa-4f61-9225-f53630c991e1 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action" status: 202 len: 0 microversion: 2.1 time: 0.171312 Sep 27 10:08:16.066216 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1650/3012] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:15 2022] POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action => generated 0 bytes in 179 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:16.074910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:16.075530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:16.084971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.085817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.086419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.087507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.088122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.088716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.121791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1d470745-2c30-452a-8840-1d2b78bbc88e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:16.130035 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1d470745-2c30-452a-8840-1d2b78bbc88e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.373564 Sep 27 10:08:16.130936 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1363/3013] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:15 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:16.136377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d6b5c588-e31f-4604-84c9-458f00bee4c0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:16.137229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d6b5c588-e31f-4604-84c9-458f00bee4c0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:16.143382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6b5c588-e31f-4604-84c9-458f00bee4c0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.144057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6b5c588-e31f-4604-84c9-458f00bee4c0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.144707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6b5c588-e31f-4604-84c9-458f00bee4c0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.145605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6b5c588-e31f-4604-84c9-458f00bee4c0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.146227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6b5c588-e31f-4604-84c9-458f00bee4c0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.146814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6b5c588-e31f-4604-84c9-458f00bee4c0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.205954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:16.215296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:16.217018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:16.223289 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-63a8b1e3-056c-4082-b791-2e4ae01fe4a5 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1538 microversion: 2.91 time: 0.154877 Sep 27 10:08:16.237349 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1651/3014] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:16 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1538 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:16.253632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0ffd9300-fbf1-4afc-80ad-a2af61b75785 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:16.253632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0ffd9300-fbf1-4afc-80ad-a2af61b75785 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:16.260835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ffd9300-fbf1-4afc-80ad-a2af61b75785 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.261325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ffd9300-fbf1-4afc-80ad-a2af61b75785 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.261991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ffd9300-fbf1-4afc-80ad-a2af61b75785 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.262709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ffd9300-fbf1-4afc-80ad-a2af61b75785 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.263163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ffd9300-fbf1-4afc-80ad-a2af61b75785 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.263582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ffd9300-fbf1-4afc-80ad-a2af61b75785 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.354678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d6b5c588-e31f-4604-84c9-458f00bee4c0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:16.373458 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d6b5c588-e31f-4604-84c9-458f00bee4c0 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1323 microversion: 2.1 time: 0.240836 Sep 27 10:08:16.374008 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1364/3015] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:16 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1323 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:16.379501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e903a005-7d94-4836-a33b-c748461d6723 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:16.380570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e903a005-7d94-4836-a33b-c748461d6723 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: d59205e7-5f65-4e51-b779-b8530a5b3f8e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:16.387756 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-e903a005-7d94-4836-a33b-c748461d6723 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance d59205e7-5f65-4e51-b779-b8530a5b3f8e could not be found. Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e903a005-7d94-4836-a33b-c748461d6723 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/d59205e7-5f65-4e51-b779-b8530a5b3f8e/action HTTP/1.1 Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 13 Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: {"pause": {}} Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance d59205e7-5f65-4e51-b779-b8530a5b3f8e could not be found. Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'd59205e7-5f65-4e51-b779-b8530a5b3f8e', 'body': {'pause': {}}} Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d59205e7-5f65-4e51-b779-b8530a5b3f8e could not be found. Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 36, in _pause Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, ctxt, id) Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d59205e7-5f65-4e51-b779-b8530a5b3f8e could not be found. Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:16.392272 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:16.396369 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:16.396369 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:16.396369 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d59205e7-5f65-4e51-b779-b8530a5b3f8e could not be found. Sep 27 10:08:16.396369 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:16.399239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e903a005-7d94-4836-a33b-c748461d6723 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance d59205e7-5f65-4e51-b779-b8530a5b3f8e could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:16.400603 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e903a005-7d94-4836-a33b-c748461d6723 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/d59205e7-5f65-4e51-b779-b8530a5b3f8e/action" status: 404 len: 111 microversion: 2.1 time: 0.025234 Sep 27 10:08:16.413215 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1365/3016] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:16 2022] POST /compute/v2.1/servers/d59205e7-5f65-4e51-b779-b8530a5b3f8e/action => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:16.416547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7296534c-3e3a-44ff-97ab-1a0bbacc829c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Action: 'action', calling method: >, body: {"unshelve": {"host": "nch2-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:16.419832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7296534c-3e3a-44ff-97ab-1a0bbacc829c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:16.437092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7296534c-3e3a-44ff-97ab-1a0bbacc829c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.438167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7296534c-3e3a-44ff-97ab-1a0bbacc829c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.439169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7296534c-3e3a-44ff-97ab-1a0bbacc829c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.440534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7296534c-3e3a-44ff-97ab-1a0bbacc829c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.441478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7296534c-3e3a-44ff-97ab-1a0bbacc829c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.442141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7296534c-3e3a-44ff-97ab-1a0bbacc829c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.500893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7296534c-3e3a-44ff-97ab-1a0bbacc829c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "nch2-master-1". {{(pid=145969) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 27 10:08:16.597246 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7296534c-3e3a-44ff-97ab-1a0bbacc829c tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618/action" status: 202 len: 0 microversion: 2.91 time: 0.185325 Sep 27 10:08:16.602664 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1366/3017] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:08:16 2022] POST /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618/action => generated 0 bytes in 192 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:08:16.621161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-34b08ad9-80ff-488c-8760-e8477867dbed tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:16.622014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-34b08ad9-80ff-488c-8760-e8477867dbed tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:16.629136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b08ad9-80ff-488c-8760-e8477867dbed tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.629655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b08ad9-80ff-488c-8760-e8477867dbed tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.630278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b08ad9-80ff-488c-8760-e8477867dbed tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.631021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b08ad9-80ff-488c-8760-e8477867dbed tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.631479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b08ad9-80ff-488c-8760-e8477867dbed tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.631907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b08ad9-80ff-488c-8760-e8477867dbed tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.895323 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0ffd9300-fbf1-4afc-80ad-a2af61b75785 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1613 microversion: 2.1 time: 0.656429 Sep 27 10:08:16.901657 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1652/3018] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:16 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1613 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:16.919767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:16.933158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:16.939982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.941819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.942417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:16.943251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:16.943814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:16.944486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:17.069229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:17.086824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:17.088256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:17.104171 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ec8d1b3b-c08f-4511-aa14-862cc974ddff tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1486 microversion: 2.91 time: 0.194309 Sep 27 10:08:17.110165 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1653/3019] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:16 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1486 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:17.115504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2aa79d95-b063-48bd-9315-f67c4a79fb90 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:17.116135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2aa79d95-b063-48bd-9315-f67c4a79fb90 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:17.144072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa79d95-b063-48bd-9315-f67c4a79fb90 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:17.144488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa79d95-b063-48bd-9315-f67c4a79fb90 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:17.144994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa79d95-b063-48bd-9315-f67c4a79fb90 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:17.145739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa79d95-b063-48bd-9315-f67c4a79fb90 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:17.146264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa79d95-b063-48bd-9315-f67c4a79fb90 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:17.147751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2aa79d95-b063-48bd-9315-f67c4a79fb90 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:17.411967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-34b08ad9-80ff-488c-8760-e8477867dbed tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:17.426059 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-34b08ad9-80ff-488c-8760-e8477867dbed tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.821108 Sep 27 10:08:17.442881 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1367/3020] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:16 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 823 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:17.442881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-04ad719f-1144-4dd5-ba78-9a4dc4cd93c4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:17.442881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-04ad719f-1144-4dd5-ba78-9a4dc4cd93c4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:17.442881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ad719f-1144-4dd5-ba78-9a4dc4cd93c4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:17.442881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ad719f-1144-4dd5-ba78-9a4dc4cd93c4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:17.444369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ad719f-1144-4dd5-ba78-9a4dc4cd93c4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:17.445477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ad719f-1144-4dd5-ba78-9a4dc4cd93c4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:17.452741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ad719f-1144-4dd5-ba78-9a4dc4cd93c4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:17.453447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-04ad719f-1144-4dd5-ba78-9a4dc4cd93c4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:17.535926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2aa79d95-b063-48bd-9315-f67c4a79fb90 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:17.545076 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2aa79d95-b063-48bd-9315-f67c4a79fb90 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.433671 Sep 27 10:08:17.546236 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1654/3021] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:17 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:17.551191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69454d02-f2a3-4fe2-a15c-cc7228134d6d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:17.552054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69454d02-f2a3-4fe2-a15c-cc7228134d6d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:17.569311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69454d02-f2a3-4fe2-a15c-cc7228134d6d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:17.570368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69454d02-f2a3-4fe2-a15c-cc7228134d6d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:17.571395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69454d02-f2a3-4fe2-a15c-cc7228134d6d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:17.572510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69454d02-f2a3-4fe2-a15c-cc7228134d6d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:17.573332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69454d02-f2a3-4fe2-a15c-cc7228134d6d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:17.573971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69454d02-f2a3-4fe2-a15c-cc7228134d6d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:17.592865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:17.890521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69454d02-f2a3-4fe2-a15c-cc7228134d6d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:17.905790 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69454d02-f2a3-4fe2-a15c-cc7228134d6d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1323 microversion: 2.1 time: 0.357980 Sep 27 10:08:17.907026 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1655/3022] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:17 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1323 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:17.914514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-521780ec-2ac4-4b97-a386-d345b9eb6235 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:17.915364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-521780ec-2ac4-4b97-a386-d345b9eb6235 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:17.925913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521780ec-2ac4-4b97-a386-d345b9eb6235 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:17.926691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521780ec-2ac4-4b97-a386-d345b9eb6235 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:17.927299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521780ec-2ac4-4b97-a386-d345b9eb6235 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:17.928167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521780ec-2ac4-4b97-a386-d345b9eb6235 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:17.928872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521780ec-2ac4-4b97-a386-d345b9eb6235 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:17.929844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521780ec-2ac4-4b97-a386-d345b9eb6235 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.008540 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-04ad719f-1144-4dd5-ba78-9a4dc4cd93c4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1613 microversion: 2.1 time: 0.580041 Sep 27 10:08:18.009446 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1368/3023] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:17 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1613 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:18.039948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-62574e8a-8de0-49ee-a4ec-0f7c8a02f7a4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:18.047267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-62574e8a-8de0-49ee-a4ec-0f7c8a02f7a4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Searching by: {'host': 'nch1-master-1', 'deleted': False} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:08:18.053185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62574e8a-8de0-49ee-a4ec-0f7c8a02f7a4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.053963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62574e8a-8de0-49ee-a4ec-0f7c8a02f7a4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.054622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62574e8a-8de0-49ee-a4ec-0f7c8a02f7a4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.055571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62574e8a-8de0-49ee-a4ec-0f7c8a02f7a4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.056099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62574e8a-8de0-49ee-a4ec-0f7c8a02f7a4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.056548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62574e8a-8de0-49ee-a4ec-0f7c8a02f7a4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.189992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-62574e8a-8de0-49ee-a4ec-0f7c8a02f7a4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Listed batch of 3 results from cell out of 1000 limit. Returned 3 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:08:18.294648 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-62574e8a-8de0-49ee-a4ec-0f7c8a02f7a4 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers?host=nch1-master-1&all_tenants=1" status: 200 len: 990 microversion: 2.1 time: 0.261564 Sep 27 10:08:18.296068 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1369/3024] 10.222.0.26 () {58 vars in 1190 bytes} [Tue Sep 27 10:08:18 2022] GET /compute/v2.1/servers?host=nch1-master-1&all_tenants=1 => generated 990 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:18.304783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:18.305699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:18.313416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.314213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.314846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.316264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.317319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.317931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.538895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:18.551634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:18.553161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:18.561388 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e9a8c2b8-dfb7-46e8-94bc-8ed9364e7fe4 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.263453 Sep 27 10:08:18.562325 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1370/3025] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:18 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:18.568386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a58df45d-c12d-41bc-9297-bb491621caeb tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:18.575050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a58df45d-c12d-41bc-9297-bb491621caeb tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Searching by: {'host': 'nonexistent_host', 'deleted': False} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:08:18.582285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a58df45d-c12d-41bc-9297-bb491621caeb tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.582984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a58df45d-c12d-41bc-9297-bb491621caeb tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.583232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a58df45d-c12d-41bc-9297-bb491621caeb tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.583982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a58df45d-c12d-41bc-9297-bb491621caeb tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.584446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a58df45d-c12d-41bc-9297-bb491621caeb tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.584963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a58df45d-c12d-41bc-9297-bb491621caeb tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.615963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-521780ec-2ac4-4b97-a386-d345b9eb6235 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:18.624397 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a58df45d-c12d-41bc-9297-bb491621caeb tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers?host=nonexistent_host&all_tenants=1" status: 200 len: 15 microversion: 2.1 time: 0.060853 Sep 27 10:08:18.626216 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1371/3026] 10.222.0.26 () {58 vars in 1196 bytes} [Tue Sep 27 10:08:18 2022] GET /compute/v2.1/servers?host=nonexistent_host&all_tenants=1 => generated 15 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:08:18.629415 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-521780ec-2ac4-4b97-a386-d345b9eb6235 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.720428 Sep 27 10:08:18.630020 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1656/3027] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:17 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 722 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:18.642966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cacb2409-3979-49bc-9186-f8e1a095da64 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:18.645273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cacb2409-3979-49bc-9186-f8e1a095da64 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:18.645906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-81576480-dd14-48d5-9e49-83b81ba8a0b9 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:18.653170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-81576480-dd14-48d5-9e49-83b81ba8a0b9 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:18.657774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cacb2409-3979-49bc-9186-f8e1a095da64 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.658289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cacb2409-3979-49bc-9186-f8e1a095da64 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.658859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cacb2409-3979-49bc-9186-f8e1a095da64 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.659478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cacb2409-3979-49bc-9186-f8e1a095da64 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.659951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cacb2409-3979-49bc-9186-f8e1a095da64 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.660387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cacb2409-3979-49bc-9186-f8e1a095da64 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.677583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81576480-dd14-48d5-9e49-83b81ba8a0b9 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.678545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81576480-dd14-48d5-9e49-83b81ba8a0b9 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.679280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81576480-dd14-48d5-9e49-83b81ba8a0b9 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.680717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81576480-dd14-48d5-9e49-83b81ba8a0b9 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.682073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81576480-dd14-48d5-9e49-83b81ba8a0b9 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.682874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81576480-dd14-48d5-9e49-83b81ba8a0b9 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.804761 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-81576480-dd14-48d5-9e49-83b81ba8a0b9 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action" status: 202 len: 0 microversion: 2.1 time: 0.165817 Sep 27 10:08:18.809520 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1657/3028] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:18 2022] POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action => generated 0 bytes in 172 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:08:18.815721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d530ec74-fd86-4cc8-981c-3c197d6297c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:18.816430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d530ec74-fd86-4cc8-981c-3c197d6297c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:18.824470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d530ec74-fd86-4cc8-981c-3c197d6297c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.825460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d530ec74-fd86-4cc8-981c-3c197d6297c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.826160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d530ec74-fd86-4cc8-981c-3c197d6297c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.827200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d530ec74-fd86-4cc8-981c-3c197d6297c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:18.827940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d530ec74-fd86-4cc8-981c-3c197d6297c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:18.828640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d530ec74-fd86-4cc8-981c-3c197d6297c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:18.957655 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d530ec74-fd86-4cc8-981c-3c197d6297c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 202 len: 0 microversion: 2.1 time: 0.146427 Sep 27 10:08:18.960748 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1658/3029] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:18 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 0 bytes in 149 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:18.982222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:18.986952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:18.996516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:19.000938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:19.001484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:19.002370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:19.002996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:19.003574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:19.139928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cacb2409-3979-49bc-9186-f8e1a095da64 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:19.148916 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cacb2409-3979-49bc-9186-f8e1a095da64 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.514753 Sep 27 10:08:19.161816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:08:19.162934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:08:19.164882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:08:19.165504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:08:19.165823 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1372/3030] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:18 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:19.166359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:08:19.171497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b043dfc-11e3-40e3-a7ca-c14fae5c8fae tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:19.172131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3b043dfc-11e3-40e3-a7ca-c14fae5c8fae tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:19.180045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b043dfc-11e3-40e3-a7ca-c14fae5c8fae tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:19.181228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b043dfc-11e3-40e3-a7ca-c14fae5c8fae tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:19.188306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b043dfc-11e3-40e3-a7ca-c14fae5c8fae tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:19.193208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b043dfc-11e3-40e3-a7ca-c14fae5c8fae tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:19.199025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b043dfc-11e3-40e3-a7ca-c14fae5c8fae tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:19.203931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b043dfc-11e3-40e3-a7ca-c14fae5c8fae tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:19.356577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:19.375694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:19.377476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:19.378935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:19.381400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:19.384000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:19.384590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:19.465680 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d2d9a43-b0fe-4f69-ac92-f4cbf93b0958 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action" status: 202 len: 1066 microversion: 2.1 time: 0.503586 Sep 27 10:08:19.472716 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1659/3031] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:18 2022] POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action => generated 1066 bytes in 507 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Sep 27 10:08:19.480985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad0f7d0d-e7a7-4d47-a693-717ffac4f498 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:19.481887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ad0f7d0d-e7a7-4d47-a693-717ffac4f498 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:19.492582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad0f7d0d-e7a7-4d47-a693-717ffac4f498 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:19.494757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad0f7d0d-e7a7-4d47-a693-717ffac4f498 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:19.495277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad0f7d0d-e7a7-4d47-a693-717ffac4f498 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:19.497402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad0f7d0d-e7a7-4d47-a693-717ffac4f498 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:19.497883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad0f7d0d-e7a7-4d47-a693-717ffac4f498 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:19.498320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad0f7d0d-e7a7-4d47-a693-717ffac4f498 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:19.684572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3b043dfc-11e3-40e3-a7ca-c14fae5c8fae tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:19.693555 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b043dfc-11e3-40e3-a7ca-c14fae5c8fae tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1323 microversion: 2.1 time: 0.526538 Sep 27 10:08:19.694139 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1373/3032] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:19 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1323 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:19.739062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dea395a6-ea2e-4492-a5f5-62b4d58813e7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:19.739876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dea395a6-ea2e-4492-a5f5-62b4d58813e7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:19.749423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dea395a6-ea2e-4492-a5f5-62b4d58813e7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:19.750240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dea395a6-ea2e-4492-a5f5-62b4d58813e7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:19.751702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dea395a6-ea2e-4492-a5f5-62b4d58813e7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:19.753536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dea395a6-ea2e-4492-a5f5-62b4d58813e7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:19.754174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dea395a6-ea2e-4492-a5f5-62b4d58813e7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:19.754814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dea395a6-ea2e-4492-a5f5-62b4d58813e7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.075738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dea395a6-ea2e-4492-a5f5-62b4d58813e7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:20.097014 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dea395a6-ea2e-4492-a5f5-62b4d58813e7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1470 microversion: 2.1 time: 0.400694 Sep 27 10:08:20.099520 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1374/3033] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:19 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1470 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:20.117359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:20.118431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:20.133167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.138271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.139227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.140371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.140989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.141632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.153146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:20.306019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:20.320136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:20.323789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:20.339912 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-450e276d-50c6-4beb-8859-6271f43d440b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.237592 Sep 27 10:08:20.340439 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1375/3034] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:20 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:20.350446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-48013c1a-d32c-4a27-8375-96ba30c20dcc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:20.356213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-48013c1a-d32c-4a27-8375-96ba30c20dcc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:20.375241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48013c1a-d32c-4a27-8375-96ba30c20dcc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.377876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48013c1a-d32c-4a27-8375-96ba30c20dcc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.378515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48013c1a-d32c-4a27-8375-96ba30c20dcc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.384571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48013c1a-d32c-4a27-8375-96ba30c20dcc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.385305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48013c1a-d32c-4a27-8375-96ba30c20dcc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.385990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48013c1a-d32c-4a27-8375-96ba30c20dcc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.472610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ad0f7d0d-e7a7-4d47-a693-717ffac4f498 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:20.484882 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ad0f7d0d-e7a7-4d47-a693-717ffac4f498 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1490 microversion: 2.1 time: 1.014094 Sep 27 10:08:20.485755 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1660/3035] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:19 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1490 bytes in 1018 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:20.512922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:20.512922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:20.519583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.520169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.520687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.521567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.522078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.522511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.551638 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Cannot 'pause' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state paused Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action HTTP/1.1 Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 13 Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: {"pause": {}} Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: exception: Cannot 'pause' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state paused Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3327ff95-5c05-4167-a73b-5fd55f7bfabd', 'body': {'pause': {}}} Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 41, in _pause Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.pause(ctxt, server) Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd in vm_state paused. Cannot pause while the instance is in this state. Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 45, in _pause Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'pause' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state paused Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:20.553558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:20.558101 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:20.558101 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:20.558101 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:20.558101 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'pause' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state paused Sep 27 10:08:20.558101 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:20.558101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 409 to user: Cannot 'pause' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state paused {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:20.558101 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-591ed223-4800-4016-93b3-7e8d22ea43e2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 409 len: 143 microversion: 2.1 time: 0.048054 Sep 27 10:08:20.559163 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1661/3036] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:20 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 143 bytes in 53 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Sep 27 10:08:20.564564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e4b4acf3-24d9-4537-9836-23a1dc21d48b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:20.565583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e4b4acf3-24d9-4537-9836-23a1dc21d48b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:20.573922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4b4acf3-24d9-4537-9836-23a1dc21d48b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.574424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4b4acf3-24d9-4537-9836-23a1dc21d48b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.574874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4b4acf3-24d9-4537-9836-23a1dc21d48b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.575605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4b4acf3-24d9-4537-9836-23a1dc21d48b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.576070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4b4acf3-24d9-4537-9836-23a1dc21d48b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.576500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4b4acf3-24d9-4537-9836-23a1dc21d48b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.698453 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e4b4acf3-24d9-4537-9836-23a1dc21d48b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 202 len: 0 microversion: 2.1 time: 0.135970 Sep 27 10:08:20.704719 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1662/3037] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:20 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 0 bytes in 145 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:20.719407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5f4e468b-57e3-4ac3-a539-fb13bc300385 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:20.720151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5f4e468b-57e3-4ac3-a539-fb13bc300385 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:20.739409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4e468b-57e3-4ac3-a539-fb13bc300385 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.739931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4e468b-57e3-4ac3-a539-fb13bc300385 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.741665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4e468b-57e3-4ac3-a539-fb13bc300385 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.743507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4e468b-57e3-4ac3-a539-fb13bc300385 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.744019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4e468b-57e3-4ac3-a539-fb13bc300385 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.746798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4e468b-57e3-4ac3-a539-fb13bc300385 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.880367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-48013c1a-d32c-4a27-8375-96ba30c20dcc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:20.903047 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-48013c1a-d32c-4a27-8375-96ba30c20dcc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.559081 Sep 27 10:08:20.907106 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1376/3038] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:20 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:20.925288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-56a99afe-0b86-4cc6-9dd0-c6bc6ed92e92 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:20.926377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-56a99afe-0b86-4cc6-9dd0-c6bc6ed92e92 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:20.949269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a99afe-0b86-4cc6-9dd0-c6bc6ed92e92 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.954469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a99afe-0b86-4cc6-9dd0-c6bc6ed92e92 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.956202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a99afe-0b86-4cc6-9dd0-c6bc6ed92e92 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:20.958088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a99afe-0b86-4cc6-9dd0-c6bc6ed92e92 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:20.962380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a99afe-0b86-4cc6-9dd0-c6bc6ed92e92 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:20.964530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a99afe-0b86-4cc6-9dd0-c6bc6ed92e92 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:21.551947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-56a99afe-0b86-4cc6-9dd0-c6bc6ed92e92 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:21.561502 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-56a99afe-0b86-4cc6-9dd0-c6bc6ed92e92 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1323 microversion: 2.1 time: 0.651572 Sep 27 10:08:21.569935 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1377/3039] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:20 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1323 bytes in 660 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:21.584546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9b48d814-836e-4a1a-8ceb-85f947823d36 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:21.586774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9b48d814-836e-4a1a-8ceb-85f947823d36 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:21.610132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b48d814-836e-4a1a-8ceb-85f947823d36 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:21.614331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b48d814-836e-4a1a-8ceb-85f947823d36 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:21.617413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b48d814-836e-4a1a-8ceb-85f947823d36 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:21.621166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b48d814-836e-4a1a-8ceb-85f947823d36 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:21.622215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b48d814-836e-4a1a-8ceb-85f947823d36 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:21.625013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b48d814-836e-4a1a-8ceb-85f947823d36 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:21.955144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9b48d814-836e-4a1a-8ceb-85f947823d36 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:21.970455 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9b48d814-836e-4a1a-8ceb-85f947823d36 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1470 microversion: 2.1 time: 0.397422 Sep 27 10:08:21.971187 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1378/3040] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:21 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1470 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:21.990694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:21.994443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:22.008352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:22.022429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:22.023145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:22.024343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:22.024726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f4e468b-57e3-4ac3-a539-fb13bc300385 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:22.025858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:22.034809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:22.054844 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5f4e468b-57e3-4ac3-a539-fb13bc300385 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1497 microversion: 2.1 time: 1.334208 Sep 27 10:08:22.065856 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1663/3041] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:20 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1497 bytes in 1351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:22.079633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2cc93ceb-5b7b-4349-aed8-fbfc9fdf9bd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:22.081464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2cc93ceb-5b7b-4349-aed8-fbfc9fdf9bd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:22.113001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cc93ceb-5b7b-4349-aed8-fbfc9fdf9bd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:22.113898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cc93ceb-5b7b-4349-aed8-fbfc9fdf9bd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:22.114558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cc93ceb-5b7b-4349-aed8-fbfc9fdf9bd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:22.115955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cc93ceb-5b7b-4349-aed8-fbfc9fdf9bd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:22.117036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cc93ceb-5b7b-4349-aed8-fbfc9fdf9bd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:22.117880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cc93ceb-5b7b-4349-aed8-fbfc9fdf9bd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:22.238560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:22.252907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:22.265194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:22.279753 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-72912fc4-2672-4d99-93f0-06609d9c5993 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.304549 Sep 27 10:08:22.281451 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1379/3042] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:21 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:22.371426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:22.596222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5b084207-6028-45e5-9087-024ff8bf9222 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:22.597350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5b084207-6028-45e5-9087-024ff8bf9222 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:22.617135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b084207-6028-45e5-9087-024ff8bf9222 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:22.620487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b084207-6028-45e5-9087-024ff8bf9222 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:22.623214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b084207-6028-45e5-9087-024ff8bf9222 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:22.627431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b084207-6028-45e5-9087-024ff8bf9222 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:22.632567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b084207-6028-45e5-9087-024ff8bf9222 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:22.633537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b084207-6028-45e5-9087-024ff8bf9222 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:22.658982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2cc93ceb-5b7b-4349-aed8-fbfc9fdf9bd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:22.668087 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2cc93ceb-5b7b-4349-aed8-fbfc9fdf9bd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.605652 Sep 27 10:08:22.671516 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1664/3043] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:22 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:22.999612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b217864e-6358-4f91-9212-a085392ec88b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:23.000937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b217864e-6358-4f91-9212-a085392ec88b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:23.015804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b217864e-6358-4f91-9212-a085392ec88b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:23.018494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b217864e-6358-4f91-9212-a085392ec88b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:23.019320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b217864e-6358-4f91-9212-a085392ec88b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:23.020347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b217864e-6358-4f91-9212-a085392ec88b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:23.023461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b217864e-6358-4f91-9212-a085392ec88b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:23.025994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b217864e-6358-4f91-9212-a085392ec88b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:23.075774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b084207-6028-45e5-9087-024ff8bf9222 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:23.102849 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5b084207-6028-45e5-9087-024ff8bf9222 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1323 microversion: 2.1 time: 0.511856 Sep 27 10:08:23.105531 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1380/3044] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:22 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1323 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:23.117995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e8b7b9cf-3bde-4671-8dcf-25842e32d545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:23.122920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e8b7b9cf-3bde-4671-8dcf-25842e32d545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:23.143626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8b7b9cf-3bde-4671-8dcf-25842e32d545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:23.181974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8b7b9cf-3bde-4671-8dcf-25842e32d545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.038s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:23.184215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8b7b9cf-3bde-4671-8dcf-25842e32d545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:23.187470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8b7b9cf-3bde-4671-8dcf-25842e32d545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:23.189897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8b7b9cf-3bde-4671-8dcf-25842e32d545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:23.194036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8b7b9cf-3bde-4671-8dcf-25842e32d545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:23.572058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e8b7b9cf-3bde-4671-8dcf-25842e32d545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:23.586091 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e8b7b9cf-3bde-4671-8dcf-25842e32d545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.478149 Sep 27 10:08:23.587121 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1381/3045] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:23 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:23.597321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-3dc5c57a-4e86-4684-be7a-1c295cac4870 req-9d1658c9-7d69-4cfe-9c12-68eddcf06866 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1b7c9d0a-630a-4c9a-8c25-56f06e0d851c", "name": "network-vif-plugged", "status": "completed", "tag": "4fd1690d-93ce-4d30-a796-27712c3d8637"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:23.614523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3dc5c57a-4e86-4684-be7a-1c295cac4870 req-9d1658c9-7d69-4cfe-9c12-68eddcf06866 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:23.615108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3dc5c57a-4e86-4684-be7a-1c295cac4870 req-9d1658c9-7d69-4cfe-9c12-68eddcf06866 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:23.615558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3dc5c57a-4e86-4684-be7a-1c295cac4870 req-9d1658c9-7d69-4cfe-9c12-68eddcf06866 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:23.644444 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-3dc5c57a-4e86-4684-be7a-1c295cac4870 req-9d1658c9-7d69-4cfe-9c12-68eddcf06866 service nova] Creating event network-vif-plugged:4fd1690d-93ce-4d30-a796-27712c3d8637 for instance 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c on nch2-master-1 Sep 27 10:08:23.648444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b217864e-6358-4f91-9212-a085392ec88b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:23.652071 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-3dc5c57a-4e86-4684-be7a-1c295cac4870 req-9d1658c9-7d69-4cfe-9c12-68eddcf06866 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063289 Sep 27 10:08:23.653127 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1382/3046] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:23.659488 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b217864e-6358-4f91-9212-a085392ec88b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1470 microversion: 2.1 time: 0.662914 Sep 27 10:08:23.661403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:23.662242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:23.676973 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1665/3047] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:22 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1470 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:23.685556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-44d95311-bcbb-4fab-8720-c2fff7f06c65 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:23.687268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-44d95311-bcbb-4fab-8720-c2fff7f06c65 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:23.696790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44d95311-bcbb-4fab-8720-c2fff7f06c65 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:23.697727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44d95311-bcbb-4fab-8720-c2fff7f06c65 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:23.698376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44d95311-bcbb-4fab-8720-c2fff7f06c65 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:23.699307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44d95311-bcbb-4fab-8720-c2fff7f06c65 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:23.700031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44d95311-bcbb-4fab-8720-c2fff7f06c65 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:23.700679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44d95311-bcbb-4fab-8720-c2fff7f06c65 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:23.710025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:23.710709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:23.711234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:23.712120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:23.712591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:23.722739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:24.026403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:24.043128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:24.048515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:24.062671 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b8c07b5a-dd5c-49f9-84b8-586f36f90069 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.407265 Sep 27 10:08:24.065766 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1383/3048] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:23 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:24.077881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a23c7231-4285-4d79-95df-9c08130468b5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:24.079313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a23c7231-4285-4d79-95df-9c08130468b5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:24.095287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23c7231-4285-4d79-95df-9c08130468b5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:24.097042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23c7231-4285-4d79-95df-9c08130468b5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:24.097717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23c7231-4285-4d79-95df-9c08130468b5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:24.100509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23c7231-4285-4d79-95df-9c08130468b5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:24.102458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23c7231-4285-4d79-95df-9c08130468b5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:24.104002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a23c7231-4285-4d79-95df-9c08130468b5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:24.369859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a23c7231-4285-4d79-95df-9c08130468b5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:24.372304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:24.385729 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a23c7231-4285-4d79-95df-9c08130468b5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.318646 Sep 27 10:08:24.402707 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1384/3049] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:24 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:24.408338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e5b081c1-c0d1-4b76-8231-1324f0d1a31c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:24.409201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e5b081c1-c0d1-4b76-8231-1324f0d1a31c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:24.416496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:24.418704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5b081c1-c0d1-4b76-8231-1324f0d1a31c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:24.419523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5b081c1-c0d1-4b76-8231-1324f0d1a31c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:24.423387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5b081c1-c0d1-4b76-8231-1324f0d1a31c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:24.424394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5b081c1-c0d1-4b76-8231-1324f0d1a31c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:24.425386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5b081c1-c0d1-4b76-8231-1324f0d1a31c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:24.425985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5b081c1-c0d1-4b76-8231-1324f0d1a31c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:24.532204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-44d95311-bcbb-4fab-8720-c2fff7f06c65 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:24.539830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:24.559555 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-44d95311-bcbb-4fab-8720-c2fff7f06c65 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.881005 Sep 27 10:08:24.573836 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1666/3050] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:23 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 896 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:24.598572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:24.627715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:24.637005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:24.637836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:24.638829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:24.639794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:24.640401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:24.641252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:24.676545 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5/action HTTP/1.1 Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 28 Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: {"reboot": {"type": "SOFT"}} Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5', 'body': {'reboot': {'type': 'SOFT'}}} Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1031, in _action_reboot Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:24.679562 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:24.685262 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:08:24.685262 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:24.685262 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:24.685262 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:24.685262 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:24.685262 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:24.685262 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:24.685262 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:24.685262 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:24.685262 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:08:24.685262 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:24.696219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:24.698962 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86e892e6-4525-466e-a346-c2be2d135b63 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5/action" status: 404 len: 111 microversion: 2.1 time: 0.111537 Sep 27 10:08:24.704490 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1667/3051] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:24 2022] POST /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5/action => generated 111 bytes in 118 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:24.714175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ccda8d3b-0c45-4798-a5dc-ebc058fb2ac4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:24.724606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ccda8d3b-0c45-4798-a5dc-ebc058fb2ac4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:24.744960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccda8d3b-0c45-4798-a5dc-ebc058fb2ac4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:24.746751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccda8d3b-0c45-4798-a5dc-ebc058fb2ac4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:24.747419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccda8d3b-0c45-4798-a5dc-ebc058fb2ac4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:24.750140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccda8d3b-0c45-4798-a5dc-ebc058fb2ac4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:24.750684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccda8d3b-0c45-4798-a5dc-ebc058fb2ac4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:24.754806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccda8d3b-0c45-4798-a5dc-ebc058fb2ac4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:24.990682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:25.093000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e5b081c1-c0d1-4b76-8231-1324f0d1a31c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:25.105244 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e5b081c1-c0d1-4b76-8231-1324f0d1a31c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1323 microversion: 2.1 time: 0.700099 Sep 27 10:08:25.107213 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1385/3052] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:24 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1323 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:25.112450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5a7d82b-e931-4777-ab57-5d1343000c4f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:25.113363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c5a7d82b-e931-4777-ab57-5d1343000c4f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:25.128936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5a7d82b-e931-4777-ab57-5d1343000c4f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:25.129980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5a7d82b-e931-4777-ab57-5d1343000c4f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:25.130826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5a7d82b-e931-4777-ab57-5d1343000c4f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:25.131857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5a7d82b-e931-4777-ab57-5d1343000c4f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:25.132765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5a7d82b-e931-4777-ab57-5d1343000c4f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:25.133393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5a7d82b-e931-4777-ab57-5d1343000c4f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:25.178139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ccda8d3b-0c45-4798-a5dc-ebc058fb2ac4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:25.187930 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ccda8d3b-0c45-4798-a5dc-ebc058fb2ac4 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1470 microversion: 2.1 time: 0.481756 Sep 27 10:08:25.193802 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1668/3053] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:24 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1470 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:25.205347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:25.206161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:25.219602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:25.220619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:25.221717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:25.222644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:25.223366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:25.223958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:25.322697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:25.355364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:25.357500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:25.378730 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0396af4b-300e-46a7-a486-fc3c883ee362 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.182977 Sep 27 10:08:25.379633 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1669/3054] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:25 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:25.414372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6acd510-b958-49c3-9700-ffdefb6d5fd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:25.415007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c6acd510-b958-49c3-9700-ffdefb6d5fd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:25.428045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:25.429903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6acd510-b958-49c3-9700-ffdefb6d5fd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:25.430560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6acd510-b958-49c3-9700-ffdefb6d5fd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:25.431202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6acd510-b958-49c3-9700-ffdefb6d5fd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:25.431932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6acd510-b958-49c3-9700-ffdefb6d5fd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:25.432418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6acd510-b958-49c3-9700-ffdefb6d5fd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:25.433028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6acd510-b958-49c3-9700-ffdefb6d5fd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:25.696349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c5a7d82b-e931-4777-ab57-5d1343000c4f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:25.714752 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c5a7d82b-e931-4777-ab57-5d1343000c4f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.606217 Sep 27 10:08:25.715362 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1386/3055] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:25 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 607 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:25.744703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cefa82df-b530-4543-9190-b8c35d1ade00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:25.745587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cefa82df-b530-4543-9190-b8c35d1ade00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:25.754652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cefa82df-b530-4543-9190-b8c35d1ade00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:25.762346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cefa82df-b530-4543-9190-b8c35d1ade00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:25.763065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cefa82df-b530-4543-9190-b8c35d1ade00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:25.764165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cefa82df-b530-4543-9190-b8c35d1ade00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:25.764813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cefa82df-b530-4543-9190-b8c35d1ade00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:25.765401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cefa82df-b530-4543-9190-b8c35d1ade00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:26.201836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cefa82df-b530-4543-9190-b8c35d1ade00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:26.217032 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cefa82df-b530-4543-9190-b8c35d1ade00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.471824 Sep 27 10:08:26.218239 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1387/3056] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:25 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:26.223720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a272020e-3450-4c75-bd64-a3cfa0d31ac5 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:26.231182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a272020e-3450-4c75-bd64-a3cfa0d31ac5 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:26.246596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a272020e-3450-4c75-bd64-a3cfa0d31ac5 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:26.247362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a272020e-3450-4c75-bd64-a3cfa0d31ac5 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:26.248000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a272020e-3450-4c75-bd64-a3cfa0d31ac5 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:26.248935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a272020e-3450-4c75-bd64-a3cfa0d31ac5 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:26.249566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a272020e-3450-4c75-bd64-a3cfa0d31ac5 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:26.250172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a272020e-3450-4c75-bd64-a3cfa0d31ac5 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:26.534056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c6acd510-b958-49c3-9700-ffdefb6d5fd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:26.548049 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c6acd510-b958-49c3-9700-ffdefb6d5fd5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 1.141370 Sep 27 10:08:26.548923 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1670/3057] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:25 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 1143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:26.558343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e24a4128-dc41-426d-b84e-9031c4bc9314 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:26.559179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e24a4128-dc41-426d-b84e-9031c4bc9314 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:26.591665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e24a4128-dc41-426d-b84e-9031c4bc9314 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:26.592883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e24a4128-dc41-426d-b84e-9031c4bc9314 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:26.593665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e24a4128-dc41-426d-b84e-9031c4bc9314 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:26.594791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e24a4128-dc41-426d-b84e-9031c4bc9314 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:26.595257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e24a4128-dc41-426d-b84e-9031c4bc9314 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:26.595685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e24a4128-dc41-426d-b84e-9031c4bc9314 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:26.730972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a272020e-3450-4c75-bd64-a3cfa0d31ac5 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:26.740844 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a272020e-3450-4c75-bd64-a3cfa0d31ac5 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1323 microversion: 2.1 time: 0.521098 Sep 27 10:08:26.767675 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1388/3058] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:26 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1323 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:26.772960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-37fcb593-fb3e-44b7-87c8-63d83f6895fc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:26.776598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-37fcb593-fb3e-44b7-87c8-63d83f6895fc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 4bf0bf8c-70ba-4f2c-9948-a37140e77ac6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:26.783871 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-37fcb593-fb3e-44b7-87c8-63d83f6895fc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 4bf0bf8c-70ba-4f2c-9948-a37140e77ac6 could not be found. Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-37fcb593-fb3e-44b7-87c8-63d83f6895fc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/4bf0bf8c-70ba-4f2c-9948-a37140e77ac6/action HTTP/1.1 Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 28 Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: {"reboot": {"type": "SOFT"}} Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 4bf0bf8c-70ba-4f2c-9948-a37140e77ac6 could not be found. Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '4bf0bf8c-70ba-4f2c-9948-a37140e77ac6', 'body': {'reboot': {'type': 'SOFT'}}} Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4bf0bf8c-70ba-4f2c-9948-a37140e77ac6 could not be found. Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1031, in _action_reboot Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4bf0bf8c-70ba-4f2c-9948-a37140e77ac6 could not be found. Sep 27 10:08:26.786136 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:26.790612 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:26.790612 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:26.790612 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:26.790612 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:26.790612 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:26.790612 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:26.790612 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:26.790612 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4bf0bf8c-70ba-4f2c-9948-a37140e77ac6 could not be found. Sep 27 10:08:26.790612 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:26.794844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-37fcb593-fb3e-44b7-87c8-63d83f6895fc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 4bf0bf8c-70ba-4f2c-9948-a37140e77ac6 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:26.796133 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-37fcb593-fb3e-44b7-87c8-63d83f6895fc tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/4bf0bf8c-70ba-4f2c-9948-a37140e77ac6/action" status: 404 len: 111 microversion: 2.1 time: 0.027331 Sep 27 10:08:26.800892 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1389/3059] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:26 2022] POST /compute/v2.1/servers/4bf0bf8c-70ba-4f2c-9948-a37140e77ac6/action => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:26.806331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:26.807175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:26.815154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:26.816424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:26.817204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:26.818185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:26.818886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:26.819393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:26.903811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e24a4128-dc41-426d-b84e-9031c4bc9314 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:26.915187 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e24a4128-dc41-426d-b84e-9031c4bc9314 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1470 microversion: 2.1 time: 0.364977 Sep 27 10:08:26.919669 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1671/3060] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:26 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1470 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:26.946658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b53390e9-0b83-495c-b7b5-75ebdf133c57 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:26.950187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b53390e9-0b83-495c-b7b5-75ebdf133c57 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:26.962031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b53390e9-0b83-495c-b7b5-75ebdf133c57 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:26.965695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b53390e9-0b83-495c-b7b5-75ebdf133c57 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:26.966367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b53390e9-0b83-495c-b7b5-75ebdf133c57 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:26.969230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b53390e9-0b83-495c-b7b5-75ebdf133c57 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:26.971296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b53390e9-0b83-495c-b7b5-75ebdf133c57 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:26.972423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b53390e9-0b83-495c-b7b5-75ebdf133c57 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:27.180885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:27.199866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:27.208781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:27.233982 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ada543d6-7588-4aa8-b8b3-5b137871fc42 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.426806 Sep 27 10:08:27.239428 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1390/3061] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:26 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:27.579815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3aa2bc3a-096c-443d-8728-0cbcdeae8f00 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:27.586116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3aa2bc3a-096c-443d-8728-0cbcdeae8f00 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:27.619074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa2bc3a-096c-443d-8728-0cbcdeae8f00 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:27.620402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa2bc3a-096c-443d-8728-0cbcdeae8f00 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:27.624287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa2bc3a-096c-443d-8728-0cbcdeae8f00 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:27.625598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa2bc3a-096c-443d-8728-0cbcdeae8f00 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:27.629376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa2bc3a-096c-443d-8728-0cbcdeae8f00 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:27.630349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa2bc3a-096c-443d-8728-0cbcdeae8f00 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:27.889236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b53390e9-0b83-495c-b7b5-75ebdf133c57 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:27.903828 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b53390e9-0b83-495c-b7b5-75ebdf133c57 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.982404 Sep 27 10:08:27.904787 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1672/3062] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:26 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 984 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:27.935949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b6e84439-7eb3-4be0-b7ca-e54c15a3a01c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:27.939735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b6e84439-7eb3-4be0-b7ca-e54c15a3a01c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:27.955913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6e84439-7eb3-4be0-b7ca-e54c15a3a01c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:27.956742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6e84439-7eb3-4be0-b7ca-e54c15a3a01c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:27.957436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6e84439-7eb3-4be0-b7ca-e54c15a3a01c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:27.959441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6e84439-7eb3-4be0-b7ca-e54c15a3a01c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:27.962335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6e84439-7eb3-4be0-b7ca-e54c15a3a01c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:27.963057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6e84439-7eb3-4be0-b7ca-e54c15a3a01c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:28.253259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b6e84439-7eb3-4be0-b7ca-e54c15a3a01c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:28.262160 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b6e84439-7eb3-4be0-b7ca-e54c15a3a01c tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1323 microversion: 2.1 time: 0.342535 Sep 27 10:08:28.269266 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1673/3063] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:27 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1323 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:28.289379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8bfcd7a-d66c-4ca5-9f2f-0d4121790610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:28.292064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f8bfcd7a-d66c-4ca5-9f2f-0d4121790610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:28.317622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3aa2bc3a-096c-443d-8728-0cbcdeae8f00 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:28.319541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8bfcd7a-d66c-4ca5-9f2f-0d4121790610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:28.320717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8bfcd7a-d66c-4ca5-9f2f-0d4121790610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:28.321677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8bfcd7a-d66c-4ca5-9f2f-0d4121790610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:28.324732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8bfcd7a-d66c-4ca5-9f2f-0d4121790610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:28.326097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8bfcd7a-d66c-4ca5-9f2f-0d4121790610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:28.328698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8bfcd7a-d66c-4ca5-9f2f-0d4121790610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:28.339843 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3aa2bc3a-096c-443d-8728-0cbcdeae8f00 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.761519 Sep 27 10:08:28.339843 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1391/3064] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:27 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 763 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:28.339843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1e9512cf-0336-4310-9843-e5ce27176fec tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:28.339843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1e9512cf-0336-4310-9843-e5ce27176fec tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:28.370264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e9512cf-0336-4310-9843-e5ce27176fec tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:28.370773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e9512cf-0336-4310-9843-e5ce27176fec tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:28.371225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e9512cf-0336-4310-9843-e5ce27176fec tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:28.371955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e9512cf-0336-4310-9843-e5ce27176fec tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:28.372422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e9512cf-0336-4310-9843-e5ce27176fec tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:28.372889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e9512cf-0336-4310-9843-e5ce27176fec tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:28.618460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=10 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:28.809899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1e9512cf-0336-4310-9843-e5ce27176fec tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:28.823445 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1e9512cf-0336-4310-9843-e5ce27176fec tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1470 microversion: 2.1 time: 0.491779 Sep 27 10:08:28.833491 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1392/3065] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:28 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1470 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:28.894500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:28.895140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:28.905417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:28.906300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:28.911226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:28.916083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:28.916647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:28.917740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:28.966316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:29.069881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:29.086123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:29.088392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:29.114398 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4603330b-d636-4a65-9f4c-afb445b87914 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.277949 Sep 27 10:08:29.116810 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1393/3066] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:28 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:29.139458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f8bfcd7a-d66c-4ca5-9f2f-0d4121790610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:29.153584 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f8bfcd7a-d66c-4ca5-9f2f-0d4121790610 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.882417 Sep 27 10:08:29.235728 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1674/3067] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:28 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 965 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:29.251634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:29.257154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:29.293266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-910c8b52-d2a2-4c71-8ad8-82bfa1324a57 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:29.293891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-910c8b52-d2a2-4c71-8ad8-82bfa1324a57 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:29.296095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:29.299710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:29.301011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:29.301954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-910c8b52-d2a2-4c71-8ad8-82bfa1324a57 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:29.302472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-910c8b52-d2a2-4c71-8ad8-82bfa1324a57 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:29.302916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-910c8b52-d2a2-4c71-8ad8-82bfa1324a57 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:29.304023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-910c8b52-d2a2-4c71-8ad8-82bfa1324a57 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:29.304538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-910c8b52-d2a2-4c71-8ad8-82bfa1324a57 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:29.305035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-910c8b52-d2a2-4c71-8ad8-82bfa1324a57 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:29.306360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:29.307018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:29.309967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:29.363247 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5/action HTTP/1.1 Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 65 Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5', 'body': {'rebuild': {'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6'}}} Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [Previous line repeated 4 more times] Sep 27 10:08:29.371748 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1160, in _action_rebuild Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:08:29.376139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:29.381186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:29.384310 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-33384e18-e4e8-4fae-bcc5-a2b33f64a46e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5/action" status: 404 len: 111 microversion: 2.1 time: 0.137703 Sep 27 10:08:29.387395 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1394/3068] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:29 2022] POST /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5/action => generated 111 bytes in 140 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:29.395384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd4dfe3b-a8ea-49b7-b2b1-7676baf270b3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:29.396381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cd4dfe3b-a8ea-49b7-b2b1-7676baf270b3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:29.422162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4dfe3b-a8ea-49b7-b2b1-7676baf270b3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:29.424224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4dfe3b-a8ea-49b7-b2b1-7676baf270b3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:29.424826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4dfe3b-a8ea-49b7-b2b1-7676baf270b3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:29.426872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4dfe3b-a8ea-49b7-b2b1-7676baf270b3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:29.428488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4dfe3b-a8ea-49b7-b2b1-7676baf270b3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:29.437022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4dfe3b-a8ea-49b7-b2b1-7676baf270b3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:29.798934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cd4dfe3b-a8ea-49b7-b2b1-7676baf270b3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:29.823566 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd4dfe3b-a8ea-49b7-b2b1-7676baf270b3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.434651 Sep 27 10:08:29.826945 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1395/3069] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:29 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:29.854352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7d7f3b6c-0b61-4a57-bdcd-33b278b4924a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:29.855953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7d7f3b6c-0b61-4a57-bdcd-33b278b4924a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:29.874223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7f3b6c-0b61-4a57-bdcd-33b278b4924a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:29.875190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7f3b6c-0b61-4a57-bdcd-33b278b4924a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:29.875877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7f3b6c-0b61-4a57-bdcd-33b278b4924a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:29.876891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7f3b6c-0b61-4a57-bdcd-33b278b4924a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:29.877566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7f3b6c-0b61-4a57-bdcd-33b278b4924a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:29.878211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d7f3b6c-0b61-4a57-bdcd-33b278b4924a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:30.010715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-910c8b52-d2a2-4c71-8ad8-82bfa1324a57 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:30.020650 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-910c8b52-d2a2-4c71-8ad8-82bfa1324a57 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1520 microversion: 2.1 time: 0.731292 Sep 27 10:08:30.027692 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1675/3070] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:29 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1520 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:30.042171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b4454df1-46ab-4e45-acbd-8a7fe41c32d2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:30.047289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b4454df1-46ab-4e45-acbd-8a7fe41c32d2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:30.067251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4454df1-46ab-4e45-acbd-8a7fe41c32d2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:30.069410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4454df1-46ab-4e45-acbd-8a7fe41c32d2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:30.070184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4454df1-46ab-4e45-acbd-8a7fe41c32d2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:30.074214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4454df1-46ab-4e45-acbd-8a7fe41c32d2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:30.075029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4454df1-46ab-4e45-acbd-8a7fe41c32d2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:30.075708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4454df1-46ab-4e45-acbd-8a7fe41c32d2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:30.129382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7d7f3b6c-0b61-4a57-bdcd-33b278b4924a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:30.138592 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7d7f3b6c-0b61-4a57-bdcd-33b278b4924a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.309226 Sep 27 10:08:30.139176 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1396/3071] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:29 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:30.170954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7f07ee16-184b-4c9a-adc9-1d894668d5bc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:30.171454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7f07ee16-184b-4c9a-adc9-1d894668d5bc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:30.199090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f07ee16-184b-4c9a-adc9-1d894668d5bc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:30.201235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f07ee16-184b-4c9a-adc9-1d894668d5bc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:30.203194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f07ee16-184b-4c9a-adc9-1d894668d5bc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:30.204216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f07ee16-184b-4c9a-adc9-1d894668d5bc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:30.206001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f07ee16-184b-4c9a-adc9-1d894668d5bc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:30.207098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f07ee16-184b-4c9a-adc9-1d894668d5bc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:30.286850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7f07ee16-184b-4c9a-adc9-1d894668d5bc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:08:30.547437 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7f07ee16-184b-4c9a-adc9-1d894668d5bc tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 204 len: 0 microversion: 2.1 time: 0.386089 Sep 27 10:08:30.550839 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1397/3072] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:08:30 2022] DELETE /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 0 bytes in 390 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:08:30.556598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:30.565973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:30.578998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:30.579709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:30.580155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:30.580936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:30.581413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:30.581843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:30.785716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:30.795733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:30.799511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:30.800325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b4454df1-46ab-4e45-acbd-8a7fe41c32d2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:30.807158 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d1c0b084-26f4-406b-a057-d61394e89fbb tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.254465 Sep 27 10:08:30.811153 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b4454df1-46ab-4e45-acbd-8a7fe41c32d2 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1470 microversion: 2.1 time: 0.775748 Sep 27 10:08:30.821852 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1398/3073] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:30 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:30.830009 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1676/3074] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:30 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1470 bytes in 802 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:30.837391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-afb7075d-3c78-4d17-a44d-b5e676cbc836 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:30.837391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-afb7075d-3c78-4d17-a44d-b5e676cbc836 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:30.845524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afb7075d-3c78-4d17-a44d-b5e676cbc836 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:30.845524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afb7075d-3c78-4d17-a44d-b5e676cbc836 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:30.845898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afb7075d-3c78-4d17-a44d-b5e676cbc836 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:30.846789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afb7075d-3c78-4d17-a44d-b5e676cbc836 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:30.847105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afb7075d-3c78-4d17-a44d-b5e676cbc836 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:30.847543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-afb7075d-3c78-4d17-a44d-b5e676cbc836 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:30.850511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d31626fb-8d73-4820-a54e-82a019a0ed83 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:30.853055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d31626fb-8d73-4820-a54e-82a019a0ed83 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:30.867708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d31626fb-8d73-4820-a54e-82a019a0ed83 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:30.870094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d31626fb-8d73-4820-a54e-82a019a0ed83 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:30.870488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d31626fb-8d73-4820-a54e-82a019a0ed83 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:30.871543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d31626fb-8d73-4820-a54e-82a019a0ed83 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:30.874314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d31626fb-8d73-4820-a54e-82a019a0ed83 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:30.874776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d31626fb-8d73-4820-a54e-82a019a0ed83 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:31.162483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d31626fb-8d73-4820-a54e-82a019a0ed83 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:31.171876 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d31626fb-8d73-4820-a54e-82a019a0ed83 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1526 microversion: 2.1 time: 0.339953 Sep 27 10:08:31.175150 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1677/3075] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:30 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1526 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:31.176462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-afb7075d-3c78-4d17-a44d-b5e676cbc836 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:31.192811 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-afb7075d-3c78-4d17-a44d-b5e676cbc836 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.369720 Sep 27 10:08:31.195618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9cabd7f7-300e-4125-97c6-3ef51e1ae623 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:31.196251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9cabd7f7-300e-4125-97c6-3ef51e1ae623 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:31.197873 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1399/3076] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:30 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:31.210760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cabd7f7-300e-4125-97c6-3ef51e1ae623 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:31.211696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cabd7f7-300e-4125-97c6-3ef51e1ae623 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:31.211696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cabd7f7-300e-4125-97c6-3ef51e1ae623 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:31.212411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cabd7f7-300e-4125-97c6-3ef51e1ae623 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:31.213083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cabd7f7-300e-4125-97c6-3ef51e1ae623 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:31.213876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cabd7f7-300e-4125-97c6-3ef51e1ae623 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:31.438409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9cabd7f7-300e-4125-97c6-3ef51e1ae623 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:31.442381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1167a9e6-d6cf-404d-8937-d4a6960cfca8 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-2050139640", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "min_count": 3, "max_count": 2}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:31.446958 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9cabd7f7-300e-4125-97c6-3ef51e1ae623 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.269186 Sep 27 10:08:31.447170 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-1167a9e6-d6cf-404d-8937-d4a6960cfca8 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] HTTP exception thrown: min_count must be <= max_count Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1167a9e6-d6cf-404d-8937-d4a6960cfca8 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Request method failure captured: Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 184 Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsuegBvsJcApiBKV_QSFXeJjgJ7n6czPsOIuZZxLciWx0dpY9IN9YV-7Q7arTvEz2Lomu53WBinK8hIkaqbm9Er5fCvePclBEXVVfZjFAPRSkrIlHhhDspQYkWW90-l1JXbkWCPyb5ZpKWhNlosDZIAebg7qu68zKlmbWNWhGrXO0JU Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 5388b7c5e001457c9ab07206893a86e8 Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5388b7c5e001457c9ab07206893a86e8"}]}] Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 5388b7c5e001457c9ab07206893a86e8 Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-MultipleCreateNegativeTestJSON-368475875-project Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: d65756281cb34a258d31b6ead74411aa Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-368475875-project Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-2050139640", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "min_count": 3, "max_count": 2}} Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: exception: min_count must be <= max_count Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-2050139640', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'min_count': 3, 'max_count': 2}}} Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 726, in create Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=msg) Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: min_count must be <= max_count Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: min_count must be <= max_count Sep 27 10:08:31.448597 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.453251 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1678/3077] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:31 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:31.456140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1167a9e6-d6cf-404d-8937-d4a6960cfca8 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Returning 400 to user: min_count must be <= max_count {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:31.457305 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1167a9e6-d6cf-404d-8937-d4a6960cfca8 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 74 microversion: 2.1 time: 0.256995 Sep 27 10:08:31.457766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-38338f03-52f3-4eb5-aaa3-73f6dae98a36 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:31.458548 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1400/3078] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:08:31 2022] POST /compute/v2.1/servers => generated 74 bytes in 260 msecs (HTTP/1.1 400) 9 headers in 380 bytes (1 switches on core 0) Sep 27 10:08:31.477937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-38338f03-52f3-4eb5-aaa3-73f6dae98a36 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: bd4ab650-cd23-4edb-b603-f5e4b57fc9de] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:31.496854 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-38338f03-52f3-4eb5-aaa3-73f6dae98a36 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance bd4ab650-cd23-4edb-b603-f5e4b57fc9de could not be found. Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-38338f03-52f3-4eb5-aaa3-73f6dae98a36 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/bd4ab650-cd23-4edb-b603-f5e4b57fc9de/action HTTP/1.1 Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 65 Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance bd4ab650-cd23-4edb-b603-f5e4b57fc9de could not be found. Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'bd4ab650-cd23-4edb-b603-f5e4b57fc9de', 'body': {'rebuild': {'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6'}}} Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bd4ab650-cd23-4edb-b603-f5e4b57fc9de could not be found. Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 4 more times] Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1160, in _action_rebuild Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, id) Sep 27 10:08:31.498561 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bd4ab650-cd23-4edb-b603-f5e4b57fc9de could not be found. Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bd4ab650-cd23-4edb-b603-f5e4b57fc9de could not be found. Sep 27 10:08:31.503379 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.505259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-38338f03-52f3-4eb5-aaa3-73f6dae98a36 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance bd4ab650-cd23-4edb-b603-f5e4b57fc9de could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:31.506709 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-38338f03-52f3-4eb5-aaa3-73f6dae98a36 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/bd4ab650-cd23-4edb-b603-f5e4b57fc9de/action" status: 404 len: 111 microversion: 2.1 time: 0.052874 Sep 27 10:08:31.509774 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1679/3079] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:31 2022] POST /compute/v2.1/servers/bd4ab650-cd23-4edb-b603-f5e4b57fc9de/action => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:31.518313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2a7f6d80-4e35-402b-bc8e-13f81be7a3e8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:31.518935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2a7f6d80-4e35-402b-bc8e-13f81be7a3e8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:31.536473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a7f6d80-4e35-402b-bc8e-13f81be7a3e8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:31.537223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a7f6d80-4e35-402b-bc8e-13f81be7a3e8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:31.539075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a7f6d80-4e35-402b-bc8e-13f81be7a3e8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:31.541037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a7f6d80-4e35-402b-bc8e-13f81be7a3e8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:31.549624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a7f6d80-4e35-402b-bc8e-13f81be7a3e8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:31.550264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a7f6d80-4e35-402b-bc8e-13f81be7a3e8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:31.553985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-63a055e7-8646-4e00-bc5e-06ba8032d053 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-1757494333", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "max_count": 0}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-63a055e7-8646-4e00-bc5e-06ba8032d053 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Request method failure captured: Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 168 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsuegBvsJcApiBKV_QSFXeJjgJ7n6czPsOIuZZxLciWx0dpY9IN9YV-7Q7arTvEz2Lomu53WBinK8hIkaqbm9Er5fCvePclBEXVVfZjFAPRSkrIlHhhDspQYkWW90-l1JXbkWCPyb5ZpKWhNlosDZIAebg7qu68zKlmbWNWhGrXO0JU Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 5388b7c5e001457c9ab07206893a86e8 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5388b7c5e001457c9ab07206893a86e8"}]}] Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 5388b7c5e001457c9ab07206893a86e8 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-MultipleCreateNegativeTestJSON-368475875-project Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: d65756281cb34a258d31b6ead74411aa Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-368475875-project Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-1757494333", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "max_count": 0}} Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-1757494333', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'max_count': 0}}} Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['server']['properties']['max_count']: Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'minLength': 1, Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'minimum': 1, Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 0 Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:08:31.564959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 Sep 27 10:08:31.570784 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.575157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-63a055e7-8646-4e00-bc5e-06ba8032d053 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Returning 400 to user: Invalid input for field/attribute max_count. Value: 0. 0 is less than the minimum of 1 {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:31.577162 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-63a055e7-8646-4e00-bc5e-06ba8032d053 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 130 microversion: 2.1 time: 0.029180 Sep 27 10:08:31.580183 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1680/3080] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:08:31 2022] POST /compute/v2.1/servers => generated 130 bytes in 33 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:08:31.670090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c5160665-8b8f-4354-b566-18518ec634e5 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-2024248567", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "max_count": 2.5}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c5160665-8b8f-4354-b566-18518ec634e5 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Request method failure captured: Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 170 Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsuegBvsJcApiBKV_QSFXeJjgJ7n6czPsOIuZZxLciWx0dpY9IN9YV-7Q7arTvEz2Lomu53WBinK8hIkaqbm9Er5fCvePclBEXVVfZjFAPRSkrIlHhhDspQYkWW90-l1JXbkWCPyb5ZpKWhNlosDZIAebg7qu68zKlmbWNWhGrXO0JU Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 5388b7c5e001457c9ab07206893a86e8 Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5388b7c5e001457c9ab07206893a86e8"}]}] Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 5388b7c5e001457c9ab07206893a86e8 Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-MultipleCreateNegativeTestJSON-368475875-project Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: d65756281cb34a258d31b6ead74411aa Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-368475875-project Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-2024248567", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "max_count": 2.5}} Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-2024248567', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'max_count': 2.5}}} Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 2.5 is not of type 'integer', 'string' Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['server']['properties']['max_count']: Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'minLength': 1, Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'minimum': 1, Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 2.5 Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:08:31.674189 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c5160665-8b8f-4354-b566-18518ec634e5 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Returning 400 to user: Invalid input for field/attribute max_count. Value: 2.5. 2.5 is not of type 'integer', 'string' {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:31.679181 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c5160665-8b8f-4354-b566-18518ec634e5 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 139 microversion: 2.1 time: 0.022057 Sep 27 10:08:31.687122 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1681/3081] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:08:31 2022] POST /compute/v2.1/servers => generated 139 bytes in 35 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:08:31.770558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5dbd023c-7bda-4148-ae0b-5aaf5159ef44 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-2037438031", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "min_count": 0}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5dbd023c-7bda-4148-ae0b-5aaf5159ef44 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Request method failure captured: Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 168 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsuegBvsJcApiBKV_QSFXeJjgJ7n6czPsOIuZZxLciWx0dpY9IN9YV-7Q7arTvEz2Lomu53WBinK8hIkaqbm9Er5fCvePclBEXVVfZjFAPRSkrIlHhhDspQYkWW90-l1JXbkWCPyb5ZpKWhNlosDZIAebg7qu68zKlmbWNWhGrXO0JU Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 5388b7c5e001457c9ab07206893a86e8 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5388b7c5e001457c9ab07206893a86e8"}]}] Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 5388b7c5e001457c9ab07206893a86e8 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-MultipleCreateNegativeTestJSON-368475875-project Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: d65756281cb34a258d31b6ead74411aa Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-368475875-project Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-2037438031", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "min_count": 0}} Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-2037438031', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'min_count': 0}}} Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['server']['properties']['min_count']: Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'minLength': 1, Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'minimum': 1, Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 0 Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:08:31.776454 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 Sep 27 10:08:31.782536 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:31.788751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5dbd023c-7bda-4148-ae0b-5aaf5159ef44 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Returning 400 to user: Invalid input for field/attribute min_count. Value: 0. 0 is less than the minimum of 1 {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:31.795565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5dbd023c-7bda-4148-ae0b-5aaf5159ef44 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 130 microversion: 2.1 time: 0.029777 Sep 27 10:08:31.797286 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1682/3082] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:08:31 2022] POST /compute/v2.1/servers => generated 130 bytes in 33 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 27 10:08:31.840995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:31.842507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:31.856585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:31.858552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:31.859907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:31.862475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:31.864358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:31.865123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:31.909595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2a7f6d80-4e35-402b-bc8e-13f81be7a3e8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:31.934033 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2a7f6d80-4e35-402b-bc8e-13f81be7a3e8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.419749 Sep 27 10:08:31.934898 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1401/3083] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:31 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:31.943145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-56768ed8-30ea-4d86-94a4-27dbcb869d2d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:31.950879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-56768ed8-30ea-4d86-94a4-27dbcb869d2d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:31.968975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56768ed8-30ea-4d86-94a4-27dbcb869d2d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:31.971190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56768ed8-30ea-4d86-94a4-27dbcb869d2d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:31.972055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56768ed8-30ea-4d86-94a4-27dbcb869d2d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:31.973166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56768ed8-30ea-4d86-94a4-27dbcb869d2d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:31.976035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56768ed8-30ea-4d86-94a4-27dbcb869d2d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:31.980617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56768ed8-30ea-4d86-94a4-27dbcb869d2d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:32.073512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:32.082409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:32.092270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:32.101754 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b27dcc7d-782d-40ce-894b-56aeb92d5b78 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.266074 Sep 27 10:08:32.117894 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1683/3084] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:31 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:08:32.123404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92266826-76a7-403c-9127-de18703da264 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-1160889636", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "min_count": 2.5}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92266826-76a7-403c-9127-de18703da264 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Request method failure captured: Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 170 Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsuegBvsJcApiBKV_QSFXeJjgJ7n6czPsOIuZZxLciWx0dpY9IN9YV-7Q7arTvEz2Lomu53WBinK8hIkaqbm9Er5fCvePclBEXVVfZjFAPRSkrIlHhhDspQYkWW90-l1JXbkWCPyb5ZpKWhNlosDZIAebg7qu68zKlmbWNWhGrXO0JU Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 5388b7c5e001457c9ab07206893a86e8 Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/5388b7c5e001457c9ab07206893a86e8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_5388b7c5e001457c9ab07206893a86e8"}]}] Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 5388b7c5e001457c9ab07206893a86e8 Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-MultipleCreateNegativeTestJSON-368475875 Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-MultipleCreateNegativeTestJSON-368475875-project Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: d65756281cb34a258d31b6ead74411aa Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-MultipleCreateNegativeTestJSON-368475875-project Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-MultipleCreateNegativeTestJSON-server-1160889636", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "min_count": 2.5}} Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': 'tempest-MultipleCreateNegativeTestJSON-server-1160889636', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'min_count': 2.5}}} Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 2.5 is not of type 'integer', 'string' Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'type' in schema['properties']['server']['properties']['min_count']: Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'minLength': 1, Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'minimum': 1, Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 2.5 Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:08:32.129325 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' Sep 27 10:08:32.134086 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:32.135697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92266826-76a7-403c-9127-de18703da264 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] Returning 400 to user: Invalid input for field/attribute min_count. Value: 2.5. 2.5 is not of type 'integer', 'string' {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:32.136753 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-92266826-76a7-403c-9127-de18703da264 tempest-MultipleCreateNegativeTestJSON-368475875 tempest-MultipleCreateNegativeTestJSON-368475875-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 139 microversion: 2.1 time: 0.017460 Sep 27 10:08:32.143871 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1684/3085] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:08:32 2022] POST /compute/v2.1/servers => generated 139 bytes in 19 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:08:32.149280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9fa2cae-13c9-4c2a-a506-f0085a852145 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:32.157291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b9fa2cae-13c9-4c2a-a506-f0085a852145 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:32.176408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9fa2cae-13c9-4c2a-a506-f0085a852145 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:32.177347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9fa2cae-13c9-4c2a-a506-f0085a852145 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:32.177964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9fa2cae-13c9-4c2a-a506-f0085a852145 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:32.178869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9fa2cae-13c9-4c2a-a506-f0085a852145 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:32.179811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9fa2cae-13c9-4c2a-a506-f0085a852145 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:32.189052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9fa2cae-13c9-4c2a-a506-f0085a852145 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:33.177919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b9fa2cae-13c9-4c2a-a506-f0085a852145 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:33.187076 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b9fa2cae-13c9-4c2a-a506-f0085a852145 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 1.041919 Sep 27 10:08:33.192332 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1685/3086] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:32 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 1048 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:33.202266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cc8f3a6e-8415-4bb3-8f19-a4fb82732d49 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:33.203129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cc8f3a6e-8415-4bb3-8f19-a4fb82732d49 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:33.218404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc8f3a6e-8415-4bb3-8f19-a4fb82732d49 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:33.219763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc8f3a6e-8415-4bb3-8f19-a4fb82732d49 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:33.221804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc8f3a6e-8415-4bb3-8f19-a4fb82732d49 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:33.224344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc8f3a6e-8415-4bb3-8f19-a4fb82732d49 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:33.225034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc8f3a6e-8415-4bb3-8f19-a4fb82732d49 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:33.225648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc8f3a6e-8415-4bb3-8f19-a4fb82732d49 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:33.501586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cc8f3a6e-8415-4bb3-8f19-a4fb82732d49 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:33.510819 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cc8f3a6e-8415-4bb3-8f19-a4fb82732d49 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1526 microversion: 2.1 time: 0.316977 Sep 27 10:08:33.515917 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1686/3087] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:33 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1526 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:33.529867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-317687a5-6852-4ddf-aaff-796213590a7d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:33.530785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-317687a5-6852-4ddf-aaff-796213590a7d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:33.541966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317687a5-6852-4ddf-aaff-796213590a7d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:33.544549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317687a5-6852-4ddf-aaff-796213590a7d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:33.547068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317687a5-6852-4ddf-aaff-796213590a7d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:33.548026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317687a5-6852-4ddf-aaff-796213590a7d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:33.551862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317687a5-6852-4ddf-aaff-796213590a7d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:33.553164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317687a5-6852-4ddf-aaff-796213590a7d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:33.555051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-56768ed8-30ea-4d86-94a4-27dbcb869d2d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:33.574040 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-56768ed8-30ea-4d86-94a4-27dbcb869d2d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1476 microversion: 2.1 time: 1.636195 Sep 27 10:08:33.574911 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1402/3088] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:31 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1476 bytes in 1639 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:33.596936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:33.599567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:33.619862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:33.621326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:33.623949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:33.626262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:33.628251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:33.632639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:33.776979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:33.807866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:33.811867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:33.825981 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4606035d-6517-4181-8b55-4d4297aa4fc6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.249315 Sep 27 10:08:33.846658 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1403/3089] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:33 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:33.858248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cf8da7c1-feb6-4d0c-b7b9-275c460c4a9e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"restore": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:33.859134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cf8da7c1-feb6-4d0c-b7b9-275c460c4a9e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: ece563b8-65f1-4587-956c-41e94f83deb0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:33.872853 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-cf8da7c1-feb6-4d0c-b7b9-275c460c4a9e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance ece563b8-65f1-4587-956c-41e94f83deb0 could not be found. Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cf8da7c1-feb6-4d0c-b7b9-275c460c4a9e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/ece563b8-65f1-4587-956c-41e94f83deb0/action HTTP/1.1 Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 15 Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: {"restore": {}} Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance ece563b8-65f1-4587-956c-41e94f83deb0 could not be found. Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'ece563b8-65f1-4587-956c-41e94f83deb0', 'body': {'restore': {}}} Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ece563b8-65f1-4587-956c-41e94f83deb0 could not be found. Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 38, in _restore Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ece563b8-65f1-4587-956c-41e94f83deb0 could not be found. Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:33.877068 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:33.881181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:33.881181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:33.881181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ece563b8-65f1-4587-956c-41e94f83deb0 could not be found. Sep 27 10:08:33.881181 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:33.885618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cf8da7c1-feb6-4d0c-b7b9-275c460c4a9e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance ece563b8-65f1-4587-956c-41e94f83deb0 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:33.890727 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cf8da7c1-feb6-4d0c-b7b9-275c460c4a9e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/ece563b8-65f1-4587-956c-41e94f83deb0/action" status: 404 len: 111 microversion: 2.1 time: 0.042377 Sep 27 10:08:33.896874 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1404/3090] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:33 2022] POST /compute/v2.1/servers/ece563b8-65f1-4587-956c-41e94f83deb0/action => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:33.904298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e6d70c37-d2ca-41e1-8df5-1735e53b7e3b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:33.906516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e6d70c37-d2ca-41e1-8df5-1735e53b7e3b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:33.920855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6d70c37-d2ca-41e1-8df5-1735e53b7e3b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:33.921724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6d70c37-d2ca-41e1-8df5-1735e53b7e3b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:33.922429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6d70c37-d2ca-41e1-8df5-1735e53b7e3b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:33.923364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6d70c37-d2ca-41e1-8df5-1735e53b7e3b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:33.924027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6d70c37-d2ca-41e1-8df5-1735e53b7e3b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:33.924696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6d70c37-d2ca-41e1-8df5-1735e53b7e3b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:33.957694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-317687a5-6852-4ddf-aaff-796213590a7d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:33.965862 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-317687a5-6852-4ddf-aaff-796213590a7d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.448407 Sep 27 10:08:33.977768 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1687/3091] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:33 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:34.224082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e6d70c37-d2ca-41e1-8df5-1735e53b7e3b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:34.234547 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e6d70c37-d2ca-41e1-8df5-1735e53b7e3b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.335912 Sep 27 10:08:34.236186 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1405/3092] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:33 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:34.274283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6fb7977-644f-40fb-812c-7e928eb8c425 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:34.279637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c6fb7977-644f-40fb-812c-7e928eb8c425 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:34.286118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6fb7977-644f-40fb-812c-7e928eb8c425 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:34.286858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6fb7977-644f-40fb-812c-7e928eb8c425 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:34.287482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6fb7977-644f-40fb-812c-7e928eb8c425 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:34.288433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6fb7977-644f-40fb-812c-7e928eb8c425 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:34.288965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6fb7977-644f-40fb-812c-7e928eb8c425 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:34.289502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6fb7977-644f-40fb-812c-7e928eb8c425 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:34.564813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b185a509-244e-4444-865c-41506606531d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:34.569491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b185a509-244e-4444-865c-41506606531d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:34.580709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b185a509-244e-4444-865c-41506606531d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:34.589639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b185a509-244e-4444-865c-41506606531d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:34.590498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b185a509-244e-4444-865c-41506606531d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:34.594251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b185a509-244e-4444-865c-41506606531d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:34.595922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b185a509-244e-4444-865c-41506606531d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:34.597984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b185a509-244e-4444-865c-41506606531d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:34.748685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c6fb7977-644f-40fb-812c-7e928eb8c425 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:34.795894 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c6fb7977-644f-40fb-812c-7e928eb8c425 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.534299 Sep 27 10:08:34.810979 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1688/3093] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:34 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:34.824585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-840be7c6-9159-4830-8363-0073bb00d5c7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:34.825504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-840be7c6-9159-4830-8363-0073bb00d5c7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:34.833514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-840be7c6-9159-4830-8363-0073bb00d5c7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:34.834719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-840be7c6-9159-4830-8363-0073bb00d5c7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:34.835331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-840be7c6-9159-4830-8363-0073bb00d5c7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:34.836537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-840be7c6-9159-4830-8363-0073bb00d5c7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:34.837226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-840be7c6-9159-4830-8363-0073bb00d5c7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:34.837825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-840be7c6-9159-4830-8363-0073bb00d5c7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:34.844173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:35.249867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-840be7c6-9159-4830-8363-0073bb00d5c7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:35.265532 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-840be7c6-9159-4830-8363-0073bb00d5c7 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1462 microversion: 2.1 time: 0.449730 Sep 27 10:08:35.268918 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1689/3094] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:34 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1462 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:35.285734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"restore": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:35.286386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:35.320968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:35.321677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:35.323396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:35.325457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:35.331172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:35.333966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:35.359122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b185a509-244e-4444-865c-41506606531d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:35.376959 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b185a509-244e-4444-865c-41506606531d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1526 microversion: 2.1 time: 0.830034 Sep 27 10:08:35.377592 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1406/3095] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:34 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1526 bytes in 834 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:35.394083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:35.398447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:35.421290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:35.421813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:35.423307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:35.424069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:35.424534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:35.426482 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Cannot 'restore' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:08:35.428604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action HTTP/1.1 Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 15 Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: {"restore": {}} Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: exception: Cannot 'restore' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3327ff95-5c05-4167-a73b-5fd55f7bfabd', 'body': {'restore': {}}} Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 42, in _restore Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.restore(context, instance) Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd in vm_state active. Cannot restore while the instance is in this state. Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/deferred_delete.py", line 46, in _restore Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'restore' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:35.431385 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:35.436457 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:35.436457 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:35.436457 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:35.436457 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'restore' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:08:35.436457 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:35.436457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 409 to user: Cannot 'restore' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:35.436457 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a013d86-f888-4644-9e60-c5eca379c4a9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 409 len: 145 microversion: 2.1 time: 0.161974 Sep 27 10:08:35.437617 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1690/3096] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:35 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 145 bytes in 167 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:08:35.453854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1593b9cb-6fa0-4130-a345-3d5b0412ad63 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:35.454720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1593b9cb-6fa0-4130-a345-3d5b0412ad63 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:35.467499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1593b9cb-6fa0-4130-a345-3d5b0412ad63 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:35.468287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1593b9cb-6fa0-4130-a345-3d5b0412ad63 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:35.470011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1593b9cb-6fa0-4130-a345-3d5b0412ad63 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:35.472110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1593b9cb-6fa0-4130-a345-3d5b0412ad63 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:35.474650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1593b9cb-6fa0-4130-a345-3d5b0412ad63 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:35.475289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1593b9cb-6fa0-4130-a345-3d5b0412ad63 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:35.633929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:35.651390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:35.653865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:35.673779 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4bc36641-7539-4545-96a0-7132eadbeb87 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.295052 Sep 27 10:08:35.674390 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1407/3097] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:35 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:35.683068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9012a5ba-11e4-4f8a-a317-b680c75a907d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:35.683930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9012a5ba-11e4-4f8a-a317-b680c75a907d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:35.717611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9012a5ba-11e4-4f8a-a317-b680c75a907d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:35.718165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9012a5ba-11e4-4f8a-a317-b680c75a907d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:35.718623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9012a5ba-11e4-4f8a-a317-b680c75a907d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:35.719369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9012a5ba-11e4-4f8a-a317-b680c75a907d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:35.719840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9012a5ba-11e4-4f8a-a317-b680c75a907d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:35.720276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9012a5ba-11e4-4f8a-a317-b680c75a907d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:35.819715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1593b9cb-6fa0-4130-a345-3d5b0412ad63 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:35.829213 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1593b9cb-6fa0-4130-a345-3d5b0412ad63 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 0.390585 Sep 27 10:08:35.829804 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1691/3098] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:35 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:35.866246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a1d404e6-7444-42af-8788-94207fd9da00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:35.874805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a1d404e6-7444-42af-8788-94207fd9da00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:35.898141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1d404e6-7444-42af-8788-94207fd9da00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:35.898967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1d404e6-7444-42af-8788-94207fd9da00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:35.901933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1d404e6-7444-42af-8788-94207fd9da00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:35.904539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1d404e6-7444-42af-8788-94207fd9da00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:35.906351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1d404e6-7444-42af-8788-94207fd9da00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:35.906814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1d404e6-7444-42af-8788-94207fd9da00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:36.181718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a1d404e6-7444-42af-8788-94207fd9da00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:36.191493 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a1d404e6-7444-42af-8788-94207fd9da00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.338927 Sep 27 10:08:36.207688 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1692/3099] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:35 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:36.222937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f040aef0-d8c7-4e59-9a09-4780f4ed80f1 req-201086f6-edd6-45cf-9a73-7f36e5543e6d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b902a024-8aa7-4843-8680-41d3e189d3af", "name": "network-vif-unplugged", "status": "completed", "tag": "df367b62-0029-488e-9897-fac934fc397e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:36.237206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f040aef0-d8c7-4e59-9a09-4780f4ed80f1 req-201086f6-edd6-45cf-9a73-7f36e5543e6d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:36.238073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f040aef0-d8c7-4e59-9a09-4780f4ed80f1 req-201086f6-edd6-45cf-9a73-7f36e5543e6d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:36.238680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f040aef0-d8c7-4e59-9a09-4780f4ed80f1 req-201086f6-edd6-45cf-9a73-7f36e5543e6d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:36.257436 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f040aef0-d8c7-4e59-9a09-4780f4ed80f1 req-201086f6-edd6-45cf-9a73-7f36e5543e6d service nova] Creating event network-vif-unplugged:df367b62-0029-488e-9897-fac934fc397e for instance b902a024-8aa7-4843-8680-41d3e189d3af on nch1-master-1 Sep 27 10:08:36.281416 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f040aef0-d8c7-4e59-9a09-4780f4ed80f1 req-201086f6-edd6-45cf-9a73-7f36e5543e6d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.071450 Sep 27 10:08:36.283073 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1693/3100] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:36 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:36.488286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9012a5ba-11e4-4f8a-a317-b680c75a907d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:36.499699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ListServersNegativeTestJSON-server-638186245", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "min_count": 3, "networks": [{"uuid": "ad4b2f7f-13bb-41da-91a3-6b593274e93b"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:36.501830 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9012a5ba-11e4-4f8a-a317-b680c75a907d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1462 microversion: 2.1 time: 0.823558 Sep 27 10:08:36.502842 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1408/3101] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:35 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1462 bytes in 825 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:36.519160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2bf9f4a7-a70b-4425-a154-43a9c4c7dc52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:36.520212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2bf9f4a7-a70b-4425-a154-43a9c4c7dc52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:36.550923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bf9f4a7-a70b-4425-a154-43a9c4c7dc52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:36.551605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bf9f4a7-a70b-4425-a154-43a9c4c7dc52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:36.552231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bf9f4a7-a70b-4425-a154-43a9c4c7dc52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:36.553207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bf9f4a7-a70b-4425-a154-43a9c4c7dc52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:36.553817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bf9f4a7-a70b-4425-a154-43a9c4c7dc52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:36.554674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bf9f4a7-a70b-4425-a154-43a9c4c7dc52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:36.604592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] validate_networks() for [('ad4b2f7f-13bb-41da-91a3-6b593274e93b', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:08:36.891291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2bf9f4a7-a70b-4425-a154-43a9c4c7dc52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:36.915152 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2bf9f4a7-a70b-4425-a154-43a9c4c7dc52 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.410511 Sep 27 10:08:36.917421 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1409/3102] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:36 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:36.932991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-edfe634e-a842-42d3-ad6e-8115c4f091a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:36.934243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-edfe634e-a842-42d3-ad6e-8115c4f091a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:36.944407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edfe634e-a842-42d3-ad6e-8115c4f091a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:36.945485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edfe634e-a842-42d3-ad6e-8115c4f091a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:36.946203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edfe634e-a842-42d3-ad6e-8115c4f091a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:36.947170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edfe634e-a842-42d3-ad6e-8115c4f091a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:36.948494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edfe634e-a842-42d3-ad6e-8115c4f091a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:36.949456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edfe634e-a842-42d3-ad6e-8115c4f091a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:37.562840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-edfe634e-a842-42d3-ad6e-8115c4f091a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:37.564676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:08:37.565185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:08:37.565949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:08:37.566368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:08:37.566848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:08:37.575608 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-edfe634e-a842-42d3-ad6e-8115c4f091a8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1526 microversion: 2.1 time: 0.656461 Sep 27 10:08:37.578992 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1410/3103] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:36 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1526 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:37.586621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:37.587122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:37.587565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:37.588307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:37.588799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:37.589235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:37.590864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:37.619062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:37.631188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:37.634772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:37.634916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Getting quotas for project 79022e5ca5764576b4feb7f4cadba80c. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:08:37.635584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:37.637472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:37.640865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Getting quotas for user 2699ef01c0eb4d5783203e6b41293d97 and project 79022e5ca5764576b4feb7f4cadba80c. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:08:37.641813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:37.642517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:37.662305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Going to run 3 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:08:37.669637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:08:37.678005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:37.725850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 4fed875e-5be1-4b37-a0f9-55e6bc6fca05] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:08:37.778035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=4fed875e-5be1-4b37-a0f9-55e6bc6fca05,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:08:37.800774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:08:37.801408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:08:37.802224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:08:37.803015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:08:37.803631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:08:37.920335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:37.933202 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-90986d3e-2e46-4957-8be5-d252f55503b1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.648315 Sep 27 10:08:37.934891 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1694/3104] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:08:36 2022] POST /compute/v2.1/servers => generated 380 bytes in 1651 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:08:37.937826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:37.938546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:37.940253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:37.943708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:37.943879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:37.944474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:37.945242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:37.947151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:37.964055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:37.964570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:37.965027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:37.965753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:37.966217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:37.966632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:38.005374 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1520b4b9-d9c6-44de-871c-c9e0089f1b95 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action" status: 202 len: 1066 microversion: 2.1 time: 0.424930 Sep 27 10:08:38.020878 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1411/3105] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:37 2022] POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action => generated 1066 bytes in 430 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Sep 27 10:08:38.025170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7876d927-ea95-489a-a3d5-5ab6a8e132a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:38.025901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7876d927-ea95-489a-a3d5-5ab6a8e132a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:38.035900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876d927-ea95-489a-a3d5-5ab6a8e132a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:38.038791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876d927-ea95-489a-a3d5-5ab6a8e132a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:38.040034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876d927-ea95-489a-a3d5-5ab6a8e132a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:38.045131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876d927-ea95-489a-a3d5-5ab6a8e132a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:38.045787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876d927-ea95-489a-a3d5-5ab6a8e132a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:38.046296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876d927-ea95-489a-a3d5-5ab6a8e132a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:38.158055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:38.176237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:38.178368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:38.189923 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7239ec2f-c200-4d11-9000-b98e31dedc0c tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.253928 Sep 27 10:08:38.191032 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1695/3106] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:37 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:38.203185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e29e2190-8ddc-40f9-8e6f-444095a7996e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:38.204086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e29e2190-8ddc-40f9-8e6f-444095a7996e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 4f8f5227-adf6-4f94-a2fa-2db95e312b3c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:38.216419 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-e29e2190-8ddc-40f9-8e6f-444095a7996e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 4f8f5227-adf6-4f94-a2fa-2db95e312b3c could not be found. Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e29e2190-8ddc-40f9-8e6f-444095a7996e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/4f8f5227-adf6-4f94-a2fa-2db95e312b3c/action HTTP/1.1 Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 14 Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: {"resume": {}} Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 4f8f5227-adf6-4f94-a2fa-2db95e312b3c could not be found. Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '4f8f5227-adf6-4f94-a2fa-2db95e312b3c', 'body': {'resume': {}}} Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4f8f5227-adf6-4f94-a2fa-2db95e312b3c could not be found. Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 55, in _resume Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, id) Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4f8f5227-adf6-4f94-a2fa-2db95e312b3c could not be found. Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:38.224088 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:38.230834 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:38.230834 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:38.230834 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4f8f5227-adf6-4f94-a2fa-2db95e312b3c could not be found. Sep 27 10:08:38.230834 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:38.239570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e29e2190-8ddc-40f9-8e6f-444095a7996e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 4f8f5227-adf6-4f94-a2fa-2db95e312b3c could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:38.239570 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e29e2190-8ddc-40f9-8e6f-444095a7996e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/4f8f5227-adf6-4f94-a2fa-2db95e312b3c/action" status: 404 len: 111 microversion: 2.1 time: 0.046475 Sep 27 10:08:38.239570 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1696/3107] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:38 2022] POST /compute/v2.1/servers/4f8f5227-adf6-4f94-a2fa-2db95e312b3c/action => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:38.246585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0a046f19-2fd1-4dd4-bfca-953a7ba73cd0 req-e53d3c86-be92-4828-98a3-ac0199ae60ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1b7c9d0a-630a-4c9a-8c25-56f06e0d851c", "name": "network-vif-deleted", "tag": "4fd1690d-93ce-4d30-a796-27712c3d8637"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:38.258255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0a046f19-2fd1-4dd4-bfca-953a7ba73cd0 req-e53d3c86-be92-4828-98a3-ac0199ae60ce service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:38.260130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0a046f19-2fd1-4dd4-bfca-953a7ba73cd0 req-e53d3c86-be92-4828-98a3-ac0199ae60ce service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:38.260909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0a046f19-2fd1-4dd4-bfca-953a7ba73cd0 req-e53d3c86-be92-4828-98a3-ac0199ae60ce service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:38.274970 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0a046f19-2fd1-4dd4-bfca-953a7ba73cd0 req-e53d3c86-be92-4828-98a3-ac0199ae60ce service nova] Creating event network-vif-deleted:4fd1690d-93ce-4d30-a796-27712c3d8637 for instance 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c on nch2-master-1 Sep 27 10:08:38.281190 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0a046f19-2fd1-4dd4-bfca-953a7ba73cd0 req-e53d3c86-be92-4828-98a3-ac0199ae60ce service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040152 Sep 27 10:08:38.282415 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1697/3108] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:38.288004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ebd405f4-df6a-4e5d-925f-1818eb1afb1a tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:38.293850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ebd405f4-df6a-4e5d-925f-1818eb1afb1a tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:38.294946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ebd405f4-df6a-4e5d-925f-1818eb1afb1a tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Searching by: {'deleted': False, 'project_id': '79022e5ca5764576b4feb7f4cadba80c'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:08:38.320253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd405f4-df6a-4e5d-925f-1818eb1afb1a tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:38.321084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd405f4-df6a-4e5d-925f-1818eb1afb1a tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:38.321894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd405f4-df6a-4e5d-925f-1818eb1afb1a tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:38.322902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd405f4-df6a-4e5d-925f-1818eb1afb1a tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:38.323692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd405f4-df6a-4e5d-925f-1818eb1afb1a tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:38.324395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd405f4-df6a-4e5d-925f-1818eb1afb1a tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:38.363032 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ebd405f4-df6a-4e5d-925f-1818eb1afb1a tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 1015 microversion: 2.1 time: 0.078933 Sep 27 10:08:38.364062 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1698/3109] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:08:38 2022] GET /compute/v2.1/servers => generated 1015 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:38.370918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f261cc2d-77ff-4d4d-a8e0-bc2717690bf3 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:38.371978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f261cc2d-77ff-4d4d-a8e0-bc2717690bf3 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:38.396158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f261cc2d-77ff-4d4d-a8e0-bc2717690bf3 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:38.398282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f261cc2d-77ff-4d4d-a8e0-bc2717690bf3 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:38.399057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f261cc2d-77ff-4d4d-a8e0-bc2717690bf3 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:38.400684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f261cc2d-77ff-4d4d-a8e0-bc2717690bf3 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:38.402376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f261cc2d-77ff-4d4d-a8e0-bc2717690bf3 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:38.404054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f261cc2d-77ff-4d4d-a8e0-bc2717690bf3 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:38.627196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f261cc2d-77ff-4d4d-a8e0-bc2717690bf3 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:38.642532 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f261cc2d-77ff-4d4d-a8e0-bc2717690bf3 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.275882 Sep 27 10:08:38.643934 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1699/3110] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:38 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:38.667022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c628f19b-a048-48da-b92c-6e4ca5326c59 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:38.668073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c628f19b-a048-48da-b92c-6e4ca5326c59 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:38.674751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c628f19b-a048-48da-b92c-6e4ca5326c59 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:38.675673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c628f19b-a048-48da-b92c-6e4ca5326c59 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:38.676740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c628f19b-a048-48da-b92c-6e4ca5326c59 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:38.677842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c628f19b-a048-48da-b92c-6e4ca5326c59 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:38.678592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c628f19b-a048-48da-b92c-6e4ca5326c59 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:38.679310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c628f19b-a048-48da-b92c-6e4ca5326c59 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:39.018834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c628f19b-a048-48da-b92c-6e4ca5326c59 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:39.040101 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c628f19b-a048-48da-b92c-6e4ca5326c59 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.390458 Sep 27 10:08:39.042077 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1700/3111] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:38 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:39.052689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d3736d4-b12b-48fd-8e52-48b85c9b75d3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:39.054391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4d3736d4-b12b-48fd-8e52-48b85c9b75d3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:39.151156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4d3736d4-b12b-48fd-8e52-48b85c9b75d3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:39.162645 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4d3736d4-b12b-48fd-8e52-48b85c9b75d3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1222 microversion: 2.1 time: 0.118860 Sep 27 10:08:39.164007 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1701/3112] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:39 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1222 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:39.169457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:39.170071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:39.194221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:39.194862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:39.197153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:39.197419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:39.198258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:39.200706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:39.352946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:39.354328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:39.355474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:39.356741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lazy-loading 'fault' on Instance uuid 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:08:39.494791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '68d9297ca0884c2fa7f5283f722451ba', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8130333e2c04a3f960dc6465e07cc46', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:39.527653 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bb70a1c1-462a-4f4b-96b4-e4e11bdef9f7 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 200 len: 1313 microversion: 2.1 time: 0.357817 Sep 27 10:08:39.530911 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1702/3113] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:39 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 1313 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:39.551925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4722d347-1bb7-48b5-81fe-0b9bfd03294c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:39.562173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4722d347-1bb7-48b5-81fe-0b9bfd03294c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:39.581687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4722d347-1bb7-48b5-81fe-0b9bfd03294c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:39.582789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4722d347-1bb7-48b5-81fe-0b9bfd03294c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:39.586639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7876d927-ea95-489a-a3d5-5ab6a8e132a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:39.591969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4722d347-1bb7-48b5-81fe-0b9bfd03294c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:39.600296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4722d347-1bb7-48b5-81fe-0b9bfd03294c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:39.612237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4722d347-1bb7-48b5-81fe-0b9bfd03294c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:39.614836 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7876d927-ea95-489a-a3d5-5ab6a8e132a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1507 microversion: 2.1 time: 1.603659 Sep 27 10:08:39.622790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4722d347-1bb7-48b5-81fe-0b9bfd03294c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:39.628819 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1412/3114] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:38 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1507 bytes in 1618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:39.647000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:39.657629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:39.675681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:39.678739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:39.682203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:39.683065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:39.683602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:39.684209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:39.899970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4722d347-1bb7-48b5-81fe-0b9bfd03294c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:39.911030 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4722d347-1bb7-48b5-81fe-0b9bfd03294c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.376884 Sep 27 10:08:39.911637 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1703/3115] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:39 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:39.919068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0074a8a9-b2f9-4eee-a5a2-897000d5bf53 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:39.920781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0074a8a9-b2f9-4eee-a5a2-897000d5bf53 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:39.928267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0074a8a9-b2f9-4eee-a5a2-897000d5bf53 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:39.929045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0074a8a9-b2f9-4eee-a5a2-897000d5bf53 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:39.929817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0074a8a9-b2f9-4eee-a5a2-897000d5bf53 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:39.930960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0074a8a9-b2f9-4eee-a5a2-897000d5bf53 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:39.931779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0074a8a9-b2f9-4eee-a5a2-897000d5bf53 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:39.932813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0074a8a9-b2f9-4eee-a5a2-897000d5bf53 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:39.955064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:39.973080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:39.978063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:40.003539 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-db75c1dc-d02c-4309-8aec-1447a5d7fec0 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.364685 Sep 27 10:08:40.006283 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1413/3116] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:39 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:40.013104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:40.013935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:40.023024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.037149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.039012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.041352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.043671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.044359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.087516 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Cannot 'resume' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action HTTP/1.1 Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 14 Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: {"resume": {}} Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: exception: Cannot 'resume' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3327ff95-5c05-4167-a73b-5fd55f7bfabd', 'body': {'resume': {}}} Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 59, in _resume Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.compute_api.resume(context, server) Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 309, in inner Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd in vm_state active. Cannot resume while the instance is in this state. Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 63, in _resume Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'resume' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:40.089622 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:40.094241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:40.094241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:40.094241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:40.094241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:40.094241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:40.094241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'resume' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:08:40.094241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:40.096175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 409 to user: Cannot 'resume' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:40.097332 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-63087ecd-8dbf-4d09-8bbf-2c0e03d23064 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 409 len: 144 microversion: 2.1 time: 0.089474 Sep 27 10:08:40.098561 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1414/3117] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:40 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 144 bytes in 91 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:08:40.114829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-39c1fca4-4799-4aea-a327-36ffbfc1e545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:40.118661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-39c1fca4-4799-4aea-a327-36ffbfc1e545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:40.129346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39c1fca4-4799-4aea-a327-36ffbfc1e545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.129885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39c1fca4-4799-4aea-a327-36ffbfc1e545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.131185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39c1fca4-4799-4aea-a327-36ffbfc1e545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.132222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39c1fca4-4799-4aea-a327-36ffbfc1e545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.133095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39c1fca4-4799-4aea-a327-36ffbfc1e545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.133998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39c1fca4-4799-4aea-a327-36ffbfc1e545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.222030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0074a8a9-b2f9-4eee-a5a2-897000d5bf53 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:40.235655 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0074a8a9-b2f9-4eee-a5a2-897000d5bf53 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.322688 Sep 27 10:08:40.236854 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1704/3118] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:39 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:40.242607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c99ac88c-a1e4-4f0e-b626-2797c2ae9bc6 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:40.243622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c99ac88c-a1e4-4f0e-b626-2797c2ae9bc6 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:40.251026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99ac88c-a1e4-4f0e-b626-2797c2ae9bc6 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.251783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99ac88c-a1e4-4f0e-b626-2797c2ae9bc6 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.252698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99ac88c-a1e4-4f0e-b626-2797c2ae9bc6 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.253659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99ac88c-a1e4-4f0e-b626-2797c2ae9bc6 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.254148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99ac88c-a1e4-4f0e-b626-2797c2ae9bc6 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.254586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99ac88c-a1e4-4f0e-b626-2797c2ae9bc6 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.333069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c99ac88c-a1e4-4f0e-b626-2797c2ae9bc6 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:40.350636 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c99ac88c-a1e4-4f0e-b626-2797c2ae9bc6 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1226 microversion: 2.1 time: 0.111970 Sep 27 10:08:40.351576 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1705/3119] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:40 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1226 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:40.554191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-39c1fca4-4799-4aea-a327-36ffbfc1e545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:40.560834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:40.561464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:40.563790 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-39c1fca4-4799-4aea-a327-36ffbfc1e545 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.460697 Sep 27 10:08:40.577030 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1415/3120] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:40 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:40.589658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.591602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.593119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.595986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.609036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-896d2013-fd71-4a87-b560-90bbe15a7401 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:40.609639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.610440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.610591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-896d2013-fd71-4a87-b560-90bbe15a7401 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:40.620429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-896d2013-fd71-4a87-b560-90bbe15a7401 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.630405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-896d2013-fd71-4a87-b560-90bbe15a7401 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.631216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-896d2013-fd71-4a87-b560-90bbe15a7401 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.632419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-896d2013-fd71-4a87-b560-90bbe15a7401 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.633479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-896d2013-fd71-4a87-b560-90bbe15a7401 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.633479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-896d2013-fd71-4a87-b560-90bbe15a7401 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.661246 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] HTTP exception thrown: Instance 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c could not be found. Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c HTTP/1.1 Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c could not be found. Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '1b7c9d0a-630a-4c9a-8c25-56f06e0d851c'} Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c could not be found. Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c could not be found. Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:40.671742 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:40.676850 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:40.676850 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:40.676850 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:40.676850 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:40.676850 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:40.676850 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:40.676850 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c could not be found. Sep 27 10:08:40.676850 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:40.676850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 404 to user: Instance 1b7c9d0a-630a-4c9a-8c25-56f06e0d851c could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:40.678840 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b4672a14-8244-4b7e-a990-1b266900b819 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c" status: 404 len: 111 microversion: 2.1 time: 0.125109 Sep 27 10:08:40.682111 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1706/3121] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:40 2022] GET /compute/v2.1/servers/1b7c9d0a-630a-4c9a-8c25-56f06e0d851c => generated 111 bytes in 129 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:08:40.699378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eb4c6fcf-f1c6-4b8e-8251-91fc9e244869 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:40.702424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eb4c6fcf-f1c6-4b8e-8251-91fc9e244869 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:40.718473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb4c6fcf-f1c6-4b8e-8251-91fc9e244869 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.719342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb4c6fcf-f1c6-4b8e-8251-91fc9e244869 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.719981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb4c6fcf-f1c6-4b8e-8251-91fc9e244869 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.720930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb4c6fcf-f1c6-4b8e-8251-91fc9e244869 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:40.721928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb4c6fcf-f1c6-4b8e-8251-91fc9e244869 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:40.722570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb4c6fcf-f1c6-4b8e-8251-91fc9e244869 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:40.950334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-eb4c6fcf-f1c6-4b8e-8251-91fc9e244869 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:40.967160 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eb4c6fcf-f1c6-4b8e-8251-91fc9e244869 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1528 microversion: 2.1 time: 0.283342 Sep 27 10:08:40.969473 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1707/3122] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:40 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1528 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:40.977579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-00d3b101-d392-4dd9-9069-4fb0a0cf5bc3 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '.version_select at 0x7f7f63815f70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:41.030575 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-00d3b101-d392-4dd9-9069-4fb0a0cf5bc3 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] HTTP exception thrown: Image not found. Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-00d3b101-d392-4dd9-9069-4fb0a0cf5bc3 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/images/c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb HTTP/1.1 Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: exception: Image not found. Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb'} Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb to delete Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb could not be found. Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:41.033966 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.040332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-00d3b101-d392-4dd9-9069-4fb0a0cf5bc3 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 404 to user: Image not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:41.043852 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-00d3b101-d392-4dd9-9069-4fb0a0cf5bc3 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "DELETE /compute/v2.1/images/c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb" status: 404 len: 62 microversion: 2.1 time: 0.065413 Sep 27 10:08:41.045061 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1708/3123] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:40 2022] DELETE /compute/v2.1/images/c8cbbdc6-3cdf-47a8-87b7-4e2bdf00b4eb => generated 62 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:08:41.065085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:41.067409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:41.107699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:41.111141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:41.111940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:41.115789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:41.116439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:41.117730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:41.296020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:41.339684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-896d2013-fd71-4a87-b560-90bbe15a7401 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:41.350149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:41.355439 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-896d2013-fd71-4a87-b560-90bbe15a7401 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.750869 Sep 27 10:08:41.355756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:41.379997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:41.383828 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d6211794-53ef-47db-a06b-92f0d78339a6 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.337340 Sep 27 10:08:41.384818 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1709/3125] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:41 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:41.394169 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1416/3125] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:40 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 782 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:08:41.396523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f26d8edd-9d75-4eed-9cd7-d0afd56d7272 req-a21a9678-36b2-4d9f-bbf0-ffe09a138b78 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b902a024-8aa7-4843-8680-41d3e189d3af", "name": "network-vif-plugged", "status": "completed", "tag": "df367b62-0029-488e-9897-fac934fc397e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:41.423687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f26d8edd-9d75-4eed-9cd7-d0afd56d7272 req-a21a9678-36b2-4d9f-bbf0-ffe09a138b78 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:41.427070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f26d8edd-9d75-4eed-9cd7-d0afd56d7272 req-a21a9678-36b2-4d9f-bbf0-ffe09a138b78 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:41.430522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f26d8edd-9d75-4eed-9cd7-d0afd56d7272 req-a21a9678-36b2-4d9f-bbf0-ffe09a138b78 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:41.450389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:41.463431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:41.465606 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f26d8edd-9d75-4eed-9cd7-d0afd56d7272 req-a21a9678-36b2-4d9f-bbf0-ffe09a138b78 service nova] Creating event network-vif-plugged:df367b62-0029-488e-9897-fac934fc397e for instance b902a024-8aa7-4843-8680-41d3e189d3af on nch1-master-1 Sep 27 10:08:41.485740 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f26d8edd-9d75-4eed-9cd7-d0afd56d7272 req-a21a9678-36b2-4d9f-bbf0-ffe09a138b78 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099347 Sep 27 10:08:41.486700 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1710/3126] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:41.495445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:41.495953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:41.496827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a2d4e4f6-e3a9-40be-b817-380b8481cd83 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:41.498761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a2d4e4f6-e3a9-40be-b817-380b8481cd83 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:41.502044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:41.505256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:41.505996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:41.508073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:41.521294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2d4e4f6-e3a9-40be-b817-380b8481cd83 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:41.522102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2d4e4f6-e3a9-40be-b817-380b8481cd83 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:41.522762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2d4e4f6-e3a9-40be-b817-380b8481cd83 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:41.528357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2d4e4f6-e3a9-40be-b817-380b8481cd83 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:41.529193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2d4e4f6-e3a9-40be-b817-380b8481cd83 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:41.529886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2d4e4f6-e3a9-40be-b817-380b8481cd83 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:41.585599 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] HTTP exception thrown: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa HTTP/1.1 Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa'} Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:41.589690 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:41.594753 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:41.594753 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.594753 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:41.594753 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.594753 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:41.594753 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:41.594753 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:41.594753 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:41.594753 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:41.594753 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:41.594753 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:41.598918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 404 to user: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:41.604441 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8ac30143-5934-4083-adea-bd95eca2bca8 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 404 len: 111 microversion: 2.1 time: 0.204967 Sep 27 10:08:41.606908 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1417/3127] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:08:41 2022] DELETE /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 111 bytes in 212 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:41.619483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-397c3187-bc55-4dd1-bd5a-a542a15e9b9d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:41.622314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-397c3187-bc55-4dd1-bd5a-a542a15e9b9d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:41.638697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-397c3187-bc55-4dd1-bd5a-a542a15e9b9d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:41.644687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-397c3187-bc55-4dd1-bd5a-a542a15e9b9d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:41.646332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-397c3187-bc55-4dd1-bd5a-a542a15e9b9d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:41.647442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-397c3187-bc55-4dd1-bd5a-a542a15e9b9d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:41.649696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-397c3187-bc55-4dd1-bd5a-a542a15e9b9d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:41.650377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-397c3187-bc55-4dd1-bd5a-a542a15e9b9d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:41.949990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a2d4e4f6-e3a9-40be-b817-380b8481cd83 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:42.000225 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a2d4e4f6-e3a9-40be-b817-380b8481cd83 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.510547 Sep 27 10:08:42.001861 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1711/3128] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:41 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 514 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:42.007756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:42.008984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:42.020054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:42.020499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:42.021875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:42.023338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:42.024395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:42.024871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:42.054738 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] HTTP exception thrown: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa HTTP/1.1 Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa'} Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.057303 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:42.063125 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.063125 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:42.063125 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:42.063125 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:42.063125 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:42.063125 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:42.063125 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. Sep 27 10:08:42.063125 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.065404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 404 to user: Instance 4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:42.066565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a89d6c1b-726a-484c-b994-8f3b65488f7b tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa" status: 404 len: 111 microversion: 2.1 time: 0.063201 Sep 27 10:08:42.067202 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1712/3129] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:42 2022] GET /compute/v2.1/servers/4b3e86ea-d4ee-4dec-ad82-5ef52b4781fa => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:42.070346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-397c3187-bc55-4dd1-bd5a-a542a15e9b9d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:42.075662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-719c0bbb-925e-41b7-9929-e7f62b8413c4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'create', calling method: >, body: {"server": {"name": "", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-719c0bbb-925e-41b7-9929-e7f62b8413c4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 160 Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "fabc7738-7394-4084-bbfe-24c62c6029b4"}]}} Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute name. Value: . '' is too short Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'server': {'name': '', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'networks': [{'uuid': 'fabc7738-7394-4084-bbfe-24c62c6029b4'}]}}} Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['server']['properties']['name']: Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi '' Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:08:42.080955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-719c0bbb-925e-41b7-9929-e7f62b8413c4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:42.088294 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-719c0bbb-925e-41b7-9929-e7f62b8413c4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 108 microversion: 2.1 time: 0.013764 Sep 27 10:08:42.094925 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1713/3130] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:08:42 2022] POST /compute/v2.1/servers => generated 108 bytes in 27 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:08:42.100551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e16b84a-f055-4758-b72c-8f93ca7fddfb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:42.106573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9e16b84a-f055-4758-b72c-8f93ca7fddfb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:42.142055 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-397c3187-bc55-4dd1-bd5a-a542a15e9b9d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1274 microversion: 2.1 time: 0.514842 Sep 27 10:08:42.142981 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1418/3131] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:41 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1274 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:42.158855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e16b84a-f055-4758-b72c-8f93ca7fddfb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:42.162709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7550326d-6426-407c-8146-d98d2c81a626 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '.version_select at 0x7f7f62b9c430>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:42.163876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e16b84a-f055-4758-b72c-8f93ca7fddfb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:42.167240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e16b84a-f055-4758-b72c-8f93ca7fddfb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:42.168013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e16b84a-f055-4758-b72c-8f93ca7fddfb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:42.168495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e16b84a-f055-4758-b72c-8f93ca7fddfb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:42.168986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e16b84a-f055-4758-b72c-8f93ca7fddfb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:42.195507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:42.226602 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7550326d-6426-407c-8146-d98d2c81a626 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1310573978" status: 202 len: 0 microversion: 2.1 time: 0.081405 Sep 27 10:08:42.227608 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1419/3132] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:08:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1310573978 => generated 0 bytes in 83 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:42.247567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1995d6d6-44a2-46b8-a3df-991fa8333224 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:42.249103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1995d6d6-44a2-46b8-a3df-991fa8333224 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:42.269505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1995d6d6-44a2-46b8-a3df-991fa8333224 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:42.274062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1995d6d6-44a2-46b8-a3df-991fa8333224 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:42.276165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1995d6d6-44a2-46b8-a3df-991fa8333224 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:42.279528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1995d6d6-44a2-46b8-a3df-991fa8333224 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:42.281300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1995d6d6-44a2-46b8-a3df-991fa8333224 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:42.281907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1995d6d6-44a2-46b8-a3df-991fa8333224 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:42.739325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9e16b84a-f055-4758-b72c-8f93ca7fddfb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:42.753316 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9e16b84a-f055-4758-b72c-8f93ca7fddfb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1528 microversion: 2.1 time: 0.656916 Sep 27 10:08:42.757082 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1714/3133] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:42 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1528 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:42.763558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:42.764176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:42.783425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:42.788943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:42.790600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:42.791795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:42.793138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:42.795178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:42.958541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:42.978531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:42.991898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:43.012583 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-41b61f61-6c11-4351-98cd-c89a27b7da3b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1544 microversion: 2.91 time: 0.253947 Sep 27 10:08:43.014405 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1715/3134] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:42 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1544 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:43.032805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8d34acdd-ee17-4849-a957-de2754ba0c5e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:43.035292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8d34acdd-ee17-4849-a957-de2754ba0c5e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:43.047479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d34acdd-ee17-4849-a957-de2754ba0c5e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:43.048301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d34acdd-ee17-4849-a957-de2754ba0c5e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:43.050234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d34acdd-ee17-4849-a957-de2754ba0c5e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:43.051905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d34acdd-ee17-4849-a957-de2754ba0c5e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:43.053690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d34acdd-ee17-4849-a957-de2754ba0c5e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:43.055400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d34acdd-ee17-4849-a957-de2754ba0c5e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:43.326299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1995d6d6-44a2-46b8-a3df-991fa8333224 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:43.340245 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1995d6d6-44a2-46b8-a3df-991fa8333224 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 1.107755 Sep 27 10:08:43.364555 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1420/3135] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:42 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 1135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:43.369942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3fe2adc4-ba41-4540-bb9b-d4754d29b01d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:43.370575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3fe2adc4-ba41-4540-bb9b-d4754d29b01d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:43.388617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe2adc4-ba41-4540-bb9b-d4754d29b01d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:43.389059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe2adc4-ba41-4540-bb9b-d4754d29b01d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:43.389544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe2adc4-ba41-4540-bb9b-d4754d29b01d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:43.390486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe2adc4-ba41-4540-bb9b-d4754d29b01d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:43.391105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe2adc4-ba41-4540-bb9b-d4754d29b01d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:43.391698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fe2adc4-ba41-4540-bb9b-d4754d29b01d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:43.480866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8d34acdd-ee17-4849-a957-de2754ba0c5e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:43.494255 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8d34acdd-ee17-4849-a957-de2754ba0c5e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.469584 Sep 27 10:08:43.505094 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1716/3136] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:43 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:08:43.510417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-24c2a8e0-c191-4656-86c4-a3be6faeaa00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:43.510417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-24c2a8e0-c191-4656-86c4-a3be6faeaa00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:43.527184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c2a8e0-c191-4656-86c4-a3be6faeaa00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:43.530244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c2a8e0-c191-4656-86c4-a3be6faeaa00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:43.530524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c2a8e0-c191-4656-86c4-a3be6faeaa00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:43.533207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c2a8e0-c191-4656-86c4-a3be6faeaa00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:43.534903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c2a8e0-c191-4656-86c4-a3be6faeaa00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:43.535363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c2a8e0-c191-4656-86c4-a3be6faeaa00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:43.619816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3fe2adc4-ba41-4540-bb9b-d4754d29b01d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:43.628573 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3fe2adc4-ba41-4540-bb9b-d4754d29b01d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1280 microversion: 2.1 time: 0.262826 Sep 27 10:08:43.629960 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1421/3137] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:43 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1280 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:43.793676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32e8cbfd-0cce-49f2-ab5d-b2a933073613 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:43.797133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32e8cbfd-0cce-49f2-ab5d-b2a933073613 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:43.805990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e8cbfd-0cce-49f2-ab5d-b2a933073613 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:43.806871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e8cbfd-0cce-49f2-ab5d-b2a933073613 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:43.812800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e8cbfd-0cce-49f2-ab5d-b2a933073613 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:43.814571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e8cbfd-0cce-49f2-ab5d-b2a933073613 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:43.815296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e8cbfd-0cce-49f2-ab5d-b2a933073613 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:43.816354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e8cbfd-0cce-49f2-ab5d-b2a933073613 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:44.255391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-24c2a8e0-c191-4656-86c4-a3be6faeaa00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:44.272058 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-24c2a8e0-c191-4656-86c4-a3be6faeaa00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.773355 Sep 27 10:08:44.276065 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1717/3138] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:43 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 779 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:44.283130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:44.283933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:44.292901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:44.293901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:44.294627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:44.297401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:44.298218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:44.299258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:44.383817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:44.394509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:44.396007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:44.402021 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-03e115ca-2d7e-45d9-a86f-8d4df8c6f99b tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1531 microversion: 2.91 time: 0.124078 Sep 27 10:08:44.402938 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1718/3139] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:44 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1531 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:44.408189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f43ca028-eb05-4796-a9f9-a6cacada0169 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:44.409094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f43ca028-eb05-4796-a9f9-a6cacada0169 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: de980c1b-26a0-4fd7-8d12-a2be2b94683f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:44.415551 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-f43ca028-eb05-4796-a9f9-a6cacada0169 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance de980c1b-26a0-4fd7-8d12-a2be2b94683f could not be found. Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f43ca028-eb05-4796-a9f9-a6cacada0169 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/de980c1b-26a0-4fd7-8d12-a2be2b94683f/action HTTP/1.1 Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 14 Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: {"shelve": {}} Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance de980c1b-26a0-4fd7-8d12-a2be2b94683f could not be found. Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'de980c1b-26a0-4fd7-8d12-a2be2b94683f', 'body': {'shelve': {}}} Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance de980c1b-26a0-4fd7-8d12-a2be2b94683f could not be found. Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 44, in _shelve Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance de980c1b-26a0-4fd7-8d12-a2be2b94683f could not be found. Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:44.421219 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:44.426191 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:44.426191 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:44.426191 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance de980c1b-26a0-4fd7-8d12-a2be2b94683f could not be found. Sep 27 10:08:44.426191 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.427395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f43ca028-eb05-4796-a9f9-a6cacada0169 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance de980c1b-26a0-4fd7-8d12-a2be2b94683f could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:44.428473 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f43ca028-eb05-4796-a9f9-a6cacada0169 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/de980c1b-26a0-4fd7-8d12-a2be2b94683f/action" status: 404 len: 111 microversion: 2.1 time: 0.024175 Sep 27 10:08:44.429432 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1719/3140] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:44 2022] POST /compute/v2.1/servers/de980c1b-26a0-4fd7-8d12-a2be2b94683f/action => generated 111 bytes in 25 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:44.437703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5501a99d-9fc3-430c-8aa1-9da12d23b20d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '.version_select at 0x7f7f6c22d700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:44.461931 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-5501a99d-9fc3-430c-8aa1-9da12d23b20d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] HTTP exception thrown: Image not found. Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5501a99d-9fc3-430c-8aa1-9da12d23b20d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/images/82527f1b-6d7a-4517-bf5e-959689e1b5d0 HTTP/1.1 Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: exception: Image not found. Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '82527f1b-6d7a-4517-bf5e-959689e1b5d0'} Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 724, in delete Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._client.call(context, 2, 'delete', args=(image_id,)) Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 191, in call Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = getattr(controller, method)(*args, **kwargs) Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/utils.py", line 670, in inner Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return RequestIdProxy(wrapped(*args, **kwargs)) Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/v2/images.py", line 394, in delete Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.http_client.delete(url) Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/adapter.py", line 410, in delete Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, 'DELETE', **kwargs) Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 380, in request Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._handle_response(resp) Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/glanceclient/common/http.py", line 120, in _handle_response Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.from_response(resp, resp.content) Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi glanceclient.exc.HTTPNotFound: HTTP 404 Not Found: Failed to find image 82527f1b-6d7a-4517-bf5e-959689e1b5d0 to delete Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 104, in delete Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._image_api.delete(context, id) Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 1255, in delete Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return session.delete(context, image_id) Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/image/glance.py", line 726, in delete Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ImageNotFound(image_id=image_id) Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ImageNotFound: Image 82527f1b-6d7a-4517-bf5e-959689e1b5d0 could not be found. Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:44.463709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/images.py", line 107, in delete Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=explanation) Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Image not found. Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Image not found. Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5501a99d-9fc3-430c-8aa1-9da12d23b20d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 404 to user: Image not found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5501a99d-9fc3-430c-8aa1-9da12d23b20d tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "DELETE /compute/v2.1/images/82527f1b-6d7a-4517-bf5e-959689e1b5d0" status: 404 len: 62 microversion: 2.1 time: 0.031561 Sep 27 10:08:44.468563 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1720/3141] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:44 2022] DELETE /compute/v2.1/images/82527f1b-6d7a-4517-bf5e-959689e1b5d0 => generated 62 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:08:44.471945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8856d7d0-6b93-4a4c-b32b-0447cb316d0d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:44.472554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8856d7d0-6b93-4a4c-b32b-0447cb316d0d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:44.479645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8856d7d0-6b93-4a4c-b32b-0447cb316d0d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:44.480171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8856d7d0-6b93-4a4c-b32b-0447cb316d0d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:44.480713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8856d7d0-6b93-4a4c-b32b-0447cb316d0d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:44.481409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8856d7d0-6b93-4a4c-b32b-0447cb316d0d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:44.481876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8856d7d0-6b93-4a4c-b32b-0447cb316d0d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:44.482303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8856d7d0-6b93-4a4c-b32b-0447cb316d0d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:44.571205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-8856d7d0-6b93-4a4c-b32b-0447cb316d0d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] Lazy-loading 'services' on Instance uuid 42709739-135c-4adb-aa6c-4fc216a5d618 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:08:44.598654 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8856d7d0-6b93-4a4c-b32b-0447cb316d0d tempest-UnshelveToHostMultiNodesTest-1811880953 tempest-UnshelveToHostMultiNodesTest-1811880953-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1936 microversion: 2.91 time: 0.130856 Sep 27 10:08:44.599897 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1721/3142] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:44 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1936 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:44.605625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-70f9f225-ab5c-4272-8d25-3e78783ee422 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:44.606490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-70f9f225-ab5c-4272-8d25-3e78783ee422 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:44.614196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70f9f225-ab5c-4272-8d25-3e78783ee422 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:44.615546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70f9f225-ab5c-4272-8d25-3e78783ee422 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:44.616171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70f9f225-ab5c-4272-8d25-3e78783ee422 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:44.617153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70f9f225-ab5c-4272-8d25-3e78783ee422 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:44.618165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70f9f225-ab5c-4272-8d25-3e78783ee422 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:44.618914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70f9f225-ab5c-4272-8d25-3e78783ee422 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:44.805470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-70f9f225-ab5c-4272-8d25-3e78783ee422 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:44.825072 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-70f9f225-ab5c-4272-8d25-3e78783ee422 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.222946 Sep 27 10:08:44.847223 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1722/3143] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:44 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:44.859040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:44.859547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:44.872298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:44.873999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:44.877584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:44.878414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:44.878935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:44.879374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:44.902162 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] HTTP exception thrown: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 HTTP/1.1 Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '0abee65c-58ee-4b9c-ac6c-65631519b216'} Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:44.903770 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 404 to user: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fa8bad90-96dc-41d5-a302-0caed7242c40 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 404 len: 111 microversion: 2.1 time: 0.057040 Sep 27 10:08:44.908133 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1723/3144] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:08:44 2022] DELETE /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:44.916519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5896eebc-d801-40a7-acac-6b8d48eaa1b1 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:44.917172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5896eebc-d801-40a7-acac-6b8d48eaa1b1 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:44.924370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5896eebc-d801-40a7-acac-6b8d48eaa1b1 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:44.924894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5896eebc-d801-40a7-acac-6b8d48eaa1b1 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:44.925330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5896eebc-d801-40a7-acac-6b8d48eaa1b1 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:44.926423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5896eebc-d801-40a7-acac-6b8d48eaa1b1 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:44.926874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5896eebc-d801-40a7-acac-6b8d48eaa1b1 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:44.927297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5896eebc-d801-40a7-acac-6b8d48eaa1b1 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.044183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32e8cbfd-0cce-49f2-ab5d-b2a933073613 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:45.057531 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32e8cbfd-0cce-49f2-ab5d-b2a933073613 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1528 microversion: 2.1 time: 1.272221 Sep 27 10:08:45.059090 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1422/3145] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:43 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1528 bytes in 1279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:45.069886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8acded61-0897-41cb-a058-b665c1387b23 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:45.070748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8acded61-0897-41cb-a058-b665c1387b23 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:45.082570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8acded61-0897-41cb-a058-b665c1387b23 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:45.083372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8acded61-0897-41cb-a058-b665c1387b23 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:45.084083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8acded61-0897-41cb-a058-b665c1387b23 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.085476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8acded61-0897-41cb-a058-b665c1387b23 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:45.086557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8acded61-0897-41cb-a058-b665c1387b23 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:45.087425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8acded61-0897-41cb-a058-b665c1387b23 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.179766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5896eebc-d801-40a7-acac-6b8d48eaa1b1 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:45.181553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8acded61-0897-41cb-a058-b665c1387b23 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:08:45.198815 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5896eebc-d801-40a7-acac-6b8d48eaa1b1 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.286507 Sep 27 10:08:45.199835 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1724/3146] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:44 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:45.214919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-637b3cd7-b63c-4f0d-b246-271c76afe119 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:45.215910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-637b3cd7-b63c-4f0d-b246-271c76afe119 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:45.223470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-637b3cd7-b63c-4f0d-b246-271c76afe119 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:45.223982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-637b3cd7-b63c-4f0d-b246-271c76afe119 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:45.224449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-637b3cd7-b63c-4f0d-b246-271c76afe119 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.230904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-637b3cd7-b63c-4f0d-b246-271c76afe119 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:45.231502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-637b3cd7-b63c-4f0d-b246-271c76afe119 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:45.232097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-637b3cd7-b63c-4f0d-b246-271c76afe119 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.345251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-637b3cd7-b63c-4f0d-b246-271c76afe119 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:45.362552 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-637b3cd7-b63c-4f0d-b246-271c76afe119 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1280 microversion: 2.1 time: 0.158649 Sep 27 10:08:45.376773 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1725/3147] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:45 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1280 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:45.376773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d23622e3-4a3a-44e9-af62-4ce5ba487f04 req-4029a874-53d7-4714-8f1b-8cf1a8c66ea1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08babbce-373e-4cf5-a138-066b6eeb02c3", "name": "network-vif-plugged", "status": "completed", "tag": "2ca9e80c-8a06-414a-81cf-a330f210be36"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:45.386162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d23622e3-4a3a-44e9-af62-4ce5ba487f04 req-4029a874-53d7-4714-8f1b-8cf1a8c66ea1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:45.386958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d23622e3-4a3a-44e9-af62-4ce5ba487f04 req-4029a874-53d7-4714-8f1b-8cf1a8c66ea1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:45.387595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d23622e3-4a3a-44e9-af62-4ce5ba487f04 req-4029a874-53d7-4714-8f1b-8cf1a8c66ea1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.402898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:45.419096 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d23622e3-4a3a-44e9-af62-4ce5ba487f04 req-4029a874-53d7-4714-8f1b-8cf1a8c66ea1 service nova] Creating event network-vif-plugged:2ca9e80c-8a06-414a-81cf-a330f210be36 for instance 08babbce-373e-4cf5-a138-066b6eeb02c3 on nch2-master-1 Sep 27 10:08:45.428471 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d23622e3-4a3a-44e9-af62-4ce5ba487f04 req-4029a874-53d7-4714-8f1b-8cf1a8c66ea1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062697 Sep 27 10:08:45.433862 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1726/3148] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:45.440551 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8acded61-0897-41cb-a058-b665c1387b23 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "DELETE /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 204 len: 0 microversion: 2.91 time: 0.377869 Sep 27 10:08:45.441718 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1423/3149] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:08:45 2022] DELETE /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:08:45.442204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30664b6b-a870-4c22-adde-c4b9ebce4455 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:45.443292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-30664b6b-a870-4c22-adde-c4b9ebce4455 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:45.449959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:45.451009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] [instance: 0abee65c-58ee-4b9c-ac6c-65631519b216] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:45.452007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30664b6b-a870-4c22-adde-c4b9ebce4455 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:45.452766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30664b6b-a870-4c22-adde-c4b9ebce4455 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:45.453389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30664b6b-a870-4c22-adde-c4b9ebce4455 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.454201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30664b6b-a870-4c22-adde-c4b9ebce4455 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:45.454844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30664b6b-a870-4c22-adde-c4b9ebce4455 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:45.455362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30664b6b-a870-4c22-adde-c4b9ebce4455 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.468107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:45.468835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:45.469540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.470756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:45.471395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:45.472049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.493384 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] HTTP exception thrown: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Request method failure captured: Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 HTTP/1.1 Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMssT-PQObbvfvbZyxMTPj5GrvTXvmHAx9FfpDtrEYhJAm00u-iKnAu7LfKrmbmlxB1hpIn78HOJuj3fO76fEHoo6cW-_mCoM2kL7kaXNbdTiZ9rmFGgqBWNTip2I5Cl3DQNzC2Y1fWgnENksTxE3NTUoroolOoVJFxjiNzwFhNeBDVI Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f8130333e2c04a3f960dc6465e07cc46"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f8130333e2c04a3f960dc6465e07cc46"}]}] Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: f8130333e2c04a3f960dc6465e07cc46 Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073 Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 68d9297ca0884c2fa7f5283f722451ba Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ImagesOneServerNegativeTestJSON-2069318073-project Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '0abee65c-58ee-4b9c-ac6c-65631519b216'} Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:45.496329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:45.500889 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:45.500889 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:45.500889 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:45.500889 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:45.500889 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:45.500889 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:45.500889 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. Sep 27 10:08:45.500889 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:45.502503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] Returning 404 to user: Instance 0abee65c-58ee-4b9c-ac6c-65631519b216 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:45.505848 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-495a42cb-dd0a-49f1-93aa-f00eb63c2d61 tempest-ImagesOneServerNegativeTestJSON-2069318073 tempest-ImagesOneServerNegativeTestJSON-2069318073-project] 10.222.0.26 "GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216" status: 404 len: 111 microversion: 2.1 time: 0.061351 Sep 27 10:08:45.507731 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1424/3150] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:45 2022] GET /compute/v2.1/servers/0abee65c-58ee-4b9c-ac6c-65631519b216 => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:08:45.521076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:45.522442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:45.534637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:45.538906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:45.539638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.541894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:45.542506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:45.543834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:45.722425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:45.732467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:45.733999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:45.751910 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7cb0ee6e-0fd9-4e9a-8819-e2dcf9874dfe tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.242862 Sep 27 10:08:45.771975 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1425/3151] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:45 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:45.984333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-30664b6b-a870-4c22-adde-c4b9ebce4455 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:45.997054 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-30664b6b-a870-4c22-adde-c4b9ebce4455 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.561690 Sep 27 10:08:46.028028 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1727/3152] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:45 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:46.043544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-77a42729-0b91-4454-b7a2-6d040cd35281 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:46.044367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-77a42729-0b91-4454-b7a2-6d040cd35281 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:46.053006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77a42729-0b91-4454-b7a2-6d040cd35281 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:46.053751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77a42729-0b91-4454-b7a2-6d040cd35281 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:46.054396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77a42729-0b91-4454-b7a2-6d040cd35281 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:46.055312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77a42729-0b91-4454-b7a2-6d040cd35281 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:46.055927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77a42729-0b91-4454-b7a2-6d040cd35281 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:46.056515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77a42729-0b91-4454-b7a2-6d040cd35281 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:46.083046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-805b51c9-bb41-4629-b3dd-1225217a39d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:46.083892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-805b51c9-bb41-4629-b3dd-1225217a39d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:46.090424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-77a42729-0b91-4454-b7a2-6d040cd35281 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lazy-loading 'flavor' on Instance uuid 3327ff95-5c05-4167-a73b-5fd55f7bfabd {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:08:46.098683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-805b51c9-bb41-4629-b3dd-1225217a39d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:46.102251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-805b51c9-bb41-4629-b3dd-1225217a39d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:46.105112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-805b51c9-bb41-4629-b3dd-1225217a39d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:46.105457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:46.106906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-805b51c9-bb41-4629-b3dd-1225217a39d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:46.107804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-805b51c9-bb41-4629-b3dd-1225217a39d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:46.111106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-805b51c9-bb41-4629-b3dd-1225217a39d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:46.395977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-805b51c9-bb41-4629-b3dd-1225217a39d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:46.408644 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-805b51c9-bb41-4629-b3dd-1225217a39d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 0.329387 Sep 27 10:08:46.423826 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1728/3153] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:46 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:08:46.443361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-40dd7c50-5621-49de-b1a8-e0e6c3e5498f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:46.446937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-40dd7c50-5621-49de-b1a8-e0e6c3e5498f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:46.463742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40dd7c50-5621-49de-b1a8-e0e6c3e5498f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:46.469853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40dd7c50-5621-49de-b1a8-e0e6c3e5498f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:46.471620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40dd7c50-5621-49de-b1a8-e0e6c3e5498f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:46.472388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40dd7c50-5621-49de-b1a8-e0e6c3e5498f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:46.476651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40dd7c50-5621-49de-b1a8-e0e6c3e5498f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:46.477203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40dd7c50-5621-49de-b1a8-e0e6c3e5498f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:46.510398 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-77a42729-0b91-4454-b7a2-6d040cd35281 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 202 len: 0 microversion: 2.1 time: 0.472801 Sep 27 10:08:46.515417 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1426/3154] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:08:46 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 0 bytes in 479 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:08:46.530905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a2805f69-80bc-4334-8f12-68a290934b9c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:46.534140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a2805f69-80bc-4334-8f12-68a290934b9c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:46.559294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2805f69-80bc-4334-8f12-68a290934b9c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:46.579582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2805f69-80bc-4334-8f12-68a290934b9c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:46.583262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2805f69-80bc-4334-8f12-68a290934b9c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:46.589264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2805f69-80bc-4334-8f12-68a290934b9c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:46.595898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2805f69-80bc-4334-8f12-68a290934b9c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:46.596462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2805f69-80bc-4334-8f12-68a290934b9c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:46.871252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-40dd7c50-5621-49de-b1a8-e0e6c3e5498f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:46.882701 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-40dd7c50-5621-49de-b1a8-e0e6c3e5498f tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.452650 Sep 27 10:08:46.899230 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1729/3155] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:46 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:46.909881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba7d2470-81f8-4164-b73a-982c1b2e7d30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:46.910839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba7d2470-81f8-4164-b73a-982c1b2e7d30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:46.924328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7d2470-81f8-4164-b73a-982c1b2e7d30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:46.925933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7d2470-81f8-4164-b73a-982c1b2e7d30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:46.926638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7d2470-81f8-4164-b73a-982c1b2e7d30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:46.930980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7d2470-81f8-4164-b73a-982c1b2e7d30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:46.931819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7d2470-81f8-4164-b73a-982c1b2e7d30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:46.932457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7d2470-81f8-4164-b73a-982c1b2e7d30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:47.185645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba7d2470-81f8-4164-b73a-982c1b2e7d30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:47.198927 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba7d2470-81f8-4164-b73a-982c1b2e7d30 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.297614 Sep 27 10:08:47.201256 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1730/3156] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:46 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:47.209991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:47.210938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:47.220841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:47.229400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:47.229941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:47.230685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:47.233172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:47.233632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:47.344580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:47.356817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:47.358418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:47.370395 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d06acc29-5a5f-4b7d-b654-5239f5e4b2dd tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.168031 Sep 27 10:08:47.389225 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1731/3157] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:47 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:08:47.443113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c18f2db2-626d-463b-b89d-895dee343658 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:47.443921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c18f2db2-626d-463b-b89d-895dee343658 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:47.452357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18f2db2-626d-463b-b89d-895dee343658 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:47.454813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18f2db2-626d-463b-b89d-895dee343658 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:47.456270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18f2db2-626d-463b-b89d-895dee343658 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:47.457687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18f2db2-626d-463b-b89d-895dee343658 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:47.460878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18f2db2-626d-463b-b89d-895dee343658 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:47.461626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18f2db2-626d-463b-b89d-895dee343658 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:47.769140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c18f2db2-626d-463b-b89d-895dee343658 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:47.780786 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c18f2db2-626d-463b-b89d-895dee343658 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 0.342222 Sep 27 10:08:47.800200 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1732/3158] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:47 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:08:47.909521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bb6c0ceb-8e6a-4cf9-8d7e-7ba40be4f489 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:47.910285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bb6c0ceb-8e6a-4cf9-8d7e-7ba40be4f489 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:47.922005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6c0ceb-8e6a-4cf9-8d7e-7ba40be4f489 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:47.922779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6c0ceb-8e6a-4cf9-8d7e-7ba40be4f489 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:47.923556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6c0ceb-8e6a-4cf9-8d7e-7ba40be4f489 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:47.925197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6c0ceb-8e6a-4cf9-8d7e-7ba40be4f489 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:47.925940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6c0ceb-8e6a-4cf9-8d7e-7ba40be4f489 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:47.926643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6c0ceb-8e6a-4cf9-8d7e-7ba40be4f489 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:48.256137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb6c0ceb-8e6a-4cf9-8d7e-7ba40be4f489 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:48.265665 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bb6c0ceb-8e6a-4cf9-8d7e-7ba40be4f489 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.361860 Sep 27 10:08:48.266262 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1733/3159] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:47 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:08:48.283842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-520ec377-d108-4faf-9f4d-e5f5a1b19316 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:48.284481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-520ec377-d108-4faf-9f4d-e5f5a1b19316 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:48.292523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-520ec377-d108-4faf-9f4d-e5f5a1b19316 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:48.293040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-520ec377-d108-4faf-9f4d-e5f5a1b19316 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:48.293475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-520ec377-d108-4faf-9f4d-e5f5a1b19316 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:48.294403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-520ec377-d108-4faf-9f4d-e5f5a1b19316 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:48.294965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-520ec377-d108-4faf-9f4d-e5f5a1b19316 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:48.295395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-520ec377-d108-4faf-9f4d-e5f5a1b19316 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:48.326780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:48.332441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:48.345757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a2805f69-80bc-4334-8f12-68a290934b9c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:48.356679 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a2805f69-80bc-4334-8f12-68a290934b9c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1322 microversion: 2.1 time: 1.839248 Sep 27 10:08:48.359835 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1427/3160] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:46 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1322 bytes in 1844 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:48.405198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:48.406081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:48.414725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:48.415468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:48.416106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:48.417222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:48.417834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:48.418545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:48.618233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:48.637505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:48.643726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'f7bf3532492e43719ce03e29f9740b5f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7fb866af50c14a0393c03c1463330238', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:48.666259 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-934bd41c-6e95-402b-aef5-4083bcc17daa tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 200 len: 1537 microversion: 2.91 time: 0.266905 Sep 27 10:08:48.668143 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1428/3161] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:48 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 1537 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:08:48.686171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-520ec377-d108-4faf-9f4d-e5f5a1b19316 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:48.704272 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-520ec377-d108-4faf-9f4d-e5f5a1b19316 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.435041 Sep 27 10:08:48.727794 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1734/3162] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:48 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:48.807970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-873a68de-1c20-47ba-b8ce-898ef28e4cb5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:48.809334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-873a68de-1c20-47ba-b8ce-898ef28e4cb5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:48.830610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873a68de-1c20-47ba-b8ce-898ef28e4cb5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:48.836689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873a68de-1c20-47ba-b8ce-898ef28e4cb5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:48.840178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873a68de-1c20-47ba-b8ce-898ef28e4cb5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:48.841812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873a68de-1c20-47ba-b8ce-898ef28e4cb5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:48.844353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873a68de-1c20-47ba-b8ce-898ef28e4cb5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:48.846774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873a68de-1c20-47ba-b8ce-898ef28e4cb5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:49.298930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-709e3da6-bef3-430a-b5d1-0bdc3e894f79 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:49.299683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-709e3da6-bef3-430a-b5d1-0bdc3e894f79 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:49.302453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:08:49.322452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-709e3da6-bef3-430a-b5d1-0bdc3e894f79 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:49.322954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-709e3da6-bef3-430a-b5d1-0bdc3e894f79 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:49.323399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-709e3da6-bef3-430a-b5d1-0bdc3e894f79 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:49.335672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-709e3da6-bef3-430a-b5d1-0bdc3e894f79 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:49.336913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-709e3da6-bef3-430a-b5d1-0bdc3e894f79 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:49.345387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-709e3da6-bef3-430a-b5d1-0bdc3e894f79 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:49.782374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-873a68de-1c20-47ba-b8ce-898ef28e4cb5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:49.796122 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-873a68de-1c20-47ba-b8ce-898ef28e4cb5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 0.992525 Sep 27 10:08:49.804720 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1429/3163] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:48 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 1000 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:50.005395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc20408e-56a8-403e-b34c-5655fceb7f5e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Action: 'create', calling method: .version_select at 0x7f7f636413a0>, body: {"keypair": {"name": "tempest-keypair-667250342", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMKUyc7treY20wRjIVDgFoiO114zWAJAyQPJPEoaPZDiQHEtsNbH9VKbSCtxLJzEHxkWjBQaxU1pkDCewPI8Bdaz/P9vJ90gsBCdCBq9qnN+BGTarvG5uK4ii2Mukoei9Q=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:50.071266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-cc20408e-56a8-403e-b34c-5655fceb7f5e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Getting quotas for project 327b414889a04e189a7f940685a4ff39. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:08:50.095974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-cc20408e-56a8-403e-b34c-5655fceb7f5e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Getting quotas for user f06d16c90a5d464696f8137612c35327 and project 327b414889a04e189a7f940685a4ff39. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:08:50.175717 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cc20408e-56a8-403e-b34c-5655fceb7f5e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.369387 Sep 27 10:08:50.176407 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1430/3164] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:08:49 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:50.195175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4954a008-d012-4c4a-b266-2d44a3d9a31d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:50.196471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4954a008-d012-4c4a-b266-2d44a3d9a31d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:50.212553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4954a008-d012-4c4a-b266-2d44a3d9a31d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:50.213103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4954a008-d012-4c4a-b266-2d44a3d9a31d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:50.213553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4954a008-d012-4c4a-b266-2d44a3d9a31d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:50.214289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4954a008-d012-4c4a-b266-2d44a3d9a31d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:50.214758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4954a008-d012-4c4a-b266-2d44a3d9a31d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:50.215199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4954a008-d012-4c4a-b266-2d44a3d9a31d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:50.708974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4954a008-d012-4c4a-b266-2d44a3d9a31d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:50.721782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-709e3da6-bef3-430a-b5d1-0bdc3e894f79 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:50.722518 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4954a008-d012-4c4a-b266-2d44a3d9a31d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1322 microversion: 2.1 time: 0.544780 Sep 27 10:08:50.731460 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1431/3165] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:50 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1322 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:50.734002 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-709e3da6-bef3-430a-b5d1-0bdc3e894f79 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 1.444101 Sep 27 10:08:50.736831 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1735/3166] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:49 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 1449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:08:50.746855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e356f624-31da-44ed-bc62-58a40e8079c5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:50.749002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:50.749637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] [instance: 42709739-135c-4adb-aa6c-4fc216a5d618] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:50.750438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e356f624-31da-44ed-bc62-58a40e8079c5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:50.761477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:50.761992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:50.768132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:50.769273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:50.771478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e356f624-31da-44ed-bc62-58a40e8079c5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:50.772889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e356f624-31da-44ed-bc62-58a40e8079c5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:50.774467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:50.774938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:50.775994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e356f624-31da-44ed-bc62-58a40e8079c5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:50.778138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e356f624-31da-44ed-bc62-58a40e8079c5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:50.779618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e356f624-31da-44ed-bc62-58a40e8079c5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:50.780602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e356f624-31da-44ed-bc62-58a40e8079c5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:50.823962 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] HTTP exception thrown: Instance 42709739-135c-4adb-aa6c-4fc216a5d618 could not be found. Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Request method failure captured: Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 HTTP/1.1 Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMssJYdJJs2u5fUL6VO3pMUcESABjkk0hAbDSojsugKbqueoO3AHTI9rDeyNre__EorzLvVXBea3fosAqWP6ZzZnv55aKHYGDnebQ_uR7xwdqqt1m5EHWAIpPRzvijn2D2AlF5KUqJqWsUjfY9l1eyshrupBkh27l0TF-XFBCDchWnOU Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.91 Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 7fb866af50c14a0393c03c1463330238 Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-UnshelveToHostMultiNodesTest-1944244026 Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fb866af50c14a0393c03c1463330238"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7fb866af50c14a0393c03c1463330238"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7fb866af50c14a0393c03c1463330238"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7fb866af50c14a0393c03c1463330238"}]}] Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-UnshelveToHostMultiNodesTest-1944244026 Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 7fb866af50c14a0393c03c1463330238 Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-UnshelveToHostMultiNodesTest-1944244026 Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-UnshelveToHostMultiNodesTest-1944244026-project Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: f7bf3532492e43719ce03e29f9740b5f Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-UnshelveToHostMultiNodesTest-1944244026-project Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 42709739-135c-4adb-aa6c-4fc216a5d618 could not be found. Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '42709739-135c-4adb-aa6c-4fc216a5d618'} Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 42709739-135c-4adb-aa6c-4fc216a5d618 could not be found. Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 42709739-135c-4adb-aa6c-4fc216a5d618 could not be found. Sep 27 10:08:50.837025 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:50.842145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:50.842145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:50.842145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:50.842145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:50.842145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:50.842145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:50.842145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:50.842145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 42709739-135c-4adb-aa6c-4fc216a5d618 could not be found. Sep 27 10:08:50.842145 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:50.843613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] Returning 404 to user: Instance 42709739-135c-4adb-aa6c-4fc216a5d618 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:50.854003 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8bbeeacc-d54c-4698-a448-a354853a5dc1 tempest-UnshelveToHostMultiNodesTest-1944244026 tempest-UnshelveToHostMultiNodesTest-1944244026-project] 10.222.0.26 "GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618" status: 404 len: 111 microversion: 2.91 time: 0.121122 Sep 27 10:08:50.857721 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1432/3167] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:08:50 2022] GET /compute/v2.1/servers/42709739-135c-4adb-aa6c-4fc216a5d618 => generated 111 bytes in 125 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:08:50.869757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1822c0d-2950-474e-9e99-98351466b16a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:50.873679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a1822c0d-2950-474e-9e99-98351466b16a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:50.894762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1822c0d-2950-474e-9e99-98351466b16a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:50.895273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1822c0d-2950-474e-9e99-98351466b16a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:50.895733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1822c0d-2950-474e-9e99-98351466b16a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:50.896458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1822c0d-2950-474e-9e99-98351466b16a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:50.896984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1822c0d-2950-474e-9e99-98351466b16a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:50.897421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1822c0d-2950-474e-9e99-98351466b16a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:51.482341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e356f624-31da-44ed-bc62-58a40e8079c5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:51.515945 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e356f624-31da-44ed-bc62-58a40e8079c5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.777646 Sep 27 10:08:51.528063 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1736/3168] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:50 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 790 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:51.778313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aa9d6d5c-723a-49f6-933c-ac3b91fa0ebc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:51.781720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aa9d6d5c-723a-49f6-933c-ac3b91fa0ebc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:51.859281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9d6d5c-723a-49f6-933c-ac3b91fa0ebc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:51.861322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9d6d5c-723a-49f6-933c-ac3b91fa0ebc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:51.863939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9d6d5c-723a-49f6-933c-ac3b91fa0ebc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:51.865872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9d6d5c-723a-49f6-933c-ac3b91fa0ebc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:51.869717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9d6d5c-723a-49f6-933c-ac3b91fa0ebc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:51.871424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9d6d5c-723a-49f6-933c-ac3b91fa0ebc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:52.050969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a1822c0d-2950-474e-9e99-98351466b16a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:52.083727 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a1822c0d-2950-474e-9e99-98351466b16a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 1.216380 Sep 27 10:08:52.084604 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1433/3169] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:50 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 1226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:52.129893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3864b85-70bf-420f-a57c-2b4430034f39 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:52.130610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e3864b85-70bf-420f-a57c-2b4430034f39 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:52.157807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3864b85-70bf-420f-a57c-2b4430034f39 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:52.158312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3864b85-70bf-420f-a57c-2b4430034f39 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:52.158757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3864b85-70bf-420f-a57c-2b4430034f39 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:52.159480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3864b85-70bf-420f-a57c-2b4430034f39 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:52.160115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3864b85-70bf-420f-a57c-2b4430034f39 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:52.160811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3864b85-70bf-420f-a57c-2b4430034f39 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:52.655932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aa9d6d5c-723a-49f6-933c-ac3b91fa0ebc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:52.678443 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aa9d6d5c-723a-49f6-933c-ac3b91fa0ebc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.918793 Sep 27 10:08:52.683829 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1737/3170] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:51 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 928 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:52.688982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6bda581a-e7a6-4c44-93bf-0c3194a42da8 req-61e79d4a-7d7b-44cf-bdd0-d8e14c6d16c9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e4e3f396-5222-478d-8a5a-333d866ba1ab", "tag": "55b7c0fb-c7dc-4b8a-bc41-1a218717dd1f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:52.723551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6bda581a-e7a6-4c44-93bf-0c3194a42da8 req-61e79d4a-7d7b-44cf-bdd0-d8e14c6d16c9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:52.738310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6bda581a-e7a6-4c44-93bf-0c3194a42da8 req-61e79d4a-7d7b-44cf-bdd0-d8e14c6d16c9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:52.738802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6bda581a-e7a6-4c44-93bf-0c3194a42da8 req-61e79d4a-7d7b-44cf-bdd0-d8e14c6d16c9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:52.830713 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6bda581a-e7a6-4c44-93bf-0c3194a42da8 req-61e79d4a-7d7b-44cf-bdd0-d8e14c6d16c9 service nova] Creating event network-changed:55b7c0fb-c7dc-4b8a-bc41-1a218717dd1f for instance e4e3f396-5222-478d-8a5a-333d866ba1ab on nch1-master-1 Sep 27 10:08:52.879100 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6bda581a-e7a6-4c44-93bf-0c3194a42da8 req-61e79d4a-7d7b-44cf-bdd0-d8e14c6d16c9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.173028 Sep 27 10:08:52.882890 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1738/3171] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:52.908862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-71d94d33-a2c8-4a4c-959e-a3e56d837ba5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:52.909875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-71d94d33-a2c8-4a4c-959e-a3e56d837ba5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:52.929332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d94d33-a2c8-4a4c-959e-a3e56d837ba5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:52.931728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d94d33-a2c8-4a4c-959e-a3e56d837ba5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:52.932207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d94d33-a2c8-4a4c-959e-a3e56d837ba5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:52.934960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d94d33-a2c8-4a4c-959e-a3e56d837ba5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:52.937360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d94d33-a2c8-4a4c-959e-a3e56d837ba5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:52.937846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d94d33-a2c8-4a4c-959e-a3e56d837ba5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:53.363536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e3864b85-70bf-420f-a57c-2b4430034f39 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:53.387709 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e3864b85-70bf-420f-a57c-2b4430034f39 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1322 microversion: 2.1 time: 1.286338 Sep 27 10:08:53.403011 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1434/3172] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:52 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1322 bytes in 1318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:53.436165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-68491db4-a7b0-45f2-bf03-c837195accca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:53.436840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-68491db4-a7b0-45f2-bf03-c837195accca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:53.460738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68491db4-a7b0-45f2-bf03-c837195accca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:53.461280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68491db4-a7b0-45f2-bf03-c837195accca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:53.461739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68491db4-a7b0-45f2-bf03-c837195accca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:53.462499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68491db4-a7b0-45f2-bf03-c837195accca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:53.466146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68491db4-a7b0-45f2-bf03-c837195accca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:53.471642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68491db4-a7b0-45f2-bf03-c837195accca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:53.943957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-71d94d33-a2c8-4a4c-959e-a3e56d837ba5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:53.970110 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-71d94d33-a2c8-4a4c-959e-a3e56d837ba5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 1.079333 Sep 27 10:08:53.970874 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1739/3173] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:52 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 1087 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:53.984426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-02e244b3-4c6d-4dd9-98fe-6c9c31231386 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:53.987660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-02e244b3-4c6d-4dd9-98fe-6c9c31231386 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:54.005868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02e244b3-4c6d-4dd9-98fe-6c9c31231386 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:54.006375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02e244b3-4c6d-4dd9-98fe-6c9c31231386 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:54.006814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02e244b3-4c6d-4dd9-98fe-6c9c31231386 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:54.010146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02e244b3-4c6d-4dd9-98fe-6c9c31231386 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:54.010713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02e244b3-4c6d-4dd9-98fe-6c9c31231386 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:54.014948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02e244b3-4c6d-4dd9-98fe-6c9c31231386 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:55.027953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-02e244b3-4c6d-4dd9-98fe-6c9c31231386 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:55.041833 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-02e244b3-4c6d-4dd9-98fe-6c9c31231386 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 1.062906 Sep 27 10:08:55.045839 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1740/3174] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:53 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 1074 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:55.058727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4f9306e3-56a8-4822-b3e3-9a49ccfdc48b tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:55.059362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4f9306e3-56a8-4822-b3e3-9a49ccfdc48b tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:55.074623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9306e3-56a8-4822-b3e3-9a49ccfdc48b tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:55.075484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9306e3-56a8-4822-b3e3-9a49ccfdc48b tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:55.076082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9306e3-56a8-4822-b3e3-9a49ccfdc48b tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:55.077320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9306e3-56a8-4822-b3e3-9a49ccfdc48b tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:55.077942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9306e3-56a8-4822-b3e3-9a49ccfdc48b tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:55.078527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f9306e3-56a8-4822-b3e3-9a49ccfdc48b tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:55.298698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-68491db4-a7b0-45f2-bf03-c837195accca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:55.319483 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-68491db4-a7b0-45f2-bf03-c837195accca tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 1.879331 Sep 27 10:08:55.325130 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1435/3175] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:53 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 1922 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:55.352104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-54d266af-0b27-4885-a089-e9c01722e837 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:55.357010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-54d266af-0b27-4885-a089-e9c01722e837 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:55.392244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54d266af-0b27-4885-a089-e9c01722e837 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:55.398017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54d266af-0b27-4885-a089-e9c01722e837 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:55.401891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54d266af-0b27-4885-a089-e9c01722e837 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:55.408351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54d266af-0b27-4885-a089-e9c01722e837 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:55.409201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54d266af-0b27-4885-a089-e9c01722e837 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:55.410207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54d266af-0b27-4885-a089-e9c01722e837 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:56.287340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-54d266af-0b27-4885-a089-e9c01722e837 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:56.296446 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-54d266af-0b27-4885-a089-e9c01722e837 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.969673 Sep 27 10:08:56.297087 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1436/3176] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:55 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 971 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:56.324877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f69ea4ca-9ea6-42c9-b148-48890c5e8f8e req-e888faea-ab86-4fe4-9939-a1ae25908fbc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd", "tag": "00a1303d-82aa-425e-b090-f9228ac6af6a"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:56.358154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f69ea4ca-9ea6-42c9-b148-48890c5e8f8e req-e888faea-ab86-4fe4-9939-a1ae25908fbc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:56.358791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f69ea4ca-9ea6-42c9-b148-48890c5e8f8e req-e888faea-ab86-4fe4-9939-a1ae25908fbc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:56.359405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f69ea4ca-9ea6-42c9-b148-48890c5e8f8e req-e888faea-ab86-4fe4-9939-a1ae25908fbc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:56.391817 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f69ea4ca-9ea6-42c9-b148-48890c5e8f8e req-e888faea-ab86-4fe4-9939-a1ae25908fbc service nova] Creating event network-changed:00a1303d-82aa-425e-b090-f9228ac6af6a for instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd on nch1-master-1 Sep 27 10:08:56.406147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4f9306e3-56a8-4822-b3e3-9a49ccfdc48b tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:56.407648 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f69ea4ca-9ea6-42c9-b148-48890c5e8f8e req-e888faea-ab86-4fe4-9939-a1ae25908fbc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.106026 Sep 27 10:08:56.408723 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1437/3177] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:56.425794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3a47245c-5f0d-4f50-bec8-99994a1a555e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:56.427360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3a47245c-5f0d-4f50-bec8-99994a1a555e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:56.450253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a47245c-5f0d-4f50-bec8-99994a1a555e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:56.454477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a47245c-5f0d-4f50-bec8-99994a1a555e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:56.455206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a47245c-5f0d-4f50-bec8-99994a1a555e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:56.456324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a47245c-5f0d-4f50-bec8-99994a1a555e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:56.456465 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4f9306e3-56a8-4822-b3e3-9a49ccfdc48b tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1322 microversion: 2.1 time: 1.409210 Sep 27 10:08:56.457248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a47245c-5f0d-4f50-bec8-99994a1a555e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:56.457978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a47245c-5f0d-4f50-bec8-99994a1a555e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:56.466398 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1741/3178] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:55 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1322 bytes in 1418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:56.530440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a54d7654-37b7-4c69-bf03-eb51e75fae26 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:56.534645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a54d7654-37b7-4c69-bf03-eb51e75fae26 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:56.566910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a54d7654-37b7-4c69-bf03-eb51e75fae26 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:56.567421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a54d7654-37b7-4c69-bf03-eb51e75fae26 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:56.567872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a54d7654-37b7-4c69-bf03-eb51e75fae26 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:56.568602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a54d7654-37b7-4c69-bf03-eb51e75fae26 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:56.583409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a54d7654-37b7-4c69-bf03-eb51e75fae26 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:56.586185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a54d7654-37b7-4c69-bf03-eb51e75fae26 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:56.894210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3a47245c-5f0d-4f50-bec8-99994a1a555e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:56.912898 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3a47245c-5f0d-4f50-bec8-99994a1a555e tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.498959 Sep 27 10:08:56.914923 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1438/3179] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:56 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:57.320314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-669a4dd6-f4f0-4575-bf9a-a801d4e6ed73 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:57.320966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-669a4dd6-f4f0-4575-bf9a-a801d4e6ed73 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:57.334054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-669a4dd6-f4f0-4575-bf9a-a801d4e6ed73 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:57.334703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-669a4dd6-f4f0-4575-bf9a-a801d4e6ed73 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:57.335199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-669a4dd6-f4f0-4575-bf9a-a801d4e6ed73 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:57.335996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-669a4dd6-f4f0-4575-bf9a-a801d4e6ed73 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:57.336460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-669a4dd6-f4f0-4575-bf9a-a801d4e6ed73 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:57.336914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-669a4dd6-f4f0-4575-bf9a-a801d4e6ed73 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:57.609324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a54d7654-37b7-4c69-bf03-eb51e75fae26 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:57.636272 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a54d7654-37b7-4c69-bf03-eb51e75fae26 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 1.145814 Sep 27 10:08:57.636966 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1742/3180] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:56 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 1170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:57.674872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-21091a31-b1a7-4ef7-8692-ab4c58b7bcc3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:57.681437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-21091a31-b1a7-4ef7-8692-ab4c58b7bcc3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:57.700503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21091a31-b1a7-4ef7-8692-ab4c58b7bcc3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:57.704052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21091a31-b1a7-4ef7-8692-ab4c58b7bcc3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:57.705085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21091a31-b1a7-4ef7-8692-ab4c58b7bcc3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:57.707509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21091a31-b1a7-4ef7-8692-ab4c58b7bcc3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:57.712324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21091a31-b1a7-4ef7-8692-ab4c58b7bcc3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:57.715165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21091a31-b1a7-4ef7-8692-ab4c58b7bcc3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:57.792189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-669a4dd6-f4f0-4575-bf9a-a801d4e6ed73 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:57.802410 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-669a4dd6-f4f0-4575-bf9a-a801d4e6ed73 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.486675 Sep 27 10:08:57.824853 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1439/3181] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:57 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 510 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:58.170748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21091a31-b1a7-4ef7-8692-ab4c58b7bcc3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:58.185524 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-21091a31-b1a7-4ef7-8692-ab4c58b7bcc3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1322 microversion: 2.1 time: 0.546752 Sep 27 10:08:58.189681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1743/3182] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:57 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1322 bytes in 552 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:58.195362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-229deeae-e82c-44d3-b725-92e9ba158caa tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:58.206116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-229deeae-e82c-44d3-b725-92e9ba158caa tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:58.225941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-229deeae-e82c-44d3-b725-92e9ba158caa tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:58.226719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-229deeae-e82c-44d3-b725-92e9ba158caa tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:58.227392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-229deeae-e82c-44d3-b725-92e9ba158caa tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:58.229120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-229deeae-e82c-44d3-b725-92e9ba158caa tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:58.229985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-229deeae-e82c-44d3-b725-92e9ba158caa tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:58.230673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-229deeae-e82c-44d3-b725-92e9ba158caa tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:58.246610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a6768eb5-b4e8-47ff-bf06-d7fdc1ec84ae tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] Action: 'create', calling method: .version_select at 0x7f7f6366d670>, body: {"server_group": {"name": "server-group", "policy": "affinity"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:58.326786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a6768eb5-b4e8-47ff-bf06-d7fdc1ec84ae tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] Getting quotas for project 2b032e83813a429bb445b5f7ee854a4f. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:08:58.352188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a6768eb5-b4e8-47ff-bf06-d7fdc1ec84ae tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] Getting quotas for user bf823d90bd69466691ba2094929a8239 and project 2b032e83813a429bb445b5f7ee854a4f. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:08:58.559838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a6768eb5-b4e8-47ff-bf06-d7fdc1ec84ae tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] Getting quotas for project 2b032e83813a429bb445b5f7ee854a4f. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:08:58.587940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a6768eb5-b4e8-47ff-bf06-d7fdc1ec84ae tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] Getting quotas for user bf823d90bd69466691ba2094929a8239 and project 2b032e83813a429bb445b5f7ee854a4f. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:08:58.631152 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a6768eb5-b4e8-47ff-bf06-d7fdc1ec84ae tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] 10.222.0.26 "POST /compute/v2.1/os-server-groups" status: 200 len: 235 microversion: 2.64 time: 0.697960 Sep 27 10:08:58.632882 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1440/3183] 10.222.0.26 () {62 vars in 1214 bytes} [Tue Sep 27 10:08:57 2022] POST /compute/v2.1/os-server-groups => generated 235 bytes in 700 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 10:08:58.639032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b9ef5e3d-6789-4bc5-8719-07a2eb6f0541 req-f5fe7b61-c0ca-4529-a504-a8c07417b4aa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4fed875e-5be1-4b37-a0f9-55e6bc6fca05", "tag": "c0a6ca20-6141-4b76-b38e-d3a32bd4803c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:58.650741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b9ef5e3d-6789-4bc5-8719-07a2eb6f0541 req-f5fe7b61-c0ca-4529-a504-a8c07417b4aa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:58.651481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b9ef5e3d-6789-4bc5-8719-07a2eb6f0541 req-f5fe7b61-c0ca-4529-a504-a8c07417b4aa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:58.652104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b9ef5e3d-6789-4bc5-8719-07a2eb6f0541 req-f5fe7b61-c0ca-4529-a504-a8c07417b4aa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:58.681622 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b9ef5e3d-6789-4bc5-8719-07a2eb6f0541 req-f5fe7b61-c0ca-4529-a504-a8c07417b4aa service nova] Creating event network-changed:c0a6ca20-6141-4b76-b38e-d3a32bd4803c for instance 4fed875e-5be1-4b37-a0f9-55e6bc6fca05 on nch2-master-1 Sep 27 10:08:58.683771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-229deeae-e82c-44d3-b725-92e9ba158caa tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:58.694995 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b9ef5e3d-6789-4bc5-8719-07a2eb6f0541 req-f5fe7b61-c0ca-4529-a504-a8c07417b4aa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057744 Sep 27 10:08:58.698537 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1441/3184] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:08:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:08:58.700629 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-229deeae-e82c-44d3-b725-92e9ba158caa tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.509374 Sep 27 10:08:58.702627 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1744/3185] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:58 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:58.709514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2cde498c-4afd-49dd-b9a0-fbcf7d49b1e6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:58.710842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2cde498c-4afd-49dd-b9a0-fbcf7d49b1e6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:58.718540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b3a347a-81a0-4efb-b86d-3efc2899b1cf tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:58.726045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cde498c-4afd-49dd-b9a0-fbcf7d49b1e6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:58.727172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cde498c-4afd-49dd-b9a0-fbcf7d49b1e6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:58.727903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cde498c-4afd-49dd-b9a0-fbcf7d49b1e6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:58.729095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cde498c-4afd-49dd-b9a0-fbcf7d49b1e6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:58.729780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cde498c-4afd-49dd-b9a0-fbcf7d49b1e6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:58.730820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cde498c-4afd-49dd-b9a0-fbcf7d49b1e6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:58.744554 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b3a347a-81a0-4efb-b86d-3efc2899b1cf tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] 10.222.0.26 "GET /compute/v2.1/os-server-groups" status: 200 len: 238 microversion: 2.64 time: 0.044166 Sep 27 10:08:58.746030 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1442/3186] 10.222.0.26 () {60 vars in 1193 bytes} [Tue Sep 27 10:08:58 2022] GET /compute/v2.1/os-server-groups => generated 238 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:08:58.764075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2efd3418-f477-4448-a6cd-293efbe143a5 tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:58.784314 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2efd3418-f477-4448-a6cd-293efbe143a5 tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] 10.222.0.26 "GET /compute/v2.1/os-server-groups/17d0b013-0b29-42df-88ab-a9c8dc610dc8" status: 200 len: 235 microversion: 2.64 time: 0.029047 Sep 27 10:08:58.785103 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1443/3187] 10.222.0.26 () {60 vars in 1304 bytes} [Tue Sep 27 10:08:58 2022] GET /compute/v2.1/os-server-groups/17d0b013-0b29-42df-88ab-a9c8dc610dc8 => generated 235 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:08:58.809984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-034c9363-d573-42e3-92b7-a52229609a49 tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:58.899627 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-034c9363-d573-42e3-92b7-a52229609a49 tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/17d0b013-0b29-42df-88ab-a9c8dc610dc8" status: 204 len: 0 microversion: 2.64 time: 0.106840 Sep 27 10:08:58.901074 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1444/3188] 10.222.0.26 () {60 vars in 1307 bytes} [Tue Sep 27 10:08:58 2022] DELETE /compute/v2.1/os-server-groups/17d0b013-0b29-42df-88ab-a9c8dc610dc8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Sep 27 10:08:58.907126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec2f0ead-7bf3-4e2e-a4cf-3638b5c59a0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:58.909186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ec2f0ead-7bf3-4e2e-a4cf-3638b5c59a0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:58.917538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2f0ead-7bf3-4e2e-a4cf-3638b5c59a0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:58.918027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2f0ead-7bf3-4e2e-a4cf-3638b5c59a0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:58.918786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2f0ead-7bf3-4e2e-a4cf-3638b5c59a0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:58.919674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2f0ead-7bf3-4e2e-a4cf-3638b5c59a0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:58.920446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2f0ead-7bf3-4e2e-a4cf-3638b5c59a0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:58.921310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec2f0ead-7bf3-4e2e-a4cf-3638b5c59a0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:59.140379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ec2f0ead-7bf3-4e2e-a4cf-3638b5c59a0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:59.159879 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ec2f0ead-7bf3-4e2e-a4cf-3638b5c59a0b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.249686 Sep 27 10:08:59.176496 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1445/3189] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:58 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:59.240577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5bc3489c-03a3-449f-aeab-251090936b2b tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:59.271160 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-5bc3489c-03a3-449f-aeab-251090936b2b tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] HTTP exception thrown: Instance group 17d0b013-0b29-42df-88ab-a9c8dc610dc8 could not be found. Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5bc3489c-03a3-449f-aeab-251090936b2b tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] Request method failure captured: Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/os-server-groups/17d0b013-0b29-42df-88ab-a9c8dc610dc8 HTTP/1.1 Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsu5pFaxEJ0iV_uu8jwP5jhF74TmweED_N7aahEUDtrg4MTpgcuCbWMYEMW1-rR77o8qFswXZ9XmVz8j9kP82YZpuUEXT9ucZrGSYfAvJ2EfDgY7aoh_lMmCuSb6aI45_ya6zl7sP9WzjuBzG0Zz9SVuXVQBJUeDeH7v_XojNA2UtRw Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.64 Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2b032e83813a429bb445b5f7ee854a4f Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerGroup264TestJSON-460584503 Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b032e83813a429bb445b5f7ee854a4f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2b032e83813a429bb445b5f7ee854a4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2b032e83813a429bb445b5f7ee854a4f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2b032e83813a429bb445b5f7ee854a4f"}]}] Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerGroup264TestJSON-460584503 Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2b032e83813a429bb445b5f7ee854a4f Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerGroup264TestJSON-460584503 Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerGroup264TestJSON-460584503-project Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: bf823d90bd69466691ba2094929a8239 Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerGroup264TestJSON-460584503-project Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance group 17d0b013-0b29-42df-88ab-a9c8dc610dc8 could not be found. Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '17d0b013-0b29-42df-88ab-a9c8dc610dc8'} Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group 17d0b013-0b29-42df-88ab-a9c8dc610dc8 could not be found. Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group 17d0b013-0b29-42df-88ab-a9c8dc610dc8 could not be found. Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:59.295327 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:08:59.300903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:08:59.300903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:08:59.300903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:08:59.300903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:08:59.300903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group 17d0b013-0b29-42df-88ab-a9c8dc610dc8 could not be found. Sep 27 10:08:59.300903 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:08:59.300903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5bc3489c-03a3-449f-aeab-251090936b2b tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] Returning 404 to user: Instance group 17d0b013-0b29-42df-88ab-a9c8dc610dc8 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:08:59.308577 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5bc3489c-03a3-449f-aeab-251090936b2b tempest-ServerGroup264TestJSON-460584503 tempest-ServerGroup264TestJSON-460584503-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/17d0b013-0b29-42df-88ab-a9c8dc610dc8" status: 404 len: 117 microversion: 2.64 time: 0.107033 Sep 27 10:08:59.313383 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1446/3190] 10.222.0.26 () {60 vars in 1307 bytes} [Tue Sep 27 10:08:59 2022] DELETE /compute/v2.1/os-server-groups/17d0b013-0b29-42df-88ab-a9c8dc610dc8 => generated 117 bytes in 136 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:08:59.321772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-144a5170-3480-42ed-bece-c56eb7fc9cde tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:08:59.331957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2cde498c-4afd-49dd-b9a0-fbcf7d49b1e6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:08:59.334920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-144a5170-3480-42ed-bece-c56eb7fc9cde tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:08:59.350304 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2cde498c-4afd-49dd-b9a0-fbcf7d49b1e6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 0.645063 Sep 27 10:08:59.353507 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1745/3191] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:58 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:08:59.376270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-144a5170-3480-42ed-bece-c56eb7fc9cde tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:59.377616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-144a5170-3480-42ed-bece-c56eb7fc9cde tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:59.379255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-144a5170-3480-42ed-bece-c56eb7fc9cde tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:59.384441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-144a5170-3480-42ed-bece-c56eb7fc9cde tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:08:59.385969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-144a5170-3480-42ed-bece-c56eb7fc9cde tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:08:59.386774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-144a5170-3480-42ed-bece-c56eb7fc9cde tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:08:59.612624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Action: 'create', calling method: >, body: {"server": {"name": "guest-instance-1.domain.com", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "accessIPv4": "2.2.2.2", "networks": [{"uuid": "2c5d842b-969b-48d9-a0c8-0ecb3d5784a3"}], "security_groups": [{"name": "tempest-securitygroup--1110063867"}], "key_name": "tempest-keypair-667250342", "user_data": "}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:08:59.954944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] validate_networks() for [('2c5d842b-969b-48d9-a0c8-0ecb3d5784a3', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:09:00.600568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-144a5170-3480-42ed-bece-c56eb7fc9cde tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:00.614442 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-144a5170-3480-42ed-bece-c56eb7fc9cde tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1322 microversion: 2.1 time: 1.299766 Sep 27 10:09:00.616454 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1447/3192] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:08:59 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1322 bytes in 1302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:00.642529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ab78aed6-2fb4-4255-ac50-f73eae3cac50 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:00.643159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ab78aed6-2fb4-4255-ac50-f73eae3cac50 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:00.681778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab78aed6-2fb4-4255-ac50-f73eae3cac50 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:00.682349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab78aed6-2fb4-4255-ac50-f73eae3cac50 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:00.682758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab78aed6-2fb4-4255-ac50-f73eae3cac50 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:00.683512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab78aed6-2fb4-4255-ac50-f73eae3cac50 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:00.683907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab78aed6-2fb4-4255-ac50-f73eae3cac50 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:00.684329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab78aed6-2fb4-4255-ac50-f73eae3cac50 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:00.996244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:09:00.997173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:09:00.997279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:09:00.997609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:09:00.998091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:09:01.012137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:01.012634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:01.013099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:01.018546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:01.022546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:01.024518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:01.074069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Getting quotas for project 327b414889a04e189a7f940685a4ff39. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:01.080290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Getting quotas for user f06d16c90a5d464696f8137612c35327 and project 327b414889a04e189a7f940685a4ff39. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:01.112784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:09:01.115554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:09:01.250072 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3a8bc3c1-3118-47da-9108-294595e20cd3 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.646541 Sep 27 10:09:01.252173 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1746/3193] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:08:59 2022] POST /compute/v2.1/servers => generated 409 bytes in 1648 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:09:01.261083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-08c1139a-da82-4209-a4d9-2e15122b715f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:01.264319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-08c1139a-da82-4209-a4d9-2e15122b715f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:01.281978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c1139a-da82-4209-a4d9-2e15122b715f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:01.287659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c1139a-da82-4209-a4d9-2e15122b715f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:01.291393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c1139a-da82-4209-a4d9-2e15122b715f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:01.293306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c1139a-da82-4209-a4d9-2e15122b715f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:01.295088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c1139a-da82-4209-a4d9-2e15122b715f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:01.299792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08c1139a-da82-4209-a4d9-2e15122b715f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:01.721227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-08c1139a-da82-4209-a4d9-2e15122b715f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:01.732973 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-08c1139a-da82-4209-a4d9-2e15122b715f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.478802 Sep 27 10:09:01.749446 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1747/3194] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:01 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:01.764758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d2386b7d-a8b8-43cd-acf9-36a7f8c23a1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:01.765395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d2386b7d-a8b8-43cd-acf9-36a7f8c23a1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:01.781913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2386b7d-a8b8-43cd-acf9-36a7f8c23a1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:01.782428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2386b7d-a8b8-43cd-acf9-36a7f8c23a1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:01.782872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2386b7d-a8b8-43cd-acf9-36a7f8c23a1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:01.790950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2386b7d-a8b8-43cd-acf9-36a7f8c23a1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:01.791555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2386b7d-a8b8-43cd-acf9-36a7f8c23a1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:01.793475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2386b7d-a8b8-43cd-acf9-36a7f8c23a1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:02.282513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d2386b7d-a8b8-43cd-acf9-36a7f8c23a1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:02.303838 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d2386b7d-a8b8-43cd-acf9-36a7f8c23a1a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 0.548533 Sep 27 10:09:02.306713 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1748/3195] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:01 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:02.332351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-09a9fdf8-c860-4600-b7fc-5530c6628fda tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:02.333063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-09a9fdf8-c860-4600-b7fc-5530c6628fda tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:02.346811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:02.596907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-09a9fdf8-c860-4600-b7fc-5530c6628fda tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:02.608923 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-09a9fdf8-c860-4600-b7fc-5530c6628fda tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1225 microversion: 2.1 time: 0.297514 Sep 27 10:09:02.622317 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1749/3196] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:02 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1225 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:02.636830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1f718079-c70c-433e-92a8-d63345dec3b8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:02.637867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1f718079-c70c-433e-92a8-d63345dec3b8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:02.648366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f718079-c70c-433e-92a8-d63345dec3b8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:02.653552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f718079-c70c-433e-92a8-d63345dec3b8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:02.654886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f718079-c70c-433e-92a8-d63345dec3b8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:02.660486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f718079-c70c-433e-92a8-d63345dec3b8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:02.661134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f718079-c70c-433e-92a8-d63345dec3b8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:02.664604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f718079-c70c-433e-92a8-d63345dec3b8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:03.020609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1f718079-c70c-433e-92a8-d63345dec3b8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:03.064851 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1f718079-c70c-433e-92a8-d63345dec3b8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1322 microversion: 2.1 time: 0.438210 Sep 27 10:09:03.067899 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1750/3197] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:02 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1322 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:03.180914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ab78aed6-2fb4-4255-ac50-f73eae3cac50 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:03.193088 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ab78aed6-2fb4-4255-ac50-f73eae3cac50 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 2.575064 Sep 27 10:09:03.193691 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1448/3198] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:00 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 2576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:03.216256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cb6e85bd-61fd-4780-a5e1-1816e908a938 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:03.229421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cb6e85bd-61fd-4780-a5e1-1816e908a938 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:03.249198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=1,nova_api:INSERT=3,nova_api:DELETE=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:03.254847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb6e85bd-61fd-4780-a5e1-1816e908a938 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:03.260733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb6e85bd-61fd-4780-a5e1-1816e908a938 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:03.261193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb6e85bd-61fd-4780-a5e1-1816e908a938 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:03.264543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb6e85bd-61fd-4780-a5e1-1816e908a938 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:03.267243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb6e85bd-61fd-4780-a5e1-1816e908a938 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:03.267696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb6e85bd-61fd-4780-a5e1-1816e908a938 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:03.339156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:03.515756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ebb6191a-d00c-4f38-8334-cbf2d1c05cdc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:03.520194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebb6191a-d00c-4f38-8334-cbf2d1c05cdc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:03.526424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebb6191a-d00c-4f38-8334-cbf2d1c05cdc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:03.527026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebb6191a-d00c-4f38-8334-cbf2d1c05cdc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:03.527853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebb6191a-d00c-4f38-8334-cbf2d1c05cdc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:03.528599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebb6191a-d00c-4f38-8334-cbf2d1c05cdc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:03.529251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebb6191a-d00c-4f38-8334-cbf2d1c05cdc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:03.575968 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ebb6191a-d00c-4f38-8334-cbf2d1c05cdc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.506502 Sep 27 10:09:03.577238 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1751/3199] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:09:03 2022] GET /compute/v2.1/os-services => generated 947 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:03.684557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-96dd5712-048d-46a7-b46e-4673a60e5dd7 tempest-ListImagesTestJSON-2128999237 tempest-ListImagesTestJSON-2128999237-project] Calling method '.version_select at 0x7f7f6308b040>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:03.771636 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-96dd5712-048d-46a7-b46e-4673a60e5dd7 tempest-ListImagesTestJSON-2128999237 tempest-ListImagesTestJSON-2128999237-project] 10.222.0.26 "GET /compute/v2.1/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6" status: 200 len: 784 microversion: 2.1 time: 0.193012 Sep 27 10:09:03.772598 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1752/3200] 10.222.0.26 () {58 vars in 1233 bytes} [Tue Sep 27 10:09:03 2022] GET /compute/v2.1/images/c1623d3a-c1bb-4649-9910-5deadb8a1ff6 => generated 784 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:03.779698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a2e71f73-199b-44c5-8fab-098a5443bd3b req-196e21b7-18ec-42c3-bf80-2a0035cd7ca5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4e3f396-5222-478d-8a5a-333d866ba1ab", "name": "network-vif-plugged", "status": "completed", "tag": "55b7c0fb-c7dc-4b8a-bc41-1a218717dd1f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:03.793067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a2e71f73-199b-44c5-8fab-098a5443bd3b req-196e21b7-18ec-42c3-bf80-2a0035cd7ca5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:03.794149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a2e71f73-199b-44c5-8fab-098a5443bd3b req-196e21b7-18ec-42c3-bf80-2a0035cd7ca5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:03.794917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a2e71f73-199b-44c5-8fab-098a5443bd3b req-196e21b7-18ec-42c3-bf80-2a0035cd7ca5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:03.811261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cb6e85bd-61fd-4780-a5e1-1816e908a938 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:03.825298 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cb6e85bd-61fd-4780-a5e1-1816e908a938 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.615099 Sep 27 10:09:03.827774 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a2e71f73-199b-44c5-8fab-098a5443bd3b req-196e21b7-18ec-42c3-bf80-2a0035cd7ca5 service nova] Creating event network-vif-plugged:55b7c0fb-c7dc-4b8a-bc41-1a218717dd1f for instance e4e3f396-5222-478d-8a5a-333d866ba1ab on nch1-master-1 Sep 27 10:09:03.829066 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1449/3201] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:03 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:03.869674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c2d22e62-0997-4836-bac5-68fa8b259afe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:03.870975 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a2e71f73-199b-44c5-8fab-098a5443bd3b req-196e21b7-18ec-42c3-bf80-2a0035cd7ca5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.090015 Sep 27 10:09:03.874068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c2d22e62-0997-4836-bac5-68fa8b259afe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:03.876707 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1753/3202] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:03.885399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92e68393-656a-4f0f-a950-00349e5e1fbc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:03.905716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2d22e62-0997-4836-bac5-68fa8b259afe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:03.908535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2d22e62-0997-4836-bac5-68fa8b259afe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:03.912020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2d22e62-0997-4836-bac5-68fa8b259afe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:03.921771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2d22e62-0997-4836-bac5-68fa8b259afe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:03.922861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92e68393-656a-4f0f-a950-00349e5e1fbc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:03.923714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2d22e62-0997-4836-bac5-68fa8b259afe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:03.925182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2d22e62-0997-4836-bac5-68fa8b259afe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:03.925901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92e68393-656a-4f0f-a950-00349e5e1fbc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:03.929326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92e68393-656a-4f0f-a950-00349e5e1fbc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:03.932571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92e68393-656a-4f0f-a950-00349e5e1fbc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:03.935268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92e68393-656a-4f0f-a950-00349e5e1fbc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:03.935959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92e68393-656a-4f0f-a950-00349e5e1fbc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.018869 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-92e68393-656a-4f0f-a950-00349e5e1fbc tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.140674 Sep 27 10:09:04.021122 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1754/3203] 10.222.0.26 () {58 vars in 1170 bytes} [Tue Sep 27 10:09:03 2022] GET /compute/v2.1/os-services?xxx=nova-compute => generated 947 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:04.028318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b6c18a14-c895-41af-b779-f3e6cd4e5a2c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:04.033840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b6c18a14-c895-41af-b779-f3e6cd4e5a2c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:04.049396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6c18a14-c895-41af-b779-f3e6cd4e5a2c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.050324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6c18a14-c895-41af-b779-f3e6cd4e5a2c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.051023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6c18a14-c895-41af-b779-f3e6cd4e5a2c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.051923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6c18a14-c895-41af-b779-f3e6cd4e5a2c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.052619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6c18a14-c895-41af-b779-f3e6cd4e5a2c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.053253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6c18a14-c895-41af-b779-f3e6cd4e5a2c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.135860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b6c18a14-c895-41af-b779-f3e6cd4e5a2c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:04.149264 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b6c18a14-c895-41af-b779-f3e6cd4e5a2c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1295 microversion: 2.1 time: 0.124629 Sep 27 10:09:04.150810 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1755/3204] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:04 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1295 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:04.157402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-47996990-b2bf-49d8-b758-4df65b9746b8 tempest-ListImagesTestJSON-2128999237 tempest-ListImagesTestJSON-2128999237-project] Calling method '.version_select at 0x7f7f630f98b0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:04.259278 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-47996990-b2bf-49d8-b758-4df65b9746b8 tempest-ListImagesTestJSON-2128999237 tempest-ListImagesTestJSON-2128999237-project] 10.222.0.26 "GET /compute/v2.1/images" status: 200 len: 445 microversion: 2.1 time: 0.106981 Sep 27 10:09:04.261078 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1756/3205] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:04 2022] GET /compute/v2.1/images => generated 445 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:04.283693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4e67ff9f-f6ae-4794-bd2a-0a435f9cd2f4 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:04.288073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e67ff9f-f6ae-4794-bd2a-0a435f9cd2f4 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.298893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e67ff9f-f6ae-4794-bd2a-0a435f9cd2f4 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.299478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e67ff9f-f6ae-4794-bd2a-0a435f9cd2f4 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.302207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e67ff9f-f6ae-4794-bd2a-0a435f9cd2f4 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.302705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e67ff9f-f6ae-4794-bd2a-0a435f9cd2f4 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.303149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e67ff9f-f6ae-4794-bd2a-0a435f9cd2f4 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.344059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:04.375993 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4e67ff9f-f6ae-4794-bd2a-0a435f9cd2f4 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.113908 Sep 27 10:09:04.388713 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1757/3206] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:09:04 2022] GET /compute/v2.1/os-services => generated 947 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:04.409282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1edd8366-ffa6-46ea-9200-80295cd9b157 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:04.409915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1edd8366-ffa6-46ea-9200-80295cd9b157 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:04.426359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1edd8366-ffa6-46ea-9200-80295cd9b157 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.426757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1edd8366-ffa6-46ea-9200-80295cd9b157 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.427299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1edd8366-ffa6-46ea-9200-80295cd9b157 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.428039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1edd8366-ffa6-46ea-9200-80295cd9b157 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.428493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1edd8366-ffa6-46ea-9200-80295cd9b157 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.429138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1edd8366-ffa6-46ea-9200-80295cd9b157 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.541362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c2d22e62-0997-4836-bac5-68fa8b259afe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:04.549985 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c2d22e62-0997-4836-bac5-68fa8b259afe tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 0.717803 Sep 27 10:09:04.563215 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1450/3207] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:03 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 733 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:04.573189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8d0b1e19-8d37-44f6-a8ad-689f1bcdf8dc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:04.573189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8d0b1e19-8d37-44f6-a8ad-689f1bcdf8dc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:04.589848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d0b1e19-8d37-44f6-a8ad-689f1bcdf8dc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.590154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d0b1e19-8d37-44f6-a8ad-689f1bcdf8dc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.590647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d0b1e19-8d37-44f6-a8ad-689f1bcdf8dc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.591401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d0b1e19-8d37-44f6-a8ad-689f1bcdf8dc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.591904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d0b1e19-8d37-44f6-a8ad-689f1bcdf8dc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.592345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d0b1e19-8d37-44f6-a8ad-689f1bcdf8dc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.649568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1edd8366-ffa6-46ea-9200-80295cd9b157 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:04.659683 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1edd8366-ffa6-46ea-9200-80295cd9b157 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1322 microversion: 2.1 time: 0.269235 Sep 27 10:09:04.674771 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1758/3208] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:04 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1322 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:04.683094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eece5fa3-ef90-4ac8-921e-2b5ff3842b09 tempest-ListImagesTestJSON-2128999237 tempest-ListImagesTestJSON-2128999237-project] Calling method '.version_select at 0x7f7f630f9c10>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:04.738828 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eece5fa3-ef90-4ac8-921e-2b5ff3842b09 tempest-ListImagesTestJSON-2128999237 tempest-ListImagesTestJSON-2128999237-project] 10.222.0.26 "GET /compute/v2.1/images/detail" status: 200 len: 787 microversion: 2.1 time: 0.062817 Sep 27 10:09:04.748321 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1759/3209] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 10:09:04 2022] GET /compute/v2.1/images/detail => generated 787 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:04.753977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2660485a-2e86-4f0c-b1fa-dbf6db39546c tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:04.761573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2660485a-2e86-4f0c-b1fa-dbf6db39546c tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.762130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2660485a-2e86-4f0c-b1fa-dbf6db39546c tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.762524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2660485a-2e86-4f0c-b1fa-dbf6db39546c tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.763280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2660485a-2e86-4f0c-b1fa-dbf6db39546c tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.763744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2660485a-2e86-4f0c-b1fa-dbf6db39546c tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.764329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2660485a-2e86-4f0c-b1fa-dbf6db39546c tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.796432 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2660485a-2e86-4f0c-b1fa-dbf6db39546c tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx" status: 200 len: 16 microversion: 2.1 time: 0.046518 Sep 27 10:09:04.797232 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1760/3210] 10.222.0.26 () {58 vars in 1196 bytes} [Tue Sep 27 10:09:04 2022] GET /compute/v2.1/os-services?host=ncdv-master-1&binary=xxx => generated 16 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:09:04.817950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c732a193-e406-4352-98c4-626311fe1879 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:04.822403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c732a193-e406-4352-98c4-626311fe1879 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.822902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c732a193-e406-4352-98c4-626311fe1879 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.823343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c732a193-e406-4352-98c4-626311fe1879 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.824285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c732a193-e406-4352-98c4-626311fe1879 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.824929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c732a193-e406-4352-98c4-626311fe1879 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.825542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c732a193-e406-4352-98c4-626311fe1879 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.857323 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c732a193-e406-4352-98c4-626311fe1879 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 947 microversion: 2.1 time: 0.051280 Sep 27 10:09:04.858674 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1761/3211] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:09:04 2022] GET /compute/v2.1/os-services => generated 947 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:04.866249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da9b0af4-9c00-4fdd-84a3-3a56f8e4f9b4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:04.868214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-da9b0af4-9c00-4fdd-84a3-3a56f8e4f9b4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:04.876530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9b0af4-9c00-4fdd-84a3-3a56f8e4f9b4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.877420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9b0af4-9c00-4fdd-84a3-3a56f8e4f9b4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.878320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9b0af4-9c00-4fdd-84a3-3a56f8e4f9b4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.879388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9b0af4-9c00-4fdd-84a3-3a56f8e4f9b4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:04.880263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9b0af4-9c00-4fdd-84a3-3a56f8e4f9b4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:04.881067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9b0af4-9c00-4fdd-84a3-3a56f8e4f9b4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:04.970264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8d0b1e19-8d37-44f6-a8ad-689f1bcdf8dc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:04.986862 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8d0b1e19-8d37-44f6-a8ad-689f1bcdf8dc tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.422506 Sep 27 10:09:04.988651 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1451/3212] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:04 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:04.999564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8d21476b-716c-458e-a6c1-4cd1f57373a3 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:05.006481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d21476b-716c-458e-a6c1-4cd1f57373a3 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:05.011057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d21476b-716c-458e-a6c1-4cd1f57373a3 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:05.014221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d21476b-716c-458e-a6c1-4cd1f57373a3 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:05.015287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d21476b-716c-458e-a6c1-4cd1f57373a3 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:05.016644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d21476b-716c-458e-a6c1-4cd1f57373a3 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:05.018604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d21476b-716c-458e-a6c1-4cd1f57373a3 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:05.096408 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8d21476b-716c-458e-a6c1-4cd1f57373a3 tempest-ServicesAdminNegativeTestJSON-713223644 tempest-ServicesAdminNegativeTestJSON-713223644-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler" status: 200 len: 16 microversion: 2.1 time: 0.104146 Sep 27 10:09:05.106445 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1452/3213] 10.222.0.26 () {58 vars in 1198 bytes} [Tue Sep 27 10:09:04 2022] GET /compute/v2.1/os-services?host=xxx&binary=nova-scheduler => generated 16 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:09:05.138541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-da9b0af4-9c00-4fdd-84a3-3a56f8e4f9b4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:05.154766 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da9b0af4-9c00-4fdd-84a3-3a56f8e4f9b4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.294604 Sep 27 10:09:05.156087 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1762/3214] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:04 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:05.188154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-778a7f2f-b75f-4c47-8813-e61eb45fce99 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:05.190095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-778a7f2f-b75f-4c47-8813-e61eb45fce99 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:05.210228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778a7f2f-b75f-4c47-8813-e61eb45fce99 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:05.215585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778a7f2f-b75f-4c47-8813-e61eb45fce99 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:05.223585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778a7f2f-b75f-4c47-8813-e61eb45fce99 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:05.224819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778a7f2f-b75f-4c47-8813-e61eb45fce99 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:05.225437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778a7f2f-b75f-4c47-8813-e61eb45fce99 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:05.226233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-778a7f2f-b75f-4c47-8813-e61eb45fce99 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:05.410993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-778a7f2f-b75f-4c47-8813-e61eb45fce99 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:05.424854 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-778a7f2f-b75f-4c47-8813-e61eb45fce99 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1283 microversion: 2.1 time: 0.245459 Sep 27 10:09:05.431352 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1763/3215] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:05 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1283 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:05.516516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce41f6fa-eceb-42ac-b77b-58c458a7217e tempest-ServicesAdminNegativeTestJSON-2087440082 tempest-ServicesAdminNegativeTestJSON-2087440082-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:05.520090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ce41f6fa-eceb-42ac-b77b-58c458a7217e tempest-ServicesAdminNegativeTestJSON-2087440082 tempest-ServicesAdminNegativeTestJSON-2087440082-project] Policy check for os_compute_api:os-services:list failed with credentials {'is_admin': False, 'user_id': 'a8323bb0ff4542a5b523dceaaf7606a4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '56687e6bb37f48e4a6152179d840d29b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce41f6fa-eceb-42ac-b77b-58c458a7217e tempest-ServicesAdminNegativeTestJSON-2087440082 tempest-ServicesAdminNegativeTestJSON-2087440082-project] Request method failure captured: Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-services HTTP/1.1 Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsu8j5xSZvE5E107xBEniwg1DygEBsEGRj5KJv10wQPTLqqpGUEoZ3NxiHXdJrcFYF26NypviFVk69T3Q8azPIV3utVrSxEoWq4LpiBxrm_ukNiM4AYwWtPXbReyvaYLrgi_JqHjevxSjurnq_O0M5gdUV2ks3YZRdVeFSwOKVbXZ60 Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 56687e6bb37f48e4a6152179d840d29b Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServicesAdminNegativeTestJSON-2087440082 Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56687e6bb37f48e4a6152179d840d29b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/56687e6bb37f48e4a6152179d840d29b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/56687e6bb37f48e4a6152179d840d29b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_56687e6bb37f48e4a6152179d840d29b"}]}] Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServicesAdminNegativeTestJSON-2087440082 Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 56687e6bb37f48e4a6152179d840d29b Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServicesAdminNegativeTestJSON-2087440082 Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServicesAdminNegativeTestJSON-2087440082-project Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a8323bb0ff4542a5b523dceaaf7606a4 Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServicesAdminNegativeTestJSON-2087440082-project Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: exception: Policy doesn't allow os_compute_api:os-services:list to be performed. Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: action_args: {} Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/services.py", line 384, in index Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi context.can(services_policies.BASE_POLICY_NAME % 'list', target={}) Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/context.py", line 240, in can Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return policy.authorize(self, action, target) Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 199, in authorize Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi LOG.debug('Policy check for %(action)s failed with credentials ' Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise self.value Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/policy.py", line 186, in authorize Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = _ENFORCER.authorize(action, target, context, Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1184, in authorize Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.enforce( Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc(*args, **kwargs) Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.PolicyNotAuthorized: Policy doesn't allow os_compute_api:os-services:list to be performed. Sep 27 10:09:05.524166 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:05.529071 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:05.529071 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:05.529071 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:05.529071 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:05.529071 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:05.529071 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 354, in __exit__ Sep 27 10:09:05.529071 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(webob.exc.HTTPForbidden( Sep 27 10:09:05.529071 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Policy doesn't allow os_compute_api:os-services:list to be performed. Sep 27 10:09:05.529071 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:05.535935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce41f6fa-eceb-42ac-b77b-58c458a7217e tempest-ServicesAdminNegativeTestJSON-2087440082 tempest-ServicesAdminNegativeTestJSON-2087440082-project] Returning 403 to user: Policy doesn't allow os_compute_api:os-services:list to be performed. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:05.538216 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ce41f6fa-eceb-42ac-b77b-58c458a7217e tempest-ServicesAdminNegativeTestJSON-2087440082 tempest-ServicesAdminNegativeTestJSON-2087440082-project] 10.222.0.26 "GET /compute/v2.1/os-services" status: 403 len: 112 microversion: 2.1 time: 0.430510 Sep 27 10:09:05.546314 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1453/3216] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:09:05 2022] GET /compute/v2.1/os-services => generated 112 bytes in 439 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:09:05.592388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3fa23fef-71dc-4895-8213-40f718d2025e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:05.592962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3fa23fef-71dc-4895-8213-40f718d2025e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:05.618902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fa23fef-71dc-4895-8213-40f718d2025e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:05.620202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fa23fef-71dc-4895-8213-40f718d2025e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:05.620675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fa23fef-71dc-4895-8213-40f718d2025e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:05.625284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fa23fef-71dc-4895-8213-40f718d2025e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:05.625886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fa23fef-71dc-4895-8213-40f718d2025e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:05.626688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fa23fef-71dc-4895-8213-40f718d2025e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:05.687214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7aec3dc0-ef4e-439c-800a-87845dc9a8be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:05.688127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7aec3dc0-ef4e-439c-800a-87845dc9a8be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:05.714905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aec3dc0-ef4e-439c-800a-87845dc9a8be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:05.717019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aec3dc0-ef4e-439c-800a-87845dc9a8be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:05.718724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aec3dc0-ef4e-439c-800a-87845dc9a8be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:05.719495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aec3dc0-ef4e-439c-800a-87845dc9a8be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:05.722812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aec3dc0-ef4e-439c-800a-87845dc9a8be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:05.723281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aec3dc0-ef4e-439c-800a-87845dc9a8be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:06.054649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3fa23fef-71dc-4895-8213-40f718d2025e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:06.069350 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3fa23fef-71dc-4895-8213-40f718d2025e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 0.481613 Sep 27 10:09:06.070013 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1764/3217] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:05 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:06.075789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f552bdb2-c18c-49e4-9fb5-99525ea435e8 req-7f8bb6f7-4a89-4f2d-bef9-3748474659fa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4fed875e-5be1-4b37-a0f9-55e6bc6fca05", "name": "network-vif-plugged", "status": "completed", "tag": "c0a6ca20-6141-4b76-b38e-d3a32bd4803c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:06.096262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f552bdb2-c18c-49e4-9fb5-99525ea435e8 req-7f8bb6f7-4a89-4f2d-bef9-3748474659fa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:06.097302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f552bdb2-c18c-49e4-9fb5-99525ea435e8 req-7f8bb6f7-4a89-4f2d-bef9-3748474659fa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:06.098088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f552bdb2-c18c-49e4-9fb5-99525ea435e8 req-7f8bb6f7-4a89-4f2d-bef9-3748474659fa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:06.142709 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f552bdb2-c18c-49e4-9fb5-99525ea435e8 req-7f8bb6f7-4a89-4f2d-bef9-3748474659fa service nova] Creating event network-vif-plugged:c0a6ca20-6141-4b76-b38e-d3a32bd4803c for instance 4fed875e-5be1-4b37-a0f9-55e6bc6fca05 on nch2-master-1 Sep 27 10:09:06.159924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f552bdb2-c18c-49e4-9fb5-99525ea435e8 req-7f8bb6f7-4a89-4f2d-bef9-3748474659fa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084631 Sep 27 10:09:06.162737 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1765/3218] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:06.179395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eefa4121-2e09-416b-8a62-eceaa8dc47d9 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:06.181861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eefa4121-2e09-416b-8a62-eceaa8dc47d9 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:06.202302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefa4121-2e09-416b-8a62-eceaa8dc47d9 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:06.204777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefa4121-2e09-416b-8a62-eceaa8dc47d9 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:06.205498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefa4121-2e09-416b-8a62-eceaa8dc47d9 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:06.206464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefa4121-2e09-416b-8a62-eceaa8dc47d9 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:06.208186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefa4121-2e09-416b-8a62-eceaa8dc47d9 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:06.210522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefa4121-2e09-416b-8a62-eceaa8dc47d9 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:06.358178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7aec3dc0-ef4e-439c-800a-87845dc9a8be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:06.373645 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7aec3dc0-ef4e-439c-800a-87845dc9a8be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1322 microversion: 2.1 time: 0.691248 Sep 27 10:09:06.381271 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1454/3219] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:05 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1322 bytes in 699 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:06.386322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ab9102f3-34c7-4e61-9ff4-f834e01c90cf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:06.386946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ab9102f3-34c7-4e61-9ff4-f834e01c90cf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:06.410922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab9102f3-34c7-4e61-9ff4-f834e01c90cf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:06.425533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab9102f3-34c7-4e61-9ff4-f834e01c90cf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:06.426266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab9102f3-34c7-4e61-9ff4-f834e01c90cf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:06.427220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab9102f3-34c7-4e61-9ff4-f834e01c90cf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:06.427944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab9102f3-34c7-4e61-9ff4-f834e01c90cf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:06.428640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ab9102f3-34c7-4e61-9ff4-f834e01c90cf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:06.924283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-eefa4121-2e09-416b-8a62-eceaa8dc47d9 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:06.935028 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eefa4121-2e09-416b-8a62-eceaa8dc47d9 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1471 microversion: 2.1 time: 0.767356 Sep 27 10:09:06.949483 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1766/3220] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:06 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1471 bytes in 786 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:06.954784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0f284339-f37b-4146-9b15-e97ec5708f7d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:06.955695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0f284339-f37b-4146-9b15-e97ec5708f7d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:06.968063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f284339-f37b-4146-9b15-e97ec5708f7d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:06.969065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f284339-f37b-4146-9b15-e97ec5708f7d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:06.969877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f284339-f37b-4146-9b15-e97ec5708f7d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:06.970895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f284339-f37b-4146-9b15-e97ec5708f7d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:06.971614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f284339-f37b-4146-9b15-e97ec5708f7d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:06.972294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f284339-f37b-4146-9b15-e97ec5708f7d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:07.354635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0f284339-f37b-4146-9b15-e97ec5708f7d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:07.389316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ab9102f3-34c7-4e61-9ff4-f834e01c90cf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:07.407410 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0f284339-f37b-4146-9b15-e97ec5708f7d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1351 microversion: 2.1 time: 0.456225 Sep 27 10:09:07.411972 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1767/3221] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:06 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1351 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:07.419328 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ab9102f3-34c7-4e61-9ff4-f834e01c90cf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 1.036795 Sep 27 10:09:07.420750 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1455/3222] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:06 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 1039 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:07.429070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fe230a0e-6b93-4790-b25d-648e3b813af3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:07.436358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5f09a8c-254f-4444-a85c-56c936b7a837 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:07.438285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5f09a8c-254f-4444-a85c-56c936b7a837 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:07.450162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f09a8c-254f-4444-a85c-56c936b7a837 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:07.451336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f09a8c-254f-4444-a85c-56c936b7a837 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:07.452001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f09a8c-254f-4444-a85c-56c936b7a837 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:07.453360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f09a8c-254f-4444-a85c-56c936b7a837 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:07.454050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f09a8c-254f-4444-a85c-56c936b7a837 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:07.454665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f09a8c-254f-4444-a85c-56c936b7a837 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:07.467707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fe230a0e-6b93-4790-b25d-648e3b813af3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:07.497101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe230a0e-6b93-4790-b25d-648e3b813af3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:07.497101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe230a0e-6b93-4790-b25d-648e3b813af3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:07.497101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe230a0e-6b93-4790-b25d-648e3b813af3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:07.497101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe230a0e-6b93-4790-b25d-648e3b813af3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:07.497101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe230a0e-6b93-4790-b25d-648e3b813af3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:07.497101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe230a0e-6b93-4790-b25d-648e3b813af3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:07.876430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fe230a0e-6b93-4790-b25d-648e3b813af3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:07.895274 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fe230a0e-6b93-4790-b25d-648e3b813af3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 0.476111 Sep 27 10:09:07.899802 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1768/3223] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:07 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:07.975387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fd8ffab3-6115-4e01-bf67-9b2858758f7b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:07.976792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fd8ffab3-6115-4e01-bf67-9b2858758f7b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:07.997823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd8ffab3-6115-4e01-bf67-9b2858758f7b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:08.001344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd8ffab3-6115-4e01-bf67-9b2858758f7b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:08.002199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd8ffab3-6115-4e01-bf67-9b2858758f7b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:08.007905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd8ffab3-6115-4e01-bf67-9b2858758f7b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:08.012247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd8ffab3-6115-4e01-bf67-9b2858758f7b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:08.012809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd8ffab3-6115-4e01-bf67-9b2858758f7b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:08.050871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5f09a8c-254f-4444-a85c-56c936b7a837 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:08.063310 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5f09a8c-254f-4444-a85c-56c936b7a837 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 200 len: 1515 microversion: 2.1 time: 0.636108 Sep 27 10:09:08.064801 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1456/3224] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:07 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 1515 bytes in 641 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:08.093464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a60b3d36-d167-433f-84c4-e2d7e4901cf5 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:08.094336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a60b3d36-d167-433f-84c4-e2d7e4901cf5 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 4fed875e-5be1-4b37-a0f9-55e6bc6fca05] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:08.103778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a60b3d36-d167-433f-84c4-e2d7e4901cf5 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:08.107275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a60b3d36-d167-433f-84c4-e2d7e4901cf5 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:08.107840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a60b3d36-d167-433f-84c4-e2d7e4901cf5 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:08.111438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a60b3d36-d167-433f-84c4-e2d7e4901cf5 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:08.114173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a60b3d36-d167-433f-84c4-e2d7e4901cf5 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:08.114880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a60b3d36-d167-433f-84c4-e2d7e4901cf5 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:08.401501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fd8ffab3-6115-4e01-bf67-9b2858758f7b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:08.412727 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fd8ffab3-6115-4e01-bf67-9b2858758f7b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1477 microversion: 2.1 time: 0.442259 Sep 27 10:09:08.429858 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1769/3225] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:07 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1477 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:09:08.437796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-09d978c1-103a-4f52-811b-297d92368b29 req-2cb9510a-9e9e-4b6d-a44c-58b581d0aa8a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd", "name": "network-vif-plugged", "status": "completed", "tag": "00a1303d-82aa-425e-b090-f9228ac6af6a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:08.449113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-09d978c1-103a-4f52-811b-297d92368b29 req-2cb9510a-9e9e-4b6d-a44c-58b581d0aa8a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:08.450026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-09d978c1-103a-4f52-811b-297d92368b29 req-2cb9510a-9e9e-4b6d-a44c-58b581d0aa8a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:08.450617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-09d978c1-103a-4f52-811b-297d92368b29 req-2cb9510a-9e9e-4b6d-a44c-58b581d0aa8a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:08.466683 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-09d978c1-103a-4f52-811b-297d92368b29 req-2cb9510a-9e9e-4b6d-a44c-58b581d0aa8a service nova] Creating event network-vif-plugged:00a1303d-82aa-425e-b090-f9228ac6af6a for instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd on nch1-master-1 Sep 27 10:09:08.473491 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-09d978c1-103a-4f52-811b-297d92368b29 req-2cb9510a-9e9e-4b6d-a44c-58b581d0aa8a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041481 Sep 27 10:09:08.474676 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1770/3226] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:08.480369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-975b0a3d-8256-4037-82f9-9db3cc5acece tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:08.481332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-975b0a3d-8256-4037-82f9-9db3cc5acece tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:08.488921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-975b0a3d-8256-4037-82f9-9db3cc5acece tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:08.489600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-975b0a3d-8256-4037-82f9-9db3cc5acece tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:08.490336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-975b0a3d-8256-4037-82f9-9db3cc5acece tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:08.491213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-975b0a3d-8256-4037-82f9-9db3cc5acece tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:08.491853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-975b0a3d-8256-4037-82f9-9db3cc5acece tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:08.492567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-975b0a3d-8256-4037-82f9-9db3cc5acece tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:08.624904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=12 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:08.684119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-975b0a3d-8256-4037-82f9-9db3cc5acece tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:08.695029 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-975b0a3d-8256-4037-82f9-9db3cc5acece tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1351 microversion: 2.1 time: 0.218944 Sep 27 10:09:08.705996 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1771/3227] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:08 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1351 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:08.728547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0c41e703-0ba6-489f-9be8-9429b9993eaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:08.729626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0c41e703-0ba6-489f-9be8-9429b9993eaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:08.752198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c41e703-0ba6-489f-9be8-9429b9993eaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:08.753600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c41e703-0ba6-489f-9be8-9429b9993eaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:08.757770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c41e703-0ba6-489f-9be8-9429b9993eaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:08.758813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c41e703-0ba6-489f-9be8-9429b9993eaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:08.759537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c41e703-0ba6-489f-9be8-9429b9993eaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:08.760146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c41e703-0ba6-489f-9be8-9429b9993eaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:09.085776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0c41e703-0ba6-489f-9be8-9429b9993eaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:09.097736 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0c41e703-0ba6-489f-9be8-9429b9993eaa tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1532 microversion: 2.1 time: 0.385788 Sep 27 10:09:09.098330 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1772/3228] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:08 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1532 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:09.113476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b34b7a83-7fcc-42ec-9d89-495bdd0e12a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:09.117044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b34b7a83-7fcc-42ec-9d89-495bdd0e12a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:09.130263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b34b7a83-7fcc-42ec-9d89-495bdd0e12a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:09.132320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b34b7a83-7fcc-42ec-9d89-495bdd0e12a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:09.132808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b34b7a83-7fcc-42ec-9d89-495bdd0e12a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:09.133549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b34b7a83-7fcc-42ec-9d89-495bdd0e12a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:09.140945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b34b7a83-7fcc-42ec-9d89-495bdd0e12a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:09.141747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b34b7a83-7fcc-42ec-9d89-495bdd0e12a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:09.149942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a60b3d36-d167-433f-84c4-e2d7e4901cf5 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:09.162560 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a60b3d36-d167-433f-84c4-e2d7e4901cf5 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05" status: 200 len: 1322 microversion: 2.1 time: 1.074311 Sep 27 10:09:09.187498 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1457/3229] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:08 2022] GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05 => generated 1322 bytes in 1100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:09.364990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ee738e80-3a32-4d49-935a-403436774730 tempest-AbsoluteLimitsTestJSON-767203783 tempest-AbsoluteLimitsTestJSON-767203783-project] Calling method '.version_select at 0x7f7f63899dc0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:09.384304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee738e80-3a32-4d49-935a-403436774730 tempest-AbsoluteLimitsTestJSON-767203783 tempest-AbsoluteLimitsTestJSON-767203783-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:09.385285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee738e80-3a32-4d49-935a-403436774730 tempest-AbsoluteLimitsTestJSON-767203783 tempest-AbsoluteLimitsTestJSON-767203783-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:09.386170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee738e80-3a32-4d49-935a-403436774730 tempest-AbsoluteLimitsTestJSON-767203783 tempest-AbsoluteLimitsTestJSON-767203783-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:09.387833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee738e80-3a32-4d49-935a-403436774730 tempest-AbsoluteLimitsTestJSON-767203783 tempest-AbsoluteLimitsTestJSON-767203783-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:09.388590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee738e80-3a32-4d49-935a-403436774730 tempest-AbsoluteLimitsTestJSON-767203783 tempest-AbsoluteLimitsTestJSON-767203783-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:09.405523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee738e80-3a32-4d49-935a-403436774730 tempest-AbsoluteLimitsTestJSON-767203783 tempest-AbsoluteLimitsTestJSON-767203783-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:09.421424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b34b7a83-7fcc-42ec-9d89-495bdd0e12a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:09.435210 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b34b7a83-7fcc-42ec-9d89-495bdd0e12a8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1523 microversion: 2.1 time: 0.332219 Sep 27 10:09:09.436926 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1773/3230] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:09 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1523 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:09.453288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a97a785b-b168-4ff7-858b-d53df1f0670a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:09.457459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a97a785b-b168-4ff7-858b-d53df1f0670a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:09.460183 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ee738e80-3a32-4d49-935a-403436774730 tempest-AbsoluteLimitsTestJSON-767203783 tempest-AbsoluteLimitsTestJSON-767203783-project] 10.222.0.26 "GET /compute/v2.1/limits" status: 200 len: 511 microversion: 2.1 time: 0.270595 Sep 27 10:09:09.474255 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1458/3231] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:09 2022] GET /compute/v2.1/limits => generated 511 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:09.493902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a97a785b-b168-4ff7-858b-d53df1f0670a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:09.494806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a97a785b-b168-4ff7-858b-d53df1f0670a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:09.496507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a97a785b-b168-4ff7-858b-d53df1f0670a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:09.497748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a97a785b-b168-4ff7-858b-d53df1f0670a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:09.500829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a97a785b-b168-4ff7-858b-d53df1f0670a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:09.505031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a97a785b-b168-4ff7-858b-d53df1f0670a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:09.731227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f7d4f2f8-2c56-433a-babc-45b7fcb1321b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:09.732072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f7d4f2f8-2c56-433a-babc-45b7fcb1321b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:09.749530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7d4f2f8-2c56-433a-babc-45b7fcb1321b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:09.752202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7d4f2f8-2c56-433a-babc-45b7fcb1321b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:09.755941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7d4f2f8-2c56-433a-babc-45b7fcb1321b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:09.756946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7d4f2f8-2c56-433a-babc-45b7fcb1321b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:09.758390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7d4f2f8-2c56-433a-babc-45b7fcb1321b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:09.760125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7d4f2f8-2c56-433a-babc-45b7fcb1321b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:09.805424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a97a785b-b168-4ff7-858b-d53df1f0670a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:09.817732 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a97a785b-b168-4ff7-858b-d53df1f0670a tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1462 microversion: 2.1 time: 0.377554 Sep 27 10:09:09.818974 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1774/3232] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:09 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1462 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:09.850226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-386553a2-2147-4cae-b9de-e35ac047d05a tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "error"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:09.857054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-386553a2-2147-4cae-b9de-e35ac047d05a tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:09.873820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-386553a2-2147-4cae-b9de-e35ac047d05a tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:09.874681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-386553a2-2147-4cae-b9de-e35ac047d05a tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:09.876119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-386553a2-2147-4cae-b9de-e35ac047d05a tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:09.879112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-386553a2-2147-4cae-b9de-e35ac047d05a tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:09.881191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-386553a2-2147-4cae-b9de-e35ac047d05a tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:09.883879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-386553a2-2147-4cae-b9de-e35ac047d05a tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:10.078019 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-386553a2-2147-4cae-b9de-e35ac047d05a tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action" status: 202 len: 0 microversion: 2.1 time: 0.232153 Sep 27 10:09:10.079499 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1775/3233] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:09:09 2022] POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action => generated 0 bytes in 234 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:09:10.087849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:10.090561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:10.104230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:10.107865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:10.109552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:10.117691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:10.119015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:10.121671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:10.131357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f7d4f2f8-2c56-433a-babc-45b7fcb1321b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:10.141935 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f7d4f2f8-2c56-433a-babc-45b7fcb1321b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1351 microversion: 2.1 time: 0.415049 Sep 27 10:09:10.147249 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1459/3234] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:09 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1351 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:09:10.153972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d068d69a-8fa3-406c-9696-3600dc081578 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:10.158818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d068d69a-8fa3-406c-9696-3600dc081578 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:10.173139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d068d69a-8fa3-406c-9696-3600dc081578 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:10.176710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d068d69a-8fa3-406c-9696-3600dc081578 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:10.177148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d068d69a-8fa3-406c-9696-3600dc081578 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:10.180586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d068d69a-8fa3-406c-9696-3600dc081578 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:10.181661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d068d69a-8fa3-406c-9696-3600dc081578 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:10.182969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d068d69a-8fa3-406c-9696-3600dc081578 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:10.221231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:10.223475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:10.226929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:10.227705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lazy-loading 'fault' on Instance uuid b902a024-8aa7-4843-8680-41d3e189d3af {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:09:10.232448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=1,nova_cell1:UPDATE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:10.456259 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4a4e2c8c-f9de-46a4-88ae-d1fad6673cd6 tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1596 microversion: 2.1 time: 0.372867 Sep 27 10:09:10.456896 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1776/3235] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:10 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1596 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:09:10.472883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6516e014-3887-45c5-b821-bb8a3990b513 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:10.472883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6516e014-3887-45c5-b821-bb8a3990b513 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 4fed875e-5be1-4b37-a0f9-55e6bc6fca05] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:10.481940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6516e014-3887-45c5-b821-bb8a3990b513 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:10.482762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6516e014-3887-45c5-b821-bb8a3990b513 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:10.483393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6516e014-3887-45c5-b821-bb8a3990b513 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:10.484346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6516e014-3887-45c5-b821-bb8a3990b513 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:10.485040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6516e014-3887-45c5-b821-bb8a3990b513 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:10.485655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6516e014-3887-45c5-b821-bb8a3990b513 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:10.682789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d068d69a-8fa3-406c-9696-3600dc081578 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:10.706562 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d068d69a-8fa3-406c-9696-3600dc081578 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1532 microversion: 2.1 time: 0.557702 Sep 27 10:09:10.709385 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1460/3236] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:10 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1532 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:10.729587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-161a2ddc-eb57-4c7e-a767-94d1046fa562 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:10.731402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-161a2ddc-eb57-4c7e-a767-94d1046fa562 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:10.746128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-161a2ddc-eb57-4c7e-a767-94d1046fa562 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:10.751299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-161a2ddc-eb57-4c7e-a767-94d1046fa562 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:10.752241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-161a2ddc-eb57-4c7e-a767-94d1046fa562 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:10.753752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-161a2ddc-eb57-4c7e-a767-94d1046fa562 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:10.754567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-161a2ddc-eb57-4c7e-a767-94d1046fa562 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:10.755155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-161a2ddc-eb57-4c7e-a767-94d1046fa562 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:11.060155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6516e014-3887-45c5-b821-bb8a3990b513 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:11.073493 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6516e014-3887-45c5-b821-bb8a3990b513 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05" status: 200 len: 1515 microversion: 2.1 time: 0.613979 Sep 27 10:09:11.081975 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1777/3237] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:10 2022] GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05 => generated 1515 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:11.096535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d492f7c8-3202-4f88-9643-3cb793c027bd tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Action: 'action', calling method: >, body: {"os-resetState": {"state": "active"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:11.099280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d492f7c8-3202-4f88-9643-3cb793c027bd tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:11.106882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d492f7c8-3202-4f88-9643-3cb793c027bd tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:11.107643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d492f7c8-3202-4f88-9643-3cb793c027bd tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:11.108243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d492f7c8-3202-4f88-9643-3cb793c027bd tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:11.109179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d492f7c8-3202-4f88-9643-3cb793c027bd tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:11.109842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d492f7c8-3202-4f88-9643-3cb793c027bd tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:11.110471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d492f7c8-3202-4f88-9643-3cb793c027bd tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:11.219102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-161a2ddc-eb57-4c7e-a767-94d1046fa562 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:11.229449 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-161a2ddc-eb57-4c7e-a767-94d1046fa562 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1505 microversion: 2.1 time: 0.516021 Sep 27 10:09:11.238001 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1461/3238] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:10 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1505 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:11.251108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b08a303b-69d3-4ad3-874b-d05c73f04486 req-f540009a-a5af-4a15-a1eb-1f08dfc4a709 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b902a024-8aa7-4843-8680-41d3e189d3af", "name": "network-vif-unplugged", "status": "completed", "tag": "df367b62-0029-488e-9897-fac934fc397e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:11.278650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b08a303b-69d3-4ad3-874b-d05c73f04486 req-f540009a-a5af-4a15-a1eb-1f08dfc4a709 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:11.279138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b08a303b-69d3-4ad3-874b-d05c73f04486 req-f540009a-a5af-4a15-a1eb-1f08dfc4a709 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:11.279631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b08a303b-69d3-4ad3-874b-d05c73f04486 req-f540009a-a5af-4a15-a1eb-1f08dfc4a709 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:11.292829 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d492f7c8-3202-4f88-9643-3cb793c027bd tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action" status: 202 len: 0 microversion: 2.1 time: 0.209227 Sep 27 10:09:11.293888 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1778/3239] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:09:11 2022] POST /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af/action => generated 0 bytes in 211 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:09:11.313964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e4eef00-2374-4f05-a13c-8c11a35e2614 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:11.315182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e4eef00-2374-4f05-a13c-8c11a35e2614 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:11.322342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4eef00-2374-4f05-a13c-8c11a35e2614 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:11.322969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4eef00-2374-4f05-a13c-8c11a35e2614 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:11.323508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4eef00-2374-4f05-a13c-8c11a35e2614 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:11.324302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4eef00-2374-4f05-a13c-8c11a35e2614 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:11.324580 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b08a303b-69d3-4ad3-874b-d05c73f04486 req-f540009a-a5af-4a15-a1eb-1f08dfc4a709 service nova] Creating event network-vif-unplugged:df367b62-0029-488e-9897-fac934fc397e for instance b902a024-8aa7-4843-8680-41d3e189d3af on nch1-master-1 Sep 27 10:09:11.325083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4eef00-2374-4f05-a13c-8c11a35e2614 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:11.325589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4eef00-2374-4f05-a13c-8c11a35e2614 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:11.339124 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b08a303b-69d3-4ad3-874b-d05c73f04486 req-f540009a-a5af-4a15-a1eb-1f08dfc4a709 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.099516 Sep 27 10:09:11.339976 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1462/3240] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:11 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:11.346021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-908fda5f-9bf3-4ff8-bc2c-eff0bda6165e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:11.346741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-908fda5f-9bf3-4ff8-bc2c-eff0bda6165e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:11.356164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-908fda5f-9bf3-4ff8-bc2c-eff0bda6165e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:11.356993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-908fda5f-9bf3-4ff8-bc2c-eff0bda6165e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:11.358120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-908fda5f-9bf3-4ff8-bc2c-eff0bda6165e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:11.359179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-908fda5f-9bf3-4ff8-bc2c-eff0bda6165e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:11.359801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-908fda5f-9bf3-4ff8-bc2c-eff0bda6165e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:11.360385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-908fda5f-9bf3-4ff8-bc2c-eff0bda6165e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:11.755452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-908fda5f-9bf3-4ff8-bc2c-eff0bda6165e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:11.764062 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-908fda5f-9bf3-4ff8-bc2c-eff0bda6165e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1351 microversion: 2.1 time: 0.422868 Sep 27 10:09:11.777458 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1463/3241] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:11 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1351 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:11.792915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-12841ad3-cf76-44f8-a7ee-b451a02b2f6d tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:11.793801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-12841ad3-cf76-44f8-a7ee-b451a02b2f6d tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:11.800843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12841ad3-cf76-44f8-a7ee-b451a02b2f6d tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:11.803052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12841ad3-cf76-44f8-a7ee-b451a02b2f6d tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:11.803567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12841ad3-cf76-44f8-a7ee-b451a02b2f6d tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:11.804468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12841ad3-cf76-44f8-a7ee-b451a02b2f6d tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:11.805006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12841ad3-cf76-44f8-a7ee-b451a02b2f6d tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:11.805620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12841ad3-cf76-44f8-a7ee-b451a02b2f6d tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:11.814263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0e4eef00-2374-4f05-a13c-8c11a35e2614 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:11.865047 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e4eef00-2374-4f05-a13c-8c11a35e2614 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd" status: 200 len: 1515 microversion: 2.1 time: 0.564737 Sep 27 10:09:11.869176 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1779/3242] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:11 2022] GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd => generated 1515 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:11.889685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-790f3556-ded4-40f7-910e-f48a5d80704a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:11.894837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-790f3556-ded4-40f7-910e-f48a5d80704a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:11.940622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-790f3556-ded4-40f7-910e-f48a5d80704a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:11.942465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-790f3556-ded4-40f7-910e-f48a5d80704a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:11.943193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-790f3556-ded4-40f7-910e-f48a5d80704a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:11.953574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-790f3556-ded4-40f7-910e-f48a5d80704a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:11.955625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-790f3556-ded4-40f7-910e-f48a5d80704a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:11.956342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-790f3556-ded4-40f7-910e-f48a5d80704a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.221708 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-12841ad3-cf76-44f8-a7ee-b451a02b2f6d tempest-ServersAdminTestJSON-1998984426 tempest-ServersAdminTestJSON-1998984426-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1613 microversion: 2.1 time: 0.443022 Sep 27 10:09:12.222608 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1464/3243] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:11 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1613 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:12.228985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f5cf1fc-69df-48a0-8675-c4b53b996e63 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:12.230616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f5cf1fc-69df-48a0-8675-c4b53b996e63 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:12.239576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f5cf1fc-69df-48a0-8675-c4b53b996e63 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:12.245364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f5cf1fc-69df-48a0-8675-c4b53b996e63 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:12.246060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f5cf1fc-69df-48a0-8675-c4b53b996e63 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.247310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f5cf1fc-69df-48a0-8675-c4b53b996e63 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:12.248567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f5cf1fc-69df-48a0-8675-c4b53b996e63 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:12.249213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f5cf1fc-69df-48a0-8675-c4b53b996e63 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.313626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f5cf1fc-69df-48a0-8675-c4b53b996e63 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:09:12.399961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-790f3556-ded4-40f7-910e-f48a5d80704a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:12.409822 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-790f3556-ded4-40f7-910e-f48a5d80704a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1532 microversion: 2.1 time: 0.537487 Sep 27 10:09:12.410242 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1780/3244] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:11 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1532 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:12.418952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f09f5283-a299-4cca-ad6b-2ef2fdb3b9be req-0a52c5ac-9fb9-490f-9d8e-9dd90ec8b876 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9ac32528-351d-4ba2-b0ec-0eeb4872e95c", "tag": "6f38bb60-56d7-44ce-ad27-ae8c1e3ac0fb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:12.450852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f09f5283-a299-4cca-ad6b-2ef2fdb3b9be req-0a52c5ac-9fb9-490f-9d8e-9dd90ec8b876 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:12.451648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f09f5283-a299-4cca-ad6b-2ef2fdb3b9be req-0a52c5ac-9fb9-490f-9d8e-9dd90ec8b876 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:12.452236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f09f5283-a299-4cca-ad6b-2ef2fdb3b9be req-0a52c5ac-9fb9-490f-9d8e-9dd90ec8b876 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.468415 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f09f5283-a299-4cca-ad6b-2ef2fdb3b9be req-0a52c5ac-9fb9-490f-9d8e-9dd90ec8b876 service nova] Creating event network-changed:6f38bb60-56d7-44ce-ad27-ae8c1e3ac0fb for instance 9ac32528-351d-4ba2-b0ec-0eeb4872e95c on nch2-master-1 Sep 27 10:09:12.484828 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f09f5283-a299-4cca-ad6b-2ef2fdb3b9be req-0a52c5ac-9fb9-490f-9d8e-9dd90ec8b876 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070810 Sep 27 10:09:12.488363 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1781/3245] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:12.494220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-40a6e7f8-689e-418d-a6d5-04de5a33d0d8 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:12.499019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-40a6e7f8-689e-418d-a6d5-04de5a33d0d8 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:12.510011 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f5cf1fc-69df-48a0-8675-c4b53b996e63 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "DELETE /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd" status: 204 len: 0 microversion: 2.1 time: 0.285885 Sep 27 10:09:12.510630 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1465/3246] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:09:12 2022] DELETE /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd => generated 0 bytes in 287 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:09:12.515635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40a6e7f8-689e-418d-a6d5-04de5a33d0d8 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:12.517232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40a6e7f8-689e-418d-a6d5-04de5a33d0d8 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:12.518921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73bf9457-a376-42ff-a5d5-cf339a8623ec tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:12.519602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40a6e7f8-689e-418d-a6d5-04de5a33d0d8 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.519743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-73bf9457-a376-42ff-a5d5-cf339a8623ec tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:12.520614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40a6e7f8-689e-418d-a6d5-04de5a33d0d8 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:12.521255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40a6e7f8-689e-418d-a6d5-04de5a33d0d8 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:12.524062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40a6e7f8-689e-418d-a6d5-04de5a33d0d8 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.542095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73bf9457-a376-42ff-a5d5-cf339a8623ec tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:12.542615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73bf9457-a376-42ff-a5d5-cf339a8623ec tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:12.543047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73bf9457-a376-42ff-a5d5-cf339a8623ec tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.543787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73bf9457-a376-42ff-a5d5-cf339a8623ec tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:12.544256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73bf9457-a376-42ff-a5d5-cf339a8623ec tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:12.544890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73bf9457-a376-42ff-a5d5-cf339a8623ec tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.577890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-40a6e7f8-689e-418d-a6d5-04de5a33d0d8 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:09:12.640321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:12.723089 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-40a6e7f8-689e-418d-a6d5-04de5a33d0d8 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "DELETE /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 204 len: 0 microversion: 2.1 time: 0.232937 Sep 27 10:09:12.723728 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1782/3247] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:09:12 2022] DELETE /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:09:12.729706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d8ecb42-b640-4f27-a88b-c72090a1ec60 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:12.730386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d8ecb42-b640-4f27-a88b-c72090a1ec60 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:12.742688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d8ecb42-b640-4f27-a88b-c72090a1ec60 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:12.743053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d8ecb42-b640-4f27-a88b-c72090a1ec60 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:12.743491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d8ecb42-b640-4f27-a88b-c72090a1ec60 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.744211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d8ecb42-b640-4f27-a88b-c72090a1ec60 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:12.751322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d8ecb42-b640-4f27-a88b-c72090a1ec60 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:12.751915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d8ecb42-b640-4f27-a88b-c72090a1ec60 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.876845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-73bf9457-a376-42ff-a5d5-cf339a8623ec tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:12.885589 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-73bf9457-a376-42ff-a5d5-cf339a8623ec tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1505 microversion: 2.1 time: 0.370867 Sep 27 10:09:12.888401 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1466/3248] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:12 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1505 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:12.901398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e6115b42-1b9c-46aa-a619-86d0abe4dc11 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:12.902309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e6115b42-1b9c-46aa-a619-86d0abe4dc11 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:12.915033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6115b42-1b9c-46aa-a619-86d0abe4dc11 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:12.915525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6115b42-1b9c-46aa-a619-86d0abe4dc11 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:12.916249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6115b42-1b9c-46aa-a619-86d0abe4dc11 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.917173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6115b42-1b9c-46aa-a619-86d0abe4dc11 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:12.917903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6115b42-1b9c-46aa-a619-86d0abe4dc11 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:12.918485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6115b42-1b9c-46aa-a619-86d0abe4dc11 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:12.965557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:13.130282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e6115b42-1b9c-46aa-a619-86d0abe4dc11 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:13.146258 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e6115b42-1b9c-46aa-a619-86d0abe4dc11 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1469 microversion: 2.1 time: 0.255721 Sep 27 10:09:13.147329 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1467/3249] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:12 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1469 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:13.166199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6152f146-4bfb-499e-bfc9-2e028c065651 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:13.167283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6152f146-4bfb-499e-bfc9-2e028c065651 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:13.176674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6152f146-4bfb-499e-bfc9-2e028c065651 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:13.177479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6152f146-4bfb-499e-bfc9-2e028c065651 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:13.178273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6152f146-4bfb-499e-bfc9-2e028c065651 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:13.181420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6152f146-4bfb-499e-bfc9-2e028c065651 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:13.181547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7d8ecb42-b640-4f27-a88b-c72090a1ec60 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:13.182478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6152f146-4bfb-499e-bfc9-2e028c065651 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:13.182932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6152f146-4bfb-499e-bfc9-2e028c065651 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:13.201764 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d8ecb42-b640-4f27-a88b-c72090a1ec60 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd" status: 200 len: 1521 microversion: 2.1 time: 0.476278 Sep 27 10:09:13.206608 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1783/3250] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:12 2022] GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd => generated 1521 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:13.455407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6152f146-4bfb-499e-bfc9-2e028c065651 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:13.466271 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6152f146-4bfb-499e-bfc9-2e028c065651 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1351 microversion: 2.1 time: 0.317521 Sep 27 10:09:13.467377 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1468/3251] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:13 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1351 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:13.474645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-dc023520-72b1-49af-b40b-765e1f211b82 req-93ae6434-6cab-4883-a449-f43bbd0f6036 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3327ff95-5c05-4167-a73b-5fd55f7bfabd", "name": "network-vif-plugged", "status": "completed", "tag": "d571b78e-f1b6-4353-b09e-cac4cd18cde7"}, {"server_uuid": "08babbce-373e-4cf5-a138-066b6eeb02c3", "name": "network-vif-unplugged", "status": "completed", "tag": "2ca9e80c-8a06-414a-81cf-a330f210be36"}, {"server_uuid": "b902a024-8aa7-4843-8680-41d3e189d3af", "name": "network-vif-plugged", "status": "completed", "tag": "df367b62-0029-488e-9897-fac934fc397e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:13.498289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dc023520-72b1-49af-b40b-765e1f211b82 req-93ae6434-6cab-4883-a449-f43bbd0f6036 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:13.498853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dc023520-72b1-49af-b40b-765e1f211b82 req-93ae6434-6cab-4883-a449-f43bbd0f6036 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:13.499221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dc023520-72b1-49af-b40b-765e1f211b82 req-93ae6434-6cab-4883-a449-f43bbd0f6036 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:13.503815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-742da409-2fa7-4d07-98a3-6a5de3d03645 tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Calling method '.version_select at 0x7f7f6c201af0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:13.518399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-742da409-2fa7-4d07-98a3-6a5de3d03645 tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:13.518710 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-dc023520-72b1-49af-b40b-765e1f211b82 req-93ae6434-6cab-4883-a449-f43bbd0f6036 service nova] Creating event network-vif-plugged:d571b78e-f1b6-4353-b09e-cac4cd18cde7 for instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd on nch2-master-1 Sep 27 10:09:13.519425 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-dc023520-72b1-49af-b40b-765e1f211b82 req-93ae6434-6cab-4883-a449-f43bbd0f6036 service nova] Creating event network-vif-unplugged:2ca9e80c-8a06-414a-81cf-a330f210be36 for instance 08babbce-373e-4cf5-a138-066b6eeb02c3 on nch2-master-1 Sep 27 10:09:13.519833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-742da409-2fa7-4d07-98a3-6a5de3d03645 tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:13.519997 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-dc023520-72b1-49af-b40b-765e1f211b82 req-93ae6434-6cab-4883-a449-f43bbd0f6036 service nova] Creating event network-vif-plugged:df367b62-0029-488e-9897-fac934fc397e for instance b902a024-8aa7-4843-8680-41d3e189d3af on nch1-master-1 Sep 27 10:09:13.520860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-742da409-2fa7-4d07-98a3-6a5de3d03645 tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:13.521941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-742da409-2fa7-4d07-98a3-6a5de3d03645 tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:13.524136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-742da409-2fa7-4d07-98a3-6a5de3d03645 tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:13.524798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-742da409-2fa7-4d07-98a3-6a5de3d03645 tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:13.560386 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-742da409-2fa7-4d07-98a3-6a5de3d03645 tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] 10.222.0.26 "GET /compute/v2.1/limits" status: 200 len: 511 microversion: 2.1 time: 0.352584 Sep 27 10:09:13.561416 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1784/3252] 10.222.0.26 () {58 vars in 1122 bytes} [Tue Sep 27 10:09:13 2022] GET /compute/v2.1/limits => generated 511 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:13.568298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-333fe123-baf5-490e-b1cd-8de772834683 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:13.569261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-333fe123-baf5-490e-b1cd-8de772834683 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:13.583659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-333fe123-baf5-490e-b1cd-8de772834683 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:13.585656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-333fe123-baf5-490e-b1cd-8de772834683 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:13.587786 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-dc023520-72b1-49af-b40b-765e1f211b82 req-93ae6434-6cab-4883-a449-f43bbd0f6036 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 527 microversion: 2.1 time: 0.096124 Sep 27 10:09:13.588713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-333fe123-baf5-490e-b1cd-8de772834683 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:13.589789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-333fe123-baf5-490e-b1cd-8de772834683 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:13.589921 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1469/3253] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:13 2022] POST /compute/v2.1/os-server-external-events => generated 527 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:13.591815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-333fe123-baf5-490e-b1cd-8de772834683 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:13.593405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-333fe123-baf5-490e-b1cd-8de772834683 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:13.726665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-063e12c4-f731-4e7f-9bf4-71ee6c49e87a tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AbsoluteLimitsNegativeTestJSON-server-1642054520", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"0": "0", "1": "1", "2": "2", "3": "3", "4": "4", "5": "5", "6": "6", "7": "7", "8": "8", "9": "9", "10": "10", "11": "11", "12": "12", "13": "13", "14": "14", "15": "15", "16": "16", "17": "17", "18": "18", "19": "19", "20": "20", "21": "21", "22": "22", "23": "23", "24": "24", "25": "25", "26": "26", "27": "27", "28": "28", "29": "29", "30": "30", "31": "31", "32": "32", "33": "33", "34": "34", "35": "35", "36": "36", "37": "37", "38": "38", "39": "39", "40": "40", "41": "41", "42": "42", "43": "43", "44": "44", "45": "45", "46": "46", "47": "47", "48": "48", "49": "49", "50": "50", "51": "51", "52": "52", "53": "53", "54": "54", "55": "55", "56": "56", "57": "57", "58": "58", "59": "59", "60": "60", "61": "61", "62": "62", "63": "63", "64": "64", "65": "65", "66": "66", "67": "67", "68": "68", "69": "69", "70": "70", "71": "71", "72": "72", "73": "73", "74": "74", "75": "75", "76": "76", "77": "77", "78": "78", "79": "79", "80": "80", "81": "81", "82": "82", "83": "83", "84": "84", "85": "85", "86": "86", "87": "87", "88": "88", "89": "89", "90": "90", "91": "91", "92": "92", "93": "93", "94": "94", "95": "95", "96": "96", "97": "97", "98": "98", "99": "99", "100": "100", "101": "101", "102": "102", "103": "103", "104": "104", "105": "105", "106": "106", "107": "107", "108": "108", "109": "109", "110": "110", "111": "111", "112": "112", "113": "113", "114": "114", "115": "115", "116": "116", "117": "117", "118": "118", "119": "119", "120": "120", "121": "121", "122": "122", "123": "123", "124": "124", "125": "125", "126": "126", "127": "127", "128": "128"}}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:13.796756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-063e12c4-f731-4e7f-9bf4-71ee6c49e87a tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:09:13.804408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-333fe123-baf5-490e-b1cd-8de772834683 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:13.813984 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-333fe123-baf5-490e-b1cd-8de772834683 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1532 microversion: 2.1 time: 0.251000 Sep 27 10:09:13.815318 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1785/3254] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:13 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1532 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:13.916525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3e20508d-5ffe-4a3e-9819-9b09955dd62f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:13.917223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3e20508d-5ffe-4a3e-9819-9b09955dd62f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:13.927116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e20508d-5ffe-4a3e-9819-9b09955dd62f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:13.927915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e20508d-5ffe-4a3e-9819-9b09955dd62f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:13.929160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e20508d-5ffe-4a3e-9819-9b09955dd62f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:13.930370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e20508d-5ffe-4a3e-9819-9b09955dd62f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:13.931037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e20508d-5ffe-4a3e-9819-9b09955dd62f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:13.931913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e20508d-5ffe-4a3e-9819-9b09955dd62f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:14.204037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3e20508d-5ffe-4a3e-9819-9b09955dd62f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:14.219173 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3e20508d-5ffe-4a3e-9819-9b09955dd62f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1505 microversion: 2.1 time: 0.306940 Sep 27 10:09:14.224308 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1786/3255] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:13 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1505 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:09:14.235433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-caa6828a-ef91-4708-b0bb-b2f2a5190a6d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:14.236408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-caa6828a-ef91-4708-b0bb-b2f2a5190a6d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:14.249652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa6828a-ef91-4708-b0bb-b2f2a5190a6d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:14.250341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa6828a-ef91-4708-b0bb-b2f2a5190a6d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:14.251139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa6828a-ef91-4708-b0bb-b2f2a5190a6d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:14.252012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa6828a-ef91-4708-b0bb-b2f2a5190a6d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:14.252630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa6828a-ef91-4708-b0bb-b2f2a5190a6d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:14.253461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-caa6828a-ef91-4708-b0bb-b2f2a5190a6d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:14.281778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-063e12c4-f731-4e7f-9bf4-71ee6c49e87a tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Getting quotas for project bf5cfea42dc94baeaaf5f1eaea296ff0. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:14.291202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-063e12c4-f731-4e7f-9bf4-71ee6c49e87a tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Getting quotas for user fd1d5ed67db946ce86f719a23ae682c3 and project bf5cfea42dc94baeaaf5f1eaea296ff0. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:14.336129 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-063e12c4-f731-4e7f-9bf4-71ee6c49e87a tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] HTTP exception thrown: Maximum number of metadata items exceeds 128 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-063e12c4-f731-4e7f-9bf4-71ee6c49e87a tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Request method failure captured: Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 1752 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsvI5CU5_R0-gQa7PZiAK-0VL8bziK-_u2yDoQzClVUqBFzpg0EHVIXHsUL7SH6TGO0ysn-19GPECR3-I60taNKsth8wMcQH1YH2t7_PtdTVgklGjHpiSeOZWjbGwdP0RrQ-8_OImQnZCkYh30WtFxzcWHbCH35jxReUFdXeD8lXgL8 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: bf5cfea42dc94baeaaf5f1eaea296ff0 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AbsoluteLimitsNegativeTestJSON-587511915 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/bf5cfea42dc94baeaaf5f1eaea296ff0"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/bf5cfea42dc94baeaaf5f1eaea296ff0"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/bf5cfea42dc94baeaaf5f1eaea296ff0"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_bf5cfea42dc94baeaaf5f1eaea296ff0"}]}] Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AbsoluteLimitsNegativeTestJSON-587511915 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: bf5cfea42dc94baeaaf5f1eaea296ff0 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AbsoluteLimitsNegativeTestJSON-587511915 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AbsoluteLimitsNegativeTestJSON-587511915-project Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: fd1d5ed67db946ce86f719a23ae682c3 Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AbsoluteLimitsNegativeTestJSON-587511915-project Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-AbsoluteLimitsNegativeTestJSON-server-1642054520", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"0": "0", "1": "1", "2": "2", "3": "3", "4": "4", "5": "5", "6": "6", "7": "7", "8": "8", "9": "9", "10": "10", "11": "11", "12": "12", "13": "13", "14": "14", "15": "15", "16": "16", "17": "17", "18": "18", "19": "19", "20": "20", "21": "21", "22": "22", "23": "23", "24": "24", "25": "25", "26": "26", "27": "27", "28": "28", "29": "29", "30": "30", "31": "31", "32": "32", "33": "33", "34": "34", "35": "35", "36": "36", "37": "37", "38": "38", "39": "39", "40": "40", "41": "41", "42": "42", "43": "43", "44": "44", "45": "45", "46": "46", "47": "47", "48": "48", "49": "49", "50": "50", "51": "51", "52": "52", "53": "53", "54": "54", "55": "55", "56": "56", "57": "57", "58": "58", "59": "59", "60": "60", "61": "61", "62": "62", "63": "63", "64": "64", "65": "65", "66": "66", "67": "67", "68": "68", "69": "69", "70": "70", "71": "71", "72": "72", "73": "73", "74": "74", "75": "75", "76": "76", "77": "77", "78": "78", "79": "79", "80": "80", "81": "81", "82": "82", "83": "83", "84": "84", "85": "85", "86": "86", "87": "87", "88": "88", "89": "89", "90": "90", "91": "91", "92": "92", "93": "93", "94": "94", "95": "95", "96": "96", "97": "97", "98": "98", "99": "99", "100": "100", "101": "101", "102": "102", "103": "103", "104": "104", "105": "105", "106": "106", "107": "107", "108": "108", "109": "109", "110": "110", "111": "111", "112": "112", "113": "113", "114": "114", "115": "115", "116": "116", "117": "117", "118": "118", "119": "119", "120": "120", "121": "121", "122": "122", "123": "123", "124": "124", "125": "125", "126": "126", "127": "127", "128": "128"}}} Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:09:14.338335 ncdv-master-1 devstack@n-api.service[145969]: exception: Maximum number of metadata items exceeds 128 Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-AbsoluteLimitsNegativeTestJSON-server-1642054520', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'metadata': {'0': '0', '1': '1', '2': '2', '3': '3', '4': '4', '5': '5', '6': '6', '7': '7', '8': '8', '9': '9', '10': '10', '11': '11', '12': '12', '13': '13', '14': '14', '15': '15', '16': '16', '17': '17', '18': '18', '19': '19', '20': '20', '21': '21', '22': '22', '23': '23', '24': '24', '25': '25', '26': '26', '27': '27', '28': '28', '29': '29', '30': '30', '31': '31', '32': '32', '33': '33', '34': '34', '35': '35', '36': '36', '37': '37', '38': '38', '39': '39', '40': '40', '41': '41', '42': '42', '43': '43', '44': '44', '45': '45', '46': '46', '47': '47', '48': '48', '49': '49', '50': '50', '51': '51', '52': '52', '53': '53', '54': '54', '55': '55', '56': '56', '57': '57', '58': '58', '59': '59', '60': '60', '61': '61', '62': '62', '63': '63', '64': '64', '65': '65', '66': '66', '67': '67', '68': '68', '69': '69', '70': '70', '71': '71', '72': '72', '73': '73', '74': '74', '75': '75', '76': '76', '77': '77', '78': '78', '79': '79', '80': '80', '81': '81', '82': '82', '83': '83', '84': '84', '85': '85', '86': '86', '87': '87', '88': '88', '89': '89', '90': '90', '91': '91', '92': '92', '93': '93', '94': '94', '95': '95', '96': '96', '97': '97', '98': '98', '99': '99', '100': '100', '101': '101', '102': '102', '103': '103', '104': '104', '105': '105', '106': '106', '107': '107', '108': '108', '109': '109', '110': '110', '111': '111', '112': '112', '113': '113', '114': '114', '115': '115', '116': '116', '117': '117', '118': '118', '119': '119', '120': '120', '121': '121', '122': '122', '123': '123', '124': '124', '125': '125', '126': '126', '127': '127', '128': '128'}}}} Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 476, in _check_metadata_properties_quota Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi objects.Quotas.limit_check(context, metadata_items=num_metadata) Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 290, in limit_check Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check( Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1118, in limit_check Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._driver.limit_check(context, self._resources, values, Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 468, in limit_check Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), quotas=quotas, Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['metadata_items'] Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 784, in create Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instances, resv_id = self.compute_api.create( Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2206, in create Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._create_instance( Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1724, in _create_instance Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._checks_for_create_and_rebuild(context, image_id, boot_meta, Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 1016, in _checks_for_create_and_rebuild Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._check_metadata_properties_quota(context, metadata) Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 483, in _check_metadata_properties_quota Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.MetadataLimitExceeded(allowed=quota_metadata) Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.MetadataLimitExceeded: Maximum number of metadata items exceeds 128 Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:14.341156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [Previous line repeated 10 more times] Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 800, in create Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Maximum number of metadata items exceeds 128 Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Maximum number of metadata items exceeds 128 Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-063e12c4-f731-4e7f-9bf4-71ee6c49e87a tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] Returning 403 to user: Maximum number of metadata items exceeds 128 {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-063e12c4-f731-4e7f-9bf4-71ee6c49e87a tempest-AbsoluteLimitsNegativeTestJSON-587511915 tempest-AbsoluteLimitsNegativeTestJSON-587511915-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 403 len: 87 microversion: 2.1 time: 0.619440 Sep 27 10:09:14.344751 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1470/3256] 10.222.0.26 () {60 vars in 1148 bytes} [Tue Sep 27 10:09:13 2022] POST /compute/v2.1/servers => generated 87 bytes in 621 msecs (HTTP/1.1 403) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:09:14.352480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c0427f17-036b-4656-8225-37eba2c0f2bf tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:14.353567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c0427f17-036b-4656-8225-37eba2c0f2bf tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:14.361874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0427f17-036b-4656-8225-37eba2c0f2bf tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:14.363131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0427f17-036b-4656-8225-37eba2c0f2bf tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:14.364038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0427f17-036b-4656-8225-37eba2c0f2bf tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:14.365152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0427f17-036b-4656-8225-37eba2c0f2bf tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:14.366384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0427f17-036b-4656-8225-37eba2c0f2bf tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:14.367433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0427f17-036b-4656-8225-37eba2c0f2bf tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:14.588841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-caa6828a-ef91-4708-b0bb-b2f2a5190a6d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:14.598860 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-caa6828a-ef91-4708-b0bb-b2f2a5190a6d tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1469 microversion: 2.1 time: 0.372831 Sep 27 10:09:14.600603 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1787/3257] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:14 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1469 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:14.611315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-efcb1691-4daa-47de-a992-27023c5dd04f tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:14.614033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-efcb1691-4daa-47de-a992-27023c5dd04f tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:14.627822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c0427f17-036b-4656-8225-37eba2c0f2bf tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:14.639077 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c0427f17-036b-4656-8225-37eba2c0f2bf tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd" status: 200 len: 1521 microversion: 2.1 time: 0.288673 Sep 27 10:09:14.640069 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1471/3258] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:14 2022] GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd => generated 1521 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:14.661237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efcb1691-4daa-47de-a992-27023c5dd04f tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:14.663960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efcb1691-4daa-47de-a992-27023c5dd04f tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:14.677148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efcb1691-4daa-47de-a992-27023c5dd04f tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:14.683033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efcb1691-4daa-47de-a992-27023c5dd04f tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:14.683778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efcb1691-4daa-47de-a992-27023c5dd04f tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:14.687449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efcb1691-4daa-47de-a992-27023c5dd04f tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:14.839591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-94f22956-6e3d-40c1-be6a-ef6246a43f62 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:14.840694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-94f22956-6e3d-40c1-be6a-ef6246a43f62 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:14.857376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94f22956-6e3d-40c1-be6a-ef6246a43f62 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:14.858176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94f22956-6e3d-40c1-be6a-ef6246a43f62 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:14.858652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94f22956-6e3d-40c1-be6a-ef6246a43f62 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:14.859477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94f22956-6e3d-40c1-be6a-ef6246a43f62 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:14.860132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94f22956-6e3d-40c1-be6a-ef6246a43f62 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:14.860671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-94f22956-6e3d-40c1-be6a-ef6246a43f62 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:15.065401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-efcb1691-4daa-47de-a992-27023c5dd04f tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:15.080614 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-efcb1691-4daa-47de-a992-27023c5dd04f tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1351 microversion: 2.1 time: 0.477234 Sep 27 10:09:15.093715 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1788/3259] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:14 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1351 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:15.251882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-20874d56-d0c2-45e7-ba0c-c672240f2906 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:15.253176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-20874d56-d0c2-45e7-ba0c-c672240f2906 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:15.263273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20874d56-d0c2-45e7-ba0c-c672240f2906 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:15.263722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20874d56-d0c2-45e7-ba0c-c672240f2906 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:15.264157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20874d56-d0c2-45e7-ba0c-c672240f2906 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:15.268113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20874d56-d0c2-45e7-ba0c-c672240f2906 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:15.268606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20874d56-d0c2-45e7-ba0c-c672240f2906 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:15.269077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20874d56-d0c2-45e7-ba0c-c672240f2906 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:15.534596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-94f22956-6e3d-40c1-be6a-ef6246a43f62 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:15.547158 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-94f22956-6e3d-40c1-be6a-ef6246a43f62 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.712011 Sep 27 10:09:15.568210 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1472/3260] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:14 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 735 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:15.627384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0d53399e-27ae-40d6-9069-8b283dbc7043 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:15.628301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0d53399e-27ae-40d6-9069-8b283dbc7043 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:15.637643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d53399e-27ae-40d6-9069-8b283dbc7043 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:15.646362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d53399e-27ae-40d6-9069-8b283dbc7043 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:15.647500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d53399e-27ae-40d6-9069-8b283dbc7043 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:15.648544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d53399e-27ae-40d6-9069-8b283dbc7043 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:15.649195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d53399e-27ae-40d6-9069-8b283dbc7043 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:15.649938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d53399e-27ae-40d6-9069-8b283dbc7043 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:15.976232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-20874d56-d0c2-45e7-ba0c-c672240f2906 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:16.012810 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-20874d56-d0c2-45e7-ba0c-c672240f2906 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 200 len: 1508 microversion: 2.1 time: 0.765882 Sep 27 10:09:16.024320 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1789/3261] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:15 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 1508 bytes in 776 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:09:16.031249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4cdaadfe-8f83-4a32-aa1a-5a0ca47ea4b4 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:16.032144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4cdaadfe-8f83-4a32-aa1a-5a0ca47ea4b4 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:16.043039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:16.045598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cdaadfe-8f83-4a32-aa1a-5a0ca47ea4b4 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:16.046193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cdaadfe-8f83-4a32-aa1a-5a0ca47ea4b4 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:16.046638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cdaadfe-8f83-4a32-aa1a-5a0ca47ea4b4 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:16.047378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cdaadfe-8f83-4a32-aa1a-5a0ca47ea4b4 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:16.047837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cdaadfe-8f83-4a32-aa1a-5a0ca47ea4b4 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:16.048352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cdaadfe-8f83-4a32-aa1a-5a0ca47ea4b4 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:16.090771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0d53399e-27ae-40d6-9069-8b283dbc7043 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:16.100599 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0d53399e-27ae-40d6-9069-8b283dbc7043 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 200 len: 1469 microversion: 2.1 time: 0.477518 Sep 27 10:09:16.102130 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1473/3262] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:15 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 1469 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:09:16.271545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4cdaadfe-8f83-4a32-aa1a-5a0ca47ea4b4 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:16.285861 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4cdaadfe-8f83-4a32-aa1a-5a0ca47ea4b4 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd" status: 200 len: 1521 microversion: 2.1 time: 0.254936 Sep 27 10:09:16.285861 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1790/3263] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:16 2022] GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd => generated 1521 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:16.303618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:16.313364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:16.323529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:16.324311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:16.325434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:16.327171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:16.328309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:16.328977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:16.366204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=2,nova_cell1:UPDATE=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:16.374346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:09:16.382418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV254Test-server-1081529821", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": "none"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:16.382678 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.compute.api [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Working on deleting snapshot 58d9fd8c-7a3d-428c-b522-1634d6694718 from shelved instance... Sep 27 10:09:16.512448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:09:16.513229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:09:16.514097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:09:16.514735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:09:16.515248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:09:16.539702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:16.541160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:16.542059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:16.544861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:16.545868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:16.547547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:16.593714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Getting quotas for project 1f83c8276958489e84b0907523036097. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:16.610196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Getting quotas for user d8cc9d27cf6944aa9f5ef81d7fb3c428 and project 1f83c8276958489e84b0907523036097. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:16.650446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:09:16.660373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:09:16.807037 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2f8cd01b-7853-4387-81e8-66d404c8a063 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.54 time: 0.703619 Sep 27 10:09:16.812791 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1474/3264] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:09:16 2022] POST /compute/v2.1/servers => generated 380 bytes in 710 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:09:16.830749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-df1fe432-a837-420b-8668-a1371b59e99a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:16.832127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-df1fe432-a837-420b-8668-a1371b59e99a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:16.879417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df1fe432-a837-420b-8668-a1371b59e99a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:16.886676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df1fe432-a837-420b-8668-a1371b59e99a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:16.887322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df1fe432-a837-420b-8668-a1371b59e99a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:16.891796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df1fe432-a837-420b-8668-a1371b59e99a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:16.892270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df1fe432-a837-420b-8668-a1371b59e99a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:16.892731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df1fe432-a837-420b-8668-a1371b59e99a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:17.897225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-df1fe432-a837-420b-8668-a1371b59e99a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:17.925657 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-df1fe432-a837-420b-8668-a1371b59e99a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1351 microversion: 2.1 time: 1.105905 Sep 27 10:09:17.929299 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1475/3265] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:16 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1351 bytes in 1116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:17.950646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-05d607e2-5bf7-467e-b01a-2c74b9d8069d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:17.951379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-05d607e2-5bf7-467e-b01a-2c74b9d8069d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:17.959656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05d607e2-5bf7-467e-b01a-2c74b9d8069d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:17.961108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05d607e2-5bf7-467e-b01a-2c74b9d8069d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:17.962446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05d607e2-5bf7-467e-b01a-2c74b9d8069d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:17.963823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05d607e2-5bf7-467e-b01a-2c74b9d8069d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:17.964445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05d607e2-5bf7-467e-b01a-2c74b9d8069d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:17.965274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05d607e2-5bf7-467e-b01a-2c74b9d8069d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:18.261750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Doing local delete in cell ed1088da-caae-4426-8048-9cba90e7755c(cell1) {{(pid=145968) _delete /opt/stack/nova/nova/compute/api.py:2546}} Sep 27 10:09:18.262837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:18.263425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:18.264011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:18.264561 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.compute.api [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] instance is in SHELVED_OFFLOADED state, cleanup the instance's info from database. Sep 27 10:09:18.276581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] deallocate_for_instance() {{(pid=145968) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1780}} Sep 27 10:09:18.906809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-05d607e2-5bf7-467e-b01a-2c74b9d8069d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:18.922868 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-05d607e2-5bf7-467e-b01a-2c74b9d8069d tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.989865 Sep 27 10:09:18.923530 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1476/3266] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:17 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 993 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:18.942544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-abd29ab0-8031-48fb-80a6-d7385d834b78 req-5e20a61a-dafa-4cc9-8206-a2df4af86ae8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bfc0bc5d-a883-4255-9869-8aaba51eaae2", "name": "network-vif-deleted", "tag": "8d8cb0aa-16a7-4483-8c75-b88ae1eeb6d4"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:18.965952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abd29ab0-8031-48fb-80a6-d7385d834b78 req-5e20a61a-dafa-4cc9-8206-a2df4af86ae8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:18.967333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abd29ab0-8031-48fb-80a6-d7385d834b78 req-5e20a61a-dafa-4cc9-8206-a2df4af86ae8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:18.967957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abd29ab0-8031-48fb-80a6-d7385d834b78 req-5e20a61a-dafa-4cc9-8206-a2df4af86ae8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:19.034978 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-abd29ab0-8031-48fb-80a6-d7385d834b78 req-5e20a61a-dafa-4cc9-8206-a2df4af86ae8 service nova] Creating event network-vif-deleted:8d8cb0aa-16a7-4483-8c75-b88ae1eeb6d4 for instance bfc0bc5d-a883-4255-9869-8aaba51eaae2 on nch1-master-1 Sep 27 10:09:19.047568 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-abd29ab0-8031-48fb-80a6-d7385d834b78 req-5e20a61a-dafa-4cc9-8206-a2df4af86ae8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.122767 Sep 27 10:09:19.052006 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1477/3267] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:19.073532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:19.074899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:19.087484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:19.089404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:19.089872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:19.090429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:19.090903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:19.091399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:20.076258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:20.090929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:20.092593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:20.093992 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-37a47d90-58c3-45a8-b197-7e262365f9c0 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1274 microversion: 2.54 time: 1.040106 Sep 27 10:09:20.095123 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1478/3268] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:19 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1274 bytes in 1042 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:20.114148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:20.132902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: bfc0bc5d-a883-4255-9869-8aaba51eaae2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:20.142463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:20.149709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:20.153264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:20.154258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:20.157862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:20.160519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:20.205348 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] HTTP exception thrown: Instance bfc0bc5d-a883-4255-9869-8aaba51eaae2 could not be found. Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Request method failure captured: Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 HTTP/1.1 Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMstY7pvZq0cwMNYMMVADM0gWwtU_Ctmt8D3P6WO9OPmRX7a6zhma5oX4hGPRVJhWPCRMjjMBtQe3DrNvI4aKKTceFl2eCz0eUV7imwxj3WQAsdS4VavncEa0g3-vfbBrzWhsnamQA0g-aTtmbyfHvm91gTZTbTU_45aI7C4p4PszrLQ Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 52efcec2c8a14640a8c3df1dc576bbf1 Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersAdminTestJSON-568530853 Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/52efcec2c8a14640a8c3df1dc576bbf1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/52efcec2c8a14640a8c3df1dc576bbf1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/52efcec2c8a14640a8c3df1dc576bbf1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_52efcec2c8a14640a8c3df1dc576bbf1"}]}] Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersAdminTestJSON-568530853 Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 52efcec2c8a14640a8c3df1dc576bbf1 Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersAdminTestJSON-568530853 Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersAdminTestJSON-568530853-project Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b4e0f20e52074142a3b525bafbb30c97 Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersAdminTestJSON-568530853-project Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance bfc0bc5d-a883-4255-9869-8aaba51eaae2 could not be found. Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'bfc0bc5d-a883-4255-9869-8aaba51eaae2'} Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bfc0bc5d-a883-4255-9869-8aaba51eaae2 could not be found. Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bfc0bc5d-a883-4255-9869-8aaba51eaae2 could not be found. Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:20.212945 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:20.218135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:20.218135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:20.218135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:20.218135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:09:20.218135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:09:20.218135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bfc0bc5d-a883-4255-9869-8aaba51eaae2 could not be found. Sep 27 10:09:20.218135 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:20.224859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Returning 404 to user: Instance bfc0bc5d-a883-4255-9869-8aaba51eaae2 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:20.226080 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1fa12467-c32e-4d78-b4cd-dc4f132c0987 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2" status: 404 len: 111 microversion: 2.1 time: 0.129643 Sep 27 10:09:20.227182 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1479/3269] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:20 2022] GET /compute/v2.1/servers/bfc0bc5d-a883-4255-9869-8aaba51eaae2 => generated 111 bytes in 131 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:09:20.243089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:20.250851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:20.281985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:20.285053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:20.288823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:20.292563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:20.293468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:20.296264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:20.380483 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] HTTP exception thrown: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Request method failure captured: Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd HTTP/1.1 Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsuj15rCUD1970qSbzxB3cw4sTAfA1YtNh4VnoapFvuP9GffqVpFgRGMeWDK-Po0RARJzDlkwtHZJxI_lb7FISD34bP5tmzbtvUeP246046Go7rEPdSKK7iPObXyQaBpwtLG8VaeId4vMcWe5OB5uDdCNIPuVd0zCfFs93HwTWDEgHo Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_79022e5ca5764576b4feb7f4cadba80c"}]}] Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2699ef01c0eb4d5783203e6b41293d97 Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd'} Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:20.392125 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:20.399067 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:20.399067 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:20.399067 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:20.399067 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:20.399067 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:09:20.399067 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:09:20.399067 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:20.399067 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:20.399067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Returning 404 to user: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:20.402537 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-538dbbfe-236e-4f1e-813b-1f44800ff414 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd" status: 404 len: 111 microversion: 2.1 time: 0.173965 Sep 27 10:09:20.408719 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1480/3270] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:20 2022] GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd => generated 111 bytes in 181 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:09:20.429671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dd34df8c-8184-4d2e-b3e8-0939b502522c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:20.449285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dd34df8c-8184-4d2e-b3e8-0939b502522c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:20.482970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd34df8c-8184-4d2e-b3e8-0939b502522c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:20.486552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd34df8c-8184-4d2e-b3e8-0939b502522c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:20.489607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd34df8c-8184-4d2e-b3e8-0939b502522c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:20.495528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd34df8c-8184-4d2e-b3e8-0939b502522c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:20.496061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd34df8c-8184-4d2e-b3e8-0939b502522c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:20.496524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd34df8c-8184-4d2e-b3e8-0939b502522c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:20.701031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Updating instance_info_cache with network_info: [] {{(pid=145968) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:09:21.112605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dd34df8c-8184-4d2e-b3e8-0939b502522c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:21.124921 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dd34df8c-8184-4d2e-b3e8-0939b502522c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1351 microversion: 2.1 time: 0.712450 Sep 27 10:09:21.125837 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1481/3271] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:20 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1351 bytes in 716 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:21.133489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bc09a6ff-6076-497e-9cbf-35c4fab41db6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:21.138773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bc09a6ff-6076-497e-9cbf-35c4fab41db6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:21.153239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc09a6ff-6076-497e-9cbf-35c4fab41db6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:21.154031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc09a6ff-6076-497e-9cbf-35c4fab41db6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:21.154627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc09a6ff-6076-497e-9cbf-35c4fab41db6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:21.155568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc09a6ff-6076-497e-9cbf-35c4fab41db6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:21.156252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc09a6ff-6076-497e-9cbf-35c4fab41db6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:21.156973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc09a6ff-6076-497e-9cbf-35c4fab41db6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:21.386974 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3985a4a5-3865-437c-bf07-789fbad6d35c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 204 len: 0 microversion: 2.1 time: 5.089027 Sep 27 10:09:21.390838 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1791/3272] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:09:16 2022] DELETE /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 0 bytes in 5102 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:09:21.395837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-47a55c0b-66db-4acf-b384-b9975f0526fe tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:21.397770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-47a55c0b-66db-4acf-b384-b9975f0526fe tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:21.412196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a55c0b-66db-4acf-b384-b9975f0526fe tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:21.412732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a55c0b-66db-4acf-b384-b9975f0526fe tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:21.413178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a55c0b-66db-4acf-b384-b9975f0526fe tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:21.413944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a55c0b-66db-4acf-b384-b9975f0526fe tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:21.414405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a55c0b-66db-4acf-b384-b9975f0526fe tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:21.419111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a55c0b-66db-4acf-b384-b9975f0526fe tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:21.505498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-47a55c0b-66db-4acf-b384-b9975f0526fe tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:09:21.733480 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-47a55c0b-66db-4acf-b384-b9975f0526fe tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 204 len: 0 microversion: 2.1 time: 0.343955 Sep 27 10:09:21.739928 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1792/3273] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:09:21 2022] DELETE /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:09:21.760366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-91aab6b9-fd4b-40c3-9559-ac93df43130c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:21.770253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-91aab6b9-fd4b-40c3-9559-ac93df43130c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:21.773732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-91aab6b9-fd4b-40c3-9559-ac93df43130c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Searching by: {'changes-since': datetime.datetime(2051, 1, 1, 12, 34, tzinfo=datetime.timezone.utc), 'vm_state': ['active'], 'project_id': '79022e5ca5764576b4feb7f4cadba80c'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:09:21.793738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91aab6b9-fd4b-40c3-9559-ac93df43130c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:21.794281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91aab6b9-fd4b-40c3-9559-ac93df43130c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:21.802424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91aab6b9-fd4b-40c3-9559-ac93df43130c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:21.803504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91aab6b9-fd4b-40c3-9559-ac93df43130c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:21.804029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91aab6b9-fd4b-40c3-9559-ac93df43130c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:21.804472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91aab6b9-fd4b-40c3-9559-ac93df43130c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:21.903974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bc09a6ff-6076-497e-9cbf-35c4fab41db6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:21.919743 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bc09a6ff-6076-497e-9cbf-35c4fab41db6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.789766 Sep 27 10:09:21.921209 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1482/3274] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:21 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 794 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:21.932312 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-91aab6b9-fd4b-40c3-9559-ac93df43130c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers?changes-since=2051-01-01T12%3A34%3A00Z&status=ACTIVE" status: 200 len: 15 microversion: 2.1 time: 0.187872 Sep 27 10:09:21.938024 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1793/3275] 10.222.0.26 () {58 vars in 1230 bytes} [Tue Sep 27 10:09:21 2022] GET /compute/v2.1/servers?changes-since=2051-01-01T12%3A34%3A00Z&status=ACTIVE => generated 15 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:09:21.939765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7bc39085-4128-4eb2-b12a-ad30b68dee6e req-8b225b1f-2d8a-4f16-a31f-82a5684311d3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "08babbce-373e-4cf5-a138-066b6eeb02c3", "name": "network-vif-deleted", "tag": "2ca9e80c-8a06-414a-81cf-a330f210be36"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:21.944875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7d3461d4-1ad8-41af-a193-3779ca8d948c req-d356cd5e-80ad-4379-af72-14ef3fc53e60 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd", "name": "network-vif-deleted", "tag": "00a1303d-82aa-425e-b090-f9228ac6af6a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:21.970361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7d3461d4-1ad8-41af-a193-3779ca8d948c req-d356cd5e-80ad-4379-af72-14ef3fc53e60 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:21.970857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7d3461d4-1ad8-41af-a193-3779ca8d948c req-d356cd5e-80ad-4379-af72-14ef3fc53e60 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:21.971293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7d3461d4-1ad8-41af-a193-3779ca8d948c req-d356cd5e-80ad-4379-af72-14ef3fc53e60 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:21.971806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:21.980249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7bc39085-4128-4eb2-b12a-ad30b68dee6e req-8b225b1f-2d8a-4f16-a31f-82a5684311d3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:21.989660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7bc39085-4128-4eb2-b12a-ad30b68dee6e req-8b225b1f-2d8a-4f16-a31f-82a5684311d3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:21.990220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7bc39085-4128-4eb2-b12a-ad30b68dee6e req-8b225b1f-2d8a-4f16-a31f-82a5684311d3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:21.995066 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7d3461d4-1ad8-41af-a193-3779ca8d948c req-d356cd5e-80ad-4379-af72-14ef3fc53e60 service nova] Creating event network-vif-deleted:00a1303d-82aa-425e-b090-f9228ac6af6a for instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd on nch1-master-1 Sep 27 10:09:22.003702 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7d3461d4-1ad8-41af-a193-3779ca8d948c req-d356cd5e-80ad-4379-af72-14ef3fc53e60 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064087 Sep 27 10:09:22.005958 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1794/3276] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:22.012236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:22.013270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:22.030348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:22.032840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:22.033604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:22.035800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:22.036519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:22.038755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:22.041620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.server_external_events [req-7bc39085-4128-4eb2-b12a-ad30b68dee6e req-8b225b1f-2d8a-4f16-a31f-82a5684311d3 service nova] Unable to find a host for instance 08babbce-373e-4cf5-a138-066b6eeb02c3. Dropping event network-vif-deleted {{(pid=145969) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 27 10:09:22.042581 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7bc39085-4128-4eb2-b12a-ad30b68dee6e req-8b225b1f-2d8a-4f16-a31f-82a5684311d3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 207 len: 180 microversion: 2.1 time: 0.120156 Sep 27 10:09:22.043199 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1483/3277] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:21 2022] POST /compute/v2.1/os-server-external-events => generated 180 bytes in 121 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Sep 27 10:09:22.080621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:22.082946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:22.113881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:22.114575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:22.117316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:22.122846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:22.123570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:22.124034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:22.205982 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 HTTP/1.1 Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '08babbce-373e-4cf5-a138-066b6eeb02c3'} Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:22.207508 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.212417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:22.212417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:22.212417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:22.212417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:09:22.212417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:09:22.212417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:09:22.212417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.212417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:22.222994 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f9961189-46f3-484e-bab5-cb018d3ee09a tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 404 len: 111 microversion: 2.1 time: 0.163446 Sep 27 10:09:22.225479 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1484/3278] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:22 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 111 bytes in 168 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:09:22.249695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0883c2ac-0c0f-42d0-8906-1937fbb9fef5 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:22.250324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0883c2ac-0c0f-42d0-8906-1937fbb9fef5 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:22.285715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0883c2ac-0c0f-42d0-8906-1937fbb9fef5 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:22.286238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0883c2ac-0c0f-42d0-8906-1937fbb9fef5 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:22.286688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0883c2ac-0c0f-42d0-8906-1937fbb9fef5 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:22.287426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0883c2ac-0c0f-42d0-8906-1937fbb9fef5 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:22.292345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0883c2ac-0c0f-42d0-8906-1937fbb9fef5 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:22.292914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0883c2ac-0c0f-42d0-8906-1937fbb9fef5 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:22.403025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:22.419245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:22.422928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:22.431941 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9624cb0a-f5ac-4736-9b5b-711d5a4c0325 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1336 microversion: 2.54 time: 0.424297 Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1795/3279] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:22 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1336 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6fb64ab1-6962-44a8-b3e5-77cc9335e728 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6fb64ab1-6962-44a8-b3e5-77cc9335e728 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Request method failure captured: Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers?changes-since=2011%2F01%2F01 HTTP/1.1 Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsuj15rCUD1970qSbzxB3cw4sTAfA1YtNh4VnoapFvuP9GffqVpFgRGMeWDK-Po0RARJzDlkwtHZJxI_lb7FISD34bP5tmzbtvUeP246046Go7rEPdSKK7iPObXyQaBpwtLG8VaeId4vMcWe5OB5uDdCNIPuVd0zCfFs93HwTWDEgHo Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_79022e5ca5764576b4feb7f4cadba80c"}]}] Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 2699ef01c0eb4d5783203e6b41293d97 Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: action_args: {} Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '2011/01/01' is not a 'date-time' Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'format' in schema['properties']['changes-since']['items']: Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'format': 'date-time', 'type': 'string'} Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance[0]: Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi '2011/01/01' Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:22.444775 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 181, in wrapper Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi if _schema_validation_helper(query_params_schema, Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6fb64ab1-6962-44a8-b3e5-77cc9335e728 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Returning 400 to user: Invalid input for query parameters changes-since. Value: 2011/01/01. '2011/01/01' is not a 'date-time' {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6fb64ab1-6962-44a8-b3e5-77cc9335e728 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers?changes-since=2011%2F01%2F01" status: 400 len: 146 microversion: 2.1 time: 0.012440 Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1796/3280] 10.222.0.26 () {58 vars in 1182 bytes} [Tue Sep 27 10:09:22 2022] GET /compute/v2.1/servers?changes-since=2011%2F01%2F01 => generated 146 bytes in 14 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ff398fe0-822c-4c54-8191-39e5d4f3052e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:22.457824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ff398fe0-822c-4c54-8191-39e5d4f3052e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:22.479780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff398fe0-822c-4c54-8191-39e5d4f3052e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:22.481769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff398fe0-822c-4c54-8191-39e5d4f3052e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:22.482493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff398fe0-822c-4c54-8191-39e5d4f3052e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:22.483629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff398fe0-822c-4c54-8191-39e5d4f3052e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:22.484704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff398fe0-822c-4c54-8191-39e5d4f3052e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:22.488322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff398fe0-822c-4c54-8191-39e5d4f3052e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:22.718867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0883c2ac-0c0f-42d0-8906-1937fbb9fef5 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:22.733638 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0883c2ac-0c0f-42d0-8906-1937fbb9fef5 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1470 microversion: 2.1 time: 0.506047 Sep 27 10:09:22.737966 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1485/3281] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:22 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1470 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:22.764782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5eff117d-a9bd-43ab-b823-cdd61900dd11 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:22.798174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5eff117d-a9bd-43ab-b823-cdd61900dd11 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:22.800991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5eff117d-a9bd-43ab-b823-cdd61900dd11 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Searching by: {'deleted': False, 'project_id': '79022e5ca5764576b4feb7f4cadba80c'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:09:22.809577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ff398fe0-822c-4c54-8191-39e5d4f3052e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:22.819757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5eff117d-a9bd-43ab-b823-cdd61900dd11 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:22.822963 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ff398fe0-822c-4c54-8191-39e5d4f3052e tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1351 microversion: 2.1 time: 0.369810 Sep 27 10:09:22.823222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5eff117d-a9bd-43ab-b823-cdd61900dd11 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:22.823991 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1797/3282] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:22 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1351 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:22.825522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5eff117d-a9bd-43ab-b823-cdd61900dd11 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:22.827499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5eff117d-a9bd-43ab-b823-cdd61900dd11 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:22.829601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5eff117d-a9bd-43ab-b823-cdd61900dd11 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:22.830327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5eff117d-a9bd-43ab-b823-cdd61900dd11 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:22.834187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1082922274", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "0117c46b-39f2-4cf5-9f8a-7710125bdf55"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:22.938606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-5eff117d-a9bd-43ab-b823-cdd61900dd11 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:09:22.944399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] validate_networks() for [('0117c46b-39f2-4cf5-9f8a-7710125bdf55', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:09:22.981708 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5eff117d-a9bd-43ab-b823-cdd61900dd11 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 681 microversion: 2.1 time: 0.241451 Sep 27 10:09:22.983102 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1486/3283] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:09:22 2022] GET /compute/v2.1/servers => generated 681 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:22.988282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-c09d032b-9b38-43bf-8201-e1c1e1e67bfb req-58efef0d-76d0-4e9e-8701-7b7301f94ad6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9ac32528-351d-4ba2-b0ec-0eeb4872e95c", "name": "network-vif-plugged", "status": "completed", "tag": "6f38bb60-56d7-44ce-ad27-ae8c1e3ac0fb"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:23.004246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c09d032b-9b38-43bf-8201-e1c1e1e67bfb req-58efef0d-76d0-4e9e-8701-7b7301f94ad6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:23.005086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c09d032b-9b38-43bf-8201-e1c1e1e67bfb req-58efef0d-76d0-4e9e-8701-7b7301f94ad6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:23.005759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c09d032b-9b38-43bf-8201-e1c1e1e67bfb req-58efef0d-76d0-4e9e-8701-7b7301f94ad6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:23.020790 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c09d032b-9b38-43bf-8201-e1c1e1e67bfb req-58efef0d-76d0-4e9e-8701-7b7301f94ad6 service nova] Creating event network-vif-plugged:6f38bb60-56d7-44ce-ad27-ae8c1e3ac0fb for instance 9ac32528-351d-4ba2-b0ec-0eeb4872e95c on nch2-master-1 Sep 27 10:09:23.026855 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-c09d032b-9b38-43bf-8201-e1c1e1e67bfb req-58efef0d-76d0-4e9e-8701-7b7301f94ad6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042108 Sep 27 10:09:23.027908 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1487/3284] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:23.033634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25303b27-7773-4b23-a2c1-34f9567ca67b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:23.034759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-25303b27-7773-4b23-a2c1-34f9567ca67b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:23.041980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25303b27-7773-4b23-a2c1-34f9567ca67b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:23.042892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25303b27-7773-4b23-a2c1-34f9567ca67b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:23.043540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25303b27-7773-4b23-a2c1-34f9567ca67b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:23.044482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25303b27-7773-4b23-a2c1-34f9567ca67b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:23.045175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25303b27-7773-4b23-a2c1-34f9567ca67b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:23.045833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25303b27-7773-4b23-a2c1-34f9567ca67b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:23.518358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:23.668025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-25303b27-7773-4b23-a2c1-34f9567ca67b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:23.679919 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-25303b27-7773-4b23-a2c1-34f9567ca67b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.650295 Sep 27 10:09:23.680866 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1488/3285] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:23 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:23.695976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:23.704546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:23.704933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.servers [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Removing options 'limit' from query {{(pid=145969) remove_invalid_options /opt/stack/nova/nova/api/openstack/compute/servers.py:1490}} Sep 27 10:09:23.705930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Searching by: {'deleted': False, 'project_id': '79022e5ca5764576b4feb7f4cadba80c'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:09:23.712495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:23.713054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:23.713479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:23.715124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:23.715865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:23.716536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:23.748326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Listed batch of 2 results from cell out of 102 limit. Returned 2 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:09:23.776230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:23.785938 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-472f838d-09c1-4ccd-8a92-b3d5e881b462 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers?limit=102" status: 200 len: 681 microversion: 2.1 time: 0.102341 Sep 27 10:09:23.795581 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1489/3286] 10.222.0.26 () {58 vars in 1144 bytes} [Tue Sep 27 10:09:23 2022] GET /compute/v2.1/servers?limit=102 => generated 681 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:23.802243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:23.802858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:23.813131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:23.813690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:23.814206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:23.815134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:23.816969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:23.817598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:23.982736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:24.008555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:24.014677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:24.018742 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-84727167-0acd-499f-b8e6-c51029ab467a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1336 microversion: 2.54 time: 0.221164 Sep 27 10:09:24.021762 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1490/3287] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:23 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1336 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:24.036025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b29922e3-d0c6-4396-87f1-e042b0eff935 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:24.037518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b29922e3-d0c6-4396-87f1-e042b0eff935 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:24.060630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b29922e3-d0c6-4396-87f1-e042b0eff935 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:24.062519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b29922e3-d0c6-4396-87f1-e042b0eff935 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:24.064472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b29922e3-d0c6-4396-87f1-e042b0eff935 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:24.067781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b29922e3-d0c6-4396-87f1-e042b0eff935 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:24.069717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b29922e3-d0c6-4396-87f1-e042b0eff935 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:24.071567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b29922e3-d0c6-4396-87f1-e042b0eff935 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:24.416897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b29922e3-d0c6-4396-87f1-e042b0eff935 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:24.430437 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b29922e3-d0c6-4396-87f1-e042b0eff935 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1470 microversion: 2.1 time: 0.406807 Sep 27 10:09:24.433398 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1491/3288] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:24 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1470 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:24.462005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bfc4e59d-b3d7-4f7c-be67-e786743f1ee9 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bfc4e59d-b3d7-4f7c-be67-e786743f1ee9 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Request method failure captured: Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers?limit=-1 HTTP/1.1 Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsuj15rCUD1970qSbzxB3cw4sTAfA1YtNh4VnoapFvuP9GffqVpFgRGMeWDK-Po0RARJzDlkwtHZJxI_lb7FISD34bP5tmzbtvUeP246046Go7rEPdSKK7iPObXyQaBpwtLG8VaeId4vMcWe5OB5uDdCNIPuVd0zCfFs93HwTWDEgHo Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_79022e5ca5764576b4feb7f4cadba80c"}]}] Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2699ef01c0eb4d5783203e6b41293d97 Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: action_args: {} Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '-1' does not match '^[0-9]*$' Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['limit']['items']: Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'minLength': 1, Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'minimum': 0, Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance[0]: Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi '-1' Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 181, in wrapper Sep 27 10:09:24.470249 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi if _schema_validation_helper(query_params_schema, Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' Sep 27 10:09:24.475467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:24.483130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bfc4e59d-b3d7-4f7c-be67-e786743f1ee9 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Returning 400 to user: Invalid input for query parameters limit. Value: -1. '-1' does not match '^[0-9]*$' {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:24.485944 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bfc4e59d-b3d7-4f7c-be67-e786743f1ee9 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers?limit=-1" status: 400 len: 127 microversion: 2.1 time: 0.031734 Sep 27 10:09:24.487713 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1492/3289] 10.222.0.26 () {58 vars in 1142 bytes} [Tue Sep 27 10:09:24 2022] GET /compute/v2.1/servers?limit=-1 => generated 127 bytes in 34 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:09:24.494588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-42ba563d-8eb5-4b4c-a74d-5c8e95d09200 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:24.497786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-42ba563d-8eb5-4b4c-a74d-5c8e95d09200 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:24.521327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42ba563d-8eb5-4b4c-a74d-5c8e95d09200 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:24.524579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42ba563d-8eb5-4b4c-a74d-5c8e95d09200 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:24.525315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42ba563d-8eb5-4b4c-a74d-5c8e95d09200 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:24.526273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42ba563d-8eb5-4b4c-a74d-5c8e95d09200 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:24.528465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42ba563d-8eb5-4b4c-a74d-5c8e95d09200 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:24.529141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42ba563d-8eb5-4b4c-a74d-5c8e95d09200 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:24.841508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:09:24.842878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:09:24.843608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:09:24.844256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:09:24.844945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:09:24.876472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:24.878751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:24.884320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:24.894035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:24.894754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:24.895270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:24.935704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:24.945227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for user 7f7c5adbac4f48b6bc141e0f888f2496 and project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:24.998723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:09:25.001399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:09:25.125870 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-06ef2c71-aa01-4012-8dc5-93508fe7e1c1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.300463 Sep 27 10:09:25.127339 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1798/3290] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:09:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 2302 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:09:25.136549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-42ba563d-8eb5-4b4c-a74d-5c8e95d09200 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:25.137136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fdd8bc74-33b1-4e6e-a563-45eab146d3e1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:25.184603 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-42ba563d-8eb5-4b4c-a74d-5c8e95d09200 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1351 microversion: 2.1 time: 0.695576 Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fdd8bc74-33b1-4e6e-a563-45eab146d3e1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Request method failure captured: Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers?limit=testing HTTP/1.1 Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsuj15rCUD1970qSbzxB3cw4sTAfA1YtNh4VnoapFvuP9GffqVpFgRGMeWDK-Po0RARJzDlkwtHZJxI_lb7FISD34bP5tmzbtvUeP246046Go7rEPdSKK7iPObXyQaBpwtLG8VaeId4vMcWe5OB5uDdCNIPuVd0zCfFs93HwTWDEgHo Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_79022e5ca5764576b4feb7f4cadba80c"}]}] Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 2699ef01c0eb4d5783203e6b41293d97 Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: action_args: {} Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'testing' does not match '^[0-9]*$' Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['limit']['items']: Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'minLength': 1, Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'minimum': 0, Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'pattern': '^[0-9]*$', Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'type': ['integer', 'string']} Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance[0]: Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'testing' Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 181, in wrapper Sep 27 10:09:25.186369 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi if _schema_validation_helper(query_params_schema, Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fdd8bc74-33b1-4e6e-a563-45eab146d3e1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Returning 400 to user: Invalid input for query parameters limit. Value: testing. 'testing' does not match '^[0-9]*$' {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:25.190909 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fdd8bc74-33b1-4e6e-a563-45eab146d3e1 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers?limit=testing" status: 400 len: 137 microversion: 2.1 time: 0.059622 Sep 27 10:09:25.192925 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1493/3291] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:24 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1351 bytes in 698 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:25.199869 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1799/3292] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 10:09:25 2022] GET /compute/v2.1/servers?limit=testing => generated 137 bytes in 72 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:09:25.210045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f945376f-d916-40e2-97c3-8f339509ca15 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:25.213862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:25.217800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f945376f-d916-40e2-97c3-8f339509ca15 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:25.223411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:25.255920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945376f-d916-40e2-97c3-8f339509ca15 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:25.256299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945376f-d916-40e2-97c3-8f339509ca15 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:25.256809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945376f-d916-40e2-97c3-8f339509ca15 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:25.261968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:25.266764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:25.267435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:25.268894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:25.269573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:25.270158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:25.275283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945376f-d916-40e2-97c3-8f339509ca15 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:25.277151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945376f-d916-40e2-97c3-8f339509ca15 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:25.277636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945376f-d916-40e2-97c3-8f339509ca15 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:25.557003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:25.582277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:25.588196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:25.589477 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-970903fb-5c0e-4b15-a8a8-0f862cdc566a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1357 microversion: 2.54 time: 0.387834 Sep 27 10:09:25.593885 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1800/3293] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:25 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1357 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:25.613539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b69396df-85e0-4b86-b17f-c4936849eb6b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:25.615681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b69396df-85e0-4b86-b17f-c4936849eb6b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:25.648517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f945376f-d916-40e2-97c3-8f339509ca15 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:25.710766 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f945376f-d916-40e2-97c3-8f339509ca15 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.509111 Sep 27 10:09:25.711480 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1494/3294] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:25 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:25.746288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6a7c7550-612a-4416-ad57-48f58b0703be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:25.769278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6a7c7550-612a-4416-ad57-48f58b0703be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:25.774204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6a7c7550-612a-4416-ad57-48f58b0703be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Searching by: {'flavor': 'non_existing_flavor', 'deleted': False, 'project_id': '79022e5ca5764576b4feb7f4cadba80c'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:09:25.792813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.servers [None req-6a7c7550-612a-4416-ad57-48f58b0703be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Flavor 'non_existing_flavor' could not be found {{(pid=145969) _get_servers /opt/stack/nova/nova/api/openstack/compute/servers.py:336}} Sep 27 10:09:25.812941 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6a7c7550-612a-4416-ad57-48f58b0703be tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers?flavor=non_existing_flavor" status: 200 len: 15 microversion: 2.1 time: 0.079800 Sep 27 10:09:25.813434 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1495/3295] 10.222.0.26 () {58 vars in 1178 bytes} [Tue Sep 27 10:09:25 2022] GET /compute/v2.1/servers?flavor=non_existing_flavor => generated 15 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:09:25.838137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-804aa5e5-c9d2-4eae-8275-f39e5d77f4e6 req-134eda3a-3e71-400b-87ef-e04b8113773c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d9bf371a-9052-4e46-b718-75e61fa47db3", "name": "network-vif-deleted", "tag": "3bdfdad8-1835-4508-a3f9-69be7eeab18d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:25.871598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-804aa5e5-c9d2-4eae-8275-f39e5d77f4e6 req-134eda3a-3e71-400b-87ef-e04b8113773c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:25.872019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-804aa5e5-c9d2-4eae-8275-f39e5d77f4e6 req-134eda3a-3e71-400b-87ef-e04b8113773c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:25.875464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-804aa5e5-c9d2-4eae-8275-f39e5d77f4e6 req-134eda3a-3e71-400b-87ef-e04b8113773c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:25.909042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b69396df-85e0-4b86-b17f-c4936849eb6b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:25.918308 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b69396df-85e0-4b86-b17f-c4936849eb6b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1215 microversion: 2.1 time: 0.322042 Sep 27 10:09:25.920029 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1801/3296] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:25 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1215 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:25.926749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:25.933607 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-804aa5e5-c9d2-4eae-8275-f39e5d77f4e6 req-134eda3a-3e71-400b-87ef-e04b8113773c service nova] Creating event network-vif-deleted:3bdfdad8-1835-4508-a3f9-69be7eeab18d for instance d9bf371a-9052-4e46-b718-75e61fa47db3 on nch1-master-1 Sep 27 10:09:25.943339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:25.959961 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-804aa5e5-c9d2-4eae-8275-f39e5d77f4e6 req-134eda3a-3e71-400b-87ef-e04b8113773c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.145230 Sep 27 10:09:25.963210 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1496/3297] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:25.971762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:25.977587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:25.978234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:25.979133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:25.981148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-82a719ec-f658-4efc-b1f6-4529beee0096 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Action: 'create', calling method: .version_select at 0x7f7f62adf5e0>, body: {"keypair": {"name": "tempest-ServerShowV254Test-keypair-1438412530", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDZCmpbyO+wQnCjSxcdk1tEWyYE7QRSUjaKRxAu5uWcj2kVkksR+MR8dHegp5QX2ZzTEOuzjT23iniR9oPaBWv52KGij3ri+8WR5xv3TqxbvHSpZhiBSGR3/MjNYqbD2NQ=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:25.982005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:25.983655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:26.097652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-82a719ec-f658-4efc-b1f6-4529beee0096 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Getting quotas for project 1f83c8276958489e84b0907523036097. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:26.104570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:26.108051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:26.110881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:26.113359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lazy-loading 'fault' on Instance uuid d9bf371a-9052-4e46-b718-75e61fa47db3 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:09:26.122560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-82a719ec-f658-4efc-b1f6-4529beee0096 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Getting quotas for user d8cc9d27cf6944aa9f5ef81d7fb3c428 and project 1f83c8276958489e84b0907523036097. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:26.221463 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-82a719ec-f658-4efc-b1f6-4529beee0096 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 420 microversion: 2.54 time: 0.256504 Sep 27 10:09:26.222850 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1497/3298] 10.222.0.26 () {62 vars in 1200 bytes} [Tue Sep 27 10:09:25 2022] POST /compute/v2.1/os-keypairs => generated 420 bytes in 258 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 27 10:09:26.230285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8646ddf0-d595-43db-a178-bc8755c30615 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:26.243968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8646ddf0-d595-43db-a178-bc8755c30615 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:26.244836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8646ddf0-d595-43db-a178-bc8755c30615 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Searching by: {'image': 'non_existing_image', 'deleted': False, 'project_id': '79022e5ca5764576b4feb7f4cadba80c'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:09:26.258595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8646ddf0-d595-43db-a178-bc8755c30615 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:26.259100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8646ddf0-d595-43db-a178-bc8755c30615 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:26.259525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8646ddf0-d595-43db-a178-bc8755c30615 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:26.260309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8646ddf0-d595-43db-a178-bc8755c30615 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:26.260802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8646ddf0-d595-43db-a178-bc8755c30615 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:26.261208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8646ddf0-d595-43db-a178-bc8755c30615 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:26.343609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:26.352231 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6317516d-11ae-4dd0-9dbe-d067b448275c tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 200 len: 1304 microversion: 2.1 time: 0.430275 Sep 27 10:09:26.358640 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1802/3299] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:25 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 1304 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:26.377566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6355e306-fee2-4246-bb97-d81e1d17782c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:26.379808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6355e306-fee2-4246-bb97-d81e1d17782c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:26.401359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6355e306-fee2-4246-bb97-d81e1d17782c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:26.410957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6355e306-fee2-4246-bb97-d81e1d17782c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:26.412293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6355e306-fee2-4246-bb97-d81e1d17782c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:26.421477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6355e306-fee2-4246-bb97-d81e1d17782c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:26.422454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6355e306-fee2-4246-bb97-d81e1d17782c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:26.442889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6355e306-fee2-4246-bb97-d81e1d17782c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:26.487077 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8646ddf0-d595-43db-a178-bc8755c30615 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers?image=non_existing_image" status: 200 len: 15 microversion: 2.1 time: 0.262650 Sep 27 10:09:26.491287 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1498/3300] 10.222.0.26 () {58 vars in 1174 bytes} [Tue Sep 27 10:09:26 2022] GET /compute/v2.1/servers?image=non_existing_image => generated 15 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:09:26.500476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Action: 'action', calling method: >, body: {"rebuild": {"key_name": "tempest-ServerShowV254Test-keypair-1438412530", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:26.505886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:26.513956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:26.514988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:26.515772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:26.516688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:26.517332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:26.517989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:26.692786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:09:26.693378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:09:26.694191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:09:26.696811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:09:26.697426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:09:26.937770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:26.950603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:26.950603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:26.951080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:26.953631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:26.954103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:26.954539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:27.023469 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a08f956f-3bc9-4bb7-b2fe-0fa08b834f26 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "POST /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2/action" status: 202 len: 1140 microversion: 2.54 time: 0.529806 Sep 27 10:09:27.031727 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1499/3301] 10.222.0.26 () {62 vars in 1320 bytes} [Tue Sep 27 10:09:26 2022] POST /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2/action => generated 1140 bytes in 539 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Sep 27 10:09:27.040581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-224ff665-1dc4-4bdd-8f14-fffbc7518ef3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:27.051855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-224ff665-1dc4-4bdd-8f14-fffbc7518ef3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:27.057854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-224ff665-1dc4-4bdd-8f14-fffbc7518ef3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Searching by: {'name': 'non_existing_server_name', 'deleted': False, 'project_id': '79022e5ca5764576b4feb7f4cadba80c'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:09:27.067394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-224ff665-1dc4-4bdd-8f14-fffbc7518ef3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:27.068068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-224ff665-1dc4-4bdd-8f14-fffbc7518ef3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:27.068625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-224ff665-1dc4-4bdd-8f14-fffbc7518ef3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:27.070026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-224ff665-1dc4-4bdd-8f14-fffbc7518ef3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:27.070856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-224ff665-1dc4-4bdd-8f14-fffbc7518ef3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:27.071407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-224ff665-1dc4-4bdd-8f14-fffbc7518ef3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:27.097232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6355e306-fee2-4246-bb97-d81e1d17782c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:27.114160 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6355e306-fee2-4246-bb97-d81e1d17782c tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1540 microversion: 2.1 time: 0.747524 Sep 27 10:09:27.117710 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1803/3302] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:26 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1540 bytes in 756 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:27.147910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1782a8f5-65c7-4136-a8da-8761a9f8a868 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:27.149555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1782a8f5-65c7-4136-a8da-8761a9f8a868 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:27.168967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1782a8f5-65c7-4136-a8da-8761a9f8a868 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:27.169478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1782a8f5-65c7-4136-a8da-8761a9f8a868 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:27.171083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1782a8f5-65c7-4136-a8da-8761a9f8a868 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:27.171862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1782a8f5-65c7-4136-a8da-8761a9f8a868 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:27.172320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1782a8f5-65c7-4136-a8da-8761a9f8a868 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:27.174958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1782a8f5-65c7-4136-a8da-8761a9f8a868 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:27.222352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=78,nova_cell1:UPDATE=17,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1,nova_cell1:DELETE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:27.260085 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-224ff665-1dc4-4bdd-8f14-fffbc7518ef3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers?name=non_existing_server_name" status: 200 len: 15 microversion: 2.1 time: 0.226502 Sep 27 10:09:27.261284 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1500/3303] 10.222.0.26 () {58 vars in 1184 bytes} [Tue Sep 27 10:09:27 2022] GET /compute/v2.1/servers?name=non_existing_server_name => generated 15 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:09:27.267814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-44424969-5046-438b-90af-56664e1a444f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:27.268442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-44424969-5046-438b-90af-56664e1a444f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:27.276429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44424969-5046-438b-90af-56664e1a444f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:27.277069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44424969-5046-438b-90af-56664e1a444f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:27.277724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44424969-5046-438b-90af-56664e1a444f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:27.279844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44424969-5046-438b-90af-56664e1a444f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:27.280531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44424969-5046-438b-90af-56664e1a444f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:27.281059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44424969-5046-438b-90af-56664e1a444f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:27.460787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-44424969-5046-438b-90af-56664e1a444f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:27.480642 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-44424969-5046-438b-90af-56664e1a444f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1267 microversion: 2.1 time: 0.216851 Sep 27 10:09:27.483370 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1501/3304] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:27 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1267 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:27.496412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:27.500243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:27.526000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:27.527634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:27.528353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:27.531096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:27.532053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:27.532816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:27.860466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:27.884786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:27.888498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:27.898356 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bf32ae8a-974d-4054-a1d4-045b3eb891ce tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.413500 Sep 27 10:09:27.899695 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1502/3305] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:27 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:27.904813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a624a0b-75c1-42a3-9e7f-98607ec17a5d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:27.908472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a624a0b-75c1-42a3-9e7f-98607ec17a5d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:27.926396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a624a0b-75c1-42a3-9e7f-98607ec17a5d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:27.927075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a624a0b-75c1-42a3-9e7f-98607ec17a5d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:27.927694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a624a0b-75c1-42a3-9e7f-98607ec17a5d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:27.928588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a624a0b-75c1-42a3-9e7f-98607ec17a5d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:27.929524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a624a0b-75c1-42a3-9e7f-98607ec17a5d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:27.939999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a624a0b-75c1-42a3-9e7f-98607ec17a5d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:28.112644 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a624a0b-75c1-42a3-9e7f-98607ec17a5d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.211355 Sep 27 10:09:28.120961 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1503/3306] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:09:27 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c/os-interface => generated 284 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:28.120961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:28.136922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:28.137289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Searching by: {'deleted': False, 'project_id': '79022e5ca5764576b4feb7f4cadba80c'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:09:28.152393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:28.152951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:28.153395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:28.154159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:28.154620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:28.155400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:28.213781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1782a8f5-65c7-4136-a8da-8761a9f8a868 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:28.231146 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1782a8f5-65c7-4136-a8da-8761a9f8a868 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 1.105181 Sep 27 10:09:28.236014 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1804/3307] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:27 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 1118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:28.248738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:09:28.251013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:28.257143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: d9bf371a-9052-4e46-b718-75e61fa47db3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:28.280073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:28.289160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:28.293464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:28.294583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:28.295299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:28.295982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:28.329379 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] HTTP exception thrown: Instance d9bf371a-9052-4e46-b718-75e61fa47db3 could not be found. Sep 27 10:09:28.331961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Request method failure captured: Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 HTTP/1.1 Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMstY7pvZq0cwMNYMMVADM0gWwtU_Ctmt8D3P6WO9OPmRX7a6zhma5oX4hGPRVJhWPCRMjjMBtQe3DrNvI4aKKTceFl2eCz0eUV7imwxj3WQAsdS4VavncEa0g3-vfbBrzWhsnamQA0g-aTtmbyfHvm91gTZTbTU_45aI7C4p4PszrLQ Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 52efcec2c8a14640a8c3df1dc576bbf1 Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersAdminTestJSON-568530853 Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/52efcec2c8a14640a8c3df1dc576bbf1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/52efcec2c8a14640a8c3df1dc576bbf1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/52efcec2c8a14640a8c3df1dc576bbf1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_52efcec2c8a14640a8c3df1dc576bbf1"}]}] Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersAdminTestJSON-568530853 Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 52efcec2c8a14640a8c3df1dc576bbf1 Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersAdminTestJSON-568530853 Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersAdminTestJSON-568530853-project Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b4e0f20e52074142a3b525bafbb30c97 Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersAdminTestJSON-568530853-project Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance d9bf371a-9052-4e46-b718-75e61fa47db3 could not be found. Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'd9bf371a-9052-4e46-b718-75e61fa47db3'} Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d9bf371a-9052-4e46-b718-75e61fa47db3 could not be found. Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d9bf371a-9052-4e46-b718-75e61fa47db3 could not be found. Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:28.338801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:28.347335 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:28.347335 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:28.347335 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:28.347335 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:09:28.347335 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:09:28.347335 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d9bf371a-9052-4e46-b718-75e61fa47db3 could not be found. Sep 27 10:09:28.347335 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:28.348709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:28.348709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:28.348709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:28.351680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Returning 404 to user: Instance d9bf371a-9052-4e46-b718-75e61fa47db3 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:28.353665 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3f8517a1-eb83-4e5b-8693-e7f5ab753769 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3" status: 404 len: 111 microversion: 2.1 time: 0.112754 Sep 27 10:09:28.354938 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1805/3308] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:28 2022] GET /compute/v2.1/servers/d9bf371a-9052-4e46-b718-75e61fa47db3 => generated 111 bytes in 118 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:09:28.376998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86cb6191-94b5-4c97-adde-074e60fe5a31 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:28.381733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86cb6191-94b5-4c97-adde-074e60fe5a31 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:28.405134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86cb6191-94b5-4c97-adde-074e60fe5a31 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:28.406223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86cb6191-94b5-4c97-adde-074e60fe5a31 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:28.411764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86cb6191-94b5-4c97-adde-074e60fe5a31 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:28.414639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86cb6191-94b5-4c97-adde-074e60fe5a31 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:28.415524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86cb6191-94b5-4c97-adde-074e60fe5a31 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:28.421524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86cb6191-94b5-4c97-adde-074e60fe5a31 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:28.483934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86cb6191-94b5-4c97-adde-074e60fe5a31 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:09:28.681150 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86cb6191-94b5-4c97-adde-074e60fe5a31 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "DELETE /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 204 len: 0 microversion: 2.1 time: 0.321225 Sep 27 10:09:28.681954 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1806/3309] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:09:28 2022] DELETE /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 0 bytes in 324 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:09:28.698428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-575c7ae8-2b4b-4f21-b9b4-87ca41762af2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:28.699952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-575c7ae8-2b4b-4f21-b9b4-87ca41762af2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:28.715558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-575c7ae8-2b4b-4f21-b9b4-87ca41762af2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:28.717545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-575c7ae8-2b4b-4f21-b9b4-87ca41762af2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:28.718949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-575c7ae8-2b4b-4f21-b9b4-87ca41762af2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:28.722013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-575c7ae8-2b4b-4f21-b9b4-87ca41762af2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:28.722760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-575c7ae8-2b4b-4f21-b9b4-87ca41762af2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:28.723454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-575c7ae8-2b4b-4f21-b9b4-87ca41762af2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:29.010084 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0061fdce-d7db-4785-a867-ae6500b4134f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/detail" status: 200 len: 3023 microversion: 2.1 time: 0.894201 Sep 27 10:09:29.010688 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1504/3310] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:09:28 2022] GET /compute/v2.1/servers/detail => generated 3023 bytes in 896 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:29.016531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-61640e2c-f7be-42e9-84e0-ab5879919406 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:29.017675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-61640e2c-f7be-42e9-84e0-ab5879919406 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:29.037094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61640e2c-f7be-42e9-84e0-ab5879919406 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:29.043043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61640e2c-f7be-42e9-84e0-ab5879919406 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:29.043765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61640e2c-f7be-42e9-84e0-ab5879919406 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:29.044628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61640e2c-f7be-42e9-84e0-ab5879919406 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:29.052866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61640e2c-f7be-42e9-84e0-ab5879919406 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:29.054382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61640e2c-f7be-42e9-84e0-ab5879919406 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:29.090539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-575c7ae8-2b4b-4f21-b9b4-87ca41762af2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:29.103906 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-575c7ae8-2b4b-4f21-b9b4-87ca41762af2 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1273 microversion: 2.1 time: 0.418730 Sep 27 10:09:29.110591 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1807/3311] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:28 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1273 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:29.131028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:29.131992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:29.155410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:29.156738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:29.157387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:29.158305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:29.160163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:29.160841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:29.384650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:29.394695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:29.396705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:29.398508 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f372c7a6-238c-43c1-9b04-4a0ea154b6a7 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.278868 Sep 27 10:09:29.400617 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1808/3312] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:29 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:29.406332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3e14396d-abec-49c5-8373-389686e69c8c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:29.411601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3e14396d-abec-49c5-8373-389686e69c8c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:29.412973 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3e14396d-abec-49c5-8373-389686e69c8c tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers?status=non_existing_status" status: 200 len: 15 microversion: 2.1 time: 0.010639 Sep 27 10:09:29.421646 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1809/3313] 10.222.0.26 () {58 vars in 1178 bytes} [Tue Sep 27 10:09:29 2022] GET /compute/v2.1/servers?status=non_existing_status => generated 15 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:09:29.427473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7db29c13-e49e-44ed-9dd8-3c5d13e92410 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:29.428800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7db29c13-e49e-44ed-9dd8-3c5d13e92410 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:29.437560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7db29c13-e49e-44ed-9dd8-3c5d13e92410 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:29.438285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7db29c13-e49e-44ed-9dd8-3c5d13e92410 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:29.438959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7db29c13-e49e-44ed-9dd8-3c5d13e92410 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:29.442126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7db29c13-e49e-44ed-9dd8-3c5d13e92410 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:29.442885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7db29c13-e49e-44ed-9dd8-3c5d13e92410 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:29.444206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7db29c13-e49e-44ed-9dd8-3c5d13e92410 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:29.842297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-61640e2c-f7be-42e9-84e0-ab5879919406 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:29.856905 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-61640e2c-f7be-42e9-84e0-ab5879919406 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1468 microversion: 2.1 time: 0.844784 Sep 27 10:09:29.861348 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1505/3314] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:29 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1468 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:29.879010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a368c616-57f7-453b-970c-7777cf62d210 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:29.895695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a368c616-57f7-453b-970c-7777cf62d210 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:29.897990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a368c616-57f7-453b-970c-7777cf62d210 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Searching by: {'deleted': False, 'project_id': '79022e5ca5764576b4feb7f4cadba80c'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:09:29.913215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a368c616-57f7-453b-970c-7777cf62d210 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:29.916864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a368c616-57f7-453b-970c-7777cf62d210 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:29.917764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a368c616-57f7-453b-970c-7777cf62d210 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:29.919118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a368c616-57f7-453b-970c-7777cf62d210 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:29.922026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a368c616-57f7-453b-970c-7777cf62d210 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:29.922945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a368c616-57f7-453b-970c-7777cf62d210 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:29.953250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=17 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:29.999026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-a368c616-57f7-453b-970c-7777cf62d210 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:09:30.067414 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a368c616-57f7-453b-970c-7777cf62d210 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 681 microversion: 2.1 time: 0.204145 Sep 27 10:09:30.069143 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1506/3315] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:09:29 2022] GET /compute/v2.1/servers => generated 681 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:30.088093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:30.089054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:30.101945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.103430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.105222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.107592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.108183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.108743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.118692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7db29c13-e49e-44ed-9dd8-3c5d13e92410 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:30.131418 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7db29c13-e49e-44ed-9dd8-3c5d13e92410 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.708336 Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] HTTP exception thrown: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Request method failure captured: Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd HTTP/1.1 Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsuj15rCUD1970qSbzxB3cw4sTAfA1YtNh4VnoapFvuP9GffqVpFgRGMeWDK-Po0RARJzDlkwtHZJxI_lb7FISD34bP5tmzbtvUeP246046Go7rEPdSKK7iPObXyQaBpwtLG8VaeId4vMcWe5OB5uDdCNIPuVd0zCfFs93HwTWDEgHo Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_79022e5ca5764576b4feb7f4cadba80c"}]}] Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2699ef01c0eb4d5783203e6b41293d97 Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd'} Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:09:30.154845 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Returning 404 to user: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-54440c0f-8012-40bc-b082-592b0ecde733 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "DELETE /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd" status: 404 len: 111 microversion: 2.1 time: 0.071919 Sep 27 10:09:30.164196 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1507/3316] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:09:30 2022] DELETE /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd => generated 111 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:09:30.166576 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1810/3317] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:29 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 744 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:30.169632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0721a619-b7e6-42b6-bea2-204dd313f043 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:30.172131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-375630ef-a906-4f8a-b528-cf74c2a0837f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:30.172916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-375630ef-a906-4f8a-b528-cf74c2a0837f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 4fed875e-5be1-4b37-a0f9-55e6bc6fca05] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:30.173438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0721a619-b7e6-42b6-bea2-204dd313f043 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:30.180544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-375630ef-a906-4f8a-b528-cf74c2a0837f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.181492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-375630ef-a906-4f8a-b528-cf74c2a0837f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.182147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-375630ef-a906-4f8a-b528-cf74c2a0837f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.183249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-375630ef-a906-4f8a-b528-cf74c2a0837f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.183999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-375630ef-a906-4f8a-b528-cf74c2a0837f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.184624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-375630ef-a906-4f8a-b528-cf74c2a0837f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.188346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0721a619-b7e6-42b6-bea2-204dd313f043 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.192068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0721a619-b7e6-42b6-bea2-204dd313f043 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.192068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0721a619-b7e6-42b6-bea2-204dd313f043 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.193857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0721a619-b7e6-42b6-bea2-204dd313f043 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.194469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0721a619-b7e6-42b6-bea2-204dd313f043 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.196493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0721a619-b7e6-42b6-bea2-204dd313f043 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.218310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-375630ef-a906-4f8a-b528-cf74c2a0837f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 4fed875e-5be1-4b37-a0f9-55e6bc6fca05] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:09:30.372040 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-375630ef-a906-4f8a-b528-cf74c2a0837f tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05" status: 204 len: 0 microversion: 2.1 time: 0.203005 Sep 27 10:09:30.373363 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1811/3318] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:09:30 2022] DELETE /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:09:30.385311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c48d4c6d-19c1-44b9-9d69-8438c17cd3b3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:30.406850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c48d4c6d-19c1-44b9-9d69-8438c17cd3b3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:30.414797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d4c6d-19c1-44b9-9d69-8438c17cd3b3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.415605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d4c6d-19c1-44b9-9d69-8438c17cd3b3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.416207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d4c6d-19c1-44b9-9d69-8438c17cd3b3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.417138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d4c6d-19c1-44b9-9d69-8438c17cd3b3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.417794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d4c6d-19c1-44b9-9d69-8438c17cd3b3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.418701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48d4c6d-19c1-44b9-9d69-8438c17cd3b3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.482449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c48d4c6d-19c1-44b9-9d69-8438c17cd3b3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:09:30.722183 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c48d4c6d-19c1-44b9-9d69-8438c17cd3b3 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 204 len: 0 microversion: 2.1 time: 0.341426 Sep 27 10:09:30.729022 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1812/3319] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:09:30 2022] DELETE /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 0 bytes in 349 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:09:30.741983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:30.759223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:30.782963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.792446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.794617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.795627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.802360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.803151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.918479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0721a619-b7e6-42b6-bea2-204dd313f043 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:30.932733 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0721a619-b7e6-42b6-bea2-204dd313f043 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1273 microversion: 2.1 time: 0.772555 Sep 27 10:09:30.935604 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1508/3320] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:30 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1273 bytes in 777 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:30.950629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:30.951532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:30.974885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.978173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.980522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.982747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:30.983645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:30.985009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:30.995976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:31.027438 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] HTTP exception thrown: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Request method failure captured: Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd HTTP/1.1 Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsuj15rCUD1970qSbzxB3cw4sTAfA1YtNh4VnoapFvuP9GffqVpFgRGMeWDK-Po0RARJzDlkwtHZJxI_lb7FISD34bP5tmzbtvUeP246046Go7rEPdSKK7iPObXyQaBpwtLG8VaeId4vMcWe5OB5uDdCNIPuVd0zCfFs93HwTWDEgHo Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_79022e5ca5764576b4feb7f4cadba80c"}]}] Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2699ef01c0eb4d5783203e6b41293d97 Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd'} Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:31.033143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:31.038443 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:31.038443 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:31.038443 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:31.038443 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:31.038443 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:09:31.038443 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:09:31.038443 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. Sep 27 10:09:31.038443 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:31.043726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Returning 404 to user: Instance 2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:31.044957 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1fcb8b6c-9e22-48e2-8378-6939b6519ded tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd" status: 404 len: 111 microversion: 2.1 time: 0.107707 Sep 27 10:09:31.045740 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1509/3321] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:30 2022] GET /compute/v2.1/servers/2dde3c32-23b6-4cd1-87de-e2d5bbfc5bbd => generated 111 bytes in 109 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:09:31.062555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8b1856e5-cfdc-4455-8ccb-6cee8fc60130 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:31.063484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8b1856e5-cfdc-4455-8ccb-6cee8fc60130 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:31.078509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:31.081959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b1856e5-cfdc-4455-8ccb-6cee8fc60130 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:31.082829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b1856e5-cfdc-4455-8ccb-6cee8fc60130 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:31.084973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b1856e5-cfdc-4455-8ccb-6cee8fc60130 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:31.086788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b1856e5-cfdc-4455-8ccb-6cee8fc60130 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:31.090062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b1856e5-cfdc-4455-8ccb-6cee8fc60130 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:31.090723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b1856e5-cfdc-4455-8ccb-6cee8fc60130 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:31.094452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:31.101320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:31.104326 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2fe41a3a-caa8-4c55-8ca5-d5ffbd7ab9c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.373790 Sep 27 10:09:31.107446 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1813/3322] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:30 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:31.117148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e4ed2893-4e33-4927-a9f2-bb78bb783aa8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:31.118467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e4ed2893-4e33-4927-a9f2-bb78bb783aa8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 4fed875e-5be1-4b37-a0f9-55e6bc6fca05] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:31.144287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4ed2893-4e33-4927-a9f2-bb78bb783aa8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:31.145102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4ed2893-4e33-4927-a9f2-bb78bb783aa8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:31.147449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4ed2893-4e33-4927-a9f2-bb78bb783aa8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:31.148830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4ed2893-4e33-4927-a9f2-bb78bb783aa8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:31.150633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4ed2893-4e33-4927-a9f2-bb78bb783aa8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:31.153087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4ed2893-4e33-4927-a9f2-bb78bb783aa8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:31.546598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e4ed2893-4e33-4927-a9f2-bb78bb783aa8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:31.575580 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e4ed2893-4e33-4927-a9f2-bb78bb783aa8 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05" status: 200 len: 1521 microversion: 2.1 time: 0.466906 Sep 27 10:09:31.596907 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1814/3323] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:31 2022] GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05 => generated 1521 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:31.596907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-117efe6c-1915-4227-949c-6619b3dae946 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:31.596907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-117efe6c-1915-4227-949c-6619b3dae946 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:31.632882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117efe6c-1915-4227-949c-6619b3dae946 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:31.655166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117efe6c-1915-4227-949c-6619b3dae946 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.022s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:31.658255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117efe6c-1915-4227-949c-6619b3dae946 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:31.667896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117efe6c-1915-4227-949c-6619b3dae946 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:31.672991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117efe6c-1915-4227-949c-6619b3dae946 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:31.673462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117efe6c-1915-4227-949c-6619b3dae946 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:32.460834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8b1856e5-cfdc-4455-8ccb-6cee8fc60130 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:32.477494 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8b1856e5-cfdc-4455-8ccb-6cee8fc60130 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1468 microversion: 2.1 time: 1.429724 Sep 27 10:09:32.478095 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1510/3324] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:31 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1468 bytes in 1431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:32.498350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a5020070-6af6-4af7-a772-2fb8669ab767 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:32.499316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a5020070-6af6-4af7-a772-2fb8669ab767 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:32.533621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5020070-6af6-4af7-a772-2fb8669ab767 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:32.545508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5020070-6af6-4af7-a772-2fb8669ab767 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:32.545965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5020070-6af6-4af7-a772-2fb8669ab767 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:32.546731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5020070-6af6-4af7-a772-2fb8669ab767 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:32.547190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5020070-6af6-4af7-a772-2fb8669ab767 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:32.547616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5020070-6af6-4af7-a772-2fb8669ab767 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:32.568867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-117efe6c-1915-4227-949c-6619b3dae946 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:32.588123 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-117efe6c-1915-4227-949c-6619b3dae946 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 1.004747 Sep 27 10:09:32.618808 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1815/3325] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:31 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 1036 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:32.625107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:32.637077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:32.654805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:32.655309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:32.655818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:32.656553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:32.657030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:32.662685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:32.884757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:32.899015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:32.910471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:32.917866 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-563ae79f-17ab-4e02-b4a8-db72205b4888 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.296834 Sep 27 10:09:32.930877 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1816/3326] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:32 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:32.949296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c3d8124c-8d86-4875-928c-c380adc6e15d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:32.956398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c3d8124c-8d86-4875-928c-c380adc6e15d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 4fed875e-5be1-4b37-a0f9-55e6bc6fca05] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:32.974840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3d8124c-8d86-4875-928c-c380adc6e15d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:32.977301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3d8124c-8d86-4875-928c-c380adc6e15d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:32.981359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3d8124c-8d86-4875-928c-c380adc6e15d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:32.985630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3d8124c-8d86-4875-928c-c380adc6e15d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:32.986412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3d8124c-8d86-4875-928c-c380adc6e15d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:32.987117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3d8124c-8d86-4875-928c-c380adc6e15d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:33.115836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a5020070-6af6-4af7-a772-2fb8669ab767 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:33.132007 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a5020070-6af6-4af7-a772-2fb8669ab767 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 0.652518 Sep 27 10:09:33.144435 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1511/3327] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:32 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 665 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:33.502982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6460ae55-5a67-46ae-b751-e20f2eb419db tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:33.506650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6460ae55-5a67-46ae-b751-e20f2eb419db tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:33.519471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6460ae55-5a67-46ae-b751-e20f2eb419db tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:33.522176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6460ae55-5a67-46ae-b751-e20f2eb419db tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:33.523020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6460ae55-5a67-46ae-b751-e20f2eb419db tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:33.524360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6460ae55-5a67-46ae-b751-e20f2eb419db tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:33.526766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6460ae55-5a67-46ae-b751-e20f2eb419db tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:33.530550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6460ae55-5a67-46ae-b751-e20f2eb419db tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:33.697472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c3d8124c-8d86-4875-928c-c380adc6e15d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:33.701567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:33.719730 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c3d8124c-8d86-4875-928c-c380adc6e15d tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05" status: 200 len: 1521 microversion: 2.1 time: 0.782387 Sep 27 10:09:33.732958 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1817/3328] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:32 2022] GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05 => generated 1521 bytes in 800 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:33.744707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-97bf9ee5-f94d-4a7d-b23a-73d4e881339a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:33.745530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-97bf9ee5-f94d-4a7d-b23a-73d4e881339a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:33.749098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=5,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:33.780629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bf9ee5-f94d-4a7d-b23a-73d4e881339a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:33.780629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bf9ee5-f94d-4a7d-b23a-73d4e881339a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:33.780629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bf9ee5-f94d-4a7d-b23a-73d4e881339a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:33.780629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bf9ee5-f94d-4a7d-b23a-73d4e881339a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:33.793243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bf9ee5-f94d-4a7d-b23a-73d4e881339a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:33.795516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bf9ee5-f94d-4a7d-b23a-73d4e881339a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:34.869217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-97bf9ee5-f94d-4a7d-b23a-73d4e881339a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:34.881429 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-97bf9ee5-f94d-4a7d-b23a-73d4e881339a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 1.147471 Sep 27 10:09:34.898482 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1818/3329] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:33 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 1165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:34.906461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:34.914400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:34.948446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=18 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:34.953277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:34.954173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:34.954855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:34.956098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:34.957051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:34.957708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:35.914343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:36.007900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:36.024871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:36.028816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:36.033834 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-efdfb275-06cb-4dac-839b-41e410f7460e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 1.132614 Sep 27 10:09:36.058992 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1819/3330] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:34 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 1159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:36.093793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2f875508-1625-4602-97e7-2eccf9558a52 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:36.096425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2f875508-1625-4602-97e7-2eccf9558a52 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:36.133574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f875508-1625-4602-97e7-2eccf9558a52 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:36.136152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f875508-1625-4602-97e7-2eccf9558a52 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:36.137540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f875508-1625-4602-97e7-2eccf9558a52 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:36.138450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f875508-1625-4602-97e7-2eccf9558a52 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:36.140678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f875508-1625-4602-97e7-2eccf9558a52 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:36.141311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f875508-1625-4602-97e7-2eccf9558a52 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:36.499911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2f875508-1625-4602-97e7-2eccf9558a52 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:36.508719 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2f875508-1625-4602-97e7-2eccf9558a52 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 0.448263 Sep 27 10:09:36.518506 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1820/3331] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:36 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:36.537218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:36.538154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 4fed875e-5be1-4b37-a0f9-55e6bc6fca05] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:36.546355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:36.547101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:36.547713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:36.548606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:36.549285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:36.550332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:36.606293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:36.607685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:36.608413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:36.609467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lazy-loading 'fault' on Instance uuid 4fed875e-5be1-4b37-a0f9-55e6bc6fca05 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:09:36.689785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6460ae55-5a67-46ae-b751-e20f2eb419db tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:36.693412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '2699ef01c0eb4d5783203e6b41293d97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '79022e5ca5764576b4feb7f4cadba80c', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:36.711259 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e6f0d93b-ca80-47e5-80d8-7aebfa44d39e tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05" status: 200 len: 1312 microversion: 2.1 time: 0.183186 Sep 27 10:09:36.711801 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1821/3332] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:36 2022] GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05 => generated 1312 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:36.732786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-2f6aab76-b380-48b8-bead-31c15aebf82b req-00e84f9d-7a5a-494d-9e02-5d945545f0c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e4e3f396-5222-478d-8a5a-333d866ba1ab", "name": "network-vif-deleted", "tag": "55b7c0fb-c7dc-4b8a-bc41-1a218717dd1f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:36.733086 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6460ae55-5a67-46ae-b751-e20f2eb419db tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1468 microversion: 2.1 time: 3.208998 Sep 27 10:09:36.734649 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1512/3333] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:33 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1468 bytes in 3238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:36.763883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2f6aab76-b380-48b8-bead-31c15aebf82b req-00e84f9d-7a5a-494d-9e02-5d945545f0c5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:36.765150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2f6aab76-b380-48b8-bead-31c15aebf82b req-00e84f9d-7a5a-494d-9e02-5d945545f0c5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:36.765598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2f6aab76-b380-48b8-bead-31c15aebf82b req-00e84f9d-7a5a-494d-9e02-5d945545f0c5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:36.774176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ee938bd5-2547-43b3-abc2-88eeef53f87d req-28c11208-432d-435c-b983-c65f2c097a1a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b902a024-8aa7-4843-8680-41d3e189d3af", "name": "network-vif-unplugged", "status": "completed", "tag": "df367b62-0029-488e-9897-fac934fc397e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:36.780460 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-2f6aab76-b380-48b8-bead-31c15aebf82b req-00e84f9d-7a5a-494d-9e02-5d945545f0c5 service nova] Creating event network-vif-deleted:55b7c0fb-c7dc-4b8a-bc41-1a218717dd1f for instance e4e3f396-5222-478d-8a5a-333d866ba1ab on nch1-master-1 Sep 27 10:09:36.788275 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-2f6aab76-b380-48b8-bead-31c15aebf82b req-00e84f9d-7a5a-494d-9e02-5d945545f0c5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074872 Sep 27 10:09:36.792315 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1822/3334] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:36.798306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-60f04e81-2c25-434e-a222-4fee6c81c300 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:36.799026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-60f04e81-2c25-434e-a222-4fee6c81c300 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:36.811249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60f04e81-2c25-434e-a222-4fee6c81c300 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:36.811755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60f04e81-2c25-434e-a222-4fee6c81c300 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:36.812199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60f04e81-2c25-434e-a222-4fee6c81c300 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:36.812991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60f04e81-2c25-434e-a222-4fee6c81c300 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:36.813454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60f04e81-2c25-434e-a222-4fee6c81c300 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:36.813886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60f04e81-2c25-434e-a222-4fee6c81c300 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:36.814443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ee938bd5-2547-43b3-abc2-88eeef53f87d req-28c11208-432d-435c-b983-c65f2c097a1a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:36.821434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ee938bd5-2547-43b3-abc2-88eeef53f87d req-28c11208-432d-435c-b983-c65f2c097a1a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:36.821889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ee938bd5-2547-43b3-abc2-88eeef53f87d req-28c11208-432d-435c-b983-c65f2c097a1a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:36.867749 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ee938bd5-2547-43b3-abc2-88eeef53f87d req-28c11208-432d-435c-b983-c65f2c097a1a service nova] Creating event network-vif-unplugged:df367b62-0029-488e-9897-fac934fc397e for instance b902a024-8aa7-4843-8680-41d3e189d3af on nch1-master-1 Sep 27 10:09:36.883740 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ee938bd5-2547-43b3-abc2-88eeef53f87d req-28c11208-432d-435c-b983-c65f2c097a1a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.142132 Sep 27 10:09:36.887462 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1513/3335] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:36 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:36.903156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a00bf955-a671-4383-a20c-c38d6bbd8488 req-e2edd7e2-f97c-4067-a684-e6fcfcbe0cb9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4fed875e-5be1-4b37-a0f9-55e6bc6fca05", "name": "network-vif-deleted", "tag": "c0a6ca20-6141-4b76-b38e-d3a32bd4803c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:36.940371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a00bf955-a671-4383-a20c-c38d6bbd8488 req-e2edd7e2-f97c-4067-a684-e6fcfcbe0cb9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:36.941084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a00bf955-a671-4383-a20c-c38d6bbd8488 req-e2edd7e2-f97c-4067-a684-e6fcfcbe0cb9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:36.942331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a00bf955-a671-4383-a20c-c38d6bbd8488 req-e2edd7e2-f97c-4067-a684-e6fcfcbe0cb9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:36.982423 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a00bf955-a671-4383-a20c-c38d6bbd8488 req-e2edd7e2-f97c-4067-a684-e6fcfcbe0cb9 service nova] Creating event network-vif-deleted:c0a6ca20-6141-4b76-b38e-d3a32bd4803c for instance 4fed875e-5be1-4b37-a0f9-55e6bc6fca05 on nch2-master-1 Sep 27 10:09:36.997544 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a00bf955-a671-4383-a20c-c38d6bbd8488 req-e2edd7e2-f97c-4067-a684-e6fcfcbe0cb9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.108047 Sep 27 10:09:37.000902 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1514/3336] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:37.052173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-60f04e81-2c25-434e-a222-4fee6c81c300 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:37.063020 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-60f04e81-2c25-434e-a222-4fee6c81c300 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.268817 Sep 27 10:09:37.069426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:37.080588 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1823/3337] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:36 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:37.083570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:37.099005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:37.099658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:37.100131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:37.101404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:37.102041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:37.105694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:37.535525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2a9d871f-54b4-4aee-a136-a3ec07338695 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:37.536499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2a9d871f-54b4-4aee-a136-a3ec07338695 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:37.547452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a9d871f-54b4-4aee-a136-a3ec07338695 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:37.548320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a9d871f-54b4-4aee-a136-a3ec07338695 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:37.550504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a9d871f-54b4-4aee-a136-a3ec07338695 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:37.565452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a9d871f-54b4-4aee-a136-a3ec07338695 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:37.566419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a9d871f-54b4-4aee-a136-a3ec07338695 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:37.570022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a9d871f-54b4-4aee-a136-a3ec07338695 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:37.638578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:37.654787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:37.660299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:37.662992 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-93e1593e-4bdb-4f89-b19c-85f9fcc6161b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.600411 Sep 27 10:09:37.668384 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1515/3338] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:37 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 607 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:09:37.736218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:37.736979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: 4fed875e-5be1-4b37-a0f9-55e6bc6fca05] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:37.755102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:37.755102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:37.755102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:37.755722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:37.756185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:37.756613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:37.796172 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] HTTP exception thrown: Instance 4fed875e-5be1-4b37-a0f9-55e6bc6fca05 could not be found. Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Request method failure captured: Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05 HTTP/1.1 Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsuj15rCUD1970qSbzxB3cw4sTAfA1YtNh4VnoapFvuP9GffqVpFgRGMeWDK-Po0RARJzDlkwtHZJxI_lb7FISD34bP5tmzbtvUeP246046Go7rEPdSKK7iPObXyQaBpwtLG8VaeId4vMcWe5OB5uDdCNIPuVd0zCfFs93HwTWDEgHo Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_79022e5ca5764576b4feb7f4cadba80c"}]}] Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2699ef01c0eb4d5783203e6b41293d97 Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 4fed875e-5be1-4b37-a0f9-55e6bc6fca05 could not be found. Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '4fed875e-5be1-4b37-a0f9-55e6bc6fca05'} Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4fed875e-5be1-4b37-a0f9-55e6bc6fca05 could not be found. Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4fed875e-5be1-4b37-a0f9-55e6bc6fca05 could not be found. Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:37.798158 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:37.802241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:37.802241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:37.802241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:37.802241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:37.802241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:09:37.802241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:09:37.802241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4fed875e-5be1-4b37-a0f9-55e6bc6fca05 could not be found. Sep 27 10:09:37.802241 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:37.802241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Returning 404 to user: Instance 4fed875e-5be1-4b37-a0f9-55e6bc6fca05 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:37.802241 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6e0c8c24-4ff9-4626-bc33-acd1dcc267c0 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05" status: 404 len: 111 microversion: 2.1 time: 0.067941 Sep 27 10:09:37.802241 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1516/3339] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:37 2022] GET /compute/v2.1/servers/4fed875e-5be1-4b37-a0f9-55e6bc6fca05 => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:09:37.808404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7c4b452a-d965-4f84-b6b1-8d58818a6d00 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:37.809120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7c4b452a-d965-4f84-b6b1-8d58818a6d00 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:37.817911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4b452a-d965-4f84-b6b1-8d58818a6d00 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:37.818423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4b452a-d965-4f84-b6b1-8d58818a6d00 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:37.818868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4b452a-d965-4f84-b6b1-8d58818a6d00 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:37.819768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4b452a-d965-4f84-b6b1-8d58818a6d00 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:37.820348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4b452a-d965-4f84-b6b1-8d58818a6d00 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:37.820986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4b452a-d965-4f84-b6b1-8d58818a6d00 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:38.064059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7c4b452a-d965-4f84-b6b1-8d58818a6d00 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b4e0f20e52074142a3b525bafbb30c97', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '52efcec2c8a14640a8c3df1dc576bbf1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:38.078978 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7c4b452a-d965-4f84-b6b1-8d58818a6d00 tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 200 len: 1426 microversion: 2.1 time: 0.276157 Sep 27 10:09:38.081752 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1517/3340] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:37 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 1426 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:38.128622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:38.129840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] [instance: e4e3f396-5222-478d-8a5a-333d866ba1ab] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:38.142501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:38.148950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:38.149643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:38.153461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:38.154795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:38.157020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:38.206373 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] HTTP exception thrown: Instance e4e3f396-5222-478d-8a5a-333d866ba1ab could not be found. Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Request method failure captured: Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab HTTP/1.1 Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsuj15rCUD1970qSbzxB3cw4sTAfA1YtNh4VnoapFvuP9GffqVpFgRGMeWDK-Po0RARJzDlkwtHZJxI_lb7FISD34bP5tmzbtvUeP246046Go7rEPdSKK7iPObXyQaBpwtLG8VaeId4vMcWe5OB5uDdCNIPuVd0zCfFs93HwTWDEgHo Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/79022e5ca5764576b4feb7f4cadba80c"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_79022e5ca5764576b4feb7f4cadba80c"}]}] Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 79022e5ca5764576b4feb7f4cadba80c Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ListServersNegativeTestJSON-2078923277 Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 2699ef01c0eb4d5783203e6b41293d97 Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ListServersNegativeTestJSON-2078923277-project Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance e4e3f396-5222-478d-8a5a-333d866ba1ab could not be found. Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'e4e3f396-5222-478d-8a5a-333d866ba1ab'} Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e4e3f396-5222-478d-8a5a-333d866ba1ab could not be found. Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e4e3f396-5222-478d-8a5a-333d866ba1ab could not be found. Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:38.209596 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:38.214765 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:38.214765 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:38.214765 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:38.214765 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:38.214765 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:09:38.214765 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:09:38.214765 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e4e3f396-5222-478d-8a5a-333d866ba1ab could not be found. Sep 27 10:09:38.214765 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:38.223965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2a9d871f-54b4-4aee-a136-a3ec07338695 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:38.236304 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2a9d871f-54b4-4aee-a136-a3ec07338695 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 0.706284 Sep 27 10:09:38.237462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] Returning 404 to user: Instance e4e3f396-5222-478d-8a5a-333d866ba1ab could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:38.238542 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4994cdf6-4bb6-41f1-97ce-2fb9ae68fd30 tempest-ListServersNegativeTestJSON-2078923277 tempest-ListServersNegativeTestJSON-2078923277-project] 10.222.0.26 "GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab" status: 404 len: 111 microversion: 2.1 time: 0.155437 Sep 27 10:09:38.239356 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1824/3341] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:37 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 710 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:38.241044 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1518/3342] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:38 2022] GET /compute/v2.1/servers/e4e3f396-5222-478d-8a5a-333d866ba1ab => generated 111 bytes in 158 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:09:38.257288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4ad66c13-3471-4813-9b96-030847ca9fdd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:38.259006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4ad66c13-3471-4813-9b96-030847ca9fdd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:38.284855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ad66c13-3471-4813-9b96-030847ca9fdd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:38.286890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ad66c13-3471-4813-9b96-030847ca9fdd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:38.287900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ad66c13-3471-4813-9b96-030847ca9fdd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:38.289797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ad66c13-3471-4813-9b96-030847ca9fdd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:38.290571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ad66c13-3471-4813-9b96-030847ca9fdd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:38.291173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ad66c13-3471-4813-9b96-030847ca9fdd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:38.665697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4ad66c13-3471-4813-9b96-030847ca9fdd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:38.675280 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4ad66c13-3471-4813-9b96-030847ca9fdd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.432244 Sep 27 10:09:38.690052 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1825/3343] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:38 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:38.697080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:38.697857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:38.700201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=18 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:38.714358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:38.717388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:38.717946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:38.718783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:38.724760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:38.725330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:39.116088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:39.117940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] [instance: b902a024-8aa7-4843-8680-41d3e189d3af] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:39.135992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:39.136510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:39.144014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:39.144929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:39.153799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:39.154791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:39.176368 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] HTTP exception thrown: Instance b902a024-8aa7-4843-8680-41d3e189d3af could not be found. Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Request method failure captured: Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af HTTP/1.1 Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMstY7pvZq0cwMNYMMVADM0gWwtU_Ctmt8D3P6WO9OPmRX7a6zhma5oX4hGPRVJhWPCRMjjMBtQe3DrNvI4aKKTceFl2eCz0eUV7imwxj3WQAsdS4VavncEa0g3-vfbBrzWhsnamQA0g-aTtmbyfHvm91gTZTbTU_45aI7C4p4PszrLQ Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 52efcec2c8a14640a8c3df1dc576bbf1 Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersAdminTestJSON-568530853 Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/52efcec2c8a14640a8c3df1dc576bbf1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/52efcec2c8a14640a8c3df1dc576bbf1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/52efcec2c8a14640a8c3df1dc576bbf1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_52efcec2c8a14640a8c3df1dc576bbf1"}]}] Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersAdminTestJSON-568530853 Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 52efcec2c8a14640a8c3df1dc576bbf1 Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersAdminTestJSON-568530853 Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersAdminTestJSON-568530853-project Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b4e0f20e52074142a3b525bafbb30c97 Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersAdminTestJSON-568530853-project Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance b902a024-8aa7-4843-8680-41d3e189d3af could not be found. Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'b902a024-8aa7-4843-8680-41d3e189d3af'} Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b902a024-8aa7-4843-8680-41d3e189d3af could not be found. Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b902a024-8aa7-4843-8680-41d3e189d3af could not be found. Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:39.180084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:39.185266 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:39.185266 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:39.185266 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:39.185266 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:09:39.185266 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:09:39.185266 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b902a024-8aa7-4843-8680-41d3e189d3af could not be found. Sep 27 10:09:39.185266 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:09:39.186897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] Returning 404 to user: Instance b902a024-8aa7-4843-8680-41d3e189d3af could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:39.187934 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-939fb747-37fd-4453-89f4-42b73691ad5b tempest-ServersAdminTestJSON-568530853 tempest-ServersAdminTestJSON-568530853-project] 10.222.0.26 "GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af" status: 404 len: 111 microversion: 2.1 time: 0.076240 Sep 27 10:09:39.189262 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1826/3344] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:39 2022] GET /compute/v2.1/servers/b902a024-8aa7-4843-8680-41d3e189d3af => generated 111 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:09:39.197638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:39.220017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:39.222042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:39.223254 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b34f0096-c4e9-4957-81fc-13d015b6f668 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.531553 Sep 27 10:09:39.240754 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1519/3345] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:38 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:39.247523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-68c89cc5-73ec-4410-9ccb-0849c65f5842 req-f0529895-6d7b-4244-98fc-8a69b4d2665d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b902a024-8aa7-4843-8680-41d3e189d3af", "name": "network-vif-deleted", "tag": "df367b62-0029-488e-9897-fac934fc397e"}, {"name": "network-changed", "server_uuid": "9ac32528-351d-4ba2-b0ec-0eeb4872e95c", "tag": "6f38bb60-56d7-44ce-ad27-ae8c1e3ac0fb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:39.265164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e3ee9d1-4fc5-4390-9079-ea97a54b053e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:39.265777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4e3ee9d1-4fc5-4390-9079-ea97a54b053e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:39.276217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3ee9d1-4fc5-4390-9079-ea97a54b053e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:39.277439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3ee9d1-4fc5-4390-9079-ea97a54b053e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:39.277884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3ee9d1-4fc5-4390-9079-ea97a54b053e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:39.278607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3ee9d1-4fc5-4390-9079-ea97a54b053e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:39.279063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3ee9d1-4fc5-4390-9079-ea97a54b053e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:39.279490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3ee9d1-4fc5-4390-9079-ea97a54b053e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:39.294647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-68c89cc5-73ec-4410-9ccb-0849c65f5842 req-f0529895-6d7b-4244-98fc-8a69b4d2665d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:39.295124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-68c89cc5-73ec-4410-9ccb-0849c65f5842 req-f0529895-6d7b-4244-98fc-8a69b4d2665d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:39.297179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-68c89cc5-73ec-4410-9ccb-0849c65f5842 req-f0529895-6d7b-4244-98fc-8a69b4d2665d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:39.346147 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-68c89cc5-73ec-4410-9ccb-0849c65f5842 req-f0529895-6d7b-4244-98fc-8a69b4d2665d service nova] Creating event network-vif-deleted:df367b62-0029-488e-9897-fac934fc397e for instance b902a024-8aa7-4843-8680-41d3e189d3af on nch1-master-1 Sep 27 10:09:39.346657 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-68c89cc5-73ec-4410-9ccb-0849c65f5842 req-f0529895-6d7b-4244-98fc-8a69b4d2665d service nova] Creating event network-changed:6f38bb60-56d7-44ce-ad27-ae8c1e3ac0fb for instance 9ac32528-351d-4ba2-b0ec-0eeb4872e95c on nch2-master-1 Sep 27 10:09:39.380384 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-68c89cc5-73ec-4410-9ccb-0849c65f5842 req-f0529895-6d7b-4244-98fc-8a69b4d2665d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.137342 Sep 27 10:09:39.384530 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1827/3346] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:39 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:39.704540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4488752e-9127-465b-9cab-7e410535ceb6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:39.705588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4488752e-9127-465b-9cab-7e410535ceb6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:39.721918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4488752e-9127-465b-9cab-7e410535ceb6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:39.724282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4488752e-9127-465b-9cab-7e410535ceb6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:39.725041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4488752e-9127-465b-9cab-7e410535ceb6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:39.728226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4488752e-9127-465b-9cab-7e410535ceb6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:39.728977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4488752e-9127-465b-9cab-7e410535ceb6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:39.731491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4488752e-9127-465b-9cab-7e410535ceb6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:39.968629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:40.481152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4488752e-9127-465b-9cab-7e410535ceb6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:40.492497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4e3ee9d1-4fc5-4390-9079-ea97a54b053e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:40.498310 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4488752e-9127-465b-9cab-7e410535ceb6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.798385 Sep 27 10:09:40.503351 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1828/3347] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:39 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 804 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:09:40.517903 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4e3ee9d1-4fc5-4390-9079-ea97a54b053e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 1.242827 Sep 27 10:09:40.517903 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1520/3348] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:39 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 1248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:09:40.525687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:40.526317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:40.551192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:40.551429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:40.551783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:40.552599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:40.553142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:40.553583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:40.827407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:40.839285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:40.840764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:40.841970 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af7ff537-4cf2-4740-bb24-cf551909fba9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.328157 Sep 27 10:09:40.849530 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1521/3349] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:40 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:41.532737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9d9674d6-e787-4ce2-8b04-21b92480208c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:41.534281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5e049da6-288e-4add-b80e-62c98f8d2dea tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:41.535483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9d9674d6-e787-4ce2-8b04-21b92480208c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:41.536379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5e049da6-288e-4add-b80e-62c98f8d2dea tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:41.551083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e049da6-288e-4add-b80e-62c98f8d2dea tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:41.552312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e049da6-288e-4add-b80e-62c98f8d2dea tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:41.552755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d9674d6-e787-4ce2-8b04-21b92480208c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:41.553269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d9674d6-e787-4ce2-8b04-21b92480208c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:41.553809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d9674d6-e787-4ce2-8b04-21b92480208c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:41.554457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d9674d6-e787-4ce2-8b04-21b92480208c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:41.556320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e049da6-288e-4add-b80e-62c98f8d2dea tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:41.558551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e049da6-288e-4add-b80e-62c98f8d2dea tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:41.559970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e049da6-288e-4add-b80e-62c98f8d2dea tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:41.562676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e049da6-288e-4add-b80e-62c98f8d2dea tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:41.564747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d9674d6-e787-4ce2-8b04-21b92480208c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:41.565390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d9674d6-e787-4ce2-8b04-21b92480208c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:41.827729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5e049da6-288e-4add-b80e-62c98f8d2dea tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:41.842558 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5e049da6-288e-4add-b80e-62c98f8d2dea tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 0.314253 Sep 27 10:09:41.861795 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1829/3350] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:41 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:41.875297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-01500d0a-6663-4da2-b6f8-5205c925402e req-03dbf7f7-2914-47db-aeaf-183b480618bd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e581dad6-2c27-4baa-9f55-c082f78b67bd", "tag": "e201ea6d-d900-45b5-8802-968cdc0d7ca9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:41.905574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-01500d0a-6663-4da2-b6f8-5205c925402e req-03dbf7f7-2914-47db-aeaf-183b480618bd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:41.908832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-01500d0a-6663-4da2-b6f8-5205c925402e req-03dbf7f7-2914-47db-aeaf-183b480618bd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:41.909731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-01500d0a-6663-4da2-b6f8-5205c925402e req-03dbf7f7-2914-47db-aeaf-183b480618bd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:41.940377 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-01500d0a-6663-4da2-b6f8-5205c925402e req-03dbf7f7-2914-47db-aeaf-183b480618bd service nova] Creating event network-changed:e201ea6d-d900-45b5-8802-968cdc0d7ca9 for instance e581dad6-2c27-4baa-9f55-c082f78b67bd on nch1-master-1 Sep 27 10:09:41.954232 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-01500d0a-6663-4da2-b6f8-5205c925402e req-03dbf7f7-2914-47db-aeaf-183b480618bd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084478 Sep 27 10:09:41.956810 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1830/3351] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:41.970425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:41.973737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:41.989489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:41.991412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:41.995203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:41.996288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:41.997009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:41.999951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:42.521960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:42.533455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:42.539099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:42.543778 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ad2ec7f5-94d4-485a-8944-d2b2c359db31 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.585466 Sep 27 10:09:42.556904 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1831/3352] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:41 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:42.788971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9d9674d6-e787-4ce2-8b04-21b92480208c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:42.800974 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9d9674d6-e787-4ce2-8b04-21b92480208c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 1.272201 Sep 27 10:09:42.802776 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1522/3353] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:41 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 1275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:42.877643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-142e102b-5bb0-4a48-b3ee-03da149b3760 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:42.878521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-142e102b-5bb0-4a48-b3ee-03da149b3760 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:42.895170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-142e102b-5bb0-4a48-b3ee-03da149b3760 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:42.897191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-142e102b-5bb0-4a48-b3ee-03da149b3760 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:42.899422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-142e102b-5bb0-4a48-b3ee-03da149b3760 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:42.901644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-142e102b-5bb0-4a48-b3ee-03da149b3760 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:42.902997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-142e102b-5bb0-4a48-b3ee-03da149b3760 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:42.903664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-142e102b-5bb0-4a48-b3ee-03da149b3760 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:43.587204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:43.589373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:43.601005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-142e102b-5bb0-4a48-b3ee-03da149b3760 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:43.615380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:43.616996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:43.617657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:43.618553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:43.619176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:43.619756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:43.627192 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-142e102b-5bb0-4a48-b3ee-03da149b3760 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 0.753693 Sep 27 10:09:43.628260 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1832/3354] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:42 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:43.734591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:43.756286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:43.761100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:43.762875 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c2b8f26c-909f-4107-ab80-23cec716a1c9 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.180380 Sep 27 10:09:43.764880 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1523/3355] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:43 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:43.827655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1aa6899f-c0a7-4a42-85d1-52f55080f4a1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:43.828636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1aa6899f-c0a7-4a42-85d1-52f55080f4a1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:43.842903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1aa6899f-c0a7-4a42-85d1-52f55080f4a1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:43.845756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1aa6899f-c0a7-4a42-85d1-52f55080f4a1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:43.846756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1aa6899f-c0a7-4a42-85d1-52f55080f4a1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:43.848189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1aa6899f-c0a7-4a42-85d1-52f55080f4a1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:43.851664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1aa6899f-c0a7-4a42-85d1-52f55080f4a1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:43.852442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1aa6899f-c0a7-4a42-85d1-52f55080f4a1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:44.367600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1aa6899f-c0a7-4a42-85d1-52f55080f4a1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:44.382760 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1aa6899f-c0a7-4a42-85d1-52f55080f4a1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.560081 Sep 27 10:09:44.389296 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1833/3356] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:43 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:44.654967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-618d8487-f1db-4317-8639-846f23d1caff tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:44.660021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-618d8487-f1db-4317-8639-846f23d1caff tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:44.684016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618d8487-f1db-4317-8639-846f23d1caff tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:44.684939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618d8487-f1db-4317-8639-846f23d1caff tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:44.686772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618d8487-f1db-4317-8639-846f23d1caff tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:44.689916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618d8487-f1db-4317-8639-846f23d1caff tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:44.691376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618d8487-f1db-4317-8639-846f23d1caff tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:44.694393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-618d8487-f1db-4317-8639-846f23d1caff tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:44.790539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:44.791400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:44.807596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:44.808114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:44.808582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:44.816313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:44.818055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:44.818521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:45.132729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:45.146027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:45.147503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:45.167085 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-601fef58-47e0-4668-8e19-f0af4b2d5523 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.379382 Sep 27 10:09:45.168255 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1834/3357] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:44 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:09:45.319825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-618d8487-f1db-4317-8639-846f23d1caff tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:45.332981 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-618d8487-f1db-4317-8639-846f23d1caff tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 0.686530 Sep 27 10:09:45.333599 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1524/3358] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:44 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 688 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:45.414101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-65f9cfdd-aac2-4b5d-a184-c8362ac1626e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:45.415904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-65f9cfdd-aac2-4b5d-a184-c8362ac1626e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:45.432521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65f9cfdd-aac2-4b5d-a184-c8362ac1626e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:45.433290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65f9cfdd-aac2-4b5d-a184-c8362ac1626e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:45.436942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65f9cfdd-aac2-4b5d-a184-c8362ac1626e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:45.442680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65f9cfdd-aac2-4b5d-a184-c8362ac1626e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:45.443471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65f9cfdd-aac2-4b5d-a184-c8362ac1626e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:45.445560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65f9cfdd-aac2-4b5d-a184-c8362ac1626e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:45.739186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b8e07ea-03a5-4df4-811f-5a436f20dbb2 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Action: 'create', calling method: .version_select at 0x7f7f635f85e0>, body: {"keypair": {"name": "tempest-keypair-624016620", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDDfbXYxxjr/L/4uZxkgJDMIDde6LrqPGjY2IfHAE+09hPuLHPzaaoqKx6SDWbRuL7Q4ZKmFV/qnxPLDf9b9ysL0VHzy38SAZg7bzx2Wl4Ok/tV4dL0qYZW5Q4tnkaTVig=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:45.768983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:45.787822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3b8e07ea-03a5-4df4-811f-5a436f20dbb2 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:45.800405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3b8e07ea-03a5-4df4-811f-5a436f20dbb2 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for user 9c748c35ea164fd391630a67c123cf2d and project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:45.864752 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b8e07ea-03a5-4df4-811f-5a436f20dbb2 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.378734 Sep 27 10:09:45.866163 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1525/3359] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:09:45 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:45.877249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b71bc742-4353-4bb7-8645-4aebc799598c tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Calling method '.version_select at 0x7f7f62e2f700>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:45.901790 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b71bc742-4353-4bb7-8645-4aebc799598c tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "GET /compute/v2.1/os-keypairs/tempest-keypair-624016620" status: 200 len: 497 microversion: 2.1 time: 0.026349 Sep 27 10:09:45.904315 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1526/3360] 10.222.0.26 () {58 vars in 1215 bytes} [Tue Sep 27 10:09:45 2022] GET /compute/v2.1/os-keypairs/tempest-keypair-624016620 => generated 497 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:09:45.918101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1da56f7f-6857-45e8-b28a-532b24e42028 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Calling method '.version_select at 0x7f7f62dc4c10>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:46.004509 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1da56f7f-6857-45e8-b28a-532b24e42028 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-624016620" status: 202 len: 0 microversion: 2.1 time: 0.090113 Sep 27 10:09:46.005381 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1527/3361] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:09:45 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-624016620 => generated 0 bytes in 94 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:09:46.017434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5fbf342b-0f75-45cc-9051-4d2ae4320451 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Action: 'create', calling method: .version_select at 0x7f7f62efde50>, body: {"keypair": {"name": "tempest-keypair-397045973", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHwtRSpWlSTJQkQGV2Z4FmvWWOsN/fOPG799PhD6zaeukpWAgQuAaYnDmcZfhHFZFUIVcMjB5msaza9n2lKxaWZdIHIYI5onZuTMZHjhkFxSlPMl/T8GYXzTlqqz6TI9KA=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:46.080008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5fbf342b-0f75-45cc-9051-4d2ae4320451 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:46.110334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5fbf342b-0f75-45cc-9051-4d2ae4320451 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for user 9c748c35ea164fd391630a67c123cf2d and project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:46.193258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-65f9cfdd-aac2-4b5d-a184-c8362ac1626e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:46.198405 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5fbf342b-0f75-45cc-9051-4d2ae4320451 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.184975 Sep 27 10:09:46.201392 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1528/3362] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:09:46 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:46.209162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:46.211429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:46.220229 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-65f9cfdd-aac2-4b5d-a184-c8362ac1626e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.803202 Sep 27 10:09:46.227767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:46.228265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:46.228729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:46.229452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:46.229912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:46.230338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:46.230982 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1835/3363] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:45 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 822 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:46.246239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5e64505a-6073-442c-83f4-409e4cc37000 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Calling method '.version_select at 0x7f7f6269f3a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:46.327717 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5e64505a-6073-442c-83f4-409e4cc37000 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-397045973" status: 202 len: 0 microversion: 2.1 time: 0.095379 Sep 27 10:09:46.343749 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1836/3364] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:09:46 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-397045973 => generated 0 bytes in 112 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:09:46.364579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-49c8939b-f381-4b80-a856-2535640fa25e tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Action: 'create', calling method: .version_select at 0x7f7f626b7700>, body: {"keypair": {"name": "tempest-keypair-6472060", "public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsNe3/1ILNCqFyfYWDeTKLD6jEXC2OQHLmietMWW+/vdaZq7KZEwO0jhglaFjU1mpqq4Gz5RX156sCTNM9vRbwKAxfsdF9laBYVsex3m3Wmui3uYrKyumsoJn2g9GNnG1PI1mrVjZ61i0GY3khna+wzlTpCCmy5HNlrmbj3XLqBUpipTOXmsnr4sChzC53KCd8LXuwc1i/CZPvF+3XipvAgFSE53pCtLOeB1kYMOBaiUPLQTWXR3JpckqFIQwhIH0zoHlJvZE8hh90XcPojYN56tI0OlrGqojbediJYD0rUsJu4weZpbn8vilb3JuDY+jwssnSA8wzBx3A/8y9Pp1B nova@ubuntu"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:46.466761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-49c8939b-f381-4b80-a856-2535640fa25e tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:46.486464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-49c8939b-f381-4b80-a856-2535640fa25e tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for user 9c748c35ea164fd391630a67c123cf2d and project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:46.571947 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-49c8939b-f381-4b80-a856-2535640fa25e tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 571 microversion: 2.1 time: 0.218788 Sep 27 10:09:46.577492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:46.580030 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1837/3365] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:09:46 2022] POST /compute/v2.1/os-keypairs => generated 571 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:46.590149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba791b86-dc0f-4786-bae2-0c86f9e2a458 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:46.592860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba791b86-dc0f-4786-bae2-0c86f9e2a458 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:46.596202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:46.597682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:46.608065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba791b86-dc0f-4786-bae2-0c86f9e2a458 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:46.613286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba791b86-dc0f-4786-bae2-0c86f9e2a458 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:46.614891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba791b86-dc0f-4786-bae2-0c86f9e2a458 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:46.616819 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-290b9a6a-a610-48d7-a58b-3314bfdb5304 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.396261 Sep 27 10:09:46.625782 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1529/3366] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:46 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:46.625990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba791b86-dc0f-4786-bae2-0c86f9e2a458 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:46.625990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba791b86-dc0f-4786-bae2-0c86f9e2a458 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:46.625990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba791b86-dc0f-4786-bae2-0c86f9e2a458 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:46.629803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-38ae3b28-a2e7-46b6-8b3c-e3407baf82ae tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Calling method '.version_select at 0x7f7f62e978b0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:46.689929 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-38ae3b28-a2e7-46b6-8b3c-e3407baf82ae tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-6472060" status: 202 len: 0 microversion: 2.1 time: 0.071290 Sep 27 10:09:46.690530 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1530/3367] 10.222.0.26 () {58 vars in 1212 bytes} [Tue Sep 27 10:09:46 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-6472060 => generated 0 bytes in 72 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:09:46.707828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7764ef45-0ecd-4f0c-93be-d5080296c69c tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Action: 'create', calling method: .version_select at 0x7f7f62e63e50>, body: {"keypair": {"name": "tempest-KeyPairsV2TestJSON-keypair-755466975", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHmWdTTU2OphJyKMvqtXiHY42Z65X7JjS+O7gjHsuuP0NNpuf66EQiQzJG6C72wAH2rYDSnnoUXdi2BY2OZUxqA0YqhBII6jiMcvV1IXUSEj1CaChFG7cIfybtS9pb27fQ=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:46.772618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7764ef45-0ecd-4f0c-93be-d5080296c69c tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:46.819784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7764ef45-0ecd-4f0c-93be-d5080296c69c tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for user 9c748c35ea164fd391630a67c123cf2d and project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:46.928349 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7764ef45-0ecd-4f0c-93be-d5080296c69c tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 404 microversion: 2.1 time: 0.224600 Sep 27 10:09:46.933673 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1531/3368] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:09:46 2022] POST /compute/v2.1/os-keypairs => generated 404 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:46.967749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-858ec6ea-3804-4999-b3f2-8b0879e64a7f tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Action: 'create', calling method: .version_select at 0x7f7f62e5e4c0>, body: {"keypair": {"name": "tempest-KeyPairsV2TestJSON-keypair-1612015467", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMd7LmFosU9P/e544UiugjOxlzQ5RXS83lyCQWwsKtWQG7SngDatc3cIuGsRmzzXKGXbCXs1MbtSj0E9l6FTnfQqf6tmYd+jt8v3ebsJAciUqe++yi95LY3M1rXNRBZGeA=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:47.062142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-858ec6ea-3804-4999-b3f2-8b0879e64a7f tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:47.093387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-858ec6ea-3804-4999-b3f2-8b0879e64a7f tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for user 9c748c35ea164fd391630a67c123cf2d and project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:47.164648 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-858ec6ea-3804-4999-b3f2-8b0879e64a7f tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 405 microversion: 2.1 time: 0.217208 Sep 27 10:09:47.176598 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1532/3369] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:09:46 2022] POST /compute/v2.1/os-keypairs => generated 405 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:47.193058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c3984622-a603-4c22-8b15-2960ccf6f48b tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Action: 'create', calling method: .version_select at 0x7f7f62e633a0>, body: {"keypair": {"name": "tempest-KeyPairsV2TestJSON-keypair-906896448", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE8XiusWnmnL3NaT0QKM1aKOyeMhjPM9aIu2v4nEw56PKluwNAGjhcPg8G2ef+o45nfu1W1gdRvhP5lpO7vOVtvLKufBSR0UTy9ZO0/VMrIS/pPe4tHbq/KVPOeeMBQ8gw=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:47.232127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c3984622-a603-4c22-8b15-2960ccf6f48b tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:47.240079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c3984622-a603-4c22-8b15-2960ccf6f48b tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Getting quotas for user 9c748c35ea164fd391630a67c123cf2d and project ddb7c4c03e254178b0e9991d54b04f41. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:47.313659 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c3984622-a603-4c22-8b15-2960ccf6f48b tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 404 microversion: 2.1 time: 0.133954 Sep 27 10:09:47.320690 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1533/3370] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:09:47 2022] POST /compute/v2.1/os-keypairs => generated 404 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:47.328672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f20ef58a-51fc-4a89-92e1-404806ea3cbe tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:47.329323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f20ef58a-51fc-4a89-92e1-404806ea3cbe tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:47.361681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f20ef58a-51fc-4a89-92e1-404806ea3cbe tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:47.362221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f20ef58a-51fc-4a89-92e1-404806ea3cbe tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:47.364093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f20ef58a-51fc-4a89-92e1-404806ea3cbe tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:47.364908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f20ef58a-51fc-4a89-92e1-404806ea3cbe tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:47.375326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f20ef58a-51fc-4a89-92e1-404806ea3cbe tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:47.376036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f20ef58a-51fc-4a89-92e1-404806ea3cbe tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:47.905151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba791b86-dc0f-4786-bae2-0c86f9e2a458 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:47.913343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:47.931581 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba791b86-dc0f-4786-bae2-0c86f9e2a458 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 1.349499 Sep 27 10:09:47.932315 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1838/3371] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:46 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 1351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:47.943823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-260460ee-2cb5-4726-8c14-e4969e982202 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Calling method '.version_select at 0x7f7f62695dc0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:47.953190 ncdv-master-1 devstack@n-api.service[145968]: WARNING oslo_db.sqlalchemy.utils [None req-260460ee-2cb5-4726-8c14-e4969e982202 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Unique keys not in sort_keys. The sorting order may be unstable. Sep 27 10:09:47.974364 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-260460ee-2cb5-4726-8c14-e4969e982202 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "GET /compute/v2.1/os-keypairs" status: 200 len: 1092 microversion: 2.1 time: 0.040609 Sep 27 10:09:47.975290 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1839/3372] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:09:47 2022] GET /compute/v2.1/os-keypairs => generated 1092 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:47.988870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:47.992062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:48.008964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:48.011320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:48.011990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:48.014726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:48.019011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:48.019783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:48.199549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:48.214986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f20ef58a-51fc-4a89-92e1-404806ea3cbe tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:48.216700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:48.218570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:48.220343 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-03a89bc1-613e-41f6-86a0-8d55e8cd362b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.242499 Sep 27 10:09:48.221507 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1840/3373] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:47 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:48.229772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-73d422f9-0fd7-43b8-98ca-cd565c78aa6a tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Calling method '.version_select at 0x7f7f626950d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:48.247711 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f20ef58a-51fc-4a89-92e1-404806ea3cbe tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.925762 Sep 27 10:09:48.253016 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1534/3374] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:47 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 932 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:48.282421 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-73d422f9-0fd7-43b8-98ca-cd565c78aa6a tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-906896448" status: 202 len: 0 microversion: 2.1 time: 0.059505 Sep 27 10:09:48.283784 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1841/3375] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:09:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-906896448 => generated 0 bytes in 61 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:09:48.293159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6252cb0d-518e-4b67-8f32-62cf0d8d7446 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Calling method '.version_select at 0x7f7f63a18790>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:48.335770 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6252cb0d-518e-4b67-8f32-62cf0d8d7446 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-1612015467" status: 202 len: 0 microversion: 2.1 time: 0.046293 Sep 27 10:09:48.336958 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1535/3376] 10.222.0.26 () {58 vars in 1278 bytes} [Tue Sep 27 10:09:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-1612015467 => generated 0 bytes in 49 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:09:48.345179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-905eb88c-0009-4bd3-bd21-c6ca9992f846 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] Calling method '.version_select at 0x7f7f62643af0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:48.402616 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-905eb88c-0009-4bd3-bd21-c6ca9992f846 tempest-KeyPairsV2TestJSON-421937052 tempest-KeyPairsV2TestJSON-421937052-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-755466975" status: 202 len: 0 microversion: 2.1 time: 0.061276 Sep 27 10:09:48.403649 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1842/3377] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:09:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-KeyPairsV2TestJSON-keypair-755466975 => generated 0 bytes in 63 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:09:48.960613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5706e19e-3422-4823-b24b-cbf426034cf8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:48.964182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5706e19e-3422-4823-b24b-cbf426034cf8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:48.991884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e19e-3422-4823-b24b-cbf426034cf8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:48.993905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e19e-3422-4823-b24b-cbf426034cf8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:48.995359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e19e-3422-4823-b24b-cbf426034cf8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:48.997125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e19e-3422-4823-b24b-cbf426034cf8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:49.003449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e19e-3422-4823-b24b-cbf426034cf8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:49.004143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5706e19e-3422-4823-b24b-cbf426034cf8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:49.636872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6aed5784-de8b-41c4-9153-697a218cf37a tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Action: 'create', calling method: .version_select at 0x7f7f6268dee0>, body: {"keypair": {"name": "tempest-keypair-2043080211", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLzyCAk5wTsXN1Gh4QK6jgEfHnyMz/4ByAwNSLTGyD+C1B7K1bFteprUzHnJV0sYRUwUou0sBSLpYD5sV95ZMIM4ieMOZrHEzJbVjgFHNLBfYBrgerm9vO96nUpGP6SbHQ=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:49.687814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6aed5784-de8b-41c4-9153-697a218cf37a tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Getting quotas for project f847bee14ff74dfbafe7c6a199f1b99f. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:49.710209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6aed5784-de8b-41c4-9153-697a218cf37a tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Getting quotas for user b8d6f6c63fe144959139278ca287e2ec and project f847bee14ff74dfbafe7c6a199f1b99f. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:49.789426 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6aed5784-de8b-41c4-9153-697a218cf37a tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.604431 Sep 27 10:09:49.790076 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1843/3378] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:09:49 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:49.813205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:49.815496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:49.837449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:49.837795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:49.844502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:49.850303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:49.851945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5706e19e-3422-4823-b24b-cbf426034cf8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:49.853042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:49.853496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:49.870101 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5706e19e-3422-4823-b24b-cbf426034cf8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 0.913599 Sep 27 10:09:49.894385 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1536/3379] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:48 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 942 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:09:49.904883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cdb23ca0-52df-4208-8b6f-f91ab5f6b590 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:49.909814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cdb23ca0-52df-4208-8b6f-f91ab5f6b590 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:49.927727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb23ca0-52df-4208-8b6f-f91ab5f6b590 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:49.928478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb23ca0-52df-4208-8b6f-f91ab5f6b590 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:49.930443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb23ca0-52df-4208-8b6f-f91ab5f6b590 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:49.933762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb23ca0-52df-4208-8b6f-f91ab5f6b590 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:49.934501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb23ca0-52df-4208-8b6f-f91ab5f6b590 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:49.935328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb23ca0-52df-4208-8b6f-f91ab5f6b590 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:50.039883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:50.049137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:50.053647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:50.057966 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-efc28235-4e7a-45ef-941e-1c93907f0a01 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.266536 Sep 27 10:09:50.058776 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1844/3380] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:49 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:50.623312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cdb23ca0-52df-4208-8b6f-f91ab5f6b590 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:50.639482 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cdb23ca0-52df-4208-8b6f-f91ab5f6b590 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1527 microversion: 2.1 time: 0.743319 Sep 27 10:09:50.642764 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1537/3381] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:49 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1527 bytes in 748 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:50.902680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-56c7fa83-1ca4-4c41-855e-87e3767f4a42 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:50.903325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-56c7fa83-1ca4-4c41-855e-87e3767f4a42 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:50.913734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c7fa83-1ca4-4c41-855e-87e3767f4a42 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:50.914492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c7fa83-1ca4-4c41-855e-87e3767f4a42 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:50.915124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c7fa83-1ca4-4c41-855e-87e3767f4a42 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:50.916145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c7fa83-1ca4-4c41-855e-87e3767f4a42 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:50.916871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c7fa83-1ca4-4c41-855e-87e3767f4a42 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:50.917482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56c7fa83-1ca4-4c41-855e-87e3767f4a42 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:51.093280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:51.093908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:51.118450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:51.119417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:51.120437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:51.121858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:51.125966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:51.126763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:51.134231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:51.223071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-56c7fa83-1ca4-4c41-855e-87e3767f4a42 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:51.240297 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-56c7fa83-1ca4-4c41-855e-87e3767f4a42 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 0.342360 Sep 27 10:09:51.241298 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1845/3382] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:50 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:51.400229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:51.416590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:51.420589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:51.425899 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-808d13fc-c526-4b05-a794-c5bf3425d702 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.336884 Sep 27 10:09:51.427917 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1538/3383] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:51 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:51.671623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dccdedde-79de-4a02-983e-22b90a9d0220 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:51.673458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dccdedde-79de-4a02-983e-22b90a9d0220 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:51.687340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccdedde-79de-4a02-983e-22b90a9d0220 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:51.689260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccdedde-79de-4a02-983e-22b90a9d0220 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:51.694333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccdedde-79de-4a02-983e-22b90a9d0220 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:51.695379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccdedde-79de-4a02-983e-22b90a9d0220 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:51.696067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccdedde-79de-4a02-983e-22b90a9d0220 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:51.696485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dccdedde-79de-4a02-983e-22b90a9d0220 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:52.264614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6b5921d7-7ba0-4357-ba47-d1c8354fba0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:52.265330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6b5921d7-7ba0-4357-ba47-d1c8354fba0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:52.278301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b5921d7-7ba0-4357-ba47-d1c8354fba0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:52.283587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b5921d7-7ba0-4357-ba47-d1c8354fba0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:52.284370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b5921d7-7ba0-4357-ba47-d1c8354fba0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:52.286258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b5921d7-7ba0-4357-ba47-d1c8354fba0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:52.288624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b5921d7-7ba0-4357-ba47-d1c8354fba0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:52.291289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b5921d7-7ba0-4357-ba47-d1c8354fba0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:52.327368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dccdedde-79de-4a02-983e-22b90a9d0220 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:52.343430 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dccdedde-79de-4a02-983e-22b90a9d0220 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1509 microversion: 2.1 time: 0.677524 Sep 27 10:09:52.344023 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1846/3384] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:51 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1509 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:52.457821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:52.458824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:52.468118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:52.470721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:52.471452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:52.485942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:52.489120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:52.493342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:52.726884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:52.754098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:52.757988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:52.767781 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c86b78db-724e-435e-a21a-d753438165fd tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1409 microversion: 2.54 time: 0.312729 Sep 27 10:09:52.781381 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1847/3385] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:52 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1409 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:09:52.856606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6b5921d7-7ba0-4357-ba47-d1c8354fba0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:52.865397 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6b5921d7-7ba0-4357-ba47-d1c8354fba0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 0.605257 Sep 27 10:09:52.879487 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1539/3386] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:52 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:53.283764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a3ff4e2c-95c6-450d-9f58-a53f01258a68 req-0971e160-3fdd-4f0c-9a08-e8161b1dc043 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e581dad6-2c27-4baa-9f55-c082f78b67bd", "name": "network-vif-plugged", "status": "completed", "tag": "e201ea6d-d900-45b5-8802-968cdc0d7ca9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:53.301076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a3ff4e2c-95c6-450d-9f58-a53f01258a68 req-0971e160-3fdd-4f0c-9a08-e8161b1dc043 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:53.317812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a3ff4e2c-95c6-450d-9f58-a53f01258a68 req-0971e160-3fdd-4f0c-9a08-e8161b1dc043 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:53.318668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a3ff4e2c-95c6-450d-9f58-a53f01258a68 req-0971e160-3fdd-4f0c-9a08-e8161b1dc043 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:53.349722 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a3ff4e2c-95c6-450d-9f58-a53f01258a68 req-0971e160-3fdd-4f0c-9a08-e8161b1dc043 service nova] Creating event network-vif-plugged:e201ea6d-d900-45b5-8802-968cdc0d7ca9 for instance e581dad6-2c27-4baa-9f55-c082f78b67bd on nch1-master-1 Sep 27 10:09:53.375188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1b215d8e-57a6-47ae-b25e-5ac15c9ca3d5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:53.376598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1b215d8e-57a6-47ae-b25e-5ac15c9ca3d5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:53.381466 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a3ff4e2c-95c6-450d-9f58-a53f01258a68 req-0971e160-3fdd-4f0c-9a08-e8161b1dc043 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087724 Sep 27 10:09:53.384126 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1848/3387] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:09:53.386981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b215d8e-57a6-47ae-b25e-5ac15c9ca3d5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:53.387728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b215d8e-57a6-47ae-b25e-5ac15c9ca3d5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:53.388434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b215d8e-57a6-47ae-b25e-5ac15c9ca3d5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:53.389815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b215d8e-57a6-47ae-b25e-5ac15c9ca3d5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:53.390496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b215d8e-57a6-47ae-b25e-5ac15c9ca3d5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:53.391138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b215d8e-57a6-47ae-b25e-5ac15c9ca3d5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:53.740078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesAdminNegativeTest-server-1435946249", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "e4d5dcb5-38db-4df7-833a-8982d96ea82d"}], "security_groups": [{"name": "tempest-securitygroup--1951658482"}], "key_name": "tempest-keypair-2043080211", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:53.951967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1b215d8e-57a6-47ae-b25e-5ac15c9ca3d5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:53.970718 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1b215d8e-57a6-47ae-b25e-5ac15c9ca3d5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1509 microversion: 2.1 time: 0.599684 Sep 27 10:09:53.974149 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1540/3388] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:53 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1509 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:53.985938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:53.989615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:54.006199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:54.016529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] validate_networks() for [('e4d5dcb5-38db-4df7-833a-8982d96ea82d', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:09:54.026771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:54.036145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.026s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:54.038994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:54.039831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:54.040551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:54.202865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:54.217063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:54.224678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:54.227559 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f29ca34f-9122-4fd7-b1aa-2a7cafab501d tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1415 microversion: 2.54 time: 0.250460 Sep 27 10:09:54.228588 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1541/3389] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:53 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1415 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:54.238642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-929fcad2-cf48-445c-881c-d53890fc74cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:54.244976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-929fcad2-cf48-445c-881c-d53890fc74cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:54.255888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929fcad2-cf48-445c-881c-d53890fc74cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:54.258556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929fcad2-cf48-445c-881c-d53890fc74cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:54.259943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929fcad2-cf48-445c-881c-d53890fc74cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:54.261727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929fcad2-cf48-445c-881c-d53890fc74cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:54.262373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929fcad2-cf48-445c-881c-d53890fc74cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:54.262931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929fcad2-cf48-445c-881c-d53890fc74cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:54.704611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-929fcad2-cf48-445c-881c-d53890fc74cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:54.714085 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-929fcad2-cf48-445c-881c-d53890fc74cc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 0.483960 Sep 27 10:09:54.716846 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1542/3390] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:54 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:54.871398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:09:54.872432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:09:54.873768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:09:54.873989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:09:54.874519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:09:54.887005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:54.887786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:54.888504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:54.889543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:54.890234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:54.891141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:54.915998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Getting quotas for project f847bee14ff74dfbafe7c6a199f1b99f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:09:54.922915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Getting quotas for user b8d6f6c63fe144959139278ca287e2ec and project f847bee14ff74dfbafe7c6a199f1b99f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:09:54.927456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=3,nova_api:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:54.975601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:09:54.978571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:09:55.006530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e562e7e1-1c89-4195-9812-f11565023ff2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:55.008295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e562e7e1-1c89-4195-9812-f11565023ff2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:55.026278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e562e7e1-1c89-4195-9812-f11565023ff2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:55.036752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e562e7e1-1c89-4195-9812-f11565023ff2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:55.036752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e562e7e1-1c89-4195-9812-f11565023ff2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:55.052772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e562e7e1-1c89-4195-9812-f11565023ff2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:55.062515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e562e7e1-1c89-4195-9812-f11565023ff2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:55.064566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e562e7e1-1c89-4195-9812-f11565023ff2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:55.108418 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8eca3ff9-1760-440c-96f0-eaf640a18107 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.374396 Sep 27 10:09:55.109885 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1849/3391] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:09:53 2022] POST /compute/v2.1/servers => generated 406 bytes in 1377 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:09:55.158516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3b35eb3-3567-44c7-a2fe-f047a6801240 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:55.162699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d3b35eb3-3567-44c7-a2fe-f047a6801240 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:55.303490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d3b35eb3-3567-44c7-a2fe-f047a6801240 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:55.313351 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d3b35eb3-3567-44c7-a2fe-f047a6801240 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1242 microversion: 2.1 time: 0.193284 Sep 27 10:09:55.314835 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1850/3392] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:55 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1242 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:55.323536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:55.325561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:55.339062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:55.339747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:55.340471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:55.341458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:55.342240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:55.342897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:55.467213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:55.485502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:55.487242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:55.489948 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a30c025e-e1d3-4ad0-8998-9e1b639057f1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1415 microversion: 2.54 time: 0.172621 Sep 27 10:09:55.489948 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1851/3393] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:55 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1415 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:55.523325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e562e7e1-1c89-4195-9812-f11565023ff2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:55.537234 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e562e7e1-1c89-4195-9812-f11565023ff2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1509 microversion: 2.1 time: 0.544236 Sep 27 10:09:55.539485 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1543/3394] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:54 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1509 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:55.746260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e058f10d-c7da-4a0f-ac47-0cb96bf5f91e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:55.746884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e058f10d-c7da-4a0f-ac47-0cb96bf5f91e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:55.761971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e058f10d-c7da-4a0f-ac47-0cb96bf5f91e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:55.762724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e058f10d-c7da-4a0f-ac47-0cb96bf5f91e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:55.765178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e058f10d-c7da-4a0f-ac47-0cb96bf5f91e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:55.766458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e058f10d-c7da-4a0f-ac47-0cb96bf5f91e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:55.767155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e058f10d-c7da-4a0f-ac47-0cb96bf5f91e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:55.770634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e058f10d-c7da-4a0f-ac47-0cb96bf5f91e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:56.340614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80a58af4-5dd5-4742-ac47-f9e1dc599171 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:56.342946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-80a58af4-5dd5-4742-ac47-f9e1dc599171 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:56.363224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a58af4-5dd5-4742-ac47-f9e1dc599171 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:56.365139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a58af4-5dd5-4742-ac47-f9e1dc599171 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:56.366716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a58af4-5dd5-4742-ac47-f9e1dc599171 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:56.378521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a58af4-5dd5-4742-ac47-f9e1dc599171 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:56.380796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a58af4-5dd5-4742-ac47-f9e1dc599171 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:56.383132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a58af4-5dd5-4742-ac47-f9e1dc599171 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:56.642669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-80a58af4-5dd5-4742-ac47-f9e1dc599171 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:56.664398 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-80a58af4-5dd5-4742-ac47-f9e1dc599171 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1246 microversion: 2.1 time: 0.327818 Sep 27 10:09:56.680765 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1544/3395] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:56 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1246 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:56.688725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:56.689433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:56.698532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:56.699545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:56.700341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:56.701609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:56.702228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:56.703343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:56.815374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e058f10d-c7da-4a0f-ac47-0cb96bf5f91e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:56.833296 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e058f10d-c7da-4a0f-ac47-0cb96bf5f91e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1315 microversion: 2.1 time: 1.089671 Sep 27 10:09:56.834487 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1852/3396] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:55 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1315 bytes in 1093 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:56.850561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dee9286d-d727-4f20-90d3-65b58edba7e0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:56.852023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dee9286d-d727-4f20-90d3-65b58edba7e0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:56.867088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:56.870842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee9286d-d727-4f20-90d3-65b58edba7e0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:56.872484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee9286d-d727-4f20-90d3-65b58edba7e0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:56.876587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee9286d-d727-4f20-90d3-65b58edba7e0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:56.878866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee9286d-d727-4f20-90d3-65b58edba7e0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:56.879613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee9286d-d727-4f20-90d3-65b58edba7e0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:56.881045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee9286d-d727-4f20-90d3-65b58edba7e0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:56.884006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:56.885467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:56.886654 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-434d1988-f8ad-4510-a7c0-cbf8b456e4ca tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1415 microversion: 2.54 time: 0.204657 Sep 27 10:09:56.902121 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1545/3397] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:56 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1415 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:57.185654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a9a94347-d256-4f83-84a0-d7d3d17c5251 req-b1978933-1dde-471e-a7fd-d28e4822363f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3327ff95-5c05-4167-a73b-5fd55f7bfabd", "name": "network-vif-unplugged", "status": "completed", "tag": "d571b78e-f1b6-4353-b09e-cac4cd18cde7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:57.219811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a9a94347-d256-4f83-84a0-d7d3d17c5251 req-b1978933-1dde-471e-a7fd-d28e4822363f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:57.226878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a9a94347-d256-4f83-84a0-d7d3d17c5251 req-b1978933-1dde-471e-a7fd-d28e4822363f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:57.228620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a9a94347-d256-4f83-84a0-d7d3d17c5251 req-b1978933-1dde-471e-a7fd-d28e4822363f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:57.299973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=20 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:57.305598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.server_external_events [req-a9a94347-d256-4f83-84a0-d7d3d17c5251 req-b1978933-1dde-471e-a7fd-d28e4822363f service nova] Unable to find a host for instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd. Dropping event network-vif-unplugged {{(pid=145969) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 27 10:09:57.308555 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a9a94347-d256-4f83-84a0-d7d3d17c5251 req-b1978933-1dde-471e-a7fd-d28e4822363f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.131642 Sep 27 10:09:57.309216 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1546/3398] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:09:57 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 133 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Sep 27 10:09:57.611725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dee9286d-d727-4f20-90d3-65b58edba7e0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:57.627110 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dee9286d-d727-4f20-90d3-65b58edba7e0 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1512 microversion: 2.1 time: 0.790040 Sep 27 10:09:57.627745 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1853/3399] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:56 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1512 bytes in 792 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:57.654753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d9fec9d9-346e-467a-9348-fe603ba99440 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:57.655320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d9fec9d9-346e-467a-9348-fe603ba99440 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:57.658769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=5,nova_api:DELETE=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:57.685142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9fec9d9-346e-467a-9348-fe603ba99440 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:57.685926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9fec9d9-346e-467a-9348-fe603ba99440 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:57.686387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9fec9d9-346e-467a-9348-fe603ba99440 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:57.687115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9fec9d9-346e-467a-9348-fe603ba99440 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:57.691422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9fec9d9-346e-467a-9348-fe603ba99440 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:57.691906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9fec9d9-346e-467a-9348-fe603ba99440 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:57.695980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c99f1287-6cf8-4870-b3d5-79abf690e909 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:57.696863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c99f1287-6cf8-4870-b3d5-79abf690e909 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:57.706451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99f1287-6cf8-4870-b3d5-79abf690e909 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:57.712376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99f1287-6cf8-4870-b3d5-79abf690e909 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:57.713221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99f1287-6cf8-4870-b3d5-79abf690e909 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:57.714168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99f1287-6cf8-4870-b3d5-79abf690e909 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:57.714642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99f1287-6cf8-4870-b3d5-79abf690e909 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:57.715076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c99f1287-6cf8-4870-b3d5-79abf690e909 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:57.931030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c99f1287-6cf8-4870-b3d5-79abf690e909 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:57.962053 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c99f1287-6cf8-4870-b3d5-79abf690e909 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1294 microversion: 2.1 time: 0.275826 Sep 27 10:09:57.966922 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1854/3400] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:57 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1294 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:57.984273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-74da15fd-1d6b-46a3-ac39-bb7dac91fa7b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:57.987728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-74da15fd-1d6b-46a3-ac39-bb7dac91fa7b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:58.007242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74da15fd-1d6b-46a3-ac39-bb7dac91fa7b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.009714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74da15fd-1d6b-46a3-ac39-bb7dac91fa7b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.010780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74da15fd-1d6b-46a3-ac39-bb7dac91fa7b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:58.016028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74da15fd-1d6b-46a3-ac39-bb7dac91fa7b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.021847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74da15fd-1d6b-46a3-ac39-bb7dac91fa7b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.022719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74da15fd-1d6b-46a3-ac39-bb7dac91fa7b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:58.274630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-74da15fd-1d6b-46a3-ac39-bb7dac91fa7b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:58.286222 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-74da15fd-1d6b-46a3-ac39-bb7dac91fa7b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1502 microversion: 2.1 time: 0.316258 Sep 27 10:09:58.287503 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1855/3401] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:57 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1502 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:58.301117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:58.302122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:58.315910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.320275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d9fec9d9-346e-467a-9348-fe603ba99440 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:58.324554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.327999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:58.330631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.332065 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d9fec9d9-346e-467a-9348-fe603ba99440 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1452 microversion: 2.1 time: 0.681343 Sep 27 10:09:58.333201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.333983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:58.335878 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1547/3402] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:57 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1452 bytes in 686 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:58.353148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-49d76008-bbfa-4f51-b847-1167396ee624 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:58.363709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49d76008-bbfa-4f51-b847-1167396ee624 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.364202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49d76008-bbfa-4f51-b847-1167396ee624 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.364639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49d76008-bbfa-4f51-b847-1167396ee624 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:58.400227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-49d76008-bbfa-4f51-b847-1167396ee624 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Going to try to stop instance {{(pid=145969) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 10:09:58.498963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:58.513982 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-49d76008-bbfa-4f51-b847-1167396ee624 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd/action" status: 202 len: 0 microversion: 2.1 time: 0.176786 Sep 27 10:09:58.514844 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1548/3403] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:09:58 2022] POST /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd/action => generated 0 bytes in 178 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:09:58.519715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:58.521355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:58.522810 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-010cb1da-796c-4705-83e9-18fa98b2971a tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1400 microversion: 2.54 time: 0.232437 Sep 27 10:09:58.526080 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1856/3404] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:58 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1400 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:09:58.531172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:58.531833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:58.538311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-87c7b885-4ed1-47a7-be00-6729af561d0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:58.539320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.539473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-87c7b885-4ed1-47a7-be00-6729af561d0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:58.539625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.541598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:58.542629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.544235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.545537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:58.555548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c7b885-4ed1-47a7-be00-6729af561d0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.556936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c7b885-4ed1-47a7-be00-6729af561d0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.561150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c7b885-4ed1-47a7-be00-6729af561d0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:58.562716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c7b885-4ed1-47a7-be00-6729af561d0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.563366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c7b885-4ed1-47a7-be00-6729af561d0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.565408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c7b885-4ed1-47a7-be00-6729af561d0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:58.612457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lazy-loading 'flavor' on Instance uuid 3327ff95-5c05-4167-a73b-5fd55f7bfabd {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:09:58.642181 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Cannot 'shelve' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state shelved_offloaded Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action HTTP/1.1 Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 14 Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: {"shelve": {}} Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: exception: Cannot 'shelve' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state shelved_offloaded Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3327ff95-5c05-4167-a73b-5fd55f7bfabd', 'body': {'shelve': {}}} Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 49, in _shelve Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.compute_api.shelve(context, instance) Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 278, in inner Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd in vm_state shelved_offloaded. Cannot shelve while the instance is in this state. Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 56, in _shelve Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 10:09:58.646953 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'shelve' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state shelved_offloaded Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'shelve' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state shelved_offloaded Sep 27 10:09:58.652493 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:09:58.654565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 409 to user: Cannot 'shelve' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state shelved_offloaded {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:09:58.658819 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8c520955-38ba-49bf-bf49-01b7bb550c75 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 409 len: 155 microversion: 2.1 time: 0.142722 Sep 27 10:09:58.660470 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1549/3405] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:09:58 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 155 bytes in 145 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:09:58.667333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8b7e9ca9-8515-4536-a9ca-3088ec486aba tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '.version_select at 0x7f7f62e970d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:58.721490 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8b7e9ca9-8515-4536-a9ca-3088ec486aba tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-ServerShowV254Test-keypair-1438412530" status: 204 len: 0 microversion: 2.54 time: 0.059174 Sep 27 10:09:58.722681 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1550/3406] 10.222.0.26 () {60 vars in 1319 bytes} [Tue Sep 27 10:09:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-ServerShowV254Test-keypair-1438412530 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:09:58.733876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0541287e-b716-4728-b53e-52b4efdec662 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:09:58.735475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0541287e-b716-4728-b53e-52b4efdec662 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:58.752882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0541287e-b716-4728-b53e-52b4efdec662 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.753404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0541287e-b716-4728-b53e-52b4efdec662 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.754676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0541287e-b716-4728-b53e-52b4efdec662 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:58.755634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0541287e-b716-4728-b53e-52b4efdec662 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.756809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0541287e-b716-4728-b53e-52b4efdec662 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.758645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0541287e-b716-4728-b53e-52b4efdec662 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:58.827947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0541287e-b716-4728-b53e-52b4efdec662 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=145969) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 27 10:09:58.977274 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0541287e-b716-4728-b53e-52b4efdec662 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 202 len: 0 microversion: 2.1 time: 0.253659 Sep 27 10:09:58.977951 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1551/3407] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:09:58 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 0 bytes in 255 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:09:58.983709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dbaae2ca-798d-498d-a902-00cf77e3213e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:58.984538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dbaae2ca-798d-498d-a902-00cf77e3213e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:58.996522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbaae2ca-798d-498d-a902-00cf77e3213e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:58.997280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbaae2ca-798d-498d-a902-00cf77e3213e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:58.997894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbaae2ca-798d-498d-a902-00cf77e3213e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:59.000095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbaae2ca-798d-498d-a902-00cf77e3213e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:59.000911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbaae2ca-798d-498d-a902-00cf77e3213e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:59.001507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbaae2ca-798d-498d-a902-00cf77e3213e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:59.041606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dbaae2ca-798d-498d-a902-00cf77e3213e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:09:59.132345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:09:59.206996 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dbaae2ca-798d-498d-a902-00cf77e3213e tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 204 len: 0 microversion: 2.54 time: 0.226983 Sep 27 10:09:59.207619 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1552/3408] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:09:58 2022] DELETE /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:09:59.240534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b72e2d66-67ff-47ab-b3f8-7b5355f6d976 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:59.271652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b72e2d66-67ff-47ab-b3f8-7b5355f6d976 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:59.293887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b72e2d66-67ff-47ab-b3f8-7b5355f6d976 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:59.294385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b72e2d66-67ff-47ab-b3f8-7b5355f6d976 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:59.294829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b72e2d66-67ff-47ab-b3f8-7b5355f6d976 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:59.295594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b72e2d66-67ff-47ab-b3f8-7b5355f6d976 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:59.296034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b72e2d66-67ff-47ab-b3f8-7b5355f6d976 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:59.317482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b72e2d66-67ff-47ab-b3f8-7b5355f6d976 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:59.615312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-87c7b885-4ed1-47a7-be00-6729af561d0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:59.626970 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-87c7b885-4ed1-47a7-be00-6729af561d0d tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 1.098315 Sep 27 10:09:59.629884 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1857/3409] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:58 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 1103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:59.636304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-153f8e3a-8900-49ec-ac82-8f6395907708 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:59.637231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-153f8e3a-8900-49ec-ac82-8f6395907708 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:59.645203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153f8e3a-8900-49ec-ac82-8f6395907708 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:59.646244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153f8e3a-8900-49ec-ac82-8f6395907708 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:59.646977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153f8e3a-8900-49ec-ac82-8f6395907708 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:59.648930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153f8e3a-8900-49ec-ac82-8f6395907708 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:59.649654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153f8e3a-8900-49ec-ac82-8f6395907708 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:59.652153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153f8e3a-8900-49ec-ac82-8f6395907708 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:59.799457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-153f8e3a-8900-49ec-ac82-8f6395907708 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:09:59.813930 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-153f8e3a-8900-49ec-ac82-8f6395907708 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1300 microversion: 2.1 time: 0.182724 Sep 27 10:09:59.815030 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1858/3410] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:59 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1300 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:09:59.830188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:09:59.831455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:09:59.844194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:59.845169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:59.845872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:59.847308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:09:59.847990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:09:59.848647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:09:59.977423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:00.026612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:00.028616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:00.033091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f09a5c8e-9ba0-4e55-817b-732aa14cd1e1 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1406 microversion: 2.54 time: 0.216613 Sep 27 10:10:00.035594 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1859/3411] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:09:59 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1406 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:00.455941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b72e2d66-67ff-47ab-b3f8-7b5355f6d976 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:00.469796 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b72e2d66-67ff-47ab-b3f8-7b5355f6d976 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1460 microversion: 2.1 time: 1.244268 Sep 27 10:10:00.475438 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1553/3412] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:09:59 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1460 bytes in 1251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:00.654636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-675f9eb9-ffae-4f1a-81f6-f4d57cdf023c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:00.655303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-675f9eb9-ffae-4f1a-81f6-f4d57cdf023c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:00.666513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675f9eb9-ffae-4f1a-81f6-f4d57cdf023c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:00.676492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675f9eb9-ffae-4f1a-81f6-f4d57cdf023c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:00.677014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675f9eb9-ffae-4f1a-81f6-f4d57cdf023c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:00.677854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675f9eb9-ffae-4f1a-81f6-f4d57cdf023c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:00.678323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675f9eb9-ffae-4f1a-81f6-f4d57cdf023c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:00.678760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675f9eb9-ffae-4f1a-81f6-f4d57cdf023c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:00.848825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8dd64045-80ec-4135-b4a2-5f143f45362c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:00.849781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8dd64045-80ec-4135-b4a2-5f143f45362c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:00.865916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dd64045-80ec-4135-b4a2-5f143f45362c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:00.868454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dd64045-80ec-4135-b4a2-5f143f45362c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:00.869304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dd64045-80ec-4135-b4a2-5f143f45362c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:00.872039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dd64045-80ec-4135-b4a2-5f143f45362c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:00.873423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dd64045-80ec-4135-b4a2-5f143f45362c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:00.874045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dd64045-80ec-4135-b4a2-5f143f45362c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:01.102531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-675f9eb9-ffae-4f1a-81f6-f4d57cdf023c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:01.122401 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-675f9eb9-ffae-4f1a-81f6-f4d57cdf023c tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 0.470001 Sep 27 10:10:01.125261 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1860/3413] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:00 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:01.147943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:01.149042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:01.180979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:01.181491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:01.185541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:01.187652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:01.188149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:01.191765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:01.497250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:01.512167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8dd64045-80ec-4135-b4a2-5f143f45362c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:01.526661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:01.530823 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8dd64045-80ec-4135-b4a2-5f143f45362c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 0.690991 Sep 27 10:10:01.535244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:01.538522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:01.539775 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1554/3414] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:00 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 701 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:01.544387 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a0d2061d-3125-4d74-8307-f2a4c460ff5b tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1406 microversion: 2.54 time: 0.400177 Sep 27 10:10:01.546320 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1861/3415] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:01 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1406 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:01.576793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-24431627-d1fb-44a6-80a0-50164d5df7a6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:01.578008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-24431627-d1fb-44a6-80a0-50164d5df7a6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:01.594565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24431627-d1fb-44a6-80a0-50164d5df7a6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:01.596721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24431627-d1fb-44a6-80a0-50164d5df7a6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:01.597353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24431627-d1fb-44a6-80a0-50164d5df7a6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:01.599055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24431627-d1fb-44a6-80a0-50164d5df7a6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:01.614646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24431627-d1fb-44a6-80a0-50164d5df7a6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:01.616058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24431627-d1fb-44a6-80a0-50164d5df7a6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:02.150894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f41189d-d04f-4446-9cf9-2a430f19dea3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:02.151734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f41189d-d04f-4446-9cf9-2a430f19dea3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:02.172720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f41189d-d04f-4446-9cf9-2a430f19dea3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:02.187129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f41189d-d04f-4446-9cf9-2a430f19dea3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:02.188309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f41189d-d04f-4446-9cf9-2a430f19dea3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:02.189518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f41189d-d04f-4446-9cf9-2a430f19dea3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:02.194910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f41189d-d04f-4446-9cf9-2a430f19dea3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:02.195878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f41189d-d04f-4446-9cf9-2a430f19dea3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:02.657590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-24431627-d1fb-44a6-80a0-50164d5df7a6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:02.671265 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-24431627-d1fb-44a6-80a0-50164d5df7a6 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 1.129972 Sep 27 10:10:02.677056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6f41189d-d04f-4446-9cf9-2a430f19dea3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:02.687114 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f41189d-d04f-4446-9cf9-2a430f19dea3 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 0.543658 Sep 27 10:10:02.687930 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1862/3416] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:02 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:02.708353 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1555/3417] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:01 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 1153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:02.708353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:02.708353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:02.709669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4465e76-c274-48f3-a87a-2a3a5b8d1d21 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:02.709669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4465e76-c274-48f3-a87a-2a3a5b8d1d21 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:02.718989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:02.719482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:02.720436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:02.720598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:02.721090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:02.721516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:02.730474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4465e76-c274-48f3-a87a-2a3a5b8d1d21 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:02.730983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4465e76-c274-48f3-a87a-2a3a5b8d1d21 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:02.731408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4465e76-c274-48f3-a87a-2a3a5b8d1d21 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:02.732129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4465e76-c274-48f3-a87a-2a3a5b8d1d21 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:02.732582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4465e76-c274-48f3-a87a-2a3a5b8d1d21 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:02.733034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4465e76-c274-48f3-a87a-2a3a5b8d1d21 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:02.818675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:02.819607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:02.820152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:02.820707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lazy-loading 'fault' on Instance uuid 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:10:03.020234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:03.028560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:03.037576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'd8cc9d27cf6944aa9f5ef81d7fb3c428', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f83c8276958489e84b0907523036097', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:03.040344 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5ee02881-af32-411c-b330-2a74acdfedc6 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 200 len: 1411 microversion: 2.54 time: 0.345346 Sep 27 10:10:03.041564 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1556/3418] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:02 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 1411 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:03.553225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4465e76-c274-48f3-a87a-2a3a5b8d1d21 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:03.562178 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4465e76-c274-48f3-a87a-2a3a5b8d1d21 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 0.873415 Sep 27 10:10:03.563380 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1863/3419] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:02 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 875 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:03.717287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-35860aca-514d-4ef6-a0cc-cbb7adad27c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:03.717963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-35860aca-514d-4ef6-a0cc-cbb7adad27c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:03.719472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-98868375-a858-47ce-a215-0bfbcfde4bd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:03.720347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-98868375-a858-47ce-a215-0bfbcfde4bd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:03.726994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35860aca-514d-4ef6-a0cc-cbb7adad27c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:03.727496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35860aca-514d-4ef6-a0cc-cbb7adad27c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:03.731484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35860aca-514d-4ef6-a0cc-cbb7adad27c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:03.732326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35860aca-514d-4ef6-a0cc-cbb7adad27c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:03.738031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35860aca-514d-4ef6-a0cc-cbb7adad27c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:03.738578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35860aca-514d-4ef6-a0cc-cbb7adad27c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:03.741641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98868375-a858-47ce-a215-0bfbcfde4bd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:03.742143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98868375-a858-47ce-a215-0bfbcfde4bd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:03.742589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98868375-a858-47ce-a215-0bfbcfde4bd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:03.743315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98868375-a858-47ce-a215-0bfbcfde4bd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:03.743770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98868375-a858-47ce-a215-0bfbcfde4bd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:03.744489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98868375-a858-47ce-a215-0bfbcfde4bd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:04.221411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-35860aca-514d-4ef6-a0cc-cbb7adad27c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:04.235769 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-35860aca-514d-4ef6-a0cc-cbb7adad27c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 0.522807 Sep 27 10:10:04.241600 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1557/3420] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:03 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:04.255611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:04.257107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] [instance: 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:04.265881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:04.267860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:04.268792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:04.269806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:04.270808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:04.271752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:04.298330 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] HTTP exception thrown: Instance 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 could not be found. Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Request method failure captured: Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 HTTP/1.1 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsvLlNY_9ZSHEa4_buoyInEWDsjQ6pAReu6zSfkc1XpEOHFb99ir8zP0fFH9ARoHorS9W1yZa-kDRICigkEXrnXaSP5I35lavuyfVpN3FBzcM_aaioguLNai3Punwo8Ev-oW09sdjAjD678_9p0YpP0jeR9mPuvgUTKTW-Q7yAXmoE8 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.54 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1f83c8276958489e84b0907523036097 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerShowV254Test-696618874 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1f83c8276958489e84b0907523036097"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1f83c8276958489e84b0907523036097"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1f83c8276958489e84b0907523036097"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1f83c8276958489e84b0907523036097"}]}] Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerShowV254Test-696618874 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1f83c8276958489e84b0907523036097 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerShowV254Test-696618874 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerShowV254Test-696618874-project Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: d8cc9d27cf6944aa9f5ef81d7fb3c428 Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerShowV254Test-696618874-project Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 could not be found. Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '7e85dfcb-035e-46c0-925f-eb0ca3ea15d2'} Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 could not be found. Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 could not be found. Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:04.303035 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:04.308436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:04.308436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:04.308436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:04.308436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:04.308436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:04.308436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 could not be found. Sep 27 10:10:04.308436 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:04.309826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] Returning 404 to user: Instance 7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:04.313103 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e3b34ec1-bc70-4fa1-9507-86af7b96f472 tempest-ServerShowV254Test-696618874 tempest-ServerShowV254Test-696618874-project] 10.222.0.26 "GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2" status: 404 len: 111 microversion: 2.54 time: 0.069144 Sep 27 10:10:04.320780 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1558/3421] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:04 2022] GET /compute/v2.1/servers/7e85dfcb-035e-46c0-925f-eb0ca3ea15d2 => generated 111 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:10:04.335752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-98868375-a858-47ce-a215-0bfbcfde4bd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:04.347875 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-98868375-a858-47ce-a215-0bfbcfde4bd5 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.635490 Sep 27 10:10:04.348883 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1864/3422] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:03 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:04.598070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d7ed38c2-d596-4ef7-8a10-1444ae1c7ef6 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:04.598972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d7ed38c2-d596-4ef7-8a10-1444ae1c7ef6 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:04.650818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ed38c2-d596-4ef7-8a10-1444ae1c7ef6 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:04.651611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ed38c2-d596-4ef7-8a10-1444ae1c7ef6 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:04.652172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ed38c2-d596-4ef7-8a10-1444ae1c7ef6 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:04.692812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ed38c2-d596-4ef7-8a10-1444ae1c7ef6 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:04.693321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ed38c2-d596-4ef7-8a10-1444ae1c7ef6 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:04.693849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ed38c2-d596-4ef7-8a10-1444ae1c7ef6 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:05.263685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9867904a-e4e3-40dd-a91f-4f66592098e1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:05.265010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9867904a-e4e3-40dd-a91f-4f66592098e1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:05.267192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:05.282208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9867904a-e4e3-40dd-a91f-4f66592098e1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:05.284139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9867904a-e4e3-40dd-a91f-4f66592098e1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:05.285838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9867904a-e4e3-40dd-a91f-4f66592098e1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:05.286783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9867904a-e4e3-40dd-a91f-4f66592098e1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:05.287418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9867904a-e4e3-40dd-a91f-4f66592098e1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:05.288066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9867904a-e4e3-40dd-a91f-4f66592098e1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:05.336267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=16 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:06.526054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9867904a-e4e3-40dd-a91f-4f66592098e1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:06.543401 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9867904a-e4e3-40dd-a91f-4f66592098e1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 1.283512 Sep 27 10:10:06.543995 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1865/3423] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:05 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 1285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:06.549539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a702475d-0812-4b23-91dc-318e5ac5c50e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:06.561374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a702475d-0812-4b23-91dc-318e5ac5c50e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:06.590648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a702475d-0812-4b23-91dc-318e5ac5c50e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:06.591423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a702475d-0812-4b23-91dc-318e5ac5c50e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:06.592033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a702475d-0812-4b23-91dc-318e5ac5c50e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:06.592951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a702475d-0812-4b23-91dc-318e5ac5c50e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:06.593598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a702475d-0812-4b23-91dc-318e5ac5c50e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:06.594187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a702475d-0812-4b23-91dc-318e5ac5c50e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:06.896074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d7ed38c2-d596-4ef7-8a10-1444ae1c7ef6 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:06.920462 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d7ed38c2-d596-4ef7-8a10-1444ae1c7ef6 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 2.329970 Sep 27 10:10:06.926464 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1559/3424] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:04 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 2345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:06.948755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2b23d571-447a-4f8c-a714-3015ba3cf09b req-02199c36-9d7c-4aae-9004-6d157adfc423 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a04c146e-494e-458e-843e-09c63c576cb3", "tag": "f45a2d54-220d-47e8-93bd-d90e6fda8e24"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:06.961654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2b23d571-447a-4f8c-a714-3015ba3cf09b req-02199c36-9d7c-4aae-9004-6d157adfc423 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:06.962629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2b23d571-447a-4f8c-a714-3015ba3cf09b req-02199c36-9d7c-4aae-9004-6d157adfc423 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:06.963232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2b23d571-447a-4f8c-a714-3015ba3cf09b req-02199c36-9d7c-4aae-9004-6d157adfc423 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:06.992733 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2b23d571-447a-4f8c-a714-3015ba3cf09b req-02199c36-9d7c-4aae-9004-6d157adfc423 service nova] Creating event network-changed:f45a2d54-220d-47e8-93bd-d90e6fda8e24 for instance a04c146e-494e-458e-843e-09c63c576cb3 on nch2-master-1 Sep 27 10:10:07.000074 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2b23d571-447a-4f8c-a714-3015ba3cf09b req-02199c36-9d7c-4aae-9004-6d157adfc423 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062893 Sep 27 10:10:07.002875 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1560/3425] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:07.140048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a702475d-0812-4b23-91dc-318e5ac5c50e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:07.148326 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a702475d-0812-4b23-91dc-318e5ac5c50e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.602214 Sep 27 10:10:07.156730 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1866/3426] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:06 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 611 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:07.577273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0496710d-f15a-43f3-8b44-ac579a9d6854 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:07.578445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0496710d-f15a-43f3-8b44-ac579a9d6854 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:07.596329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0496710d-f15a-43f3-8b44-ac579a9d6854 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:07.597241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0496710d-f15a-43f3-8b44-ac579a9d6854 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:07.598522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0496710d-f15a-43f3-8b44-ac579a9d6854 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:07.599830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0496710d-f15a-43f3-8b44-ac579a9d6854 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:07.602005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0496710d-f15a-43f3-8b44-ac579a9d6854 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:07.602746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0496710d-f15a-43f3-8b44-ac579a9d6854 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:07.956589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1710ccf1-824a-4f6a-9359-7b146908ee55 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:07.964189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1710ccf1-824a-4f6a-9359-7b146908ee55 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:07.986400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1710ccf1-824a-4f6a-9359-7b146908ee55 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:07.987978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1710ccf1-824a-4f6a-9359-7b146908ee55 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:07.988435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1710ccf1-824a-4f6a-9359-7b146908ee55 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:07.991636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1710ccf1-824a-4f6a-9359-7b146908ee55 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:07.994305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1710ccf1-824a-4f6a-9359-7b146908ee55 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:07.995851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1710ccf1-824a-4f6a-9359-7b146908ee55 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:08.242999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1710ccf1-824a-4f6a-9359-7b146908ee55 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:08.257081 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1710ccf1-824a-4f6a-9359-7b146908ee55 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 0.309022 Sep 27 10:10:08.259641 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1867/3427] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:07 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:08.620910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-64302b34-aa03-49f7-a4d1-8b4528854985 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Action: 'create', calling method: .version_select at 0x7f7f6273eb80>, body: {"keypair": {"name": "tempest-keypair-790039360", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ45jgSZ6EJOfX+pO/9Ju8hiBvy4OF9M9o+o57yqToC8rw3SrJFPOd9AeLmRyjhcYakwj4H6qns0a7c9R6jAmXTOhhobhzv38jqFY3iPeq3x0h9/8DNxrtpc+Dt9/5CDbw=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:08.640853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-64302b34-aa03-49f7-a4d1-8b4528854985 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Getting quotas for project a74a11e81faa4ca59f7c5662a3e8321f. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:10:08.651736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-64302b34-aa03-49f7-a4d1-8b4528854985 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Getting quotas for user a83af7f06aa94823989c466abc0ffa3e and project a74a11e81faa4ca59f7c5662a3e8321f. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:10:08.708297 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-64302b34-aa03-49f7-a4d1-8b4528854985 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.447198 Sep 27 10:10:08.709054 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1868/3428] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:10:08 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:08.743957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3d40e709-8569-4c85-8ed3-b5cc457a19e1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:08.745454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3d40e709-8569-4c85-8ed3-b5cc457a19e1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:08.757977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d40e709-8569-4c85-8ed3-b5cc457a19e1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:08.760129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d40e709-8569-4c85-8ed3-b5cc457a19e1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:08.762886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d40e709-8569-4c85-8ed3-b5cc457a19e1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:08.763856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d40e709-8569-4c85-8ed3-b5cc457a19e1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:08.766314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d40e709-8569-4c85-8ed3-b5cc457a19e1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:08.766944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d40e709-8569-4c85-8ed3-b5cc457a19e1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:08.786958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0496710d-f15a-43f3-8b44-ac579a9d6854 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:08.806367 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0496710d-f15a-43f3-8b44-ac579a9d6854 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 1.236481 Sep 27 10:10:08.830229 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1561/3429] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:07 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 1259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:09.283468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dbd069cb-d879-464c-932a-562183e31d70 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:09.284105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dbd069cb-d879-464c-932a-562183e31d70 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:09.308205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbd069cb-d879-464c-932a-562183e31d70 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:09.308774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbd069cb-d879-464c-932a-562183e31d70 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:09.310203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbd069cb-d879-464c-932a-562183e31d70 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:09.311542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbd069cb-d879-464c-932a-562183e31d70 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:09.324434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbd069cb-d879-464c-932a-562183e31d70 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:09.334687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbd069cb-d879-464c-932a-562183e31d70 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:09.575162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3d40e709-8569-4c85-8ed3-b5cc457a19e1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:09.604423 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3d40e709-8569-4c85-8ed3-b5cc457a19e1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.875760 Sep 27 10:10:09.608369 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1869/3430] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:08 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 898 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:09.838231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aa665794-1392-487b-bd21-12b1ff4eb73e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:09.839434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aa665794-1392-487b-bd21-12b1ff4eb73e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:09.895702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa665794-1392-487b-bd21-12b1ff4eb73e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:09.900877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa665794-1392-487b-bd21-12b1ff4eb73e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:09.902143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa665794-1392-487b-bd21-12b1ff4eb73e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:09.907990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa665794-1392-487b-bd21-12b1ff4eb73e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:09.911057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa665794-1392-487b-bd21-12b1ff4eb73e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:09.911678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa665794-1392-487b-bd21-12b1ff4eb73e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:10.507767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dbd069cb-d879-464c-932a-562183e31d70 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:10.519027 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dbd069cb-d879-464c-932a-562183e31d70 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 1.240569 Sep 27 10:10:10.519621 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1562/3431] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:09 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 1242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:10.636421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ccb42eba-6475-4708-a9cc-0fa3115603f2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:10.637314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ccb42eba-6475-4708-a9cc-0fa3115603f2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:10.659975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ccb42eba-6475-4708-a9cc-0fa3115603f2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:10.662172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aa665794-1392-487b-bd21-12b1ff4eb73e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:10.667482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ccb42eba-6475-4708-a9cc-0fa3115603f2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:10.668090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ccb42eba-6475-4708-a9cc-0fa3115603f2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:10.668876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ccb42eba-6475-4708-a9cc-0fa3115603f2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:10.669351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ccb42eba-6475-4708-a9cc-0fa3115603f2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:10.673155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ccb42eba-6475-4708-a9cc-0fa3115603f2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:10.678682 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aa665794-1392-487b-bd21-12b1ff4eb73e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 0.843487 Sep 27 10:10:10.693053 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1870/3432] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:09 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 847 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:11.089221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueTestJSON-server-1546971046", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:11.170351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] validate_networks() for [('4e799abf-8fbf-4193-9ed7-52de886b50f0', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:10:11.826297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ccb42eba-6475-4708-a9cc-0fa3115603f2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:11.840120 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ccb42eba-6475-4708-a9cc-0fa3115603f2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 1.205522 Sep 27 10:10:11.842903 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1563/3433] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:10 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 1212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:11.849268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-36575fa6-61db-4e8b-8135-7e0b20dd67ef tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:11.850145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-36575fa6-61db-4e8b-8135-7e0b20dd67ef tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:11.863330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36575fa6-61db-4e8b-8135-7e0b20dd67ef tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:11.863993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36575fa6-61db-4e8b-8135-7e0b20dd67ef tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:11.866092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36575fa6-61db-4e8b-8135-7e0b20dd67ef tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:11.867055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36575fa6-61db-4e8b-8135-7e0b20dd67ef tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:11.875584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36575fa6-61db-4e8b-8135-7e0b20dd67ef tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:11.876301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-36575fa6-61db-4e8b-8135-7e0b20dd67ef tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:12.112390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-36575fa6-61db-4e8b-8135-7e0b20dd67ef tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:12.126038 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-36575fa6-61db-4e8b-8135-7e0b20dd67ef tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 0.281768 Sep 27 10:10:12.142719 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1564/3434] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:11 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:12.155307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ceea285e-f716-49af-bb25-861047bfe967 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:12.161161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ceea285e-f716-49af-bb25-861047bfe967 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:12.177317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceea285e-f716-49af-bb25-861047bfe967 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:12.178213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceea285e-f716-49af-bb25-861047bfe967 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:12.178944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceea285e-f716-49af-bb25-861047bfe967 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:12.183839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceea285e-f716-49af-bb25-861047bfe967 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:12.184634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceea285e-f716-49af-bb25-861047bfe967 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:12.185882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceea285e-f716-49af-bb25-861047bfe967 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:12.452105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:10:12.453413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:10:12.454390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:10:12.454965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:10:12.455636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:10:12.472496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:12.473795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:12.479232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:12.480425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:12.481472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:12.482071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:12.534393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Getting quotas for project c24fd5e915af4b6c872d69b99e7375ce. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:10:12.545892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Getting quotas for user b11313762b324174a9a5f362d1cf4bdd and project c24fd5e915af4b6c872d69b99e7375ce. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:10:12.575628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:10:12.585721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:10:12.676566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ceea285e-f716-49af-bb25-861047bfe967 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:12.691522 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ceea285e-f716-49af-bb25-861047bfe967 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 0.542328 Sep 27 10:10:12.693401 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1565/3435] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:12 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:12.700376 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-598b5ec4-8ab3-4584-9420-69371110fed4 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.846042 Sep 27 10:10:12.709737 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1871/3436] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:10:10 2022] POST /compute/v2.1/servers => generated 383 bytes in 1856 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:10:12.715743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-46df0694-2167-4822-a233-3e112494baf6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:12.721112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-46df0694-2167-4822-a233-3e112494baf6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:12.891905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-de3b9f2d-c924-4e3e-b274-345d1bbdcb00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:12.895201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-de3b9f2d-c924-4e3e-b274-345d1bbdcb00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:12.915187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de3b9f2d-c924-4e3e-b274-345d1bbdcb00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:12.917005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de3b9f2d-c924-4e3e-b274-345d1bbdcb00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:12.917989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de3b9f2d-c924-4e3e-b274-345d1bbdcb00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:12.919019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de3b9f2d-c924-4e3e-b274-345d1bbdcb00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:12.920390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de3b9f2d-c924-4e3e-b274-345d1bbdcb00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:12.921305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de3b9f2d-c924-4e3e-b274-345d1bbdcb00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:13.264023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-46df0694-2167-4822-a233-3e112494baf6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:13.298486 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-46df0694-2167-4822-a233-3e112494baf6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1214 microversion: 2.1 time: 0.570944 Sep 27 10:10:13.299936 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1566/3437] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:12 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1214 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:13.314504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5c9b7314-1e56-469b-86c7-9f0152492f82 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:13.315729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5c9b7314-1e56-469b-86c7-9f0152492f82 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:13.340394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c9b7314-1e56-469b-86c7-9f0152492f82 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:13.341600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c9b7314-1e56-469b-86c7-9f0152492f82 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:13.342639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c9b7314-1e56-469b-86c7-9f0152492f82 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:13.343560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c9b7314-1e56-469b-86c7-9f0152492f82 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:13.346868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c9b7314-1e56-469b-86c7-9f0152492f82 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:13.347718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c9b7314-1e56-469b-86c7-9f0152492f82 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:13.828690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5c9b7314-1e56-469b-86c7-9f0152492f82 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:13.854641 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5c9b7314-1e56-469b-86c7-9f0152492f82 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 0.549405 Sep 27 10:10:13.875273 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1567/3438] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:13 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:13.886499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-1286626852", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"hello": "world"}, "accessIPv4": "1.1.1.1", "accessIPv6": "0000:0000:0000:0000:0000:babe:220.12.22.2", "adminPass": "***", "networks": [{"uuid": "3f981001-2e8b-4ada-9b48-357a00f08343"}], "security_groups": [{"name": "tempest-securitygroup--745445245"}], "key_name": "tempest-keypair-790039360", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=", "OS-DCF:diskConfig": "}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:14.323963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] validate_networks() for [('3f981001-2e8b-4ada-9b48-357a00f08343', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:10:14.794221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-de3b9f2d-c924-4e3e-b274-345d1bbdcb00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:14.804909 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-de3b9f2d-c924-4e3e-b274-345d1bbdcb00 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 1.937811 Sep 27 10:10:14.806549 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1872/3439] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:12 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 1941 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:14.815581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-546797fd-39bc-4b13-b44d-c4e36ae4cb86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:14.817103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-546797fd-39bc-4b13-b44d-c4e36ae4cb86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:14.825146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-546797fd-39bc-4b13-b44d-c4e36ae4cb86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:14.825926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-546797fd-39bc-4b13-b44d-c4e36ae4cb86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:14.827512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-546797fd-39bc-4b13-b44d-c4e36ae4cb86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:14.828731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-546797fd-39bc-4b13-b44d-c4e36ae4cb86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:14.829377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-546797fd-39bc-4b13-b44d-c4e36ae4cb86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:14.829977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-546797fd-39bc-4b13-b44d-c4e36ae4cb86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:15.777800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-546797fd-39bc-4b13-b44d-c4e36ae4cb86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:15.799730 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-546797fd-39bc-4b13-b44d-c4e36ae4cb86 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 0.990966 Sep 27 10:10:15.800404 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1873/3440] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:14 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 993 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:15.821748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-956eb1cd-5f56-4850-9eec-6b7a8c9ec449 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:15.822383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-956eb1cd-5f56-4850-9eec-6b7a8c9ec449 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:15.849222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-956eb1cd-5f56-4850-9eec-6b7a8c9ec449 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:15.858096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-956eb1cd-5f56-4850-9eec-6b7a8c9ec449 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:15.861007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-956eb1cd-5f56-4850-9eec-6b7a8c9ec449 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:15.862018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-956eb1cd-5f56-4850-9eec-6b7a8c9ec449 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:15.865329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-956eb1cd-5f56-4850-9eec-6b7a8c9ec449 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:15.866017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-956eb1cd-5f56-4850-9eec-6b7a8c9ec449 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:16.382844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-956eb1cd-5f56-4850-9eec-6b7a8c9ec449 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:16.393517 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-956eb1cd-5f56-4850-9eec-6b7a8c9ec449 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1272 microversion: 2.1 time: 0.591947 Sep 27 10:10:16.412392 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1874/3441] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:15 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1272 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:16.434260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-245ba980-0516-4298-942d-6f69f35e2b72 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:16.434840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-245ba980-0516-4298-942d-6f69f35e2b72 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:16.457877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-245ba980-0516-4298-942d-6f69f35e2b72 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:16.461089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-245ba980-0516-4298-942d-6f69f35e2b72 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:16.462790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-245ba980-0516-4298-942d-6f69f35e2b72 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:16.465252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-245ba980-0516-4298-942d-6f69f35e2b72 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:16.467368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-245ba980-0516-4298-942d-6f69f35e2b72 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:16.470758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-245ba980-0516-4298-942d-6f69f35e2b72 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:16.918913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Getting quotas for project a74a11e81faa4ca59f7c5662a3e8321f. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:10:16.945938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Getting quotas for user a83af7f06aa94823989c466abc0ffa3e and project a74a11e81faa4ca59f7c5662a3e8321f. Resources: dict_keys(['metadata_items']) {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:10:16.989684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:10:16.993681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:10:16.994556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:10:16.997827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:10:16.998468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:10:17.020150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:17.024418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:17.031444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:17.032446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:17.033100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:17.033665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:17.091520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Getting quotas for project a74a11e81faa4ca59f7c5662a3e8321f. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:10:17.113379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Getting quotas for user a83af7f06aa94823989c466abc0ffa3e and project a74a11e81faa4ca59f7c5662a3e8321f. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:10:17.154893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-245ba980-0516-4298-942d-6f69f35e2b72 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:17.165585 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-245ba980-0516-4298-942d-6f69f35e2b72 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 0.748928 Sep 27 10:10:17.165819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:10:17.169194 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1875/3442] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:16 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 756 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:17.174685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:10:17.176930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07070bb6-f052-4833-8741-0094396d0d8e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:17.181693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-07070bb6-f052-4833-8741-0094396d0d8e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:17.200524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07070bb6-f052-4833-8741-0094396d0d8e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:17.202573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07070bb6-f052-4833-8741-0094396d0d8e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:17.203050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07070bb6-f052-4833-8741-0094396d0d8e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:17.203796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07070bb6-f052-4833-8741-0094396d0d8e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:17.204259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07070bb6-f052-4833-8741-0094396d0d8e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:17.207660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07070bb6-f052-4833-8741-0094396d0d8e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:17.309093 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-867b104e-6c12-4346-9c59-0ee8efed9ff8 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 3.428626 Sep 27 10:10:17.309750 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1568/3443] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:10:13 2022] POST /compute/v2.1/servers => generated 406 bytes in 3434 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:10:17.328874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa2316a0-a9c2-43b3-972e-eeb549bf3f72 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Action: 'create', calling method: .version_select at 0x7f7f62defe50>, body: {"keypair": {"name": "tempest-keypair-1829774399", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBkz2Wnupau+HPfcjK2eKr2uSZZU24UHPUdge2HTtfz+KCw9DpYcc8Cb391+1UHJ6T+Iyvn+S5NhIX++hwowJw3HFHvOyYYi50NBMyqdKVBWqXiiyrVsxIq9WnRBLXSq4A=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:17.374870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-aa2316a0-a9c2-43b3-972e-eeb549bf3f72 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Getting quotas for project f86d6a96f893464f81868717f0531bd3. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:10:17.388891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-aa2316a0-a9c2-43b3-972e-eeb549bf3f72 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Getting quotas for user 04a5f2b1d0134e529f25c227954b5f20 and project f86d6a96f893464f81868717f0531bd3. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:10:17.464557 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aa2316a0-a9c2-43b3-972e-eeb549bf3f72 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.42 time: 0.147720 Sep 27 10:10:17.465376 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1569/3444] 10.222.0.26 () {62 vars in 1200 bytes} [Tue Sep 27 10:10:17 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 155 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 27 10:10:17.477776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-09c90a92-2f64-4fd6-b2b5-86c1e5737adc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:17.479808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-09c90a92-2f64-4fd6-b2b5-86c1e5737adc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:17.496783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c90a92-2f64-4fd6-b2b5-86c1e5737adc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:17.496783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c90a92-2f64-4fd6-b2b5-86c1e5737adc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:17.502873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c90a92-2f64-4fd6-b2b5-86c1e5737adc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:17.504178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c90a92-2f64-4fd6-b2b5-86c1e5737adc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:17.505052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c90a92-2f64-4fd6-b2b5-86c1e5737adc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:17.509945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c90a92-2f64-4fd6-b2b5-86c1e5737adc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:18.001600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-09c90a92-2f64-4fd6-b2b5-86c1e5737adc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:18.013588 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-09c90a92-2f64-4fd6-b2b5-86c1e5737adc tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 0.546868 Sep 27 10:10:18.015554 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1570/3445] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:17 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:18.033848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dc710c2f-1f9c-460f-ae87-84f40d83b468 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:18.035345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dc710c2f-1f9c-460f-ae87-84f40d83b468 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:18.039420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=93,nova_api:DELETE=1,nova_api:UPDATE=2,nova_api:INSERT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:18.174651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dc710c2f-1f9c-460f-ae87-84f40d83b468 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:18.187228 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dc710c2f-1f9c-460f-ae87-84f40d83b468 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1269 microversion: 2.1 time: 0.170210 Sep 27 10:10:18.201217 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1571/3446] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:18 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1269 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:18.213618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-48227c9d-f97c-4142-a967-128b9c66f2a6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:18.214771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-48227c9d-f97c-4142-a967-128b9c66f2a6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:18.222318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48227c9d-f97c-4142-a967-128b9c66f2a6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:18.223193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48227c9d-f97c-4142-a967-128b9c66f2a6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:18.223923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48227c9d-f97c-4142-a967-128b9c66f2a6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:18.224932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48227c9d-f97c-4142-a967-128b9c66f2a6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:18.225665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48227c9d-f97c-4142-a967-128b9c66f2a6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:18.226527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48227c9d-f97c-4142-a967-128b9c66f2a6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:18.498317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-07070bb6-f052-4833-8741-0094396d0d8e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:18.508757 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-07070bb6-f052-4833-8741-0094396d0d8e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 1.338104 Sep 27 10:10:18.512951 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1876/3447] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:17 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 1343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:18.526173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e36b6b4-b30e-4a36-acc5-e241f78bf4c7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:18.527418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2e36b6b4-b30e-4a36-acc5-e241f78bf4c7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:18.552265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e36b6b4-b30e-4a36-acc5-e241f78bf4c7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:18.572970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e36b6b4-b30e-4a36-acc5-e241f78bf4c7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:18.575116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e36b6b4-b30e-4a36-acc5-e241f78bf4c7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:18.589023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e36b6b4-b30e-4a36-acc5-e241f78bf4c7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:18.592477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e36b6b4-b30e-4a36-acc5-e241f78bf4c7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:18.593724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e36b6b4-b30e-4a36-acc5-e241f78bf4c7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:19.178535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-48227c9d-f97c-4142-a967-128b9c66f2a6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:19.190581 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-48227c9d-f97c-4142-a967-128b9c66f2a6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1314 microversion: 2.1 time: 0.987154 Sep 27 10:10:19.193250 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1572/3448] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:18 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1314 bytes in 991 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:19.204862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1a9cd504-eed0-4b33-b9be-54e9a33f2015 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:19.215291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1a9cd504-eed0-4b33-b9be-54e9a33f2015 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:19.239919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a9cd504-eed0-4b33-b9be-54e9a33f2015 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:19.240537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a9cd504-eed0-4b33-b9be-54e9a33f2015 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:19.241018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a9cd504-eed0-4b33-b9be-54e9a33f2015 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:19.256088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a9cd504-eed0-4b33-b9be-54e9a33f2015 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:19.258633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a9cd504-eed0-4b33-b9be-54e9a33f2015 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:19.259400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a9cd504-eed0-4b33-b9be-54e9a33f2015 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:19.372285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2e36b6b4-b30e-4a36-acc5-e241f78bf4c7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:19.389911 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2e36b6b4-b30e-4a36-acc5-e241f78bf4c7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 0.873695 Sep 27 10:10:19.391978 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1877/3449] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:18 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 878 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:19.411247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-74d39b43-9b58-4ce5-946c-d4724fcdfc74 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:19.413482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-74d39b43-9b58-4ce5-946c-d4724fcdfc74 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:19.432170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74d39b43-9b58-4ce5-946c-d4724fcdfc74 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:19.433562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74d39b43-9b58-4ce5-946c-d4724fcdfc74 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:19.435609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74d39b43-9b58-4ce5-946c-d4724fcdfc74 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:19.438986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74d39b43-9b58-4ce5-946c-d4724fcdfc74 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:19.439745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74d39b43-9b58-4ce5-946c-d4724fcdfc74 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:19.440851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74d39b43-9b58-4ce5-946c-d4724fcdfc74 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:19.594655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-74d39b43-9b58-4ce5-946c-d4724fcdfc74 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:19.627052 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-74d39b43-9b58-4ce5-946c-d4724fcdfc74 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1321 microversion: 2.1 time: 0.233030 Sep 27 10:10:19.644693 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1878/3450] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:19 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1321 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:19.651370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9625d041-f419-4a7a-bda1-d31b06014edd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:19.652146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9625d041-f419-4a7a-bda1-d31b06014edd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:19.666978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9625d041-f419-4a7a-bda1-d31b06014edd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:19.669070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9625d041-f419-4a7a-bda1-d31b06014edd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:19.669717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9625d041-f419-4a7a-bda1-d31b06014edd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:19.670579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9625d041-f419-4a7a-bda1-d31b06014edd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:19.673616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9625d041-f419-4a7a-bda1-d31b06014edd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:19.674216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9625d041-f419-4a7a-bda1-d31b06014edd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:19.812106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1a9cd504-eed0-4b33-b9be-54e9a33f2015 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:19.856983 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1a9cd504-eed0-4b33-b9be-54e9a33f2015 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 0.639877 Sep 27 10:10:19.858329 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1573/3451] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:19 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:20.151058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9625d041-f419-4a7a-bda1-d31b06014edd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:20.186288 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9625d041-f419-4a7a-bda1-d31b06014edd tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.551095 Sep 27 10:10:20.188676 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1879/3452] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:19 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:20.222485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ae62fcf5-1135-4ad2-80b7-815d632de97a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:20.223786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ae62fcf5-1135-4ad2-80b7-815d632de97a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:20.245605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae62fcf5-1135-4ad2-80b7-815d632de97a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:20.254409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae62fcf5-1135-4ad2-80b7-815d632de97a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:20.255255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae62fcf5-1135-4ad2-80b7-815d632de97a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:20.258024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae62fcf5-1135-4ad2-80b7-815d632de97a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:20.259557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae62fcf5-1135-4ad2-80b7-815d632de97a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:20.264175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae62fcf5-1135-4ad2-80b7-815d632de97a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:20.285265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:20.429063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a2a4eccc-d68d-4d01-b789-921085b586ed tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:20.431384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a2a4eccc-d68d-4d01-b789-921085b586ed tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:20.452445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2a4eccc-d68d-4d01-b789-921085b586ed tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:20.452979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2a4eccc-d68d-4d01-b789-921085b586ed tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:20.453440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2a4eccc-d68d-4d01-b789-921085b586ed tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:20.454210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2a4eccc-d68d-4d01-b789-921085b586ed tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:20.454624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2a4eccc-d68d-4d01-b789-921085b586ed tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:20.455050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2a4eccc-d68d-4d01-b789-921085b586ed tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:20.459716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:21.155856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a2a4eccc-d68d-4d01-b789-921085b586ed tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:21.177549 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a2a4eccc-d68d-4d01-b789-921085b586ed tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 0.755902 Sep 27 10:10:21.178151 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1880/3453] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:20 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 758 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:21.185972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dad00915-aa8d-4744-8fc1-e0f0bbc3a071 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:21.187106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dad00915-aa8d-4744-8fc1-e0f0bbc3a071 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:21.206541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dad00915-aa8d-4744-8fc1-e0f0bbc3a071 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:21.207328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dad00915-aa8d-4744-8fc1-e0f0bbc3a071 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:21.208132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dad00915-aa8d-4744-8fc1-e0f0bbc3a071 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:21.209086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dad00915-aa8d-4744-8fc1-e0f0bbc3a071 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:21.209717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dad00915-aa8d-4744-8fc1-e0f0bbc3a071 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:21.210355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dad00915-aa8d-4744-8fc1-e0f0bbc3a071 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:21.399268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ae62fcf5-1135-4ad2-80b7-815d632de97a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:21.423156 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ae62fcf5-1135-4ad2-80b7-815d632de97a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1314 microversion: 2.1 time: 1.206017 Sep 27 10:10:21.424319 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1574/3454] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:20 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1314 bytes in 1208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:21.476646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fab59ac3-4370-4a00-9fcc-49ca026f371b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:21.478018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fab59ac3-4370-4a00-9fcc-49ca026f371b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:21.497551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fab59ac3-4370-4a00-9fcc-49ca026f371b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:21.498403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fab59ac3-4370-4a00-9fcc-49ca026f371b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:21.499082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fab59ac3-4370-4a00-9fcc-49ca026f371b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:21.500022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fab59ac3-4370-4a00-9fcc-49ca026f371b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:21.501558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fab59ac3-4370-4a00-9fcc-49ca026f371b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:21.502512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fab59ac3-4370-4a00-9fcc-49ca026f371b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:21.789937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dad00915-aa8d-4744-8fc1-e0f0bbc3a071 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:21.805295 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dad00915-aa8d-4744-8fc1-e0f0bbc3a071 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1327 microversion: 2.1 time: 0.621469 Sep 27 10:10:21.808223 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1881/3455] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:21 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1327 bytes in 629 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:21.822230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-74035377-d225-492b-ae96-1fe5db47f19b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:21.826285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-74035377-d225-492b-ae96-1fe5db47f19b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:21.842821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74035377-d225-492b-ae96-1fe5db47f19b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:21.843857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74035377-d225-492b-ae96-1fe5db47f19b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:21.844523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74035377-d225-492b-ae96-1fe5db47f19b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:21.846250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74035377-d225-492b-ae96-1fe5db47f19b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:21.846891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74035377-d225-492b-ae96-1fe5db47f19b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:21.847477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74035377-d225-492b-ae96-1fe5db47f19b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:22.304077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fab59ac3-4370-4a00-9fcc-49ca026f371b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:22.314841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-74035377-d225-492b-ae96-1fe5db47f19b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:22.322062 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fab59ac3-4370-4a00-9fcc-49ca026f371b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 0.859514 Sep 27 10:10:22.327438 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-74035377-d225-492b-ae96-1fe5db47f19b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.517775 Sep 27 10:10:22.332998 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1575/3456] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:21 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 908 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:22.337105 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1882/3457] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:21 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:22.351929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e9840a11-874d-4244-af66-2f53dc0bde4d req-808211bf-e1ab-4037-ad96-d1316962517b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a04c146e-494e-458e-843e-09c63c576cb3", "name": "network-vif-plugged", "status": "completed", "tag": "f45a2d54-220d-47e8-93bd-d90e6fda8e24"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:22.352721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-113f5516-d784-4c65-bdf1-3bf185e203dd tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:22.355143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-113f5516-d784-4c65-bdf1-3bf185e203dd tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:22.367604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:22.374277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113f5516-d784-4c65-bdf1-3bf185e203dd tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:22.375638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113f5516-d784-4c65-bdf1-3bf185e203dd tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:22.375883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113f5516-d784-4c65-bdf1-3bf185e203dd tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:22.378922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113f5516-d784-4c65-bdf1-3bf185e203dd tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:22.379405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113f5516-d784-4c65-bdf1-3bf185e203dd tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:22.379848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113f5516-d784-4c65-bdf1-3bf185e203dd tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:22.380782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e9840a11-874d-4244-af66-2f53dc0bde4d req-808211bf-e1ab-4037-ad96-d1316962517b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:22.384298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e9840a11-874d-4244-af66-2f53dc0bde4d req-808211bf-e1ab-4037-ad96-d1316962517b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:22.384849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e9840a11-874d-4244-af66-2f53dc0bde4d req-808211bf-e1ab-4037-ad96-d1316962517b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:22.419480 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e9840a11-874d-4244-af66-2f53dc0bde4d req-808211bf-e1ab-4037-ad96-d1316962517b service nova] Creating event network-vif-plugged:f45a2d54-220d-47e8-93bd-d90e6fda8e24 for instance a04c146e-494e-458e-843e-09c63c576cb3 on nch2-master-1 Sep 27 10:10:22.428597 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e9840a11-874d-4244-af66-2f53dc0bde4d req-808211bf-e1ab-4037-ad96-d1316962517b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091831 Sep 27 10:10:22.430994 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1576/3458] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:22.462276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-12fe9c6e-2c8b-42d0-aee1-6304713f6d56 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:22.463004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-12fe9c6e-2c8b-42d0-aee1-6304713f6d56 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:22.469527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12fe9c6e-2c8b-42d0-aee1-6304713f6d56 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:22.470251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12fe9c6e-2c8b-42d0-aee1-6304713f6d56 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:22.471229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12fe9c6e-2c8b-42d0-aee1-6304713f6d56 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:22.472177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12fe9c6e-2c8b-42d0-aee1-6304713f6d56 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:22.472902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12fe9c6e-2c8b-42d0-aee1-6304713f6d56 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:22.473562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12fe9c6e-2c8b-42d0-aee1-6304713f6d56 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:22.508372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:22.566989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=10 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:22.721882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-113f5516-d784-4c65-bdf1-3bf185e203dd tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:22.737488 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-113f5516-d784-4c65-bdf1-3bf185e203dd tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 0.399253 Sep 27 10:10:22.738705 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1883/3459] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:22 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:22.743864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-60194c6a-2bb2-436e-bc88-2e6b5f5cc95e req-75db9eca-2ee1-47ab-b53b-c79ac34ed51e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3651e4da-f0ce-4e37-82ac-9b81a1564a1d", "tag": "87d895c7-9525-40c2-baef-ff15ac876653"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:22.766424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-60194c6a-2bb2-436e-bc88-2e6b5f5cc95e req-75db9eca-2ee1-47ab-b53b-c79ac34ed51e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:22.766902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-60194c6a-2bb2-436e-bc88-2e6b5f5cc95e req-75db9eca-2ee1-47ab-b53b-c79ac34ed51e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:22.767514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-60194c6a-2bb2-436e-bc88-2e6b5f5cc95e req-75db9eca-2ee1-47ab-b53b-c79ac34ed51e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:22.794113 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-60194c6a-2bb2-436e-bc88-2e6b5f5cc95e req-75db9eca-2ee1-47ab-b53b-c79ac34ed51e service nova] Creating event network-changed:87d895c7-9525-40c2-baef-ff15ac876653 for instance 3651e4da-f0ce-4e37-82ac-9b81a1564a1d on nch1-master-1 Sep 27 10:10:22.800814 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-60194c6a-2bb2-436e-bc88-2e6b5f5cc95e req-75db9eca-2ee1-47ab-b53b-c79ac34ed51e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060717 Sep 27 10:10:22.801951 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1884/3460] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:22.846233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8ea77810-5149-4458-ac61-bf949de54586 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:22.847031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8ea77810-5149-4458-ac61-bf949de54586 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:22.858042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ea77810-5149-4458-ac61-bf949de54586 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:22.859948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ea77810-5149-4458-ac61-bf949de54586 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:22.862139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ea77810-5149-4458-ac61-bf949de54586 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:22.863911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ea77810-5149-4458-ac61-bf949de54586 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:22.864948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ea77810-5149-4458-ac61-bf949de54586 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:22.880809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ea77810-5149-4458-ac61-bf949de54586 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:22.959527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-12fe9c6e-2c8b-42d0-aee1-6304713f6d56 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:22.968062 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-12fe9c6e-2c8b-42d0-aee1-6304713f6d56 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1314 microversion: 2.1 time: 0.510507 Sep 27 10:10:22.968916 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1577/3461] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:22 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1314 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:23.016798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8ea77810-5149-4458-ac61-bf949de54586 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:23.030866 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8ea77810-5149-4458-ac61-bf949de54586 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1327 microversion: 2.1 time: 0.189557 Sep 27 10:10:23.032797 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1885/3462] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:22 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1327 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:23.371303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b7326e9-33fe-4127-878c-8b0384bfe578 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:23.372081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3b7326e9-33fe-4127-878c-8b0384bfe578 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:23.372563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07287833-58b2-447a-a8e8-933719100be8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:23.373216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-07287833-58b2-447a-a8e8-933719100be8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:23.384955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b7326e9-33fe-4127-878c-8b0384bfe578 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:23.385596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b7326e9-33fe-4127-878c-8b0384bfe578 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:23.390679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07287833-58b2-447a-a8e8-933719100be8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:23.391182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07287833-58b2-447a-a8e8-933719100be8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:23.391614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b7326e9-33fe-4127-878c-8b0384bfe578 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:23.391771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07287833-58b2-447a-a8e8-933719100be8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:23.392327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07287833-58b2-447a-a8e8-933719100be8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:23.392697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b7326e9-33fe-4127-878c-8b0384bfe578 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:23.392820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07287833-58b2-447a-a8e8-933719100be8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:23.393215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07287833-58b2-447a-a8e8-933719100be8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:23.393971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b7326e9-33fe-4127-878c-8b0384bfe578 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:23.394499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b7326e9-33fe-4127-878c-8b0384bfe578 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:23.696456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3b7326e9-33fe-4127-878c-8b0384bfe578 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:23.705298 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b7326e9-33fe-4127-878c-8b0384bfe578 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1512 microversion: 2.1 time: 0.337983 Sep 27 10:10:23.708065 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1578/3463] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:23 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1512 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:23.733188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-07287833-58b2-447a-a8e8-933719100be8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:23.743675 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-07287833-58b2-447a-a8e8-933719100be8 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.374826 Sep 27 10:10:23.745156 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1886/3464] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:23 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:23.769095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d9795d61-06e5-414b-b2de-75ec81930314 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:23.769819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d9795d61-06e5-414b-b2de-75ec81930314 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:23.788015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9795d61-06e5-414b-b2de-75ec81930314 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:23.788529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9795d61-06e5-414b-b2de-75ec81930314 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:23.788994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9795d61-06e5-414b-b2de-75ec81930314 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:23.789708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9795d61-06e5-414b-b2de-75ec81930314 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:23.806655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9795d61-06e5-414b-b2de-75ec81930314 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:23.807251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9795d61-06e5-414b-b2de-75ec81930314 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:23.986052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-device-tagging-server-1535827868", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "config_drive": true, "networks": [{"uuid": "d5f1dbd2-2aae-4893-b238-a525ee70f3b8"}, {"port": "5c514831-defe-416f-aec5-712e750a0e3b", "tag": "port-1"}, {"port": "ee0efbef-fa4b-4072-b28f-b4422ccb9afa", "tag": "port-2"}, {"uuid": "fb36e9dc-c611-4d79-9c91-a07d03a719a3", "tag": "net-1"}, {"uuid": "fb36e9dc-c611-4d79-9c91-a07d03a719a3"}, {"uuid": "494d8f5e-38aa-4723-896f-a3615050b661", "fixed_ip": "10.2.2.100", "tag": "net-2-100"}, {"uuid": "494d8f5e-38aa-4723-896f-a3615050b661", "fixed_ip": "10.2.2.200", "tag": "net-2-200"}], "block_device_mapping_v2": [{"uuid": "52fbc488-d91e-4ac4-b7b6-28284e31df30", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "tag": "boot"}, {"uuid": "1454250c-f356-4053-83eb-5c69c0a805d7", "source_type": "volume", "destination_type": "volume", "boot_index": 1, "tag": "other"}, {"uuid": "1caf2529-26ae-48b5-83d4-b87bb80b3024", "source_type": "volume", "destination_type": "volume", "boot_index": 2}], "security_groups": [{"name": "tempest-securitygroup--163036777"}], "key_name": "tempest-keypair-1829774399", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:24.187618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d9795d61-06e5-414b-b2de-75ec81930314 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:24.199290 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d9795d61-06e5-414b-b2de-75ec81930314 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 0.434515 Sep 27 10:10:24.202418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] validate_networks() for [('d5f1dbd2-2aae-4893-b238-a525ee70f3b8', None, None, None, None, None), (None, None, '5c514831-defe-416f-aec5-712e750a0e3b', None, None, None), (None, None, 'ee0efbef-fa4b-4072-b28f-b4422ccb9afa', None, None, None), ('fb36e9dc-c611-4d79-9c91-a07d03a719a3', None, None, None, None, None), ('fb36e9dc-c611-4d79-9c91-a07d03a719a3', None, None, None, None, None), ('494d8f5e-38aa-4723-896f-a3615050b661', '10.2.2.100', None, None, None, None), ('494d8f5e-38aa-4723-896f-a3615050b661', '10.2.2.200', None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:10:24.209562 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1579/3465] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:23 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:24.215560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-17ca1ca5-04b6-402b-9539-8895122da2a7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:24.216281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-17ca1ca5-04b6-402b-9539-8895122da2a7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:24.223520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17ca1ca5-04b6-402b-9539-8895122da2a7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:24.224279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17ca1ca5-04b6-402b-9539-8895122da2a7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:24.225773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17ca1ca5-04b6-402b-9539-8895122da2a7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:24.226727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17ca1ca5-04b6-402b-9539-8895122da2a7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:24.227361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17ca1ca5-04b6-402b-9539-8895122da2a7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:24.227975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17ca1ca5-04b6-402b-9539-8895122da2a7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:24.430745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-17ca1ca5-04b6-402b-9539-8895122da2a7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:24.462670 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-17ca1ca5-04b6-402b-9539-8895122da2a7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1314 microversion: 2.1 time: 0.251554 Sep 27 10:10:24.463586 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1580/3466] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:24 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1314 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:24.469630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0eb4745d-a13b-44db-a5be-9efa9a917d39 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:24.470514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0eb4745d-a13b-44db-a5be-9efa9a917d39 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:24.478610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0eb4745d-a13b-44db-a5be-9efa9a917d39 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:24.480244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0eb4745d-a13b-44db-a5be-9efa9a917d39 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:24.480866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0eb4745d-a13b-44db-a5be-9efa9a917d39 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:24.482081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0eb4745d-a13b-44db-a5be-9efa9a917d39 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:24.482732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0eb4745d-a13b-44db-a5be-9efa9a917d39 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:24.483363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0eb4745d-a13b-44db-a5be-9efa9a917d39 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:24.878876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0eb4745d-a13b-44db-a5be-9efa9a917d39 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:24.893118 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0eb4745d-a13b-44db-a5be-9efa9a917d39 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1394 microversion: 2.1 time: 0.427481 Sep 27 10:10:24.894032 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1581/3467] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:24 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1394 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:24.900373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-24deb325-4639-47de-bf92-429fb08b8397 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:24.901621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-24deb325-4639-47de-bf92-429fb08b8397 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:24.911916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24deb325-4639-47de-bf92-429fb08b8397 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:24.912966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24deb325-4639-47de-bf92-429fb08b8397 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:24.913704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24deb325-4639-47de-bf92-429fb08b8397 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:24.914697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24deb325-4639-47de-bf92-429fb08b8397 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:24.915337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24deb325-4639-47de-bf92-429fb08b8397 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:24.916246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24deb325-4639-47de-bf92-429fb08b8397 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:25.460851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-24deb325-4639-47de-bf92-429fb08b8397 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:25.469834 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-24deb325-4639-47de-bf92-429fb08b8397 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1489 microversion: 2.1 time: 0.574514 Sep 27 10:10:25.481310 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1582/3468] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:24 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1489 bytes in 586 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:25.495912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8214cbfc-a85d-40a7-9d14-0bf32f72716a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:25.496539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8214cbfc-a85d-40a7-9d14-0bf32f72716a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:25.502969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8214cbfc-a85d-40a7-9d14-0bf32f72716a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:25.503456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8214cbfc-a85d-40a7-9d14-0bf32f72716a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:25.503887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8214cbfc-a85d-40a7-9d14-0bf32f72716a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:25.504859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8214cbfc-a85d-40a7-9d14-0bf32f72716a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:25.505472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8214cbfc-a85d-40a7-9d14-0bf32f72716a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:25.505895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8214cbfc-a85d-40a7-9d14-0bf32f72716a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:25.826760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8214cbfc-a85d-40a7-9d14-0bf32f72716a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:25.841587 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8214cbfc-a85d-40a7-9d14-0bf32f72716a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.359212 Sep 27 10:10:25.842291 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1583/3469] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:25 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:25.852926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-72ba3361-ce99-4f40-9333-73d75f489af0 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:25.852926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-72ba3361-ce99-4f40-9333-73d75f489af0 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:25.854369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72ba3361-ce99-4f40-9333-73d75f489af0 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:25.855128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72ba3361-ce99-4f40-9333-73d75f489af0 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:25.855792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72ba3361-ce99-4f40-9333-73d75f489af0 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:25.857131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72ba3361-ce99-4f40-9333-73d75f489af0 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:25.857810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72ba3361-ce99-4f40-9333-73d75f489af0 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:25.858464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72ba3361-ce99-4f40-9333-73d75f489af0 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:26.098647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-72ba3361-ce99-4f40-9333-73d75f489af0 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:26.112923 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-72ba3361-ce99-4f40-9333-73d75f489af0 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1368 microversion: 2.1 time: 0.267426 Sep 27 10:10:26.113562 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1584/3470] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:25 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1368 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:26.125335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ed9d27e3-275f-44d5-9f56-2e75f21d8125 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:26.126243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ed9d27e3-275f-44d5-9f56-2e75f21d8125 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:26.143241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed9d27e3-275f-44d5-9f56-2e75f21d8125 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:26.144572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed9d27e3-275f-44d5-9f56-2e75f21d8125 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:26.145340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed9d27e3-275f-44d5-9f56-2e75f21d8125 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:26.146168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed9d27e3-275f-44d5-9f56-2e75f21d8125 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:26.146662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed9d27e3-275f-44d5-9f56-2e75f21d8125 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:26.149338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed9d27e3-275f-44d5-9f56-2e75f21d8125 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:26.454753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ed9d27e3-275f-44d5-9f56-2e75f21d8125 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:26.463876 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ed9d27e3-275f-44d5-9f56-2e75f21d8125 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1314 microversion: 2.1 time: 0.345353 Sep 27 10:10:26.465737 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1585/3471] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:26 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1314 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:26.471452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:26.472183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:26.480183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:26.480896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:26.481672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:26.482556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:26.483161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:26.483736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:26.515068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:10:26.642718 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 204 len: 0 microversion: 2.1 time: 0.175395 Sep 27 10:10:26.643653 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1586/3472] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:10:26 2022] DELETE /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 0 bytes in 177 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:10:26.648879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:26.650075 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-29cc48c8-2cda-4332-a2a9-076a910afb40 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute//" status: 200 len: 377 microversion: - time: 0.002066 Sep 27 10:10:26.659370 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1587/3473] 10.222.0.26 () {56 vars in 886 bytes} [Tue Sep 27 10:10:26 2022] GET /compute/ => generated 377 bytes in 15 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Sep 27 10:10:26.666420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bcca6c7d-ea9a-4984-b89e-aaefe82ed507 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:26.667154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bcca6c7d-ea9a-4984-b89e-aaefe82ed507 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:26.673831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcca6c7d-ea9a-4984-b89e-aaefe82ed507 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:26.674249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcca6c7d-ea9a-4984-b89e-aaefe82ed507 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:26.674704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcca6c7d-ea9a-4984-b89e-aaefe82ed507 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:26.675557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcca6c7d-ea9a-4984-b89e-aaefe82ed507 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:26.676929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcca6c7d-ea9a-4984-b89e-aaefe82ed507 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:26.676929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcca6c7d-ea9a-4984-b89e-aaefe82ed507 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:27.068283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:27.199141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bcca6c7d-ea9a-4984-b89e-aaefe82ed507 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:27.211859 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bcca6c7d-ea9a-4984-b89e-aaefe82ed507 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1394 microversion: 2.1 time: 0.551010 Sep 27 10:10:27.214934 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1588/3474] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:26 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1394 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:27.228507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-284f980b-14c8-4ae2-a1b9-c748002e0dbe req-7293b2b7-267a-4733-936e-ed841bdebf27 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "451a2ed4-e57c-46fc-8e29-09ad474cfa9d", "tag": "8ab5011a-0825-464f-88c1-5874577eff8e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:27.241702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-284f980b-14c8-4ae2-a1b9-c748002e0dbe req-7293b2b7-267a-4733-936e-ed841bdebf27 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:27.242331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-284f980b-14c8-4ae2-a1b9-c748002e0dbe req-7293b2b7-267a-4733-936e-ed841bdebf27 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:27.242845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-284f980b-14c8-4ae2-a1b9-c748002e0dbe req-7293b2b7-267a-4733-936e-ed841bdebf27 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:27.255771 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-284f980b-14c8-4ae2-a1b9-c748002e0dbe req-7293b2b7-267a-4733-936e-ed841bdebf27 service nova] Creating event network-changed:8ab5011a-0825-464f-88c1-5874577eff8e for instance 451a2ed4-e57c-46fc-8e29-09ad474cfa9d on nch1-master-1 Sep 27 10:10:27.262107 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-284f980b-14c8-4ae2-a1b9-c748002e0dbe req-7293b2b7-267a-4733-936e-ed841bdebf27 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043507 Sep 27 10:10:27.263043 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1589/3475] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:27.441241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9575b567-2ff0-4810-aaac-977ae069b102 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:27.443191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9575b567-2ff0-4810-aaac-977ae069b102 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:27.452422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9575b567-2ff0-4810-aaac-977ae069b102 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:27.453234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9575b567-2ff0-4810-aaac-977ae069b102 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:27.454200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9575b567-2ff0-4810-aaac-977ae069b102 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:27.454941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9575b567-2ff0-4810-aaac-977ae069b102 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:27.455413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9575b567-2ff0-4810-aaac-977ae069b102 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:27.455850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9575b567-2ff0-4810-aaac-977ae069b102 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:27.548971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:10:27.550190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:10:27.550878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:10:27.551389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:10:27.551985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:10:27.561175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:27.561662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:27.562099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:27.562829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:27.563289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:27.563824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:27.581738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Getting quotas for project f86d6a96f893464f81868717f0531bd3. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:10:27.587992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Getting quotas for user 04a5f2b1d0134e529f25c227954b5f20 and project f86d6a96f893464f81868717f0531bd3. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:10:27.731857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9575b567-2ff0-4810-aaac-977ae069b102 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:27.742638 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9575b567-2ff0-4810-aaac-977ae069b102 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1495 microversion: 2.1 time: 0.478286 Sep 27 10:10:27.759383 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1590/3476] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:27 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1495 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:27.764177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe835723-a69d-449d-9be8-dda8415d214b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:27.767603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fe835723-a69d-449d-9be8-dda8415d214b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:27.774621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe835723-a69d-449d-9be8-dda8415d214b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:27.775414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe835723-a69d-449d-9be8-dda8415d214b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:27.776038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe835723-a69d-449d-9be8-dda8415d214b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:27.777119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe835723-a69d-449d-9be8-dda8415d214b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:27.782471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:10:27.785650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='boot',updated_at=,uuid=,volume_id='52fbc488-d91e-4ac4-b7b6-28284e31df30',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=1,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='other',updated_at=,uuid=,volume_id='1454250c-f356-4053-83eb-5c69c0a805d7',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=2,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='1caf2529-26ae-48b5-83d4-b87bb80b3024',volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:10:27.803231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe835723-a69d-449d-9be8-dda8415d214b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.026s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:27.803705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe835723-a69d-449d-9be8-dda8415d214b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:27.867434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:28.187326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=18 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:28.363788 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-489761dd-a4d7-4c3f-b750-bcb16e1f6ca7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.42 time: 4.386722 Sep 27 10:10:28.364514 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1887/3477] 10.222.0.26 () {62 vars in 1189 bytes} [Tue Sep 27 10:10:23 2022] POST /compute/v2.1/servers => generated 405 bytes in 4388 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:10:28.370111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3eda8408-b252-4f60-843d-f6467b580258 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:28.371317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3eda8408-b252-4f60-843d-f6467b580258 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:28.379846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3eda8408-b252-4f60-843d-f6467b580258 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:28.380614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3eda8408-b252-4f60-843d-f6467b580258 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:28.381266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3eda8408-b252-4f60-843d-f6467b580258 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:28.382200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3eda8408-b252-4f60-843d-f6467b580258 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:28.383044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3eda8408-b252-4f60-843d-f6467b580258 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:28.383815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3eda8408-b252-4f60-843d-f6467b580258 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:28.497102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fe835723-a69d-449d-9be8-dda8415d214b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:28.515005 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fe835723-a69d-449d-9be8-dda8415d214b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.754417 Sep 27 10:10:28.535336 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1591/3478] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:27 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 775 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:28.550725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9eb7ee43-0bca-4330-9133-117e3ccf39a2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:28.551354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9eb7ee43-0bca-4330-9133-117e3ccf39a2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:28.561644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb7ee43-0bca-4330-9133-117e3ccf39a2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:28.562452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb7ee43-0bca-4330-9133-117e3ccf39a2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:28.562893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb7ee43-0bca-4330-9133-117e3ccf39a2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:28.563613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb7ee43-0bca-4330-9133-117e3ccf39a2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:28.564066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb7ee43-0bca-4330-9133-117e3ccf39a2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:28.564693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb7ee43-0bca-4330-9133-117e3ccf39a2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:28.690532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3eda8408-b252-4f60-843d-f6467b580258 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:28.700646 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3eda8408-b252-4f60-843d-f6467b580258 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1557 microversion: 2.1 time: 0.334811 Sep 27 10:10:28.701265 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1888/3479] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:28 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1557 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:28.715059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82e5ed01-d960-4c3d-9003-bce573eeb678 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:28.715973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-82e5ed01-d960-4c3d-9003-bce573eeb678 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:28.732410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e5ed01-d960-4c3d-9003-bce573eeb678 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:28.733084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e5ed01-d960-4c3d-9003-bce573eeb678 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:28.735754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e5ed01-d960-4c3d-9003-bce573eeb678 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:28.736619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e5ed01-d960-4c3d-9003-bce573eeb678 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:28.744976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e5ed01-d960-4c3d-9003-bce573eeb678 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:28.745639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e5ed01-d960-4c3d-9003-bce573eeb678 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:29.014217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-82e5ed01-d960-4c3d-9003-bce573eeb678 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:29.023382 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-82e5ed01-d960-4c3d-9003-bce573eeb678 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1394 microversion: 2.1 time: 0.317542 Sep 27 10:10:29.024594 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1889/3480] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:28 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1394 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:29.029585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f65f29f7-55d2-4365-801e-6fbe613d715c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:29.030310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f65f29f7-55d2-4365-801e-6fbe613d715c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:29.110866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f65f29f7-55d2-4365-801e-6fbe613d715c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:29.122871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f65f29f7-55d2-4365-801e-6fbe613d715c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:29.124712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f65f29f7-55d2-4365-801e-6fbe613d715c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:29.126162 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f65f29f7-55d2-4365-801e-6fbe613d715c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1286 microversion: 2.42 time: 0.100299 Sep 27 10:10:29.126902 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1890/3481] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:29 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1286 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:29.135909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4732002-0e88-43c9-bdd2-811094641e05 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:29.137244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d4732002-0e88-43c9-bdd2-811094641e05 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:29.157507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4732002-0e88-43c9-bdd2-811094641e05 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:29.158243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4732002-0e88-43c9-bdd2-811094641e05 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:29.158921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4732002-0e88-43c9-bdd2-811094641e05 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:29.159811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4732002-0e88-43c9-bdd2-811094641e05 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:29.162190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4732002-0e88-43c9-bdd2-811094641e05 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:29.162920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4732002-0e88-43c9-bdd2-811094641e05 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:29.351005 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4732002-0e88-43c9-bdd2-811094641e05 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.222219 Sep 27 10:10:29.351609 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1891/3482] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:10:29 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-interface => generated 285 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:29.357407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41b87a59-c234-4ee9-bec4-632df9fe42c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:29.358039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-41b87a59-c234-4ee9-bec4-632df9fe42c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:29.371998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b87a59-c234-4ee9-bec4-632df9fe42c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:29.372514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b87a59-c234-4ee9-bec4-632df9fe42c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:29.375812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b87a59-c234-4ee9-bec4-632df9fe42c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:29.378085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b87a59-c234-4ee9-bec4-632df9fe42c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:29.378610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b87a59-c234-4ee9-bec4-632df9fe42c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:29.379060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41b87a59-c234-4ee9-bec4-632df9fe42c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:29.675375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9eb7ee43-0bca-4330-9133-117e3ccf39a2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:29.689864 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9eb7ee43-0bca-4330-9133-117e3ccf39a2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1314 microversion: 2.1 time: 1.151819 Sep 27 10:10:29.706610 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1592/3483] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:28 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1314 bytes in 1170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:29.722089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-92c06171-2e57-425f-956e-1b08af6284c3 req-3cb027f1-1a74-4f84-ae80-a8ea59a4eb96 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3651e4da-f0ce-4e37-82ac-9b81a1564a1d", "name": "network-vif-plugged", "status": "completed", "tag": "87d895c7-9525-40c2-baef-ff15ac876653"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:29.752131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-92c06171-2e57-425f-956e-1b08af6284c3 req-3cb027f1-1a74-4f84-ae80-a8ea59a4eb96 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:29.752743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-92c06171-2e57-425f-956e-1b08af6284c3 req-3cb027f1-1a74-4f84-ae80-a8ea59a4eb96 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:29.753546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-92c06171-2e57-425f-956e-1b08af6284c3 req-3cb027f1-1a74-4f84-ae80-a8ea59a4eb96 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:29.785251 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-92c06171-2e57-425f-956e-1b08af6284c3 req-3cb027f1-1a74-4f84-ae80-a8ea59a4eb96 service nova] Creating event network-vif-plugged:87d895c7-9525-40c2-baef-ff15ac876653 for instance 3651e4da-f0ce-4e37-82ac-9b81a1564a1d on nch1-master-1 Sep 27 10:10:29.798615 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-92c06171-2e57-425f-956e-1b08af6284c3 req-3cb027f1-1a74-4f84-ae80-a8ea59a4eb96 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.086453 Sep 27 10:10:29.800455 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1593/3484] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:29.817063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-18f58147-ffed-43f3-89ba-eb22ea4b4df7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:29.818403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-18f58147-ffed-43f3-89ba-eb22ea4b4df7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:29.839302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18f58147-ffed-43f3-89ba-eb22ea4b4df7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:29.842660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18f58147-ffed-43f3-89ba-eb22ea4b4df7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:29.844874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18f58147-ffed-43f3-89ba-eb22ea4b4df7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:29.846897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18f58147-ffed-43f3-89ba-eb22ea4b4df7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:29.849311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18f58147-ffed-43f3-89ba-eb22ea4b4df7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:29.850830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18f58147-ffed-43f3-89ba-eb22ea4b4df7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:29.899035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-41b87a59-c234-4ee9-bec4-632df9fe42c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:29.912621 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-41b87a59-c234-4ee9-bec4-632df9fe42c0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1495 microversion: 2.1 time: 0.559908 Sep 27 10:10:29.913302 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1892/3485] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:29 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1495 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:30.048866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-80b351f7-3a9f-49fa-8fee-efc5c015caf0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:30.049484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-80b351f7-3a9f-49fa-8fee-efc5c015caf0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:30.060926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b351f7-3a9f-49fa-8fee-efc5c015caf0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:30.064096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b351f7-3a9f-49fa-8fee-efc5c015caf0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:30.064560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b351f7-3a9f-49fa-8fee-efc5c015caf0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:30.065592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b351f7-3a9f-49fa-8fee-efc5c015caf0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:30.066074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b351f7-3a9f-49fa-8fee-efc5c015caf0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:30.066510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80b351f7-3a9f-49fa-8fee-efc5c015caf0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:30.393261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-18f58147-ffed-43f3-89ba-eb22ea4b4df7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:30.406822 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-18f58147-ffed-43f3-89ba-eb22ea4b4df7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.604356 Sep 27 10:10:30.422040 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1594/3486] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:29 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:30.422040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:30.422040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:30.431284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:30.431976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:30.432584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:30.433578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:30.434204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:30.434796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:30.485666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-80b351f7-3a9f-49fa-8fee-efc5c015caf0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:30.498919 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-80b351f7-3a9f-49fa-8fee-efc5c015caf0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1394 microversion: 2.1 time: 0.453882 Sep 27 10:10:30.499608 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1893/3487] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:30 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1394 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:30.509284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f9982428-24f8-4c32-a4e1-c6f98f843cae req-f5ccddd5-a889-4c0d-a276-67a7efc1f163 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e581dad6-2c27-4baa-9f55-c082f78b67bd", "name": "network-vif-deleted", "tag": "e201ea6d-d900-45b5-8802-968cdc0d7ca9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:30.537213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f9982428-24f8-4c32-a4e1-c6f98f843cae req-f5ccddd5-a889-4c0d-a276-67a7efc1f163 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:30.537703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f9982428-24f8-4c32-a4e1-c6f98f843cae req-f5ccddd5-a889-4c0d-a276-67a7efc1f163 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:30.538213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f9982428-24f8-4c32-a4e1-c6f98f843cae req-f5ccddd5-a889-4c0d-a276-67a7efc1f163 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:30.552804 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f9982428-24f8-4c32-a4e1-c6f98f843cae req-f5ccddd5-a889-4c0d-a276-67a7efc1f163 service nova] Creating event network-vif-deleted:e201ea6d-d900-45b5-8802-968cdc0d7ca9 for instance e581dad6-2c27-4baa-9f55-c082f78b67bd on nch1-master-1 Sep 27 10:10:30.557856 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f9982428-24f8-4c32-a4e1-c6f98f843cae req-f5ccddd5-a889-4c0d-a276-67a7efc1f163 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057088 Sep 27 10:10:30.558796 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1894/3488] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:30.586120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:30.601418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:30.601418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:30.601418 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7798e37d-eb34-4a40-add2-c9386c948200 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1576 microversion: 2.42 time: 0.183267 Sep 27 10:10:30.601418 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1595/3489] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:30 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1576 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:30.719892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0ec3b87-7ee9-41ae-99c6-451f168dd013 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:30.721432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f0ec3b87-7ee9-41ae-99c6-451f168dd013 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:30.729610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ec3b87-7ee9-41ae-99c6-451f168dd013 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:30.730456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ec3b87-7ee9-41ae-99c6-451f168dd013 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:30.731124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ec3b87-7ee9-41ae-99c6-451f168dd013 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:30.732563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ec3b87-7ee9-41ae-99c6-451f168dd013 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:30.733978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ec3b87-7ee9-41ae-99c6-451f168dd013 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:30.734644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ec3b87-7ee9-41ae-99c6-451f168dd013 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:30.936449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:30.937831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:30.951311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:30.954453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:30.955354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:30.963628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:30.965060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:30.969546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:31.063491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:31.067066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:31.068694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:31.068694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lazy-loading 'fault' on Instance uuid e581dad6-2c27-4baa-9f55-c082f78b67bd {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:10:31.274685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:31.293651 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e2510a63-be31-416b-a052-751637d4bba1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 200 len: 1305 microversion: 2.1 time: 0.361338 Sep 27 10:10:31.333726 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1596/3490] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:30 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 1305 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:31.344467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f0ec3b87-7ee9-41ae-99c6-451f168dd013 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:31.358766 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f0ec3b87-7ee9-41ae-99c6-451f168dd013 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1314 microversion: 2.1 time: 0.639007 Sep 27 10:10:31.378491 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1895/3491] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:30 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1314 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:31.437139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-92c55f12-4d90-4809-9f3a-1469c760000c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:31.437825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-92c55f12-4d90-4809-9f3a-1469c760000c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:31.448379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c55f12-4d90-4809-9f3a-1469c760000c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:31.448379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c55f12-4d90-4809-9f3a-1469c760000c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:31.448744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c55f12-4d90-4809-9f3a-1469c760000c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:31.449462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c55f12-4d90-4809-9f3a-1469c760000c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:31.449924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c55f12-4d90-4809-9f3a-1469c760000c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:31.450355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c55f12-4d90-4809-9f3a-1469c760000c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:31.522004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0d83ebf9-765a-4d2c-9088-8e42bc1441a6 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:31.523732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0d83ebf9-765a-4d2c-9088-8e42bc1441a6 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:31.533674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d83ebf9-765a-4d2c-9088-8e42bc1441a6 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:31.534276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d83ebf9-765a-4d2c-9088-8e42bc1441a6 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:31.534784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d83ebf9-765a-4d2c-9088-8e42bc1441a6 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:31.535599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d83ebf9-765a-4d2c-9088-8e42bc1441a6 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:31.536123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d83ebf9-765a-4d2c-9088-8e42bc1441a6 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:31.536671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d83ebf9-765a-4d2c-9088-8e42bc1441a6 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:31.833585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0d83ebf9-765a-4d2c-9088-8e42bc1441a6 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:31.874523 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0d83ebf9-765a-4d2c-9088-8e42bc1441a6 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1394 microversion: 2.1 time: 0.352663 Sep 27 10:10:31.876153 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1896/3492] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:31 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1394 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:31.888613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:31.889590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:31.895382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-92c55f12-4d90-4809-9f3a-1469c760000c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:31.903945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:31.906082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:31.908256 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-92c55f12-4d90-4809-9f3a-1469c760000c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.474709 Sep 27 10:10:31.908480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:31.911385 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1597/3493] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:31 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:31.915504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:31.916238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:31.916902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:31.919297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-0538afae-221e-4921-b186-0237a0896fe1 req-9eca250d-5c09-4e83-a3c9-8c666da9be66 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a04c146e-494e-458e-843e-09c63c576cb3", "tag": "f45a2d54-220d-47e8-93bd-d90e6fda8e24"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:31.946794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0538afae-221e-4921-b186-0237a0896fe1 req-9eca250d-5c09-4e83-a3c9-8c666da9be66 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:31.948084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0538afae-221e-4921-b186-0237a0896fe1 req-9eca250d-5c09-4e83-a3c9-8c666da9be66 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:31.948788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0538afae-221e-4921-b186-0237a0896fe1 req-9eca250d-5c09-4e83-a3c9-8c666da9be66 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:31.975000 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-0538afae-221e-4921-b186-0237a0896fe1 req-9eca250d-5c09-4e83-a3c9-8c666da9be66 service nova] Creating event network-changed:f45a2d54-220d-47e8-93bd-d90e6fda8e24 for instance a04c146e-494e-458e-843e-09c63c576cb3 on nch2-master-1 Sep 27 10:10:31.984337 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-0538afae-221e-4921-b186-0237a0896fe1 req-9eca250d-5c09-4e83-a3c9-8c666da9be66 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071348 Sep 27 10:10:31.990419 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1598/3494] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:31.996499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-9895d978-5e32-4dd4-85b7-041dfb2db819 req-853c24f7-4307-41c1-bc07-26882bda7e3d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3327ff95-5c05-4167-a73b-5fd55f7bfabd", "name": "network-vif-plugged", "status": "completed", "tag": "d571b78e-f1b6-4353-b09e-cac4cd18cde7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:32.023563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9895d978-5e32-4dd4-85b7-041dfb2db819 req-853c24f7-4307-41c1-bc07-26882bda7e3d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:32.024204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9895d978-5e32-4dd4-85b7-041dfb2db819 req-853c24f7-4307-41c1-bc07-26882bda7e3d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:32.028788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9895d978-5e32-4dd4-85b7-041dfb2db819 req-853c24f7-4307-41c1-bc07-26882bda7e3d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:32.056638 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-9895d978-5e32-4dd4-85b7-041dfb2db819 req-853c24f7-4307-41c1-bc07-26882bda7e3d service nova] Creating event network-vif-plugged:d571b78e-f1b6-4353-b09e-cac4cd18cde7 for instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd on nch2-master-1 Sep 27 10:10:32.070355 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-9895d978-5e32-4dd4-85b7-041dfb2db819 req-853c24f7-4307-41c1-bc07-26882bda7e3d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.078094 Sep 27 10:10:32.073700 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1599/3495] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:32.082484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:32.112450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:32.126207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:32.130869 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fb8f5abd-d942-40b1-83cc-50bec45551be tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1594 microversion: 2.42 time: 0.253265 Sep 27 10:10:32.133760 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1897/3496] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:31 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1594 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:32.326458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:32.332587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:32.340283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:32.356583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:32.357652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:32.358669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:32.359360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:32.361219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:32.395142 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:10:32.397575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-28abcff2-5b6d-45b4-b188-3b00f90b3fbe tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:32.398203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-28abcff2-5b6d-45b4-b188-3b00f90b3fbe tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd HTTP/1.1 Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'e581dad6-2c27-4baa-9f55-c082f78b67bd'} Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:32.398842 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:32.403262 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:32.403262 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:32.403262 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:32.403262 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:32.403262 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:32.403262 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:10:32.403262 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:32.405327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:32.406450 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f5a21db7-f790-46c6-8a18-0b9243cc9392 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 404 len: 111 microversion: 2.1 time: 0.084703 Sep 27 10:10:32.411259 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1600/3497] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:32 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 111 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:10:32.421126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28abcff2-5b6d-45b4-b188-3b00f90b3fbe tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:32.432142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28abcff2-5b6d-45b4-b188-3b00f90b3fbe tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:32.436130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28abcff2-5b6d-45b4-b188-3b00f90b3fbe tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:32.436938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28abcff2-5b6d-45b4-b188-3b00f90b3fbe tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:32.442114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28abcff2-5b6d-45b4-b188-3b00f90b3fbe tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:32.442599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-28abcff2-5b6d-45b4-b188-3b00f90b3fbe tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:32.498995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1072905428", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "0117c46b-39f2-4cf5-9f8a-7710125bdf55"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:32.587346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] validate_networks() for [('0117c46b-39f2-4cf5-9f8a-7710125bdf55', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:10:32.654605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-28abcff2-5b6d-45b4-b188-3b00f90b3fbe tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:32.666684 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-28abcff2-5b6d-45b4-b188-3b00f90b3fbe tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1499 microversion: 2.1 time: 0.284698 Sep 27 10:10:32.667907 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1898/3498] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:32 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1499 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:32.698926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6c5d6173-384c-47d7-82b9-d5d4d800427f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:32.712461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6c5d6173-384c-47d7-82b9-d5d4d800427f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:32.747199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c5d6173-384c-47d7-82b9-d5d4d800427f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:32.748092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c5d6173-384c-47d7-82b9-d5d4d800427f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:32.748896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c5d6173-384c-47d7-82b9-d5d4d800427f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:32.752622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c5d6173-384c-47d7-82b9-d5d4d800427f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:32.753315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c5d6173-384c-47d7-82b9-d5d4d800427f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:32.753903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c5d6173-384c-47d7-82b9-d5d4d800427f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:32.817324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-6c5d6173-384c-47d7-82b9-d5d4d800427f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lazy-loading 'flavor' on Instance uuid 3651e4da-f0ce-4e37-82ac-9b81a1564a1d {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:10:32.844813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:32.956105 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6c5d6173-384c-47d7-82b9-d5d4d800427f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "POST /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d/action" status: 200 len: 32 microversion: 2.1 time: 0.263662 Sep 27 10:10:32.958488 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1899/3499] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:10:32 2022] POST /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d/action => generated 32 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:10:32.965707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-97f80a2b-f068-48e0-9ab7-7e52b5f824c0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:32.966843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-97f80a2b-f068-48e0-9ab7-7e52b5f824c0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:32.977318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97f80a2b-f068-48e0-9ab7-7e52b5f824c0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:32.978505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97f80a2b-f068-48e0-9ab7-7e52b5f824c0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:32.978821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97f80a2b-f068-48e0-9ab7-7e52b5f824c0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:32.980588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97f80a2b-f068-48e0-9ab7-7e52b5f824c0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:32.981005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97f80a2b-f068-48e0-9ab7-7e52b5f824c0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:32.982193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97f80a2b-f068-48e0-9ab7-7e52b5f824c0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:33.158851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-97f80a2b-f068-48e0-9ab7-7e52b5f824c0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:33.181457 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-97f80a2b-f068-48e0-9ab7-7e52b5f824c0 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1394 microversion: 2.1 time: 0.221072 Sep 27 10:10:33.182168 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1900/3500] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:32 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1394 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:33.186647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4a5b4d92-ca3c-4c22-b73c-70d3763db8d2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:33.187262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4a5b4d92-ca3c-4c22-b73c-70d3763db8d2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:33.202932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a5b4d92-ca3c-4c22-b73c-70d3763db8d2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:33.205045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a5b4d92-ca3c-4c22-b73c-70d3763db8d2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:33.205509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a5b4d92-ca3c-4c22-b73c-70d3763db8d2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:33.209387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a5b4d92-ca3c-4c22-b73c-70d3763db8d2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:33.209873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a5b4d92-ca3c-4c22-b73c-70d3763db8d2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:33.210262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a5b4d92-ca3c-4c22-b73c-70d3763db8d2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:33.406283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4a5b4d92-ca3c-4c22-b73c-70d3763db8d2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:33.426634 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4a5b4d92-ca3c-4c22-b73c-70d3763db8d2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.242484 Sep 27 10:10:33.430681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1901/3501] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:33 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:33.436314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9752c717-ba4f-425c-b024-a49626434281 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:33.436958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9752c717-ba4f-425c-b024-a49626434281 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:33.459904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9752c717-ba4f-425c-b024-a49626434281 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:33.464705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9752c717-ba4f-425c-b024-a49626434281 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:33.465195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9752c717-ba4f-425c-b024-a49626434281 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:33.466098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9752c717-ba4f-425c-b024-a49626434281 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:33.466659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9752c717-ba4f-425c-b024-a49626434281 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:33.467097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9752c717-ba4f-425c-b024-a49626434281 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:33.650192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9752c717-ba4f-425c-b024-a49626434281 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:33.659522 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9752c717-ba4f-425c-b024-a49626434281 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.227279 Sep 27 10:10:33.660125 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1902/3502] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:33 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:33.665396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:33.666969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:33.680323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:33.680978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:33.681566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:33.682636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:33.683291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:33.683747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:33.752466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:33.762528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:33.764138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:33.765416 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d4dec8e-c5ac-4883-8484-f21c3cfa4d79 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1594 microversion: 2.42 time: 0.104133 Sep 27 10:10:33.766077 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1903/3503] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:33 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1594 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:34.208936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-97edc33f-260d-4c6c-8d49-18baaacc81ed tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:34.210377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-97edc33f-260d-4c6c-8d49-18baaacc81ed tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:34.227818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97edc33f-260d-4c6c-8d49-18baaacc81ed tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:34.229072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97edc33f-260d-4c6c-8d49-18baaacc81ed tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:34.230175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97edc33f-260d-4c6c-8d49-18baaacc81ed tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:34.231218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97edc33f-260d-4c6c-8d49-18baaacc81ed tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:34.231832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97edc33f-260d-4c6c-8d49-18baaacc81ed tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:34.232412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97edc33f-260d-4c6c-8d49-18baaacc81ed tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:34.955169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-97edc33f-260d-4c6c-8d49-18baaacc81ed tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:34.969525 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-97edc33f-260d-4c6c-8d49-18baaacc81ed tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1394 microversion: 2.1 time: 0.761292 Sep 27 10:10:34.973565 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1904/3504] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:34 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1394 bytes in 769 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:34.983602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-31accd18-4944-4b58-8976-4af005ea9eae tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:34.986807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-31accd18-4944-4b58-8976-4af005ea9eae tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:35.002910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31accd18-4944-4b58-8976-4af005ea9eae tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:35.003422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31accd18-4944-4b58-8976-4af005ea9eae tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:35.005965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31accd18-4944-4b58-8976-4af005ea9eae tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:35.008718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31accd18-4944-4b58-8976-4af005ea9eae tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:35.009493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31accd18-4944-4b58-8976-4af005ea9eae tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:35.011580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-31accd18-4944-4b58-8976-4af005ea9eae tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:35.175744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:10:35.176900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:10:35.177720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:10:35.178300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:10:35.179291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:10:35.194872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:35.197302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:35.199837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:35.200910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:35.204267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:35.204952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:35.256316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:10:35.266148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Getting quotas for user 7f7c5adbac4f48b6bc141e0f888f2496 and project 6c681ff8fe354f97abb5fe0d5181e6f1. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:10:35.327212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:10:35.333909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:10:35.391565 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-853f4bc9-8f9c-47b8-bd46-ac812816c115 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.896330 Sep 27 10:10:35.399822 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1601/3505] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:10:32 2022] POST /compute/v2.1/servers => generated 380 bytes in 2905 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:10:35.407358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80ca919f-fb6c-44e4-baaa-cd93dd2fcd7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:35.409895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-80ca919f-fb6c-44e4-baaa-cd93dd2fcd7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:35.423053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80ca919f-fb6c-44e4-baaa-cd93dd2fcd7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:35.423959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80ca919f-fb6c-44e4-baaa-cd93dd2fcd7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:35.424749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80ca919f-fb6c-44e4-baaa-cd93dd2fcd7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:35.427066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80ca919f-fb6c-44e4-baaa-cd93dd2fcd7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:35.427878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80ca919f-fb6c-44e4-baaa-cd93dd2fcd7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:35.428644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80ca919f-fb6c-44e4-baaa-cd93dd2fcd7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:35.495323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-31accd18-4944-4b58-8976-4af005ea9eae tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:35.505699 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-31accd18-4944-4b58-8976-4af005ea9eae tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1518 microversion: 2.1 time: 0.530715 Sep 27 10:10:35.512036 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1905/3506] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:34 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1518 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:35.521147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:35.521825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:35.533673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:35.534381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:35.534984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:35.535900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:35.536796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:35.537783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:35.936383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-80ca919f-fb6c-44e4-baaa-cd93dd2fcd7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:35.961449 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-80ca919f-fb6c-44e4-baaa-cd93dd2fcd7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.558850 Sep 27 10:10:35.963152 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1602/3507] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:35 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:35.985622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-29bc6244-50f2-48c7-9160-9e2bf2fc34f6 req-9147337a-7f9f-4ff0-9213-bf6af8b9b1f6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "451a2ed4-e57c-46fc-8e29-09ad474cfa9d", "name": "network-vif-plugged", "status": "completed", "tag": "8ab5011a-0825-464f-88c1-5874577eff8e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:36.002241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-29bc6244-50f2-48c7-9160-9e2bf2fc34f6 req-9147337a-7f9f-4ff0-9213-bf6af8b9b1f6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:36.002796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-29bc6244-50f2-48c7-9160-9e2bf2fc34f6 req-9147337a-7f9f-4ff0-9213-bf6af8b9b1f6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:36.003574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-29bc6244-50f2-48c7-9160-9e2bf2fc34f6 req-9147337a-7f9f-4ff0-9213-bf6af8b9b1f6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:36.018599 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-29bc6244-50f2-48c7-9160-9e2bf2fc34f6 req-9147337a-7f9f-4ff0-9213-bf6af8b9b1f6 service nova] Creating event network-vif-plugged:8ab5011a-0825-464f-88c1-5874577eff8e for instance 451a2ed4-e57c-46fc-8e29-09ad474cfa9d on nch1-master-1 Sep 27 10:10:36.025160 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-29bc6244-50f2-48c7-9160-9e2bf2fc34f6 req-9147337a-7f9f-4ff0-9213-bf6af8b9b1f6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056978 Sep 27 10:10:36.025876 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1603/3508] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:36.033971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6ee0ef75-6155-4a55-b3b3-5a9b10898473 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:36.035196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6ee0ef75-6155-4a55-b3b3-5a9b10898473 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:36.084830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:36.128547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:36.144611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:36.148762 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6694f2ab-7ce2-4c4c-b55b-6d064dd88e70 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1661 microversion: 2.42 time: 0.634050 Sep 27 10:10:36.150024 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1906/3509] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:35 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1661 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:36.177821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c62c3dc1-3fc4-4945-b1ed-53ae67c8a733 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:36.180199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c62c3dc1-3fc4-4945-b1ed-53ae67c8a733 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:36.190907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c62c3dc1-3fc4-4945-b1ed-53ae67c8a733 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:36.191532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c62c3dc1-3fc4-4945-b1ed-53ae67c8a733 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:36.192134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c62c3dc1-3fc4-4945-b1ed-53ae67c8a733 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:36.193707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c62c3dc1-3fc4-4945-b1ed-53ae67c8a733 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:36.194552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c62c3dc1-3fc4-4945-b1ed-53ae67c8a733 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:36.195292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c62c3dc1-3fc4-4945-b1ed-53ae67c8a733 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:36.242321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6ee0ef75-6155-4a55-b3b3-5a9b10898473 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:36.251691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6ee0ef75-6155-4a55-b3b3-5a9b10898473 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1215 microversion: 2.1 time: 0.224513 Sep 27 10:10:36.252293 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1604/3510] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:36 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1215 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:36.541287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0956b1d-5a7b-4924-bad8-1b8f25a0d27f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:36.541925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b0956b1d-5a7b-4924-bad8-1b8f25a0d27f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:36.552682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0956b1d-5a7b-4924-bad8-1b8f25a0d27f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:36.553434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0956b1d-5a7b-4924-bad8-1b8f25a0d27f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:36.554932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0956b1d-5a7b-4924-bad8-1b8f25a0d27f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:36.556008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0956b1d-5a7b-4924-bad8-1b8f25a0d27f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:36.556872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0956b1d-5a7b-4924-bad8-1b8f25a0d27f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:36.557936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0956b1d-5a7b-4924-bad8-1b8f25a0d27f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:36.560817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c62c3dc1-3fc4-4945-b1ed-53ae67c8a733 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:36.572228 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c62c3dc1-3fc4-4945-b1ed-53ae67c8a733 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1394 microversion: 2.1 time: 0.419539 Sep 27 10:10:36.572907 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1907/3511] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:36 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1394 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:36.855103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b0956b1d-5a7b-4924-bad8-1b8f25a0d27f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:36.863860 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0956b1d-5a7b-4924-bad8-1b8f25a0d27f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.331041 Sep 27 10:10:36.868468 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1605/3512] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:36 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:36.893906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1f8f28bd-7e49-40bf-9e9a-fc636471978b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:36.894630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1f8f28bd-7e49-40bf-9e9a-fc636471978b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:36.913869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f8f28bd-7e49-40bf-9e9a-fc636471978b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:36.916775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f8f28bd-7e49-40bf-9e9a-fc636471978b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:36.929306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f8f28bd-7e49-40bf-9e9a-fc636471978b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:36.932064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f8f28bd-7e49-40bf-9e9a-fc636471978b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:36.934239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f8f28bd-7e49-40bf-9e9a-fc636471978b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:36.936590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f8f28bd-7e49-40bf-9e9a-fc636471978b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:36.985956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1341ca2-c307-4d21-8d63-2b3499810714 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:36.986665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a1341ca2-c307-4d21-8d63-2b3499810714 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:36.995587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1341ca2-c307-4d21-8d63-2b3499810714 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.001277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1341ca2-c307-4d21-8d63-2b3499810714 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.001811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1341ca2-c307-4d21-8d63-2b3499810714 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:37.002626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1341ca2-c307-4d21-8d63-2b3499810714 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.003309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1341ca2-c307-4d21-8d63-2b3499810714 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.004112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1341ca2-c307-4d21-8d63-2b3499810714 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:37.140693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1f8f28bd-7e49-40bf-9e9a-fc636471978b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:37.151782 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1f8f28bd-7e49-40bf-9e9a-fc636471978b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.262988 Sep 27 10:10:37.152890 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1908/3513] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:36 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:37.180386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:37.181311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:37.191684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.194538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.195675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:37.196512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.196916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.197542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:37.311101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a1341ca2-c307-4d21-8d63-2b3499810714 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:37.325872 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a1341ca2-c307-4d21-8d63-2b3499810714 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.343045 Sep 27 10:10:37.329463 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1606/3514] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:36 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:37.339589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-002d9a1e-ab50-4934-a209-3f40207839c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:37.343304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-002d9a1e-ab50-4934-a209-3f40207839c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:37.354675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-002d9a1e-ab50-4934-a209-3f40207839c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.355728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-002d9a1e-ab50-4934-a209-3f40207839c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.366863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-002d9a1e-ab50-4934-a209-3f40207839c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:37.372931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-002d9a1e-ab50-4934-a209-3f40207839c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.374474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-002d9a1e-ab50-4934-a209-3f40207839c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.374875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-002d9a1e-ab50-4934-a209-3f40207839c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:37.488132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:37.500178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:37.501656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:37.503308 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-03ab175b-b7a6-46b7-bc65-5f2fdbd28a24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1707 microversion: 2.42 time: 0.327159 Sep 27 10:10:37.506211 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1909/3515] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:37 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1707 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:37.511796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33ffd8d3-cc27-4634-bcdf-9a93f46a0045 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:37.512619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33ffd8d3-cc27-4634-bcdf-9a93f46a0045 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:37.519433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ffd8d3-cc27-4634-bcdf-9a93f46a0045 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.520210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ffd8d3-cc27-4634-bcdf-9a93f46a0045 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.520844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ffd8d3-cc27-4634-bcdf-9a93f46a0045 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:37.521717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ffd8d3-cc27-4634-bcdf-9a93f46a0045 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.522373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ffd8d3-cc27-4634-bcdf-9a93f46a0045 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.523066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ffd8d3-cc27-4634-bcdf-9a93f46a0045 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:37.604348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:37.646546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-002d9a1e-ab50-4934-a209-3f40207839c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:37.660548 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-002d9a1e-ab50-4934-a209-3f40207839c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.329278 Sep 27 10:10:37.661182 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1607/3516] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:37 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:37.666897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-222b442a-13ab-44b4-8957-b989a689d77a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:37.670659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-222b442a-13ab-44b4-8957-b989a689d77a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:37.686440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-222b442a-13ab-44b4-8957-b989a689d77a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.689170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-222b442a-13ab-44b4-8957-b989a689d77a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.689859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-222b442a-13ab-44b4-8957-b989a689d77a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:37.690694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-222b442a-13ab-44b4-8957-b989a689d77a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.694630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-222b442a-13ab-44b4-8957-b989a689d77a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.695799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-222b442a-13ab-44b4-8957-b989a689d77a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:37.705867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33ffd8d3-cc27-4634-bcdf-9a93f46a0045 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:37.719021 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33ffd8d3-cc27-4634-bcdf-9a93f46a0045 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1267 microversion: 2.1 time: 0.211436 Sep 27 10:10:37.720376 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1910/3517] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:37 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1267 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:37.725513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af93ddc9-c5d5-45e2-b77f-2106646b6f98 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:37.732054 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-af93ddc9-c5d5-45e2-b77f-2106646b6f98 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance a81a7005-8154-4d08-830d-912704e244ca has no mapping to a cell. Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af93ddc9-c5d5-45e2-b77f-2106646b6f98 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/a81a7005-8154-4d08-830d-912704e244ca/action HTTP/1.1 Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 15 Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: {"os-stop": {}} Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance a81a7005-8154-4d08-830d-912704e244ca has no mapping to a cell. Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a81a7005-8154-4d08-830d-912704e244ca', 'body': {'os-stop': {}}} Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1411, in _get_instance Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi mapping = objects.InstanceMapping.get_by_instance_uuid( Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_mapping.py", line 112, in get_by_instance_uuid Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_mapping = cls._get_by_instance_uuid_from_db(context, instance_uuid) Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_mapping.py", line 106, in _get_by_instance_uuid_from_db Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceMappingNotFound(uuid=instance_uuid) Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceMappingNotFound: Instance a81a7005-8154-4d08-830d-912704e244ca has no mapping to a cell. Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1444, in _stop_server Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, id) Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1418, in _get_instance Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:10:37.742839 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a81a7005-8154-4d08-830d-912704e244ca has no mapping to a cell. Sep 27 10:10:37.748195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:37.748195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:37.748195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:37.748195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:37.748195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:37.748195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:37.748195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:37.748195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:37.748195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a81a7005-8154-4d08-830d-912704e244ca has no mapping to a cell. Sep 27 10:10:37.748195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:37.749743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af93ddc9-c5d5-45e2-b77f-2106646b6f98 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance a81a7005-8154-4d08-830d-912704e244ca has no mapping to a cell. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:37.754812 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af93ddc9-c5d5-45e2-b77f-2106646b6f98 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/a81a7005-8154-4d08-830d-912704e244ca/action" status: 404 len: 117 microversion: 2.1 time: 0.029389 Sep 27 10:10:37.760341 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1911/3518] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:10:37 2022] POST /compute/v2.1/servers/a81a7005-8154-4d08-830d-912704e244ca/action => generated 117 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:10:37.772888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4081c0e0-2ded-4df3-90a9-2b23ab443571 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:37.776344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4081c0e0-2ded-4df3-90a9-2b23ab443571 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:37.794310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4081c0e0-2ded-4df3-90a9-2b23ab443571 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.797999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4081c0e0-2ded-4df3-90a9-2b23ab443571 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.799486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4081c0e0-2ded-4df3-90a9-2b23ab443571 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:37.801386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4081c0e0-2ded-4df3-90a9-2b23ab443571 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:37.804525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4081c0e0-2ded-4df3-90a9-2b23ab443571 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:37.806869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4081c0e0-2ded-4df3-90a9-2b23ab443571 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.011141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4081c0e0-2ded-4df3-90a9-2b23ab443571 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:38.032191 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4081c0e0-2ded-4df3-90a9-2b23ab443571 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.260844 Sep 27 10:10:38.032191 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1912/3519] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:37 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:38.051625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a51591fc-64c1-4ff1-a3a3-94418bcd2ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:38.052250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a51591fc-64c1-4ff1-a3a3-94418bcd2ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:38.059521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a51591fc-64c1-4ff1-a3a3-94418bcd2ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.060015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a51591fc-64c1-4ff1-a3a3-94418bcd2ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.061345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a51591fc-64c1-4ff1-a3a3-94418bcd2ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.062076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a51591fc-64c1-4ff1-a3a3-94418bcd2ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.062566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a51591fc-64c1-4ff1-a3a3-94418bcd2ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.062983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a51591fc-64c1-4ff1-a3a3-94418bcd2ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.204860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-222b442a-13ab-44b4-8957-b989a689d77a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:38.212345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a51591fc-64c1-4ff1-a3a3-94418bcd2ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:38.224393 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a51591fc-64c1-4ff1-a3a3-94418bcd2ed1 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.176935 Sep 27 10:10:38.225009 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1913/3520] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:38 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:38.227892 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-222b442a-13ab-44b4-8957-b989a689d77a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1394 microversion: 2.1 time: 0.565236 Sep 27 10:10:38.241487 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1608/3521] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:37 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1394 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:38.250123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cc630b77-8a64-4611-8bb8-db39286e9a87 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:38.250676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cc630b77-8a64-4611-8bb8-db39286e9a87 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 5fca9197-2f37-4080-b1f4-59ce544b14a8] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:38.259110 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-cc630b77-8a64-4611-8bb8-db39286e9a87 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 5fca9197-2f37-4080-b1f4-59ce544b14a8 could not be found. Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cc630b77-8a64-4611-8bb8-db39286e9a87 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/5fca9197-2f37-4080-b1f4-59ce544b14a8/action HTTP/1.1 Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 15 Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: {"suspend": {}} Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 5fca9197-2f37-4080-b1f4-59ce544b14a8 could not be found. Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '5fca9197-2f37-4080-b1f4-59ce544b14a8', 'body': {'suspend': {}}} Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5fca9197-2f37-4080-b1f4-59ce544b14a8 could not be found. Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 35, in _suspend Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, id) Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5fca9197-2f37-4080-b1f4-59ce544b14a8 could not be found. Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:38.260843 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:38.265827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:38.265827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:38.265827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5fca9197-2f37-4080-b1f4-59ce544b14a8 could not be found. Sep 27 10:10:38.265827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:38.267349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cc630b77-8a64-4611-8bb8-db39286e9a87 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 5fca9197-2f37-4080-b1f4-59ce544b14a8 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:38.268356 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cc630b77-8a64-4611-8bb8-db39286e9a87 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/5fca9197-2f37-4080-b1f4-59ce544b14a8/action" status: 404 len: 111 microversion: 2.1 time: 0.022600 Sep 27 10:10:38.269613 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1914/3522] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:10:38 2022] POST /compute/v2.1/servers/5fca9197-2f37-4080-b1f4-59ce544b14a8/action => generated 111 bytes in 24 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:10:38.277628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a7fb4a0-6093-4676-9de2-a5eda5c8ef43 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:38.278528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a7fb4a0-6093-4676-9de2-a5eda5c8ef43 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:38.282047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:38.285892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a7fb4a0-6093-4676-9de2-a5eda5c8ef43 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.286623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a7fb4a0-6093-4676-9de2-a5eda5c8ef43 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.287226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a7fb4a0-6093-4676-9de2-a5eda5c8ef43 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.287941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a7fb4a0-6093-4676-9de2-a5eda5c8ef43 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.288625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a7fb4a0-6093-4676-9de2-a5eda5c8ef43 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.289137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a7fb4a0-6093-4676-9de2-a5eda5c8ef43 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.358592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-849894dd-a1da-4a6c-8231-cbd35dc0e8e2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:38.359710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-849894dd-a1da-4a6c-8231-cbd35dc0e8e2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:38.369222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-849894dd-a1da-4a6c-8231-cbd35dc0e8e2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.369938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-849894dd-a1da-4a6c-8231-cbd35dc0e8e2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.370604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-849894dd-a1da-4a6c-8231-cbd35dc0e8e2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.371378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-849894dd-a1da-4a6c-8231-cbd35dc0e8e2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.371846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-849894dd-a1da-4a6c-8231-cbd35dc0e8e2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.372578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-849894dd-a1da-4a6c-8231-cbd35dc0e8e2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.578511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-849894dd-a1da-4a6c-8231-cbd35dc0e8e2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:38.588852 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-849894dd-a1da-4a6c-8231-cbd35dc0e8e2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.236453 Sep 27 10:10:38.590083 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1915/3523] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:38 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:38.608247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:38.608944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:38.618913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.621491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.622121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.622772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.633570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.634096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.791511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0a7fb4a0-6093-4676-9de2-a5eda5c8ef43 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:38.794564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:38.801708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:38.803516 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a7fb4a0-6093-4676-9de2-a5eda5c8ef43 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.529672 Sep 27 10:10:38.805920 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1609/3524] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:38 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:38.807885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:38.809600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:38.810981 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c90f7cc5-48de-4c36-9468-2066ab453af7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1741 microversion: 2.42 time: 0.219682 Sep 27 10:10:38.811594 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1916/3525] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:38 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1741 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:38.812617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b91c1530-e961-4789-b3f1-dc4a4700ebd0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:38.821931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b91c1530-e961-4789-b3f1-dc4a4700ebd0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:38.835723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c1530-e961-4789-b3f1-dc4a4700ebd0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.836346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c1530-e961-4789-b3f1-dc4a4700ebd0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.837527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c1530-e961-4789-b3f1-dc4a4700ebd0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.838448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c1530-e961-4789-b3f1-dc4a4700ebd0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.839063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c1530-e961-4789-b3f1-dc4a4700ebd0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.839795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c1530-e961-4789-b3f1-dc4a4700ebd0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.841642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-631db3a6-d184-450f-88d8-406c59cae5a4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:38.842378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-631db3a6-d184-450f-88d8-406c59cae5a4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:38.855641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-631db3a6-d184-450f-88d8-406c59cae5a4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.856612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-631db3a6-d184-450f-88d8-406c59cae5a4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.857167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-631db3a6-d184-450f-88d8-406c59cae5a4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.858143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-631db3a6-d184-450f-88d8-406c59cae5a4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:38.859848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-631db3a6-d184-450f-88d8-406c59cae5a4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:38.860996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-631db3a6-d184-450f-88d8-406c59cae5a4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:38.932515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b91c1530-e961-4789-b3f1-dc4a4700ebd0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:38.945372 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b91c1530-e961-4789-b3f1-dc4a4700ebd0 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1273 microversion: 2.1 time: 0.138034 Sep 27 10:10:38.945972 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1610/3526] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:38 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1273 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:39.032137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-631db3a6-d184-450f-88d8-406c59cae5a4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:39.042387 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-631db3a6-d184-450f-88d8-406c59cae5a4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.212212 Sep 27 10:10:39.057031 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1917/3527] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:38 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:39.066736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-02a0ff2a-6001-4e2e-961c-4746ba7bcbca tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:39.067645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-02a0ff2a-6001-4e2e-961c-4746ba7bcbca tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:39.077100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02a0ff2a-6001-4e2e-961c-4746ba7bcbca tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.077766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02a0ff2a-6001-4e2e-961c-4746ba7bcbca tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.078312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02a0ff2a-6001-4e2e-961c-4746ba7bcbca tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.079701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02a0ff2a-6001-4e2e-961c-4746ba7bcbca tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.080262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02a0ff2a-6001-4e2e-961c-4746ba7bcbca tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.081442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02a0ff2a-6001-4e2e-961c-4746ba7bcbca tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.107374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-02a0ff2a-6001-4e2e-961c-4746ba7bcbca tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lazy-loading 'flavor' on Instance uuid 3327ff95-5c05-4167-a73b-5fd55f7bfabd {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:10:39.154924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:39.202992 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-02a0ff2a-6001-4e2e-961c-4746ba7bcbca tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 202 len: 0 microversion: 2.1 time: 0.140390 Sep 27 10:10:39.204845 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1611/3528] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:10:39 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 0 bytes in 143 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:10:39.213286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5b67409-0a44-40ca-90fd-465f3dcb842e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:39.214587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b5b67409-0a44-40ca-90fd-465f3dcb842e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:39.216894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:39.223787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5b67409-0a44-40ca-90fd-465f3dcb842e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.224612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5b67409-0a44-40ca-90fd-465f3dcb842e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.225275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5b67409-0a44-40ca-90fd-465f3dcb842e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.226418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5b67409-0a44-40ca-90fd-465f3dcb842e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.227199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5b67409-0a44-40ca-90fd-465f3dcb842e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.227955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5b67409-0a44-40ca-90fd-465f3dcb842e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.259220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d6277cd9-a662-4f65-ae9b-232aae6e586d tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:39.260185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d6277cd9-a662-4f65-ae9b-232aae6e586d tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:39.267760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6277cd9-a662-4f65-ae9b-232aae6e586d tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.269204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6277cd9-a662-4f65-ae9b-232aae6e586d tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.269958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6277cd9-a662-4f65-ae9b-232aae6e586d tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.270940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6277cd9-a662-4f65-ae9b-232aae6e586d tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.271659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6277cd9-a662-4f65-ae9b-232aae6e586d tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.272294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6277cd9-a662-4f65-ae9b-232aae6e586d tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.389708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b5b67409-0a44-40ca-90fd-465f3dcb842e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:39.398196 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b5b67409-0a44-40ca-90fd-465f3dcb842e tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1513 microversion: 2.1 time: 0.188779 Sep 27 10:10:39.398801 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1918/3529] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:39 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1513 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:39.615514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a1a629df-cf22-425d-9f66-b014b1d2bec1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:39.616086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a1a629df-cf22-425d-9f66-b014b1d2bec1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:39.621139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d6277cd9-a662-4f65-ae9b-232aae6e586d tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:39.624554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1a629df-cf22-425d-9f66-b014b1d2bec1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.625061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1a629df-cf22-425d-9f66-b014b1d2bec1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.625801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1a629df-cf22-425d-9f66-b014b1d2bec1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.626218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1a629df-cf22-425d-9f66-b014b1d2bec1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.626641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1a629df-cf22-425d-9f66-b014b1d2bec1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.629363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1a629df-cf22-425d-9f66-b014b1d2bec1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.635383 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d6277cd9-a662-4f65-ae9b-232aae6e586d tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1575 microversion: 2.1 time: 0.382922 Sep 27 10:10:39.651363 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1612/3530] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:39 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1575 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:39.668176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-881a5320-2dcc-4da3-ab2c-76bab407f00a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:39.669829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-881a5320-2dcc-4da3-ab2c-76bab407f00a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:39.679522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881a5320-2dcc-4da3-ab2c-76bab407f00a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.680047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881a5320-2dcc-4da3-ab2c-76bab407f00a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.680574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881a5320-2dcc-4da3-ab2c-76bab407f00a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.681314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881a5320-2dcc-4da3-ab2c-76bab407f00a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.681775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881a5320-2dcc-4da3-ab2c-76bab407f00a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.683380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-881a5320-2dcc-4da3-ab2c-76bab407f00a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.803593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a1a629df-cf22-425d-9f66-b014b1d2bec1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:39.812956 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a1a629df-cf22-425d-9f66-b014b1d2bec1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.201398 Sep 27 10:10:39.833461 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1919/3531] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:39 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:39.842445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:39.843113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:39.852631 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-881a5320-2dcc-4da3-ab2c-76bab407f00a tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.191291 Sep 27 10:10:39.854155 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1613/3532] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:10:39 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d/os-interface => generated 284 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:39.862004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.862529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.862983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.863721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.864185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.864613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.970027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b4f26f7e-8498-46d9-9f59-afbb2a4229ef tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:39.970519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b4f26f7e-8498-46d9-9f59-afbb2a4229ef tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:39.982128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f26f7e-8498-46d9-9f59-afbb2a4229ef tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.986865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f26f7e-8498-46d9-9f59-afbb2a4229ef tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.987515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f26f7e-8498-46d9-9f59-afbb2a4229ef tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:39.992005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f26f7e-8498-46d9-9f59-afbb2a4229ef tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:39.992930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f26f7e-8498-46d9-9f59-afbb2a4229ef tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:39.993971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4f26f7e-8498-46d9-9f59-afbb2a4229ef tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:40.189363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:40.204594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:40.207064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:40.208512 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e8ad5042-0dd5-4c64-bd09-8009ea90a31e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1741 microversion: 2.42 time: 0.371413 Sep 27 10:10:40.209343 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1920/3533] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:39 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1741 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:40.422038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-baecd33e-21a2-43df-a4f8-fd081d1d94ee tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:40.423920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-baecd33e-21a2-43df-a4f8-fd081d1d94ee tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:40.427967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b4f26f7e-8498-46d9-9f59-afbb2a4229ef tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:40.433716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baecd33e-21a2-43df-a4f8-fd081d1d94ee tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:40.434911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baecd33e-21a2-43df-a4f8-fd081d1d94ee tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:40.435690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baecd33e-21a2-43df-a4f8-fd081d1d94ee tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:40.436635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baecd33e-21a2-43df-a4f8-fd081d1d94ee tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:40.437230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baecd33e-21a2-43df-a4f8-fd081d1d94ee tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:40.437744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baecd33e-21a2-43df-a4f8-fd081d1d94ee tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:40.443636 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b4f26f7e-8498-46d9-9f59-afbb2a4229ef tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1315 microversion: 2.1 time: 0.477352 Sep 27 10:10:40.444284 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1614/3534] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:39 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1315 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:40.725231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-baecd33e-21a2-43df-a4f8-fd081d1d94ee tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:40.735958 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-baecd33e-21a2-43df-a4f8-fd081d1d94ee tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1496 microversion: 2.1 time: 0.318191 Sep 27 10:10:40.742559 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1921/3535] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:40 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1496 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:40.760067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:40.761222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:40.777678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:40.779197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:40.779924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:40.781983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:40.782531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:40.783257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:40.826997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lazy-loading 'flavor' on Instance uuid 3327ff95-5c05-4167-a73b-5fd55f7bfabd {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:10:40.842558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1626591d-3b87-421a-8ac9-d4f4ef23337a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:40.843183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1626591d-3b87-421a-8ac9-d4f4ef23337a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:40.844502 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Cannot 'suspend' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state suspended Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action HTTP/1.1 Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 15 Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: {"suspend": {}} Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: exception: Cannot 'suspend' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state suspended Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3327ff95-5c05-4167-a73b-5fd55f7bfabd', 'body': {'suspend': {}}} Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 40, in _suspend Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.compute_api.suspend(context, server) Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 309, in inner Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 259, in inner Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd in vm_state suspended. Cannot suspend while the instance is in this state. Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/suspend_server.py", line 44, in _suspend Sep 27 10:10:40.846911 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'suspend' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state suspended Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'suspend' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state suspended Sep 27 10:10:40.851454 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:40.854740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1626591d-3b87-421a-8ac9-d4f4ef23337a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:40.856533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1626591d-3b87-421a-8ac9-d4f4ef23337a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:40.859810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1626591d-3b87-421a-8ac9-d4f4ef23337a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:40.862684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1626591d-3b87-421a-8ac9-d4f4ef23337a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:40.863364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1626591d-3b87-421a-8ac9-d4f4ef23337a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:40.864754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1626591d-3b87-421a-8ac9-d4f4ef23337a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:40.871963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 409 to user: Cannot 'suspend' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state suspended {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:40.873775 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32e7f5b6-a35c-484f-92d5-d31a414e8e03 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 409 len: 148 microversion: 2.1 time: 0.117749 Sep 27 10:10:40.874687 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1615/3536] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:10:40 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 148 bytes in 120 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Sep 27 10:10:40.886218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc178011-6486-4d0a-bc99-3aeec59e517c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:40.887082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc178011-6486-4d0a-bc99-3aeec59e517c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:40.894878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc178011-6486-4d0a-bc99-3aeec59e517c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:40.895659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc178011-6486-4d0a-bc99-3aeec59e517c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:40.896426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc178011-6486-4d0a-bc99-3aeec59e517c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:40.897356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc178011-6486-4d0a-bc99-3aeec59e517c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:40.898447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc178011-6486-4d0a-bc99-3aeec59e517c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:40.899032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc178011-6486-4d0a-bc99-3aeec59e517c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:40.997936 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cc178011-6486-4d0a-bc99-3aeec59e517c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 202 len: 0 microversion: 2.1 time: 0.116625 Sep 27 10:10:41.000688 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1616/3537] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:10:40 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 0 bytes in 118 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:10:41.018426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-741a0ab9-8300-4d4b-bb57-1ed0cfd456c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:41.019751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-741a0ab9-8300-4d4b-bb57-1ed0cfd456c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:41.030343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-741a0ab9-8300-4d4b-bb57-1ed0cfd456c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:41.031004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-741a0ab9-8300-4d4b-bb57-1ed0cfd456c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:41.031652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-741a0ab9-8300-4d4b-bb57-1ed0cfd456c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:41.032515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-741a0ab9-8300-4d4b-bb57-1ed0cfd456c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:41.033218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-741a0ab9-8300-4d4b-bb57-1ed0cfd456c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:41.033989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-741a0ab9-8300-4d4b-bb57-1ed0cfd456c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:41.178924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1626591d-3b87-421a-8ac9-d4f4ef23337a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:41.194153 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1626591d-3b87-421a-8ac9-d4f4ef23337a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.356768 Sep 27 10:10:41.206238 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1922/3538] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:40 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:41.234958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:41.235946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:41.242141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:41.242880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:41.243308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:41.244160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:41.244624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:41.245076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:41.501014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-741a0ab9-8300-4d4b-bb57-1ed0cfd456c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:41.510562 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-741a0ab9-8300-4d4b-bb57-1ed0cfd456c9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1502 microversion: 2.1 time: 0.503990 Sep 27 10:10:41.511303 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1617/3539] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:41 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1502 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:41.532285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58aa570c-d662-4d16-b7c4-4cf9ed1ce0f5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:41.533934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-58aa570c-d662-4d16-b7c4-4cf9ed1ce0f5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:41.542874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58aa570c-d662-4d16-b7c4-4cf9ed1ce0f5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:41.543389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58aa570c-d662-4d16-b7c4-4cf9ed1ce0f5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:41.544612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58aa570c-d662-4d16-b7c4-4cf9ed1ce0f5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:41.545465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58aa570c-d662-4d16-b7c4-4cf9ed1ce0f5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:41.546007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58aa570c-d662-4d16-b7c4-4cf9ed1ce0f5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:41.547411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58aa570c-d662-4d16-b7c4-4cf9ed1ce0f5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:41.797042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:41.811260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:41.813281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:41.815494 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7dd1a9f9-ed85-435e-8824-e2ead70048de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1787 microversion: 2.42 time: 0.584684 Sep 27 10:10:41.816565 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1923/3540] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:41 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1787 bytes in 586 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:10:41.854318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4e26c1cb-3581-4564-9b79-be12e5e5e425 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:41.855090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4e26c1cb-3581-4564-9b79-be12e5e5e425 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:41.858201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:41.863942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e26c1cb-3581-4564-9b79-be12e5e5e425 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:41.864528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e26c1cb-3581-4564-9b79-be12e5e5e425 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:41.865211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e26c1cb-3581-4564-9b79-be12e5e5e425 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:41.866000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e26c1cb-3581-4564-9b79-be12e5e5e425 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:41.866672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e26c1cb-3581-4564-9b79-be12e5e5e425 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:41.867214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e26c1cb-3581-4564-9b79-be12e5e5e425 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.020871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4e26c1cb-3581-4564-9b79-be12e5e5e425 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:42.029924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4e26c1cb-3581-4564-9b79-be12e5e5e425 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1575 microversion: 2.1 time: 0.179778 Sep 27 10:10:42.030528 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1924/3541] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:41 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1575 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:42.035709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-23695b7e-e5e6-4e64-961f-a8d64c5bb96a req-9bb2be98-2ce8-4d80-ba9e-a1c1dea46980 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "451a2ed4-e57c-46fc-8e29-09ad474cfa9d", "tag": "8ab5011a-0825-464f-88c1-5874577eff8e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:42.046805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-23695b7e-e5e6-4e64-961f-a8d64c5bb96a req-9bb2be98-2ce8-4d80-ba9e-a1c1dea46980 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.047569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-23695b7e-e5e6-4e64-961f-a8d64c5bb96a req-9bb2be98-2ce8-4d80-ba9e-a1c1dea46980 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.048168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-23695b7e-e5e6-4e64-961f-a8d64c5bb96a req-9bb2be98-2ce8-4d80-ba9e-a1c1dea46980 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.062245 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-23695b7e-e5e6-4e64-961f-a8d64c5bb96a req-9bb2be98-2ce8-4d80-ba9e-a1c1dea46980 service nova] Creating event network-changed:8ab5011a-0825-464f-88c1-5874577eff8e for instance 451a2ed4-e57c-46fc-8e29-09ad474cfa9d on nch1-master-1 Sep 27 10:10:42.068773 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-23695b7e-e5e6-4e64-961f-a8d64c5bb96a req-9bb2be98-2ce8-4d80-ba9e-a1c1dea46980 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036325 Sep 27 10:10:42.070485 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1925/3542] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:42.076576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c701398d-f853-4340-a219-367e58ce1d61 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:42.077407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c701398d-f853-4340-a219-367e58ce1d61 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:42.090934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c701398d-f853-4340-a219-367e58ce1d61 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.091438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c701398d-f853-4340-a219-367e58ce1d61 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.092477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c701398d-f853-4340-a219-367e58ce1d61 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.093268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c701398d-f853-4340-a219-367e58ce1d61 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.093912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c701398d-f853-4340-a219-367e58ce1d61 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.094353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c701398d-f853-4340-a219-367e58ce1d61 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.123269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c701398d-f853-4340-a219-367e58ce1d61 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:10:42.225098 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c701398d-f853-4340-a219-367e58ce1d61 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 204 len: 0 microversion: 2.1 time: 0.152820 Sep 27 10:10:42.226853 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1926/3543] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:10:42 2022] DELETE /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:10:42.235353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad1852ec-1205-4d60-9527-77ef9683579c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:42.236211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ad1852ec-1205-4d60-9527-77ef9683579c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:42.244436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad1852ec-1205-4d60-9527-77ef9683579c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.245138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad1852ec-1205-4d60-9527-77ef9683579c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.245602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad1852ec-1205-4d60-9527-77ef9683579c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.246318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad1852ec-1205-4d60-9527-77ef9683579c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.246786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad1852ec-1205-4d60-9527-77ef9683579c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.247198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad1852ec-1205-4d60-9527-77ef9683579c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.348963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-58aa570c-d662-4d16-b7c4-4cf9ed1ce0f5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:42.364239 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-58aa570c-d662-4d16-b7c4-4cf9ed1ce0f5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1315 microversion: 2.1 time: 0.838932 Sep 27 10:10:42.365709 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1618/3544] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:41 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1315 bytes in 841 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:42.372542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4a8d38e0-bbe3-4f49-b970-6e2fd581e84d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:42.373410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4a8d38e0-bbe3-4f49-b970-6e2fd581e84d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:42.382994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a8d38e0-bbe3-4f49-b970-6e2fd581e84d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.383599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a8d38e0-bbe3-4f49-b970-6e2fd581e84d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.384040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a8d38e0-bbe3-4f49-b970-6e2fd581e84d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.384833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a8d38e0-bbe3-4f49-b970-6e2fd581e84d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.385471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a8d38e0-bbe3-4f49-b970-6e2fd581e84d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.386007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a8d38e0-bbe3-4f49-b970-6e2fd581e84d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.406481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ad1852ec-1205-4d60-9527-77ef9683579c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:42.416464 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ad1852ec-1205-4d60-9527-77ef9683579c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.187358 Sep 27 10:10:42.417556 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1927/3545] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:42 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:42.536250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-bef0561f-ea37-43d3-a3a4-3a8428fb6b4d req-aae18193-698b-4dbf-af5c-7d65860c9254 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "668efb48-676d-4a61-837f-eba6c5365546", "tag": "48a8d3b8-b284-4793-8bb8-001085caddfc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:42.551878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bef0561f-ea37-43d3-a3a4-3a8428fb6b4d req-aae18193-698b-4dbf-af5c-7d65860c9254 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.552556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bef0561f-ea37-43d3-a3a4-3a8428fb6b4d req-aae18193-698b-4dbf-af5c-7d65860c9254 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.553831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bef0561f-ea37-43d3-a3a4-3a8428fb6b4d req-aae18193-698b-4dbf-af5c-7d65860c9254 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.566568 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-bef0561f-ea37-43d3-a3a4-3a8428fb6b4d req-aae18193-698b-4dbf-af5c-7d65860c9254 service nova] Creating event network-changed:48a8d3b8-b284-4793-8bb8-001085caddfc for instance 668efb48-676d-4a61-837f-eba6c5365546 on nch1-master-1 Sep 27 10:10:42.571840 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-bef0561f-ea37-43d3-a3a4-3a8428fb6b4d req-aae18193-698b-4dbf-af5c-7d65860c9254 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041979 Sep 27 10:10:42.572764 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1928/3546] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:42.577990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-27948471-7120-425f-a50b-adf7537ad8d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:42.579053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-27948471-7120-425f-a50b-adf7537ad8d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:42.586580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27948471-7120-425f-a50b-adf7537ad8d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.587149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27948471-7120-425f-a50b-adf7537ad8d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.587768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27948471-7120-425f-a50b-adf7537ad8d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.589075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27948471-7120-425f-a50b-adf7537ad8d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.589604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27948471-7120-425f-a50b-adf7537ad8d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.590102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27948471-7120-425f-a50b-adf7537ad8d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.762117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-27948471-7120-425f-a50b-adf7537ad8d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:42.771427 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-27948471-7120-425f-a50b-adf7537ad8d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1502 microversion: 2.1 time: 0.197265 Sep 27 10:10:42.772021 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1929/3547] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:42 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1502 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:42.845892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:42.846898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:42.855169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.855733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.856271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.857105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:42.857623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:42.858234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:42.941629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4a8d38e0-bbe3-4f49-b970-6e2fd581e84d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:42.952147 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4a8d38e0-bbe3-4f49-b970-6e2fd581e84d tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1665 microversion: 2.1 time: 0.584594 Sep 27 10:10:42.969042 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1619/3548] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:42 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1665 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:43.121740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:43.135118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:43.136932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:43.138722 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-39aded4e-39eb-475b-8c85-0236c95cfb29 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1833 microversion: 2.42 time: 0.298193 Sep 27 10:10:43.152680 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1930/3549] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:42 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1833 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:43.392639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0a71d2f-2b03-4048-8944-690d9f53b67e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:43.394214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b0a71d2f-2b03-4048-8944-690d9f53b67e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:43.402866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0a71d2f-2b03-4048-8944-690d9f53b67e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:43.403419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0a71d2f-2b03-4048-8944-690d9f53b67e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:43.404065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0a71d2f-2b03-4048-8944-690d9f53b67e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:43.404773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0a71d2f-2b03-4048-8944-690d9f53b67e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:43.405416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0a71d2f-2b03-4048-8944-690d9f53b67e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:43.405824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0a71d2f-2b03-4048-8944-690d9f53b67e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:43.445412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af081ea5-5cd2-41b8-ab12-6989c543b156 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:43.446731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af081ea5-5cd2-41b8-ab12-6989c543b156 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:43.455554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af081ea5-5cd2-41b8-ab12-6989c543b156 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:43.456406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af081ea5-5cd2-41b8-ab12-6989c543b156 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:43.456905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af081ea5-5cd2-41b8-ab12-6989c543b156 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:43.457749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af081ea5-5cd2-41b8-ab12-6989c543b156 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:43.458329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af081ea5-5cd2-41b8-ab12-6989c543b156 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:43.458759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af081ea5-5cd2-41b8-ab12-6989c543b156 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:43.645780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af081ea5-5cd2-41b8-ab12-6989c543b156 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:43.655113 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af081ea5-5cd2-41b8-ab12-6989c543b156 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.218629 Sep 27 10:10:43.655705 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1931/3550] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:43 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:43.798403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a480f19a-e732-4209-90c7-7cd80ac99ce2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:43.799118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a480f19a-e732-4209-90c7-7cd80ac99ce2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:43.805387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a480f19a-e732-4209-90c7-7cd80ac99ce2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:43.805966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a480f19a-e732-4209-90c7-7cd80ac99ce2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:43.806648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a480f19a-e732-4209-90c7-7cd80ac99ce2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:43.807909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a480f19a-e732-4209-90c7-7cd80ac99ce2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:43.808764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a480f19a-e732-4209-90c7-7cd80ac99ce2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:43.809262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a480f19a-e732-4209-90c7-7cd80ac99ce2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:43.913353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b0a71d2f-2b03-4048-8944-690d9f53b67e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:43.935203 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0a71d2f-2b03-4048-8944-690d9f53b67e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1315 microversion: 2.1 time: 0.547650 Sep 27 10:10:43.950543 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1620/3551] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:43 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1315 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:43.977811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e246e7ea-b8f5-465e-985d-8c9475e99734 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:43.978429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e246e7ea-b8f5-465e-985d-8c9475e99734 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:43.984819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e246e7ea-b8f5-465e-985d-8c9475e99734 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:43.985300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e246e7ea-b8f5-465e-985d-8c9475e99734 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:43.985734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e246e7ea-b8f5-465e-985d-8c9475e99734 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:43.986441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e246e7ea-b8f5-465e-985d-8c9475e99734 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:43.986893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e246e7ea-b8f5-465e-985d-8c9475e99734 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:43.987376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e246e7ea-b8f5-465e-985d-8c9475e99734 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:44.001029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a480f19a-e732-4209-90c7-7cd80ac99ce2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:44.012200 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a480f19a-e732-4209-90c7-7cd80ac99ce2 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1502 microversion: 2.1 time: 0.217719 Sep 27 10:10:44.013512 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1932/3552] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:43 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1502 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:44.165759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:44.166370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:44.173437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:44.173932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:44.174395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:44.175112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:44.175564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:44.175989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:44.180799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:44.372609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:44.383189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:44.384897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:44.386327 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e453ec1b-b5f1-4d3a-8c21-df7847d3c7b8 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1833 microversion: 2.42 time: 0.225496 Sep 27 10:10:44.401858 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1933/3553] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:44 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1833 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:44.684011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bd91127d-cdf3-426f-91b7-be762003e527 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:44.685673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bd91127d-cdf3-426f-91b7-be762003e527 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:44.697849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd91127d-cdf3-426f-91b7-be762003e527 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:44.698507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd91127d-cdf3-426f-91b7-be762003e527 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:44.698711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e246e7ea-b8f5-465e-985d-8c9475e99734 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:44.699673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd91127d-cdf3-426f-91b7-be762003e527 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:44.700982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd91127d-cdf3-426f-91b7-be762003e527 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:44.701653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd91127d-cdf3-426f-91b7-be762003e527 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:44.702390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd91127d-cdf3-426f-91b7-be762003e527 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:44.709151 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e246e7ea-b8f5-465e-985d-8c9475e99734 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1665 microversion: 2.1 time: 0.736111 Sep 27 10:10:44.725554 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1621/3554] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:43 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1665 bytes in 745 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:44.869823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bd91127d-cdf3-426f-91b7-be762003e527 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:44.878533 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bd91127d-cdf3-426f-91b7-be762003e527 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.200469 Sep 27 10:10:44.879723 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1934/3555] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:44 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:44.964472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b81200cf-ee83-4bd4-bc17-51e39dee1751 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:44.964601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b81200cf-ee83-4bd4-bc17-51e39dee1751 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:44.971783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b81200cf-ee83-4bd4-bc17-51e39dee1751 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:44.972502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b81200cf-ee83-4bd4-bc17-51e39dee1751 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:44.973282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b81200cf-ee83-4bd4-bc17-51e39dee1751 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:44.974218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b81200cf-ee83-4bd4-bc17-51e39dee1751 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:44.974950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b81200cf-ee83-4bd4-bc17-51e39dee1751 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:44.975757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b81200cf-ee83-4bd4-bc17-51e39dee1751 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:45.036870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fc5f31f0-e896-41d3-8882-17de75982221 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:45.037469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fc5f31f0-e896-41d3-8882-17de75982221 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:45.044852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc5f31f0-e896-41d3-8882-17de75982221 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:45.046291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc5f31f0-e896-41d3-8882-17de75982221 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:45.047163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc5f31f0-e896-41d3-8882-17de75982221 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:45.047929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc5f31f0-e896-41d3-8882-17de75982221 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:45.048622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc5f31f0-e896-41d3-8882-17de75982221 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:45.049224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc5f31f0-e896-41d3-8882-17de75982221 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:45.219398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc5f31f0-e896-41d3-8882-17de75982221 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:45.225441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:45.230013 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fc5f31f0-e896-41d3-8882-17de75982221 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1502 microversion: 2.1 time: 0.197510 Sep 27 10:10:45.231454 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1935/3556] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:45 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1502 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:45.414637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b81200cf-ee83-4bd4-bc17-51e39dee1751 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:45.417733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:45.421201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:45.428956 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b81200cf-ee83-4bd4-bc17-51e39dee1751 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1315 microversion: 2.1 time: 0.469414 Sep 27 10:10:45.429639 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1622/3557] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:44 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1315 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:45.443543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:45.443543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:45.443543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:45.443543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:45.444239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:45.445785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:45.668549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:45.678621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:45.680533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:45.681934 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13e7016e-e6fa-49b7-8e8b-e8c587137cd0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1833 microversion: 2.42 time: 0.271104 Sep 27 10:10:45.682564 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1936/3558] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:45 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1833 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:45.733202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d67e1d93-6092-4d5e-8968-de94955d0ffa tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:45.736102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d67e1d93-6092-4d5e-8968-de94955d0ffa tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:45.742892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d67e1d93-6092-4d5e-8968-de94955d0ffa tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:45.743934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d67e1d93-6092-4d5e-8968-de94955d0ffa tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:45.744526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d67e1d93-6092-4d5e-8968-de94955d0ffa tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:45.745477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d67e1d93-6092-4d5e-8968-de94955d0ffa tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:45.746100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d67e1d93-6092-4d5e-8968-de94955d0ffa tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:45.746974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d67e1d93-6092-4d5e-8968-de94955d0ffa tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:45.864202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a31fa754-39fd-4276-8686-6315c25d1c23 req-b7d3b585-844d-4cb0-b9c1-17f074bcde3d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "tag": "8cd8623a-a3f6-4d1c-aced-9707f8353fa6"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:45.881048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a31fa754-39fd-4276-8686-6315c25d1c23 req-b7d3b585-844d-4cb0-b9c1-17f074bcde3d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:45.881956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a31fa754-39fd-4276-8686-6315c25d1c23 req-b7d3b585-844d-4cb0-b9c1-17f074bcde3d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:45.882642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a31fa754-39fd-4276-8686-6315c25d1c23 req-b7d3b585-844d-4cb0-b9c1-17f074bcde3d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:45.896839 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a31fa754-39fd-4276-8686-6315c25d1c23 req-b7d3b585-844d-4cb0-b9c1-17f074bcde3d service nova] Creating event network-changed:8cd8623a-a3f6-4d1c-aced-9707f8353fa6 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:10:45.903562 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a31fa754-39fd-4276-8686-6315c25d1c23 req-b7d3b585-844d-4cb0-b9c1-17f074bcde3d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045965 Sep 27 10:10:45.904270 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1937/3559] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:45.911244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d12e381-69a0-4039-ac08-3e9c3df98497 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:45.912035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d12e381-69a0-4039-ac08-3e9c3df98497 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:45.921225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d12e381-69a0-4039-ac08-3e9c3df98497 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:45.922243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d12e381-69a0-4039-ac08-3e9c3df98497 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:45.922599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d67e1d93-6092-4d5e-8968-de94955d0ffa tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:45.922877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d12e381-69a0-4039-ac08-3e9c3df98497 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:45.923679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d12e381-69a0-4039-ac08-3e9c3df98497 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:45.927363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d12e381-69a0-4039-ac08-3e9c3df98497 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:45.928110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d12e381-69a0-4039-ac08-3e9c3df98497 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:45.936014 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d67e1d93-6092-4d5e-8968-de94955d0ffa tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1597 microversion: 2.1 time: 0.206573 Sep 27 10:10:45.936613 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1623/3560] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:45 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1597 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:46.096687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1d12e381-69a0-4039-ac08-3e9c3df98497 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:46.108371 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d12e381-69a0-4039-ac08-3e9c3df98497 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.202538 Sep 27 10:10:46.109088 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1938/3561] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:45 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:46.260700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7f38afd7-8a38-431f-8edf-ba23c15a58bf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:46.264217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7f38afd7-8a38-431f-8edf-ba23c15a58bf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:46.275505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:46.279975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f38afd7-8a38-431f-8edf-ba23c15a58bf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:46.281062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f38afd7-8a38-431f-8edf-ba23c15a58bf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:46.282020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f38afd7-8a38-431f-8edf-ba23c15a58bf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:46.283337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f38afd7-8a38-431f-8edf-ba23c15a58bf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:46.283964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f38afd7-8a38-431f-8edf-ba23c15a58bf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:46.284561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7f38afd7-8a38-431f-8edf-ba23c15a58bf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:46.455878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d467fc6-91e7-499e-8fa4-549bab45627b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:46.457122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d467fc6-91e7-499e-8fa4-549bab45627b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:46.465417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d467fc6-91e7-499e-8fa4-549bab45627b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:46.466510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d467fc6-91e7-499e-8fa4-549bab45627b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:46.467335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d467fc6-91e7-499e-8fa4-549bab45627b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:46.468338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d467fc6-91e7-499e-8fa4-549bab45627b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:46.468988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d467fc6-91e7-499e-8fa4-549bab45627b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:46.469830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d467fc6-91e7-499e-8fa4-549bab45627b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:46.635320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1d467fc6-91e7-499e-8fa4-549bab45627b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:46.651009 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d467fc6-91e7-499e-8fa4-549bab45627b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1315 microversion: 2.1 time: 0.194989 Sep 27 10:10:46.651689 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1939/3562] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:46 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1315 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:46.711082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:46.712100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:46.720102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:46.721006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:46.721737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:46.723051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:46.723741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:46.724324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:46.908401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:46.924089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:46.925814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:46.927698 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-91251817-d763-4c3f-8667-0d90cdaaed4c tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1833 microversion: 2.42 time: 0.221017 Sep 27 10:10:46.928275 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1940/3563] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:46 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1833 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:46.969018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-afdb6505-7c54-4fd8-8b11-b8ea1d2f315a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:46.969893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-afdb6505-7c54-4fd8-8b11-b8ea1d2f315a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:46.976556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afdb6505-7c54-4fd8-8b11-b8ea1d2f315a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:46.977177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afdb6505-7c54-4fd8-8b11-b8ea1d2f315a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:46.977617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afdb6505-7c54-4fd8-8b11-b8ea1d2f315a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:46.978328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afdb6505-7c54-4fd8-8b11-b8ea1d2f315a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:46.978757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afdb6505-7c54-4fd8-8b11-b8ea1d2f315a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:46.979247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-afdb6505-7c54-4fd8-8b11-b8ea1d2f315a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.006090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7f38afd7-8a38-431f-8edf-ba23c15a58bf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:47.019954 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7f38afd7-8a38-431f-8edf-ba23c15a58bf tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.768783 Sep 27 10:10:47.027059 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1624/3564] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:46 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 777 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:47.053422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e4e81045-a3b1-46a2-8e30-1061fbadd35f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:47.054346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e4e81045-a3b1-46a2-8e30-1061fbadd35f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:47.061502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-afdb6505-7c54-4fd8-8b11-b8ea1d2f315a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:47.062418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4e81045-a3b1-46a2-8e30-1061fbadd35f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:47.070462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4e81045-a3b1-46a2-8e30-1061fbadd35f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:47.071060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4e81045-a3b1-46a2-8e30-1061fbadd35f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.071944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4e81045-a3b1-46a2-8e30-1061fbadd35f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:47.072058 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-afdb6505-7c54-4fd8-8b11-b8ea1d2f315a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1597 microversion: 2.1 time: 0.109556 Sep 27 10:10:47.072592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4e81045-a3b1-46a2-8e30-1061fbadd35f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:47.072775 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1941/3565] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:46 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1597 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:47.073362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4e81045-a3b1-46a2-8e30-1061fbadd35f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.132963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8ece67e-7e3d-476f-9813-9a75b30f8630 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:47.133740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c8ece67e-7e3d-476f-9813-9a75b30f8630 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:47.141349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8ece67e-7e3d-476f-9813-9a75b30f8630 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:47.142143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8ece67e-7e3d-476f-9813-9a75b30f8630 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:47.142748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8ece67e-7e3d-476f-9813-9a75b30f8630 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.143835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8ece67e-7e3d-476f-9813-9a75b30f8630 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:47.144307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8ece67e-7e3d-476f-9813-9a75b30f8630 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:47.144759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8ece67e-7e3d-476f-9813-9a75b30f8630 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.332777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c8ece67e-7e3d-476f-9813-9a75b30f8630 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:47.346161 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c8ece67e-7e3d-476f-9813-9a75b30f8630 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.217094 Sep 27 10:10:47.347859 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1942/3566] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:47 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:47.390677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e4e81045-a3b1-46a2-8e30-1061fbadd35f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:47.415014 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e4e81045-a3b1-46a2-8e30-1061fbadd35f tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.364005 Sep 27 10:10:47.431327 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1625/3567] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:47 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:47.439271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8ee3ca3-eb07-4213-beac-8a43aa7df20b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:47.440107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c8ee3ca3-eb07-4213-beac-8a43aa7df20b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 27a3a03d-3fd1-4d45-93c9-a9e70ecd7372] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:47.450291 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-c8ee3ca3-eb07-4213-beac-8a43aa7df20b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 27a3a03d-3fd1-4d45-93c9-a9e70ecd7372 could not be found. Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8ee3ca3-eb07-4213-beac-8a43aa7df20b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/27a3a03d-3fd1-4d45-93c9-a9e70ecd7372/action HTTP/1.1 Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 15 Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: {"unpause": {}} Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 27a3a03d-3fd1-4d45-93c9-a9e70ecd7372 could not be found. Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '27a3a03d-3fd1-4d45-93c9-a9e70ecd7372', 'body': {'unpause': {}}} Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 27a3a03d-3fd1-4d45-93c9-a9e70ecd7372 could not be found. Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 58, in _unpause Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, ctxt, id) Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 27a3a03d-3fd1-4d45-93c9-a9e70ecd7372 could not be found. Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:47.452769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:47.459800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:47.459800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:47.459800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 27a3a03d-3fd1-4d45-93c9-a9e70ecd7372 could not be found. Sep 27 10:10:47.459800 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:47.461387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8ee3ca3-eb07-4213-beac-8a43aa7df20b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 27a3a03d-3fd1-4d45-93c9-a9e70ecd7372 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:47.462629 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c8ee3ca3-eb07-4213-beac-8a43aa7df20b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/27a3a03d-3fd1-4d45-93c9-a9e70ecd7372/action" status: 404 len: 111 microversion: 2.1 time: 0.028033 Sep 27 10:10:47.463429 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1943/3568] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:10:47 2022] POST /compute/v2.1/servers/27a3a03d-3fd1-4d45-93c9-a9e70ecd7372/action => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:10:47.473269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2fa19596-1591-416b-ace2-ec21cc74a7c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:47.473832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2fa19596-1591-416b-ace2-ec21cc74a7c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:47.480443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fa19596-1591-416b-ace2-ec21cc74a7c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:47.480956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fa19596-1591-416b-ace2-ec21cc74a7c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:47.481564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fa19596-1591-416b-ace2-ec21cc74a7c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.482499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fa19596-1591-416b-ace2-ec21cc74a7c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:47.483125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fa19596-1591-416b-ace2-ec21cc74a7c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:47.483810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fa19596-1591-416b-ace2-ec21cc74a7c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.676162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af1b1c1e-9e9c-4ee0-a7c5-fb5972976094 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:47.676831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af1b1c1e-9e9c-4ee0-a7c5-fb5972976094 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:47.684795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af1b1c1e-9e9c-4ee0-a7c5-fb5972976094 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:47.685289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af1b1c1e-9e9c-4ee0-a7c5-fb5972976094 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:47.685920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af1b1c1e-9e9c-4ee0-a7c5-fb5972976094 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.686779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af1b1c1e-9e9c-4ee0-a7c5-fb5972976094 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:47.687455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af1b1c1e-9e9c-4ee0-a7c5-fb5972976094 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:47.688039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af1b1c1e-9e9c-4ee0-a7c5-fb5972976094 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.859245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af1b1c1e-9e9c-4ee0-a7c5-fb5972976094 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:47.868280 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af1b1c1e-9e9c-4ee0-a7c5-fb5972976094 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1315 microversion: 2.1 time: 0.196366 Sep 27 10:10:47.888273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2fa19596-1591-416b-ace2-ec21cc74a7c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:47.892830 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1944/3569] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:47 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1315 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:47.907172 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2fa19596-1591-416b-ace2-ec21cc74a7c3 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.438559 Sep 27 10:10:47.908329 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1626/3570] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:47 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:47.927731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-11ee06d9-2427-428b-9880-c024fc609bbe req-420c52dc-308d-4dce-8d1e-a0857a3b67e1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9ac32528-351d-4ba2-b0ec-0eeb4872e95c", "name": "network-vif-deleted", "tag": "6f38bb60-56d7-44ce-ad27-ae8c1e3ac0fb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:47.930909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c594577d-7a00-42a8-9af7-7d11ebef0b71 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:47.932068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c594577d-7a00-42a8-9af7-7d11ebef0b71 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:47.939835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c594577d-7a00-42a8-9af7-7d11ebef0b71 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:47.940579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c594577d-7a00-42a8-9af7-7d11ebef0b71 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:47.941203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c594577d-7a00-42a8-9af7-7d11ebef0b71 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.942130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c594577d-7a00-42a8-9af7-7d11ebef0b71 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:47.942751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c594577d-7a00-42a8-9af7-7d11ebef0b71 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:47.943679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c594577d-7a00-42a8-9af7-7d11ebef0b71 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.945083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-11ee06d9-2427-428b-9880-c024fc609bbe req-420c52dc-308d-4dce-8d1e-a0857a3b67e1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:47.945673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-11ee06d9-2427-428b-9880-c024fc609bbe req-420c52dc-308d-4dce-8d1e-a0857a3b67e1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:47.946291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-11ee06d9-2427-428b-9880-c024fc609bbe req-420c52dc-308d-4dce-8d1e-a0857a3b67e1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:47.970437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=96,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:47.974866 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-11ee06d9-2427-428b-9880-c024fc609bbe req-420c52dc-308d-4dce-8d1e-a0857a3b67e1 service nova] Creating event network-vif-deleted:6f38bb60-56d7-44ce-ad27-ae8c1e3ac0fb for instance 9ac32528-351d-4ba2-b0ec-0eeb4872e95c on nch2-master-1 Sep 27 10:10:47.983242 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-11ee06d9-2427-428b-9880-c024fc609bbe req-420c52dc-308d-4dce-8d1e-a0857a3b67e1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058530 Sep 27 10:10:47.984542 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1945/3571] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:47.993444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:47.994310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:48.002700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.003292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.003840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.004607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.005464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.006209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.207686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c594577d-7a00-42a8-9af7-7d11ebef0b71 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:48.217464 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c594577d-7a00-42a8-9af7-7d11ebef0b71 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.290731 Sep 27 10:10:48.220796 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1627/3572] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:47 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:48.225972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:48.231887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:48.243622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:48.250106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.250624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.251077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.251798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.255514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.256279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.261351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:48.264512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:48.266257 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2fe8545b-03ff-42a6-b902-56a869b6bdaa tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1854 microversion: 2.42 time: 0.278698 Sep 27 10:10:48.267598 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1946/3573] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:47 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1854 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:48.275046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:48.275952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:48.286722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.287402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.289476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.290475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.291143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.292213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.339192 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Cannot 'unpause' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action HTTP/1.1 Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 15 Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: {"unpause": {}} Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: exception: Cannot 'unpause' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3327ff95-5c05-4167-a73b-5fd55f7bfabd', 'body': {'unpause': {}}} Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 62, in _unpause Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.unpause(ctxt, server) Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd in vm_state active. Cannot unpause while the instance is in this state. Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/pause_server.py", line 66, in _unpause Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'unpause' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:48.343416 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:48.348864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:48.348864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:48.348864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:48.348864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'unpause' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:10:48.348864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:48.350309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 409 to user: Cannot 'unpause' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:48.351517 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-45aae6ce-f213-4e16-a83f-d8641a705791 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 409 len: 145 microversion: 2.1 time: 0.081843 Sep 27 10:10:48.352919 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1947/3574] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:10:48 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 145 bytes in 84 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:10:48.367323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.368007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9e289ab-e80a-47e6-b7a7-d31e520b376c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:48.368142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.368524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.369014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lazy-loading 'fault' on Instance uuid 9ac32528-351d-4ba2-b0ec-0eeb4872e95c {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:10:48.369341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b9e289ab-e80a-47e6-b7a7-d31e520b376c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:48.376233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9e289ab-e80a-47e6-b7a7-d31e520b376c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.377130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9e289ab-e80a-47e6-b7a7-d31e520b376c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.377760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9e289ab-e80a-47e6-b7a7-d31e520b376c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.378781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9e289ab-e80a-47e6-b7a7-d31e520b376c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.379458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9e289ab-e80a-47e6-b7a7-d31e520b376c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.381200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9e289ab-e80a-47e6-b7a7-d31e520b376c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.486518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f06d16c90a5d464696f8137612c35327', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '327b414889a04e189a7f940685a4ff39', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:48.499858 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9eb50b79-979f-4902-832c-d140e7d31006 tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 200 len: 1315 microversion: 2.1 time: 0.277569 Sep 27 10:10:48.502805 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1628/3575] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:48 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 1315 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:48.510246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4ba7b0ed-25c8-4e63-be75-6ee56f04ba47 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:48.511013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4ba7b0ed-25c8-4e63-be75-6ee56f04ba47 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:48.520550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ba7b0ed-25c8-4e63-be75-6ee56f04ba47 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.528948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ba7b0ed-25c8-4e63-be75-6ee56f04ba47 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.530708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ba7b0ed-25c8-4e63-be75-6ee56f04ba47 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.531620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ba7b0ed-25c8-4e63-be75-6ee56f04ba47 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.534898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ba7b0ed-25c8-4e63-be75-6ee56f04ba47 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.535796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ba7b0ed-25c8-4e63-be75-6ee56f04ba47 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.588301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b9e289ab-e80a-47e6-b7a7-d31e520b376c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:48.596814 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b9e289ab-e80a-47e6-b7a7-d31e520b376c tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.238213 Sep 27 10:10:48.597602 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1948/3576] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:48 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:48.621273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-93cdb49d-ab2e-4a3b-8b7c-bdd084f7ddab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:48.621896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-93cdb49d-ab2e-4a3b-8b7c-bdd084f7ddab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:48.628375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93cdb49d-ab2e-4a3b-8b7c-bdd084f7ddab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.628933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93cdb49d-ab2e-4a3b-8b7c-bdd084f7ddab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.629385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93cdb49d-ab2e-4a3b-8b7c-bdd084f7ddab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.630096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93cdb49d-ab2e-4a3b-8b7c-bdd084f7ddab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.630580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93cdb49d-ab2e-4a3b-8b7c-bdd084f7ddab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.630995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93cdb49d-ab2e-4a3b-8b7c-bdd084f7ddab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.807271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-93cdb49d-ab2e-4a3b-8b7c-bdd084f7ddab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:48.815612 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-93cdb49d-ab2e-4a3b-8b7c-bdd084f7ddab tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.198169 Sep 27 10:10:48.816745 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1949/3577] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:48 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:48.840231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-52984a1a-485d-4084-a597-252799107b1b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:48.841502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4ba7b0ed-25c8-4e63-be75-6ee56f04ba47 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:48.841638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-52984a1a-485d-4084-a597-252799107b1b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: d673999a-cb2c-41e9-b1a7-dc336cc1f52e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:48.849581 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-52984a1a-485d-4084-a597-252799107b1b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance d673999a-cb2c-41e9-b1a7-dc336cc1f52e could not be found. Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-52984a1a-485d-4084-a597-252799107b1b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/d673999a-cb2c-41e9-b1a7-dc336cc1f52e/action HTTP/1.1 Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 18 Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: {"unshelve": null} Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance d673999a-cb2c-41e9-b1a7-dc336cc1f52e could not be found. Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'd673999a-cb2c-41e9-b1a7-dc336cc1f52e', 'body': {'unshelve': None}} Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d673999a-cb2c-41e9-b1a7-dc336cc1f52e could not be found. Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 102, in _unshelve Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d673999a-cb2c-41e9-b1a7-dc336cc1f52e could not be found. Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:48.851530 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:48.860640 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:48.860640 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:48.860640 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:48.860640 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:48.860640 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:48.860640 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:48.860640 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d673999a-cb2c-41e9-b1a7-dc336cc1f52e could not be found. Sep 27 10:10:48.860640 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:48.861376 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4ba7b0ed-25c8-4e63-be75-6ee56f04ba47 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.348055 Sep 27 10:10:48.861376 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1629/3578] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:48 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:48.863171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-52984a1a-485d-4084-a597-252799107b1b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance d673999a-cb2c-41e9-b1a7-dc336cc1f52e could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:48.864331 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-52984a1a-485d-4084-a597-252799107b1b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/d673999a-cb2c-41e9-b1a7-dc336cc1f52e/action" status: 404 len: 111 microversion: 2.1 time: 0.028239 Sep 27 10:10:48.865201 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1950/3579] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:10:48 2022] POST /compute/v2.1/servers/d673999a-cb2c-41e9-b1a7-dc336cc1f52e/action => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:10:48.874561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a4effa8a-d2a2-434d-b0f9-aad74ffa7985 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:48.875914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a4effa8a-d2a2-434d-b0f9-aad74ffa7985 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:48.882306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4effa8a-d2a2-434d-b0f9-aad74ffa7985 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.883141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4effa8a-d2a2-434d-b0f9-aad74ffa7985 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.883744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4effa8a-d2a2-434d-b0f9-aad74ffa7985 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.884459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4effa8a-d2a2-434d-b0f9-aad74ffa7985 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.884960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4effa8a-d2a2-434d-b0f9-aad74ffa7985 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.885523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4effa8a-d2a2-434d-b0f9-aad74ffa7985 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.896500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7ea86765-4476-402f-9d12-6713c6a4c7ad tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:48.897381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7ea86765-4476-402f-9d12-6713c6a4c7ad tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:48.912819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea86765-4476-402f-9d12-6713c6a4c7ad tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.915891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea86765-4476-402f-9d12-6713c6a4c7ad tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.917160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea86765-4476-402f-9d12-6713c6a4c7ad tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:48.919665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea86765-4476-402f-9d12-6713c6a4c7ad tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:48.920385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea86765-4476-402f-9d12-6713c6a4c7ad tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:48.921238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea86765-4476-402f-9d12-6713c6a4c7ad tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.130105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7ea86765-4476-402f-9d12-6713c6a4c7ad tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:49.141329 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7ea86765-4476-402f-9d12-6713c6a4c7ad tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1315 microversion: 2.1 time: 0.249735 Sep 27 10:10:49.141893 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1951/3580] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:48 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1315 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:49.195923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a4effa8a-d2a2-434d-b0f9-aad74ffa7985 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:49.206979 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a4effa8a-d2a2-434d-b0f9-aad74ffa7985 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.336771 Sep 27 10:10:49.207870 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1630/3581] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:48 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:49.232020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a587e8a9-6f8a-4a06-9b74-2bd7bf98af68 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:49.232881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a587e8a9-6f8a-4a06-9b74-2bd7bf98af68 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:49.240170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a587e8a9-6f8a-4a06-9b74-2bd7bf98af68 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.240977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a587e8a9-6f8a-4a06-9b74-2bd7bf98af68 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.242056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a587e8a9-6f8a-4a06-9b74-2bd7bf98af68 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.242778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a587e8a9-6f8a-4a06-9b74-2bd7bf98af68 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.243241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a587e8a9-6f8a-4a06-9b74-2bd7bf98af68 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.243667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a587e8a9-6f8a-4a06-9b74-2bd7bf98af68 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.293711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:49.294332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:49.301971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.302931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.302931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.303610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.304146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.304582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.430594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a587e8a9-6f8a-4a06-9b74-2bd7bf98af68 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:49.439966 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a587e8a9-6f8a-4a06-9b74-2bd7bf98af68 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.211856 Sep 27 10:10:49.440608 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1952/3582] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:49 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:49.463857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:49.464448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:49.470760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.471249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.471697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.472389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.472882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.473494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.493333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:49.514963 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Cannot 'unshelve' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action HTTP/1.1 Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 18 Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: {"unshelve": null} Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: exception: Cannot 'unshelve' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3327ff95-5c05-4167-a73b-5fd55f7bfabd', 'body': {'unshelve': None}} Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 124, in _unshelve Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.unshelve( Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd in vm_state active. Cannot unshelve while the instance is in this state. Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/shelve.py", line 137, in _unshelve Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state( Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:10:49.516880 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'unshelve' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'unshelve' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:49.521815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 409 to user: Cannot 'unshelve' instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd while it is in vm_state active {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:49.524098 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a2408064-6541-4b0a-8091-12d5a6e7b6d4 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action" status: 409 len: 146 microversion: 2.1 time: 0.064825 Sep 27 10:10:49.526450 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1953/3583] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:10:49 2022] POST /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd/action => generated 146 bytes in 68 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:10:49.532099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:49.532940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] [instance: 9ac32528-351d-4ba2-b0ec-0eeb4872e95c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:49.539511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.540140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.540714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.541543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.542128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.542657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.563977 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] HTTP exception thrown: Instance 9ac32528-351d-4ba2-b0ec-0eeb4872e95c could not be found. Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Request method failure captured: Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c HTTP/1.1 Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsuxLfR8HD8yzacO5X_AOkZmoonFZnPAN0WrmHTMGVGO9QRR_PNCaz-7-FVeQVrEGyjR4linxzKK8JUhgRbfM4gUPLznR20fLvHgm9snqhzInlyJTa5eEBW4UFO6PVeCBA9DR_m8AQY__ttmY2uUbWrLDDHBJQCFfpNObd905YTKQLc Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 327b414889a04e189a7f940685a4ff39 Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestFqdnHostnames-2007165513 Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/327b414889a04e189a7f940685a4ff39"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/327b414889a04e189a7f940685a4ff39"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/327b414889a04e189a7f940685a4ff39"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_327b414889a04e189a7f940685a4ff39"}]}] Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestFqdnHostnames-2007165513 Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 327b414889a04e189a7f940685a4ff39 Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestFqdnHostnames-2007165513 Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestFqdnHostnames-2007165513-project Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: f06d16c90a5d464696f8137612c35327 Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestFqdnHostnames-2007165513-project Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 9ac32528-351d-4ba2-b0ec-0eeb4872e95c could not be found. Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '9ac32528-351d-4ba2-b0ec-0eeb4872e95c'} Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9ac32528-351d-4ba2-b0ec-0eeb4872e95c could not be found. Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9ac32528-351d-4ba2-b0ec-0eeb4872e95c could not be found. Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:49.566273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:49.571672 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:49.571672 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:49.571672 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:49.571672 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:49.571672 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:49.571672 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9ac32528-351d-4ba2-b0ec-0eeb4872e95c could not be found. Sep 27 10:10:49.571672 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:49.571672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Returning 404 to user: Instance 9ac32528-351d-4ba2-b0ec-0eeb4872e95c could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:49.572452 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-57defeea-2b68-45e9-9132-7ef8c5b0d50b tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c" status: 404 len: 111 microversion: 2.1 time: 0.043486 Sep 27 10:10:49.572918 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1954/3584] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:49 2022] GET /compute/v2.1/servers/9ac32528-351d-4ba2-b0ec-0eeb4872e95c => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:10:49.581546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-18e9f382-fd11-444d-8d0a-310578b35a11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:49.582440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-18e9f382-fd11-444d-8d0a-310578b35a11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:49.595856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-18e9f382-fd11-444d-8d0a-310578b35a11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.596389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-18e9f382-fd11-444d-8d0a-310578b35a11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.596891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-18e9f382-fd11-444d-8d0a-310578b35a11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.597601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-18e9f382-fd11-444d-8d0a-310578b35a11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.598092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-18e9f382-fd11-444d-8d0a-310578b35a11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.598534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-18e9f382-fd11-444d-8d0a-310578b35a11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.653809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:49.669230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:49.671316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:49.672796 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-58bff503-8007-4e33-9f98-02a697b66652 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.383323 Sep 27 10:10:49.674003 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1631/3585] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:49 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:10:49.679349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6ee9a619-b22a-4e71-b86c-0ae8c4e4bd3a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] Calling method '.version_select at 0x7f7f62bfec10>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:49.706445 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6ee9a619-b22a-4e71-b86c-0ae8c4e4bd3a tempest-ServersTestFqdnHostnames-2007165513 tempest-ServersTestFqdnHostnames-2007165513-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-667250342" status: 202 len: 0 microversion: 2.1 time: 0.031143 Sep 27 10:10:49.707079 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1632/3586] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:10:49 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-667250342 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:10:49.880069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd04f1a6-3031-4b49-98fb-c546eaf63b7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:49.880797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd04f1a6-3031-4b49-98fb-c546eaf63b7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:49.881256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-18e9f382-fd11-444d-8d0a-310578b35a11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:49.888261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd04f1a6-3031-4b49-98fb-c546eaf63b7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.889055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd04f1a6-3031-4b49-98fb-c546eaf63b7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.889719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd04f1a6-3031-4b49-98fb-c546eaf63b7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.890653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd04f1a6-3031-4b49-98fb-c546eaf63b7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.891344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd04f1a6-3031-4b49-98fb-c546eaf63b7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.892148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd04f1a6-3031-4b49-98fb-c546eaf63b7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.894375 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-18e9f382-fd11-444d-8d0a-310578b35a11 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.319652 Sep 27 10:10:49.900057 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1955/3587] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:49 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:49.924315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-939f1394-e7eb-400e-96d2-207043064c39 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:49.925120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-939f1394-e7eb-400e-96d2-207043064c39 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:49.931561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939f1394-e7eb-400e-96d2-207043064c39 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.932139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939f1394-e7eb-400e-96d2-207043064c39 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.933091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939f1394-e7eb-400e-96d2-207043064c39 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:49.935032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939f1394-e7eb-400e-96d2-207043064c39 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:49.935786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939f1394-e7eb-400e-96d2-207043064c39 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:49.936558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-939f1394-e7eb-400e-96d2-207043064c39 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:50.115050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd04f1a6-3031-4b49-98fb-c546eaf63b7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:50.131722 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd04f1a6-3031-4b49-98fb-c546eaf63b7f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.255508 Sep 27 10:10:50.133270 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1633/3588] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:49 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:50.142923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5d19f959-207a-493d-b27e-fe61dc08ecef req-4b7aa64e-17d6-480b-af3f-7c430def4e5e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "tag": "5c514831-defe-416f-aec5-712e750a0e3b"}, {"name": "network-changed", "server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "tag": "ee0efbef-fa4b-4072-b28f-b4422ccb9afa"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:50.168046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5d19f959-207a-493d-b27e-fe61dc08ecef req-4b7aa64e-17d6-480b-af3f-7c430def4e5e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:50.168537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5d19f959-207a-493d-b27e-fe61dc08ecef req-4b7aa64e-17d6-480b-af3f-7c430def4e5e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:50.169003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5d19f959-207a-493d-b27e-fe61dc08ecef req-4b7aa64e-17d6-480b-af3f-7c430def4e5e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:50.192531 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5d19f959-207a-493d-b27e-fe61dc08ecef req-4b7aa64e-17d6-480b-af3f-7c430def4e5e service nova] Creating event network-changed:5c514831-defe-416f-aec5-712e750a0e3b for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:10:50.193707 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5d19f959-207a-493d-b27e-fe61dc08ecef req-4b7aa64e-17d6-480b-af3f-7c430def4e5e service nova] Creating event network-changed:ee0efbef-fa4b-4072-b28f-b4422ccb9afa for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:10:50.199272 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5d19f959-207a-493d-b27e-fe61dc08ecef req-4b7aa64e-17d6-480b-af3f-7c430def4e5e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 346 microversion: 2.1 time: 0.062961 Sep 27 10:10:50.200285 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1634/3589] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:50 2022] POST /compute/v2.1/os-server-external-events => generated 346 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:50.205997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d502c492-9a71-4747-b246-92fbb343500e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:50.207042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d502c492-9a71-4747-b246-92fbb343500e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:50.217621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d502c492-9a71-4747-b246-92fbb343500e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:50.218575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d502c492-9a71-4747-b246-92fbb343500e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:50.219163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d502c492-9a71-4747-b246-92fbb343500e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:50.220388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d502c492-9a71-4747-b246-92fbb343500e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:50.220888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d502c492-9a71-4747-b246-92fbb343500e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:50.221326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d502c492-9a71-4747-b246-92fbb343500e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:50.256488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-939f1394-e7eb-400e-96d2-207043064c39 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:50.273343 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-939f1394-e7eb-400e-96d2-207043064c39 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.353611 Sep 27 10:10:50.274918 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1956/3590] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:49 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:50.300965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e346806f-2d7d-4329-8bb0-70184cfd7a2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'update', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1851409494_updated"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:50.304153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e346806f-2d7d-4329-8bb0-70184cfd7a2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 82601883-1e83-4b6b-96db-9799d025aca2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:50.313573 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-e346806f-2d7d-4329-8bb0-70184cfd7a2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 82601883-1e83-4b6b-96db-9799d025aca2 could not be found. Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e346806f-2d7d-4329-8bb0-70184cfd7a2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: request: PUT /compute/v2.1/servers/82601883-1e83-4b6b-96db-9799d025aca2 HTTP/1.1 Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 81 Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1851409494_updated"}} Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 82601883-1e83-4b6b-96db-9799d025aca2 could not be found. Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '82601883-1e83-4b6b-96db-9799d025aca2', 'body': {'server': {'name': 'tempest-ServersNegativeTestJSON-server-1851409494_updated'}}} Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82601883-1e83-4b6b-96db-9799d025aca2 could not be found. Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 915, in update Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(ctxt, req, id, is_detail=True) Sep 27 10:10:50.315495 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82601883-1e83-4b6b-96db-9799d025aca2 could not be found. Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82601883-1e83-4b6b-96db-9799d025aca2 could not be found. Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e346806f-2d7d-4329-8bb0-70184cfd7a2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 82601883-1e83-4b6b-96db-9799d025aca2 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:50.320987 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e346806f-2d7d-4329-8bb0-70184cfd7a2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "PUT /compute/v2.1/servers/82601883-1e83-4b6b-96db-9799d025aca2" status: 404 len: 111 microversion: 2.1 time: 0.025881 Sep 27 10:10:50.322762 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1957/3591] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:10:50 2022] PUT /compute/v2.1/servers/82601883-1e83-4b6b-96db-9799d025aca2 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:10:50.331275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6a16f540-c9e4-456a-803e-7ba125c142ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:50.331896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6a16f540-c9e4-456a-803e-7ba125c142ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:50.339586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a16f540-c9e4-456a-803e-7ba125c142ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:50.340229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a16f540-c9e4-456a-803e-7ba125c142ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:50.340691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a16f540-c9e4-456a-803e-7ba125c142ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:50.341420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a16f540-c9e4-456a-803e-7ba125c142ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:50.341877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a16f540-c9e4-456a-803e-7ba125c142ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:50.342304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a16f540-c9e4-456a-803e-7ba125c142ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:50.591450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6a16f540-c9e4-456a-803e-7ba125c142ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:50.604781 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6a16f540-c9e4-456a-803e-7ba125c142ad tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.277363 Sep 27 10:10:50.608912 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1958/3592] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:50 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:50.636717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3ac3e775-a61b-4ff7-9872-15d24d5deb2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:50.637354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3ac3e775-a61b-4ff7-9872-15d24d5deb2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:50.646003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ac3e775-a61b-4ff7-9872-15d24d5deb2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:50.646505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ac3e775-a61b-4ff7-9872-15d24d5deb2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:50.646947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ac3e775-a61b-4ff7-9872-15d24d5deb2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:50.658799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ac3e775-a61b-4ff7-9872-15d24d5deb2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:50.660894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ac3e775-a61b-4ff7-9872-15d24d5deb2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:50.664534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ac3e775-a61b-4ff7-9872-15d24d5deb2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:50.747276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d502c492-9a71-4747-b246-92fbb343500e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:50.766118 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d502c492-9a71-4747-b246-92fbb343500e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1315 microversion: 2.1 time: 0.554683 Sep 27 10:10:50.766792 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1635/3593] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:50 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1315 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:50.778685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:50.779310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:50.792929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:50.793368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:50.793784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:50.794497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:50.794957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:50.795381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:51.014338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3ac3e775-a61b-4ff7-9872-15d24d5deb2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:51.022628 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3ac3e775-a61b-4ff7-9872-15d24d5deb2a tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.390543 Sep 27 10:10:51.023481 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1959/3594] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:50 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:51.033522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-784885f7-9415-41cc-951e-9cdd4208d401 req-0ec41698-d931-4387-8a12-f993c55f3865 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "668efb48-676d-4a61-837f-eba6c5365546", "name": "network-vif-plugged", "status": "completed", "tag": "48a8d3b8-b284-4793-8bb8-001085caddfc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:51.053173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-784885f7-9415-41cc-951e-9cdd4208d401 req-0ec41698-d931-4387-8a12-f993c55f3865 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:51.054061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-784885f7-9415-41cc-951e-9cdd4208d401 req-0ec41698-d931-4387-8a12-f993c55f3865 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:51.054703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-784885f7-9415-41cc-951e-9cdd4208d401 req-0ec41698-d931-4387-8a12-f993c55f3865 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:51.068844 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-784885f7-9415-41cc-951e-9cdd4208d401 req-0ec41698-d931-4387-8a12-f993c55f3865 service nova] Creating event network-vif-plugged:48a8d3b8-b284-4793-8bb8-001085caddfc for instance 668efb48-676d-4a61-837f-eba6c5365546 on nch1-master-1 Sep 27 10:10:51.074821 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-784885f7-9415-41cc-951e-9cdd4208d401 req-0ec41698-d931-4387-8a12-f993c55f3865 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048008 Sep 27 10:10:51.080648 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1960/3595] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:51.087151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7594e253-868f-4078-af86-2a2fc48ac9ef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'update', calling method: >, body: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7594e253-868f-4078-af86-2a2fc48ac9ef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: request: PUT /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd HTTP/1.1 Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 280 Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"}} Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3327ff95-5c05-4167-a73b-5fd55f7bfabd', 'body': {'server': {'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'}}} Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'maxLength' in schema['properties']['server']['properties']['name']: Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.091613 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long Sep 27 10:10:51.095111 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.099345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7594e253-868f-4078-af86-2a2fc48ac9ef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Invalid input for field/attribute name. Value: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa. 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:51.100855 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7594e253-868f-4078-af86-2a2fc48ac9ef tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "PUT /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 400 len: 619 microversion: 2.1 time: 0.018394 Sep 27 10:10:51.101790 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1961/3596] 10.222.0.26 () {60 vars in 1257 bytes} [Tue Sep 27 10:10:51 2022] PUT /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 619 bytes in 20 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:10:51.112604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5628d77e-71ab-4e76-adf8-65defb6ebaf7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:51.113643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5628d77e-71ab-4e76-adf8-65defb6ebaf7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:51.123087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5628d77e-71ab-4e76-adf8-65defb6ebaf7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:51.123954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5628d77e-71ab-4e76-adf8-65defb6ebaf7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:51.124689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5628d77e-71ab-4e76-adf8-65defb6ebaf7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:51.125673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5628d77e-71ab-4e76-adf8-65defb6ebaf7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:51.126598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5628d77e-71ab-4e76-adf8-65defb6ebaf7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:51.127821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5628d77e-71ab-4e76-adf8-65defb6ebaf7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:51.349702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:51.361415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:51.363121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:51.364516 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-52f15cdb-4dac-45ca-91e5-c9db6eeb0984 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.596773 Sep 27 10:10:51.365347 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1636/3597] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:50 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 598 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:51.370462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ef19ef13-cea2-4ece-b5e2-62e77272a04c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:51.371353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ef19ef13-cea2-4ece-b5e2-62e77272a04c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:51.377671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef19ef13-cea2-4ece-b5e2-62e77272a04c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:51.378428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef19ef13-cea2-4ece-b5e2-62e77272a04c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:51.379110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef19ef13-cea2-4ece-b5e2-62e77272a04c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:51.380041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef19ef13-cea2-4ece-b5e2-62e77272a04c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:51.380715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef19ef13-cea2-4ece-b5e2-62e77272a04c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:51.381322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef19ef13-cea2-4ece-b5e2-62e77272a04c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:51.389178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5628d77e-71ab-4e76-adf8-65defb6ebaf7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:51.399146 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5628d77e-71ab-4e76-adf8-65defb6ebaf7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.292346 Sep 27 10:10:51.403648 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1962/3598] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:51 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:51.428870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fda78507-e35f-4a85-8d39-1b5de1befb17 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:51.429789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fda78507-e35f-4a85-8d39-1b5de1befb17 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:51.436766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fda78507-e35f-4a85-8d39-1b5de1befb17 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:51.437468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fda78507-e35f-4a85-8d39-1b5de1befb17 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:51.437909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fda78507-e35f-4a85-8d39-1b5de1befb17 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:51.439408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fda78507-e35f-4a85-8d39-1b5de1befb17 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:51.440122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fda78507-e35f-4a85-8d39-1b5de1befb17 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:51.440774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fda78507-e35f-4a85-8d39-1b5de1befb17 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:51.669627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ef19ef13-cea2-4ece-b5e2-62e77272a04c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:51.699082 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ef19ef13-cea2-4ece-b5e2-62e77272a04c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.332381 Sep 27 10:10:51.715959 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1637/3599] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:51 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:51.716723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fda78507-e35f-4a85-8d39-1b5de1befb17 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:51.725314 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fda78507-e35f-4a85-8d39-1b5de1befb17 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.301160 Sep 27 10:10:51.726840 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1963/3600] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:51 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:51.749388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e37d7fb0-12f3-4f97-b6fb-55bb99ac3586 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Action: 'update', calling method: >, body: {"server": {"name": ""}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e37d7fb0-12f3-4f97-b6fb-55bb99ac3586 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: request: PUT /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd HTTP/1.1 Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 24 Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": ""}} Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute name. Value: . '' is too short Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3327ff95-5c05-4167-a73b-5fd55f7bfabd', 'body': {'server': {'name': ''}}} Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['server']['properties']['name']: Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'format': 'name', 'maxLength': 255, 'minLength': 1, 'type': 'string'} Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi '' Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:10:51.755607 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:10:51.761349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute name. Value: . '' is too short Sep 27 10:10:51.761349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.761349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:51.761349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.761349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:51.761349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:51.761349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:51.761349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:10:51.761349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:10:51.761349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute name. Value: . '' is too short Sep 27 10:10:51.761349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:51.763962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e37d7fb0-12f3-4f97-b6fb-55bb99ac3586 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 400 to user: Invalid input for field/attribute name. Value: . '' is too short {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:51.765258 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e37d7fb0-12f3-4f97-b6fb-55bb99ac3586 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "PUT /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 400 len: 108 microversion: 2.1 time: 0.019962 Sep 27 10:10:51.770402 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1638/3601] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:10:51 2022] PUT /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 108 bytes in 26 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:10:51.775907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-26f1a8cd-ad73-4e4c-9a5c-73fc3cab466b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:51.776588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-26f1a8cd-ad73-4e4c-9a5c-73fc3cab466b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:51.784473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26f1a8cd-ad73-4e4c-9a5c-73fc3cab466b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:51.785853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26f1a8cd-ad73-4e4c-9a5c-73fc3cab466b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:51.786639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26f1a8cd-ad73-4e4c-9a5c-73fc3cab466b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:51.788478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26f1a8cd-ad73-4e4c-9a5c-73fc3cab466b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:51.790015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26f1a8cd-ad73-4e4c-9a5c-73fc3cab466b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:51.791829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4684b8f7-8daf-4a2d-ab17-dcc820b06418 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:51.792229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26f1a8cd-ad73-4e4c-9a5c-73fc3cab466b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:51.792765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4684b8f7-8daf-4a2d-ab17-dcc820b06418 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:51.806899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4684b8f7-8daf-4a2d-ab17-dcc820b06418 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:51.812705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4684b8f7-8daf-4a2d-ab17-dcc820b06418 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:51.813134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4684b8f7-8daf-4a2d-ab17-dcc820b06418 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:51.813889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4684b8f7-8daf-4a2d-ab17-dcc820b06418 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:51.814360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4684b8f7-8daf-4a2d-ab17-dcc820b06418 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:51.814785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4684b8f7-8daf-4a2d-ab17-dcc820b06418 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:52.069518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-26f1a8cd-ad73-4e4c-9a5c-73fc3cab466b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:52.082007 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-26f1a8cd-ad73-4e4c-9a5c-73fc3cab466b tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1505 microversion: 2.1 time: 0.310540 Sep 27 10:10:52.082948 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1964/3602] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:51 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1505 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:52.227988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-df7d5a52-e2d9-4e0a-8db7-472161411783 req-f09032ea-fd9d-4177-8e12-c239c5b7ba72 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "tag": "2b32bc4b-33c7-4c6b-918d-8458f3eee387"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:52.228490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4684b8f7-8daf-4a2d-ab17-dcc820b06418 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:52.239524 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4684b8f7-8daf-4a2d-ab17-dcc820b06418 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1315 microversion: 2.1 time: 0.452873 Sep 27 10:10:52.240116 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1639/3603] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:51 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1315 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:52.243366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-df7d5a52-e2d9-4e0a-8db7-472161411783 req-f09032ea-fd9d-4177-8e12-c239c5b7ba72 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:52.244123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-df7d5a52-e2d9-4e0a-8db7-472161411783 req-f09032ea-fd9d-4177-8e12-c239c5b7ba72 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:52.244796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-df7d5a52-e2d9-4e0a-8db7-472161411783 req-f09032ea-fd9d-4177-8e12-c239c5b7ba72 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:52.284529 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-df7d5a52-e2d9-4e0a-8db7-472161411783 req-f09032ea-fd9d-4177-8e12-c239c5b7ba72 service nova] Creating event network-changed:2b32bc4b-33c7-4c6b-918d-8458f3eee387 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:10:52.293970 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-df7d5a52-e2d9-4e0a-8db7-472161411783 req-f09032ea-fd9d-4177-8e12-c239c5b7ba72 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070069 Sep 27 10:10:52.295200 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1965/3604] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:52.402955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:52.404258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:52.413102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:52.414796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:52.418365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:52.419832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:52.420610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:52.421798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:52.723661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cf8b2a79-2d18-4fbc-b26b-b4fd58530a50 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:52.724533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cf8b2a79-2d18-4fbc-b26b-b4fd58530a50 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:52.733734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8b2a79-2d18-4fbc-b26b-b4fd58530a50 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:52.734690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8b2a79-2d18-4fbc-b26b-b4fd58530a50 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:52.735470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8b2a79-2d18-4fbc-b26b-b4fd58530a50 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:52.737441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8b2a79-2d18-4fbc-b26b-b4fd58530a50 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:52.738112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8b2a79-2d18-4fbc-b26b-b4fd58530a50 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:52.738914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8b2a79-2d18-4fbc-b26b-b4fd58530a50 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:53.046876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:53.062308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:53.064214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:53.065805 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f7a4ad46-968d-49b4-af45-a8f8f2319758 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.673859 Sep 27 10:10:53.067117 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1640/3605] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:52 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:53.094702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cf8b2a79-2d18-4fbc-b26b-b4fd58530a50 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:53.105166 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cf8b2a79-2d18-4fbc-b26b-b4fd58530a50 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.386124 Sep 27 10:10:53.107916 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1966/3606] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:52 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:53.263790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9a1fec7f-0e57-4f64-b881-075079416e54 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:53.265898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9a1fec7f-0e57-4f64-b881-075079416e54 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:53.275080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a1fec7f-0e57-4f64-b881-075079416e54 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:53.275838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a1fec7f-0e57-4f64-b881-075079416e54 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:53.276585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a1fec7f-0e57-4f64-b881-075079416e54 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:53.277513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a1fec7f-0e57-4f64-b881-075079416e54 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:53.278127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a1fec7f-0e57-4f64-b881-075079416e54 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:53.278707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a1fec7f-0e57-4f64-b881-075079416e54 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:53.797874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9a1fec7f-0e57-4f64-b881-075079416e54 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:53.827271 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9a1fec7f-0e57-4f64-b881-075079416e54 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1502 microversion: 2.1 time: 0.567243 Sep 27 10:10:53.831476 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1641/3607] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:53 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1502 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:53.857482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-667b9197-be8a-4661-8260-b75304e77a41 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:53.858893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-667b9197-be8a-4661-8260-b75304e77a41 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:53.868446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-667b9197-be8a-4661-8260-b75304e77a41 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:53.869365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-667b9197-be8a-4661-8260-b75304e77a41 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:53.870063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-667b9197-be8a-4661-8260-b75304e77a41 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:53.871115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-667b9197-be8a-4661-8260-b75304e77a41 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:53.871800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-667b9197-be8a-4661-8260-b75304e77a41 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:53.872460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-667b9197-be8a-4661-8260-b75304e77a41 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:53.907277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-667b9197-be8a-4661-8260-b75304e77a41 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lazy-loading 'flavor' on Instance uuid 668efb48-676d-4a61-837f-eba6c5365546 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:10:54.042076 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-667b9197-be8a-4661-8260-b75304e77a41 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "POST /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546/action" status: 202 len: 0 microversion: 2.1 time: 0.189535 Sep 27 10:10:54.043318 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1967/3608] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:10:53 2022] POST /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546/action => generated 0 bytes in 192 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:10:54.051250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1678dcba-dba0-4608-b669-643a7aa2a210 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:54.057811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1678dcba-dba0-4608-b669-643a7aa2a210 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:54.072267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1678dcba-dba0-4608-b669-643a7aa2a210 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.072784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1678dcba-dba0-4608-b669-643a7aa2a210 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.073229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1678dcba-dba0-4608-b669-643a7aa2a210 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.073946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1678dcba-dba0-4608-b669-643a7aa2a210 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.077677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1678dcba-dba0-4608-b669-643a7aa2a210 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.081263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1678dcba-dba0-4608-b669-643a7aa2a210 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.102899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:54.104043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:54.117915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.118431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.119376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.120472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.121377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.121961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.554277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:54.569587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:54.571342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:54.572828 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-00ecb2a5-102a-4a6a-8551-245e02bb08de tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.480234 Sep 27 10:10:54.574188 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1968/3609] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:54 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:10:54.580830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f26f94d2-08e1-4f6d-bf48-1897166b93a5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:54.581699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f26f94d2-08e1-4f6d-bf48-1897166b93a5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:54.590570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f26f94d2-08e1-4f6d-bf48-1897166b93a5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.591473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f26f94d2-08e1-4f6d-bf48-1897166b93a5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.592095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f26f94d2-08e1-4f6d-bf48-1897166b93a5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.593067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f26f94d2-08e1-4f6d-bf48-1897166b93a5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.593932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f26f94d2-08e1-4f6d-bf48-1897166b93a5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.595337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f26f94d2-08e1-4f6d-bf48-1897166b93a5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.677823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1678dcba-dba0-4608-b669-643a7aa2a210 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:54.691719 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1678dcba-dba0-4608-b669-643a7aa2a210 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1510 microversion: 2.1 time: 0.644283 Sep 27 10:10:54.707233 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1642/3610] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:54 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1510 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:54.707233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:54.707233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:54.710527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.711100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.711605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.712532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.713188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.714506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.735622 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 HTTP/1.1 Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5'} Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:10:54.737846 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1a6261dd-ac13-4ac3-91f5-c519427ffc41 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "DELETE /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 404 len: 111 microversion: 2.1 time: 0.044830 Sep 27 10:10:54.742379 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1643/3611] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:10:54 2022] DELETE /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:10:54.745793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-eb3c9c8f-b725-48ba-8632-38492df2d2a4 req-faa5bd57-e47d-400d-88ed-b2e58536aa7f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "tag": "bcc2c185-ff79-4bda-a3f2-d915e7d3752c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:54.761185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-eb3c9c8f-b725-48ba-8632-38492df2d2a4 req-faa5bd57-e47d-400d-88ed-b2e58536aa7f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.761765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-eb3c9c8f-b725-48ba-8632-38492df2d2a4 req-faa5bd57-e47d-400d-88ed-b2e58536aa7f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.762208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-eb3c9c8f-b725-48ba-8632-38492df2d2a4 req-faa5bd57-e47d-400d-88ed-b2e58536aa7f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.777748 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-eb3c9c8f-b725-48ba-8632-38492df2d2a4 req-faa5bd57-e47d-400d-88ed-b2e58536aa7f service nova] Creating event network-changed:bcc2c185-ff79-4bda-a3f2-d915e7d3752c for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:10:54.784088 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-eb3c9c8f-b725-48ba-8632-38492df2d2a4 req-faa5bd57-e47d-400d-88ed-b2e58536aa7f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042750 Sep 27 10:10:54.784861 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1644/3612] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:54.791989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:54.792836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:54.812841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.814201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.815013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.815987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.816610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.817499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.855096 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 HTTP/1.1 Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5'} Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:10:54.857351 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:54.864300 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:10:54.864300 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:10:54.864300 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:10:54.864300 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:10:54.864300 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:10:54.864300 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. Sep 27 10:10:54.864300 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:10:54.867021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:10:54.868476 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4fd80d69-863b-47e6-a5aa-75f4ddb14d31 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5" status: 404 len: 111 microversion: 2.1 time: 0.082182 Sep 27 10:10:54.869416 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1645/3613] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:54 2022] GET /compute/v2.1/servers/8a6e7ef6-14e2-4958-a5db-3e7f74daf6a5 => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:10:54.886385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0ce2b0fd-2569-4522-a64d-fe724ff754f9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:54.887988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0ce2b0fd-2569-4522-a64d-fe724ff754f9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:54.904449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ce2b0fd-2569-4522-a64d-fe724ff754f9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.906293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ce2b0fd-2569-4522-a64d-fe724ff754f9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.908222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ce2b0fd-2569-4522-a64d-fe724ff754f9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.914711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ce2b0fd-2569-4522-a64d-fe724ff754f9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:54.915823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ce2b0fd-2569-4522-a64d-fe724ff754f9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:54.916694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ce2b0fd-2569-4522-a64d-fe724ff754f9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:54.957496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0ce2b0fd-2569-4522-a64d-fe724ff754f9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:10:55.093191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:55.161795 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0ce2b0fd-2569-4522-a64d-fe724ff754f9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 204 len: 0 microversion: 2.1 time: 0.286926 Sep 27 10:10:55.163404 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1646/3614] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:10:54 2022] DELETE /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 0 bytes in 290 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:10:55.182209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-44e29a26-fffa-4c6e-a598-9a144ab04260 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:55.185925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-44e29a26-fffa-4c6e-a598-9a144ab04260 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:55.201481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44e29a26-fffa-4c6e-a598-9a144ab04260 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:55.202300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44e29a26-fffa-4c6e-a598-9a144ab04260 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:55.202902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44e29a26-fffa-4c6e-a598-9a144ab04260 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:55.203836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44e29a26-fffa-4c6e-a598-9a144ab04260 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:55.204498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44e29a26-fffa-4c6e-a598-9a144ab04260 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:55.205237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44e29a26-fffa-4c6e-a598-9a144ab04260 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:55.383006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f26f94d2-08e1-4f6d-bf48-1897166b93a5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:55.392297 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f26f94d2-08e1-4f6d-bf48-1897166b93a5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.816625 Sep 27 10:10:55.393350 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1969/3615] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:54 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 818 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:55.434597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-44e29a26-fffa-4c6e-a598-9a144ab04260 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:55.449296 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-44e29a26-fffa-4c6e-a598-9a144ab04260 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.280504 Sep 27 10:10:55.451279 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1647/3616] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:55 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:55.611773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:55.613164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:55.622440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:55.623036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:55.623598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:55.624395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:55.624957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:55.626188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:55.715990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7a3b6720-6345-43b5-8054-72ab288942d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:55.716947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7a3b6720-6345-43b5-8054-72ab288942d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:55.723607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3b6720-6345-43b5-8054-72ab288942d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:55.724124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3b6720-6345-43b5-8054-72ab288942d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:55.724561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3b6720-6345-43b5-8054-72ab288942d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:55.725299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3b6720-6345-43b5-8054-72ab288942d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:55.725813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3b6720-6345-43b5-8054-72ab288942d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:55.726235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3b6720-6345-43b5-8054-72ab288942d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:55.913147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:55.923834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a3b6720-6345-43b5-8054-72ab288942d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:55.936311 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7a3b6720-6345-43b5-8054-72ab288942d8 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1493 microversion: 2.1 time: 0.224544 Sep 27 10:10:55.938762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:55.942622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:55.945855 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-06d3f4e7-1fe1-46fe-ac9b-486eab969c38 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.348171 Sep 27 10:10:55.960947 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1648/3617] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:55 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1493 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:55.962222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba54449b-b3a7-484a-9a41-c77972d8e4f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:55.963562 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1970/3618] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:55 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:55.963934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ba54449b-b3a7-484a-9a41-c77972d8e4f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:55.970877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba54449b-b3a7-484a-9a41-c77972d8e4f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:55.971764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba54449b-b3a7-484a-9a41-c77972d8e4f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:55.972543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba54449b-b3a7-484a-9a41-c77972d8e4f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:55.973545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba54449b-b3a7-484a-9a41-c77972d8e4f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:55.974062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba54449b-b3a7-484a-9a41-c77972d8e4f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:55.974704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba54449b-b3a7-484a-9a41-c77972d8e4f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:56.013799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ba54449b-b3a7-484a-9a41-c77972d8e4f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:10:56.144131 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ba54449b-b3a7-484a-9a41-c77972d8e4f1 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 204 len: 0 microversion: 2.1 time: 0.186181 Sep 27 10:10:56.144829 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1649/3619] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:10:55 2022] DELETE /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:10:56.154944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-52ac78a7-4133-4b73-a2c7-a3d77be36ecb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:56.155661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-52ac78a7-4133-4b73-a2c7-a3d77be36ecb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:56.164797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52ac78a7-4133-4b73-a2c7-a3d77be36ecb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:56.165541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52ac78a7-4133-4b73-a2c7-a3d77be36ecb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:56.166140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52ac78a7-4133-4b73-a2c7-a3d77be36ecb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:56.167474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52ac78a7-4133-4b73-a2c7-a3d77be36ecb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:56.168737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52ac78a7-4133-4b73-a2c7-a3d77be36ecb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:56.169393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52ac78a7-4133-4b73-a2c7-a3d77be36ecb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:56.417789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a5df3dd4-1a36-49b4-be5d-36b925ff3793 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:56.418842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a5df3dd4-1a36-49b4-be5d-36b925ff3793 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:56.425690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5df3dd4-1a36-49b4-be5d-36b925ff3793 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:56.426270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5df3dd4-1a36-49b4-be5d-36b925ff3793 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:56.426770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5df3dd4-1a36-49b4-be5d-36b925ff3793 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:56.427567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5df3dd4-1a36-49b4-be5d-36b925ff3793 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:56.428108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5df3dd4-1a36-49b4-be5d-36b925ff3793 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:56.428862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5df3dd4-1a36-49b4-be5d-36b925ff3793 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:56.457337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-52ac78a7-4133-4b73-a2c7-a3d77be36ecb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:56.468431 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-52ac78a7-4133-4b73-a2c7-a3d77be36ecb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1499 microversion: 2.1 time: 0.319048 Sep 27 10:10:56.470508 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1971/3620] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:56 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1499 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:56.475332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2cf24eb3-9262-408d-8637-08ad37f69508 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:56.475936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2cf24eb3-9262-408d-8637-08ad37f69508 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:56.492575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:56.498741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cf24eb3-9262-408d-8637-08ad37f69508 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:56.499396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cf24eb3-9262-408d-8637-08ad37f69508 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:56.499703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cf24eb3-9262-408d-8637-08ad37f69508 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:56.501375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cf24eb3-9262-408d-8637-08ad37f69508 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:56.503285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cf24eb3-9262-408d-8637-08ad37f69508 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:56.504375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cf24eb3-9262-408d-8637-08ad37f69508 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:56.596502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a5df3dd4-1a36-49b4-be5d-36b925ff3793 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:56.610889 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a5df3dd4-1a36-49b4-be5d-36b925ff3793 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.197424 Sep 27 10:10:56.612401 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1650/3621] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:56 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:56.814316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2cf24eb3-9262-408d-8637-08ad37f69508 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:56.826904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-41749fbd-3746-4589-93c5-092dc741574a req-4a14b82c-1036-46bf-9289-53be33d9f017 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "tag": "73444bba-797d-4fc6-88a3-e126deb3c605"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:56.831191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:56.855174 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2cf24eb3-9262-408d-8637-08ad37f69508 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.383359 Sep 27 10:10:56.859318 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1972/3622] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:56 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:56.876699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-41749fbd-3746-4589-93c5-092dc741574a req-4a14b82c-1036-46bf-9289-53be33d9f017 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:56.881235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-41749fbd-3746-4589-93c5-092dc741574a req-4a14b82c-1036-46bf-9289-53be33d9f017 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:56.881729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-41749fbd-3746-4589-93c5-092dc741574a req-4a14b82c-1036-46bf-9289-53be33d9f017 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:56.898476 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-41749fbd-3746-4589-93c5-092dc741574a req-4a14b82c-1036-46bf-9289-53be33d9f017 service nova] Creating event network-changed:73444bba-797d-4fc6-88a3-e126deb3c605 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:10:56.905691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-41749fbd-3746-4589-93c5-092dc741574a req-4a14b82c-1036-46bf-9289-53be33d9f017 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.088280 Sep 27 10:10:56.907138 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1651/3623] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:56.977214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:56.977986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:56.985686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:56.986496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:56.987191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:56.988129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:56.988880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:56.989797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:57.245989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:57.259143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:57.260645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:57.261853 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-55df659b-17f2-4b58-9be9-42ef2719f2d7 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.289952 Sep 27 10:10:57.262420 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1973/3624] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:56 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:57.495236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8bc01573-6f7d-4e88-ba86-45df6b1f0b58 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:57.496962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8bc01573-6f7d-4e88-ba86-45df6b1f0b58 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:57.500332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=97,nova_api:DELETE=1,nova_api:UPDATE=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:10:57.506127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bc01573-6f7d-4e88-ba86-45df6b1f0b58 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:57.506613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bc01573-6f7d-4e88-ba86-45df6b1f0b58 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:57.507098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bc01573-6f7d-4e88-ba86-45df6b1f0b58 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:57.508118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bc01573-6f7d-4e88-ba86-45df6b1f0b58 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:57.508581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bc01573-6f7d-4e88-ba86-45df6b1f0b58 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:57.509047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bc01573-6f7d-4e88-ba86-45df6b1f0b58 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:57.636142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e112f50-63a0-4131-a5f2-654419b6bb8c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:57.636781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7e112f50-63a0-4131-a5f2-654419b6bb8c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:57.643631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e112f50-63a0-4131-a5f2-654419b6bb8c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:57.644350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e112f50-63a0-4131-a5f2-654419b6bb8c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:57.644893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e112f50-63a0-4131-a5f2-654419b6bb8c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:57.645622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e112f50-63a0-4131-a5f2-654419b6bb8c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:57.646590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e112f50-63a0-4131-a5f2-654419b6bb8c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:57.647320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e112f50-63a0-4131-a5f2-654419b6bb8c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:57.814332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7e112f50-63a0-4131-a5f2-654419b6bb8c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:57.825572 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7e112f50-63a0-4131-a5f2-654419b6bb8c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.194644 Sep 27 10:10:57.826370 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1974/3625] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:57 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:57.887993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a7b86208-1843-49d4-9ad6-28ca90bc01b9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:57.889879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a7b86208-1843-49d4-9ad6-28ca90bc01b9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:57.904164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7b86208-1843-49d4-9ad6-28ca90bc01b9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:57.905037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7b86208-1843-49d4-9ad6-28ca90bc01b9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:57.905703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7b86208-1843-49d4-9ad6-28ca90bc01b9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:57.906532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7b86208-1843-49d4-9ad6-28ca90bc01b9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:57.907091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7b86208-1843-49d4-9ad6-28ca90bc01b9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:57.907740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7b86208-1843-49d4-9ad6-28ca90bc01b9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:58.077157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8bc01573-6f7d-4e88-ba86-45df6b1f0b58 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:58.095107 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8bc01573-6f7d-4e88-ba86-45df6b1f0b58 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1499 microversion: 2.1 time: 0.605516 Sep 27 10:10:58.112949 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1652/3626] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:57 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1499 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:58.160275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a7b86208-1843-49d4-9ad6-28ca90bc01b9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:58.171980 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a7b86208-1843-49d4-9ad6-28ca90bc01b9 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1511 microversion: 2.1 time: 0.290410 Sep 27 10:10:58.173194 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1975/3627] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:57 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1511 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:58.289744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:58.290598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:58.302745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:58.304208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:58.305594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:58.306492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:58.307567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:58.308408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:58.853794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dbe56334-f587-406d-ae67-b1698ae80821 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:58.854759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dbe56334-f587-406d-ae67-b1698ae80821 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:58.865714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbe56334-f587-406d-ae67-b1698ae80821 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:58.866523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbe56334-f587-406d-ae67-b1698ae80821 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:58.867240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbe56334-f587-406d-ae67-b1698ae80821 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:58.868203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbe56334-f587-406d-ae67-b1698ae80821 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:58.868977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbe56334-f587-406d-ae67-b1698ae80821 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:58.870236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbe56334-f587-406d-ae67-b1698ae80821 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:59.083690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dbe56334-f587-406d-ae67-b1698ae80821 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:59.092608 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dbe56334-f587-406d-ae67-b1698ae80821 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.246206 Sep 27 10:10:59.105013 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1976/3628] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:58 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:10:59.106077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:59.123739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-2ed341ee-8c3f-489d-b9f7-ab81c4ea814d req-a5ce600c-2a72-4118-add3-7e41e8235835 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "tag": "dbf7486a-85d8-4b37-a8d2-b1f467142580"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:10:59.125779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:59.127251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:59.128872 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-034c9314-e2fc-4f46-8341-3d93487a8234 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.844331 Sep 27 10:10:59.145528 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1653/3629] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:10:58 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 862 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:10:59.147657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2ed341ee-8c3f-489d-b9f7-ab81c4ea814d req-a5ce600c-2a72-4118-add3-7e41e8235835 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:59.148345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2ed341ee-8c3f-489d-b9f7-ab81c4ea814d req-a5ce600c-2a72-4118-add3-7e41e8235835 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:59.153087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-700c14b4-9aa6-4439-8bd6-ed4225ae1544 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:59.153779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2ed341ee-8c3f-489d-b9f7-ab81c4ea814d req-a5ce600c-2a72-4118-add3-7e41e8235835 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:59.157124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-700c14b4-9aa6-4439-8bd6-ed4225ae1544 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:59.165211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-700c14b4-9aa6-4439-8bd6-ed4225ae1544 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:59.165712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-700c14b4-9aa6-4439-8bd6-ed4225ae1544 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:59.166457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-700c14b4-9aa6-4439-8bd6-ed4225ae1544 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:59.167503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-700c14b4-9aa6-4439-8bd6-ed4225ae1544 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:59.168129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-700c14b4-9aa6-4439-8bd6-ed4225ae1544 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:59.168882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-700c14b4-9aa6-4439-8bd6-ed4225ae1544 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:59.171925 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-2ed341ee-8c3f-489d-b9f7-ab81c4ea814d req-a5ce600c-2a72-4118-add3-7e41e8235835 service nova] Creating event network-changed:dbf7486a-85d8-4b37-a8d2-b1f467142580 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:10:59.178135 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-2ed341ee-8c3f-489d-b9f7-ab81c4ea814d req-a5ce600c-2a72-4118-add3-7e41e8235835 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083416 Sep 27 10:10:59.179387 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1977/3630] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:10:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:10:59.202272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4f51b490-9eca-4d25-9801-86f8b9e003a7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:10:59.204520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4f51b490-9eca-4d25-9801-86f8b9e003a7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:10:59.218720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f51b490-9eca-4d25-9801-86f8b9e003a7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:59.226285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f51b490-9eca-4d25-9801-86f8b9e003a7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:59.227009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f51b490-9eca-4d25-9801-86f8b9e003a7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:59.229902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f51b490-9eca-4d25-9801-86f8b9e003a7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:10:59.230463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f51b490-9eca-4d25-9801-86f8b9e003a7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:10:59.231198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f51b490-9eca-4d25-9801-86f8b9e003a7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:10:59.365896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4f51b490-9eca-4d25-9801-86f8b9e003a7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '92b0186cd92744dabd001b86560b5c90', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027612746ecc44f4ab8e7af121ec27a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:59.379374 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4f51b490-9eca-4d25-9801-86f8b9e003a7 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 200 len: 1469 microversion: 2.1 time: 0.181932 Sep 27 10:10:59.380945 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1978/3631] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:59 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 1469 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:10:59.713477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-700c14b4-9aa6-4439-8bd6-ed4225ae1544 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:10:59.724308 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-700c14b4-9aa6-4439-8bd6-ed4225ae1544 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1499 microversion: 2.1 time: 0.576660 Sep 27 10:10:59.725854 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1654/3632] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:10:59 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1499 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:00.119619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-91cb4eaf-e92e-403a-87d5-2b7f88e603fd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:00.120903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-91cb4eaf-e92e-403a-87d5-2b7f88e603fd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:00.128511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91cb4eaf-e92e-403a-87d5-2b7f88e603fd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:00.129252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91cb4eaf-e92e-403a-87d5-2b7f88e603fd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:00.129894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91cb4eaf-e92e-403a-87d5-2b7f88e603fd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:00.130783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91cb4eaf-e92e-403a-87d5-2b7f88e603fd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:00.131392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91cb4eaf-e92e-403a-87d5-2b7f88e603fd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:00.132026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-91cb4eaf-e92e-403a-87d5-2b7f88e603fd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:00.159022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:00.159631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:00.169667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:00.170401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:00.179286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:00.180593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:00.181443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:00.182026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:00.331287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-91cb4eaf-e92e-403a-87d5-2b7f88e603fd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:00.340177 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-91cb4eaf-e92e-403a-87d5-2b7f88e603fd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.225369 Sep 27 10:11:00.341045 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1979/3633] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:00 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:00.407306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:00.407970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] [instance: 3327ff95-5c05-4167-a73b-5fd55f7bfabd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:00.416185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:00.416638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:00.417125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:00.417966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:00.418438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:00.422081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:00.440354 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] HTTP exception thrown: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd could not be found. Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Request method failure captured: Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd HTTP/1.1 Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsscJSmhrmWubxjqHOnh0ubnTyKuJ-0uTCcJDNR-AXZiwIDWE5R_t4I9kiHYDWgjHdkaoZUYuHDY1qxlVP4nQZhKq5Fj2xJpd4g9ISE1PlY-NwqmB_BFRbJkLZ5io_jiQmHH3QJ1dSBSsRcHzhRQD7ywqzkgSTz-Dozzy425zwTAfrk Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/027612746ecc44f4ab8e7af121ec27a1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_027612746ecc44f4ab8e7af121ec27a1"}]}] Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 027612746ecc44f4ab8e7af121ec27a1 Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-2136329196 Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 92b0186cd92744dabd001b86560b5c90 Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-2136329196-project Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd could not be found. Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3327ff95-5c05-4167-a73b-5fd55f7bfabd'} Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd could not be found. Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd could not be found. Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:00.442741 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:00.448438 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:00.448438 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:00.448438 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:00.448438 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:00.448438 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:00.448438 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd could not be found. Sep 27 10:11:00.448438 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:00.450762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] Returning 404 to user: Instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:00.452568 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fde6e293-7678-4540-aa2f-5168af1cdf40 tempest-ServersNegativeTestJSON-2136329196 tempest-ServersNegativeTestJSON-2136329196-project] 10.222.0.26 "GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd" status: 404 len: 111 microversion: 2.1 time: 0.049129 Sep 27 10:11:00.453214 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1980/3634] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:00 2022] GET /compute/v2.1/servers/3327ff95-5c05-4167-a73b-5fd55f7bfabd => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:00.658626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:00.670660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:00.673158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:00.674341 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-537cb1bd-30d5-4f55-9cee-e9397fac0412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.519519 Sep 27 10:11:00.674891 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1655/3635] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:00 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:00.752535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:00.753735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:00.761132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:00.762334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:00.763090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:00.764191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:00.764942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:00.769190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:00.832084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:00.832908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:00.833353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:00.834001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lazy-loading 'fault' on Instance uuid 668efb48-676d-4a61-837f-eba6c5365546 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:11:00.908179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7f7c5adbac4f48b6bc141e0f888f2496', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c681ff8fe354f97abb5fe0d5181e6f1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:00.919188 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-957bce13-5204-4df5-b264-f9a7897073da tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 200 len: 1305 microversion: 2.1 time: 0.172196 Sep 27 10:11:00.935813 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1981/3636] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:00 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 1305 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:01.335404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-503b0672-162f-4cf6-af3f-95a3618afd3b req-6c8fac19-d9db-43c4-9eaf-76d606d4b7fa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3327ff95-5c05-4167-a73b-5fd55f7bfabd", "name": "network-vif-deleted", "tag": "d571b78e-f1b6-4353-b09e-cac4cd18cde7"}, {"server_uuid": "668efb48-676d-4a61-837f-eba6c5365546", "name": "network-vif-deleted", "tag": "48a8d3b8-b284-4793-8bb8-001085caddfc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:01.364239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d1cd2e01-f9ef-4fcb-a0d1-adda61c6f6d3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:01.364915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d1cd2e01-f9ef-4fcb-a0d1-adda61c6f6d3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:01.369303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-503b0672-162f-4cf6-af3f-95a3618afd3b req-6c8fac19-d9db-43c4-9eaf-76d606d4b7fa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:01.371486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-503b0672-162f-4cf6-af3f-95a3618afd3b req-6c8fac19-d9db-43c4-9eaf-76d606d4b7fa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:01.372598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-503b0672-162f-4cf6-af3f-95a3618afd3b req-6c8fac19-d9db-43c4-9eaf-76d606d4b7fa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:01.373502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cd2e01-f9ef-4fcb-a0d1-adda61c6f6d3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:01.378657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cd2e01-f9ef-4fcb-a0d1-adda61c6f6d3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:01.383112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cd2e01-f9ef-4fcb-a0d1-adda61c6f6d3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:01.384214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cd2e01-f9ef-4fcb-a0d1-adda61c6f6d3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:01.385653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cd2e01-f9ef-4fcb-a0d1-adda61c6f6d3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:01.387957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1cd2e01-f9ef-4fcb-a0d1-adda61c6f6d3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:01.389531 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-503b0672-162f-4cf6-af3f-95a3618afd3b req-6c8fac19-d9db-43c4-9eaf-76d606d4b7fa service nova] Creating event network-vif-deleted:d571b78e-f1b6-4353-b09e-cac4cd18cde7 for instance 3327ff95-5c05-4167-a73b-5fd55f7bfabd on nch2-master-1 Sep 27 10:11:01.390343 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-503b0672-162f-4cf6-af3f-95a3618afd3b req-6c8fac19-d9db-43c4-9eaf-76d606d4b7fa service nova] Creating event network-vif-deleted:48a8d3b8-b284-4793-8bb8-001085caddfc for instance 668efb48-676d-4a61-837f-eba6c5365546 on nch1-master-1 Sep 27 10:11:01.399891 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-503b0672-162f-4cf6-af3f-95a3618afd3b req-6c8fac19-d9db-43c4-9eaf-76d606d4b7fa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.070450 Sep 27 10:11:01.401333 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1656/3637] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:01 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:01.624314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d1cd2e01-f9ef-4fcb-a0d1-adda61c6f6d3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:01.634665 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d1cd2e01-f9ef-4fcb-a0d1-adda61c6f6d3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.274620 Sep 27 10:11:01.649280 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1982/3638] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:01 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:01.705516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:01.706377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:01.715793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:01.716516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:01.717154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:01.718136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:01.718800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:01.719428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:01.948393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:01.949224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:01.959027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:01.960923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:01.961802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:01.963172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:01.963857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:01.970346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.013300 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 HTTP/1.1 Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '668efb48-676d-4a61-837f-eba6c5365546'} Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.020142 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.031198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.031198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.031198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.031198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:02.031198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:02.031198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.031198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.031198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:02.031198 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0ca036ff-126b-49a8-a419-0b0fc200f5a5 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 404 len: 111 microversion: 2.1 time: 0.079926 Sep 27 10:11:02.031198 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1983/3639] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:01 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:11:02.045393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:02.045914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:02.062582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.063087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.063520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.070313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.070811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.071246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.120999 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 HTTP/1.1 Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '668efb48-676d-4a61-837f-eba6c5365546'} Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:02.122896 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:02.127300 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.127300 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.127300 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.127300 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.127300 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.127300 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.127300 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.127300 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:02.127300 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:02.127300 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.127300 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.131705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:02.133071 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dab47748-f7b6-402c-9f44-10043e1b7d17 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 404 len: 111 microversion: 2.1 time: 0.099359 Sep 27 10:11:02.133869 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1984/3640] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:11:02 2022] DELETE /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 111 bytes in 105 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:02.143092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:02.143851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 668efb48-676d-4a61-837f-eba6c5365546] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:02.158686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.159822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.160585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.161769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.162452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.163196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.211171 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 HTTP/1.1 Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '668efb48-676d-4a61-837f-eba6c5365546'} Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.218174 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.222539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.222539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.222539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.222539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:02.222539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:02.222539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. Sep 27 10:11:02.222539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.224564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 668efb48-676d-4a61-837f-eba6c5365546 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:02.226003 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4abfbcdc-f19e-414b-abb7-1ce6b10adbeb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546" status: 404 len: 111 microversion: 2.1 time: 0.087933 Sep 27 10:11:02.227513 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1985/3641] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:02 2022] GET /compute/v2.1/servers/668efb48-676d-4a61-837f-eba6c5365546 => generated 111 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:11:02.236528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:02.236746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:02.243983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.244581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.245071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.245919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.246373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.255757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.287658 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd HTTP/1.1 Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'e581dad6-2c27-4baa-9f55-c082f78b67bd'} Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:02.290106 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:02.295057 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:11:02.295057 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.295057 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.295057 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.295057 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.295057 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.295057 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.295057 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:02.295057 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:02.295057 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:11:02.295057 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.298798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:02.317841 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4854ca0c-6f6f-4f56-8c90-809bbf20e1d6 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 404 len: 111 microversion: 2.1 time: 0.070739 Sep 27 10:11:02.322927 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1986/3642] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:11:02 2022] DELETE /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 111 bytes in 92 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:02.338117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:02.339185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: e581dad6-2c27-4baa-9f55-c082f78b67bd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:02.347668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.348460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.349155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.350192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.351243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.351871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.381836 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd HTTP/1.1 Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'e581dad6-2c27-4baa-9f55-c082f78b67bd'} Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.387531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.392576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.392576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.392576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.392576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:02.392576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:02.392576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. Sep 27 10:11:02.392576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.394982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance e581dad6-2c27-4baa-9f55-c082f78b67bd could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:02.395880 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ae1a6d16-72e5-466c-89e3-b718cff7b0de tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd" status: 404 len: 111 microversion: 2.1 time: 0.068880 Sep 27 10:11:02.396461 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1987/3643] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:02 2022] GET /compute/v2.1/servers/e581dad6-2c27-4baa-9f55-c082f78b67bd => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:02.406425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:02.406934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:02.414863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.416959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.417137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.417907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.418425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.418984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.436841 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 HTTP/1.1 Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '08babbce-373e-4cf5-a138-066b6eeb02c3'} Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:02.439808 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:02.445151 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-906326ae-bac7-495c-a2f3-f7e5a4b2e975 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 404 len: 111 microversion: 2.1 time: 0.039858 Sep 27 10:11:02.449183 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1988/3644] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:11:02 2022] DELETE /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:02.455638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:02.456383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 08babbce-373e-4cf5-a138-066b6eeb02c3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:02.465420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.465944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.466998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.467999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.468524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.469127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.486681 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 HTTP/1.1 Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '08babbce-373e-4cf5-a138-066b6eeb02c3'} Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.488757 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.493845 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.493845 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.493845 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.493845 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:02.493845 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:02.493845 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. Sep 27 10:11:02.493845 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.493845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 08babbce-373e-4cf5-a138-066b6eeb02c3 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:02.493845 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-359027cc-9061-469a-af25-3d48a8d2bfd9 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3" status: 404 len: 111 microversion: 2.1 time: 0.040970 Sep 27 10:11:02.493845 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1989/3645] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:02 2022] GET /compute/v2.1/servers/08babbce-373e-4cf5-a138-066b6eeb02c3 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:02.501777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:02.503091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:02.511909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.512501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.513182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.514129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.515065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.516131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.536117 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e HTTP/1.1 Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '918832ca-5183-4a17-b0ed-01fc0c661f3e'} Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:02.540153 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:02.544989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:11:02.544989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.544989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.544989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.544989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.544989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.544989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.544989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:02.544989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:02.544989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:11:02.544989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.547075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:02.548282 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b62198b4-ebd5-40f8-a8f8-e758cedfc03e tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 404 len: 111 microversion: 2.1 time: 0.051627 Sep 27 10:11:02.552425 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1990/3646] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:11:02 2022] DELETE /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:11:02.580314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:02.582343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 918832ca-5183-4a17-b0ed-01fc0c661f3e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:02.614282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.614784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.615224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.615943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.616640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.621573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.638778 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e HTTP/1.1 Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '918832ca-5183-4a17-b0ed-01fc0c661f3e'} Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.640439 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.644732 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.644732 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.644732 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.644732 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:02.644732 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:02.644732 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. Sep 27 10:11:02.644732 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.644732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 918832ca-5183-4a17-b0ed-01fc0c661f3e could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:02.644732 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7b6dedcc-cd57-4a56-9b0a-1c2be0f12941 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e" status: 404 len: 111 microversion: 2.1 time: 0.088298 Sep 27 10:11:02.644732 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1991/3647] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:02 2022] GET /compute/v2.1/servers/918832ca-5183-4a17-b0ed-01fc0c661f3e => generated 111 bytes in 89 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:02.652175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:02.653084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:02.661630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.662525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.663469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.664717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.665445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.666139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.685073 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 HTTP/1.1 Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '1c7bdeda-4ac1-4db1-a521-41316a9cf5c0'} Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:02.687280 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:02.693047 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:11:02.693047 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.693047 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.693047 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.693047 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.693047 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.693047 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.693047 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:02.693047 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:02.693047 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:11:02.693047 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.703800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:02.704900 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7bc1bcd1-a792-40d9-99fd-64ec2626d26b tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0" status: 404 len: 111 microversion: 2.1 time: 0.057488 Sep 27 10:11:02.706809 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1992/3648] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:11:02 2022] DELETE /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:02.713462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f976e81-6466-4bd4-b3e8-096c95e41547 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:02.714550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f976e81-6466-4bd4-b3e8-096c95e41547 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:02.724887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:02.730134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f976e81-6466-4bd4-b3e8-096c95e41547 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.730867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f976e81-6466-4bd4-b3e8-096c95e41547 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.731673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f976e81-6466-4bd4-b3e8-096c95e41547 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.732814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f976e81-6466-4bd4-b3e8-096c95e41547 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.740384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:02.744124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:02.747716 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-09c9f803-373d-4ccb-8d64-62d00a941af2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 1.046892 Sep 27 10:11:02.750620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f976e81-6466-4bd4-b3e8-096c95e41547 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.750882 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1657/3649] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:01 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 1052 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:02.751928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f976e81-6466-4bd4-b3e8-096c95e41547 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.764063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:02.764815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:02.782228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.783044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.784769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.785612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:02.786351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:02.787276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:02.832869 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 HTTP/1.1 Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1c7bdeda-4ac1-4db1-a521-41316a9cf5c0'} Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:02.835417 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.839875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:02.839875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:02.839875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:02.839875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:02.839875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:02.839875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. Sep 27 10:11:02.839875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:02.843907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:02.846735 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2225e49d-7806-42e5-8e3f-6973f0128c2f tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0" status: 404 len: 111 microversion: 2.1 time: 0.094731 Sep 27 10:11:02.848286 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1658/3650] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:02 2022] GET /compute/v2.1/servers/1c7bdeda-4ac1-4db1-a521-41316a9cf5c0 => generated 111 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:03.579282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6f976e81-6466-4bd4-b3e8-096c95e41547 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:03.589446 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f976e81-6466-4bd4-b3e8-096c95e41547 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1505 microversion: 2.1 time: 0.880673 Sep 27 10:11:03.590648 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1993/3651] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:02 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1505 bytes in 882 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:03.793131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:03.793857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:03.812240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:03.815235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:03.815783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:03.820860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:03.821688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:03.822226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:04.093244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:04.093925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:04.106973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:04.109136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:04.109994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:04.112566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:04.114462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:04.115384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:04.144896 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee HTTP/1.1 Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '4e215381-8752-4c41-aae8-d3b564df3dee'} Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:04.147171 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:04.152680 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:11:04.152680 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.152680 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:04.152680 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.152680 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.152680 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:04.152680 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:04.152680 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:04.152680 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:04.152680 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:11:04.152680 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.158916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:04.160248 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7f00b24d-0f82-4759-8bcb-1b36a3510e13 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 404 len: 111 microversion: 2.1 time: 0.071265 Sep 27 10:11:04.161389 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1994/3652] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:11:04 2022] DELETE /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 111 bytes in 73 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:04.204486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:04.207872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 4e215381-8752-4c41-aae8-d3b564df3dee] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:04.227723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:04.230925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:04.231532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:04.232182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:04.232735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:04.233180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:04.274361 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee HTTP/1.1 Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '4e215381-8752-4c41-aae8-d3b564df3dee'} Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:04.275939 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.280235 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.280235 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:04.280235 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:04.280235 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:04.280235 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:04.280235 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. Sep 27 10:11:04.280235 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.280235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 4e215381-8752-4c41-aae8-d3b564df3dee could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:04.280235 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d844985f-b98e-4e51-9805-6b2354365660 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee" status: 404 len: 111 microversion: 2.1 time: 0.110768 Sep 27 10:11:04.280235 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1995/3653] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:04 2022] GET /compute/v2.1/servers/4e215381-8752-4c41-aae8-d3b564df3dee => generated 111 bytes in 113 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:04.288304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:04.289192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:04.296014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:04.297136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:04.297867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:04.299018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:04.299663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:04.300279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:04.320176 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 HTTP/1.1 Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '35973885-926e-4fa3-80a3-0df40511ebd2'} Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:04.322628 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.328071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:04.329384 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a55f6c33-3105-4564-b988-be447e1a4bcb tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "DELETE /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 404 len: 111 microversion: 2.1 time: 0.046100 Sep 27 10:11:04.330285 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1996/3654] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:11:04 2022] DELETE /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:11:04.338729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:04.339455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] [instance: 35973885-926e-4fa3-80a3-0df40511ebd2] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:04.346024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:04.346605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:04.347126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:04.348086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:04.348989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:04.349615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:04.367421 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] HTTP exception thrown: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Request method failure captured: Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 HTTP/1.1 Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsriWWu9kgn76J7OGJTZ6qnGEs2PYLZ_was6JmUq0xBGwbMJzq6ZX3XkPBUsAZpSGdszikn6dDOqXMyeuDt4vDd9c_AMcau3Z8EwV0CxH1dvf6TkyW6On5XvMEJE-3mkxe-PIQZ39WPztXtXxaVzEV64xL8t-CFvoGh4RwpwpCKMZ_8 Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/6c681ff8fe354f97abb5fe0d5181e6f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_6c681ff8fe354f97abb5fe0d5181e6f1"}]}] Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 6c681ff8fe354f97abb5fe0d5181e6f1 Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-1957619751 Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 7f7c5adbac4f48b6bc141e0f888f2496 Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-1957619751-project Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '35973885-926e-4fa3-80a3-0df40511ebd2'} Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:04.369943 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.374955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:04.374955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:04.374955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:04.374955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:04.374955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:04.374955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. Sep 27 10:11:04.374955 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:04.376858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] Returning 404 to user: Instance 35973885-926e-4fa3-80a3-0df40511ebd2 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:04.377955 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-47b159dd-831c-40b3-939d-1bc00ce2c950 tempest-DeleteServersTestJSON-1957619751 tempest-DeleteServersTestJSON-1957619751-project] 10.222.0.26 "GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2" status: 404 len: 111 microversion: 2.1 time: 0.043644 Sep 27 10:11:04.380920 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1997/3655] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:04 2022] GET /compute/v2.1/servers/35973885-926e-4fa3-80a3-0df40511ebd2 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:04.592180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:04.608961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:04.610691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:04.612094 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-78635d45-f83e-4d1f-a52c-28615ba40a81 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.833009 Sep 27 10:11:04.615716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c773e47c-40c5-41d6-a17f-cb0384ee37b5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:04.616484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c773e47c-40c5-41d6-a17f-cb0384ee37b5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:04.620170 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1659/3656] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:03 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 842 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:04.646672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c773e47c-40c5-41d6-a17f-cb0384ee37b5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:04.648129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c773e47c-40c5-41d6-a17f-cb0384ee37b5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:04.648641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c773e47c-40c5-41d6-a17f-cb0384ee37b5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:04.650501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c773e47c-40c5-41d6-a17f-cb0384ee37b5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:04.651686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c773e47c-40c5-41d6-a17f-cb0384ee37b5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:04.653461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c773e47c-40c5-41d6-a17f-cb0384ee37b5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:05.125037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c773e47c-40c5-41d6-a17f-cb0384ee37b5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:05.143487 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c773e47c-40c5-41d6-a17f-cb0384ee37b5 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1485 microversion: 2.1 time: 0.532123 Sep 27 10:11:05.145349 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1998/3657] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:04 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1485 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:05.293179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueTestJSON-server-517520090", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:05.364471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] validate_networks() for [('4e799abf-8fbf-4193-9ed7-52de886b50f0', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:11:05.647070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:05.648426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:05.655483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:05.656260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:05.656882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:05.657763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:05.658371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:05.658948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:06.088019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:06.091881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:06.111534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:06.113001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:06.114182 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-673fd264-a734-46a2-b36c-61f7bbc68615 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.471161 Sep 27 10:11:06.114809 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 1999/3658] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:05 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:06.736864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:11:06.737421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:11:06.738298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:11:06.739720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:11:06.741640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:11:06.762281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:06.762802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:06.766162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:06.769338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:06.769747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:06.770187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:06.812211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Getting quotas for project c24fd5e915af4b6c872d69b99e7375ce. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:11:06.828735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Getting quotas for user b11313762b324174a9a5f362d1cf4bdd and project c24fd5e915af4b6c872d69b99e7375ce. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:11:06.869997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:11:06.878382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:11:06.974415 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-63e66759-6e3d-45a6-9c8c-16786c13c215 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.685822 Sep 27 10:11:06.982324 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1660/3659] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:11:05 2022] POST /compute/v2.1/servers => generated 383 bytes in 1694 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:11:06.986933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b6301dd4-4e6c-408f-af9e-e3d5200f5df7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:06.987550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b6301dd4-4e6c-408f-af9e-e3d5200f5df7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:07.140698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:07.141494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:07.151019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:07.151366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:07.153847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:07.159502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:07.160079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:07.160818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:07.198590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b6301dd4-4e6c-408f-af9e-e3d5200f5df7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:07.205593 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b6301dd4-4e6c-408f-af9e-e3d5200f5df7 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1213 microversion: 2.1 time: 0.222461 Sep 27 10:11:07.219114 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2000/3660] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:06 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1213 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:07.518196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:07.535615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:07.537098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:07.538424 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aaa1b609-0658-4ce6-a310-28729326397f tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.401742 Sep 27 10:11:07.541429 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1661/3661] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:07 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:11:08.231964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-26a38895-9b4f-460f-937c-79da573f72eb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:08.233528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-26a38895-9b4f-460f-937c-79da573f72eb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:08.258017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a38895-9b4f-460f-937c-79da573f72eb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:08.258526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a38895-9b4f-460f-937c-79da573f72eb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:08.259014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a38895-9b4f-460f-937c-79da573f72eb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:08.260255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a38895-9b4f-460f-937c-79da573f72eb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:08.260255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a38895-9b4f-460f-937c-79da573f72eb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:08.260550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a38895-9b4f-460f-937c-79da573f72eb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:08.572107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:08.572974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:08.590451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:08.592865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:08.594791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:08.597179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:08.601676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:08.602538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:08.654544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-26a38895-9b4f-460f-937c-79da573f72eb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:08.665099 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-26a38895-9b4f-460f-937c-79da573f72eb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1265 microversion: 2.1 time: 0.437357 Sep 27 10:11:08.674770 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2001/3662] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:08 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1265 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:09.849787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3eb04eeb-4206-4c9b-81fb-fe521bf8277a tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Calling method '.version_select at 0x7f7f625b60d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:10.043640 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3eb04eeb-4206-4c9b-81fb-fe521bf8277a tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/d729807b74a241d7a17b184897aa595e/defaults" status: 200 len: 371 microversion: 2.1 time: 0.388500 Sep 27 10:11:10.044547 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2002/3663] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:11:09 2022] GET /compute/v2.1/os-quota-sets/d729807b74a241d7a17b184897aa595e/defaults => generated 371 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:10.055857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33c18ad3-1d9c-4a34-921c-c06adb2c49de tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:10.057586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33c18ad3-1d9c-4a34-921c-c06adb2c49de tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:10.083325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:10.090204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33c18ad3-1d9c-4a34-921c-c06adb2c49de tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:10.091653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33c18ad3-1d9c-4a34-921c-c06adb2c49de tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:10.093183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33c18ad3-1d9c-4a34-921c-c06adb2c49de tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:10.096072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33c18ad3-1d9c-4a34-921c-c06adb2c49de tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:10.098960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33c18ad3-1d9c-4a34-921c-c06adb2c49de tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:10.100130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33c18ad3-1d9c-4a34-921c-c06adb2c49de tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:10.112135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:10.113633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:10.114718 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-50ae0c99-7a76-4882-8395-62ed8556fb6b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 1.549146 Sep 27 10:11:10.122113 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1662/3664] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:08 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 1557 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:10.134379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-96182029-7b18-48a2-82aa-77d073ba22d5 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Calling method '.version_select at 0x7f7f6270d940>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:10.298070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33c18ad3-1d9c-4a34-921c-c06adb2c49de tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:10.310897 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33c18ad3-1d9c-4a34-921c-c06adb2c49de tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1271 microversion: 2.1 time: 0.260846 Sep 27 10:11:10.333666 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2003/3665] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:10 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1271 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:10.392046 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-96182029-7b18-48a2-82aa-77d073ba22d5 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/d729807b74a241d7a17b184897aa595e" status: 200 len: 371 microversion: 2.1 time: 0.268194 Sep 27 10:11:10.392811 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1663/3666] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:11:10 2022] GET /compute/v2.1/os-quota-sets/d729807b74a241d7a17b184897aa595e => generated 371 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:10.409267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8dbee1e1-a2ef-44bc-9609-0cf4d40be3ae tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Calling method '.version_select at 0x7f7f625bcca0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:10.485280 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8dbee1e1-a2ef-44bc-9609-0cf4d40be3ae tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/d729807b74a241d7a17b184897aa595e/defaults" status: 200 len: 371 microversion: 2.1 time: 0.080433 Sep 27 10:11:10.486978 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2004/3667] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:11:10 2022] GET /compute/v2.1/os-quota-sets/d729807b74a241d7a17b184897aa595e/defaults => generated 371 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:11:10.502213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e806654-0566-4bc5-85bf-38160ae08090 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Calling method '.version_select at 0x7f7f626d1670>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:10.618878 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4e806654-0566-4bc5-85bf-38160ae08090 tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/d729807b74a241d7a17b184897aa595e" status: 200 len: 371 microversion: 2.1 time: 0.120103 Sep 27 10:11:10.626322 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1664/3668] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:11:10 2022] GET /compute/v2.1/os-quota-sets/d729807b74a241d7a17b184897aa595e => generated 371 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:10.633859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d559b499-5ad9-4215-84d0-10e3de14c1db tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] Calling method '.version_select at 0x7f7f625b61f0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:10.735125 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d559b499-5ad9-4215-84d0-10e3de14c1db tempest-QuotasTestJSON-1721900347 tempest-QuotasTestJSON-1721900347-project] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/d729807b74a241d7a17b184897aa595e?user_id=7d3c9ae0a3b34cd487cf069002f6d70e" status: 200 len: 371 microversion: 2.1 time: 0.106707 Sep 27 10:11:10.736044 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2005/3669] 10.222.0.26 () {58 vars in 1323 bytes} [Tue Sep 27 10:11:10 2022] GET /compute/v2.1/os-quota-sets/d729807b74a241d7a17b184897aa595e?user_id=7d3c9ae0a3b34cd487cf069002f6d70e => generated 371 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:11.158917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:11.159516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:11.168176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:11.168688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:11.169175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:11.169895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:11.170347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:11.170960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:11.340142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9582d8ca-af97-4c77-97d0-c736a8603ffb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:11.342149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9582d8ca-af97-4c77-97d0-c736a8603ffb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:11.359499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9582d8ca-af97-4c77-97d0-c736a8603ffb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:11.360395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9582d8ca-af97-4c77-97d0-c736a8603ffb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:11.361106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9582d8ca-af97-4c77-97d0-c736a8603ffb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:11.362127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9582d8ca-af97-4c77-97d0-c736a8603ffb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:11.362847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9582d8ca-af97-4c77-97d0-c736a8603ffb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:11.363691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9582d8ca-af97-4c77-97d0-c736a8603ffb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:11.487043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9582d8ca-af97-4c77-97d0-c736a8603ffb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:11.496189 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9582d8ca-af97-4c77-97d0-c736a8603ffb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1271 microversion: 2.1 time: 0.161945 Sep 27 10:11:11.496993 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2006/3670] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:11 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1271 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:11.797252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:11.806705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:11.808175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:11.809435 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f45021d7-c926-43ad-9a16-b9c915024e9b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.661430 Sep 27 10:11:11.809999 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1665/3671] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:11 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:12.520098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b528cb7-b2e4-471a-991f-793478599ca2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:12.521605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3b528cb7-b2e4-471a-991f-793478599ca2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:12.540627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b528cb7-b2e4-471a-991f-793478599ca2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:12.541186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b528cb7-b2e4-471a-991f-793478599ca2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:12.541639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b528cb7-b2e4-471a-991f-793478599ca2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:12.542366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b528cb7-b2e4-471a-991f-793478599ca2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:12.542832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b528cb7-b2e4-471a-991f-793478599ca2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:12.543265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b528cb7-b2e4-471a-991f-793478599ca2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:12.989707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1216678942", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {}, "networks": [{"uuid": "9c10d900-7e77-42f0-8e1b-f46336bf67ee"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:13.122202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] validate_networks() for [('9c10d900-7e77-42f0-8e1b-f46336bf67ee', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:11:13.564318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3b528cb7-b2e4-471a-991f-793478599ca2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:13.574172 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3b528cb7-b2e4-471a-991f-793478599ca2 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1313 microversion: 2.1 time: 1.058182 Sep 27 10:11:13.579241 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2007/3672] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:12 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1313 bytes in 1064 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:13.590243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:13.592916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:13.605549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:13.609113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:13.609784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:13.610720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:13.613000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:13.614024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:13.974475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:13.987514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:14.006476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:14.009501 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2639d55f-3926-437e-ad7a-aa88fd9511ba tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.428795 Sep 27 10:11:14.018293 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2008/3673] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:13 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:14.605822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-55dee43e-3687-4c78-bcb7-f40c9b8328ea tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:14.606913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-55dee43e-3687-4c78-bcb7-f40c9b8328ea tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:14.631179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55dee43e-3687-4c78-bcb7-f40c9b8328ea tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:14.633261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55dee43e-3687-4c78-bcb7-f40c9b8328ea tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:14.635822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55dee43e-3687-4c78-bcb7-f40c9b8328ea tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:14.637647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55dee43e-3687-4c78-bcb7-f40c9b8328ea tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:14.638132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55dee43e-3687-4c78-bcb7-f40c9b8328ea tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:14.642788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55dee43e-3687-4c78-bcb7-f40c9b8328ea tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:14.899835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:11:14.900305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:11:14.902831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:11:14.904221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:11:14.904667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:11:14.937431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:14.947257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:14.947834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:14.948708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:14.949199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:14.949652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:14.973063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55dee43e-3687-4c78-bcb7-f40c9b8328ea tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:14.984632 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-55dee43e-3687-4c78-bcb7-f40c9b8328ea tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1313 microversion: 2.1 time: 0.383636 Sep 27 10:11:14.985620 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2009/3674] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:14 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1313 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:15.004714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:15.024067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Getting quotas for project be55c1fb4e6f4870b25e986907d41101. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:11:15.031351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Getting quotas for user 96e48f22b8e7478986f912e64df670b0 and project be55c1fb4e6f4870b25e986907d41101. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:11:15.047901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:11:15.050460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:11:15.050646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:15.051112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:15.063818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:15.065834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:15.066331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:15.070302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:15.070624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:15.071539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:15.153804 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f2871318-e173-4e21-9b27-c2be266b0c94 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.452638 Sep 27 10:11:15.154947 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1666/3675] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:11:12 2022] POST /compute/v2.1/servers => generated 380 bytes in 2455 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:11:15.171977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4424ce31-79b7-4bbb-a04f-cb9b16e899f0 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:15.173774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4424ce31-79b7-4bbb-a04f-cb9b16e899f0 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:15.431059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4424ce31-79b7-4bbb-a04f-cb9b16e899f0 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:15.440425 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4424ce31-79b7-4bbb-a04f-cb9b16e899f0 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1224 microversion: 2.1 time: 0.277904 Sep 27 10:11:15.452952 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1667/3676] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:15 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1224 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:15.452952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-9e109cf2-df02-4777-8495-be14ef226879 req-68b3bc2a-d793-481e-ac69-41eade1bed26 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "87e69339-c527-44d1-8566-ce64d503d7a6", "tag": "eaf55525-ffa8-40a5-9a88-b2f94df3274b"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:15.465155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:15.465314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9e109cf2-df02-4777-8495-be14ef226879 req-68b3bc2a-d793-481e-ac69-41eade1bed26 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:15.466440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9e109cf2-df02-4777-8495-be14ef226879 req-68b3bc2a-d793-481e-ac69-41eade1bed26 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:15.467679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9e109cf2-df02-4777-8495-be14ef226879 req-68b3bc2a-d793-481e-ac69-41eade1bed26 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:15.481049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:15.486306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:15.491490 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1cc4c505-149f-45fb-ae75-54264583b412 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.447261 Sep 27 10:11:15.492259 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2010/3677] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:15 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:11:15.497095 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-9e109cf2-df02-4777-8495-be14ef226879 req-68b3bc2a-d793-481e-ac69-41eade1bed26 service nova] Creating event network-changed:eaf55525-ffa8-40a5-9a88-b2f94df3274b for instance 87e69339-c527-44d1-8566-ce64d503d7a6 on nch1-master-1 Sep 27 10:11:15.509824 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-9e109cf2-df02-4777-8495-be14ef226879 req-68b3bc2a-d793-481e-ac69-41eade1bed26 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066798 Sep 27 10:11:15.510477 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1668/3678] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:15.721435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-08c04a6b-a957-47c3-aa3a-ead9e26d4925 tempest-ComputeTenantNetworksTest-1331271593 tempest-ComputeTenantNetworksTest-1331271593-project] Calling method '.version_select at 0x7f7f62615dc0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:15.966376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d60e9551-946a-4243-9888-4fe45b87245e req-caef81bc-1982-4a3c-ba72-f7c27a13409e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-plugged", "status": "completed", "tag": "8cd8623a-a3f6-4d1c-aced-9707f8353fa6"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:15.985189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d60e9551-946a-4243-9888-4fe45b87245e req-caef81bc-1982-4a3c-ba72-f7c27a13409e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:15.986823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d60e9551-946a-4243-9888-4fe45b87245e req-caef81bc-1982-4a3c-ba72-f7c27a13409e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:15.987569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d60e9551-946a-4243-9888-4fe45b87245e req-caef81bc-1982-4a3c-ba72-f7c27a13409e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:16.004780 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d60e9551-946a-4243-9888-4fe45b87245e req-caef81bc-1982-4a3c-ba72-f7c27a13409e service nova] Creating event network-vif-plugged:8cd8623a-a3f6-4d1c-aced-9707f8353fa6 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:11:16.012014 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d60e9551-946a-4243-9888-4fe45b87245e req-caef81bc-1982-4a3c-ba72-f7c27a13409e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051692 Sep 27 10:11:16.013711 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1669/3679] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:11:16.022704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-15476480-1c20-46b1-8acf-ece16ee25ac3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:16.023980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-15476480-1c20-46b1-8acf-ece16ee25ac3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:16.034573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15476480-1c20-46b1-8acf-ece16ee25ac3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:16.035402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15476480-1c20-46b1-8acf-ece16ee25ac3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:16.036422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15476480-1c20-46b1-8acf-ece16ee25ac3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:16.038857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15476480-1c20-46b1-8acf-ece16ee25ac3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:16.039790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15476480-1c20-46b1-8acf-ece16ee25ac3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:16.041016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15476480-1c20-46b1-8acf-ece16ee25ac3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:16.282529 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-08c04a6b-a957-47c3-aa3a-ead9e26d4925 tempest-ComputeTenantNetworksTest-1331271593 tempest-ComputeTenantNetworksTest-1331271593-project] 10.222.0.26 "GET /compute/v2.1/os-tenant-networks" status: 200 len: 308 microversion: 2.1 time: 0.788888 Sep 27 10:11:16.283124 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2011/3680] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 10:11:15 2022] GET /compute/v2.1/os-tenant-networks => generated 308 bytes in 790 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:16.295811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f9f937b8-180b-4438-a324-a3f23f4a6f8e tempest-ComputeTenantNetworksTest-1331271593 tempest-ComputeTenantNetworksTest-1331271593-project] Calling method '.version_select at 0x7f7f625511f0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:16.360414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-15476480-1c20-46b1-8acf-ece16ee25ac3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:16.369148 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-15476480-1c20-46b1-8acf-ece16ee25ac3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1313 microversion: 2.1 time: 0.353593 Sep 27 10:11:16.386072 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1670/3681] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:16 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1313 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:16.452260 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f9f937b8-180b-4438-a324-a3f23f4a6f8e tempest-ComputeTenantNetworksTest-1331271593 tempest-ComputeTenantNetworksTest-1331271593-project] 10.222.0.26 "GET /compute/v2.1/os-tenant-networks/64e5ca0b-2f73-4fdf-8e52-e92221a450f5" status: 200 len: 139 microversion: 2.1 time: 0.162107 Sep 27 10:11:16.452962 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2012/3682] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:11:16 2022] GET /compute/v2.1/os-tenant-networks/64e5ca0b-2f73-4fdf-8e52-e92221a450f5 => generated 139 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:11:16.469194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b4ec15b1-3430-46a7-8d42-480ceeecb058 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:16.469881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b4ec15b1-3430-46a7-8d42-480ceeecb058 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:16.478400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4ec15b1-3430-46a7-8d42-480ceeecb058 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:16.478644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4ec15b1-3430-46a7-8d42-480ceeecb058 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:16.499113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4ec15b1-3430-46a7-8d42-480ceeecb058 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:16.502099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4ec15b1-3430-46a7-8d42-480ceeecb058 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:16.502819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4ec15b1-3430-46a7-8d42-480ceeecb058 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:16.503928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4ec15b1-3430-46a7-8d42-480ceeecb058 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:16.520138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:16.520927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:16.538410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:16.539497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:16.540099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:16.541182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:16.541799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:16.542620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:16.681261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b4ec15b1-3430-46a7-8d42-480ceeecb058 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:16.692453 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b4ec15b1-3430-46a7-8d42-480ceeecb058 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1276 microversion: 2.1 time: 0.228652 Sep 27 10:11:16.697303 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1671/3683] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:16 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1276 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:16.762813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3e74952f-7d44-4e32-8ad2-2be1ede153be tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:16.773941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3e74952f-7d44-4e32-8ad2-2be1ede153be tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:16.778492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e74952f-7d44-4e32-8ad2-2be1ede153be tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Searching by: {'deleted': False, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:11:16.793747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e74952f-7d44-4e32-8ad2-2be1ede153be tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:16.794875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e74952f-7d44-4e32-8ad2-2be1ede153be tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:16.798756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e74952f-7d44-4e32-8ad2-2be1ede153be tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:16.802057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e74952f-7d44-4e32-8ad2-2be1ede153be tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:16.802828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e74952f-7d44-4e32-8ad2-2be1ede153be tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:16.806587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e74952f-7d44-4e32-8ad2-2be1ede153be tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:16.841282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-3e74952f-7d44-4e32-8ad2-2be1ede153be tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:11:16.925511 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3e74952f-7d44-4e32-8ad2-2be1ede153be tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 334 microversion: 2.1 time: 0.165696 Sep 27 10:11:16.931574 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1672/3684] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:11:16 2022] GET /compute/v2.1/servers => generated 334 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:16.953974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:16.959636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:16.972917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Searching by: {'deleted': False, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:11:16.982734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:16.989816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:16.990522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:16.991770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:16.992409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:16.994001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:17.040142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:17.042262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:11:17.060014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:17.065219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:17.068142 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2ef2903e-894c-45e8-aa76-b0f2407ca8a6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.551048 Sep 27 10:11:17.071065 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2013/3685] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:16 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:17.118729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:17.128574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:17.129325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:17.129783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:17.393762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-137cb8dd-a192-4039-bab2-06236cfb9d40 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:17.396125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-137cb8dd-a192-4039-bab2-06236cfb9d40 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:17.413975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-137cb8dd-a192-4039-bab2-06236cfb9d40 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:17.416968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-137cb8dd-a192-4039-bab2-06236cfb9d40 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:17.418032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-137cb8dd-a192-4039-bab2-06236cfb9d40 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:17.418971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-137cb8dd-a192-4039-bab2-06236cfb9d40 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:17.420046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-137cb8dd-a192-4039-bab2-06236cfb9d40 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:17.420726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-137cb8dd-a192-4039-bab2-06236cfb9d40 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:17.681293 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4bd88a49-6090-4b61-98aa-b09a3e08cb26 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/detail" status: 200 len: 1697 microversion: 2.1 time: 0.732601 Sep 27 10:11:17.682222 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1673/3686] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:11:16 2022] GET /compute/v2.1/servers/detail => generated 1697 bytes in 734 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:17.708675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9ca297a4-9432-49f5-97f4-e306c4fb0aa7 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:17.724534 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9ca297a4-9432-49f5-97f4-e306c4fb0aa7 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.019634 Sep 27 10:11:17.727518 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1674/3687] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:11:17 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:17.738476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9aff762a-036d-4e8b-badd-6b10ebc05099 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:17.739224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9aff762a-036d-4e8b-badd-6b10ebc05099 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:17.755746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-137cb8dd-a192-4039-bab2-06236cfb9d40 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:17.767942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aff762a-036d-4e8b-badd-6b10ebc05099 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:17.768091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-137cb8dd-a192-4039-bab2-06236cfb9d40 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1313 microversion: 2.1 time: 0.379962 Sep 27 10:11:17.768590 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2014/3688] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:17 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1313 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:17.774508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aff762a-036d-4e8b-badd-6b10ebc05099 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:17.779524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aff762a-036d-4e8b-badd-6b10ebc05099 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:17.780310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aff762a-036d-4e8b-badd-6b10ebc05099 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:17.794718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aff762a-036d-4e8b-badd-6b10ebc05099 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:17.795179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aff762a-036d-4e8b-badd-6b10ebc05099 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:18.018505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9aff762a-036d-4e8b-badd-6b10ebc05099 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:18.032528 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9aff762a-036d-4e8b-badd-6b10ebc05099 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1282 microversion: 2.1 time: 0.303807 Sep 27 10:11:18.033524 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1675/3689] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:17 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1282 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:18.072731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-779e0f52-78a7-4938-b7c9-df5230fc5295 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:18.074541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-779e0f52-78a7-4938-b7c9-df5230fc5295 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:18.086120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-779e0f52-78a7-4938-b7c9-df5230fc5295 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:18.097153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:18.097802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:18.100203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-779e0f52-78a7-4938-b7c9-df5230fc5295 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:18.102403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-779e0f52-78a7-4938-b7c9-df5230fc5295 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:18.103764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-779e0f52-78a7-4938-b7c9-df5230fc5295 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:18.105499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-779e0f52-78a7-4938-b7c9-df5230fc5295 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:18.105985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-779e0f52-78a7-4938-b7c9-df5230fc5295 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:18.108051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:18.111808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:18.113389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:18.115283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:18.118029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:18.121279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:18.182182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-779e0f52-78a7-4938-b7c9-df5230fc5295 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:11:18.435161 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-779e0f52-78a7-4938-b7c9-df5230fc5295 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "DELETE /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 204 len: 0 microversion: 2.1 time: 0.366233 Sep 27 10:11:18.438093 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2015/3690] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:11:18 2022] DELETE /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 0 bytes in 370 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:11:18.454028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-710223ee-8af4-4990-ad12-f6d280099e98 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:18.456444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-710223ee-8af4-4990-ad12-f6d280099e98 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:18.488838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-710223ee-8af4-4990-ad12-f6d280099e98 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:18.491084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-710223ee-8af4-4990-ad12-f6d280099e98 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:18.492072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-710223ee-8af4-4990-ad12-f6d280099e98 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:18.494461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-710223ee-8af4-4990-ad12-f6d280099e98 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:18.499392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-710223ee-8af4-4990-ad12-f6d280099e98 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:18.500012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-710223ee-8af4-4990-ad12-f6d280099e98 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:18.804888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:18.816263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:18.817862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:18.819629 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-55998627-8804-4578-968c-690778de523b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.726575 Sep 27 10:11:18.820831 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1676/3691] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:18 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:18.825762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-acd17c35-2d5d-4131-bdb7-cc979144f7ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:18.826379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-acd17c35-2d5d-4131-bdb7-cc979144f7ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:18.840044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acd17c35-2d5d-4131-bdb7-cc979144f7ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:18.840543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acd17c35-2d5d-4131-bdb7-cc979144f7ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:18.841087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acd17c35-2d5d-4131-bdb7-cc979144f7ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:18.841747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acd17c35-2d5d-4131-bdb7-cc979144f7ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:18.842212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acd17c35-2d5d-4131-bdb7-cc979144f7ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:18.842641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acd17c35-2d5d-4131-bdb7-cc979144f7ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:18.847485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-710223ee-8af4-4990-ad12-f6d280099e98 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:18.865532 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-710223ee-8af4-4990-ad12-f6d280099e98 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1700 microversion: 2.1 time: 0.423105 Sep 27 10:11:18.867260 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2016/3692] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:18 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1700 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:19.059364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-be2b2d05-1787-4d17-b4e2-5d05a2e15c96 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:19.066296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-be2b2d05-1787-4d17-b4e2-5d05a2e15c96 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:19.073769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:19.076299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be2b2d05-1787-4d17-b4e2-5d05a2e15c96 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:19.078297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be2b2d05-1787-4d17-b4e2-5d05a2e15c96 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:19.079457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be2b2d05-1787-4d17-b4e2-5d05a2e15c96 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:19.080801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be2b2d05-1787-4d17-b4e2-5d05a2e15c96 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:19.081490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be2b2d05-1787-4d17-b4e2-5d05a2e15c96 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:19.083066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be2b2d05-1787-4d17-b4e2-5d05a2e15c96 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:19.164577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-acd17c35-2d5d-4131-bdb7-cc979144f7ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:19.177406 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-acd17c35-2d5d-4131-bdb7-cc979144f7ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1313 microversion: 2.1 time: 0.355209 Sep 27 10:11:19.178201 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1677/3693] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:18 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1313 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:19.490271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-be2b2d05-1787-4d17-b4e2-5d05a2e15c96 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:19.504890 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-be2b2d05-1787-4d17-b4e2-5d05a2e15c96 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.451821 Sep 27 10:11:19.521712 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2017/3694] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:19 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:19.850698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:19.851964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:19.864263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:19.866405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:19.868964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:19.870335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:19.871985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:19.874718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:19.891881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67d365b0-6d84-4c9c-98da-c458063c6666 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:19.893401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67d365b0-6d84-4c9c-98da-c458063c6666 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:19.912065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67d365b0-6d84-4c9c-98da-c458063c6666 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:19.915484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67d365b0-6d84-4c9c-98da-c458063c6666 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:19.916030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67d365b0-6d84-4c9c-98da-c458063c6666 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:19.917490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67d365b0-6d84-4c9c-98da-c458063c6666 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:19.919156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67d365b0-6d84-4c9c-98da-c458063c6666 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:19.922556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67d365b0-6d84-4c9c-98da-c458063c6666 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:20.236290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-67d365b0-6d84-4c9c-98da-c458063c6666 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:20.253287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67d365b0-6d84-4c9c-98da-c458063c6666 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1700 microversion: 2.1 time: 0.360039 Sep 27 10:11:20.253287 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2018/3695] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:19 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1700 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:20.264077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af83ccea-678e-4827-badd-aaa51708eb4b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:20.264732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af83ccea-678e-4827-badd-aaa51708eb4b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:20.277142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af83ccea-678e-4827-badd-aaa51708eb4b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:20.278193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af83ccea-678e-4827-badd-aaa51708eb4b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:20.279883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af83ccea-678e-4827-badd-aaa51708eb4b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:20.280685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af83ccea-678e-4827-badd-aaa51708eb4b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:20.281167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af83ccea-678e-4827-badd-aaa51708eb4b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:20.281604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af83ccea-678e-4827-badd-aaa51708eb4b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:20.575493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af83ccea-678e-4827-badd-aaa51708eb4b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:20.591185 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af83ccea-678e-4827-badd-aaa51708eb4b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1313 microversion: 2.1 time: 0.339653 Sep 27 10:11:20.592484 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2019/3696] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:20 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1313 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:20.606521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d176425d-8466-45b2-94e7-37e44e464725 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:20.607394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d176425d-8466-45b2-94e7-37e44e464725 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:20.626901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d176425d-8466-45b2-94e7-37e44e464725 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:20.627485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d176425d-8466-45b2-94e7-37e44e464725 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:20.627930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d176425d-8466-45b2-94e7-37e44e464725 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:20.628780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d176425d-8466-45b2-94e7-37e44e464725 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:20.629266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d176425d-8466-45b2-94e7-37e44e464725 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:20.629699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d176425d-8466-45b2-94e7-37e44e464725 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:20.726878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:20.978142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d176425d-8466-45b2-94e7-37e44e464725 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:20.982982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:20.990815 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d176425d-8466-45b2-94e7-37e44e464725 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.388521 Sep 27 10:11:20.991770 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2020/3697] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:20 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:21.006320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:21.021572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:21.023667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:21.026386 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f4367e1-c410-433a-ae5c-4e2119bf6e68 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 1.183833 Sep 27 10:11:21.050013 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1678/3698] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:19 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 1207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:21.270446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25656b98-20bd-47a9-b995-66e3947f75fe tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:21.271067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-25656b98-20bd-47a9-b995-66e3947f75fe tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:21.290762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25656b98-20bd-47a9-b995-66e3947f75fe tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:21.291270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25656b98-20bd-47a9-b995-66e3947f75fe tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:21.291721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25656b98-20bd-47a9-b995-66e3947f75fe tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:21.292509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25656b98-20bd-47a9-b995-66e3947f75fe tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:21.292931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25656b98-20bd-47a9-b995-66e3947f75fe tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:21.293359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25656b98-20bd-47a9-b995-66e3947f75fe tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:21.359336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-31490e69-13c7-409a-b69d-77bb01df0e1a tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] Calling method '.version_select at 0x7f7f62e63820>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:21.422727 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-31490e69-13c7-409a-b69d-77bb01df0e1a tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/b313c2a7-2f8f-4fa4-b158-8b23b1a99360" status: 200 len: 344 microversion: 2.1 time: 0.069145 Sep 27 10:11:21.441833 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1679/3699] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:11:21 2022] GET /compute/v2.1/os-volumes/b313c2a7-2f8f-4fa4-b158-8b23b1a99360 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:11:21.619914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-800d9dea-c00c-4a50-9953-9aebce1f3f0a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:21.620547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-800d9dea-c00c-4a50-9953-9aebce1f3f0a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:21.637310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-800d9dea-c00c-4a50-9953-9aebce1f3f0a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:21.640101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-800d9dea-c00c-4a50-9953-9aebce1f3f0a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:21.642923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-800d9dea-c00c-4a50-9953-9aebce1f3f0a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:21.645056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-800d9dea-c00c-4a50-9953-9aebce1f3f0a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:21.649625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-800d9dea-c00c-4a50-9953-9aebce1f3f0a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:21.650387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-800d9dea-c00c-4a50-9953-9aebce1f3f0a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:21.772241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-25656b98-20bd-47a9-b995-66e3947f75fe tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:21.789854 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-25656b98-20bd-47a9-b995-66e3947f75fe tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1700 microversion: 2.1 time: 0.523051 Sep 27 10:11:21.811779 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2021/3700] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:21 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1700 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:22.014507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a53d0904-ea72-43ee-a083-2a7ab831795a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:22.015139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a53d0904-ea72-43ee-a083-2a7ab831795a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:22.029070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a53d0904-ea72-43ee-a083-2a7ab831795a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:22.029813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a53d0904-ea72-43ee-a083-2a7ab831795a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:22.030445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a53d0904-ea72-43ee-a083-2a7ab831795a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:22.031849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a53d0904-ea72-43ee-a083-2a7ab831795a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:22.036768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a53d0904-ea72-43ee-a083-2a7ab831795a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:22.037410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a53d0904-ea72-43ee-a083-2a7ab831795a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:22.334582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a53d0904-ea72-43ee-a083-2a7ab831795a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:22.352849 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a53d0904-ea72-43ee-a083-2a7ab831795a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.342203 Sep 27 10:11:22.356706 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2022/3701] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:22 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:22.369394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:22.370222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:22.399297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:22.399831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:22.400285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:22.401526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:22.402182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:22.404308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:22.666587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-800d9dea-c00c-4a50-9953-9aebce1f3f0a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:22.693129 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-800d9dea-c00c-4a50-9953-9aebce1f3f0a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1313 microversion: 2.1 time: 1.076700 Sep 27 10:11:22.693464 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1680/3702] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:21 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1313 bytes in 1079 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:22.703931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2dc93e52-4ad3-47df-bee6-d7513740bb36 req-723e5092-9ef9-4870-957d-400e316bf005 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2ad21db4-8d06-48e5-b469-be50b03f701a", "tag": "55eb0fa6-bb39-46b7-889c-e034fd5114c2"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:22.748933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2dc93e52-4ad3-47df-bee6-d7513740bb36 req-723e5092-9ef9-4870-957d-400e316bf005 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:22.749388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2dc93e52-4ad3-47df-bee6-d7513740bb36 req-723e5092-9ef9-4870-957d-400e316bf005 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:22.749841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2dc93e52-4ad3-47df-bee6-d7513740bb36 req-723e5092-9ef9-4870-957d-400e316bf005 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:22.778130 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2dc93e52-4ad3-47df-bee6-d7513740bb36 req-723e5092-9ef9-4870-957d-400e316bf005 service nova] Creating event network-changed:55eb0fa6-bb39-46b7-889c-e034fd5114c2 for instance 2ad21db4-8d06-48e5-b469-be50b03f701a on nch2-master-1 Sep 27 10:11:22.798034 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2dc93e52-4ad3-47df-bee6-d7513740bb36 req-723e5092-9ef9-4870-957d-400e316bf005 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.102032 Sep 27 10:11:22.799324 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1681/3703] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:22.837582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6f4aa76c-3210-43de-b6ba-12ad2fa918b5 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:22.838757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6f4aa76c-3210-43de-b6ba-12ad2fa918b5 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:22.849616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f4aa76c-3210-43de-b6ba-12ad2fa918b5 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:22.851609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f4aa76c-3210-43de-b6ba-12ad2fa918b5 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:22.852990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f4aa76c-3210-43de-b6ba-12ad2fa918b5 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:22.855641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f4aa76c-3210-43de-b6ba-12ad2fa918b5 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:22.858537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f4aa76c-3210-43de-b6ba-12ad2fa918b5 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:22.859312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f4aa76c-3210-43de-b6ba-12ad2fa918b5 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:22.899514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:22.911050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:22.916745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:22.923805 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b6319e2a-bbf9-4fab-b2e2-1368e43c1e32 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.562181 Sep 27 10:11:22.925404 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2023/3704] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:22 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:23.086981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6f4aa76c-3210-43de-b6ba-12ad2fa918b5 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:23.095287 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6f4aa76c-3210-43de-b6ba-12ad2fa918b5 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1633 microversion: 2.1 time: 0.270708 Sep 27 10:11:23.109532 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1682/3705] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:22 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1633 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:23.171066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-00fff673-1eda-424f-a406-86d204634366 tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] Calling method '.version_select at 0x7f7f625b6700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:23.236860 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-00fff673-1eda-424f-a406-86d204634366 tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/581dba67-f337-4542-9d7a-c4f99fb86ae0" status: 200 len: 344 microversion: 2.1 time: 0.069486 Sep 27 10:11:23.237727 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2024/3706] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:11:23 2022] GET /compute/v2.1/os-volumes/581dba67-f337-4542-9d7a-c4f99fb86ae0 => generated 344 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:23.382162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c2c2022-4244-44d2-a4bf-866312bf5197 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:23.382775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c2c2022-4244-44d2-a4bf-866312bf5197 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:23.395048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c2c2022-4244-44d2-a4bf-866312bf5197 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:23.395892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c2c2022-4244-44d2-a4bf-866312bf5197 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:23.396888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c2c2022-4244-44d2-a4bf-866312bf5197 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:23.399585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c2c2022-4244-44d2-a4bf-866312bf5197 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:23.400267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c2c2022-4244-44d2-a4bf-866312bf5197 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:23.400901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c2c2022-4244-44d2-a4bf-866312bf5197 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:23.732880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f4bf2c89-a042-4aa1-9e81-5d71beda1e87 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:23.744363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f4bf2c89-a042-4aa1-9e81-5d71beda1e87 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:23.759644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bf2c89-a042-4aa1-9e81-5d71beda1e87 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:23.760511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bf2c89-a042-4aa1-9e81-5d71beda1e87 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:23.761388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bf2c89-a042-4aa1-9e81-5d71beda1e87 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:23.766895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bf2c89-a042-4aa1-9e81-5d71beda1e87 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:23.768117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bf2c89-a042-4aa1-9e81-5d71beda1e87 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:23.769491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4bf2c89-a042-4aa1-9e81-5d71beda1e87 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:23.992606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c2c2022-4244-44d2-a4bf-866312bf5197 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:24.042212 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c2c2022-4244-44d2-a4bf-866312bf5197 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.662559 Sep 27 10:11:24.048112 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1683/3707] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:23 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:24.063096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:24.065322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:24.105886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:24.108831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:24.109909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:24.110907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:24.111908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:24.113579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:24.284245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f4bf2c89-a042-4aa1-9e81-5d71beda1e87 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:24.298760 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f4bf2c89-a042-4aa1-9e81-5d71beda1e87 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1313 microversion: 2.1 time: 0.578125 Sep 27 10:11:24.299933 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2025/3708] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:23 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1313 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:24.340839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:24.342495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:24.352680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:24.353432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:24.354105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:24.355076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:24.355735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:24.356375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:24.412427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:24.413899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:24.414595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:24.415220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lazy-loading 'fault' on Instance uuid 451a2ed4-e57c-46fc-8e29-09ad474cfa9d {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:11:24.553859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a83af7f06aa94823989c466abc0ffa3e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a74a11e81faa4ca59f7c5662a3e8321f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:24.567750 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-72c294e4-ad6a-48eb-b8d7-dca416619ec9 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 200 len: 1359 microversion: 2.1 time: 0.266545 Sep 27 10:11:24.569631 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2026/3709] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:24 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 1359 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:24.634736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:24.647177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:24.649847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:24.651229 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f0fb547e-ec97-4693-916c-4a403a88a230 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.600937 Sep 27 10:11:24.652132 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1684/3710] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:24 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 603 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:24.806877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4f7eb262-1ff2-4017-bda5-19286df3b118 tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] Calling method '.version_select at 0x7f7f626c9ca0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:24.823490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-fc9a518f-bbc3-4461-a527-2e9f0d7de333 req-b6abc5bf-92b9-4189-a9b8-685bc629f44a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "451a2ed4-e57c-46fc-8e29-09ad474cfa9d", "name": "network-vif-deleted", "tag": "8ab5011a-0825-464f-88c1-5874577eff8e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:24.832553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:24.846248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fc9a518f-bbc3-4461-a527-2e9f0d7de333 req-b6abc5bf-92b9-4189-a9b8-685bc629f44a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:24.847050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fc9a518f-bbc3-4461-a527-2e9f0d7de333 req-b6abc5bf-92b9-4189-a9b8-685bc629f44a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:24.847729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fc9a518f-bbc3-4461-a527-2e9f0d7de333 req-b6abc5bf-92b9-4189-a9b8-685bc629f44a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:24.869459 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-fc9a518f-bbc3-4461-a527-2e9f0d7de333 req-b6abc5bf-92b9-4189-a9b8-685bc629f44a service nova] Creating event network-vif-deleted:8ab5011a-0825-464f-88c1-5874577eff8e for instance 451a2ed4-e57c-46fc-8e29-09ad474cfa9d on nch1-master-1 Sep 27 10:11:24.878941 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-fc9a518f-bbc3-4461-a527-2e9f0d7de333 req-b6abc5bf-92b9-4189-a9b8-685bc629f44a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062140 Sep 27 10:11:24.884891 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1685/3711] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:24.911893 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4f7eb262-1ff2-4017-bda5-19286df3b118 tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/a4cf7c86-4e26-462a-a552-ac303e88f53d" status: 200 len: 344 microversion: 2.1 time: 0.112211 Sep 27 10:11:24.912597 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2027/3712] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:11:24 2022] GET /compute/v2.1/os-volumes/a4cf7c86-4e26-462a-a552-ac303e88f53d => generated 344 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:24.937495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0d03441b-e8ab-4813-9bd9-e46ec796869e tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] Calling method '.version_select at 0x7f7f6270dee0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:25.071900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58476dee-8c0f-48df-a707-f620ef9dd07d tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:25.072721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58476dee-8c0f-48df-a707-f620ef9dd07d tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:25.085947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58476dee-8c0f-48df-a707-f620ef9dd07d tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:25.089843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58476dee-8c0f-48df-a707-f620ef9dd07d tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:25.090614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58476dee-8c0f-48df-a707-f620ef9dd07d tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:25.092749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58476dee-8c0f-48df-a707-f620ef9dd07d tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:25.098300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58476dee-8c0f-48df-a707-f620ef9dd07d tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:25.100672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58476dee-8c0f-48df-a707-f620ef9dd07d tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:25.104144 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0d03441b-e8ab-4813-9bd9-e46ec796869e tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] 10.222.0.26 "GET /compute/v2.1/os-volumes" status: 200 len: 1015 microversion: 2.1 time: 0.175194 Sep 27 10:11:25.115643 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1686/3713] 10.222.0.26 () {58 vars in 1134 bytes} [Tue Sep 27 10:11:24 2022] GET /compute/v2.1/os-volumes => generated 1015 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:25.123487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e2ef9495-c86a-46bb-b506-0ecf049ab3d5 tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] Calling method '.version_select at 0x7f7f6271ff70>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:25.317519 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e2ef9495-c86a-46bb-b506-0ecf049ab3d5 tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] 10.222.0.26 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 681 microversion: 2.1 time: 0.198884 Sep 27 10:11:25.324632 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1687/3714] 10.222.0.26 () {58 vars in 1149 bytes} [Tue Sep 27 10:11:25 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 681 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:11:25.331999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-733ee628-3bdb-4998-979d-d2bb1591ca8b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:25.334267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-733ee628-3bdb-4998-979d-d2bb1591ca8b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:25.341626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733ee628-3bdb-4998-979d-d2bb1591ca8b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:25.342549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733ee628-3bdb-4998-979d-d2bb1591ca8b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:25.343706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733ee628-3bdb-4998-979d-d2bb1591ca8b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:25.344605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733ee628-3bdb-4998-979d-d2bb1591ca8b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:25.345256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733ee628-3bdb-4998-979d-d2bb1591ca8b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:25.345850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733ee628-3bdb-4998-979d-d2bb1591ca8b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:25.479621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58476dee-8c0f-48df-a707-f620ef9dd07d tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:25.496304 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58476dee-8c0f-48df-a707-f620ef9dd07d tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.427744 Sep 27 10:11:25.506702 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2028/3715] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:25 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:25.514706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d491c7d-7356-4c2e-bf89-72f9914017cb tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] Calling method '.version_select at 0x7f7f626c9820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:25.577057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-733ee628-3bdb-4998-979d-d2bb1591ca8b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:25.588573 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-733ee628-3bdb-4998-979d-d2bb1591ca8b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1313 microversion: 2.1 time: 0.261912 Sep 27 10:11:25.589683 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1688/3716] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:25 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1313 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:25.595122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:25.596051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] [instance: 451a2ed4-e57c-46fc-8e29-09ad474cfa9d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:25.619811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:25.621208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:25.622004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:25.622963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:25.623635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:25.624072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:25.624736 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d491c7d-7356-4c2e-bf89-72f9914017cb tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] 10.222.0.26 "GET /compute/v2.1/os-volumes" status: 200 len: 1015 microversion: 2.1 time: 0.114403 Sep 27 10:11:25.625821 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2029/3717] 10.222.0.26 () {58 vars in 1134 bytes} [Tue Sep 27 10:11:25 2022] GET /compute/v2.1/os-volumes => generated 1015 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:25.641241 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] HTTP exception thrown: Instance 451a2ed4-e57c-46fc-8e29-09ad474cfa9d could not be found. Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Request method failure captured: Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d HTTP/1.1 Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsv_XrWIKtexahZQVBxYd97F8ehIUcU8IEgaDFrN5ira3DDEA6_WuehdCNv7Li9FErgVjchFMZKhKOGL5ZfP4sCi1HBLshVQnyUTfkDkk2x7Xiitgocnvqj9R9ZnNXuwdd8iHEYt9y6SK424rDHHVerfVr2G2VRkPND-pmyHs0Ofs7s Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: a74a11e81faa4ca59f7c5662a3e8321f Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersTestJSON-1683068214 Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a74a11e81faa4ca59f7c5662a3e8321f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a74a11e81faa4ca59f7c5662a3e8321f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a74a11e81faa4ca59f7c5662a3e8321f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a74a11e81faa4ca59f7c5662a3e8321f"}]}] Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersTestJSON-1683068214 Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: a74a11e81faa4ca59f7c5662a3e8321f Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersTestJSON-1683068214 Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersTestJSON-1683068214-project Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a83af7f06aa94823989c466abc0ffa3e Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersTestJSON-1683068214-project Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 451a2ed4-e57c-46fc-8e29-09ad474cfa9d could not be found. Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '451a2ed4-e57c-46fc-8e29-09ad474cfa9d'} Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 451a2ed4-e57c-46fc-8e29-09ad474cfa9d could not be found. Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 451a2ed4-e57c-46fc-8e29-09ad474cfa9d could not be found. Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:25.642951 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:25.647922 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:25.647922 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:25.647922 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:25.647922 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:25.647922 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:25.647922 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 451a2ed4-e57c-46fc-8e29-09ad474cfa9d could not be found. Sep 27 10:11:25.647922 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:25.647922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Returning 404 to user: Instance 451a2ed4-e57c-46fc-8e29-09ad474cfa9d could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:25.647922 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d2f79b97-4e92-46c8-bf6e-cb5725a05978 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d" status: 404 len: 111 microversion: 2.1 time: 0.053272 Sep 27 10:11:25.647922 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1689/3718] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:25 2022] GET /compute/v2.1/servers/451a2ed4-e57c-46fc-8e29-09ad474cfa9d => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:25.650222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eb8a33df-8e59-4f31-8d86-89f3997e1a0c tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] Calling method '.version_select at 0x7f7f6283b3a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:25.655067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d68f6458-34f2-4699-a254-b528713e7af7 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] Calling method '.version_select at 0x7f7f626d8d30>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:25.686324 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d68f6458-34f2-4699-a254-b528713e7af7 tempest-ServersTestJSON-1683068214 tempest-ServersTestJSON-1683068214-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-790039360" status: 202 len: 0 microversion: 2.1 time: 0.036236 Sep 27 10:11:25.690136 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1690/3719] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:11:25 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-790039360 => generated 0 bytes in 38 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:11:25.712025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:25.715565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:25.740892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:25.741865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:25.745025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:25.746019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:25.747735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:25.748409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:25.773744 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eb8a33df-8e59-4f31-8d86-89f3997e1a0c tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] 10.222.0.26 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 347 microversion: 2.1 time: 0.135196 Sep 27 10:11:25.774123 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2030/3720] 10.222.0.26 () {58 vars in 1167 bytes} [Tue Sep 27 10:11:25 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 347 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:11:25.793564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-66bd7a60-96e2-4e2e-b8d1-2a3ee4bd124b tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] Calling method '.version_select at 0x7f7f6283bd30>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:25.970065 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-66bd7a60-96e2-4e2e-b8d1-2a3ee4bd124b tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 681 microversion: 2.1 time: 0.180681 Sep 27 10:11:25.980830 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2031/3721] 10.222.0.26 () {58 vars in 1170 bytes} [Tue Sep 27 10:11:25 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 681 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:25.987270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-82930e54-9ed9-4a77-a85c-e159dd251d07 req-523bea72-9053-4a2e-a00d-4e7c7c435a60 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "87e69339-c527-44d1-8566-ce64d503d7a6", "name": "network-vif-plugged", "status": "completed", "tag": "eaf55525-ffa8-40a5-9a88-b2f94df3274b"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:26.002655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-82930e54-9ed9-4a77-a85c-e159dd251d07 req-523bea72-9053-4a2e-a00d-4e7c7c435a60 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:26.004501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-82930e54-9ed9-4a77-a85c-e159dd251d07 req-523bea72-9053-4a2e-a00d-4e7c7c435a60 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:26.010836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-82930e54-9ed9-4a77-a85c-e159dd251d07 req-523bea72-9053-4a2e-a00d-4e7c7c435a60 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:26.053800 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-82930e54-9ed9-4a77-a85c-e159dd251d07 req-523bea72-9053-4a2e-a00d-4e7c7c435a60 service nova] Creating event network-vif-plugged:eaf55525-ffa8-40a5-9a88-b2f94df3274b for instance 87e69339-c527-44d1-8566-ce64d503d7a6 on nch1-master-1 Sep 27 10:11:26.069359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:26.072214 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-82930e54-9ed9-4a77-a85c-e159dd251d07 req-523bea72-9053-4a2e-a00d-4e7c7c435a60 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082598 Sep 27 10:11:26.074122 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2032/3722] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:26.081682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:26.086642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6b175a48-daa7-409c-bbfd-97cc21868f25 tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] Calling method '.version_select at 0x7f7f62fdb9d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:26.100132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:26.118715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:26.120775 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-463f42a3-971e-4297-b04d-730c7b255b41 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.431982 Sep 27 10:11:26.121358 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1691/3723] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:25 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:26.225153 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6b175a48-daa7-409c-bbfd-97cc21868f25 tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1015 microversion: 2.1 time: 0.149510 Sep 27 10:11:26.226295 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2033/3724] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 10:11:26 2022] GET /compute/v2.1/os-volumes/detail => generated 1015 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:26.240423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-92b01521-a183-4008-aaa4-daff542d5ac9 tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] Calling method '.version_select at 0x7f7f636293a0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:26.373866 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-92b01521-a183-4008-aaa4-daff542d5ac9 tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 347 microversion: 2.1 time: 0.137621 Sep 27 10:11:26.374495 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1692/3725] 10.222.0.26 () {58 vars in 1188 bytes} [Tue Sep 27 10:11:26 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 347 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:26.389177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-719e2b07-24cb-4695-bf27-eb371b77f79c tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] Calling method '.version_select at 0x7f7f62551a60>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:26.518486 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-719e2b07-24cb-4695-bf27-eb371b77f79c tempest-VolumesTestJSON-50992764 tempest-VolumesTestJSON-50992764-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1015 microversion: 2.1 time: 0.133430 Sep 27 10:11:26.520686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cbde947a-42e7-4bc9-ad73-dff66ad39e25 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:26.521549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cbde947a-42e7-4bc9-ad73-dff66ad39e25 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:26.528577 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2034/3726] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 10:11:26 2022] GET /compute/v2.1/os-volumes/detail => generated 1015 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:26.543125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbde947a-42e7-4bc9-ad73-dff66ad39e25 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:26.548913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbde947a-42e7-4bc9-ad73-dff66ad39e25 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:26.549735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbde947a-42e7-4bc9-ad73-dff66ad39e25 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:26.550715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbde947a-42e7-4bc9-ad73-dff66ad39e25 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:26.551614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbde947a-42e7-4bc9-ad73-dff66ad39e25 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:26.554144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbde947a-42e7-4bc9-ad73-dff66ad39e25 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:26.615905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-16ecba65-fd1a-4bf2-8223-1caf3485f352 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:26.617253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-16ecba65-fd1a-4bf2-8223-1caf3485f352 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:26.642397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ecba65-fd1a-4bf2-8223-1caf3485f352 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:26.643120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ecba65-fd1a-4bf2-8223-1caf3485f352 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:26.643858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ecba65-fd1a-4bf2-8223-1caf3485f352 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:26.645547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ecba65-fd1a-4bf2-8223-1caf3485f352 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:26.646355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ecba65-fd1a-4bf2-8223-1caf3485f352 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:26.646992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16ecba65-fd1a-4bf2-8223-1caf3485f352 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:26.955729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-16ecba65-fd1a-4bf2-8223-1caf3485f352 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:26.965910 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-16ecba65-fd1a-4bf2-8223-1caf3485f352 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1313 microversion: 2.1 time: 0.354457 Sep 27 10:11:26.967080 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2035/3727] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:26 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1313 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:27.012290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:27.114060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=20 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:27.162699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:27.165622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:27.216555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:27.217756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:27.220224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:27.221744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:27.227210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:27.228177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:27.345059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cbde947a-42e7-4bc9-ad73-dff66ad39e25 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:27.362612 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cbde947a-42e7-4bc9-ad73-dff66ad39e25 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.845932 Sep 27 10:11:27.364389 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1693/3728] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:26 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 848 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:27.902927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:27.922108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:27.923963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:27.925447 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aaa02f0c-ad84-4a61-9f88-234e799726d1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.778516 Sep 27 10:11:27.925990 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2036/3729] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:27 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 782 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:11:27.994953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-716298af-5e2d-460e-9880-7ef267757b2f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:27.996232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-716298af-5e2d-460e-9880-7ef267757b2f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:28.007866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-716298af-5e2d-460e-9880-7ef267757b2f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:28.011533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-716298af-5e2d-460e-9880-7ef267757b2f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:28.013506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-716298af-5e2d-460e-9880-7ef267757b2f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:28.016124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-716298af-5e2d-460e-9880-7ef267757b2f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:28.021530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-716298af-5e2d-460e-9880-7ef267757b2f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:28.022269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-716298af-5e2d-460e-9880-7ef267757b2f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:28.139107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8163faf6-2c99-4ee7-bc5a-41b3e0b18598 req-18f2c4ab-2512-4407-8cb5-eaff6e10ce02 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-plugged", "status": "completed", "tag": "dbf7486a-85d8-4b37-a8d2-b1f467142580"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:28.155544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8163faf6-2c99-4ee7-bc5a-41b3e0b18598 req-18f2c4ab-2512-4407-8cb5-eaff6e10ce02 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:28.156444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8163faf6-2c99-4ee7-bc5a-41b3e0b18598 req-18f2c4ab-2512-4407-8cb5-eaff6e10ce02 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:28.157496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8163faf6-2c99-4ee7-bc5a-41b3e0b18598 req-18f2c4ab-2512-4407-8cb5-eaff6e10ce02 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:28.188572 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8163faf6-2c99-4ee7-bc5a-41b3e0b18598 req-18f2c4ab-2512-4407-8cb5-eaff6e10ce02 service nova] Creating event network-vif-plugged:dbf7486a-85d8-4b37-a8d2-b1f467142580 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:11:28.207858 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8163faf6-2c99-4ee7-bc5a-41b3e0b18598 req-18f2c4ab-2512-4407-8cb5-eaff6e10ce02 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069009 Sep 27 10:11:28.215811 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2037/3730] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:11:28.308225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-716298af-5e2d-460e-9880-7ef267757b2f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:28.318314 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-716298af-5e2d-460e-9880-7ef267757b2f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1498 microversion: 2.1 time: 0.327975 Sep 27 10:11:28.320807 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1694/3731] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:27 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1498 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:28.364499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-47a41ef1-9290-4592-86ea-e5fff6f27f1f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:28.370321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-47a41ef1-9290-4592-86ea-e5fff6f27f1f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:28.386801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a41ef1-9290-4592-86ea-e5fff6f27f1f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:28.390627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5f97a07-1e5c-45b1-b3eb-f5a9e2eb9856 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:28.391587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5f97a07-1e5c-45b1-b3eb-f5a9e2eb9856 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:28.391821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a41ef1-9290-4592-86ea-e5fff6f27f1f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:28.392152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a41ef1-9290-4592-86ea-e5fff6f27f1f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:28.403684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f97a07-1e5c-45b1-b3eb-f5a9e2eb9856 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:28.405423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a41ef1-9290-4592-86ea-e5fff6f27f1f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:28.405770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f97a07-1e5c-45b1-b3eb-f5a9e2eb9856 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:28.406008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a41ef1-9290-4592-86ea-e5fff6f27f1f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:28.406210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f97a07-1e5c-45b1-b3eb-f5a9e2eb9856 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:28.406404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47a41ef1-9290-4592-86ea-e5fff6f27f1f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:28.412315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f97a07-1e5c-45b1-b3eb-f5a9e2eb9856 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:28.413289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f97a07-1e5c-45b1-b3eb-f5a9e2eb9856 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:28.413973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5f97a07-1e5c-45b1-b3eb-f5a9e2eb9856 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:28.493403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-47a41ef1-9290-4592-86ea-e5fff6f27f1f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lazy-loading 'flavor' on Instance uuid 87e69339-c527-44d1-8566-ce64d503d7a6 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:11:28.638723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5f97a07-1e5c-45b1-b3eb-f5a9e2eb9856 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:28.664692 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5f97a07-1e5c-45b1-b3eb-f5a9e2eb9856 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.279251 Sep 27 10:11:28.668596 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1695/3732] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:28 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:28.671032 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-47a41ef1-9290-4592-86ea-e5fff6f27f1f tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "POST /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6/action" status: 200 len: 32 microversion: 2.1 time: 0.324538 Sep 27 10:11:28.671705 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2038/3733] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:11:28 2022] POST /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6/action => generated 32 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:11:28.682407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fc65b43c-d96e-4dc1-9916-45eab465ae21 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:28.683092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fc65b43c-d96e-4dc1-9916-45eab465ae21 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:28.707432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc65b43c-d96e-4dc1-9916-45eab465ae21 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:28.708934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc65b43c-d96e-4dc1-9916-45eab465ae21 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:28.712945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc65b43c-d96e-4dc1-9916-45eab465ae21 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:28.715825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc65b43c-d96e-4dc1-9916-45eab465ae21 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:28.716702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc65b43c-d96e-4dc1-9916-45eab465ae21 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:28.717506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc65b43c-d96e-4dc1-9916-45eab465ae21 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:28.956433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:28.957207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:28.967943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:28.968814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:28.969509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:28.972044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:28.972608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:28.973579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:29.067196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fc65b43c-d96e-4dc1-9916-45eab465ae21 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:29.088186 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fc65b43c-d96e-4dc1-9916-45eab465ae21 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.408859 Sep 27 10:11:29.090864 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1696/3734] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:28 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:29.362730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:29.377555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:29.382473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:29.384356 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-29c38b09-5727-4d11-a23b-cfc01cdcaa65 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.434129 Sep 27 10:11:29.397510 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2039/3735] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:28 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:11:29.701544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9a983195-262d-4645-b4bc-36edd7f47934 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:29.702831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9a983195-262d-4645-b4bc-36edd7f47934 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:29.717463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a983195-262d-4645-b4bc-36edd7f47934 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:29.717988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a983195-262d-4645-b4bc-36edd7f47934 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:29.718416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a983195-262d-4645-b4bc-36edd7f47934 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:29.719134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a983195-262d-4645-b4bc-36edd7f47934 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:29.719605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a983195-262d-4645-b4bc-36edd7f47934 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:29.720018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a983195-262d-4645-b4bc-36edd7f47934 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:30.116175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3decdc2c-9d58-4707-bc8f-b8aef8bfbf03 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:30.116935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3decdc2c-9d58-4707-bc8f-b8aef8bfbf03 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:30.137992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3decdc2c-9d58-4707-bc8f-b8aef8bfbf03 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:30.138698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3decdc2c-9d58-4707-bc8f-b8aef8bfbf03 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:30.139157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3decdc2c-9d58-4707-bc8f-b8aef8bfbf03 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:30.139910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3decdc2c-9d58-4707-bc8f-b8aef8bfbf03 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:30.140371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3decdc2c-9d58-4707-bc8f-b8aef8bfbf03 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:30.140831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3decdc2c-9d58-4707-bc8f-b8aef8bfbf03 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:30.390434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9a983195-262d-4645-b4bc-36edd7f47934 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:30.403961 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9a983195-262d-4645-b4bc-36edd7f47934 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.709842 Sep 27 10:11:30.406394 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1697/3736] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:29 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:30.421389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-11e0e471-2f23-4072-b766-31375b62360a req-3bc86c3c-af68-4bca-b7e5-dd86f66d54df service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-plugged", "status": "completed", "tag": "2b32bc4b-33c7-4c6b-918d-8458f3eee387"}, {"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-plugged", "status": "completed", "tag": "bcc2c185-ff79-4bda-a3f2-d915e7d3752c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:30.445070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-11e0e471-2f23-4072-b766-31375b62360a req-3bc86c3c-af68-4bca-b7e5-dd86f66d54df service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:30.447537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-11e0e471-2f23-4072-b766-31375b62360a req-3bc86c3c-af68-4bca-b7e5-dd86f66d54df service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:30.448175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-11e0e471-2f23-4072-b766-31375b62360a req-3bc86c3c-af68-4bca-b7e5-dd86f66d54df service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:30.466895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3decdc2c-9d58-4707-bc8f-b8aef8bfbf03 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:30.475099 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-11e0e471-2f23-4072-b766-31375b62360a req-3bc86c3c-af68-4bca-b7e5-dd86f66d54df service nova] Creating event network-vif-plugged:2b32bc4b-33c7-4c6b-918d-8458f3eee387 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:11:30.480467 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-11e0e471-2f23-4072-b766-31375b62360a req-3bc86c3c-af68-4bca-b7e5-dd86f66d54df service nova] Creating event network-vif-plugged:bcc2c185-ff79-4bda-a3f2-d915e7d3752c for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:11:30.481450 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3decdc2c-9d58-4707-bc8f-b8aef8bfbf03 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.369662 Sep 27 10:11:30.499279 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2040/3737] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:30 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:30.501230 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-11e0e471-2f23-4072-b766-31375b62360a req-3bc86c3c-af68-4bca-b7e5-dd86f66d54df service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.090651 Sep 27 10:11:30.502275 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1698/3738] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:30 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:30.506749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:30.507810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:30.517519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:30.518219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:30.519609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:30.520531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:30.521286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:30.522080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:30.977764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:30.994636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:30.996115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:30.997396 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7587924f-69eb-430d-b324-1912fff2aa5b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.496945 Sep 27 10:11:30.997975 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2041/3739] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:30 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:31.435580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af79be18-6c45-449c-a059-086fe80ad1b9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:31.436286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-af79be18-6c45-449c-a059-086fe80ad1b9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:31.451949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af79be18-6c45-449c-a059-086fe80ad1b9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:31.452899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af79be18-6c45-449c-a059-086fe80ad1b9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:31.453522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af79be18-6c45-449c-a059-086fe80ad1b9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:31.454424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af79be18-6c45-449c-a059-086fe80ad1b9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:31.455068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af79be18-6c45-449c-a059-086fe80ad1b9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:31.455804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af79be18-6c45-449c-a059-086fe80ad1b9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:31.509886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d94f22f5-2286-40f7-8af3-514e6c460a53 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:31.511100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d94f22f5-2286-40f7-8af3-514e6c460a53 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:31.525543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d94f22f5-2286-40f7-8af3-514e6c460a53 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:31.526355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d94f22f5-2286-40f7-8af3-514e6c460a53 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:31.527350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d94f22f5-2286-40f7-8af3-514e6c460a53 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:31.528395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d94f22f5-2286-40f7-8af3-514e6c460a53 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:31.529065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d94f22f5-2286-40f7-8af3-514e6c460a53 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:31.529659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d94f22f5-2286-40f7-8af3-514e6c460a53 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:31.774022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d94f22f5-2286-40f7-8af3-514e6c460a53 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:31.785651 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d94f22f5-2286-40f7-8af3-514e6c460a53 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.280850 Sep 27 10:11:31.787356 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2042/3740] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:31 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:32.027636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:32.028272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:32.040541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:32.041285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:32.041912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:32.042647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:32.043517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:32.043517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:32.086212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:32.380095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:32.392053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:32.393776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:32.395225 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da0b5b9c-2e56-4cb5-b95f-62e8bb55dee3 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.371634 Sep 27 10:11:32.409548 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2043/3741] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:32 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:11:32.584137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af79be18-6c45-449c-a059-086fe80ad1b9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:32.617037 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af79be18-6c45-449c-a059-086fe80ad1b9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 1.185631 Sep 27 10:11:32.618551 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1699/3742] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:31 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 1189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:32.684773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-67c22f79-1499-4420-9f5a-e5ce6408880a req-1195105c-61b5-415c-97a5-a6a0be347dd6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-plugged", "status": "completed", "tag": "5c514831-defe-416f-aec5-712e750a0e3b"}, {"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-plugged", "status": "completed", "tag": "73444bba-797d-4fc6-88a3-e126deb3c605"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:32.699285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-67c22f79-1499-4420-9f5a-e5ce6408880a req-1195105c-61b5-415c-97a5-a6a0be347dd6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:32.700326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-67c22f79-1499-4420-9f5a-e5ce6408880a req-1195105c-61b5-415c-97a5-a6a0be347dd6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:32.701040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-67c22f79-1499-4420-9f5a-e5ce6408880a req-1195105c-61b5-415c-97a5-a6a0be347dd6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:32.721167 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-67c22f79-1499-4420-9f5a-e5ce6408880a req-1195105c-61b5-415c-97a5-a6a0be347dd6 service nova] Creating event network-vif-plugged:5c514831-defe-416f-aec5-712e750a0e3b for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:11:32.722980 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-67c22f79-1499-4420-9f5a-e5ce6408880a req-1195105c-61b5-415c-97a5-a6a0be347dd6 service nova] Creating event network-vif-plugged:73444bba-797d-4fc6-88a3-e126deb3c605 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:11:32.730054 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-67c22f79-1499-4420-9f5a-e5ce6408880a req-1195105c-61b5-415c-97a5-a6a0be347dd6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.050475 Sep 27 10:11:32.731375 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2044/3743] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:32 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:32.821754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-285051cb-330d-451a-bb9c-42b0ef440dfa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:32.827380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-285051cb-330d-451a-bb9c-42b0ef440dfa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:32.839834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-285051cb-330d-451a-bb9c-42b0ef440dfa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:32.841472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-285051cb-330d-451a-bb9c-42b0ef440dfa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:32.842142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-285051cb-330d-451a-bb9c-42b0ef440dfa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:32.843123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-285051cb-330d-451a-bb9c-42b0ef440dfa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:32.843806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-285051cb-330d-451a-bb9c-42b0ef440dfa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:32.844438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-285051cb-330d-451a-bb9c-42b0ef440dfa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:33.250474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-285051cb-330d-451a-bb9c-42b0ef440dfa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:33.262360 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-285051cb-330d-451a-bb9c-42b0ef440dfa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.456012 Sep 27 10:11:33.265645 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1700/3744] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:32 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:33.425106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:33.425943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:33.439761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:33.441481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:33.443811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:33.446464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:33.447193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:33.448372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:33.667439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-70b626f5-0082-4f1c-8d18-61b558447755 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:33.669979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-70b626f5-0082-4f1c-8d18-61b558447755 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:33.684942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70b626f5-0082-4f1c-8d18-61b558447755 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:33.687231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70b626f5-0082-4f1c-8d18-61b558447755 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:33.689519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70b626f5-0082-4f1c-8d18-61b558447755 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:33.690539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70b626f5-0082-4f1c-8d18-61b558447755 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:33.692041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70b626f5-0082-4f1c-8d18-61b558447755 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:33.695016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70b626f5-0082-4f1c-8d18-61b558447755 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:34.351841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-70b626f5-0082-4f1c-8d18-61b558447755 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:34.360805 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-70b626f5-0082-4f1c-8d18-61b558447755 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.719868 Sep 27 10:11:34.364850 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1701/3745] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:33 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 725 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:34.383899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0addf1c7-5b19-4485-b1c9-82a25df2039d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:34.386709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0addf1c7-5b19-4485-b1c9-82a25df2039d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:34.405720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0addf1c7-5b19-4485-b1c9-82a25df2039d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:34.406535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0addf1c7-5b19-4485-b1c9-82a25df2039d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:34.407142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0addf1c7-5b19-4485-b1c9-82a25df2039d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:34.408157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0addf1c7-5b19-4485-b1c9-82a25df2039d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:34.408843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0addf1c7-5b19-4485-b1c9-82a25df2039d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:34.409497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0addf1c7-5b19-4485-b1c9-82a25df2039d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:34.874573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:34.894069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:34.900023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:34.902336 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c63770c8-db8c-4d36-b106-4edd1e26b866 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 1.482979 Sep 27 10:11:34.904908 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2045/3746] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:33 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 1487 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:34.917788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ad4cccbd-b88f-4f1e-9358-feb30c966e13 req-cba7ef42-1fae-4dc6-b89d-5925846abdab service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-plugged", "status": "completed", "tag": "ee0efbef-fa4b-4072-b28f-b4422ccb9afa"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:34.951309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ad4cccbd-b88f-4f1e-9358-feb30c966e13 req-cba7ef42-1fae-4dc6-b89d-5925846abdab service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:34.953628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ad4cccbd-b88f-4f1e-9358-feb30c966e13 req-cba7ef42-1fae-4dc6-b89d-5925846abdab service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:34.955113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ad4cccbd-b88f-4f1e-9358-feb30c966e13 req-cba7ef42-1fae-4dc6-b89d-5925846abdab service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:35.016953 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ad4cccbd-b88f-4f1e-9358-feb30c966e13 req-cba7ef42-1fae-4dc6-b89d-5925846abdab service nova] Creating event network-vif-plugged:ee0efbef-fa4b-4072-b28f-b4422ccb9afa for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:11:35.050105 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ad4cccbd-b88f-4f1e-9358-feb30c966e13 req-cba7ef42-1fae-4dc6-b89d-5925846abdab service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.138335 Sep 27 10:11:35.055052 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2046/3747] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:35.388702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e902ad47-e54d-47fc-89c0-91be95e7dd78 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:35.389844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e902ad47-e54d-47fc-89c0-91be95e7dd78 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:35.408604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e902ad47-e54d-47fc-89c0-91be95e7dd78 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:35.409169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e902ad47-e54d-47fc-89c0-91be95e7dd78 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:35.411842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e902ad47-e54d-47fc-89c0-91be95e7dd78 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:35.415172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e902ad47-e54d-47fc-89c0-91be95e7dd78 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:35.415865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e902ad47-e54d-47fc-89c0-91be95e7dd78 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:35.419004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e902ad47-e54d-47fc-89c0-91be95e7dd78 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:35.630433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0addf1c7-5b19-4485-b1c9-82a25df2039d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:35.650141 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0addf1c7-5b19-4485-b1c9-82a25df2039d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 1.272140 Sep 27 10:11:35.664770 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1702/3748] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:34 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 1285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:35.847816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e902ad47-e54d-47fc-89c0-91be95e7dd78 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:35.858624 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e902ad47-e54d-47fc-89c0-91be95e7dd78 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.474568 Sep 27 10:11:35.861588 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2047/3749] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:35 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:35.937926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:35.938501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:35.957463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:35.957976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:35.958393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:35.959139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:35.959573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:35.960176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:36.435892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:36.451678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:36.453559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:36.456944 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fa56f751-4f34-472b-b8f9-26846dbad210 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.523730 Sep 27 10:11:36.486426 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1703/3750] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:35 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:36.678366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8656c188-6019-49e8-8c56-72c81a0b6c80 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:36.679087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8656c188-6019-49e8-8c56-72c81a0b6c80 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:36.696496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8656c188-6019-49e8-8c56-72c81a0b6c80 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:36.697304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8656c188-6019-49e8-8c56-72c81a0b6c80 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:36.697924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8656c188-6019-49e8-8c56-72c81a0b6c80 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:36.698863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8656c188-6019-49e8-8c56-72c81a0b6c80 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:36.700997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8656c188-6019-49e8-8c56-72c81a0b6c80 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:36.702073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8656c188-6019-49e8-8c56-72c81a0b6c80 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:36.890340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-59a5dcb2-3591-4080-96a0-1f0c38f1e6b8 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:36.891019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-59a5dcb2-3591-4080-96a0-1f0c38f1e6b8 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:36.911308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59a5dcb2-3591-4080-96a0-1f0c38f1e6b8 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:36.911308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59a5dcb2-3591-4080-96a0-1f0c38f1e6b8 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:36.911498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59a5dcb2-3591-4080-96a0-1f0c38f1e6b8 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:36.912116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59a5dcb2-3591-4080-96a0-1f0c38f1e6b8 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:36.912849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59a5dcb2-3591-4080-96a0-1f0c38f1e6b8 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:36.915151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59a5dcb2-3591-4080-96a0-1f0c38f1e6b8 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:37.260807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8656c188-6019-49e8-8c56-72c81a0b6c80 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:37.280421 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8656c188-6019-49e8-8c56-72c81a0b6c80 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.607011 Sep 27 10:11:37.281428 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2048/3751] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:36 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:37.310173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-929c576d-bac3-455c-b35d-5bf1a07b8c01 req-9af5b1a1-196e-4076-9a4d-4e84a7b47180 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ad21db4-8d06-48e5-b469-be50b03f701a", "name": "network-vif-plugged", "status": "completed", "tag": "55eb0fa6-bb39-46b7-889c-e034fd5114c2"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:37.311762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-59a5dcb2-3591-4080-96a0-1f0c38f1e6b8 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:37.321308 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-59a5dcb2-3591-4080-96a0-1f0c38f1e6b8 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.435848 Sep 27 10:11:37.333336 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1704/3752] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:36 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:37.348719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-929c576d-bac3-455c-b35d-5bf1a07b8c01 req-9af5b1a1-196e-4076-9a4d-4e84a7b47180 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:37.349675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-929c576d-bac3-455c-b35d-5bf1a07b8c01 req-9af5b1a1-196e-4076-9a4d-4e84a7b47180 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:37.351919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-929c576d-bac3-455c-b35d-5bf1a07b8c01 req-9af5b1a1-196e-4076-9a4d-4e84a7b47180 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:37.395675 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-929c576d-bac3-455c-b35d-5bf1a07b8c01 req-9af5b1a1-196e-4076-9a4d-4e84a7b47180 service nova] Creating event network-vif-plugged:55eb0fa6-bb39-46b7-889c-e034fd5114c2 for instance 2ad21db4-8d06-48e5-b469-be50b03f701a on nch2-master-1 Sep 27 10:11:37.412079 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-929c576d-bac3-455c-b35d-5bf1a07b8c01 req-9af5b1a1-196e-4076-9a4d-4e84a7b47180 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.129081 Sep 27 10:11:37.414130 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2049/3753] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:37.487853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:37.488490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:37.504416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:37.504966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:37.507180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:37.507950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:37.508425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:37.508901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:37.978072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:37.993462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:37.997844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:38.003679 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9338ed2a-844c-4f22-9dfa-52e0a725848b tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1875 microversion: 2.42 time: 0.519925 Sep 27 10:11:38.004501 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1705/3754] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:37 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1875 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:38.305586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7fa1b8f8-5313-47f8-abdc-dff7ee3d1407 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:38.306466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7fa1b8f8-5313-47f8-abdc-dff7ee3d1407 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:38.314150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1b8f8-5313-47f8-abdc-dff7ee3d1407 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:38.314960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1b8f8-5313-47f8-abdc-dff7ee3d1407 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:38.315611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1b8f8-5313-47f8-abdc-dff7ee3d1407 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:38.317018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1b8f8-5313-47f8-abdc-dff7ee3d1407 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:38.317722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1b8f8-5313-47f8-abdc-dff7ee3d1407 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:38.318364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fa1b8f8-5313-47f8-abdc-dff7ee3d1407 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:38.352152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-437bcd6f-b699-4a87-8798-d18466ace56a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:38.353332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-437bcd6f-b699-4a87-8798-d18466ace56a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:38.373212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437bcd6f-b699-4a87-8798-d18466ace56a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:38.374201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437bcd6f-b699-4a87-8798-d18466ace56a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:38.374576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437bcd6f-b699-4a87-8798-d18466ace56a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:38.377412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437bcd6f-b699-4a87-8798-d18466ace56a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:38.378084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437bcd6f-b699-4a87-8798-d18466ace56a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:38.379258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437bcd6f-b699-4a87-8798-d18466ace56a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:38.573877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7fa1b8f8-5313-47f8-abdc-dff7ee3d1407 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:38.582566 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7fa1b8f8-5313-47f8-abdc-dff7ee3d1407 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.281250 Sep 27 10:11:38.583496 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2050/3755] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:38 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:38.652020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-437bcd6f-b699-4a87-8798-d18466ace56a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:38.662444 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-437bcd6f-b699-4a87-8798-d18466ace56a tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1324 microversion: 2.1 time: 0.316008 Sep 27 10:11:38.676986 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1706/3756] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:38 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1324 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:39.046913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:39.049809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:39.070996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:39.073272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:39.073920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:39.074693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:39.076700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:39.084678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:39.451674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:39.476377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:39.482547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:39.487371 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-59b38170-f458-4b1e-a559-8974df0a9e24 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2832 microversion: 2.42 time: 0.457107 Sep 27 10:11:39.488521 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2051/3757] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:11:39 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2832 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:39.517485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ec49674-ba16-415b-ac17-46e9bfb3854e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:39.518628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ec49674-ba16-415b-ac17-46e9bfb3854e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:39.538130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec49674-ba16-415b-ac17-46e9bfb3854e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:39.543346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec49674-ba16-415b-ac17-46e9bfb3854e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:39.544118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec49674-ba16-415b-ac17-46e9bfb3854e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:39.547785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec49674-ba16-415b-ac17-46e9bfb3854e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:39.557819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec49674-ba16-415b-ac17-46e9bfb3854e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:39.558455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ec49674-ba16-415b-ac17-46e9bfb3854e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:39.611076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c51732c4-b6cf-4940-9ff0-fea4f23b224c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:39.612292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c51732c4-b6cf-4940-9ff0-fea4f23b224c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:39.625903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c51732c4-b6cf-4940-9ff0-fea4f23b224c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:39.627255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c51732c4-b6cf-4940-9ff0-fea4f23b224c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:39.628074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c51732c4-b6cf-4940-9ff0-fea4f23b224c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:39.632714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c51732c4-b6cf-4940-9ff0-fea4f23b224c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:39.633535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c51732c4-b6cf-4940-9ff0-fea4f23b224c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:39.633983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c51732c4-b6cf-4940-9ff0-fea4f23b224c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:39.846626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c51732c4-b6cf-4940-9ff0-fea4f23b224c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:39.857795 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c51732c4-b6cf-4940-9ff0-fea4f23b224c tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.252759 Sep 27 10:11:39.858516 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2052/3758] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:39 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:39.864452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-48861bf3-42f5-4edb-85ef-6de9ce12702e tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:39.865221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-48861bf3-42f5-4edb-85ef-6de9ce12702e tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:39.872346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48861bf3-42f5-4edb-85ef-6de9ce12702e tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:39.872941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48861bf3-42f5-4edb-85ef-6de9ce12702e tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:39.873749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48861bf3-42f5-4edb-85ef-6de9ce12702e tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:39.874683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48861bf3-42f5-4edb-85ef-6de9ce12702e tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:39.875378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48861bf3-42f5-4edb-85ef-6de9ce12702e tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:39.876302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48861bf3-42f5-4edb-85ef-6de9ce12702e tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:39.880649 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8ec49674-ba16-415b-ac17-46e9bfb3854e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b/os-interface" status: 200 len: 1831 microversion: 2.42 time: 0.367611 Sep 27 10:11:39.881323 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1707/3759] 10.222.0.26 () {60 vars in 1316 bytes} [Tue Sep 27 10:11:39 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b/os-interface => generated 1831 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:11:40.353996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-48861bf3-42f5-4edb-85ef-6de9ce12702e tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:40.363946 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-48861bf3-42f5-4edb-85ef-6de9ce12702e tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1519 microversion: 2.1 time: 0.503960 Sep 27 10:11:40.408541 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2053/3760] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:39 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1519 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:40.625244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe53e52c-4353-409b-912e-f89df4755940 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1767972726", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"": "data1"}, "networks": [{"uuid": "9c10d900-7e77-42f0-8e1b-f46336bf67ee"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe53e52c-4353-409b-912e-f89df4755940 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Request method failure captured: Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 243 Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsxArI2BrDRc_CFHMqHahDxz0SscM67RJwatpFvOsFPcb_7cW9eGCWj_T78Wq2MrI0SNXZkUQjV0dl_4-Odxa1JIz-DZjNE1TSToEaLsr5cEidhPthrnIxer4_dQPDtEx1Uy6izg17U34URr01sqNWP7JOLvN6_kaTi24Rfn7Mbe3vI Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_be55c1fb4e6f4870b25e986907d41101"}]}] Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 96e48f22b8e7478986f912e64df670b0 Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1767972726", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"": "data1"}, "networks": [{"uuid": "9c10d900-7e77-42f0-8e1b-f46336bf67ee"}]}} Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-1767972726', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'metadata': {'': 'data1'}, 'networks': [{'uuid': '9c10d900-7e77-42f0-8e1b-f46336bf67ee'}]}}} Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'object'} Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'': 'data1'} Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:40.634383 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.640000 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.643657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe53e52c-4353-409b-912e-f89df4755940 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'': 'data1'}. '' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:40.646692 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fe53e52c-4353-409b-912e-f89df4755940 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 175 microversion: 2.1 time: 0.025807 Sep 27 10:11:40.649502 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1708/3761] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:11:40 2022] POST /compute/v2.1/servers => generated 175 bytes in 30 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:11:40.661099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c23f93c-e184-457c-8695-2ba22a132c93 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:40.661736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5c23f93c-e184-457c-8695-2ba22a132c93 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 62baa93e-c594-46bd-8ece-3865856afdb7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:40.682408 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-5c23f93c-e184-457c-8695-2ba22a132c93 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] HTTP exception thrown: Instance 62baa93e-c594-46bd-8ece-3865856afdb7 could not be found. Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c23f93c-e184-457c-8695-2ba22a132c93 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Request method failure captured: Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/62baa93e-c594-46bd-8ece-3865856afdb7/metadata HTTP/1.1 Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsxArI2BrDRc_CFHMqHahDxz0SscM67RJwatpFvOsFPcb_7cW9eGCWj_T78Wq2MrI0SNXZkUQjV0dl_4-Odxa1JIz-DZjNE1TSToEaLsr5cEidhPthrnIxer4_dQPDtEx1Uy6izg17U34URr01sqNWP7JOLvN6_kaTi24Rfn7Mbe3vI Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_be55c1fb4e6f4870b25e986907d41101"}]}] Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 96e48f22b8e7478986f912e64df670b0 Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 62baa93e-c594-46bd-8ece-3865856afdb7 could not be found. Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'server_id': '62baa93e-c594-46bd-8ece-3865856afdb7'} Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 62baa93e-c594-46bd-8ece-3865856afdb7 could not be found. Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_metadata.py", line 54, in index Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, server_id) Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 62baa93e-c594-46bd-8ece-3865856afdb7 could not be found. Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:40.698150 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:40.703253 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:40.703253 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:40.703253 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 62baa93e-c594-46bd-8ece-3865856afdb7 could not be found. Sep 27 10:11:40.703253 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.703253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c23f93c-e184-457c-8695-2ba22a132c93 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Returning 404 to user: Instance 62baa93e-c594-46bd-8ece-3865856afdb7 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:40.703253 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c23f93c-e184-457c-8695-2ba22a132c93 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/62baa93e-c594-46bd-8ece-3865856afdb7/metadata" status: 404 len: 111 microversion: 2.1 time: 0.043251 Sep 27 10:11:40.712632 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2054/3762] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:11:40 2022] GET /compute/v2.1/servers/62baa93e-c594-46bd-8ece-3865856afdb7/metadata => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:40.786169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5be59ac0-cd5e-4bbc-b067-a0de20854782 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1294435696", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "9c10d900-7e77-42f0-8e1b-f46336bf67ee"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5be59ac0-cd5e-4bbc-b067-a0de20854782 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Request method failure captured: Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 499 Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsxArI2BrDRc_CFHMqHahDxz0SscM67RJwatpFvOsFPcb_7cW9eGCWj_T78Wq2MrI0SNXZkUQjV0dl_4-Odxa1JIz-DZjNE1TSToEaLsr5cEidhPthrnIxer4_dQPDtEx1Uy6izg17U34URr01sqNWP7JOLvN6_kaTi24Rfn7Mbe3vI Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_be55c1fb4e6f4870b25e986907d41101"}]}] Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 96e48f22b8e7478986f912e64df670b0 Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1294435696", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "9c10d900-7e77-42f0-8e1b-f46336bf67ee"}]}} Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-1294435696', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': '9c10d900-7e77-42f0-8e1b-f46336bf67ee'}]}}} Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Sep 27 10:11:40.791552 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'object'} Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.796047 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.805861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5be59ac0-cd5e-4bbc-b067-a0de20854782 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:40.806969 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5be59ac0-cd5e-4bbc-b067-a0de20854782 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 687 microversion: 2.1 time: 0.025799 Sep 27 10:11:40.807741 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1709/3763] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:11:40 2022] POST /compute/v2.1/servers => generated 687 bytes in 27 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:11:40.886530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3feec6c-c2f0-4b12-b8d6-c8f4438a41bf tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:40.887236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3feec6c-c2f0-4b12-b8d6-c8f4438a41bf tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:40.901389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3feec6c-c2f0-4b12-b8d6-c8f4438a41bf tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:40.902173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3feec6c-c2f0-4b12-b8d6-c8f4438a41bf tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:40.902761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3feec6c-c2f0-4b12-b8d6-c8f4438a41bf tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:40.905136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4aa0679a-9715-4683-a649-41da3ead7a95 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1197961653", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "9c10d900-7e77-42f0-8e1b-f46336bf67ee"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4aa0679a-9715-4683-a649-41da3ead7a95 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Request method failure captured: Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 500 Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsxArI2BrDRc_CFHMqHahDxz0SscM67RJwatpFvOsFPcb_7cW9eGCWj_T78Wq2MrI0SNXZkUQjV0dl_4-Odxa1JIz-DZjNE1TSToEaLsr5cEidhPthrnIxer4_dQPDtEx1Uy6izg17U34URr01sqNWP7JOLvN6_kaTi24Rfn7Mbe3vI Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_be55c1fb4e6f4870b25e986907d41101"}]}] Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 96e48f22b8e7478986f912e64df670b0 Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1197961653", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "9c10d900-7e77-42f0-8e1b-f46336bf67ee"}]}} Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-1197961653', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': '9c10d900-7e77-42f0-8e1b-f46336bf67ee'}]}}} Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Sep 27 10:11:40.909618 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'object'} Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:40.913252 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:40.915574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3feec6c-c2f0-4b12-b8d6-c8f4438a41bf tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:40.915574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3feec6c-c2f0-4b12-b8d6-c8f4438a41bf tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:40.916447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3feec6c-c2f0-4b12-b8d6-c8f4438a41bf tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:40.920318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4aa0679a-9715-4683-a649-41da3ead7a95 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:40.921353 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4aa0679a-9715-4683-a649-41da3ead7a95 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 689 microversion: 2.1 time: 0.020401 Sep 27 10:11:40.922036 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1710/3764] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:11:40 2022] POST /compute/v2.1/servers => generated 689 bytes in 22 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 27 10:11:41.005424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1c890be2-3134-47d8-8dfc-42efc97d3e09 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-373521811", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "9c10d900-7e77-42f0-8e1b-f46336bf67ee"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1c890be2-3134-47d8-8dfc-42efc97d3e09 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Request method failure captured: Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 753 Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsxArI2BrDRc_CFHMqHahDxz0SscM67RJwatpFvOsFPcb_7cW9eGCWj_T78Wq2MrI0SNXZkUQjV0dl_4-Odxa1JIz-DZjNE1TSToEaLsr5cEidhPthrnIxer4_dQPDtEx1Uy6izg17U34URr01sqNWP7JOLvN6_kaTi24Rfn7Mbe3vI Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_be55c1fb4e6f4870b25e986907d41101"}]}] Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 96e48f22b8e7478986f912e64df670b0 Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-373521811", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "9c10d900-7e77-42f0-8e1b-f46336bf67ee"}]}} Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-373521811', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': '9c10d900-7e77-42f0-8e1b-f46336bf67ee'}]}}} Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:11:41.015666 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'object'} Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:11:41.019228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:11:41.022316 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:41.022316 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.024580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1c890be2-3134-47d8-8dfc-42efc97d3e09 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:41.026074 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1c890be2-3134-47d8-8dfc-42efc97d3e09 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 1197 microversion: 2.1 time: 0.025376 Sep 27 10:11:41.036173 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1711/3765] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:11:40 2022] POST /compute/v2.1/servers => generated 1197 bytes in 36 msecs (HTTP/1.1 400) 9 headers in 382 bytes (1 switches on core 0) Sep 27 10:11:41.188500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-50873121-4b15-4845-abd6-21627cd9037f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1076579131", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "9c10d900-7e77-42f0-8e1b-f46336bf67ee"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-50873121-4b15-4845-abd6-21627cd9037f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Request method failure captured: Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers HTTP/1.1 Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 1266 Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsxArI2BrDRc_CFHMqHahDxz0SscM67RJwatpFvOsFPcb_7cW9eGCWj_T78Wq2MrI0SNXZkUQjV0dl_4-Odxa1JIz-DZjNE1TSToEaLsr5cEidhPthrnIxer4_dQPDtEx1Uy6izg17U34URr01sqNWP7JOLvN6_kaTi24Rfn7Mbe3vI Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_be55c1fb4e6f4870b25e986907d41101"}]}] Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 96e48f22b8e7478986f912e64df670b0 Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: {"server": {"name": "tempest-ServerMetadataNegativeTestJSON-server-1076579131", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk": "data1"}, "networks": [{"uuid": "9c10d900-7e77-42f0-8e1b-f46336bf67ee"}]}} Sep 27 10:11:41.199130 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'server': {'name': 'tempest-ServerMetadataNegativeTestJSON-server-1076579131', 'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6', 'flavorRef': '452', 'metadata': {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}, 'networks': [{'uuid': '9c10d900-7e77-42f0-8e1b-f46336bf67ee'}]}}} Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'additionalProperties' in schema['properties']['server']['properties']['metadata']: Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'additionalProperties': False, Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'patternProperties': {'^[a-zA-Z0-9-_:. ]{1,255}$': {'maxLength': 255, Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'string'}}, Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi 'type': 'object'} Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.206795 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['server']: Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'} Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:11:41.209173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:11:41.211329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' Sep 27 10:11:41.211329 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.211329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-50873121-4b15-4845-abd6-21627cd9037f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Returning 400 to user: Invalid input for field/attribute metadata. Value: {'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk': 'data1'}. 'kkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk' does not match any of the regexes: '^[a-zA-Z0-9-_:. ]{1,255}$' {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:41.211329 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-50873121-4b15-4845-abd6-21627cd9037f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 400 len: 2221 microversion: 2.1 time: 0.016737 Sep 27 10:11:41.211329 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1712/3766] 10.222.0.26 () {60 vars in 1148 bytes} [Tue Sep 27 10:11:41 2022] POST /compute/v2.1/servers => generated 2221 bytes in 25 msecs (HTTP/1.1 400) 9 headers in 382 bytes (1 switches on core 0) Sep 27 10:11:41.213788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c4667735-23f0-49ae-8cfe-dab4fa06fd2b tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:41.214288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c4667735-23f0-49ae-8cfe-dab4fa06fd2b tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 0a8da770-8ed0-461c-bfd5-b2953ab6a2ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:41.229709 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-c4667735-23f0-49ae-8cfe-dab4fa06fd2b tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] HTTP exception thrown: Instance 0a8da770-8ed0-461c-bfd5-b2953ab6a2ea could not be found. Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c4667735-23f0-49ae-8cfe-dab4fa06fd2b tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Request method failure captured: Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/0a8da770-8ed0-461c-bfd5-b2953ab6a2ea/metadata/test2 HTTP/1.1 Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsxArI2BrDRc_CFHMqHahDxz0SscM67RJwatpFvOsFPcb_7cW9eGCWj_T78Wq2MrI0SNXZkUQjV0dl_4-Odxa1JIz-DZjNE1TSToEaLsr5cEidhPthrnIxer4_dQPDtEx1Uy6izg17U34URr01sqNWP7JOLvN6_kaTi24Rfn7Mbe3vI Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_be55c1fb4e6f4870b25e986907d41101"}]}] Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 96e48f22b8e7478986f912e64df670b0 Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 0a8da770-8ed0-461c-bfd5-b2953ab6a2ea could not be found. Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'server_id': '0a8da770-8ed0-461c-bfd5-b2953ab6a2ea', 'id': 'test2'} Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0a8da770-8ed0-461c-bfd5-b2953ab6a2ea could not be found. Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_metadata.py", line 129, in show Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi server = common.get_instance(self.compute_api, context, server_id) Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0a8da770-8ed0-461c-bfd5-b2953ab6a2ea could not be found. Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:41.238141 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:41.243730 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:41.243730 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:41.243730 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0a8da770-8ed0-461c-bfd5-b2953ab6a2ea could not be found. Sep 27 10:11:41.243730 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:41.247135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c4667735-23f0-49ae-8cfe-dab4fa06fd2b tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Returning 404 to user: Instance 0a8da770-8ed0-461c-bfd5-b2953ab6a2ea could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:41.247135 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c4667735-23f0-49ae-8cfe-dab4fa06fd2b tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/0a8da770-8ed0-461c-bfd5-b2953ab6a2ea/metadata/test2" status: 404 len: 111 microversion: 2.1 time: 0.036674 Sep 27 10:11:41.247390 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1713/3767] 10.222.0.26 () {58 vars in 1281 bytes} [Tue Sep 27 10:11:41 2022] GET /compute/v2.1/servers/0a8da770-8ed0-461c-bfd5-b2953ab6a2ea/metadata/test2 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:11:41.262550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea84716f-1aef-424b-95e1-f591d71f0a3f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:41.263474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ea84716f-1aef-424b-95e1-f591d71f0a3f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:41.274265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea84716f-1aef-424b-95e1-f591d71f0a3f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:41.277863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea84716f-1aef-424b-95e1-f591d71f0a3f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:41.279153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea84716f-1aef-424b-95e1-f591d71f0a3f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:41.283436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea84716f-1aef-424b-95e1-f591d71f0a3f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:41.284110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea84716f-1aef-424b-95e1-f591d71f0a3f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:41.285724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea84716f-1aef-424b-95e1-f591d71f0a3f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:41.356123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ea84716f-1aef-424b-95e1-f591d71f0a3f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:11:41.373382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b3feec6c-c2f0-4b12-b8d6-c8f4438a41bf tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:41.384620 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3feec6c-c2f0-4b12-b8d6-c8f4438a41bf tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.500987 Sep 27 10:11:41.401333 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2055/3768] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:40 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:41.488384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:41.563716 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea84716f-1aef-424b-95e1-f591d71f0a3f tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "DELETE /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 204 len: 0 microversion: 2.1 time: 0.306077 Sep 27 10:11:41.565064 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1714/3769] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:11:41 2022] DELETE /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:11:41.575338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fec75b01-8a5d-4299-8ed4-003c2f37f7a6 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:41.576559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fec75b01-8a5d-4299-8ed4-003c2f37f7a6 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:41.590596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fec75b01-8a5d-4299-8ed4-003c2f37f7a6 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:41.592874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fec75b01-8a5d-4299-8ed4-003c2f37f7a6 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:41.594678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fec75b01-8a5d-4299-8ed4-003c2f37f7a6 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:41.596528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fec75b01-8a5d-4299-8ed4-003c2f37f7a6 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:41.597558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fec75b01-8a5d-4299-8ed4-003c2f37f7a6 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:41.597999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fec75b01-8a5d-4299-8ed4-003c2f37f7a6 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:42.413417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-781fcbaa-c48b-4375-8a1e-602316041fa9 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:42.414257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-781fcbaa-c48b-4375-8a1e-602316041fa9 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:42.427410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-781fcbaa-c48b-4375-8a1e-602316041fa9 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:42.427846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-781fcbaa-c48b-4375-8a1e-602316041fa9 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:42.431653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-781fcbaa-c48b-4375-8a1e-602316041fa9 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:42.432801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-781fcbaa-c48b-4375-8a1e-602316041fa9 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:42.433462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-781fcbaa-c48b-4375-8a1e-602316041fa9 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:42.435400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-781fcbaa-c48b-4375-8a1e-602316041fa9 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:42.649398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fec75b01-8a5d-4299-8ed4-003c2f37f7a6 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:42.659577 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fec75b01-8a5d-4299-8ed4-003c2f37f7a6 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1525 microversion: 2.1 time: 1.089214 Sep 27 10:11:42.672381 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2056/3770] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:41 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1525 bytes in 1091 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:42.871266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-781fcbaa-c48b-4375-8a1e-602316041fa9 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:42.887604 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-781fcbaa-c48b-4375-8a1e-602316041fa9 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.479908 Sep 27 10:11:42.889814 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1715/3771] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:42 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:43.029580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7a07fece-fc31-4ac0-8178-a718444b35a3 req-2018aa73-9458-4d55-b172-7e7255bf49df service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "tag": "8cd8623a-a3f6-4d1c-aced-9707f8353fa6"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:43.057297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a07fece-fc31-4ac0-8178-a718444b35a3 req-2018aa73-9458-4d55-b172-7e7255bf49df service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:43.057297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a07fece-fc31-4ac0-8178-a718444b35a3 req-2018aa73-9458-4d55-b172-7e7255bf49df service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:43.057297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a07fece-fc31-4ac0-8178-a718444b35a3 req-2018aa73-9458-4d55-b172-7e7255bf49df service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:43.081038 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7a07fece-fc31-4ac0-8178-a718444b35a3 req-2018aa73-9458-4d55-b172-7e7255bf49df service nova] Creating event network-changed:8cd8623a-a3f6-4d1c-aced-9707f8353fa6 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:11:43.108099 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7a07fece-fc31-4ac0-8178-a718444b35a3 req-2018aa73-9458-4d55-b172-7e7255bf49df service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082227 Sep 27 10:11:43.112459 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2057/3772] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:43.684068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f0b47300-8db4-40c2-9ed5-432e1730cace tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:43.684703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f0b47300-8db4-40c2-9ed5-432e1730cace tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:43.696341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b47300-8db4-40c2-9ed5-432e1730cace tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:43.697298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b47300-8db4-40c2-9ed5-432e1730cace tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:43.697901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b47300-8db4-40c2-9ed5-432e1730cace tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:43.698784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b47300-8db4-40c2-9ed5-432e1730cace tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:43.699409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b47300-8db4-40c2-9ed5-432e1730cace tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:43.699839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b47300-8db4-40c2-9ed5-432e1730cace tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:43.919426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-adc72e45-9b76-4c01-9ee7-37af27a6481d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:43.920902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-adc72e45-9b76-4c01-9ee7-37af27a6481d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:43.936384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-adc72e45-9b76-4c01-9ee7-37af27a6481d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:43.936941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-adc72e45-9b76-4c01-9ee7-37af27a6481d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:43.941981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-adc72e45-9b76-4c01-9ee7-37af27a6481d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:43.945800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-adc72e45-9b76-4c01-9ee7-37af27a6481d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:43.946514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-adc72e45-9b76-4c01-9ee7-37af27a6481d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:43.948394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-adc72e45-9b76-4c01-9ee7-37af27a6481d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:44.213668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-adc72e45-9b76-4c01-9ee7-37af27a6481d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:44.225040 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-adc72e45-9b76-4c01-9ee7-37af27a6481d tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.310438 Sep 27 10:11:44.226883 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2058/3773] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:43 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:45.024988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f0b47300-8db4-40c2-9ed5-432e1730cace tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:45.045867 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f0b47300-8db4-40c2-9ed5-432e1730cace tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1525 microversion: 2.1 time: 1.364198 Sep 27 10:11:45.049139 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1716/3774] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:43 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1525 bytes in 1371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:45.259650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8fa9176-e18d-4f61-a0da-958b034d46e8 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:45.260276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c8fa9176-e18d-4f61-a0da-958b034d46e8 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:45.271687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8fa9176-e18d-4f61-a0da-958b034d46e8 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:45.272462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8fa9176-e18d-4f61-a0da-958b034d46e8 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:45.273294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8fa9176-e18d-4f61-a0da-958b034d46e8 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:45.275354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8fa9176-e18d-4f61-a0da-958b034d46e8 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:45.276687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8fa9176-e18d-4f61-a0da-958b034d46e8 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:45.282017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8fa9176-e18d-4f61-a0da-958b034d46e8 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:45.723264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7c52cf03-020c-457e-b569-cf1ff34f8ae4 req-5c1c9c6a-2877-4a1b-9cd6-40274a905e44 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2ad21db4-8d06-48e5-b469-be50b03f701a", "name": "network-vif-deleted", "tag": "55eb0fa6-bb39-46b7-889c-e034fd5114c2"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:45.756281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c8fa9176-e18d-4f61-a0da-958b034d46e8 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:45.784557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7c52cf03-020c-457e-b569-cf1ff34f8ae4 req-5c1c9c6a-2877-4a1b-9cd6-40274a905e44 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:45.785420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7c52cf03-020c-457e-b569-cf1ff34f8ae4 req-5c1c9c6a-2877-4a1b-9cd6-40274a905e44 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:45.787281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7c52cf03-020c-457e-b569-cf1ff34f8ae4 req-5c1c9c6a-2877-4a1b-9cd6-40274a905e44 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:45.796857 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c8fa9176-e18d-4f61-a0da-958b034d46e8 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.546116 Sep 27 10:11:45.801284 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2059/3775] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:45 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:45.831691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7c52cf03-020c-457e-b569-cf1ff34f8ae4 req-5c1c9c6a-2877-4a1b-9cd6-40274a905e44 service nova] Creating event network-vif-deleted:55eb0fa6-bb39-46b7-889c-e034fd5114c2 for instance 2ad21db4-8d06-48e5-b469-be50b03f701a on nch2-master-1 Sep 27 10:11:45.840320 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7c52cf03-020c-457e-b569-cf1ff34f8ae4 req-5c1c9c6a-2877-4a1b-9cd6-40274a905e44 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.121335 Sep 27 10:11:45.843202 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1717/3776] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:11:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:11:46.074429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:46.076215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:46.094672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:46.095188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:46.095639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:46.104930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:46.106619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:46.121626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:46.221260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:46.222031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:46.222756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:46.223437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lazy-loading 'fault' on Instance uuid 2ad21db4-8d06-48e5-b469-be50b03f701a {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:11:46.313805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96e48f22b8e7478986f912e64df670b0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'be55c1fb4e6f4870b25e986907d41101', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:46.332992 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2e23f9ab-f436-4dbc-ad48-043ea846aad9 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 200 len: 1314 microversion: 2.1 time: 0.262649 Sep 27 10:11:46.336026 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2060/3777] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:46 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 1314 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:46.831283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23ee2409-1cee-4e3b-894a-9faf5047307e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:46.833449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-23ee2409-1cee-4e3b-894a-9faf5047307e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:46.867311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ee2409-1cee-4e3b-894a-9faf5047307e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:46.867973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ee2409-1cee-4e3b-894a-9faf5047307e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:46.868546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ee2409-1cee-4e3b-894a-9faf5047307e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:46.870377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ee2409-1cee-4e3b-894a-9faf5047307e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:46.871005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ee2409-1cee-4e3b-894a-9faf5047307e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:46.871523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ee2409-1cee-4e3b-894a-9faf5047307e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:47.289791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af441a36-cce9-481b-8924-2fb03ec868ff tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f605c0e1-aee3-46fe-a624-b9b13bb6cb46"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:47.292305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af441a36-cce9-481b-8924-2fb03ec868ff tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:47.307198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-23ee2409-1cee-4e3b-894a-9faf5047307e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:47.316936 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-23ee2409-1cee-4e3b-894a-9faf5047307e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.493968 Sep 27 10:11:47.319674 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1718/3778] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:46 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:47.320013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af441a36-cce9-481b-8924-2fb03ec868ff tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:47.323664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af441a36-cce9-481b-8924-2fb03ec868ff tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:47.324376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af441a36-cce9-481b-8924-2fb03ec868ff tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:47.325657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af441a36-cce9-481b-8924-2fb03ec868ff tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:47.326156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af441a36-cce9-481b-8924-2fb03ec868ff tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:47.327942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af441a36-cce9-481b-8924-2fb03ec868ff tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:47.362137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:47.363012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] [instance: 2ad21db4-8d06-48e5-b469-be50b03f701a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:47.379659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:47.395890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:47.398753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:47.399614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:47.400101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:47.405641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:47.473674 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] HTTP exception thrown: Instance 2ad21db4-8d06-48e5-b469-be50b03f701a could not be found. Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Request method failure captured: Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a HTTP/1.1 Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsxArI2BrDRc_CFHMqHahDxz0SscM67RJwatpFvOsFPcb_7cW9eGCWj_T78Wq2MrI0SNXZkUQjV0dl_4-Odxa1JIz-DZjNE1TSToEaLsr5cEidhPthrnIxer4_dQPDtEx1Uy6izg17U34URr01sqNWP7JOLvN6_kaTi24Rfn7Mbe3vI Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/be55c1fb4e6f4870b25e986907d41101"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_be55c1fb4e6f4870b25e986907d41101"}]}] Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: be55c1fb4e6f4870b25e986907d41101 Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerMetadataNegativeTestJSON-1635680230 Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 96e48f22b8e7478986f912e64df670b0 Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerMetadataNegativeTestJSON-1635680230-project Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 2ad21db4-8d06-48e5-b469-be50b03f701a could not be found. Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '2ad21db4-8d06-48e5-b469-be50b03f701a'} Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2ad21db4-8d06-48e5-b469-be50b03f701a could not be found. Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2ad21db4-8d06-48e5-b469-be50b03f701a could not be found. Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:47.485159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:47.490484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:47.490484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:47.490484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:47.490484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:47.490484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:47.490484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:47.490484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2ad21db4-8d06-48e5-b469-be50b03f701a could not be found. Sep 27 10:11:47.490484 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:11:47.494404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] Returning 404 to user: Instance 2ad21db4-8d06-48e5-b469-be50b03f701a could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:47.496101 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a52f62b5-a43c-4563-8d82-4f9275d26d23 tempest-ServerMetadataNegativeTestJSON-1635680230 tempest-ServerMetadataNegativeTestJSON-1635680230-project] 10.222.0.26 "GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a" status: 404 len: 111 microversion: 2.1 time: 0.140621 Sep 27 10:11:47.498119 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1719/3779] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:47 2022] GET /compute/v2.1/servers/2ad21db4-8d06-48e5-b469-be50b03f701a => generated 111 bytes in 143 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:11:47.888449 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af441a36-cce9-481b-8924-2fb03ec868ff tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "POST /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.604163 Sep 27 10:11:47.889272 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2061/3780] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:11:47 2022] POST /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-volume_attachments => generated 194 bytes in 606 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:11:48.349076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea087ea7-cf82-4383-bd33-2b61994cdd99 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:48.350806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ea087ea7-cf82-4383-bd33-2b61994cdd99 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:48.368610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea087ea7-cf82-4383-bd33-2b61994cdd99 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:48.370856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea087ea7-cf82-4383-bd33-2b61994cdd99 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:48.372572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea087ea7-cf82-4383-bd33-2b61994cdd99 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:48.373615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea087ea7-cf82-4383-bd33-2b61994cdd99 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:48.386890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea087ea7-cf82-4383-bd33-2b61994cdd99 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:48.393000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea087ea7-cf82-4383-bd33-2b61994cdd99 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:48.919936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ea087ea7-cf82-4383-bd33-2b61994cdd99 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:48.941936 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea087ea7-cf82-4383-bd33-2b61994cdd99 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.599412 Sep 27 10:11:48.964471 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1720/3781] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:48 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:49.977934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed65beb0-51b2-4c74-ad15-d4c80f2426f6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:49.978659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed65beb0-51b2-4c74-ad15-d4c80f2426f6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:49.983046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=80 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:50.018044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed65beb0-51b2-4c74-ad15-d4c80f2426f6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:50.019040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed65beb0-51b2-4c74-ad15-d4c80f2426f6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:50.021205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed65beb0-51b2-4c74-ad15-d4c80f2426f6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:50.025378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed65beb0-51b2-4c74-ad15-d4c80f2426f6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:50.026038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed65beb0-51b2-4c74-ad15-d4c80f2426f6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:50.029308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed65beb0-51b2-4c74-ad15-d4c80f2426f6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:50.097674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:51.150380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ed65beb0-51b2-4c74-ad15-d4c80f2426f6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:51.173814 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed65beb0-51b2-4c74-ad15-d4c80f2426f6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 1.202838 Sep 27 10:11:51.188918 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2062/3782] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:49 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 1209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:52.206572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ef12cccf-a177-4e85-9e77-82ff032271ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:52.207276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ef12cccf-a177-4e85-9e77-82ff032271ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:52.235557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef12cccf-a177-4e85-9e77-82ff032271ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:52.238202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef12cccf-a177-4e85-9e77-82ff032271ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:52.239430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef12cccf-a177-4e85-9e77-82ff032271ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:52.241167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef12cccf-a177-4e85-9e77-82ff032271ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:52.248277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef12cccf-a177-4e85-9e77-82ff032271ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:52.253331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef12cccf-a177-4e85-9e77-82ff032271ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:52.941107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ef12cccf-a177-4e85-9e77-82ff032271ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:52.953271 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ef12cccf-a177-4e85-9e77-82ff032271ad tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.749686 Sep 27 10:11:52.969072 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1721/3783] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:52 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 768 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:53.605168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Action: 'update', calling method: >, body: {"volumeAttachment": {"volumeId": "31944d6c-ca2b-4b09-ba07-082833be12a6"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:53.608963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:53.625071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:53.626371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:53.632777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:53.633601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:53.634099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:53.634549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:53.913548 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] HTTP exception thrown: Volume 31944d6c-ca2b-4b09-ba07-082833be12a6 could not be found. Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Request method failure captured: Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: request: PUT /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-volume_attachments/f605c0e1-aee3-46fe-a624-b9b13bb6cb46 HTTP/1.1 Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 74 Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsvsXYS9VZ3zLv66ebSIwOPabHVzHI_2IQRkJzQYxVaADGwQSuJEqfGvPXthG1Mxtuuf2tI5uRd-RxKyFaY0uxKmOOhdqeA-P28oeT2sCTk-ZmdPb6Ll2KLIrOgF33UiPDp3dNMOXvK4CgxlYhEZx_sHxoKu_tYKb2svV8e6NHev3Z0 Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 688812897dc54f40ad95e885b40a23f5 Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesAdminNegativeTest-1082822564 Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/688812897dc54f40ad95e885b40a23f5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/688812897dc54f40ad95e885b40a23f5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/688812897dc54f40ad95e885b40a23f5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_688812897dc54f40ad95e885b40a23f5"}]}] Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesAdminNegativeTest-1082822564 Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 688812897dc54f40ad95e885b40a23f5 Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-1082822564 Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesAdminNegativeTest-1082822564-project-admin Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 31e653b281b64ade86e15660158ff07e Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesAdminNegativeTest-1082822564-project-admin Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: {"volumeAttachment": {"volumeId": "31944d6c-ca2b-4b09-ba07-082833be12a6"}} Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: exception: Volume 31944d6c-ca2b-4b09-ba07-082833be12a6 could not be found. Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'server_id': 'a04c146e-494e-458e-843e-09c63c576cb3', 'id': 'f605c0e1-aee3-46fe-a624-b9b13bb6cb46', 'body': {'volumeAttachment': {'volumeId': '31944d6c-ca2b-4b09-ba07-082833be12a6'}}} Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 31944d6c-ca2b-4b09-ba07-082833be12a6 could not be found. (HTTP 404) (Request-ID: req-31c102c1-385a-4111-afc7-6a53910667b1) Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 436, in _update_volume_swap Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi new_volume = self.volume_api.get(context, new_volume_id) Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 10:11:53.958328 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 31944d6c-ca2b-4b09-ba07-082833be12a6 could not be found. Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 522, in update Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._update_volume_swap(req, instance, id, body) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 445, in _update_volume_swap Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Volume 31944d6c-ca2b-4b09-ba07-082833be12a6 could not be found. Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 31944d6c-ca2b-4b09-ba07-082833be12a6 could not be found. Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Returning 400 to user: Volume 31944d6c-ca2b-4b09-ba07-082833be12a6 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dd3ce0b5-24a0-4b4f-b776-02df001afd88 tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] 10.222.0.26 "PUT /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-volume_attachments/f605c0e1-aee3-46fe-a624-b9b13bb6cb46" status: 400 len: 107 microversion: 2.1 time: 1.164904 Sep 27 10:11:53.964321 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2063/3784] 10.222.0.26 () {60 vars in 1433 bytes} [Tue Sep 27 10:11:52 2022] PUT /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-volume_attachments/f605c0e1-aee3-46fe-a624-b9b13bb6cb46 => generated 107 bytes in 1166 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:11:53.978624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4cabac48-97a4-4874-8bea-4a54dbeed0a3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:53.979234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4cabac48-97a4-4874-8bea-4a54dbeed0a3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:53.990587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cabac48-97a4-4874-8bea-4a54dbeed0a3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:53.992978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cabac48-97a4-4874-8bea-4a54dbeed0a3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:53.993445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cabac48-97a4-4874-8bea-4a54dbeed0a3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:53.994183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cabac48-97a4-4874-8bea-4a54dbeed0a3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:53.994646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cabac48-97a4-4874-8bea-4a54dbeed0a3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:53.995072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cabac48-97a4-4874-8bea-4a54dbeed0a3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:54.063741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6a091cb1-4b09-422c-accc-7d57f53c6215 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:54.064252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6a091cb1-4b09-422c-accc-7d57f53c6215 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:54.076129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a091cb1-4b09-422c-accc-7d57f53c6215 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:54.076628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a091cb1-4b09-422c-accc-7d57f53c6215 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:54.088689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a091cb1-4b09-422c-accc-7d57f53c6215 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:54.090303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a091cb1-4b09-422c-accc-7d57f53c6215 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:54.092401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a091cb1-4b09-422c-accc-7d57f53c6215 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:54.092983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a091cb1-4b09-422c-accc-7d57f53c6215 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:54.295174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-6a091cb1-4b09-422c-accc-7d57f53c6215 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lazy-loading 'services' on Instance uuid a04c146e-494e-458e-843e-09c63c576cb3 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:11:54.459647 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6a091cb1-4b09-422c-accc-7d57f53c6215 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-volume_attachments/f605c0e1-aee3-46fe-a624-b9b13bb6cb46" status: 202 len: 0 microversion: 2.1 time: 0.399705 Sep 27 10:11:54.460388 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2064/3785] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:11:54 2022] DELETE /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-volume_attachments/f605c0e1-aee3-46fe-a624-b9b13bb6cb46 => generated 0 bytes in 402 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:11:54.469844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eddf4c9c-8750-45a0-8b4e-8330e462adc5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:54.472446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eddf4c9c-8750-45a0-8b4e-8330e462adc5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:54.486765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eddf4c9c-8750-45a0-8b4e-8330e462adc5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:54.487650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eddf4c9c-8750-45a0-8b4e-8330e462adc5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:54.488575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eddf4c9c-8750-45a0-8b4e-8330e462adc5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:54.489596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eddf4c9c-8750-45a0-8b4e-8330e462adc5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:54.490244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eddf4c9c-8750-45a0-8b4e-8330e462adc5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:54.490866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eddf4c9c-8750-45a0-8b4e-8330e462adc5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:54.546997 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eddf4c9c-8750-45a0-8b4e-8330e462adc5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.081516 Sep 27 10:11:54.548205 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2065/3786] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:11:54 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-volume_attachments => generated 197 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:11:54.997252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f83d973-218c-4846-9d44-429ed7ecb615 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Action: 'create', calling method: .version_select at 0x7f7f630a6f70>, body: {"keypair": {"name": "tempest-keypair-1960047129", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPewm27PgftlxTc4OLgMoDRB6jxClA8g3WDe+9NOPgpU/GzYrKzr8/EjGgZRiftg6FkeE1S8iVkdO1EEMaIqaW6ZMoUDy8F2XH/cLPscJF7U/rETmt6ug/SYRZgad98C/g=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:55.019284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8f83d973-218c-4846-9d44-429ed7ecb615 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Getting quotas for project 0a1ced5ea0884fc99d7644f37ca3f376. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:11:55.027488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8f83d973-218c-4846-9d44-429ed7ecb615 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Getting quotas for user 015d442ea03c434bb63aa9ba9b537ce8 and project 0a1ced5ea0884fc99d7644f37ca3f376. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:11:55.056421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4cabac48-97a4-4874-8bea-4a54dbeed0a3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:55.071724 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4cabac48-97a4-4874-8bea-4a54dbeed0a3 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 1.094687 Sep 27 10:11:55.073563 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1722/3787] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:53 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 1100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:11:55.085980 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8f83d973-218c-4846-9d44-429ed7ecb615 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.278131 Sep 27 10:11:55.087158 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2066/3788] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:11:54 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:11:55.560525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f70dbbd3-10a7-4519-b212-e492996100f7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:55.562812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f70dbbd3-10a7-4519-b212-e492996100f7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:55.565343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=65,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:11:55.590357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f70dbbd3-10a7-4519-b212-e492996100f7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:55.592213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f70dbbd3-10a7-4519-b212-e492996100f7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:55.592728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f70dbbd3-10a7-4519-b212-e492996100f7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:55.593508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f70dbbd3-10a7-4519-b212-e492996100f7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:55.593967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f70dbbd3-10a7-4519-b212-e492996100f7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:55.594768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f70dbbd3-10a7-4519-b212-e492996100f7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:55.647171 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f70dbbd3-10a7-4519-b212-e492996100f7 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.091125 Sep 27 10:11:55.653113 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1723/3789] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:11:55 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3/os-volume_attachments => generated 25 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:11:56.004791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesAdminNegativeTest-server-1570779536", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "e4d5dcb5-38db-4df7-833a-8982d96ea82d"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:56.097419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-462265b2-62cb-4faf-a452-a9d7f450cb73 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:56.098628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-462265b2-62cb-4faf-a452-a9d7f450cb73 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:56.108043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462265b2-62cb-4faf-a452-a9d7f450cb73 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:56.110042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462265b2-62cb-4faf-a452-a9d7f450cb73 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:56.110895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462265b2-62cb-4faf-a452-a9d7f450cb73 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:56.113422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462265b2-62cb-4faf-a452-a9d7f450cb73 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:56.115124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462265b2-62cb-4faf-a452-a9d7f450cb73 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:56.123783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462265b2-62cb-4faf-a452-a9d7f450cb73 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:56.134231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] validate_networks() for [('e4d5dcb5-38db-4df7-833a-8982d96ea82d', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:11:56.887233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-462265b2-62cb-4faf-a452-a9d7f450cb73 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:56.942118 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-462265b2-62cb-4faf-a452-a9d7f450cb73 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.849138 Sep 27 10:11:56.964173 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1724/3790] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:56 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 857 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:57.015548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7ee2887d-3216-46a0-8710-d903e750eeab tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Action: 'create', calling method: .version_select at 0x7f7f6268b310>, body: {"keypair": {"name": "tempest-keypair-1657625992", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEA7IOQ81RJ0Ydy8vt2NxfhucT3NyKwDPncHKceUzluF/Wp5fZbqsfVucqq8FmlK6MirVX/mbO0tMAXI2vplN8iK878jRDyRVb8TnDcJ6OqADhTp0yImAjwFMrMSy1lUdw=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:57.067099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7ee2887d-3216-46a0-8710-d903e750eeab tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Getting quotas for project 9c60d0e3692444e39346d2c296c32ac2. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:11:57.095102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7ee2887d-3216-46a0-8710-d903e750eeab tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Getting quotas for user e4772d35d57f478d929eea70fd11e147 and project 9c60d0e3692444e39346d2c296c32ac2. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:11:57.184864 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7ee2887d-3216-46a0-8710-d903e750eeab tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.32 time: 0.186170 Sep 27 10:11:57.199965 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1725/3791] 10.222.0.26 () {62 vars in 1200 bytes} [Tue Sep 27 10:11:56 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 203 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 27 10:11:57.853164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:11:57.853718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:11:57.854061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:11:57.854472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:11:57.854885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:11:57.875678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:57.876419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:57.877062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:57.878213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:57.879080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:57.879254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:57.953323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Getting quotas for project f847bee14ff74dfbafe7c6a199f1b99f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:11:57.972039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7af94eaa-a453-45b8-a18e-a9e502329386 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:57.973661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7af94eaa-a453-45b8-a18e-a9e502329386 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:57.985009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7af94eaa-a453-45b8-a18e-a9e502329386 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:57.985519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7af94eaa-a453-45b8-a18e-a9e502329386 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:57.985968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7af94eaa-a453-45b8-a18e-a9e502329386 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:57.986722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7af94eaa-a453-45b8-a18e-a9e502329386 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:57.987195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7af94eaa-a453-45b8-a18e-a9e502329386 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:57.988223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Getting quotas for user b8d6f6c63fe144959139278ca287e2ec and project f847bee14ff74dfbafe7c6a199f1b99f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:11:57.996420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7af94eaa-a453-45b8-a18e-a9e502329386 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:58.017758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:11:58.023083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:11:58.151275 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6ea92cbd-0210-4eca-9df4-86a7b1378f88 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.164620 Sep 27 10:11:58.154939 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2067/3792] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:11:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 2171 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:11:58.167979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5bc77d56-1b8f-4153-bead-e3ec8143d854 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:58.170100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5bc77d56-1b8f-4153-bead-e3ec8143d854 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:58.516142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5bc77d56-1b8f-4153-bead-e3ec8143d854 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:58.523818 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5bc77d56-1b8f-4153-bead-e3ec8143d854 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1218 microversion: 2.1 time: 0.362281 Sep 27 10:11:58.531996 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2068/3793] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:58 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1218 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:58.831942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7af94eaa-a453-45b8-a18e-a9e502329386 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:11:58.846348 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7af94eaa-a453-45b8-a18e-a9e502329386 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1484 microversion: 2.1 time: 0.876902 Sep 27 10:11:58.869249 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1726/3794] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:57 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1484 bytes in 902 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:11:58.876647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f66e2510-cc64-4ddf-89ac-7cc1dbbff903 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:11:58.877974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f66e2510-cc64-4ddf-89ac-7cc1dbbff903 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:58.889784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f66e2510-cc64-4ddf-89ac-7cc1dbbff903 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:58.891656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f66e2510-cc64-4ddf-89ac-7cc1dbbff903 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:58.892328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f66e2510-cc64-4ddf-89ac-7cc1dbbff903 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:58.893148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f66e2510-cc64-4ddf-89ac-7cc1dbbff903 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:58.893634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f66e2510-cc64-4ddf-89ac-7cc1dbbff903 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:58.894256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f66e2510-cc64-4ddf-89ac-7cc1dbbff903 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:59.016344 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f66e2510-cc64-4ddf-89ac-7cc1dbbff903 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "POST /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6/action" status: 202 len: 0 microversion: 2.1 time: 0.142894 Sep 27 10:11:59.027909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba89348d-b04c-4ae8-a86b-d87c2bf5bfd6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:59.028899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ba89348d-b04c-4ae8-a86b-d87c2bf5bfd6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:59.034666 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2069/3795] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:11:58 2022] POST /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6/action => generated 0 bytes in 162 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:11:59.039539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba89348d-b04c-4ae8-a86b-d87c2bf5bfd6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:59.040356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba89348d-b04c-4ae8-a86b-d87c2bf5bfd6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:59.041011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba89348d-b04c-4ae8-a86b-d87c2bf5bfd6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:59.041938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba89348d-b04c-4ae8-a86b-d87c2bf5bfd6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:59.042996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba89348d-b04c-4ae8-a86b-d87c2bf5bfd6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:59.043680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba89348d-b04c-4ae8-a86b-d87c2bf5bfd6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:59.561104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-03e1a994-afea-4428-bd14-b039f72f1fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:11:59.562635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-03e1a994-afea-4428-bd14-b039f72f1fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:11:59.598023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e1a994-afea-4428-bd14-b039f72f1fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:59.613682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e1a994-afea-4428-bd14-b039f72f1fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:59.616384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e1a994-afea-4428-bd14-b039f72f1fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:11:59.618985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e1a994-afea-4428-bd14-b039f72f1fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:11:59.622367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e1a994-afea-4428-bd14-b039f72f1fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:11:59.623493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03e1a994-afea-4428-bd14-b039f72f1fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:00.147742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03e1a994-afea-4428-bd14-b039f72f1fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:00.165647 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-03e1a994-afea-4428-bd14-b039f72f1fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1222 microversion: 2.1 time: 0.609984 Sep 27 10:12:00.166597 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2070/3796] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:59 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1222 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:12:00.207212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ba89348d-b04c-4ae8-a86b-d87c2bf5bfd6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:00.220678 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ba89348d-b04c-4ae8-a86b-d87c2bf5bfd6 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1492 microversion: 2.1 time: 1.197187 Sep 27 10:12:00.221879 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1727/3797] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:11:59 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1492 bytes in 1199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:01.108874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestManualDisk-server-247641393", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "metadata": {"hello": "world"}, "accessIPv4": "1.1.1.1", "accessIPv6": "0000:0000:0000:0000:0000:babe:220.12.22.2", "adminPass": "***", "networks": [{"uuid": "59d33d2e-6b88-467e-9679-67e3a6431c9a"}], "security_groups": [{"name": "tempest-securitygroup--324794081"}], "key_name": "tempest-keypair-1960047129", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=", "OS-DCF:diskConfig": "}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:01.196759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-53006e7a-c99c-420e-a51c-1fbee068e2a5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:01.200775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-53006e7a-c99c-420e-a51c-1fbee068e2a5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:01.231260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53006e7a-c99c-420e-a51c-1fbee068e2a5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:01.233938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53006e7a-c99c-420e-a51c-1fbee068e2a5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:01.234585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53006e7a-c99c-420e-a51c-1fbee068e2a5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:01.237930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53006e7a-c99c-420e-a51c-1fbee068e2a5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:01.239155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53006e7a-c99c-420e-a51c-1fbee068e2a5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:01.240601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53006e7a-c99c-420e-a51c-1fbee068e2a5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:01.387108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-53006e7a-c99c-420e-a51c-1fbee068e2a5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:01.400972 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-53006e7a-c99c-420e-a51c-1fbee068e2a5 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1276 microversion: 2.1 time: 0.213464 Sep 27 10:12:01.401752 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1728/3798] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:01 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1276 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:01.407008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f4f4373a-075c-42f6-a94e-72236b5b32aa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:01.407980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f4f4373a-075c-42f6-a94e-72236b5b32aa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:01.419229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4f4373a-075c-42f6-a94e-72236b5b32aa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:01.420291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4f4373a-075c-42f6-a94e-72236b5b32aa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:01.420958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4f4373a-075c-42f6-a94e-72236b5b32aa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:01.422369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4f4373a-075c-42f6-a94e-72236b5b32aa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:01.423890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4f4373a-075c-42f6-a94e-72236b5b32aa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:01.424596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4f4373a-075c-42f6-a94e-72236b5b32aa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:01.499666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] validate_networks() for [('59d33d2e-6b88-467e-9679-67e3a6431c9a', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:12:01.672772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f4f4373a-075c-42f6-a94e-72236b5b32aa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:01.684210 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f4f4373a-075c-42f6-a94e-72236b5b32aa tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1492 microversion: 2.1 time: 0.281196 Sep 27 10:12:01.686444 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1729/3799] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:01 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1492 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:02.430661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e9ad9a98-bf62-4200-8243-7c2e4fed9da2 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:02.431371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e9ad9a98-bf62-4200-8243-7c2e4fed9da2 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:02.450353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9ad9a98-bf62-4200-8243-7c2e4fed9da2 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:02.461877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9ad9a98-bf62-4200-8243-7c2e4fed9da2 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.022s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:02.462735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9ad9a98-bf62-4200-8243-7c2e4fed9da2 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:02.463760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9ad9a98-bf62-4200-8243-7c2e4fed9da2 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:02.464465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9ad9a98-bf62-4200-8243-7c2e4fed9da2 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:02.465180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9ad9a98-bf62-4200-8243-7c2e4fed9da2 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:02.799707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e9ad9a98-bf62-4200-8243-7c2e4fed9da2 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:02.809784 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e9ad9a98-bf62-4200-8243-7c2e4fed9da2 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1318 microversion: 2.1 time: 0.383842 Sep 27 10:12:02.829319 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1730/3800] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:02 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1318 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:02.874987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Getting quotas for project 0a1ced5ea0884fc99d7644f37ca3f376. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:12:02.886912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-84b19ce7-def7-43f0-9f5c-a66896bd639b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:02.890972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Getting quotas for user 015d442ea03c434bb63aa9ba9b537ce8 and project 0a1ced5ea0884fc99d7644f37ca3f376. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:12:02.892235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-84b19ce7-def7-43f0-9f5c-a66896bd639b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:02.917684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b19ce7-def7-43f0-9f5c-a66896bd639b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:02.918485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b19ce7-def7-43f0-9f5c-a66896bd639b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:02.919174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b19ce7-def7-43f0-9f5c-a66896bd639b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:02.920154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b19ce7-def7-43f0-9f5c-a66896bd639b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:02.923529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b19ce7-def7-43f0-9f5c-a66896bd639b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:02.925788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b19ce7-def7-43f0-9f5c-a66896bd639b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:02.976207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:12:02.976986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:12:02.977661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:12:02.978470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:12:02.979061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:12:02.995405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:02.995908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:02.996352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:02.997756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:02.998293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:02.998728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:03.051593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Getting quotas for project 0a1ced5ea0884fc99d7644f37ca3f376. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:12:03.064975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Getting quotas for user 015d442ea03c434bb63aa9ba9b537ce8 and project 0a1ced5ea0884fc99d7644f37ca3f376. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:12:03.081457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:12:03.084449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:12:03.153902 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cbdc0a87-a966-466a-aed1-8a2c606509f8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 2.055803 Sep 27 10:12:03.155330 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2071/3801] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:12:01 2022] POST /compute/v2.1/servers => generated 408 bytes in 2058 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:12:03.316390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-84b19ce7-def7-43f0-9f5c-a66896bd639b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:03.334900 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-84b19ce7-def7-43f0-9f5c-a66896bd639b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1498 microversion: 2.1 time: 0.466161 Sep 27 10:12:03.355005 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1731/3802] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:02 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1498 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:03.367872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b7d23251-e3bf-443e-a769-abf482934a9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:03.369197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b7d23251-e3bf-443e-a769-abf482934a9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:03.483101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV247Test-server-671728454", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": "none"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:03.534284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b7d23251-e3bf-443e-a769-abf482934a9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:03.550746 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b7d23251-e3bf-443e-a769-abf482934a9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1277 microversion: 2.1 time: 0.189097 Sep 27 10:12:03.555257 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1732/3803] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:03 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1277 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:03.576007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-device-tagging-server-2031672087", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "config_drive": true, "networks": [{"uuid": "30c41b30-0e6b-4f26-89c6-c173c276fcf2"}, {"port": "c8585417-f76b-4df9-869f-bf4ecaf46641", "tag": "port-1"}, {"port": "db1377f2-7ffe-4edd-befc-8a1ac1aba3ea", "tag": "port-2"}, {"uuid": "09ad34d7-4332-4a25-8dba-ed1a55ea1836", "tag": "net-1"}, {"uuid": "09ad34d7-4332-4a25-8dba-ed1a55ea1836"}, {"uuid": "6d7d4986-27b2-4541-a4e9-deb2e9ea3873", "fixed_ip": "10.2.2.100", "tag": "net-2-100"}, {"uuid": "6d7d4986-27b2-4541-a4e9-deb2e9ea3873", "fixed_ip": "10.2.2.200", "tag": "net-2-200"}], "block_device_mapping_v2": [{"uuid": "cceacac7-f31d-4e36-b764-2231c93d64eb", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "tag": "boot"}, {"uuid": "8943dda0-13e7-42e3-bdeb-f9cf11b10058", "source_type": "volume", "destination_type": "volume", "boot_index": 1, "tag": "other"}, {"uuid": "4b2f9c8c-667b-4bff-8590-c93510f8acf3", "source_type": "volume", "destination_type": "volume", "boot_index": 2}], "security_groups": [{"name": "tempest-securitygroup--581371344"}], "key_name": "tempest-keypair-1657625992", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:03.610278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:12:03.612527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:12:03.615371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:12:03.617627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:12:03.618640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:12:03.690264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:03.690873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:03.691313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:03.692054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:03.692509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:03.704510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:03.761500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Getting quotas for project b5087b7fd4b3482082e5dd90fdcef18f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:12:03.774438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Getting quotas for user 976e7f61517e4e3f9f2417078225ca62 and project b5087b7fd4b3482082e5dd90fdcef18f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:12:03.827565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:12:03.830563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: cde27309-a349-4e7e-a4a8-1032336b61da] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:12:03.876988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] validate_networks() for [('30c41b30-0e6b-4f26-89c6-c173c276fcf2', None, None, None, None, None), (None, None, 'c8585417-f76b-4df9-869f-bf4ecaf46641', None, None, None), (None, None, 'db1377f2-7ffe-4edd-befc-8a1ac1aba3ea', None, None, None), ('09ad34d7-4332-4a25-8dba-ed1a55ea1836', None, None, None, None, None), ('09ad34d7-4332-4a25-8dba-ed1a55ea1836', None, None, None, None, None), ('6d7d4986-27b2-4541-a4e9-deb2e9ea3873', '10.2.2.100', None, None, None, None), ('6d7d4986-27b2-4541-a4e9-deb2e9ea3873', '10.2.2.200', None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:12:03.932828 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3f020558-8af3-4740-91f1-41099d9e5d55 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.47 time: 0.771784 Sep 27 10:12:03.933958 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2072/3804] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:12:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 778 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:12:03.959211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0dabc323-06f7-4602-a69d-b841ed5ef65e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:03.960812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0dabc323-06f7-4602-a69d-b841ed5ef65e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:03.970960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0dabc323-06f7-4602-a69d-b841ed5ef65e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:03.972584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0dabc323-06f7-4602-a69d-b841ed5ef65e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:03.974368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0dabc323-06f7-4602-a69d-b841ed5ef65e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:03.975823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0dabc323-06f7-4602-a69d-b841ed5ef65e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:03.976650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0dabc323-06f7-4602-a69d-b841ed5ef65e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:03.977446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0dabc323-06f7-4602-a69d-b841ed5ef65e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:04.015040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0dabc323-06f7-4602-a69d-b841ed5ef65e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:12:04.219226 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0dabc323-06f7-4602-a69d-b841ed5ef65e tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "DELETE /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 204 len: 0 microversion: 2.1 time: 0.283767 Sep 27 10:12:04.226006 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2073/3805] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:12:03 2022] DELETE /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:12:04.245673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-85689404-b010-4961-b4e6-cc431d848671 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:04.251125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-85689404-b010-4961-b4e6-cc431d848671 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:04.270990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85689404-b010-4961-b4e6-cc431d848671 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:04.272298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85689404-b010-4961-b4e6-cc431d848671 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:04.275795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85689404-b010-4961-b4e6-cc431d848671 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:04.277695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85689404-b010-4961-b4e6-cc431d848671 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:04.278728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85689404-b010-4961-b4e6-cc431d848671 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:04.281265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85689404-b010-4961-b4e6-cc431d848671 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:04.690916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-85689404-b010-4961-b4e6-cc431d848671 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:04.705326 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-85689404-b010-4961-b4e6-cc431d848671 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1318 microversion: 2.1 time: 0.473247 Sep 27 10:12:04.706163 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2074/3806] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:04 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1318 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:04.722928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fd4ace80-a2ea-43b3-b6f8-b9059b507197 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:04.724114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fd4ace80-a2ea-43b3-b6f8-b9059b507197 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: cde27309-a349-4e7e-a4a8-1032336b61da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:04.857808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fd4ace80-a2ea-43b3-b6f8-b9059b507197 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:04.895900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fd4ace80-a2ea-43b3-b6f8-b9059b507197 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:04.897781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fd4ace80-a2ea-43b3-b6f8-b9059b507197 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:04.901387 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fd4ace80-a2ea-43b3-b6f8-b9059b507197 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/cde27309-a349-4e7e-a4a8-1032336b61da" status: 200 len: 1277 microversion: 2.47 time: 0.190717 Sep 27 10:12:04.915188 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2075/3807] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:04 2022] GET /compute/v2.1/servers/cde27309-a349-4e7e-a4a8-1032336b61da => generated 1277 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:04.934936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-402dd97e-61d7-49eb-a73f-61526f74a854 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:04.936221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-402dd97e-61d7-49eb-a73f-61526f74a854 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:04.944957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=10,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:04.947354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-402dd97e-61d7-49eb-a73f-61526f74a854 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:04.948026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-402dd97e-61d7-49eb-a73f-61526f74a854 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:04.948677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-402dd97e-61d7-49eb-a73f-61526f74a854 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:04.949902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-402dd97e-61d7-49eb-a73f-61526f74a854 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:04.950886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-402dd97e-61d7-49eb-a73f-61526f74a854 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:04.951631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-402dd97e-61d7-49eb-a73f-61526f74a854 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:05.446180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-402dd97e-61d7-49eb-a73f-61526f74a854 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:05.459959 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-402dd97e-61d7-49eb-a73f-61526f74a854 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.542926 Sep 27 10:12:05.463684 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2076/3808] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:04 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:05.473199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bcc5576d-cc34-4097-8682-6983663d4420 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:05.474219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bcc5576d-cc34-4097-8682-6983663d4420 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:05.486054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc5576d-cc34-4097-8682-6983663d4420 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:05.488153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc5576d-cc34-4097-8682-6983663d4420 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:05.488846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc5576d-cc34-4097-8682-6983663d4420 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:05.490976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc5576d-cc34-4097-8682-6983663d4420 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:05.491878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc5576d-cc34-4097-8682-6983663d4420 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:05.493256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc5576d-cc34-4097-8682-6983663d4420 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:05.644210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bcc5576d-cc34-4097-8682-6983663d4420 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:05.654398 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bcc5576d-cc34-4097-8682-6983663d4420 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1281 microversion: 2.1 time: 0.189275 Sep 27 10:12:05.655244 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2077/3809] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:05 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1281 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:05.660587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ea66bb9e-0211-444c-ba00-485d8cb4cdcd req-9476fea0-634a-448a-a9a9-35f989fb63f6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e7f6860d-d3b1-4897-85dc-3653f32efa7d", "tag": "850e37cd-72bd-4ecc-89d4-fc4545104ce6"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:05.672648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ea66bb9e-0211-444c-ba00-485d8cb4cdcd req-9476fea0-634a-448a-a9a9-35f989fb63f6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:05.673249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ea66bb9e-0211-444c-ba00-485d8cb4cdcd req-9476fea0-634a-448a-a9a9-35f989fb63f6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:05.673667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ea66bb9e-0211-444c-ba00-485d8cb4cdcd req-9476fea0-634a-448a-a9a9-35f989fb63f6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:05.688401 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ea66bb9e-0211-444c-ba00-485d8cb4cdcd req-9476fea0-634a-448a-a9a9-35f989fb63f6 service nova] Creating event network-changed:850e37cd-72bd-4ecc-89d4-fc4545104ce6 for instance e7f6860d-d3b1-4897-85dc-3653f32efa7d on nch2-master-1 Sep 27 10:12:05.693674 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ea66bb9e-0211-444c-ba00-485d8cb4cdcd req-9476fea0-634a-448a-a9a9-35f989fb63f6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036861 Sep 27 10:12:05.695502 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2078/3810] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:05.702361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV247Test-server-1176514218", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": "none"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:05.769062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:12:05.769496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:12:05.770193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:12:05.770801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:12:05.771425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:12:05.782657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:05.783492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:05.784120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:05.785448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:05.786160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:05.786784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:05.808948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Getting quotas for project b5087b7fd4b3482082e5dd90fdcef18f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:12:05.815861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Getting quotas for user 976e7f61517e4e3f9f2417078225ca62 and project b5087b7fd4b3482082e5dd90fdcef18f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:12:05.832738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:12:05.836093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:12:05.905786 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c6db4cb8-26e9-4e55-889b-0287cca29670 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.47 time: 0.208658 Sep 27 10:12:05.906877 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2079/3811] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:12:05 2022] POST /compute/v2.1/servers => generated 380 bytes in 211 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:12:05.913415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fb24bd1a-b782-457a-bccf-9695de87b41e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:05.913415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fb24bd1a-b782-457a-bccf-9695de87b41e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:05.922124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb24bd1a-b782-457a-bccf-9695de87b41e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:05.922755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb24bd1a-b782-457a-bccf-9695de87b41e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:05.923564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb24bd1a-b782-457a-bccf-9695de87b41e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:05.924679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb24bd1a-b782-457a-bccf-9695de87b41e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:05.925308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb24bd1a-b782-457a-bccf-9695de87b41e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:05.925917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb24bd1a-b782-457a-bccf-9695de87b41e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:06.428298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fb24bd1a-b782-457a-bccf-9695de87b41e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:06.459149 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fb24bd1a-b782-457a-bccf-9695de87b41e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1318 microversion: 2.1 time: 0.538466 Sep 27 10:12:06.460157 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2080/3812] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:05 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1318 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:06.469345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f62f80b6-721e-4529-91d0-1de986cbf02d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Action: 'update', calling method: >, body: {"server": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:06.478666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f62f80b6-721e-4529-91d0-1de986cbf02d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:06.550508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f62f80b6-721e-4529-91d0-1de986cbf02d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:06.551116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f62f80b6-721e-4529-91d0-1de986cbf02d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:06.551710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f62f80b6-721e-4529-91d0-1de986cbf02d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:06.717371 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f62f80b6-721e-4529-91d0-1de986cbf02d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "PUT /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 992 microversion: 2.47 time: 0.254506 Sep 27 10:12:06.718533 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2081/3813] 10.222.0.26 () {62 vars in 1297 bytes} [Tue Sep 27 10:12:06 2022] PUT /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 992 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:12:06.729252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-45e71943-e4d0-447b-9bc5-3e279b9c9adb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:06.730278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-45e71943-e4d0-447b-9bc5-3e279b9c9adb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:06.740838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e71943-e4d0-447b-9bc5-3e279b9c9adb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:06.742304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e71943-e4d0-447b-9bc5-3e279b9c9adb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:06.743127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e71943-e4d0-447b-9bc5-3e279b9c9adb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:06.744968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e71943-e4d0-447b-9bc5-3e279b9c9adb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:06.745711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e71943-e4d0-447b-9bc5-3e279b9c9adb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:06.746389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e71943-e4d0-447b-9bc5-3e279b9c9adb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:06.778312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:07.307619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-45e71943-e4d0-447b-9bc5-3e279b9c9adb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:07.318850 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-45e71943-e4d0-447b-9bc5-3e279b9c9adb tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 0.598521 Sep 27 10:12:07.333421 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2082/3814] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:06 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:07.342956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0ddeae0e-6889-4953-ade2-87c3bc15cdaa tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:07.343862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0ddeae0e-6889-4953-ade2-87c3bc15cdaa tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:07.352281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ddeae0e-6889-4953-ade2-87c3bc15cdaa tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:07.353559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ddeae0e-6889-4953-ade2-87c3bc15cdaa tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:07.354213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ddeae0e-6889-4953-ade2-87c3bc15cdaa tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:07.355572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ddeae0e-6889-4953-ade2-87c3bc15cdaa tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:07.356674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ddeae0e-6889-4953-ade2-87c3bc15cdaa tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:07.358247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ddeae0e-6889-4953-ade2-87c3bc15cdaa tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:07.502773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0ddeae0e-6889-4953-ade2-87c3bc15cdaa tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:07.513089 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0ddeae0e-6889-4953-ade2-87c3bc15cdaa tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1329 microversion: 2.1 time: 0.178112 Sep 27 10:12:07.513873 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2083/3815] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:07 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1329 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:07.518944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:07.519946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:07.526325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:07.527080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:07.527883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:07.528854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:07.529668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:07.530307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:07.651927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:07.670445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:07.677677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:07.681272 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-005385d3-b322-4335-bb32-233d543c5312 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1274 microversion: 2.47 time: 0.164684 Sep 27 10:12:07.682786 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2084/3816] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:07 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1274 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:07.698369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-57387f46-47b9-4ed6-a87d-bb08e9851407 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:07.699450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-57387f46-47b9-4ed6-a87d-bb08e9851407 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:07.713464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57387f46-47b9-4ed6-a87d-bb08e9851407 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:07.714387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57387f46-47b9-4ed6-a87d-bb08e9851407 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:07.715181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57387f46-47b9-4ed6-a87d-bb08e9851407 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:07.723206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57387f46-47b9-4ed6-a87d-bb08e9851407 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:07.727532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57387f46-47b9-4ed6-a87d-bb08e9851407 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:07.727933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57387f46-47b9-4ed6-a87d-bb08e9851407 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:08.201893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-57387f46-47b9-4ed6-a87d-bb08e9851407 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:08.213917 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-57387f46-47b9-4ed6-a87d-bb08e9851407 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1318 microversion: 2.1 time: 0.528450 Sep 27 10:12:08.215151 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2085/3817] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:07 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1318 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:08.346961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fb9e2d82-de9e-4a44-acc7-c080782c4fdd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:08.347936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fb9e2d82-de9e-4a44-acc7-c080782c4fdd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:08.356634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb9e2d82-de9e-4a44-acc7-c080782c4fdd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:08.363807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb9e2d82-de9e-4a44-acc7-c080782c4fdd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:08.364508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb9e2d82-de9e-4a44-acc7-c080782c4fdd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:08.365492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb9e2d82-de9e-4a44-acc7-c080782c4fdd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:08.366375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb9e2d82-de9e-4a44-acc7-c080782c4fdd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:08.367228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb9e2d82-de9e-4a44-acc7-c080782c4fdd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:09.349564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:12:09.350724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:12:09.352181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:12:09.352847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:12:09.353473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:12:09.366959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:09.367923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:09.368706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:09.369419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:09.370154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:09.370749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:09.395851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Getting quotas for project 9c60d0e3692444e39346d2c296c32ac2. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:12:09.407179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Getting quotas for user e4772d35d57f478d929eea70fd11e147 and project 9c60d0e3692444e39346d2c296c32ac2. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:12:09.548780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fb9e2d82-de9e-4a44-acc7-c080782c4fdd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:09.558593 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fb9e2d82-de9e-4a44-acc7-c080782c4fdd tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1504 microversion: 2.1 time: 1.217227 Sep 27 10:12:09.561590 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2086/3818] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:08 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1504 bytes in 1221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:12:09.572609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-59a66133-b0d2-4c43-8355-a32e5e7d5173 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:09.573829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-59a66133-b0d2-4c43-8355-a32e5e7d5173 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:09.590183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a66133-b0d2-4c43-8355-a32e5e7d5173 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:09.591026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a66133-b0d2-4c43-8355-a32e5e7d5173 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:09.592249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a66133-b0d2-4c43-8355-a32e5e7d5173 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:09.593506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a66133-b0d2-4c43-8355-a32e5e7d5173 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:09.593905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a66133-b0d2-4c43-8355-a32e5e7d5173 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:09.594382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a66133-b0d2-4c43-8355-a32e5e7d5173 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:09.661960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:12:09.665629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='boot',updated_at=,uuid=,volume_id='cceacac7-f31d-4e36-b764-2231c93d64eb',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=1,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag='other',updated_at=,uuid=,volume_id='8943dda0-13e7-42e3-bdeb-f9cf11b10058',volume_size=None,volume_type=None), BlockDeviceMapping(attachment_id=,boot_index=2,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='4b2f9c8c-667b-4bff-8590-c93510f8acf3',volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:12:09.757333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59a66133-b0d2-4c43-8355-a32e5e7d5173 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:09.776588 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-59a66133-b0d2-4c43-8355-a32e5e7d5173 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1335 microversion: 2.1 time: 0.213094 Sep 27 10:12:09.777271 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2087/3819] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:09 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1335 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:09.787748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:09.788615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:09.801425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:09.802530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:09.803457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:09.804698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:09.805436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:09.807087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:09.933912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:09.952907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:09.954701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:09.956560 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e77086fa-998f-4e28-a349-45b0491c08d5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1338 microversion: 2.47 time: 0.175601 Sep 27 10:12:09.958288 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2088/3820] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:09 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1338 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:09.978633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aa23722b-b805-42d2-a42b-80707bbfc0d4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:09.980593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aa23722b-b805-42d2-a42b-80707bbfc0d4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:10.006267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa23722b-b805-42d2-a42b-80707bbfc0d4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:10.006847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa23722b-b805-42d2-a42b-80707bbfc0d4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:10.007317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa23722b-b805-42d2-a42b-80707bbfc0d4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:10.008042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa23722b-b805-42d2-a42b-80707bbfc0d4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:10.008506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa23722b-b805-42d2-a42b-80707bbfc0d4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:10.009174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa23722b-b805-42d2-a42b-80707bbfc0d4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:10.407521 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3aa31067-4bde-422c-93fb-6b3bf7180858 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.32 time: 6.848295 Sep 27 10:12:10.409345 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1733/3821] 10.222.0.26 () {62 vars in 1189 bytes} [Tue Sep 27 10:12:03 2022] POST /compute/v2.1/servers => generated 405 bytes in 6853 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:12:10.419476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a7092f3d-afe4-4106-aa5a-f4f0a86b2f2e req-2b10a5a5-6111-40c0-a97d-2036bc88642f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "87e69339-c527-44d1-8566-ce64d503d7a6", "name": "network-vif-deleted", "tag": "eaf55525-ffa8-40a5-9a88-b2f94df3274b"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:10.442740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a7092f3d-afe4-4106-aa5a-f4f0a86b2f2e req-2b10a5a5-6111-40c0-a97d-2036bc88642f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:10.448784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a7092f3d-afe4-4106-aa5a-f4f0a86b2f2e req-2b10a5a5-6111-40c0-a97d-2036bc88642f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:10.449778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a7092f3d-afe4-4106-aa5a-f4f0a86b2f2e req-2b10a5a5-6111-40c0-a97d-2036bc88642f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:10.481515 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a7092f3d-afe4-4106-aa5a-f4f0a86b2f2e req-2b10a5a5-6111-40c0-a97d-2036bc88642f service nova] Creating event network-vif-deleted:eaf55525-ffa8-40a5-9a88-b2f94df3274b for instance 87e69339-c527-44d1-8566-ce64d503d7a6 on nch1-master-1 Sep 27 10:12:10.487942 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a7092f3d-afe4-4106-aa5a-f4f0a86b2f2e req-2b10a5a5-6111-40c0-a97d-2036bc88642f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077050 Sep 27 10:12:10.488645 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1734/3822] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:10.498551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5ad441c2-8148-4d42-8f47-d3f6981306e1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:10.499443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5ad441c2-8148-4d42-8f47-d3f6981306e1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:10.639993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5ad441c2-8148-4d42-8f47-d3f6981306e1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:10.646689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aa23722b-b805-42d2-a42b-80707bbfc0d4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:10.652096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5ad441c2-8148-4d42-8f47-d3f6981306e1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:10.656874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5ad441c2-8148-4d42-8f47-d3f6981306e1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:10.657139 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aa23722b-b805-42d2-a42b-80707bbfc0d4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1318 microversion: 2.1 time: 0.696048 Sep 27 10:12:10.661881 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2089/3823] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:09 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1318 bytes in 703 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:10.667229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:10.668972 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5ad441c2-8148-4d42-8f47-d3f6981306e1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1286 microversion: 2.32 time: 0.165836 Sep 27 10:12:10.673511 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1735/3824] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:10 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1286 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:10.674377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:10.680902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:10.681393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:10.682636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:10.683364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:10.683821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:10.684533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:10.742977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:10.743743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:10.744884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:10.745633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lazy-loading 'fault' on Instance uuid 87e69339-c527-44d1-8566-ce64d503d7a6 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:12:10.808323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90ee6023-bf48-47db-a4a8-638df6aafe7a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:10.810537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-90ee6023-bf48-47db-a4a8-638df6aafe7a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:10.822150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ee6023-bf48-47db-a4a8-638df6aafe7a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:10.823267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ee6023-bf48-47db-a4a8-638df6aafe7a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:10.823905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ee6023-bf48-47db-a4a8-638df6aafe7a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:10.824991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ee6023-bf48-47db-a4a8-638df6aafe7a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:10.825888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ee6023-bf48-47db-a4a8-638df6aafe7a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:10.827276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90ee6023-bf48-47db-a4a8-638df6aafe7a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:10.839472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:10.880846 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2711c1ef-bb11-461e-a814-311728dd9da1 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1303 microversion: 2.1 time: 0.217384 Sep 27 10:12:10.910787 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2090/3825] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:10 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1303 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:10.984563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:10.986916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:10.994259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:10.994767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:10.995229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:10.996139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:10.996611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:11.000775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:11.229130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-90ee6023-bf48-47db-a4a8-638df6aafe7a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:11.246525 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-90ee6023-bf48-47db-a4a8-638df6aafe7a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1402 microversion: 2.1 time: 0.446323 Sep 27 10:12:11.264111 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1736/3826] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:10 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1402 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:11.355072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:11.389292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:11.390873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:11.395616 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f09ee12-16f8-4acd-a9c3-524a1ab380d4 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1336 microversion: 2.47 time: 0.415856 Sep 27 10:12:11.406796 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2091/3827] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:10 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1336 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:11.682729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9868571f-f159-49e0-a4c7-7cfb9d559721 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:11.683601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9868571f-f159-49e0-a4c7-7cfb9d559721 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:11.693245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9868571f-f159-49e0-a4c7-7cfb9d559721 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:11.695113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9868571f-f159-49e0-a4c7-7cfb9d559721 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:11.695903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9868571f-f159-49e0-a4c7-7cfb9d559721 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:11.698864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9868571f-f159-49e0-a4c7-7cfb9d559721 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:11.699597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9868571f-f159-49e0-a4c7-7cfb9d559721 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:11.700481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9868571f-f159-49e0-a4c7-7cfb9d559721 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:11.703999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41f0bc5f-a1b8-4bcf-b9a7-466eb9009fb1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:11.705697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-41f0bc5f-a1b8-4bcf-b9a7-466eb9009fb1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:11.709804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=99 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:11.805548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-41f0bc5f-a1b8-4bcf-b9a7-466eb9009fb1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:11.815901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-41f0bc5f-a1b8-4bcf-b9a7-466eb9009fb1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:11.822086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-41f0bc5f-a1b8-4bcf-b9a7-466eb9009fb1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:11.823188 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-41f0bc5f-a1b8-4bcf-b9a7-466eb9009fb1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1286 microversion: 2.32 time: 0.123622 Sep 27 10:12:11.827273 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2092/3828] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:11 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1286 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:11.912056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:11.912787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:11.925139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:11.925940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:11.926569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:11.927805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:11.931354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:11.937757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:11.991309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:11.992433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:11.993211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:11.993878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lazy-loading 'fault' on Instance uuid 87e69339-c527-44d1-8566-ce64d503d7a6 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:12:12.060003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:12.070564 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-642a5274-b0ec-4449-8a3f-3debd6338d76 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 200 len: 1303 microversion: 2.1 time: 0.162663 Sep 27 10:12:12.073529 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2093/3829] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:11 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 1303 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:12:12.152490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9868571f-f159-49e0-a4c7-7cfb9d559721 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:12.164824 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9868571f-f159-49e0-a4c7-7cfb9d559721 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1318 microversion: 2.1 time: 0.485258 Sep 27 10:12:12.165697 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1737/3830] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:11 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1318 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:12.277460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d744020-d108-44a7-bf7b-5b7d3d058b0a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:12.278560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d744020-d108-44a7-bf7b-5b7d3d058b0a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:12.292097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d744020-d108-44a7-bf7b-5b7d3d058b0a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:12.292615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d744020-d108-44a7-bf7b-5b7d3d058b0a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:12.293112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d744020-d108-44a7-bf7b-5b7d3d058b0a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:12.295174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d744020-d108-44a7-bf7b-5b7d3d058b0a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:12.300118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d744020-d108-44a7-bf7b-5b7d3d058b0a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:12.300704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d744020-d108-44a7-bf7b-5b7d3d058b0a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:12.430039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:12.431221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:12.453934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:12.457191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:12.457677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:12.462596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:12.463114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:12.463602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:12.564035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:12.590578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:12.593906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:12.616705 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0c1f0b52-46e2-47e4-af77-f0702439dfc2 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1336 microversion: 2.47 time: 0.176258 Sep 27 10:12:12.629209 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1738/3831] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:12 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1336 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:12.782763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1d744020-d108-44a7-bf7b-5b7d3d058b0a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:12.792540 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d744020-d108-44a7-bf7b-5b7d3d058b0a tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1402 microversion: 2.1 time: 0.521926 Sep 27 10:12:12.793992 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2094/3832] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:12 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1402 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:12.857376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:12.858113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:12.868831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:12.870077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:12.872558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:12.873471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:12.876364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:12.877085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:13.041086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:13.054177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:13.056479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:13.058177 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5eecd58c-7e68-4c0c-be45-f82fab097e80 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1576 microversion: 2.32 time: 0.207019 Sep 27 10:12:13.059307 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2095/3833] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:12 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1576 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:13.219098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1167410d-52f8-4a2f-8558-792c5ad03397 req-090fcd6f-9e26-4739-a9e6-845fb75bbae4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e7f6860d-d3b1-4897-85dc-3653f32efa7d", "name": "network-vif-plugged", "status": "completed", "tag": "850e37cd-72bd-4ecc-89d4-fc4545104ce6"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:13.247607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1167410d-52f8-4a2f-8558-792c5ad03397 req-090fcd6f-9e26-4739-a9e6-845fb75bbae4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:13.248577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1167410d-52f8-4a2f-8558-792c5ad03397 req-090fcd6f-9e26-4739-a9e6-845fb75bbae4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:13.249441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1167410d-52f8-4a2f-8558-792c5ad03397 req-090fcd6f-9e26-4739-a9e6-845fb75bbae4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:13.271498 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1167410d-52f8-4a2f-8558-792c5ad03397 req-090fcd6f-9e26-4739-a9e6-845fb75bbae4 service nova] Creating event network-vif-plugged:850e37cd-72bd-4ecc-89d4-fc4545104ce6 for instance e7f6860d-d3b1-4897-85dc-3653f32efa7d on nch2-master-1 Sep 27 10:12:13.278059 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1167410d-52f8-4a2f-8558-792c5ad03397 req-090fcd6f-9e26-4739-a9e6-845fb75bbae4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.535162 Sep 27 10:12:13.279472 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1739/3834] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:12:13.287783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:13.288495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 87e69339-c527-44d1-8566-ce64d503d7a6] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:13.299003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:13.299497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:13.300129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:13.300909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:13.301535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:13.302170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:13.321107 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] HTTP exception thrown: Instance 87e69339-c527-44d1-8566-ce64d503d7a6 could not be found. Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Request method failure captured: Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 HTTP/1.1 Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMswCOsUZbybhIYReNau0E3ElRAtfR2bnFpLmwRsb-LPvULYpqx6LqeBYgEKXjjNit5JFjk4cu1DuhT6kpKW9St9V6DtAjlf-AMYYwU7iwfuW7RFmA250kQxr3SYx6fH2VaUrV8oUBwZCqKLuFWusFTeQDAN5cZKRNUblrKVhcVOJcDI Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: c24fd5e915af4b6c872d69b99e7375ce Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerRescueTestJSON-2121572987 Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c24fd5e915af4b6c872d69b99e7375ce"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c24fd5e915af4b6c872d69b99e7375ce"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c24fd5e915af4b6c872d69b99e7375ce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c24fd5e915af4b6c872d69b99e7375ce"}]}] Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerRescueTestJSON-2121572987 Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: c24fd5e915af4b6c872d69b99e7375ce Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerRescueTestJSON-2121572987 Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerRescueTestJSON-2121572987-project Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b11313762b324174a9a5f362d1cf4bdd Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerRescueTestJSON-2121572987-project Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 87e69339-c527-44d1-8566-ce64d503d7a6 could not be found. Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '87e69339-c527-44d1-8566-ce64d503d7a6'} Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 87e69339-c527-44d1-8566-ce64d503d7a6 could not be found. Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 87e69339-c527-44d1-8566-ce64d503d7a6 could not be found. Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:13.336986 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:12:13.342801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:13.342801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:13.342801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:13.342801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:12:13.342801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:12:13.342801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 87e69339-c527-44d1-8566-ce64d503d7a6 could not be found. Sep 27 10:12:13.342801 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:12:13.345879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Returning 404 to user: Instance 87e69339-c527-44d1-8566-ce64d503d7a6 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:12:13.349149 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a4210739-2c6f-47f4-abf7-56006de6093a tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6" status: 404 len: 111 microversion: 2.1 time: 0.066372 Sep 27 10:12:13.354832 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1740/3835] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:13 2022] GET /compute/v2.1/servers/87e69339-c527-44d1-8566-ce64d503d7a6 => generated 111 bytes in 74 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:12:13.360395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-004b9855-fb47-4238-9d6b-b71977201300 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:13.364548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-004b9855-fb47-4238-9d6b-b71977201300 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:13.373421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-004b9855-fb47-4238-9d6b-b71977201300 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:13.374117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-004b9855-fb47-4238-9d6b-b71977201300 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:13.374738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-004b9855-fb47-4238-9d6b-b71977201300 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:13.375644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-004b9855-fb47-4238-9d6b-b71977201300 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:13.376747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-004b9855-fb47-4238-9d6b-b71977201300 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:13.377407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-004b9855-fb47-4238-9d6b-b71977201300 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:13.431363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e56e4188-69b0-43e7-8c2e-f843e977b208 req-df67f239-f498-4965-9801-ef10de45c819 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6cece583-0807-44b9-a6c1-ea77401b7f51", "tag": "ff323f5a-e504-4eb7-9433-e9c539434972"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:13.466953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e56e4188-69b0-43e7-8c2e-f843e977b208 req-df67f239-f498-4965-9801-ef10de45c819 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:13.471971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e56e4188-69b0-43e7-8c2e-f843e977b208 req-df67f239-f498-4965-9801-ef10de45c819 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:13.475212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e56e4188-69b0-43e7-8c2e-f843e977b208 req-df67f239-f498-4965-9801-ef10de45c819 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:13.540536 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e56e4188-69b0-43e7-8c2e-f843e977b208 req-df67f239-f498-4965-9801-ef10de45c819 service nova] Creating event network-changed:ff323f5a-e504-4eb7-9433-e9c539434972 for instance 6cece583-0807-44b9-a6c1-ea77401b7f51 on nch1-master-1 Sep 27 10:12:13.562430 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e56e4188-69b0-43e7-8c2e-f843e977b208 req-df67f239-f498-4965-9801-ef10de45c819 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.498224 Sep 27 10:12:13.564073 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2096/3836] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:13.584579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1e57a856-a219-425b-8f33-fcb91f3ac51b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:13.590420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1e57a856-a219-425b-8f33-fcb91f3ac51b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:13.613528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e57a856-a219-425b-8f33-fcb91f3ac51b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:13.614265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e57a856-a219-425b-8f33-fcb91f3ac51b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:13.615503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e57a856-a219-425b-8f33-fcb91f3ac51b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:13.624203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e57a856-a219-425b-8f33-fcb91f3ac51b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:13.624993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e57a856-a219-425b-8f33-fcb91f3ac51b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:13.625614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e57a856-a219-425b-8f33-fcb91f3ac51b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:13.688861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1e57a856-a219-425b-8f33-fcb91f3ac51b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:12:13.875034 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1e57a856-a219-425b-8f33-fcb91f3ac51b tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 204 len: 0 microversion: 2.1 time: 0.309322 Sep 27 10:12:13.878967 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2097/3837] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:12:13 2022] DELETE /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:12:13.886552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:13.887548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:13.906467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:13.910035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:13.910553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:13.911798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:13.913367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:13.914089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:13.947767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:14.022465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.045812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.048980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.053614 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e3699eda-f313-42fa-8549-a13b5b5aeef5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1357 microversion: 2.47 time: 0.170852 Sep 27 10:12:14.069607 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2098/3838] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:13 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1357 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:14.084111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-edbd45c6-f13b-4d50-ab72-9d88a54a1fde tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:14.085460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-edbd45c6-f13b-4d50-ab72-9d88a54a1fde tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:14.097157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbd45c6-f13b-4d50-ab72-9d88a54a1fde tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.097989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbd45c6-f13b-4d50-ab72-9d88a54a1fde tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.098532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbd45c6-f13b-4d50-ab72-9d88a54a1fde tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.099653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbd45c6-f13b-4d50-ab72-9d88a54a1fde tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.100419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbd45c6-f13b-4d50-ab72-9d88a54a1fde tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.101783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbd45c6-f13b-4d50-ab72-9d88a54a1fde tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.143409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-004b9855-fb47-4238-9d6b-b71977201300 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.158061 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-004b9855-fb47-4238-9d6b-b71977201300 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1318 microversion: 2.1 time: 0.801440 Sep 27 10:12:14.162592 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1741/3839] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:13 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1318 bytes in 806 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:14.171919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eca9b53b-b382-4448-b3ab-81c0ca589159 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:14.171919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eca9b53b-b382-4448-b3ab-81c0ca589159 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:14.180910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eca9b53b-b382-4448-b3ab-81c0ca589159 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.182066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eca9b53b-b382-4448-b3ab-81c0ca589159 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.183731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eca9b53b-b382-4448-b3ab-81c0ca589159 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.184844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eca9b53b-b382-4448-b3ab-81c0ca589159 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.186033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eca9b53b-b382-4448-b3ab-81c0ca589159 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.188197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eca9b53b-b382-4448-b3ab-81c0ca589159 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.297142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-edbd45c6-f13b-4d50-ab72-9d88a54a1fde tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.307095 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-edbd45c6-f13b-4d50-ab72-9d88a54a1fde tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1402 microversion: 2.1 time: 0.234372 Sep 27 10:12:14.308204 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2099/3840] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:14 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1402 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:14.322666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:14.327923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:14.335323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.336120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.336808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.337709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.338718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.339358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.463721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:12:14.465021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:12:14.466160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:12:14.467268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:12:14.467852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:12:14.503944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eca9b53b-b382-4448-b3ab-81c0ca589159 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.518425 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eca9b53b-b382-4448-b3ab-81c0ca589159 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1491 microversion: 2.1 time: 0.353636 Sep 27 10:12:14.520598 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1742/3841] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:14 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1491 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:14.541665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:14.542618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:14.551652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.552584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.553144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.554631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.555414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.557921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.636416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:14.653020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.653972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.654675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.657173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.660418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.661153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.730291 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e9676e3-aa98-41e4-838a-e603c17742a8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "POST /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea/action" status: 202 len: 1079 microversion: 2.47 time: 0.420763 Sep 27 10:12:14.736843 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2100/3842] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:12:14 2022] POST /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea/action => generated 1079 bytes in 428 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Sep 27 10:12:14.762614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:14.763332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:14.763527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.774187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.774658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.775099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.781026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:14.781526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:14.781983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:14.782620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.784252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.788693 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8bd57954-a0c1-4471-9005-c143ea65eb11 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1594 microversion: 2.32 time: 0.265789 Sep 27 10:12:14.789484 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1743/3843] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:14 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1594 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:14.912400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.927332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.931702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:14.934743 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-85cdf29d-0e5f-4fe0-9f79-7fdd7d6a6cc7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.185680 Sep 27 10:12:14.938536 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2101/3844] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:14 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:15.188444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c201b11-f269-4570-837b-5aadb4a484b9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:15.189413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c201b11-f269-4570-837b-5aadb4a484b9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:15.197431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c201b11-f269-4570-837b-5aadb4a484b9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:15.197972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c201b11-f269-4570-837b-5aadb4a484b9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:15.198453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c201b11-f269-4570-837b-5aadb4a484b9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:15.199174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c201b11-f269-4570-837b-5aadb4a484b9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:15.199919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c201b11-f269-4570-837b-5aadb4a484b9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:15.200711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c201b11-f269-4570-837b-5aadb4a484b9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:15.335354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-095a8452-99ad-4dc0-b71a-cfa1bdf2ed33 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:15.336875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-095a8452-99ad-4dc0-b71a-cfa1bdf2ed33 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:15.353680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-095a8452-99ad-4dc0-b71a-cfa1bdf2ed33 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:15.354756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-095a8452-99ad-4dc0-b71a-cfa1bdf2ed33 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:15.356003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-095a8452-99ad-4dc0-b71a-cfa1bdf2ed33 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:15.358720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-095a8452-99ad-4dc0-b71a-cfa1bdf2ed33 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:15.359290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-095a8452-99ad-4dc0-b71a-cfa1bdf2ed33 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:15.359936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-095a8452-99ad-4dc0-b71a-cfa1bdf2ed33 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:15.487522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c201b11-f269-4570-837b-5aadb4a484b9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:15.506227 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c201b11-f269-4570-837b-5aadb4a484b9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1318 microversion: 2.1 time: 0.323149 Sep 27 10:12:15.525569 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1744/3845] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:15 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1318 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:15.544228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1deee1b7-f25c-4fe5-b41a-31e3bf99a905 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:15.545080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1deee1b7-f25c-4fe5-b41a-31e3bf99a905 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:15.555599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1deee1b7-f25c-4fe5-b41a-31e3bf99a905 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:15.556102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1deee1b7-f25c-4fe5-b41a-31e3bf99a905 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:15.556538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1deee1b7-f25c-4fe5-b41a-31e3bf99a905 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:15.558189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1deee1b7-f25c-4fe5-b41a-31e3bf99a905 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:15.561021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1deee1b7-f25c-4fe5-b41a-31e3bf99a905 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:15.562560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1deee1b7-f25c-4fe5-b41a-31e3bf99a905 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:15.604571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-095a8452-99ad-4dc0-b71a-cfa1bdf2ed33 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:15.617369 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-095a8452-99ad-4dc0-b71a-cfa1bdf2ed33 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1402 microversion: 2.1 time: 0.287538 Sep 27 10:12:15.618484 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2102/3846] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:15 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1402 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:12:15.818302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:15.824394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:15.825329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:15.837761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:15.838704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:15.848128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:15.849245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:15.850535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:15.851556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:15.991294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:16.010979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:16.014161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:16.015892 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2824c558-5ae1-4a6a-bd1f-6ac00e5e1752 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1594 microversion: 2.32 time: 0.199553 Sep 27 10:12:16.022718 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2103/3847] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:15 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1594 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:16.028445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:16.038453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:16.052707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:16.053912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:16.055802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:16.060104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:16.061025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:16.062042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:16.192206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:16.210315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:16.211973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:16.213614 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b034b638-0b95-4324-a4e5-375521fc4227 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.189337 Sep 27 10:12:16.214491 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2104/3848] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:16 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:16.408720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1deee1b7-f25c-4fe5-b41a-31e3bf99a905 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:16.423874 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1deee1b7-f25c-4fe5-b41a-31e3bf99a905 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1491 microversion: 2.1 time: 0.884189 Sep 27 10:12:16.426289 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1745/3849] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:15 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1491 bytes in 887 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:16.533477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e13be300-d8f2-4109-9e06-66278c95cae4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:16.534705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e13be300-d8f2-4109-9e06-66278c95cae4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:16.545076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e13be300-d8f2-4109-9e06-66278c95cae4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:16.545828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e13be300-d8f2-4109-9e06-66278c95cae4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:16.546335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e13be300-d8f2-4109-9e06-66278c95cae4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:16.547084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e13be300-d8f2-4109-9e06-66278c95cae4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:16.547889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e13be300-d8f2-4109-9e06-66278c95cae4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:16.548280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e13be300-d8f2-4109-9e06-66278c95cae4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:16.644650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-09ff6412-3fbe-4e49-9853-c7d78234ea49 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:16.645912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-09ff6412-3fbe-4e49-9853-c7d78234ea49 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:16.653337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09ff6412-3fbe-4e49-9853-c7d78234ea49 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:16.658251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09ff6412-3fbe-4e49-9853-c7d78234ea49 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:16.658693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09ff6412-3fbe-4e49-9853-c7d78234ea49 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:16.659635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09ff6412-3fbe-4e49-9853-c7d78234ea49 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:16.660103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09ff6412-3fbe-4e49-9853-c7d78234ea49 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:16.660531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09ff6412-3fbe-4e49-9853-c7d78234ea49 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:16.868787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e13be300-d8f2-4109-9e06-66278c95cae4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:16.882376 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e13be300-d8f2-4109-9e06-66278c95cae4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1506 microversion: 2.1 time: 0.352492 Sep 27 10:12:16.893047 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2105/3850] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:16 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1506 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:17.052998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:17.055136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:17.068003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:17.069585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:17.071068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:17.072150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:17.086704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:17.088244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:17.220068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-09ff6412-3fbe-4e49-9853-c7d78234ea49 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:17.234052 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-09ff6412-3fbe-4e49-9853-c7d78234ea49 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1402 microversion: 2.1 time: 0.593222 Sep 27 10:12:17.235494 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1746/3851] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:16 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1402 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:17.259828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:17.264038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:17.280208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:17.281063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:17.283360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:17.284305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:17.286015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:17.288438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:17.504750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:17.505077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:17.530482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:17.534302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:17.538053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:17.538980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:17.541984 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d993a689-d180-4a4f-b5e3-807fa20394f9 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1661 microversion: 2.32 time: 0.495034 Sep 27 10:12:17.543964 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-47a086aa-c24e-46b7-acd2-2bf42fe0a4dc tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.303436 Sep 27 10:12:17.546769 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1747/3852] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:17 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:17.557153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-201fb0bb-cbe1-4f7c-a67f-3bc126f7cf33 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:17.558017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-201fb0bb-cbe1-4f7c-a67f-3bc126f7cf33 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:17.559114 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2106/3853] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:17 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1661 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:17.573417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-201fb0bb-cbe1-4f7c-a67f-3bc126f7cf33 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:17.577458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-201fb0bb-cbe1-4f7c-a67f-3bc126f7cf33 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:17.578117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-201fb0bb-cbe1-4f7c-a67f-3bc126f7cf33 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:17.579043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-201fb0bb-cbe1-4f7c-a67f-3bc126f7cf33 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:17.579722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-201fb0bb-cbe1-4f7c-a67f-3bc126f7cf33 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:17.580320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-201fb0bb-cbe1-4f7c-a67f-3bc126f7cf33 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:18.259904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7cd19ba6-a401-49f3-a0ae-1b785007972f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:18.261392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7cd19ba6-a401-49f3-a0ae-1b785007972f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:18.274860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:18.280464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cd19ba6-a401-49f3-a0ae-1b785007972f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:18.282109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cd19ba6-a401-49f3-a0ae-1b785007972f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:18.283619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cd19ba6-a401-49f3-a0ae-1b785007972f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:18.285136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cd19ba6-a401-49f3-a0ae-1b785007972f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:18.285585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cd19ba6-a401-49f3-a0ae-1b785007972f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:18.287221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cd19ba6-a401-49f3-a0ae-1b785007972f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:18.577809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-201fb0bb-cbe1-4f7c-a67f-3bc126f7cf33 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b11313762b324174a9a5f362d1cf4bdd', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c24fd5e915af4b6c872d69b99e7375ce', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:18.590871 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-201fb0bb-cbe1-4f7c-a67f-3bc126f7cf33 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 200 len: 1491 microversion: 2.1 time: 1.041038 Sep 27 10:12:18.595251 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1748/3854] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:17 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 1491 bytes in 1048 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:18.606797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:18.607433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:18.619905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:18.620637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:18.621354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:18.625551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:18.626232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:18.626840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:18.828799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7cd19ba6-a401-49f3-a0ae-1b785007972f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:18.842891 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7cd19ba6-a401-49f3-a0ae-1b785007972f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1402 microversion: 2.1 time: 0.588091 Sep 27 10:12:18.844054 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2107/3855] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:18 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1402 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:18.849900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:18.852553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:18.860371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:18.861180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:18.876359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:18.877587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:18.878130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:18.878732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:19.047768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:19.060361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:19.061859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:19.063057 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b113d20f-8d19-4d06-a34a-13f03a62dac7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.217413 Sep 27 10:12:19.063639 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2108/3856] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:18 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:19.082111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Action: 'update', calling method: >, body: {"volumeAttachment": {"volumeId": "a57cf025-29a4-477b-9b8a-c89e9db47889"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:19.086150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:19.095509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:19.096312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:19.098210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:19.099345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:19.100786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:19.101650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:19.180571 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] HTTP exception thrown: Volume 2f927731-1f80-4cb7-9cdc-e9d23dc7dd93 could not be found. Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Request method failure captured: Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: request: PUT /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d/os-volume_attachments/2f927731-1f80-4cb7-9cdc-e9d23dc7dd93 HTTP/1.1 Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 74 Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsvsXYS9VZ3zLv66ebSIwOPabHVzHI_2IQRkJzQYxVaADGwQSuJEqfGvPXthG1Mxtuuf2tI5uRd-RxKyFaY0uxKmOOhdqeA-P28oeT2sCTk-ZmdPb6Ll2KLIrOgF33UiPDp3dNMOXvK4CgxlYhEZx_sHxoKu_tYKb2svV8e6NHev3Z0 Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 688812897dc54f40ad95e885b40a23f5 Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesAdminNegativeTest-1082822564 Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/688812897dc54f40ad95e885b40a23f5"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/688812897dc54f40ad95e885b40a23f5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/688812897dc54f40ad95e885b40a23f5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_688812897dc54f40ad95e885b40a23f5"}]}] Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesAdminNegativeTest-1082822564 Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 688812897dc54f40ad95e885b40a23f5 Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-1082822564 Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesAdminNegativeTest-1082822564-project-admin Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 31e653b281b64ade86e15660158ff07e Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesAdminNegativeTest-1082822564-project-admin Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: {"volumeAttachment": {"volumeId": "a57cf025-29a4-477b-9b8a-c89e9db47889"}} Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: exception: Volume 2f927731-1f80-4cb7-9cdc-e9d23dc7dd93 could not be found. Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'server_id': 'e7f6860d-d3b1-4897-85dc-3653f32efa7d', 'id': '2f927731-1f80-4cb7-9cdc-e9d23dc7dd93', 'body': {'volumeAttachment': {'volumeId': 'a57cf025-29a4-477b-9b8a-c89e9db47889'}}} Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 2f927731-1f80-4cb7-9cdc-e9d23dc7dd93 could not be found. (HTTP 404) (Request-ID: req-ac66e885-6bc1-4e8d-8cf4-2c70ad99dff6) Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 430, in _update_volume_swap Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi old_volume = self.volume_api.get(context, old_volume_id) Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 10:12:19.183068 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 2f927731-1f80-4cb7-9cdc-e9d23dc7dd93 could not be found. Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 522, in update Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._update_volume_swap(req, instance, id, body) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 432, in _update_volume_swap Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 2f927731-1f80-4cb7-9cdc-e9d23dc7dd93 could not be found. Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 2f927731-1f80-4cb7-9cdc-e9d23dc7dd93 could not be found. Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] Returning 404 to user: Volume 2f927731-1f80-4cb7-9cdc-e9d23dc7dd93 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-41524de9-7b8a-468e-9158-dea9f718812a tempest-VolumesAdminNegativeTest-1082822564 tempest-VolumesAdminNegativeTest-1082822564-project-admin] 10.222.0.26 "PUT /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d/os-volume_attachments/2f927731-1f80-4cb7-9cdc-e9d23dc7dd93" status: 404 len: 109 microversion: 2.1 time: 0.120048 Sep 27 10:12:19.196315 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2109/3857] 10.222.0.26 () {60 vars in 1433 bytes} [Tue Sep 27 10:12:19 2022] PUT /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d/os-volume_attachments/2f927731-1f80-4cb7-9cdc-e9d23dc7dd93 => generated 109 bytes in 121 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:12:19.204982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-12cf045f-6d8f-4dbe-860c-8cafde4e078a req-901f20ec-9c7d-4de2-bb59-1deda4b07c4b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3651e4da-f0ce-4e37-82ac-9b81a1564a1d", "name": "network-vif-deleted", "tag": "87d895c7-9525-40c2-baef-ff15ac876653"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:19.214722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-12cf045f-6d8f-4dbe-860c-8cafde4e078a req-901f20ec-9c7d-4de2-bb59-1deda4b07c4b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:19.215227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-12cf045f-6d8f-4dbe-860c-8cafde4e078a req-901f20ec-9c7d-4de2-bb59-1deda4b07c4b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:19.215668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-12cf045f-6d8f-4dbe-860c-8cafde4e078a req-901f20ec-9c7d-4de2-bb59-1deda4b07c4b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:19.232574 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-12cf045f-6d8f-4dbe-860c-8cafde4e078a req-901f20ec-9c7d-4de2-bb59-1deda4b07c4b service nova] Creating event network-vif-deleted:87d895c7-9525-40c2-baef-ff15ac876653 for instance 3651e4da-f0ce-4e37-82ac-9b81a1564a1d on nch1-master-1 Sep 27 10:12:19.239301 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-12cf045f-6d8f-4dbe-860c-8cafde4e078a req-901f20ec-9c7d-4de2-bb59-1deda4b07c4b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052684 Sep 27 10:12:19.239959 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2110/3858] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:19.255495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:19.273324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:19.275141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:19.276331 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2f273d42-05b0-47e2-b377-0ef6f1e08a7a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1661 microversion: 2.32 time: 0.679910 Sep 27 10:12:19.278119 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1749/3859] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:18 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1661 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:19.636142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:19.637083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] [instance: 3651e4da-f0ce-4e37-82ac-9b81a1564a1d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:19.653387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:19.654249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:19.655131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:19.656097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:19.656895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:19.657529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:19.684542 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] HTTP exception thrown: Instance 3651e4da-f0ce-4e37-82ac-9b81a1564a1d could not be found. Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Request method failure captured: Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d HTTP/1.1 Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMswCOsUZbybhIYReNau0E3ElRAtfR2bnFpLmwRsb-LPvULYpqx6LqeBYgEKXjjNit5JFjk4cu1DuhT6kpKW9St9V6DtAjlf-AMYYwU7iwfuW7RFmA250kQxr3SYx6fH2VaUrV8oUBwZCqKLuFWusFTeQDAN5cZKRNUblrKVhcVOJcDI Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: c24fd5e915af4b6c872d69b99e7375ce Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerRescueTestJSON-2121572987 Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c24fd5e915af4b6c872d69b99e7375ce"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c24fd5e915af4b6c872d69b99e7375ce"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c24fd5e915af4b6c872d69b99e7375ce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c24fd5e915af4b6c872d69b99e7375ce"}]}] Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerRescueTestJSON-2121572987 Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: c24fd5e915af4b6c872d69b99e7375ce Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerRescueTestJSON-2121572987 Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerRescueTestJSON-2121572987-project Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b11313762b324174a9a5f362d1cf4bdd Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerRescueTestJSON-2121572987-project Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 3651e4da-f0ce-4e37-82ac-9b81a1564a1d could not be found. Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3651e4da-f0ce-4e37-82ac-9b81a1564a1d'} Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3651e4da-f0ce-4e37-82ac-9b81a1564a1d could not be found. Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3651e4da-f0ce-4e37-82ac-9b81a1564a1d could not be found. Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:19.687117 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.692868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:19.692868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:19.692868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:19.692868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:12:19.692868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:12:19.692868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3651e4da-f0ce-4e37-82ac-9b81a1564a1d could not be found. Sep 27 10:12:19.692868 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:19.695598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] Returning 404 to user: Instance 3651e4da-f0ce-4e37-82ac-9b81a1564a1d could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:12:19.696729 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-29b4734a-551a-42d7-9c50-ac9813f19021 tempest-ServerRescueTestJSON-2121572987 tempest-ServerRescueTestJSON-2121572987-project] 10.222.0.26 "GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d" status: 404 len: 111 microversion: 2.1 time: 0.077461 Sep 27 10:12:19.697961 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2111/3860] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:19 2022] GET /compute/v2.1/servers/3651e4da-f0ce-4e37-82ac-9b81a1564a1d => generated 111 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:12:19.863953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:19.870676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d36cedf2-1b16-4860-9093-03538a02a2ce tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:19.871332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d36cedf2-1b16-4860-9093-03538a02a2ce tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:19.879378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d36cedf2-1b16-4860-9093-03538a02a2ce tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:19.879871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d36cedf2-1b16-4860-9093-03538a02a2ce tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:19.880312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d36cedf2-1b16-4860-9093-03538a02a2ce tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:19.881116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d36cedf2-1b16-4860-9093-03538a02a2ce tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:19.882051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d36cedf2-1b16-4860-9093-03538a02a2ce tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:19.882930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d36cedf2-1b16-4860-9093-03538a02a2ce tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:20.093085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:20.093698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:20.105347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:20.105922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:20.106802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:20.107783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:20.108254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:20.108725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:20.216672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:20.236545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:20.241137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:20.243949 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-678b33b6-548b-44aa-8448-03f1ed03a2b3 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.154418 Sep 27 10:12:20.244936 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2112/3861] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:20 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:12:20.269094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d36cedf2-1b16-4860-9093-03538a02a2ce tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:20.283173 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d36cedf2-1b16-4860-9093-03538a02a2ce tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1402 microversion: 2.1 time: 0.420112 Sep 27 10:12:20.283755 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1750/3862] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:19 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1402 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:20.308824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:20.309865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:20.318311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:20.320868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:20.321541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:20.322453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:20.323087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:20.323678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:20.480346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1ca9122c-f476-4047-89c1-2a1881140cfe tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:20.480870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1ca9122c-f476-4047-89c1-2a1881140cfe tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:20.489334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ca9122c-f476-4047-89c1-2a1881140cfe tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:20.489986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ca9122c-f476-4047-89c1-2a1881140cfe tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:20.490563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ca9122c-f476-4047-89c1-2a1881140cfe tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:20.491770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ca9122c-f476-4047-89c1-2a1881140cfe tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:20.492323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ca9122c-f476-4047-89c1-2a1881140cfe tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:20.492924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ca9122c-f476-4047-89c1-2a1881140cfe tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:20.538753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1ca9122c-f476-4047-89c1-2a1881140cfe tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:12:20.664861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=14 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:20.703596 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1ca9122c-f476-4047-89c1-2a1881140cfe tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 204 len: 0 microversion: 2.1 time: 0.224942 Sep 27 10:12:20.713615 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1751/3863] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:12:20 2022] DELETE /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:12:20.723504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-08b30a68-b5c9-40eb-b723-83ad40170400 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:20.724231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-08b30a68-b5c9-40eb-b723-83ad40170400 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:20.727813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:20.740292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08b30a68-b5c9-40eb-b723-83ad40170400 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:20.741274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08b30a68-b5c9-40eb-b723-83ad40170400 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:20.741827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:20.742934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08b30a68-b5c9-40eb-b723-83ad40170400 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:20.744894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08b30a68-b5c9-40eb-b723-83ad40170400 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:20.745588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08b30a68-b5c9-40eb-b723-83ad40170400 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:20.746191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08b30a68-b5c9-40eb-b723-83ad40170400 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:20.749109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:20.760003 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-57d74ed2-deb6-49ca-86d9-49144ffbbb09 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1695 microversion: 2.32 time: 0.449392 Sep 27 10:12:20.763720 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2113/3864] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:20 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1695 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:21.064622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-08b30a68-b5c9-40eb-b723-83ad40170400 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:21.080380 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-08b30a68-b5c9-40eb-b723-83ad40170400 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1512 microversion: 2.1 time: 0.361050 Sep 27 10:12:21.081571 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1752/3865] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:20 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1512 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:21.276954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:21.278817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:21.289417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:21.289417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:21.289417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:21.289417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:21.289417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:21.309164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:21.311277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e0a8769b-e54b-4d95-b909-203225bb0c9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:21.312141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e0a8769b-e54b-4d95-b909-203225bb0c9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:21.320914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0a8769b-e54b-4d95-b909-203225bb0c9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:21.321748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0a8769b-e54b-4d95-b909-203225bb0c9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:21.323125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0a8769b-e54b-4d95-b909-203225bb0c9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:21.328633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0a8769b-e54b-4d95-b909-203225bb0c9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:21.330120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0a8769b-e54b-4d95-b909-203225bb0c9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:21.330879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0a8769b-e54b-4d95-b909-203225bb0c9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:21.360566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:21.456103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:21.460505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:21.471590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:21.473744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:21.474941 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bf2786f5-65c7-48f4-8722-25330826575d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.205694 Sep 27 10:12:21.476546 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2114/3866] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:21 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:21.526035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e0a8769b-e54b-4d95-b909-203225bb0c9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:21.533987 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e0a8769b-e54b-4d95-b909-203225bb0c9d tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1402 microversion: 2.1 time: 0.228668 Sep 27 10:12:21.534548 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1753/3867] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:21 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1402 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:12:21.802766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:21.806433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:21.819292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=38 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:21.822575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:21.823160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:21.823821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:21.824829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:21.825494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:21.826284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:22.124769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea6b7222-3b8c-4cc2-8a14-54ba043c751e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:22.127177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ea6b7222-3b8c-4cc2-8a14-54ba043c751e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:22.136757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea6b7222-3b8c-4cc2-8a14-54ba043c751e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:22.138387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea6b7222-3b8c-4cc2-8a14-54ba043c751e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:22.139534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea6b7222-3b8c-4cc2-8a14-54ba043c751e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:22.140788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea6b7222-3b8c-4cc2-8a14-54ba043c751e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:22.141604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea6b7222-3b8c-4cc2-8a14-54ba043c751e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:22.142465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea6b7222-3b8c-4cc2-8a14-54ba043c751e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:22.217006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:22.229958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:22.233432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:22.235364 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5f76b29c-eb5e-46d5-8758-cf2ed51afc0d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1741 microversion: 2.32 time: 0.438557 Sep 27 10:12:22.236735 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2115/3868] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:21 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1741 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:22.448804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ea6b7222-3b8c-4cc2-8a14-54ba043c751e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:22.461492 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea6b7222-3b8c-4cc2-8a14-54ba043c751e tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1512 microversion: 2.1 time: 0.344079 Sep 27 10:12:22.469075 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1754/3869] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:22 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1512 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:22.502823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:22.504135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:22.513197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:22.514445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:22.515049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:22.515951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:22.516564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:22.517173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:22.561849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e5c733db-071f-440b-a7b5-611a529df45f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:22.562563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e5c733db-071f-440b-a7b5-611a529df45f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:22.572255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:22.574842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c733db-071f-440b-a7b5-611a529df45f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:22.575844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c733db-071f-440b-a7b5-611a529df45f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:22.577380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c733db-071f-440b-a7b5-611a529df45f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:22.578759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c733db-071f-440b-a7b5-611a529df45f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:22.579534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c733db-071f-440b-a7b5-611a529df45f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:22.580610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c733db-071f-440b-a7b5-611a529df45f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:22.659917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:22.674569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:22.676225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:22.677658 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c7dee516-3dcf-4890-a4c7-0e05fae5da93 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.180710 Sep 27 10:12:22.678478 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2116/3870] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:22 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:22.978442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e5c733db-071f-440b-a7b5-611a529df45f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:22.990708 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e5c733db-071f-440b-a7b5-611a529df45f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1402 microversion: 2.1 time: 0.436213 Sep 27 10:12:22.991804 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1755/3871] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:22 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1402 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:23.265857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:23.266636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:23.274285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:23.274772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:23.275482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:23.276451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:23.277295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:23.277899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:23.425539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5de2724d-9230-4412-a0aa-3aa7a3cc938c req-b2f3d294-190e-44c1-994f-1453c5dc0f54 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cece583-0807-44b9-a6c1-ea77401b7f51", "name": "network-vif-plugged", "status": "completed", "tag": "ff323f5a-e504-4eb7-9433-e9c539434972"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:23.456365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5de2724d-9230-4412-a0aa-3aa7a3cc938c req-b2f3d294-190e-44c1-994f-1453c5dc0f54 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:23.457013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5de2724d-9230-4412-a0aa-3aa7a3cc938c req-b2f3d294-190e-44c1-994f-1453c5dc0f54 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:23.467137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5de2724d-9230-4412-a0aa-3aa7a3cc938c req-b2f3d294-190e-44c1-994f-1453c5dc0f54 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:23.507158 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5de2724d-9230-4412-a0aa-3aa7a3cc938c req-b2f3d294-190e-44c1-994f-1453c5dc0f54 service nova] Creating event network-vif-plugged:ff323f5a-e504-4eb7-9433-e9c539434972 for instance 6cece583-0807-44b9-a6c1-ea77401b7f51 on nch1-master-1 Sep 27 10:12:23.514369 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5de2724d-9230-4412-a0aa-3aa7a3cc938c req-b2f3d294-190e-44c1-994f-1453c5dc0f54 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.092677 Sep 27 10:12:23.524539 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1756/3872] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:23.537653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a6bc906b-97c8-4d75-a8aa-7cccd8c6cc36 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:23.550993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a6bc906b-97c8-4d75-a8aa-7cccd8c6cc36 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:23.563788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bc906b-97c8-4d75-a8aa-7cccd8c6cc36 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:23.564582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bc906b-97c8-4d75-a8aa-7cccd8c6cc36 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:23.565406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bc906b-97c8-4d75-a8aa-7cccd8c6cc36 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:23.566406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bc906b-97c8-4d75-a8aa-7cccd8c6cc36 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:23.567047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bc906b-97c8-4d75-a8aa-7cccd8c6cc36 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:23.567688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6bc906b-97c8-4d75-a8aa-7cccd8c6cc36 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:23.775109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a6bc906b-97c8-4d75-a8aa-7cccd8c6cc36 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:23.790147 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a6bc906b-97c8-4d75-a8aa-7cccd8c6cc36 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1470 microversion: 2.1 time: 0.261194 Sep 27 10:12:23.809081 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1757/3873] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:23 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1470 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:23.821814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:23.822843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:23.839808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:23.840783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:23.841695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:23.843238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:23.843928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:23.844576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:24.164773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:24.180365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:24.181988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:24.183229 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c841cd51-2b7e-4a7b-ab00-eb90ceeda6c8 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1787 microversion: 2.32 time: 0.921960 Sep 27 10:12:24.187288 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2117/3874] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:23 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1787 bytes in 927 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:24.201656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fcf3b5fa-2bfa-4400-bab9-7ac759a8522f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:24.209405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fcf3b5fa-2bfa-4400-bab9-7ac759a8522f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:24.218723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcf3b5fa-2bfa-4400-bab9-7ac759a8522f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:24.219516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcf3b5fa-2bfa-4400-bab9-7ac759a8522f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:24.220216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcf3b5fa-2bfa-4400-bab9-7ac759a8522f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:24.221217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcf3b5fa-2bfa-4400-bab9-7ac759a8522f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:24.221860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcf3b5fa-2bfa-4400-bab9-7ac759a8522f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:24.222607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcf3b5fa-2bfa-4400-bab9-7ac759a8522f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:24.383202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:24.396210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:24.399035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:24.400507 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ae8ccf5f-b47c-433d-a2c8-0562f9be37bd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.589937 Sep 27 10:12:24.401243 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1758/3875] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:23 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:24.494717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-8ad53321-e909-4a2f-a55a-91f39373f72d req-2cba0011-62e4-4268-ac86-56d257822d64 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e7f6860d-d3b1-4897-85dc-3653f32efa7d", "name": "network-vif-deleted", "tag": "850e37cd-72bd-4ecc-89d4-fc4545104ce6"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:24.523242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8ad53321-e909-4a2f-a55a-91f39373f72d req-2cba0011-62e4-4268-ac86-56d257822d64 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:24.524139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8ad53321-e909-4a2f-a55a-91f39373f72d req-2cba0011-62e4-4268-ac86-56d257822d64 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:24.525007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8ad53321-e909-4a2f-a55a-91f39373f72d req-2cba0011-62e4-4268-ac86-56d257822d64 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:24.526007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fcf3b5fa-2bfa-4400-bab9-7ac759a8522f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:24.541873 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8ad53321-e909-4a2f-a55a-91f39373f72d req-2cba0011-62e4-4268-ac86-56d257822d64 service nova] Creating event network-vif-deleted:850e37cd-72bd-4ecc-89d4-fc4545104ce6 for instance e7f6860d-d3b1-4897-85dc-3653f32efa7d on nch2-master-1 Sep 27 10:12:24.547720 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fcf3b5fa-2bfa-4400-bab9-7ac759a8522f tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1402 microversion: 2.1 time: 0.353246 Sep 27 10:12:24.549031 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-8ad53321-e909-4a2f-a55a-91f39373f72d req-2cba0011-62e4-4268-ac86-56d257822d64 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061716 Sep 27 10:12:24.550468 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1759/3876] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:24.551222 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2118/3877] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:24 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1402 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:24.823224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:24.824008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:24.832315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:24.832808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:24.833815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:24.834914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:24.835661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:24.836381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:24.879545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:24.880617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:24.881279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:24.882060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lazy-loading 'fault' on Instance uuid e7f6860d-d3b1-4897-85dc-3653f32efa7d {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:12:24.976491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:24.988693 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d08e40dd-ffc9-4e75-9f5a-83a32d0b76d9 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 200 len: 1308 microversion: 2.1 time: 0.173571 Sep 27 10:12:24.993460 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1760/3878] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:24 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 1308 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:25.219964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:25.220309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:25.236596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:25.237331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:25.237878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:25.238709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:25.239986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:25.241100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:25.435801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:25.436757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:25.447089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:25.450065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:25.451839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:25.454174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:25.454851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:25.455283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:25.577847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:25.591052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:25.593329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:25.594670 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9f983630-ea3a-4450-8b34-91e72f4d5a39 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.163493 Sep 27 10:12:25.609518 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1761/3879] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:25 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:12:25.609518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8e280028-771a-4c95-96f8-35663d249044 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:25.609518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8e280028-771a-4c95-96f8-35663d249044 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:25.627857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e280028-771a-4c95-96f8-35663d249044 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:25.629008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e280028-771a-4c95-96f8-35663d249044 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:25.631333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e280028-771a-4c95-96f8-35663d249044 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:25.646402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e280028-771a-4c95-96f8-35663d249044 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:25.649248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e280028-771a-4c95-96f8-35663d249044 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:25.653485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8e280028-771a-4c95-96f8-35663d249044 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:25.692390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:25.702867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:25.704308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:25.718514 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3927a2dc-78a9-4a69-b067-b09929ef701c tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1787 microversion: 2.32 time: 0.507588 Sep 27 10:12:25.722634 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2119/3880] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:25 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1787 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:25.941889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8e280028-771a-4c95-96f8-35663d249044 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:25.952166 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8e280028-771a-4c95-96f8-35663d249044 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1402 microversion: 2.1 time: 0.353850 Sep 27 10:12:25.952784 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1762/3881] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:25 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1402 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:26.015692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:26.016338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: e7f6860d-d3b1-4897-85dc-3653f32efa7d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:26.025343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:26.025810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:26.026445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:26.027335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:26.028196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:26.028633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:26.045817 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] HTTP exception thrown: Instance e7f6860d-d3b1-4897-85dc-3653f32efa7d could not be found. Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Request method failure captured: Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d HTTP/1.1 Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsvSKjxFpoBIP2u6EFp3My1Yqd9bBj3m8CgbWWapDBUvT3WdvklXX3eu9ZGf4jBd8D17DcVKcKM43nenVqW9JlM_cE_xo90emZytpqfuRzVgzxepCgMSBWlLsDXm8aqn6P7_ohi25VJhTY4FOf80OJBz243mXcrX8PJe8YbrGMeVpkM Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f847bee14ff74dfbafe7c6a199f1b99f Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesAdminNegativeTest-1801268990 Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f847bee14ff74dfbafe7c6a199f1b99f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f847bee14ff74dfbafe7c6a199f1b99f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f847bee14ff74dfbafe7c6a199f1b99f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f847bee14ff74dfbafe7c6a199f1b99f"}]}] Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesAdminNegativeTest-1801268990 Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f847bee14ff74dfbafe7c6a199f1b99f Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-1801268990 Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesAdminNegativeTest-1801268990-project Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b8d6f6c63fe144959139278ca287e2ec Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesAdminNegativeTest-1801268990-project Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance e7f6860d-d3b1-4897-85dc-3653f32efa7d could not be found. Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'e7f6860d-d3b1-4897-85dc-3653f32efa7d'} Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e7f6860d-d3b1-4897-85dc-3653f32efa7d could not be found. Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e7f6860d-d3b1-4897-85dc-3653f32efa7d could not be found. Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:26.047625 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:26.057003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:26.057003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:26.057003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:26.057003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:12:26.057003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:12:26.057003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e7f6860d-d3b1-4897-85dc-3653f32efa7d could not be found. Sep 27 10:12:26.057003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:26.057003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Returning 404 to user: Instance e7f6860d-d3b1-4897-85dc-3653f32efa7d could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:12:26.057003 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-01ad5fbd-9cfe-4efd-9049-d08269fd4995 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d" status: 404 len: 111 microversion: 2.1 time: 0.038430 Sep 27 10:12:26.064770 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2120/3882] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:26 2022] GET /compute/v2.1/servers/e7f6860d-d3b1-4897-85dc-3653f32efa7d => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:12:26.631963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:26.632815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:26.656624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:26.666393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:26.667526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:26.668685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:26.670835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:26.671418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:26.759544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:26.761217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:26.782062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:26.782280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:26.782549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:26.783171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:26.784050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:26.784701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:26.786212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:26.824155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:26.830450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:26.843326 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-657db4e6-8d7e-4c36-8496-4f9ae0b4fc01 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.209781 Sep 27 10:12:26.843326 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1763/3883] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:26 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:26.979155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c39c7eae-d390-420b-8cee-ce1dcf742c91 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:26.982299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c39c7eae-d390-420b-8cee-ce1dcf742c91 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:26.992267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c39c7eae-d390-420b-8cee-ce1dcf742c91 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:26.992997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c39c7eae-d390-420b-8cee-ce1dcf742c91 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:26.993943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c39c7eae-d390-420b-8cee-ce1dcf742c91 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:26.995069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c39c7eae-d390-420b-8cee-ce1dcf742c91 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:26.995575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c39c7eae-d390-420b-8cee-ce1dcf742c91 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:26.996241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c39c7eae-d390-420b-8cee-ce1dcf742c91 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:27.258802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:27.287210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:27.293026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:27.294938 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8c173490-1cf9-41ed-9982-17148313959f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1833 microversion: 2.32 time: 0.543304 Sep 27 10:12:27.300152 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2121/3884] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:26 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1833 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:27.323626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8e55cf39-b940-4445-a8f6-774bcf0bb165 req-2feb727a-0ed8-4f63-8f6a-833a2e36eef2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "tag": "10f31ce3-e11c-4b44-bb7f-173c2919d9b7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:27.369679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8e55cf39-b940-4445-a8f6-774bcf0bb165 req-2feb727a-0ed8-4f63-8f6a-833a2e36eef2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:27.373575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8e55cf39-b940-4445-a8f6-774bcf0bb165 req-2feb727a-0ed8-4f63-8f6a-833a2e36eef2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:27.374134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8e55cf39-b940-4445-a8f6-774bcf0bb165 req-2feb727a-0ed8-4f63-8f6a-833a2e36eef2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:27.418112 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8e55cf39-b940-4445-a8f6-774bcf0bb165 req-2feb727a-0ed8-4f63-8f6a-833a2e36eef2 service nova] Creating event network-changed:10f31ce3-e11c-4b44-bb7f-173c2919d9b7 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:12:27.429232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c39c7eae-d390-420b-8cee-ce1dcf742c91 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:27.432621 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8e55cf39-b940-4445-a8f6-774bcf0bb165 req-2feb727a-0ed8-4f63-8f6a-833a2e36eef2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.130666 Sep 27 10:12:27.439614 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2122/3885] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:27.446656 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c39c7eae-d390-420b-8cee-ce1dcf742c91 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1590 microversion: 2.1 time: 0.473958 Sep 27 10:12:27.449323 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1764/3886] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:26 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1590 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:27.477525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d66ff63-4eeb-4854-968b-f7bc8018fa13 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:27.480787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4d66ff63-4eeb-4854-968b-f7bc8018fa13 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:27.511885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d66ff63-4eeb-4854-968b-f7bc8018fa13 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:27.513905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d66ff63-4eeb-4854-968b-f7bc8018fa13 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:27.514898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d66ff63-4eeb-4854-968b-f7bc8018fa13 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:27.518030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d66ff63-4eeb-4854-968b-f7bc8018fa13 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:27.525411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d66ff63-4eeb-4854-968b-f7bc8018fa13 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:27.525578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d66ff63-4eeb-4854-968b-f7bc8018fa13 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:27.759404 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4d66ff63-4eeb-4854-968b-f7bc8018fa13 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.288496 Sep 27 10:12:27.761829 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2123/3887] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:12:27 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51/os-interface => generated 285 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:27.865254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:27.867751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:27.885135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:27.885877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:27.887442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:27.888493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:27.891178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:27.896988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:28.092995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:28.107507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:28.109844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:28.113292 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-90445e0e-8bd2-4bd0-9099-2f93aabc495d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.251937 Sep 27 10:12:28.133800 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1765/3888] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:27 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:28.340997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:28.342704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:28.354437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:28.356221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:28.356913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:28.358085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:28.358837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:28.360195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:28.383642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3e0c7260-f955-4fb7-b6a4-4726ced19d7d tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:28.384591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e0c7260-f955-4fb7-b6a4-4726ced19d7d tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:28.401024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e0c7260-f955-4fb7-b6a4-4726ced19d7d tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:28.403250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e0c7260-f955-4fb7-b6a4-4726ced19d7d tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:28.405642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e0c7260-f955-4fb7-b6a4-4726ced19d7d tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:28.409812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e0c7260-f955-4fb7-b6a4-4726ced19d7d tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:28.410309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e0c7260-f955-4fb7-b6a4-4726ced19d7d tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:28.413387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e0c7260-f955-4fb7-b6a4-4726ced19d7d tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:28.513273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e0c7260-f955-4fb7-b6a4-4726ced19d7d tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:12:28.707212 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3e0c7260-f955-4fb7-b6a4-4726ced19d7d tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 204 len: 0 microversion: 2.1 time: 0.329224 Sep 27 10:12:28.711614 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1766/3889] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:12:28 2022] DELETE /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:12:28.717230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d059a674-7523-4e3e-bd25-b7d384a1a48f tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:28.717876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d059a674-7523-4e3e-bd25-b7d384a1a48f tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:28.742006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d059a674-7523-4e3e-bd25-b7d384a1a48f tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:28.742506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d059a674-7523-4e3e-bd25-b7d384a1a48f tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:28.742953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d059a674-7523-4e3e-bd25-b7d384a1a48f tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:28.744339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d059a674-7523-4e3e-bd25-b7d384a1a48f tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:28.744965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d059a674-7523-4e3e-bd25-b7d384a1a48f tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:28.747459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d059a674-7523-4e3e-bd25-b7d384a1a48f tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:28.817632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:28.839314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:28.844311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:28.847803 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-105c9154-e714-4aa4-9b24-d4537ffa6bd5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1854 microversion: 2.32 time: 0.512553 Sep 27 10:12:28.864972 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2124/3890] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:28 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1854 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:29.058843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d059a674-7523-4e3e-bd25-b7d384a1a48f tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:29.068839 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d059a674-7523-4e3e-bd25-b7d384a1a48f tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1682 microversion: 2.1 time: 0.355959 Sep 27 10:12:29.073416 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1767/3891] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:28 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1682 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:29.148358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:29.149857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:29.164652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:29.166792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:29.167486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:29.173475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:29.173977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:29.174421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:29.317812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:29.329267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:29.330955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:29.332250 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cb887d74-397c-4e2d-89cf-3742cdb95622 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.191552 Sep 27 10:12:29.342311 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2125/3892] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:29 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:29.474271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ce7c6d81-9453-4992-8373-9d848ee34a8d req-0a837031-6b46-42e2-b7bf-5e2eecbb4def service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "tag": "c8585417-f76b-4df9-869f-bf4ecaf46641"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:29.491804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ce7c6d81-9453-4992-8373-9d848ee34a8d req-0a837031-6b46-42e2-b7bf-5e2eecbb4def service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:29.492924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ce7c6d81-9453-4992-8373-9d848ee34a8d req-0a837031-6b46-42e2-b7bf-5e2eecbb4def service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:29.493881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ce7c6d81-9453-4992-8373-9d848ee34a8d req-0a837031-6b46-42e2-b7bf-5e2eecbb4def service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:29.514647 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ce7c6d81-9453-4992-8373-9d848ee34a8d req-0a837031-6b46-42e2-b7bf-5e2eecbb4def service nova] Creating event network-changed:c8585417-f76b-4df9-869f-bf4ecaf46641 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:12:29.526819 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ce7c6d81-9453-4992-8373-9d848ee34a8d req-0a837031-6b46-42e2-b7bf-5e2eecbb4def service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055076 Sep 27 10:12:29.530735 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1768/3893] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:29.920205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:29.922013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:29.933773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:29.934274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:29.934955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:29.936036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:29.937234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:29.938398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:30.018865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e338f77-d628-4c91-9e19-ca6f0ca4cbbb tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:30.019801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4e338f77-d628-4c91-9e19-ca6f0ca4cbbb tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:30.045504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e338f77-d628-4c91-9e19-ca6f0ca4cbbb tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:30.049538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e338f77-d628-4c91-9e19-ca6f0ca4cbbb tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:30.050071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e338f77-d628-4c91-9e19-ca6f0ca4cbbb tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:30.051206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e338f77-d628-4c91-9e19-ca6f0ca4cbbb tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:30.051895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e338f77-d628-4c91-9e19-ca6f0ca4cbbb tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:30.053262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e338f77-d628-4c91-9e19-ca6f0ca4cbbb tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:30.295917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4e338f77-d628-4c91-9e19-ca6f0ca4cbbb tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:30.308952 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4e338f77-d628-4c91-9e19-ca6f0ca4cbbb tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1590 microversion: 2.1 time: 0.300257 Sep 27 10:12:30.311267 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1769/3894] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:30 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1590 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:30.360064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6af9cbab-2a67-4bbd-9d8d-ecbee19df558 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:30.360725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6af9cbab-2a67-4bbd-9d8d-ecbee19df558 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:30.374632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6af9cbab-2a67-4bbd-9d8d-ecbee19df558 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:30.375137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6af9cbab-2a67-4bbd-9d8d-ecbee19df558 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:30.375583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6af9cbab-2a67-4bbd-9d8d-ecbee19df558 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:30.381051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6af9cbab-2a67-4bbd-9d8d-ecbee19df558 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:30.381803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6af9cbab-2a67-4bbd-9d8d-ecbee19df558 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:30.382173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6af9cbab-2a67-4bbd-9d8d-ecbee19df558 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:30.396129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:30.822196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:30.836293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:30.838004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:30.842256 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dd2c6688-d559-415e-9895-f4f68203d834 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1854 microversion: 2.32 time: 0.942908 Sep 27 10:12:30.842863 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2126/3895] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:29 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1854 bytes in 972 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:31.277864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e1f228dc-1793-4726-9b28-dbaca66fbcb9 req-bba0550d-1454-4342-b214-dc3a114e41ba service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6cece583-0807-44b9-a6c1-ea77401b7f51", "tag": "ff323f5a-e504-4eb7-9433-e9c539434972"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:31.293163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e1f228dc-1793-4726-9b28-dbaca66fbcb9 req-bba0550d-1454-4342-b214-dc3a114e41ba service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:31.298426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e1f228dc-1793-4726-9b28-dbaca66fbcb9 req-bba0550d-1454-4342-b214-dc3a114e41ba service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:31.300709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e1f228dc-1793-4726-9b28-dbaca66fbcb9 req-bba0550d-1454-4342-b214-dc3a114e41ba service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:31.331770 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e1f228dc-1793-4726-9b28-dbaca66fbcb9 req-bba0550d-1454-4342-b214-dc3a114e41ba service nova] Creating event network-changed:ff323f5a-e504-4eb7-9433-e9c539434972 for instance 6cece583-0807-44b9-a6c1-ea77401b7f51 on nch1-master-1 Sep 27 10:12:31.342693 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e1f228dc-1793-4726-9b28-dbaca66fbcb9 req-bba0550d-1454-4342-b214-dc3a114e41ba service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.476500 Sep 27 10:12:31.343724 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2127/3896] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:31.361223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:31.362793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:31.376093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:31.377052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6af9cbab-2a67-4bbd-9d8d-ecbee19df558 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:31.379161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:31.379829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:31.383462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:31.384145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:31.386243 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6af9cbab-2a67-4bbd-9d8d-ecbee19df558 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1682 microversion: 2.1 time: 1.070129 Sep 27 10:12:31.393546 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1770/3897] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:30 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1682 bytes in 1078 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:31.398323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:31.559921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:31.580903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:31.586697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:31.590611 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e68a033e-82e5-43e5-9244-a375d983a2c5 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.244790 Sep 27 10:12:31.611437 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2128/3898] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:31 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:31.873564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:31.874413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:31.882925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:31.883856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:31.884466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:31.885861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:31.886512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:31.887100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:32.411491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-70375bd1-1784-4bac-9739-8d80c95b4952 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:32.412902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-70375bd1-1784-4bac-9739-8d80c95b4952 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:32.422106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70375bd1-1784-4bac-9739-8d80c95b4952 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:32.422929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70375bd1-1784-4bac-9739-8d80c95b4952 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:32.423536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70375bd1-1784-4bac-9739-8d80c95b4952 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:32.424477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70375bd1-1784-4bac-9739-8d80c95b4952 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:32.425189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70375bd1-1784-4bac-9739-8d80c95b4952 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:32.425819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-70375bd1-1784-4bac-9739-8d80c95b4952 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:32.763826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-70375bd1-1784-4bac-9739-8d80c95b4952 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:32.773833 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-70375bd1-1784-4bac-9739-8d80c95b4952 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1682 microversion: 2.1 time: 0.366429 Sep 27 10:12:32.777356 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2129/3899] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:32 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1682 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:12:32.796518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:32.797621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:32.805807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:32.806344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:32.806867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:32.807757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:32.808459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:32.809079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:32.926400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:32.943328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:32.949072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:32.957290 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-68e7ea7e-907d-45a6-bb28-7bb29cf6efcd tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.178477 Sep 27 10:12:32.957926 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2130/3900] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:32 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:33.183995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:33.197330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:33.200202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:33.201849 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9f5c70d7-b518-4917-adef-2bee268a579e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 1.333698 Sep 27 10:12:33.203132 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1771/3901] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:31 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 1335 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:33.626357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8bd79a3e-0280-4577-a419-d5dc2310888b req-c59f6e0c-952e-411d-b846-65bf2bb40b52 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "tag": "db1377f2-7ffe-4edd-befc-8a1ac1aba3ea"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:33.656387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8bd79a3e-0280-4577-a419-d5dc2310888b req-c59f6e0c-952e-411d-b846-65bf2bb40b52 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:33.657348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8bd79a3e-0280-4577-a419-d5dc2310888b req-c59f6e0c-952e-411d-b846-65bf2bb40b52 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:33.658012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8bd79a3e-0280-4577-a419-d5dc2310888b req-c59f6e0c-952e-411d-b846-65bf2bb40b52 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:33.675211 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8bd79a3e-0280-4577-a419-d5dc2310888b req-c59f6e0c-952e-411d-b846-65bf2bb40b52 service nova] Creating event network-changed:db1377f2-7ffe-4edd-befc-8a1ac1aba3ea for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:12:33.683954 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8bd79a3e-0280-4577-a419-d5dc2310888b req-c59f6e0c-952e-411d-b846-65bf2bb40b52 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071415 Sep 27 10:12:33.684649 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2131/3902] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:33.799620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d1e156a6-2c10-4234-935a-7ea399368db4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:33.800224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d1e156a6-2c10-4234-935a-7ea399368db4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:33.807006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1e156a6-2c10-4234-935a-7ea399368db4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:33.807762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1e156a6-2c10-4234-935a-7ea399368db4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:33.808403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1e156a6-2c10-4234-935a-7ea399368db4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:33.809150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1e156a6-2c10-4234-935a-7ea399368db4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:33.809784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1e156a6-2c10-4234-935a-7ea399368db4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:33.810215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1e156a6-2c10-4234-935a-7ea399368db4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:33.986776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:33.987476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:33.995437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:33.995970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:33.996411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:33.997289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:33.997955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:33.998569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:34.114450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:34.128865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:34.132990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:34.136427 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c1315586-a6dc-4adb-8cd6-b1691d8e1a5d tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.154542 Sep 27 10:12:34.137976 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2132/3903] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:33 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:12:34.233314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:34.236307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:34.245384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:34.246220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:34.246799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:34.247840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:34.248396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:34.249044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:34.491043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d1e156a6-2c10-4234-935a-7ea399368db4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:34.508029 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d1e156a6-2c10-4234-935a-7ea399368db4 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1682 microversion: 2.1 time: 0.711684 Sep 27 10:12:34.513556 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1772/3904] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:33 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1682 bytes in 718 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:35.162595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:35.163312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:35.172679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:35.174484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:35.175126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:35.176084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:35.176651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:35.179566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:35.259788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:35.282764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:35.285541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:35.289357 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-daa884bd-730a-4d97-80a9-06b53ba4f3a0 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 1.065424 Sep 27 10:12:35.291058 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2133/3905] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:34 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 1068 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:12:35.298388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:35.310301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:35.312951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:35.315977 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-057486ae-28fe-4581-a92a-5cf618eef916 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.157245 Sep 27 10:12:35.327279 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1773/3906] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:35 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:35.543983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-673cf948-9b7b-421d-9e84-6fc518556e8c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:35.544996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-673cf948-9b7b-421d-9e84-6fc518556e8c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:35.560089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673cf948-9b7b-421d-9e84-6fc518556e8c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:35.564583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673cf948-9b7b-421d-9e84-6fc518556e8c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:35.565481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673cf948-9b7b-421d-9e84-6fc518556e8c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:35.571553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673cf948-9b7b-421d-9e84-6fc518556e8c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:35.573348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673cf948-9b7b-421d-9e84-6fc518556e8c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:35.574316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-673cf948-9b7b-421d-9e84-6fc518556e8c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:35.654612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:35.708079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2854424d-f370-4400-8766-cf9ddcf28b5d req-28511b9e-712a-4a01-9d80-dbfd31cf41a8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a04c146e-494e-458e-843e-09c63c576cb3", "name": "network-vif-deleted", "tag": "f45a2d54-220d-47e8-93bd-d90e6fda8e24"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:35.725629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2854424d-f370-4400-8766-cf9ddcf28b5d req-28511b9e-712a-4a01-9d80-dbfd31cf41a8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:35.725852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2854424d-f370-4400-8766-cf9ddcf28b5d req-28511b9e-712a-4a01-9d80-dbfd31cf41a8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:35.726362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2854424d-f370-4400-8766-cf9ddcf28b5d req-28511b9e-712a-4a01-9d80-dbfd31cf41a8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:35.759994 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2854424d-f370-4400-8766-cf9ddcf28b5d req-28511b9e-712a-4a01-9d80-dbfd31cf41a8 service nova] Creating event network-vif-deleted:f45a2d54-220d-47e8-93bd-d90e6fda8e24 for instance a04c146e-494e-458e-843e-09c63c576cb3 on nch2-master-1 Sep 27 10:12:35.773763 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2854424d-f370-4400-8766-cf9ddcf28b5d req-28511b9e-712a-4a01-9d80-dbfd31cf41a8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070769 Sep 27 10:12:35.785043 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1774/3907] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:12:35.929215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-673cf948-9b7b-421d-9e84-6fc518556e8c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b8d6f6c63fe144959139278ca287e2ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f847bee14ff74dfbafe7c6a199f1b99f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:35.941654 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-673cf948-9b7b-421d-9e84-6fc518556e8c tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 200 len: 1327 microversion: 2.1 time: 0.403003 Sep 27 10:12:35.947009 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2134/3908] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:35 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 1327 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:36.317309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:36.318709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:36.336918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:36.337861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:36.338577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:36.339556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:36.342131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:36.342781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:36.351083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:36.353857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:36.371595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:36.372920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:36.376931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:36.379909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:36.384613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:36.385421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:36.579829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:36.594589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:36.603227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:36.604885 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7e2848ae-b2e6-4b04-bd5a-1c2d5cf8b105 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.260185 Sep 27 10:12:36.608377 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2135/3909] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:36 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:36.977638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:36.978310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] [instance: a04c146e-494e-458e-843e-09c63c576cb3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:36.990717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:36.993516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:36.994301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:36.998103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:37.000159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:37.001241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:37.035078 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] HTTP exception thrown: Instance a04c146e-494e-458e-843e-09c63c576cb3 could not be found. Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Request method failure captured: Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 HTTP/1.1 Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsvSKjxFpoBIP2u6EFp3My1Yqd9bBj3m8CgbWWapDBUvT3WdvklXX3eu9ZGf4jBd8D17DcVKcKM43nenVqW9JlM_cE_xo90emZytpqfuRzVgzxepCgMSBWlLsDXm8aqn6P7_ohi25VJhTY4FOf80OJBz243mXcrX8PJe8YbrGMeVpkM Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f847bee14ff74dfbafe7c6a199f1b99f Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesAdminNegativeTest-1801268990 Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f847bee14ff74dfbafe7c6a199f1b99f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f847bee14ff74dfbafe7c6a199f1b99f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f847bee14ff74dfbafe7c6a199f1b99f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f847bee14ff74dfbafe7c6a199f1b99f"}]}] Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesAdminNegativeTest-1801268990 Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f847bee14ff74dfbafe7c6a199f1b99f Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesAdminNegativeTest-1801268990 Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesAdminNegativeTest-1801268990-project Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b8d6f6c63fe144959139278ca287e2ec Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesAdminNegativeTest-1801268990-project Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance a04c146e-494e-458e-843e-09c63c576cb3 could not be found. Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a04c146e-494e-458e-843e-09c63c576cb3'} Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a04c146e-494e-458e-843e-09c63c576cb3 could not be found. Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a04c146e-494e-458e-843e-09c63c576cb3 could not be found. Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:37.038559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:37.046075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:37.046075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:37.046075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:37.046075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:12:37.046075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:12:37.046075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a04c146e-494e-458e-843e-09c63c576cb3 could not be found. Sep 27 10:12:37.046075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:12:37.048229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Returning 404 to user: Instance a04c146e-494e-458e-843e-09c63c576cb3 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:12:37.050412 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-02d07258-7515-46c4-8a50-24432f4ad123 tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3" status: 404 len: 111 microversion: 2.1 time: 0.079166 Sep 27 10:12:37.061244 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2136/3910] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:36 2022] GET /compute/v2.1/servers/a04c146e-494e-458e-843e-09c63c576cb3 => generated 111 bytes in 88 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:12:37.073332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-116805f0-e37e-4086-81bb-3c184d315fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] Calling method '.version_select at 0x7f7f626c8f70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:37.128473 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-116805f0-e37e-4086-81bb-3c184d315fab tempest-VolumesAdminNegativeTest-1801268990 tempest-VolumesAdminNegativeTest-1801268990-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2043080211" status: 202 len: 0 microversion: 2.1 time: 0.064432 Sep 27 10:12:37.130207 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2137/3911] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:12:37 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2043080211 => generated 0 bytes in 68 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:12:37.638913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:37.639718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:37.666098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:37.668860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:37.671557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:37.673259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:37.675840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:37.682321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:37.683065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:37.685682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:37.687136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:37.688490 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0b22311f-9448-4a2a-8df2-2650f093fa83 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 1.375669 Sep 27 10:12:37.696803 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1775/3912] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:36 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 1385 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:37.801939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:37.812346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:37.814138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:37.815799 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-22c9ab1d-6954-4362-86a6-fc3d679a9f0e tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.183535 Sep 27 10:12:37.833778 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2138/3913] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:37 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:37.849611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e82ff7e0-b027-4e9e-867b-980624bcb92b req-0175c9cd-82bd-4ff2-b1c0-a3ce700f9c9e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "tag": "4a6d853f-cf26-46e4-ad6f-27d3139a8f84"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:37.866403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e82ff7e0-b027-4e9e-867b-980624bcb92b req-0175c9cd-82bd-4ff2-b1c0-a3ce700f9c9e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:37.867120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e82ff7e0-b027-4e9e-867b-980624bcb92b req-0175c9cd-82bd-4ff2-b1c0-a3ce700f9c9e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:37.867726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e82ff7e0-b027-4e9e-867b-980624bcb92b req-0175c9cd-82bd-4ff2-b1c0-a3ce700f9c9e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:37.894590 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e82ff7e0-b027-4e9e-867b-980624bcb92b req-0175c9cd-82bd-4ff2-b1c0-a3ce700f9c9e service nova] Creating event network-changed:4a6d853f-cf26-46e4-ad6f-27d3139a8f84 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:12:37.902460 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e82ff7e0-b027-4e9e-867b-980624bcb92b req-0175c9cd-82bd-4ff2-b1c0-a3ce700f9c9e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056504 Sep 27 10:12:37.903633 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1776/3914] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:38.727967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:38.728601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:38.741292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:38.746550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:38.747041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:38.747861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:38.748461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:38.748934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:38.851384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:38.852154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:38.864045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:38.864562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:38.865066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:38.865900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:38.866471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:38.872900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:39.079703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:39.097898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:39.104106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:39.105820 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-43f8b5bd-a0c8-4985-9e0d-b2825b37b980 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.263945 Sep 27 10:12:39.125454 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1777/3915] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:38 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:12:39.944943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:39.961287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:39.963532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:39.965235 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d943d3d0-50f7-46a1-b002-c10af1c42bf4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 1.242029 Sep 27 10:12:39.965817 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2139/3916] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:38 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 1243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:39.989291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-c8ac27a7-d444-44e3-8d18-1b3f3f1c2faf req-bacf7fe8-ed7e-4395-9a43-2ab7ff82216d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "tag": "0edf08ff-1fc9-48d8-be71-8d8e715e7c93"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:40.020811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c8ac27a7-d444-44e3-8d18-1b3f3f1c2faf req-bacf7fe8-ed7e-4395-9a43-2ab7ff82216d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:40.027379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c8ac27a7-d444-44e3-8d18-1b3f3f1c2faf req-bacf7fe8-ed7e-4395-9a43-2ab7ff82216d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:40.028226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c8ac27a7-d444-44e3-8d18-1b3f3f1c2faf req-bacf7fe8-ed7e-4395-9a43-2ab7ff82216d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:40.052797 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c8ac27a7-d444-44e3-8d18-1b3f3f1c2faf req-bacf7fe8-ed7e-4395-9a43-2ab7ff82216d service nova] Creating event network-changed:0edf08ff-1fc9-48d8-be71-8d8e715e7c93 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:12:40.062798 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-c8ac27a7-d444-44e3-8d18-1b3f3f1c2faf req-bacf7fe8-ed7e-4395-9a43-2ab7ff82216d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.078256 Sep 27 10:12:40.064768 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1778/3917] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:40.145229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:40.146185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:40.157285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:40.158356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:40.160305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:40.162137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:40.162896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:40.167558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:40.490570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:40.520800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:40.525845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:40.530894 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7a9dc081-62ff-4bca-8a85-5a922bebc6f7 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.395025 Sep 27 10:12:40.545192 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2140/3918] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:40 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:40.995820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:40.998224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:41.050307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:41.056640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:41.060460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:41.066260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:41.071003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:41.074689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:41.571019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:41.574171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:41.589544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:41.590610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:41.591154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:41.592189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:41.593185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:41.595318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:41.719982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:41.732345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:41.741270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:41.742524 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-34c1238a-a790-4fce-a003-0244e6d51424 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.184652 Sep 27 10:12:41.743998 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2141/3919] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:41 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:41.784957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:41.802302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:41.804159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:41.805650 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-974ca482-0b34-4193-b0d4-b0b7bb2ef662 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.814922 Sep 27 10:12:41.821371 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1779/3920] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:40 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:42.778482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:42.781712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:42.790924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:42.791729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:42.792342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:42.793355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:42.794009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:42.794610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:42.837979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:42.837979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:42.851253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:42.852855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:42.853517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:42.854457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:42.855072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:42.855610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:42.895652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:42.907538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:42.909401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:42.910931 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-055545a9-cd9d-4139-8a20-6ce14a3edaa0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.139550 Sep 27 10:12:42.911739 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2142/3921] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:42 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:43.642436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:43.665518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:43.668703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:43.672536 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0c31243e-8d9c-4846-b8be-d53298f3db1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.843489 Sep 27 10:12:43.684188 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1780/3922] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:42 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 853 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:12:43.773166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-568fc192-aae9-46b4-a1d2-5b4792a59304 req-f0415859-8c87-4551-9c90-0ecfae26e108 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "tag": "86cc38ea-2a41-41be-a24f-432ba40420c5"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:43.789597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-568fc192-aae9-46b4-a1d2-5b4792a59304 req-f0415859-8c87-4551-9c90-0ecfae26e108 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:43.792584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-568fc192-aae9-46b4-a1d2-5b4792a59304 req-f0415859-8c87-4551-9c90-0ecfae26e108 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:43.793384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-568fc192-aae9-46b4-a1d2-5b4792a59304 req-f0415859-8c87-4551-9c90-0ecfae26e108 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:43.818453 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-568fc192-aae9-46b4-a1d2-5b4792a59304 req-f0415859-8c87-4551-9c90-0ecfae26e108 service nova] Creating event network-changed:86cc38ea-2a41-41be-a24f-432ba40420c5 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:12:43.829246 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-568fc192-aae9-46b4-a1d2-5b4792a59304 req-f0415859-8c87-4551-9c90-0ecfae26e108 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059707 Sep 27 10:12:43.830878 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2143/3923] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:12:43.945971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:43.946595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:43.959743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:43.960715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:43.961347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:43.963505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:43.964159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:43.966514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:44.129202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:44.144818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:44.152803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:44.154222 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8d8618e7-f3cd-4bc0-afd2-4a1a03ba7b4b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.213181 Sep 27 10:12:44.154830 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1781/3924] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:43 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:44.472272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueTestJSONUnderV235-server-1933867046", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:44.569075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] validate_networks() for [('3dbae7e4-7497-4220-9492-7ccbe8a206d1', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:12:44.703011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:44.703630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:44.718473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:44.718973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:44.719412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:44.720152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:44.720616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:44.721064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:45.465407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:12:45.466476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:12:45.467032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:12:45.467820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:12:45.468437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:12:45.480950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:45.481869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:45.483405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:45.484474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:45.485180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:45.485880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:45.515034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Getting quotas for project f23e44d8a5094104b35d0288d33f0ab6. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:12:45.521877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Getting quotas for user 241251b91c43444b900922f942681cc5 and project f23e44d8a5094104b35d0288d33f0ab6. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:12:45.538292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:12:45.541218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:12:45.612112 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e7c22b33-ad08-46df-b4ea-5dc09383d3c5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.386098 Sep 27 10:12:45.622532 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2144/3925] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:12:44 2022] POST /compute/v2.1/servers => generated 383 bytes in 1397 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:12:45.630092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:45.631027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:45.641966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:45.645496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:45.647933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:45.648912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:45.649401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:45.651438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:45.815094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:45.841222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:45.852479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:45.860767 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e8f1aefc-48c7-4c73-bbaa-cca3c8a081fc tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 1.160102 Sep 27 10:12:45.861437 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1782/3926] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:44 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 1163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:12:45.870870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ff49ffd-bde6-4f41-8964-d28558d48537 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:45.871699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ff49ffd-bde6-4f41-8964-d28558d48537 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:46.087964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:46.106928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:46.116543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:46.119398 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a97fbbe-28b9-404d-b6c3-d1900972334a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.495182 Sep 27 10:12:46.122681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2145/3927] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:45 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:46.199442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8ff49ffd-bde6-4f41-8964-d28558d48537 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:46.218065 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8ff49ffd-bde6-4f41-8964-d28558d48537 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1223 microversion: 2.1 time: 0.352325 Sep 27 10:12:46.230103 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1783/3928] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:45 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1223 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:46.894513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:46.895203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:46.911951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:46.927621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:46.928601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:46.929868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:46.930736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:46.932763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:47.153719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:47.154402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:47.166859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:47.167364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:47.167930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:47.171225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:47.173237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:47.185496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:47.399337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:47.414320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:47.417219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:47.418776 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c62da3e3-5be1-4aea-b7f4-131785299f63 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1366 microversion: 2.47 time: 0.270639 Sep 27 10:12:47.419725 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1784/3929] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:47 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1366 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:12:47.424862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-33bbace0-250b-4f19-a275-cbf97a3d868f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:47.425541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-33bbace0-250b-4f19-a275-cbf97a3d868f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:47.431664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bbace0-250b-4f19-a275-cbf97a3d868f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:47.432220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bbace0-250b-4f19-a275-cbf97a3d868f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:47.432769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bbace0-250b-4f19-a275-cbf97a3d868f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:47.433566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bbace0-250b-4f19-a275-cbf97a3d868f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:47.434130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bbace0-250b-4f19-a275-cbf97a3d868f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:47.434658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33bbace0-250b-4f19-a275-cbf97a3d868f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:47.487279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:47.501590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:47.503272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:47.504973 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f0d6ace1-f52d-4335-9786-3e4e3b692d40 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.615887 Sep 27 10:12:47.507030 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2146/3930] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:46 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:47.680849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-33bbace0-250b-4f19-a275-cbf97a3d868f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:47.700627 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-33bbace0-250b-4f19-a275-cbf97a3d868f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1275 microversion: 2.1 time: 0.279405 Sep 27 10:12:47.721064 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1785/3931] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:47 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1275 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:48.450918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:48.452365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:48.469072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=92,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:48.472710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:48.475428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:48.477279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:48.480262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:48.482759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:48.483901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:48.541551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:48.542259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:48.560367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:48.562886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:48.563597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:48.564823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:48.565561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:48.566406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:48.674992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:48.688697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:48.690252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:48.691592 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5f4de6a1-30ca-4a3b-8248-66e2d78bb205 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1372 microversion: 2.47 time: 0.245863 Sep 27 10:12:48.697993 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2147/3932] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:48 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1372 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:48.706317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-41f97afc-36e2-453d-bb02-a05466c0ac0f req-b8252808-6825-4f2d-8731-d1b0fc0c100d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "tag": "78c9b09b-0886-456d-897a-ad2c11dd56c3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:48.729822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-41f97afc-36e2-453d-bb02-a05466c0ac0f req-b8252808-6825-4f2d-8731-d1b0fc0c100d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:48.730327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-41f97afc-36e2-453d-bb02-a05466c0ac0f req-b8252808-6825-4f2d-8731-d1b0fc0c100d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:48.730827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-41f97afc-36e2-453d-bb02-a05466c0ac0f req-b8252808-6825-4f2d-8731-d1b0fc0c100d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:48.758776 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-41f97afc-36e2-453d-bb02-a05466c0ac0f req-b8252808-6825-4f2d-8731-d1b0fc0c100d service nova] Creating event network-changed:78c9b09b-0886-456d-897a-ad2c11dd56c3 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:12:48.765346 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-41f97afc-36e2-453d-bb02-a05466c0ac0f req-b8252808-6825-4f2d-8731-d1b0fc0c100d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066019 Sep 27 10:12:48.766108 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2148/3933] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:12:48.771988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-df839fa3-e18b-425d-9a7e-18a7055fd375 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:48.773537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-df839fa3-e18b-425d-9a7e-18a7055fd375 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:48.781709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df839fa3-e18b-425d-9a7e-18a7055fd375 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:48.782600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df839fa3-e18b-425d-9a7e-18a7055fd375 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:48.783301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df839fa3-e18b-425d-9a7e-18a7055fd375 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:48.784405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df839fa3-e18b-425d-9a7e-18a7055fd375 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:48.785076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df839fa3-e18b-425d-9a7e-18a7055fd375 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:48.786061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df839fa3-e18b-425d-9a7e-18a7055fd375 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:48.880008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-df839fa3-e18b-425d-9a7e-18a7055fd375 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:48.893845 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-df839fa3-e18b-425d-9a7e-18a7055fd375 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1281 microversion: 2.1 time: 0.126325 Sep 27 10:12:48.895352 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2149/3934] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:48 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1281 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:49.469443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:49.515760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:49.527262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:49.541393 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-da420450-064b-468f-8885-819e35c97bd4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.997634 Sep 27 10:12:49.542808 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1786/3935] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:48 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 1007 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:12:49.728761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:49.731187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:49.742437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:49.744784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:49.745476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:49.746502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:49.747176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:49.747932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:49.915636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:49.919582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0cdba300-2d76-4af0-abd5-9a081255b86a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:49.920214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0cdba300-2d76-4af0-abd5-9a081255b86a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:49.933821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:49.938409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:49.941979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cdba300-2d76-4af0-abd5-9a081255b86a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:49.942825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cdba300-2d76-4af0-abd5-9a081255b86a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:49.943644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cdba300-2d76-4af0-abd5-9a081255b86a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:49.944695 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-42441033-a018-459a-a2b1-a3e9a7cf7677 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1372 microversion: 2.47 time: 0.219184 Sep 27 10:12:49.944929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cdba300-2d76-4af0-abd5-9a081255b86a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:49.946173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cdba300-2d76-4af0-abd5-9a081255b86a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:49.946779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cdba300-2d76-4af0-abd5-9a081255b86a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:49.957169 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2150/3936] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:49 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1372 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:50.437801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0cdba300-2d76-4af0-abd5-9a081255b86a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:50.448771 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0cdba300-2d76-4af0-abd5-9a081255b86a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1281 microversion: 2.1 time: 0.533036 Sep 27 10:12:50.450240 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1787/3937] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:49 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1281 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:12:50.572238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:50.573017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:50.585834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:50.586423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:50.587052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:50.594371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:50.595278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:50.596050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:50.972326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:50.973041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:50.983406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:50.983932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:50.989425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:50.990393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:50.994072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:50.995143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:51.517468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:51.530783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:51.551373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:51.551809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:51.555582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:51.558761 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1845b9f1-e325-476e-ace5-eeff4f5b19f5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.991050 Sep 27 10:12:51.559835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:51.569237 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2151/3938] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:50 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 994 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:51.569458 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0cf99728-b728-4a27-89f2-aab64063b818 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1357 microversion: 2.47 time: 0.593257 Sep 27 10:12:51.573656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e956f205-dab5-4216-a1d1-28d37954c1f5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:51.574291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e956f205-dab5-4216-a1d1-28d37954c1f5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:51.578159 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1788/3939] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:50 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1357 bytes in 610 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:12:51.588541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e956f205-dab5-4216-a1d1-28d37954c1f5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:51.593937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e956f205-dab5-4216-a1d1-28d37954c1f5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:51.594495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e956f205-dab5-4216-a1d1-28d37954c1f5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:51.595427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e956f205-dab5-4216-a1d1-28d37954c1f5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:51.595967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e956f205-dab5-4216-a1d1-28d37954c1f5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:51.596517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e956f205-dab5-4216-a1d1-28d37954c1f5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:51.625507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:51.631600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:51.633810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Searching by: {'deleted': False, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:12:51.640861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:51.641806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:51.642446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:51.644827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:51.645488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:51.646104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:51.847790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Listed batch of 2 results from cell out of 1000 limit. Returned 2 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:12:51.935804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:51.948701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:51.957434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:51.958269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:51.959023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:51.973813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:51.975786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:52.192688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e956f205-dab5-4216-a1d1-28d37954c1f5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:52.226116 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e956f205-dab5-4216-a1d1-28d37954c1f5 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1323 microversion: 2.1 time: 0.653175 Sep 27 10:12:52.242712 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2152/3940] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:51 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1323 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:52.282922 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-64861283-ca4a-4f7c-bceb-d20f6085cf18 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/detail" status: 200 len: 2706 microversion: 2.47 time: 0.669420 Sep 27 10:12:52.299805 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1789/3941] 10.222.0.26 () {60 vars in 1187 bytes} [Tue Sep 27 10:12:51 2022] GET /compute/v2.1/servers/detail => generated 2706 bytes in 691 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:52.329458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-823dfa78-cc41-4a73-8de4-01c39dc8c659 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:52.330085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-823dfa78-cc41-4a73-8de4-01c39dc8c659 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:52.339597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-823dfa78-cc41-4a73-8de4-01c39dc8c659 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:52.340149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-823dfa78-cc41-4a73-8de4-01c39dc8c659 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:52.341741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-823dfa78-cc41-4a73-8de4-01c39dc8c659 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:52.342639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-823dfa78-cc41-4a73-8de4-01c39dc8c659 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:52.343103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-823dfa78-cc41-4a73-8de4-01c39dc8c659 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:52.343563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-823dfa78-cc41-4a73-8de4-01c39dc8c659 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:52.378440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-823dfa78-cc41-4a73-8de4-01c39dc8c659 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:12:52.481727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:52.566850 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-823dfa78-cc41-4a73-8de4-01c39dc8c659 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "DELETE /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 204 len: 0 microversion: 2.47 time: 0.244705 Sep 27 10:12:52.568212 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2153/3942] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:12:52 2022] DELETE /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 0 bytes in 249 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:12:52.581772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:52.583047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:52.600612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:52.603845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:52.604623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:52.605309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:52.607399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:52.608072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:52.608733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:52.610265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:52.624715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:52.627858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:52.628701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:52.629684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:52.631212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:52.633098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:52.810519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:52.822350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:52.824314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:52.825835 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1b70cb6d-be6a-466a-ba42-f031d758d343 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1363 microversion: 2.47 time: 0.252164 Sep 27 10:12:52.827027 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1790/3943] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:52 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1363 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:53.211738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:53.238046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:53.243706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:53.246200 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7f1139b8-84ce-4db4-95b3-f8ef7f37364e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.655138 Sep 27 10:12:53.251787 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2154/3944] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:52 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 662 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:53.256257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-30e862b6-5184-4efb-a20b-225d65104a29 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:53.257385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-30e862b6-5184-4efb-a20b-225d65104a29 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:53.259988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=83,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:53.274730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30e862b6-5184-4efb-a20b-225d65104a29 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:53.275260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30e862b6-5184-4efb-a20b-225d65104a29 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:53.275765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30e862b6-5184-4efb-a20b-225d65104a29 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:53.276582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30e862b6-5184-4efb-a20b-225d65104a29 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:53.280124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30e862b6-5184-4efb-a20b-225d65104a29 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:53.280594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30e862b6-5184-4efb-a20b-225d65104a29 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:53.854572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:53.855869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:53.863203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:53.863997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:53.864633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:53.865558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:53.866328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:53.867328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:53.947931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-30e862b6-5184-4efb-a20b-225d65104a29 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:53.957031 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-30e862b6-5184-4efb-a20b-225d65104a29 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1323 microversion: 2.1 time: 0.705167 Sep 27 10:12:53.969138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:53.970692 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1791/3945] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:53 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1323 bytes in 719 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:53.975919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:53.980708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:53.985364 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-08fe15b9-87bb-435c-91cd-25410d733ee0 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1363 microversion: 2.47 time: 0.133032 Sep 27 10:12:53.988865 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2155/3946] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:53 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1363 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:54.285645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:54.288007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:54.309676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:54.310223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:54.311030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:54.312293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:54.313099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:54.313539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:54.490870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-b101d7c9-e567-4ce9-b708-9753fe2e6655 req-4b30ba2f-562d-4c03-9b9e-56a4171f73b9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ce015738-c16b-4f29-8cd0-b96245268712", "tag": "1871cdde-9d60-4d78-b0d8-e7defafdce3e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:12:54.508929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b101d7c9-e567-4ce9-b708-9753fe2e6655 req-4b30ba2f-562d-4c03-9b9e-56a4171f73b9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:54.512850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b101d7c9-e567-4ce9-b708-9753fe2e6655 req-4b30ba2f-562d-4c03-9b9e-56a4171f73b9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:54.515328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b101d7c9-e567-4ce9-b708-9753fe2e6655 req-4b30ba2f-562d-4c03-9b9e-56a4171f73b9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:54.543625 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b101d7c9-e567-4ce9-b708-9753fe2e6655 req-4b30ba2f-562d-4c03-9b9e-56a4171f73b9 service nova] Creating event network-changed:1871cdde-9d60-4d78-b0d8-e7defafdce3e for instance ce015738-c16b-4f29-8cd0-b96245268712 on nch2-master-1 Sep 27 10:12:54.552777 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-b101d7c9-e567-4ce9-b708-9753fe2e6655 req-4b30ba2f-562d-4c03-9b9e-56a4171f73b9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065484 Sep 27 10:12:54.560595 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2156/3947] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:12:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:12:54.986704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2cd7ffd2-c6aa-4793-818f-87f1349a4ce7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:54.988747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2cd7ffd2-c6aa-4793-818f-87f1349a4ce7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:55.011023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cd7ffd2-c6aa-4793-818f-87f1349a4ce7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:55.011714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cd7ffd2-c6aa-4793-818f-87f1349a4ce7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:55.012320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cd7ffd2-c6aa-4793-818f-87f1349a4ce7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:55.013508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cd7ffd2-c6aa-4793-818f-87f1349a4ce7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:55.014497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cd7ffd2-c6aa-4793-818f-87f1349a4ce7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:55.015038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cd7ffd2-c6aa-4793-818f-87f1349a4ce7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:55.504913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:55.531733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:12:55.625944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:55.639186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:55.642014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:55.643746 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-414b3e3c-858f-4090-90de-2db2029e1628 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 1.370878 Sep 27 10:12:55.645511 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1792/3948] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:54 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 1373 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:55.654453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:55.655980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:55.665131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:55.666442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:55.667209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:55.668175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:55.670982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:55.671630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:55.688287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2cd7ffd2-c6aa-4793-818f-87f1349a4ce7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:55.702787 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2cd7ffd2-c6aa-4793-818f-87f1349a4ce7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1323 microversion: 2.1 time: 0.721698 Sep 27 10:12:55.705912 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2157/3949] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:54 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1323 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:12:55.728227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:55.729086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:55.729697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:55.730433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lazy-loading 'fault' on Instance uuid 18b10791-31a2-4cfd-9236-9953856f52ea {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:12:55.839170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:55.857811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:55.859489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:55.860999 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a4ef8cb6-9c45-4b33-bb6a-83cd119b3f16 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 200 len: 1368 microversion: 2.47 time: 0.212931 Sep 27 10:12:55.861972 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1793/3950] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:55 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 1368 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:56.672182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:56.672978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:56.687684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:56.688259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:56.690904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:56.695285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:56.696356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:56.697017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:56.734747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0392f671-4db0-4bd6-b2c4-a8aeaec9a83e tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:56.736130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0392f671-4db0-4bd6-b2c4-a8aeaec9a83e tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:56.750200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0392f671-4db0-4bd6-b2c4-a8aeaec9a83e tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:56.751146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0392f671-4db0-4bd6-b2c4-a8aeaec9a83e tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:56.751937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0392f671-4db0-4bd6-b2c4-a8aeaec9a83e tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:56.755751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0392f671-4db0-4bd6-b2c4-a8aeaec9a83e tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:56.756704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0392f671-4db0-4bd6-b2c4-a8aeaec9a83e tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:56.757610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0392f671-4db0-4bd6-b2c4-a8aeaec9a83e tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:57.036611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0392f671-4db0-4bd6-b2c4-a8aeaec9a83e tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:57.049272 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0392f671-4db0-4bd6-b2c4-a8aeaec9a83e tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1323 microversion: 2.1 time: 0.318232 Sep 27 10:12:57.054063 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1794/3951] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:56 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1323 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:57.085333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:57.088280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: 18b10791-31a2-4cfd-9236-9953856f52ea] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:57.106436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:57.108417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:57.109743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:57.111856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:57.113902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:57.116080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:57.174237 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] HTTP exception thrown: Instance 18b10791-31a2-4cfd-9236-9953856f52ea could not be found. Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Request method failure captured: Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea HTTP/1.1 Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsxyk0Q3RjEpBi8ornxPhJiUJ7CJ-ozZGkh2W9gNB9_tixstUZXNx3GhilODAiBJpXNmFiAvFNweOaPmmDtx1qpejl7KKsPLdRrruy9CHCguWk878jOYEI3sLp0_CGho24ujIDVqsC9ppJg88QBbNptj_HiGtLvBUAROC2rnUs9QrHQ Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.47 Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: b5087b7fd4b3482082e5dd90fdcef18f Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerShowV247Test-1411625565 Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b5087b7fd4b3482082e5dd90fdcef18f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b5087b7fd4b3482082e5dd90fdcef18f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b5087b7fd4b3482082e5dd90fdcef18f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b5087b7fd4b3482082e5dd90fdcef18f"}]}] Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerShowV247Test-1411625565 Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: b5087b7fd4b3482082e5dd90fdcef18f Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerShowV247Test-1411625565 Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerShowV247Test-1411625565-project Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 976e7f61517e4e3f9f2417078225ca62 Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerShowV247Test-1411625565-project Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 18b10791-31a2-4cfd-9236-9953856f52ea could not be found. Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '18b10791-31a2-4cfd-9236-9953856f52ea'} Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 18b10791-31a2-4cfd-9236-9953856f52ea could not be found. Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 18b10791-31a2-4cfd-9236-9953856f52ea could not be found. Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:12:57.176054 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:12:57.183566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:12:57.183566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:12:57.183566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:12:57.183566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:12:57.183566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:12:57.183566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 18b10791-31a2-4cfd-9236-9953856f52ea could not be found. Sep 27 10:12:57.183566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:12:57.183566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Returning 404 to user: Instance 18b10791-31a2-4cfd-9236-9953856f52ea could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:12:57.183566 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-80c73291-1088-474e-af3a-211b79923e9a tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea" status: 404 len: 111 microversion: 2.47 time: 0.121353 Sep 27 10:12:57.183566 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1795/3952] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:57 2022] GET /compute/v2.1/servers/18b10791-31a2-4cfd-9236-9953856f52ea => generated 111 bytes in 123 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:12:57.195919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-66684d94-c62b-4840-a7b1-c5fdd519a266 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:57.196416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-66684d94-c62b-4840-a7b1-c5fdd519a266 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: cde27309-a349-4e7e-a4a8-1032336b61da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:57.205308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:57.217492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66684d94-c62b-4840-a7b1-c5fdd519a266 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:57.218166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66684d94-c62b-4840-a7b1-c5fdd519a266 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:57.218599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66684d94-c62b-4840-a7b1-c5fdd519a266 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:57.219380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66684d94-c62b-4840-a7b1-c5fdd519a266 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:57.219855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66684d94-c62b-4840-a7b1-c5fdd519a266 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:57.220294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66684d94-c62b-4840-a7b1-c5fdd519a266 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:57.229444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:57.231784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:57.233395 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fd0f7ca5-bb17-4612-8eee-fe4562c4f00b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.566034 Sep 27 10:12:57.234688 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2158/3953] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:56 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:57.279552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-66684d94-c62b-4840-a7b1-c5fdd519a266 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: cde27309-a349-4e7e-a4a8-1032336b61da] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:12:57.461536 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-66684d94-c62b-4840-a7b1-c5fdd519a266 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "DELETE /compute/v2.1/servers/cde27309-a349-4e7e-a4a8-1032336b61da" status: 204 len: 0 microversion: 2.47 time: 0.276943 Sep 27 10:12:57.462237 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1796/3954] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:12:57 2022] DELETE /compute/v2.1/servers/cde27309-a349-4e7e-a4a8-1032336b61da => generated 0 bytes in 279 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:12:57.471208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:57.472140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: cde27309-a349-4e7e-a4a8-1032336b61da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:57.484102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:57.486516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:57.487636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:57.490114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:57.493366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:57.494100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:57.723350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:57.738034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:57.745152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:57.747134 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c686a338-cf84-4bdb-a103-5ade0ef8931b tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/cde27309-a349-4e7e-a4a8-1032336b61da" status: 200 len: 1362 microversion: 2.47 time: 0.280572 Sep 27 10:12:57.747955 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2159/3955] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:57 2022] GET /compute/v2.1/servers/cde27309-a349-4e7e-a4a8-1032336b61da => generated 1362 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:58.081318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c665eabb-32f0-4b65-9b43-23db1908f9a4 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:58.082660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c665eabb-32f0-4b65-9b43-23db1908f9a4 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:58.091612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c665eabb-32f0-4b65-9b43-23db1908f9a4 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:58.092372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c665eabb-32f0-4b65-9b43-23db1908f9a4 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:58.093028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c665eabb-32f0-4b65-9b43-23db1908f9a4 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:58.094099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c665eabb-32f0-4b65-9b43-23db1908f9a4 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:58.094791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c665eabb-32f0-4b65-9b43-23db1908f9a4 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:58.095390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c665eabb-32f0-4b65-9b43-23db1908f9a4 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:58.267953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:58.268575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:58.279745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:58.280198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:58.280643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:58.281406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:58.281855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:58.282286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:58.518282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c665eabb-32f0-4b65-9b43-23db1908f9a4 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:58.536815 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c665eabb-32f0-4b65-9b43-23db1908f9a4 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1323 microversion: 2.1 time: 0.460251 Sep 27 10:12:58.548446 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1797/3956] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:58 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1323 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:58.779072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:58.780481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: cde27309-a349-4e7e-a4a8-1032336b61da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:58.796336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:58.796901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:58.797425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:58.798236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:58.799115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:58.799924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:58.909443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:58.927552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:58.929573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:58.930908 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9b6822f3-85d8-48d5-8887-1f1e897c64d7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.666906 Sep 27 10:12:58.945486 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2160/3957] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:58 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:12:59.223522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:59.240347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:59.242641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '976e7f61517e4e3f9f2417078225ca62', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b5087b7fd4b3482082e5dd90fdcef18f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:59.244343 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-385fe51a-4836-40a7-bc4c-b889014a34d8 tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/cde27309-a349-4e7e-a4a8-1032336b61da" status: 200 len: 1362 microversion: 2.47 time: 0.470500 Sep 27 10:12:59.245614 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1798/3958] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:58 2022] GET /compute/v2.1/servers/cde27309-a349-4e7e-a4a8-1032336b61da => generated 1362 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:12:59.572833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-68cd0eb4-d1ce-409e-b54d-e141f27943ee tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:59.572833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-68cd0eb4-d1ce-409e-b54d-e141f27943ee tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:59.575408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68cd0eb4-d1ce-409e-b54d-e141f27943ee tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:59.576597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68cd0eb4-d1ce-409e-b54d-e141f27943ee tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:59.576999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68cd0eb4-d1ce-409e-b54d-e141f27943ee tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:59.577748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68cd0eb4-d1ce-409e-b54d-e141f27943ee tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:59.581366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68cd0eb4-d1ce-409e-b54d-e141f27943ee tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:59.581904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68cd0eb4-d1ce-409e-b54d-e141f27943ee tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:59.914505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-68cd0eb4-d1ce-409e-b54d-e141f27943ee tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:12:59.925211 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-68cd0eb4-d1ce-409e-b54d-e141f27943ee tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1323 microversion: 2.1 time: 0.364072 Sep 27 10:12:59.928403 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2161/3959] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:12:59 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1323 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:12:59.959842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:12:59.960680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:12:59.970961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:59.971421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:59.971849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:12:59.972574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:12:59.976470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:12:59.977213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:00.278112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:00.279010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] [instance: cde27309-a349-4e7e-a4a8-1032336b61da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:00.286548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:00.287121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:00.287641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:00.289907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:00.290444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:00.290891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:00.306389 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] HTTP exception thrown: Instance cde27309-a349-4e7e-a4a8-1032336b61da could not be found. Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Request method failure captured: Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/cde27309-a349-4e7e-a4a8-1032336b61da HTTP/1.1 Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsxyk0Q3RjEpBi8ornxPhJiUJ7CJ-ozZGkh2W9gNB9_tixstUZXNx3GhilODAiBJpXNmFiAvFNweOaPmmDtx1qpejl7KKsPLdRrruy9CHCguWk878jOYEI3sLp0_CGho24ujIDVqsC9ppJg88QBbNptj_HiGtLvBUAROC2rnUs9QrHQ Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.47 Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b5087b7fd4b3482082e5dd90fdcef18f Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerShowV247Test-1411625565 Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b5087b7fd4b3482082e5dd90fdcef18f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b5087b7fd4b3482082e5dd90fdcef18f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b5087b7fd4b3482082e5dd90fdcef18f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b5087b7fd4b3482082e5dd90fdcef18f"}]}] Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerShowV247Test-1411625565 Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b5087b7fd4b3482082e5dd90fdcef18f Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerShowV247Test-1411625565 Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerShowV247Test-1411625565-project Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 976e7f61517e4e3f9f2417078225ca62 Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerShowV247Test-1411625565-project Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance cde27309-a349-4e7e-a4a8-1032336b61da could not be found. Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'cde27309-a349-4e7e-a4a8-1032336b61da'} Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cde27309-a349-4e7e-a4a8-1032336b61da could not be found. Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cde27309-a349-4e7e-a4a8-1032336b61da could not be found. Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:13:00.308179 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:13:00.318059 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:13:00.318059 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:13:00.318059 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:13:00.318059 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:13:00.318059 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:13:00.318059 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cde27309-a349-4e7e-a4a8-1032336b61da could not be found. Sep 27 10:13:00.318059 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:13:00.318059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] Returning 404 to user: Instance cde27309-a349-4e7e-a4a8-1032336b61da could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:13:00.318059 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c5935aa-83e4-4e4e-8876-e82b395a906c tempest-ServerShowV247Test-1411625565 tempest-ServerShowV247Test-1411625565-project] 10.222.0.26 "GET /compute/v2.1/servers/cde27309-a349-4e7e-a4a8-1032336b61da" status: 404 len: 111 microversion: 2.47 time: 0.035940 Sep 27 10:13:00.318059 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2162/3960] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:00 2022] GET /compute/v2.1/servers/cde27309-a349-4e7e-a4a8-1032336b61da => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 27 10:13:00.528815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:00.541636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:00.545133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:00.546760 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8b432d50-56a3-43bc-a049-6ee892eb9f8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.591043 Sep 27 10:13:00.547517 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1799/3961] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:12:59 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:00.954103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9a33ba12-d365-45ba-9be0-67534a99b65f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:00.954780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9a33ba12-d365-45ba-9be0-67534a99b65f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:00.982455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a33ba12-d365-45ba-9be0-67534a99b65f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:00.983517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a33ba12-d365-45ba-9be0-67534a99b65f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:00.984546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a33ba12-d365-45ba-9be0-67534a99b65f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:00.985922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a33ba12-d365-45ba-9be0-67534a99b65f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:00.986696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a33ba12-d365-45ba-9be0-67534a99b65f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:00.987440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a33ba12-d365-45ba-9be0-67534a99b65f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:01.577243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:01.578063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:01.586723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:01.587567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:01.588115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:01.589167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:01.589878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:01.590578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:01.644061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9a33ba12-d365-45ba-9be0-67534a99b65f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:01.655447 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9a33ba12-d365-45ba-9be0-67534a99b65f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1323 microversion: 2.1 time: 0.707810 Sep 27 10:13:01.656374 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2163/3962] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:00 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1323 bytes in 710 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:01.774430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:01.946100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:02.135414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:02.166937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:02.168720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:02.170007 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8cc94f2b-06c7-4323-86cc-7391866aac9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.599400 Sep 27 10:13:02.175429 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1800/3963] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:01 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:13:02.679202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69816fad-97b5-42bb-8ff0-6fa20aab4c65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:02.680090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69816fad-97b5-42bb-8ff0-6fa20aab4c65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:02.701242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69816fad-97b5-42bb-8ff0-6fa20aab4c65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:02.701963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69816fad-97b5-42bb-8ff0-6fa20aab4c65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:02.702822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69816fad-97b5-42bb-8ff0-6fa20aab4c65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:02.703814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69816fad-97b5-42bb-8ff0-6fa20aab4c65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:02.704348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69816fad-97b5-42bb-8ff0-6fa20aab4c65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:02.704848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69816fad-97b5-42bb-8ff0-6fa20aab4c65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:02.996334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69816fad-97b5-42bb-8ff0-6fa20aab4c65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:03.016758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ff5d4621-efb7-4c41-8bcf-2aee4ba1b30a req-29af041e-9951-4f90-94e7-7200c78bbe8c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ce015738-c16b-4f29-8cd0-b96245268712", "name": "network-vif-plugged", "status": "completed", "tag": "1871cdde-9d60-4d78-b0d8-e7defafdce3e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:03.022853 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69816fad-97b5-42bb-8ff0-6fa20aab4c65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1323 microversion: 2.1 time: 0.347100 Sep 27 10:13:03.028216 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2164/3964] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:02 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1323 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:03.035094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ff5d4621-efb7-4c41-8bcf-2aee4ba1b30a req-29af041e-9951-4f90-94e7-7200c78bbe8c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:03.035907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ff5d4621-efb7-4c41-8bcf-2aee4ba1b30a req-29af041e-9951-4f90-94e7-7200c78bbe8c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:03.036340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ff5d4621-efb7-4c41-8bcf-2aee4ba1b30a req-29af041e-9951-4f90-94e7-7200c78bbe8c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:03.068720 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ff5d4621-efb7-4c41-8bcf-2aee4ba1b30a req-29af041e-9951-4f90-94e7-7200c78bbe8c service nova] Creating event network-vif-plugged:1871cdde-9d60-4d78-b0d8-e7defafdce3e for instance ce015738-c16b-4f29-8cd0-b96245268712 on nch2-master-1 Sep 27 10:13:03.095958 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ff5d4621-efb7-4c41-8bcf-2aee4ba1b30a req-29af041e-9951-4f90-94e7-7200c78bbe8c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081415 Sep 27 10:13:03.095958 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1801/3965] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:13:03.198712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:03.199496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:03.209112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:03.209720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:03.210256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:03.211068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:03.211710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:03.212151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:03.285829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ff5d4621-efb7-4c41-8bcf-2aee4ba1b30a req-29af041e-9951-4f90-94e7-7200c78bbe8c service nova] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:03.287185 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ff5d4621-efb7-4c41-8bcf-2aee4ba1b30a req-29af041e-9951-4f90-94e7-7200c78bbe8c service nova] 10.222.0.26 "GET /compute//" status: 200 len: 377 microversion: - time: 0.002352 Sep 27 10:13:03.296576 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1802/3966] 10.222.0.26 () {56 vars in 886 bytes} [Tue Sep 27 10:13:03 2022] GET /compute/ => generated 377 bytes in 12 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Sep 27 10:13:03.553963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:03.572767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:03.574701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:03.576004 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-26aed04c-779e-471a-8814-59a2ee06fba6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.382115 Sep 27 10:13:03.576477 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2165/3967] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:03 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:04.052770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f36d998b-21a5-469f-a612-feb8a0afe26a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:04.054804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f36d998b-21a5-469f-a612-feb8a0afe26a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:04.065046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36d998b-21a5-469f-a612-feb8a0afe26a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:04.066620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36d998b-21a5-469f-a612-feb8a0afe26a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:04.066620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36d998b-21a5-469f-a612-feb8a0afe26a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:04.067278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36d998b-21a5-469f-a612-feb8a0afe26a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:04.068002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36d998b-21a5-469f-a612-feb8a0afe26a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:04.068643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36d998b-21a5-469f-a612-feb8a0afe26a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:04.323971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f36d998b-21a5-469f-a612-feb8a0afe26a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:04.339253 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f36d998b-21a5-469f-a612-feb8a0afe26a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1323 microversion: 2.1 time: 0.292179 Sep 27 10:13:04.354153 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1803/3968] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:04 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1323 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:04.607785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:04.609287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:04.621865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:04.623921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:04.625702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:04.626795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:04.627551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:04.628521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:05.023816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:05.040471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:05.044458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:05.046378 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f6ef5abb-5593-4809-bc66-8cbad6d7ba49 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.446701 Sep 27 10:13:05.047367 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2166/3969] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:04 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:05.369559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b25f2d04-c4a3-48e9-8bc2-0eeeaae49310 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:05.370300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b25f2d04-c4a3-48e9-8bc2-0eeeaae49310 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:05.383739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b25f2d04-c4a3-48e9-8bc2-0eeeaae49310 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:05.384545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b25f2d04-c4a3-48e9-8bc2-0eeeaae49310 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:05.385738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b25f2d04-c4a3-48e9-8bc2-0eeeaae49310 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:05.388150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b25f2d04-c4a3-48e9-8bc2-0eeeaae49310 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:05.390397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b25f2d04-c4a3-48e9-8bc2-0eeeaae49310 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:05.390826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b25f2d04-c4a3-48e9-8bc2-0eeeaae49310 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:05.634826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b25f2d04-c4a3-48e9-8bc2-0eeeaae49310 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:05.654635 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b25f2d04-c4a3-48e9-8bc2-0eeeaae49310 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1323 microversion: 2.1 time: 0.291482 Sep 27 10:13:05.664849 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1804/3970] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:05 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1323 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:06.093063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:06.096174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:06.116809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:06.119013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:06.120499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:06.123708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:06.124214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:06.125229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:06.566064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:06.582942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:06.584479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:06.587611 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d78bee3e-844b-4aca-be84-165ea84b3690 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.517160 Sep 27 10:13:06.589203 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2167/3971] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:06 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:06.688678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9b002453-865c-422f-95b6-6aed148867e0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:06.689306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9b002453-865c-422f-95b6-6aed148867e0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:06.706905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b002453-865c-422f-95b6-6aed148867e0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:06.710067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b002453-865c-422f-95b6-6aed148867e0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:06.712221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b002453-865c-422f-95b6-6aed148867e0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:06.713322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b002453-865c-422f-95b6-6aed148867e0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:06.714010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b002453-865c-422f-95b6-6aed148867e0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:06.717268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9b002453-865c-422f-95b6-6aed148867e0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:07.168955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9b002453-865c-422f-95b6-6aed148867e0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:07.182843 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9b002453-865c-422f-95b6-6aed148867e0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1519 microversion: 2.1 time: 0.498729 Sep 27 10:13:07.185174 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1805/3972] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:06 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1519 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:07.215467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-deca15ad-aaad-4fd1-9b7b-e366c81cb6a2 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:07.218068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-deca15ad-aaad-4fd1-9b7b-e366c81cb6a2 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:07.228743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-deca15ad-aaad-4fd1-9b7b-e366c81cb6a2 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:07.229732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-deca15ad-aaad-4fd1-9b7b-e366c81cb6a2 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:07.230205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-deca15ad-aaad-4fd1-9b7b-e366c81cb6a2 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:07.231183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-deca15ad-aaad-4fd1-9b7b-e366c81cb6a2 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:07.231676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-deca15ad-aaad-4fd1-9b7b-e366c81cb6a2 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:07.232106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-deca15ad-aaad-4fd1-9b7b-e366c81cb6a2 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:07.292100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-deca15ad-aaad-4fd1-9b7b-e366c81cb6a2 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lazy-loading 'flavor' on Instance uuid ce015738-c16b-4f29-8cd0-b96245268712 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:13:07.437988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75a7d0ad-06eb-467f-a0b2-4001f64c419b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:07.446145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-75a7d0ad-06eb-467f-a0b2-4001f64c419b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:07.448093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-75a7d0ad-06eb-467f-a0b2-4001f64c419b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Searching by: {'deleted': False, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:13:07.461942 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-deca15ad-aaad-4fd1-9b7b-e366c81cb6a2 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "POST /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712/action" status: 200 len: 32 microversion: 2.1 time: 0.251567 Sep 27 10:13:07.463325 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2168/3973] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:13:07 2022] POST /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712/action => generated 32 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:13:07.474419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d67f88fb-975f-4c9a-9640-71779803e95c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:07.476482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d67f88fb-975f-4c9a-9640-71779803e95c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:07.484144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7d0ad-06eb-467f-a0b2-4001f64c419b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:07.486005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d67f88fb-975f-4c9a-9640-71779803e95c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:07.487329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d67f88fb-975f-4c9a-9640-71779803e95c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:07.487703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7d0ad-06eb-467f-a0b2-4001f64c419b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:07.488336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d67f88fb-975f-4c9a-9640-71779803e95c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:07.489042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7d0ad-06eb-467f-a0b2-4001f64c419b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:07.489505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d67f88fb-975f-4c9a-9640-71779803e95c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:07.490152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d67f88fb-975f-4c9a-9640-71779803e95c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:07.491121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d67f88fb-975f-4c9a-9640-71779803e95c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:07.491973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7d0ad-06eb-467f-a0b2-4001f64c419b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:07.493552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7d0ad-06eb-467f-a0b2-4001f64c419b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:07.494358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7d0ad-06eb-467f-a0b2-4001f64c419b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:07.497364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:07.537326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:07.570118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-75a7d0ad-06eb-467f-a0b2-4001f64c419b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:13:07.635861 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-75a7d0ad-06eb-467f-a0b2-4001f64c419b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 339 microversion: 2.1 time: 0.203233 Sep 27 10:13:07.638065 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1806/3974] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:13:07 2022] GET /compute/v2.1/servers => generated 339 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:07.646233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:07.647336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:07.676987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:07.677812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:07.678701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:07.680004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:07.680869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:07.684156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:07.898282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d67f88fb-975f-4c9a-9640-71779803e95c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:07.911925 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d67f88fb-975f-4c9a-9640-71779803e95c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.442826 Sep 27 10:13:07.913986 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2169/3975] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:07 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:07.929517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:07.937236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:07.938916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Searching by: {'deleted': False, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:13:07.947354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:07.948333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:07.949057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:07.950301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:07.951130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:07.951775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:07.989285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:13:08.058095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:08.066382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:08.068459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:08.068459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:08.270602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:08.294583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:08.298335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:08.299703 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b91c327c-2c78-4ffc-a773-43b708b2a194 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.660486 Sep 27 10:13:08.300622 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1807/3976] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:07 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 662 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:08.315800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:08.317205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:08.326104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:08.327938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:08.328703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:08.331099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:08.331924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:08.333157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:08.882921 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e82b7f0f-7d14-42ef-96bb-d6f32ee74a4e tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/detail" status: 200 len: 1712 microversion: 2.1 time: 0.958356 Sep 27 10:13:08.883839 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2170/3977] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:13:07 2022] GET /compute/v2.1/servers/detail => generated 1712 bytes in 961 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:08.912502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-937e65ae-3492-434e-acbe-4a4cf83044a8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:08.927333 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-937e65ae-3492-434e-acbe-4a4cf83044a8 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/flavors/452" status: 200 len: 370 microversion: 2.1 time: 0.018962 Sep 27 10:13:08.928359 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2171/3978] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:13:08 2022] GET /compute/v2.1/flavors/452 => generated 370 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:08.939628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-45bb84f7-e7ac-4c6d-9f60-fd7ffa44891f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:08.940253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-45bb84f7-e7ac-4c6d-9f60-fd7ffa44891f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:08.958609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45bb84f7-e7ac-4c6d-9f60-fd7ffa44891f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:08.959100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45bb84f7-e7ac-4c6d-9f60-fd7ffa44891f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:08.959624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45bb84f7-e7ac-4c6d-9f60-fd7ffa44891f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:08.960610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45bb84f7-e7ac-4c6d-9f60-fd7ffa44891f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:08.961102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45bb84f7-e7ac-4c6d-9f60-fd7ffa44891f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:08.961528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45bb84f7-e7ac-4c6d-9f60-fd7ffa44891f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:08.999806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:09.011101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:09.012947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:09.014661 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3dce124f-a04d-4a35-9ce4-3854c44c5102 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2951 microversion: 2.42 time: 0.704222 Sep 27 10:13:09.015486 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1808/3979] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:08 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2951 bytes in 714 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:09.043333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9bb760d0-ef6d-4939-b578-c58febb207c2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:09.043841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9bb760d0-ef6d-4939-b578-c58febb207c2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:09.060850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9bb760d0-ef6d-4939-b578-c58febb207c2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:09.061309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9bb760d0-ef6d-4939-b578-c58febb207c2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:09.061765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9bb760d0-ef6d-4939-b578-c58febb207c2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:09.062557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9bb760d0-ef6d-4939-b578-c58febb207c2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:09.062972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9bb760d0-ef6d-4939-b578-c58febb207c2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:09.063629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9bb760d0-ef6d-4939-b578-c58febb207c2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:09.335046 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9bb760d0-ef6d-4939-b578-c58febb207c2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b/os-interface" status: 200 len: 1831 microversion: 2.42 time: 0.295607 Sep 27 10:13:09.340072 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1809/3980] 10.222.0.26 () {60 vars in 1316 bytes} [Tue Sep 27 10:13:09 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b/os-interface => generated 1831 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:13:09.355993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-45bb84f7-e7ac-4c6d-9f60-fd7ffa44891f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:09.370442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-628d7388-7b72-466d-a3f8-605e227cddd5 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:09.371969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-628d7388-7b72-466d-a3f8-605e227cddd5 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:09.386492 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-45bb84f7-e7ac-4c6d-9f60-fd7ffa44891f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.449457 Sep 27 10:13:09.388900 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2172/3981] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:08 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:09.396768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-628d7388-7b72-466d-a3f8-605e227cddd5 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:09.398793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-628d7388-7b72-466d-a3f8-605e227cddd5 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:09.399908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-628d7388-7b72-466d-a3f8-605e227cddd5 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:09.404274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-628d7388-7b72-466d-a3f8-605e227cddd5 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:09.405364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-628d7388-7b72-466d-a3f8-605e227cddd5 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:09.406256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-628d7388-7b72-466d-a3f8-605e227cddd5 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:09.408185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:09.409149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:09.429795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:09.430657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:09.433200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:09.435867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:09.436970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:09.438487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:09.502604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-628d7388-7b72-466d-a3f8-605e227cddd5 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:13:09.660340 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-628d7388-7b72-466d-a3f8-605e227cddd5 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "DELETE /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 204 len: 0 microversion: 2.1 time: 0.317898 Sep 27 10:13:09.665197 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1810/3982] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:13:09 2022] DELETE /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:13:09.675402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fdc09de0-c01c-444f-9c30-2e96c6c2f986 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:09.676853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fdc09de0-c01c-444f-9c30-2e96c6c2f986 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:09.687188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdc09de0-c01c-444f-9c30-2e96c6c2f986 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:09.687833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdc09de0-c01c-444f-9c30-2e96c6c2f986 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:09.688386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdc09de0-c01c-444f-9c30-2e96c6c2f986 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:09.689369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdc09de0-c01c-444f-9c30-2e96c6c2f986 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:09.690236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdc09de0-c01c-444f-9c30-2e96c6c2f986 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:09.690366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fdc09de0-c01c-444f-9c30-2e96c6c2f986 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:10.026791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:10.041344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:10.042820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:10.052529 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-15c8cce5-4d13-4d64-be18-ce9bc2cc912b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.662243 Sep 27 10:13:10.064910 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2173/3983] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:09 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:10.163979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fdc09de0-c01c-444f-9c30-2e96c6c2f986 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:10.173060 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fdc09de0-c01c-444f-9c30-2e96c6c2f986 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1715 microversion: 2.1 time: 0.506039 Sep 27 10:13:10.181652 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1811/3984] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:09 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1715 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:10.422722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-145ce871-4e03-4579-9e66-7a56cb75aa93 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:10.433121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-145ce871-4e03-4579-9e66-7a56cb75aa93 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:10.465533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145ce871-4e03-4579-9e66-7a56cb75aa93 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:10.466028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145ce871-4e03-4579-9e66-7a56cb75aa93 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:10.474573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145ce871-4e03-4579-9e66-7a56cb75aa93 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:10.477467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145ce871-4e03-4579-9e66-7a56cb75aa93 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:10.478170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145ce871-4e03-4579-9e66-7a56cb75aa93 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:10.478793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145ce871-4e03-4579-9e66-7a56cb75aa93 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:11.084141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:11.085224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:11.105209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:11.106446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:11.107374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:11.108339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:11.109085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:11.109767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:11.162214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-145ce871-4e03-4579-9e66-7a56cb75aa93 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:11.171344 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-145ce871-4e03-4579-9e66-7a56cb75aa93 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.755461 Sep 27 10:13:11.171948 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2174/3985] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:10 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 757 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:11.213373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-328ee9fe-eb46-482d-b349-793ace5a83da tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:11.214292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-328ee9fe-eb46-482d-b349-793ace5a83da tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:11.222000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-328ee9fe-eb46-482d-b349-793ace5a83da tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:11.222696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-328ee9fe-eb46-482d-b349-793ace5a83da tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:11.223323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-328ee9fe-eb46-482d-b349-793ace5a83da tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:11.224256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-328ee9fe-eb46-482d-b349-793ace5a83da tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:11.224899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-328ee9fe-eb46-482d-b349-793ace5a83da tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:11.225557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-328ee9fe-eb46-482d-b349-793ace5a83da tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:11.670303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:11.685640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:11.713512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:11.715828 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a3ff2c67-a603-4d37-b855-67215528dd43 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.635810 Sep 27 10:13:11.724823 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1812/3986] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:11 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:13:11.749088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-328ee9fe-eb46-482d-b349-793ace5a83da tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:11.773116 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-328ee9fe-eb46-482d-b349-793ace5a83da tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1715 microversion: 2.1 time: 0.569920 Sep 27 10:13:11.774028 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2175/3987] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:11 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1715 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:12.199619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd43b9e8-629b-4ed7-8c71-1e27df3ea89a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:12.200824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd43b9e8-629b-4ed7-8c71-1e27df3ea89a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:12.221817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd43b9e8-629b-4ed7-8c71-1e27df3ea89a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:12.222323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd43b9e8-629b-4ed7-8c71-1e27df3ea89a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:12.222758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd43b9e8-629b-4ed7-8c71-1e27df3ea89a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:12.223480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd43b9e8-629b-4ed7-8c71-1e27df3ea89a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:12.223938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd43b9e8-629b-4ed7-8c71-1e27df3ea89a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:12.224364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd43b9e8-629b-4ed7-8c71-1e27df3ea89a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:12.751728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:12.752978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:12.767710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:12.770413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:12.771055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:12.774328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:12.775122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:12.776508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:13.268179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd43b9e8-629b-4ed7-8c71-1e27df3ea89a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:13.286551 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd43b9e8-629b-4ed7-8c71-1e27df3ea89a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 1.092722 Sep 27 10:13:13.287872 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1813/3988] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:12 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 1094 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:13.293275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-907f2e41-9953-4584-b907-04629b86f51b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:13.296416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-907f2e41-9953-4584-b907-04629b86f51b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:13.321377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-907f2e41-9953-4584-b907-04629b86f51b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:13.325888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-907f2e41-9953-4584-b907-04629b86f51b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:13.326868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-907f2e41-9953-4584-b907-04629b86f51b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:13.327625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-907f2e41-9953-4584-b907-04629b86f51b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:13.328035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-907f2e41-9953-4584-b907-04629b86f51b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:13.332804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-907f2e41-9953-4584-b907-04629b86f51b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:13.766225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-907f2e41-9953-4584-b907-04629b86f51b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:13.775749 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-907f2e41-9953-4584-b907-04629b86f51b tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1715 microversion: 2.1 time: 0.486356 Sep 27 10:13:13.790178 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1814/3989] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:13 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1715 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:13.984150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:14.005461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:14.011553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:14.020394 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fe9377ed-1500-4ae9-9739-763b505fb95a tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 1.273043 Sep 27 10:13:14.045109 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2176/3990] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:12 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 1298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:14.319422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-60947d6c-dcf4-4022-a4ae-0aaf8dda0804 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:14.321183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-60947d6c-dcf4-4022-a4ae-0aaf8dda0804 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:14.332833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60947d6c-dcf4-4022-a4ae-0aaf8dda0804 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:14.338276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60947d6c-dcf4-4022-a4ae-0aaf8dda0804 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:14.339049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60947d6c-dcf4-4022-a4ae-0aaf8dda0804 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:14.340019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60947d6c-dcf4-4022-a4ae-0aaf8dda0804 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:14.345225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60947d6c-dcf4-4022-a4ae-0aaf8dda0804 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:14.349566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60947d6c-dcf4-4022-a4ae-0aaf8dda0804 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:14.802367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae2985ad-856f-442d-9c82-00ccdc4de717 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:14.816017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ae2985ad-856f-442d-9c82-00ccdc4de717 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:14.831718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae2985ad-856f-442d-9c82-00ccdc4de717 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:14.832340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae2985ad-856f-442d-9c82-00ccdc4de717 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:14.832914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae2985ad-856f-442d-9c82-00ccdc4de717 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:14.833506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae2985ad-856f-442d-9c82-00ccdc4de717 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:14.845156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae2985ad-856f-442d-9c82-00ccdc4de717 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:14.846169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae2985ad-856f-442d-9c82-00ccdc4de717 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:15.007118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ae2985ad-856f-442d-9c82-00ccdc4de717 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '015d442ea03c434bb63aa9ba9b537ce8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0a1ced5ea0884fc99d7644f37ca3f376', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:15.025743 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ae2985ad-856f-442d-9c82-00ccdc4de717 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 200 len: 1648 microversion: 2.1 time: 0.225036 Sep 27 10:13:15.025743 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2177/3991] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:14 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 1648 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:15.041180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d36eceb6-af08-4c46-a17d-44e8dfa46a24 req-fe079305-6c39-4a9c-9322-c0cbf19a3742 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6cece583-0807-44b9-a6c1-ea77401b7f51", "name": "network-vif-deleted", "tag": "ff323f5a-e504-4eb7-9433-e9c539434972"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:15.070287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d36eceb6-af08-4c46-a17d-44e8dfa46a24 req-fe079305-6c39-4a9c-9322-c0cbf19a3742 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:15.077358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d36eceb6-af08-4c46-a17d-44e8dfa46a24 req-fe079305-6c39-4a9c-9322-c0cbf19a3742 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:15.078043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d36eceb6-af08-4c46-a17d-44e8dfa46a24 req-fe079305-6c39-4a9c-9322-c0cbf19a3742 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:15.098844 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d36eceb6-af08-4c46-a17d-44e8dfa46a24 req-fe079305-6c39-4a9c-9322-c0cbf19a3742 service nova] Creating event network-vif-deleted:ff323f5a-e504-4eb7-9433-e9c539434972 for instance 6cece583-0807-44b9-a6c1-ea77401b7f51 on nch1-master-1 Sep 27 10:13:15.126034 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d36eceb6-af08-4c46-a17d-44e8dfa46a24 req-fe079305-6c39-4a9c-9322-c0cbf19a3742 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100159 Sep 27 10:13:15.133817 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2178/3992] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:15.146088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:15.150821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:15.181652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:15.182180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:15.182642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:15.183367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:15.183834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:15.184253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:15.284975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-60947d6c-dcf4-4022-a4ae-0aaf8dda0804 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:15.300089 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-60947d6c-dcf4-4022-a4ae-0aaf8dda0804 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.985430 Sep 27 10:13:15.316684 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1815/3993] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:14 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 990 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:15.918642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:15.946405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:15.955176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:15.957924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-21f653c3-313d-4efe-a8b9-86046428b206 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.822595 Sep 27 10:13:15.960894 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2179/3994] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:15 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 826 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:16.055397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:16.056148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] [instance: 6cece583-0807-44b9-a6c1-ea77401b7f51] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:16.074909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:16.076285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:16.078372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:16.079966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:16.081578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:16.083904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:16.122792 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] HTTP exception thrown: Instance 6cece583-0807-44b9-a6c1-ea77401b7f51 could not be found. Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Request method failure captured: Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 HTTP/1.1 Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsxqUBV-8qXw4iUnAA7pB_wyhhlg7y5zeLHqO6_bXhrIHfF9izYSNpEY8pnBZyTzO496I2MZKfFu40z0QJqbDh1XGBA-sYmYoA6eGeTdMj6Y3aMbSbRPu2bQLIO15NRtMGJmu_RKzhN8YEVNn_3wggcwVbo3IZEpjd1pBsEPxC58iwk Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 0a1ced5ea0884fc99d7644f37ca3f376 Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersTestManualDisk-1093943796 Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0a1ced5ea0884fc99d7644f37ca3f376"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0a1ced5ea0884fc99d7644f37ca3f376"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0a1ced5ea0884fc99d7644f37ca3f376"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0a1ced5ea0884fc99d7644f37ca3f376"}]}] Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersTestManualDisk-1093943796 Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 0a1ced5ea0884fc99d7644f37ca3f376 Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersTestManualDisk-1093943796 Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersTestManualDisk-1093943796-project Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 015d442ea03c434bb63aa9ba9b537ce8 Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersTestManualDisk-1093943796-project Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 6cece583-0807-44b9-a6c1-ea77401b7f51 could not be found. Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '6cece583-0807-44b9-a6c1-ea77401b7f51'} Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6cece583-0807-44b9-a6c1-ea77401b7f51 could not be found. Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6cece583-0807-44b9-a6c1-ea77401b7f51 could not be found. Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:13:16.127693 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:13:16.133774 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:13:16.133774 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:13:16.133774 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:13:16.133774 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:13:16.133774 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:13:16.133774 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6cece583-0807-44b9-a6c1-ea77401b7f51 could not be found. Sep 27 10:13:16.133774 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:13:16.133774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Returning 404 to user: Instance 6cece583-0807-44b9-a6c1-ea77401b7f51 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:13:16.138196 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b99ee0f9-4a20-44e4-9479-9c6157b9fbcd tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51" status: 404 len: 111 microversion: 2.1 time: 0.088781 Sep 27 10:13:16.139779 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1816/3995] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:16 2022] GET /compute/v2.1/servers/6cece583-0807-44b9-a6c1-ea77401b7f51 => generated 111 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:13:16.151690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e7439aa9-a81c-4ded-b73c-ecb12543a701 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] Calling method '.version_select at 0x7f7f63ac3e50>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:16.193673 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e7439aa9-a81c-4ded-b73c-ecb12543a701 tempest-ServersTestManualDisk-1093943796 tempest-ServersTestManualDisk-1093943796-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1960047129" status: 202 len: 0 microversion: 2.1 time: 0.046843 Sep 27 10:13:16.196554 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2180/3996] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:13:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1960047129 => generated 0 bytes in 54 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:13:16.328652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32f31676-45c3-4a73-adb5-54d26abba9d1 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:16.329684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32f31676-45c3-4a73-adb5-54d26abba9d1 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:16.349487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32f31676-45c3-4a73-adb5-54d26abba9d1 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:16.350356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32f31676-45c3-4a73-adb5-54d26abba9d1 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:16.350984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32f31676-45c3-4a73-adb5-54d26abba9d1 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:16.351931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32f31676-45c3-4a73-adb5-54d26abba9d1 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:16.352567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32f31676-45c3-4a73-adb5-54d26abba9d1 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:16.353206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32f31676-45c3-4a73-adb5-54d26abba9d1 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:16.989922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:16.990549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:17.011113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:17.011616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:17.012073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:17.022255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:17.023447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:17.029001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:17.355312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32f31676-45c3-4a73-adb5-54d26abba9d1 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:17.371493 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32f31676-45c3-4a73-adb5-54d26abba9d1 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 1.047993 Sep 27 10:13:17.373816 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1817/3997] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:16 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 1051 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:13:17.670340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:17.692284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:17.697282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:17.699143 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cb32a55d-5bd3-432e-9771-e054f6370e03 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.714453 Sep 27 10:13:17.699956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:17.731274 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2181/3998] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:16 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 747 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:13:17.966550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-99d52dd4-36e9-4117-adc8-a12c5c62d040 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] Action: 'create', calling method: .version_select at 0x7f7f626d8dc0>, body: {"pool": null} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:18.395640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62e01271-8ea8-4ed0-bb37-1fab6a1a240f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:18.396571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-62e01271-8ea8-4ed0-bb37-1fab6a1a240f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:18.410599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:18.420965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:18.423563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e01271-8ea8-4ed0-bb37-1fab6a1a240f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:18.425082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e01271-8ea8-4ed0-bb37-1fab6a1a240f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:18.425633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e01271-8ea8-4ed0-bb37-1fab6a1a240f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:18.432066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e01271-8ea8-4ed0-bb37-1fab6a1a240f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:18.434619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e01271-8ea8-4ed0-bb37-1fab6a1a240f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:18.435237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e01271-8ea8-4ed0-bb37-1fab6a1a240f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:19.380650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-62e01271-8ea8-4ed0-bb37-1fab6a1a240f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:19.395570 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-62e01271-8ea8-4ed0-bb37-1fab6a1a240f tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 1.002695 Sep 27 10:13:19.414015 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2182/3999] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:18 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 1021 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:19.430231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:19.431117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:19.434171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=80,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:19.444164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:19.450210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:19.451045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:19.451945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:19.452536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:19.453148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:20.265179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:20.285329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:20.287150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:20.288986 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8c68c989-a329-4291-a88a-c7d28b122970 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.875189 Sep 27 10:13:20.290174 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2183/4000] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:19 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 877 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:20.421720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c4d2ff5c-9e86-483a-aa23-e0feedb5ee46 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:20.423071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c4d2ff5c-9e86-483a-aa23-e0feedb5ee46 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:20.436440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4d2ff5c-9e86-483a-aa23-e0feedb5ee46 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:20.437259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4d2ff5c-9e86-483a-aa23-e0feedb5ee46 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:20.437886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4d2ff5c-9e86-483a-aa23-e0feedb5ee46 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:20.438796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4d2ff5c-9e86-483a-aa23-e0feedb5ee46 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:20.449549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4d2ff5c-9e86-483a-aa23-e0feedb5ee46 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:20.450226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4d2ff5c-9e86-483a-aa23-e0feedb5ee46 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:20.625587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c4d2ff5c-9e86-483a-aa23-e0feedb5ee46 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:20.639430 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c4d2ff5c-9e86-483a-aa23-e0feedb5ee46 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.221587 Sep 27 10:13:20.640049 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2184/4001] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:20 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:13:21.276702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-64390ba9-5ea3-4461-9a0d-31353bc923bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Action: 'create', calling method: .version_select at 0x7f7f6c2ccd30>, body: {"keypair": {"name": "tempest-keypair-436902387", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDAmpgqxKxJS5APvV0NEy9Rcu+zVDU+xfB77bmsoMNLQcnydLifMqr7ZLZlqjTCDjdo+iZnGahFKtezUYk8LEIWX9B5oubtLELfTJQsCTEHz6EGmI38Qan21MKMBZc7Ekw=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:21.327753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-64390ba9-5ea3-4461-9a0d-31353bc923bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Getting quotas for project e2e00c90216b4d1b80ac7892ee23550f. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:13:21.352082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-64390ba9-5ea3-4461-9a0d-31353bc923bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Getting quotas for user 279d3662f60c4a979c22ee3e8fa92865 and project e2e00c90216b4d1b80ac7892ee23550f. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:13:21.429930 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-64390ba9-5ea3-4461-9a0d-31353bc923bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.259619 Sep 27 10:13:21.431194 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2185/4002] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:13:21 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:13:21.447207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:21.450479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:21.476720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:21.479816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:21.481304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:21.485226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:21.485898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:21.487582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:21.647467 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-99d52dd4-36e9-4117-adc8-a12c5c62d040 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] 10.222.0.26 "POST /compute/v2.1/os-floating-ips" status: 200 len: 141 microversion: 2.1 time: 3.971880 Sep 27 10:13:21.649908 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1818/4003] 10.222.0.26 () {60 vars in 1170 bytes} [Tue Sep 27 10:13:17 2022] POST /compute/v2.1/os-floating-ips => generated 141 bytes in 3975 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:13:21.663307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fff37b2e-b56e-4f42-a355-7421f4460436 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] Action: 'create', calling method: .version_select at 0x7f7f62e63820>, body: {"pool": null} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:22.162294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:22.190320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:22.200680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:22.205058 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-635db48b-4f81-495b-8f97-ece4267c84de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.772179 Sep 27 10:13:22.209587 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2186/4004] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:21 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 778 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:22.227291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d5aa56f4-b432-4988-a8e2-910a8d0393ab tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:22.230444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d5aa56f4-b432-4988-a8e2-910a8d0393ab tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:22.252950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5aa56f4-b432-4988-a8e2-910a8d0393ab tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:22.253893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5aa56f4-b432-4988-a8e2-910a8d0393ab tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:22.255897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5aa56f4-b432-4988-a8e2-910a8d0393ab tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:22.257110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5aa56f4-b432-4988-a8e2-910a8d0393ab tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:22.258964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5aa56f4-b432-4988-a8e2-910a8d0393ab tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:22.260639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5aa56f4-b432-4988-a8e2-910a8d0393ab tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:22.579364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d5aa56f4-b432-4988-a8e2-910a8d0393ab tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:22.603367 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d5aa56f4-b432-4988-a8e2-910a8d0393ab tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.392438 Sep 27 10:13:22.608549 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2187/4005] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:22 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:23.235893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:23.236992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:23.258565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:23.259477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:23.265313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:23.266649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:23.269429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:23.270738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:23.947293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:23.952736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:23.969312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:23.971154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:23.977552 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c70273c7-2bde-4311-9464-67a14bbe7be4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.744977 Sep 27 10:13:23.989746 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2188/4006] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:23 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:23.995539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba1ed6cc-2e20-4933-8db1-b46ad2d0be65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:24.004903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba1ed6cc-2e20-4933-8db1-b46ad2d0be65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:24.024286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba1ed6cc-2e20-4933-8db1-b46ad2d0be65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:24.025683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba1ed6cc-2e20-4933-8db1-b46ad2d0be65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:24.026493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba1ed6cc-2e20-4933-8db1-b46ad2d0be65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:24.027520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba1ed6cc-2e20-4933-8db1-b46ad2d0be65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:24.028105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba1ed6cc-2e20-4933-8db1-b46ad2d0be65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:24.028552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba1ed6cc-2e20-4933-8db1-b46ad2d0be65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:24.260111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba1ed6cc-2e20-4933-8db1-b46ad2d0be65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:24.279363 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba1ed6cc-2e20-4933-8db1-b46ad2d0be65 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.287960 Sep 27 10:13:24.280727 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2189/4007] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:23 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:25.016796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:25.019331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:25.029827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:25.030643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:25.031088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:25.031813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:25.032451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:25.033057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:25.593864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:25.610392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:25.611862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:25.620401 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13d575bc-f0d9-481e-bb4d-5315d446f404 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.608560 Sep 27 10:13:25.633812 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2190/4008] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:25 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:13:25.653417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f2eb0faa-d27a-49a5-9642-31bc7740b109 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:25.654063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f2eb0faa-d27a-49a5-9642-31bc7740b109 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:25.670563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2eb0faa-d27a-49a5-9642-31bc7740b109 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:25.671985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2eb0faa-d27a-49a5-9642-31bc7740b109 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:25.673629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2eb0faa-d27a-49a5-9642-31bc7740b109 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:25.677112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2eb0faa-d27a-49a5-9642-31bc7740b109 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:25.677599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2eb0faa-d27a-49a5-9642-31bc7740b109 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:25.678293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2eb0faa-d27a-49a5-9642-31bc7740b109 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:25.822551 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fff37b2e-b56e-4f42-a355-7421f4460436 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] 10.222.0.26 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 4.163327 Sep 27 10:13:25.824514 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1819/4009] 10.222.0.26 () {60 vars in 1170 bytes} [Tue Sep 27 10:13:21 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 4166 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:25.868838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d372df17-d51c-417e-bb1d-0c589316e98e tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] Action: 'create', calling method: .version_select at 0x7f7f635f8dc0>, body: {"pool": null} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:26.093664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f2eb0faa-d27a-49a5-9642-31bc7740b109 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:26.115459 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f2eb0faa-d27a-49a5-9642-31bc7740b109 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.479661 Sep 27 10:13:26.129707 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2191/4010] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:25 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:26.350350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=63,nova_api:UPDATE=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:26.482429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-28ce4844-16d6-4864-8a68-2d9ac1f199ce req-3be7b61c-0f38-4a96-8578-bfc291d3fe46 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-plugged", "status": "completed", "tag": "86cc38ea-2a41-41be-a24f-432ba40420c5"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:26.506164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-28ce4844-16d6-4864-8a68-2d9ac1f199ce req-3be7b61c-0f38-4a96-8578-bfc291d3fe46 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:26.506781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-28ce4844-16d6-4864-8a68-2d9ac1f199ce req-3be7b61c-0f38-4a96-8578-bfc291d3fe46 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:26.508131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-28ce4844-16d6-4864-8a68-2d9ac1f199ce req-3be7b61c-0f38-4a96-8578-bfc291d3fe46 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:26.535383 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-28ce4844-16d6-4864-8a68-2d9ac1f199ce req-3be7b61c-0f38-4a96-8578-bfc291d3fe46 service nova] Creating event network-vif-plugged:86cc38ea-2a41-41be-a24f-432ba40420c5 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:13:26.551593 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-28ce4844-16d6-4864-8a68-2d9ac1f199ce req-3be7b61c-0f38-4a96-8578-bfc291d3fe46 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073546 Sep 27 10:13:26.554185 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2192/4011] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:26.654025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:26.656443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:26.666225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:26.667436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:26.668289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:26.669820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:26.670677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:26.671615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:27.250195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:27.270231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:27.274443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:27.276603 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7e452148-624d-4bea-a694-f6367ed6659d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.628134 Sep 27 10:13:27.300616 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2193/4012] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:26 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:27.300616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1056083199", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "dc711087-d8a4-45fc-8efb-f4217b6d9ff1"}], "security_groups": [{"name": "tempest-securitygroup--1702213767"}], "key_name": "tempest-keypair-436902387", "user_data": "}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:27.361366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=75,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:27.586727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] validate_networks() for [('dc711087-d8a4-45fc-8efb-f4217b6d9ff1', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:13:27.945041 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d372df17-d51c-417e-bb1d-0c589316e98e tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] 10.222.0.26 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 2.103385 Sep 27 10:13:27.945855 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1820/4013] 10.222.0.26 () {60 vars in 1170 bytes} [Tue Sep 27 10:13:25 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 2113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:27.958271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7dda6229-b59d-453a-a03e-495d62d9fa1d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:27.959841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7dda6229-b59d-453a-a03e-495d62d9fa1d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:27.992928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dda6229-b59d-453a-a03e-495d62d9fa1d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:27.993814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dda6229-b59d-453a-a03e-495d62d9fa1d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:27.995050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dda6229-b59d-453a-a03e-495d62d9fa1d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:27.996475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dda6229-b59d-453a-a03e-495d62d9fa1d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:27.997257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dda6229-b59d-453a-a03e-495d62d9fa1d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:27.998386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dda6229-b59d-453a-a03e-495d62d9fa1d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:28.279153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7dda6229-b59d-453a-a03e-495d62d9fa1d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:28.302212 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7dda6229-b59d-453a-a03e-495d62d9fa1d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.344449 Sep 27 10:13:28.305588 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1821/4014] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:27 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:28.311996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8d3c6223-8289-4d1e-b720-a761e78ac6c0 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] Action: 'create', calling method: .version_select at 0x7f7f635d4040>, body: {"pool": null} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:28.805536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:13:28.805536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:13:28.805536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:13:28.805536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:13:28.805536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:13:28.815066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:28.815723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:28.816255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:28.817280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:28.817992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:28.818611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:28.839868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Getting quotas for project e2e00c90216b4d1b80ac7892ee23550f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:13:28.846246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Getting quotas for user 279d3662f60c4a979c22ee3e8fa92865 and project e2e00c90216b4d1b80ac7892ee23550f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:13:28.860832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:13:28.863526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:13:28.922739 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53f91026-1053-4fb9-b636-f2e9b6020336 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.635488 Sep 27 10:13:28.930539 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2194/4015] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:13:27 2022] POST /compute/v2.1/servers => generated 402 bytes in 1647 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:13:28.936740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:28.937764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:28.952189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:28.952833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:28.953397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:28.955044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:28.956007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:28.956679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:29.241484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:29.258674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:29.265260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:29.266720 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2dd2bb51-a521-4302-a1a5-d62dea3a28c6 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.335029 Sep 27 10:13:29.269381 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2195/4016] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:28 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:29.280527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-cdd46389-e04b-4c1c-8862-4d74b67f4672 req-61a1fe7e-ccbd-41df-93ff-29224cf42820 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-plugged", "status": "completed", "tag": "78c9b09b-0886-456d-897a-ad2c11dd56c3"}, {"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-plugged", "status": "completed", "tag": "10f31ce3-e11c-4b44-bb7f-173c2919d9b7"}, {"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-plugged", "status": "completed", "tag": "c8585417-f76b-4df9-869f-bf4ecaf46641"}, {"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-plugged", "status": "completed", "tag": "4a6d853f-cf26-46e4-ad6f-27d3139a8f84"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:29.314725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cdd46389-e04b-4c1c-8862-4d74b67f4672 req-61a1fe7e-ccbd-41df-93ff-29224cf42820 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:29.315484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cdd46389-e04b-4c1c-8862-4d74b67f4672 req-61a1fe7e-ccbd-41df-93ff-29224cf42820 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:29.316206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cdd46389-e04b-4c1c-8862-4d74b67f4672 req-61a1fe7e-ccbd-41df-93ff-29224cf42820 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:29.352430 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-cdd46389-e04b-4c1c-8862-4d74b67f4672 req-61a1fe7e-ccbd-41df-93ff-29224cf42820 service nova] Creating event network-vif-plugged:78c9b09b-0886-456d-897a-ad2c11dd56c3 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:13:29.353283 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-cdd46389-e04b-4c1c-8862-4d74b67f4672 req-61a1fe7e-ccbd-41df-93ff-29224cf42820 service nova] Creating event network-vif-plugged:10f31ce3-e11c-4b44-bb7f-173c2919d9b7 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:13:29.355272 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-cdd46389-e04b-4c1c-8862-4d74b67f4672 req-61a1fe7e-ccbd-41df-93ff-29224cf42820 service nova] Creating event network-vif-plugged:c8585417-f76b-4df9-869f-bf4ecaf46641 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:13:29.355983 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-cdd46389-e04b-4c1c-8862-4d74b67f4672 req-61a1fe7e-ccbd-41df-93ff-29224cf42820 service nova] Creating event network-vif-plugged:4a6d853f-cf26-46e4-ad6f-27d3139a8f84 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:13:29.369461 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-cdd46389-e04b-4c1c-8862-4d74b67f4672 req-61a1fe7e-ccbd-41df-93ff-29224cf42820 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 696 microversion: 2.1 time: 0.095297 Sep 27 10:13:29.370432 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2196/4017] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:29 2022] POST /compute/v2.1/os-server-external-events => generated 696 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:29.377296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-63b8a469-4e56-4fd3-9a6f-0e7d3f52a6de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:29.378655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-63b8a469-4e56-4fd3-9a6f-0e7d3f52a6de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:29.459314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-63b8a469-4e56-4fd3-9a6f-0e7d3f52a6de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:29.467985 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-63b8a469-4e56-4fd3-9a6f-0e7d3f52a6de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1237 microversion: 2.1 time: 0.096184 Sep 27 10:13:29.476049 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2197/4018] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:29 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1237 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:29.483221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e16f0ebc-2c95-4a9e-808d-76a5bd101469 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:29.484033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e16f0ebc-2c95-4a9e-808d-76a5bd101469 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:29.492816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e16f0ebc-2c95-4a9e-808d-76a5bd101469 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:29.495817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e16f0ebc-2c95-4a9e-808d-76a5bd101469 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:29.497157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e16f0ebc-2c95-4a9e-808d-76a5bd101469 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:29.498246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e16f0ebc-2c95-4a9e-808d-76a5bd101469 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:29.503655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e16f0ebc-2c95-4a9e-808d-76a5bd101469 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:29.504292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e16f0ebc-2c95-4a9e-808d-76a5bd101469 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:29.855966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e16f0ebc-2c95-4a9e-808d-76a5bd101469 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:29.873098 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e16f0ebc-2c95-4a9e-808d-76a5bd101469 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.390117 Sep 27 10:13:29.874350 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2198/4019] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:29 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:30.298125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:30.298757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:30.314657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:30.315160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:30.315601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:30.316321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:30.316810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:30.321770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:30.698591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:30.712154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:30.713754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:30.715129 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1cc0725a-c545-4aef-956e-b069259d2934 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.421732 Sep 27 10:13:30.715928 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2199/4020] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:30 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:13:30.732162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0f80ce49-b697-4bc9-a9ed-bb0cbf8408aa tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:30.733106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0f80ce49-b697-4bc9-a9ed-bb0cbf8408aa tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:30.748875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f80ce49-b697-4bc9-a9ed-bb0cbf8408aa tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:30.749628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f80ce49-b697-4bc9-a9ed-bb0cbf8408aa tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:30.752956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f80ce49-b697-4bc9-a9ed-bb0cbf8408aa tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:30.753892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f80ce49-b697-4bc9-a9ed-bb0cbf8408aa tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:30.756944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f80ce49-b697-4bc9-a9ed-bb0cbf8408aa tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:30.758545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f80ce49-b697-4bc9-a9ed-bb0cbf8408aa tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:30.909419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0f80ce49-b697-4bc9-a9ed-bb0cbf8408aa tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:30.926084 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0f80ce49-b697-4bc9-a9ed-bb0cbf8408aa tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1289 microversion: 2.1 time: 0.208593 Sep 27 10:13:30.935976 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2200/4021] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:30 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1289 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:30.935976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f1a1a76-728b-4850-a90f-75840db7a14d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:30.935976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7f1a1a76-728b-4850-a90f-75840db7a14d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:30.954373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1a1a76-728b-4850-a90f-75840db7a14d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:30.954901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1a1a76-728b-4850-a90f-75840db7a14d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:30.955395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1a1a76-728b-4850-a90f-75840db7a14d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:30.956391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1a1a76-728b-4850-a90f-75840db7a14d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:30.957586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1a1a76-728b-4850-a90f-75840db7a14d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:30.958253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f1a1a76-728b-4850-a90f-75840db7a14d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:31.273431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7f1a1a76-728b-4850-a90f-75840db7a14d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:31.297821 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7f1a1a76-728b-4850-a90f-75840db7a14d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.370025 Sep 27 10:13:31.299850 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2201/4022] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:30 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:31.397082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-46b4579a-65e9-4374-b433-b2ec09d2ebbb req-c003340d-67a1-41d2-b9f7-c1e72a0c41d1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-plugged", "status": "completed", "tag": "db1377f2-7ffe-4edd-befc-8a1ac1aba3ea"}, {"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-plugged", "status": "completed", "tag": "0edf08ff-1fc9-48d8-be71-8d8e715e7c93"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:31.413074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-46b4579a-65e9-4374-b433-b2ec09d2ebbb req-c003340d-67a1-41d2-b9f7-c1e72a0c41d1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:31.413875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-46b4579a-65e9-4374-b433-b2ec09d2ebbb req-c003340d-67a1-41d2-b9f7-c1e72a0c41d1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:31.414201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-46b4579a-65e9-4374-b433-b2ec09d2ebbb req-c003340d-67a1-41d2-b9f7-c1e72a0c41d1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:31.427801 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-46b4579a-65e9-4374-b433-b2ec09d2ebbb req-c003340d-67a1-41d2-b9f7-c1e72a0c41d1 service nova] Creating event network-vif-plugged:db1377f2-7ffe-4edd-befc-8a1ac1aba3ea for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:13:31.428687 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-46b4579a-65e9-4374-b433-b2ec09d2ebbb req-c003340d-67a1-41d2-b9f7-c1e72a0c41d1 service nova] Creating event network-vif-plugged:0edf08ff-1fc9-48d8-be71-8d8e715e7c93 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:13:31.437183 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-46b4579a-65e9-4374-b433-b2ec09d2ebbb req-c003340d-67a1-41d2-b9f7-c1e72a0c41d1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.044966 Sep 27 10:13:31.444314 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2202/4023] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:31 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:31.736110 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8d3c6223-8289-4d1e-b720-a761e78ac6c0 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] 10.222.0.26 "POST /compute/v2.1/os-floating-ips" status: 200 len: 142 microversion: 2.1 time: 3.429230 Sep 27 10:13:31.738556 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1822/4024] 10.222.0.26 () {60 vars in 1170 bytes} [Tue Sep 27 10:13:28 2022] POST /compute/v2.1/os-floating-ips => generated 142 bytes in 3431 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:31.750286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:31.751009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:31.751133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8eb2c1cb-57e7-41e9-a095-078fbfef17e3 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] Calling method '.version_select at 0x7f7f6c1a7160>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:31.767869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:31.768966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:31.770263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:31.771164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:31.771781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:31.772547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:32.409311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:32.425487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:32.427257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:32.428747 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c5840de7-ae95-4cad-86b1-91d1935933b4 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.688365 Sep 27 10:13:32.451863 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2203/4025] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:31 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:32.459209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f94a7e14-87a5-4069-afad-c04d8ceb3bb2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:32.459951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f94a7e14-87a5-4069-afad-c04d8ceb3bb2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:32.467324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94a7e14-87a5-4069-afad-c04d8ceb3bb2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:32.467829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94a7e14-87a5-4069-afad-c04d8ceb3bb2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:32.468324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94a7e14-87a5-4069-afad-c04d8ceb3bb2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:32.469311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94a7e14-87a5-4069-afad-c04d8ceb3bb2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:32.469939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94a7e14-87a5-4069-afad-c04d8ceb3bb2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:32.470561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94a7e14-87a5-4069-afad-c04d8ceb3bb2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:32.592448 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8eb2c1cb-57e7-41e9-a095-078fbfef17e3 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] 10.222.0.26 "GET /compute/v2.1/os-floating-ips/c8059d41-5e9c-4bc5-abe5-dde953f152bc" status: 200 len: 142 microversion: 2.1 time: 0.844742 Sep 27 10:13:32.595212 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1823/4026] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:13:31 2022] GET /compute/v2.1/os-floating-ips/c8059d41-5e9c-4bc5-abe5-dde953f152bc => generated 142 bytes in 849 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:32.608074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bce2b5c3-7c6a-49dd-93d3-b94b83f9ba07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:32.613275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bce2b5c3-7c6a-49dd-93d3-b94b83f9ba07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:32.620108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f94a7e14-87a5-4069-afad-c04d8ceb3bb2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:32.628992 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f94a7e14-87a5-4069-afad-c04d8ceb3bb2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1295 microversion: 2.1 time: 0.175631 Sep 27 10:13:32.635312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce2b5c3-7c6a-49dd-93d3-b94b83f9ba07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:32.636760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce2b5c3-7c6a-49dd-93d3-b94b83f9ba07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:32.637584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce2b5c3-7c6a-49dd-93d3-b94b83f9ba07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:32.638490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce2b5c3-7c6a-49dd-93d3-b94b83f9ba07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:32.640249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce2b5c3-7c6a-49dd-93d3-b94b83f9ba07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:32.640896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bce2b5c3-7c6a-49dd-93d3-b94b83f9ba07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:32.648836 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2204/4027] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:32 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1295 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:32.657000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c1b8e4ce-84f8-4ad4-adfa-e6694e5d0e94 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] Calling method '.version_select at 0x7f7f636c49d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:33.016324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bce2b5c3-7c6a-49dd-93d3-b94b83f9ba07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:33.027512 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bce2b5c3-7c6a-49dd-93d3-b94b83f9ba07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.428352 Sep 27 10:13:33.028156 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1824/4028] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:32 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:33.463429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:33.464514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:33.483631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:33.484833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:33.485533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:33.486505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:33.487169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:33.487818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:34.040919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:34.066541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:34.072391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:34.073666 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0de0c457-12d3-4007-860d-d5105206b009 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1875 microversion: 2.32 time: 0.615757 Sep 27 10:13:34.081582 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1825/4029] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:33 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1875 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:34.089585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a151de89-5c84-4e4a-a4ce-8723ed1b56f1 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:34.090220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a151de89-5c84-4e4a-a4ce-8723ed1b56f1 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:34.097483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a151de89-5c84-4e4a-a4ce-8723ed1b56f1 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:34.098932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a151de89-5c84-4e4a-a4ce-8723ed1b56f1 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:34.099689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a151de89-5c84-4e4a-a4ce-8723ed1b56f1 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:34.101893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a151de89-5c84-4e4a-a4ce-8723ed1b56f1 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:34.102543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a151de89-5c84-4e4a-a4ce-8723ed1b56f1 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:34.103123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a151de89-5c84-4e4a-a4ce-8723ed1b56f1 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:34.600751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a151de89-5c84-4e4a-a4ce-8723ed1b56f1 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:34.610839 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a151de89-5c84-4e4a-a4ce-8723ed1b56f1 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1363 microversion: 2.1 time: 0.528029 Sep 27 10:13:34.614283 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1826/4030] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:34 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1363 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:34.632576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-462ed445-b9d3-47e8-88dd-e6462b9cf7fb tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:34.633605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-462ed445-b9d3-47e8-88dd-e6462b9cf7fb tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:34.641123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462ed445-b9d3-47e8-88dd-e6462b9cf7fb tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:34.641837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462ed445-b9d3-47e8-88dd-e6462b9cf7fb tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:34.642612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462ed445-b9d3-47e8-88dd-e6462b9cf7fb tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:34.643663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462ed445-b9d3-47e8-88dd-e6462b9cf7fb tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:34.644304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462ed445-b9d3-47e8-88dd-e6462b9cf7fb tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:34.644935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-462ed445-b9d3-47e8-88dd-e6462b9cf7fb tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:34.895356 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c1b8e4ce-84f8-4ad4-adfa-e6694e5d0e94 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/c8059d41-5e9c-4bc5-abe5-dde953f152bc" status: 202 len: 0 microversion: 2.1 time: 2.243326 Sep 27 10:13:34.896204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-462ed445-b9d3-47e8-88dd-e6462b9cf7fb tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:34.896698 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2205/4031] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:13:32 2022] DELETE /compute/v2.1/os-floating-ips/c8059d41-5e9c-4bc5-abe5-dde953f152bc => generated 0 bytes in 2247 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:13:34.906655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-124287d9-2e57-473e-8f8e-32d2aa027bc6 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] Calling method '.version_select at 0x7f7f636c4550>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:34.909094 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-462ed445-b9d3-47e8-88dd-e6462b9cf7fb tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.293386 Sep 27 10:13:34.925930 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1827/4032] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:34 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:35.096796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-3806c819-6f4b-4c64-b2e3-4d42dd0c5d62 req-2cb3c017-4d47-440b-9faa-29712ed6ccfd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "49f9e4be-7594-4603-8f69-8765513f8b89", "tag": "8c35173c-f9f3-41ad-a61d-50ef6ca5f054"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:35.117935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3806c819-6f4b-4c64-b2e3-4d42dd0c5d62 req-2cb3c017-4d47-440b-9faa-29712ed6ccfd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:35.118648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3806c819-6f4b-4c64-b2e3-4d42dd0c5d62 req-2cb3c017-4d47-440b-9faa-29712ed6ccfd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:35.119432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3806c819-6f4b-4c64-b2e3-4d42dd0c5d62 req-2cb3c017-4d47-440b-9faa-29712ed6ccfd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:35.138342 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-3806c819-6f4b-4c64-b2e3-4d42dd0c5d62 req-2cb3c017-4d47-440b-9faa-29712ed6ccfd service nova] Creating event network-changed:8c35173c-f9f3-41ad-a61d-50ef6ca5f054 for instance 49f9e4be-7594-4603-8f69-8765513f8b89 on nch1-master-1 Sep 27 10:13:35.145558 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-3806c819-6f4b-4c64-b2e3-4d42dd0c5d62 req-2cb3c017-4d47-440b-9faa-29712ed6ccfd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054129 Sep 27 10:13:35.146781 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1828/4033] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:35.169288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:35.170493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:35.183212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:35.183662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:35.184204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:35.184956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:35.185595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:35.186013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:35.268627 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-124287d9-2e57-473e-8f8e-32d2aa027bc6 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] 10.222.0.26 "GET /compute/v2.1/os-floating-ip-pools" status: 200 len: 43 microversion: 2.1 time: 0.366303 Sep 27 10:13:35.269772 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2206/4034] 10.222.0.26 () {58 vars in 1164 bytes} [Tue Sep 27 10:13:34 2022] GET /compute/v2.1/os-floating-ip-pools => generated 43 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:13:35.281396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b4010bc8-04b7-46b6-a545-f5f747697398 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] Calling method '.version_select at 0x7f7f63763310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:35.499344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:35.510924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:35.512393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:35.513751 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-92922835-9570-4ee2-b49a-5f506b6d9a6f tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2828 microversion: 2.32 time: 0.365430 Sep 27 10:13:35.514303 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1829/4035] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:13:35 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2828 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:13:35.542594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0801672b-1436-436c-8e08-849d88d7cf12 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:35.543372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0801672b-1436-436c-8e08-849d88d7cf12 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:35.550580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0801672b-1436-436c-8e08-849d88d7cf12 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:35.551348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0801672b-1436-436c-8e08-849d88d7cf12 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:35.552253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0801672b-1436-436c-8e08-849d88d7cf12 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:35.553277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0801672b-1436-436c-8e08-849d88d7cf12 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:35.554020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0801672b-1436-436c-8e08-849d88d7cf12 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:35.554979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0801672b-1436-436c-8e08-849d88d7cf12 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:35.723353 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0801672b-1436-436c-8e08-849d88d7cf12 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd/os-interface" status: 200 len: 1831 microversion: 2.32 time: 0.186335 Sep 27 10:13:35.724703 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1830/4036] 10.222.0.26 () {60 vars in 1316 bytes} [Tue Sep 27 10:13:35 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd/os-interface => generated 1831 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:13:35.731744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b24ef42e-a30d-4f6a-9ea3-9ede66b1f115 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:35.733401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b24ef42e-a30d-4f6a-9ea3-9ede66b1f115 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:35.740952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b24ef42e-a30d-4f6a-9ea3-9ede66b1f115 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:35.742527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b24ef42e-a30d-4f6a-9ea3-9ede66b1f115 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:35.742981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b24ef42e-a30d-4f6a-9ea3-9ede66b1f115 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:35.743854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b24ef42e-a30d-4f6a-9ea3-9ede66b1f115 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:35.744322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b24ef42e-a30d-4f6a-9ea3-9ede66b1f115 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:35.744798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b24ef42e-a30d-4f6a-9ea3-9ede66b1f115 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:36.229097 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b4010bc8-04b7-46b6-a545-f5f747697398 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] 10.222.0.26 "GET /compute/v2.1/os-floating-ips" status: 200 len: 398 microversion: 2.1 time: 0.951295 Sep 27 10:13:36.229785 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2207/4037] 10.222.0.26 () {58 vars in 1149 bytes} [Tue Sep 27 10:13:35 2022] GET /compute/v2.1/os-floating-ips => generated 398 bytes in 953 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:13:36.234887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a7c8e7ea-ecfc-47fa-a26f-b2977ecfa386 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:36.235577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a7c8e7ea-ecfc-47fa-a26f-b2977ecfa386 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:36.244789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c8e7ea-ecfc-47fa-a26f-b2977ecfa386 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:36.246351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c8e7ea-ecfc-47fa-a26f-b2977ecfa386 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:36.247007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c8e7ea-ecfc-47fa-a26f-b2977ecfa386 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:36.247792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c8e7ea-ecfc-47fa-a26f-b2977ecfa386 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:36.248323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c8e7ea-ecfc-47fa-a26f-b2977ecfa386 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:36.248848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c8e7ea-ecfc-47fa-a26f-b2977ecfa386 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:36.354383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b24ef42e-a30d-4f6a-9ea3-9ede66b1f115 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:36.362722 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b24ef42e-a30d-4f6a-9ea3-9ede66b1f115 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1363 microversion: 2.1 time: 0.636028 Sep 27 10:13:36.378336 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1831/4038] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:35 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1363 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:36.378336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1f0c230e-58ac-4ecc-ae98-b16d5b043410 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] Calling method '.version_select at 0x7f7f63faf0d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:36.604613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a7c8e7ea-ecfc-47fa-a26f-b2977ecfa386 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:36.619607 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a7c8e7ea-ecfc-47fa-a26f-b2977ecfa386 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.388486 Sep 27 10:13:36.625749 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2208/4039] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:36 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:37.395934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ce985d07-2108-4932-b649-9aeac6ab157e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:37.399505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ce985d07-2108-4932-b649-9aeac6ab157e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:37.421234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ce985d07-2108-4932-b649-9aeac6ab157e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:37.423947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ce985d07-2108-4932-b649-9aeac6ab157e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:37.426803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ce985d07-2108-4932-b649-9aeac6ab157e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:37.427905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ce985d07-2108-4932-b649-9aeac6ab157e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:37.428743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ce985d07-2108-4932-b649-9aeac6ab157e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:37.429774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ce985d07-2108-4932-b649-9aeac6ab157e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:37.901700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ce985d07-2108-4932-b649-9aeac6ab157e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:37.912074 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ce985d07-2108-4932-b649-9aeac6ab157e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1363 microversion: 2.1 time: 0.530522 Sep 27 10:13:37.922552 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2209/4040] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:37 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1363 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:13:37.927927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6ec0e168-9364-41fc-9a87-e251f8b8a4b7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:37.929117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6ec0e168-9364-41fc-9a87-e251f8b8a4b7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:37.935823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec0e168-9364-41fc-9a87-e251f8b8a4b7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:37.936550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec0e168-9364-41fc-9a87-e251f8b8a4b7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:37.937162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec0e168-9364-41fc-9a87-e251f8b8a4b7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:37.938038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec0e168-9364-41fc-9a87-e251f8b8a4b7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:37.938504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec0e168-9364-41fc-9a87-e251f8b8a4b7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:37.938932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ec0e168-9364-41fc-9a87-e251f8b8a4b7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:38.323523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6ec0e168-9364-41fc-9a87-e251f8b8a4b7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:38.328672 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1f0c230e-58ac-4ecc-ae98-b16d5b043410 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/c241f21d-d3ed-4f35-9714-7050800ce0c7" status: 202 len: 0 microversion: 2.1 time: 1.963847 Sep 27 10:13:38.329289 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1832/4041] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:13:36 2022] DELETE /compute/v2.1/os-floating-ips/c241f21d-d3ed-4f35-9714-7050800ce0c7 => generated 0 bytes in 1965 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:13:38.338762 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6ec0e168-9364-41fc-9a87-e251f8b8a4b7 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1525 microversion: 2.1 time: 0.414676 Sep 27 10:13:38.339365 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2210/4042] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:37 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1525 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:38.342841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-c49e56fc-e417-4d0c-9db3-72749df24f9b req-a0f09e44-5b3f-44fc-a12e-a99447159e98 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "tag": "10f31ce3-e11c-4b44-bb7f-173c2919d9b7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:38.358916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af3d8856-5587-474d-8f15-0f142ad28953 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] Calling method '.version_select at 0x7f7f629363a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:38.368878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c49e56fc-e417-4d0c-9db3-72749df24f9b req-a0f09e44-5b3f-44fc-a12e-a99447159e98 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:38.371471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c49e56fc-e417-4d0c-9db3-72749df24f9b req-a0f09e44-5b3f-44fc-a12e-a99447159e98 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:38.376016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c49e56fc-e417-4d0c-9db3-72749df24f9b req-a0f09e44-5b3f-44fc-a12e-a99447159e98 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:38.405734 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c49e56fc-e417-4d0c-9db3-72749df24f9b req-a0f09e44-5b3f-44fc-a12e-a99447159e98 service nova] Creating event network-changed:10f31ce3-e11c-4b44-bb7f-173c2919d9b7 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:13:38.412794 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-c49e56fc-e417-4d0c-9db3-72749df24f9b req-a0f09e44-5b3f-44fc-a12e-a99447159e98 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082031 Sep 27 10:13:38.413966 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1833/4043] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:38.826770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:38.940934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9c53c521-0701-4a0a-900c-6d90a76eda27 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:38.941712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9c53c521-0701-4a0a-900c-6d90a76eda27 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:38.972827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c53c521-0701-4a0a-900c-6d90a76eda27 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:38.973334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c53c521-0701-4a0a-900c-6d90a76eda27 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:38.973808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c53c521-0701-4a0a-900c-6d90a76eda27 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:38.974597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c53c521-0701-4a0a-900c-6d90a76eda27 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:38.975153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c53c521-0701-4a0a-900c-6d90a76eda27 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:38.975585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c53c521-0701-4a0a-900c-6d90a76eda27 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:39.316709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9c53c521-0701-4a0a-900c-6d90a76eda27 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:39.331563 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9c53c521-0701-4a0a-900c-6d90a76eda27 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1363 microversion: 2.1 time: 0.394931 Sep 27 10:13:39.333843 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1834/4044] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:38 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1363 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:13:39.362012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-be524755-5418-457d-a39c-e451be44d1cc tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:39.364366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-be524755-5418-457d-a39c-e451be44d1cc tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:39.379405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be524755-5418-457d-a39c-e451be44d1cc tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:39.381518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be524755-5418-457d-a39c-e451be44d1cc tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:39.382199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be524755-5418-457d-a39c-e451be44d1cc tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:39.383424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be524755-5418-457d-a39c-e451be44d1cc tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:39.385384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be524755-5418-457d-a39c-e451be44d1cc tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:39.387730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be524755-5418-457d-a39c-e451be44d1cc tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:39.469876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=12 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:40.405701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-be524755-5418-457d-a39c-e451be44d1cc tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:40.420368 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-be524755-5418-457d-a39c-e451be44d1cc tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1505 microversion: 2.1 time: 1.062352 Sep 27 10:13:40.421274 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1835/4045] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:39 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1505 bytes in 1064 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:40.438083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4dd86d39-c167-4a4f-8b4b-0819124772e8 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:40.439003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4dd86d39-c167-4a4f-8b4b-0819124772e8 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:40.447549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd86d39-c167-4a4f-8b4b-0819124772e8 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:40.448358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd86d39-c167-4a4f-8b4b-0819124772e8 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:40.448917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd86d39-c167-4a4f-8b4b-0819124772e8 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:40.450752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd86d39-c167-4a4f-8b4b-0819124772e8 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:40.451541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd86d39-c167-4a4f-8b4b-0819124772e8 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:40.452438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd86d39-c167-4a4f-8b4b-0819124772e8 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:40.628416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4dd86d39-c167-4a4f-8b4b-0819124772e8 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:40.636528 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4dd86d39-c167-4a4f-8b4b-0819124772e8 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1363 microversion: 2.1 time: 0.213844 Sep 27 10:13:40.637335 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1836/4046] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:40 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1363 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:40.644201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-860745ac-2081-4f30-b299-efc96158b18c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Action: 'create', calling method: .version_select at 0x7f7f6277ec10>, body: {"security_group": {"name": "tempest-ServerRescueTestJSONUnderV235-securitygroup-613346231", "description": "tempest-description-1006871651"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:40.863503 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-860745ac-2081-4f30-b299-efc96158b18c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "POST /compute/v2.1/os-security-groups" status: 200 len: 250 microversion: 2.1 time: 0.224039 Sep 27 10:13:40.866486 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1837/4047] 10.222.0.26 () {60 vars in 1180 bytes} [Tue Sep 27 10:13:40 2022] POST /compute/v2.1/os-security-groups => generated 250 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:40.882879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-530ec34e-12d8-4d64-8881-72b6cca6f037 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-ServerRescueTestJSONUnderV235-securitygroup-613346231"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:40.883756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-530ec34e-12d8-4d64-8881-72b6cca6f037 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:40.890312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-530ec34e-12d8-4d64-8881-72b6cca6f037 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:40.890822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-530ec34e-12d8-4d64-8881-72b6cca6f037 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:40.891267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-530ec34e-12d8-4d64-8881-72b6cca6f037 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:40.891996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-530ec34e-12d8-4d64-8881-72b6cca6f037 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:40.892806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-530ec34e-12d8-4d64-8881-72b6cca6f037 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:40.893465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-530ec34e-12d8-4d64-8881-72b6cca6f037 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:41.110207 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.network.security_group_api [None req-530ec34e-12d8-4d64-8881-72b6cca6f037 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Adding security group 30bf3323-30cf-45ba-8587-ed0b3621f2b8 to port 1871cdde-9d60-4d78-b0d8-e7defafdce3e Sep 27 10:13:41.760939 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af3d8856-5587-474d-8f15-0f142ad28953 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/e77722dd-e2de-4bcd-bfeb-447a2427a6c1" status: 202 len: 0 microversion: 2.1 time: 3.420177 Sep 27 10:13:41.762193 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2211/4048] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:13:38 2022] DELETE /compute/v2.1/os-floating-ips/e77722dd-e2de-4bcd-bfeb-447a2427a6c1 => generated 0 bytes in 3422 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:13:41.768859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0fadf843-5ab3-43a3-af39-0a30bd73bf2c req-e2adcf4b-5707-4982-ab45-173fc73a5dcd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "49f9e4be-7594-4603-8f69-8765513f8b89", "name": "network-vif-plugged", "status": "completed", "tag": "8c35173c-f9f3-41ad-a61d-50ef6ca5f054"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:41.786240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0fadf843-5ab3-43a3-af39-0a30bd73bf2c req-e2adcf4b-5707-4982-ab45-173fc73a5dcd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:41.787173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0fadf843-5ab3-43a3-af39-0a30bd73bf2c req-e2adcf4b-5707-4982-ab45-173fc73a5dcd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:41.787924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0fadf843-5ab3-43a3-af39-0a30bd73bf2c req-e2adcf4b-5707-4982-ab45-173fc73a5dcd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:41.837116 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0fadf843-5ab3-43a3-af39-0a30bd73bf2c req-e2adcf4b-5707-4982-ab45-173fc73a5dcd service nova] Creating event network-vif-plugged:8c35173c-f9f3-41ad-a61d-50ef6ca5f054 for instance 49f9e4be-7594-4603-8f69-8765513f8b89 on nch1-master-1 Sep 27 10:13:41.849792 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0fadf843-5ab3-43a3-af39-0a30bd73bf2c req-e2adcf4b-5707-4982-ab45-173fc73a5dcd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084320 Sep 27 10:13:41.850881 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2212/4049] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:41.853869 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-530ec34e-12d8-4d64-8881-72b6cca6f037 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "POST /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712/action" status: 202 len: 0 microversion: 2.1 time: 0.977913 Sep 27 10:13:41.855573 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1838/4050] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:13:40 2022] POST /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712/action => generated 0 bytes in 980 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:13:41.863534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-371afc45-0a2c-43ca-98ba-353a514b0a59 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:41.864507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-371afc45-0a2c-43ca-98ba-353a514b0a59 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:41.872378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-52854656-38e1-4f2b-85ac-18455902d1d4 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] Calling method '.version_select at 0x7f7f62def3a0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:41.873141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-371afc45-0a2c-43ca-98ba-353a514b0a59 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:41.873890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-371afc45-0a2c-43ca-98ba-353a514b0a59 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:41.885269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-371afc45-0a2c-43ca-98ba-353a514b0a59 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:41.888947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-371afc45-0a2c-43ca-98ba-353a514b0a59 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:41.889844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-371afc45-0a2c-43ca-98ba-353a514b0a59 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:41.891355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-371afc45-0a2c-43ca-98ba-353a514b0a59 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:42.432041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-371afc45-0a2c-43ca-98ba-353a514b0a59 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:42.435682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:42.449855 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-371afc45-0a2c-43ca-98ba-353a514b0a59 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1363 microversion: 2.1 time: 0.597265 Sep 27 10:13:42.457124 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2213/4051] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:41 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1363 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:42.474946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f208459-4a41-4690-8a97-7570341a9224 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-ServerRescueTestJSONUnderV235-securitygroup-613346231"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:42.475758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f208459-4a41-4690-8a97-7570341a9224 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:42.484522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f208459-4a41-4690-8a97-7570341a9224 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:42.485311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f208459-4a41-4690-8a97-7570341a9224 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:42.485948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f208459-4a41-4690-8a97-7570341a9224 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:42.486945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f208459-4a41-4690-8a97-7570341a9224 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:42.487562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f208459-4a41-4690-8a97-7570341a9224 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:42.488224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f208459-4a41-4690-8a97-7570341a9224 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:43.245758 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.network.security_group_api [None req-6f208459-4a41-4690-8a97-7570341a9224 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Removing security group 30bf3323-30cf-45ba-8587-ed0b3621f2b8 from port 1871cdde-9d60-4d78-b0d8-e7defafdce3e Sep 27 10:13:43.648293 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-52854656-38e1-4f2b-85ac-18455902d1d4 tempest-FloatingIPDetailsTestJSON-1554985925 tempest-FloatingIPDetailsTestJSON-1554985925-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/7ce9cd6a-a871-45a1-b5ec-f16509091f1b" status: 202 len: 0 microversion: 2.1 time: 1.791554 Sep 27 10:13:43.649166 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1839/4052] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:13:41 2022] DELETE /compute/v2.1/os-floating-ips/7ce9cd6a-a871-45a1-b5ec-f16509091f1b => generated 0 bytes in 1793 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:13:43.657880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a397750e-bc87-4df6-b4ad-e12589eb105b req-b5240b46-c2f2-48a2-8610-bb0e5f446287 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ce015738-c16b-4f29-8cd0-b96245268712", "tag": "1871cdde-9d60-4d78-b0d8-e7defafdce3e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:43.686768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a397750e-bc87-4df6-b4ad-e12589eb105b req-b5240b46-c2f2-48a2-8610-bb0e5f446287 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:43.688622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a397750e-bc87-4df6-b4ad-e12589eb105b req-b5240b46-c2f2-48a2-8610-bb0e5f446287 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:43.689312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a397750e-bc87-4df6-b4ad-e12589eb105b req-b5240b46-c2f2-48a2-8610-bb0e5f446287 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:43.740129 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a397750e-bc87-4df6-b4ad-e12589eb105b req-b5240b46-c2f2-48a2-8610-bb0e5f446287 service nova] Creating event network-changed:1871cdde-9d60-4d78-b0d8-e7defafdce3e for instance ce015738-c16b-4f29-8cd0-b96245268712 on nch2-master-1 Sep 27 10:13:43.749895 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a397750e-bc87-4df6-b4ad-e12589eb105b req-b5240b46-c2f2-48a2-8610-bb0e5f446287 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.096353 Sep 27 10:13:43.753184 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1840/4053] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:43.758698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1ac564c2-49d3-498a-abf5-54756f957b5f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:43.759408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1ac564c2-49d3-498a-abf5-54756f957b5f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:43.778051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ac564c2-49d3-498a-abf5-54756f957b5f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:43.778561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ac564c2-49d3-498a-abf5-54756f957b5f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:43.779001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ac564c2-49d3-498a-abf5-54756f957b5f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:43.779722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ac564c2-49d3-498a-abf5-54756f957b5f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:43.780179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ac564c2-49d3-498a-abf5-54756f957b5f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:43.781010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ac564c2-49d3-498a-abf5-54756f957b5f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:44.163409 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f208459-4a41-4690-8a97-7570341a9224 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "POST /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712/action" status: 202 len: 0 microversion: 2.1 time: 1.704609 Sep 27 10:13:44.164987 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2214/4054] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:13:42 2022] POST /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712/action => generated 0 bytes in 1707 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:13:44.298882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1ac564c2-49d3-498a-abf5-54756f957b5f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:44.315596 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1ac564c2-49d3-498a-abf5-54756f957b5f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1549 microversion: 2.1 time: 0.561087 Sep 27 10:13:44.323581 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1841/4055] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:43 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1549 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:44.335981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-233a5279-bed8-4ec4-aa2e-b9c004cfce33 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Action: 'create', calling method: .version_select at 0x7f7f62eaf820>, body: {"pool": null} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:44.674185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-tempest.common.compute-instance-1884155628", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "min_count": 2, "networks": [{"uuid": "3168a5ae-5200-46e4-9fa9-362c59caaed1"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:44.751282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] validate_networks() for [('3168a5ae-5200-46e4-9fa9-362c59caaed1', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:13:45.783503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:13:45.784132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:13:45.785208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:13:45.785808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:13:45.786239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:13:45.802732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:45.803549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:45.805526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:45.807085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:45.809066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:45.810372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:45.860316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Getting quotas for project 326c35c0bc52463e95a56ba3371a276a. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:13:45.874358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Getting quotas for user a5e050c7389844a78199faef5a0c9da4 and project 326c35c0bc52463e95a56ba3371a276a. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:13:45.904398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Going to run 2 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:13:45.908057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: e60a1d1f-7e9a-44e8-88a3-fae7a0133c18] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:13:45.976314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=e60a1d1f-7e9a-44e8-88a3-fae7a0133c18,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:13:46.025517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:46.074837 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-71f07776-4610-4462-abd1-10ed5baed14f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.908376 Sep 27 10:13:46.076040 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2215/4056] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:13:44 2022] POST /compute/v2.1/servers => generated 380 bytes in 1910 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:13:46.098015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-58151739-f8a5-4fc2-bfd8-cf9416276580 req-1b0d88b6-3955-46ae-96c1-e6a2a9b46fc4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ce015738-c16b-4f29-8cd0-b96245268712", "tag": "1871cdde-9d60-4d78-b0d8-e7defafdce3e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:46.125294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-58151739-f8a5-4fc2-bfd8-cf9416276580 req-1b0d88b6-3955-46ae-96c1-e6a2a9b46fc4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:46.129178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-58151739-f8a5-4fc2-bfd8-cf9416276580 req-1b0d88b6-3955-46ae-96c1-e6a2a9b46fc4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:46.129786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-58151739-f8a5-4fc2-bfd8-cf9416276580 req-1b0d88b6-3955-46ae-96c1-e6a2a9b46fc4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:46.174081 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-58151739-f8a5-4fc2-bfd8-cf9416276580 req-1b0d88b6-3955-46ae-96c1-e6a2a9b46fc4 service nova] Creating event network-changed:1871cdde-9d60-4d78-b0d8-e7defafdce3e for instance ce015738-c16b-4f29-8cd0-b96245268712 on nch2-master-1 Sep 27 10:13:46.188476 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-58151739-f8a5-4fc2-bfd8-cf9416276580 req-1b0d88b6-3955-46ae-96c1-e6a2a9b46fc4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.110878 Sep 27 10:13:46.189320 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2216/4057] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:46.208642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d4c6367-6b87-45e0-9630-d659c4e0eb79 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:46.210772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d4c6367-6b87-45e0-9630-d659c4e0eb79 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:46.228401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4c6367-6b87-45e0-9630-d659c4e0eb79 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:46.229004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4c6367-6b87-45e0-9630-d659c4e0eb79 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:46.229542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4c6367-6b87-45e0-9630-d659c4e0eb79 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:46.230458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4c6367-6b87-45e0-9630-d659c4e0eb79 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:46.231351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4c6367-6b87-45e0-9630-d659c4e0eb79 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:46.232251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4c6367-6b87-45e0-9630-d659c4e0eb79 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:46.366984 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d4c6367-6b87-45e0-9630-d659c4e0eb79 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.168889 Sep 27 10:13:46.368083 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2217/4058] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:13:46 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/os-interface => generated 285 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:46.384957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c21e06eb-67a0-41b1-9136-44934c4d0629 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:46.399945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c21e06eb-67a0-41b1-9136-44934c4d0629 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:46.401030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c21e06eb-67a0-41b1-9136-44934c4d0629 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Searching by: {'deleted': False, 'project_id': '326c35c0bc52463e95a56ba3371a276a'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:13:46.463780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21e06eb-67a0-41b1-9136-44934c4d0629 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:46.465710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21e06eb-67a0-41b1-9136-44934c4d0629 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:46.466243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21e06eb-67a0-41b1-9136-44934c4d0629 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:46.468918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21e06eb-67a0-41b1-9136-44934c4d0629 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:46.469843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21e06eb-67a0-41b1-9136-44934c4d0629 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:46.471973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21e06eb-67a0-41b1-9136-44934c4d0629 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:46.567961 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c21e06eb-67a0-41b1-9136-44934c4d0629 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 677 microversion: 2.1 time: 0.198275 Sep 27 10:13:46.568853 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2218/4059] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:13:46 2022] GET /compute/v2.1/servers => generated 677 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:46.582058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9ea7510c-2c53-46a2-a3f4-0708ab7b2841 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:46.583226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ea7510c-2c53-46a2-a3f4-0708ab7b2841 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:47.336536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ea7510c-2c53-46a2-a3f4-0708ab7b2841 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:47.365477 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9ea7510c-2c53-46a2-a3f4-0708ab7b2841 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1220 microversion: 2.1 time: 0.787971 Sep 27 10:13:47.371870 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2219/4060] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:46 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1220 bytes in 795 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:48.399452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a9ee00b-c7aa-4592-9939-893e470681e1 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:48.403225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3a9ee00b-c7aa-4592-9939-893e470681e1 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:48.424292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a9ee00b-c7aa-4592-9939-893e470681e1 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:48.426014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a9ee00b-c7aa-4592-9939-893e470681e1 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:48.427253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a9ee00b-c7aa-4592-9939-893e470681e1 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:48.428997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a9ee00b-c7aa-4592-9939-893e470681e1 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:48.429819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a9ee00b-c7aa-4592-9939-893e470681e1 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:48.430657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a9ee00b-c7aa-4592-9939-893e470681e1 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:48.628651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3a9ee00b-c7aa-4592-9939-893e470681e1 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:48.641615 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3a9ee00b-c7aa-4592-9939-893e470681e1 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1272 microversion: 2.1 time: 0.248470 Sep 27 10:13:48.644104 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2220/4061] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:48 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1272 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:13:49.087063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-698c7506-eebc-4241-b252-bd7e9cf38262 req-4ce74c78-5d6a-4def-9fe4-7a3e53eb18b3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "49f9e4be-7594-4603-8f69-8765513f8b89", "tag": "8c35173c-f9f3-41ad-a61d-50ef6ca5f054"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:49.118188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-698c7506-eebc-4241-b252-bd7e9cf38262 req-4ce74c78-5d6a-4def-9fe4-7a3e53eb18b3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:49.118818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-698c7506-eebc-4241-b252-bd7e9cf38262 req-4ce74c78-5d6a-4def-9fe4-7a3e53eb18b3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:49.119598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-698c7506-eebc-4241-b252-bd7e9cf38262 req-4ce74c78-5d6a-4def-9fe4-7a3e53eb18b3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:49.139164 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-698c7506-eebc-4241-b252-bd7e9cf38262 req-4ce74c78-5d6a-4def-9fe4-7a3e53eb18b3 service nova] Creating event network-changed:8c35173c-f9f3-41ad-a61d-50ef6ca5f054 for instance 49f9e4be-7594-4603-8f69-8765513f8b89 on nch1-master-1 Sep 27 10:13:49.145518 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-698c7506-eebc-4241-b252-bd7e9cf38262 req-4ce74c78-5d6a-4def-9fe4-7a3e53eb18b3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064335 Sep 27 10:13:49.146650 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2221/4062] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:13:49.670821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f6d693a2-1212-4a8f-9e51-c1be2feb72e2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:49.671590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f6d693a2-1212-4a8f-9e51-c1be2feb72e2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:49.685471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6d693a2-1212-4a8f-9e51-c1be2feb72e2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:49.687522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6d693a2-1212-4a8f-9e51-c1be2feb72e2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:49.688993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6d693a2-1212-4a8f-9e51-c1be2feb72e2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:49.690766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6d693a2-1212-4a8f-9e51-c1be2feb72e2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:49.693058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6d693a2-1212-4a8f-9e51-c1be2feb72e2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:49.693655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6d693a2-1212-4a8f-9e51-c1be2feb72e2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:49.876386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f6d693a2-1212-4a8f-9e51-c1be2feb72e2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:49.891125 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f6d693a2-1212-4a8f-9e51-c1be2feb72e2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1278 microversion: 2.1 time: 0.227251 Sep 27 10:13:49.893772 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2222/4063] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:49 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1278 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:13:50.928722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c032347a-81ff-42b3-8e2c-0d72416730c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:50.930367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c032347a-81ff-42b3-8e2c-0d72416730c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:50.940741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c032347a-81ff-42b3-8e2c-0d72416730c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:50.943181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c032347a-81ff-42b3-8e2c-0d72416730c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:50.945190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c032347a-81ff-42b3-8e2c-0d72416730c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:50.946477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c032347a-81ff-42b3-8e2c-0d72416730c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:50.951648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c032347a-81ff-42b3-8e2c-0d72416730c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:50.953799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c032347a-81ff-42b3-8e2c-0d72416730c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:51.085900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c032347a-81ff-42b3-8e2c-0d72416730c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:51.094562 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c032347a-81ff-42b3-8e2c-0d72416730c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1278 microversion: 2.1 time: 0.178762 Sep 27 10:13:51.109629 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2223/4064] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:50 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1278 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:13:52.119661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1eafa04e-5b50-4b70-b095-552637fdc444 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:52.120306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1eafa04e-5b50-4b70-b095-552637fdc444 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:52.130990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eafa04e-5b50-4b70-b095-552637fdc444 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:52.131729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eafa04e-5b50-4b70-b095-552637fdc444 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:52.132485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eafa04e-5b50-4b70-b095-552637fdc444 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:52.133289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eafa04e-5b50-4b70-b095-552637fdc444 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:52.133753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eafa04e-5b50-4b70-b095-552637fdc444 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:52.134345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eafa04e-5b50-4b70-b095-552637fdc444 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:52.324913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1eafa04e-5b50-4b70-b095-552637fdc444 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:52.334220 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1eafa04e-5b50-4b70-b095-552637fdc444 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1320 microversion: 2.1 time: 0.219591 Sep 27 10:13:52.343400 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2224/4065] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:52 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1320 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:13:52.357467 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-233a5279-bed8-4ec4-aa2e-b9c004cfce33 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "POST /compute/v2.1/os-floating-ips" status: 200 len: 141 microversion: 2.1 time: 8.031580 Sep 27 10:13:52.366297 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1842/4066] 10.222.0.26 () {60 vars in 1170 bytes} [Tue Sep 27 10:13:44 2022] POST /compute/v2.1/os-floating-ips => generated 141 bytes in 8033 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:52.373652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Action: 'action', calling method: , body: {"addFloatingIp": {"address": "172.35.1.39"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:52.376967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:52.389740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:52.391508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:52.396418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:52.397504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:52.398230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:52.398878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:52.463374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "refresh_cache-ce015738-c16b-4f29-8cd0-b96245268712" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 10:13:52.464300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquired lock "refresh_cache-ce015738-c16b-4f29-8cd0-b96245268712" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 10:13:53.365651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-96a9040f-46e9-43ed-8a39-4c067dc66f6f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:53.369304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-96a9040f-46e9-43ed-8a39-4c067dc66f6f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:53.386390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96a9040f-46e9-43ed-8a39-4c067dc66f6f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:53.388346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96a9040f-46e9-43ed-8a39-4c067dc66f6f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:53.389432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96a9040f-46e9-43ed-8a39-4c067dc66f6f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:53.391532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96a9040f-46e9-43ed-8a39-4c067dc66f6f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:53.395067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96a9040f-46e9-43ed-8a39-4c067dc66f6f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:53.397705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96a9040f-46e9-43ed-8a39-4c067dc66f6f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:53.922428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-96a9040f-46e9-43ed-8a39-4c067dc66f6f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:53.933543 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-96a9040f-46e9-43ed-8a39-4c067dc66f6f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1320 microversion: 2.1 time: 0.579004 Sep 27 10:13:53.934581 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1843/4067] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:53 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1320 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:13:54.962386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a14299cf-eac3-4c72-ad8e-63275b347c83 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:54.963017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a14299cf-eac3-4c72-ad8e-63275b347c83 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:54.970213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a14299cf-eac3-4c72-ad8e-63275b347c83 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:54.971112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a14299cf-eac3-4c72-ad8e-63275b347c83 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:54.971680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a14299cf-eac3-4c72-ad8e-63275b347c83 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:54.972522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a14299cf-eac3-4c72-ad8e-63275b347c83 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:54.973126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a14299cf-eac3-4c72-ad8e-63275b347c83 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:54.973691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a14299cf-eac3-4c72-ad8e-63275b347c83 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:55.557102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Building network info cache for instance {{(pid=145968) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Sep 27 10:13:55.667899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a14299cf-eac3-4c72-ad8e-63275b347c83 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:55.681100 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a14299cf-eac3-4c72-ad8e-63275b347c83 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1320 microversion: 2.1 time: 0.724394 Sep 27 10:13:55.683594 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1844/4068] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:54 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1320 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:13:55.690797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-77d369ba-0356-431a-9d1b-a9e1eec93e25 req-ffd63db7-4fcd-4d78-a84d-8af2f38468ac service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eef2d4e4-d9aa-48ca-bd73-b538541465fa", "tag": "fe94630e-b20d-425b-8f30-a2c5c4298db8"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:55.705789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-77d369ba-0356-431a-9d1b-a9e1eec93e25 req-ffd63db7-4fcd-4d78-a84d-8af2f38468ac service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:55.706549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-77d369ba-0356-431a-9d1b-a9e1eec93e25 req-ffd63db7-4fcd-4d78-a84d-8af2f38468ac service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:55.707054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-77d369ba-0356-431a-9d1b-a9e1eec93e25 req-ffd63db7-4fcd-4d78-a84d-8af2f38468ac service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:55.763993 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-77d369ba-0356-431a-9d1b-a9e1eec93e25 req-ffd63db7-4fcd-4d78-a84d-8af2f38468ac service nova] Creating event network-changed:fe94630e-b20d-425b-8f30-a2c5c4298db8 for instance eef2d4e4-d9aa-48ca-bd73-b538541465fa on nch1-master-1 Sep 27 10:13:55.774820 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-77d369ba-0356-431a-9d1b-a9e1eec93e25 req-ffd63db7-4fcd-4d78-a84d-8af2f38468ac service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089671 Sep 27 10:13:55.775742 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1845/4069] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:55.784635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-bb417881-ab61-44c0-812e-7a537b4aa065 req-df1bc7bf-f4c8-4148-a129-cfccc8a3bcdb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ce015738-c16b-4f29-8cd0-b96245268712", "tag": "1871cdde-9d60-4d78-b0d8-e7defafdce3e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:55.806977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bb417881-ab61-44c0-812e-7a537b4aa065 req-df1bc7bf-f4c8-4148-a129-cfccc8a3bcdb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:55.807522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bb417881-ab61-44c0-812e-7a537b4aa065 req-df1bc7bf-f4c8-4148-a129-cfccc8a3bcdb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:55.807895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bb417881-ab61-44c0-812e-7a537b4aa065 req-df1bc7bf-f4c8-4148-a129-cfccc8a3bcdb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:55.829875 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-bb417881-ab61-44c0-812e-7a537b4aa065 req-df1bc7bf-f4c8-4148-a129-cfccc8a3bcdb service nova] Creating event network-changed:1871cdde-9d60-4d78-b0d8-e7defafdce3e for instance ce015738-c16b-4f29-8cd0-b96245268712 on nch2-master-1 Sep 27 10:13:55.841845 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-bb417881-ab61-44c0-812e-7a537b4aa065 req-df1bc7bf-f4c8-4148-a129-cfccc8a3bcdb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063115 Sep 27 10:13:55.843592 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1846/4070] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:56.524738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:56.709470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-10b75170-78ed-47e6-ab0d-84356da1ff12 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:56.710959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-10b75170-78ed-47e6-ab0d-84356da1ff12 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:56.714274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=41 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:56.728944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b75170-78ed-47e6-ab0d-84356da1ff12 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:56.730040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b75170-78ed-47e6-ab0d-84356da1ff12 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:56.730645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b75170-78ed-47e6-ab0d-84356da1ff12 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:56.731644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b75170-78ed-47e6-ab0d-84356da1ff12 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:56.732905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b75170-78ed-47e6-ab0d-84356da1ff12 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:56.733495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b75170-78ed-47e6-ab0d-84356da1ff12 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:56.783933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:57.114171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-10b75170-78ed-47e6-ab0d-84356da1ff12 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:57.129485 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-10b75170-78ed-47e6-ab0d-84356da1ff12 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1320 microversion: 2.1 time: 0.422404 Sep 27 10:13:57.130256 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1847/4071] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:56 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1320 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:13:57.364374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:13:57.775430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Updating instance_info_cache with network_info: [{"id": "1871cdde-9d60-4d78-b0d8-e7defafdce3e", "address": "fa:16:3e:bf:44:f2", "network": {"id": "3dbae7e4-7497-4220-9492-7ccbe8a206d1", "bridge": null, "label": "tempest-ServerRescueTestJSONUnderV235-2011235172-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.35.1.39", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f23e44d8a5094104b35d0288d33f0ab6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap1871cdde-9d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=145968) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:13:57.801196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Releasing lock "refresh_cache-ce015738-c16b-4f29-8cd0-b96245268712" {{(pid=145968) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 10:13:57.803692 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-942291f3-47a5-40d9-aad7-5f306378a480 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "POST /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712/action" status: 202 len: 0 microversion: 2.1 time: 5.434680 Sep 27 10:13:57.806375 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2225/4072] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:13:52 2022] POST /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712/action => generated 0 bytes in 5439 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Sep 27 10:13:57.815141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Action: 'action', calling method: , body: {"removeFloatingIp": {"address": "172.35.1.39"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:57.852226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-08e9f815-3049-42a9-a303-ab57b90ef8f6 req-f29db438-5b7e-4e0d-8c88-79afda468461 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e60a1d1f-7e9a-44e8-88a3-fae7a0133c18", "tag": "ffd4ab48-3933-4905-98a4-38405d3efd95"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:13:57.870803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-08e9f815-3049-42a9-a303-ab57b90ef8f6 req-f29db438-5b7e-4e0d-8c88-79afda468461 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:57.871637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-08e9f815-3049-42a9-a303-ab57b90ef8f6 req-f29db438-5b7e-4e0d-8c88-79afda468461 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:57.872234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-08e9f815-3049-42a9-a303-ab57b90ef8f6 req-f29db438-5b7e-4e0d-8c88-79afda468461 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:57.898484 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-08e9f815-3049-42a9-a303-ab57b90ef8f6 req-f29db438-5b7e-4e0d-8c88-79afda468461 service nova] Creating event network-changed:ffd4ab48-3933-4905-98a4-38405d3efd95 for instance e60a1d1f-7e9a-44e8-88a3-fae7a0133c18 on nch1-master-1 Sep 27 10:13:57.924558 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-08e9f815-3049-42a9-a303-ab57b90ef8f6 req-f29db438-5b7e-4e0d-8c88-79afda468461 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077690 Sep 27 10:13:57.927129 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2226/4073] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:13:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:13:58.163917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8ad11389-cd07-4537-b91b-f3165f1140eb tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:58.165289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8ad11389-cd07-4537-b91b-f3165f1140eb tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:58.175966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad11389-cd07-4537-b91b-f3165f1140eb tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:58.176754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad11389-cd07-4537-b91b-f3165f1140eb tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:58.177387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad11389-cd07-4537-b91b-f3165f1140eb tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:58.178426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad11389-cd07-4537-b91b-f3165f1140eb tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:58.179539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad11389-cd07-4537-b91b-f3165f1140eb tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:58.180164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad11389-cd07-4537-b91b-f3165f1140eb tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:58.241443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:58.265568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:58.266137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:58.266516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:58.269187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:58.269935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:58.270728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:58.497807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8ad11389-cd07-4537-b91b-f3165f1140eb tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:58.520112 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8ad11389-cd07-4537-b91b-f3165f1140eb tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1320 microversion: 2.1 time: 0.365189 Sep 27 10:13:58.522455 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2227/4074] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:58 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1320 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:13:58.635105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "refresh_cache-ce015738-c16b-4f29-8cd0-b96245268712" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 27 10:13:58.635841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquired lock "refresh_cache-ce015738-c16b-4f29-8cd0-b96245268712" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 27 10:13:59.547991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a840114b-b11e-4678-ae34-74fab1f9b16f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:13:59.549392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a840114b-b11e-4678-ae34-74fab1f9b16f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:13:59.579015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a840114b-b11e-4678-ae34-74fab1f9b16f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:59.581379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a840114b-b11e-4678-ae34-74fab1f9b16f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:59.582027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a840114b-b11e-4678-ae34-74fab1f9b16f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:59.586010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a840114b-b11e-4678-ae34-74fab1f9b16f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:13:59.586808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a840114b-b11e-4678-ae34-74fab1f9b16f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:13:59.588484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a840114b-b11e-4678-ae34-74fab1f9b16f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:13:59.950580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a840114b-b11e-4678-ae34-74fab1f9b16f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:13:59.973937 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a840114b-b11e-4678-ae34-74fab1f9b16f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1320 microversion: 2.1 time: 0.428435 Sep 27 10:13:59.975757 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2228/4075] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:13:59 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1320 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:00.737077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Building network info cache for instance {{(pid=145969) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:1987}} Sep 27 10:14:00.748803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6f5d2d40-ceaa-4362-b596-1aff04065061 req-8d305976-a3e0-455c-aed0-e8b2419eefdc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ce015738-c16b-4f29-8cd0-b96245268712", "tag": "1871cdde-9d60-4d78-b0d8-e7defafdce3e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:00.771126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6f5d2d40-ceaa-4362-b596-1aff04065061 req-8d305976-a3e0-455c-aed0-e8b2419eefdc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:00.772056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6f5d2d40-ceaa-4362-b596-1aff04065061 req-8d305976-a3e0-455c-aed0-e8b2419eefdc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:00.776358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6f5d2d40-ceaa-4362-b596-1aff04065061 req-8d305976-a3e0-455c-aed0-e8b2419eefdc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:00.806850 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6f5d2d40-ceaa-4362-b596-1aff04065061 req-8d305976-a3e0-455c-aed0-e8b2419eefdc service nova] Creating event network-changed:1871cdde-9d60-4d78-b0d8-e7defafdce3e for instance ce015738-c16b-4f29-8cd0-b96245268712 on nch2-master-1 Sep 27 10:14:00.817657 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6f5d2d40-ceaa-4362-b596-1aff04065061 req-8d305976-a3e0-455c-aed0-e8b2419eefdc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.073355 Sep 27 10:14:00.819011 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2229/4076] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:14:01.556884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07a2ab0d-0aa5-4255-bd0f-d9a0aa977c49 tempest-AZV2TestJSON-1499224937 tempest-AZV2TestJSON-1499224937-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:01.558730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a2ab0d-0aa5-4255-bd0f-d9a0aa977c49 tempest-AZV2TestJSON-1499224937 tempest-AZV2TestJSON-1499224937-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:01.559432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a2ab0d-0aa5-4255-bd0f-d9a0aa977c49 tempest-AZV2TestJSON-1499224937 tempest-AZV2TestJSON-1499224937-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:01.560079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a2ab0d-0aa5-4255-bd0f-d9a0aa977c49 tempest-AZV2TestJSON-1499224937 tempest-AZV2TestJSON-1499224937-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:01.562066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a2ab0d-0aa5-4255-bd0f-d9a0aa977c49 tempest-AZV2TestJSON-1499224937 tempest-AZV2TestJSON-1499224937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:01.563652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a2ab0d-0aa5-4255-bd0f-d9a0aa977c49 tempest-AZV2TestJSON-1499224937 tempest-AZV2TestJSON-1499224937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:01.564964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07a2ab0d-0aa5-4255-bd0f-d9a0aa977c49 tempest-AZV2TestJSON-1499224937 tempest-AZV2TestJSON-1499224937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:01.619691 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-07a2ab0d-0aa5-4255-bd0f-d9a0aa977c49 tempest-AZV2TestJSON-1499224937 tempest-AZV2TestJSON-1499224937-project] 10.222.0.26 "GET /compute/v2.1/os-availability-zone" status: 200 len: 97 microversion: 2.1 time: 0.680391 Sep 27 10:14:01.620927 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2230/4077] 10.222.0.26 () {58 vars in 1164 bytes} [Tue Sep 27 10:14:00 2022] GET /compute/v2.1/os-availability-zone => generated 97 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:14:01.627372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f06bcb05-4983-4fa9-b7c2-d835e12b22c3 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:01.650418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f06bcb05-4983-4fa9-b7c2-d835e12b22c3 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:01.693152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06bcb05-4983-4fa9-b7c2-d835e12b22c3 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:01.693711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06bcb05-4983-4fa9-b7c2-d835e12b22c3 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:01.694188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06bcb05-4983-4fa9-b7c2-d835e12b22c3 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:01.700800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06bcb05-4983-4fa9-b7c2-d835e12b22c3 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:01.701551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06bcb05-4983-4fa9-b7c2-d835e12b22c3 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:01.702140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f06bcb05-4983-4fa9-b7c2-d835e12b22c3 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:02.157750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f06bcb05-4983-4fa9-b7c2-d835e12b22c3 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:02.170678 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f06bcb05-4983-4fa9-b7c2-d835e12b22c3 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1320 microversion: 2.1 time: 0.548153 Sep 27 10:14:02.171294 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2231/4078] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:01 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1320 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:02.398461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Updating instance_info_cache with network_info: [{"id": "1871cdde-9d60-4d78-b0d8-e7defafdce3e", "address": "fa:16:3e:bf:44:f2", "network": {"id": "3dbae7e4-7497-4220-9492-7ccbe8a206d1", "bridge": null, "label": "tempest-ServerRescueTestJSONUnderV235-2011235172-network", "subnets": [{"cidr": "172.20.0.0/28", "dns": [], "gateway": {"address": "172.20.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "172.20.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {}}], "meta": {"injected": false, "tenant_id": "f23e44d8a5094104b35d0288d33f0ab6", "mtu": 1500, "physical_network": "physnet1", "tunneled": false}}, "type": "hyperv", "details": {"connectivity": "legacy", "port_filter": false, "bound_drivers": {"0": "hyperv"}}, "devname": "tap1871cdde-9d", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=145969) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:14:02.436850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Releasing lock "refresh_cache-ce015738-c16b-4f29-8cd0-b96245268712" {{(pid=145969) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 27 10:14:02.440267 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eb05992e-92f1-42d8-96cb-c2b390cecb71 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "POST /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712/action" status: 202 len: 0 microversion: 2.1 time: 4.627617 Sep 27 10:14:02.441108 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1848/4079] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:13:57 2022] POST /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712/action => generated 0 bytes in 4631 msecs (HTTP/1.1 202) 9 headers in 369 bytes (1 switches on core 0) Sep 27 10:14:02.451479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f30beab-3874-4979-a81a-a30531cf673d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '.version_select at 0x7f7f63763d30>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:03.013891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5533720a-38f2-45af-bbd5-af7a412d1fe2 req-ee6ae65e-a4c3-4ba3-9c34-7a645c248074 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eef2d4e4-d9aa-48ca-bd73-b538541465fa", "name": "network-vif-plugged", "status": "completed", "tag": "fe94630e-b20d-425b-8f30-a2c5c4298db8"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:03.043693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5533720a-38f2-45af-bbd5-af7a412d1fe2 req-ee6ae65e-a4c3-4ba3-9c34-7a645c248074 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:03.044629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5533720a-38f2-45af-bbd5-af7a412d1fe2 req-ee6ae65e-a4c3-4ba3-9c34-7a645c248074 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:03.045579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5533720a-38f2-45af-bbd5-af7a412d1fe2 req-ee6ae65e-a4c3-4ba3-9c34-7a645c248074 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:03.063512 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5533720a-38f2-45af-bbd5-af7a412d1fe2 req-ee6ae65e-a4c3-4ba3-9c34-7a645c248074 service nova] Creating event network-vif-plugged:fe94630e-b20d-425b-8f30-a2c5c4298db8 for instance eef2d4e4-d9aa-48ca-bd73-b538541465fa on nch1-master-1 Sep 27 10:14:03.070848 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5533720a-38f2-45af-bbd5-af7a412d1fe2 req-ee6ae65e-a4c3-4ba3-9c34-7a645c248074 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061373 Sep 27 10:14:03.072380 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1849/4080] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:03.204140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b15df927-d099-44a7-ab64-dd8d7ca7351c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:03.207794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b15df927-d099-44a7-ab64-dd8d7ca7351c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:03.230741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b15df927-d099-44a7-ab64-dd8d7ca7351c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:03.231995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b15df927-d099-44a7-ab64-dd8d7ca7351c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:03.235698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b15df927-d099-44a7-ab64-dd8d7ca7351c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:03.236960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b15df927-d099-44a7-ab64-dd8d7ca7351c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:03.237456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b15df927-d099-44a7-ab64-dd8d7ca7351c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:03.237897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b15df927-d099-44a7-ab64-dd8d7ca7351c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:03.521269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b15df927-d099-44a7-ab64-dd8d7ca7351c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:03.542663 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b15df927-d099-44a7-ab64-dd8d7ca7351c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1320 microversion: 2.1 time: 0.348445 Sep 27 10:14:03.548173 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1850/4081] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:03 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1320 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:04.417959 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9f30beab-3874-4979-a81a-a30531cf673d tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "DELETE /compute/v2.1/os-floating-ips/2bf41389-36bb-41f8-99de-5bd02082abe5" status: 202 len: 0 microversion: 2.1 time: 1.972037 Sep 27 10:14:04.418648 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2232/4082] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:14:02 2022] DELETE /compute/v2.1/os-floating-ips/2bf41389-36bb-41f8-99de-5bd02082abe5 => generated 0 bytes in 1974 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:14:04.432135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-baeeef26-99af-456c-b5eb-d6d17715e5ac tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '.version_select at 0x7f7f62f341f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:04.574419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-de97b891-602a-4b16-9f4a-5906e8f8daff tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:04.575363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-de97b891-602a-4b16-9f4a-5906e8f8daff tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:04.585424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de97b891-602a-4b16-9f4a-5906e8f8daff tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:04.585424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de97b891-602a-4b16-9f4a-5906e8f8daff tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:04.586352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de97b891-602a-4b16-9f4a-5906e8f8daff tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:04.586537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de97b891-602a-4b16-9f4a-5906e8f8daff tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:04.587345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de97b891-602a-4b16-9f4a-5906e8f8daff tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:04.588323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de97b891-602a-4b16-9f4a-5906e8f8daff tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:04.783019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-de97b891-602a-4b16-9f4a-5906e8f8daff tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:04.794585 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-de97b891-602a-4b16-9f4a-5906e8f8daff tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 200 len: 1509 microversion: 2.1 time: 0.225013 Sep 27 10:14:04.797697 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2233/4083] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:04 2022] GET /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 1509 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:04.826386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3a7dcef-58d2-4302-9659-9063d2e793f8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:04.827750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e3a7dcef-58d2-4302-9659-9063d2e793f8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: e60a1d1f-7e9a-44e8-88a3-fae7a0133c18] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:04.839405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3a7dcef-58d2-4302-9659-9063d2e793f8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:04.840503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3a7dcef-58d2-4302-9659-9063d2e793f8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:04.841097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3a7dcef-58d2-4302-9659-9063d2e793f8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:04.846460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3a7dcef-58d2-4302-9659-9063d2e793f8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:04.847143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3a7dcef-58d2-4302-9659-9063d2e793f8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:04.847626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3a7dcef-58d2-4302-9659-9063d2e793f8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:04.881446 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-baeeef26-99af-456c-b5eb-d6d17715e5ac tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "DELETE /compute/v2.1/os-security-groups/30bf3323-30cf-45ba-8587-ed0b3621f2b8" status: 202 len: 0 microversion: 2.1 time: 0.455541 Sep 27 10:14:04.882297 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1851/4084] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:14:04 2022] DELETE /compute/v2.1/os-security-groups/30bf3323-30cf-45ba-8587-ed0b3621f2b8 => generated 0 bytes in 458 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:14:04.891667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e4265067-5454-470a-b0ce-5d2fa34afc07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:04.892226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e4265067-5454-470a-b0ce-5d2fa34afc07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:04.908089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4265067-5454-470a-b0ce-5d2fa34afc07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:04.908818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4265067-5454-470a-b0ce-5d2fa34afc07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:04.909459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4265067-5454-470a-b0ce-5d2fa34afc07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:04.910228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4265067-5454-470a-b0ce-5d2fa34afc07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:04.910689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4265067-5454-470a-b0ce-5d2fa34afc07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:04.913508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e4265067-5454-470a-b0ce-5d2fa34afc07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:05.020506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e4265067-5454-470a-b0ce-5d2fa34afc07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:14:05.154395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e3a7dcef-58d2-4302-9659-9063d2e793f8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:05.182565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e3a7dcef-58d2-4302-9659-9063d2e793f8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/e60a1d1f-7e9a-44e8-88a3-fae7a0133c18" status: 200 len: 1320 microversion: 2.1 time: 0.361313 Sep 27 10:14:05.187662 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2234/4085] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:04 2022] GET /compute/v2.1/servers/e60a1d1f-7e9a-44e8-88a3-fae7a0133c18 => generated 1320 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:14:05.227654 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e4265067-5454-470a-b0ce-5d2fa34afc07 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "DELETE /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 204 len: 0 microversion: 2.1 time: 0.338548 Sep 27 10:14:05.228868 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1852/4086] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:14:04 2022] DELETE /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:14:05.237398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-38ca3e47-b800-447b-b4b9-7d66ffd55af0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:05.238640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-38ca3e47-b800-447b-b4b9-7d66ffd55af0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:05.249628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38ca3e47-b800-447b-b4b9-7d66ffd55af0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:05.250392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38ca3e47-b800-447b-b4b9-7d66ffd55af0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:05.251144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38ca3e47-b800-447b-b4b9-7d66ffd55af0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:05.252217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38ca3e47-b800-447b-b4b9-7d66ffd55af0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:05.253113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38ca3e47-b800-447b-b4b9-7d66ffd55af0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:05.253732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38ca3e47-b800-447b-b4b9-7d66ffd55af0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:05.540828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-38ca3e47-b800-447b-b4b9-7d66ffd55af0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:05.551464 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-38ca3e47-b800-447b-b4b9-7d66ffd55af0 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1511 microversion: 2.1 time: 0.318114 Sep 27 10:14:05.555064 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2235/4087] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:05 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1511 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:05.659164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-c488c8c3-12ec-455f-a313-d073e7437959 req-0c9b0ffb-ddbf-48b2-adce-6eef9b351fec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e60a1d1f-7e9a-44e8-88a3-fae7a0133c18", "name": "network-vif-plugged", "status": "completed", "tag": "ffd4ab48-3933-4905-98a4-38405d3efd95"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:05.677738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c488c8c3-12ec-455f-a313-d073e7437959 req-0c9b0ffb-ddbf-48b2-adce-6eef9b351fec service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:05.681526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c488c8c3-12ec-455f-a313-d073e7437959 req-0c9b0ffb-ddbf-48b2-adce-6eef9b351fec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:05.682151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c488c8c3-12ec-455f-a313-d073e7437959 req-0c9b0ffb-ddbf-48b2-adce-6eef9b351fec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:05.708792 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c488c8c3-12ec-455f-a313-d073e7437959 req-0c9b0ffb-ddbf-48b2-adce-6eef9b351fec service nova] Creating event network-vif-plugged:ffd4ab48-3933-4905-98a4-38405d3efd95 for instance e60a1d1f-7e9a-44e8-88a3-fae7a0133c18 on nch1-master-1 Sep 27 10:14:05.715013 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-c488c8c3-12ec-455f-a313-d073e7437959 req-0c9b0ffb-ddbf-48b2-adce-6eef9b351fec service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060726 Sep 27 10:14:05.716373 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1853/4088] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:06.212917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-251e041d-4189-4118-8226-575c7e52672f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:06.214958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-251e041d-4189-4118-8226-575c7e52672f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: e60a1d1f-7e9a-44e8-88a3-fae7a0133c18] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:06.226864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251e041d-4189-4118-8226-575c7e52672f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:06.228519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251e041d-4189-4118-8226-575c7e52672f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:06.229326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251e041d-4189-4118-8226-575c7e52672f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:06.230181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251e041d-4189-4118-8226-575c7e52672f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:06.230862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251e041d-4189-4118-8226-575c7e52672f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:06.232626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251e041d-4189-4118-8226-575c7e52672f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:06.539246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-251e041d-4189-4118-8226-575c7e52672f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:06.550986 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-251e041d-4189-4118-8226-575c7e52672f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/e60a1d1f-7e9a-44e8-88a3-fae7a0133c18" status: 200 len: 1320 microversion: 2.1 time: 0.341909 Sep 27 10:14:06.552648 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2236/4089] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:06 2022] GET /compute/v2.1/servers/e60a1d1f-7e9a-44e8-88a3-fae7a0133c18 => generated 1320 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:06.586721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-258c016b-c9fc-45ac-b510-7b36a276c604 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:06.589038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-258c016b-c9fc-45ac-b510-7b36a276c604 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:06.634896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-258c016b-c9fc-45ac-b510-7b36a276c604 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:06.635480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-258c016b-c9fc-45ac-b510-7b36a276c604 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:06.635922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-258c016b-c9fc-45ac-b510-7b36a276c604 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:06.638209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-258c016b-c9fc-45ac-b510-7b36a276c604 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:06.641655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-258c016b-c9fc-45ac-b510-7b36a276c604 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:06.642480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-258c016b-c9fc-45ac-b510-7b36a276c604 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:07.080184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-258c016b-c9fc-45ac-b510-7b36a276c604 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:07.096723 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-258c016b-c9fc-45ac-b510-7b36a276c604 tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1511 microversion: 2.1 time: 0.520489 Sep 27 10:14:07.099118 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1854/4090] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:06 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1511 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:07.597415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-00642a61-1751-413a-bfd4-9e15dff28c65 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:07.625073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-00642a61-1751-413a-bfd4-9e15dff28c65 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: e60a1d1f-7e9a-44e8-88a3-fae7a0133c18] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:07.641828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00642a61-1751-413a-bfd4-9e15dff28c65 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:07.642745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00642a61-1751-413a-bfd4-9e15dff28c65 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:07.643490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00642a61-1751-413a-bfd4-9e15dff28c65 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:07.645024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00642a61-1751-413a-bfd4-9e15dff28c65 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:07.645983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00642a61-1751-413a-bfd4-9e15dff28c65 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:07.646647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00642a61-1751-413a-bfd4-9e15dff28c65 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:07.653399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:08.009779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-00642a61-1751-413a-bfd4-9e15dff28c65 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:08.023588 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-00642a61-1751-413a-bfd4-9e15dff28c65 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/e60a1d1f-7e9a-44e8-88a3-fae7a0133c18" status: 200 len: 1508 microversion: 2.1 time: 0.443995 Sep 27 10:14:08.025932 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2237/4091] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:07 2022] GET /compute/v2.1/servers/e60a1d1f-7e9a-44e8-88a3-fae7a0133c18 => generated 1508 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:08.057396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0afae1f3-786e-4b40-b12c-9de0f0022bef tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:08.059081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0afae1f3-786e-4b40-b12c-9de0f0022bef tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: e60a1d1f-7e9a-44e8-88a3-fae7a0133c18] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:08.069005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0afae1f3-786e-4b40-b12c-9de0f0022bef tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:08.075799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0afae1f3-786e-4b40-b12c-9de0f0022bef tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:08.077758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0afae1f3-786e-4b40-b12c-9de0f0022bef tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:08.080090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0afae1f3-786e-4b40-b12c-9de0f0022bef tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:08.081515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0afae1f3-786e-4b40-b12c-9de0f0022bef tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:08.082905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0afae1f3-786e-4b40-b12c-9de0f0022bef tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:08.131432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b99eb336-9626-4664-9273-4f482b8bb69c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:08.132110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b99eb336-9626-4664-9273-4f482b8bb69c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:08.145044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b99eb336-9626-4664-9273-4f482b8bb69c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:08.148302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b99eb336-9626-4664-9273-4f482b8bb69c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:08.150143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b99eb336-9626-4664-9273-4f482b8bb69c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:08.151364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b99eb336-9626-4664-9273-4f482b8bb69c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:08.152394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b99eb336-9626-4664-9273-4f482b8bb69c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:08.156226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b99eb336-9626-4664-9273-4f482b8bb69c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:08.158906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0afae1f3-786e-4b40-b12c-9de0f0022bef tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: e60a1d1f-7e9a-44e8-88a3-fae7a0133c18] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:14:08.331746 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0afae1f3-786e-4b40-b12c-9de0f0022bef tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e60a1d1f-7e9a-44e8-88a3-fae7a0133c18" status: 204 len: 0 microversion: 2.1 time: 0.284171 Sep 27 10:14:08.334206 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1855/4092] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:14:08 2022] DELETE /compute/v2.1/servers/e60a1d1f-7e9a-44e8-88a3-fae7a0133c18 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:14:08.346449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cb87c30b-e453-4837-b92a-1996f9e7190d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:08.347429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cb87c30b-e453-4837-b92a-1996f9e7190d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:08.357991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb87c30b-e453-4837-b92a-1996f9e7190d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:08.358845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb87c30b-e453-4837-b92a-1996f9e7190d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:08.359584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb87c30b-e453-4837-b92a-1996f9e7190d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:08.360627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb87c30b-e453-4837-b92a-1996f9e7190d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:08.361915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb87c30b-e453-4837-b92a-1996f9e7190d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:08.363301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cb87c30b-e453-4837-b92a-1996f9e7190d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:08.406963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cb87c30b-e453-4837-b92a-1996f9e7190d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: eef2d4e4-d9aa-48ca-bd73-b538541465fa] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:14:08.501640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b99eb336-9626-4664-9273-4f482b8bb69c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:08.524044 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b99eb336-9626-4664-9273-4f482b8bb69c tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1511 microversion: 2.1 time: 0.401385 Sep 27 10:14:08.540035 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cb87c30b-e453-4837-b92a-1996f9e7190d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "DELETE /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa" status: 204 len: 0 microversion: 2.1 time: 0.200059 Sep 27 10:14:08.540954 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2238/4093] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:08 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1511 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:08.541472 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1856/4094] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:14:08 2022] DELETE /compute/v2.1/servers/eef2d4e4-d9aa-48ca-bd73-b538541465fa => generated 0 bytes in 203 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:14:08.693974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-MultipleCreateTestJSON-server-902322088", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "min_count": 1, "max_count": 2, "return_reservation_id": true, "networks": [{"uuid": "3168a5ae-5200-46e4-9fa9-362c59caaed1"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:08.831409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] validate_networks() for [('3168a5ae-5200-46e4-9fa9-362c59caaed1', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:14:09.134086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-6ba0fc4a-4d8a-4451-a0eb-54c204541f28 req-a6b59f63-b122-4381-8bab-f82b44f89f69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ce015738-c16b-4f29-8cd0-b96245268712", "name": "network-vif-deleted", "tag": "1871cdde-9d60-4d78-b0d8-e7defafdce3e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:09.159435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6ba0fc4a-4d8a-4451-a0eb-54c204541f28 req-a6b59f63-b122-4381-8bab-f82b44f89f69 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:09.159983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6ba0fc4a-4d8a-4451-a0eb-54c204541f28 req-a6b59f63-b122-4381-8bab-f82b44f89f69 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:09.163087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6ba0fc4a-4d8a-4451-a0eb-54c204541f28 req-a6b59f63-b122-4381-8bab-f82b44f89f69 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:09.196741 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-6ba0fc4a-4d8a-4451-a0eb-54c204541f28 req-a6b59f63-b122-4381-8bab-f82b44f89f69 service nova] Creating event network-vif-deleted:1871cdde-9d60-4d78-b0d8-e7defafdce3e for instance ce015738-c16b-4f29-8cd0-b96245268712 on nch2-master-1 Sep 27 10:14:09.210914 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-6ba0fc4a-4d8a-4451-a0eb-54c204541f28 req-a6b59f63-b122-4381-8bab-f82b44f89f69 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083457 Sep 27 10:14:09.214580 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1857/4095] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:09.548858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:09.549792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:09.562777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:09.563330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:09.563794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:09.564564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:09.565080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:09.565521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:09.632377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:09.632910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:09.633368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:09.633835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lazy-loading 'fault' on Instance uuid ce015738-c16b-4f29-8cd0-b96245268712 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:14:09.705020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '241251b91c43444b900922f942681cc5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f23e44d8a5094104b35d0288d33f0ab6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:09.715982 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-90b83f88-7e39-4cd4-8881-16bf3699280a tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 200 len: 1313 microversion: 2.1 time: 0.171763 Sep 27 10:14:09.716571 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1858/4096] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:09 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 1313 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:14:10.173741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:14:10.176260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:14:10.177056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:14:10.178831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:14:10.180300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:14:10.197868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:10.198482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:10.198959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:10.200110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:10.200867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:10.201624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:10.234079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Getting quotas for project 326c35c0bc52463e95a56ba3371a276a. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:14:10.245768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Getting quotas for user a5e050c7389844a78199faef5a0c9da4 and project 326c35c0bc52463e95a56ba3371a276a. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:14:10.291624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Going to run 2 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:14:10.295039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: af870ec8-bdb2-4a04-9688-ab4bb5ae8331] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:14:10.401088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=af870ec8-bdb2-4a04-9688-ab4bb5ae8331,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:14:10.514850 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8f26a001-6e78-4e67-bf22-647c99e69e74 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 32 microversion: 2.1 time: 1.826653 Sep 27 10:14:10.518937 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2239/4097] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:14:08 2022] POST /compute/v2.1/servers => generated 32 bytes in 1832 msecs (HTTP/1.1 202) 9 headers in 362 bytes (1 switches on core 0) Sep 27 10:14:10.528322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f31b1522-185f-471c-8f1e-3dfacc79a420 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:10.549266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f31b1522-185f-471c-8f1e-3dfacc79a420 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:10.550549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f31b1522-185f-471c-8f1e-3dfacc79a420 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Searching by: {'deleted': False, 'project_id': '326c35c0bc52463e95a56ba3371a276a'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:14:10.587630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f31b1522-185f-471c-8f1e-3dfacc79a420 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:10.589481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f31b1522-185f-471c-8f1e-3dfacc79a420 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:10.594619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f31b1522-185f-471c-8f1e-3dfacc79a420 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:10.598345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f31b1522-185f-471c-8f1e-3dfacc79a420 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:10.599097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f31b1522-185f-471c-8f1e-3dfacc79a420 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:10.599757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f31b1522-185f-471c-8f1e-3dfacc79a420 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:10.696781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-f31b1522-185f-471c-8f1e-3dfacc79a420 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Listed batch of 2 results from cell out of 998 limit. Returned 2 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:14:10.746038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:10.747194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] [instance: ce015738-c16b-4f29-8cd0-b96245268712] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:10.759276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:10.759826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:10.760291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:10.766228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:10.767004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:10.767355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:10.779687 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f31b1522-185f-471c-8f1e-3dfacc79a420 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 1335 microversion: 2.1 time: 0.255242 Sep 27 10:14:10.788837 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1859/4098] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:14:10 2022] GET /compute/v2.1/servers => generated 1335 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:10.794616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ac97a301-b472-4387-b96f-b8f3905bf9f4 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:10.795739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ac97a301-b472-4387-b96f-b8f3905bf9f4 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:10.804599 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] HTTP exception thrown: Instance ce015738-c16b-4f29-8cd0-b96245268712 could not be found. Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Request method failure captured: Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 HTTP/1.1 Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsycZ-P2Sl_QD5AYj8kUj_2215IpF-FpGMEOHjgujdQjZuZWBiqPIzaDKNJpVCorgcuKKONUtQuifZpqgiCHasL4XlHWKcYwiyb15kCYsIvL3EpoCtU3ibpUgmWVa2FRcGXVDZ-QyYgS1cdGlXmYqBfh01Jym7s9TKmWXAoQYsNIjyo Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f23e44d8a5094104b35d0288d33f0ab6 Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerRescueTestJSONUnderV235-2070851702 Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f23e44d8a5094104b35d0288d33f0ab6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f23e44d8a5094104b35d0288d33f0ab6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f23e44d8a5094104b35d0288d33f0ab6"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f23e44d8a5094104b35d0288d33f0ab6"}]}] Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerRescueTestJSONUnderV235-2070851702 Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f23e44d8a5094104b35d0288d33f0ab6 Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerRescueTestJSONUnderV235-2070851702 Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerRescueTestJSONUnderV235-2070851702-project Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 241251b91c43444b900922f942681cc5 Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerRescueTestJSONUnderV235-2070851702-project Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance ce015738-c16b-4f29-8cd0-b96245268712 could not be found. Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'ce015738-c16b-4f29-8cd0-b96245268712'} Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ce015738-c16b-4f29-8cd0-b96245268712 could not be found. Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ce015738-c16b-4f29-8cd0-b96245268712 could not be found. Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:10.806413 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:14:10.811079 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:10.811079 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:10.811079 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:10.811079 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:10.811079 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:14:10.811079 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:14:10.811079 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ce015738-c16b-4f29-8cd0-b96245268712 could not be found. Sep 27 10:14:10.811079 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:10.811079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] Returning 404 to user: Instance ce015738-c16b-4f29-8cd0-b96245268712 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:14:10.811079 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-39e845e2-e7e2-43f2-9f31-c33fe96778ad tempest-ServerRescueTestJSONUnderV235-2070851702 tempest-ServerRescueTestJSONUnderV235-2070851702-project] 10.222.0.26 "GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712" status: 404 len: 111 microversion: 2.1 time: 0.068399 Sep 27 10:14:10.817762 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2240/4099] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:10 2022] GET /compute/v2.1/servers/ce015738-c16b-4f29-8cd0-b96245268712 => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:14:10.964323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ac97a301-b472-4387-b96f-b8f3905bf9f4 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:10.973470 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ac97a301-b472-4387-b96f-b8f3905bf9f4 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1217 microversion: 2.1 time: 0.183214 Sep 27 10:14:10.974059 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1860/4100] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:10 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1217 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:11.996620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d7f4a19d-7c69-45cd-a81e-aca8644835c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:11.997451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d7f4a19d-7c69-45cd-a81e-aca8644835c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:12.011627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7f4a19d-7c69-45cd-a81e-aca8644835c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:12.012323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7f4a19d-7c69-45cd-a81e-aca8644835c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:12.012868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7f4a19d-7c69-45cd-a81e-aca8644835c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:12.013755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7f4a19d-7c69-45cd-a81e-aca8644835c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:12.014827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7f4a19d-7c69-45cd-a81e-aca8644835c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:12.015530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7f4a19d-7c69-45cd-a81e-aca8644835c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:12.159548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d7f4a19d-7c69-45cd-a81e-aca8644835c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:12.171549 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d7f4a19d-7c69-45cd-a81e-aca8644835c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1213 microversion: 2.1 time: 0.176358 Sep 27 10:14:12.172234 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2241/4101] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:11 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1213 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:12.754907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d40d2537-671a-4fd9-a95c-ae32a09864b4 req-d1b4c5ec-c8ef-4750-a6c2-fc45d24d0f2b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e60a1d1f-7e9a-44e8-88a3-fae7a0133c18", "name": "network-vif-deleted", "tag": "ffd4ab48-3933-4905-98a4-38405d3efd95"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:12.791837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d40d2537-671a-4fd9-a95c-ae32a09864b4 req-d1b4c5ec-c8ef-4750-a6c2-fc45d24d0f2b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:12.796359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d40d2537-671a-4fd9-a95c-ae32a09864b4 req-d1b4c5ec-c8ef-4750-a6c2-fc45d24d0f2b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:12.796480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d40d2537-671a-4fd9-a95c-ae32a09864b4 req-d1b4c5ec-c8ef-4750-a6c2-fc45d24d0f2b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:12.828079 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d40d2537-671a-4fd9-a95c-ae32a09864b4 req-d1b4c5ec-c8ef-4750-a6c2-fc45d24d0f2b service nova] Creating event network-vif-deleted:ffd4ab48-3933-4905-98a4-38405d3efd95 for instance e60a1d1f-7e9a-44e8-88a3-fae7a0133c18 on nch1-master-1 Sep 27 10:14:12.852791 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d40d2537-671a-4fd9-a95c-ae32a09864b4 req-d1b4c5ec-c8ef-4750-a6c2-fc45d24d0f2b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.097285 Sep 27 10:14:12.856101 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1861/4102] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:13.200239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6b5f616d-d490-414b-b3cd-867cc39da3fe tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:13.203251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6b5f616d-d490-414b-b3cd-867cc39da3fe tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:13.216537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b5f616d-d490-414b-b3cd-867cc39da3fe tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:13.217416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b5f616d-d490-414b-b3cd-867cc39da3fe tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:13.218062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b5f616d-d490-414b-b3cd-867cc39da3fe tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:13.219507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b5f616d-d490-414b-b3cd-867cc39da3fe tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:13.220163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b5f616d-d490-414b-b3cd-867cc39da3fe tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:13.221476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b5f616d-d490-414b-b3cd-867cc39da3fe tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:13.360550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6b5f616d-d490-414b-b3cd-867cc39da3fe tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:13.372363 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6b5f616d-d490-414b-b3cd-867cc39da3fe tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1269 microversion: 2.1 time: 0.180113 Sep 27 10:14:13.373475 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2242/4103] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:13 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1269 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:14.402584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a796c483-1d32-42f4-a95f-c34a88a12426 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:14.403311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a796c483-1d32-42f4-a95f-c34a88a12426 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:14.433744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a796c483-1d32-42f4-a95f-c34a88a12426 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:14.434312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a796c483-1d32-42f4-a95f-c34a88a12426 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:14.434797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a796c483-1d32-42f4-a95f-c34a88a12426 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:14.435579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a796c483-1d32-42f4-a95f-c34a88a12426 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:14.436115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a796c483-1d32-42f4-a95f-c34a88a12426 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:14.436554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a796c483-1d32-42f4-a95f-c34a88a12426 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:14.518524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=91,nova_cell1:SAVEPOINT=1,nova_cell1:UPDATE=4,nova_cell1:RELEASE=1,nova_cell1:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:14.734157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a796c483-1d32-42f4-a95f-c34a88a12426 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:14.751252 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a796c483-1d32-42f4-a95f-c34a88a12426 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1275 microversion: 2.1 time: 0.355047 Sep 27 10:14:14.751974 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1862/4104] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:14 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1275 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:15.135268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e332e768-581b-4753-b707-3d461e4364d9 req-18ac8370-8647-4757-be54-71419b943d66 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eef2d4e4-d9aa-48ca-bd73-b538541465fa", "name": "network-vif-deleted", "tag": "fe94630e-b20d-425b-8f30-a2c5c4298db8"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:15.183461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e332e768-581b-4753-b707-3d461e4364d9 req-18ac8370-8647-4757-be54-71419b943d66 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:15.185007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e332e768-581b-4753-b707-3d461e4364d9 req-18ac8370-8647-4757-be54-71419b943d66 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:15.190501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e332e768-581b-4753-b707-3d461e4364d9 req-18ac8370-8647-4757-be54-71419b943d66 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:15.244249 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e332e768-581b-4753-b707-3d461e4364d9 req-18ac8370-8647-4757-be54-71419b943d66 service nova] Creating event network-vif-deleted:fe94630e-b20d-425b-8f30-a2c5c4298db8 for instance eef2d4e4-d9aa-48ca-bd73-b538541465fa on nch1-master-1 Sep 27 10:14:15.253772 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e332e768-581b-4753-b707-3d461e4364d9 req-18ac8370-8647-4757-be54-71419b943d66 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120953 Sep 27 10:14:15.260488 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2243/4105] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:15.780171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ff2c797e-55cc-44c6-84cb-8d0d9e15a547 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:15.781467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ff2c797e-55cc-44c6-84cb-8d0d9e15a547 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:15.795722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff2c797e-55cc-44c6-84cb-8d0d9e15a547 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:15.797988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff2c797e-55cc-44c6-84cb-8d0d9e15a547 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:15.799678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff2c797e-55cc-44c6-84cb-8d0d9e15a547 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:15.801875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff2c797e-55cc-44c6-84cb-8d0d9e15a547 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:15.803221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff2c797e-55cc-44c6-84cb-8d0d9e15a547 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:15.809176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff2c797e-55cc-44c6-84cb-8d0d9e15a547 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:15.958244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ff2c797e-55cc-44c6-84cb-8d0d9e15a547 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:15.970280 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ff2c797e-55cc-44c6-84cb-8d0d9e15a547 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1275 microversion: 2.1 time: 0.196432 Sep 27 10:14:15.975396 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1863/4106] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:15 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1275 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:17.002846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e67ee79f-5576-40a1-a2ab-d40d2b2660b8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:17.004547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e67ee79f-5576-40a1-a2ab-d40d2b2660b8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:17.016672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:INSERT=9,nova_api:SELECT=71 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:17.032549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e67ee79f-5576-40a1-a2ab-d40d2b2660b8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:17.034588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e67ee79f-5576-40a1-a2ab-d40d2b2660b8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:17.035295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e67ee79f-5576-40a1-a2ab-d40d2b2660b8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:17.036274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e67ee79f-5576-40a1-a2ab-d40d2b2660b8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:17.045240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e67ee79f-5576-40a1-a2ab-d40d2b2660b8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:17.045728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e67ee79f-5576-40a1-a2ab-d40d2b2660b8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:17.219793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e67ee79f-5576-40a1-a2ab-d40d2b2660b8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:17.233044 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e67ee79f-5576-40a1-a2ab-d40d2b2660b8 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1275 microversion: 2.1 time: 0.237473 Sep 27 10:14:17.236806 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2244/4107] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:16 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1275 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:18.264476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-71576fa3-22b1-4615-97fd-23c250493736 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:18.265823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-71576fa3-22b1-4615-97fd-23c250493736 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:18.281641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71576fa3-22b1-4615-97fd-23c250493736 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:18.282712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71576fa3-22b1-4615-97fd-23c250493736 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:18.285033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71576fa3-22b1-4615-97fd-23c250493736 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:18.285807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71576fa3-22b1-4615-97fd-23c250493736 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:18.286282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71576fa3-22b1-4615-97fd-23c250493736 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:18.287402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71576fa3-22b1-4615-97fd-23c250493736 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:18.453912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-71576fa3-22b1-4615-97fd-23c250493736 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:18.464818 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-71576fa3-22b1-4615-97fd-23c250493736 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1275 microversion: 2.1 time: 0.206511 Sep 27 10:14:18.466105 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1864/4108] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:18 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1275 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:19.094834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-InstanceActionsNegativeTestJSON-server-1294892951", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "e7678fe5-ddd2-4798-ac50-a03b6010a9f3"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:19.172138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] validate_networks() for [('e7678fe5-ddd2-4798-ac50-a03b6010a9f3', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:14:19.493429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-727e4290-7237-4837-8aaa-c280fc4f075c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:19.494669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-727e4290-7237-4837-8aaa-c280fc4f075c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:19.508311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-727e4290-7237-4837-8aaa-c280fc4f075c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:19.510702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-727e4290-7237-4837-8aaa-c280fc4f075c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:19.514052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-727e4290-7237-4837-8aaa-c280fc4f075c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:19.514872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-727e4290-7237-4837-8aaa-c280fc4f075c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:19.515395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-727e4290-7237-4837-8aaa-c280fc4f075c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:19.516947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-727e4290-7237-4837-8aaa-c280fc4f075c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:19.842797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-727e4290-7237-4837-8aaa-c280fc4f075c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:19.857411 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-727e4290-7237-4837-8aaa-c280fc4f075c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 0.369585 Sep 27 10:14:19.859338 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1865/4109] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:19 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:14:20.215686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:20.281512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:20.763323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1e7d4ec8-e1a7-4f7c-bbee-b79915555443 req-9ee04e5e-4872-4542-bf0c-f4603f36daee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "af870ec8-bdb2-4a04-9688-ab4bb5ae8331", "tag": "c157e508-9e0f-4763-a230-1bf760fe648b"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:20.791812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:20.805647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1e7d4ec8-e1a7-4f7c-bbee-b79915555443 req-9ee04e5e-4872-4542-bf0c-f4603f36daee service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:20.808020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1e7d4ec8-e1a7-4f7c-bbee-b79915555443 req-9ee04e5e-4872-4542-bf0c-f4603f36daee service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:20.809631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1e7d4ec8-e1a7-4f7c-bbee-b79915555443 req-9ee04e5e-4872-4542-bf0c-f4603f36daee service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:20.846327 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1e7d4ec8-e1a7-4f7c-bbee-b79915555443 req-9ee04e5e-4872-4542-bf0c-f4603f36daee service nova] Creating event network-changed:c157e508-9e0f-4763-a230-1bf760fe648b for instance af870ec8-bdb2-4a04-9688-ab4bb5ae8331 on nch2-master-1 Sep 27 10:14:20.855919 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1e7d4ec8-e1a7-4f7c-bbee-b79915555443 req-9ee04e5e-4872-4542-bf0c-f4603f36daee service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.097269 Sep 27 10:14:20.857047 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1866/4110] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:20.891546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b1f8e21b-e72e-439f-a1b3-b71d9959e45f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:20.892272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b1f8e21b-e72e-439f-a1b3-b71d9959e45f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:20.907647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1f8e21b-e72e-439f-a1b3-b71d9959e45f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:20.908333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1f8e21b-e72e-439f-a1b3-b71d9959e45f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:20.909057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1f8e21b-e72e-439f-a1b3-b71d9959e45f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:20.917945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1f8e21b-e72e-439f-a1b3-b71d9959e45f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:20.918710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1f8e21b-e72e-439f-a1b3-b71d9959e45f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:20.919191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1f8e21b-e72e-439f-a1b3-b71d9959e45f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:20.977264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:21.296078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:14:21.296904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:14:21.297477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:14:21.298043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:14:21.298605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:14:21.357376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:21.358043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:21.362373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:21.363296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:21.364240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:21.364240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:21.446049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Getting quotas for project 1e7e90f89b4844dbafdc1c5e5f17b407. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:14:21.461184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Getting quotas for user c28c5e12dbb444cab4e376a3015c6bd1 and project 1e7e90f89b4844dbafdc1c5e5f17b407. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:14:21.510431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:14:21.518702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:14:21.735517 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a73384c-5de2-41b4-b848-ddd438e8d959 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.839359 Sep 27 10:14:21.744555 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2245/4111] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:14:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 2849 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:14:21.767878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-970c4ebb-bd54-4b7d-9611-0527cbfc75b3 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:21.771887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-970c4ebb-bd54-4b7d-9611-0527cbfc75b3 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:21.981508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b1f8e21b-e72e-439f-a1b3-b71d9959e45f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:22.005620 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b1f8e21b-e72e-439f-a1b3-b71d9959e45f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 1.122797 Sep 27 10:14:22.007157 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1867/4112] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:20 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 1126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:22.196211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-970c4ebb-bd54-4b7d-9611-0527cbfc75b3 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:22.208410 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-970c4ebb-bd54-4b7d-9611-0527cbfc75b3 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1225 microversion: 2.1 time: 0.456181 Sep 27 10:14:22.209038 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2246/4113] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:21 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1225 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:22.921471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-abbbd8ab-bd5e-4f2b-893a-40e32a467b89 req-3a64d5e6-f411-4e4e-8263-865e86a63eec service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ac85843-17b5-4435-b528-cb0d37b70631", "tag": "3a6fbbd7-4580-4281-aac2-fa84df8853b2"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:22.935344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abbbd8ab-bd5e-4f2b-893a-40e32a467b89 req-3a64d5e6-f411-4e4e-8263-865e86a63eec service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:22.936388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abbbd8ab-bd5e-4f2b-893a-40e32a467b89 req-3a64d5e6-f411-4e4e-8263-865e86a63eec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:22.937118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abbbd8ab-bd5e-4f2b-893a-40e32a467b89 req-3a64d5e6-f411-4e4e-8263-865e86a63eec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:22.961769 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-abbbd8ab-bd5e-4f2b-893a-40e32a467b89 req-3a64d5e6-f411-4e4e-8263-865e86a63eec service nova] Creating event network-changed:3a6fbbd7-4580-4281-aac2-fa84df8853b2 for instance 4ac85843-17b5-4435-b528-cb0d37b70631 on nch2-master-1 Sep 27 10:14:22.971432 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-abbbd8ab-bd5e-4f2b-893a-40e32a467b89 req-3a64d5e6-f411-4e4e-8263-865e86a63eec service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054919 Sep 27 10:14:22.974078 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1868/4114] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:22.976690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-839ea009-20d1-4353-88ff-5026b577c107 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:22.980637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-839ea009-20d1-4353-88ff-5026b577c107 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:23.005824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839ea009-20d1-4353-88ff-5026b577c107 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:23.006482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839ea009-20d1-4353-88ff-5026b577c107 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:23.006940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839ea009-20d1-4353-88ff-5026b577c107 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:23.007811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839ea009-20d1-4353-88ff-5026b577c107 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:23.008237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839ea009-20d1-4353-88ff-5026b577c107 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:23.014416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-839ea009-20d1-4353-88ff-5026b577c107 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:23.041038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-96f28ad6-31b7-4118-8d30-9d69526e7716 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:23.043100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-96f28ad6-31b7-4118-8d30-9d69526e7716 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:23.052865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96f28ad6-31b7-4118-8d30-9d69526e7716 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:23.053635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96f28ad6-31b7-4118-8d30-9d69526e7716 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:23.065610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96f28ad6-31b7-4118-8d30-9d69526e7716 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:23.066687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96f28ad6-31b7-4118-8d30-9d69526e7716 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:23.067417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96f28ad6-31b7-4118-8d30-9d69526e7716 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:23.086538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96f28ad6-31b7-4118-8d30-9d69526e7716 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:23.117597 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-839ea009-20d1-4353-88ff-5026b577c107 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/ips" status: 200 len: 145 microversion: 2.1 time: 0.141572 Sep 27 10:14:23.118518 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2247/4115] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:14:22 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/ips => generated 145 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:14:23.237554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5f33bcc3-7e6b-4171-9b79-cc6b93b8c75b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:23.249698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5f33bcc3-7e6b-4171-9b79-cc6b93b8c75b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:23.261844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f33bcc3-7e6b-4171-9b79-cc6b93b8c75b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:23.275615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f33bcc3-7e6b-4171-9b79-cc6b93b8c75b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:23.276364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f33bcc3-7e6b-4171-9b79-cc6b93b8c75b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:23.280317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f33bcc3-7e6b-4171-9b79-cc6b93b8c75b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:23.282264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f33bcc3-7e6b-4171-9b79-cc6b93b8c75b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:23.283569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f33bcc3-7e6b-4171-9b79-cc6b93b8c75b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:23.518507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-96f28ad6-31b7-4118-8d30-9d69526e7716 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:23.534605 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-96f28ad6-31b7-4118-8d30-9d69526e7716 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 0.494232 Sep 27 10:14:23.545385 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1869/4116] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:23 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:23.729744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f33bcc3-7e6b-4171-9b79-cc6b93b8c75b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:23.740689 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5f33bcc3-7e6b-4171-9b79-cc6b93b8c75b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1277 microversion: 2.1 time: 0.507368 Sep 27 10:14:23.777417 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2248/4117] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:23 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1277 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:14:24.562061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d355e9b4-f44b-4024-a41f-d6eb40fbbbd6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:24.562669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d355e9b4-f44b-4024-a41f-d6eb40fbbbd6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:24.579942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d355e9b4-f44b-4024-a41f-d6eb40fbbbd6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:24.580474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d355e9b4-f44b-4024-a41f-d6eb40fbbbd6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:24.582321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d355e9b4-f44b-4024-a41f-d6eb40fbbbd6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:24.583072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d355e9b4-f44b-4024-a41f-d6eb40fbbbd6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:24.584608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d355e9b4-f44b-4024-a41f-d6eb40fbbbd6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:24.585450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d355e9b4-f44b-4024-a41f-d6eb40fbbbd6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:24.803286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c50813ff-969e-4ea3-9997-4540c77e7439 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:24.804566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c50813ff-969e-4ea3-9997-4540c77e7439 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:24.818511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c50813ff-969e-4ea3-9997-4540c77e7439 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:24.819081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c50813ff-969e-4ea3-9997-4540c77e7439 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:24.823361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c50813ff-969e-4ea3-9997-4540c77e7439 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:24.824470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c50813ff-969e-4ea3-9997-4540c77e7439 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:24.825031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c50813ff-969e-4ea3-9997-4540c77e7439 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:24.825471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c50813ff-969e-4ea3-9997-4540c77e7439 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:24.973223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c50813ff-969e-4ea3-9997-4540c77e7439 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:24.987420 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c50813ff-969e-4ea3-9997-4540c77e7439 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1283 microversion: 2.1 time: 0.186737 Sep 27 10:14:24.987420 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2249/4118] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:24 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1283 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:25.689673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d355e9b4-f44b-4024-a41f-d6eb40fbbbd6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:25.712268 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d355e9b4-f44b-4024-a41f-d6eb40fbbbd6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 1.150220 Sep 27 10:14:25.741873 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1870/4119] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:24 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 1184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:25.847789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e5af107c-5b71-461c-a3e1-deb10425a0f9"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:25.853251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:25.869229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:25.870096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:25.870749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:25.871883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:25.872622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:25.873285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:26.012761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5574b64f-c969-45f3-844e-0267ed82e6bc tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:26.013396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5574b64f-c969-45f3-844e-0267ed82e6bc tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:26.029375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5574b64f-c969-45f3-844e-0267ed82e6bc tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:26.033079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5574b64f-c969-45f3-844e-0267ed82e6bc tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:26.033583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5574b64f-c969-45f3-844e-0267ed82e6bc tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:26.034427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5574b64f-c969-45f3-844e-0267ed82e6bc tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:26.038393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5574b64f-c969-45f3-844e-0267ed82e6bc tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:26.038903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5574b64f-c969-45f3-844e-0267ed82e6bc tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:26.395270 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-40ffd251-c2a2-4fbc-b8f4-da1c0a7c7ce9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "POST /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.553004 Sep 27 10:14:26.397626 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2250/4120] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:14:25 2022] POST /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/os-volume_attachments => generated 194 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:26.548782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5574b64f-c969-45f3-844e-0267ed82e6bc tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:26.565342 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5574b64f-c969-45f3-844e-0267ed82e6bc tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1283 microversion: 2.1 time: 0.559469 Sep 27 10:14:26.565959 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1871/4121] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:26 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1283 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:26.743189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0452b917-0deb-4602-9a1b-091e06a2c8d6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:26.744336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0452b917-0deb-4602-9a1b-091e06a2c8d6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:26.759862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0452b917-0deb-4602-9a1b-091e06a2c8d6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:26.760702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0452b917-0deb-4602-9a1b-091e06a2c8d6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:26.762828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0452b917-0deb-4602-9a1b-091e06a2c8d6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:26.764839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0452b917-0deb-4602-9a1b-091e06a2c8d6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:26.766237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0452b917-0deb-4602-9a1b-091e06a2c8d6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:26.766899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0452b917-0deb-4602-9a1b-091e06a2c8d6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:27.033326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0452b917-0deb-4602-9a1b-091e06a2c8d6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:27.042876 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0452b917-0deb-4602-9a1b-091e06a2c8d6 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 0.306380 Sep 27 10:14:27.047379 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2251/4122] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:26 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:27.592040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c81d1711-7549-4c5b-80cb-1b5aceaedac2 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:27.592692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c81d1711-7549-4c5b-80cb-1b5aceaedac2 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:27.596034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:27.639720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c81d1711-7549-4c5b-80cb-1b5aceaedac2 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:27.649222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c81d1711-7549-4c5b-80cb-1b5aceaedac2 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:27.651695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c81d1711-7549-4c5b-80cb-1b5aceaedac2 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:27.652642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c81d1711-7549-4c5b-80cb-1b5aceaedac2 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:27.657602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c81d1711-7549-4c5b-80cb-1b5aceaedac2 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:27.658286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c81d1711-7549-4c5b-80cb-1b5aceaedac2 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:28.076201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3345e57-4c74-4ad2-bb4c-68edd95acd00 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:28.083296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d3345e57-4c74-4ad2-bb4c-68edd95acd00 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:28.108491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3345e57-4c74-4ad2-bb4c-68edd95acd00 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:28.110115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3345e57-4c74-4ad2-bb4c-68edd95acd00 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:28.112417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3345e57-4c74-4ad2-bb4c-68edd95acd00 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:28.114791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3345e57-4c74-4ad2-bb4c-68edd95acd00 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:28.116074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3345e57-4c74-4ad2-bb4c-68edd95acd00 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:28.117838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3345e57-4c74-4ad2-bb4c-68edd95acd00 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:28.419132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d3345e57-4c74-4ad2-bb4c-68edd95acd00 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:28.431781 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d3345e57-4c74-4ad2-bb4c-68edd95acd00 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 0.360903 Sep 27 10:14:28.432916 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2252/4123] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:28 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:29.204160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c81d1711-7549-4c5b-80cb-1b5aceaedac2 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:29.219899 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c81d1711-7549-4c5b-80cb-1b5aceaedac2 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1325 microversion: 2.1 time: 1.631805 Sep 27 10:14:29.220513 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1872/4124] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:27 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1325 bytes in 1633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:29.462586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-89ebb3c8-e54e-4b96-b45c-c9e452075c93 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:29.463218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-89ebb3c8-e54e-4b96-b45c-c9e452075c93 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:29.478839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89ebb3c8-e54e-4b96-b45c-c9e452075c93 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:29.479475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89ebb3c8-e54e-4b96-b45c-c9e452075c93 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:29.480087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89ebb3c8-e54e-4b96-b45c-c9e452075c93 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:29.480865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89ebb3c8-e54e-4b96-b45c-c9e452075c93 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:29.481603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89ebb3c8-e54e-4b96-b45c-c9e452075c93 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:29.493548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89ebb3c8-e54e-4b96-b45c-c9e452075c93 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:29.885643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-89ebb3c8-e54e-4b96-b45c-c9e452075c93 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:29.899646 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-89ebb3c8-e54e-4b96-b45c-c9e452075c93 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 0.442017 Sep 27 10:14:29.901158 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2253/4125] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:29 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:30.095209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e874e72-0b50-47e6-9eb8-1ae5e968d1b4 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:30.107016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e874e72-0b50-47e6-9eb8-1ae5e968d1b4 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:30.108839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e874e72-0b50-47e6-9eb8-1ae5e968d1b4 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:30.111505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e874e72-0b50-47e6-9eb8-1ae5e968d1b4 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:30.161382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4e874e72-0b50-47e6-9eb8-1ae5e968d1b4 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Going to try to stop instance {{(pid=145969) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 10:14:30.247623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1ef75f52-c290-47ac-adc2-930cafd5670f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:30.248646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1ef75f52-c290-47ac-adc2-930cafd5670f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:30.268130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef75f52-c290-47ac-adc2-930cafd5670f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:30.272190 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4e874e72-0b50-47e6-9eb8-1ae5e968d1b4 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "POST /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/action" status: 202 len: 0 microversion: 2.1 time: 0.180851 Sep 27 10:14:30.273073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef75f52-c290-47ac-adc2-930cafd5670f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:30.274150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef75f52-c290-47ac-adc2-930cafd5670f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:30.275266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef75f52-c290-47ac-adc2-930cafd5670f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:30.275961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef75f52-c290-47ac-adc2-930cafd5670f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:30.277897 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1873/4126] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:14:30 2022] POST /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/action => generated 0 bytes in 188 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:14:30.278756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef75f52-c290-47ac-adc2-930cafd5670f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:30.290282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-98a40fb2-d944-4ba1-9b82-351c2e23f025 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:30.292072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-98a40fb2-d944-4ba1-9b82-351c2e23f025 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:30.307162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98a40fb2-d944-4ba1-9b82-351c2e23f025 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:30.313610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98a40fb2-d944-4ba1-9b82-351c2e23f025 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:30.314404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98a40fb2-d944-4ba1-9b82-351c2e23f025 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:30.315372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98a40fb2-d944-4ba1-9b82-351c2e23f025 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:30.316058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98a40fb2-d944-4ba1-9b82-351c2e23f025 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:30.316747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98a40fb2-d944-4ba1-9b82-351c2e23f025 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:30.531601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1ef75f52-c290-47ac-adc2-930cafd5670f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:30.552895 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1ef75f52-c290-47ac-adc2-930cafd5670f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1325 microversion: 2.1 time: 0.310177 Sep 27 10:14:30.555042 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2254/4127] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:30 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1325 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:30.907431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-98a40fb2-d944-4ba1-9b82-351c2e23f025 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:30.919033 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-98a40fb2-d944-4ba1-9b82-351c2e23f025 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1723 microversion: 2.1 time: 0.633946 Sep 27 10:14:30.920370 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1874/4128] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:30 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1723 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:30.928038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8fdc6407-1703-4c4b-9b82-70d213f3f592 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:30.929331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8fdc6407-1703-4c4b-9b82-70d213f3f592 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:30.940411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fdc6407-1703-4c4b-9b82-70d213f3f592 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:30.940937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fdc6407-1703-4c4b-9b82-70d213f3f592 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:30.941425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fdc6407-1703-4c4b-9b82-70d213f3f592 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:30.942138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fdc6407-1703-4c4b-9b82-70d213f3f592 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:30.942598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fdc6407-1703-4c4b-9b82-70d213f3f592 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:30.943024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fdc6407-1703-4c4b-9b82-70d213f3f592 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:31.196277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8fdc6407-1703-4c4b-9b82-70d213f3f592 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:31.209383 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8fdc6407-1703-4c4b-9b82-70d213f3f592 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 0.285095 Sep 27 10:14:31.218112 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2255/4129] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:30 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:31.578963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd8778d7-18cd-41c5-9a34-9ba289ab3c13 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:31.580366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd8778d7-18cd-41c5-9a34-9ba289ab3c13 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:31.592051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd8778d7-18cd-41c5-9a34-9ba289ab3c13 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:31.593232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd8778d7-18cd-41c5-9a34-9ba289ab3c13 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:31.593913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd8778d7-18cd-41c5-9a34-9ba289ab3c13 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:31.594879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd8778d7-18cd-41c5-9a34-9ba289ab3c13 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:31.595552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd8778d7-18cd-41c5-9a34-9ba289ab3c13 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:31.596193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd8778d7-18cd-41c5-9a34-9ba289ab3c13 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:31.877713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3886f99a-1223-4899-ae3a-a893a5a9fbab req-b49b3a7d-6a4b-414a-8d94-383340a5efd6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "af870ec8-bdb2-4a04-9688-ab4bb5ae8331", "name": "network-vif-plugged", "status": "completed", "tag": "c157e508-9e0f-4763-a230-1bf760fe648b"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:31.885010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:31.900497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3886f99a-1223-4899-ae3a-a893a5a9fbab req-b49b3a7d-6a4b-414a-8d94-383340a5efd6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:31.911469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3886f99a-1223-4899-ae3a-a893a5a9fbab req-b49b3a7d-6a4b-414a-8d94-383340a5efd6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:31.911923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3886f99a-1223-4899-ae3a-a893a5a9fbab req-b49b3a7d-6a4b-414a-8d94-383340a5efd6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:31.930597 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3886f99a-1223-4899-ae3a-a893a5a9fbab req-b49b3a7d-6a4b-414a-8d94-383340a5efd6 service nova] Creating event network-vif-plugged:c157e508-9e0f-4763-a230-1bf760fe648b for instance af870ec8-bdb2-4a04-9688-ab4bb5ae8331 on nch2-master-1 Sep 27 10:14:31.945187 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3886f99a-1223-4899-ae3a-a893a5a9fbab req-b49b3a7d-6a4b-414a-8d94-383340a5efd6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071096 Sep 27 10:14:31.948984 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2256/4130] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:31.960174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-24b7e6cd-f4a2-4006-ac50-0dc63bfd9bfc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:31.961233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-24b7e6cd-f4a2-4006-ac50-0dc63bfd9bfc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:31.972321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24b7e6cd-f4a2-4006-ac50-0dc63bfd9bfc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:31.973466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24b7e6cd-f4a2-4006-ac50-0dc63bfd9bfc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:31.974170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24b7e6cd-f4a2-4006-ac50-0dc63bfd9bfc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:31.975585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24b7e6cd-f4a2-4006-ac50-0dc63bfd9bfc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:31.977348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24b7e6cd-f4a2-4006-ac50-0dc63bfd9bfc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:31.978170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24b7e6cd-f4a2-4006-ac50-0dc63bfd9bfc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:31.987391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:32.203104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:32.620350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd8778d7-18cd-41c5-9a34-9ba289ab3c13 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:32.672970 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd8778d7-18cd-41c5-9a34-9ba289ab3c13 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1325 microversion: 2.1 time: 1.098386 Sep 27 10:14:32.676935 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1875/4131] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:31 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1325 bytes in 1102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:32.691535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-53929de0-6e09-45fb-a443-1235d2ce82f9 req-72c8e955-2e25-4240-80be-690466eff78e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "59d9d393-afa0-4c17-a358-09fb4e415b60", "tag": "b41cbc51-7a4c-4d78-adfd-0a78a0690ee0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:32.729213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-53929de0-6e09-45fb-a443-1235d2ce82f9 req-72c8e955-2e25-4240-80be-690466eff78e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:32.731205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-53929de0-6e09-45fb-a443-1235d2ce82f9 req-72c8e955-2e25-4240-80be-690466eff78e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:32.733020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-53929de0-6e09-45fb-a443-1235d2ce82f9 req-72c8e955-2e25-4240-80be-690466eff78e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:32.779603 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-53929de0-6e09-45fb-a443-1235d2ce82f9 req-72c8e955-2e25-4240-80be-690466eff78e service nova] Creating event network-changed:b41cbc51-7a4c-4d78-adfd-0a78a0690ee0 for instance 59d9d393-afa0-4c17-a358-09fb4e415b60 on nch1-master-1 Sep 27 10:14:32.780197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-24b7e6cd-f4a2-4006-ac50-0dc63bfd9bfc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:32.794043 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-24b7e6cd-f4a2-4006-ac50-0dc63bfd9bfc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1723 microversion: 2.1 time: 0.842602 Sep 27 10:14:32.795468 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-53929de0-6e09-45fb-a443-1235d2ce82f9 req-72c8e955-2e25-4240-80be-690466eff78e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.115229 Sep 27 10:14:32.796854 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2257/4132] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:31 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1723 bytes in 847 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:32.797619 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1876/4133] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:32.807837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-712aa355-0ff4-4332-a917-00a007573e2c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:32.809853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-712aa355-0ff4-4332-a917-00a007573e2c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:32.824012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-712aa355-0ff4-4332-a917-00a007573e2c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:32.824933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-712aa355-0ff4-4332-a917-00a007573e2c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:32.825603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-712aa355-0ff4-4332-a917-00a007573e2c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:32.826854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-712aa355-0ff4-4332-a917-00a007573e2c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:32.827515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-712aa355-0ff4-4332-a917-00a007573e2c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:32.828107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-712aa355-0ff4-4332-a917-00a007573e2c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:33.104611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-712aa355-0ff4-4332-a917-00a007573e2c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:33.127091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-712aa355-0ff4-4332-a917-00a007573e2c tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 0.328845 Sep 27 10:14:33.130228 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2258/4134] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:32 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:33.701409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6795dab3-3601-4366-996c-1d3fff32f04e tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:33.703010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6795dab3-3601-4366-996c-1d3fff32f04e tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:33.711399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6795dab3-3601-4366-996c-1d3fff32f04e tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:33.712153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6795dab3-3601-4366-996c-1d3fff32f04e tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:33.712823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6795dab3-3601-4366-996c-1d3fff32f04e tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:33.713767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6795dab3-3601-4366-996c-1d3fff32f04e tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:33.714233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6795dab3-3601-4366-996c-1d3fff32f04e tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:33.714669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6795dab3-3601-4366-996c-1d3fff32f04e tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:33.826377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-84bef9e2-d060-4a18-ae96-b5e10fa5b5bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:33.827007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-84bef9e2-d060-4a18-ae96-b5e10fa5b5bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:33.836795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bef9e2-d060-4a18-ae96-b5e10fa5b5bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:33.837735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bef9e2-d060-4a18-ae96-b5e10fa5b5bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:33.838492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bef9e2-d060-4a18-ae96-b5e10fa5b5bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:33.839212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bef9e2-d060-4a18-ae96-b5e10fa5b5bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:33.839846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bef9e2-d060-4a18-ae96-b5e10fa5b5bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:33.840267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84bef9e2-d060-4a18-ae96-b5e10fa5b5bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:34.007492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-84bef9e2-d060-4a18-ae96-b5e10fa5b5bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:34.017782 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-84bef9e2-d060-4a18-ae96-b5e10fa5b5bc tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1723 microversion: 2.1 time: 0.198429 Sep 27 10:14:34.018449 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2259/4135] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:33 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1723 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:34.156109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a625dcee-3588-474f-85af-e1b73507f3c0 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:34.156985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a625dcee-3588-474f-85af-e1b73507f3c0 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:34.177029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a625dcee-3588-474f-85af-e1b73507f3c0 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:34.177535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a625dcee-3588-474f-85af-e1b73507f3c0 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:34.180751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6795dab3-3601-4366-996c-1d3fff32f04e tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:34.185121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a625dcee-3588-474f-85af-e1b73507f3c0 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:34.191663 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6795dab3-3601-4366-996c-1d3fff32f04e tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1325 microversion: 2.1 time: 0.495451 Sep 27 10:14:34.192621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a625dcee-3588-474f-85af-e1b73507f3c0 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:34.207203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a625dcee-3588-474f-85af-e1b73507f3c0 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:34.208285 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1877/4136] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:33 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1325 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:34.217637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a625dcee-3588-474f-85af-e1b73507f3c0 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:34.400397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a625dcee-3588-474f-85af-e1b73507f3c0 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:34.412960 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a625dcee-3588-474f-85af-e1b73507f3c0 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 0.261975 Sep 27 10:14:34.414799 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2260/4137] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:34 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:34.995110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e94bd755-239c-4120-a070-71d0beff876a req-c0e7ed85-9eab-4b20-bd1e-af83374ba9ec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ac85843-17b5-4435-b528-cb0d37b70631", "name": "network-vif-plugged", "status": "completed", "tag": "3a6fbbd7-4580-4281-aac2-fa84df8853b2"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:35.017859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e94bd755-239c-4120-a070-71d0beff876a req-c0e7ed85-9eab-4b20-bd1e-af83374ba9ec service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:35.018353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e94bd755-239c-4120-a070-71d0beff876a req-c0e7ed85-9eab-4b20-bd1e-af83374ba9ec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:35.021461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e94bd755-239c-4120-a070-71d0beff876a req-c0e7ed85-9eab-4b20-bd1e-af83374ba9ec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:35.043881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3bf76078-789d-4eb1-bf36-164163c8a415 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:35.045051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3bf76078-789d-4eb1-bf36-164163c8a415 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:35.055575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf76078-789d-4eb1-bf36-164163c8a415 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:35.056234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf76078-789d-4eb1-bf36-164163c8a415 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:35.065546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf76078-789d-4eb1-bf36-164163c8a415 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:35.065691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e94bd755-239c-4120-a070-71d0beff876a req-c0e7ed85-9eab-4b20-bd1e-af83374ba9ec service nova] Creating event network-vif-plugged:3a6fbbd7-4580-4281-aac2-fa84df8853b2 for instance 4ac85843-17b5-4435-b528-cb0d37b70631 on nch2-master-1 Sep 27 10:14:35.066499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf76078-789d-4eb1-bf36-164163c8a415 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:35.066991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf76078-789d-4eb1-bf36-164163c8a415 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:35.067433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf76078-789d-4eb1-bf36-164163c8a415 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:35.079305 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e94bd755-239c-4120-a070-71d0beff876a req-c0e7ed85-9eab-4b20-bd1e-af83374ba9ec service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.086567 Sep 27 10:14:35.080793 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1878/4138] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:35.248501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-83f8dd0c-5e3b-4971-a35b-17b9e954de73 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:35.249524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-83f8dd0c-5e3b-4971-a35b-17b9e954de73 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:35.262442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f8dd0c-5e3b-4971-a35b-17b9e954de73 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:35.277128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f8dd0c-5e3b-4971-a35b-17b9e954de73 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:35.281137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f8dd0c-5e3b-4971-a35b-17b9e954de73 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:35.283833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f8dd0c-5e3b-4971-a35b-17b9e954de73 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:35.285595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f8dd0c-5e3b-4971-a35b-17b9e954de73 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:35.288286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f8dd0c-5e3b-4971-a35b-17b9e954de73 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:35.437556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3bf76078-789d-4eb1-bf36-164163c8a415 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:35.451581 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3bf76078-789d-4eb1-bf36-164163c8a415 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1723 microversion: 2.1 time: 0.411989 Sep 27 10:14:35.452630 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2261/4139] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:35 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1723 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:35.458082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a9755faf-709a-477b-aeea-13eae353c085 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:35.458795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a9755faf-709a-477b-aeea-13eae353c085 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:35.466625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9755faf-709a-477b-aeea-13eae353c085 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:35.467434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9755faf-709a-477b-aeea-13eae353c085 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:35.468062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9755faf-709a-477b-aeea-13eae353c085 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:35.469081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9755faf-709a-477b-aeea-13eae353c085 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:35.471151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9755faf-709a-477b-aeea-13eae353c085 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:35.471887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9755faf-709a-477b-aeea-13eae353c085 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:35.683119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a9755faf-709a-477b-aeea-13eae353c085 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:35.693430 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a9755faf-709a-477b-aeea-13eae353c085 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 0.239192 Sep 27 10:14:35.700813 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2262/4140] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:35 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:35.862580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-83f8dd0c-5e3b-4971-a35b-17b9e954de73 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:35.871482 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-83f8dd0c-5e3b-4971-a35b-17b9e954de73 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1325 microversion: 2.1 time: 0.632775 Sep 27 10:14:35.887797 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1879/4141] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:35 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1325 bytes in 650 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:36.479711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-44bc72fa-c7d1-4dec-bde6-0afdff86ae0d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:36.481668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-44bc72fa-c7d1-4dec-bde6-0afdff86ae0d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:36.504018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44bc72fa-c7d1-4dec-bde6-0afdff86ae0d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:36.508482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44bc72fa-c7d1-4dec-bde6-0afdff86ae0d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:36.509346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44bc72fa-c7d1-4dec-bde6-0afdff86ae0d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:36.510504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44bc72fa-c7d1-4dec-bde6-0afdff86ae0d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:36.513242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44bc72fa-c7d1-4dec-bde6-0afdff86ae0d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:36.513997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44bc72fa-c7d1-4dec-bde6-0afdff86ae0d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:36.725495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-20d3ba0d-6a90-4284-a029-72e9762731f7 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:36.726998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-20d3ba0d-6a90-4284-a029-72e9762731f7 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:36.737028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20d3ba0d-6a90-4284-a029-72e9762731f7 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:36.737560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20d3ba0d-6a90-4284-a029-72e9762731f7 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:36.738433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20d3ba0d-6a90-4284-a029-72e9762731f7 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:36.739746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20d3ba0d-6a90-4284-a029-72e9762731f7 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:36.741057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20d3ba0d-6a90-4284-a029-72e9762731f7 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:36.741749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20d3ba0d-6a90-4284-a029-72e9762731f7 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:36.797247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-44bc72fa-c7d1-4dec-bde6-0afdff86ae0d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:36.816925 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-44bc72fa-c7d1-4dec-bde6-0afdff86ae0d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1723 microversion: 2.1 time: 0.333517 Sep 27 10:14:36.816925 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2263/4142] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:36 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1723 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:36.897681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e418ea3d-c855-4c51-9696-e3f48c7da23f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:36.898600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e418ea3d-c855-4c51-9696-e3f48c7da23f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:36.906851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418ea3d-c855-4c51-9696-e3f48c7da23f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:36.907859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418ea3d-c855-4c51-9696-e3f48c7da23f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:36.910158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418ea3d-c855-4c51-9696-e3f48c7da23f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:36.911287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418ea3d-c855-4c51-9696-e3f48c7da23f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:36.912149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418ea3d-c855-4c51-9696-e3f48c7da23f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:36.915122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418ea3d-c855-4c51-9696-e3f48c7da23f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:37.133467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e418ea3d-c855-4c51-9696-e3f48c7da23f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:37.145667 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e418ea3d-c855-4c51-9696-e3f48c7da23f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1325 microversion: 2.1 time: 0.254055 Sep 27 10:14:37.146850 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2264/4143] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:36 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1325 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:37.298455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-20d3ba0d-6a90-4284-a029-72e9762731f7 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:37.303609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:37.321952 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-20d3ba0d-6a90-4284-a029-72e9762731f7 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1317 microversion: 2.1 time: 0.607227 Sep 27 10:14:37.325506 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1880/4144] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:36 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1317 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:14:37.836534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f00618b2-fe38-47a7-9b5e-a1e645dd5ebd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:37.837990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f00618b2-fe38-47a7-9b5e-a1e645dd5ebd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:37.860625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00618b2-fe38-47a7-9b5e-a1e645dd5ebd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:37.861118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00618b2-fe38-47a7-9b5e-a1e645dd5ebd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:37.861561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00618b2-fe38-47a7-9b5e-a1e645dd5ebd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:37.862286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00618b2-fe38-47a7-9b5e-a1e645dd5ebd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:37.869687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00618b2-fe38-47a7-9b5e-a1e645dd5ebd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:37.870345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f00618b2-fe38-47a7-9b5e-a1e645dd5ebd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:38.153840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f00618b2-fe38-47a7-9b5e-a1e645dd5ebd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:38.166518 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f00618b2-fe38-47a7-9b5e-a1e645dd5ebd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1723 microversion: 2.1 time: 0.335284 Sep 27 10:14:38.167604 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2265/4145] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:37 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1723 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:38.190337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1f5f42b8-edc1-4e0e-94c0-90a89c7a797d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:38.191417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1f5f42b8-edc1-4e0e-94c0-90a89c7a797d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:38.199619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f5f42b8-edc1-4e0e-94c0-90a89c7a797d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:38.200411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f5f42b8-edc1-4e0e-94c0-90a89c7a797d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:38.201087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f5f42b8-edc1-4e0e-94c0-90a89c7a797d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:38.202262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f5f42b8-edc1-4e0e-94c0-90a89c7a797d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:38.202894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f5f42b8-edc1-4e0e-94c0-90a89c7a797d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:38.203529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f5f42b8-edc1-4e0e-94c0-90a89c7a797d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:38.323856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-\u20a1-854116495", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "662e23c5-f7e1-4e14-af6c-49d672076632"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:38.392801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] validate_networks() for [('662e23c5-f7e1-4e14-af6c-49d672076632', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:14:38.677960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1f5f42b8-edc1-4e0e-94c0-90a89c7a797d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:38.691874 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1f5f42b8-edc1-4e0e-94c0-90a89c7a797d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1325 microversion: 2.1 time: 0.509500 Sep 27 10:14:38.693189 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2266/4146] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:38 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1325 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:38.700691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-94bebb9f-f901-48cb-94f5-61372acbb2c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:38.701621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-94bebb9f-f901-48cb-94f5-61372acbb2c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:38.710225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94bebb9f-f901-48cb-94f5-61372acbb2c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:38.710969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94bebb9f-f901-48cb-94f5-61372acbb2c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:38.711574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94bebb9f-f901-48cb-94f5-61372acbb2c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:38.712432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94bebb9f-f901-48cb-94f5-61372acbb2c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:38.713486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94bebb9f-f901-48cb-94f5-61372acbb2c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:38.714276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94bebb9f-f901-48cb-94f5-61372acbb2c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:38.963576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-94bebb9f-f901-48cb-94f5-61372acbb2c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:38.989794 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-94bebb9f-f901-48cb-94f5-61372acbb2c2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 200 len: 1505 microversion: 2.1 time: 0.293892 Sep 27 10:14:38.997275 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2267/4147] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:38 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 1505 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:39.015152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-187316d7-61df-4c08-a187-6e6b326c070d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:39.017228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-187316d7-61df-4c08-a187-6e6b326c070d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: af870ec8-bdb2-4a04-9688-ab4bb5ae8331] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:39.024315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-187316d7-61df-4c08-a187-6e6b326c070d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:39.025362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-187316d7-61df-4c08-a187-6e6b326c070d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:39.026289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-187316d7-61df-4c08-a187-6e6b326c070d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:39.027532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-187316d7-61df-4c08-a187-6e6b326c070d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:39.028166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-187316d7-61df-4c08-a187-6e6b326c070d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:39.028793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-187316d7-61df-4c08-a187-6e6b326c070d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:39.457523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:14:39.458553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:14:39.459598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:14:39.460474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:14:39.461144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:14:39.487549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:39.488100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:39.489391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:39.490184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:39.493827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:39.495191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:39.550424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:14:39.562759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for user a7b8599d0b40424087e52d8e35b71ec4 and project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:14:39.591003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:14:39.597076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:14:39.704030 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [ca62d266-f6ab-4184-ab27-495ae2071470] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 27 10:14:39.797965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-187316d7-61df-4c08-a187-6e6b326c070d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:39.812967 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-187316d7-61df-4c08-a187-6e6b326c070d tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331" status: 200 len: 1505 microversion: 2.1 time: 0.802044 Sep 27 10:14:39.832005 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2268/4148] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:39 2022] GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331 => generated 1505 bytes in 816 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:14:39.844752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2131d6d3-bd23-48db-8919-4099138466bd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:39.845336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2131d6d3-bd23-48db-8919-4099138466bd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:39.856014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2131d6d3-bd23-48db-8919-4099138466bd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:39.856521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2131d6d3-bd23-48db-8919-4099138466bd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:39.857078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2131d6d3-bd23-48db-8919-4099138466bd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:39.857963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2131d6d3-bd23-48db-8919-4099138466bd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:39.858485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2131d6d3-bd23-48db-8919-4099138466bd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:39.858928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2131d6d3-bd23-48db-8919-4099138466bd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:40.076757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2131d6d3-bd23-48db-8919-4099138466bd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:40.088989 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2131d6d3-bd23-48db-8919-4099138466bd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1723 microversion: 2.1 time: 0.255342 Sep 27 10:14:40.089825 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2269/4149] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:39 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1723 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:40.099339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fdb2b62d-5df3-4954-8c5b-1b30209951a0 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:40.100366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fdb2b62d-5df3-4954-8c5b-1b30209951a0 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:40.109819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdb2b62d-5df3-4954-8c5b-1b30209951a0 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:40.110611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdb2b62d-5df3-4954-8c5b-1b30209951a0 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:40.111209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdb2b62d-5df3-4954-8c5b-1b30209951a0 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:40.111996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdb2b62d-5df3-4954-8c5b-1b30209951a0 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:40.112865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdb2b62d-5df3-4954-8c5b-1b30209951a0 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:40.113452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdb2b62d-5df3-4954-8c5b-1b30209951a0 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:40.406014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fdb2b62d-5df3-4954-8c5b-1b30209951a0 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:40.420497 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fdb2b62d-5df3-4954-8c5b-1b30209951a0 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1325 microversion: 2.1 time: 0.329370 Sep 27 10:14:40.421129 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2270/4150] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:40 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1325 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:40.462164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8614cb55-93d1-4cc3-9f7b-f07a00027e02 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:40.462808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8614cb55-93d1-4cc3-9f7b-f07a00027e02 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: af870ec8-bdb2-4a04-9688-ab4bb5ae8331] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:40.477399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8614cb55-93d1-4cc3-9f7b-f07a00027e02 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:40.478307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8614cb55-93d1-4cc3-9f7b-f07a00027e02 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:40.478917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8614cb55-93d1-4cc3-9f7b-f07a00027e02 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:40.480345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8614cb55-93d1-4cc3-9f7b-f07a00027e02 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:40.480998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8614cb55-93d1-4cc3-9f7b-f07a00027e02 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:40.481641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8614cb55-93d1-4cc3-9f7b-f07a00027e02 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:40.528920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8614cb55-93d1-4cc3-9f7b-f07a00027e02 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: af870ec8-bdb2-4a04-9688-ab4bb5ae8331] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:14:40.705361 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8614cb55-93d1-4cc3-9f7b-f07a00027e02 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "DELETE /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331" status: 204 len: 0 microversion: 2.1 time: 0.258911 Sep 27 10:14:40.711430 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2271/4151] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:14:40 2022] DELETE /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:14:40.713897 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [ca62d266-f6ab-4184-ab27-495ae2071470] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 49484. Sep 27 10:14:40.723297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f31b055-4af6-432c-9807-fad29fdd0c9f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:40.724176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f31b055-4af6-432c-9807-fad29fdd0c9f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:40.727693 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-74972f43-332a-4875-87da-26e5b80222d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.599720 Sep 27 10:14:40.728602 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1881/4152] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:14:38 2022] POST /compute/v2.1/servers => generated 380 bytes in 2602 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:14:40.735444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f31b055-4af6-432c-9807-fad29fdd0c9f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:40.737001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f31b055-4af6-432c-9807-fad29fdd0c9f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:40.738198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f31b055-4af6-432c-9807-fad29fdd0c9f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:40.742144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f31b055-4af6-432c-9807-fad29fdd0c9f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:40.743106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f31b055-4af6-432c-9807-fad29fdd0c9f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:40.743729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f31b055-4af6-432c-9807-fad29fdd0c9f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:40.749983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:40.754004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-78f7aa14-d669-4876-b51d-9dfc8b4d2938 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:40.766001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-78f7aa14-d669-4876-b51d-9dfc8b4d2938 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:40.780087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f31b055-4af6-432c-9807-fad29fdd0c9f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:14:40.891080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-78f7aa14-d669-4876-b51d-9dfc8b4d2938 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:40.899745 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-78f7aa14-d669-4876-b51d-9dfc8b4d2938 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1192 microversion: 2.1 time: 0.162592 Sep 27 10:14:40.910392 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1882/4153] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:40 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1192 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:40.929211 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f31b055-4af6-432c-9807-fad29fdd0c9f tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 204 len: 0 microversion: 2.1 time: 0.215100 Sep 27 10:14:40.929750 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2272/4154] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:14:40 2022] DELETE /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:14:40.939105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0ede41d7-e200-40d2-b2a4-227e490a5976 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:40.939880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0ede41d7-e200-40d2-b2a4-227e490a5976 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: af870ec8-bdb2-4a04-9688-ab4bb5ae8331] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:40.951278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ede41d7-e200-40d2-b2a4-227e490a5976 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:40.951830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ede41d7-e200-40d2-b2a4-227e490a5976 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:40.952280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ede41d7-e200-40d2-b2a4-227e490a5976 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:40.952280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ede41d7-e200-40d2-b2a4-227e490a5976 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:40.952280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ede41d7-e200-40d2-b2a4-227e490a5976 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:40.952280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ede41d7-e200-40d2-b2a4-227e490a5976 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:41.127955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-271ebd88-f1b6-44b5-909f-24e0779baee9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:41.129456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-271ebd88-f1b6-44b5-909f-24e0779baee9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:41.146527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-271ebd88-f1b6-44b5-909f-24e0779baee9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:41.147258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-271ebd88-f1b6-44b5-909f-24e0779baee9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:41.147745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-271ebd88-f1b6-44b5-909f-24e0779baee9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:41.149626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-271ebd88-f1b6-44b5-909f-24e0779baee9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:41.150218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-271ebd88-f1b6-44b5-909f-24e0779baee9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:41.151584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-271ebd88-f1b6-44b5-909f-24e0779baee9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:41.415246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0ede41d7-e200-40d2-b2a4-227e490a5976 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:41.429654 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0ede41d7-e200-40d2-b2a4-227e490a5976 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331" status: 200 len: 1511 microversion: 2.1 time: 0.493029 Sep 27 10:14:41.431547 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1883/4155] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:40 2022] GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331 => generated 1511 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:41.460338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b90adda8-8206-46de-92ef-080149b14b55 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:41.466766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b90adda8-8206-46de-92ef-080149b14b55 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:41.474030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b90adda8-8206-46de-92ef-080149b14b55 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:41.475670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b90adda8-8206-46de-92ef-080149b14b55 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:41.476458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b90adda8-8206-46de-92ef-080149b14b55 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:41.478368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b90adda8-8206-46de-92ef-080149b14b55 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:41.479645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b90adda8-8206-46de-92ef-080149b14b55 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:41.480282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b90adda8-8206-46de-92ef-080149b14b55 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:41.687035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-271ebd88-f1b6-44b5-909f-24e0779baee9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:41.706440 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-271ebd88-f1b6-44b5-909f-24e0779baee9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1723 microversion: 2.1 time: 0.594148 Sep 27 10:14:41.707102 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2273/4156] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:41 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1723 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:41.865226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b90adda8-8206-46de-92ef-080149b14b55 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:41.881155 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b90adda8-8206-46de-92ef-080149b14b55 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1325 microversion: 2.1 time: 0.429821 Sep 27 10:14:41.882763 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1884/4157] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:41 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1325 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:41.938717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fc36f3cd-eb33-4669-b4bc-08e137c22e5f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:41.939411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fc36f3cd-eb33-4669-b4bc-08e137c22e5f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:41.960304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc36f3cd-eb33-4669-b4bc-08e137c22e5f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:41.960839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc36f3cd-eb33-4669-b4bc-08e137c22e5f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:41.961276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc36f3cd-eb33-4669-b4bc-08e137c22e5f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:41.963495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc36f3cd-eb33-4669-b4bc-08e137c22e5f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:41.963736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc36f3cd-eb33-4669-b4bc-08e137c22e5f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:41.964190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc36f3cd-eb33-4669-b4bc-08e137c22e5f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:42.039324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ef4d069b-2774-48d3-86ea-507c3b31f11c req-df5df092-9191-4dfd-9d38-81f39143ff24 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59d9d393-afa0-4c17-a358-09fb4e415b60", "name": "network-vif-plugged", "status": "completed", "tag": "b41cbc51-7a4c-4d78-adfd-0a78a0690ee0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:42.074932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ef4d069b-2774-48d3-86ea-507c3b31f11c req-df5df092-9191-4dfd-9d38-81f39143ff24 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:42.075440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ef4d069b-2774-48d3-86ea-507c3b31f11c req-df5df092-9191-4dfd-9d38-81f39143ff24 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:42.075828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ef4d069b-2774-48d3-86ea-507c3b31f11c req-df5df092-9191-4dfd-9d38-81f39143ff24 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:42.113910 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ef4d069b-2774-48d3-86ea-507c3b31f11c req-df5df092-9191-4dfd-9d38-81f39143ff24 service nova] Creating event network-vif-plugged:b41cbc51-7a4c-4d78-adfd-0a78a0690ee0 for instance 59d9d393-afa0-4c17-a358-09fb4e415b60 on nch1-master-1 Sep 27 10:14:42.123668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc36f3cd-eb33-4669-b4bc-08e137c22e5f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:42.134534 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fc36f3cd-eb33-4669-b4bc-08e137c22e5f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1188 microversion: 2.1 time: 0.199759 Sep 27 10:14:42.136515 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2274/4158] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:41 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1188 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:42.138223 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ef4d069b-2774-48d3-86ea-507c3b31f11c req-df5df092-9191-4dfd-9d38-81f39143ff24 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100061 Sep 27 10:14:42.139196 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1885/4159] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:42.458869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-50148586-4ba6-4ea9-b221-1f03919b3d34 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:42.460277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-50148586-4ba6-4ea9-b221-1f03919b3d34 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: af870ec8-bdb2-4a04-9688-ab4bb5ae8331] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:42.478614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50148586-4ba6-4ea9-b221-1f03919b3d34 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:42.479720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50148586-4ba6-4ea9-b221-1f03919b3d34 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:42.480385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50148586-4ba6-4ea9-b221-1f03919b3d34 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:42.481401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50148586-4ba6-4ea9-b221-1f03919b3d34 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:42.482072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50148586-4ba6-4ea9-b221-1f03919b3d34 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:42.482703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50148586-4ba6-4ea9-b221-1f03919b3d34 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:42.735924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-030035e6-5bc2-45b0-84c5-f121024cb3dd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:42.736680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-030035e6-5bc2-45b0-84c5-f121024cb3dd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:42.738205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-50148586-4ba6-4ea9-b221-1f03919b3d34 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:42.748572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-030035e6-5bc2-45b0-84c5-f121024cb3dd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:42.748930 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-50148586-4ba6-4ea9-b221-1f03919b3d34 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331" status: 200 len: 1511 microversion: 2.1 time: 0.294727 Sep 27 10:14:42.749196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-030035e6-5bc2-45b0-84c5-f121024cb3dd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:42.749680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-030035e6-5bc2-45b0-84c5-f121024cb3dd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:42.750510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-030035e6-5bc2-45b0-84c5-f121024cb3dd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:42.761080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-030035e6-5bc2-45b0-84c5-f121024cb3dd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:42.762131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-030035e6-5bc2-45b0-84c5-f121024cb3dd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:42.762277 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2275/4160] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:42 2022] GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331 => generated 1511 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:42.910812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ccb54e48-d08b-4ca9-bd59-ab6ffb5d5cc5 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:42.911730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ccb54e48-d08b-4ca9-bd59-ab6ffb5d5cc5 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:42.922962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccb54e48-d08b-4ca9-bd59-ab6ffb5d5cc5 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:42.924274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccb54e48-d08b-4ca9-bd59-ab6ffb5d5cc5 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:42.926039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccb54e48-d08b-4ca9-bd59-ab6ffb5d5cc5 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:42.928434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccb54e48-d08b-4ca9-bd59-ab6ffb5d5cc5 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:42.929144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccb54e48-d08b-4ca9-bd59-ab6ffb5d5cc5 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:42.930432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccb54e48-d08b-4ca9-bd59-ab6ffb5d5cc5 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:42.981744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-030035e6-5bc2-45b0-84c5-f121024cb3dd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:42.996565 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-030035e6-5bc2-45b0-84c5-f121024cb3dd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1723 microversion: 2.1 time: 0.265082 Sep 27 10:14:42.997427 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1886/4161] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:42 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1723 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:14:43.165783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7d5c89d1-224e-428f-b46b-775742b53067 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:43.166722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7d5c89d1-224e-428f-b46b-775742b53067 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:43.181466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d5c89d1-224e-428f-b46b-775742b53067 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:43.183665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d5c89d1-224e-428f-b46b-775742b53067 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:43.184145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d5c89d1-224e-428f-b46b-775742b53067 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:43.185007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d5c89d1-224e-428f-b46b-775742b53067 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:43.185519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d5c89d1-224e-428f-b46b-775742b53067 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:43.187137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d5c89d1-224e-428f-b46b-775742b53067 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:43.245900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ccb54e48-d08b-4ca9-bd59-ab6ffb5d5cc5 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:43.260007 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ccb54e48-d08b-4ca9-bd59-ab6ffb5d5cc5 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1325 microversion: 2.1 time: 0.353197 Sep 27 10:14:43.264303 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2276/4162] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:42 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1325 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:43.555885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7d5c89d1-224e-428f-b46b-775742b53067 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:43.567151 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7d5c89d1-224e-428f-b46b-775742b53067 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1252 microversion: 2.1 time: 0.404606 Sep 27 10:14:43.573608 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1887/4163] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:43 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1252 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:43.774734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e6ef5705-8c05-4d3e-9f3b-8c970da2f675 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:43.775381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e6ef5705-8c05-4d3e-9f3b-8c970da2f675 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: af870ec8-bdb2-4a04-9688-ab4bb5ae8331] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:43.791416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ef5705-8c05-4d3e-9f3b-8c970da2f675 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:43.791913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ef5705-8c05-4d3e-9f3b-8c970da2f675 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:43.792362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ef5705-8c05-4d3e-9f3b-8c970da2f675 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:43.805878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ef5705-8c05-4d3e-9f3b-8c970da2f675 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:43.806165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ef5705-8c05-4d3e-9f3b-8c970da2f675 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:43.806691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ef5705-8c05-4d3e-9f3b-8c970da2f675 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:44.024443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e6ef5705-8c05-4d3e-9f3b-8c970da2f675 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:44.033204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3c3ec44d-c88c-4c6c-8773-e9bc28313910 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:44.034575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3c3ec44d-c88c-4c6c-8773-e9bc28313910 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:44.047458 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e6ef5705-8c05-4d3e-9f3b-8c970da2f675 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331" status: 200 len: 1511 microversion: 2.1 time: 0.276981 Sep 27 10:14:44.048344 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2277/4164] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:43 2022] GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331 => generated 1511 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:44.050367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c3ec44d-c88c-4c6c-8773-e9bc28313910 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:44.051127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c3ec44d-c88c-4c6c-8773-e9bc28313910 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:44.051724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c3ec44d-c88c-4c6c-8773-e9bc28313910 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:44.052782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c3ec44d-c88c-4c6c-8773-e9bc28313910 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:44.053589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c3ec44d-c88c-4c6c-8773-e9bc28313910 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:44.054172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c3ec44d-c88c-4c6c-8773-e9bc28313910 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:44.290635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8b0b4c13-0ed1-4be6-baeb-067015293c59 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:44.291550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8b0b4c13-0ed1-4be6-baeb-067015293c59 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:44.302524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b0b4c13-0ed1-4be6-baeb-067015293c59 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:44.304512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b0b4c13-0ed1-4be6-baeb-067015293c59 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:44.305328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b0b4c13-0ed1-4be6-baeb-067015293c59 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:44.311432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b0b4c13-0ed1-4be6-baeb-067015293c59 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:44.312355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b0b4c13-0ed1-4be6-baeb-067015293c59 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:44.313110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b0b4c13-0ed1-4be6-baeb-067015293c59 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:44.563063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8b0b4c13-0ed1-4be6-baeb-067015293c59 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:44.576455 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8b0b4c13-0ed1-4be6-baeb-067015293c59 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1523 microversion: 2.1 time: 0.295083 Sep 27 10:14:44.582518 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2278/4165] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:44 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1523 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:44.605774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-59b176a2-4b16-4a29-9b9b-fbc5bc424eef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:44.606741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-59b176a2-4b16-4a29-9b9b-fbc5bc424eef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:44.625160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b176a2-4b16-4a29-9b9b-fbc5bc424eef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:44.625999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b176a2-4b16-4a29-9b9b-fbc5bc424eef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:44.626530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b176a2-4b16-4a29-9b9b-fbc5bc424eef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:44.627508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b176a2-4b16-4a29-9b9b-fbc5bc424eef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:44.628178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b176a2-4b16-4a29-9b9b-fbc5bc424eef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:44.628951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59b176a2-4b16-4a29-9b9b-fbc5bc424eef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:44.736357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59b176a2-4b16-4a29-9b9b-fbc5bc424eef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:44.753351 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-59b176a2-4b16-4a29-9b9b-fbc5bc424eef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1250 microversion: 2.1 time: 0.154862 Sep 27 10:14:44.754641 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2279/4166] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:44 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1250 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:14:44.768302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:44.771939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:44.782814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:44.784986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:44.785864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:44.786480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:44.788481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:44.789401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:44.792574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:44.848251 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] HTTP exception thrown: Action 999 not found Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Request method failure captured: Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60/os-instance-actions/999 HTTP/1.1 Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsz6PLM9dpPu51tGocEF6MbfdcT4o6g5_j6G9tKh3OtmxJOxCpWeUdhEj00-cfx4R5mxwLOgrVD6Qr72H5IjVpztuTavzw-rZicoZhcIVjIgZ1iBPTD-N8m77pt_aZgdGWyIFtWgKmHuyon1mDmPFFVEhrHWOSyrnFPL4hldN3EKxUg Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1e7e90f89b4844dbafdc1c5e5f17b407 Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-InstanceActionsNegativeTestJSON-1110616879 Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1e7e90f89b4844dbafdc1c5e5f17b407"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1e7e90f89b4844dbafdc1c5e5f17b407"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1e7e90f89b4844dbafdc1c5e5f17b407"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1e7e90f89b4844dbafdc1c5e5f17b407"}]}] Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-InstanceActionsNegativeTestJSON-1110616879 Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1e7e90f89b4844dbafdc1c5e5f17b407 Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-InstanceActionsNegativeTestJSON-1110616879 Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-InstanceActionsNegativeTestJSON-1110616879-project Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: c28c5e12dbb444cab4e376a3015c6bd1 Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-InstanceActionsNegativeTestJSON-1110616879-project Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: exception: Action 999 not found Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'server_id': '59d9d393-afa0-4c17-a358-09fb4e415b60', 'id': '999'} Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_actions.py", line 152, in show Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Action 999 not found Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Action 999 not found Sep 27 10:14:44.856332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:44.860099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Returning 404 to user: Action 999 not found {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:14:44.860918 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-acde7b14-5d3e-4a7a-ae7f-225377b2767d tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60/os-instance-actions/999" status: 404 len: 66 microversion: 2.1 time: 0.103199 Sep 27 10:14:44.861485 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2280/4167] 10.222.0.26 () {58 vars in 1308 bytes} [Tue Sep 27 10:14:44 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60/os-instance-actions/999 => generated 66 bytes in 106 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:14:44.878060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d11e8bb3-0a12-490d-8206-126ac17a8303 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '.version_select at 0x7f7f63715040>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:44.878943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d11e8bb3-0a12-490d-8206-126ac17a8303 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: c6bcd2ec-4da3-4d9f-8c4e-0d64cdb05755] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:44.888832 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-d11e8bb3-0a12-490d-8206-126ac17a8303 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] HTTP exception thrown: Instance c6bcd2ec-4da3-4d9f-8c4e-0d64cdb05755 could not be found. Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d11e8bb3-0a12-490d-8206-126ac17a8303 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Request method failure captured: Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/c6bcd2ec-4da3-4d9f-8c4e-0d64cdb05755/os-instance-actions HTTP/1.1 Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsz6PLM9dpPu51tGocEF6MbfdcT4o6g5_j6G9tKh3OtmxJOxCpWeUdhEj00-cfx4R5mxwLOgrVD6Qr72H5IjVpztuTavzw-rZicoZhcIVjIgZ1iBPTD-N8m77pt_aZgdGWyIFtWgKmHuyon1mDmPFFVEhrHWOSyrnFPL4hldN3EKxUg Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1e7e90f89b4844dbafdc1c5e5f17b407 Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-InstanceActionsNegativeTestJSON-1110616879 Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1e7e90f89b4844dbafdc1c5e5f17b407"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1e7e90f89b4844dbafdc1c5e5f17b407"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1e7e90f89b4844dbafdc1c5e5f17b407"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1e7e90f89b4844dbafdc1c5e5f17b407"}]}] Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-InstanceActionsNegativeTestJSON-1110616879 Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1e7e90f89b4844dbafdc1c5e5f17b407 Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-InstanceActionsNegativeTestJSON-1110616879 Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-InstanceActionsNegativeTestJSON-1110616879-project Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: c28c5e12dbb444cab4e376a3015c6bd1 Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-InstanceActionsNegativeTestJSON-1110616879-project Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance c6bcd2ec-4da3-4d9f-8c4e-0d64cdb05755 could not be found. Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'server_id': 'c6bcd2ec-4da3-4d9f-8c4e-0d64cdb05755'} Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c6bcd2ec-4da3-4d9f-8c4e-0d64cdb05755 could not be found. Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_actions.py", line 89, in index Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(req, context, server_id) Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/instance_actions.py", line 77, in _get_instance Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return common.get_instance(self.compute_api, context, server_id) Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:14:44.891214 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:14:44.895198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c6bcd2ec-4da3-4d9f-8c4e-0d64cdb05755 could not be found. Sep 27 10:14:44.895198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:44.895198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:14:44.895198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:44.895198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:44.895198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:44.895198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:44.895198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:14:44.895198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:14:44.895198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c6bcd2ec-4da3-4d9f-8c4e-0d64cdb05755 could not be found. Sep 27 10:14:44.895198 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:44.896284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d11e8bb3-0a12-490d-8206-126ac17a8303 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Returning 404 to user: Instance c6bcd2ec-4da3-4d9f-8c4e-0d64cdb05755 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:14:44.898468 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d11e8bb3-0a12-490d-8206-126ac17a8303 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/c6bcd2ec-4da3-4d9f-8c4e-0d64cdb05755/os-instance-actions" status: 404 len: 111 microversion: 2.1 time: 0.026804 Sep 27 10:14:44.899625 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2281/4168] 10.222.0.26 () {58 vars in 1296 bytes} [Tue Sep 27 10:14:44 2022] GET /compute/v2.1/servers/c6bcd2ec-4da3-4d9f-8c4e-0d64cdb05755/os-instance-actions => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:14:44.913460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-87a323ca-59f5-46eb-a201-70675b4fa105 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:44.915583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-87a323ca-59f5-46eb-a201-70675b4fa105 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:44.925186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87a323ca-59f5-46eb-a201-70675b4fa105 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:44.927180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87a323ca-59f5-46eb-a201-70675b4fa105 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:44.927844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87a323ca-59f5-46eb-a201-70675b4fa105 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:44.929394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87a323ca-59f5-46eb-a201-70675b4fa105 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:44.929590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c3ec44d-c88c-4c6c-8773-e9bc28313910 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:44.930444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87a323ca-59f5-46eb-a201-70675b4fa105 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:44.931066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87a323ca-59f5-46eb-a201-70675b4fa105 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:44.942908 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3c3ec44d-c88c-4c6c-8773-e9bc28313910 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1723 microversion: 2.1 time: 0.921020 Sep 27 10:14:44.944941 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1888/4169] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:44 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1723 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:44.983896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-87a323ca-59f5-46eb-a201-70675b4fa105 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:14:45.076190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7aac7fc1-0f3d-4f94-83b1-2ac5f8b67653 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:45.076941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7aac7fc1-0f3d-4f94-83b1-2ac5f8b67653 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: af870ec8-bdb2-4a04-9688-ab4bb5ae8331] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:45.091490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aac7fc1-0f3d-4f94-83b1-2ac5f8b67653 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:45.092340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aac7fc1-0f3d-4f94-83b1-2ac5f8b67653 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:45.093091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aac7fc1-0f3d-4f94-83b1-2ac5f8b67653 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:45.094050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aac7fc1-0f3d-4f94-83b1-2ac5f8b67653 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:45.094665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aac7fc1-0f3d-4f94-83b1-2ac5f8b67653 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:45.095427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7aac7fc1-0f3d-4f94-83b1-2ac5f8b67653 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:45.146015 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-87a323ca-59f5-46eb-a201-70675b4fa105 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "DELETE /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 204 len: 0 microversion: 2.1 time: 0.238752 Sep 27 10:14:45.151583 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2282/4170] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:14:44 2022] DELETE /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:14:45.157560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bdf1ff82-b489-4090-be46-c94de8518b43 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:45.158866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bdf1ff82-b489-4090-be46-c94de8518b43 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:45.166581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdf1ff82-b489-4090-be46-c94de8518b43 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:45.167394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdf1ff82-b489-4090-be46-c94de8518b43 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:45.176568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdf1ff82-b489-4090-be46-c94de8518b43 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:45.180740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdf1ff82-b489-4090-be46-c94de8518b43 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:45.181252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdf1ff82-b489-4090-be46-c94de8518b43 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:45.181809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdf1ff82-b489-4090-be46-c94de8518b43 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:45.470367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bdf1ff82-b489-4090-be46-c94de8518b43 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:45.485967 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bdf1ff82-b489-4090-be46-c94de8518b43 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1529 microversion: 2.1 time: 0.332784 Sep 27 10:14:45.487582 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2283/4171] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:45 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1529 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:45.503602 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 10:14:45.505788 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 10:14:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331 (ip 10.222.0.26) !!! Sep 27 10:14:45.778671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3ee2d07-faea-4767-8c1b-959769ed559d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:45.779837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d3ee2d07-faea-4767-8c1b-959769ed559d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:45.790515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ee2d07-faea-4767-8c1b-959769ed559d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:45.793796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ee2d07-faea-4767-8c1b-959769ed559d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:45.795301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ee2d07-faea-4767-8c1b-959769ed559d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:45.797566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ee2d07-faea-4767-8c1b-959769ed559d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:45.798515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ee2d07-faea-4767-8c1b-959769ed559d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:45.799666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ee2d07-faea-4767-8c1b-959769ed559d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:45.905982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7aac7fc1-0f3d-4f94-83b1-2ac5f8b67653 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:45.916465 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7aac7fc1-0f3d-4f94-83b1-2ac5f8b67653 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331" status: 200 len: 1511 microversion: 2.1 time: 0.846901 Sep 27 10:14:45.925487 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1889/4172] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:45 2022] GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331 => generated 1511 bytes in 857 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:45.951034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d3ee2d07-faea-4767-8c1b-959769ed559d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:45.965809 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d3ee2d07-faea-4767-8c1b-959769ed559d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1250 microversion: 2.1 time: 0.191626 Sep 27 10:14:45.970624 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2284/4173] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:45 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1250 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:14:45.972327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-317da84e-ecb4-491e-907c-7b6c483a992b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:45.973161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-317da84e-ecb4-491e-907c-7b6c483a992b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:45.985932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-317da84e-ecb4-491e-907c-7b6c483a992b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:45.986787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-317da84e-ecb4-491e-907c-7b6c483a992b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:45.987479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-317da84e-ecb4-491e-907c-7b6c483a992b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:45.988442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-317da84e-ecb4-491e-907c-7b6c483a992b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:45.989257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-317da84e-ecb4-491e-907c-7b6c483a992b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:45.989898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-317da84e-ecb4-491e-907c-7b6c483a992b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:46.458355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-317da84e-ecb4-491e-907c-7b6c483a992b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:46.473916 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-317da84e-ecb4-491e-907c-7b6c483a992b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1723 microversion: 2.1 time: 0.505580 Sep 27 10:14:46.475218 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1890/4174] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:45 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1723 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:14:46.524084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0854e0b7-759a-4be3-af37-051b650ede90 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:46.527477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0854e0b7-759a-4be3-af37-051b650ede90 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:46.544053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0854e0b7-759a-4be3-af37-051b650ede90 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:46.545194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0854e0b7-759a-4be3-af37-051b650ede90 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:46.546099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0854e0b7-759a-4be3-af37-051b650ede90 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:46.547683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0854e0b7-759a-4be3-af37-051b650ede90 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:46.549131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0854e0b7-759a-4be3-af37-051b650ede90 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:46.549850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0854e0b7-759a-4be3-af37-051b650ede90 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:46.950621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-99dd160f-6d2b-495b-8ddd-bda6a91e1542 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:46.951329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-99dd160f-6d2b-495b-8ddd-bda6a91e1542 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: af870ec8-bdb2-4a04-9688-ab4bb5ae8331] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:46.962572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99dd160f-6d2b-495b-8ddd-bda6a91e1542 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:46.964581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99dd160f-6d2b-495b-8ddd-bda6a91e1542 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:46.965122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99dd160f-6d2b-495b-8ddd-bda6a91e1542 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:46.965836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99dd160f-6d2b-495b-8ddd-bda6a91e1542 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:46.966303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99dd160f-6d2b-495b-8ddd-bda6a91e1542 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:46.970001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99dd160f-6d2b-495b-8ddd-bda6a91e1542 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:46.975944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0854e0b7-759a-4be3-af37-051b650ede90 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:46.985851 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0854e0b7-759a-4be3-af37-051b650ede90 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1529 microversion: 2.1 time: 0.477500 Sep 27 10:14:46.990224 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2285/4175] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:46 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1529 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:46.998447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-72c92c07-a01f-4b93-b8f9-6e55ba90e575 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:46.999286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-72c92c07-a01f-4b93-b8f9-6e55ba90e575 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:47.014224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c92c07-a01f-4b93-b8f9-6e55ba90e575 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:47.014729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c92c07-a01f-4b93-b8f9-6e55ba90e575 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:47.015175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c92c07-a01f-4b93-b8f9-6e55ba90e575 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:47.015905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c92c07-a01f-4b93-b8f9-6e55ba90e575 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:47.016374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c92c07-a01f-4b93-b8f9-6e55ba90e575 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:47.016839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72c92c07-a01f-4b93-b8f9-6e55ba90e575 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:47.049368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:47.128641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-72c92c07-a01f-4b93-b8f9-6e55ba90e575 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:47.143732 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-72c92c07-a01f-4b93-b8f9-6e55ba90e575 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1250 microversion: 2.1 time: 0.149655 Sep 27 10:14:47.144345 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2286/4176] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:46 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1250 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:47.258004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6936d136-4e60-4a92-87b7-39a1426706b8 req-2886db26-11ee-4cb2-9883-7ae30c3f4ffe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ac85843-17b5-4435-b528-cb0d37b70631", "name": "network-vif-deleted", "tag": "3a6fbbd7-4580-4281-aac2-fa84df8853b2"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:47.281411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6936d136-4e60-4a92-87b7-39a1426706b8 req-2886db26-11ee-4cb2-9883-7ae30c3f4ffe service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:47.289367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6936d136-4e60-4a92-87b7-39a1426706b8 req-2886db26-11ee-4cb2-9883-7ae30c3f4ffe service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:47.289854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6936d136-4e60-4a92-87b7-39a1426706b8 req-2886db26-11ee-4cb2-9883-7ae30c3f4ffe service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:47.313076 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6936d136-4e60-4a92-87b7-39a1426706b8 req-2886db26-11ee-4cb2-9883-7ae30c3f4ffe service nova] Creating event network-vif-deleted:3a6fbbd7-4580-4281-aac2-fa84df8853b2 for instance 4ac85843-17b5-4435-b528-cb0d37b70631 on nch2-master-1 Sep 27 10:14:47.321068 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6936d136-4e60-4a92-87b7-39a1426706b8 req-2886db26-11ee-4cb2-9883-7ae30c3f4ffe service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068475 Sep 27 10:14:47.323470 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2287/4177] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:47.347445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-99dd160f-6d2b-495b-8ddd-bda6a91e1542 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a5e050c7389844a78199faef5a0c9da4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '326c35c0bc52463e95a56ba3371a276a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:47.371944 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-99dd160f-6d2b-495b-8ddd-bda6a91e1542 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331" status: 200 len: 1469 microversion: 2.1 time: 0.426114 Sep 27 10:14:47.419098 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1891/4178] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:46 2022] GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331 => generated 1469 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:47.505820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-167777ae-ea77-44b0-8fea-8b5963cbe47e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:47.509848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-167777ae-ea77-44b0-8fea-8b5963cbe47e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:47.525411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167777ae-ea77-44b0-8fea-8b5963cbe47e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:47.527519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167777ae-ea77-44b0-8fea-8b5963cbe47e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:47.528483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167777ae-ea77-44b0-8fea-8b5963cbe47e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:47.529665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167777ae-ea77-44b0-8fea-8b5963cbe47e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:47.530281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167777ae-ea77-44b0-8fea-8b5963cbe47e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:47.530888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167777ae-ea77-44b0-8fea-8b5963cbe47e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:47.816236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-959426810", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "925eede8-afc8-48d1-a08f-9f66e37bf771"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:47.899617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-167777ae-ea77-44b0-8fea-8b5963cbe47e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:47.916449 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-167777ae-ea77-44b0-8fea-8b5963cbe47e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1700 microversion: 2.1 time: 0.417915 Sep 27 10:14:47.918022 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2288/4179] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:47 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1700 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:47.938500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] validate_networks() for [('925eede8-afc8-48d1-a08f-9f66e37bf771', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:14:47.948784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c1d86934-2138-4473-bff6-e59b19b22fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:47.969936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1d86934-2138-4473-bff6-e59b19b22fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:47.970939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1d86934-2138-4473-bff6-e59b19b22fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:47.972481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1d86934-2138-4473-bff6-e59b19b22fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:48.052500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c1d86934-2138-4473-bff6-e59b19b22fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Going to try to start instance {{(pid=145968) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 27 10:14:48.251733 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c1d86934-2138-4473-bff6-e59b19b22fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "POST /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/action" status: 202 len: 0 microversion: 2.1 time: 0.309386 Sep 27 10:14:48.259244 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2289/4180] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:14:47 2022] POST /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/action => generated 0 bytes in 320 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:14:48.281796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-694b00cf-2606-44ef-8741-702aa793466b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:48.291482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-694b00cf-2606-44ef-8741-702aa793466b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:48.320682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-694b00cf-2606-44ef-8741-702aa793466b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:48.320682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-694b00cf-2606-44ef-8741-702aa793466b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:48.320682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-694b00cf-2606-44ef-8741-702aa793466b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:48.320682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-694b00cf-2606-44ef-8741-702aa793466b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:48.320682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-694b00cf-2606-44ef-8741-702aa793466b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:48.320682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-694b00cf-2606-44ef-8741-702aa793466b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:48.703422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-694b00cf-2606-44ef-8741-702aa793466b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:48.713251 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-694b00cf-2606-44ef-8741-702aa793466b tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1529 microversion: 2.1 time: 0.451947 Sep 27 10:14:48.714090 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2290/4181] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:48 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1529 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:48.732485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f7d12b24-2fb5-4652-802f-ed0e90f2dfbe tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:48.733467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f7d12b24-2fb5-4652-802f-ed0e90f2dfbe tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:48.742290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7d12b24-2fb5-4652-802f-ed0e90f2dfbe tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:48.742890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7d12b24-2fb5-4652-802f-ed0e90f2dfbe tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:48.743728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7d12b24-2fb5-4652-802f-ed0e90f2dfbe tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:48.744612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7d12b24-2fb5-4652-802f-ed0e90f2dfbe tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:48.745509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7d12b24-2fb5-4652-802f-ed0e90f2dfbe tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:48.746291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7d12b24-2fb5-4652-802f-ed0e90f2dfbe tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.152720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f7d12b24-2fb5-4652-802f-ed0e90f2dfbe tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:49.160943 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f7d12b24-2fb5-4652-802f-ed0e90f2dfbe tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1292 microversion: 2.1 time: 0.445357 Sep 27 10:14:49.172434 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2291/4182] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:48 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1292 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:49.180691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bfb088ba-8240-485a-98f7-5ce2b2ccda30 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:49.182299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bfb088ba-8240-485a-98f7-5ce2b2ccda30 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:49.190301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb088ba-8240-485a-98f7-5ce2b2ccda30 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.190792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb088ba-8240-485a-98f7-5ce2b2ccda30 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.191236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb088ba-8240-485a-98f7-5ce2b2ccda30 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.191933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb088ba-8240-485a-98f7-5ce2b2ccda30 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.192456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb088ba-8240-485a-98f7-5ce2b2ccda30 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.192838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfb088ba-8240-485a-98f7-5ce2b2ccda30 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.515697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:49.578392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bfb088ba-8240-485a-98f7-5ce2b2ccda30 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:49.605094 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bfb088ba-8240-485a-98f7-5ce2b2ccda30 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1709 microversion: 2.1 time: 0.441404 Sep 27 10:14:49.606310 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2292/4183] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:49 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1709 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:49.617295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:49.617295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: af870ec8-bdb2-4a04-9688-ab4bb5ae8331] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:49.624818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.625736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.626411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.627362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.655859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.658528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.027s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.692895 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] HTTP exception thrown: Instance af870ec8-bdb2-4a04-9688-ab4bb5ae8331 could not be found. Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Request method failure captured: Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331 HTTP/1.1 Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMszX-a341IC9_IAKhEBxAOqf1mij6xDuSSVVkGrssNWEw_dCoLkA1xWTzR4Rwr2Gns9Ok-NlHM7M6S5l1bagLeNEjZlWXfRV92-RjADtRurWMFky-4UZS6Kd0aZkdeapysCh8o_qe0TBQ7poqS-O59YrFSS4LX_iBhzIQN1K8he4j-s Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 326c35c0bc52463e95a56ba3371a276a Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-MultipleCreateTestJSON-1528938377 Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/326c35c0bc52463e95a56ba3371a276a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/326c35c0bc52463e95a56ba3371a276a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/326c35c0bc52463e95a56ba3371a276a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_326c35c0bc52463e95a56ba3371a276a"}]}] Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-MultipleCreateTestJSON-1528938377 Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 326c35c0bc52463e95a56ba3371a276a Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-MultipleCreateTestJSON-1528938377 Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-MultipleCreateTestJSON-1528938377-project Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a5e050c7389844a78199faef5a0c9da4 Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-MultipleCreateTestJSON-1528938377-project Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance af870ec8-bdb2-4a04-9688-ab4bb5ae8331 could not be found. Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'af870ec8-bdb2-4a04-9688-ab4bb5ae8331'} Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance af870ec8-bdb2-4a04-9688-ab4bb5ae8331 could not be found. Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance af870ec8-bdb2-4a04-9688-ab4bb5ae8331 could not be found. Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:14:49.695343 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:49.700906 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:49.700906 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:49.700906 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:49.700906 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:14:49.700906 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:14:49.700906 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance af870ec8-bdb2-4a04-9688-ab4bb5ae8331 could not be found. Sep 27 10:14:49.700906 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:49.700906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Returning 404 to user: Instance af870ec8-bdb2-4a04-9688-ab4bb5ae8331 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:14:49.700906 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9011b82d-c64e-4d71-b546-494b651c9cd2 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331" status: 404 len: 111 microversion: 2.1 time: 0.089554 Sep 27 10:14:49.702440 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2293/4184] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:49 2022] GET /compute/v2.1/servers/af870ec8-bdb2-4a04-9688-ab4bb5ae8331 => generated 111 bytes in 95 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:14:49.707637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-04a765e3-a36f-4f93-a204-ab550c3b0d3a req-9b0b7e90-a33c-40c8-b10a-1ffa3e03f947 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "af870ec8-bdb2-4a04-9688-ab4bb5ae8331", "name": "network-vif-deleted", "tag": "c157e508-9e0f-4763-a230-1bf760fe648b"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:49.719034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04a765e3-a36f-4f93-a204-ab550c3b0d3a req-9b0b7e90-a33c-40c8-b10a-1ffa3e03f947 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.720170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04a765e3-a36f-4f93-a204-ab550c3b0d3a req-9b0b7e90-a33c-40c8-b10a-1ffa3e03f947 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.720730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04a765e3-a36f-4f93-a204-ab550c3b0d3a req-9b0b7e90-a33c-40c8-b10a-1ffa3e03f947 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.736391 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-04a765e3-a36f-4f93-a204-ab550c3b0d3a req-9b0b7e90-a33c-40c8-b10a-1ffa3e03f947 service nova] Creating event network-vif-deleted:c157e508-9e0f-4763-a230-1bf760fe648b for instance af870ec8-bdb2-4a04-9688-ab4bb5ae8331 on nch2-master-1 Sep 27 10:14:49.743538 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-04a765e3-a36f-4f93-a204-ab550c3b0d3a req-9b0b7e90-a33c-40c8-b10a-1ffa3e03f947 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039282 Sep 27 10:14:49.744189 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2294/4185] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:49.750849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:49.751960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] [instance: 4ac85843-17b5-4435-b528-cb0d37b70631] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:49.759237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.760067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.760840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.762005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.762723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.763414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.781620 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] HTTP exception thrown: Instance 4ac85843-17b5-4435-b528-cb0d37b70631 could not be found. Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Request method failure captured: Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 HTTP/1.1 Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMszX-a341IC9_IAKhEBxAOqf1mij6xDuSSVVkGrssNWEw_dCoLkA1xWTzR4Rwr2Gns9Ok-NlHM7M6S5l1bagLeNEjZlWXfRV92-RjADtRurWMFky-4UZS6Kd0aZkdeapysCh8o_qe0TBQ7poqS-O59YrFSS4LX_iBhzIQN1K8he4j-s Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 326c35c0bc52463e95a56ba3371a276a Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-MultipleCreateTestJSON-1528938377 Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/326c35c0bc52463e95a56ba3371a276a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/326c35c0bc52463e95a56ba3371a276a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/326c35c0bc52463e95a56ba3371a276a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_326c35c0bc52463e95a56ba3371a276a"}]}] Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-MultipleCreateTestJSON-1528938377 Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 326c35c0bc52463e95a56ba3371a276a Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-MultipleCreateTestJSON-1528938377 Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-MultipleCreateTestJSON-1528938377-project Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a5e050c7389844a78199faef5a0c9da4 Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-MultipleCreateTestJSON-1528938377-project Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 4ac85843-17b5-4435-b528-cb0d37b70631 could not be found. Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '4ac85843-17b5-4435-b528-cb0d37b70631'} Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ac85843-17b5-4435-b528-cb0d37b70631 could not be found. Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ac85843-17b5-4435-b528-cb0d37b70631 could not be found. Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:14:49.784240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:49.789332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:49.789332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:49.789332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:49.789332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:14:49.789332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:14:49.789332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ac85843-17b5-4435-b528-cb0d37b70631 could not be found. Sep 27 10:14:49.789332 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:14:49.789332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] Returning 404 to user: Instance 4ac85843-17b5-4435-b528-cb0d37b70631 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:14:49.789332 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1b881e7d-8256-483e-ba4a-a99258eba0c5 tempest-MultipleCreateTestJSON-1528938377 tempest-MultipleCreateTestJSON-1528938377-project] 10.222.0.26 "GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631" status: 404 len: 111 microversion: 2.1 time: 0.041173 Sep 27 10:14:49.789332 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2295/4186] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:49 2022] GET /compute/v2.1/servers/4ac85843-17b5-4435-b528-cb0d37b70631 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:14:49.798226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:49.798899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:49.807879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.808420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.809064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.809755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.810199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.811001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.856266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:14:49.856714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:14:49.857231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:14:49.857821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:14:49.858283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:14:49.867463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.868356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.870279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.871648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lazy-loading 'fault' on Instance uuid 59d9d393-afa0-4c17-a358-09fb4e415b60 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:14:49.873031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.874333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.874965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.877042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:49.877847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:49.879514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:49.912052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Getting quotas for project 27c0d3daa08340d89d288bbf811fece9. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:14:49.920142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Getting quotas for user dc369ca971424247b96050e09368a7bb and project 27c0d3daa08340d89d288bbf811fece9. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:14:49.940226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:14:49.942864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:14:50.010140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c28c5e12dbb444cab4e376a3015c6bd1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1e7e90f89b4844dbafdc1c5e5f17b407', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:50.013177 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-33d183a4-fb18-4455-a533-d6ff657a6628 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.200872 Sep 27 10:14:50.014236 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1892/4187] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:14:47 2022] POST /compute/v2.1/servers => generated 380 bytes in 2203 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:14:50.052857 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6ee26e15-17cc-479c-a31f-65fc4bcb7b79 tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 200 len: 1315 microversion: 2.1 time: 0.257191 Sep 27 10:14:50.058850 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2296/4188] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:49 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 1315 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:50.061927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-451896ed-1f99-4ca8-97cf-2f986b3d929b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:50.062584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-451896ed-1f99-4ca8-97cf-2f986b3d929b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:50.196403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-451896ed-1f99-4ca8-97cf-2f986b3d929b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:50.202010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f21cdeea-3fb2-46cb-ae84-31d04cd9d504 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:50.203270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f21cdeea-3fb2-46cb-ae84-31d04cd9d504 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:50.217924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21cdeea-3fb2-46cb-ae84-31d04cd9d504 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:50.218392 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-451896ed-1f99-4ca8-97cf-2f986b3d929b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1213 microversion: 2.1 time: 0.178944 Sep 27 10:14:50.219537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21cdeea-3fb2-46cb-ae84-31d04cd9d504 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:50.219710 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1893/4189] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:50 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1213 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:50.220190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21cdeea-3fb2-46cb-ae84-31d04cd9d504 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:50.220990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21cdeea-3fb2-46cb-ae84-31d04cd9d504 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:50.221559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21cdeea-3fb2-46cb-ae84-31d04cd9d504 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:50.222041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21cdeea-3fb2-46cb-ae84-31d04cd9d504 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:50.639042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5e6e9d56-600b-4269-97a2-62d3c49f89de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:50.640072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5e6e9d56-600b-4269-97a2-62d3c49f89de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:50.660069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e6e9d56-600b-4269-97a2-62d3c49f89de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:50.661161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e6e9d56-600b-4269-97a2-62d3c49f89de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:50.661746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e6e9d56-600b-4269-97a2-62d3c49f89de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:50.664143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e6e9d56-600b-4269-97a2-62d3c49f89de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:50.665182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e6e9d56-600b-4269-97a2-62d3c49f89de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:50.667107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e6e9d56-600b-4269-97a2-62d3c49f89de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:51.080305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5e6e9d56-600b-4269-97a2-62d3c49f89de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:51.096883 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5e6e9d56-600b-4269-97a2-62d3c49f89de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1713 microversion: 2.1 time: 0.461409 Sep 27 10:14:51.098145 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1894/4190] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:50 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1713 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:51.106720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:51.107658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] [instance: 59d9d393-afa0-4c17-a358-09fb4e415b60] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:51.116806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:51.117675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:51.118317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:51.119315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:51.120522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:51.121074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:51.127643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f21cdeea-3fb2-46cb-ae84-31d04cd9d504 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:51.140787 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f21cdeea-3fb2-46cb-ae84-31d04cd9d504 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1292 microversion: 2.1 time: 0.953480 Sep 27 10:14:51.141853 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2297/4191] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:50 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1292 bytes in 955 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:51.146510 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] HTTP exception thrown: Instance 59d9d393-afa0-4c17-a358-09fb4e415b60 could not be found. Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Request method failure captured: Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 HTTP/1.1 Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsz6PLM9dpPu51tGocEF6MbfdcT4o6g5_j6G9tKh3OtmxJOxCpWeUdhEj00-cfx4R5mxwLOgrVD6Qr72H5IjVpztuTavzw-rZicoZhcIVjIgZ1iBPTD-N8m77pt_aZgdGWyIFtWgKmHuyon1mDmPFFVEhrHWOSyrnFPL4hldN3EKxUg Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1e7e90f89b4844dbafdc1c5e5f17b407 Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-InstanceActionsNegativeTestJSON-1110616879 Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1e7e90f89b4844dbafdc1c5e5f17b407"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1e7e90f89b4844dbafdc1c5e5f17b407"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1e7e90f89b4844dbafdc1c5e5f17b407"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1e7e90f89b4844dbafdc1c5e5f17b407"}]}] Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-InstanceActionsNegativeTestJSON-1110616879 Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1e7e90f89b4844dbafdc1c5e5f17b407 Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-InstanceActionsNegativeTestJSON-1110616879 Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-InstanceActionsNegativeTestJSON-1110616879-project Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: c28c5e12dbb444cab4e376a3015c6bd1 Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-InstanceActionsNegativeTestJSON-1110616879-project Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 59d9d393-afa0-4c17-a358-09fb4e415b60 could not be found. Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '59d9d393-afa0-4c17-a358-09fb4e415b60'} Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 59d9d393-afa0-4c17-a358-09fb4e415b60 could not be found. Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 59d9d393-afa0-4c17-a358-09fb4e415b60 could not be found. Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:14:51.149021 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:14:51.154139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:14:51.154139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:14:51.154139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:14:51.154139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:14:51.154139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:14:51.154139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:14:51.154139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 59d9d393-afa0-4c17-a358-09fb4e415b60 could not be found. Sep 27 10:14:51.154139 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:14:51.156454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] Returning 404 to user: Instance 59d9d393-afa0-4c17-a358-09fb4e415b60 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:14:51.157499 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-58e6790e-865a-4531-8f0c-7deee96b467f tempest-InstanceActionsNegativeTestJSON-1110616879 tempest-InstanceActionsNegativeTestJSON-1110616879-project] 10.222.0.26 "GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60" status: 404 len: 111 microversion: 2.1 time: 0.058027 Sep 27 10:14:51.158134 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1895/4192] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:51 2022] GET /compute/v2.1/servers/59d9d393-afa0-4c17-a358-09fb4e415b60 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:14:51.246737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f4fb8a1b-65bd-41e3-aa0e-0db3c10ef8e2 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:51.247413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f4fb8a1b-65bd-41e3-aa0e-0db3c10ef8e2 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:51.256251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4fb8a1b-65bd-41e3-aa0e-0db3c10ef8e2 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:51.256819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4fb8a1b-65bd-41e3-aa0e-0db3c10ef8e2 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:51.269479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4fb8a1b-65bd-41e3-aa0e-0db3c10ef8e2 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:51.270506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4fb8a1b-65bd-41e3-aa0e-0db3c10ef8e2 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:51.271143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4fb8a1b-65bd-41e3-aa0e-0db3c10ef8e2 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:51.271763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4fb8a1b-65bd-41e3-aa0e-0db3c10ef8e2 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:51.391559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f4fb8a1b-65bd-41e3-aa0e-0db3c10ef8e2 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:51.413289 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f4fb8a1b-65bd-41e3-aa0e-0db3c10ef8e2 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1265 microversion: 2.1 time: 0.170858 Sep 27 10:14:51.417735 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2298/4193] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:51 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1265 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:51.865893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5a25341b-276d-48ce-8dcb-4335d19091e5 req-60b5587a-4990-4817-84b2-56865f775bca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "59d9d393-afa0-4c17-a358-09fb4e415b60", "name": "network-vif-deleted", "tag": "b41cbc51-7a4c-4d78-adfd-0a78a0690ee0"}, {"name": "network-changed", "server_uuid": "3cd54bff-350d-4106-9562-7ccfce2a106c", "tag": "51432875-d706-4015-876d-392752eeb855"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:51.893706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5a25341b-276d-48ce-8dcb-4335d19091e5 req-60b5587a-4990-4817-84b2-56865f775bca service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:51.894261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5a25341b-276d-48ce-8dcb-4335d19091e5 req-60b5587a-4990-4817-84b2-56865f775bca service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:51.894710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5a25341b-276d-48ce-8dcb-4335d19091e5 req-60b5587a-4990-4817-84b2-56865f775bca service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:51.917845 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5a25341b-276d-48ce-8dcb-4335d19091e5 req-60b5587a-4990-4817-84b2-56865f775bca service nova] Creating event network-vif-deleted:b41cbc51-7a4c-4d78-adfd-0a78a0690ee0 for instance 59d9d393-afa0-4c17-a358-09fb4e415b60 on nch1-master-1 Sep 27 10:14:51.918612 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5a25341b-276d-48ce-8dcb-4335d19091e5 req-60b5587a-4990-4817-84b2-56865f775bca service nova] Creating event network-changed:51432875-d706-4015-876d-392752eeb855 for instance 3cd54bff-350d-4106-9562-7ccfce2a106c on nch1-master-1 Sep 27 10:14:51.932825 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5a25341b-276d-48ce-8dcb-4335d19091e5 req-60b5587a-4990-4817-84b2-56865f775bca service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.074056 Sep 27 10:14:51.934840 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1896/4194] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:51 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:52.175021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e9fc7f46-92ee-4b4d-969d-ac78f82b6bf4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:52.175481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e9fc7f46-92ee-4b4d-969d-ac78f82b6bf4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:52.185899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9fc7f46-92ee-4b4d-969d-ac78f82b6bf4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:52.186752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9fc7f46-92ee-4b4d-969d-ac78f82b6bf4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:52.187609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9fc7f46-92ee-4b4d-969d-ac78f82b6bf4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:52.188881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9fc7f46-92ee-4b4d-969d-ac78f82b6bf4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:52.189711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9fc7f46-92ee-4b4d-969d-ac78f82b6bf4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:52.190456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9fc7f46-92ee-4b4d-969d-ac78f82b6bf4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:52.446721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6710ae31-a856-4c1c-89f5-4c82223ee304 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:52.447462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6710ae31-a856-4c1c-89f5-4c82223ee304 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:52.461420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6710ae31-a856-4c1c-89f5-4c82223ee304 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:52.462384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6710ae31-a856-4c1c-89f5-4c82223ee304 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:52.462886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6710ae31-a856-4c1c-89f5-4c82223ee304 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:52.464248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6710ae31-a856-4c1c-89f5-4c82223ee304 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:52.465803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6710ae31-a856-4c1c-89f5-4c82223ee304 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:52.466757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6710ae31-a856-4c1c-89f5-4c82223ee304 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:52.538514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e9fc7f46-92ee-4b4d-969d-ac78f82b6bf4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:52.553448 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e9fc7f46-92ee-4b4d-969d-ac78f82b6bf4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1292 microversion: 2.1 time: 0.384547 Sep 27 10:14:52.569112 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2299/4195] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:52 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1292 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:52.650923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6710ae31-a856-4c1c-89f5-4c82223ee304 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:52.660135 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6710ae31-a856-4c1c-89f5-4c82223ee304 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1271 microversion: 2.1 time: 0.219701 Sep 27 10:14:52.661710 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1897/4196] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:52 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1271 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:53.583069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-55eea8da-a444-4999-bf8b-1f58027e544b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:53.587524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-55eea8da-a444-4999-bf8b-1f58027e544b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:53.597194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55eea8da-a444-4999-bf8b-1f58027e544b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:53.598557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55eea8da-a444-4999-bf8b-1f58027e544b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:53.599101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55eea8da-a444-4999-bf8b-1f58027e544b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:53.617930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55eea8da-a444-4999-bf8b-1f58027e544b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:53.618851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55eea8da-a444-4999-bf8b-1f58027e544b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:53.620056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55eea8da-a444-4999-bf8b-1f58027e544b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:53.686151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5941509-2231-4da6-98b8-5199c51b1ebd tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:53.688067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5941509-2231-4da6-98b8-5199c51b1ebd tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:53.705540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5941509-2231-4da6-98b8-5199c51b1ebd tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:53.706263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5941509-2231-4da6-98b8-5199c51b1ebd tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:53.706983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5941509-2231-4da6-98b8-5199c51b1ebd tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:53.708477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5941509-2231-4da6-98b8-5199c51b1ebd tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:53.709169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5941509-2231-4da6-98b8-5199c51b1ebd tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:53.709893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5941509-2231-4da6-98b8-5199c51b1ebd tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:53.909798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5941509-2231-4da6-98b8-5199c51b1ebd tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:53.933246 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5941509-2231-4da6-98b8-5199c51b1ebd tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1271 microversion: 2.1 time: 0.250006 Sep 27 10:14:53.944954 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1898/4197] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:53 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1271 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:54.552020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55eea8da-a444-4999-bf8b-1f58027e544b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:54.586934 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-55eea8da-a444-4999-bf8b-1f58027e544b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1292 microversion: 2.1 time: 1.011282 Sep 27 10:14:54.588912 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2300/4198] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:53 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1292 bytes in 1016 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:54.968159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe4005b7-e5df-4f1d-af5c-06ff36dc467f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:54.970108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fe4005b7-e5df-4f1d-af5c-06ff36dc467f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:54.984570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe4005b7-e5df-4f1d-af5c-06ff36dc467f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:54.985386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe4005b7-e5df-4f1d-af5c-06ff36dc467f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:54.986073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe4005b7-e5df-4f1d-af5c-06ff36dc467f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:54.986723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe4005b7-e5df-4f1d-af5c-06ff36dc467f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:54.988998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe4005b7-e5df-4f1d-af5c-06ff36dc467f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:54.991387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe4005b7-e5df-4f1d-af5c-06ff36dc467f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:55.531766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fe4005b7-e5df-4f1d-af5c-06ff36dc467f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:55.550925 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fe4005b7-e5df-4f1d-af5c-06ff36dc467f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1313 microversion: 2.1 time: 0.586717 Sep 27 10:14:55.553585 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1899/4199] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:54 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1313 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:55.616338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7056649e-af7c-4e94-a7c2-64d437cab250 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:55.617650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7056649e-af7c-4e94-a7c2-64d437cab250 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:55.629357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7056649e-af7c-4e94-a7c2-64d437cab250 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:55.629857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7056649e-af7c-4e94-a7c2-64d437cab250 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:55.630318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7056649e-af7c-4e94-a7c2-64d437cab250 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:55.636116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7056649e-af7c-4e94-a7c2-64d437cab250 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:55.637125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7056649e-af7c-4e94-a7c2-64d437cab250 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:55.637766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7056649e-af7c-4e94-a7c2-64d437cab250 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:56.080199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7056649e-af7c-4e94-a7c2-64d437cab250 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:56.101822 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7056649e-af7c-4e94-a7c2-64d437cab250 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1292 microversion: 2.1 time: 0.489877 Sep 27 10:14:56.103313 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2301/4200] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:55 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1292 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:56.581942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2e2e468e-b977-47e8-9e2f-29d6125d1cee tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:56.583166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2e2e468e-b977-47e8-9e2f-29d6125d1cee tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:56.597766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2e468e-b977-47e8-9e2f-29d6125d1cee tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:56.598283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2e468e-b977-47e8-9e2f-29d6125d1cee tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:56.598731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2e468e-b977-47e8-9e2f-29d6125d1cee tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:56.606889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2e468e-b977-47e8-9e2f-29d6125d1cee tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:56.613703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2e468e-b977-47e8-9e2f-29d6125d1cee tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:56.614329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e2e468e-b977-47e8-9e2f-29d6125d1cee tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:56.956770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a4a548ea-a4b7-4a9a-9773-7b019ee9a614 req-84e529fc-6d33-4780-b960-1768b43f2d48 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c74b0db7-5f7b-484b-8d2e-feea845f3264", "tag": "c16a5b67-0bb0-445b-9ef4-349421b3b566"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:56.970782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a4a548ea-a4b7-4a9a-9773-7b019ee9a614 req-84e529fc-6d33-4780-b960-1768b43f2d48 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:56.971430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a4a548ea-a4b7-4a9a-9773-7b019ee9a614 req-84e529fc-6d33-4780-b960-1768b43f2d48 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:56.972369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a4a548ea-a4b7-4a9a-9773-7b019ee9a614 req-84e529fc-6d33-4780-b960-1768b43f2d48 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:56.987309 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a4a548ea-a4b7-4a9a-9773-7b019ee9a614 req-84e529fc-6d33-4780-b960-1768b43f2d48 service nova] Creating event network-changed:c16a5b67-0bb0-445b-9ef4-349421b3b566 for instance c74b0db7-5f7b-484b-8d2e-feea845f3264 on nch2-master-1 Sep 27 10:14:56.992943 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a4a548ea-a4b7-4a9a-9773-7b019ee9a614 req-84e529fc-6d33-4780-b960-1768b43f2d48 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040612 Sep 27 10:14:56.993923 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2302/4201] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:14:57.016818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2e2e468e-b977-47e8-9e2f-29d6125d1cee tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:57.031540 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2e2e468e-b977-47e8-9e2f-29d6125d1cee tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1313 microversion: 2.1 time: 0.459342 Sep 27 10:14:57.032606 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1900/4202] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:56 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1313 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:57.130620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-372bee51-a22b-4d55-8b83-bc5f45e238ef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:57.131455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-372bee51-a22b-4d55-8b83-bc5f45e238ef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:57.140111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-372bee51-a22b-4d55-8b83-bc5f45e238ef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:57.140916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-372bee51-a22b-4d55-8b83-bc5f45e238ef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:57.141508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-372bee51-a22b-4d55-8b83-bc5f45e238ef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:57.142660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-372bee51-a22b-4d55-8b83-bc5f45e238ef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:57.143610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-372bee51-a22b-4d55-8b83-bc5f45e238ef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:57.144288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-372bee51-a22b-4d55-8b83-bc5f45e238ef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:57.593408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-372bee51-a22b-4d55-8b83-bc5f45e238ef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:57.604223 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-372bee51-a22b-4d55-8b83-bc5f45e238ef tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1292 microversion: 2.1 time: 0.477805 Sep 27 10:14:57.605387 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2303/4203] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:57 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1292 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:57.969262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-04f6de40-7463-4e98-a1a0-732448a44173 req-0b0e6218-b4af-4ca3-ba19-4723575a86b3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3cd54bff-350d-4106-9562-7ccfce2a106c", "name": "network-vif-plugged", "status": "completed", "tag": "51432875-d706-4015-876d-392752eeb855"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:14:57.984909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:57.987822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-04f6de40-7463-4e98-a1a0-732448a44173 req-0b0e6218-b4af-4ca3-ba19-4723575a86b3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:57.988523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-04f6de40-7463-4e98-a1a0-732448a44173 req-0b0e6218-b4af-4ca3-ba19-4723575a86b3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:57.989189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-04f6de40-7463-4e98-a1a0-732448a44173 req-0b0e6218-b4af-4ca3-ba19-4723575a86b3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:58.005516 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-04f6de40-7463-4e98-a1a0-732448a44173 req-0b0e6218-b4af-4ca3-ba19-4723575a86b3 service nova] Creating event network-vif-plugged:51432875-d706-4015-876d-392752eeb855 for instance 3cd54bff-350d-4106-9562-7ccfce2a106c on nch1-master-1 Sep 27 10:14:58.019786 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-04f6de40-7463-4e98-a1a0-732448a44173 req-0b0e6218-b4af-4ca3-ba19-4723575a86b3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057720 Sep 27 10:14:58.021551 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1901/4204] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:14:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:14:58.056985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2daab212-de94-43ff-9f49-824bd130887b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:58.062043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2daab212-de94-43ff-9f49-824bd130887b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:58.069048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2daab212-de94-43ff-9f49-824bd130887b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:58.071697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2daab212-de94-43ff-9f49-824bd130887b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:58.072321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2daab212-de94-43ff-9f49-824bd130887b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:58.073518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2daab212-de94-43ff-9f49-824bd130887b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:58.074599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2daab212-de94-43ff-9f49-824bd130887b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:58.075036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2daab212-de94-43ff-9f49-824bd130887b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:58.543801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2daab212-de94-43ff-9f49-824bd130887b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:58.568080 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2daab212-de94-43ff-9f49-824bd130887b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1313 microversion: 2.1 time: 0.515016 Sep 27 10:14:58.572106 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2304/4205] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:58 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1313 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:58.629877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6aa4f05a-4d50-4517-9bfa-cb4048d65529 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:58.630600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6aa4f05a-4d50-4517-9bfa-cb4048d65529 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:58.642750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6aa4f05a-4d50-4517-9bfa-cb4048d65529 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:58.643242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6aa4f05a-4d50-4517-9bfa-cb4048d65529 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:58.643690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6aa4f05a-4d50-4517-9bfa-cb4048d65529 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:58.644407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6aa4f05a-4d50-4517-9bfa-cb4048d65529 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:58.649324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6aa4f05a-4d50-4517-9bfa-cb4048d65529 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:58.649846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6aa4f05a-4d50-4517-9bfa-cb4048d65529 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:58.891702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6aa4f05a-4d50-4517-9bfa-cb4048d65529 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:14:58.903050 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6aa4f05a-4d50-4517-9bfa-cb4048d65529 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1292 microversion: 2.1 time: 0.277502 Sep 27 10:14:58.904137 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1902/4206] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:58 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1292 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:14:59.597147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3cbf8e8a-0606-435e-91e3-d5c1e25555d5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:59.608853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3cbf8e8a-0606-435e-91e3-d5c1e25555d5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:59.634964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cbf8e8a-0606-435e-91e3-d5c1e25555d5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:59.643415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cbf8e8a-0606-435e-91e3-d5c1e25555d5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:59.645716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cbf8e8a-0606-435e-91e3-d5c1e25555d5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:59.649611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cbf8e8a-0606-435e-91e3-d5c1e25555d5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:59.650404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cbf8e8a-0606-435e-91e3-d5c1e25555d5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:59.651170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cbf8e8a-0606-435e-91e3-d5c1e25555d5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:59.704430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:59.930967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-494e2bd0-76a6-4989-9067-dbaccf66319d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:14:59.931527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-494e2bd0-76a6-4989-9067-dbaccf66319d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:14:59.937537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:14:59.948862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-494e2bd0-76a6-4989-9067-dbaccf66319d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:59.949578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-494e2bd0-76a6-4989-9067-dbaccf66319d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:59.950411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-494e2bd0-76a6-4989-9067-dbaccf66319d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:59.951477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-494e2bd0-76a6-4989-9067-dbaccf66319d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:14:59.952591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-494e2bd0-76a6-4989-9067-dbaccf66319d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:14:59.953297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-494e2bd0-76a6-4989-9067-dbaccf66319d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:14:59.975834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:00.191924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3cbf8e8a-0606-435e-91e3-d5c1e25555d5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:00.206741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:00.213573 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3cbf8e8a-0606-435e-91e3-d5c1e25555d5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1313 microversion: 2.1 time: 0.621806 Sep 27 10:15:00.227046 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2305/4207] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:59 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1313 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:00.402523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-494e2bd0-76a6-4989-9067-dbaccf66319d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:00.414058 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-494e2bd0-76a6-4989-9067-dbaccf66319d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1472 microversion: 2.1 time: 0.490338 Sep 27 10:15:00.421585 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1903/4208] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:14:59 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1472 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:00.682404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-280757005", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:00.765001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] validate_networks() for [('662e23c5-f7e1-4e14-af6c-49d672076632', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:15:01.391870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAaction247Test-server-946539687", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": "none"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:01.471193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:15:01.474402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:15:01.476028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:15:01.476830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:15:01.479956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:15:01.494433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:01.495575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:01.496595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:01.497797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:01.498982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:01.499754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:01.528581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Getting quotas for project c8e7620229c349319071db0ff7426b62. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:15:01.538037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Getting quotas for user c43755ee4f0f49d8bf1d15921c2aed13 and project c8e7620229c349319071db0ff7426b62. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:15:01.554951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:15:01.558510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:15:01.600619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:15:01.601648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:15:01.602578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:15:01.603437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:15:01.605193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:15:01.620688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:01.621345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:01.622286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:01.623269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:01.623874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:01.624494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:01.633891 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-589b8736-61b8-4643-b7dd-f3d2519a6000 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.47 time: 0.475657 Sep 27 10:15:01.635248 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1904/4209] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:15:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 477 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Sep 27 10:15:01.640292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2c92a016-dc64-4e9a-af78-bd0f99b9affc tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:01.641030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2c92a016-dc64-4e9a-af78-bd0f99b9affc tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:01.646219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:15:01.648754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c92a016-dc64-4e9a-af78-bd0f99b9affc tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:01.649464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c92a016-dc64-4e9a-af78-bd0f99b9affc tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:01.650125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c92a016-dc64-4e9a-af78-bd0f99b9affc tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:01.651030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c92a016-dc64-4e9a-af78-bd0f99b9affc tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:01.651782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c92a016-dc64-4e9a-af78-bd0f99b9affc tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:01.652422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c92a016-dc64-4e9a-af78-bd0f99b9affc tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:01.653136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for user a7b8599d0b40424087e52d8e35b71ec4 and project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:15:01.669406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:15:01.672842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:15:01.764101 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-68a362cb-081f-4c93-a5d1-28ad22e43abd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.089934 Sep 27 10:15:01.776145 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2306/4210] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:15:00 2022] POST /compute/v2.1/servers => generated 380 bytes in 1101 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:15:01.784862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8085d274-7e42-420e-8dca-b70ae5342cda tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:01.785507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8085d274-7e42-420e-8dca-b70ae5342cda tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:01.939404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8085d274-7e42-420e-8dca-b70ae5342cda tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:01.947067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2c92a016-dc64-4e9a-af78-bd0f99b9affc tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:01.961341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8085d274-7e42-420e-8dca-b70ae5342cda tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:01.961913 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2c92a016-dc64-4e9a-af78-bd0f99b9affc tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1313 microversion: 2.1 time: 0.325471 Sep 27 10:15:01.962513 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1905/4211] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:01 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1313 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:01.970543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0e6b2204-b8ca-4ea3-8a0a-ce5e98494060 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:01.973484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0e6b2204-b8ca-4ea3-8a0a-ce5e98494060 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:01.980990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8085d274-7e42-420e-8dca-b70ae5342cda tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:01.982316 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8085d274-7e42-420e-8dca-b70ae5342cda tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925" status: 200 len: 1280 microversion: 2.47 time: 0.204779 Sep 27 10:15:01.996300 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2307/4212] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:01 2022] GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925 => generated 1280 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:02.010262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0e6b2204-b8ca-4ea3-8a0a-ce5e98494060 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:15:02.555748 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.scheduler.client.report [None req-0e6b2204-b8ca-4ea3-8a0a-ce5e98494060 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Deleted allocations for instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa Sep 27 10:15:02.566390 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.compute.api [None req-0e6b2204-b8ca-4ea3-8a0a-ce5e98494060 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa] Instance Mapping does not exist while attempting local delete cleanup. Sep 27 10:15:02.567247 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0e6b2204-b8ca-4ea3-8a0a-ce5e98494060 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa" status: 204 len: 0 microversion: 2.1 time: 0.603451 Sep 27 10:15:02.576205 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1906/4213] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:15:01 2022] DELETE /compute/v2.1/servers/b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa => generated 0 bytes in 613 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:15:02.585974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86524890-e281-4584-95ee-dea99fef00c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:02.586934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86524890-e281-4584-95ee-dea99fef00c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:02.596582 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-86524890-e281-4584-95ee-dea99fef00c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86524890-e281-4584-95ee-dea99fef00c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa HTTP/1.1 Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa'} Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:02.598579 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:02.602629 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:02.602629 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:15:02.602629 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:15:02.602629 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:15:02.602629 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:02.604549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86524890-e281-4584-95ee-dea99fef00c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:15:02.605660 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86524890-e281-4584-95ee-dea99fef00c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa" status: 404 len: 111 microversion: 2.1 time: 0.025219 Sep 27 10:15:02.624645 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2308/4214] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:02 2022] GET /compute/v2.1/servers/b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:15:02.784235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-769005154", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "accessIPv6": "2001:2001::3", "networks": [{"uuid": "662e23c5-f7e1-4e14-af6c-49d672076632"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:02.861324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] validate_networks() for [('662e23c5-f7e1-4e14-af6c-49d672076632', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:15:02.986613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-433351a6-9b32-4442-a57b-ed01f77a26a6 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:02.987457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-433351a6-9b32-4442-a57b-ed01f77a26a6 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:03.001996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-433351a6-9b32-4442-a57b-ed01f77a26a6 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:03.002500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-433351a6-9b32-4442-a57b-ed01f77a26a6 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:03.003013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-433351a6-9b32-4442-a57b-ed01f77a26a6 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:03.003796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-433351a6-9b32-4442-a57b-ed01f77a26a6 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:03.004268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-433351a6-9b32-4442-a57b-ed01f77a26a6 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:03.005200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-433351a6-9b32-4442-a57b-ed01f77a26a6 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:03.309121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-433351a6-9b32-4442-a57b-ed01f77a26a6 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:03.319183 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-433351a6-9b32-4442-a57b-ed01f77a26a6 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1313 microversion: 2.1 time: 0.338006 Sep 27 10:15:03.336603 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2309/4215] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:02 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1313 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:03.344225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:03.345309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:03.354563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:03.355408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:03.356033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:03.356934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:03.357690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:03.358584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:03.463332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:03.486214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:03.488157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:03.489724 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9117e82-a521-4a8a-9de1-d0ee20374843 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925" status: 200 len: 1332 microversion: 2.47 time: 0.150512 Sep 27 10:15:03.490607 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2310/4216] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:03 2022] GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925 => generated 1332 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:03.748587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:15:03.750007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:15:03.751023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:15:03.751685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:15:03.752198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:15:03.763529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:03.764094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:03.764803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:03.766430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:03.767107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:03.767957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:03.792243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:15:03.798152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for user a7b8599d0b40424087e52d8e35b71ec4 and project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:15:03.816092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:15:03.819299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:15:03.879167 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8fbe9811-2797-4bbd-964f-3257e80c9f8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.099087 Sep 27 10:15:03.880528 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1907/4217] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:15:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 1102 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:15:03.895800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b2d72101-a7cd-4859-8c0a-8cd01fcb8af7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:03.896897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b2d72101-a7cd-4859-8c0a-8cd01fcb8af7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:03.981912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b2d72101-a7cd-4859-8c0a-8cd01fcb8af7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:03.992523 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b2d72101-a7cd-4859-8c0a-8cd01fcb8af7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1220 microversion: 2.1 time: 0.104808 Sep 27 10:15:03.994430 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2311/4218] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:03 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1220 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:04.348815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-38292e82-036f-483f-a789-2d376895a5a3 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:04.349972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-38292e82-036f-483f-a789-2d376895a5a3 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:04.368378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38292e82-036f-483f-a789-2d376895a5a3 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:04.369155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38292e82-036f-483f-a789-2d376895a5a3 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:04.371286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38292e82-036f-483f-a789-2d376895a5a3 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:04.372630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38292e82-036f-483f-a789-2d376895a5a3 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:04.373491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38292e82-036f-483f-a789-2d376895a5a3 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:04.374090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38292e82-036f-483f-a789-2d376895a5a3 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:04.525182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:04.526336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:04.564295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:04.566392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:04.567182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:04.574183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:04.577019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:04.577561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:04.701216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:04.733944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:04.735735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:04.737409 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-013c8092-b32d-48e2-afd2-d2f32100d04d tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925" status: 200 len: 1338 microversion: 2.47 time: 0.223446 Sep 27 10:15:04.738340 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2312/4219] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:04 2022] GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925 => generated 1338 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:04.875316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-38292e82-036f-483f-a789-2d376895a5a3 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:04.909636 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-38292e82-036f-483f-a789-2d376895a5a3 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1313 microversion: 2.1 time: 0.569335 Sep 27 10:15:04.910659 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1908/4220] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:04 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1313 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:05.019647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-944737ba-4ae4-4651-9455-59690ed3acd0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:05.021036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-944737ba-4ae4-4651-9455-59690ed3acd0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:05.039653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-944737ba-4ae4-4651-9455-59690ed3acd0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:05.040413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-944737ba-4ae4-4651-9455-59690ed3acd0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:05.041139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-944737ba-4ae4-4651-9455-59690ed3acd0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:05.042003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-944737ba-4ae4-4651-9455-59690ed3acd0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:05.042616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-944737ba-4ae4-4651-9455-59690ed3acd0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:05.043179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-944737ba-4ae4-4651-9455-59690ed3acd0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:05.255340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-944737ba-4ae4-4651-9455-59690ed3acd0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:05.273672 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-944737ba-4ae4-4651-9455-59690ed3acd0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1272 microversion: 2.1 time: 0.254111 Sep 27 10:15:05.274612 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2313/4221] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:05 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1272 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:05.765326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:05.766377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:05.786501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:05.788819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:05.792528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:05.793933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:05.796542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:05.797249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:05.914496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:05.929550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:05.931194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:05.937299 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cc5e3527-5812-4f04-9b0b-a870819548bf tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925" status: 200 len: 1338 microversion: 2.47 time: 0.172653 Sep 27 10:15:05.956707 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1909/4222] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:05 2022] GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925 => generated 1338 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:05.961271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4682a791-3cf3-4a50-baff-43ac9372e50a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:05.962022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4682a791-3cf3-4a50-baff-43ac9372e50a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:05.974074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4682a791-3cf3-4a50-baff-43ac9372e50a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:05.977241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4682a791-3cf3-4a50-baff-43ac9372e50a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:05.977879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4682a791-3cf3-4a50-baff-43ac9372e50a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:05.978794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4682a791-3cf3-4a50-baff-43ac9372e50a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:05.979466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4682a791-3cf3-4a50-baff-43ac9372e50a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:05.980145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4682a791-3cf3-4a50-baff-43ac9372e50a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:06.298818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2f6cb4ec-da59-4c64-9889-b1eefa69beba tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:06.299691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2f6cb4ec-da59-4c64-9889-b1eefa69beba tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:06.312434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6cb4ec-da59-4c64-9889-b1eefa69beba tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:06.322393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6cb4ec-da59-4c64-9889-b1eefa69beba tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:06.326892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6cb4ec-da59-4c64-9889-b1eefa69beba tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:06.327945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6cb4ec-da59-4c64-9889-b1eefa69beba tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:06.330167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6cb4ec-da59-4c64-9889-b1eefa69beba tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:06.331736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f6cb4ec-da59-4c64-9889-b1eefa69beba tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:06.388978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4682a791-3cf3-4a50-baff-43ac9372e50a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:06.398087 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4682a791-3cf3-4a50-baff-43ac9372e50a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1313 microversion: 2.1 time: 0.463860 Sep 27 10:15:06.406044 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2314/4223] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:05 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1313 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:06.671974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2f6cb4ec-da59-4c64-9889-b1eefa69beba tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:06.687485 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2f6cb4ec-da59-4c64-9889-b1eefa69beba tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1278 microversion: 2.1 time: 0.392599 Sep 27 10:15:06.688086 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1910/4224] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:06 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1278 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:06.977230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:06.977920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:06.992775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:06.993344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:06.993807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:06.996558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:06.999324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:06.999904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:07.054386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-afefb0ae-696a-48f7-91a1-75d15716508f req-1f8c21bf-28e0-47cd-9a94-e38abe218d21 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c74b0db7-5f7b-484b-8d2e-feea845f3264", "name": "network-vif-plugged", "status": "completed", "tag": "c16a5b67-0bb0-445b-9ef4-349421b3b566"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:07.068063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-afefb0ae-696a-48f7-91a1-75d15716508f req-1f8c21bf-28e0-47cd-9a94-e38abe218d21 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:07.069215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-afefb0ae-696a-48f7-91a1-75d15716508f req-1f8c21bf-28e0-47cd-9a94-e38abe218d21 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:07.069892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-afefb0ae-696a-48f7-91a1-75d15716508f req-1f8c21bf-28e0-47cd-9a94-e38abe218d21 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:07.084761 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-afefb0ae-696a-48f7-91a1-75d15716508f req-1f8c21bf-28e0-47cd-9a94-e38abe218d21 service nova] Creating event network-vif-plugged:c16a5b67-0bb0-445b-9ef4-349421b3b566 for instance c74b0db7-5f7b-484b-8d2e-feea845f3264 on nch2-master-1 Sep 27 10:15:07.091756 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-afefb0ae-696a-48f7-91a1-75d15716508f req-1f8c21bf-28e0-47cd-9a94-e38abe218d21 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041261 Sep 27 10:15:07.093320 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1911/4225] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:07.093891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:07.103592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:07.105119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:07.106517 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-24e294bb-bfbf-4d24-86f9-9415553c09e3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925" status: 200 len: 1359 microversion: 2.47 time: 0.136681 Sep 27 10:15:07.107790 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2315/4226] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:06 2022] GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925 => generated 1359 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:07.135762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5e945662-dd14-4d32-b908-2538341640c3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-1-713483896"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:07.153287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5e945662-dd14-4d32-b908-2538341640c3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:07.182478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e945662-dd14-4d32-b908-2538341640c3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:07.183716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e945662-dd14-4d32-b908-2538341640c3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:07.207490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e945662-dd14-4d32-b908-2538341640c3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:07.210760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e945662-dd14-4d32-b908-2538341640c3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:07.211782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e945662-dd14-4d32-b908-2538341640c3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:07.215486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e945662-dd14-4d32-b908-2538341640c3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:07.437578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15b26a1e-7b69-44d2-9baf-a8a61aa9fa1b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:07.438211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-15b26a1e-7b69-44d2-9baf-a8a61aa9fa1b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:07.453337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15b26a1e-7b69-44d2-9baf-a8a61aa9fa1b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:07.454528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15b26a1e-7b69-44d2-9baf-a8a61aa9fa1b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:07.455169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15b26a1e-7b69-44d2-9baf-a8a61aa9fa1b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:07.457344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15b26a1e-7b69-44d2-9baf-a8a61aa9fa1b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:07.461665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15b26a1e-7b69-44d2-9baf-a8a61aa9fa1b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:07.464366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15b26a1e-7b69-44d2-9baf-a8a61aa9fa1b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:07.696445 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5e945662-dd14-4d32-b908-2538341640c3 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "POST /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925/action" status: 202 len: 52 microversion: 2.47 time: 0.566401 Sep 27 10:15:07.703866 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1912/4227] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:15:07 2022] POST /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925/action => generated 52 bytes in 575 msecs (HTTP/1.1 202) 9 headers in 364 bytes (1 switches on core 0) Sep 27 10:15:07.723641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba82b177-4d01-465c-aeb3-aa17023cb407 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:07.728280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ba82b177-4d01-465c-aeb3-aa17023cb407 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:07.751360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba82b177-4d01-465c-aeb3-aa17023cb407 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:07.754130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba82b177-4d01-465c-aeb3-aa17023cb407 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:07.755818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba82b177-4d01-465c-aeb3-aa17023cb407 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:07.758463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba82b177-4d01-465c-aeb3-aa17023cb407 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:07.759259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba82b177-4d01-465c-aeb3-aa17023cb407 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:07.761523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba82b177-4d01-465c-aeb3-aa17023cb407 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:07.843208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ba82b177-4d01-465c-aeb3-aa17023cb407 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:15:08.007464 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ba82b177-4d01-465c-aeb3-aa17023cb407 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "DELETE /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925" status: 204 len: 0 microversion: 2.47 time: 0.293404 Sep 27 10:15:08.012522 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1913/4228] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:15:07 2022] DELETE /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:15:08.016225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4b08bf06-7a92-4a1d-bcb4-dec4c3ba3690 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:08.017421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4b08bf06-7a92-4a1d-bcb4-dec4c3ba3690 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:08.026523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b08bf06-7a92-4a1d-bcb4-dec4c3ba3690 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:08.027830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b08bf06-7a92-4a1d-bcb4-dec4c3ba3690 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:08.028435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b08bf06-7a92-4a1d-bcb4-dec4c3ba3690 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:08.029537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b08bf06-7a92-4a1d-bcb4-dec4c3ba3690 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:08.030426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b08bf06-7a92-4a1d-bcb4-dec4c3ba3690 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:08.031350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b08bf06-7a92-4a1d-bcb4-dec4c3ba3690 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:08.247407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-15b26a1e-7b69-44d2-9baf-a8a61aa9fa1b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:08.268088 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-15b26a1e-7b69-44d2-9baf-a8a61aa9fa1b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1313 microversion: 2.1 time: 0.841572 Sep 27 10:15:08.270845 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2316/4229] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:07 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1313 bytes in 845 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:08.286021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:08.288442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:08.296700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:08.303858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4b08bf06-7a92-4a1d-bcb4-dec4c3ba3690 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:08.315975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:08.316746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:08.317895 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4b08bf06-7a92-4a1d-bcb4-dec4c3ba3690 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1320 microversion: 2.1 time: 0.303966 Sep 27 10:15:08.318618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:08.320770 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1914/4230] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:08 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1320 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:08.321473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:08.322134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:08.325004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:09.046008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:09.064496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:09.067327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:09.069350 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f7f0d92b-4b87-4a38-a337-ac17dc34137b tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925" status: 200 len: 1365 microversion: 2.47 time: 0.794623 Sep 27 10:15:09.070690 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2317/4231] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:08 2022] GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925 => generated 1365 bytes in 799 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:09.083318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:09.084257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:09.096777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:09.097430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:09.097873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:09.098817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:09.099426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:09.100229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:09.294379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a1478a65-b3dd-43e0-a8a0-d736ffc3fd5b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:09.295288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a1478a65-b3dd-43e0-a8a0-d736ffc3fd5b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:09.305093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1478a65-b3dd-43e0-a8a0-d736ffc3fd5b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:09.305606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1478a65-b3dd-43e0-a8a0-d736ffc3fd5b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:09.309064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1478a65-b3dd-43e0-a8a0-d736ffc3fd5b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:09.311108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1478a65-b3dd-43e0-a8a0-d736ffc3fd5b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:09.314580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1478a65-b3dd-43e0-a8a0-d736ffc3fd5b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:09.315641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1478a65-b3dd-43e0-a8a0-d736ffc3fd5b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:09.539772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:09.550173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:09.552869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:09.554879 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f9cbaea1-0d1c-45f2-af66-ced05b41788e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2947 microversion: 2.32 time: 0.475992 Sep 27 10:15:09.556692 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1915/4232] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:09 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2947 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:09.564360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a785a115-aead-4019-9c8a-54e9da109de9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:09.564998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a785a115-aead-4019-9c8a-54e9da109de9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:09.575218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a785a115-aead-4019-9c8a-54e9da109de9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:09.575811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a785a115-aead-4019-9c8a-54e9da109de9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:09.576254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a785a115-aead-4019-9c8a-54e9da109de9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:09.589535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a785a115-aead-4019-9c8a-54e9da109de9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:09.592373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a785a115-aead-4019-9c8a-54e9da109de9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:09.592871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a785a115-aead-4019-9c8a-54e9da109de9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:09.795140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a1478a65-b3dd-43e0-a8a0-d736ffc3fd5b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:09.819706 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a1478a65-b3dd-43e0-a8a0-d736ffc3fd5b tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1498 microversion: 2.1 time: 0.519238 Sep 27 10:15:09.821576 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2318/4233] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:09 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1498 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:09.856093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef711f75-4c60-4a1b-a110-2f3f51261ad5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:09.857874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ef711f75-4c60-4a1b-a110-2f3f51261ad5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:09.892099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef711f75-4c60-4a1b-a110-2f3f51261ad5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:09.893210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef711f75-4c60-4a1b-a110-2f3f51261ad5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:09.893659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef711f75-4c60-4a1b-a110-2f3f51261ad5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:09.897601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef711f75-4c60-4a1b-a110-2f3f51261ad5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:09.901712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef711f75-4c60-4a1b-a110-2f3f51261ad5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:09.902335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef711f75-4c60-4a1b-a110-2f3f51261ad5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:10.003218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a785a115-aead-4019-9c8a-54e9da109de9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:10.014469 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a785a115-aead-4019-9c8a-54e9da109de9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1320 microversion: 2.1 time: 0.456340 Sep 27 10:15:10.015068 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1916/4234] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:09 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1320 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:10.020227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b875d6a1-7d2a-4f2e-a753-ca73274b1784 req-77e38f4b-d273-4dc7-a9df-cdb78714664c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "430df9fb-7e75-4f60-8d96-0910ec4e8243", "tag": "b69beb62-d074-4135-8cde-13301d0421e3"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:10.058478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b875d6a1-7d2a-4f2e-a753-ca73274b1784 req-77e38f4b-d273-4dc7-a9df-cdb78714664c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:10.058959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b875d6a1-7d2a-4f2e-a753-ca73274b1784 req-77e38f4b-d273-4dc7-a9df-cdb78714664c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:10.059394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b875d6a1-7d2a-4f2e-a753-ca73274b1784 req-77e38f4b-d273-4dc7-a9df-cdb78714664c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:10.090620 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b875d6a1-7d2a-4f2e-a753-ca73274b1784 req-77e38f4b-d273-4dc7-a9df-cdb78714664c service nova] Creating event network-changed:b69beb62-d074-4135-8cde-13301d0421e3 for instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 on nch2-master-1 Sep 27 10:15:10.103546 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b875d6a1-7d2a-4f2e-a753-ca73274b1784 req-77e38f4b-d273-4dc7-a9df-cdb78714664c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085613 Sep 27 10:15:10.106390 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1917/4235] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:10.123766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:10.127223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:10.143954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:10.145234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:10.149412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:10.150424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:10.151042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:10.151677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:10.366625 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef711f75-4c60-4a1b-a110-2f3f51261ad5 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd/os-interface" status: 200 len: 1831 microversion: 2.32 time: 0.542710 Sep 27 10:15:10.368204 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2319/4236] 10.222.0.26 () {60 vars in 1316 bytes} [Tue Sep 27 10:15:09 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd/os-interface => generated 1831 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:10.459939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:10.481270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:10.487024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:10.490559 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ac60e2f7-a350-4619-9967-6371055bf0bd tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925" status: 200 len: 1365 microversion: 2.47 time: 0.380734 Sep 27 10:15:10.493646 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1918/4237] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:10 2022] GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925 => generated 1365 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:10.853272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f5b7e3f2-fe36-4da3-ae22-85528319c5b0 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:10.853797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f5b7e3f2-fe36-4da3-ae22-85528319c5b0 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:10.869405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5b7e3f2-fe36-4da3-ae22-85528319c5b0 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:10.870185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5b7e3f2-fe36-4da3-ae22-85528319c5b0 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:10.870799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5b7e3f2-fe36-4da3-ae22-85528319c5b0 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:10.872400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5b7e3f2-fe36-4da3-ae22-85528319c5b0 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:10.873059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5b7e3f2-fe36-4da3-ae22-85528319c5b0 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:10.882737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f5b7e3f2-fe36-4da3-ae22-85528319c5b0 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:10.937948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f5b7e3f2-fe36-4da3-ae22-85528319c5b0 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:15:11.043535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-14659854-cb28-4af2-8d53-045359fc70d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:11.044373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-14659854-cb28-4af2-8d53-045359fc70d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:11.097991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14659854-cb28-4af2-8d53-045359fc70d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:11.099095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14659854-cb28-4af2-8d53-045359fc70d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:11.100682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14659854-cb28-4af2-8d53-045359fc70d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:11.102998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14659854-cb28-4af2-8d53-045359fc70d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:11.103798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14659854-cb28-4af2-8d53-045359fc70d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:11.104530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14659854-cb28-4af2-8d53-045359fc70d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:11.237609 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f5b7e3f2-fe36-4da3-ae22-85528319c5b0 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "DELETE /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 204 len: 0 microversion: 2.1 time: 0.388330 Sep 27 10:15:11.241926 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2320/4238] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:15:10 2022] DELETE /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:15:11.249997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-597faf03-0f3a-4fa4-aa2c-717290f95ce8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:11.251055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-597faf03-0f3a-4fa4-aa2c-717290f95ce8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:11.263046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-597faf03-0f3a-4fa4-aa2c-717290f95ce8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:11.263621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-597faf03-0f3a-4fa4-aa2c-717290f95ce8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:11.264071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-597faf03-0f3a-4fa4-aa2c-717290f95ce8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:11.266156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-597faf03-0f3a-4fa4-aa2c-717290f95ce8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:11.267083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-597faf03-0f3a-4fa4-aa2c-717290f95ce8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:11.267554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-597faf03-0f3a-4fa4-aa2c-717290f95ce8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:11.443257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-14659854-cb28-4af2-8d53-045359fc70d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:11.454271 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-14659854-cb28-4af2-8d53-045359fc70d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1320 microversion: 2.1 time: 0.417281 Sep 27 10:15:11.466458 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1919/4239] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:11 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1320 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:11.530108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:11.531142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:11.538433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:11.539218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:11.539824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:11.540799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:11.541421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:11.542777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:11.598375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:11.599240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:11.600109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:11.600920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lazy-loading 'fault' on Instance uuid 099800ce-4c93-4ba9-87f4-aa5dda057925 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:15:11.603421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:11.636139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:11.650847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-597faf03-0f3a-4fa4-aa2c-717290f95ce8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:11.665339 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-597faf03-0f3a-4fa4-aa2c-717290f95ce8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1504 microversion: 2.1 time: 0.421504 Sep 27 10:15:11.668580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:11.689130 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2321/4240] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:11 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1504 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:11.694810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:11.696768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'c43755ee4f0f49d8bf1d15921c2aed13', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c8e7620229c349319071db0ff7426b62', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:11.698671 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-56a69074-2610-486e-b16e-2c1c271e4e12 tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925" status: 200 len: 1370 microversion: 2.47 time: 0.178582 Sep 27 10:15:11.705173 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1920/4241] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:11 2022] GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925 => generated 1370 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:15:12.480498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-435bef8b-8875-446d-8431-1427273df7e9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:12.481580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-435bef8b-8875-446d-8431-1427273df7e9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:12.504626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-435bef8b-8875-446d-8431-1427273df7e9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:12.505175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-435bef8b-8875-446d-8431-1427273df7e9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:12.505620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-435bef8b-8875-446d-8431-1427273df7e9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:12.512789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-435bef8b-8875-446d-8431-1427273df7e9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:12.515344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-435bef8b-8875-446d-8431-1427273df7e9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:12.517285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-435bef8b-8875-446d-8431-1427273df7e9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:12.702476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-730870e1-7d8b-4519-8cd9-8fab861c5bbf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:12.703101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-730870e1-7d8b-4519-8cd9-8fab861c5bbf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:12.731408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730870e1-7d8b-4519-8cd9-8fab861c5bbf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:12.734613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730870e1-7d8b-4519-8cd9-8fab861c5bbf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:12.735347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730870e1-7d8b-4519-8cd9-8fab861c5bbf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:12.738009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730870e1-7d8b-4519-8cd9-8fab861c5bbf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:12.740931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730870e1-7d8b-4519-8cd9-8fab861c5bbf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:12.741983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730870e1-7d8b-4519-8cd9-8fab861c5bbf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:12.856124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-435bef8b-8875-446d-8431-1427273df7e9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:12.872290 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-435bef8b-8875-446d-8431-1427273df7e9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1320 microversion: 2.1 time: 0.392134 Sep 27 10:15:12.874136 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2322/4242] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:12 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1320 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:12.893635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:12.901137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] [instance: 099800ce-4c93-4ba9-87f4-aa5dda057925] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:12.927374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:12.930918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:12.944007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:12.945526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:12.946572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:12.947307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:12.976384 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] HTTP exception thrown: Instance 099800ce-4c93-4ba9-87f4-aa5dda057925 could not be found. Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Request method failure captured: Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925 HTTP/1.1 Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0l9_KTm8j2on6gNKd1WJqkRuwPG6EIrE4S_fSWRHsqIaiusZClvJpbPOkVV20BAkKIS7QWgyrAS-ziKbQJ7YQO44hhYDd3Zuj2HLsWqtI7BHiT0-ongSJGHLvC1zr7bvigf7YNJrHIc-RwFkd4Yzyik_Z9T4EE-Uv_AqctAvXFAHs Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.47 Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: c8e7620229c349319071db0ff7426b62 Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersAaction247Test-99574241 Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8e7620229c349319071db0ff7426b62"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c8e7620229c349319071db0ff7426b62"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c8e7620229c349319071db0ff7426b62"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c8e7620229c349319071db0ff7426b62"}]}] Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersAaction247Test-99574241 Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: c8e7620229c349319071db0ff7426b62 Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersAaction247Test-99574241 Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersAaction247Test-99574241-project Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: c43755ee4f0f49d8bf1d15921c2aed13 Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersAaction247Test-99574241-project Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 099800ce-4c93-4ba9-87f4-aa5dda057925 could not be found. Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '099800ce-4c93-4ba9-87f4-aa5dda057925'} Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 099800ce-4c93-4ba9-87f4-aa5dda057925 could not be found. Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 099800ce-4c93-4ba9-87f4-aa5dda057925 could not be found. Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:12.979735 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:12.983698 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:12.983698 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:12.983698 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:12.983698 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:15:12.983698 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:15:12.983698 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 099800ce-4c93-4ba9-87f4-aa5dda057925 could not be found. Sep 27 10:15:12.983698 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:12.990111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] Returning 404 to user: Instance 099800ce-4c93-4ba9-87f4-aa5dda057925 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:15:12.992341 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6d974747-b7f6-4d0e-b5a2-ad819982015a tempest-ServersAaction247Test-99574241 tempest-ServersAaction247Test-99574241-project] 10.222.0.26 "GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925" status: 404 len: 111 microversion: 2.47 time: 0.114644 Sep 27 10:15:12.994405 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2323/4243] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:12 2022] GET /compute/v2.1/servers/099800ce-4c93-4ba9-87f4-aa5dda057925 => generated 111 bytes in 118 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:15:13.029560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-730870e1-7d8b-4519-8cd9-8fab861c5bbf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:13.038976 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-730870e1-7d8b-4519-8cd9-8fab861c5bbf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1504 microversion: 2.1 time: 0.341940 Sep 27 10:15:13.048573 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1921/4244] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:12 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1504 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:13.915880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6d4abedc-ce7e-4323-bf4a-5e1790094095 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:13.918446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6d4abedc-ce7e-4323-bf4a-5e1790094095 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:13.955036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d4abedc-ce7e-4323-bf4a-5e1790094095 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:13.956274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d4abedc-ce7e-4323-bf4a-5e1790094095 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:13.956973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d4abedc-ce7e-4323-bf4a-5e1790094095 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:13.958590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d4abedc-ce7e-4323-bf4a-5e1790094095 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:13.960135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d4abedc-ce7e-4323-bf4a-5e1790094095 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:13.960938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d4abedc-ce7e-4323-bf4a-5e1790094095 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:13.995297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=10 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:14.064096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7822d21e-200d-41f1-8cee-5d1f0fda5feb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:14.064810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7822d21e-200d-41f1-8cee-5d1f0fda5feb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:14.072451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:14.076709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=14 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:14.078877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7822d21e-200d-41f1-8cee-5d1f0fda5feb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:14.079563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7822d21e-200d-41f1-8cee-5d1f0fda5feb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:14.080006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7822d21e-200d-41f1-8cee-5d1f0fda5feb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:14.080817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7822d21e-200d-41f1-8cee-5d1f0fda5feb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:14.081290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7822d21e-200d-41f1-8cee-5d1f0fda5feb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:14.081731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7822d21e-200d-41f1-8cee-5d1f0fda5feb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:14.327565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6d4abedc-ce7e-4323-bf4a-5e1790094095 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:14.335797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:14.337190 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6d4abedc-ce7e-4323-bf4a-5e1790094095 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1320 microversion: 2.1 time: 0.438701 Sep 27 10:15:14.338230 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2324/4245] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:13 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1320 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:14.343365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7822d21e-200d-41f1-8cee-5d1f0fda5feb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:14.355017 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7822d21e-200d-41f1-8cee-5d1f0fda5feb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1504 microversion: 2.1 time: 0.295200 Sep 27 10:15:14.369635 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1922/4246] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:14 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1504 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:15.365185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-18c9e254-7c2b-4280-b598-8dc95f64ea8c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:15.366233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-18c9e254-7c2b-4280-b598-8dc95f64ea8c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:15.382485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18c9e254-7c2b-4280-b598-8dc95f64ea8c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:15.383006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18c9e254-7c2b-4280-b598-8dc95f64ea8c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:15.385373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18c9e254-7c2b-4280-b598-8dc95f64ea8c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:15.387536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18c9e254-7c2b-4280-b598-8dc95f64ea8c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:15.389792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18c9e254-7c2b-4280-b598-8dc95f64ea8c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:15.391730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18c9e254-7c2b-4280-b598-8dc95f64ea8c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:15.399164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-282cbc24-1a50-49ce-a4f2-21c2558ad688 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:15.400487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-282cbc24-1a50-49ce-a4f2-21c2558ad688 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:15.415866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-282cbc24-1a50-49ce-a4f2-21c2558ad688 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:15.417201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-282cbc24-1a50-49ce-a4f2-21c2558ad688 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:15.419311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-282cbc24-1a50-49ce-a4f2-21c2558ad688 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:15.421073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-282cbc24-1a50-49ce-a4f2-21c2558ad688 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:15.422135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-282cbc24-1a50-49ce-a4f2-21c2558ad688 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:15.423039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-282cbc24-1a50-49ce-a4f2-21c2558ad688 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:15.520013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-282cbc24-1a50-49ce-a4f2-21c2558ad688 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:15:15.754086 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-282cbc24-1a50-49ce-a4f2-21c2558ad688 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "DELETE /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 204 len: 0 microversion: 2.42 time: 0.653837 Sep 27 10:15:15.760782 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2325/4247] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:15:15 2022] DELETE /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 0 bytes in 657 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:15:15.769629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-35a16c43-f789-4bbd-8645-1b5b9bf635c5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:15.770948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-35a16c43-f789-4bbd-8645-1b5b9bf635c5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:15.781372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a16c43-f789-4bbd-8645-1b5b9bf635c5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:15.782359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a16c43-f789-4bbd-8645-1b5b9bf635c5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:15.783019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a16c43-f789-4bbd-8645-1b5b9bf635c5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:15.783887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a16c43-f789-4bbd-8645-1b5b9bf635c5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:15.784487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a16c43-f789-4bbd-8645-1b5b9bf635c5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:15.785324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a16c43-f789-4bbd-8645-1b5b9bf635c5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:15.920442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-18c9e254-7c2b-4280-b598-8dc95f64ea8c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:15.935233 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-18c9e254-7c2b-4280-b598-8dc95f64ea8c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1320 microversion: 2.1 time: 0.574641 Sep 27 10:15:15.936800 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1923/4248] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:15 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1320 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:15.942840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:15.944248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:15.965262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:15.965756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:15.966199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:15.966916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:15.967579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:15.968010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:16.227768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-35a16c43-f789-4bbd-8645-1b5b9bf635c5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:16.237870 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-35a16c43-f789-4bbd-8645-1b5b9bf635c5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1504 microversion: 2.1 time: 0.475159 Sep 27 10:15:16.241483 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2326/4249] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:15 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1504 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:16.471530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:16.486216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:16.490188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:16.494201 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-51709bf0-fb10-4262-a424-e2ea843bc2f0 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2957 microversion: 2.42 time: 0.554876 Sep 27 10:15:16.494825 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1924/4250] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:15 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2957 bytes in 557 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:16.659829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-659372c3-d027-4e9c-b903-1f82a02ddb17 req-e5569698-9270-4487-9554-bb3ef8e18c06 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c74b0db7-5f7b-484b-8d2e-feea845f3264", "name": "network-vif-deleted", "tag": "c16a5b67-0bb0-445b-9ef4-349421b3b566"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:16.682534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-659372c3-d027-4e9c-b903-1f82a02ddb17 req-e5569698-9270-4487-9554-bb3ef8e18c06 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:16.683311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-659372c3-d027-4e9c-b903-1f82a02ddb17 req-e5569698-9270-4487-9554-bb3ef8e18c06 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:16.683998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-659372c3-d027-4e9c-b903-1f82a02ddb17 req-e5569698-9270-4487-9554-bb3ef8e18c06 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:16.712792 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-659372c3-d027-4e9c-b903-1f82a02ddb17 req-e5569698-9270-4487-9554-bb3ef8e18c06 service nova] Creating event network-vif-deleted:c16a5b67-0bb0-445b-9ef4-349421b3b566 for instance c74b0db7-5f7b-484b-8d2e-feea845f3264 on nch2-master-1 Sep 27 10:15:16.735565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-659372c3-d027-4e9c-b903-1f82a02ddb17 req-e5569698-9270-4487-9554-bb3ef8e18c06 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079632 Sep 27 10:15:16.736646 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2327/4251] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:16.962914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e2cc2dd8-3141-4b61-80d0-3f901c792080 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:16.963546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e2cc2dd8-3141-4b61-80d0-3f901c792080 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:16.980980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2cc2dd8-3141-4b61-80d0-3f901c792080 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:16.981367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2cc2dd8-3141-4b61-80d0-3f901c792080 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:16.981816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2cc2dd8-3141-4b61-80d0-3f901c792080 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:16.982554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2cc2dd8-3141-4b61-80d0-3f901c792080 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:16.983028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2cc2dd8-3141-4b61-80d0-3f901c792080 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:16.983503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2cc2dd8-3141-4b61-80d0-3f901c792080 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:17.269013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e6c59369-3442-484d-8b48-9617dd7f780c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:17.269742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e6c59369-3442-484d-8b48-9617dd7f780c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:17.288141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6c59369-3442-484d-8b48-9617dd7f780c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:17.289777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6c59369-3442-484d-8b48-9617dd7f780c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:17.292532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6c59369-3442-484d-8b48-9617dd7f780c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:17.295822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6c59369-3442-484d-8b48-9617dd7f780c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:17.296532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6c59369-3442-484d-8b48-9617dd7f780c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:17.298152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6c59369-3442-484d-8b48-9617dd7f780c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:17.501992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e2cc2dd8-3141-4b61-80d0-3f901c792080 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:17.523362 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e2cc2dd8-3141-4b61-80d0-3f901c792080 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1320 microversion: 2.1 time: 0.564449 Sep 27 10:15:17.524284 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1925/4252] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:16 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1320 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:17.532497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:17.545777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:17.563566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:17.564158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:17.564697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:17.565521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:17.566044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:17.566538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:17.661033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e6c59369-3442-484d-8b48-9617dd7f780c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:17.682781 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e6c59369-3442-484d-8b48-9617dd7f780c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1462 microversion: 2.1 time: 0.419779 Sep 27 10:15:17.686985 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2328/4253] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:17 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1462 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:18.241836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:18.255841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:18.257644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:18.258983 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7babdb80-e64d-472f-abe8-5385db270cc1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2957 microversion: 2.42 time: 0.732239 Sep 27 10:15:18.259669 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1926/4254] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:17 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2957 bytes in 733 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:18.547878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed2b75e3-4508-42b1-a420-664699bfd208 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:18.548941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed2b75e3-4508-42b1-a420-664699bfd208 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:18.566516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed2b75e3-4508-42b1-a420-664699bfd208 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:18.568372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed2b75e3-4508-42b1-a420-664699bfd208 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:18.573999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed2b75e3-4508-42b1-a420-664699bfd208 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:18.576106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed2b75e3-4508-42b1-a420-664699bfd208 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:18.582678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed2b75e3-4508-42b1-a420-664699bfd208 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:18.583550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed2b75e3-4508-42b1-a420-664699bfd208 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:18.712133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:18.712819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:18.740981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:18.744712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:18.749562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:18.750670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:18.752263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:18.757285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:18.924790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=6,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:18.929221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:18.930131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:18.930835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:18.931477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lazy-loading 'fault' on Instance uuid c74b0db7-5f7b-484b-8d2e-feea845f3264 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:15:19.076978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:19.093630 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e47ef06b-123a-48fd-9929-edd305de3739 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 200 len: 1303 microversion: 2.1 time: 0.385719 Sep 27 10:15:19.095138 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1927/4255] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:18 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 1303 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:19.104319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ed2b75e3-4508-42b1-a420-664699bfd208 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:19.115903 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed2b75e3-4508-42b1-a420-664699bfd208 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1320 microversion: 2.1 time: 0.571731 Sep 27 10:15:19.133245 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2329/4256] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:18 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1320 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:19.289179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:19.291821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:19.307427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:19.310242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:19.313530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:19.314578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:19.316891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:19.317581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:19.958105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:19.976321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:19.978107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:19.982718 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-612573bc-3efb-4000-8c55-7f2386708015 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2957 microversion: 2.42 time: 0.699205 Sep 27 10:15:20.005000 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1928/4257] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:19 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2957 bytes in 716 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:20.120053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:20.121333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: c74b0db7-5f7b-484b-8d2e-feea845f3264] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:20.135038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:20.135549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:20.149166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d1eedcc0-e3b7-401e-8824-86c78046090e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:20.149788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d1eedcc0-e3b7-401e-8824-86c78046090e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:20.155450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:20.156532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:20.157085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:20.157511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:20.165304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1eedcc0-e3b7-401e-8824-86c78046090e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:20.165807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1eedcc0-e3b7-401e-8824-86c78046090e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:20.168989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1eedcc0-e3b7-401e-8824-86c78046090e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:20.169794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1eedcc0-e3b7-401e-8824-86c78046090e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:20.170452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1eedcc0-e3b7-401e-8824-86c78046090e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:20.170682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1eedcc0-e3b7-401e-8824-86c78046090e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:20.209305 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] HTTP exception thrown: Instance c74b0db7-5f7b-484b-8d2e-feea845f3264 could not be found. Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Request method failure captured: Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 HTTP/1.1 Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0VRQ9Vwh0k9D0EAs-3tF0VMfIoDIEXdI3eWAXQDXn-Z8TKcLn92Voq3d5s3RtKnblVl635LVxbLNKzN4b9KSrEJ27lRH7pXq2u8sALvQuWlHykNdj3cFOzdA_IhsLJwIn3XBAlRvOt_A_owDCabdp-jPJYg4aAzCUTqVlVNq7ioE8 Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 27c0d3daa08340d89d288bbf811fece9 Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesActionsTest-1551375736 Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/27c0d3daa08340d89d288bbf811fece9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/27c0d3daa08340d89d288bbf811fece9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/27c0d3daa08340d89d288bbf811fece9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_27c0d3daa08340d89d288bbf811fece9"}]}] Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesActionsTest-1551375736 Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 27c0d3daa08340d89d288bbf811fece9 Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesActionsTest-1551375736 Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesActionsTest-1551375736-project Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: dc369ca971424247b96050e09368a7bb Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesActionsTest-1551375736-project Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance c74b0db7-5f7b-484b-8d2e-feea845f3264 could not be found. Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'c74b0db7-5f7b-484b-8d2e-feea845f3264'} Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c74b0db7-5f7b-484b-8d2e-feea845f3264 could not be found. Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c74b0db7-5f7b-484b-8d2e-feea845f3264 could not be found. Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:20.216421 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:20.221404 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:20.221404 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:20.221404 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:20.221404 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:15:20.221404 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:15:20.221404 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c74b0db7-5f7b-484b-8d2e-feea845f3264 could not be found. Sep 27 10:15:20.221404 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:20.221404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Returning 404 to user: Instance c74b0db7-5f7b-484b-8d2e-feea845f3264 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:15:20.225051 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5fb8ab0e-6dab-454c-bffa-6da5c3fd3c1c tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264" status: 404 len: 111 microversion: 2.1 time: 0.103889 Sep 27 10:15:20.226283 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2330/4258] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:20 2022] GET /compute/v2.1/servers/c74b0db7-5f7b-484b-8d2e-feea845f3264 => generated 111 bytes in 111 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:15:20.415138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1437496609", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "925eede8-afc8-48d1-a08f-9f66e37bf771"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:20.541965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] validate_networks() for [('925eede8-afc8-48d1-a08f-9f66e37bf771', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:15:20.703716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d1eedcc0-e3b7-401e-8824-86c78046090e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:20.722999 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d1eedcc0-e3b7-401e-8824-86c78046090e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1320 microversion: 2.1 time: 0.569056 Sep 27 10:15:20.737947 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1929/4259] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:20 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1320 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:21.015019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:21.016492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:21.029053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:21.029512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:21.030132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:21.031061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:21.040797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:21.041458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:21.684574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:15:21.685976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:15:21.686572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:15:21.687113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:15:21.687718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:15:21.707358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:21.708042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:21.708577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:21.709718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:21.710395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:21.724410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:21.771275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Getting quotas for project 27c0d3daa08340d89d288bbf811fece9. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:15:21.778268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Getting quotas for user dc369ca971424247b96050e09368a7bb and project 27c0d3daa08340d89d288bbf811fece9. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:15:21.804164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:15:21.809524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:15:21.884283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:21.898991 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d3b4f9f7-faa8-47c1-ae84-398198090420 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.488943 Sep 27 10:15:21.901297 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2331/4260] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:15:20 2022] POST /compute/v2.1/servers => generated 380 bytes in 1492 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:15:21.912983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d8b78c7c-0152-4b23-8151-b24ab90c7597 req-b0bc5c47-9e8f-4dc8-a87a-35ed57bf8cc0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "430df9fb-7e75-4f60-8d96-0910ec4e8243", "name": "network-vif-plugged", "status": "completed", "tag": "b69beb62-d074-4135-8cde-13301d0421e3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:21.914671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:21.921669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:21.923291 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9611616a-75ce-43a7-b19f-d26170c636ac tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2957 microversion: 2.42 time: 0.912631 Sep 27 10:15:21.924735 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1930/4261] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:21 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2957 bytes in 915 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:21.941266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1e556825-cb6a-46b8-9f11-dfab77cdbbcc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:21.942140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1e556825-cb6a-46b8-9f11-dfab77cdbbcc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:21.956795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e556825-cb6a-46b8-9f11-dfab77cdbbcc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:21.957575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e556825-cb6a-46b8-9f11-dfab77cdbbcc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:21.958159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e556825-cb6a-46b8-9f11-dfab77cdbbcc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:21.961224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e556825-cb6a-46b8-9f11-dfab77cdbbcc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:21.962461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e556825-cb6a-46b8-9f11-dfab77cdbbcc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:21.964699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e556825-cb6a-46b8-9f11-dfab77cdbbcc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:21.978824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d8b78c7c-0152-4b23-8151-b24ab90c7597 req-b0bc5c47-9e8f-4dc8-a87a-35ed57bf8cc0 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:21.979316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d8b78c7c-0152-4b23-8151-b24ab90c7597 req-b0bc5c47-9e8f-4dc8-a87a-35ed57bf8cc0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:21.982497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d8b78c7c-0152-4b23-8151-b24ab90c7597 req-b0bc5c47-9e8f-4dc8-a87a-35ed57bf8cc0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:22.026776 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d8b78c7c-0152-4b23-8151-b24ab90c7597 req-b0bc5c47-9e8f-4dc8-a87a-35ed57bf8cc0 service nova] Creating event network-vif-plugged:b69beb62-d074-4135-8cde-13301d0421e3 for instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 on nch2-master-1 Sep 27 10:15:22.047359 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d8b78c7c-0152-4b23-8151-b24ab90c7597 req-b0bc5c47-9e8f-4dc8-a87a-35ed57bf8cc0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.142735 Sep 27 10:15:22.050869 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2332/4262] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:22.063146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3fb20eca-7b05-405f-819f-868edabe06fb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:22.063572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3fb20eca-7b05-405f-819f-868edabe06fb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:22.323798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3fb20eca-7b05-405f-819f-868edabe06fb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:22.349637 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3fb20eca-7b05-405f-819f-868edabe06fb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1214 microversion: 2.1 time: 0.292798 Sep 27 10:15:22.352379 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2333/4263] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:22 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1214 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:22.916947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1e556825-cb6a-46b8-9f11-dfab77cdbbcc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:22.930544 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1e556825-cb6a-46b8-9f11-dfab77cdbbcc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1320 microversion: 2.1 time: 0.993062 Sep 27 10:15:22.931212 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1931/4264] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:21 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1320 bytes in 1006 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:22.961629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:22.973251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:22.996015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:22.996935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:22.998293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:22.999607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:23.000243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:23.000856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:23.379814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5bd8945a-35a0-44ab-b966-8f09b353eb4f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:23.380721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5bd8945a-35a0-44ab-b966-8f09b353eb4f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:23.410385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bd8945a-35a0-44ab-b966-8f09b353eb4f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:23.410817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bd8945a-35a0-44ab-b966-8f09b353eb4f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:23.411419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bd8945a-35a0-44ab-b966-8f09b353eb4f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:23.413496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bd8945a-35a0-44ab-b966-8f09b353eb4f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:23.419266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bd8945a-35a0-44ab-b966-8f09b353eb4f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:23.420021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bd8945a-35a0-44ab-b966-8f09b353eb4f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:23.540722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5bd8945a-35a0-44ab-b966-8f09b353eb4f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:23.554615 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5bd8945a-35a0-44ab-b966-8f09b353eb4f tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1210 microversion: 2.1 time: 0.182470 Sep 27 10:15:23.555355 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1932/4265] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:23 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1210 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:23.973425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b910ab61-f501-4192-9250-0456db86ee2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:23.980008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b910ab61-f501-4192-9250-0456db86ee2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:24.002717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b910ab61-f501-4192-9250-0456db86ee2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:24.003297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b910ab61-f501-4192-9250-0456db86ee2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:24.003820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b910ab61-f501-4192-9250-0456db86ee2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:24.004797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b910ab61-f501-4192-9250-0456db86ee2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:24.005773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b910ab61-f501-4192-9250-0456db86ee2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:24.006089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b910ab61-f501-4192-9250-0456db86ee2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:24.611075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b910ab61-f501-4192-9250-0456db86ee2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:24.628616 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b910ab61-f501-4192-9250-0456db86ee2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1320 microversion: 2.1 time: 0.676710 Sep 27 10:15:24.634188 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1933/4266] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:23 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1320 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:24.647354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:24.648614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:24.681939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:24.682587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:24.683110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:24.685855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:24.686421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:24.692215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:24.767852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:24.791769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:24.809351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:24.813566 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cd175559-7a77-4e0a-9234-69bd17571fb1 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2915 microversion: 2.42 time: 1.859519 Sep 27 10:15:24.814778 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2334/4267] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:22 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2915 bytes in 1861 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:24.821369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d088789e-af5d-48f4-b1f9-f29dd9b5cb84 req-130062b7-cad4-4549-84e3-0a8198840c34 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-unplugged", "status": "completed", "tag": "dbf7486a-85d8-4b37-a8d2-b1f467142580"}, {"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-unplugged", "status": "completed", "tag": "2b32bc4b-33c7-4c6b-918d-8458f3eee387"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:24.855031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d088789e-af5d-48f4-b1f9-f29dd9b5cb84 req-130062b7-cad4-4549-84e3-0a8198840c34 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:24.857433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d088789e-af5d-48f4-b1f9-f29dd9b5cb84 req-130062b7-cad4-4549-84e3-0a8198840c34 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:24.858119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d088789e-af5d-48f4-b1f9-f29dd9b5cb84 req-130062b7-cad4-4549-84e3-0a8198840c34 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:24.893028 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d088789e-af5d-48f4-b1f9-f29dd9b5cb84 req-130062b7-cad4-4549-84e3-0a8198840c34 service nova] Creating event network-vif-unplugged:dbf7486a-85d8-4b37-a8d2-b1f467142580 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:15:24.894992 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d088789e-af5d-48f4-b1f9-f29dd9b5cb84 req-130062b7-cad4-4549-84e3-0a8198840c34 service nova] Creating event network-vif-unplugged:2b32bc4b-33c7-4c6b-918d-8458f3eee387 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:15:24.901164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lazy-loading 'services' on Instance uuid 49f9e4be-7594-4603-8f69-8765513f8b89 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:15:24.904886 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d088789e-af5d-48f4-b1f9-f29dd9b5cb84 req-130062b7-cad4-4549-84e3-0a8198840c34 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 358 microversion: 2.1 time: 0.086540 Sep 27 10:15:24.906168 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2335/4268] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:24 2022] POST /compute/v2.1/os-server-external-events => generated 358 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:24.914154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b9bc4fd-9feb-435c-9487-345b3047e690 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:24.915317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1b9bc4fd-9feb-435c-9487-345b3047e690 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:24.923519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b9bc4fd-9feb-435c-9487-345b3047e690 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:24.924256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b9bc4fd-9feb-435c-9487-345b3047e690 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:24.924896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b9bc4fd-9feb-435c-9487-345b3047e690 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:24.925795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b9bc4fd-9feb-435c-9487-345b3047e690 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:24.926426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b9bc4fd-9feb-435c-9487-345b3047e690 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:24.927051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b9bc4fd-9feb-435c-9487-345b3047e690 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:25.067573 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e5ca1faf-6272-4701-bce4-8f1eaf197fbd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "DELETE /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/os-volume_attachments/e5af107c-5b71-461c-a3e1-deb10425a0f9" status: 202 len: 0 microversion: 2.1 time: 0.431718 Sep 27 10:15:25.073869 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1934/4269] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:15:24 2022] DELETE /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/os-volume_attachments/e5af107c-5b71-461c-a3e1-deb10425a0f9 => generated 0 bytes in 438 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:15:25.130801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1b9bc4fd-9feb-435c-9487-345b3047e690 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:25.175814 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1b9bc4fd-9feb-435c-9487-345b3047e690 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1272 microversion: 2.1 time: 0.251349 Sep 27 10:15:25.186988 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2336/4270] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:24 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1272 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:25.663901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e055767d-b64f-4340-8109-63a7641ef925 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:25.665922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e055767d-b64f-4340-8109-63a7641ef925 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:25.686354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e055767d-b64f-4340-8109-63a7641ef925 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:25.687065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e055767d-b64f-4340-8109-63a7641ef925 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:25.688392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e055767d-b64f-4340-8109-63a7641ef925 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:25.689687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e055767d-b64f-4340-8109-63a7641ef925 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:25.690508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e055767d-b64f-4340-8109-63a7641ef925 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:25.691400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e055767d-b64f-4340-8109-63a7641ef925 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:25.854137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:25.856438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:25.873561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:25.874769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:25.875883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:25.877180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:25.880574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:25.887775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:26.092688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e055767d-b64f-4340-8109-63a7641ef925 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:26.126083 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e055767d-b64f-4340-8109-63a7641ef925 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1500 microversion: 2.1 time: 0.466571 Sep 27 10:15:26.127895 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1935/4271] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:25 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1500 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:26.150024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ec06e15-29ef-4483-a884-458c369cd258 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:26.150863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ec06e15-29ef-4483-a884-458c369cd258 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:26.157609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec06e15-29ef-4483-a884-458c369cd258 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:26.158400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec06e15-29ef-4483-a884-458c369cd258 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:26.159230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec06e15-29ef-4483-a884-458c369cd258 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:26.160235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec06e15-29ef-4483-a884-458c369cd258 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:26.160910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec06e15-29ef-4483-a884-458c369cd258 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:26.161511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec06e15-29ef-4483-a884-458c369cd258 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:26.186342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:26.513364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:26.538448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:26.542426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:26.550118 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba708a3b-4724-4996-a856-548b015b0cc9 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2869 microversion: 2.42 time: 0.702987 Sep 27 10:15:26.551601 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2337/4272] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:25 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2869 bytes in 707 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:26.560529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9fede8d4-887d-433a-9d1a-9ab7750ffed4 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:26.562351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9fede8d4-887d-433a-9d1a-9ab7750ffed4 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:26.584430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fede8d4-887d-433a-9d1a-9ab7750ffed4 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:26.585291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fede8d4-887d-433a-9d1a-9ab7750ffed4 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:26.586062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fede8d4-887d-433a-9d1a-9ab7750ffed4 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:26.586962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fede8d4-887d-433a-9d1a-9ab7750ffed4 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:26.588100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fede8d4-887d-433a-9d1a-9ab7750ffed4 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:26.588739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fede8d4-887d-433a-9d1a-9ab7750ffed4 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:26.697095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9fede8d4-887d-433a-9d1a-9ab7750ffed4 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:26.713642 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9fede8d4-887d-433a-9d1a-9ab7750ffed4 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1272 microversion: 2.1 time: 0.160480 Sep 27 10:15:26.715178 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2338/4273] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:26 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1272 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:26.719960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6aa1ec67-70a8-4143-9603-7be75610c593 req-e571daf8-f9ab-4306-b33a-14c4e92849ab service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-deleted", "tag": "73444bba-797d-4fc6-88a3-e126deb3c605"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:26.739819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6aa1ec67-70a8-4143-9603-7be75610c593 req-e571daf8-f9ab-4306-b33a-14c4e92849ab service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:26.740838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6aa1ec67-70a8-4143-9603-7be75610c593 req-e571daf8-f9ab-4306-b33a-14c4e92849ab service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:26.741452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6aa1ec67-70a8-4143-9603-7be75610c593 req-e571daf8-f9ab-4306-b33a-14c4e92849ab service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:26.755537 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6aa1ec67-70a8-4143-9603-7be75610c593 req-e571daf8-f9ab-4306-b33a-14c4e92849ab service nova] Creating event network-vif-deleted:73444bba-797d-4fc6-88a3-e126deb3c605 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:15:26.761529 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6aa1ec67-70a8-4143-9603-7be75610c593 req-e571daf8-f9ab-4306-b33a-14c4e92849ab service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044968 Sep 27 10:15:26.763664 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2339/4274] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:26.772785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f72a45b8-8857-446e-abf6-0ee90ec00804 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:26.783340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f72a45b8-8857-446e-abf6-0ee90ec00804 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:26.784417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f72a45b8-8857-446e-abf6-0ee90ec00804 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:26.785043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f72a45b8-8857-446e-abf6-0ee90ec00804 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:26.824633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f72a45b8-8857-446e-abf6-0ee90ec00804 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Going to try to stop instance {{(pid=145968) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 10:15:26.914213 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f72a45b8-8857-446e-abf6-0ee90ec00804 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "POST /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/action" status: 202 len: 0 microversion: 2.1 time: 0.148191 Sep 27 10:15:26.936411 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2340/4275] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:15:26 2022] POST /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/action => generated 0 bytes in 172 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:15:26.979273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2ec06e15-29ef-4483-a884-458c369cd258 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:26.988842 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ec06e15-29ef-4483-a884-458c369cd258 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1500 microversion: 2.1 time: 0.842637 Sep 27 10:15:26.993198 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1936/4276] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:26 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1500 bytes in 848 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:27.016860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25604ef1-d039-45aa-bd5a-e011c855015b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:27.018964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-25604ef1-d039-45aa-bd5a-e011c855015b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:27.070989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25604ef1-d039-45aa-bd5a-e011c855015b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:27.071524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25604ef1-d039-45aa-bd5a-e011c855015b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:27.073175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25604ef1-d039-45aa-bd5a-e011c855015b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:27.076963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25604ef1-d039-45aa-bd5a-e011c855015b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:27.077479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25604ef1-d039-45aa-bd5a-e011c855015b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:27.078231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25604ef1-d039-45aa-bd5a-e011c855015b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:27.255096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestMultiTenantJSON-server-797289382", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "bca35e23-a9b6-4c56-ac63-30d4627c354f"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:27.348164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] validate_networks() for [('bca35e23-a9b6-4c56-ac63-30d4627c354f', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:15:27.830536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-25604ef1-d039-45aa-bd5a-e011c855015b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:27.843369 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-25604ef1-d039-45aa-bd5a-e011c855015b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1677 microversion: 2.1 time: 0.845756 Sep 27 10:15:27.844009 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1937/4277] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:26 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1677 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:27.866371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-bb57d6a5-0418-4d71-bbb2-cfd1c35d4238 req-524637ce-dac4-4a06-801a-27ccaa656254 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-unplugged", "status": "completed", "tag": "8cd8623a-a3f6-4d1c-aced-9707f8353fa6"}, {"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-unplugged", "status": "completed", "tag": "bcc2c185-ff79-4bda-a3f2-d915e7d3752c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:27.888991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bb57d6a5-0418-4d71-bbb2-cfd1c35d4238 req-524637ce-dac4-4a06-801a-27ccaa656254 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:27.889473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bb57d6a5-0418-4d71-bbb2-cfd1c35d4238 req-524637ce-dac4-4a06-801a-27ccaa656254 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:27.890010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bb57d6a5-0418-4d71-bbb2-cfd1c35d4238 req-524637ce-dac4-4a06-801a-27ccaa656254 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:27.905872 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-bb57d6a5-0418-4d71-bbb2-cfd1c35d4238 req-524637ce-dac4-4a06-801a-27ccaa656254 service nova] Creating event network-vif-unplugged:8cd8623a-a3f6-4d1c-aced-9707f8353fa6 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:15:27.906379 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-bb57d6a5-0418-4d71-bbb2-cfd1c35d4238 req-524637ce-dac4-4a06-801a-27ccaa656254 service nova] Creating event network-vif-unplugged:bcc2c185-ff79-4bda-a3f2-d915e7d3752c for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:15:27.924930 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-bb57d6a5-0418-4d71-bbb2-cfd1c35d4238 req-524637ce-dac4-4a06-801a-27ccaa656254 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 358 microversion: 2.1 time: 0.066706 Sep 27 10:15:27.925877 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1938/4278] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:27 2022] POST /compute/v2.1/os-server-external-events => generated 358 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:27.931578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2bb15991-4fea-4164-beda-3d0b222e083e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:27.932166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2bb15991-4fea-4164-beda-3d0b222e083e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:27.956546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bb15991-4fea-4164-beda-3d0b222e083e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:27.956994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bb15991-4fea-4164-beda-3d0b222e083e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:27.957434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bb15991-4fea-4164-beda-3d0b222e083e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:27.958155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bb15991-4fea-4164-beda-3d0b222e083e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:27.958610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bb15991-4fea-4164-beda-3d0b222e083e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:27.959068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bb15991-4fea-4164-beda-3d0b222e083e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:28.014690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2bb15991-4fea-4164-beda-3d0b222e083e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:15:28.231024 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2bb15991-4fea-4164-beda-3d0b222e083e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 204 len: 0 microversion: 2.1 time: 0.302616 Sep 27 10:15:28.233026 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1939/4279] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:15:27 2022] DELETE /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:15:28.244727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:28.246635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:28.254469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:28.255321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:28.256041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:28.257997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:28.258723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:28.259402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:28.839025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:28.855840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:28.867863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:28.869587 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1e23730f-b5bd-43b8-8097-76e6ffa190f6 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2755 microversion: 2.42 time: 0.634372 Sep 27 10:15:28.873462 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1940/4280] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:28 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2755 bytes in 639 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:28.881438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-884f00bc-a3a4-45b0-b582-c00bbdf5171d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:28.882354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-884f00bc-a3a4-45b0-b582-c00bbdf5171d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:28.889214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-884f00bc-a3a4-45b0-b582-c00bbdf5171d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:28.890092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-884f00bc-a3a4-45b0-b582-c00bbdf5171d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:28.890985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-884f00bc-a3a4-45b0-b582-c00bbdf5171d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:28.896429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-884f00bc-a3a4-45b0-b582-c00bbdf5171d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:28.897888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-884f00bc-a3a4-45b0-b582-c00bbdf5171d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:28.898674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-884f00bc-a3a4-45b0-b582-c00bbdf5171d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:29.248381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-884f00bc-a3a4-45b0-b582-c00bbdf5171d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:29.265600 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-884f00bc-a3a4-45b0-b582-c00bbdf5171d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1314 microversion: 2.1 time: 0.390610 Sep 27 10:15:29.285343 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1941/4281] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:28 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1314 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:29.296233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ceedaa20-31fe-4510-9198-7783e68257f2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:29.296559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ceedaa20-31fe-4510-9198-7783e68257f2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:29.324196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceedaa20-31fe-4510-9198-7783e68257f2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:29.325083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceedaa20-31fe-4510-9198-7783e68257f2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:29.325670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceedaa20-31fe-4510-9198-7783e68257f2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:29.326404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceedaa20-31fe-4510-9198-7783e68257f2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:29.326924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceedaa20-31fe-4510-9198-7783e68257f2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:29.327488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceedaa20-31fe-4510-9198-7783e68257f2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:29.887934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:15:29.890314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:15:29.893367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:15:29.894041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:15:29.894613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:15:29.907323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:29.908226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:29.908890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:29.909935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:29.910736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:29.911342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:29.925358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ceedaa20-31fe-4510-9198-7783e68257f2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:29.937848 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ceedaa20-31fe-4510-9198-7783e68257f2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1506 microversion: 2.1 time: 0.651187 Sep 27 10:15:29.938454 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1942/4282] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:29 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1506 bytes in 653 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:29.954105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Getting quotas for project 4a125c6e9abd46b68b5b5c9cea53ec96. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:15:29.961055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-692c6be6-4802-45ae-a14b-cb24624e3dfd req-57c89fb3-925c-445c-be55-d02df8680e67 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-deleted", "tag": "2b32bc4b-33c7-4c6b-918d-8458f3eee387"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:29.965341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Getting quotas for user 02c331abed624c5f88835fa9fdfcfa5c and project 4a125c6e9abd46b68b5b5c9cea53ec96. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:15:29.984496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-692c6be6-4802-45ae-a14b-cb24624e3dfd req-57c89fb3-925c-445c-be55-d02df8680e67 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:29.985181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-692c6be6-4802-45ae-a14b-cb24624e3dfd req-57c89fb3-925c-445c-be55-d02df8680e67 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:29.985976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-692c6be6-4802-45ae-a14b-cb24624e3dfd req-57c89fb3-925c-445c-be55-d02df8680e67 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:29.995070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:15:30.001030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:15:30.011597 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-692c6be6-4802-45ae-a14b-cb24624e3dfd req-57c89fb3-925c-445c-be55-d02df8680e67 service nova] Creating event network-vif-deleted:2b32bc4b-33c7-4c6b-918d-8458f3eee387 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:15:30.022960 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-692c6be6-4802-45ae-a14b-cb24624e3dfd req-57c89fb3-925c-445c-be55-d02df8680e67 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068775 Sep 27 10:15:30.024592 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1943/4283] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:30.040400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a0612a3e-42a6-45ba-aaae-c9b885edaa3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:30.041445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a0612a3e-42a6-45ba-aaae-c9b885edaa3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:30.055396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0612a3e-42a6-45ba-aaae-c9b885edaa3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:30.056167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0612a3e-42a6-45ba-aaae-c9b885edaa3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:30.065401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0612a3e-42a6-45ba-aaae-c9b885edaa3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:30.066507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0612a3e-42a6-45ba-aaae-c9b885edaa3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:30.067174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0612a3e-42a6-45ba-aaae-c9b885edaa3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:30.067761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0612a3e-42a6-45ba-aaae-c9b885edaa3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:30.101977 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d9e957e8-6c8b-4351-99d2-560d1936571a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.156990 Sep 27 10:15:30.103569 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2341/4284] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:15:26 2022] POST /compute/v2.1/servers => generated 380 bytes in 3166 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:15:30.112999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:30.114599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:30.135217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:30.136119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:30.136844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:30.137971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:30.139141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:30.139904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:30.596590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a0612a3e-42a6-45ba-aaae-c9b885edaa3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:30.612332 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a0612a3e-42a6-45ba-aaae-c9b885edaa3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1677 microversion: 2.1 time: 0.582522 Sep 27 10:15:30.619924 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1944/4285] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:30 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1677 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:30.635217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a12c95df-2168-4efc-bdab-2201abe80713 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:30.635847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a12c95df-2168-4efc-bdab-2201abe80713 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:30.825070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a12c95df-2168-4efc-bdab-2201abe80713 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:30.841324 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a12c95df-2168-4efc-bdab-2201abe80713 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1227 microversion: 2.1 time: 0.220209 Sep 27 10:15:30.842489 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1945/4286] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:30 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1227 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:30.853197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8e3873a-98ac-4278-ac6b-f09e9c1fe09d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:30.856281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f8e3873a-98ac-4278-ac6b-f09e9c1fe09d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:30.887780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8e3873a-98ac-4278-ac6b-f09e9c1fe09d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:30.892152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:30.897281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8e3873a-98ac-4278-ac6b-f09e9c1fe09d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:30.898041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8e3873a-98ac-4278-ac6b-f09e9c1fe09d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:30.911426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:30.918561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8e3873a-98ac-4278-ac6b-f09e9c1fe09d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:30.918775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:30.920018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8e3873a-98ac-4278-ac6b-f09e9c1fe09d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:30.921296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8e3873a-98ac-4278-ac6b-f09e9c1fe09d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:30.930765 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d1b35e2-db99-4639-80a0-ac6139dce5ff tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2596 microversion: 2.42 time: 0.824849 Sep 27 10:15:30.934192 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2342/4287] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:30 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2596 bytes in 830 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:30.970420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d4948c3-ab15-4bb9-972c-10ab35797790 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:30.971582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9d4948c3-ab15-4bb9-972c-10ab35797790 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:30.995743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d4948c3-ab15-4bb9-972c-10ab35797790 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:30.997259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d4948c3-ab15-4bb9-972c-10ab35797790 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:30.999236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d4948c3-ab15-4bb9-972c-10ab35797790 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:31.005274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d4948c3-ab15-4bb9-972c-10ab35797790 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:31.009314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d4948c3-ab15-4bb9-972c-10ab35797790 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:31.012055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d4948c3-ab15-4bb9-972c-10ab35797790 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:31.592147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f8e3873a-98ac-4278-ac6b-f09e9c1fe09d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:31.599438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9d4948c3-ab15-4bb9-972c-10ab35797790 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:31.615474 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d4948c3-ab15-4bb9-972c-10ab35797790 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1506 microversion: 2.1 time: 0.649095 Sep 27 10:15:31.621419 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f8e3873a-98ac-4278-ac6b-f09e9c1fe09d tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1314 microversion: 2.1 time: 0.777648 Sep 27 10:15:31.622210 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2343/4288] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:30 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1506 bytes in 659 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:31.623613 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1946/4289] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:30 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1314 bytes in 780 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:31.650517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9696519b-94da-4672-809c-045dbc626441 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:31.651142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9696519b-94da-4672-809c-045dbc626441 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:31.657946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9696519b-94da-4672-809c-045dbc626441 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:31.658442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9696519b-94da-4672-809c-045dbc626441 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:31.667909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9696519b-94da-4672-809c-045dbc626441 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:31.668590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9696519b-94da-4672-809c-045dbc626441 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:31.672368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9696519b-94da-4672-809c-045dbc626441 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:31.672900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9696519b-94da-4672-809c-045dbc626441 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:31.769193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:31.866139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5795879b-fd9d-4018-b568-c4b2961240e8 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:31.866766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5795879b-fd9d-4018-b568-c4b2961240e8 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:31.878675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5795879b-fd9d-4018-b568-c4b2961240e8 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:31.879182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5795879b-fd9d-4018-b568-c4b2961240e8 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:31.890756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5795879b-fd9d-4018-b568-c4b2961240e8 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:31.898050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5795879b-fd9d-4018-b568-c4b2961240e8 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:31.898730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5795879b-fd9d-4018-b568-c4b2961240e8 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:31.899181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5795879b-fd9d-4018-b568-c4b2961240e8 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:32.219469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5795879b-fd9d-4018-b568-c4b2961240e8 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:32.234675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9696519b-94da-4672-809c-045dbc626441 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:32.238862 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5795879b-fd9d-4018-b568-c4b2961240e8 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1279 microversion: 2.1 time: 0.372725 Sep 27 10:15:32.239531 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1947/4290] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:31 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1279 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:32.258343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:32.259480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:32.265120 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9696519b-94da-4672-809c-045dbc626441 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1677 microversion: 2.1 time: 0.619643 Sep 27 10:15:32.275391 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2344/4291] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:31 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1677 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:32.279180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:32.280211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:32.281062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:32.285883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:32.286524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:32.290684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:32.305369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-15929d98-b4c7-43a9-9bf2-97a62eb22d3d req-0c389fe1-eb5a-4710-941a-0e7761157b0f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-deleted", "tag": "dbf7486a-85d8-4b37-a8d2-b1f467142580"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:32.335227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-15929d98-b4c7-43a9-9bf2-97a62eb22d3d req-0c389fe1-eb5a-4710-941a-0e7761157b0f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:32.336059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-15929d98-b4c7-43a9-9bf2-97a62eb22d3d req-0c389fe1-eb5a-4710-941a-0e7761157b0f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:32.336672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-15929d98-b4c7-43a9-9bf2-97a62eb22d3d req-0c389fe1-eb5a-4710-941a-0e7761157b0f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:32.391916 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-15929d98-b4c7-43a9-9bf2-97a62eb22d3d req-0c389fe1-eb5a-4710-941a-0e7761157b0f service nova] Creating event network-vif-deleted:dbf7486a-85d8-4b37-a8d2-b1f467142580 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:15:32.401130 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-15929d98-b4c7-43a9-9bf2-97a62eb22d3d req-0c389fe1-eb5a-4710-941a-0e7761157b0f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.108280 Sep 27 10:15:32.402076 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2345/4292] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:32.645438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-768a97e2-039e-4b2c-acff-238d460f5c25 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:32.648537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-768a97e2-039e-4b2c-acff-238d460f5c25 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:32.657642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=2,nova_api:INSERT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:32.707530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-768a97e2-039e-4b2c-acff-238d460f5c25 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:32.709515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-768a97e2-039e-4b2c-acff-238d460f5c25 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:32.710103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-768a97e2-039e-4b2c-acff-238d460f5c25 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:32.718623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-768a97e2-039e-4b2c-acff-238d460f5c25 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:32.720797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-768a97e2-039e-4b2c-acff-238d460f5c25 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:32.721216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-768a97e2-039e-4b2c-acff-238d460f5c25 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:33.138468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-768a97e2-039e-4b2c-acff-238d460f5c25 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:33.149649 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-768a97e2-039e-4b2c-acff-238d460f5c25 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1506 microversion: 2.1 time: 0.513481 Sep 27 10:15:33.150260 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2346/4293] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:32 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1506 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:33.161430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f93ab62-3e0c-48a6-ae52-7067b7381421 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:33.165564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9f93ab62-3e0c-48a6-ae52-7067b7381421 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:33.191859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f93ab62-3e0c-48a6-ae52-7067b7381421 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:33.193256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f93ab62-3e0c-48a6-ae52-7067b7381421 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:33.195420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f93ab62-3e0c-48a6-ae52-7067b7381421 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:33.196337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f93ab62-3e0c-48a6-ae52-7067b7381421 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:33.197725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f93ab62-3e0c-48a6-ae52-7067b7381421 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:33.199262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f93ab62-3e0c-48a6-ae52-7067b7381421 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:33.304755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:33.316142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:33.317607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:33.321829 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-196ea3a7-f1e7-492f-9e66-faddc1892f73 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2550 microversion: 2.42 time: 1.070014 Sep 27 10:15:33.345239 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1948/4294] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:32 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2550 bytes in 1104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:33.351290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-247dde58-2f24-45c5-af7d-a882f8bff4dd tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:33.352246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-247dde58-2f24-45c5-af7d-a882f8bff4dd tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:33.358741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-247dde58-2f24-45c5-af7d-a882f8bff4dd tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:33.359440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-247dde58-2f24-45c5-af7d-a882f8bff4dd tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:33.360040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-247dde58-2f24-45c5-af7d-a882f8bff4dd tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:33.361112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-247dde58-2f24-45c5-af7d-a882f8bff4dd tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:33.361716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-247dde58-2f24-45c5-af7d-a882f8bff4dd tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:33.362426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-247dde58-2f24-45c5-af7d-a882f8bff4dd tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:33.508983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-247dde58-2f24-45c5-af7d-a882f8bff4dd tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:33.526383 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-247dde58-2f24-45c5-af7d-a882f8bff4dd tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1285 microversion: 2.1 time: 0.179696 Sep 27 10:15:33.527655 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1949/4295] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:33 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1285 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:33.533752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-44287d8e-951f-4518-b035-61709232c63f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:33.535346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-44287d8e-951f-4518-b035-61709232c63f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:33.547635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44287d8e-951f-4518-b035-61709232c63f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:33.549745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44287d8e-951f-4518-b035-61709232c63f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:33.550516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44287d8e-951f-4518-b035-61709232c63f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:33.568952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44287d8e-951f-4518-b035-61709232c63f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:33.571926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44287d8e-951f-4518-b035-61709232c63f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:33.572619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44287d8e-951f-4518-b035-61709232c63f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:33.799535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9f93ab62-3e0c-48a6-ae52-7067b7381421 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:33.826734 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9f93ab62-3e0c-48a6-ae52-7067b7381421 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1314 microversion: 2.1 time: 0.670780 Sep 27 10:15:33.829615 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2347/4296] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:33 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1314 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:34.177669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:34.178438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:34.195414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:34.196548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:34.197247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:34.198125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:34.199171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:34.199677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:34.236199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:34.236977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:34.237592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:34.238194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lazy-loading 'fault' on Instance uuid 430df9fb-7e75-4f60-8d96-0910ec4e8243 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:15:34.334107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:34.344161 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-63f5e69f-1449-40ef-8ab0-d61e2d0ccd8e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 200 len: 1310 microversion: 2.1 time: 0.170370 Sep 27 10:15:34.357618 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2348/4297] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:34 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 1310 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:34.370538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:34.371413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:34.381533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:34.388363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:34.399552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:34.401604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:34.402868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:34.407071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:34.591500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-44287d8e-951f-4518-b035-61709232c63f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:34.616854 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-44287d8e-951f-4518-b035-61709232c63f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1677 microversion: 2.1 time: 1.087523 Sep 27 10:15:34.617458 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1950/4298] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:33 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1677 bytes in 1089 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:34.634849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e39314e1-fc69-43e3-807b-3d5f9da461e8 req-e70bdc90-4184-406b-8f80-fa494580ccf0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "168c4335-87f5-4cce-b73b-f525c65f5166", "tag": "dcc4c715-0cd8-4eaf-a0ad-3f50253460d0"}, {"server_uuid": "430df9fb-7e75-4f60-8d96-0910ec4e8243", "name": "network-vif-deleted", "tag": "b69beb62-d074-4135-8cde-13301d0421e3"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:34.663280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e39314e1-fc69-43e3-807b-3d5f9da461e8 req-e70bdc90-4184-406b-8f80-fa494580ccf0 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:34.664545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e39314e1-fc69-43e3-807b-3d5f9da461e8 req-e70bdc90-4184-406b-8f80-fa494580ccf0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:34.665268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e39314e1-fc69-43e3-807b-3d5f9da461e8 req-e70bdc90-4184-406b-8f80-fa494580ccf0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:34.701675 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e39314e1-fc69-43e3-807b-3d5f9da461e8 req-e70bdc90-4184-406b-8f80-fa494580ccf0 service nova] Creating event network-changed:dcc4c715-0cd8-4eaf-a0ad-3f50253460d0 for instance 168c4335-87f5-4cce-b73b-f525c65f5166 on nch1-master-1 Sep 27 10:15:34.702352 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e39314e1-fc69-43e3-807b-3d5f9da461e8 req-e70bdc90-4184-406b-8f80-fa494580ccf0 service nova] Creating event network-vif-deleted:b69beb62-d074-4135-8cde-13301d0421e3 for instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 on nch2-master-1 Sep 27 10:15:34.721332 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e39314e1-fc69-43e3-807b-3d5f9da461e8 req-e70bdc90-4184-406b-8f80-fa494580ccf0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.099120 Sep 27 10:15:34.722760 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1951/4299] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:34 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:34.739892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da020407-4c5c-4e63-a119-891bec860ef3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:34.743236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-da020407-4c5c-4e63-a119-891bec860ef3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:34.750646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da020407-4c5c-4e63-a119-891bec860ef3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:34.751225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da020407-4c5c-4e63-a119-891bec860ef3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:34.752046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da020407-4c5c-4e63-a119-891bec860ef3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:34.753061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da020407-4c5c-4e63-a119-891bec860ef3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:34.753733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da020407-4c5c-4e63-a119-891bec860ef3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:34.754357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da020407-4c5c-4e63-a119-891bec860ef3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:34.768021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:34.782386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:34.784278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:34.785630 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bb990de9-f22b-4d3d-a312-78492bbeb68a tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2346 microversion: 2.42 time: 0.426602 Sep 27 10:15:34.786537 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2349/4300] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:34 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2346 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:34.851713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2019b377-ce76-48cc-bd70-39c91b3aad7a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:34.852357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2019b377-ce76-48cc-bd70-39c91b3aad7a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:34.858756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2019b377-ce76-48cc-bd70-39c91b3aad7a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:34.859246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2019b377-ce76-48cc-bd70-39c91b3aad7a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:34.859684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2019b377-ce76-48cc-bd70-39c91b3aad7a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:34.860479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2019b377-ce76-48cc-bd70-39c91b3aad7a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:34.860961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2019b377-ce76-48cc-bd70-39c91b3aad7a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:34.861690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2019b377-ce76-48cc-bd70-39c91b3aad7a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:34.899604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da020407-4c5c-4e63-a119-891bec860ef3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:34.922745 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-da020407-4c5c-4e63-a119-891bec860ef3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1285 microversion: 2.1 time: 0.198474 Sep 27 10:15:34.925349 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1952/4301] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:34 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1285 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:35.019520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerAddressesNegativeTestJSON-server-552932344", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "2447b5ca-ed96-4532-bcc1-3470e951a9d7"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:35.032283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2019b377-ce76-48cc-bd70-39c91b3aad7a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:35.042302 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2019b377-ce76-48cc-bd70-39c91b3aad7a tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1314 microversion: 2.1 time: 0.194618 Sep 27 10:15:35.058357 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2350/4302] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:34 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1314 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:35.103529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] validate_networks() for [('2447b5ca-ed96-4532-bcc1-3470e951a9d7', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:15:35.373558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:35.374452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:35.383485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:35.384489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:35.385075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:35.385913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:35.386441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:35.386934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:35.403076 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 HTTP/1.1 Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '430df9fb-7e75-4f60-8d96-0910ec4e8243'} Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:35.404915 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:35.408706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:35.408706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:35.408706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:35.408706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:15:35.408706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:15:35.408706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:15:35.408706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:35.408706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:15:35.408706 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9e225bee-0213-43cf-9694-547aa03d2cf3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 404 len: 111 microversion: 2.1 time: 0.036988 Sep 27 10:15:35.408706 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2351/4303] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:35 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:15:35.424128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3846be80-903b-4955-9269-cdee0362491f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'create', calling method: .version_select at 0x7f7f62529dc0>, body: {"keypair": {"name": "tempest-key-848436781", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCV3RAhJU+pmBS18qppWxUSLPufGGGg4PLs+vRntlVjueaJmLsimQv4ZJOWRsTmUmfsB0PtT9mr9SJGCLNKoHWwQUozrBO6ePaGymLqn8dHxvtZ55bwjB529Fb62f3s9oQ=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:35.440126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3846be80-903b-4955-9269-cdee0362491f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for project 1d229e5b879b481082284236c9f8c446. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:15:35.446318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3846be80-903b-4955-9269-cdee0362491f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for user a7b8599d0b40424087e52d8e35b71ec4 and project 1d229e5b879b481082284236c9f8c446. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:15:35.473948 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3846be80-903b-4955-9269-cdee0362491f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 381 microversion: 2.1 time: 0.053888 Sep 27 10:15:35.474643 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2352/4304] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:15:35 2022] POST /compute/v2.1/os-keypairs => generated 381 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:35.487874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a3b32eba-3a6a-4973-bf7f-a048c63f2cb1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '.version_select at 0x7f7f62529ca0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:35.492110 ncdv-master-1 devstack@n-api.service[145968]: WARNING oslo_db.sqlalchemy.utils [None req-a3b32eba-3a6a-4973-bf7f-a048c63f2cb1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Unique keys not in sort_keys. The sorting order may be unstable. Sep 27 10:15:35.499315 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a3b32eba-3a6a-4973-bf7f-a048c63f2cb1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/os-keypairs" status: 200 len: 350 microversion: 2.1 time: 0.016304 Sep 27 10:15:35.499995 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2353/4305] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:15:35 2022] GET /compute/v2.1/os-keypairs => generated 350 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:35.642647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-edbeec4e-0c25-4cad-8539-4ac6e219bb10 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:35.643265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-edbeec4e-0c25-4cad-8539-4ac6e219bb10 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:35.649936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbeec4e-0c25-4cad-8539-4ac6e219bb10 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:35.650431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbeec4e-0c25-4cad-8539-4ac6e219bb10 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:35.650869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbeec4e-0c25-4cad-8539-4ac6e219bb10 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:35.651581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbeec4e-0c25-4cad-8539-4ac6e219bb10 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:35.652039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbeec4e-0c25-4cad-8539-4ac6e219bb10 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:35.652463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edbeec4e-0c25-4cad-8539-4ac6e219bb10 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:35.821192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-edbeec4e-0c25-4cad-8539-4ac6e219bb10 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:35.831977 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-edbeec4e-0c25-4cad-8539-4ac6e219bb10 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1677 microversion: 2.1 time: 0.193029 Sep 27 10:15:35.851147 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2354/4306] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:35 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1677 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:35.856807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-327969323", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "key_name": "tempest-key-848436781", "networks": [{"uuid": "662e23c5-f7e1-4e14-af6c-49d672076632"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:35.924197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] validate_networks() for [('662e23c5-f7e1-4e14-af6c-49d672076632', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:15:36.533923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:15:36.538818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:15:36.541219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:15:36.542931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:15:36.546230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:15:36.576596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:36.577416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:36.582277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:36.583702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:36.584431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:36.585010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:36.618319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:15:36.637045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for user a7b8599d0b40424087e52d8e35b71ec4 and project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:15:36.669921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:15:36.673250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:15:36.760770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:15:36.766677 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dff2b422-10a9-4d38-ae95-35e0d0c0d57f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.913906 Sep 27 10:15:36.767587 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2355/4307] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:15:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 916 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:15:36.774796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:15:36.774796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:15:36.774796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:15:36.774796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:15:36.782773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:36.783812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:36.793690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:36.795001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:36.797939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:36.798998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:36.799630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:36.800221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:36.812197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:36.812827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:36.813288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:36.814122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:36.818152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:36.820437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:36.879085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Getting quotas for project 4b17e56b43f04d14b1c5135d1566bef9. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:15:36.898064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Getting quotas for user 219bef5cfcfc496892611dce4bc1d89d and project 4b17e56b43f04d14b1c5135d1566bef9. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:15:36.927890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:15:36.930872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:15:37.055587 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4a07fb27-e494-495f-b486-76e1983be430 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.128329 Sep 27 10:15:37.057937 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1953/4308] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:15:34 2022] POST /compute/v2.1/servers => generated 380 bytes in 2131 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:15:37.066685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e88212a5-918d-4053-9521-67899ae4800e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:37.067649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e88212a5-918d-4053-9521-67899ae4800e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:37.078435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:37.080616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e88212a5-918d-4053-9521-67899ae4800e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:37.095329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:37.097052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:37.102822 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67739361-8ac4-47b4-8bf6-782f5cc13468 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2279 microversion: 2.42 time: 0.326019 Sep 27 10:15:37.113514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e88212a5-918d-4053-9521-67899ae4800e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.035s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:37.116316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e88212a5-918d-4053-9521-67899ae4800e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:37.118090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e88212a5-918d-4053-9521-67899ae4800e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:37.120337 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2356/4309] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:36 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2279 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:37.121612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e88212a5-918d-4053-9521-67899ae4800e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:37.128248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e88212a5-918d-4053-9521-67899ae4800e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:37.135100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e38dddf7-b5dd-4284-8364-8c9e555c1bcf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:37.135793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e38dddf7-b5dd-4284-8364-8c9e555c1bcf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:37.143812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e38dddf7-b5dd-4284-8364-8c9e555c1bcf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:37.144313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e38dddf7-b5dd-4284-8364-8c9e555c1bcf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:37.144814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e38dddf7-b5dd-4284-8364-8c9e555c1bcf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:37.145549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e38dddf7-b5dd-4284-8364-8c9e555c1bcf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:37.146001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e38dddf7-b5dd-4284-8364-8c9e555c1bcf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:37.146532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e38dddf7-b5dd-4284-8364-8c9e555c1bcf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:37.551188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e38dddf7-b5dd-4284-8364-8c9e555c1bcf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:37.561259 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e38dddf7-b5dd-4284-8364-8c9e555c1bcf tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1314 microversion: 2.1 time: 0.439638 Sep 27 10:15:37.561856 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2357/4310] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:37 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1314 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:37.585716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-76a3c91c-6eae-44de-85aa-b4a676304e53 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:37.588571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-76a3c91c-6eae-44de-85aa-b4a676304e53 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:37.722512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-76a3c91c-6eae-44de-85aa-b4a676304e53 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:37.730826 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-76a3c91c-6eae-44de-85aa-b4a676304e53 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1227 microversion: 2.1 time: 0.163163 Sep 27 10:15:37.742376 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2358/4311] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:37 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1227 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:37.755661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8eba6de9-327d-4775-8798-24761c765d03 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:37.756372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8eba6de9-327d-4775-8798-24761c765d03 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:37.784300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eba6de9-327d-4775-8798-24761c765d03 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:37.784942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eba6de9-327d-4775-8798-24761c765d03 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:37.785445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eba6de9-327d-4775-8798-24761c765d03 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:37.786243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eba6de9-327d-4775-8798-24761c765d03 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:37.786704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eba6de9-327d-4775-8798-24761c765d03 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:37.787335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eba6de9-327d-4775-8798-24761c765d03 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:37.862133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e88212a5-918d-4053-9521-67899ae4800e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:37.866495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:37.885583 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e88212a5-918d-4053-9521-67899ae4800e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1327 microversion: 2.1 time: 0.824821 Sep 27 10:15:37.908691 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1954/4312] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:37 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1327 bytes in 838 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:37.917663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5d7e1455-e2ae-416e-8a5c-2065d05fd5ed req-48261d68-4a02-4ca7-b31c-c1fc48953778 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-deleted", "tag": "bcc2c185-ff79-4bda-a3f2-d915e7d3752c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:37.940767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5d7e1455-e2ae-416e-8a5c-2065d05fd5ed req-48261d68-4a02-4ca7-b31c-c1fc48953778 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:37.954238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5d7e1455-e2ae-416e-8a5c-2065d05fd5ed req-48261d68-4a02-4ca7-b31c-c1fc48953778 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:37.955049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5d7e1455-e2ae-416e-8a5c-2065d05fd5ed req-48261d68-4a02-4ca7-b31c-c1fc48953778 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:37.983813 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5d7e1455-e2ae-416e-8a5c-2065d05fd5ed req-48261d68-4a02-4ca7-b31c-c1fc48953778 service nova] Creating event network-vif-deleted:bcc2c185-ff79-4bda-a3f2-d915e7d3752c for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:15:37.990443 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5d7e1455-e2ae-416e-8a5c-2065d05fd5ed req-48261d68-4a02-4ca7-b31c-c1fc48953778 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.086464 Sep 27 10:15:37.991571 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1955/4313] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:37.997597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-44bacf0a-3308-4789-b70c-b8f3a18bdff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:37.998953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-44bacf0a-3308-4789-b70c-b8f3a18bdff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:38.184738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8eba6de9-327d-4775-8798-24761c765d03 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:38.194525 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8eba6de9-327d-4775-8798-24761c765d03 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1654 microversion: 2.1 time: 0.450949 Sep 27 10:15:38.203947 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2359/4314] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:37 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1654 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:38.217507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:38.218423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:38.229978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:38.236557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:38.240065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:38.256318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:38.257282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:38.262497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:38.300052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-44bacf0a-3308-4789-b70c-b8f3a18bdff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:38.308615 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-44bacf0a-3308-4789-b70c-b8f3a18bdff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1224 microversion: 2.1 time: 0.315763 Sep 27 10:15:38.314070 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1956/4315] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:37 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1224 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:38.330298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d68ea2a6-7173-497c-89d8-ff108deaf689 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:38.342887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d68ea2a6-7173-497c-89d8-ff108deaf689 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:38.352537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d68ea2a6-7173-497c-89d8-ff108deaf689 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:38.353299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d68ea2a6-7173-497c-89d8-ff108deaf689 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:38.414267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d68ea2a6-7173-497c-89d8-ff108deaf689 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Going to try to start instance {{(pid=145969) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 27 10:15:38.530368 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d68ea2a6-7173-497c-89d8-ff108deaf689 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "POST /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/action" status: 202 len: 0 microversion: 2.1 time: 0.214594 Sep 27 10:15:38.531789 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1957/4316] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:15:38 2022] POST /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/action => generated 0 bytes in 217 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:15:38.541970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9eaca9b1-0a50-406a-ac2c-4301f4d4ac86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:38.544925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9eaca9b1-0a50-406a-ac2c-4301f4d4ac86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:38.564558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eaca9b1-0a50-406a-ac2c-4301f4d4ac86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:38.565412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eaca9b1-0a50-406a-ac2c-4301f4d4ac86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:38.566014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eaca9b1-0a50-406a-ac2c-4301f4d4ac86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:38.567112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eaca9b1-0a50-406a-ac2c-4301f4d4ac86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:38.567731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eaca9b1-0a50-406a-ac2c-4301f4d4ac86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:38.569620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9eaca9b1-0a50-406a-ac2c-4301f4d4ac86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:39.256632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:39.290010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:39.296465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:39.297869 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-738e2e7b-c70c-45f6-a696-8b5678ac6982 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 2165 microversion: 2.42 time: 1.092502 Sep 27 10:15:39.302696 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2360/4317] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:38 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 2165 bytes in 1096 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:39.317754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4a8956b9-1600-46c9-9b68-9365cf33e6b1 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:39.319602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4a8956b9-1600-46c9-9b68-9365cf33e6b1 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:39.340248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a8956b9-1600-46c9-9b68-9365cf33e6b1 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:39.341228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a8956b9-1600-46c9-9b68-9365cf33e6b1 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:39.342698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a8956b9-1600-46c9-9b68-9365cf33e6b1 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:39.346468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a8956b9-1600-46c9-9b68-9365cf33e6b1 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:39.347712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a8956b9-1600-46c9-9b68-9365cf33e6b1 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:39.348130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a8956b9-1600-46c9-9b68-9365cf33e6b1 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:39.661468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9eaca9b1-0a50-406a-ac2c-4301f4d4ac86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:39.670414 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9eaca9b1-0a50-406a-ac2c-4301f4d4ac86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1663 microversion: 2.1 time: 1.134068 Sep 27 10:15:39.686207 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1958/4318] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:38 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1663 bytes in 1151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:39.694115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f90cc033-6768-41f5-94d8-5753c2a81760 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:39.696447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f90cc033-6768-41f5-94d8-5753c2a81760 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:39.703723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90cc033-6768-41f5-94d8-5753c2a81760 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:39.704694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90cc033-6768-41f5-94d8-5753c2a81760 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:39.705281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90cc033-6768-41f5-94d8-5753c2a81760 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:39.706150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90cc033-6768-41f5-94d8-5753c2a81760 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:39.706743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90cc033-6768-41f5-94d8-5753c2a81760 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:39.707299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90cc033-6768-41f5-94d8-5753c2a81760 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:39.772026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4a8956b9-1600-46c9-9b68-9365cf33e6b1 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:39.791523 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4a8956b9-1600-46c9-9b68-9365cf33e6b1 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1314 microversion: 2.1 time: 0.487107 Sep 27 10:15:39.792895 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2361/4319] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:39 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1314 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:39.797798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-335f9689-ca58-45b7-88fe-603d1af6c44a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:39.801463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-335f9689-ca58-45b7-88fe-603d1af6c44a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:39.834667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-335f9689-ca58-45b7-88fe-603d1af6c44a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:39.840513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-335f9689-ca58-45b7-88fe-603d1af6c44a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:39.841375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-335f9689-ca58-45b7-88fe-603d1af6c44a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:39.842261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-335f9689-ca58-45b7-88fe-603d1af6c44a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:39.842845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-335f9689-ca58-45b7-88fe-603d1af6c44a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:39.843568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-335f9689-ca58-45b7-88fe-603d1af6c44a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:40.032110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f90cc033-6768-41f5-94d8-5753c2a81760 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:40.050672 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f90cc033-6768-41f5-94d8-5753c2a81760 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1285 microversion: 2.1 time: 0.362994 Sep 27 10:15:40.062279 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1959/4320] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:39 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1285 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:40.071494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-56840390-db96-47bb-8edb-96406c956b59 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:40.072001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-56840390-db96-47bb-8edb-96406c956b59 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:40.092231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56840390-db96-47bb-8edb-96406c956b59 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:40.092746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56840390-db96-47bb-8edb-96406c956b59 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:40.098770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56840390-db96-47bb-8edb-96406c956b59 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:40.099929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56840390-db96-47bb-8edb-96406c956b59 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:40.100640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56840390-db96-47bb-8edb-96406c956b59 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:40.101265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56840390-db96-47bb-8edb-96406c956b59 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:40.412294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-56840390-db96-47bb-8edb-96406c956b59 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:40.427419 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-56840390-db96-47bb-8edb-96406c956b59 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1284 microversion: 2.1 time: 0.354275 Sep 27 10:15:40.441114 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1960/4321] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:40 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1284 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:40.454153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-997658a8-9989-4751-bde0-52d4d2b2147d req-2367fa8d-f1d8-476a-8f7e-c5b67383e2ad service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "488d3e73-8e6a-4f18-8dbe-34bb094c2d1b", "name": "network-vif-deleted", "tag": "8cd8623a-a3f6-4d1c-aced-9707f8353fa6"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:40.457934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-335f9689-ca58-45b7-88fe-603d1af6c44a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:40.474463 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-335f9689-ca58-45b7-88fe-603d1af6c44a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1327 microversion: 2.1 time: 0.678483 Sep 27 10:15:40.475969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-997658a8-9989-4751-bde0-52d4d2b2147d req-2367fa8d-f1d8-476a-8f7e-c5b67383e2ad service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:40.476460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-997658a8-9989-4751-bde0-52d4d2b2147d req-2367fa8d-f1d8-476a-8f7e-c5b67383e2ad service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:40.481745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-997658a8-9989-4751-bde0-52d4d2b2147d req-2367fa8d-f1d8-476a-8f7e-c5b67383e2ad service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:40.494331 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-997658a8-9989-4751-bde0-52d4d2b2147d req-2367fa8d-f1d8-476a-8f7e-c5b67383e2ad service nova] Creating event network-vif-deleted:8cd8623a-a3f6-4d1c-aced-9707f8353fa6 for instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b on nch2-master-1 Sep 27 10:15:40.498980 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2362/4322] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:39 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1327 bytes in 705 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:40.508550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-29ed25b2-0bf8-4ba6-a5fc-668cb4f328e3 req-d2a0336c-7378-4b7c-831e-acbfba78f185 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "168c4335-87f5-4cce-b73b-f525c65f5166", "name": "network-vif-plugged", "status": "completed", "tag": "dcc4c715-0cd8-4eaf-a0ad-3f50253460d0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:40.509536 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-997658a8-9989-4751-bde0-52d4d2b2147d req-2367fa8d-f1d8-476a-8f7e-c5b67383e2ad service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060812 Sep 27 10:15:40.511172 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1961/4323] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:40.516912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:40.526384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:40.539956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:40.540924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:40.542713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:40.544028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:40.544744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:40.545383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:40.550948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-29ed25b2-0bf8-4ba6-a5fc-668cb4f328e3 req-d2a0336c-7378-4b7c-831e-acbfba78f185 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:40.551709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-29ed25b2-0bf8-4ba6-a5fc-668cb4f328e3 req-d2a0336c-7378-4b7c-831e-acbfba78f185 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:40.555588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-29ed25b2-0bf8-4ba6-a5fc-668cb4f328e3 req-d2a0336c-7378-4b7c-831e-acbfba78f185 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:40.590188 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-29ed25b2-0bf8-4ba6-a5fc-668cb4f328e3 req-d2a0336c-7378-4b7c-831e-acbfba78f185 service nova] Creating event network-vif-plugged:dcc4c715-0cd8-4eaf-a0ad-3f50253460d0 for instance 168c4335-87f5-4cce-b73b-f525c65f5166 on nch1-master-1 Sep 27 10:15:40.599220 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-29ed25b2-0bf8-4ba6-a5fc-668cb4f328e3 req-d2a0336c-7378-4b7c-831e-acbfba78f185 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098142 Sep 27 10:15:40.600816 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2363/4324] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:40.706453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e11d4a7f-a254-4d9b-8fff-40083f6a8cc7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:40.707602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e11d4a7f-a254-4d9b-8fff-40083f6a8cc7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:40.722810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e11d4a7f-a254-4d9b-8fff-40083f6a8cc7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:40.725596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e11d4a7f-a254-4d9b-8fff-40083f6a8cc7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:40.728182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e11d4a7f-a254-4d9b-8fff-40083f6a8cc7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:40.729360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e11d4a7f-a254-4d9b-8fff-40083f6a8cc7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:40.730256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e11d4a7f-a254-4d9b-8fff-40083f6a8cc7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:40.731655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e11d4a7f-a254-4d9b-8fff-40083f6a8cc7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:40.879679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:40.891738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:40.893648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '04a5f2b1d0134e529f25c227954b5f20', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f86d6a96f893464f81868717f0531bd3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:40.895157 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d2670d0c-6982-4372-bb47-a005d288e814 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 200 len: 1605 microversion: 2.42 time: 0.382619 Sep 27 10:15:40.896319 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1962/4325] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:40 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 1605 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:15:40.901738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25448c2f-1c16-411d-8f29-bab6be6f07a5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:40.904006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-25448c2f-1c16-411d-8f29-bab6be6f07a5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:40.910216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25448c2f-1c16-411d-8f29-bab6be6f07a5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:40.910954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25448c2f-1c16-411d-8f29-bab6be6f07a5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:40.911561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25448c2f-1c16-411d-8f29-bab6be6f07a5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:40.912841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25448c2f-1c16-411d-8f29-bab6be6f07a5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:40.913447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25448c2f-1c16-411d-8f29-bab6be6f07a5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:40.914005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25448c2f-1c16-411d-8f29-bab6be6f07a5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:41.169629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e11d4a7f-a254-4d9b-8fff-40083f6a8cc7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:41.192211 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e11d4a7f-a254-4d9b-8fff-40083f6a8cc7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1663 microversion: 2.1 time: 0.494595 Sep 27 10:15:41.194299 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2364/4326] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:40 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1663 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:41.202647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2438e817-5be4-4212-b8e6-69996de668ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:41.203449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2438e817-5be4-4212-b8e6-69996de668ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:41.215951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2438e817-5be4-4212-b8e6-69996de668ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:41.219236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2438e817-5be4-4212-b8e6-69996de668ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:41.220898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2438e817-5be4-4212-b8e6-69996de668ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:41.221863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2438e817-5be4-4212-b8e6-69996de668ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:41.223948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2438e817-5be4-4212-b8e6-69996de668ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:41.224571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2438e817-5be4-4212-b8e6-69996de668ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:41.329678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2438e817-5be4-4212-b8e6-69996de668ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:41.339413 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2438e817-5be4-4212-b8e6-69996de668ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1285 microversion: 2.1 time: 0.143580 Sep 27 10:15:41.364723 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2365/4327] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:41 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1285 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:41.377114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-25448c2f-1c16-411d-8f29-bab6be6f07a5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:41.390066 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-25448c2f-1c16-411d-8f29-bab6be6f07a5 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1314 microversion: 2.1 time: 0.492107 Sep 27 10:15:41.400239 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1963/4328] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:40 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1314 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:41.457167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-305c4204-ee8f-4371-bc5a-c3473e33aff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:41.457938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-305c4204-ee8f-4371-bc5a-c3473e33aff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:41.472055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305c4204-ee8f-4371-bc5a-c3473e33aff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:41.482999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305c4204-ee8f-4371-bc5a-c3473e33aff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:41.483956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305c4204-ee8f-4371-bc5a-c3473e33aff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:41.484983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305c4204-ee8f-4371-bc5a-c3473e33aff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:41.486141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305c4204-ee8f-4371-bc5a-c3473e33aff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:41.486798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-305c4204-ee8f-4371-bc5a-c3473e33aff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:41.504813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-55e1cab2-e03b-4434-a16a-a494284fe901 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:41.505534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-55e1cab2-e03b-4434-a16a-a494284fe901 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:41.517498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e1cab2-e03b-4434-a16a-a494284fe901 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:41.517986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e1cab2-e03b-4434-a16a-a494284fe901 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:41.518638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e1cab2-e03b-4434-a16a-a494284fe901 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:41.519563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e1cab2-e03b-4434-a16a-a494284fe901 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:41.524924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e1cab2-e03b-4434-a16a-a494284fe901 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:41.525642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e1cab2-e03b-4434-a16a-a494284fe901 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:41.549307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:41.681218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-305c4204-ee8f-4371-bc5a-c3473e33aff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:41.690722 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-305c4204-ee8f-4371-bc5a-c3473e33aff9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1282 microversion: 2.1 time: 0.239206 Sep 27 10:15:41.713204 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2366/4329] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:41 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1282 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:41.926337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:41.926948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:41.933974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:41.934473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:41.934908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:41.935789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:41.936247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:41.936689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:41.951678 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] HTTP exception thrown: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Request method failure captured: Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b HTTP/1.1 Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsvgxfK_CAwoqcba7BQGnchVqYOI8SjEEpj1Bd8mHr8wxQIL632sPlmvWCfDaVDmpDijsyEywn23EvKDQ_k8h9ZtNRoriEweN6Ns4n2sKYHKLmaqN37UmHJmPOAqNy6ApVcX45SV1VoE8AQUh4dO70DbRA5FUaa-A0CMhZJiKJmWDbA Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.42 Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f86d6a96f893464f81868717f0531bd3 Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TaggedBootDevicesTest_v242-1873294123 Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f86d6a96f893464f81868717f0531bd3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f86d6a96f893464f81868717f0531bd3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f86d6a96f893464f81868717f0531bd3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f86d6a96f893464f81868717f0531bd3"}]}] Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TaggedBootDevicesTest_v242-1873294123 Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f86d6a96f893464f81868717f0531bd3 Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TaggedBootDevicesTest_v242-1873294123 Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TaggedBootDevicesTest_v242-1873294123-project Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 04a5f2b1d0134e529f25c227954b5f20 Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TaggedBootDevicesTest_v242-1873294123-project Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '488d3e73-8e6a-4f18-8dbe-34bb094c2d1b'} Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:41.953804 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:41.957932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:41.957932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:41.957932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:41.957932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:41.957932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:15:41.957932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:15:41.957932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:41.957932 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:41.959179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Returning 404 to user: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:15:41.960320 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9534c7fc-f4d7-4e2a-a069-4ea4796973b2 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 404 len: 111 microversion: 2.42 time: 0.037881 Sep 27 10:15:41.960918 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2367/4330] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:41 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 27 10:15:41.969462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e55716c6-3ec2-4e78-b9d4-f76b2fd74b21 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '.version_select at 0x7f7f624ddf70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:41.995292 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e55716c6-3ec2-4e78-b9d4-f76b2fd74b21 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1829774399" status: 204 len: 0 microversion: 2.42 time: 0.029579 Sep 27 10:15:41.996071 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2368/4331] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 10:15:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1829774399 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Sep 27 10:15:42.073318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-55e1cab2-e03b-4434-a16a-a494284fe901 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:42.083408 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-55e1cab2-e03b-4434-a16a-a494284fe901 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1327 microversion: 2.1 time: 0.582705 Sep 27 10:15:42.107250 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1964/4332] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:41 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1327 bytes in 607 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:42.221840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-42b6eca3-c0e6-4fd6-ba62-229bc3e04cd0 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:42.222855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-42b6eca3-c0e6-4fd6-ba62-229bc3e04cd0 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:42.246913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42b6eca3-c0e6-4fd6-ba62-229bc3e04cd0 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:42.247729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42b6eca3-c0e6-4fd6-ba62-229bc3e04cd0 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:42.248385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42b6eca3-c0e6-4fd6-ba62-229bc3e04cd0 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:42.249780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42b6eca3-c0e6-4fd6-ba62-229bc3e04cd0 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:42.251069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42b6eca3-c0e6-4fd6-ba62-229bc3e04cd0 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:42.251699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42b6eca3-c0e6-4fd6-ba62-229bc3e04cd0 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:42.363966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7cb97e7a-3060-440b-b06f-fd8dbd1fecaa tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:42.364527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7cb97e7a-3060-440b-b06f-fd8dbd1fecaa tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:42.367261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:42.381658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb97e7a-3060-440b-b06f-fd8dbd1fecaa tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:42.382285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb97e7a-3060-440b-b06f-fd8dbd1fecaa tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:42.382731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb97e7a-3060-440b-b06f-fd8dbd1fecaa tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:42.383492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb97e7a-3060-440b-b06f-fd8dbd1fecaa tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:42.383955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb97e7a-3060-440b-b06f-fd8dbd1fecaa tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:42.384386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7cb97e7a-3060-440b-b06f-fd8dbd1fecaa tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:42.635099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-42b6eca3-c0e6-4fd6-ba62-229bc3e04cd0 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:42.645016 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-42b6eca3-c0e6-4fd6-ba62-229bc3e04cd0 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1667 microversion: 2.1 time: 0.426708 Sep 27 10:15:42.660881 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2369/4333] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:42 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1667 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:42.672321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae02f283-6cad-46d9-8a95-cdaa0209e964 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:42.673286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ae02f283-6cad-46d9-8a95-cdaa0209e964 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:42.680008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae02f283-6cad-46d9-8a95-cdaa0209e964 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:42.680776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae02f283-6cad-46d9-8a95-cdaa0209e964 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:42.681433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae02f283-6cad-46d9-8a95-cdaa0209e964 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:42.682319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae02f283-6cad-46d9-8a95-cdaa0209e964 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:42.682933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae02f283-6cad-46d9-8a95-cdaa0209e964 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:42.683511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae02f283-6cad-46d9-8a95-cdaa0209e964 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:42.915520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ae02f283-6cad-46d9-8a95-cdaa0209e964 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:42.925727 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ae02f283-6cad-46d9-8a95-cdaa0209e964 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1498 microversion: 2.1 time: 0.262681 Sep 27 10:15:42.946073 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2370/4334] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:42 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1498 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:42.953714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8bb68fcc-4bc5-4af9-95c3-a3bf9e646d67 req-c395a778-a474-4a5a-83e6-9287efcf60b7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "775df75d-9fec-4be2-bccb-b8c28dd3a404", "tag": "ea7e2974-9ba7-4f04-a5d9-79df1c299f3f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:42.970127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8bb68fcc-4bc5-4af9-95c3-a3bf9e646d67 req-c395a778-a474-4a5a-83e6-9287efcf60b7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:42.970955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8bb68fcc-4bc5-4af9-95c3-a3bf9e646d67 req-c395a778-a474-4a5a-83e6-9287efcf60b7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:42.971583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8bb68fcc-4bc5-4af9-95c3-a3bf9e646d67 req-c395a778-a474-4a5a-83e6-9287efcf60b7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:42.992955 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8bb68fcc-4bc5-4af9-95c3-a3bf9e646d67 req-c395a778-a474-4a5a-83e6-9287efcf60b7 service nova] Creating event network-changed:ea7e2974-9ba7-4f04-a5d9-79df1c299f3f for instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 on nch1-master-1 Sep 27 10:15:42.999750 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8bb68fcc-4bc5-4af9-95c3-a3bf9e646d67 req-c395a778-a474-4a5a-83e6-9287efcf60b7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051302 Sep 27 10:15:43.001663 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2371/4335] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:43.009324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b8cca2e7-0455-421e-98d7-3ae62d510be1 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:43.010536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b8cca2e7-0455-421e-98d7-3ae62d510be1 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:43.019321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8cca2e7-0455-421e-98d7-3ae62d510be1 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:43.019973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8cca2e7-0455-421e-98d7-3ae62d510be1 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:43.020562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8cca2e7-0455-421e-98d7-3ae62d510be1 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:43.021778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8cca2e7-0455-421e-98d7-3ae62d510be1 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:43.022384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8cca2e7-0455-421e-98d7-3ae62d510be1 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:43.023144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8cca2e7-0455-421e-98d7-3ae62d510be1 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:43.371406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b8cca2e7-0455-421e-98d7-3ae62d510be1 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:43.383728 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b8cca2e7-0455-421e-98d7-3ae62d510be1 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 0.379121 Sep 27 10:15:43.384652 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2372/4336] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:43 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:43.398499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b626aad3-d263-4d09-b863-00bdd7dc9fb1 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:43.399136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b626aad3-d263-4d09-b863-00bdd7dc9fb1 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:43.406515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b626aad3-d263-4d09-b863-00bdd7dc9fb1 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:43.407145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b626aad3-d263-4d09-b863-00bdd7dc9fb1 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:43.407447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b626aad3-d263-4d09-b863-00bdd7dc9fb1 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:43.408347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b626aad3-d263-4d09-b863-00bdd7dc9fb1 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:43.409003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b626aad3-d263-4d09-b863-00bdd7dc9fb1 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:43.409429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b626aad3-d263-4d09-b863-00bdd7dc9fb1 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:43.523681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7cb97e7a-3060-440b-b06f-fd8dbd1fecaa tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:43.539653 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7cb97e7a-3060-440b-b06f-fd8dbd1fecaa tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1327 microversion: 2.1 time: 1.179791 Sep 27 10:15:43.541216 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1965/4337] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:42 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1327 bytes in 1182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:43.632283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7d74afd-1467-4096-94f4-30fee35faae7 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:43.633166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7d74afd-1467-4096-94f4-30fee35faae7 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:43.640731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7d74afd-1467-4096-94f4-30fee35faae7 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:43.641534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7d74afd-1467-4096-94f4-30fee35faae7 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:43.642051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7d74afd-1467-4096-94f4-30fee35faae7 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:43.642847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7d74afd-1467-4096-94f4-30fee35faae7 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:43.643378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7d74afd-1467-4096-94f4-30fee35faae7 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:43.643871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7d74afd-1467-4096-94f4-30fee35faae7 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:43.673096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7d74afd-1467-4096-94f4-30fee35faae7 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:15:43.736860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b626aad3-d263-4d09-b863-00bdd7dc9fb1 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:43.748323 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b626aad3-d263-4d09-b863-00bdd7dc9fb1 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1327 microversion: 2.1 time: 0.361822 Sep 27 10:15:43.751984 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2373/4338] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:43 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1327 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:43.829407 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c7d74afd-1467-4096-94f4-30fee35faae7 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "DELETE /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 204 len: 0 microversion: 2.1 time: 0.201914 Sep 27 10:15:43.831042 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1966/4339] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:15:43 2022] DELETE /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:15:43.843739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d5b025aa-ff9a-4455-b6d7-b108442a66eb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:43.844863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d5b025aa-ff9a-4455-b6d7-b108442a66eb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:43.853195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b025aa-ff9a-4455-b6d7-b108442a66eb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:43.853701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b025aa-ff9a-4455-b6d7-b108442a66eb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:43.854140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b025aa-ff9a-4455-b6d7-b108442a66eb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:43.856712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b025aa-ff9a-4455-b6d7-b108442a66eb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:43.857365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b025aa-ff9a-4455-b6d7-b108442a66eb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:43.872951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b025aa-ff9a-4455-b6d7-b108442a66eb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:44.124924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d5b025aa-ff9a-4455-b6d7-b108442a66eb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:44.135638 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d5b025aa-ff9a-4455-b6d7-b108442a66eb tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1504 microversion: 2.1 time: 0.299839 Sep 27 10:15:44.137354 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2374/4340] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:43 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1504 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:44.410150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5c8052af-8f9e-42ca-adc1-51ac921ecce7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:44.410857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5c8052af-8f9e-42ca-adc1-51ac921ecce7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:44.422970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c8052af-8f9e-42ca-adc1-51ac921ecce7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:44.424269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c8052af-8f9e-42ca-adc1-51ac921ecce7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:44.431531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c8052af-8f9e-42ca-adc1-51ac921ecce7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:44.439397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c8052af-8f9e-42ca-adc1-51ac921ecce7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:44.440542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c8052af-8f9e-42ca-adc1-51ac921ecce7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:44.441423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c8052af-8f9e-42ca-adc1-51ac921ecce7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:44.565471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0f172bf-e03f-420e-a2e9-2f6894de9603 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:44.566236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b0f172bf-e03f-420e-a2e9-2f6894de9603 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:44.578481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f172bf-e03f-420e-a2e9-2f6894de9603 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:44.579291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f172bf-e03f-420e-a2e9-2f6894de9603 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:44.580058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f172bf-e03f-420e-a2e9-2f6894de9603 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:44.581333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f172bf-e03f-420e-a2e9-2f6894de9603 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:44.582011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f172bf-e03f-420e-a2e9-2f6894de9603 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:44.582784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f172bf-e03f-420e-a2e9-2f6894de9603 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:44.961988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5c8052af-8f9e-42ca-adc1-51ac921ecce7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:44.977828 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5c8052af-8f9e-42ca-adc1-51ac921ecce7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 0.568680 Sep 27 10:15:44.979139 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1967/4341] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:44 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:44.985141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1a4d6c36-ffa2-4446-9bd4-032906b6ac94 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:44.985974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1a4d6c36-ffa2-4446-9bd4-032906b6ac94 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:44.996256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a4d6c36-ffa2-4446-9bd4-032906b6ac94 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:44.997199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a4d6c36-ffa2-4446-9bd4-032906b6ac94 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:45.003866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a4d6c36-ffa2-4446-9bd4-032906b6ac94 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:45.005007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a4d6c36-ffa2-4446-9bd4-032906b6ac94 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:45.005570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a4d6c36-ffa2-4446-9bd4-032906b6ac94 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:45.006283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a4d6c36-ffa2-4446-9bd4-032906b6ac94 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:45.382362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1a4d6c36-ffa2-4446-9bd4-032906b6ac94 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:45.392865 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1a4d6c36-ffa2-4446-9bd4-032906b6ac94 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1327 microversion: 2.1 time: 0.411681 Sep 27 10:15:45.396169 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1968/4342] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:44 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1327 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:45.417601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-56330358-8316-44c4-b297-f12b76789a79 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:45.419477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-56330358-8316-44c4-b297-f12b76789a79 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:45.432690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56330358-8316-44c4-b297-f12b76789a79 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:45.433407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56330358-8316-44c4-b297-f12b76789a79 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:45.434726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56330358-8316-44c4-b297-f12b76789a79 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:45.435701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56330358-8316-44c4-b297-f12b76789a79 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:45.436651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56330358-8316-44c4-b297-f12b76789a79 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:45.437411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56330358-8316-44c4-b297-f12b76789a79 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:45.497883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b0f172bf-e03f-420e-a2e9-2f6894de9603 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:45.512016 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b0f172bf-e03f-420e-a2e9-2f6894de9603 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1327 microversion: 2.1 time: 0.950817 Sep 27 10:15:45.513002 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2375/4343] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:44 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1327 bytes in 953 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:45.518391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-659fd949-575b-4b83-be9c-0af9c6234fb0 req-718e86d7-b9b2-4045-9214-80314cfb497e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "521b4b27-ea54-4de8-a96d-add9170f03c5", "tag": "30c45679-2756-42d1-a719-42e9e2b8c137"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:45.528874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-659fd949-575b-4b83-be9c-0af9c6234fb0 req-718e86d7-b9b2-4045-9214-80314cfb497e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:45.529635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-659fd949-575b-4b83-be9c-0af9c6234fb0 req-718e86d7-b9b2-4045-9214-80314cfb497e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:45.530148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-659fd949-575b-4b83-be9c-0af9c6234fb0 req-718e86d7-b9b2-4045-9214-80314cfb497e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:45.544559 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-659fd949-575b-4b83-be9c-0af9c6234fb0 req-718e86d7-b9b2-4045-9214-80314cfb497e service nova] Creating event network-changed:30c45679-2756-42d1-a719-42e9e2b8c137 for instance 521b4b27-ea54-4de8-a96d-add9170f03c5 on nch2-master-1 Sep 27 10:15:45.549364 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-659fd949-575b-4b83-be9c-0af9c6234fb0 req-718e86d7-b9b2-4045-9214-80314cfb497e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034950 Sep 27 10:15:45.549967 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2376/4344] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:45.775903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-56330358-8316-44c4-b297-f12b76789a79 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:45.788025 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-56330358-8316-44c4-b297-f12b76789a79 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1504 microversion: 2.1 time: 0.388984 Sep 27 10:15:45.789488 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1969/4345] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:45 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1504 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:46.004711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a6edb9c3-9056-4cd7-ab55-8b5530e99bb7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:46.005634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a6edb9c3-9056-4cd7-ab55-8b5530e99bb7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:46.012918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6edb9c3-9056-4cd7-ab55-8b5530e99bb7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:46.013562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6edb9c3-9056-4cd7-ab55-8b5530e99bb7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:46.014170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6edb9c3-9056-4cd7-ab55-8b5530e99bb7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:46.014884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6edb9c3-9056-4cd7-ab55-8b5530e99bb7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:46.015500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6edb9c3-9056-4cd7-ab55-8b5530e99bb7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:46.015927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a6edb9c3-9056-4cd7-ab55-8b5530e99bb7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:46.431187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0cce6fa-d798-428e-ac69-99853a7a3fed tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:46.432633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b0cce6fa-d798-428e-ac69-99853a7a3fed tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:46.445841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0cce6fa-d798-428e-ac69-99853a7a3fed tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:46.447032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0cce6fa-d798-428e-ac69-99853a7a3fed tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:46.447803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0cce6fa-d798-428e-ac69-99853a7a3fed tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:46.449768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0cce6fa-d798-428e-ac69-99853a7a3fed tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:46.450785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0cce6fa-d798-428e-ac69-99853a7a3fed tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:46.451510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0cce6fa-d798-428e-ac69-99853a7a3fed tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:46.484114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a6edb9c3-9056-4cd7-ab55-8b5530e99bb7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:46.509265 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a6edb9c3-9056-4cd7-ab55-8b5530e99bb7 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 0.507631 Sep 27 10:15:46.520233 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2377/4346] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:45 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:46.538557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-97bdb515-68d9-4dac-ad17-908748e46eb9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:46.540509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-97bdb515-68d9-4dac-ad17-908748e46eb9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:46.555802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bdb515-68d9-4dac-ad17-908748e46eb9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:46.556315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bdb515-68d9-4dac-ad17-908748e46eb9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:46.558830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bdb515-68d9-4dac-ad17-908748e46eb9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:46.559961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bdb515-68d9-4dac-ad17-908748e46eb9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:46.560457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bdb515-68d9-4dac-ad17-908748e46eb9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:46.561125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bdb515-68d9-4dac-ad17-908748e46eb9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:46.601127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:46.853386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:47.133934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b0cce6fa-d798-428e-ac69-99853a7a3fed tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:47.144608 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0cce6fa-d798-428e-ac69-99853a7a3fed tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1327 microversion: 2.1 time: 0.726951 Sep 27 10:15:47.146093 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1970/4347] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:46 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1327 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:47.151752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dd42640a-b236-4316-8211-56dcf71d76e8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:47.152832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dd42640a-b236-4316-8211-56dcf71d76e8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:47.159651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd42640a-b236-4316-8211-56dcf71d76e8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:47.160570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd42640a-b236-4316-8211-56dcf71d76e8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:47.161225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd42640a-b236-4316-8211-56dcf71d76e8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:47.162287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd42640a-b236-4316-8211-56dcf71d76e8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:47.163150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd42640a-b236-4316-8211-56dcf71d76e8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:47.163760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd42640a-b236-4316-8211-56dcf71d76e8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:47.167459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:47.242891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-97bdb515-68d9-4dac-ad17-908748e46eb9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:47.254769 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-97bdb515-68d9-4dac-ad17-908748e46eb9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1327 microversion: 2.1 time: 0.723146 Sep 27 10:15:47.255993 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2378/4348] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:46 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1327 bytes in 725 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:47.487096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dd42640a-b236-4316-8211-56dcf71d76e8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc369ca971424247b96050e09368a7bb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '27c0d3daa08340d89d288bbf811fece9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:47.498001 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dd42640a-b236-4316-8211-56dcf71d76e8 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 200 len: 1504 microversion: 2.1 time: 0.350448 Sep 27 10:15:47.499713 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1971/4349] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:47 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 1504 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:47.542255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-db773e57-4459-42e7-ad54-589cf6dae9d2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:47.543480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-db773e57-4459-42e7-ad54-589cf6dae9d2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:47.549667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db773e57-4459-42e7-ad54-589cf6dae9d2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:47.550155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db773e57-4459-42e7-ad54-589cf6dae9d2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:47.550661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db773e57-4459-42e7-ad54-589cf6dae9d2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:47.551366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db773e57-4459-42e7-ad54-589cf6dae9d2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:47.551849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db773e57-4459-42e7-ad54-589cf6dae9d2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:47.552988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db773e57-4459-42e7-ad54-589cf6dae9d2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:47.582796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b7a83ea8-7d0f-422b-a4a5-df2e261af589 req-e95e70d1-704a-44fa-ba6b-f2d14fdd8f8e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec", "tag": "3d3f5182-aaf9-4555-a437-c8414a163222"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:47.630658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b7a83ea8-7d0f-422b-a4a5-df2e261af589 req-e95e70d1-704a-44fa-ba6b-f2d14fdd8f8e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:47.632670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b7a83ea8-7d0f-422b-a4a5-df2e261af589 req-e95e70d1-704a-44fa-ba6b-f2d14fdd8f8e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:47.635365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b7a83ea8-7d0f-422b-a4a5-df2e261af589 req-e95e70d1-704a-44fa-ba6b-f2d14fdd8f8e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:47.679015 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b7a83ea8-7d0f-422b-a4a5-df2e261af589 req-e95e70d1-704a-44fa-ba6b-f2d14fdd8f8e service nova] Creating event network-changed:3d3f5182-aaf9-4555-a437-c8414a163222 for instance 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec on nch2-master-1 Sep 27 10:15:47.693296 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b7a83ea8-7d0f-422b-a4a5-df2e261af589 req-e95e70d1-704a-44fa-ba6b-f2d14fdd8f8e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.126137 Sep 27 10:15:47.693958 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1972/4350] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:47.708011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1b03adbb-0c58-48ee-90b3-3b15df4ce393 req-28a70844-0e57-48cf-8b10-3e728f597580 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "168c4335-87f5-4cce-b73b-f525c65f5166", "name": "network-vif-deleted", "tag": "dcc4c715-0cd8-4eaf-a0ad-3f50253460d0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:47.727383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=26 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:47.729754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1b03adbb-0c58-48ee-90b3-3b15df4ce393 req-28a70844-0e57-48cf-8b10-3e728f597580 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:47.730246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1b03adbb-0c58-48ee-90b3-3b15df4ce393 req-28a70844-0e57-48cf-8b10-3e728f597580 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:47.730685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1b03adbb-0c58-48ee-90b3-3b15df4ce393 req-28a70844-0e57-48cf-8b10-3e728f597580 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:47.757980 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1b03adbb-0c58-48ee-90b3-3b15df4ce393 req-28a70844-0e57-48cf-8b10-3e728f597580 service nova] Creating event network-vif-deleted:dcc4c715-0cd8-4eaf-a0ad-3f50253460d0 for instance 168c4335-87f5-4cce-b73b-f525c65f5166 on nch1-master-1 Sep 27 10:15:47.767969 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1b03adbb-0c58-48ee-90b3-3b15df4ce393 req-28a70844-0e57-48cf-8b10-3e728f597580 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.069026 Sep 27 10:15:47.769043 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1973/4351] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:47.843972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-db773e57-4459-42e7-ad54-589cf6dae9d2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:47.857211 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-db773e57-4459-42e7-ad54-589cf6dae9d2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 0.318577 Sep 27 10:15:47.860793 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2379/4352] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:47 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:48.169808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e5e00347-91b4-4c63-b296-88c0eb9a5831 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:48.170533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e5e00347-91b4-4c63-b296-88c0eb9a5831 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:48.180755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e00347-91b4-4c63-b296-88c0eb9a5831 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.181263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e00347-91b4-4c63-b296-88c0eb9a5831 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.181711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e00347-91b4-4c63-b296-88c0eb9a5831 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:48.183486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e00347-91b4-4c63-b296-88c0eb9a5831 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.183934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e00347-91b4-4c63-b296-88c0eb9a5831 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.184372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e00347-91b4-4c63-b296-88c0eb9a5831 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:48.278888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b95472f8-02b1-4fc4-b6e0-1acece5d0e29 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:48.279503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b95472f8-02b1-4fc4-b6e0-1acece5d0e29 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:48.289207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:DELETE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:48.290869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b95472f8-02b1-4fc4-b6e0-1acece5d0e29 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.291577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b95472f8-02b1-4fc4-b6e0-1acece5d0e29 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.292007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b95472f8-02b1-4fc4-b6e0-1acece5d0e29 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:48.293106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b95472f8-02b1-4fc4-b6e0-1acece5d0e29 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.293869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b95472f8-02b1-4fc4-b6e0-1acece5d0e29 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.294487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b95472f8-02b1-4fc4-b6e0-1acece5d0e29 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:48.307934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=10 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:48.354639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e5e00347-91b4-4c63-b296-88c0eb9a5831 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:48.365014 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e5e00347-91b4-4c63-b296-88c0eb9a5831 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1327 microversion: 2.1 time: 0.200756 Sep 27 10:15:48.366644 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1974/4353] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:48 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1327 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:48.527023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:48.528855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] [instance: 168c4335-87f5-4cce-b73b-f525c65f5166] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:48.539561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.540609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.541235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:48.542298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.543954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.544581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:48.568464 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] HTTP exception thrown: Instance 168c4335-87f5-4cce-b73b-f525c65f5166 could not be found. Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Request method failure captured: Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 HTTP/1.1 Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs0VRQ9Vwh0k9D0EAs-3tF0VMfIoDIEXdI3eWAXQDXn-Z8TKcLn92Voq3d5s3RtKnblVl635LVxbLNKzN4b9KSrEJ27lRH7pXq2u8sALvQuWlHykNdj3cFOzdA_IhsLJwIn3XBAlRvOt_A_owDCabdp-jPJYg4aAzCUTqVlVNq7ioE8 Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 27c0d3daa08340d89d288bbf811fece9 Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-VolumesActionsTest-1551375736 Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/27c0d3daa08340d89d288bbf811fece9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/27c0d3daa08340d89d288bbf811fece9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/27c0d3daa08340d89d288bbf811fece9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_27c0d3daa08340d89d288bbf811fece9"}]}] Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-VolumesActionsTest-1551375736 Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 27c0d3daa08340d89d288bbf811fece9 Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-VolumesActionsTest-1551375736 Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-VolumesActionsTest-1551375736-project Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: dc369ca971424247b96050e09368a7bb Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-VolumesActionsTest-1551375736-project Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 168c4335-87f5-4cce-b73b-f525c65f5166 could not be found. Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '168c4335-87f5-4cce-b73b-f525c65f5166'} Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 168c4335-87f5-4cce-b73b-f525c65f5166 could not be found. Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 168c4335-87f5-4cce-b73b-f525c65f5166 could not be found. Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:48.571320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.575829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:48.575829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:48.575829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:48.575829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:15:48.575829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:15:48.575829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 168c4335-87f5-4cce-b73b-f525c65f5166 could not be found. Sep 27 10:15:48.575829 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.575829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] Returning 404 to user: Instance 168c4335-87f5-4cce-b73b-f525c65f5166 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:15:48.575829 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-80a421c3-700e-460d-9e1e-a3e00233e409 tempest-VolumesActionsTest-1551375736 tempest-VolumesActionsTest-1551375736-project] 10.222.0.26 "GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166" status: 404 len: 111 microversion: 2.1 time: 0.055095 Sep 27 10:15:48.576917 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1975/4354] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:48 2022] GET /compute/v2.1/servers/168c4335-87f5-4cce-b73b-f525c65f5166 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:15:48.603734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b95472f8-02b1-4fc4-b6e0-1acece5d0e29 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:48.621592 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b95472f8-02b1-4fc4-b6e0-1acece5d0e29 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1327 microversion: 2.1 time: 0.346588 Sep 27 10:15:48.624415 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2380/4355] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:48 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1327 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:48.752550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:48.753487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:48.761597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.762013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.762539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:48.763249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.763862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.764441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:48.780076 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] HTTP exception thrown: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Request method failure captured: Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b HTTP/1.1 Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsvgxfK_CAwoqcba7BQGnchVqYOI8SjEEpj1Bd8mHr8wxQIL632sPlmvWCfDaVDmpDijsyEywn23EvKDQ_k8h9ZtNRoriEweN6Ns4n2sKYHKLmaqN37UmHJmPOAqNy6ApVcX45SV1VoE8AQUh4dO70DbRA5FUaa-A0CMhZJiKJmWDbA Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.42 Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: f86d6a96f893464f81868717f0531bd3 Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TaggedBootDevicesTest_v242-1873294123 Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f86d6a96f893464f81868717f0531bd3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f86d6a96f893464f81868717f0531bd3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f86d6a96f893464f81868717f0531bd3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f86d6a96f893464f81868717f0531bd3"}]}] Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TaggedBootDevicesTest_v242-1873294123 Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: f86d6a96f893464f81868717f0531bd3 Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TaggedBootDevicesTest_v242-1873294123 Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TaggedBootDevicesTest_v242-1873294123-project Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 04a5f2b1d0134e529f25c227954b5f20 Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TaggedBootDevicesTest_v242-1873294123-project Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '488d3e73-8e6a-4f18-8dbe-34bb094c2d1b'} Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:15:48.781657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:15:48.785781 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:48.785781 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.785781 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:48.785781 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.785781 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:48.785781 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:48.785781 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:48.785781 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:15:48.785781 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:15:48.785781 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:48.785781 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.787817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Returning 404 to user: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:15:48.788967 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aaf86cff-6b8d-48b0-8343-f2ea4bf42d80 tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "DELETE /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 404 len: 111 microversion: 2.42 time: 0.043756 Sep 27 10:15:48.789679 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1976/4356] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:15:48 2022] DELETE /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:15:48.799255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:48.800188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] [instance: 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:48.807382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.807994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.808467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:48.809293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.809768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.810203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:48.829020 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] HTTP exception thrown: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Request method failure captured: Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b HTTP/1.1 Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsvgxfK_CAwoqcba7BQGnchVqYOI8SjEEpj1Bd8mHr8wxQIL632sPlmvWCfDaVDmpDijsyEywn23EvKDQ_k8h9ZtNRoriEweN6Ns4n2sKYHKLmaqN37UmHJmPOAqNy6ApVcX45SV1VoE8AQUh4dO70DbRA5FUaa-A0CMhZJiKJmWDbA Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.42 Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: f86d6a96f893464f81868717f0531bd3 Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TaggedBootDevicesTest_v242-1873294123 Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f86d6a96f893464f81868717f0531bd3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f86d6a96f893464f81868717f0531bd3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f86d6a96f893464f81868717f0531bd3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f86d6a96f893464f81868717f0531bd3"}]}] Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TaggedBootDevicesTest_v242-1873294123 Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: f86d6a96f893464f81868717f0531bd3 Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TaggedBootDevicesTest_v242-1873294123 Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TaggedBootDevicesTest_v242-1873294123-project Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 04a5f2b1d0134e529f25c227954b5f20 Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TaggedBootDevicesTest_v242-1873294123-project Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '488d3e73-8e6a-4f18-8dbe-34bb094c2d1b'} Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.830879 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:48.835186 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.835186 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:48.835186 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:48.835186 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:48.835186 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:15:48.835186 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:15:48.835186 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. Sep 27 10:15:48.835186 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:48.836553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] Returning 404 to user: Instance 488d3e73-8e6a-4f18-8dbe-34bb094c2d1b could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:15:48.837582 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-06039905-5fb8-47af-b933-9ac092bdc66e tempest-TaggedBootDevicesTest_v242-1873294123 tempest-TaggedBootDevicesTest_v242-1873294123-project] 10.222.0.26 "GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b" status: 404 len: 111 microversion: 2.42 time: 0.042710 Sep 27 10:15:48.841056 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2381/4357] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:15:48 2022] GET /compute/v2.1/servers/488d3e73-8e6a-4f18-8dbe-34bb094c2d1b => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:15:48.887467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ef8ec266-5b9b-4d5e-8b5f-845747721ac2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:48.888535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ef8ec266-5b9b-4d5e-8b5f-845747721ac2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:48.898518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef8ec266-5b9b-4d5e-8b5f-845747721ac2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.899140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef8ec266-5b9b-4d5e-8b5f-845747721ac2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.899832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef8ec266-5b9b-4d5e-8b5f-845747721ac2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:48.901225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef8ec266-5b9b-4d5e-8b5f-845747721ac2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:48.901811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef8ec266-5b9b-4d5e-8b5f-845747721ac2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:48.902452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef8ec266-5b9b-4d5e-8b5f-845747721ac2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:49.394170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d994159f-64d8-46f9-8f55-e314afa40bae tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:49.398257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d994159f-64d8-46f9-8f55-e314afa40bae tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:49.414175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d994159f-64d8-46f9-8f55-e314afa40bae tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:49.414584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d994159f-64d8-46f9-8f55-e314afa40bae tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:49.417412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d994159f-64d8-46f9-8f55-e314afa40bae tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:49.422591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d994159f-64d8-46f9-8f55-e314afa40bae tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:49.423187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d994159f-64d8-46f9-8f55-e314afa40bae tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:49.423676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d994159f-64d8-46f9-8f55-e314afa40bae tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:49.439164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ef8ec266-5b9b-4d5e-8b5f-845747721ac2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:49.454533 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ef8ec266-5b9b-4d5e-8b5f-845747721ac2 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 0.572810 Sep 27 10:15:49.456229 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1977/4358] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:48 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:49.613941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d994159f-64d8-46f9-8f55-e314afa40bae tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:49.632725 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d994159f-64d8-46f9-8f55-e314afa40bae tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1327 microversion: 2.1 time: 0.245565 Sep 27 10:15:49.646532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-336762aa-1a07-4e6a-b616-d665eb3e22f8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:49.647128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-336762aa-1a07-4e6a-b616-d665eb3e22f8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:49.658440 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2382/4359] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:49 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1327 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:49.660612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-336762aa-1a07-4e6a-b616-d665eb3e22f8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:49.662675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-336762aa-1a07-4e6a-b616-d665eb3e22f8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:49.663386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-336762aa-1a07-4e6a-b616-d665eb3e22f8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:49.665169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-336762aa-1a07-4e6a-b616-d665eb3e22f8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:49.666041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-336762aa-1a07-4e6a-b616-d665eb3e22f8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:49.666722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-336762aa-1a07-4e6a-b616-d665eb3e22f8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:50.050285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-35a61183-b62f-4cf9-ae96-f16478008fe5 req-4626b370-2e96-45f6-b62c-bf1598a5676e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "775df75d-9fec-4be2-bccb-b8c28dd3a404", "name": "network-vif-plugged", "status": "completed", "tag": "ea7e2974-9ba7-4f04-a5d9-79df1c299f3f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:50.084501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-35a61183-b62f-4cf9-ae96-f16478008fe5 req-4626b370-2e96-45f6-b62c-bf1598a5676e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:50.085274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-35a61183-b62f-4cf9-ae96-f16478008fe5 req-4626b370-2e96-45f6-b62c-bf1598a5676e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:50.085856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-35a61183-b62f-4cf9-ae96-f16478008fe5 req-4626b370-2e96-45f6-b62c-bf1598a5676e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:50.102493 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-35a61183-b62f-4cf9-ae96-f16478008fe5 req-4626b370-2e96-45f6-b62c-bf1598a5676e service nova] Creating event network-vif-plugged:ea7e2974-9ba7-4f04-a5d9-79df1c299f3f for instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 on nch1-master-1 Sep 27 10:15:50.109877 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-35a61183-b62f-4cf9-ae96-f16478008fe5 req-4626b370-2e96-45f6-b62c-bf1598a5676e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064103 Sep 27 10:15:50.112556 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2383/4360] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:15:50.217992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-336762aa-1a07-4e6a-b616-d665eb3e22f8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:50.239008 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-336762aa-1a07-4e6a-b616-d665eb3e22f8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1327 microversion: 2.1 time: 0.596444 Sep 27 10:15:50.256696 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1978/4361] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:49 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1327 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:50.480885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-42996fb2-f70c-4963-9b12-7a4990fcf370 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:50.481672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-42996fb2-f70c-4963-9b12-7a4990fcf370 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:50.492036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42996fb2-f70c-4963-9b12-7a4990fcf370 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:50.492725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42996fb2-f70c-4963-9b12-7a4990fcf370 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:50.493306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42996fb2-f70c-4963-9b12-7a4990fcf370 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:50.494227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42996fb2-f70c-4963-9b12-7a4990fcf370 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:50.494959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42996fb2-f70c-4963-9b12-7a4990fcf370 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:50.495622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42996fb2-f70c-4963-9b12-7a4990fcf370 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:50.655761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2fda9633-bf37-4057-9b71-b54d27c4118f tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:50.656385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2fda9633-bf37-4057-9b71-b54d27c4118f tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:50.670002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fda9633-bf37-4057-9b71-b54d27c4118f tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:50.670524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fda9633-bf37-4057-9b71-b54d27c4118f tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:50.671161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fda9633-bf37-4057-9b71-b54d27c4118f tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:50.672083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fda9633-bf37-4057-9b71-b54d27c4118f tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:50.672541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fda9633-bf37-4057-9b71-b54d27c4118f tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:50.673007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fda9633-bf37-4057-9b71-b54d27c4118f tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:50.780850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-42996fb2-f70c-4963-9b12-7a4990fcf370 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:50.792396 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-42996fb2-f70c-4963-9b12-7a4990fcf370 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 0.316930 Sep 27 10:15:50.812993 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2384/4362] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:50 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:50.997858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2fda9633-bf37-4057-9b71-b54d27c4118f tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:51.007211 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2fda9633-bf37-4057-9b71-b54d27c4118f tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1327 microversion: 2.1 time: 0.355401 Sep 27 10:15:51.011226 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1979/4363] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:50 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1327 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:51.267961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ced09481-81ab-49a6-a93e-7a1dc2850ea0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:51.268814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ced09481-81ab-49a6-a93e-7a1dc2850ea0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:51.279075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ced09481-81ab-49a6-a93e-7a1dc2850ea0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:51.279993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ced09481-81ab-49a6-a93e-7a1dc2850ea0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:51.280856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ced09481-81ab-49a6-a93e-7a1dc2850ea0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:51.282258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ced09481-81ab-49a6-a93e-7a1dc2850ea0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:51.283214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ced09481-81ab-49a6-a93e-7a1dc2850ea0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:51.284519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ced09481-81ab-49a6-a93e-7a1dc2850ea0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:51.658496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ced09481-81ab-49a6-a93e-7a1dc2850ea0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:51.673260 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ced09481-81ab-49a6-a93e-7a1dc2850ea0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1327 microversion: 2.1 time: 0.405470 Sep 27 10:15:51.674619 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2385/4364] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:51 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1327 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:51.823508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-81a0b2c8-2f3e-4aef-b532-acf160b6508b tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:51.824534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-81a0b2c8-2f3e-4aef-b532-acf160b6508b tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:51.837495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0b2c8-2f3e-4aef-b532-acf160b6508b tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:51.843980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0b2c8-2f3e-4aef-b532-acf160b6508b tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:51.847555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0b2c8-2f3e-4aef-b532-acf160b6508b tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:51.850257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0b2c8-2f3e-4aef-b532-acf160b6508b tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:51.851134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0b2c8-2f3e-4aef-b532-acf160b6508b tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:51.852816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0b2c8-2f3e-4aef-b532-acf160b6508b tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:52.031948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1268e6b2-a751-458b-b2cf-123f8fdc048b tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:52.032831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1268e6b2-a751-458b-b2cf-123f8fdc048b tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:52.047491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1268e6b2-a751-458b-b2cf-123f8fdc048b tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:52.048207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1268e6b2-a751-458b-b2cf-123f8fdc048b tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:52.051912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1268e6b2-a751-458b-b2cf-123f8fdc048b tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:52.059406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1268e6b2-a751-458b-b2cf-123f8fdc048b tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:52.060173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1268e6b2-a751-458b-b2cf-123f8fdc048b tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:52.060786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1268e6b2-a751-458b-b2cf-123f8fdc048b tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:52.417445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1268e6b2-a751-458b-b2cf-123f8fdc048b tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:52.437931 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1268e6b2-a751-458b-b2cf-123f8fdc048b tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1527 microversion: 2.1 time: 0.410346 Sep 27 10:15:52.439891 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2386/4365] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:52 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1527 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:52.478388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dbbb113d-71d6-4a9f-afa4-4c4129912c97 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:52.480842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dbbb113d-71d6-4a9f-afa4-4c4129912c97 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:52.501566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbbb113d-71d6-4a9f-afa4-4c4129912c97 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:52.502848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbbb113d-71d6-4a9f-afa4-4c4129912c97 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:52.503509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbbb113d-71d6-4a9f-afa4-4c4129912c97 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:52.505097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbbb113d-71d6-4a9f-afa4-4c4129912c97 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:52.505766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbbb113d-71d6-4a9f-afa4-4c4129912c97 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:52.506694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dbbb113d-71d6-4a9f-afa4-4c4129912c97 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:52.558253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:15:52.595183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-81a0b2c8-2f3e-4aef-b532-acf160b6508b tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:52.610205 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-81a0b2c8-2f3e-4aef-b532-acf160b6508b tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 0.790600 Sep 27 10:15:52.629468 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1980/4366] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:51 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 812 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:52.699320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-18df9335-5e54-492e-8676-044bbc74ea55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:52.699940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-18df9335-5e54-492e-8676-044bbc74ea55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:52.709816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18df9335-5e54-492e-8676-044bbc74ea55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:52.713445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18df9335-5e54-492e-8676-044bbc74ea55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:52.714107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18df9335-5e54-492e-8676-044bbc74ea55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:52.715021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18df9335-5e54-492e-8676-044bbc74ea55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:52.725840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18df9335-5e54-492e-8676-044bbc74ea55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:52.726283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18df9335-5e54-492e-8676-044bbc74ea55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:52.743188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dbbb113d-71d6-4a9f-afa4-4c4129912c97 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:52.760222 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dbbb113d-71d6-4a9f-afa4-4c4129912c97 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1527 microversion: 2.1 time: 0.299150 Sep 27 10:15:52.763179 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2387/4367] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:52 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1527 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:52.854980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:52.855502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:52.862729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:52.863667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:52.864343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:52.865440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:52.867328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:52.869008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:52.890535 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] HTTP exception thrown: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Request method failure captured: Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 HTTP/1.1 Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0-kCQkCMsP9WjS96O6yVE_vl8368AMI2KivqATiZIz_DaNN4VX3HUUJ9AwI89wfW5rDiTWeDuUv-SPWWL95SEwz6zF7pRo09Bxwij2G5kATMxzoSVpbjSfcckJgYlVcXBRjjhAqNXnP8_z-JMSMscc50PiQ_W65W-0upBpjeFoeHY Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 3e98ba3d755a4493aa1f6f88259b47f1 Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestMultiTenantJSON-1443059272 Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3e98ba3d755a4493aa1f6f88259b47f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3e98ba3d755a4493aa1f6f88259b47f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3e98ba3d755a4493aa1f6f88259b47f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3e98ba3d755a4493aa1f6f88259b47f1"}]}] Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestMultiTenantJSON-1443059272 Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 3e98ba3d755a4493aa1f6f88259b47f1 Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestMultiTenantJSON-1443059272 Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestMultiTenantJSON-1443059272-project Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 3a42c94a1e6b45b8af1e99ccdc73edf4 Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestMultiTenantJSON-1443059272-project Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '775df75d-9fec-4be2-bccb-b8c28dd3a404'} Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:15:52.892852 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:15:52.899248 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:52.899248 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:52.899248 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:52.899248 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:52.899248 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:52.899248 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:52.899248 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:52.899248 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:15:52.899248 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:15:52.899248 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:52.899248 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:52.903055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Returning 404 to user: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:15:52.905990 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7e10d9d9-711f-4d48-a179-6fb5c7681c3a tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] 10.222.0.26 "DELETE /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 404 len: 111 microversion: 2.1 time: 0.120664 Sep 27 10:15:52.907974 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2388/4368] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:15:52 2022] DELETE /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 111 bytes in 123 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:15:52.918894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-66e91ebf-4a03-40e5-aafe-45d43329ae8a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:52.920334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-66e91ebf-4a03-40e5-aafe-45d43329ae8a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:52.929692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66e91ebf-4a03-40e5-aafe-45d43329ae8a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:52.930475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66e91ebf-4a03-40e5-aafe-45d43329ae8a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:52.931155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66e91ebf-4a03-40e5-aafe-45d43329ae8a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:52.932991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66e91ebf-4a03-40e5-aafe-45d43329ae8a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:52.933768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66e91ebf-4a03-40e5-aafe-45d43329ae8a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:52.934394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66e91ebf-4a03-40e5-aafe-45d43329ae8a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:53.102775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-66e91ebf-4a03-40e5-aafe-45d43329ae8a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:53.112355 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-66e91ebf-4a03-40e5-aafe-45d43329ae8a tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1527 microversion: 2.1 time: 0.198048 Sep 27 10:15:53.113291 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2389/4369] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:52 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1527 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:53.143921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Action: 'update', calling method: >, body: {"server": {"name": "775df75d-9fec-4be2-bccb-b8c28dd3a404_new"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:53.147457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:53.156123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:53.156622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:53.157167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:53.157984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:53.158429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:53.158948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:53.176001 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] HTTP exception thrown: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Request method failure captured: Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: request: PUT /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 HTTP/1.1 Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 64 Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0-kCQkCMsP9WjS96O6yVE_vl8368AMI2KivqATiZIz_DaNN4VX3HUUJ9AwI89wfW5rDiTWeDuUv-SPWWL95SEwz6zF7pRo09Bxwij2G5kATMxzoSVpbjSfcckJgYlVcXBRjjhAqNXnP8_z-JMSMscc50PiQ_W65W-0upBpjeFoeHY Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 3e98ba3d755a4493aa1f6f88259b47f1 Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestMultiTenantJSON-1443059272 Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3e98ba3d755a4493aa1f6f88259b47f1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/3e98ba3d755a4493aa1f6f88259b47f1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/3e98ba3d755a4493aa1f6f88259b47f1"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_3e98ba3d755a4493aa1f6f88259b47f1"}]}] Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestMultiTenantJSON-1443059272 Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 3e98ba3d755a4493aa1f6f88259b47f1 Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestMultiTenantJSON-1443059272 Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestMultiTenantJSON-1443059272-project Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 3a42c94a1e6b45b8af1e99ccdc73edf4 Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestMultiTenantJSON-1443059272-project Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: {"server": {"name": "775df75d-9fec-4be2-bccb-b8c28dd3a404_new"}} Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '775df75d-9fec-4be2-bccb-b8c28dd3a404', 'body': {'server': {'name': '775df75d-9fec-4be2-bccb-b8c28dd3a404_new'}}} Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:15:53.178114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi [Previous line repeated 1 more time] Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 915, in update Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(ctxt, req, id, is_detail=True) Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] Returning 404 to user: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ad6008d1-a5e4-4354-aeaf-b32dc8220841 tempest-ServersNegativeTestMultiTenantJSON-1443059272 tempest-ServersNegativeTestMultiTenantJSON-1443059272-project] 10.222.0.26 "PUT /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 404 len: 111 microversion: 2.1 time: 0.047977 Sep 27 10:15:53.182539 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2390/4370] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:15:53 2022] PUT /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:15:53.191014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-40434d39-c710-404a-84fa-3b20b0fdeda1 req-e80f2ee4-7cda-4345-9ff0-092fcfd4a99d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "521b4b27-ea54-4de8-a96d-add9170f03c5", "name": "network-vif-plugged", "status": "completed", "tag": "30c45679-2756-42d1-a719-42e9e2b8c137"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:53.202211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-40434d39-c710-404a-84fa-3b20b0fdeda1 req-e80f2ee4-7cda-4345-9ff0-092fcfd4a99d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:53.202945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-40434d39-c710-404a-84fa-3b20b0fdeda1 req-e80f2ee4-7cda-4345-9ff0-092fcfd4a99d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:53.203484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-40434d39-c710-404a-84fa-3b20b0fdeda1 req-e80f2ee4-7cda-4345-9ff0-092fcfd4a99d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:53.217573 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-40434d39-c710-404a-84fa-3b20b0fdeda1 req-e80f2ee4-7cda-4345-9ff0-092fcfd4a99d service nova] Creating event network-vif-plugged:30c45679-2756-42d1-a719-42e9e2b8c137 for instance 521b4b27-ea54-4de8-a96d-add9170f03c5 on nch2-master-1 Sep 27 10:15:53.223477 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-40434d39-c710-404a-84fa-3b20b0fdeda1 req-e80f2ee4-7cda-4345-9ff0-092fcfd4a99d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038563 Sep 27 10:15:53.224262 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2391/4371] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:15:53.230585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cd185eec-547d-4c63-91a3-168d9224d1a3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:53.231261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cd185eec-547d-4c63-91a3-168d9224d1a3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:53.237635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd185eec-547d-4c63-91a3-168d9224d1a3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:53.238129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd185eec-547d-4c63-91a3-168d9224d1a3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:53.238573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd185eec-547d-4c63-91a3-168d9224d1a3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:53.239327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd185eec-547d-4c63-91a3-168d9224d1a3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:53.239789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd185eec-547d-4c63-91a3-168d9224d1a3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:53.240254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd185eec-547d-4c63-91a3-168d9224d1a3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:53.267146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cd185eec-547d-4c63-91a3-168d9224d1a3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:15:53.358771 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cd185eec-547d-4c63-91a3-168d9224d1a3 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "DELETE /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 204 len: 0 microversion: 2.1 time: 0.132267 Sep 27 10:15:53.359606 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2392/4372] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:15:53 2022] DELETE /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:15:53.368791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-786ca12c-ff6d-4e2f-809e-d61947602b4d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:53.369681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-786ca12c-ff6d-4e2f-809e-d61947602b4d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:53.375829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786ca12c-ff6d-4e2f-809e-d61947602b4d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:53.376409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786ca12c-ff6d-4e2f-809e-d61947602b4d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:53.376957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786ca12c-ff6d-4e2f-809e-d61947602b4d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:53.377725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786ca12c-ff6d-4e2f-809e-d61947602b4d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:53.378247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786ca12c-ff6d-4e2f-809e-d61947602b4d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:53.378656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786ca12c-ff6d-4e2f-809e-d61947602b4d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:53.435038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-18df9335-5e54-492e-8676-044bbc74ea55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:53.456501 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-18df9335-5e54-492e-8676-044bbc74ea55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1327 microversion: 2.1 time: 0.762619 Sep 27 10:15:53.457126 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1981/4373] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:52 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1327 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:53.562893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-786ca12c-ff6d-4e2f-809e-d61947602b4d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:53.574915 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-786ca12c-ff6d-4e2f-809e-d61947602b4d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1533 microversion: 2.1 time: 0.210414 Sep 27 10:15:53.575899 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2393/4374] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:53 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1533 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:53.638050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-55edc17f-2a86-4d7a-8f89-2c962476f5f4 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:53.638984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-55edc17f-2a86-4d7a-8f89-2c962476f5f4 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:53.645866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55edc17f-2a86-4d7a-8f89-2c962476f5f4 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:53.646369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55edc17f-2a86-4d7a-8f89-2c962476f5f4 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:53.646921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55edc17f-2a86-4d7a-8f89-2c962476f5f4 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:53.647666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55edc17f-2a86-4d7a-8f89-2c962476f5f4 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:53.648129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55edc17f-2a86-4d7a-8f89-2c962476f5f4 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:53.648832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55edc17f-2a86-4d7a-8f89-2c962476f5f4 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:53.874250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-55edc17f-2a86-4d7a-8f89-2c962476f5f4 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:53.889984 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-55edc17f-2a86-4d7a-8f89-2c962476f5f4 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 0.255946 Sep 27 10:15:53.890796 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1982/4375] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:53 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:54.484045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c968ee0a-63c9-4d57-80ee-6a41b1e05be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:54.485530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c968ee0a-63c9-4d57-80ee-6a41b1e05be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:54.495509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c968ee0a-63c9-4d57-80ee-6a41b1e05be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:54.496824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c968ee0a-63c9-4d57-80ee-6a41b1e05be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:54.497892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c968ee0a-63c9-4d57-80ee-6a41b1e05be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:54.499468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c968ee0a-63c9-4d57-80ee-6a41b1e05be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:54.500762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c968ee0a-63c9-4d57-80ee-6a41b1e05be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:54.502090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c968ee0a-63c9-4d57-80ee-6a41b1e05be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:54.610171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f8727898-0677-46a1-a6b5-80da9d73346d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:54.611421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f8727898-0677-46a1-a6b5-80da9d73346d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:54.652736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8727898-0677-46a1-a6b5-80da9d73346d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:54.655616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8727898-0677-46a1-a6b5-80da9d73346d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:54.656280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8727898-0677-46a1-a6b5-80da9d73346d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:54.662984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8727898-0677-46a1-a6b5-80da9d73346d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:54.663758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8727898-0677-46a1-a6b5-80da9d73346d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:54.664408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f8727898-0677-46a1-a6b5-80da9d73346d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:55.069021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c968ee0a-63c9-4d57-80ee-6a41b1e05be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:55.079230 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c968ee0a-63c9-4d57-80ee-6a41b1e05be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1327 microversion: 2.1 time: 0.599414 Sep 27 10:15:55.092995 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2394/4376] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:54 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1327 bytes in 608 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:55.092995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7da29d4f-617c-43e7-acc7-f3e86e3d15e6 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:55.093525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7da29d4f-617c-43e7-acc7-f3e86e3d15e6 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:55.114638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f8727898-0677-46a1-a6b5-80da9d73346d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:55.124966 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f8727898-0677-46a1-a6b5-80da9d73346d tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1533 microversion: 2.1 time: 0.523947 Sep 27 10:15:55.125433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7da29d4f-617c-43e7-acc7-f3e86e3d15e6 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:55.133523 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1983/4377] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:54 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1533 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:55.133617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7da29d4f-617c-43e7-acc7-f3e86e3d15e6 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:55.133617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7da29d4f-617c-43e7-acc7-f3e86e3d15e6 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:55.133617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7da29d4f-617c-43e7-acc7-f3e86e3d15e6 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:55.133617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7da29d4f-617c-43e7-acc7-f3e86e3d15e6 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:55.133617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7da29d4f-617c-43e7-acc7-f3e86e3d15e6 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:56.113515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2e999293-5d68-4929-aa10-505efa58d9e8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:56.116585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2e999293-5d68-4929-aa10-505efa58d9e8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:56.125116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7da29d4f-617c-43e7-acc7-f3e86e3d15e6 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:56.137009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e999293-5d68-4929-aa10-505efa58d9e8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:56.138597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e999293-5d68-4929-aa10-505efa58d9e8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:56.140941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e999293-5d68-4929-aa10-505efa58d9e8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:56.142980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e999293-5d68-4929-aa10-505efa58d9e8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:56.144279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e999293-5d68-4929-aa10-505efa58d9e8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:56.144951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e999293-5d68-4929-aa10-505efa58d9e8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:56.146904 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7da29d4f-617c-43e7-acc7-f3e86e3d15e6 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 1.056507 Sep 27 10:15:56.148767 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2395/4378] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:55 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 1062 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:56.160404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bc347fb9-c7d9-4a34-9e98-6f43558d8301 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:56.161436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bc347fb9-c7d9-4a34-9e98-6f43558d8301 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:56.181307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc347fb9-c7d9-4a34-9e98-6f43558d8301 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:56.182311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc347fb9-c7d9-4a34-9e98-6f43558d8301 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:56.183004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc347fb9-c7d9-4a34-9e98-6f43558d8301 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:56.184167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc347fb9-c7d9-4a34-9e98-6f43558d8301 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:56.184878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc347fb9-c7d9-4a34-9e98-6f43558d8301 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:56.185549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc347fb9-c7d9-4a34-9e98-6f43558d8301 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:56.409515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2e999293-5d68-4929-aa10-505efa58d9e8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:56.431006 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2e999293-5d68-4929-aa10-505efa58d9e8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1507 microversion: 2.1 time: 0.321606 Sep 27 10:15:56.433577 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1984/4379] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:56 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1507 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:56.466828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1fb2d549-93f3-4e53-adba-f43faead5f77 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:56.467969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1fb2d549-93f3-4e53-adba-f43faead5f77 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:56.481671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fb2d549-93f3-4e53-adba-f43faead5f77 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:56.482488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fb2d549-93f3-4e53-adba-f43faead5f77 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:56.483150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fb2d549-93f3-4e53-adba-f43faead5f77 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:56.485605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fb2d549-93f3-4e53-adba-f43faead5f77 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:56.487945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fb2d549-93f3-4e53-adba-f43faead5f77 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:56.488592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1fb2d549-93f3-4e53-adba-f43faead5f77 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:56.506022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bc347fb9-c7d9-4a34-9e98-6f43558d8301 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:56.516893 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bc347fb9-c7d9-4a34-9e98-6f43558d8301 tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1533 microversion: 2.1 time: 0.366711 Sep 27 10:15:56.522029 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2396/4380] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:56 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1533 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:56.814238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1fb2d549-93f3-4e53-adba-f43faead5f77 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:56.823758 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1fb2d549-93f3-4e53-adba-f43faead5f77 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1507 microversion: 2.1 time: 0.361893 Sep 27 10:15:56.826213 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1985/4381] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:56 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1507 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:15:56.850624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-702d4796-0273-4b42-8e4a-b57614415623 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:56.851054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-702d4796-0273-4b42-8e4a-b57614415623 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:56.873717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-702d4796-0273-4b42-8e4a-b57614415623 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:56.874244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-702d4796-0273-4b42-8e4a-b57614415623 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:56.874705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-702d4796-0273-4b42-8e4a-b57614415623 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:56.875456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-702d4796-0273-4b42-8e4a-b57614415623 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:56.875923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-702d4796-0273-4b42-8e4a-b57614415623 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:56.876352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-702d4796-0273-4b42-8e4a-b57614415623 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:56.942269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-702d4796-0273-4b42-8e4a-b57614415623 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:15:57.056879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-71f977a3-6e95-41a0-aa92-6246a045d583 req-9adaa4d6-c013-4758-9546-c76e39fb16a6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "775df75d-9fec-4be2-bccb-b8c28dd3a404", "name": "network-vif-deleted", "tag": "ea7e2974-9ba7-4f04-a5d9-79df1c299f3f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:57.110201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-71f977a3-6e95-41a0-aa92-6246a045d583 req-9adaa4d6-c013-4758-9546-c76e39fb16a6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:57.110975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-71f977a3-6e95-41a0-aa92-6246a045d583 req-9adaa4d6-c013-4758-9546-c76e39fb16a6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:57.114378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-71f977a3-6e95-41a0-aa92-6246a045d583 req-9adaa4d6-c013-4758-9546-c76e39fb16a6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:57.172938 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-71f977a3-6e95-41a0-aa92-6246a045d583 req-9adaa4d6-c013-4758-9546-c76e39fb16a6 service nova] Creating event network-vif-deleted:ea7e2974-9ba7-4f04-a5d9-79df1c299f3f for instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 on nch1-master-1 Sep 27 10:15:57.198533 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-71f977a3-6e95-41a0-aa92-6246a045d583 req-9adaa4d6-c013-4758-9546-c76e39fb16a6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.145403 Sep 27 10:15:57.198894 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1986/4382] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:15:57.248824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-19554bd9-0d87-47dd-82a4-13a192def714 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:57.261246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-19554bd9-0d87-47dd-82a4-13a192def714 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:57.290605 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-702d4796-0273-4b42-8e4a-b57614415623 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 204 len: 0 microversion: 2.1 time: 0.443765 Sep 27 10:15:57.292353 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2397/4383] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:15:56 2022] DELETE /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:15:57.299021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19554bd9-0d87-47dd-82a4-13a192def714 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:57.299813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19554bd9-0d87-47dd-82a4-13a192def714 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:57.301467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19554bd9-0d87-47dd-82a4-13a192def714 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:57.302415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19554bd9-0d87-47dd-82a4-13a192def714 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:57.305977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19554bd9-0d87-47dd-82a4-13a192def714 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:57.306415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1082d387-2a6a-4d68-b919-ad8c4cb71307 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:57.308117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-19554bd9-0d87-47dd-82a4-13a192def714 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:57.309815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1082d387-2a6a-4d68-b919-ad8c4cb71307 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:57.318563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1082d387-2a6a-4d68-b919-ad8c4cb71307 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:57.319426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1082d387-2a6a-4d68-b919-ad8c4cb71307 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:57.320012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1082d387-2a6a-4d68-b919-ad8c4cb71307 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:57.320953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1082d387-2a6a-4d68-b919-ad8c4cb71307 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:57.321557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1082d387-2a6a-4d68-b919-ad8c4cb71307 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:57.322319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1082d387-2a6a-4d68-b919-ad8c4cb71307 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:57.615590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-19554bd9-0d87-47dd-82a4-13a192def714 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:57.624605 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-19554bd9-0d87-47dd-82a4-13a192def714 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 0.399616 Sep 27 10:15:57.626309 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1987/4384] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:57 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:57.633037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:57.633653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:57.658557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:57.658972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:57.659424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:57.660153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:57.662137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:57.662745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:57.699398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:57.700200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:57.701300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:57.701925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lazy-loading 'fault' on Instance uuid 775df75d-9fec-4be2-bccb-b8c28dd3a404 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:15:57.746765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '02c331abed624c5f88835fa9fdfcfa5c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4a125c6e9abd46b68b5b5c9cea53ec96', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:57.757251 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1c678ad7-7af7-4508-9ee4-0cc84fcb991e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 200 len: 1317 microversion: 2.1 time: 0.129517 Sep 27 10:15:57.758457 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1988/4385] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:57 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 1317 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:57.843619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1082d387-2a6a-4d68-b919-ad8c4cb71307 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:57.866183 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1082d387-2a6a-4d68-b919-ad8c4cb71307 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1513 microversion: 2.1 time: 0.568646 Sep 27 10:15:57.869662 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2398/4386] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:57 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1513 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:58.651977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f98ddef2-c5b5-4ba9-af0d-e3bb8d36a285 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:58.652622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f98ddef2-c5b5-4ba9-af0d-e3bb8d36a285 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:58.675854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f98ddef2-c5b5-4ba9-af0d-e3bb8d36a285 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:58.679211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f98ddef2-c5b5-4ba9-af0d-e3bb8d36a285 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:58.680875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f98ddef2-c5b5-4ba9-af0d-e3bb8d36a285 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:58.690047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f98ddef2-c5b5-4ba9-af0d-e3bb8d36a285 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:58.696513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f98ddef2-c5b5-4ba9-af0d-e3bb8d36a285 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:58.697913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f98ddef2-c5b5-4ba9-af0d-e3bb8d36a285 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:58.781248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:58.782513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] [instance: 775df75d-9fec-4be2-bccb-b8c28dd3a404] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:58.807409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:58.807917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:58.808369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:58.809130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:58.809598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:58.810026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:58.905326 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] HTTP exception thrown: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Request method failure captured: Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 HTTP/1.1 Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0wpOLBaqXeiSRqNJR6Btgj_HbHu-n7ysjQmWsrDP4p4Pof9Y_U5mRgiPsELuFL56GmWn36oyZwDr_PXZTV7qQncc_hBxOytu4gQEm0utHSO-qQkqt-01jn8FzJhqc9UtoT5uy_gOyiQf1MLB_PYf8N0t3IX-N0Ti4Le_QPooaMGgk Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 4a125c6e9abd46b68b5b5c9cea53ec96 Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestMultiTenantJSON-306347937 Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4a125c6e9abd46b68b5b5c9cea53ec96"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4a125c6e9abd46b68b5b5c9cea53ec96"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/4a125c6e9abd46b68b5b5c9cea53ec96"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_4a125c6e9abd46b68b5b5c9cea53ec96"}]}] Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestMultiTenantJSON-306347937 Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 4a125c6e9abd46b68b5b5c9cea53ec96 Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestMultiTenantJSON-306347937 Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestMultiTenantJSON-306347937-project Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 02c331abed624c5f88835fa9fdfcfa5c Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestMultiTenantJSON-306347937-project Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '775df75d-9fec-4be2-bccb-b8c28dd3a404'} Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:58.910412 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:58.914596 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:15:58.914596 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:58.914596 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:15:58.914596 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:15:58.914596 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:15:58.914596 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:15:58.914596 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:15:58.914596 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. Sep 27 10:15:58.914596 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:15:58.914596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] Returning 404 to user: Instance 775df75d-9fec-4be2-bccb-b8c28dd3a404 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:15:58.916217 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d875f5b4-a456-4faf-a669-3f1149746c3e tempest-ServersNegativeTestMultiTenantJSON-306347937 tempest-ServersNegativeTestMultiTenantJSON-306347937-project] 10.222.0.26 "GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404" status: 404 len: 111 microversion: 2.1 time: 0.138126 Sep 27 10:15:58.916217 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2399/4387] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:58 2022] GET /compute/v2.1/servers/775df75d-9fec-4be2-bccb-b8c28dd3a404 => generated 111 bytes in 140 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:15:58.944833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9742f6dc-30cc-4490-a9c6-ec3d5ba35b68 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:15:58.945484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9742f6dc-30cc-4490-a9c6-ec3d5ba35b68 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:15:58.971442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9742f6dc-30cc-4490-a9c6-ec3d5ba35b68 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:58.979185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9742f6dc-30cc-4490-a9c6-ec3d5ba35b68 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:58.979680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9742f6dc-30cc-4490-a9c6-ec3d5ba35b68 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:58.981000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9742f6dc-30cc-4490-a9c6-ec3d5ba35b68 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:58.981484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9742f6dc-30cc-4490-a9c6-ec3d5ba35b68 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:58.981920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9742f6dc-30cc-4490-a9c6-ec3d5ba35b68 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:59.851279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f98ddef2-c5b5-4ba9-af0d-e3bb8d36a285 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:15:59.870009 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f98ddef2-c5b5-4ba9-af0d-e3bb8d36a285 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 1.219836 Sep 27 10:15:59.889836 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1989/4388] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:58 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 1242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:15:59.910642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-826f2332-02c5-41bf-9787-0823358e12d1 req-41a88596-c0d0-4b05-8816-0dc028bf066e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec", "name": "network-vif-plugged", "status": "completed", "tag": "3d3f5182-aaf9-4555-a437-c8414a163222"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:15:59.940077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-826f2332-02c5-41bf-9787-0823358e12d1 req-41a88596-c0d0-4b05-8816-0dc028bf066e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:15:59.940915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-826f2332-02c5-41bf-9787-0823358e12d1 req-41a88596-c0d0-4b05-8816-0dc028bf066e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:15:59.941571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-826f2332-02c5-41bf-9787-0823358e12d1 req-41a88596-c0d0-4b05-8816-0dc028bf066e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:15:59.979304 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-826f2332-02c5-41bf-9787-0823358e12d1 req-41a88596-c0d0-4b05-8816-0dc028bf066e service nova] Creating event network-vif-plugged:3d3f5182-aaf9-4555-a437-c8414a163222 for instance 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec on nch2-master-1 Sep 27 10:15:59.987908 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-826f2332-02c5-41bf-9787-0823358e12d1 req-41a88596-c0d0-4b05-8816-0dc028bf066e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093758 Sep 27 10:15:59.994535 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1990/4389] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:15:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:16:00.230434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9742f6dc-30cc-4490-a9c6-ec3d5ba35b68 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:00.238775 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9742f6dc-30cc-4490-a9c6-ec3d5ba35b68 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1513 microversion: 2.1 time: 1.309307 Sep 27 10:16:00.239358 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2400/4390] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:15:58 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1513 bytes in 1323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:00.894567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af9faf73-6e08-495b-ac03-016118b4f2cd tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:00.895888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-af9faf73-6e08-495b-ac03-016118b4f2cd tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:00.910750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9faf73-6e08-495b-ac03-016118b4f2cd tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:00.911589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9faf73-6e08-495b-ac03-016118b4f2cd tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:00.912386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9faf73-6e08-495b-ac03-016118b4f2cd tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:00.916919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9faf73-6e08-495b-ac03-016118b4f2cd tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:00.917660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9faf73-6e08-495b-ac03-016118b4f2cd tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:00.918119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af9faf73-6e08-495b-ac03-016118b4f2cd tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:00.975848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:01.268866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-143d85ec-8afe-408c-9930-8095ff0b4a30 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:01.269515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-143d85ec-8afe-408c-9930-8095ff0b4a30 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:01.285183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-143d85ec-8afe-408c-9930-8095ff0b4a30 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:01.285724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-143d85ec-8afe-408c-9930-8095ff0b4a30 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:01.287935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-143d85ec-8afe-408c-9930-8095ff0b4a30 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:01.289544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-143d85ec-8afe-408c-9930-8095ff0b4a30 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:01.290033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-143d85ec-8afe-408c-9930-8095ff0b4a30 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:01.295612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-143d85ec-8afe-408c-9930-8095ff0b4a30 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:01.672567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af9faf73-6e08-495b-ac03-016118b4f2cd tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:01.681725 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af9faf73-6e08-495b-ac03-016118b4f2cd tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1324 microversion: 2.1 time: 0.791309 Sep 27 10:16:01.685385 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1991/4391] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:00 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1324 bytes in 796 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:01.695558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-8a4a3bca-1a2e-47aa-8fe9-1078d1d9fe8c req-667d9379-cb3f-4530-a4e1-be818323b835 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "521b4b27-ea54-4de8-a96d-add9170f03c5", "name": "network-vif-deleted", "tag": "30c45679-2756-42d1-a719-42e9e2b8c137"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:01.720977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a4a3bca-1a2e-47aa-8fe9-1078d1d9fe8c req-667d9379-cb3f-4530-a4e1-be818323b835 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:01.721653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a4a3bca-1a2e-47aa-8fe9-1078d1d9fe8c req-667d9379-cb3f-4530-a4e1-be818323b835 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:01.722238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a4a3bca-1a2e-47aa-8fe9-1078d1d9fe8c req-667d9379-cb3f-4530-a4e1-be818323b835 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:01.744246 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8a4a3bca-1a2e-47aa-8fe9-1078d1d9fe8c req-667d9379-cb3f-4530-a4e1-be818323b835 service nova] Creating event network-vif-deleted:30c45679-2756-42d1-a719-42e9e2b8c137 for instance 521b4b27-ea54-4de8-a96d-add9170f03c5 on nch2-master-1 Sep 27 10:16:01.758233 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-8a4a3bca-1a2e-47aa-8fe9-1078d1d9fe8c req-667d9379-cb3f-4530-a4e1-be818323b835 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071358 Sep 27 10:16:01.760854 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1992/4392] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:16:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:16:01.820441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-143d85ec-8afe-408c-9930-8095ff0b4a30 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:01.845271 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-143d85ec-8afe-408c-9930-8095ff0b4a30 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 200 len: 1471 microversion: 2.1 time: 0.582656 Sep 27 10:16:01.864745 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2401/4393] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:01 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 1471 bytes in 603 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:02.712132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bd5aa73e-154c-459c-b6e3-46ce30007798 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:02.712890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bd5aa73e-154c-459c-b6e3-46ce30007798 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:02.760379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd5aa73e-154c-459c-b6e3-46ce30007798 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:02.762494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd5aa73e-154c-459c-b6e3-46ce30007798 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:02.763638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd5aa73e-154c-459c-b6e3-46ce30007798 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:02.764607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd5aa73e-154c-459c-b6e3-46ce30007798 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:02.767172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd5aa73e-154c-459c-b6e3-46ce30007798 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:02.767859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd5aa73e-154c-459c-b6e3-46ce30007798 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:02.873479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:02.874119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:02.901760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:02.904439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:02.905497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:02.909956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:02.911598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:02.915987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:02.966858 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 HTTP/1.1 Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '521b4b27-ea54-4de8-a96d-add9170f03c5'} Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:16:02.969803 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:16:02.973948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:02.973948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:16:02.973948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:16:02.973948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:16:02.973948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:16:02.973948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:16:02.973948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:16:02.983623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:16:02.985012 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3a7ef021-5b7d-46cc-ab8a-e4505c8e7768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 404 len: 111 microversion: 2.1 time: 0.116667 Sep 27 10:16:02.986177 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2402/4394] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:02 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 111 bytes in 119 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:16:02.999089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-977d6897-8f16-4684-a568-ec170fe8e333 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '.version_select at 0x7f7f62538670>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:03.063088 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-977d6897-8f16-4684-a568-ec170fe8e333 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-key-848436781" status: 202 len: 0 microversion: 2.1 time: 0.069808 Sep 27 10:16:03.074037 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2403/4395] 10.222.0.26 () {58 vars in 1206 bytes} [Tue Sep 27 10:16:02 2022] DELETE /compute/v2.1/os-keypairs/tempest-key-848436781 => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:16:03.178809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-677102652", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "662e23c5-f7e1-4e14-af6c-49d672076632"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:03.283132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bd5aa73e-154c-459c-b6e3-46ce30007798 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:03.311916 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bd5aa73e-154c-459c-b6e3-46ce30007798 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1521 microversion: 2.1 time: 0.606740 Sep 27 10:16:03.315413 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1993/4396] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:02 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1521 bytes in 611 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:03.337233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:03.338096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:03.347551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:03.352245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:03.353164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:03.354184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:03.355770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:03.356559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:03.383925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] validate_networks() for [('662e23c5-f7e1-4e14-af6c-49d672076632', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:16:03.413065 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] HTTP exception thrown: Instance is not a member of specified network Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Request method failure captured: Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec/ips/invalid HTTP/1.1 Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs1GR0U2hicUWNUHxB5yMCD8jsE0HlLueNAm4NZTJHLeiMR-9Ywo3mHrCEGzZezRKFYkMNvLAAmHsuDB85D_RQWEkk4HomdEspz73mLX62mVCxJYIhAagjNUSaEDbSZV4APjkty_-BSH_JG3rgfft0MoHCoiI4hP-fYJGh5hJXf-3pQ Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 4b17e56b43f04d14b1c5135d1566bef9 Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerAddressesNegativeTestJSON-1256384555 Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4b17e56b43f04d14b1c5135d1566bef9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4b17e56b43f04d14b1c5135d1566bef9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/4b17e56b43f04d14b1c5135d1566bef9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_4b17e56b43f04d14b1c5135d1566bef9"}]}] Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerAddressesNegativeTestJSON-1256384555 Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 4b17e56b43f04d14b1c5135d1566bef9 Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerAddressesNegativeTestJSON-1256384555 Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerAddressesNegativeTestJSON-1256384555-project Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 219bef5cfcfc496892611dce4bc1d89d Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerAddressesNegativeTestJSON-1256384555-project Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance is not a member of specified network Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'server_id': '2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec', 'id': 'invalid'} Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/ips.py", line 52, in show Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance is not a member of specified network Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance is not a member of specified network Sep 27 10:16:03.437001 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:03.445142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Returning 404 to user: Instance is not a member of specified network {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:16:03.445142 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-27dd4ca9-6f9d-407f-82b9-bfac2d2488a9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec/ips/invalid" status: 404 len: 91 microversion: 2.1 time: 0.107212 Sep 27 10:16:03.445142 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1994/4397] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 10:16:03 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec/ips/invalid => generated 91 bytes in 109 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Sep 27 10:16:03.451377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8a71a9ed-68a9-494a-8610-4f2447c7b875 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:03.451858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8a71a9ed-68a9-494a-8610-4f2447c7b875 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Failed to fetch instance by id 999 {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2981}} Sep 27 10:16:03.452363 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-8a71a9ed-68a9-494a-8610-4f2447c7b875 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] HTTP exception thrown: Instance 999 could not be found. Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8a71a9ed-68a9-494a-8610-4f2447c7b875 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Request method failure captured: Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/999/ips HTTP/1.1 Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs1GR0U2hicUWNUHxB5yMCD8jsE0HlLueNAm4NZTJHLeiMR-9Ywo3mHrCEGzZezRKFYkMNvLAAmHsuDB85D_RQWEkk4HomdEspz73mLX62mVCxJYIhAagjNUSaEDbSZV4APjkty_-BSH_JG3rgfft0MoHCoiI4hP-fYJGh5hJXf-3pQ Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 4b17e56b43f04d14b1c5135d1566bef9 Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerAddressesNegativeTestJSON-1256384555 Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4b17e56b43f04d14b1c5135d1566bef9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4b17e56b43f04d14b1c5135d1566bef9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/4b17e56b43f04d14b1c5135d1566bef9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_4b17e56b43f04d14b1c5135d1566bef9"}]}] Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerAddressesNegativeTestJSON-1256384555 Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 4b17e56b43f04d14b1c5135d1566bef9 Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerAddressesNegativeTestJSON-1256384555 Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerAddressesNegativeTestJSON-1256384555-project Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 219bef5cfcfc496892611dce4bc1d89d Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerAddressesNegativeTestJSON-1256384555-project Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 999 could not be found. Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'server_id': '999'} Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2982, in get Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_id) Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 999 could not be found. Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/ips.py", line 37, in index Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self._compute_api, context, server_id) Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 999 could not be found. Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 999 could not be found. Sep 27 10:16:03.454459 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:03.463234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8a71a9ed-68a9-494a-8610-4f2447c7b875 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Returning 404 to user: Instance 999 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:16:03.463234 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8a71a9ed-68a9-494a-8610-4f2447c7b875 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/999/ips" status: 404 len: 78 microversion: 2.1 time: 0.009421 Sep 27 10:16:03.463234 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1995/4398] 10.222.0.26 () {58 vars in 1149 bytes} [Tue Sep 27 10:16:03 2022] GET /compute/v2.1/servers/999/ips => generated 78 bytes in 11 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:16:03.467400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-01cade5a-3ab9-470a-abae-aa8610ae9ba0 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:03.467902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-01cade5a-3ab9-470a-abae-aa8610ae9ba0 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:03.491955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01cade5a-3ab9-470a-abae-aa8610ae9ba0 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:03.492693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01cade5a-3ab9-470a-abae-aa8610ae9ba0 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:03.493172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01cade5a-3ab9-470a-abae-aa8610ae9ba0 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:03.496522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01cade5a-3ab9-470a-abae-aa8610ae9ba0 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:03.497107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01cade5a-3ab9-470a-abae-aa8610ae9ba0 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:03.497590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-01cade5a-3ab9-470a-abae-aa8610ae9ba0 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:03.534357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-01cade5a-3ab9-470a-abae-aa8610ae9ba0 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:16:03.733305 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-01cade5a-3ab9-470a-abae-aa8610ae9ba0 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "DELETE /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 204 len: 0 microversion: 2.1 time: 0.270251 Sep 27 10:16:03.734820 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1996/4399] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:16:03 2022] DELETE /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 0 bytes in 273 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:16:03.752840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-70d6e42e-91ce-4e8e-8cc5-b758079c75db tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:03.755203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-70d6e42e-91ce-4e8e-8cc5-b758079c75db tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:03.765453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70d6e42e-91ce-4e8e-8cc5-b758079c75db tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:03.766329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70d6e42e-91ce-4e8e-8cc5-b758079c75db tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:03.774386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70d6e42e-91ce-4e8e-8cc5-b758079c75db tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:03.775857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70d6e42e-91ce-4e8e-8cc5-b758079c75db tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:03.776755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70d6e42e-91ce-4e8e-8cc5-b758079c75db tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:03.777410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70d6e42e-91ce-4e8e-8cc5-b758079c75db tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:04.051701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-70d6e42e-91ce-4e8e-8cc5-b758079c75db tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:04.061176 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-70d6e42e-91ce-4e8e-8cc5-b758079c75db tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1527 microversion: 2.1 time: 0.321474 Sep 27 10:16:04.064053 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1997/4400] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:03 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1527 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:05.091849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c0c662f0-a360-4441-b120-f86360fd2542 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:05.092617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c0c662f0-a360-4441-b120-f86360fd2542 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:05.114387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c662f0-a360-4441-b120-f86360fd2542 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:05.120282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c662f0-a360-4441-b120-f86360fd2542 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:05.121298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c662f0-a360-4441-b120-f86360fd2542 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:05.122550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c662f0-a360-4441-b120-f86360fd2542 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:05.125045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c662f0-a360-4441-b120-f86360fd2542 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:05.126397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c662f0-a360-4441-b120-f86360fd2542 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:05.212388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:16:05.213904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:16:05.216766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:16:05.217269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:16:05.218628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:16:05.233293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:05.234161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:05.234879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:05.236146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:05.236867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:05.237550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:05.268597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:16:05.289369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for user a7b8599d0b40424087e52d8e35b71ec4 and project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:16:05.309739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:16:05.313095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:16:05.406167 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9b4cfbc8-8dd9-4c86-9a46-984e6d3cd844 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.237242 Sep 27 10:16:05.417055 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2404/4401] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:16:03 2022] POST /compute/v2.1/servers => generated 380 bytes in 2249 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:16:05.426609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-38c38aad-b472-4cc6-9e4a-1f8ee224dac1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:05.446124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-38c38aad-b472-4cc6-9e4a-1f8ee224dac1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:05.546803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-38c38aad-b472-4cc6-9e4a-1f8ee224dac1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:05.560471 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-38c38aad-b472-4cc6-9e4a-1f8ee224dac1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1208 microversion: 2.1 time: 0.139050 Sep 27 10:16:05.581056 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2405/4402] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:05 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1208 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:06.589237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f8a78b4-89e2-4c91-9876-3783358432b2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:06.590301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7f8a78b4-89e2-4c91-9876-3783358432b2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:06.621901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f8a78b4-89e2-4c91-9876-3783358432b2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:06.622463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f8a78b4-89e2-4c91-9876-3783358432b2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:06.624717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f8a78b4-89e2-4c91-9876-3783358432b2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:06.625756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f8a78b4-89e2-4c91-9876-3783358432b2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:06.627894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f8a78b4-89e2-4c91-9876-3783358432b2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:06.628519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f8a78b4-89e2-4c91-9876-3783358432b2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:06.874158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7f8a78b4-89e2-4c91-9876-3783358432b2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:06.885660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:06.894997 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7f8a78b4-89e2-4c91-9876-3783358432b2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1212 microversion: 2.1 time: 0.310729 Sep 27 10:16:06.911768 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2406/4403] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:06 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1212 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:06.937276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c0c662f0-a360-4441-b120-f86360fd2542 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:06.963975 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c0c662f0-a360-4441-b120-f86360fd2542 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1527 microversion: 2.1 time: 1.876135 Sep 27 10:16:06.970728 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1998/4404] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:05 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1527 bytes in 1884 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:07.929901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9aea065b-292b-46e7-ba65-a34b11fb50ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:07.931214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9aea065b-292b-46e7-ba65-a34b11fb50ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:07.955094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9aea065b-292b-46e7-ba65-a34b11fb50ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:07.955870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9aea065b-292b-46e7-ba65-a34b11fb50ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:07.956315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9aea065b-292b-46e7-ba65-a34b11fb50ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:07.959507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9aea065b-292b-46e7-ba65-a34b11fb50ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:07.962816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9aea065b-292b-46e7-ba65-a34b11fb50ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:07.963883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9aea065b-292b-46e7-ba65-a34b11fb50ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:07.996105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c4d176e-c4d9-40f9-a5f6-594eb43363d9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:07.996892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c4d176e-c4d9-40f9-a5f6-594eb43363d9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:08.004496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c4d176e-c4d9-40f9-a5f6-594eb43363d9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:08.005015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c4d176e-c4d9-40f9-a5f6-594eb43363d9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:08.005451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c4d176e-c4d9-40f9-a5f6-594eb43363d9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:08.006196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c4d176e-c4d9-40f9-a5f6-594eb43363d9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:08.006650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c4d176e-c4d9-40f9-a5f6-594eb43363d9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:08.007074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c4d176e-c4d9-40f9-a5f6-594eb43363d9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:08.143406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9aea065b-292b-46e7-ba65-a34b11fb50ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:08.161514 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9aea065b-292b-46e7-ba65-a34b11fb50ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1266 microversion: 2.1 time: 0.240227 Sep 27 10:16:08.165504 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2407/4405] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:07 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1266 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:08.205253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c4d176e-c4d9-40f9-a5f6-594eb43363d9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '219bef5cfcfc496892611dce4bc1d89d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4b17e56b43f04d14b1c5135d1566bef9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:08.219688 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c4d176e-c4d9-40f9-a5f6-594eb43363d9 tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 200 len: 1485 microversion: 2.1 time: 0.227291 Sep 27 10:16:08.220576 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 1999/4406] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:07 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 1485 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:08.337266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-b246377f-75f7-4a7d-8880-84fac875e85a req-48d807c1-8532-4770-844d-7f1f7a49a1ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec", "name": "network-vif-deleted", "tag": "3d3f5182-aaf9-4555-a437-c8414a163222"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:08.366789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b246377f-75f7-4a7d-8880-84fac875e85a req-48d807c1-8532-4770-844d-7f1f7a49a1ce service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:08.367313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b246377f-75f7-4a7d-8880-84fac875e85a req-48d807c1-8532-4770-844d-7f1f7a49a1ce service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:08.367762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b246377f-75f7-4a7d-8880-84fac875e85a req-48d807c1-8532-4770-844d-7f1f7a49a1ce service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:08.399694 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b246377f-75f7-4a7d-8880-84fac875e85a req-48d807c1-8532-4770-844d-7f1f7a49a1ce service nova] Creating event network-vif-deleted:3d3f5182-aaf9-4555-a437-c8414a163222 for instance 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec on nch2-master-1 Sep 27 10:16:08.441618 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-b246377f-75f7-4a7d-8880-84fac875e85a req-48d807c1-8532-4770-844d-7f1f7a49a1ce service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.114725 Sep 27 10:16:08.446267 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2408/4407] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:16:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:16:09.188381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b1456b86-c96c-4dbd-b4a9-0a0b2dfbddbc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:09.189149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b1456b86-c96c-4dbd-b4a9-0a0b2dfbddbc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:09.205134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1456b86-c96c-4dbd-b4a9-0a0b2dfbddbc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:09.206963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1456b86-c96c-4dbd-b4a9-0a0b2dfbddbc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:09.207596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1456b86-c96c-4dbd-b4a9-0a0b2dfbddbc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:09.209969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1456b86-c96c-4dbd-b4a9-0a0b2dfbddbc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:09.211370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1456b86-c96c-4dbd-b4a9-0a0b2dfbddbc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:09.213623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1456b86-c96c-4dbd-b4a9-0a0b2dfbddbc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:09.245270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:09.245968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] [instance: 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:09.260197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:09.260985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:09.261629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:09.265643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:09.271641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:09.272354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:09.316324 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] HTTP exception thrown: Instance 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec could not be found. Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Request method failure captured: Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec HTTP/1.1 Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs1GR0U2hicUWNUHxB5yMCD8jsE0HlLueNAm4NZTJHLeiMR-9Ywo3mHrCEGzZezRKFYkMNvLAAmHsuDB85D_RQWEkk4HomdEspz73mLX62mVCxJYIhAagjNUSaEDbSZV4APjkty_-BSH_JG3rgfft0MoHCoiI4hP-fYJGh5hJXf-3pQ Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 4b17e56b43f04d14b1c5135d1566bef9 Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerAddressesNegativeTestJSON-1256384555 Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4b17e56b43f04d14b1c5135d1566bef9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4b17e56b43f04d14b1c5135d1566bef9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/4b17e56b43f04d14b1c5135d1566bef9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_4b17e56b43f04d14b1c5135d1566bef9"}]}] Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerAddressesNegativeTestJSON-1256384555 Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 4b17e56b43f04d14b1c5135d1566bef9 Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerAddressesNegativeTestJSON-1256384555 Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerAddressesNegativeTestJSON-1256384555-project Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 219bef5cfcfc496892611dce4bc1d89d Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerAddressesNegativeTestJSON-1256384555-project Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec could not be found. Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec'} Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec could not be found. Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec could not be found. Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:16:09.318145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:16:09.322032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:16:09.322032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:09.322032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:16:09.322032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:16:09.322032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:16:09.322032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:16:09.322032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec could not be found. Sep 27 10:16:09.322032 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:16:09.322032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] Returning 404 to user: Instance 2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:16:09.322032 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fcd7726d-5353-447d-9bef-cc43c83aa69a tempest-ServerAddressesNegativeTestJSON-1256384555 tempest-ServerAddressesNegativeTestJSON-1256384555-project] 10.222.0.26 "GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec" status: 404 len: 111 microversion: 2.1 time: 0.079859 Sep 27 10:16:09.333924 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2409/4408] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:09 2022] GET /compute/v2.1/servers/2bb2d3e8-0e09-432d-b76b-f2dfa8efd8ec => generated 111 bytes in 95 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:16:09.349414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b1456b86-c96c-4dbd-b4a9-0a0b2dfbddbc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:09.362195 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b1456b86-c96c-4dbd-b4a9-0a0b2dfbddbc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1266 microversion: 2.1 time: 0.178552 Sep 27 10:16:09.366198 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2000/4409] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:09 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1266 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:10.387683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-922b3433-96f1-4831-bb87-0d28b3c775a7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:10.389131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-922b3433-96f1-4831-bb87-0d28b3c775a7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:10.402848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-922b3433-96f1-4831-bb87-0d28b3c775a7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:10.403809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-922b3433-96f1-4831-bb87-0d28b3c775a7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:10.403809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-922b3433-96f1-4831-bb87-0d28b3c775a7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:10.404430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-922b3433-96f1-4831-bb87-0d28b3c775a7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:10.409589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-922b3433-96f1-4831-bb87-0d28b3c775a7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:10.410318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-922b3433-96f1-4831-bb87-0d28b3c775a7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:10.632511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-922b3433-96f1-4831-bb87-0d28b3c775a7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:10.651557 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-922b3433-96f1-4831-bb87-0d28b3c775a7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1266 microversion: 2.1 time: 0.267697 Sep 27 10:16:10.652347 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2410/4410] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:10 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1266 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:11.680835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-03bfaeb7-745a-402d-8eef-24a5ea798aac tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:11.682391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-03bfaeb7-745a-402d-8eef-24a5ea798aac tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:11.719183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03bfaeb7-745a-402d-8eef-24a5ea798aac tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:11.726955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03bfaeb7-745a-402d-8eef-24a5ea798aac tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:11.727852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03bfaeb7-745a-402d-8eef-24a5ea798aac tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:11.733004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03bfaeb7-745a-402d-8eef-24a5ea798aac tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:11.733656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03bfaeb7-745a-402d-8eef-24a5ea798aac tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:11.737044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-03bfaeb7-745a-402d-8eef-24a5ea798aac tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:12.141050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-03bfaeb7-745a-402d-8eef-24a5ea798aac tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:12.157151 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-03bfaeb7-745a-402d-8eef-24a5ea798aac tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1308 microversion: 2.1 time: 0.479684 Sep 27 10:16:12.176063 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2001/4411] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:11 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1308 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:13.184866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d29415f-3bf8-48b4-925c-2d613121a98b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:13.186870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d29415f-3bf8-48b4-925c-2d613121a98b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:13.215515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d29415f-3bf8-48b4-925c-2d613121a98b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:13.216899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d29415f-3bf8-48b4-925c-2d613121a98b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:13.220834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d29415f-3bf8-48b4-925c-2d613121a98b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:13.225914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d29415f-3bf8-48b4-925c-2d613121a98b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:13.229258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d29415f-3bf8-48b4-925c-2d613121a98b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:13.229915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d29415f-3bf8-48b4-925c-2d613121a98b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:14.055114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1d29415f-3bf8-48b4-925c-2d613121a98b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:14.075686 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d29415f-3bf8-48b4-925c-2d613121a98b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1308 microversion: 2.1 time: 0.894305 Sep 27 10:16:14.079499 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2411/4412] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:13 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1308 bytes in 900 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:15.105129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4845815e-f6e6-4047-9741-36d543af795b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:15.105739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4845815e-f6e6-4047-9741-36d543af795b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:15.121050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=84,nova_api:UPDATE=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:15.125288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4845815e-f6e6-4047-9741-36d543af795b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:15.126180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4845815e-f6e6-4047-9741-36d543af795b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:15.126639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4845815e-f6e6-4047-9741-36d543af795b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:15.127394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4845815e-f6e6-4047-9741-36d543af795b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:15.127858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4845815e-f6e6-4047-9741-36d543af795b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:15.128287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4845815e-f6e6-4047-9741-36d543af795b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:15.229965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4f920097-fcb7-4458-9b65-b29b179676de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:15.233283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4f920097-fcb7-4458-9b65-b29b179676de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:15.240902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f920097-fcb7-4458-9b65-b29b179676de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:15.241723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f920097-fcb7-4458-9b65-b29b179676de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:15.242402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f920097-fcb7-4458-9b65-b29b179676de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:15.243363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f920097-fcb7-4458-9b65-b29b179676de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:15.244044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f920097-fcb7-4458-9b65-b29b179676de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:15.244730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f920097-fcb7-4458-9b65-b29b179676de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:15.266965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:15.280377 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4f920097-fcb7-4458-9b65-b29b179676de tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.054339 Sep 27 10:16:15.281396 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2412/4413] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:16:15 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89/os-volume_attachments => generated 25 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 27 10:16:15.389094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b256f8e3-869f-4a6e-b35b-9525eb8db1d9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:15.391152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b256f8e3-869f-4a6e-b35b-9525eb8db1d9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:15.410559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b256f8e3-869f-4a6e-b35b-9525eb8db1d9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:15.420034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b256f8e3-869f-4a6e-b35b-9525eb8db1d9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:15.420743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b256f8e3-869f-4a6e-b35b-9525eb8db1d9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:15.425604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b256f8e3-869f-4a6e-b35b-9525eb8db1d9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:15.426250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b256f8e3-869f-4a6e-b35b-9525eb8db1d9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:15.426800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b256f8e3-869f-4a6e-b35b-9525eb8db1d9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:15.477553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b256f8e3-869f-4a6e-b35b-9525eb8db1d9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:16:15.561354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:15.591857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4845815e-f6e6-4047-9741-36d543af795b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:15.600693 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4845815e-f6e6-4047-9741-36d543af795b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1308 microversion: 2.1 time: 0.499406 Sep 27 10:16:15.601305 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2002/4414] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:15 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1308 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:15.665472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:15.674565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b256f8e3-869f-4a6e-b35b-9525eb8db1d9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "DELETE /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 204 len: 0 microversion: 2.1 time: 0.290072 Sep 27 10:16:15.675479 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2413/4415] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:16:15 2022] DELETE /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:16:15.684394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9ed70468-bd12-4418-85d9-b08d7a8cf021 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:15.685263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9ed70468-bd12-4418-85d9-b08d7a8cf021 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:15.693009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ed70468-bd12-4418-85d9-b08d7a8cf021 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:15.697326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ed70468-bd12-4418-85d9-b08d7a8cf021 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:15.699328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ed70468-bd12-4418-85d9-b08d7a8cf021 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:15.700724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ed70468-bd12-4418-85d9-b08d7a8cf021 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:15.705302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ed70468-bd12-4418-85d9-b08d7a8cf021 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:15.706188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ed70468-bd12-4418-85d9-b08d7a8cf021 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:15.955372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9ed70468-bd12-4418-85d9-b08d7a8cf021 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:15.964051 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9ed70468-bd12-4418-85d9-b08d7a8cf021 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1673 microversion: 2.1 time: 0.283528 Sep 27 10:16:15.969467 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2003/4416] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:15 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1673 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:16.470274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-159c2272-20d6-4a16-87d7-660c98cb99ac req-92b5f0c1-e9d7-43f5-b781-667df7983add service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f13eb59c-b2be-4b37-addc-676d74eff6ed", "tag": "2a400220-e798-40d4-8e16-33e898bba375"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:16.501848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-159c2272-20d6-4a16-87d7-660c98cb99ac req-92b5f0c1-e9d7-43f5-b781-667df7983add service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:16.503801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-159c2272-20d6-4a16-87d7-660c98cb99ac req-92b5f0c1-e9d7-43f5-b781-667df7983add service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:16.511672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-159c2272-20d6-4a16-87d7-660c98cb99ac req-92b5f0c1-e9d7-43f5-b781-667df7983add service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:16.565255 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-159c2272-20d6-4a16-87d7-660c98cb99ac req-92b5f0c1-e9d7-43f5-b781-667df7983add service nova] Creating event network-changed:2a400220-e798-40d4-8e16-33e898bba375 for instance f13eb59c-b2be-4b37-addc-676d74eff6ed on nch1-master-1 Sep 27 10:16:16.576893 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-159c2272-20d6-4a16-87d7-660c98cb99ac req-92b5f0c1-e9d7-43f5-b781-667df7983add service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.115048 Sep 27 10:16:16.580328 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2414/4417] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:16:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:16:16.624205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3f5108b6-3133-401e-ac0f-307489298a0b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:16.624904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3f5108b6-3133-401e-ac0f-307489298a0b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:16.634373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f5108b6-3133-401e-ac0f-307489298a0b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:16.634373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f5108b6-3133-401e-ac0f-307489298a0b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:16.634373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f5108b6-3133-401e-ac0f-307489298a0b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:16.634803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f5108b6-3133-401e-ac0f-307489298a0b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:16.635246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f5108b6-3133-401e-ac0f-307489298a0b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:16.635675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f5108b6-3133-401e-ac0f-307489298a0b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:16.996916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5e22c440-cc5b-4057-8446-fa19a1dd3014 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:17.001339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5e22c440-cc5b-4057-8446-fa19a1dd3014 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:17.038588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e22c440-cc5b-4057-8446-fa19a1dd3014 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:17.041238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e22c440-cc5b-4057-8446-fa19a1dd3014 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:17.043123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e22c440-cc5b-4057-8446-fa19a1dd3014 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:17.044263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e22c440-cc5b-4057-8446-fa19a1dd3014 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:17.047455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e22c440-cc5b-4057-8446-fa19a1dd3014 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:17.048160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e22c440-cc5b-4057-8446-fa19a1dd3014 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:17.728589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5e22c440-cc5b-4057-8446-fa19a1dd3014 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:17.743265 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5e22c440-cc5b-4057-8446-fa19a1dd3014 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1673 microversion: 2.1 time: 0.753973 Sep 27 10:16:17.764496 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2415/4418] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:16 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1673 bytes in 776 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:18.178437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3f5108b6-3133-401e-ac0f-307489298a0b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:18.193958 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3f5108b6-3133-401e-ac0f-307489298a0b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1308 microversion: 2.1 time: 1.569442 Sep 27 10:16:18.197053 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2004/4419] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:16 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1308 bytes in 1577 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:18.778870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53736e8a-44dd-49c1-a4dd-e3478d4f8e21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:18.779768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53736e8a-44dd-49c1-a4dd-e3478d4f8e21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:18.796132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53736e8a-44dd-49c1-a4dd-e3478d4f8e21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:18.796640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53736e8a-44dd-49c1-a4dd-e3478d4f8e21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:18.797111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53736e8a-44dd-49c1-a4dd-e3478d4f8e21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:18.797837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53736e8a-44dd-49c1-a4dd-e3478d4f8e21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:18.798294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53736e8a-44dd-49c1-a4dd-e3478d4f8e21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:18.810189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53736e8a-44dd-49c1-a4dd-e3478d4f8e21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:19.225414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-196387c9-c53d-446a-a5e7-2e333537ee49 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:19.226763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-196387c9-c53d-446a-a5e7-2e333537ee49 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:19.262513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196387c9-c53d-446a-a5e7-2e333537ee49 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:19.267297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196387c9-c53d-446a-a5e7-2e333537ee49 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:19.268344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196387c9-c53d-446a-a5e7-2e333537ee49 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:19.269749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196387c9-c53d-446a-a5e7-2e333537ee49 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:19.271866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196387c9-c53d-446a-a5e7-2e333537ee49 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:19.272521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-196387c9-c53d-446a-a5e7-2e333537ee49 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:20.087479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-53736e8a-44dd-49c1-a4dd-e3478d4f8e21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:20.102117 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53736e8a-44dd-49c1-a4dd-e3478d4f8e21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1673 microversion: 2.1 time: 1.334121 Sep 27 10:16:20.110163 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2416/4420] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:18 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1673 bytes in 1343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:20.162690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-196387c9-c53d-446a-a5e7-2e333537ee49 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:20.175370 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-196387c9-c53d-446a-a5e7-2e333537ee49 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1308 microversion: 2.1 time: 0.955735 Sep 27 10:16:20.178315 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2005/4421] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:19 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1308 bytes in 959 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:20.982323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-2e58e66d-3df2-4ca5-a858-f1695182fb79 req-56f4ccd0-a24f-4423-88fa-20e08b9403d6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "49f9e4be-7594-4603-8f69-8765513f8b89", "name": "network-vif-deleted", "tag": "8c35173c-f9f3-41ad-a61d-50ef6ca5f054"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:20.999326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2e58e66d-3df2-4ca5-a858-f1695182fb79 req-56f4ccd0-a24f-4423-88fa-20e08b9403d6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:21.000162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2e58e66d-3df2-4ca5-a858-f1695182fb79 req-56f4ccd0-a24f-4423-88fa-20e08b9403d6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:21.001064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2e58e66d-3df2-4ca5-a858-f1695182fb79 req-56f4ccd0-a24f-4423-88fa-20e08b9403d6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:21.016466 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-2e58e66d-3df2-4ca5-a858-f1695182fb79 req-56f4ccd0-a24f-4423-88fa-20e08b9403d6 service nova] Creating event network-vif-deleted:8c35173c-f9f3-41ad-a61d-50ef6ca5f054 for instance 49f9e4be-7594-4603-8f69-8765513f8b89 on nch1-master-1 Sep 27 10:16:21.024408 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-2e58e66d-3df2-4ca5-a858-f1695182fb79 req-56f4ccd0-a24f-4423-88fa-20e08b9403d6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052884 Sep 27 10:16:21.026043 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2417/4422] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:16:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:16:21.139342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3ee4bd96-92f8-4d40-bc6e-a7b7ec9220fe tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:21.140170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3ee4bd96-92f8-4d40-bc6e-a7b7ec9220fe tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:21.157965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ee4bd96-92f8-4d40-bc6e-a7b7ec9220fe tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:21.159761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ee4bd96-92f8-4d40-bc6e-a7b7ec9220fe tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:21.160481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ee4bd96-92f8-4d40-bc6e-a7b7ec9220fe tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:21.162893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ee4bd96-92f8-4d40-bc6e-a7b7ec9220fe tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:21.164021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ee4bd96-92f8-4d40-bc6e-a7b7ec9220fe tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:21.165259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ee4bd96-92f8-4d40-bc6e-a7b7ec9220fe tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:21.204426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8ebb4e34-cdeb-4a0a-ac69-21f656bffeb4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:21.205291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8ebb4e34-cdeb-4a0a-ac69-21f656bffeb4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:21.230509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ebb4e34-cdeb-4a0a-ac69-21f656bffeb4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:21.231017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ebb4e34-cdeb-4a0a-ac69-21f656bffeb4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:21.231458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ebb4e34-cdeb-4a0a-ac69-21f656bffeb4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:21.236936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ebb4e34-cdeb-4a0a-ac69-21f656bffeb4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:21.237842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ebb4e34-cdeb-4a0a-ac69-21f656bffeb4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:21.243059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ebb4e34-cdeb-4a0a-ac69-21f656bffeb4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:21.458528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3ee4bd96-92f8-4d40-bc6e-a7b7ec9220fe tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:21.482878 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3ee4bd96-92f8-4d40-bc6e-a7b7ec9220fe tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 200 len: 1322 microversion: 2.1 time: 0.350890 Sep 27 10:16:21.486853 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2006/4423] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:21 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 1322 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:21.998032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8ebb4e34-cdeb-4a0a-ac69-21f656bffeb4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:22.017618 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8ebb4e34-cdeb-4a0a-ac69-21f656bffeb4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1308 microversion: 2.1 time: 0.818803 Sep 27 10:16:22.020388 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2418/4424] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:21 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1308 bytes in 824 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:22.521100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:22.522376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:22.553878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:22.554634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:22.555834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:22.557240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:22.558034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:22.561335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:22.584950 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] HTTP exception thrown: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Request method failure captured: Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 HTTP/1.1 Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMszABSno_2kqm3BW6SUWKFiDAujuW5Xnb7TMluE7Z34HQ5ZZ0Gk_H4nclSs5JgEczlL-fW3yspIl5udeCuxAqzrKWf09XSghDeGo6ZAMbfaXvl-0XHv0hoOQwx_Y5MP4or9tGcXTOMVnEVPWO9TJ2xA61_3oGcuZds7KIPUHECLVq_Q Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e2e00c90216b4d1b80ac7892ee23550f"}]}] Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 279d3662f60c4a979c22ee3e8fa92865 Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '49f9e4be-7594-4603-8f69-8765513f8b89'} Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:16:22.588228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:22.596868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:16:22.596868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:16:22.596868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:16:22.596868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:16:22.596868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:16:22.596868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:16:22.596868 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:16:22.596868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Returning 404 to user: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:16:22.596868 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-07bd7bbb-fcdf-4384-91e8-c33adaa64b55 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 404 len: 111 microversion: 2.1 time: 0.082811 Sep 27 10:16:22.596868 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2007/4425] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:22 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 111 bytes in 85 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:16:22.600787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5a3d81e5-e310-499d-b885-7953fbb1cb2b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '.version_select at 0x7f7f637305e0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:22.661518 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5a3d81e5-e310-499d-b885-7953fbb1cb2b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-436902387" status: 202 len: 0 microversion: 2.1 time: 0.065037 Sep 27 10:16:22.666121 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2419/4426] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:16:22 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-436902387 => generated 0 bytes in 71 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:16:23.048634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-14593811-6450-40e2-a953-2b87ec566855 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:23.049351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-14593811-6450-40e2-a953-2b87ec566855 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:23.059093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14593811-6450-40e2-a953-2b87ec566855 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:23.059598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14593811-6450-40e2-a953-2b87ec566855 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:23.060039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14593811-6450-40e2-a953-2b87ec566855 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:23.060780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14593811-6450-40e2-a953-2b87ec566855 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:23.063851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14593811-6450-40e2-a953-2b87ec566855 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:23.066993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14593811-6450-40e2-a953-2b87ec566855 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:23.500264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-14593811-6450-40e2-a953-2b87ec566855 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:23.518131 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-14593811-6450-40e2-a953-2b87ec566855 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1308 microversion: 2.1 time: 0.471856 Sep 27 10:16:23.520411 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2008/4427] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:23 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1308 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:24.548017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c0b2d51d-eee3-4dd5-807e-7e8324a4b212 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:24.551810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c0b2d51d-eee3-4dd5-807e-7e8324a4b212 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:24.579039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0b2d51d-eee3-4dd5-807e-7e8324a4b212 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:24.581263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0b2d51d-eee3-4dd5-807e-7e8324a4b212 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:24.581763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0b2d51d-eee3-4dd5-807e-7e8324a4b212 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:24.582591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0b2d51d-eee3-4dd5-807e-7e8324a4b212 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:24.583066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0b2d51d-eee3-4dd5-807e-7e8324a4b212 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:24.583541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0b2d51d-eee3-4dd5-807e-7e8324a4b212 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:25.166588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c0b2d51d-eee3-4dd5-807e-7e8324a4b212 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:25.181891 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c0b2d51d-eee3-4dd5-807e-7e8324a4b212 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1308 microversion: 2.1 time: 0.638443 Sep 27 10:16:25.182907 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2420/4428] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:24 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1308 bytes in 641 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:26.209721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b6059f3-15f4-4c05-af49-b2f4c0030436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:26.211875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3b6059f3-15f4-4c05-af49-b2f4c0030436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:26.230746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b6059f3-15f4-4c05-af49-b2f4c0030436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:26.232379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b6059f3-15f4-4c05-af49-b2f4c0030436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:26.235250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b6059f3-15f4-4c05-af49-b2f4c0030436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:26.236053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b6059f3-15f4-4c05-af49-b2f4c0030436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:26.236710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b6059f3-15f4-4c05-af49-b2f4c0030436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:26.237283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b6059f3-15f4-4c05-af49-b2f4c0030436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:26.242956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:26.858977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3b6059f3-15f4-4c05-af49-b2f4c0030436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:26.883508 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b6059f3-15f4-4c05-af49-b2f4c0030436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1308 microversion: 2.1 time: 0.678088 Sep 27 10:16:26.884156 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2009/4429] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:26 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1308 bytes in 680 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:27.632010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25a3cc74-702f-43ec-8df2-5d3321168d65 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Action: 'create', calling method: .version_select at 0x7f7f628feaf0>, body: {"keypair": {"name": "tempest-keypair-1072571665", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL98/OjjWusuU1tl1le4wGZV+CndKfqu+Rv9FJN8sCWos+jOAAGXMQco62vb2AIhvBWvsdydLd76P5aot7BCQLKWXWZ2ZIVTxSUZj6WEzJ826MPxmSycdSZQulGmOBSmPg=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:27.732926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-25a3cc74-702f-43ec-8df2-5d3321168d65 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Getting quotas for project e2e00c90216b4d1b80ac7892ee23550f. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:16:27.761854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-25a3cc74-702f-43ec-8df2-5d3321168d65 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Getting quotas for user 279d3662f60c4a979c22ee3e8fa92865 and project e2e00c90216b4d1b80ac7892ee23550f. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:16:27.847063 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-25a3cc74-702f-43ec-8df2-5d3321168d65 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.220602 Sep 27 10:16:27.854022 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2421/4430] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:16:27 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:16:27.914314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-08861b05-6552-4623-8f0f-329f7850b1c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:27.914992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-08861b05-6552-4623-8f0f-329f7850b1c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:27.942091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08861b05-6552-4623-8f0f-329f7850b1c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:27.942592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08861b05-6552-4623-8f0f-329f7850b1c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:27.943039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08861b05-6552-4623-8f0f-329f7850b1c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:27.944232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08861b05-6552-4623-8f0f-329f7850b1c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:27.957908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08861b05-6552-4623-8f0f-329f7850b1c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:27.958617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-08861b05-6552-4623-8f0f-329f7850b1c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:28.759731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-33266006-43c7-45e5-8c5f-78097f5ee0ef req-ff6e078d-55da-4dac-8c40-5fb8d693d5dd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f13eb59c-b2be-4b37-addc-676d74eff6ed", "name": "network-vif-plugged", "status": "completed", "tag": "2a400220-e798-40d4-8e16-33e898bba375"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:28.800120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-33266006-43c7-45e5-8c5f-78097f5ee0ef req-ff6e078d-55da-4dac-8c40-5fb8d693d5dd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:28.801179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-33266006-43c7-45e5-8c5f-78097f5ee0ef req-ff6e078d-55da-4dac-8c40-5fb8d693d5dd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:28.801794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-33266006-43c7-45e5-8c5f-78097f5ee0ef req-ff6e078d-55da-4dac-8c40-5fb8d693d5dd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:28.816848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-08861b05-6552-4623-8f0f-329f7850b1c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:28.833356 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-08861b05-6552-4623-8f0f-329f7850b1c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1308 microversion: 2.1 time: 0.924275 Sep 27 10:16:28.836332 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2010/4431] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:27 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1308 bytes in 928 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:28.854164 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-33266006-43c7-45e5-8c5f-78097f5ee0ef req-ff6e078d-55da-4dac-8c40-5fb8d693d5dd service nova] Creating event network-vif-plugged:2a400220-e798-40d4-8e16-33e898bba375 for instance f13eb59c-b2be-4b37-addc-676d74eff6ed on nch1-master-1 Sep 27 10:16:28.867997 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-33266006-43c7-45e5-8c5f-78097f5ee0ef req-ff6e078d-55da-4dac-8c40-5fb8d693d5dd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.131566 Sep 27 10:16:28.870629 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2422/4432] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:16:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:16:29.862650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-06053a64-d4ef-4a66-87fa-020bc8add017 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:29.863285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-06053a64-d4ef-4a66-87fa-020bc8add017 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:29.883801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06053a64-d4ef-4a66-87fa-020bc8add017 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:29.884327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06053a64-d4ef-4a66-87fa-020bc8add017 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:29.884833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06053a64-d4ef-4a66-87fa-020bc8add017 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:29.885569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06053a64-d4ef-4a66-87fa-020bc8add017 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:29.886093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06053a64-d4ef-4a66-87fa-020bc8add017 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:29.886653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06053a64-d4ef-4a66-87fa-020bc8add017 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:30.800492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-06053a64-d4ef-4a66-87fa-020bc8add017 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:30.816835 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-06053a64-d4ef-4a66-87fa-020bc8add017 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1308 microversion: 2.1 time: 0.961039 Sep 27 10:16:30.817444 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2011/4433] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:29 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1308 bytes in 962 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:31.849028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae82b96d-7fc5-4ce5-9426-37f27fb2d2e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:31.850097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ae82b96d-7fc5-4ce5-9426-37f27fb2d2e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:31.875237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae82b96d-7fc5-4ce5-9426-37f27fb2d2e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:31.876878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae82b96d-7fc5-4ce5-9426-37f27fb2d2e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:31.880249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae82b96d-7fc5-4ce5-9426-37f27fb2d2e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:31.882041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae82b96d-7fc5-4ce5-9426-37f27fb2d2e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:31.884496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae82b96d-7fc5-4ce5-9426-37f27fb2d2e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:31.885160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae82b96d-7fc5-4ce5-9426-37f27fb2d2e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:32.472646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ae82b96d-7fc5-4ce5-9426-37f27fb2d2e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:32.482737 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ae82b96d-7fc5-4ce5-9426-37f27fb2d2e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1489 microversion: 2.1 time: 0.642545 Sep 27 10:16:32.483738 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2423/4434] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:31 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1489 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:32.736940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-677102652", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "662e23c5-f7e1-4e14-af6c-49d672076632"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:33.307271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] validate_networks() for [('662e23c5-f7e1-4e14-af6c-49d672076632', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:16:35.589686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-915180871", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "dc711087-d8a4-45fc-8efb-f4217b6d9ff1"}], "security_groups": [{"name": "tempest-securitygroup--1094711544"}], "key_name": "tempest-keypair-1072571665", "user_data": "}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:35.986602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:16:35.987103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:16:35.987650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:16:35.988295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:16:35.988726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:16:36.009634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:36.018080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:36.019205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:36.021013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:36.031324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:36.031964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:36.137280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:16:36.165310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for user a7b8599d0b40424087e52d8e35b71ec4 and project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:16:36.212507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:16:36.214979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:16:36.321480 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-991ca0d3-27cc-429f-a51b-915a4f954443 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.588408 Sep 27 10:16:36.323362 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2012/4435] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:16:32 2022] POST /compute/v2.1/servers => generated 380 bytes in 3591 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:16:36.336965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ca10116-89c4-4e28-9308-472aa4b61051 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:36.337874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ca10116-89c4-4e28-9308-472aa4b61051 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:36.533451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] validate_networks() for [('dc711087-d8a4-45fc-8efb-f4217b6d9ff1', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:16:36.599479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2ca10116-89c4-4e28-9308-472aa4b61051 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:36.613960 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ca10116-89c4-4e28-9308-472aa4b61051 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1208 microversion: 2.1 time: 0.282951 Sep 27 10:16:36.636400 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2013/4436] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:36 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1208 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:37.786844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:37.792146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-1787003544", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "1dfd4252-9b03-463e-9f71-8ecdabb54d49"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:37.913651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] validate_networks() for [('1dfd4252-9b03-463e-9f71-8ecdabb54d49', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:16:38.868584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:16:38.870545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:16:38.871194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:16:38.871765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:16:38.872491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:16:38.900895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:38.903799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:38.909483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:38.914064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:38.917695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:38.918308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:38.949181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Getting quotas for project 8070262519034021a2b6de1a1e1ec88a. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:16:38.956176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Getting quotas for user dbf876e1c23642578363f9c5995f3d86 and project 8070262519034021a2b6de1a1e1ec88a. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:16:38.971987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:16:38.977264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:16:39.111604 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-068a1769-7568-40a4-b8eb-3867a854a24f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.871244 Sep 27 10:16:39.117871 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2014/4437] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:16:37 2022] POST /compute/v2.1/servers => generated 380 bytes in 1878 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:16:39.123045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1d88660e-7df5-49fd-ba21-b7fc5dc38adc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:39.127688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1d88660e-7df5-49fd-ba21-b7fc5dc38adc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:39.141245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d88660e-7df5-49fd-ba21-b7fc5dc38adc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:39.145893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d88660e-7df5-49fd-ba21-b7fc5dc38adc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:39.146908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d88660e-7df5-49fd-ba21-b7fc5dc38adc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:39.148184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d88660e-7df5-49fd-ba21-b7fc5dc38adc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:39.148892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d88660e-7df5-49fd-ba21-b7fc5dc38adc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:39.159974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d88660e-7df5-49fd-ba21-b7fc5dc38adc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:39.242110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:16:39.242569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:16:39.243021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:16:39.243436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:16:39.243857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:16:39.261498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:39.262356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:39.267171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:39.268507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:39.269467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:39.270060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:39.292534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=79,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:39.337382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Getting quotas for project e2e00c90216b4d1b80ac7892ee23550f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:16:39.339533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1d88660e-7df5-49fd-ba21-b7fc5dc38adc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:39.348579 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1d88660e-7df5-49fd-ba21-b7fc5dc38adc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1266 microversion: 2.1 time: 0.229318 Sep 27 10:16:39.355036 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2015/4438] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:39 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1266 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:39.362543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Getting quotas for user 279d3662f60c4a979c22ee3e8fa92865 and project e2e00c90216b4d1b80ac7892ee23550f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:16:39.370275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73f7f713-a9e7-4aad-83eb-2b065a887a71 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:39.389008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-73f7f713-a9e7-4aad-83eb-2b065a887a71 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:39.404951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:16:39.408262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:16:39.575968 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a9ab12ff-6de9-408b-84ba-bb4a37ff3050 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 3.981279 Sep 27 10:16:39.581239 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2424/4439] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:16:35 2022] POST /compute/v2.1/servers => generated 402 bytes in 3997 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:16:39.596904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0be61596-8913-43c8-80ad-69c8ecd37970 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:39.598782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0be61596-8913-43c8-80ad-69c8ecd37970 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:39.783468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-73f7f713-a9e7-4aad-83eb-2b065a887a71 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:39.799675 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-73f7f713-a9e7-4aad-83eb-2b065a887a71 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1218 microversion: 2.1 time: 0.442123 Sep 27 10:16:39.807713 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2016/4440] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:39 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1218 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:40.059340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0be61596-8913-43c8-80ad-69c8ecd37970 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:40.076630 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0be61596-8913-43c8-80ad-69c8ecd37970 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1237 microversion: 2.1 time: 0.488470 Sep 27 10:16:40.089119 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2425/4441] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:39 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1237 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:40.384213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4f54f06d-e1a2-4a94-835f-df16f69c5be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:40.384861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4f54f06d-e1a2-4a94-835f-df16f69c5be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:40.413490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f54f06d-e1a2-4a94-835f-df16f69c5be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:40.414742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f54f06d-e1a2-4a94-835f-df16f69c5be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:40.417179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f54f06d-e1a2-4a94-835f-df16f69c5be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:40.419880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f54f06d-e1a2-4a94-835f-df16f69c5be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:40.420625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f54f06d-e1a2-4a94-835f-df16f69c5be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:40.422571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f54f06d-e1a2-4a94-835f-df16f69c5be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:40.779327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4f54f06d-e1a2-4a94-835f-df16f69c5be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:40.789449 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4f54f06d-e1a2-4a94-835f-df16f69c5be2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1266 microversion: 2.1 time: 0.410861 Sep 27 10:16:40.790054 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2017/4442] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:40 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1266 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:40.838649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e4aafbd-8a1c-4011-8912-f414b90f7e2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:40.843043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e4aafbd-8a1c-4011-8912-f414b90f7e2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:40.856211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4aafbd-8a1c-4011-8912-f414b90f7e2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:40.856725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4aafbd-8a1c-4011-8912-f414b90f7e2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:40.857170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4aafbd-8a1c-4011-8912-f414b90f7e2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:40.857954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4aafbd-8a1c-4011-8912-f414b90f7e2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:40.858312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4aafbd-8a1c-4011-8912-f414b90f7e2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:40.858734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e4aafbd-8a1c-4011-8912-f414b90f7e2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:41.113900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-202d4caf-f9da-4de9-a509-82a74e5b721e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:41.114742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-202d4caf-f9da-4de9-a509-82a74e5b721e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:41.125402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0e4aafbd-8a1c-4011-8912-f414b90f7e2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:41.127657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-202d4caf-f9da-4de9-a509-82a74e5b721e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:41.128171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-202d4caf-f9da-4de9-a509-82a74e5b721e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:41.128647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-202d4caf-f9da-4de9-a509-82a74e5b721e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:41.128647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-202d4caf-f9da-4de9-a509-82a74e5b721e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:41.128647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-202d4caf-f9da-4de9-a509-82a74e5b721e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:41.128647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-202d4caf-f9da-4de9-a509-82a74e5b721e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:41.139646 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e4aafbd-8a1c-4011-8912-f414b90f7e2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1214 microversion: 2.1 time: 0.308485 Sep 27 10:16:41.144410 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2426/4443] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:40 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1214 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:41.433586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-202d4caf-f9da-4de9-a509-82a74e5b721e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:41.456275 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-202d4caf-f9da-4de9-a509-82a74e5b721e tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1241 microversion: 2.1 time: 0.351207 Sep 27 10:16:41.457513 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2018/4444] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:41 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1241 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:41.832705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b4fe04e-a85f-43f6-87d0-83566093b7a6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:41.837423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1b4fe04e-a85f-43f6-87d0-83566093b7a6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:41.864444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b4fe04e-a85f-43f6-87d0-83566093b7a6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:41.865190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b4fe04e-a85f-43f6-87d0-83566093b7a6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:41.865560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b4fe04e-a85f-43f6-87d0-83566093b7a6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:41.866320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b4fe04e-a85f-43f6-87d0-83566093b7a6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:41.866781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b4fe04e-a85f-43f6-87d0-83566093b7a6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:41.867210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b4fe04e-a85f-43f6-87d0-83566093b7a6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:42.166399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-82c5f435-bd0f-4947-9254-e6bfeac2f3f7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:42.167405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-82c5f435-bd0f-4947-9254-e6bfeac2f3f7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:42.184120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82c5f435-bd0f-4947-9254-e6bfeac2f3f7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:42.185356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82c5f435-bd0f-4947-9254-e6bfeac2f3f7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:42.187834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82c5f435-bd0f-4947-9254-e6bfeac2f3f7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:42.191035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82c5f435-bd0f-4947-9254-e6bfeac2f3f7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:42.191922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82c5f435-bd0f-4947-9254-e6bfeac2f3f7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:42.192587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82c5f435-bd0f-4947-9254-e6bfeac2f3f7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:42.201117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1b4fe04e-a85f-43f6-87d0-83566093b7a6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:42.221073 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1b4fe04e-a85f-43f6-87d0-83566093b7a6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1266 microversion: 2.1 time: 0.406633 Sep 27 10:16:42.228844 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2427/4445] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:41 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1266 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:42.481788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9238e546-ff3a-492a-9bc3-a616c9c79386 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:42.482836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9238e546-ff3a-492a-9bc3-a616c9c79386 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:42.495671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238e546-ff3a-492a-9bc3-a616c9c79386 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:42.498532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238e546-ff3a-492a-9bc3-a616c9c79386 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:42.500405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238e546-ff3a-492a-9bc3-a616c9c79386 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:42.502383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238e546-ff3a-492a-9bc3-a616c9c79386 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:42.503045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238e546-ff3a-492a-9bc3-a616c9c79386 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:42.506034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238e546-ff3a-492a-9bc3-a616c9c79386 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:42.730652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9238e546-ff3a-492a-9bc3-a616c9c79386 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:42.753332 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9238e546-ff3a-492a-9bc3-a616c9c79386 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1297 microversion: 2.1 time: 0.271328 Sep 27 10:16:42.754419 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2428/4446] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:42 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1297 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:42.899940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-82c5f435-bd0f-4947-9254-e6bfeac2f3f7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:42.934755 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-82c5f435-bd0f-4947-9254-e6bfeac2f3f7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1276 microversion: 2.1 time: 0.769314 Sep 27 10:16:42.936958 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2019/4447] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:42 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1276 bytes in 778 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:43.248542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86e2d48f-e4f8-4dc5-ab92-f4673841160d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:43.249956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86e2d48f-e4f8-4dc5-ab92-f4673841160d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:43.267899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e2d48f-e4f8-4dc5-ab92-f4673841160d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:43.268886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e2d48f-e4f8-4dc5-ab92-f4673841160d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:43.269554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e2d48f-e4f8-4dc5-ab92-f4673841160d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:43.270516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e2d48f-e4f8-4dc5-ab92-f4673841160d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:43.271271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e2d48f-e4f8-4dc5-ab92-f4673841160d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:43.272050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86e2d48f-e4f8-4dc5-ab92-f4673841160d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:43.800570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-67e28535-9ed6-4bbc-b5ba-7156c4832f3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:43.802750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-67e28535-9ed6-4bbc-b5ba-7156c4832f3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:43.823129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e28535-9ed6-4bbc-b5ba-7156c4832f3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:43.828531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e28535-9ed6-4bbc-b5ba-7156c4832f3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:43.829497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e28535-9ed6-4bbc-b5ba-7156c4832f3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:43.830719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e28535-9ed6-4bbc-b5ba-7156c4832f3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:43.832551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e28535-9ed6-4bbc-b5ba-7156c4832f3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:43.836452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e28535-9ed6-4bbc-b5ba-7156c4832f3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:44.113419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-67e28535-9ed6-4bbc-b5ba-7156c4832f3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:44.130103 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-67e28535-9ed6-4bbc-b5ba-7156c4832f3f tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1295 microversion: 2.1 time: 0.339910 Sep 27 10:16:44.135416 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2020/4448] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:43 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1295 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:44.143016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d4713a5e-a943-4483-b750-3f605e959c61 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:44.147722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d4713a5e-a943-4483-b750-3f605e959c61 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:44.163722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4713a5e-a943-4483-b750-3f605e959c61 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:44.164673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4713a5e-a943-4483-b750-3f605e959c61 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:44.165113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4713a5e-a943-4483-b750-3f605e959c61 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:44.165873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4713a5e-a943-4483-b750-3f605e959c61 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:44.166343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4713a5e-a943-4483-b750-3f605e959c61 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:44.166756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4713a5e-a943-4483-b750-3f605e959c61 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:44.494030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d4713a5e-a943-4483-b750-3f605e959c61 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:44.502261 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d4713a5e-a943-4483-b750-3f605e959c61 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1276 microversion: 2.1 time: 0.365352 Sep 27 10:16:44.503106 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2021/4449] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:44 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1276 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:44.725397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-86e2d48f-e4f8-4dc5-ab92-f4673841160d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:44.745765 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86e2d48f-e4f8-4dc5-ab92-f4673841160d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1308 microversion: 2.1 time: 1.500906 Sep 27 10:16:44.750038 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2429/4450] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:43 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1308 bytes in 1505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:45.155674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f3472581-f20f-4aba-b009-858ee42ed1ca tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:45.157059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f3472581-f20f-4aba-b009-858ee42ed1ca tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:45.183284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3472581-f20f-4aba-b009-858ee42ed1ca tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:45.186271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3472581-f20f-4aba-b009-858ee42ed1ca tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:45.186745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3472581-f20f-4aba-b009-858ee42ed1ca tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:45.190560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3472581-f20f-4aba-b009-858ee42ed1ca tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:45.192719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3472581-f20f-4aba-b009-858ee42ed1ca tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:45.199913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3472581-f20f-4aba-b009-858ee42ed1ca tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:45.529824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-94c64fc3-1738-4870-8d4d-1b017b7948cf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:45.530590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-94c64fc3-1738-4870-8d4d-1b017b7948cf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:45.553042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94c64fc3-1738-4870-8d4d-1b017b7948cf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:45.554121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94c64fc3-1738-4870-8d4d-1b017b7948cf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:45.554825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94c64fc3-1738-4870-8d4d-1b017b7948cf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:45.555719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94c64fc3-1738-4870-8d4d-1b017b7948cf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:45.556398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94c64fc3-1738-4870-8d4d-1b017b7948cf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:45.557045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94c64fc3-1738-4870-8d4d-1b017b7948cf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:45.598260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f3472581-f20f-4aba-b009-858ee42ed1ca tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:45.607751 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f3472581-f20f-4aba-b009-858ee42ed1ca tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1295 microversion: 2.1 time: 0.456775 Sep 27 10:16:45.611522 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2022/4451] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:45 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1295 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:45.773945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-61d5346b-6f6c-4988-91d2-a6cc0bcf635a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:45.774804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-61d5346b-6f6c-4988-91d2-a6cc0bcf635a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:45.783505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=68,nova_api:INSERT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:45.785516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d5346b-6f6c-4988-91d2-a6cc0bcf635a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:45.786091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d5346b-6f6c-4988-91d2-a6cc0bcf635a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:45.790424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d5346b-6f6c-4988-91d2-a6cc0bcf635a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:45.791207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d5346b-6f6c-4988-91d2-a6cc0bcf635a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:45.791840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d5346b-6f6c-4988-91d2-a6cc0bcf635a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:45.792286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61d5346b-6f6c-4988-91d2-a6cc0bcf635a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:46.174662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-94c64fc3-1738-4870-8d4d-1b017b7948cf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:46.183587 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-94c64fc3-1738-4870-8d4d-1b017b7948cf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1318 microversion: 2.1 time: 0.658578 Sep 27 10:16:46.193759 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2430/4452] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:45 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1318 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:46.634554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99142153-9a5f-4059-a3e2-0e944fac9fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:46.635457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-99142153-9a5f-4059-a3e2-0e944fac9fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:46.642307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=62,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:46.643942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99142153-9a5f-4059-a3e2-0e944fac9fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:46.644578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99142153-9a5f-4059-a3e2-0e944fac9fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:46.645197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99142153-9a5f-4059-a3e2-0e944fac9fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:46.660377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99142153-9a5f-4059-a3e2-0e944fac9fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:46.661470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99142153-9a5f-4059-a3e2-0e944fac9fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:46.663343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99142153-9a5f-4059-a3e2-0e944fac9fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:46.832223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-99142153-9a5f-4059-a3e2-0e944fac9fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:46.843489 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-99142153-9a5f-4059-a3e2-0e944fac9fd5 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1295 microversion: 2.1 time: 0.213502 Sep 27 10:16:46.844081 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2431/4453] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:46 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1295 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:47.079732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-61d5346b-6f6c-4988-91d2-a6cc0bcf635a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:47.093987 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-61d5346b-6f6c-4988-91d2-a6cc0bcf635a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1308 microversion: 2.1 time: 1.319129 Sep 27 10:16:47.094819 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2023/4454] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:45 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1308 bytes in 1326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:47.220572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-22240d4a-31a7-4281-97c1-e629d3b29bf5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:47.221267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-22240d4a-31a7-4281-97c1-e629d3b29bf5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:47.231840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22240d4a-31a7-4281-97c1-e629d3b29bf5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:47.232347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22240d4a-31a7-4281-97c1-e629d3b29bf5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:47.232815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22240d4a-31a7-4281-97c1-e629d3b29bf5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:47.236955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22240d4a-31a7-4281-97c1-e629d3b29bf5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:47.237457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22240d4a-31a7-4281-97c1-e629d3b29bf5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:47.241564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22240d4a-31a7-4281-97c1-e629d3b29bf5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:47.295827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2d7e3868-be02-44a9-a0d5-caae4e17aa6b req-75145abe-5c9b-4947-ba62-1b5274bce7a8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e5de27fc-1301-4e5a-ad28-2cd333ab9d6e", "tag": "3634b03e-3d6f-45bc-8360-05f7f1e1a4fb"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:47.319851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2d7e3868-be02-44a9-a0d5-caae4e17aa6b req-75145abe-5c9b-4947-ba62-1b5274bce7a8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:47.320341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2d7e3868-be02-44a9-a0d5-caae4e17aa6b req-75145abe-5c9b-4947-ba62-1b5274bce7a8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:47.320791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2d7e3868-be02-44a9-a0d5-caae4e17aa6b req-75145abe-5c9b-4947-ba62-1b5274bce7a8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:47.356993 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2d7e3868-be02-44a9-a0d5-caae4e17aa6b req-75145abe-5c9b-4947-ba62-1b5274bce7a8 service nova] Creating event network-changed:3634b03e-3d6f-45bc-8360-05f7f1e1a4fb for instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e on nch1-master-1 Sep 27 10:16:47.370776 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2d7e3868-be02-44a9-a0d5-caae4e17aa6b req-75145abe-5c9b-4947-ba62-1b5274bce7a8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.079033 Sep 27 10:16:47.373780 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2024/4455] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:16:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:16:47.873462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1661ed11-bd25-4332-a12d-6106a04953c6 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:47.873987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1661ed11-bd25-4332-a12d-6106a04953c6 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:47.885865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1661ed11-bd25-4332-a12d-6106a04953c6 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:47.893455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1661ed11-bd25-4332-a12d-6106a04953c6 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:47.893916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1661ed11-bd25-4332-a12d-6106a04953c6 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:47.894749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1661ed11-bd25-4332-a12d-6106a04953c6 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:47.895280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1661ed11-bd25-4332-a12d-6106a04953c6 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:47.895713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1661ed11-bd25-4332-a12d-6106a04953c6 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:48.045206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-22240d4a-31a7-4281-97c1-e629d3b29bf5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:48.057927 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-22240d4a-31a7-4281-97c1-e629d3b29bf5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1318 microversion: 2.1 time: 0.843156 Sep 27 10:16:48.060994 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2432/4456] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:47 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1318 bytes in 847 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:48.123712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-20a75a42-31fc-4f41-b165-024e291de3b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:48.125111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-20a75a42-31fc-4f41-b165-024e291de3b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:48.143428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20a75a42-31fc-4f41-b165-024e291de3b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:48.145414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20a75a42-31fc-4f41-b165-024e291de3b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:48.147118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20a75a42-31fc-4f41-b165-024e291de3b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:48.148585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20a75a42-31fc-4f41-b165-024e291de3b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:48.149614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20a75a42-31fc-4f41-b165-024e291de3b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:48.150764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20a75a42-31fc-4f41-b165-024e291de3b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:48.186769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1661ed11-bd25-4332-a12d-6106a04953c6 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:48.196272 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1661ed11-bd25-4332-a12d-6106a04953c6 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1295 microversion: 2.1 time: 0.327253 Sep 27 10:16:48.209894 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2025/4457] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:47 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1295 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:48.460515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-4a49412b-d6b8-442f-8a68-e44271cacaf9 req-01479dd7-0ba6-49d6-84b7-de3fb33633e4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5a7e608b-b82d-43c7-a528-d4b5de120a71", "tag": "0aea9992-4077-42ac-b40b-1f0fab059b1d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:48.491605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4a49412b-d6b8-442f-8a68-e44271cacaf9 req-01479dd7-0ba6-49d6-84b7-de3fb33633e4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:48.495712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4a49412b-d6b8-442f-8a68-e44271cacaf9 req-01479dd7-0ba6-49d6-84b7-de3fb33633e4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:48.498951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4a49412b-d6b8-442f-8a68-e44271cacaf9 req-01479dd7-0ba6-49d6-84b7-de3fb33633e4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:48.529155 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-4a49412b-d6b8-442f-8a68-e44271cacaf9 req-01479dd7-0ba6-49d6-84b7-de3fb33633e4 service nova] Creating event network-changed:0aea9992-4077-42ac-b40b-1f0fab059b1d for instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 on nch2-master-1 Sep 27 10:16:48.554238 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-4a49412b-d6b8-442f-8a68-e44271cacaf9 req-01479dd7-0ba6-49d6-84b7-de3fb33633e4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.097698 Sep 27 10:16:48.558406 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2026/4458] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:16:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:16:48.561954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-20a75a42-31fc-4f41-b165-024e291de3b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:48.570381 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-20a75a42-31fc-4f41-b165-024e291de3b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1308 microversion: 2.1 time: 0.451897 Sep 27 10:16:48.573118 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2433/4459] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:48 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1308 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:49.083467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:49.089594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-34c03879-2d0d-4312-b0de-cd758514e850 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:49.090618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-34c03879-2d0d-4312-b0de-cd758514e850 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:49.098774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c03879-2d0d-4312-b0de-cd758514e850 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:49.099648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c03879-2d0d-4312-b0de-cd758514e850 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:49.100455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c03879-2d0d-4312-b0de-cd758514e850 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:49.101462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c03879-2d0d-4312-b0de-cd758514e850 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:49.103076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c03879-2d0d-4312-b0de-cd758514e850 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:49.103803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c03879-2d0d-4312-b0de-cd758514e850 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:49.233338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-19a142d1-4169-49cc-b548-eabdae30cd5b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:49.233966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-19a142d1-4169-49cc-b548-eabdae30cd5b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:49.247745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19a142d1-4169-49cc-b548-eabdae30cd5b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:49.248855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19a142d1-4169-49cc-b548-eabdae30cd5b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:49.249486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19a142d1-4169-49cc-b548-eabdae30cd5b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:49.250437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19a142d1-4169-49cc-b548-eabdae30cd5b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:49.250911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19a142d1-4169-49cc-b548-eabdae30cd5b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:49.251350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19a142d1-4169-49cc-b548-eabdae30cd5b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:49.296572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:49.713109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-19a142d1-4169-49cc-b548-eabdae30cd5b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:49.723310 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-19a142d1-4169-49cc-b548-eabdae30cd5b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1363 microversion: 2.1 time: 0.493939 Sep 27 10:16:49.725120 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2434/4460] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:49 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1363 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:49.732055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cef31c81-c93b-4ab0-9ab5-aa0af4c49ad6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:49.733048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cef31c81-c93b-4ab0-9ab5-aa0af4c49ad6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:49.741602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cef31c81-c93b-4ab0-9ab5-aa0af4c49ad6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:49.742900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cef31c81-c93b-4ab0-9ab5-aa0af4c49ad6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:49.743558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cef31c81-c93b-4ab0-9ab5-aa0af4c49ad6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:49.744925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cef31c81-c93b-4ab0-9ab5-aa0af4c49ad6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:49.746392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cef31c81-c93b-4ab0-9ab5-aa0af4c49ad6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:49.747047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cef31c81-c93b-4ab0-9ab5-aa0af4c49ad6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:49.780871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-34c03879-2d0d-4312-b0de-cd758514e850 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:49.792991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:49.799898 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-34c03879-2d0d-4312-b0de-cd758514e850 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1318 microversion: 2.1 time: 0.717658 Sep 27 10:16:49.801119 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2027/4461] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:49 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1318 bytes in 720 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:50.070023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:50.196898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cef31c81-c93b-4ab0-9ab5-aa0af4c49ad6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:50.207655 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cef31c81-c93b-4ab0-9ab5-aa0af4c49ad6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1308 microversion: 2.1 time: 0.480865 Sep 27 10:16:50.209374 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2435/4462] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:49 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1308 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:50.752873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4121e402-2d06-44cd-acf7-3f2f69053680 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:50.753652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4121e402-2d06-44cd-acf7-3f2f69053680 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:50.770154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4121e402-2d06-44cd-acf7-3f2f69053680 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:50.773182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4121e402-2d06-44cd-acf7-3f2f69053680 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:50.776079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4121e402-2d06-44cd-acf7-3f2f69053680 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:50.779363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4121e402-2d06-44cd-acf7-3f2f69053680 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:50.782807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4121e402-2d06-44cd-acf7-3f2f69053680 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:50.788822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4121e402-2d06-44cd-acf7-3f2f69053680 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:51.027960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerPasswordTestJSON-server-73341355", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "d7fe0e74-6749-42c8-a5fa-36cd2bd4c621"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:51.158281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] validate_networks() for [('d7fe0e74-6749-42c8-a5fa-36cd2bd4c621', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:16:51.340191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4121e402-2d06-44cd-acf7-3f2f69053680 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:51.357963 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4121e402-2d06-44cd-acf7-3f2f69053680 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1363 microversion: 2.1 time: 0.607484 Sep 27 10:16:51.373755 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2028/4463] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:50 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1363 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:51.383650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a6095e99-b676-45a1-b95a-a3a4702fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:51.384596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a6095e99-b676-45a1-b95a-a3a4702fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:51.392264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6095e99-b676-45a1-b95a-a3a4702fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:51.393187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6095e99-b676-45a1-b95a-a3a4702fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:51.393901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6095e99-b676-45a1-b95a-a3a4702fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:51.394854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6095e99-b676-45a1-b95a-a3a4702fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:51.395585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6095e99-b676-45a1-b95a-a3a4702fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:51.396558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6095e99-b676-45a1-b95a-a3a4702fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:51.733033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:16:51.733656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:16:51.734293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:16:51.735364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:16:51.735949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:16:51.745411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:51.746064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:51.746658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:51.747730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:51.748343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:51.748948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:51.759314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a6095e99-b676-45a1-b95a-a3a4702fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:51.770398 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a6095e99-b676-45a1-b95a-a3a4702fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1318 microversion: 2.1 time: 0.395350 Sep 27 10:16:51.774740 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2029/4464] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:51 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1318 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:51.777460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Getting quotas for project 898c5ffacd6247c6a8bff965ebaf0591. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:16:51.787376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5f1ca327-da53-452a-b817-57ae13855388 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:51.789289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5f1ca327-da53-452a-b817-57ae13855388 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:51.791285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Getting quotas for user 04440378e21043b2a287e9b4c800080d and project 898c5ffacd6247c6a8bff965ebaf0591. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:16:51.803911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f1ca327-da53-452a-b817-57ae13855388 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:51.806202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f1ca327-da53-452a-b817-57ae13855388 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:51.807536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f1ca327-da53-452a-b817-57ae13855388 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:51.808843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f1ca327-da53-452a-b817-57ae13855388 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:51.810862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f1ca327-da53-452a-b817-57ae13855388 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:51.812534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f1ca327-da53-452a-b817-57ae13855388 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:51.824457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:16:51.836083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:51.839049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:16:51.915652 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d360be8a-6f30-467f-8497-4e154937a2f2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.126309 Sep 27 10:16:51.916884 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2436/4465] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:16:50 2022] POST /compute/v2.1/servers => generated 380 bytes in 1128 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:16:51.930382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-171e9f71-ec13-4f00-bfd2-7ef73b35aa7c tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:51.931586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-171e9f71-ec13-4f00-bfd2-7ef73b35aa7c tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:52.047078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-171e9f71-ec13-4f00-bfd2-7ef73b35aa7c tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:52.058414 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-171e9f71-ec13-4f00-bfd2-7ef73b35aa7c tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1214 microversion: 2.1 time: 0.134234 Sep 27 10:16:52.074216 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2437/4466] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:51 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1214 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:52.303839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ed5b9e03-a62b-4779-8a80-1dd087d60cc9 req-1a61fd20-8a36-4bb0-ae2f-1c0a069a9d6c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "779046b2-2e5b-4946-af25-c111c590a14e", "tag": "37f44fa9-2c33-45ba-a074-f37e0af9c6dc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:52.321004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ed5b9e03-a62b-4779-8a80-1dd087d60cc9 req-1a61fd20-8a36-4bb0-ae2f-1c0a069a9d6c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:52.321394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ed5b9e03-a62b-4779-8a80-1dd087d60cc9 req-1a61fd20-8a36-4bb0-ae2f-1c0a069a9d6c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:52.321830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ed5b9e03-a62b-4779-8a80-1dd087d60cc9 req-1a61fd20-8a36-4bb0-ae2f-1c0a069a9d6c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:52.358975 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ed5b9e03-a62b-4779-8a80-1dd087d60cc9 req-1a61fd20-8a36-4bb0-ae2f-1c0a069a9d6c service nova] Creating event network-changed:37f44fa9-2c33-45ba-a074-f37e0af9c6dc for instance 779046b2-2e5b-4946-af25-c111c590a14e on nch1-master-1 Sep 27 10:16:52.375070 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ed5b9e03-a62b-4779-8a80-1dd087d60cc9 req-1a61fd20-8a36-4bb0-ae2f-1c0a069a9d6c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070251 Sep 27 10:16:52.375997 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2438/4467] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:16:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:16:52.408464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1f44dccb-6edb-49e0-a43e-92fd3658ca86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:52.415413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1f44dccb-6edb-49e0-a43e-92fd3658ca86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:52.432549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f44dccb-6edb-49e0-a43e-92fd3658ca86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:52.436224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f44dccb-6edb-49e0-a43e-92fd3658ca86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:52.436901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f44dccb-6edb-49e0-a43e-92fd3658ca86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:52.438582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f44dccb-6edb-49e0-a43e-92fd3658ca86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:52.444556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f44dccb-6edb-49e0-a43e-92fd3658ca86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:52.445239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f44dccb-6edb-49e0-a43e-92fd3658ca86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:52.468287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5f1ca327-da53-452a-b817-57ae13855388 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:52.486458 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5f1ca327-da53-452a-b817-57ae13855388 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1308 microversion: 2.1 time: 0.710248 Sep 27 10:16:52.487451 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2030/4468] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:51 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1308 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:52.797907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d31eb86f-5bce-4145-8e1f-639fd7702cf3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:52.799676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d31eb86f-5bce-4145-8e1f-639fd7702cf3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:52.811905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d31eb86f-5bce-4145-8e1f-639fd7702cf3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:52.813835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d31eb86f-5bce-4145-8e1f-639fd7702cf3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:52.815414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d31eb86f-5bce-4145-8e1f-639fd7702cf3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:52.817115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d31eb86f-5bce-4145-8e1f-639fd7702cf3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:52.819506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d31eb86f-5bce-4145-8e1f-639fd7702cf3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:52.821245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d31eb86f-5bce-4145-8e1f-639fd7702cf3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:52.858749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1f44dccb-6edb-49e0-a43e-92fd3658ca86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:52.879165 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1f44dccb-6edb-49e0-a43e-92fd3658ca86 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1363 microversion: 2.1 time: 0.487229 Sep 27 10:16:52.894562 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2439/4469] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:52 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1363 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:53.084213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c2a8080e-bc7a-4175-a93f-1256501015fe tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:53.084894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c2a8080e-bc7a-4175-a93f-1256501015fe tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:53.096999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2a8080e-bc7a-4175-a93f-1256501015fe tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:53.103591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2a8080e-bc7a-4175-a93f-1256501015fe tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:53.106146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2a8080e-bc7a-4175-a93f-1256501015fe tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:53.110755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2a8080e-bc7a-4175-a93f-1256501015fe tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:53.111541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2a8080e-bc7a-4175-a93f-1256501015fe tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:53.114335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c2a8080e-bc7a-4175-a93f-1256501015fe tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:53.190774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d31eb86f-5bce-4145-8e1f-639fd7702cf3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:53.208502 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d31eb86f-5bce-4145-8e1f-639fd7702cf3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1318 microversion: 2.1 time: 0.415709 Sep 27 10:16:53.213574 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2031/4470] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:52 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1318 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:53.396810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c2a8080e-bc7a-4175-a93f-1256501015fe tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:53.411647 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c2a8080e-bc7a-4175-a93f-1256501015fe tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1218 microversion: 2.1 time: 0.331865 Sep 27 10:16:53.427245 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2440/4471] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:53 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1218 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:53.513108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-13b1e953-d479-4261-bb81-74c315ee5ee4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:53.514145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-13b1e953-d479-4261-bb81-74c315ee5ee4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:53.529738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13b1e953-d479-4261-bb81-74c315ee5ee4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:53.532756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13b1e953-d479-4261-bb81-74c315ee5ee4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:53.539637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13b1e953-d479-4261-bb81-74c315ee5ee4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:53.542487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13b1e953-d479-4261-bb81-74c315ee5ee4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:53.543253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13b1e953-d479-4261-bb81-74c315ee5ee4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:53.549463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13b1e953-d479-4261-bb81-74c315ee5ee4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:53.904164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33b59b9c-9f91-4433-bc87-60a7af6bfa1b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:53.904926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33b59b9c-9f91-4433-bc87-60a7af6bfa1b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:53.917763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33b59b9c-9f91-4433-bc87-60a7af6bfa1b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:53.918264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33b59b9c-9f91-4433-bc87-60a7af6bfa1b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:53.918718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33b59b9c-9f91-4433-bc87-60a7af6bfa1b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:53.919456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33b59b9c-9f91-4433-bc87-60a7af6bfa1b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:53.919922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33b59b9c-9f91-4433-bc87-60a7af6bfa1b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:53.920358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33b59b9c-9f91-4433-bc87-60a7af6bfa1b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:54.214685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33b59b9c-9f91-4433-bc87-60a7af6bfa1b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:54.227298 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33b59b9c-9f91-4433-bc87-60a7af6bfa1b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1363 microversion: 2.1 time: 0.327310 Sep 27 10:16:54.231071 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2441/4472] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:53 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1363 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:54.245454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-13b1e953-d479-4261-bb81-74c315ee5ee4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:54.256247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d9fbdc0-4e0b-4142-ac51-104148618892 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:54.262041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4d9fbdc0-4e0b-4142-ac51-104148618892 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:54.264850 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-13b1e953-d479-4261-bb81-74c315ee5ee4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1308 microversion: 2.1 time: 0.756608 Sep 27 10:16:54.265456 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2032/4473] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:53 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1308 bytes in 758 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:54.274829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d9fbdc0-4e0b-4142-ac51-104148618892 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:54.275326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d9fbdc0-4e0b-4142-ac51-104148618892 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:54.275770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d9fbdc0-4e0b-4142-ac51-104148618892 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:54.276490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d9fbdc0-4e0b-4142-ac51-104148618892 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:54.279779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d9fbdc0-4e0b-4142-ac51-104148618892 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:54.279779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d9fbdc0-4e0b-4142-ac51-104148618892 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:54.439007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7ed336f4-b2fb-4902-ba99-f6a7fe7724d2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:54.440715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7ed336f4-b2fb-4902-ba99-f6a7fe7724d2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:54.462118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ed336f4-b2fb-4902-ba99-f6a7fe7724d2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:54.463038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ed336f4-b2fb-4902-ba99-f6a7fe7724d2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:54.463787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ed336f4-b2fb-4902-ba99-f6a7fe7724d2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:54.465365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ed336f4-b2fb-4902-ba99-f6a7fe7724d2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:54.466094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ed336f4-b2fb-4902-ba99-f6a7fe7724d2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:54.466986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ed336f4-b2fb-4902-ba99-f6a7fe7724d2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:54.689965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4d9fbdc0-4e0b-4142-ac51-104148618892 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:54.722814 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4d9fbdc0-4e0b-4142-ac51-104148618892 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1318 microversion: 2.1 time: 0.473030 Sep 27 10:16:54.723418 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2442/4474] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:54 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1318 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:54.745986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0e21e668-0a81-4418-b282-9b10bc2aa666 req-bf41a583-2119-4aa8-8ff8-ac5fa229b302 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e5de27fc-1301-4e5a-ad28-2cd333ab9d6e", "name": "network-vif-plugged", "status": "completed", "tag": "3634b03e-3d6f-45bc-8360-05f7f1e1a4fb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:54.760678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0e21e668-0a81-4418-b282-9b10bc2aa666 req-bf41a583-2119-4aa8-8ff8-ac5fa229b302 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:54.765925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0e21e668-0a81-4418-b282-9b10bc2aa666 req-bf41a583-2119-4aa8-8ff8-ac5fa229b302 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:54.767262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0e21e668-0a81-4418-b282-9b10bc2aa666 req-bf41a583-2119-4aa8-8ff8-ac5fa229b302 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:54.809973 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0e21e668-0a81-4418-b282-9b10bc2aa666 req-bf41a583-2119-4aa8-8ff8-ac5fa229b302 service nova] Creating event network-vif-plugged:3634b03e-3d6f-45bc-8360-05f7f1e1a4fb for instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e on nch1-master-1 Sep 27 10:16:54.823926 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0e21e668-0a81-4418-b282-9b10bc2aa666 req-bf41a583-2119-4aa8-8ff8-ac5fa229b302 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085448 Sep 27 10:16:54.834979 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2443/4475] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:16:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:16:54.948890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7ed336f4-b2fb-4902-ba99-f6a7fe7724d2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:54.962475 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7ed336f4-b2fb-4902-ba99-f6a7fe7724d2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1274 microversion: 2.1 time: 0.530637 Sep 27 10:16:54.964097 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2033/4476] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:54 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1274 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:55.256310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c85505a9-a6bd-4bd8-a637-1532dd151619 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:55.257786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c85505a9-a6bd-4bd8-a637-1532dd151619 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:55.277611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c85505a9-a6bd-4bd8-a637-1532dd151619 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:55.277611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c85505a9-a6bd-4bd8-a637-1532dd151619 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:55.277899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c85505a9-a6bd-4bd8-a637-1532dd151619 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:55.278796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c85505a9-a6bd-4bd8-a637-1532dd151619 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:55.279479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c85505a9-a6bd-4bd8-a637-1532dd151619 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:55.280089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c85505a9-a6bd-4bd8-a637-1532dd151619 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:55.293367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-98926b48-67f5-458e-a42a-a1a7f63806d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:55.294012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-98926b48-67f5-458e-a42a-a1a7f63806d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:55.303289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98926b48-67f5-458e-a42a-a1a7f63806d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:55.304042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98926b48-67f5-458e-a42a-a1a7f63806d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:55.304331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98926b48-67f5-458e-a42a-a1a7f63806d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:55.329849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98926b48-67f5-458e-a42a-a1a7f63806d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:55.330372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98926b48-67f5-458e-a42a-a1a7f63806d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:55.330826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98926b48-67f5-458e-a42a-a1a7f63806d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:55.819271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c85505a9-a6bd-4bd8-a637-1532dd151619 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:55.832647 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c85505a9-a6bd-4bd8-a637-1532dd151619 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1363 microversion: 2.1 time: 0.581612 Sep 27 10:16:55.837199 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2444/4477] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:55 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1363 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:55.845540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e9496b9-575c-446d-8045-2333786dafc3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:55.847062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8e9496b9-575c-446d-8045-2333786dafc3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:55.854591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e9496b9-575c-446d-8045-2333786dafc3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:55.855456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e9496b9-575c-446d-8045-2333786dafc3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:55.856098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e9496b9-575c-446d-8045-2333786dafc3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:55.860268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e9496b9-575c-446d-8045-2333786dafc3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:55.860268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e9496b9-575c-446d-8045-2333786dafc3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:55.860268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e9496b9-575c-446d-8045-2333786dafc3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:55.875374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:16:56.039845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-98926b48-67f5-458e-a42a-a1a7f63806d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:56.050984 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-98926b48-67f5-458e-a42a-a1a7f63806d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1308 microversion: 2.1 time: 0.764030 Sep 27 10:16:56.053743 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2034/4478] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:55 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1308 bytes in 768 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:56.064388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8b3adbc4-57a3-4172-b5f4-778c10dbb788 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:56.067898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8b3adbc4-57a3-4172-b5f4-778c10dbb788 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:56.084115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b3adbc4-57a3-4172-b5f4-778c10dbb788 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:56.085685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b3adbc4-57a3-4172-b5f4-778c10dbb788 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:56.086198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b3adbc4-57a3-4172-b5f4-778c10dbb788 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:56.087266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b3adbc4-57a3-4172-b5f4-778c10dbb788 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:56.088539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b3adbc4-57a3-4172-b5f4-778c10dbb788 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:56.089521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b3adbc4-57a3-4172-b5f4-778c10dbb788 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:56.106718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8e9496b9-575c-446d-8045-2333786dafc3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:56.118910 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e9496b9-575c-446d-8045-2333786dafc3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1318 microversion: 2.1 time: 0.279728 Sep 27 10:16:56.123341 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2445/4479] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:55 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1318 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:56.296295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8b3adbc4-57a3-4172-b5f4-778c10dbb788 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:56.309710 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8b3adbc4-57a3-4172-b5f4-778c10dbb788 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1272 microversion: 2.1 time: 0.252016 Sep 27 10:16:56.312388 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2035/4480] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:56 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1272 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:56.859190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-324932c5-21a1-41d6-a88f-b86bf595ff31 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:56.860760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-324932c5-21a1-41d6-a88f-b86bf595ff31 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:56.871532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-324932c5-21a1-41d6-a88f-b86bf595ff31 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:56.872570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-324932c5-21a1-41d6-a88f-b86bf595ff31 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:56.874252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-324932c5-21a1-41d6-a88f-b86bf595ff31 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:56.877135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-324932c5-21a1-41d6-a88f-b86bf595ff31 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:56.877960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-324932c5-21a1-41d6-a88f-b86bf595ff31 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:56.879071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-324932c5-21a1-41d6-a88f-b86bf595ff31 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:57.067628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-324932c5-21a1-41d6-a88f-b86bf595ff31 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:57.079144 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-324932c5-21a1-41d6-a88f-b86bf595ff31 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1363 microversion: 2.1 time: 0.227378 Sep 27 10:16:57.079749 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2446/4481] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:56 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1363 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:57.086258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0f9c7c81-df18-4d0d-9db1-ab0bea0b181a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:57.087215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0f9c7c81-df18-4d0d-9db1-ab0bea0b181a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:57.119404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f9c7c81-df18-4d0d-9db1-ab0bea0b181a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:57.120309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f9c7c81-df18-4d0d-9db1-ab0bea0b181a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:57.120918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f9c7c81-df18-4d0d-9db1-ab0bea0b181a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:57.121844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f9c7c81-df18-4d0d-9db1-ab0bea0b181a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:57.122410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f9c7c81-df18-4d0d-9db1-ab0bea0b181a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:57.123076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f9c7c81-df18-4d0d-9db1-ab0bea0b181a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:57.153819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1e27dc69-df7e-4f05-9a88-3aa26932f0a5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:57.154597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1e27dc69-df7e-4f05-9a88-3aa26932f0a5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:57.166415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e27dc69-df7e-4f05-9a88-3aa26932f0a5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:57.169314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e27dc69-df7e-4f05-9a88-3aa26932f0a5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:57.169775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e27dc69-df7e-4f05-9a88-3aa26932f0a5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:57.170517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e27dc69-df7e-4f05-9a88-3aa26932f0a5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:57.174145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e27dc69-df7e-4f05-9a88-3aa26932f0a5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:57.175978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e27dc69-df7e-4f05-9a88-3aa26932f0a5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:57.582441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1e27dc69-df7e-4f05-9a88-3aa26932f0a5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:57.593042 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1e27dc69-df7e-4f05-9a88-3aa26932f0a5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1508 microversion: 2.1 time: 0.444246 Sep 27 10:16:57.593857 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2447/4482] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:57 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1508 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:57.613832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e64ebf3d-666e-499f-b42a-00c7713760f8 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:57.614945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e64ebf3d-666e-499f-b42a-00c7713760f8 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:57.638431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64ebf3d-666e-499f-b42a-00c7713760f8 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:57.639263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64ebf3d-666e-499f-b42a-00c7713760f8 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:57.639898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64ebf3d-666e-499f-b42a-00c7713760f8 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:57.642689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64ebf3d-666e-499f-b42a-00c7713760f8 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:57.644497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64ebf3d-666e-499f-b42a-00c7713760f8 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:57.647695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e64ebf3d-666e-499f-b42a-00c7713760f8 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:57.866764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e64ebf3d-666e-499f-b42a-00c7713760f8 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:57.877043 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e64ebf3d-666e-499f-b42a-00c7713760f8 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1314 microversion: 2.1 time: 0.278920 Sep 27 10:16:57.880747 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2448/4483] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:57 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1314 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:57.889001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7b5e3b22-0020-4c3f-b928-e28c6d41f164 req-5c47631d-f9dc-47d7-9e81-f14caea762a2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5a7e608b-b82d-43c7-a528-d4b5de120a71", "name": "network-vif-plugged", "status": "completed", "tag": "0aea9992-4077-42ac-b40b-1f0fab059b1d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:57.903458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7b5e3b22-0020-4c3f-b928-e28c6d41f164 req-5c47631d-f9dc-47d7-9e81-f14caea762a2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:57.904346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7b5e3b22-0020-4c3f-b928-e28c6d41f164 req-5c47631d-f9dc-47d7-9e81-f14caea762a2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:57.905037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7b5e3b22-0020-4c3f-b928-e28c6d41f164 req-5c47631d-f9dc-47d7-9e81-f14caea762a2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:57.919730 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7b5e3b22-0020-4c3f-b928-e28c6d41f164 req-5c47631d-f9dc-47d7-9e81-f14caea762a2 service nova] Creating event network-vif-plugged:0aea9992-4077-42ac-b40b-1f0fab059b1d for instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 on nch2-master-1 Sep 27 10:16:57.926468 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7b5e3b22-0020-4c3f-b928-e28c6d41f164 req-5c47631d-f9dc-47d7-9e81-f14caea762a2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043740 Sep 27 10:16:57.932969 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2449/4484] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:16:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:16:57.943516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-695afa28-e404-4891-8330-465b744e82d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:57.944578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-695afa28-e404-4891-8330-465b744e82d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:57.953799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-695afa28-e404-4891-8330-465b744e82d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:57.954225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-695afa28-e404-4891-8330-465b744e82d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:57.954726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-695afa28-e404-4891-8330-465b744e82d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:57.955794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-695afa28-e404-4891-8330-465b744e82d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:57.956438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-695afa28-e404-4891-8330-465b744e82d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:57.957017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-695afa28-e404-4891-8330-465b744e82d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:58.177456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-695afa28-e404-4891-8330-465b744e82d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:58.188413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0f9c7c81-df18-4d0d-9db1-ab0bea0b181a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:58.190396 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-695afa28-e404-4891-8330-465b744e82d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1508 microversion: 2.1 time: 0.254247 Sep 27 10:16:58.199535 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0f9c7c81-df18-4d0d-9db1-ab0bea0b181a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1308 microversion: 2.1 time: 1.122378 Sep 27 10:16:58.201343 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2036/4485] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:57 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1308 bytes in 1125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:58.206751 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2450/4486] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:57 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1508 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:58.217039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "OS-DCF:diskConfig": "AUTO"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:58.219157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6a823252-4e22-47c4-95a4-3e5c1f5caa21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:58.221392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:58.222513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6a823252-4e22-47c4-95a4-3e5c1f5caa21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:58.230642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:58.231448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:58.232237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:58.233503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:58.234283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:58.234866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:58.237466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a823252-4e22-47c4-95a4-3e5c1f5caa21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:58.240670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a823252-4e22-47c4-95a4-3e5c1f5caa21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:58.247586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a823252-4e22-47c4-95a4-3e5c1f5caa21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:58.250153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a823252-4e22-47c4-95a4-3e5c1f5caa21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:58.250658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a823252-4e22-47c4-95a4-3e5c1f5caa21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:58.254414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a823252-4e22-47c4-95a4-3e5c1f5caa21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:58.372253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:16:58.374711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:16:58.375202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:16:58.378804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:16:58.379460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:16:58.534005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:58.544053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:58.545005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:58.546304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:58.547365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:58.548242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:58.549047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:58.589475 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-15c1b837-72a1-465f-bd83-e507a7768298 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "POST /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e/action" status: 202 len: 1110 microversion: 2.1 time: 0.377608 Sep 27 10:16:58.590972 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2451/4487] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:16:58 2022] POST /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e/action => generated 1110 bytes in 380 msecs (HTTP/1.1 202) 10 headers in 452 bytes (2 switches on core 0) Sep 27 10:16:58.612709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0ab0807-6081-49f4-9ffa-99b2dba1f532 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:58.613626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f0ab0807-6081-49f4-9ffa-99b2dba1f532 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:58.626866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ab0807-6081-49f4-9ffa-99b2dba1f532 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:58.627617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ab0807-6081-49f4-9ffa-99b2dba1f532 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:58.629748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ab0807-6081-49f4-9ffa-99b2dba1f532 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:58.633264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ab0807-6081-49f4-9ffa-99b2dba1f532 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:58.634266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ab0807-6081-49f4-9ffa-99b2dba1f532 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:58.637959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ab0807-6081-49f4-9ffa-99b2dba1f532 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:58.726632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6a823252-4e22-47c4-95a4-3e5c1f5caa21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:58.736195 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6a823252-4e22-47c4-95a4-3e5c1f5caa21 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1363 microversion: 2.1 time: 0.533545 Sep 27 10:16:58.740894 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2037/4488] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:58 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1363 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:58.904930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe549304-fafd-4161-8bc5-7e68be6fd3fd tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:58.907927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fe549304-fafd-4161-8bc5-7e68be6fd3fd tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:58.917521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe549304-fafd-4161-8bc5-7e68be6fd3fd tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:58.922764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe549304-fafd-4161-8bc5-7e68be6fd3fd tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:58.923455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe549304-fafd-4161-8bc5-7e68be6fd3fd tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:58.924469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe549304-fafd-4161-8bc5-7e68be6fd3fd tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:58.925120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe549304-fafd-4161-8bc5-7e68be6fd3fd tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:58.925778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe549304-fafd-4161-8bc5-7e68be6fd3fd tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:59.090971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f0ab0807-6081-49f4-9ffa-99b2dba1f532 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:59.106141 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f0ab0807-6081-49f4-9ffa-99b2dba1f532 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.497324 Sep 27 10:16:59.107502 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2452/4489] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:58 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:59.224951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3476af07-4d29-41dc-8a81-98bd661254ee tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:59.225893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3476af07-4d29-41dc-8a81-98bd661254ee tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:59.234298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3476af07-4d29-41dc-8a81-98bd661254ee tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:59.234936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3476af07-4d29-41dc-8a81-98bd661254ee tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:59.242108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3476af07-4d29-41dc-8a81-98bd661254ee tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:59.242964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3476af07-4d29-41dc-8a81-98bd661254ee tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:59.243466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3476af07-4d29-41dc-8a81-98bd661254ee tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:59.243903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3476af07-4d29-41dc-8a81-98bd661254ee tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:59.248471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fe549304-fafd-4161-8bc5-7e68be6fd3fd tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:59.265306 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fe549304-fafd-4161-8bc5-7e68be6fd3fd tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1314 microversion: 2.1 time: 0.366731 Sep 27 10:16:59.266282 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2038/4490] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:58 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1314 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:16:59.393997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3476af07-4d29-41dc-8a81-98bd661254ee tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:16:59.406253 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3476af07-4d29-41dc-8a81-98bd661254ee tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1308 microversion: 2.1 time: 0.186191 Sep 27 10:16:59.407647 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2453/4491] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:59 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1308 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:16:59.773048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f505031f-bf37-4dc9-b874-3d1cc6a8eee3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:16:59.774712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f505031f-bf37-4dc9-b874-3d1cc6a8eee3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:16:59.799366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505031f-bf37-4dc9-b874-3d1cc6a8eee3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:59.802337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505031f-bf37-4dc9-b874-3d1cc6a8eee3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:59.804199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505031f-bf37-4dc9-b874-3d1cc6a8eee3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:59.811101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505031f-bf37-4dc9-b874-3d1cc6a8eee3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:59.815630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505031f-bf37-4dc9-b874-3d1cc6a8eee3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:59.821530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505031f-bf37-4dc9-b874-3d1cc6a8eee3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:16:59.947319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5ed438a5-915f-478e-b0ca-83ac9c89a6ad req-dabf5c8c-3513-482f-a17f-e59e366d8b3e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "779046b2-2e5b-4946-af25-c111c590a14e", "name": "network-vif-plugged", "status": "completed", "tag": "37f44fa9-2c33-45ba-a074-f37e0af9c6dc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:16:59.984683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5ed438a5-915f-478e-b0ca-83ac9c89a6ad req-dabf5c8c-3513-482f-a17f-e59e366d8b3e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:16:59.994093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5ed438a5-915f-478e-b0ca-83ac9c89a6ad req-dabf5c8c-3513-482f-a17f-e59e366d8b3e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:16:59.994572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5ed438a5-915f-478e-b0ca-83ac9c89a6ad req-dabf5c8c-3513-482f-a17f-e59e366d8b3e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:00.016643 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5ed438a5-915f-478e-b0ca-83ac9c89a6ad req-dabf5c8c-3513-482f-a17f-e59e366d8b3e service nova] Creating event network-vif-plugged:37f44fa9-2c33-45ba-a074-f37e0af9c6dc for instance 779046b2-2e5b-4946-af25-c111c590a14e on nch1-master-1 Sep 27 10:17:00.026263 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5ed438a5-915f-478e-b0ca-83ac9c89a6ad req-dabf5c8c-3513-482f-a17f-e59e366d8b3e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082946 Sep 27 10:17:00.027187 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2454/4492] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:16:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:17:00.135121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5f1d1d07-fda3-4c5c-aec5-427ecdf1760b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:00.135889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5f1d1d07-fda3-4c5c-aec5-427ecdf1760b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:00.151963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f1d1d07-fda3-4c5c-aec5-427ecdf1760b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:00.153403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f1d1d07-fda3-4c5c-aec5-427ecdf1760b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:00.153860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f1d1d07-fda3-4c5c-aec5-427ecdf1760b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:00.155546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f1d1d07-fda3-4c5c-aec5-427ecdf1760b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:00.159600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f1d1d07-fda3-4c5c-aec5-427ecdf1760b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:00.161640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f1d1d07-fda3-4c5c-aec5-427ecdf1760b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:00.407580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f505031f-bf37-4dc9-b874-3d1cc6a8eee3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:00.411536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f1d1d07-fda3-4c5c-aec5-427ecdf1760b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:00.424168 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f505031f-bf37-4dc9-b874-3d1cc6a8eee3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1363 microversion: 2.1 time: 0.658657 Sep 27 10:17:00.427904 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2039/4493] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:16:59 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1363 bytes in 665 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:00.442316 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5f1d1d07-fda3-4c5c-aec5-427ecdf1760b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.299024 Sep 27 10:17:00.443114 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2455/4494] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:00 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:00.446076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-57250efa-2ce0-4899-ba7d-0a4f2f30b8d3 req-d6b3c3fb-d51c-45f5-9f13-3abd998c2d75 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f7d07191-c8dd-4778-85ab-87262a4d21ed", "tag": "068e1ae8-487b-49d1-92b7-bca512800ab7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:00.456350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c573dec1-cffc-4537-a5cc-5fcb04614d65 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:00.457276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c573dec1-cffc-4537-a5cc-5fcb04614d65 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:00.464851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c573dec1-cffc-4537-a5cc-5fcb04614d65 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:00.465623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c573dec1-cffc-4537-a5cc-5fcb04614d65 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:00.466241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c573dec1-cffc-4537-a5cc-5fcb04614d65 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:00.468097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c573dec1-cffc-4537-a5cc-5fcb04614d65 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:00.468752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c573dec1-cffc-4537-a5cc-5fcb04614d65 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:00.469327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c573dec1-cffc-4537-a5cc-5fcb04614d65 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:00.469553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-57250efa-2ce0-4899-ba7d-0a4f2f30b8d3 req-d6b3c3fb-d51c-45f5-9f13-3abd998c2d75 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:00.470053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-57250efa-2ce0-4899-ba7d-0a4f2f30b8d3 req-d6b3c3fb-d51c-45f5-9f13-3abd998c2d75 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:00.470490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-57250efa-2ce0-4899-ba7d-0a4f2f30b8d3 req-d6b3c3fb-d51c-45f5-9f13-3abd998c2d75 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:00.504146 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-57250efa-2ce0-4899-ba7d-0a4f2f30b8d3 req-d6b3c3fb-d51c-45f5-9f13-3abd998c2d75 service nova] Creating event network-changed:068e1ae8-487b-49d1-92b7-bca512800ab7 for instance f7d07191-c8dd-4778-85ab-87262a4d21ed on nch2-master-1 Sep 27 10:17:00.513652 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-57250efa-2ce0-4899-ba7d-0a4f2f30b8d3 req-d6b3c3fb-d51c-45f5-9f13-3abd998c2d75 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083883 Sep 27 10:17:00.514736 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2040/4495] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:00.522134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-34b6a7ba-8f5b-4761-9126-0f72ae2876e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:00.522879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-34b6a7ba-8f5b-4761-9126-0f72ae2876e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:00.531947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b6a7ba-8f5b-4761-9126-0f72ae2876e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:00.532529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b6a7ba-8f5b-4761-9126-0f72ae2876e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:00.533004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b6a7ba-8f5b-4761-9126-0f72ae2876e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:00.533911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b6a7ba-8f5b-4761-9126-0f72ae2876e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:00.534376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b6a7ba-8f5b-4761-9126-0f72ae2876e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:00.535079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34b6a7ba-8f5b-4761-9126-0f72ae2876e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:00.675088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c573dec1-cffc-4537-a5cc-5fcb04614d65 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:00.685394 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c573dec1-cffc-4537-a5cc-5fcb04614d65 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1314 microversion: 2.1 time: 0.241065 Sep 27 10:17:00.686417 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2456/4496] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:00 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1314 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:01.005028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-34b6a7ba-8f5b-4761-9126-0f72ae2876e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:01.013393 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-34b6a7ba-8f5b-4761-9126-0f72ae2876e3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1488 microversion: 2.1 time: 0.497305 Sep 27 10:17:01.013960 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2041/4497] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:00 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1488 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:01.040591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-132f28a1-1690-40ab-94ae-646ad0bfd7f0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:01.041714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-132f28a1-1690-40ab-94ae-646ad0bfd7f0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:01.049151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132f28a1-1690-40ab-94ae-646ad0bfd7f0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:01.049648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132f28a1-1690-40ab-94ae-646ad0bfd7f0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:01.050123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132f28a1-1690-40ab-94ae-646ad0bfd7f0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:01.050839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132f28a1-1690-40ab-94ae-646ad0bfd7f0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:01.051303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132f28a1-1690-40ab-94ae-646ad0bfd7f0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:01.051772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-132f28a1-1690-40ab-94ae-646ad0bfd7f0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:01.305950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-132f28a1-1690-40ab-94ae-646ad0bfd7f0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:01.365964 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-132f28a1-1690-40ab-94ae-646ad0bfd7f0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1489 microversion: 2.1 time: 0.326588 Sep 27 10:17:01.367526 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2457/4498] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:01 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1489 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:01.390569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cfe282df-ea6a-4c1d-a022-59bbb27b8d39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:01.391200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cfe282df-ea6a-4c1d-a022-59bbb27b8d39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:01.400830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfe282df-ea6a-4c1d-a022-59bbb27b8d39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:01.401328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfe282df-ea6a-4c1d-a022-59bbb27b8d39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:01.401774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfe282df-ea6a-4c1d-a022-59bbb27b8d39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:01.402487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfe282df-ea6a-4c1d-a022-59bbb27b8d39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:01.403173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfe282df-ea6a-4c1d-a022-59bbb27b8d39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:01.403843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfe282df-ea6a-4c1d-a022-59bbb27b8d39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:01.455589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b1a8f215-e64c-4b54-b8c5-70be321e79ae tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:01.456584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b1a8f215-e64c-4b54-b8c5-70be321e79ae tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:01.471507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a8f215-e64c-4b54-b8c5-70be321e79ae tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:01.472805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a8f215-e64c-4b54-b8c5-70be321e79ae tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:01.474913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a8f215-e64c-4b54-b8c5-70be321e79ae tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:01.476606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a8f215-e64c-4b54-b8c5-70be321e79ae tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:01.477691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a8f215-e64c-4b54-b8c5-70be321e79ae tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:01.478735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a8f215-e64c-4b54-b8c5-70be321e79ae tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:01.690245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cfe282df-ea6a-4c1d-a022-59bbb27b8d39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:01.701033 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cfe282df-ea6a-4c1d-a022-59bbb27b8d39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1488 microversion: 2.1 time: 0.313613 Sep 27 10:17:01.702141 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2042/4499] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:01 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1488 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:01.706924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0012e790-a783-4610-9ca5-c35da6363279 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:01.707528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0012e790-a783-4610-9ca5-c35da6363279 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:01.714738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0012e790-a783-4610-9ca5-c35da6363279 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:01.715467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0012e790-a783-4610-9ca5-c35da6363279 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:01.716051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0012e790-a783-4610-9ca5-c35da6363279 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:01.716933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0012e790-a783-4610-9ca5-c35da6363279 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:01.717552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0012e790-a783-4610-9ca5-c35da6363279 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:01.718299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0012e790-a783-4610-9ca5-c35da6363279 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:01.769257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:01.810610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b1a8f215-e64c-4b54-b8c5-70be321e79ae tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:01.820230 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b1a8f215-e64c-4b54-b8c5-70be321e79ae tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1363 microversion: 2.1 time: 0.369988 Sep 27 10:17:01.820859 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2458/4500] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:01 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1363 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:01.846250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d94e940-bac1-48d4-b4c8-5c41cc9835c1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:01.847033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9d94e940-bac1-48d4-b4c8-5c41cc9835c1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:01.861165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d94e940-bac1-48d4-b4c8-5c41cc9835c1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:01.861856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d94e940-bac1-48d4-b4c8-5c41cc9835c1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:01.864835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d94e940-bac1-48d4-b4c8-5c41cc9835c1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:01.868220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d94e940-bac1-48d4-b4c8-5c41cc9835c1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:01.869762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d94e940-bac1-48d4-b4c8-5c41cc9835c1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:01.870521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d94e940-bac1-48d4-b4c8-5c41cc9835c1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:02.029386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0012e790-a783-4610-9ca5-c35da6363279 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:02.050296 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0012e790-a783-4610-9ca5-c35da6363279 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.346802 Sep 27 10:17:02.052221 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2043/4501] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:01 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:02.065626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:02.065840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3c56beac-764e-4d05-8913-6e4a16ab408a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:02.066592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3c56beac-764e-4d05-8913-6e4a16ab408a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:02.084081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c56beac-764e-4d05-8913-6e4a16ab408a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:02.088323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c56beac-764e-4d05-8913-6e4a16ab408a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:02.088954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c56beac-764e-4d05-8913-6e4a16ab408a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:02.090769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c56beac-764e-4d05-8913-6e4a16ab408a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:02.091379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c56beac-764e-4d05-8913-6e4a16ab408a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:02.092015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c56beac-764e-4d05-8913-6e4a16ab408a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:02.099915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9d94e940-bac1-48d4-b4c8-5c41cc9835c1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:02.109947 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d94e940-bac1-48d4-b4c8-5c41cc9835c1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1314 microversion: 2.1 time: 0.284305 Sep 27 10:17:02.110549 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2459/4502] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:01 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1314 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:02.124029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3c56beac-764e-4d05-8913-6e4a16ab408a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:17:02.220074 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3c56beac-764e-4d05-8913-6e4a16ab408a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 204 len: 0 microversion: 2.1 time: 0.167617 Sep 27 10:17:02.221050 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2044/4503] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:17:02 2022] DELETE /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:17:02.233702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a2104b85-6a7c-42c1-92cc-117d4369f768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:02.235196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a2104b85-6a7c-42c1-92cc-117d4369f768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:02.244003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2104b85-6a7c-42c1-92cc-117d4369f768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:02.245023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2104b85-6a7c-42c1-92cc-117d4369f768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:02.246112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2104b85-6a7c-42c1-92cc-117d4369f768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:02.246905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2104b85-6a7c-42c1-92cc-117d4369f768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:02.247730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2104b85-6a7c-42c1-92cc-117d4369f768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:02.248348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2104b85-6a7c-42c1-92cc-117d4369f768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:02.431949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a2104b85-6a7c-42c1-92cc-117d4369f768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:02.441204 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a2104b85-6a7c-42c1-92cc-117d4369f768 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1494 microversion: 2.1 time: 0.215313 Sep 27 10:17:02.442230 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2460/4504] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:02 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1494 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:02.852698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7876151e-45d2-49d4-b857-eb05f6a85535 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:02.853198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7876151e-45d2-49d4-b857-eb05f6a85535 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:02.859967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876151e-45d2-49d4-b857-eb05f6a85535 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:02.860605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876151e-45d2-49d4-b857-eb05f6a85535 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:02.861192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876151e-45d2-49d4-b857-eb05f6a85535 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:02.862401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876151e-45d2-49d4-b857-eb05f6a85535 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:02.862960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876151e-45d2-49d4-b857-eb05f6a85535 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:02.863586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7876151e-45d2-49d4-b857-eb05f6a85535 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:03.057009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7876151e-45d2-49d4-b857-eb05f6a85535 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:03.065415 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7876151e-45d2-49d4-b857-eb05f6a85535 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1548 microversion: 2.1 time: 0.222380 Sep 27 10:17:03.066715 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2045/4505] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:02 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1548 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:03.081520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d11aceea-9a47-4af9-888b-ae2991777174 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:03.082465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d11aceea-9a47-4af9-888b-ae2991777174 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:03.090866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d11aceea-9a47-4af9-888b-ae2991777174 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:03.091853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d11aceea-9a47-4af9-888b-ae2991777174 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:03.092516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d11aceea-9a47-4af9-888b-ae2991777174 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:03.093630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d11aceea-9a47-4af9-888b-ae2991777174 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:03.094326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d11aceea-9a47-4af9-888b-ae2991777174 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:03.095109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d11aceea-9a47-4af9-888b-ae2991777174 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:03.097242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7271e7ec-8424-4eb0-a1a1-cb569e6b2aef tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:03.097758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7271e7ec-8424-4eb0-a1a1-cb569e6b2aef tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:03.112363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7271e7ec-8424-4eb0-a1a1-cb569e6b2aef tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:03.114347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:03.114776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7271e7ec-8424-4eb0-a1a1-cb569e6b2aef tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:03.116266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7271e7ec-8424-4eb0-a1a1-cb569e6b2aef tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:03.117793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7271e7ec-8424-4eb0-a1a1-cb569e6b2aef tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:03.118646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7271e7ec-8424-4eb0-a1a1-cb569e6b2aef tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:03.119981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7271e7ec-8424-4eb0-a1a1-cb569e6b2aef tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:03.125170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:03.232717 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7271e7ec-8424-4eb0-a1a1-cb569e6b2aef tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.142538 Sep 27 10:17:03.241058 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2046/4506] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:17:03 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-interface => generated 284 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:03.248186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9878f75c-e40b-4730-b4d3-e2582436bf2f tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:03.249526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9878f75c-e40b-4730-b4d3-e2582436bf2f tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:03.257849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9878f75c-e40b-4730-b4d3-e2582436bf2f tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:03.258544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9878f75c-e40b-4730-b4d3-e2582436bf2f tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:03.259275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9878f75c-e40b-4730-b4d3-e2582436bf2f tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:03.260650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9878f75c-e40b-4730-b4d3-e2582436bf2f tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:03.261714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9878f75c-e40b-4730-b4d3-e2582436bf2f tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:03.262361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9878f75c-e40b-4730-b4d3-e2582436bf2f tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:03.464679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9878f75c-e40b-4730-b4d3-e2582436bf2f tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:03.479558 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9878f75c-e40b-4730-b4d3-e2582436bf2f tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1314 microversion: 2.1 time: 0.240957 Sep 27 10:17:03.496635 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2047/4507] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:03 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1314 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:03.505437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-accf624d-6280-46aa-8d51-b8382d4685bb tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:03.506217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-accf624d-6280-46aa-8d51-b8382d4685bb tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:03.512523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accf624d-6280-46aa-8d51-b8382d4685bb tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:03.514735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accf624d-6280-46aa-8d51-b8382d4685bb tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:03.515171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accf624d-6280-46aa-8d51-b8382d4685bb tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:03.516104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accf624d-6280-46aa-8d51-b8382d4685bb tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:03.516920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accf624d-6280-46aa-8d51-b8382d4685bb tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:03.517388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-accf624d-6280-46aa-8d51-b8382d4685bb tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:03.719923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-accf624d-6280-46aa-8d51-b8382d4685bb tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:03.731080 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-accf624d-6280-46aa-8d51-b8382d4685bb tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1494 microversion: 2.1 time: 0.229439 Sep 27 10:17:03.731685 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2048/4508] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:03 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1494 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:03.825859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d11aceea-9a47-4af9-888b-ae2991777174 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:03.834245 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d11aceea-9a47-4af9-888b-ae2991777174 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.756706 Sep 27 10:17:03.849041 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2461/4509] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:03 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 772 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:04.507796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dd43ad54-6b32-471d-bc45-10855401248a tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:04.508762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dd43ad54-6b32-471d-bc45-10855401248a tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:04.526172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd43ad54-6b32-471d-bc45-10855401248a tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:04.527068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd43ad54-6b32-471d-bc45-10855401248a tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:04.527736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd43ad54-6b32-471d-bc45-10855401248a tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:04.528916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd43ad54-6b32-471d-bc45-10855401248a tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:04.530437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd43ad54-6b32-471d-bc45-10855401248a tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:04.531664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd43ad54-6b32-471d-bc45-10855401248a tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:04.758704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25b58ab0-6d51-4ac4-bca1-a0b15484b683 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:04.760212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-25b58ab0-6d51-4ac4-bca1-a0b15484b683 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:04.760814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dd43ad54-6b32-471d-bc45-10855401248a tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:04.768153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b58ab0-6d51-4ac4-bca1-a0b15484b683 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:04.768938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b58ab0-6d51-4ac4-bca1-a0b15484b683 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:04.769540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b58ab0-6d51-4ac4-bca1-a0b15484b683 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:04.770033 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dd43ad54-6b32-471d-bc45-10855401248a tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1314 microversion: 2.1 time: 0.269502 Sep 27 10:17:04.770742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b58ab0-6d51-4ac4-bca1-a0b15484b683 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:04.771479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b58ab0-6d51-4ac4-bca1-a0b15484b683 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:04.772058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b58ab0-6d51-4ac4-bca1-a0b15484b683 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:04.786622 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2049/4510] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:04 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1314 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:04.862289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7c4a5e3b-cdc4-4fc7-b75f-35e69e708fd9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:04.863486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7c4a5e3b-cdc4-4fc7-b75f-35e69e708fd9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:04.871481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4a5e3b-cdc4-4fc7-b75f-35e69e708fd9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:04.872130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4a5e3b-cdc4-4fc7-b75f-35e69e708fd9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:04.872644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4a5e3b-cdc4-4fc7-b75f-35e69e708fd9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:04.874010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4a5e3b-cdc4-4fc7-b75f-35e69e708fd9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:04.874472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4a5e3b-cdc4-4fc7-b75f-35e69e708fd9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:04.874908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7c4a5e3b-cdc4-4fc7-b75f-35e69e708fd9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:05.140027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7c4a5e3b-cdc4-4fc7-b75f-35e69e708fd9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:05.176417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-25b58ab0-6d51-4ac4-bca1-a0b15484b683 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:05.179721 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7c4a5e3b-cdc4-4fc7-b75f-35e69e708fd9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.322259 Sep 27 10:17:05.188759 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-25b58ab0-6d51-4ac4-bca1-a0b15484b683 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1494 microversion: 2.1 time: 0.436351 Sep 27 10:17:05.189379 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2462/4511] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:04 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1494 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:05.205333 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2050/4512] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:04 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:05.594820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-b4a82e3a-ad94-4565-8ed8-e2903c35ed6d req-128dd6b1-3bd1-49b8-b132-8d361c696ad4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "779046b2-2e5b-4946-af25-c111c590a14e", "tag": "37f44fa9-2c33-45ba-a074-f37e0af9c6dc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:05.611476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b4a82e3a-ad94-4565-8ed8-e2903c35ed6d req-128dd6b1-3bd1-49b8-b132-8d361c696ad4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:05.613384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b4a82e3a-ad94-4565-8ed8-e2903c35ed6d req-128dd6b1-3bd1-49b8-b132-8d361c696ad4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:05.614048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b4a82e3a-ad94-4565-8ed8-e2903c35ed6d req-128dd6b1-3bd1-49b8-b132-8d361c696ad4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:05.649748 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b4a82e3a-ad94-4565-8ed8-e2903c35ed6d req-128dd6b1-3bd1-49b8-b132-8d361c696ad4 service nova] Creating event network-changed:37f44fa9-2c33-45ba-a074-f37e0af9c6dc for instance 779046b2-2e5b-4946-af25-c111c590a14e on nch1-master-1 Sep 27 10:17:05.665882 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-b4a82e3a-ad94-4565-8ed8-e2903c35ed6d req-128dd6b1-3bd1-49b8-b132-8d361c696ad4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.074669 Sep 27 10:17:05.666676 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2463/4513] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:05.796363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5b26a830-75cf-4129-8f64-3a5939cd7ab2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:05.797022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5b26a830-75cf-4129-8f64-3a5939cd7ab2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:05.807197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b26a830-75cf-4129-8f64-3a5939cd7ab2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:05.807683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b26a830-75cf-4129-8f64-3a5939cd7ab2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:05.812561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b26a830-75cf-4129-8f64-3a5939cd7ab2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:05.813614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b26a830-75cf-4129-8f64-3a5939cd7ab2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:05.814500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b26a830-75cf-4129-8f64-3a5939cd7ab2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:05.824417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b26a830-75cf-4129-8f64-3a5939cd7ab2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:06.226956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a04346ad-3586-4fbb-a96c-82604aa00955 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:06.227922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a04346ad-3586-4fbb-a96c-82604aa00955 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:06.238590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a04346ad-3586-4fbb-a96c-82604aa00955 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:06.241320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a04346ad-3586-4fbb-a96c-82604aa00955 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:06.253189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a04346ad-3586-4fbb-a96c-82604aa00955 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:06.254006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a04346ad-3586-4fbb-a96c-82604aa00955 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:06.254499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a04346ad-3586-4fbb-a96c-82604aa00955 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:06.254940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a04346ad-3586-4fbb-a96c-82604aa00955 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:06.527115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a04346ad-3586-4fbb-a96c-82604aa00955 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:06.548246 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a04346ad-3586-4fbb-a96c-82604aa00955 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.320107 Sep 27 10:17:06.550175 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2464/4514] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:06 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:06.561584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b26a830-75cf-4129-8f64-3a5939cd7ab2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:06.576866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c22cf807-0c9c-4053-aa5b-1174ba01f8d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:06.577574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c22cf807-0c9c-4053-aa5b-1174ba01f8d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:06.585548 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5b26a830-75cf-4129-8f64-3a5939cd7ab2 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1314 microversion: 2.1 time: 0.793909 Sep 27 10:17:06.587322 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2051/4515] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:05 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1314 bytes in 797 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:06.596452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22cf807-0c9c-4053-aa5b-1174ba01f8d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:06.596997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22cf807-0c9c-4053-aa5b-1174ba01f8d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:06.597442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22cf807-0c9c-4053-aa5b-1174ba01f8d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:06.598172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22cf807-0c9c-4053-aa5b-1174ba01f8d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:06.598624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22cf807-0c9c-4053-aa5b-1174ba01f8d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:06.603419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22cf807-0c9c-4053-aa5b-1174ba01f8d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:06.836154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c22cf807-0c9c-4053-aa5b-1174ba01f8d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:06.847192 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c22cf807-0c9c-4053-aa5b-1174ba01f8d5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1494 microversion: 2.1 time: 0.293518 Sep 27 10:17:06.848088 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2465/4516] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:06 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1494 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:07.577850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3e6e353f-9988-43de-b02d-be95d879bbbf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:07.579239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e6e353f-9988-43de-b02d-be95d879bbbf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:07.594377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6e353f-9988-43de-b02d-be95d879bbbf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:07.594913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6e353f-9988-43de-b02d-be95d879bbbf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:07.595334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6e353f-9988-43de-b02d-be95d879bbbf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:07.599833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6e353f-9988-43de-b02d-be95d879bbbf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:07.600338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6e353f-9988-43de-b02d-be95d879bbbf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:07.601805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e6e353f-9988-43de-b02d-be95d879bbbf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:07.621525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-230c2e82-09c5-4f3a-86de-0a79977f4c81 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:07.622259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-230c2e82-09c5-4f3a-86de-0a79977f4c81 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:07.637776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-230c2e82-09c5-4f3a-86de-0a79977f4c81 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:07.638384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-230c2e82-09c5-4f3a-86de-0a79977f4c81 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:07.640215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-230c2e82-09c5-4f3a-86de-0a79977f4c81 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:07.641209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-230c2e82-09c5-4f3a-86de-0a79977f4c81 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:07.641686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-230c2e82-09c5-4f3a-86de-0a79977f4c81 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:07.642175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-230c2e82-09c5-4f3a-86de-0a79977f4c81 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:07.936185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3e6e353f-9988-43de-b02d-be95d879bbbf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:07.953796 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3e6e353f-9988-43de-b02d-be95d879bbbf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.380519 Sep 27 10:17:07.955025 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2052/4517] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:07 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:07.962138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-abe18159-dc68-46fb-8a32-a105eba44c78 req-20e8bf4b-4e6e-4850-8178-3e094725635f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5a7e608b-b82d-43c7-a528-d4b5de120a71", "name": "network-vif-deleted", "tag": "0aea9992-4077-42ac-b40b-1f0fab059b1d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:07.974826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abe18159-dc68-46fb-8a32-a105eba44c78 req-20e8bf4b-4e6e-4850-8178-3e094725635f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:07.975596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abe18159-dc68-46fb-8a32-a105eba44c78 req-20e8bf4b-4e6e-4850-8178-3e094725635f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:07.975964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abe18159-dc68-46fb-8a32-a105eba44c78 req-20e8bf4b-4e6e-4850-8178-3e094725635f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:07.989788 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-abe18159-dc68-46fb-8a32-a105eba44c78 req-20e8bf4b-4e6e-4850-8178-3e094725635f service nova] Creating event network-vif-deleted:0aea9992-4077-42ac-b40b-1f0fab059b1d for instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 on nch2-master-1 Sep 27 10:17:07.996758 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-abe18159-dc68-46fb-8a32-a105eba44c78 req-20e8bf4b-4e6e-4850-8178-3e094725635f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039690 Sep 27 10:17:07.997690 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2053/4518] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:08.009789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:08.010810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:08.019723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:08.020432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:08.021117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:08.022109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:08.022884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:08.023464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:08.065343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:08.068146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:08.068844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:08.069645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lazy-loading 'fault' on Instance uuid 5a7e608b-b82d-43c7-a528-d4b5de120a71 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:17:08.090550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-230c2e82-09c5-4f3a-86de-0a79977f4c81 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:08.110269 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-230c2e82-09c5-4f3a-86de-0a79977f4c81 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1314 microversion: 2.1 time: 0.500407 Sep 27 10:17:08.133627 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2466/4519] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:07 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1314 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:08.156137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:08.173113 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dbfda5b3-8036-4b1c-a04a-2083b02d06f5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 200 len: 1298 microversion: 2.1 time: 0.168073 Sep 27 10:17:08.174352 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2054/4520] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:08 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 1298 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:08.290428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7037ff69-f8f1-406b-aa59-4f0c82b5fe4c req-d1849a6c-d3b9-43a3-9921-e46d756fdfda service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f7d07191-c8dd-4778-85ab-87262a4d21ed", "name": "network-vif-plugged", "status": "completed", "tag": "068e1ae8-487b-49d1-92b7-bca512800ab7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:08.305068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7037ff69-f8f1-406b-aa59-4f0c82b5fe4c req-d1849a6c-d3b9-43a3-9921-e46d756fdfda service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:08.306100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7037ff69-f8f1-406b-aa59-4f0c82b5fe4c req-d1849a6c-d3b9-43a3-9921-e46d756fdfda service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:08.307178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7037ff69-f8f1-406b-aa59-4f0c82b5fe4c req-d1849a6c-d3b9-43a3-9921-e46d756fdfda service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:08.326629 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7037ff69-f8f1-406b-aa59-4f0c82b5fe4c req-d1849a6c-d3b9-43a3-9921-e46d756fdfda service nova] Creating event network-vif-plugged:068e1ae8-487b-49d1-92b7-bca512800ab7 for instance f7d07191-c8dd-4778-85ab-87262a4d21ed on nch2-master-1 Sep 27 10:17:08.332590 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7037ff69-f8f1-406b-aa59-4f0c82b5fe4c req-d1849a6c-d3b9-43a3-9921-e46d756fdfda service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047615 Sep 27 10:17:08.334254 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2467/4521] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:08.981417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2a2c1795-82cc-4688-830a-afec1494df42 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:08.982034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2a2c1795-82cc-4688-830a-afec1494df42 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:08.994136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a2c1795-82cc-4688-830a-afec1494df42 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:08.996919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a2c1795-82cc-4688-830a-afec1494df42 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:08.997401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a2c1795-82cc-4688-830a-afec1494df42 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:08.998147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a2c1795-82cc-4688-830a-afec1494df42 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:08.998611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a2c1795-82cc-4688-830a-afec1494df42 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:09.001483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a2c1795-82cc-4688-830a-afec1494df42 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:09.150248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75f8fd60-bc4f-41e3-818a-61649cfee450 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:09.151058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-75f8fd60-bc4f-41e3-818a-61649cfee450 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:09.160733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f8fd60-bc4f-41e3-818a-61649cfee450 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:09.161616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f8fd60-bc4f-41e3-818a-61649cfee450 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:09.162481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f8fd60-bc4f-41e3-818a-61649cfee450 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:09.163623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f8fd60-bc4f-41e3-818a-61649cfee450 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:09.164305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f8fd60-bc4f-41e3-818a-61649cfee450 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:09.165209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f8fd60-bc4f-41e3-818a-61649cfee450 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:09.362672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-75f8fd60-bc4f-41e3-818a-61649cfee450 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:09.375201 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-75f8fd60-bc4f-41e3-818a-61649cfee450 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1314 microversion: 2.1 time: 0.241161 Sep 27 10:17:09.376233 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2468/4522] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:09 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1314 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:09.398727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:09.399880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:09.407621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:09.408477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:09.409160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:09.410201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:09.410881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:09.411545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:09.442024 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:17:09.445709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2a2c1795-82cc-4688-830a-afec1494df42 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:09.455303 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2a2c1795-82cc-4688-830a-afec1494df42 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.477863 Sep 27 10:17:09.455904 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2055/4523] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:08 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 HTTP/1.1 Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '5a7e608b-b82d-43c7-a528-d4b5de120a71'} Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:09.460055 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:09.464209 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:09.464209 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:09.464209 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:09.464209 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:17:09.464209 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:17:09.464209 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:17:09.464209 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:09.465515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:17:09.466835 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8da5ce97-2e50-4808-be13-72a4ce539a55 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 404 len: 111 microversion: 2.1 time: 0.076094 Sep 27 10:17:09.475985 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2469/4524] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:09 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:17:09.477794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1387db1d-76da-4c96-81d3-b984c167da96 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:09.481992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1387db1d-76da-4c96-81d3-b984c167da96 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:09.496409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1387db1d-76da-4c96-81d3-b984c167da96 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:09.496934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1387db1d-76da-4c96-81d3-b984c167da96 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:09.497379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1387db1d-76da-4c96-81d3-b984c167da96 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:09.498093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1387db1d-76da-4c96-81d3-b984c167da96 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:09.502343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1387db1d-76da-4c96-81d3-b984c167da96 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:09.503924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1387db1d-76da-4c96-81d3-b984c167da96 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:09.546859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1387db1d-76da-4c96-81d3-b984c167da96 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:17:09.685859 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1387db1d-76da-4c96-81d3-b984c167da96 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 204 len: 0 microversion: 2.1 time: 0.204320 Sep 27 10:17:09.688120 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2056/4525] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:17:09 2022] DELETE /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 0 bytes in 216 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:17:09.697475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-68aaac56-0117-4ef2-8cc6-eb5e6d2f70d4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:09.698781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-68aaac56-0117-4ef2-8cc6-eb5e6d2f70d4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:09.707787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68aaac56-0117-4ef2-8cc6-eb5e6d2f70d4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:09.708940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68aaac56-0117-4ef2-8cc6-eb5e6d2f70d4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:09.709383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68aaac56-0117-4ef2-8cc6-eb5e6d2f70d4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:09.710200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68aaac56-0117-4ef2-8cc6-eb5e6d2f70d4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:09.711189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68aaac56-0117-4ef2-8cc6-eb5e6d2f70d4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:09.711626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68aaac56-0117-4ef2-8cc6-eb5e6d2f70d4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:09.969042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-68aaac56-0117-4ef2-8cc6-eb5e6d2f70d4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:09.979735 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-68aaac56-0117-4ef2-8cc6-eb5e6d2f70d4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1495 microversion: 2.1 time: 0.287416 Sep 27 10:17:09.982955 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2470/4526] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:09 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1495 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:10.406154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b613b5d7-8208-45c8-8b37-2538410364ec tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:10.407601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b613b5d7-8208-45c8-8b37-2538410364ec tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:10.424078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b613b5d7-8208-45c8-8b37-2538410364ec tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:10.425654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b613b5d7-8208-45c8-8b37-2538410364ec tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:10.427401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b613b5d7-8208-45c8-8b37-2538410364ec tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:10.432412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b613b5d7-8208-45c8-8b37-2538410364ec tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:10.433466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b613b5d7-8208-45c8-8b37-2538410364ec tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:10.434976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b613b5d7-8208-45c8-8b37-2538410364ec tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:10.482670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bce8bc61-b889-4497-83d7-9c01ea7dea15 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:10.484293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bce8bc61-b889-4497-83d7-9c01ea7dea15 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:10.488929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:10.511188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bce8bc61-b889-4497-83d7-9c01ea7dea15 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:10.511685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bce8bc61-b889-4497-83d7-9c01ea7dea15 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:10.516506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bce8bc61-b889-4497-83d7-9c01ea7dea15 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:10.522692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bce8bc61-b889-4497-83d7-9c01ea7dea15 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:10.525943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bce8bc61-b889-4497-83d7-9c01ea7dea15 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:10.528301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bce8bc61-b889-4497-83d7-9c01ea7dea15 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:10.810376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b613b5d7-8208-45c8-8b37-2538410364ec tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:10.832992 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b613b5d7-8208-45c8-8b37-2538410364ec tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1501 microversion: 2.1 time: 0.432935 Sep 27 10:17:10.838142 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2057/4527] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:10 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1501 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:10.864429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-68a5f340-71f9-41de-a902-88095f73ba2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:10.865459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-68a5f340-71f9-41de-a902-88095f73ba2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:10.872640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68a5f340-71f9-41de-a902-88095f73ba2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:10.873590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68a5f340-71f9-41de-a902-88095f73ba2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:10.874279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68a5f340-71f9-41de-a902-88095f73ba2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:10.875229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68a5f340-71f9-41de-a902-88095f73ba2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:10.875914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68a5f340-71f9-41de-a902-88095f73ba2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:10.876585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68a5f340-71f9-41de-a902-88095f73ba2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:10.897195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bce8bc61-b889-4497-83d7-9c01ea7dea15 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:10.921031 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bce8bc61-b889-4497-83d7-9c01ea7dea15 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.443285 Sep 27 10:17:10.921625 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2471/4528] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:10 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:10.969335 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-68a5f340-71f9-41de-a902-88095f73ba2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed/os-server-password" status: 204 len: 0 microversion: 2.1 time: 0.109640 Sep 27 10:17:10.975814 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2058/4529] 10.222.0.26 () {58 vars in 1296 bytes} [Tue Sep 27 10:17:10 2022] DELETE /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed/os-server-password => generated 0 bytes in 112 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:17:10.982154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58cb9e6a-4442-4fe4-a456-c196dfc1ff2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:10.983135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58cb9e6a-4442-4fe4-a456-c196dfc1ff2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:10.992693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cb9e6a-4442-4fe4-a456-c196dfc1ff2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:10.993625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cb9e6a-4442-4fe4-a456-c196dfc1ff2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:10.995225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cb9e6a-4442-4fe4-a456-c196dfc1ff2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:10.996570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cb9e6a-4442-4fe4-a456-c196dfc1ff2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:10.997862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cb9e6a-4442-4fe4-a456-c196dfc1ff2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:11.007166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ece3a169-b743-4b42-a430-e3ba079cd0c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:11.008188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ece3a169-b743-4b42-a430-e3ba079cd0c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:11.009417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cb9e6a-4442-4fe4-a456-c196dfc1ff2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:11.027866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ece3a169-b743-4b42-a430-e3ba079cd0c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:11.028788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ece3a169-b743-4b42-a430-e3ba079cd0c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:11.030004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ece3a169-b743-4b42-a430-e3ba079cd0c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:11.043385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ece3a169-b743-4b42-a430-e3ba079cd0c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:11.043941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ece3a169-b743-4b42-a430-e3ba079cd0c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:11.044392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ece3a169-b743-4b42-a430-e3ba079cd0c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:11.092017 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58cb9e6a-4442-4fe4-a456-c196dfc1ff2e tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed/os-server-password" status: 200 len: 16 microversion: 2.1 time: 0.114245 Sep 27 10:17:11.093195 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2472/4530] 10.222.0.26 () {58 vars in 1293 bytes} [Tue Sep 27 10:17:10 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed/os-server-password => generated 16 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 27 10:17:11.120887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c5b98e4c-5a7a-45bf-ae9d-72ed6f256bb1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:11.121696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c5b98e4c-5a7a-45bf-ae9d-72ed6f256bb1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:11.136147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5b98e4c-5a7a-45bf-ae9d-72ed6f256bb1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:11.139710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5b98e4c-5a7a-45bf-ae9d-72ed6f256bb1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:11.141946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5b98e4c-5a7a-45bf-ae9d-72ed6f256bb1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:11.143282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5b98e4c-5a7a-45bf-ae9d-72ed6f256bb1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:11.150781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5b98e4c-5a7a-45bf-ae9d-72ed6f256bb1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:11.153424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5b98e4c-5a7a-45bf-ae9d-72ed6f256bb1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:11.228254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c5b98e4c-5a7a-45bf-ae9d-72ed6f256bb1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:17:11.443733 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c5b98e4c-5a7a-45bf-ae9d-72ed6f256bb1 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 204 len: 0 microversion: 2.1 time: 0.340319 Sep 27 10:17:11.448132 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2473/4531] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:17:11 2022] DELETE /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 0 bytes in 342 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:17:11.477572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6c11be6d-ef10-49ef-9d3d-0ab5766d3d15 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:11.478371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6c11be6d-ef10-49ef-9d3d-0ab5766d3d15 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:11.496531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c11be6d-ef10-49ef-9d3d-0ab5766d3d15 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:11.498524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c11be6d-ef10-49ef-9d3d-0ab5766d3d15 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:11.502809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c11be6d-ef10-49ef-9d3d-0ab5766d3d15 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:11.504150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c11be6d-ef10-49ef-9d3d-0ab5766d3d15 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:11.505969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c11be6d-ef10-49ef-9d3d-0ab5766d3d15 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:11.508476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c11be6d-ef10-49ef-9d3d-0ab5766d3d15 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:11.628222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ece3a169-b743-4b42-a430-e3ba079cd0c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:11.649416 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ece3a169-b743-4b42-a430-e3ba079cd0c2 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1495 microversion: 2.1 time: 0.647471 Sep 27 10:17:11.667348 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2059/4532] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:11 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1495 bytes in 666 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:11.885625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6c11be6d-ef10-49ef-9d3d-0ab5766d3d15 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:11.905337 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6c11be6d-ef10-49ef-9d3d-0ab5766d3d15 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1507 microversion: 2.1 time: 0.455786 Sep 27 10:17:11.921664 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2474/4533] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:11 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1507 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:11.944758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-553e20a8-8bb7-4b08-8dab-9c36a94f4b9f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:11.945394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-553e20a8-8bb7-4b08-8dab-9c36a94f4b9f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:11.963987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-553e20a8-8bb7-4b08-8dab-9c36a94f4b9f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:11.966793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-553e20a8-8bb7-4b08-8dab-9c36a94f4b9f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:11.967292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-553e20a8-8bb7-4b08-8dab-9c36a94f4b9f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:11.969509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-553e20a8-8bb7-4b08-8dab-9c36a94f4b9f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:11.973167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-553e20a8-8bb7-4b08-8dab-9c36a94f4b9f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:11.973656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-553e20a8-8bb7-4b08-8dab-9c36a94f4b9f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:12.015049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=2,nova_cell1:INSERT=5 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:12.814176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-553e20a8-8bb7-4b08-8dab-9c36a94f4b9f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:12.830544 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-553e20a8-8bb7-4b08-8dab-9c36a94f4b9f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.890669 Sep 27 10:17:12.831149 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2060/4534] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:11 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 892 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:12.840269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:12.840920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:12.862010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:12.862708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:12.863771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:12.864394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:12.865113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:12.866121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:12.937673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:12.938180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:12.938683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:12.939465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lazy-loading 'fault' on Instance uuid f13eb59c-b2be-4b37-addc-676d74eff6ed {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:17:13.028946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:13.037252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5250a728-293e-4b13-b587-c1d7efd0aa12 req-d5976e91-bf39-4fcb-b286-d9bbfc7ef186 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f13eb59c-b2be-4b37-addc-676d74eff6ed", "name": "network-vif-deleted", "tag": "2a400220-e798-40d4-8e16-33e898bba375"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:13.052052 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c66e6d47-6bdb-4597-ad0b-1123b8701469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 200 len: 1298 microversion: 2.1 time: 0.217575 Sep 27 10:17:13.052685 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2061/4535] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:12 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 1298 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:13.073703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9022744a-ea2a-4e51-8ae4-4209ea0a0248 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:13.074629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9022744a-ea2a-4e51-8ae4-4209ea0a0248 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:13.083267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5250a728-293e-4b13-b587-c1d7efd0aa12 req-d5976e91-bf39-4fcb-b286-d9bbfc7ef186 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:13.089760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5250a728-293e-4b13-b587-c1d7efd0aa12 req-d5976e91-bf39-4fcb-b286-d9bbfc7ef186 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:13.090886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9022744a-ea2a-4e51-8ae4-4209ea0a0248 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:13.091393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9022744a-ea2a-4e51-8ae4-4209ea0a0248 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:13.091844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9022744a-ea2a-4e51-8ae4-4209ea0a0248 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:13.092574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9022744a-ea2a-4e51-8ae4-4209ea0a0248 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:13.093869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5250a728-293e-4b13-b587-c1d7efd0aa12 req-d5976e91-bf39-4fcb-b286-d9bbfc7ef186 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:13.094940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9022744a-ea2a-4e51-8ae4-4209ea0a0248 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:13.095569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9022744a-ea2a-4e51-8ae4-4209ea0a0248 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:13.154346 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5250a728-293e-4b13-b587-c1d7efd0aa12 req-d5976e91-bf39-4fcb-b286-d9bbfc7ef186 service nova] Creating event network-vif-deleted:2a400220-e798-40d4-8e16-33e898bba375 for instance f13eb59c-b2be-4b37-addc-676d74eff6ed on nch1-master-1 Sep 27 10:17:13.171287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5250a728-293e-4b13-b587-c1d7efd0aa12 req-d5976e91-bf39-4fcb-b286-d9bbfc7ef186 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.515615 Sep 27 10:17:13.176959 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2475/4536] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:17:13.517056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9022744a-ea2a-4e51-8ae4-4209ea0a0248 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:13.531829 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9022744a-ea2a-4e51-8ae4-4209ea0a0248 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1507 microversion: 2.1 time: 0.461936 Sep 27 10:17:13.533941 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2062/4537] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:13 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1507 bytes in 481 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:13.859473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a96e2dcf-3343-4e1f-977c-537984e2d6ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:13.860193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a96e2dcf-3343-4e1f-977c-537984e2d6ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:13.878474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96e2dcf-3343-4e1f-977c-537984e2d6ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:13.878994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96e2dcf-3343-4e1f-977c-537984e2d6ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:13.884182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96e2dcf-3343-4e1f-977c-537984e2d6ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:13.885036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96e2dcf-3343-4e1f-977c-537984e2d6ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:13.885528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96e2dcf-3343-4e1f-977c-537984e2d6ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:13.885966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a96e2dcf-3343-4e1f-977c-537984e2d6ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:14.078258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:14.078916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:14.100322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:14.105639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:14.108426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:14.110276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:14.115875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:14.116546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:14.161046 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed HTTP/1.1 Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'f13eb59c-b2be-4b37-addc-676d74eff6ed'} Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:14.163063 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:14.167698 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:14.167698 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:14.167698 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:14.167698 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:17:14.167698 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:17:14.167698 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:17:14.167698 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:14.167698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:17:14.167698 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-74ddc58e-1edc-4bea-a800-95a0df1ee9f9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 404 len: 111 microversion: 2.1 time: 0.092920 Sep 27 10:17:14.167698 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2063/4538] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:14 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 111 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:17:14.489620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a96e2dcf-3343-4e1f-977c-537984e2d6ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:14.503117 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a96e2dcf-3343-4e1f-977c-537984e2d6ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.648629 Sep 27 10:17:14.504400 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2476/4539] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:13 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:14.561888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:14.562510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:14.579552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:14.580379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:14.580488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:14.581253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:14.581801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:14.592539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:14.716449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:14.722959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:14.723922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:14.726506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:14.728819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lazy-loading 'fault' on Instance uuid f7d07191-c8dd-4778-85ab-87262a4d21ed {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:17:14.791246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-623260353", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "662e23c5-f7e1-4e14-af6c-49d672076632"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:14.812335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '04440378e21043b2a287e9b4c800080d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '898c5ffacd6247c6a8bff965ebaf0591', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:14.832434 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-739fed1a-6674-4a04-a919-2014950da726 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 200 len: 1304 microversion: 2.1 time: 0.273686 Sep 27 10:17:14.832434 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2064/4540] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:14 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 1304 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:14.894782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] validate_networks() for [('662e23c5-f7e1-4e14-af6c-49d672076632', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:17:15.684352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-314b3619-f581-44a7-b147-b4f6a33bdaea tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:15.684965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-314b3619-f581-44a7-b147-b4f6a33bdaea tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:15.698337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-314b3619-f581-44a7-b147-b4f6a33bdaea tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:15.699195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:17:15.699669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:17:15.701511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:17:15.701939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:17:15.702361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:17:15.702920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-314b3619-f581-44a7-b147-b4f6a33bdaea tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:15.703479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-314b3619-f581-44a7-b147-b4f6a33bdaea tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:15.706062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-314b3619-f581-44a7-b147-b4f6a33bdaea tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:15.711389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-314b3619-f581-44a7-b147-b4f6a33bdaea tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:15.712611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-314b3619-f581-44a7-b147-b4f6a33bdaea tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:15.723118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:15.723823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:15.724274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:15.727364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:15.727849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:15.728285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:15.777110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:17:15.785738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-314b3619-f581-44a7-b147-b4f6a33bdaea tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:17:15.792522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for user a7b8599d0b40424087e52d8e35b71ec4 and project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:17:15.827725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:17:15.833608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:17:15.939698 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1260fdaf-7d34-4de0-acfc-6cbdee2d9db7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.152990 Sep 27 10:17:15.951972 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2477/4541] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:17:14 2022] POST /compute/v2.1/servers => generated 380 bytes in 1155 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:17:15.981453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-74cb3cc2-5dee-47a6-a33e-b31b6e4a5f65 req-299b7675-2a2b-40a4-9119-89cfdf98d238 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f7d07191-c8dd-4778-85ab-87262a4d21ed", "name": "network-vif-deleted", "tag": "068e1ae8-487b-49d1-92b7-bca512800ab7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:16.010778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-74cb3cc2-5dee-47a6-a33e-b31b6e4a5f65 req-299b7675-2a2b-40a4-9119-89cfdf98d238 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:16.011496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-74cb3cc2-5dee-47a6-a33e-b31b6e4a5f65 req-299b7675-2a2b-40a4-9119-89cfdf98d238 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:16.012282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-74cb3cc2-5dee-47a6-a33e-b31b6e4a5f65 req-299b7675-2a2b-40a4-9119-89cfdf98d238 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:16.036346 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-74cb3cc2-5dee-47a6-a33e-b31b6e4a5f65 req-299b7675-2a2b-40a4-9119-89cfdf98d238 service nova] Creating event network-vif-deleted:068e1ae8-487b-49d1-92b7-bca512800ab7 for instance f7d07191-c8dd-4778-85ab-87262a4d21ed on nch2-master-1 Sep 27 10:17:16.042801 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-74cb3cc2-5dee-47a6-a33e-b31b6e4a5f65 req-299b7675-2a2b-40a4-9119-89cfdf98d238 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080442 Sep 27 10:17:16.043798 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2478/4542] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:16.051871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cca2ca20-8622-47bd-9c3e-a51afb873158 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:16.052943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cca2ca20-8622-47bd-9c3e-a51afb873158 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:16.056310 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-314b3619-f581-44a7-b147-b4f6a33bdaea tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 204 len: 0 microversion: 2.32 time: 1.061544 Sep 27 10:17:16.058796 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2065/4543] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:17:14 2022] DELETE /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 0 bytes in 1065 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Sep 27 10:17:16.062716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cca2ca20-8622-47bd-9c3e-a51afb873158 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:16.064786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cca2ca20-8622-47bd-9c3e-a51afb873158 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:16.065412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cca2ca20-8622-47bd-9c3e-a51afb873158 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:16.066551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cca2ca20-8622-47bd-9c3e-a51afb873158 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:16.067309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cca2ca20-8622-47bd-9c3e-a51afb873158 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:16.067961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cca2ca20-8622-47bd-9c3e-a51afb873158 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:16.078249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:16.079017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] [instance: f7d07191-c8dd-4778-85ab-87262a4d21ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:16.103661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:16.106506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:16.111197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:16.114432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:16.115870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:16.115870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:16.157783 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] HTTP exception thrown: Instance f7d07191-c8dd-4778-85ab-87262a4d21ed could not be found. Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Request method failure captured: Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed HTTP/1.1 Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs2ST7LCbc7ZPDG5n7Ixt2My_w-mjsV76VFaK8DtKqUYtW-Qb1K-xajWKtUwBNC67ctvrsikq7zcJxtJRYXiilJMpaQLw6J8V0a-6mZFuBXXuRWxu07tefL9EUdm7iaEIWoH3MfZCdNi92NxVgpVF7Yin7HcB_HlUVjgpIPUACbtODo Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 898c5ffacd6247c6a8bff965ebaf0591 Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerPasswordTestJSON-1296246252 Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/898c5ffacd6247c6a8bff965ebaf0591"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/898c5ffacd6247c6a8bff965ebaf0591"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/898c5ffacd6247c6a8bff965ebaf0591"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_898c5ffacd6247c6a8bff965ebaf0591"}]}] Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerPasswordTestJSON-1296246252 Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 898c5ffacd6247c6a8bff965ebaf0591 Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerPasswordTestJSON-1296246252 Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerPasswordTestJSON-1296246252-project Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 04440378e21043b2a287e9b4c800080d Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerPasswordTestJSON-1296246252-project Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance f7d07191-c8dd-4778-85ab-87262a4d21ed could not be found. Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'f7d07191-c8dd-4778-85ab-87262a4d21ed'} Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f7d07191-c8dd-4778-85ab-87262a4d21ed could not be found. Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f7d07191-c8dd-4778-85ab-87262a4d21ed could not be found. Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:16.159528 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:16.164142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:16.164142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:16.164142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:16.164142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:17:16.164142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:17:16.164142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f7d07191-c8dd-4778-85ab-87262a4d21ed could not be found. Sep 27 10:17:16.164142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:16.164142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] Returning 404 to user: Instance f7d07191-c8dd-4778-85ab-87262a4d21ed could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:17:16.164142 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a1fa8d5b-bcab-49ea-bc2e-5d9bf365f0e3 tempest-ServerPasswordTestJSON-1296246252 tempest-ServerPasswordTestJSON-1296246252-project] 10.222.0.26 "GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed" status: 404 len: 111 microversion: 2.1 time: 0.100149 Sep 27 10:17:16.172838 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2066/4544] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:16 2022] GET /compute/v2.1/servers/f7d07191-c8dd-4778-85ab-87262a4d21ed => generated 111 bytes in 109 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:17:16.183881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6aac7047-11ea-493e-934e-0ac80b584ce6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:16.185923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6aac7047-11ea-493e-934e-0ac80b584ce6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:16.438626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6aac7047-11ea-493e-934e-0ac80b584ce6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:16.443401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cca2ca20-8622-47bd-9c3e-a51afb873158 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:16.449559 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6aac7047-11ea-493e-934e-0ac80b584ce6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1208 microversion: 2.1 time: 0.275184 Sep 27 10:17:16.452938 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2067/4545] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:16 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1208 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:16.462688 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cca2ca20-8622-47bd-9c3e-a51afb873158 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.415633 Sep 27 10:17:16.465665 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2479/4546] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:16 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:16.477800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:16.490366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:16.510951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:16.511459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:16.511944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:16.531799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:16.535456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:16.536901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:16.989524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82b6dad3-db8d-4e08-b7e5-c84d2399b9fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Action: 'create', calling method: .version_select at 0x7f7f6249ae50>, body: {"keypair": {"name": "tempest-TestNetworkBasicOps-762744049", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBInNAGbGNM4eDutTHPcd3rtnGO4yFA+e0GbnKa6xLzTh72v/hacW4XbIgMmGBQK/P83ULh2mCWR9uq9qNUOK5N2Z9UZ9ZXEO5U4UWcbnIIX2BxFph8UrekWBb/EP+nSk0w=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:17.030480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-82b6dad3-db8d-4e08-b7e5-c84d2399b9fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:17:17.041843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-82b6dad3-db8d-4e08-b7e5-c84d2399b9fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for user 1d92f7d2ee2d47beaa4f5d43ac40cd82 and project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:17:17.095040 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-82b6dad3-db8d-4e08-b7e5-c84d2399b9fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.110311 Sep 27 10:17:17.095711 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2480/4547] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:17:16 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:17.238967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:17.264343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:17.267482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:17.269990 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-da7ee4a8-13d7-419c-b7ab-64ad340035c1 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2953 microversion: 2.32 time: 0.815152 Sep 27 10:17:17.277290 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2068/4548] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:16 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2953 bytes in 823 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:17:17.479216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3cfd108b-4433-4c20-97b3-c7dfaef5e566 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:17.480105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3cfd108b-4433-4c20-97b3-c7dfaef5e566 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:17.487937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ed892300-6890-45d8-9ca9-6e292e880cb9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:17.488926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ed892300-6890-45d8-9ca9-6e292e880cb9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:17.510945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed892300-6890-45d8-9ca9-6e292e880cb9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:17.512189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfd108b-4433-4c20-97b3-c7dfaef5e566 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:17.516423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfd108b-4433-4c20-97b3-c7dfaef5e566 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:17.517034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed892300-6890-45d8-9ca9-6e292e880cb9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:17.517941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed892300-6890-45d8-9ca9-6e292e880cb9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:17.518996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfd108b-4433-4c20-97b3-c7dfaef5e566 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:17.520284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed892300-6890-45d8-9ca9-6e292e880cb9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:17.521833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed892300-6890-45d8-9ca9-6e292e880cb9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:17.522113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfd108b-4433-4c20-97b3-c7dfaef5e566 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:17.524917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfd108b-4433-4c20-97b3-c7dfaef5e566 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:17.525304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed892300-6890-45d8-9ca9-6e292e880cb9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:17.525928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfd108b-4433-4c20-97b3-c7dfaef5e566 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:17.706065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3cfd108b-4433-4c20-97b3-c7dfaef5e566 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:17.723384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:17.736555 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3cfd108b-4433-4c20-97b3-c7dfaef5e566 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1212 microversion: 2.1 time: 0.261005 Sep 27 10:17:17.739098 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2481/4549] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:17 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1212 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:17.946495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ed892300-6890-45d8-9ca9-6e292e880cb9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:17.957066 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ed892300-6890-45d8-9ca9-6e292e880cb9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.473179 Sep 27 10:17:17.958299 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2069/4550] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:17 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:18.313974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:18.315204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:18.324766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:18.325541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:18.326468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:18.331498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:18.332734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:18.333408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:18.709973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:18.726162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:18.734003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:18.739889 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dc0f50fe-39d9-417a-8a3d-2509f725405b tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2953 microversion: 2.32 time: 0.434181 Sep 27 10:17:18.744908 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2482/4551] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:18 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2953 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:17:18.775715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c349658-c6f1-4512-b5d9-55392b1b6345 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:18.776363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c349658-c6f1-4512-b5d9-55392b1b6345 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:18.801824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c349658-c6f1-4512-b5d9-55392b1b6345 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:18.802638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c349658-c6f1-4512-b5d9-55392b1b6345 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:18.803597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c349658-c6f1-4512-b5d9-55392b1b6345 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:18.804899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c349658-c6f1-4512-b5d9-55392b1b6345 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:18.806681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c349658-c6f1-4512-b5d9-55392b1b6345 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:18.810691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c349658-c6f1-4512-b5d9-55392b1b6345 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:18.983043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-40aee706-d4e0-4ca4-a678-00fc12e532e5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:18.983944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-40aee706-d4e0-4ca4-a678-00fc12e532e5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:18.995966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40aee706-d4e0-4ca4-a678-00fc12e532e5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:18.998254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40aee706-d4e0-4ca4-a678-00fc12e532e5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:18.998894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40aee706-d4e0-4ca4-a678-00fc12e532e5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:19.000206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40aee706-d4e0-4ca4-a678-00fc12e532e5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:19.001266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40aee706-d4e0-4ca4-a678-00fc12e532e5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:19.002089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-40aee706-d4e0-4ca4-a678-00fc12e532e5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:19.110552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c349658-c6f1-4512-b5d9-55392b1b6345 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:19.120361 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c349658-c6f1-4512-b5d9-55392b1b6345 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1260 microversion: 2.1 time: 0.353883 Sep 27 10:17:19.126750 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2070/4552] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:18 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1260 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:19.528309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-40aee706-d4e0-4ca4-a678-00fc12e532e5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:19.567840 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-40aee706-d4e0-4ca4-a678-00fc12e532e5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.589279 Sep 27 10:17:19.571094 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2483/4553] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:18 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:19.774880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:19.775756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:19.788341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:19.789856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:19.792170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:19.793328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:19.794980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:19.797233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:20.143626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b86c0d06-3125-4e00-9823-9cba0067758d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:20.144498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b86c0d06-3125-4e00-9823-9cba0067758d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:20.152103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b86c0d06-3125-4e00-9823-9cba0067758d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:20.152793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b86c0d06-3125-4e00-9823-9cba0067758d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:20.153503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b86c0d06-3125-4e00-9823-9cba0067758d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:20.154373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b86c0d06-3125-4e00-9823-9cba0067758d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:20.155043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b86c0d06-3125-4e00-9823-9cba0067758d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:20.155617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b86c0d06-3125-4e00-9823-9cba0067758d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:20.284439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b86c0d06-3125-4e00-9823-9cba0067758d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:20.309503 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b86c0d06-3125-4e00-9823-9cba0067758d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1266 microversion: 2.1 time: 0.167388 Sep 27 10:17:20.314823 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2484/4554] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:20 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1266 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:20.414722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkBasicOps-server-933767991", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "38982d31-0a02-4150-ae5b-83724f9c8d2e"}], "key_name": "tempest-TestNetworkBasicOps-762744049", "security_groups": [{"name": "tempest-secgroup-smoke-807275213"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:20.553069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] validate_networks() for [('38982d31-0a02-4150-ae5b-83724f9c8d2e', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:17:20.780717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:20.790441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:20.791884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:20.793172 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4735d51e-ab62-496d-a090-140975a91b00 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2953 microversion: 2.32 time: 1.024719 Sep 27 10:17:20.793741 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2071/4555] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:19 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2953 bytes in 1026 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:17:20.809279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95229c44-61ae-4eb7-9373-46dcd4c095de tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:20.816815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-95229c44-61ae-4eb7-9373-46dcd4c095de tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:20.824728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95229c44-61ae-4eb7-9373-46dcd4c095de tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:20.825226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95229c44-61ae-4eb7-9373-46dcd4c095de tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:20.826011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95229c44-61ae-4eb7-9373-46dcd4c095de tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:20.826731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95229c44-61ae-4eb7-9373-46dcd4c095de tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:20.827187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95229c44-61ae-4eb7-9373-46dcd4c095de tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:20.827619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95229c44-61ae-4eb7-9373-46dcd4c095de tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:21.211769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:17:21.212410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:17:21.213093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:17:21.213651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:17:21.214431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:17:21.237745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:21.239153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:21.239846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:21.241979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:21.242628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:21.245091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:21.290712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:17:21.301160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for user 1d92f7d2ee2d47beaa4f5d43ac40cd82 and project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:17:21.322587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:17:21.328286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:17:21.440980 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69c1ab6c-46af-4c30-9d53-010dbc484de7 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.027937 Sep 27 10:17:21.442514 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2485/4556] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:17:20 2022] POST /compute/v2.1/servers => generated 405 bytes in 1032 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:17:21.459383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5019337-92e7-4c41-9fcc-5282a36903ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:21.460211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a5019337-92e7-4c41-9fcc-5282a36903ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:21.469009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-95229c44-61ae-4eb7-9373-46dcd4c095de tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:21.473829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5019337-92e7-4c41-9fcc-5282a36903ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:21.476530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5019337-92e7-4c41-9fcc-5282a36903ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:21.477145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5019337-92e7-4c41-9fcc-5282a36903ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:21.477906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5019337-92e7-4c41-9fcc-5282a36903ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:21.478383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5019337-92e7-4c41-9fcc-5282a36903ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:21.478817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5019337-92e7-4c41-9fcc-5282a36903ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:21.508281 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-95229c44-61ae-4eb7-9373-46dcd4c095de tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.713394 Sep 27 10:17:21.511431 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2072/4557] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:20 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 717 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:21.525973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb37befb-4639-4e16-b859-c12ed6270005 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:21.530618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fb37befb-4639-4e16-b859-c12ed6270005 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:21.647509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a5019337-92e7-4c41-9fcc-5282a36903ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:21.661526 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a5019337-92e7-4c41-9fcc-5282a36903ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1266 microversion: 2.1 time: 0.216634 Sep 27 10:17:21.668023 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2486/4558] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:21 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1266 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:21.751021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fb37befb-4639-4e16-b859-c12ed6270005 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:21.764126 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fb37befb-4639-4e16-b859-c12ed6270005 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1247 microversion: 2.1 time: 0.246709 Sep 27 10:17:21.767898 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2073/4559] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:21 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1247 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:21.826074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:21.827017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:21.841267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:21.842538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:21.845500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:21.847347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:21.848049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:21.849620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:22.280895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:22.297614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:22.299834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:22.302249 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-016b1e2d-f8a0-455f-ba7f-fd75d85d8f91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2953 microversion: 2.32 time: 0.483058 Sep 27 10:17:22.303164 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2487/4560] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:21 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2953 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:17:22.543886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e1d54586-be22-43d6-9cc4-655f1d71fc2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:22.545021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e1d54586-be22-43d6-9cc4-655f1d71fc2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:22.580260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1d54586-be22-43d6-9cc4-655f1d71fc2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:22.580808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1d54586-be22-43d6-9cc4-655f1d71fc2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:22.581259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1d54586-be22-43d6-9cc4-655f1d71fc2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:22.582037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1d54586-be22-43d6-9cc4-655f1d71fc2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:22.582464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1d54586-be22-43d6-9cc4-655f1d71fc2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:22.582888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e1d54586-be22-43d6-9cc4-655f1d71fc2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:22.690832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aea0a620-268f-402d-8dfd-c87fe2f6571a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:22.692019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aea0a620-268f-402d-8dfd-c87fe2f6571a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:22.700176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea0a620-268f-402d-8dfd-c87fe2f6571a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:22.705015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea0a620-268f-402d-8dfd-c87fe2f6571a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:22.709361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea0a620-268f-402d-8dfd-c87fe2f6571a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:22.710326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea0a620-268f-402d-8dfd-c87fe2f6571a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:22.710958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea0a620-268f-402d-8dfd-c87fe2f6571a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:22.711545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea0a620-268f-402d-8dfd-c87fe2f6571a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:23.030677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aea0a620-268f-402d-8dfd-c87fe2f6571a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:23.047273 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aea0a620-268f-402d-8dfd-c87fe2f6571a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.354812 Sep 27 10:17:23.051344 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2488/4561] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:22 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:23.067352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-98d45783-03f9-4482-a166-4b07427ee1c0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:23.068412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-98d45783-03f9-4482-a166-4b07427ee1c0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:23.077829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d45783-03f9-4482-a166-4b07427ee1c0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:23.079243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d45783-03f9-4482-a166-4b07427ee1c0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:23.082142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d45783-03f9-4482-a166-4b07427ee1c0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:23.083524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d45783-03f9-4482-a166-4b07427ee1c0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:23.086721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d45783-03f9-4482-a166-4b07427ee1c0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:23.087462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98d45783-03f9-4482-a166-4b07427ee1c0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:23.200245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-98d45783-03f9-4482-a166-4b07427ee1c0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:23.212372 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-98d45783-03f9-4482-a166-4b07427ee1c0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1299 microversion: 2.1 time: 0.156360 Sep 27 10:17:23.224692 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2489/4562] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:23 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1299 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:23.335608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:23.336229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:23.342944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:23.343629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:23.344241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:23.344988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:23.345437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:23.346204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:23.666853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e1d54586-be22-43d6-9cc4-655f1d71fc2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:23.676793 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e1d54586-be22-43d6-9cc4-655f1d71fc2f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 1.141804 Sep 27 10:17:23.682274 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2074/4563] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:22 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 1148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:23.794707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:23.825056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:23.832049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:23.835044 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58a0d3a8-3ea5-4b15-ab21-8ad59ca74a84 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2932 microversion: 2.32 time: 0.503525 Sep 27 10:17:23.848687 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2490/4564] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:23 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2932 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:17:24.052227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1be7dfbd-1a1d-4c24-9361-d74324604f9d req-c8ccc8ec-09d4-45dc-b7dc-cbf4ac748642 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-unplugged", "status": "completed", "tag": "86cc38ea-2a41-41be-a24f-432ba40420c5"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:24.063076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1be7dfbd-1a1d-4c24-9361-d74324604f9d req-c8ccc8ec-09d4-45dc-b7dc-cbf4ac748642 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:24.063622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1be7dfbd-1a1d-4c24-9361-d74324604f9d req-c8ccc8ec-09d4-45dc-b7dc-cbf4ac748642 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:24.064063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1be7dfbd-1a1d-4c24-9361-d74324604f9d req-c8ccc8ec-09d4-45dc-b7dc-cbf4ac748642 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:24.074258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29c5aa7a-dbd2-471b-a4c8-a72d81c02a2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:24.076819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-29c5aa7a-dbd2-471b-a4c8-a72d81c02a2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:24.079696 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1be7dfbd-1a1d-4c24-9361-d74324604f9d req-c8ccc8ec-09d4-45dc-b7dc-cbf4ac748642 service nova] Creating event network-vif-unplugged:86cc38ea-2a41-41be-a24f-432ba40420c5 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:17:24.084246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c5aa7a-dbd2-471b-a4c8-a72d81c02a2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:24.084877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c5aa7a-dbd2-471b-a4c8-a72d81c02a2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:24.085598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c5aa7a-dbd2-471b-a4c8-a72d81c02a2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:24.087045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c5aa7a-dbd2-471b-a4c8-a72d81c02a2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:24.087918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c5aa7a-dbd2-471b-a4c8-a72d81c02a2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:24.088034 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1be7dfbd-1a1d-4c24-9361-d74324604f9d req-c8ccc8ec-09d4-45dc-b7dc-cbf4ac748642 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.292564 Sep 27 10:17:24.088914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c5aa7a-dbd2-471b-a4c8-a72d81c02a2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:24.089029 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2075/4565] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:23 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:17:24.238761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c0df126-2867-47c8-a6e6-e50caaa7af98 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:24.239465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c0df126-2867-47c8-a6e6-e50caaa7af98 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:24.248148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c0df126-2867-47c8-a6e6-e50caaa7af98 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:24.249066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c0df126-2867-47c8-a6e6-e50caaa7af98 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:24.249665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c0df126-2867-47c8-a6e6-e50caaa7af98 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:24.250580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c0df126-2867-47c8-a6e6-e50caaa7af98 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:24.251237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c0df126-2867-47c8-a6e6-e50caaa7af98 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:24.251843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c0df126-2867-47c8-a6e6-e50caaa7af98 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:24.270079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29c5aa7a-dbd2-471b-a4c8-a72d81c02a2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:24.280000 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-29c5aa7a-dbd2-471b-a4c8-a72d81c02a2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.210014 Sep 27 10:17:24.281282 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2491/4566] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:24 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:24.421716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c0df126-2867-47c8-a6e6-e50caaa7af98 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:24.433296 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c0df126-2867-47c8-a6e6-e50caaa7af98 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1305 microversion: 2.1 time: 0.198677 Sep 27 10:17:24.434267 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2076/4567] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:24 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1305 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:24.704091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2ba9baaf-c967-4ea7-98fd-88ce45994a82 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:24.704900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2ba9baaf-c967-4ea7-98fd-88ce45994a82 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:24.712441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ba9baaf-c967-4ea7-98fd-88ce45994a82 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:24.712966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ba9baaf-c967-4ea7-98fd-88ce45994a82 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:24.713399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ba9baaf-c967-4ea7-98fd-88ce45994a82 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:24.714339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ba9baaf-c967-4ea7-98fd-88ce45994a82 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:24.714790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ba9baaf-c967-4ea7-98fd-88ce45994a82 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:24.715468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ba9baaf-c967-4ea7-98fd-88ce45994a82 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:24.869111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:24.870570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:24.881190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:24.882975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:24.883696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:24.884722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:24.885667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:24.887309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:25.031429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2ba9baaf-c967-4ea7-98fd-88ce45994a82 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:25.043124 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2ba9baaf-c967-4ea7-98fd-88ce45994a82 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.343237 Sep 27 10:17:25.045110 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2492/4568] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:24 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:25.306808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0224e606-019f-4112-9581-b5f0c830a656 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:25.307737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0224e606-019f-4112-9581-b5f0c830a656 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:25.320762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0224e606-019f-4112-9581-b5f0c830a656 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:25.321481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0224e606-019f-4112-9581-b5f0c830a656 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:25.323209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0224e606-019f-4112-9581-b5f0c830a656 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:25.326191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0224e606-019f-4112-9581-b5f0c830a656 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:25.330569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0224e606-019f-4112-9581-b5f0c830a656 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:25.331359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0224e606-019f-4112-9581-b5f0c830a656 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:25.598584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:25.632941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:25.635130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:25.637774 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a5afe69a-4dcd-4b95-85ff-943b6f7108f7 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2932 microversion: 2.32 time: 0.773468 Sep 27 10:17:25.640696 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2077/4569] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:24 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2932 bytes in 777 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:17:25.650917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-506e0c22-1313-4e89-8ca8-dcde115c569f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:25.653440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-506e0c22-1313-4e89-8ca8-dcde115c569f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:25.662583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-506e0c22-1313-4e89-8ca8-dcde115c569f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:25.663514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-506e0c22-1313-4e89-8ca8-dcde115c569f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:25.664172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-506e0c22-1313-4e89-8ca8-dcde115c569f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:25.665145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-506e0c22-1313-4e89-8ca8-dcde115c569f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:25.666764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-506e0c22-1313-4e89-8ca8-dcde115c569f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:25.667458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-506e0c22-1313-4e89-8ca8-dcde115c569f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:25.670274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0224e606-019f-4112-9581-b5f0c830a656 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:25.689207 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0224e606-019f-4112-9581-b5f0c830a656 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.387188 Sep 27 10:17:25.692374 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2493/4570] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:25 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:26.069572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-71341038-9e3f-4469-ad6e-fbcd225094c1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:26.071720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-71341038-9e3f-4469-ad6e-fbcd225094c1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:26.091474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71341038-9e3f-4469-ad6e-fbcd225094c1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:26.095645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71341038-9e3f-4469-ad6e-fbcd225094c1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:26.096109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71341038-9e3f-4469-ad6e-fbcd225094c1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:26.098586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71341038-9e3f-4469-ad6e-fbcd225094c1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:26.099070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71341038-9e3f-4469-ad6e-fbcd225094c1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:26.099652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71341038-9e3f-4469-ad6e-fbcd225094c1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:26.172781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-506e0c22-1313-4e89-8ca8-dcde115c569f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:26.182174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:26.191896 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-506e0c22-1313-4e89-8ca8-dcde115c569f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1305 microversion: 2.1 time: 0.542537 Sep 27 10:17:26.192896 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2078/4571] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:25 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1305 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:26.235807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-c15c6cab-61a9-46c8-8c53-5a7d01b68873 req-56f4a134-3cc9-4c50-825a-1024eabbf529 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-unplugged", "status": "completed", "tag": "78c9b09b-0886-456d-897a-ad2c11dd56c3"}, {"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-unplugged", "status": "completed", "tag": "0edf08ff-1fc9-48d8-be71-8d8e715e7c93"}, {"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-unplugged", "status": "completed", "tag": "10f31ce3-e11c-4b44-bb7f-173c2919d9b7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:26.291742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c15c6cab-61a9-46c8-8c53-5a7d01b68873 req-56f4a134-3cc9-4c50-825a-1024eabbf529 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:26.292228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c15c6cab-61a9-46c8-8c53-5a7d01b68873 req-56f4a134-3cc9-4c50-825a-1024eabbf529 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:26.292709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c15c6cab-61a9-46c8-8c53-5a7d01b68873 req-56f4a134-3cc9-4c50-825a-1024eabbf529 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:26.327764 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c15c6cab-61a9-46c8-8c53-5a7d01b68873 req-56f4a134-3cc9-4c50-825a-1024eabbf529 service nova] Creating event network-vif-unplugged:78c9b09b-0886-456d-897a-ad2c11dd56c3 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:17:26.328522 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c15c6cab-61a9-46c8-8c53-5a7d01b68873 req-56f4a134-3cc9-4c50-825a-1024eabbf529 service nova] Creating event network-vif-unplugged:0edf08ff-1fc9-48d8-be71-8d8e715e7c93 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:17:26.330325 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c15c6cab-61a9-46c8-8c53-5a7d01b68873 req-56f4a134-3cc9-4c50-825a-1024eabbf529 service nova] Creating event network-vif-unplugged:10f31ce3-e11c-4b44-bb7f-173c2919d9b7 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:17:26.341834 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-c15c6cab-61a9-46c8-8c53-5a7d01b68873 req-56f4a134-3cc9-4c50-825a-1024eabbf529 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 531 microversion: 2.1 time: 0.129925 Sep 27 10:17:26.342756 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2079/4572] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:26 2022] POST /compute/v2.1/os-server-external-events => generated 531 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:26.433263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-71341038-9e3f-4469-ad6e-fbcd225094c1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:26.448466 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-71341038-9e3f-4469-ad6e-fbcd225094c1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.382266 Sep 27 10:17:26.461443 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2494/4573] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:26 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:26.671023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:26.672533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:26.694125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:26.694863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:26.696752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:26.699313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:26.701680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:26.702139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:26.732234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-12586b02-4b79-49d0-8ffc-203abeb8dfce tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:26.733536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12586b02-4b79-49d0-8ffc-203abeb8dfce tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:26.751672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12586b02-4b79-49d0-8ffc-203abeb8dfce tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:26.754394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12586b02-4b79-49d0-8ffc-203abeb8dfce tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:26.756462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12586b02-4b79-49d0-8ffc-203abeb8dfce tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:26.757627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12586b02-4b79-49d0-8ffc-203abeb8dfce tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:26.758360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12586b02-4b79-49d0-8ffc-203abeb8dfce tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:26.760648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12586b02-4b79-49d0-8ffc-203abeb8dfce tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:27.001730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12586b02-4b79-49d0-8ffc-203abeb8dfce tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:27.013000 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-12586b02-4b79-49d0-8ffc-203abeb8dfce tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.292464 Sep 27 10:17:27.032980 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2495/4574] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:26 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:27.158013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0b844614-b19d-4187-a35b-cb61fc23f123 req-b1296bd9-9ca8-42f3-a28a-d44999c55ca7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dd547164-97da-4ba4-9cc3-b1448d2ab806", "tag": "44b8cebe-e56b-418e-a02d-05d82b186787"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:27.174963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0b844614-b19d-4187-a35b-cb61fc23f123 req-b1296bd9-9ca8-42f3-a28a-d44999c55ca7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:27.175766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0b844614-b19d-4187-a35b-cb61fc23f123 req-b1296bd9-9ca8-42f3-a28a-d44999c55ca7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:27.176574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0b844614-b19d-4187-a35b-cb61fc23f123 req-b1296bd9-9ca8-42f3-a28a-d44999c55ca7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:27.192752 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0b844614-b19d-4187-a35b-cb61fc23f123 req-b1296bd9-9ca8-42f3-a28a-d44999c55ca7 service nova] Creating event network-changed:44b8cebe-e56b-418e-a02d-05d82b186787 for instance dd547164-97da-4ba4-9cc3-b1448d2ab806 on nch2-master-1 Sep 27 10:17:27.213478 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0b844614-b19d-4187-a35b-cb61fc23f123 req-b1296bd9-9ca8-42f3-a28a-d44999c55ca7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062029 Sep 27 10:17:27.215237 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2496/4575] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:17:27.222057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-54917bbd-1adc-4672-b127-ba0b9688604e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:27.223031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-54917bbd-1adc-4672-b127-ba0b9688604e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:27.233071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54917bbd-1adc-4672-b127-ba0b9688604e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:27.233841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54917bbd-1adc-4672-b127-ba0b9688604e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:27.234547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54917bbd-1adc-4672-b127-ba0b9688604e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:27.235973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54917bbd-1adc-4672-b127-ba0b9688604e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:27.236637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54917bbd-1adc-4672-b127-ba0b9688604e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:27.237291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54917bbd-1adc-4672-b127-ba0b9688604e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:27.423830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:27.440905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:27.444374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:27.449227 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a17bdf4b-1685-4250-aad1-30fbd36c2764 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2911 microversion: 2.32 time: 0.785337 Sep 27 10:17:27.451986 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2080/4576] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:26 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2911 bytes in 789 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:17:27.476878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f035dd36-96a0-42d0-bc94-b220348df6d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:27.477797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f035dd36-96a0-42d0-bc94-b220348df6d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:27.486915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-54917bbd-1adc-4672-b127-ba0b9688604e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:27.488295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f035dd36-96a0-42d0-bc94-b220348df6d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:27.493082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f035dd36-96a0-42d0-bc94-b220348df6d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:27.493593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f035dd36-96a0-42d0-bc94-b220348df6d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:27.494353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f035dd36-96a0-42d0-bc94-b220348df6d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:27.495005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f035dd36-96a0-42d0-bc94-b220348df6d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:27.495594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f035dd36-96a0-42d0-bc94-b220348df6d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:27.501904 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-54917bbd-1adc-4672-b127-ba0b9688604e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1372 microversion: 2.1 time: 0.284658 Sep 27 10:17:27.502973 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2497/4577] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:27 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1372 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:28.048579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1a406a65-6ce6-42d4-9572-ffab6045e152 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:28.049243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1a406a65-6ce6-42d4-9572-ffab6045e152 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:28.065883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=1,nova_api:INSERT=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:28.067395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a406a65-6ce6-42d4-9572-ffab6045e152 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:28.069107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a406a65-6ce6-42d4-9572-ffab6045e152 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:28.069548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a406a65-6ce6-42d4-9572-ffab6045e152 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:28.085960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a406a65-6ce6-42d4-9572-ffab6045e152 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:28.087857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a406a65-6ce6-42d4-9572-ffab6045e152 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:28.090617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a406a65-6ce6-42d4-9572-ffab6045e152 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:28.091838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f035dd36-96a0-42d0-bc94-b220348df6d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:28.107553 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f035dd36-96a0-42d0-bc94-b220348df6d7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.632909 Sep 27 10:17:28.109986 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2081/4578] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:27 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:28.339931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1a406a65-6ce6-42d4-9572-ffab6045e152 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:28.374453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-256d7c80-a905-4ea2-8ffd-6e9bb45c537f req-198cb132-9488-49bf-9cba-b5fe7d278328 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-unplugged", "status": "completed", "tag": "4a6d853f-cf26-46e4-ad6f-27d3139a8f84"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:28.393071 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1a406a65-6ce6-42d4-9572-ffab6045e152 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.354734 Sep 27 10:17:28.395984 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2498/4579] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:28 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:28.419488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-256d7c80-a905-4ea2-8ffd-6e9bb45c537f req-198cb132-9488-49bf-9cba-b5fe7d278328 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:28.423581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-256d7c80-a905-4ea2-8ffd-6e9bb45c537f req-198cb132-9488-49bf-9cba-b5fe7d278328 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:28.424122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-256d7c80-a905-4ea2-8ffd-6e9bb45c537f req-198cb132-9488-49bf-9cba-b5fe7d278328 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:28.452456 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-256d7c80-a905-4ea2-8ffd-6e9bb45c537f req-198cb132-9488-49bf-9cba-b5fe7d278328 service nova] Creating event network-vif-unplugged:4a6d853f-cf26-46e4-ad6f-27d3139a8f84 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:17:28.472066 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-256d7c80-a905-4ea2-8ffd-6e9bb45c537f req-198cb132-9488-49bf-9cba-b5fe7d278328 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.102903 Sep 27 10:17:28.472710 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2082/4580] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:28 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:17:28.491383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:28.492415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:28.502255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:28.510397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:28.511392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:28.516571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:28.518295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:28.520047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:28.539985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e665fe01-ac61-4540-ac86-d9bf1ef645d9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:28.541001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e665fe01-ac61-4540-ac86-d9bf1ef645d9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:28.561638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e665fe01-ac61-4540-ac86-d9bf1ef645d9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:28.563522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e665fe01-ac61-4540-ac86-d9bf1ef645d9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:28.564976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e665fe01-ac61-4540-ac86-d9bf1ef645d9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:28.566932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e665fe01-ac61-4540-ac86-d9bf1ef645d9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:28.573218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e665fe01-ac61-4540-ac86-d9bf1ef645d9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:28.573719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e665fe01-ac61-4540-ac86-d9bf1ef645d9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:28.779933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:28.787812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:28.794318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:28.796324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:28.798632 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8bf401a8-53be-43aa-9914-d839e39a3076 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2865 microversion: 2.32 time: 0.314572 Sep 27 10:17:28.799294 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2499/4581] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:28 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2865 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:17:29.137571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-938f5a22-7030-46b3-984f-64733375e08c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:29.138463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-938f5a22-7030-46b3-984f-64733375e08c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:29.151964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-938f5a22-7030-46b3-984f-64733375e08c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:29.152582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-938f5a22-7030-46b3-984f-64733375e08c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:29.153437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-938f5a22-7030-46b3-984f-64733375e08c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:29.155281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-938f5a22-7030-46b3-984f-64733375e08c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:29.155965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-938f5a22-7030-46b3-984f-64733375e08c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:29.156638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-938f5a22-7030-46b3-984f-64733375e08c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:29.401432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-938f5a22-7030-46b3-984f-64733375e08c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:29.428252 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-938f5a22-7030-46b3-984f-64733375e08c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.295128 Sep 27 10:17:29.429597 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2500/4582] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:29 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:29.437443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e47aebfb-b17b-4be2-9c99-7fe224b67d63 req-13f16313-4489-4001-a906-148f7ec86e0c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-deleted", "tag": "4a6d853f-cf26-46e4-ad6f-27d3139a8f84"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:29.448402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e47aebfb-b17b-4be2-9c99-7fe224b67d63 req-13f16313-4489-4001-a906-148f7ec86e0c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:29.449013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e47aebfb-b17b-4be2-9c99-7fe224b67d63 req-13f16313-4489-4001-a906-148f7ec86e0c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:29.449741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e47aebfb-b17b-4be2-9c99-7fe224b67d63 req-13f16313-4489-4001-a906-148f7ec86e0c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:29.490226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e665fe01-ac61-4540-ac86-d9bf1ef645d9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:29.491317 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e47aebfb-b17b-4be2-9c99-7fe224b67d63 req-13f16313-4489-4001-a906-148f7ec86e0c service nova] Creating event network-vif-deleted:4a6d853f-cf26-46e4-ad6f-27d3139a8f84 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:17:29.505682 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e665fe01-ac61-4540-ac86-d9bf1ef645d9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1372 microversion: 2.1 time: 0.975265 Sep 27 10:17:29.506293 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2083/4583] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:28 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1372 bytes in 978 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:29.506694 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e47aebfb-b17b-4be2-9c99-7fe224b67d63 req-13f16313-4489-4001-a906-148f7ec86e0c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075513 Sep 27 10:17:29.507773 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2501/4584] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:29.526505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7d1a94c2-4aa4-482c-99f2-3f62524b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:29.527120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7d1a94c2-4aa4-482c-99f2-3f62524b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:29.533778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d1a94c2-4aa4-482c-99f2-3f62524b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:29.534517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d1a94c2-4aa4-482c-99f2-3f62524b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:29.535143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d1a94c2-4aa4-482c-99f2-3f62524b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:29.536359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d1a94c2-4aa4-482c-99f2-3f62524b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:29.542289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d1a94c2-4aa4-482c-99f2-3f62524b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:29.542845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d1a94c2-4aa4-482c-99f2-3f62524b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:29.833101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:29.833941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:29.844941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:29.845975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:29.846578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:29.847862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:29.848909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:29.850038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:29.989446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7d1a94c2-4aa4-482c-99f2-3f62524b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:29.998508 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7d1a94c2-4aa4-482c-99f2-3f62524b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.476038 Sep 27 10:17:29.999099 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2084/4585] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:29 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:30.142430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:30.156156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:30.161513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:30.177350 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4fcb627b-3333-467f-9a8b-fdd4599fdb9e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2751 microversion: 2.32 time: 0.351577 Sep 27 10:17:30.184987 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2502/4586] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:29 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2751 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:17:30.452044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-837e4f06-cae4-4e18-94ee-806a18f1ecc7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:30.452687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-837e4f06-cae4-4e18-94ee-806a18f1ecc7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:30.462393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-837e4f06-cae4-4e18-94ee-806a18f1ecc7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:30.465303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-837e4f06-cae4-4e18-94ee-806a18f1ecc7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:30.465850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-837e4f06-cae4-4e18-94ee-806a18f1ecc7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:30.466670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-837e4f06-cae4-4e18-94ee-806a18f1ecc7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:30.469835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-837e4f06-cae4-4e18-94ee-806a18f1ecc7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:30.470521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-837e4f06-cae4-4e18-94ee-806a18f1ecc7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:30.531625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-93b536bf-8dd8-41eb-bd92-2071e4fca7fd tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:30.533104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-93b536bf-8dd8-41eb-bd92-2071e4fca7fd tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:30.550947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93b536bf-8dd8-41eb-bd92-2071e4fca7fd tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:30.554012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93b536bf-8dd8-41eb-bd92-2071e4fca7fd tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:30.558234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93b536bf-8dd8-41eb-bd92-2071e4fca7fd tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:30.559256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93b536bf-8dd8-41eb-bd92-2071e4fca7fd tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:30.562641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93b536bf-8dd8-41eb-bd92-2071e4fca7fd tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:30.567028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93b536bf-8dd8-41eb-bd92-2071e4fca7fd tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:30.894462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-93b536bf-8dd8-41eb-bd92-2071e4fca7fd tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:30.914783 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-93b536bf-8dd8-41eb-bd92-2071e4fca7fd tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1372 microversion: 2.1 time: 0.386549 Sep 27 10:17:30.917614 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2503/4587] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:30 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1372 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:31.024029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-be8b3c24-3457-4b24-aaa1-92d6d7220368 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:31.024871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-be8b3c24-3457-4b24-aaa1-92d6d7220368 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:31.036307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be8b3c24-3457-4b24-aaa1-92d6d7220368 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:31.037343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be8b3c24-3457-4b24-aaa1-92d6d7220368 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:31.038253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be8b3c24-3457-4b24-aaa1-92d6d7220368 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:31.039027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be8b3c24-3457-4b24-aaa1-92d6d7220368 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:31.041523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be8b3c24-3457-4b24-aaa1-92d6d7220368 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:31.042302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be8b3c24-3457-4b24-aaa1-92d6d7220368 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:31.149345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-837e4f06-cae4-4e18-94ee-806a18f1ecc7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:31.158135 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-837e4f06-cae4-4e18-94ee-806a18f1ecc7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1515 microversion: 2.1 time: 0.710392 Sep 27 10:17:31.177083 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2085/4588] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:30 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1515 bytes in 727 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:31.218149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:31.218984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:31.230745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:31.233472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:31.233995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:31.234945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:31.237665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:31.238372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:31.271724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:31.296274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-be8b3c24-3457-4b24-aaa1-92d6d7220368 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:31.309545 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-be8b3c24-3457-4b24-aaa1-92d6d7220368 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.290234 Sep 27 10:17:31.310433 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2504/4589] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:31 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:31.443091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:31.455223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:31.457088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:31.459237 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b2a634f-e969-4120-be87-77e14ea8c497 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2705 microversion: 2.32 time: 0.248285 Sep 27 10:17:31.460512 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2086/4590] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:31 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2705 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:17:31.585435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-b186901a-f435-4703-a33b-d7a26c4acd7b req-7ff589ea-5f53-4b7e-b05c-46ac0ec2a068 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f0d9e453-3f14-4cd2-b7a7-31a47762c436", "tag": "e1e18a0a-7b42-40a1-b724-3c904a45faa3"}, {"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-deleted", "tag": "10f31ce3-e11c-4b44-bb7f-173c2919d9b7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:31.596962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=16 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:31.608573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b186901a-f435-4703-a33b-d7a26c4acd7b req-7ff589ea-5f53-4b7e-b05c-46ac0ec2a068 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:31.615130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b186901a-f435-4703-a33b-d7a26c4acd7b req-7ff589ea-5f53-4b7e-b05c-46ac0ec2a068 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:31.616063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b186901a-f435-4703-a33b-d7a26c4acd7b req-7ff589ea-5f53-4b7e-b05c-46ac0ec2a068 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:31.650616 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b186901a-f435-4703-a33b-d7a26c4acd7b req-7ff589ea-5f53-4b7e-b05c-46ac0ec2a068 service nova] Creating event network-changed:e1e18a0a-7b42-40a1-b724-3c904a45faa3 for instance f0d9e453-3f14-4cd2-b7a7-31a47762c436 on nch2-master-1 Sep 27 10:17:31.652876 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b186901a-f435-4703-a33b-d7a26c4acd7b req-7ff589ea-5f53-4b7e-b05c-46ac0ec2a068 service nova] Creating event network-vif-deleted:10f31ce3-e11c-4b44-bb7f-173c2919d9b7 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:17:31.667176 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-b186901a-f435-4703-a33b-d7a26c4acd7b req-7ff589ea-5f53-4b7e-b05c-46ac0ec2a068 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 350 microversion: 2.1 time: 0.085914 Sep 27 10:17:31.682099 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2505/4591] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:31 2022] POST /compute/v2.1/os-server-external-events => generated 350 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:31.939667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9029428a-0eb1-46e8-8bb6-67de4c1244a6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:31.940808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9029428a-0eb1-46e8-8bb6-67de4c1244a6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:31.952950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:31.958780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9029428a-0eb1-46e8-8bb6-67de4c1244a6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:31.960393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9029428a-0eb1-46e8-8bb6-67de4c1244a6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:31.963254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9029428a-0eb1-46e8-8bb6-67de4c1244a6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:31.965606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9029428a-0eb1-46e8-8bb6-67de4c1244a6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:31.966324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9029428a-0eb1-46e8-8bb6-67de4c1244a6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:31.968128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9029428a-0eb1-46e8-8bb6-67de4c1244a6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:32.188823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c0500ec1-61ae-4229-b26b-c3d4b25de8bc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:32.189729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c0500ec1-61ae-4229-b26b-c3d4b25de8bc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:32.197383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0500ec1-61ae-4229-b26b-c3d4b25de8bc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:32.199571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0500ec1-61ae-4229-b26b-c3d4b25de8bc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:32.200211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0500ec1-61ae-4229-b26b-c3d4b25de8bc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:32.204045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0500ec1-61ae-4229-b26b-c3d4b25de8bc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:32.204539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0500ec1-61ae-4229-b26b-c3d4b25de8bc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:32.205010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0500ec1-61ae-4229-b26b-c3d4b25de8bc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:32.395936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c0500ec1-61ae-4229-b26b-c3d4b25de8bc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:32.407951 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c0500ec1-61ae-4229-b26b-c3d4b25de8bc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1521 microversion: 2.1 time: 0.226817 Sep 27 10:17:32.408558 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2506/4592] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:32 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1521 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:32.416335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e418cb2d-478e-4a5d-b3eb-740dd2a6232a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:32.418616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e418cb2d-478e-4a5d-b3eb-740dd2a6232a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:32.433259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418cb2d-478e-4a5d-b3eb-740dd2a6232a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:32.433835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418cb2d-478e-4a5d-b3eb-740dd2a6232a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:32.434371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418cb2d-478e-4a5d-b3eb-740dd2a6232a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:32.435157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418cb2d-478e-4a5d-b3eb-740dd2a6232a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:32.435664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418cb2d-478e-4a5d-b3eb-740dd2a6232a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:32.436097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e418cb2d-478e-4a5d-b3eb-740dd2a6232a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:32.701523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e418cb2d-478e-4a5d-b3eb-740dd2a6232a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:32.715514 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e418cb2d-478e-4a5d-b3eb-740dd2a6232a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.305609 Sep 27 10:17:32.716125 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2507/4593] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:32 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:32.726245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:32.727399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:32.735730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:32.736440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:32.737342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:32.738766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:32.740011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:32.741597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:33.007277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9029428a-0eb1-46e8-8bb6-67de4c1244a6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:33.015719 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9029428a-0eb1-46e8-8bb6-67de4c1244a6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1372 microversion: 2.1 time: 1.080887 Sep 27 10:17:33.031174 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2087/4594] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:31 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1372 bytes in 1097 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:33.060240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:33.070216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:33.072361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:33.073830 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-690fdd94-e187-4e3c-827c-9e64b75d58de tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2705 microversion: 2.32 time: 0.356538 Sep 27 10:17:33.097306 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2508/4595] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:32 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2705 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:17:33.432887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-35c27fa8-9bb4-47a4-afd0-360cdca11340 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:33.433543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-35c27fa8-9bb4-47a4-afd0-360cdca11340 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:33.441624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35c27fa8-9bb4-47a4-afd0-360cdca11340 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:33.442136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35c27fa8-9bb4-47a4-afd0-360cdca11340 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:33.442572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35c27fa8-9bb4-47a4-afd0-360cdca11340 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:33.446154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35c27fa8-9bb4-47a4-afd0-360cdca11340 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:33.447047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35c27fa8-9bb4-47a4-afd0-360cdca11340 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:33.448612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35c27fa8-9bb4-47a4-afd0-360cdca11340 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:33.742243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f16e124d-bb3a-4cb1-9a2c-8cb198ae9934 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:33.743626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f16e124d-bb3a-4cb1-9a2c-8cb198ae9934 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:33.762668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f16e124d-bb3a-4cb1-9a2c-8cb198ae9934 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:33.763679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f16e124d-bb3a-4cb1-9a2c-8cb198ae9934 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:33.764348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f16e124d-bb3a-4cb1-9a2c-8cb198ae9934 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:33.765356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f16e124d-bb3a-4cb1-9a2c-8cb198ae9934 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:33.766890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f16e124d-bb3a-4cb1-9a2c-8cb198ae9934 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:33.767721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f16e124d-bb3a-4cb1-9a2c-8cb198ae9934 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:34.090849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f16e124d-bb3a-4cb1-9a2c-8cb198ae9934 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:34.103091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f16e124d-bb3a-4cb1-9a2c-8cb198ae9934 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.365162 Sep 27 10:17:34.104012 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2509/4596] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:33 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:34.126862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-164ffd5c-5a0f-47e4-b3a4-dc4a5d8ade00 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:34.127506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-164ffd5c-5a0f-47e4-b3a4-dc4a5d8ade00 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:34.154498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-164ffd5c-5a0f-47e4-b3a4-dc4a5d8ade00 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:34.155000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-164ffd5c-5a0f-47e4-b3a4-dc4a5d8ade00 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:34.155445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-164ffd5c-5a0f-47e4-b3a4-dc4a5d8ade00 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:34.156156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-164ffd5c-5a0f-47e4-b3a4-dc4a5d8ade00 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:34.156616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-164ffd5c-5a0f-47e4-b3a4-dc4a5d8ade00 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:34.159303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-164ffd5c-5a0f-47e4-b3a4-dc4a5d8ade00 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:34.269694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-35c27fa8-9bb4-47a4-afd0-360cdca11340 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:34.279803 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-35c27fa8-9bb4-47a4-afd0-360cdca11340 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1521 microversion: 2.1 time: 0.851305 Sep 27 10:17:34.284430 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2088/4597] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:33 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1521 bytes in 857 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:34.289350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:34.290048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:34.297566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:34.298214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:34.298830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:34.299580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:34.300038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:34.300431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:34.430617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-164ffd5c-5a0f-47e4-b3a4-dc4a5d8ade00 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:34.441855 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-164ffd5c-5a0f-47e4-b3a4-dc4a5d8ade00 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1372 microversion: 2.1 time: 0.331197 Sep 27 10:17:34.456828 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2510/4598] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:34 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1372 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:34.465070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-dd4f36de-bb59-47e7-a8bf-8f5cc7ea71ef req-d2e52809-c980-4829-bd24-ebaef9dab1b8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-deleted", "tag": "0edf08ff-1fc9-48d8-be71-8d8e715e7c93"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:34.483588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-dd4f36de-bb59-47e7-a8bf-8f5cc7ea71ef req-d2e52809-c980-4829-bd24-ebaef9dab1b8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:34.484348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-dd4f36de-bb59-47e7-a8bf-8f5cc7ea71ef req-d2e52809-c980-4829-bd24-ebaef9dab1b8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:34.485024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-dd4f36de-bb59-47e7-a8bf-8f5cc7ea71ef req-d2e52809-c980-4829-bd24-ebaef9dab1b8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:34.502389 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-dd4f36de-bb59-47e7-a8bf-8f5cc7ea71ef req-d2e52809-c980-4829-bd24-ebaef9dab1b8 service nova] Creating event network-vif-deleted:0edf08ff-1fc9-48d8-be71-8d8e715e7c93 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:17:34.509458 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-dd4f36de-bb59-47e7-a8bf-8f5cc7ea71ef req-d2e52809-c980-4829-bd24-ebaef9dab1b8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062874 Sep 27 10:17:34.510735 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2511/4599] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:34.595441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:34.625454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:34.626550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:34.629767 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-da4214ab-eadb-42e9-81a5-af730a8cd337 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2659 microversion: 2.32 time: 0.344309 Sep 27 10:17:34.630354 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2089/4600] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:34 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2659 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:17:34.845841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7920a811-ec22-4d8e-924a-44fb01047f31 req-a3952bee-de83-48a6-b29a-87f85898a858 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e5de27fc-1301-4e5a-ad28-2cd333ab9d6e", "name": "network-vif-unplugged", "status": "completed", "tag": "3634b03e-3d6f-45bc-8360-05f7f1e1a4fb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:34.870884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7920a811-ec22-4d8e-924a-44fb01047f31 req-a3952bee-de83-48a6-b29a-87f85898a858 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:34.873227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7920a811-ec22-4d8e-924a-44fb01047f31 req-a3952bee-de83-48a6-b29a-87f85898a858 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:34.874243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7920a811-ec22-4d8e-924a-44fb01047f31 req-a3952bee-de83-48a6-b29a-87f85898a858 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:34.897221 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7920a811-ec22-4d8e-924a-44fb01047f31 req-a3952bee-de83-48a6-b29a-87f85898a858 service nova] Creating event network-vif-unplugged:3634b03e-3d6f-45bc-8360-05f7f1e1a4fb for instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e on nch1-master-1 Sep 27 10:17:34.921627 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7920a811-ec22-4d8e-924a-44fb01047f31 req-a3952bee-de83-48a6-b29a-87f85898a858 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.081396 Sep 27 10:17:34.922261 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2512/4601] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:34 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:35.129810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba0623e4-8e60-464e-81b1-3c6ea2ed1d6e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:35.130423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ba0623e4-8e60-464e-81b1-3c6ea2ed1d6e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:35.141382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba0623e4-8e60-464e-81b1-3c6ea2ed1d6e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:35.145480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba0623e4-8e60-464e-81b1-3c6ea2ed1d6e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:35.145941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba0623e4-8e60-464e-81b1-3c6ea2ed1d6e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:35.146772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba0623e4-8e60-464e-81b1-3c6ea2ed1d6e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:35.147247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba0623e4-8e60-464e-81b1-3c6ea2ed1d6e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:35.147680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ba0623e4-8e60-464e-81b1-3c6ea2ed1d6e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:35.314543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-63e74535-ea99-4745-b612-c87b5cbc4eaa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:35.315152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-63e74535-ea99-4745-b612-c87b5cbc4eaa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:35.324593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63e74535-ea99-4745-b612-c87b5cbc4eaa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:35.325109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63e74535-ea99-4745-b612-c87b5cbc4eaa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:35.328580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63e74535-ea99-4745-b612-c87b5cbc4eaa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:35.329352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63e74535-ea99-4745-b612-c87b5cbc4eaa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:35.329822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63e74535-ea99-4745-b612-c87b5cbc4eaa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:35.336751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63e74535-ea99-4745-b612-c87b5cbc4eaa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:35.430668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ba0623e4-8e60-464e-81b1-3c6ea2ed1d6e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:35.439024 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ba0623e4-8e60-464e-81b1-3c6ea2ed1d6e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.314291 Sep 27 10:17:35.439608 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2090/4602] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:35 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:35.467596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0c6c5560-c93e-4015-8f12-f219712ba26b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:35.468375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0c6c5560-c93e-4015-8f12-f219712ba26b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:35.475375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c6c5560-c93e-4015-8f12-f219712ba26b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:35.475880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c6c5560-c93e-4015-8f12-f219712ba26b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:35.476321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c6c5560-c93e-4015-8f12-f219712ba26b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:35.477123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c6c5560-c93e-4015-8f12-f219712ba26b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:35.477583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c6c5560-c93e-4015-8f12-f219712ba26b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:35.478007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c6c5560-c93e-4015-8f12-f219712ba26b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:35.634684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-63e74535-ea99-4745-b612-c87b5cbc4eaa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:35.648361 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-63e74535-ea99-4745-b612-c87b5cbc4eaa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1521 microversion: 2.1 time: 0.338163 Sep 27 10:17:35.649011 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2513/4603] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:35 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1521 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:35.661983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:35.663022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:35.668438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c6c5560-c93e-4015-8f12-f219712ba26b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:35.673973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:35.674510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:35.674950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:35.678195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:35.678688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:35.679136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:35.679262 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0c6c5560-c93e-4015-8f12-f219712ba26b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1372 microversion: 2.1 time: 0.216088 Sep 27 10:17:35.680256 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2091/4604] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:35 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1372 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:35.881147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:35.892345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:35.894578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:35.895957 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c48f182b-c411-4894-b93d-d84276667b8e tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2659 microversion: 2.32 time: 0.237911 Sep 27 10:17:35.896761 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2514/4605] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:35 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2659 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:17:36.463496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3826b32f-ca6a-4f52-b8c3-2866d7b3930a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:36.464128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3826b32f-ca6a-4f52-b8c3-2866d7b3930a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:36.476227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3826b32f-ca6a-4f52-b8c3-2866d7b3930a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:36.476943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3826b32f-ca6a-4f52-b8c3-2866d7b3930a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:36.492729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3826b32f-ca6a-4f52-b8c3-2866d7b3930a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:36.492729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3826b32f-ca6a-4f52-b8c3-2866d7b3930a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:36.492729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3826b32f-ca6a-4f52-b8c3-2866d7b3930a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:36.492729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3826b32f-ca6a-4f52-b8c3-2866d7b3930a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:36.676349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4b1888e5-df37-4b6a-8aa0-cc5d92bb6ff5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:36.677011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4b1888e5-df37-4b6a-8aa0-cc5d92bb6ff5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:36.691795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b1888e5-df37-4b6a-8aa0-cc5d92bb6ff5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:36.698160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b1888e5-df37-4b6a-8aa0-cc5d92bb6ff5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:36.699119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b1888e5-df37-4b6a-8aa0-cc5d92bb6ff5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:36.701793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b1888e5-df37-4b6a-8aa0-cc5d92bb6ff5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:36.704208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b1888e5-df37-4b6a-8aa0-cc5d92bb6ff5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:36.712155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b1888e5-df37-4b6a-8aa0-cc5d92bb6ff5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:36.872436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3826b32f-ca6a-4f52-b8c3-2866d7b3930a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:36.895565 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3826b32f-ca6a-4f52-b8c3-2866d7b3930a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.435882 Sep 27 10:17:36.898303 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2092/4606] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:36 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:36.904165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-295eee4b-5f77-4ad3-a5ed-39243500da6e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:36.904881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-295eee4b-5f77-4ad3-a5ed-39243500da6e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:36.914894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-295eee4b-5f77-4ad3-a5ed-39243500da6e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:36.915762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-295eee4b-5f77-4ad3-a5ed-39243500da6e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:36.916416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-295eee4b-5f77-4ad3-a5ed-39243500da6e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:36.917501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-295eee4b-5f77-4ad3-a5ed-39243500da6e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:36.919926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-295eee4b-5f77-4ad3-a5ed-39243500da6e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:36.920524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-295eee4b-5f77-4ad3-a5ed-39243500da6e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:36.950245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4b1888e5-df37-4b6a-8aa0-cc5d92bb6ff5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:36.964628 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4b1888e5-df37-4b6a-8aa0-cc5d92bb6ff5 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1521 microversion: 2.1 time: 0.292778 Sep 27 10:17:36.965681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2515/4607] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:36 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1521 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:36.972186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:36.976801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:36.984426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:36.985481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:36.986096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:36.987611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:36.988368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:36.988982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:37.171962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:37.184476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:37.187474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:37.190162 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e49553d0-0499-4cba-a8a9-19fe0d9c9f1d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2546 microversion: 2.32 time: 0.223426 Sep 27 10:17:37.191002 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2516/4608] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:36 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2546 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:17:37.227642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-65c45e82-20f5-4786-9589-ef73b1b042d2 req-5f34bc9b-5401-4f40-9b8f-f9f3b5194450 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd547164-97da-4ba4-9cc3-b1448d2ab806", "name": "network-vif-plugged", "status": "completed", "tag": "44b8cebe-e56b-418e-a02d-05d82b186787"}, {"server_uuid": "e5de27fc-1301-4e5a-ad28-2cd333ab9d6e", "name": "network-vif-plugged", "status": "completed", "tag": "3634b03e-3d6f-45bc-8360-05f7f1e1a4fb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:37.242154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-65c45e82-20f5-4786-9589-ef73b1b042d2 req-5f34bc9b-5401-4f40-9b8f-f9f3b5194450 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:37.242977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-65c45e82-20f5-4786-9589-ef73b1b042d2 req-5f34bc9b-5401-4f40-9b8f-f9f3b5194450 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:37.243598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-65c45e82-20f5-4786-9589-ef73b1b042d2 req-5f34bc9b-5401-4f40-9b8f-f9f3b5194450 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:37.270843 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-65c45e82-20f5-4786-9589-ef73b1b042d2 req-5f34bc9b-5401-4f40-9b8f-f9f3b5194450 service nova] Creating event network-vif-plugged:44b8cebe-e56b-418e-a02d-05d82b186787 for instance dd547164-97da-4ba4-9cc3-b1448d2ab806 on nch2-master-1 Sep 27 10:17:37.272411 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-65c45e82-20f5-4786-9589-ef73b1b042d2 req-5f34bc9b-5401-4f40-9b8f-f9f3b5194450 service nova] Creating event network-vif-plugged:3634b03e-3d6f-45bc-8360-05f7f1e1a4fb for instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e on nch1-master-1 Sep 27 10:17:37.292636 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-65c45e82-20f5-4786-9589-ef73b1b042d2 req-5f34bc9b-5401-4f40-9b8f-f9f3b5194450 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.088161 Sep 27 10:17:37.293280 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2517/4609] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:37 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:37.350653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-295eee4b-5f77-4ad3-a5ed-39243500da6e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:37.384753 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-295eee4b-5f77-4ad3-a5ed-39243500da6e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1372 microversion: 2.1 time: 0.482664 Sep 27 10:17:37.389176 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2093/4610] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:36 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1372 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:37.416809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-63a1103e-b07a-4ec0-8a8d-fc0231fc1e81 req-ec789a16-149c-4baa-89c2-8e762ba1a203 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-deleted", "tag": "86cc38ea-2a41-41be-a24f-432ba40420c5"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:37.430664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-63a1103e-b07a-4ec0-8a8d-fc0231fc1e81 req-ec789a16-149c-4baa-89c2-8e762ba1a203 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:37.431384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-63a1103e-b07a-4ec0-8a8d-fc0231fc1e81 req-ec789a16-149c-4baa-89c2-8e762ba1a203 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:37.431989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-63a1103e-b07a-4ec0-8a8d-fc0231fc1e81 req-ec789a16-149c-4baa-89c2-8e762ba1a203 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:37.448060 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-63a1103e-b07a-4ec0-8a8d-fc0231fc1e81 req-ec789a16-149c-4baa-89c2-8e762ba1a203 service nova] Creating event network-vif-deleted:86cc38ea-2a41-41be-a24f-432ba40420c5 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:17:37.455448 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-63a1103e-b07a-4ec0-8a8d-fc0231fc1e81 req-ec789a16-149c-4baa-89c2-8e762ba1a203 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043502 Sep 27 10:17:37.456075 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2518/4611] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:17:37.923514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ae1cbd07-d0ae-4d37-ae0c-e5e176a49d5d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:37.924087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ae1cbd07-d0ae-4d37-ae0c-e5e176a49d5d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:37.937638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae1cbd07-d0ae-4d37-ae0c-e5e176a49d5d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:37.938399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae1cbd07-d0ae-4d37-ae0c-e5e176a49d5d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:37.939090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae1cbd07-d0ae-4d37-ae0c-e5e176a49d5d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:37.941603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae1cbd07-d0ae-4d37-ae0c-e5e176a49d5d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:37.942306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae1cbd07-d0ae-4d37-ae0c-e5e176a49d5d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:37.944293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae1cbd07-d0ae-4d37-ae0c-e5e176a49d5d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:38.002848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-05fe92da-5a40-4896-bfcb-35d08cc286f9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:38.002848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-05fe92da-5a40-4896-bfcb-35d08cc286f9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:38.009822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05fe92da-5a40-4896-bfcb-35d08cc286f9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:38.015501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05fe92da-5a40-4896-bfcb-35d08cc286f9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:38.024142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05fe92da-5a40-4896-bfcb-35d08cc286f9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:38.025746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05fe92da-5a40-4896-bfcb-35d08cc286f9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:38.025986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05fe92da-5a40-4896-bfcb-35d08cc286f9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:38.027950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05fe92da-5a40-4896-bfcb-35d08cc286f9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:38.332681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-05fe92da-5a40-4896-bfcb-35d08cc286f9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:38.342714 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-05fe92da-5a40-4896-bfcb-35d08cc286f9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1521 microversion: 2.1 time: 0.353954 Sep 27 10:17:38.345393 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2519/4612] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:37 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1521 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:38.365460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:38.368383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:38.375851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:38.377664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:38.378359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:38.381552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:38.382294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:38.382831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:38.419931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:38.472896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ae1cbd07-d0ae-4d37-ae0c-e5e176a49d5d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:38.485305 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ae1cbd07-d0ae-4d37-ae0c-e5e176a49d5d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.563206 Sep 27 10:17:38.486087 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2094/4613] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:37 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:38.493968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bcef38eb-6163-4ecf-8606-eb84c32cd5da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:38.497415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bcef38eb-6163-4ecf-8606-eb84c32cd5da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:38.515035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcef38eb-6163-4ecf-8606-eb84c32cd5da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:38.515780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcef38eb-6163-4ecf-8606-eb84c32cd5da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:38.516466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcef38eb-6163-4ecf-8606-eb84c32cd5da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:38.517399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcef38eb-6163-4ecf-8606-eb84c32cd5da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:38.518185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcef38eb-6163-4ecf-8606-eb84c32cd5da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:38.518794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bcef38eb-6163-4ecf-8606-eb84c32cd5da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:38.647565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:38.680220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:38.682043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:38.683425 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c1f37e0-ce9f-422f-93a4-ebe2f7e920bb tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 2326 microversion: 2.32 time: 0.328032 Sep 27 10:17:38.684879 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2520/4614] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:38 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 2326 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:17:38.762463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bcef38eb-6163-4ecf-8606-eb84c32cd5da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:38.774742 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bcef38eb-6163-4ecf-8606-eb84c32cd5da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1372 microversion: 2.1 time: 0.286513 Sep 27 10:17:38.775823 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2095/4615] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:38 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1372 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:39.374474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da281e01-2152-411e-896d-c0dd83fd8af9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:39.375097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-da281e01-2152-411e-896d-c0dd83fd8af9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:39.387083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da281e01-2152-411e-896d-c0dd83fd8af9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:39.387575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da281e01-2152-411e-896d-c0dd83fd8af9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:39.393484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da281e01-2152-411e-896d-c0dd83fd8af9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:39.394239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da281e01-2152-411e-896d-c0dd83fd8af9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:39.394714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da281e01-2152-411e-896d-c0dd83fd8af9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:39.395141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da281e01-2152-411e-896d-c0dd83fd8af9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:39.475858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ed1539d3-1185-43cc-adbf-65703585a232 req-0f2c9a11-7b12-4813-ac65-ddc8dd2601aa service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0aaf3916-68ea-45f6-96a9-fc8d8addfedd", "name": "network-vif-deleted", "tag": "78c9b09b-0886-456d-897a-ad2c11dd56c3"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:39.494488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ed1539d3-1185-43cc-adbf-65703585a232 req-0f2c9a11-7b12-4813-ac65-ddc8dd2601aa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:39.495374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ed1539d3-1185-43cc-adbf-65703585a232 req-0f2c9a11-7b12-4813-ac65-ddc8dd2601aa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:39.496090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ed1539d3-1185-43cc-adbf-65703585a232 req-0f2c9a11-7b12-4813-ac65-ddc8dd2601aa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:39.536379 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ed1539d3-1185-43cc-adbf-65703585a232 req-0f2c9a11-7b12-4813-ac65-ddc8dd2601aa service nova] Creating event network-vif-deleted:78c9b09b-0886-456d-897a-ad2c11dd56c3 for instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd on nch2-master-1 Sep 27 10:17:39.557505 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ed1539d3-1185-43cc-adbf-65703585a232 req-0f2c9a11-7b12-4813-ac65-ddc8dd2601aa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.086271 Sep 27 10:17:39.560834 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2096/4616] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:39.578583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-40a9e3f0-dcd4-475b-9a06-7c5a1791a6d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:39.580792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-40a9e3f0-dcd4-475b-9a06-7c5a1791a6d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:39.605375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a9e3f0-dcd4-475b-9a06-7c5a1791a6d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:39.605886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a9e3f0-dcd4-475b-9a06-7c5a1791a6d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:39.606330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a9e3f0-dcd4-475b-9a06-7c5a1791a6d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:39.607074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a9e3f0-dcd4-475b-9a06-7c5a1791a6d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:39.612553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a9e3f0-dcd4-475b-9a06-7c5a1791a6d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:39.614156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a9e3f0-dcd4-475b-9a06-7c5a1791a6d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:39.638331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-da281e01-2152-411e-896d-c0dd83fd8af9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:39.647731 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da281e01-2152-411e-896d-c0dd83fd8af9 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1521 microversion: 2.1 time: 0.280214 Sep 27 10:17:39.653405 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2521/4617] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:39 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1521 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:39.713987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:39.714740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:39.723016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:39.724368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:39.726429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:39.727312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:39.728245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:39.728972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:39.732891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:39.827213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:39.840363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:39.843047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:39.845906 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f9efb245-e4bd-4201-b50f-0e500efb0c91 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1605 microversion: 2.32 time: 0.136800 Sep 27 10:17:39.847320 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2522/4618] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:39 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1605 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:17:39.853245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dc41fbab-2795-4c27-b590-6556037fcde4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:39.854235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dc41fbab-2795-4c27-b590-6556037fcde4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:39.861424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc41fbab-2795-4c27-b590-6556037fcde4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:39.862438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc41fbab-2795-4c27-b590-6556037fcde4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:39.863191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc41fbab-2795-4c27-b590-6556037fcde4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:39.864214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc41fbab-2795-4c27-b590-6556037fcde4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:39.865117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc41fbab-2795-4c27-b590-6556037fcde4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:39.866112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc41fbab-2795-4c27-b590-6556037fcde4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:40.133601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-40a9e3f0-dcd4-475b-9a06-7c5a1791a6d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:40.144705 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-40a9e3f0-dcd4-475b-9a06-7c5a1791a6d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1308 microversion: 2.1 time: 0.580490 Sep 27 10:17:40.150872 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2097/4619] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:39 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1308 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:40.159150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bece8739-b68b-43c5-8d93-231661de9a54 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:40.164206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bece8739-b68b-43c5-8d93-231661de9a54 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:40.174313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bece8739-b68b-43c5-8d93-231661de9a54 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:40.174919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bece8739-b68b-43c5-8d93-231661de9a54 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:40.175068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dc41fbab-2795-4c27-b590-6556037fcde4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:40.175555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bece8739-b68b-43c5-8d93-231661de9a54 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:40.176409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bece8739-b68b-43c5-8d93-231661de9a54 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:40.176979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bece8739-b68b-43c5-8d93-231661de9a54 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:40.177551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bece8739-b68b-43c5-8d93-231661de9a54 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:40.212827 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dc41fbab-2795-4c27-b590-6556037fcde4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1372 microversion: 2.1 time: 0.361078 Sep 27 10:17:40.215255 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2523/4620] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:39 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1372 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:40.238063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-09f69f46-91a6-4194-adb4-f5cdb43323e2 req-079916af-df4a-48e9-b6b4-10cc2fafc2bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f0d9e453-3f14-4cd2-b7a7-31a47762c436", "name": "network-vif-plugged", "status": "completed", "tag": "e1e18a0a-7b42-40a1-b724-3c904a45faa3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:40.253977 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bece8739-b68b-43c5-8d93-231661de9a54 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/ips" status: 200 len: 145 microversion: 2.1 time: 0.099858 Sep 27 10:17:40.254997 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2098/4621] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:17:40 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/ips => generated 145 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:40.286631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-09f69f46-91a6-4194-adb4-f5cdb43323e2 req-079916af-df4a-48e9-b6b4-10cc2fafc2bc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:40.293985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-09f69f46-91a6-4194-adb4-f5cdb43323e2 req-079916af-df4a-48e9-b6b4-10cc2fafc2bc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:40.294556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-09f69f46-91a6-4194-adb4-f5cdb43323e2 req-079916af-df4a-48e9-b6b4-10cc2fafc2bc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:40.335837 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-09f69f46-91a6-4194-adb4-f5cdb43323e2 req-079916af-df4a-48e9-b6b4-10cc2fafc2bc service nova] Creating event network-vif-plugged:e1e18a0a-7b42-40a1-b724-3c904a45faa3 for instance f0d9e453-3f14-4cd2-b7a7-31a47762c436 on nch2-master-1 Sep 27 10:17:40.342987 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-09f69f46-91a6-4194-adb4-f5cdb43323e2 req-079916af-df4a-48e9-b6b4-10cc2fafc2bc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.122818 Sep 27 10:17:40.343837 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2524/4622] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:40.682469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc0cb9cc-d1cc-466a-b4a8-ec6750bbf1e6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:40.683550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc0cb9cc-d1cc-466a-b4a8-ec6750bbf1e6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:40.707127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0cb9cc-d1cc-466a-b4a8-ec6750bbf1e6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:40.707560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0cb9cc-d1cc-466a-b4a8-ec6750bbf1e6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:40.708005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0cb9cc-d1cc-466a-b4a8-ec6750bbf1e6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:40.708863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0cb9cc-d1cc-466a-b4a8-ec6750bbf1e6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:40.709454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0cb9cc-d1cc-466a-b4a8-ec6750bbf1e6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:40.709918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc0cb9cc-d1cc-466a-b4a8-ec6750bbf1e6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:40.879472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:40.880193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:40.891394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:40.891889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:40.892336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:40.893171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:40.893631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:40.894213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:40.942956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:40.944413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:40.948779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:40.949790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lazy-loading 'fault' on Instance uuid 0aaf3916-68ea-45f6-96a9-fc8d8addfedd {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:17:41.004087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cc0cb9cc-d1cc-466a-b4a8-ec6750bbf1e6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:41.014940 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cc0cb9cc-d1cc-466a-b4a8-ec6750bbf1e6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1506 microversion: 2.1 time: 0.339230 Sep 27 10:17:41.016247 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2099/4623] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:40 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1506 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:41.026547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:41.042503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80fd221b-68de-47ab-9a82-85f69c0f37cb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:41.043324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-80fd221b-68de-47ab-9a82-85f69c0f37cb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:41.048260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:41.051613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fd221b-68de-47ab-9a82-85f69c0f37cb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.051759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e4772d35d57f478d929eea70fd11e147', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9c60d0e3692444e39346d2c296c32ac2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:41.052522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fd221b-68de-47ab-9a82-85f69c0f37cb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.053290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fd221b-68de-47ab-9a82-85f69c0f37cb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.054158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fd221b-68de-47ab-9a82-85f69c0f37cb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.054640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fd221b-68de-47ab-9a82-85f69c0f37cb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.055072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fd221b-68de-47ab-9a82-85f69c0f37cb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.057281 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aa9ea6fe-8009-4633-a2f4-2297046cfb72 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 200 len: 1610 microversion: 2.32 time: 0.184195 Sep 27 10:17:41.059752 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2525/4624] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:40 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 1610 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:17:41.179514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-17851c56-d347-4fa0-830f-232081b14907 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:41.180607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-17851c56-d347-4fa0-830f-232081b14907 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:41.187662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17851c56-d347-4fa0-830f-232081b14907 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.188382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17851c56-d347-4fa0-830f-232081b14907 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.189059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17851c56-d347-4fa0-830f-232081b14907 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.190713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17851c56-d347-4fa0-830f-232081b14907 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.192064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17851c56-d347-4fa0-830f-232081b14907 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.193046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17851c56-d347-4fa0-830f-232081b14907 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.252342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-80fd221b-68de-47ab-9a82-85f69c0f37cb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:41.267661 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-80fd221b-68de-47ab-9a82-85f69c0f37cb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 200 len: 1506 microversion: 2.1 time: 0.230169 Sep 27 10:17:41.269160 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2100/4625] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:41 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 1506 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:41.274884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9102b72e-abdb-4d77-a723-48fba6a2c995 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:41.275805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9102b72e-abdb-4d77-a723-48fba6a2c995 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:41.283301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9102b72e-abdb-4d77-a723-48fba6a2c995 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.284041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9102b72e-abdb-4d77-a723-48fba6a2c995 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.284675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9102b72e-abdb-4d77-a723-48fba6a2c995 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.285576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9102b72e-abdb-4d77-a723-48fba6a2c995 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.286306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9102b72e-abdb-4d77-a723-48fba6a2c995 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.286985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9102b72e-abdb-4d77-a723-48fba6a2c995 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.432318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17851c56-d347-4fa0-830f-232081b14907 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:41.444685 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-17851c56-d347-4fa0-830f-232081b14907 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1488 microversion: 2.1 time: 0.274019 Sep 27 10:17:41.445690 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2526/4626] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:41 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1488 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:41.451447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d5b54fdf-87ab-46ba-866f-875f07ccb410 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:41.452133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d5b54fdf-87ab-46ba-866f-875f07ccb410 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:41.458997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b54fdf-87ab-46ba-866f-875f07ccb410 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.460343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b54fdf-87ab-46ba-866f-875f07ccb410 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.460974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b54fdf-87ab-46ba-866f-875f07ccb410 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.461880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b54fdf-87ab-46ba-866f-875f07ccb410 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.462682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b54fdf-87ab-46ba-866f-875f07ccb410 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.463253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5b54fdf-87ab-46ba-866f-875f07ccb410 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.492778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9102b72e-abdb-4d77-a723-48fba6a2c995 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:41.505067 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9102b72e-abdb-4d77-a723-48fba6a2c995 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1372 microversion: 2.1 time: 0.234126 Sep 27 10:17:41.507596 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2101/4627] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:41 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1372 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:41.513112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-590c4d25-104b-4e03-af29-8832335bae2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'update', calling method: >, body: {"server": {"accessIPv4": "1.1.1.1", "accessIPv6": "::babe:202:202"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:41.513753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d5b54fdf-87ab-46ba-866f-875f07ccb410 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:17:41.516048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-590c4d25-104b-4e03-af29-8832335bae2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:41.523172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-590c4d25-104b-4e03-af29-8832335bae2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.523809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-590c4d25-104b-4e03-af29-8832335bae2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.524475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-590c4d25-104b-4e03-af29-8832335bae2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.525307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-590c4d25-104b-4e03-af29-8832335bae2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.525930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-590c4d25-104b-4e03-af29-8832335bae2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.526457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-590c4d25-104b-4e03-af29-8832335bae2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.650280 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-590c4d25-104b-4e03-af29-8832335bae2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "PUT /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1083 microversion: 2.1 time: 0.140836 Sep 27 10:17:41.650941 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2102/4628] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:17:41 2022] PUT /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1083 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:41.684703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1959c436-b464-4ef7-90e9-0fdfc31940c7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:41.686116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1959c436-b464-4ef7-90e9-0fdfc31940c7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:41.696344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1959c436-b464-4ef7-90e9-0fdfc31940c7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.696963 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d5b54fdf-87ab-46ba-866f-875f07ccb410 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 204 len: 0 microversion: 2.1 time: 0.249659 Sep 27 10:17:41.697605 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2527/4629] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:17:41 2022] DELETE /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 0 bytes in 251 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:17:41.698999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1959c436-b464-4ef7-90e9-0fdfc31940c7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.699732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1959c436-b464-4ef7-90e9-0fdfc31940c7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.700803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1959c436-b464-4ef7-90e9-0fdfc31940c7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:41.701979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1959c436-b464-4ef7-90e9-0fdfc31940c7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:41.702625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1959c436-b464-4ef7-90e9-0fdfc31940c7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:41.887775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-1969627161", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "1dfd4252-9b03-463e-9f71-8ecdabb54d49"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:41.963484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] validate_networks() for [('1dfd4252-9b03-463e-9f71-8ecdabb54d49', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:17:42.022737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1959c436-b464-4ef7-90e9-0fdfc31940c7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:42.034115 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1959c436-b464-4ef7-90e9-0fdfc31940c7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1509 microversion: 2.1 time: 0.365270 Sep 27 10:17:42.036513 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2103/4630] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:41 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1509 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:42.042335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-105a79ae-22ae-4029-b776-0e5331134b29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3bc57786-1606-4e4f-81f1-868696186ba0"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:42.045672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-105a79ae-22ae-4029-b776-0e5331134b29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:42.054141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-105a79ae-22ae-4029-b776-0e5331134b29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:42.054948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-105a79ae-22ae-4029-b776-0e5331134b29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:42.055792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-105a79ae-22ae-4029-b776-0e5331134b29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:42.056565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-105a79ae-22ae-4029-b776-0e5331134b29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:42.057481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-105a79ae-22ae-4029-b776-0e5331134b29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:42.058074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-105a79ae-22ae-4029-b776-0e5331134b29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:42.551471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:17:42.552165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:17:42.553423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:17:42.554242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:17:42.554780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:17:42.565665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:42.566293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:42.566833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:42.567877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:42.569114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:42.569653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:42.595473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Getting quotas for project 8070262519034021a2b6de1a1e1ec88a. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:17:42.602760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Getting quotas for user dbf876e1c23642578363f9c5995f3d86 and project 8070262519034021a2b6de1a1e1ec88a. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:17:42.626299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:17:42.628987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:17:42.695141 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-564c1bc4-b5cb-43a4-a27c-5f76d85009ca tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.811548 Sep 27 10:17:42.697064 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2528/4631] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:17:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 814 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:17:42.703643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e7498c1-5cb1-449e-b28e-8af69bbd4ecf tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:42.704784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e7498c1-5cb1-449e-b28e-8af69bbd4ecf tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:42.715072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7498c1-5cb1-449e-b28e-8af69bbd4ecf tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:42.715578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7498c1-5cb1-449e-b28e-8af69bbd4ecf tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:42.716021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7498c1-5cb1-449e-b28e-8af69bbd4ecf tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:42.716873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7498c1-5cb1-449e-b28e-8af69bbd4ecf tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:42.717332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7498c1-5cb1-449e-b28e-8af69bbd4ecf tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:42.717757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7498c1-5cb1-449e-b28e-8af69bbd4ecf tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:42.994053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0e7498c1-5cb1-449e-b28e-8af69bbd4ecf tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:43.007674 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e7498c1-5cb1-449e-b28e-8af69bbd4ecf tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1509 microversion: 2.1 time: 0.309009 Sep 27 10:17:43.012832 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2529/4632] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:42 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1509 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:43.022603 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-105a79ae-22ae-4029-b776-0e5331134b29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "POST /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.981158 Sep 27 10:17:43.025525 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2104/4633] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:17:42 2022] POST /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments => generated 194 bytes in 988 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:43.033648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bdd61287-a4e9-4425-b36b-5c89391fad59 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:43.033813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:43.034334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:43.034525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bdd61287-a4e9-4425-b36b-5c89391fad59 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:43.042774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdd61287-a4e9-4425-b36b-5c89391fad59 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:43.043399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdd61287-a4e9-4425-b36b-5c89391fad59 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:43.043757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdd61287-a4e9-4425-b36b-5c89391fad59 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:43.044479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdd61287-a4e9-4425-b36b-5c89391fad59 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:43.044983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdd61287-a4e9-4425-b36b-5c89391fad59 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:43.046083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdd61287-a4e9-4425-b36b-5c89391fad59 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:43.055163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:43.055699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:43.057346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:43.057997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:43.058427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:43.060893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:43.097951 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] HTTP exception thrown: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Request method failure captured: Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd HTTP/1.1 Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsxOHJXKUjF2pu5Bia53-PT-XWlcxuvtCPdsiUS4bMil6yKiHgblBQDDiGCzt7KTpYRXeinQpppoSaTIC-gq1oyD96RwtW1XnJaRyugXTOLre2TnQGUINHXYwiBnFD0_tFinV0paio7fWJBPlJyGSMo5fZ5xg5w34sRZmxPh4h4HCyA Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.32 Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 9c60d0e3692444e39346d2c296c32ac2 Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TaggedBootDevicesTest-1222598264 Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9c60d0e3692444e39346d2c296c32ac2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9c60d0e3692444e39346d2c296c32ac2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9c60d0e3692444e39346d2c296c32ac2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9c60d0e3692444e39346d2c296c32ac2"}]}] Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TaggedBootDevicesTest-1222598264 Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 9c60d0e3692444e39346d2c296c32ac2 Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TaggedBootDevicesTest-1222598264 Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TaggedBootDevicesTest-1222598264-project Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: e4772d35d57f478d929eea70fd11e147 Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TaggedBootDevicesTest-1222598264-project Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '0aaf3916-68ea-45f6-96a9-fc8d8addfedd'} Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:43.102736 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:43.106866 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:43.106866 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:43.106866 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:43.106866 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:43.106866 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:17:43.106866 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:17:43.106866 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:43.106866 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:43.112254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Returning 404 to user: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:17:43.113316 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a097c696-6243-4f4b-98d7-7382fa7bc87d tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 404 len: 111 microversion: 2.32 time: 0.103565 Sep 27 10:17:43.114380 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2530/4634] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:43 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 111 bytes in 105 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:17:43.121322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b5a49d5-1e05-4d0f-917a-fd31bd5a285d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:43.125612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7b5a49d5-1e05-4d0f-917a-fd31bd5a285d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:43.269645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7b5a49d5-1e05-4d0f-917a-fd31bd5a285d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:43.280752 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7b5a49d5-1e05-4d0f-917a-fd31bd5a285d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1218 microversion: 2.1 time: 0.164999 Sep 27 10:17:43.301654 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2531/4635] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:43 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1218 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:43.319680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67a9438e-7a58-4ae7-9016-b5337f585b47 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:43.320489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67a9438e-7a58-4ae7-9016-b5337f585b47 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:43.330259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a9438e-7a58-4ae7-9016-b5337f585b47 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:43.331050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a9438e-7a58-4ae7-9016-b5337f585b47 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:43.331650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a9438e-7a58-4ae7-9016-b5337f585b47 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:43.332571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a9438e-7a58-4ae7-9016-b5337f585b47 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:43.333860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a9438e-7a58-4ae7-9016-b5337f585b47 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:43.334463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67a9438e-7a58-4ae7-9016-b5337f585b47 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:43.389385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bdd61287-a4e9-4425-b36b-5c89391fad59 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:43.397895 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bdd61287-a4e9-4425-b36b-5c89391fad59 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1545 microversion: 2.1 time: 0.370139 Sep 27 10:17:43.403721 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2105/4636] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:43 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1545 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:43.405808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67a9438e-7a58-4ae7-9016-b5337f585b47 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:17:43.408538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-889fc951-cb46-4097-b396-2c59e5f38928 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '.version_select at 0x7f7f62908700>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:43.462108 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-889fc951-cb46-4097-b396-2c59e5f38928 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1657625992" status: 204 len: 0 microversion: 2.32 time: 0.057190 Sep 27 10:17:43.463068 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2106/4637] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 10:17:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1657625992 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:17:43.469366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0e274fe9-ede5-43fc-93c7-8d48fbe11c85 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:43.471340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0e274fe9-ede5-43fc-93c7-8d48fbe11c85 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:43.486404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e274fe9-ede5-43fc-93c7-8d48fbe11c85 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:43.489334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e274fe9-ede5-43fc-93c7-8d48fbe11c85 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:43.490895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e274fe9-ede5-43fc-93c7-8d48fbe11c85 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:43.491819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e274fe9-ede5-43fc-93c7-8d48fbe11c85 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:43.493647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e274fe9-ede5-43fc-93c7-8d48fbe11c85 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:43.494264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e274fe9-ede5-43fc-93c7-8d48fbe11c85 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:43.610724 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67a9438e-7a58-4ae7-9016-b5337f585b47 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 204 len: 0 microversion: 2.1 time: 0.304325 Sep 27 10:17:43.610939 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2532/4638] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:17:43 2022] DELETE /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:17:43.634504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e8174cf0-19a7-4c9b-97fe-cac38f91c207 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:43.639109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e8174cf0-19a7-4c9b-97fe-cac38f91c207 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:43.644157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=2,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:43.678995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8174cf0-19a7-4c9b-97fe-cac38f91c207 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:43.683350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8174cf0-19a7-4c9b-97fe-cac38f91c207 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:43.684013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8174cf0-19a7-4c9b-97fe-cac38f91c207 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:43.684967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8174cf0-19a7-4c9b-97fe-cac38f91c207 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:43.685579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8174cf0-19a7-4c9b-97fe-cac38f91c207 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:43.688772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8174cf0-19a7-4c9b-97fe-cac38f91c207 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:43.733513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0e274fe9-ede5-43fc-93c7-8d48fbe11c85 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:43.745442 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0e274fe9-ede5-43fc-93c7-8d48fbe11c85 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1545 microversion: 2.1 time: 0.281053 Sep 27 10:17:43.746290 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2107/4639] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:43 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1545 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:44.222939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e8174cf0-19a7-4c9b-97fe-cac38f91c207 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:44.247242 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e8174cf0-19a7-4c9b-97fe-cac38f91c207 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1515 microversion: 2.1 time: 0.629776 Sep 27 10:17:44.270958 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2533/4640] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:43 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1515 bytes in 654 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:44.309213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d501bf7c-3aae-4990-8f54-5fab59cb2f45 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:44.309967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d501bf7c-3aae-4990-8f54-5fab59cb2f45 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:44.318574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:44.327961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d501bf7c-3aae-4990-8f54-5fab59cb2f45 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:44.328954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d501bf7c-3aae-4990-8f54-5fab59cb2f45 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:44.329738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d501bf7c-3aae-4990-8f54-5fab59cb2f45 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:44.330779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d501bf7c-3aae-4990-8f54-5fab59cb2f45 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:44.331555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d501bf7c-3aae-4990-8f54-5fab59cb2f45 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:44.332196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d501bf7c-3aae-4990-8f54-5fab59cb2f45 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:44.517113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d501bf7c-3aae-4990-8f54-5fab59cb2f45 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:44.527436 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d501bf7c-3aae-4990-8f54-5fab59cb2f45 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1270 microversion: 2.1 time: 0.222591 Sep 27 10:17:44.529285 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2108/4641] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:44 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1270 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:44.580440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-096d6704-1ea2-4cc6-a179-b143ce5ae261 req-2d6374a3-03f3-41d4-b563-fa3b6d60f641 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e5de27fc-1301-4e5a-ad28-2cd333ab9d6e", "name": "network-vif-deleted", "tag": "3634b03e-3d6f-45bc-8360-05f7f1e1a4fb"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:44.609658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-096d6704-1ea2-4cc6-a179-b143ce5ae261 req-2d6374a3-03f3-41d4-b563-fa3b6d60f641 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:44.609658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-096d6704-1ea2-4cc6-a179-b143ce5ae261 req-2d6374a3-03f3-41d4-b563-fa3b6d60f641 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:44.609658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-096d6704-1ea2-4cc6-a179-b143ce5ae261 req-2d6374a3-03f3-41d4-b563-fa3b6d60f641 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:44.666769 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-096d6704-1ea2-4cc6-a179-b143ce5ae261 req-2d6374a3-03f3-41d4-b563-fa3b6d60f641 service nova] Creating event network-vif-deleted:3634b03e-3d6f-45bc-8360-05f7f1e1a4fb for instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e on nch1-master-1 Sep 27 10:17:44.680287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-096d6704-1ea2-4cc6-a179-b143ce5ae261 req-2d6374a3-03f3-41d4-b563-fa3b6d60f641 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.104466 Sep 27 10:17:44.681189 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2534/4642] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:45.278729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d835fbef-4a59-406d-811e-138b6eed3269 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:45.279361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d835fbef-4a59-406d-811e-138b6eed3269 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:45.291148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d835fbef-4a59-406d-811e-138b6eed3269 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:45.291901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d835fbef-4a59-406d-811e-138b6eed3269 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:45.292344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d835fbef-4a59-406d-811e-138b6eed3269 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:45.293099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d835fbef-4a59-406d-811e-138b6eed3269 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:45.305573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d835fbef-4a59-406d-811e-138b6eed3269 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:45.306299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d835fbef-4a59-406d-811e-138b6eed3269 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:45.557176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92000e43-ae2f-46c3-977c-70cad69d702e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:45.558040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-92000e43-ae2f-46c3-977c-70cad69d702e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:45.571536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92000e43-ae2f-46c3-977c-70cad69d702e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:45.576926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92000e43-ae2f-46c3-977c-70cad69d702e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:45.578102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92000e43-ae2f-46c3-977c-70cad69d702e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:45.581062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92000e43-ae2f-46c3-977c-70cad69d702e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:45.582436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92000e43-ae2f-46c3-977c-70cad69d702e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:45.584301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92000e43-ae2f-46c3-977c-70cad69d702e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:45.807578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-92000e43-ae2f-46c3-977c-70cad69d702e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:45.817684 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-92000e43-ae2f-46c3-977c-70cad69d702e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1276 microversion: 2.1 time: 0.265891 Sep 27 10:17:45.820728 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2535/4643] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:45 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1276 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:46.421001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d835fbef-4a59-406d-811e-138b6eed3269 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:46.434831 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d835fbef-4a59-406d-811e-138b6eed3269 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1515 microversion: 2.1 time: 1.158902 Sep 27 10:17:46.448685 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2109/4644] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:45 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1515 bytes in 1162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:46.853871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e5fc3266-b530-49ee-8214-e9b45bb75354 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:46.854756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e5fc3266-b530-49ee-8214-e9b45bb75354 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:46.863778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5fc3266-b530-49ee-8214-e9b45bb75354 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:46.864711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5fc3266-b530-49ee-8214-e9b45bb75354 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:46.865542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5fc3266-b530-49ee-8214-e9b45bb75354 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:46.866523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5fc3266-b530-49ee-8214-e9b45bb75354 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:46.867213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5fc3266-b530-49ee-8214-e9b45bb75354 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:46.867986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5fc3266-b530-49ee-8214-e9b45bb75354 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:47.016934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e5fc3266-b530-49ee-8214-e9b45bb75354 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:47.038163 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e5fc3266-b530-49ee-8214-e9b45bb75354 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1276 microversion: 2.1 time: 0.192948 Sep 27 10:17:47.044783 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2536/4645] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:46 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1276 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:47.217338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-3f56c4a0-bacc-41f7-8de5-fe5a866731e1 req-a19d4954-bc34-40b8-823e-3b120785c589 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd547164-97da-4ba4-9cc3-b1448d2ab806", "name": "network-vif-deleted", "tag": "44b8cebe-e56b-418e-a02d-05d82b186787"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:47.238242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3f56c4a0-bacc-41f7-8de5-fe5a866731e1 req-a19d4954-bc34-40b8-823e-3b120785c589 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:47.242356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3f56c4a0-bacc-41f7-8de5-fe5a866731e1 req-a19d4954-bc34-40b8-823e-3b120785c589 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:47.244268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3f56c4a0-bacc-41f7-8de5-fe5a866731e1 req-a19d4954-bc34-40b8-823e-3b120785c589 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:47.300594 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-3f56c4a0-bacc-41f7-8de5-fe5a866731e1 req-a19d4954-bc34-40b8-823e-3b120785c589 service nova] Creating event network-vif-deleted:44b8cebe-e56b-418e-a02d-05d82b186787 for instance dd547164-97da-4ba4-9cc3-b1448d2ab806 on nch2-master-1 Sep 27 10:17:47.335745 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-3f56c4a0-bacc-41f7-8de5-fe5a866731e1 req-a19d4954-bc34-40b8-823e-3b120785c589 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.122514 Sep 27 10:17:47.337273 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2110/4646] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:47.468081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:47.471335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:47.487695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:47.488253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:47.489019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:47.490305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:47.496430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:47.498076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:47.506338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:47.552160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:47.553295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:47.553787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:47.554398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lazy-loading 'fault' on Instance uuid dd547164-97da-4ba4-9cc3-b1448d2ab806 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:17:47.693065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:47.720178 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0c7fcfb9-079e-42a3-b5cd-f8eec9b22332 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 200 len: 1319 microversion: 2.1 time: 0.262132 Sep 27 10:17:47.720935 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2537/4647] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:47 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 1319 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:47.807916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-570fdf0d-5e9e-423b-9a1a-fa6d8b9a1410 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:47.810359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-570fdf0d-5e9e-423b-9a1a-fa6d8b9a1410 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:47.821360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-570fdf0d-5e9e-423b-9a1a-fa6d8b9a1410 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:47.822007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-570fdf0d-5e9e-423b-9a1a-fa6d8b9a1410 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:47.823064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-570fdf0d-5e9e-423b-9a1a-fa6d8b9a1410 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:47.823859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-570fdf0d-5e9e-423b-9a1a-fa6d8b9a1410 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:47.824323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-570fdf0d-5e9e-423b-9a1a-fa6d8b9a1410 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:47.824999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-570fdf0d-5e9e-423b-9a1a-fa6d8b9a1410 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:47.887969 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-570fdf0d-5e9e-423b-9a1a-fa6d8b9a1410 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.084201 Sep 27 10:17:47.895321 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2111/4648] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:17:47 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments => generated 197 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:47.902810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ff232ec6-d5f1-4cf9-9e4a-aa6a4c4e7de3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:47.903543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ff232ec6-d5f1-4cf9-9e4a-aa6a4c4e7de3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:47.922884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff232ec6-d5f1-4cf9-9e4a-aa6a4c4e7de3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:47.925073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff232ec6-d5f1-4cf9-9e4a-aa6a4c4e7de3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:47.926494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff232ec6-d5f1-4cf9-9e4a-aa6a4c4e7de3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:47.927703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff232ec6-d5f1-4cf9-9e4a-aa6a4c4e7de3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:47.929594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff232ec6-d5f1-4cf9-9e4a-aa6a4c4e7de3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:47.930051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff232ec6-d5f1-4cf9-9e4a-aa6a4c4e7de3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:48.054095 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ff232ec6-d5f1-4cf9-9e4a-aa6a4c4e7de3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments/3bc57786-1606-4e4f-81f1-868696186ba0" status: 200 len: 194 microversion: 2.1 time: 0.155349 Sep 27 10:17:48.061190 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2538/4649] 10.222.0.26 () {58 vars in 1413 bytes} [Tue Sep 27 10:17:47 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments/3bc57786-1606-4e4f-81f1-868696186ba0 => generated 194 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:48.077017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d7ca9a11-aab8-4ba5-985c-51ea6e1e5acc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:48.078905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d7ca9a11-aab8-4ba5-985c-51ea6e1e5acc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:48.099505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ca9a11-aab8-4ba5-985c-51ea6e1e5acc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:48.104288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ca9a11-aab8-4ba5-985c-51ea6e1e5acc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:48.106562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ca9a11-aab8-4ba5-985c-51ea6e1e5acc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:48.114139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ca9a11-aab8-4ba5-985c-51ea6e1e5acc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:48.118730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ca9a11-aab8-4ba5-985c-51ea6e1e5acc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:48.119712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7ca9a11-aab8-4ba5-985c-51ea6e1e5acc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:48.688445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d7ca9a11-aab8-4ba5-985c-51ea6e1e5acc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:48.699683 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d7ca9a11-aab8-4ba5-985c-51ea6e1e5acc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1318 microversion: 2.1 time: 0.632381 Sep 27 10:17:48.705545 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2112/4650] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:48 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1318 bytes in 639 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:48.749982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:48.751150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:48.765162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:48.769372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:48.769960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:48.770746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:48.773112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:48.777255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:49.252497 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 HTTP/1.1 Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'dd547164-97da-4ba4-9cc3-b1448d2ab806'} Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:49.254998 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:49.259922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:49.259922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:49.259922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:49.259922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:17:49.259922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:17:49.259922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:17:49.259922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:49.262087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:17:49.262399 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86d66934-59c9-4c4c-9aef-743e2225602d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 404 len: 111 microversion: 2.1 time: 0.518901 Sep 27 10:17:49.266888 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2539/4651] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:48 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 111 bytes in 520 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:17:49.526083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersTestJSON-server-1706614547", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "662e23c5-f7e1-4e14-af6c-49d672076632"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:49.528027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-82fb98ee-e3fc-4db9-b81a-db4e9283cf79 req-36a662ea-c3df-4a0e-92b8-9c08d4270607 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3c45ec10-5528-48be-bc7b-4e628abd6d85", "tag": "d2a12997-e121-466a-a120-ced0de0c1108"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:49.547813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-82fb98ee-e3fc-4db9-b81a-db4e9283cf79 req-36a662ea-c3df-4a0e-92b8-9c08d4270607 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:49.549683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-82fb98ee-e3fc-4db9-b81a-db4e9283cf79 req-36a662ea-c3df-4a0e-92b8-9c08d4270607 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:49.550149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-82fb98ee-e3fc-4db9-b81a-db4e9283cf79 req-36a662ea-c3df-4a0e-92b8-9c08d4270607 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:49.577208 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-82fb98ee-e3fc-4db9-b81a-db4e9283cf79 req-36a662ea-c3df-4a0e-92b8-9c08d4270607 service nova] Creating event network-changed:d2a12997-e121-466a-a120-ced0de0c1108 for instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 on nch1-master-1 Sep 27 10:17:49.584473 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-82fb98ee-e3fc-4db9-b81a-db4e9283cf79 req-36a662ea-c3df-4a0e-92b8-9c08d4270607 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060259 Sep 27 10:17:49.585578 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2540/4652] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:17:49.646672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] validate_networks() for [('662e23c5-f7e1-4e14-af6c-49d672076632', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:17:49.728296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2dd49695-1a09-4ada-80c7-922bfb7c2068 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:49.729042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2dd49695-1a09-4ada-80c7-922bfb7c2068 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:49.742196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd49695-1a09-4ada-80c7-922bfb7c2068 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:49.742705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd49695-1a09-4ada-80c7-922bfb7c2068 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:49.743156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd49695-1a09-4ada-80c7-922bfb7c2068 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:49.743880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd49695-1a09-4ada-80c7-922bfb7c2068 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:49.744342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd49695-1a09-4ada-80c7-922bfb7c2068 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:49.744807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dd49695-1a09-4ada-80c7-922bfb7c2068 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:50.108555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2dd49695-1a09-4ada-80c7-922bfb7c2068 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:50.125663 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2dd49695-1a09-4ada-80c7-922bfb7c2068 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1318 microversion: 2.1 time: 0.401814 Sep 27 10:17:50.126266 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2541/4653] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:49 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1318 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:50.343433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-48330850-01ad-4101-b889-40a8d9569482 req-8b64536f-6f7c-4c10-99cd-bcbcb61fc906 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f0d9e453-3f14-4cd2-b7a7-31a47762c436", "tag": "e1e18a0a-7b42-40a1-b724-3c904a45faa3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:50.360795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-48330850-01ad-4101-b889-40a8d9569482 req-8b64536f-6f7c-4c10-99cd-bcbcb61fc906 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:50.363554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-48330850-01ad-4101-b889-40a8d9569482 req-8b64536f-6f7c-4c10-99cd-bcbcb61fc906 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:50.364058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-48330850-01ad-4101-b889-40a8d9569482 req-8b64536f-6f7c-4c10-99cd-bcbcb61fc906 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:50.395323 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-48330850-01ad-4101-b889-40a8d9569482 req-8b64536f-6f7c-4c10-99cd-bcbcb61fc906 service nova] Creating event network-changed:e1e18a0a-7b42-40a1-b724-3c904a45faa3 for instance f0d9e453-3f14-4cd2-b7a7-31a47762c436 on nch2-master-1 Sep 27 10:17:50.416601 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-48330850-01ad-4101-b889-40a8d9569482 req-8b64536f-6f7c-4c10-99cd-bcbcb61fc906 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.288822 Sep 27 10:17:50.417451 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2542/4654] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:50.428195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d8ac5cbc-2d8c-4e85-a955-ecde664d63a4"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:50.434254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:50.451940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:50.452460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:50.452947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:50.453687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:50.454155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:50.454592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:50.954565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e7f56ed1-990a-4c93-924e-d5ee7d9e3209 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "POST /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.534192 Sep 27 10:17:50.962416 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2543/4655] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:17:50 2022] POST /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments => generated 194 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:51.153812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2718af17-e958-4d9b-a7bd-5dc3e2b55d31 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:51.154751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2718af17-e958-4d9b-a7bd-5dc3e2b55d31 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:51.171084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2718af17-e958-4d9b-a7bd-5dc3e2b55d31 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:51.173110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2718af17-e958-4d9b-a7bd-5dc3e2b55d31 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:51.177948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2718af17-e958-4d9b-a7bd-5dc3e2b55d31 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:51.180191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2718af17-e958-4d9b-a7bd-5dc3e2b55d31 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:51.180716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2718af17-e958-4d9b-a7bd-5dc3e2b55d31 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:51.181277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2718af17-e958-4d9b-a7bd-5dc3e2b55d31 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:51.651037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2718af17-e958-4d9b-a7bd-5dc3e2b55d31 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:51.662910 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2718af17-e958-4d9b-a7bd-5dc3e2b55d31 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1318 microversion: 2.1 time: 0.512974 Sep 27 10:17:51.664075 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2544/4656] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:51 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1318 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:51.941445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:17:51.942431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:17:51.943608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:17:51.944339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:17:51.944598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:17:51.976425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:51.977716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:51.980486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:51.984216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:51.985241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:51.987331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:52.073364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:17:52.090473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Getting quotas for user a7b8599d0b40424087e52d8e35b71ec4 and project 1d229e5b879b481082284236c9f8c446. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:17:52.139456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:17:52.143099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:17:52.276009 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-735d5340-7cdb-4210-bcbb-28196f7af7b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.751535 Sep 27 10:17:52.284951 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2113/4657] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:17:49 2022] POST /compute/v2.1/servers => generated 380 bytes in 2760 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:17:52.292208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8d6f16c-2f77-431c-a608-46791b076c07 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:52.292879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c8d6f16c-2f77-431c-a608-46791b076c07 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:52.533217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c8d6f16c-2f77-431c-a608-46791b076c07 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:52.545684 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c8d6f16c-2f77-431c-a608-46791b076c07 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1209 microversion: 2.1 time: 0.257738 Sep 27 10:17:52.548294 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2545/4658] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:52 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1209 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:52.691016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a3a0a31-8e3b-46d1-ad66-6ea0a8a49522 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:52.693247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a3a0a31-8e3b-46d1-ad66-6ea0a8a49522 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:52.708256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3a0a31-8e3b-46d1-ad66-6ea0a8a49522 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:52.715160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3a0a31-8e3b-46d1-ad66-6ea0a8a49522 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:52.715760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3a0a31-8e3b-46d1-ad66-6ea0a8a49522 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:52.716700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3a0a31-8e3b-46d1-ad66-6ea0a8a49522 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:52.717414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3a0a31-8e3b-46d1-ad66-6ea0a8a49522 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:52.717848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3a0a31-8e3b-46d1-ad66-6ea0a8a49522 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:53.570995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f66e6b3-5052-4e8b-9500-3f1ab002858e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:53.571629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8f66e6b3-5052-4e8b-9500-3f1ab002858e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:53.574530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:53.589872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f66e6b3-5052-4e8b-9500-3f1ab002858e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:53.593436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f66e6b3-5052-4e8b-9500-3f1ab002858e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:53.594119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f66e6b3-5052-4e8b-9500-3f1ab002858e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:53.595036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f66e6b3-5052-4e8b-9500-3f1ab002858e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:53.597305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f66e6b3-5052-4e8b-9500-3f1ab002858e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:53.598231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f66e6b3-5052-4e8b-9500-3f1ab002858e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:53.734274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a3a0a31-8e3b-46d1-ad66-6ea0a8a49522 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:53.738755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:17:53.795340 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a3a0a31-8e3b-46d1-ad66-6ea0a8a49522 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1318 microversion: 2.1 time: 1.099540 Sep 27 10:17:53.815898 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2114/4659] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:52 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1318 bytes in 1131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:53.864512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8f66e6b3-5052-4e8b-9500-3f1ab002858e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:53.873187 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8f66e6b3-5052-4e8b-9500-3f1ab002858e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1205 microversion: 2.1 time: 0.307148 Sep 27 10:17:53.881104 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2546/4660] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:53 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1205 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:54.826792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-690ef16e-9c59-4857-9496-4cd664871ea1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:54.828454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-690ef16e-9c59-4857-9496-4cd664871ea1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:54.850951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-690ef16e-9c59-4857-9496-4cd664871ea1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:54.851812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-690ef16e-9c59-4857-9496-4cd664871ea1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:54.852419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-690ef16e-9c59-4857-9496-4cd664871ea1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:54.853376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-690ef16e-9c59-4857-9496-4cd664871ea1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:54.854341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-690ef16e-9c59-4857-9496-4cd664871ea1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:54.855574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-690ef16e-9c59-4857-9496-4cd664871ea1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:54.914289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2b2efe89-5cd7-43d8-81c1-659cf24a910f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:54.915739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2b2efe89-5cd7-43d8-81c1-659cf24a910f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:54.933680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2efe89-5cd7-43d8-81c1-659cf24a910f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:54.934491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2efe89-5cd7-43d8-81c1-659cf24a910f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:54.938050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2efe89-5cd7-43d8-81c1-659cf24a910f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:54.939662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2efe89-5cd7-43d8-81c1-659cf24a910f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:54.942081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2efe89-5cd7-43d8-81c1-659cf24a910f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:54.943634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b2efe89-5cd7-43d8-81c1-659cf24a910f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:55.039790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2b2efe89-5cd7-43d8-81c1-659cf24a910f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:55.050763 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2b2efe89-5cd7-43d8-81c1-659cf24a910f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1267 microversion: 2.1 time: 0.148643 Sep 27 10:17:55.053528 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2547/4661] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:54 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1267 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:55.529611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-690ef16e-9c59-4857-9496-4cd664871ea1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:55.542560 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-690ef16e-9c59-4857-9496-4cd664871ea1 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1318 microversion: 2.1 time: 0.720079 Sep 27 10:17:55.543197 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2115/4662] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:54 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1318 bytes in 723 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:55.637893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:55.638659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:55.653449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:55.654193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:55.654955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:55.657896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:55.658641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:55.659230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:55.681393 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] HTTP exception thrown: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Request method failure captured: Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd HTTP/1.1 Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMsxOHJXKUjF2pu5Bia53-PT-XWlcxuvtCPdsiUS4bMil6yKiHgblBQDDiGCzt7KTpYRXeinQpppoSaTIC-gq1oyD96RwtW1XnJaRyugXTOLre2TnQGUINHXYwiBnFD0_tFinV0paio7fWJBPlJyGSMo5fZ5xg5w34sRZmxPh4h4HCyA Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.32 Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 9c60d0e3692444e39346d2c296c32ac2 Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TaggedBootDevicesTest-1222598264 Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9c60d0e3692444e39346d2c296c32ac2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9c60d0e3692444e39346d2c296c32ac2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9c60d0e3692444e39346d2c296c32ac2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9c60d0e3692444e39346d2c296c32ac2"}]}] Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TaggedBootDevicesTest-1222598264 Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 9c60d0e3692444e39346d2c296c32ac2 Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TaggedBootDevicesTest-1222598264 Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TaggedBootDevicesTest-1222598264-project Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: e4772d35d57f478d929eea70fd11e147 Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TaggedBootDevicesTest-1222598264-project Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '0aaf3916-68ea-45f6-96a9-fc8d8addfedd'} Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:17:55.683249 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Returning 404 to user: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:17:55.687993 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-07f8da07-c5a9-458f-a270-6f6f1bf86543 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 404 len: 111 microversion: 2.32 time: 0.052335 Sep 27 10:17:55.689078 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2548/4663] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:17:55 2022] DELETE /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:17:55.695433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:55.696057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] [instance: 0aaf3916-68ea-45f6-96a9-fc8d8addfedd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:55.704465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:55.704997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:55.705439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:55.710598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:55.717675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:55.718617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:55.738046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-36d1aa03-7d36-4d2f-8d31-b72fb1513a29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:55.745802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-36d1aa03-7d36-4d2f-8d31-b72fb1513a29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:55.756549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36d1aa03-7d36-4d2f-8d31-b72fb1513a29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:55.757349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36d1aa03-7d36-4d2f-8d31-b72fb1513a29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:55.757986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36d1aa03-7d36-4d2f-8d31-b72fb1513a29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:55.759711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36d1aa03-7d36-4d2f-8d31-b72fb1513a29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:55.760461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36d1aa03-7d36-4d2f-8d31-b72fb1513a29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:55.761111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36d1aa03-7d36-4d2f-8d31-b72fb1513a29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:55.770856 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] HTTP exception thrown: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Request method failure captured: Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd HTTP/1.1 Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMsxOHJXKUjF2pu5Bia53-PT-XWlcxuvtCPdsiUS4bMil6yKiHgblBQDDiGCzt7KTpYRXeinQpppoSaTIC-gq1oyD96RwtW1XnJaRyugXTOLre2TnQGUINHXYwiBnFD0_tFinV0paio7fWJBPlJyGSMo5fZ5xg5w34sRZmxPh4h4HCyA Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.32 Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 9c60d0e3692444e39346d2c296c32ac2 Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TaggedBootDevicesTest-1222598264 Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9c60d0e3692444e39346d2c296c32ac2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9c60d0e3692444e39346d2c296c32ac2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9c60d0e3692444e39346d2c296c32ac2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9c60d0e3692444e39346d2c296c32ac2"}]}] Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TaggedBootDevicesTest-1222598264 Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 9c60d0e3692444e39346d2c296c32ac2 Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TaggedBootDevicesTest-1222598264 Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TaggedBootDevicesTest-1222598264-project Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: e4772d35d57f478d929eea70fd11e147 Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TaggedBootDevicesTest-1222598264-project Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '0aaf3916-68ea-45f6-96a9-fc8d8addfedd'} Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:55.773235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:17:55.777817 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:55.777817 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:17:55.777817 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:17:55.777817 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:17:55.777817 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:17:55.777817 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:17:55.777817 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. Sep 27 10:17:55.777817 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:17:55.780237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] Returning 404 to user: Instance 0aaf3916-68ea-45f6-96a9-fc8d8addfedd could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:17:55.781527 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32daae6c-f280-4449-a445-e44934708e27 tempest-TaggedBootDevicesTest-1222598264 tempest-TaggedBootDevicesTest-1222598264-project] 10.222.0.26 "GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd" status: 404 len: 111 microversion: 2.32 time: 0.090942 Sep 27 10:17:55.782309 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2116/4664] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:17:55 2022] GET /compute/v2.1/servers/0aaf3916-68ea-45f6-96a9-fc8d8addfedd => generated 111 bytes in 92 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:17:55.840900 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-36d1aa03-7d36-4d2f-8d31-b72fb1513a29 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.110984 Sep 27 10:17:55.843474 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2549/4665] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:17:55 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments => generated 371 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:55.984148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c80091ea-dbd4-4e23-968d-05db484a65b9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:55.984805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c80091ea-dbd4-4e23-968d-05db484a65b9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:55.996174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80091ea-dbd4-4e23-968d-05db484a65b9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:55.998635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80091ea-dbd4-4e23-968d-05db484a65b9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:55.999339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80091ea-dbd4-4e23-968d-05db484a65b9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:56.000078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80091ea-dbd4-4e23-968d-05db484a65b9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:56.000546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80091ea-dbd4-4e23-968d-05db484a65b9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:56.001010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c80091ea-dbd4-4e23-968d-05db484a65b9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:56.053255 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c80091ea-dbd4-4e23-968d-05db484a65b9 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments/3bc57786-1606-4e4f-81f1-868696186ba0" status: 200 len: 194 microversion: 2.1 time: 0.072583 Sep 27 10:17:56.054345 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2117/4666] 10.222.0.26 () {58 vars in 1413 bytes} [Tue Sep 27 10:17:55 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments/3bc57786-1606-4e4f-81f1-868696186ba0 => generated 194 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:56.065318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:56.065881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:56.075378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:56.077427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-64e4fe5f-8fe5-41b8-a05d-d6beb33b5d43 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:56.078495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:56.079158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:56.079584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-64e4fe5f-8fe5-41b8-a05d-d6beb33b5d43 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:56.081590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:56.085072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:56.085827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:56.094835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e4fe5f-8fe5-41b8-a05d-d6beb33b5d43 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:56.097397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e4fe5f-8fe5-41b8-a05d-d6beb33b5d43 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:56.100405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e4fe5f-8fe5-41b8-a05d-d6beb33b5d43 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:56.103334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e4fe5f-8fe5-41b8-a05d-d6beb33b5d43 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:56.106911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e4fe5f-8fe5-41b8-a05d-d6beb33b5d43 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:56.106911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e4fe5f-8fe5-41b8-a05d-d6beb33b5d43 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:56.233631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lazy-loading 'services' on Instance uuid 779046b2-2e5b-4946-af25-c111c590a14e {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:17:56.260836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-64e4fe5f-8fe5-41b8-a05d-d6beb33b5d43 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:56.299529 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-64e4fe5f-8fe5-41b8-a05d-d6beb33b5d43 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1267 microversion: 2.1 time: 0.227869 Sep 27 10:17:56.318297 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2118/4667] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:56 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1267 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:56.441285 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1ca7cc2a-8c07-4698-8528-129c31e0d4ea tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "DELETE /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments/3bc57786-1606-4e4f-81f1-868696186ba0" status: 202 len: 0 microversion: 2.1 time: 0.379900 Sep 27 10:17:56.442748 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2550/4668] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:17:56 2022] DELETE /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments/3bc57786-1606-4e4f-81f1-868696186ba0 => generated 0 bytes in 382 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:17:56.569242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb5ac93e-d4c3-4377-a1dd-db5fe47af949 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:56.570006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fb5ac93e-d4c3-4377-a1dd-db5fe47af949 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:56.593179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5ac93e-d4c3-4377-a1dd-db5fe47af949 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:56.593986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5ac93e-d4c3-4377-a1dd-db5fe47af949 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:56.597838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5ac93e-d4c3-4377-a1dd-db5fe47af949 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:56.599773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5ac93e-d4c3-4377-a1dd-db5fe47af949 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:56.601597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5ac93e-d4c3-4377-a1dd-db5fe47af949 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:56.605608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5ac93e-d4c3-4377-a1dd-db5fe47af949 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:57.066772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fb5ac93e-d4c3-4377-a1dd-db5fe47af949 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:57.086369 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fb5ac93e-d4c3-4377-a1dd-db5fe47af949 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1318 microversion: 2.1 time: 0.521551 Sep 27 10:17:57.090167 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2119/4669] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:56 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1318 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:57.332956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-021e1819-d301-4d70-9ad5-3e0b05060d1b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:57.334091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-021e1819-d301-4d70-9ad5-3e0b05060d1b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:57.343447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-021e1819-d301-4d70-9ad5-3e0b05060d1b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:57.344222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-021e1819-d301-4d70-9ad5-3e0b05060d1b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:57.344855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-021e1819-d301-4d70-9ad5-3e0b05060d1b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:57.345822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-021e1819-d301-4d70-9ad5-3e0b05060d1b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:57.346574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-021e1819-d301-4d70-9ad5-3e0b05060d1b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:57.347152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-021e1819-d301-4d70-9ad5-3e0b05060d1b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:57.366511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-73b683b2-3e7a-4b25-9e42-4a57e2ce9e8b req-5423b6be-2139-4178-8c77-f17abbbf08ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c45ec10-5528-48be-bc7b-4e628abd6d85", "name": "network-vif-plugged", "status": "completed", "tag": "d2a12997-e121-466a-a120-ced0de0c1108"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:57.385936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-73b683b2-3e7a-4b25-9e42-4a57e2ce9e8b req-5423b6be-2139-4178-8c77-f17abbbf08ea service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:57.386769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-73b683b2-3e7a-4b25-9e42-4a57e2ce9e8b req-5423b6be-2139-4178-8c77-f17abbbf08ea service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:57.388185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-73b683b2-3e7a-4b25-9e42-4a57e2ce9e8b req-5423b6be-2139-4178-8c77-f17abbbf08ea service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:57.413790 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-73b683b2-3e7a-4b25-9e42-4a57e2ce9e8b req-5423b6be-2139-4178-8c77-f17abbbf08ea service nova] Creating event network-vif-plugged:d2a12997-e121-466a-a120-ced0de0c1108 for instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 on nch1-master-1 Sep 27 10:17:57.429444 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-73b683b2-3e7a-4b25-9e42-4a57e2ce9e8b req-5423b6be-2139-4178-8c77-f17abbbf08ea service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068007 Sep 27 10:17:57.447170 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2120/4670] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:57.732587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-021e1819-d301-4d70-9ad5-3e0b05060d1b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:57.747596 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-021e1819-d301-4d70-9ad5-3e0b05060d1b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1309 microversion: 2.1 time: 0.419472 Sep 27 10:17:57.748204 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2551/4671] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:57 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1309 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:57.782326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9a5f9d20-05cc-4b64-9a27-8d4525447d70 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:57.783151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9a5f9d20-05cc-4b64-9a27-8d4525447d70 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:57.796275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a5f9d20-05cc-4b64-9a27-8d4525447d70 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:57.796858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a5f9d20-05cc-4b64-9a27-8d4525447d70 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:57.799609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a5f9d20-05cc-4b64-9a27-8d4525447d70 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:57.800357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a5f9d20-05cc-4b64-9a27-8d4525447d70 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:57.802265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a5f9d20-05cc-4b64-9a27-8d4525447d70 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:57.802750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a5f9d20-05cc-4b64-9a27-8d4525447d70 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:57.879248 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9a5f9d20-05cc-4b64-9a27-8d4525447d70 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments/d8ac5cbc-2d8c-4e85-a955-ecde664d63a4" status: 200 len: 194 microversion: 2.1 time: 0.102046 Sep 27 10:17:57.880446 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2121/4672] 10.222.0.26 () {58 vars in 1413 bytes} [Tue Sep 27 10:17:57 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments/d8ac5cbc-2d8c-4e85-a955-ecde664d63a4 => generated 194 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:57.893923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c45dfb8-dedf-4342-b447-471698568987 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:57.894975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5c45dfb8-dedf-4342-b447-471698568987 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:57.907241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c45dfb8-dedf-4342-b447-471698568987 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:57.908462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c45dfb8-dedf-4342-b447-471698568987 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:57.909411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c45dfb8-dedf-4342-b447-471698568987 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:57.910377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c45dfb8-dedf-4342-b447-471698568987 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:57.911889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c45dfb8-dedf-4342-b447-471698568987 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:57.915120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c45dfb8-dedf-4342-b447-471698568987 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:58.129398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2fcf245a-011c-4ae0-90df-b9987256c759 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:58.131226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2fcf245a-011c-4ae0-90df-b9987256c759 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:58.133477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-5c45dfb8-dedf-4342-b447-471698568987 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lazy-loading 'services' on Instance uuid 779046b2-2e5b-4946-af25-c111c590a14e {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:17:58.167432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fcf245a-011c-4ae0-90df-b9987256c759 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:58.171489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fcf245a-011c-4ae0-90df-b9987256c759 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:58.172460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fcf245a-011c-4ae0-90df-b9987256c759 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:58.173461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fcf245a-011c-4ae0-90df-b9987256c759 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:58.175746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fcf245a-011c-4ae0-90df-b9987256c759 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:58.177195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2fcf245a-011c-4ae0-90df-b9987256c759 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:58.304997 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c45dfb8-dedf-4342-b447-471698568987 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "DELETE /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments/d8ac5cbc-2d8c-4e85-a955-ecde664d63a4" status: 202 len: 0 microversion: 2.1 time: 0.416956 Sep 27 10:17:58.305728 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2552/4673] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:17:57 2022] DELETE /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments/d8ac5cbc-2d8c-4e85-a955-ecde664d63a4 => generated 0 bytes in 419 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:17:58.432956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2fcf245a-011c-4ae0-90df-b9987256c759 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:58.443326 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2fcf245a-011c-4ae0-90df-b9987256c759 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1318 microversion: 2.1 time: 0.331548 Sep 27 10:17:58.445901 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2122/4674] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:58 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1318 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:17:58.624363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-489bc4a2-95f9-428d-8035-5509f8ed84ef req-13cace17-79e4-452e-88b3-ab4a7cca427a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8696cba0-e2e2-4f51-858d-3b502152a6ed", "tag": "1e961036-096a-4369-b689-86c91be70173"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:17:58.635650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-489bc4a2-95f9-428d-8035-5509f8ed84ef req-13cace17-79e4-452e-88b3-ab4a7cca427a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:58.636393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-489bc4a2-95f9-428d-8035-5509f8ed84ef req-13cace17-79e4-452e-88b3-ab4a7cca427a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:58.637011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-489bc4a2-95f9-428d-8035-5509f8ed84ef req-13cace17-79e4-452e-88b3-ab4a7cca427a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:58.662262 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-489bc4a2-95f9-428d-8035-5509f8ed84ef req-13cace17-79e4-452e-88b3-ab4a7cca427a service nova] Creating event network-changed:1e961036-096a-4369-b689-86c91be70173 for instance 8696cba0-e2e2-4f51-858d-3b502152a6ed on nch2-master-1 Sep 27 10:17:58.670097 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-489bc4a2-95f9-428d-8035-5509f8ed84ef req-13cace17-79e4-452e-88b3-ab4a7cca427a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052226 Sep 27 10:17:58.670504 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2553/4675] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:17:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:17:58.770914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7ba024b0-56c7-42fd-a8c3-4d9d9969dce7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:58.771667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7ba024b0-56c7-42fd-a8c3-4d9d9969dce7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:58.778213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ba024b0-56c7-42fd-a8c3-4d9d9969dce7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:58.778711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ba024b0-56c7-42fd-a8c3-4d9d9969dce7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:58.779144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ba024b0-56c7-42fd-a8c3-4d9d9969dce7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:58.780018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ba024b0-56c7-42fd-a8c3-4d9d9969dce7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:58.780821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ba024b0-56c7-42fd-a8c3-4d9d9969dce7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:58.781431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ba024b0-56c7-42fd-a8c3-4d9d9969dce7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:59.176500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7ba024b0-56c7-42fd-a8c3-4d9d9969dce7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:59.195159 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7ba024b0-56c7-42fd-a8c3-4d9d9969dce7 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1309 microversion: 2.1 time: 0.428011 Sep 27 10:17:59.210233 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2123/4676] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:58 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1309 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:59.473544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6102a3a1-b567-4ba5-8516-fd9bdedcaeba tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:59.476431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6102a3a1-b567-4ba5-8516-fd9bdedcaeba tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:59.488720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6102a3a1-b567-4ba5-8516-fd9bdedcaeba tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:59.489657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6102a3a1-b567-4ba5-8516-fd9bdedcaeba tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:59.491663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6102a3a1-b567-4ba5-8516-fd9bdedcaeba tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:59.493124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6102a3a1-b567-4ba5-8516-fd9bdedcaeba tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:59.493882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6102a3a1-b567-4ba5-8516-fd9bdedcaeba tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:59.494649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6102a3a1-b567-4ba5-8516-fd9bdedcaeba tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:59.800214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6102a3a1-b567-4ba5-8516-fd9bdedcaeba tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:17:59.812057 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6102a3a1-b567-4ba5-8516-fd9bdedcaeba tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1508 microversion: 2.1 time: 0.346153 Sep 27 10:17:59.816998 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2554/4677] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:59 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1508 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:17:59.842705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dfc0df39-f4a9-4083-af35-17a92b1b5473 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:17:59.843667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dfc0df39-f4a9-4083-af35-17a92b1b5473 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:17:59.852019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfc0df39-f4a9-4083-af35-17a92b1b5473 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:59.853884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfc0df39-f4a9-4083-af35-17a92b1b5473 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:59.854284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfc0df39-f4a9-4083-af35-17a92b1b5473 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:17:59.855263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfc0df39-f4a9-4083-af35-17a92b1b5473 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:17:59.855964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfc0df39-f4a9-4083-af35-17a92b1b5473 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:17:59.856773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dfc0df39-f4a9-4083-af35-17a92b1b5473 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:00.226707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-781cd17b-d761-4956-8d80-f88429228ae1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:00.227602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-781cd17b-d761-4956-8d80-f88429228ae1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:00.240286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-781cd17b-d761-4956-8d80-f88429228ae1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:00.241061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-781cd17b-d761-4956-8d80-f88429228ae1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:00.241673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-781cd17b-d761-4956-8d80-f88429228ae1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:00.242600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-781cd17b-d761-4956-8d80-f88429228ae1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:00.244118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-781cd17b-d761-4956-8d80-f88429228ae1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:00.244567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-781cd17b-d761-4956-8d80-f88429228ae1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:00.250587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:00.471429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dfc0df39-f4a9-4083-af35-17a92b1b5473 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:00.480242 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dfc0df39-f4a9-4083-af35-17a92b1b5473 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1508 microversion: 2.1 time: 0.642398 Sep 27 10:18:00.482083 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2124/4678] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:17:59 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1508 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:00.504365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3675db3f-9f76-4b1f-a43b-e163290e6ca8 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "AUTO"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:00.507286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3675db3f-9f76-4b1f-a43b-e163290e6ca8 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:00.514590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-781cd17b-d761-4956-8d80-f88429228ae1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:00.529619 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-781cd17b-d761-4956-8d80-f88429228ae1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1309 microversion: 2.1 time: 0.305566 Sep 27 10:18:00.544984 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2555/4679] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:00 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1309 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:00.546751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3675db3f-9f76-4b1f-a43b-e163290e6ca8 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:00.549944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3675db3f-9f76-4b1f-a43b-e163290e6ca8 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:00.550690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3675db3f-9f76-4b1f-a43b-e163290e6ca8 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:00.551625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3675db3f-9f76-4b1f-a43b-e163290e6ca8 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:00.552267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3675db3f-9f76-4b1f-a43b-e163290e6ca8 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:00.552939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3675db3f-9f76-4b1f-a43b-e163290e6ca8 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:00.665660 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3675db3f-9f76-4b1f-a43b-e163290e6ca8 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "PUT /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1080 microversion: 2.1 time: 0.166344 Sep 27 10:18:00.667281 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2125/4680] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:18:00 2022] PUT /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1080 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:00.692526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4fec0b2-8811-4313-a53d-7b89879723b0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:00.693625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4fec0b2-8811-4313-a53d-7b89879723b0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:00.703680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4fec0b2-8811-4313-a53d-7b89879723b0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:00.704180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4fec0b2-8811-4313-a53d-7b89879723b0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:00.704627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4fec0b2-8811-4313-a53d-7b89879723b0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:00.706852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4fec0b2-8811-4313-a53d-7b89879723b0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:00.707347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4fec0b2-8811-4313-a53d-7b89879723b0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:00.707782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4fec0b2-8811-4313-a53d-7b89879723b0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:00.815545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9f8fc9ba-4b91-408a-92d1-88c91639594b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:00.819036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9f8fc9ba-4b91-408a-92d1-88c91639594b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:00.832789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f8fc9ba-4b91-408a-92d1-88c91639594b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:00.833297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f8fc9ba-4b91-408a-92d1-88c91639594b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:00.833792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f8fc9ba-4b91-408a-92d1-88c91639594b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:00.834514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f8fc9ba-4b91-408a-92d1-88c91639594b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:00.834969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f8fc9ba-4b91-408a-92d1-88c91639594b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:00.835397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f8fc9ba-4b91-408a-92d1-88c91639594b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:00.892038 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9f8fc9ba-4b91-408a-92d1-88c91639594b tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.081083 Sep 27 10:18:00.892957 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2126/4681] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:18:00 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments => generated 25 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:18:01.044167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4fec0b2-8811-4313-a53d-7b89879723b0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:01.058236 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4fec0b2-8811-4313-a53d-7b89879723b0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1506 microversion: 2.1 time: 0.372274 Sep 27 10:18:01.067453 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2556/4682] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:00 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1506 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:01.120451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d80c6f8-311b-4613-ac39-f6a6101fe55d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:01.123013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5d80c6f8-311b-4613-ac39-f6a6101fe55d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:01.128232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3a517bea-f42c-434d-997a-b51ad20e6d1c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:01.129334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3a517bea-f42c-434d-997a-b51ad20e6d1c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:01.139445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d80c6f8-311b-4613-ac39-f6a6101fe55d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.148276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d80c6f8-311b-4613-ac39-f6a6101fe55d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.149228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d80c6f8-311b-4613-ac39-f6a6101fe55d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:01.150154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d80c6f8-311b-4613-ac39-f6a6101fe55d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.150787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d80c6f8-311b-4613-ac39-f6a6101fe55d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.152131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d80c6f8-311b-4613-ac39-f6a6101fe55d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:01.156336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a517bea-f42c-434d-997a-b51ad20e6d1c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.157633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a517bea-f42c-434d-997a-b51ad20e6d1c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.160081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a517bea-f42c-434d-997a-b51ad20e6d1c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:01.161138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a517bea-f42c-434d-997a-b51ad20e6d1c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.162937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a517bea-f42c-434d-997a-b51ad20e6d1c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.163742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a517bea-f42c-434d-997a-b51ad20e6d1c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:01.241003 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5d80c6f8-311b-4613-ac39-f6a6101fe55d tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.124218 Sep 27 10:18:01.241621 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2557/4683] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:18:01 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e/os-volume_attachments => generated 25 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 27 10:18:01.356930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3d34514-85c1-4cb8-b03a-4b0aa4223735 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:01.357631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3d34514-85c1-4cb8-b03a-4b0aa4223735 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:01.365146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d34514-85c1-4cb8-b03a-4b0aa4223735 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.365661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d34514-85c1-4cb8-b03a-4b0aa4223735 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.366126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d34514-85c1-4cb8-b03a-4b0aa4223735 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:01.367258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d34514-85c1-4cb8-b03a-4b0aa4223735 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.367736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d34514-85c1-4cb8-b03a-4b0aa4223735 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.368356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d34514-85c1-4cb8-b03a-4b0aa4223735 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:01.398989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3d34514-85c1-4cb8-b03a-4b0aa4223735 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:18:01.528175 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3d34514-85c1-4cb8-b03a-4b0aa4223735 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "DELETE /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 204 len: 0 microversion: 2.1 time: 0.177133 Sep 27 10:18:01.529392 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2558/4684] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:01 2022] DELETE /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:18:01.539430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4815fa0d-e324-4609-a498-6fa275c1aca7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:01.540566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4815fa0d-e324-4609-a498-6fa275c1aca7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:01.547319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4815fa0d-e324-4609-a498-6fa275c1aca7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.548146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4815fa0d-e324-4609-a498-6fa275c1aca7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.549398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4815fa0d-e324-4609-a498-6fa275c1aca7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:01.550329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4815fa0d-e324-4609-a498-6fa275c1aca7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.552729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4815fa0d-e324-4609-a498-6fa275c1aca7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.553405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4815fa0d-e324-4609-a498-6fa275c1aca7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:01.622390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3a517bea-f42c-434d-997a-b51ad20e6d1c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:01.633429 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3a517bea-f42c-434d-997a-b51ad20e6d1c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1506 microversion: 2.1 time: 0.542222 Sep 27 10:18:01.634610 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2127/4685] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:01 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1506 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:01.640851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-801955fc-1bb1-46a6-8259-90501a05fdec tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:01.641563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-801955fc-1bb1-46a6-8259-90501a05fdec tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:01.649482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-801955fc-1bb1-46a6-8259-90501a05fdec tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.650231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-801955fc-1bb1-46a6-8259-90501a05fdec tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.650824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-801955fc-1bb1-46a6-8259-90501a05fdec tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:01.651705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-801955fc-1bb1-46a6-8259-90501a05fdec tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.652312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-801955fc-1bb1-46a6-8259-90501a05fdec tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.652931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-801955fc-1bb1-46a6-8259-90501a05fdec tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:01.773018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4815fa0d-e324-4609-a498-6fa275c1aca7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:01.806050 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4815fa0d-e324-4609-a498-6fa275c1aca7 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1673 microversion: 2.1 time: 0.271256 Sep 27 10:18:01.819727 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2559/4686] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:01 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1673 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:01.836045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "OS-DCF:diskConfig": "MANUAL"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:01.842846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:01.855891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.857428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.858201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:01.859173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:01.859828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:01.860832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:02.041345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:02.111812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:18:02.112492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:18:02.116037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:18:02.116587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:02.118569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:18:02.123395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:18:02.153916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-801955fc-1bb1-46a6-8259-90501a05fdec tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:02.163568 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-801955fc-1bb1-46a6-8259-90501a05fdec tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1309 microversion: 2.1 time: 0.527569 Sep 27 10:18:02.169627 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2128/4687] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:01 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1309 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:02.283730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:02.312405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:02.316292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:02.316810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:02.319470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:02.319964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:02.320402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:02.423569 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8271bfd7-9938-4b05-8c8f-e62b1a84f572 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "POST /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85/action" status: 202 len: 1112 microversion: 2.1 time: 0.593472 Sep 27 10:18:02.424239 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2560/4688] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:18:01 2022] POST /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85/action => generated 1112 bytes in 604 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Sep 27 10:18:02.452646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d99b9112-9bea-49fb-a26d-1b73f3bd7c80 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:02.453725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d99b9112-9bea-49fb-a26d-1b73f3bd7c80 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:02.464699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99b9112-9bea-49fb-a26d-1b73f3bd7c80 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:02.471844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99b9112-9bea-49fb-a26d-1b73f3bd7c80 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:02.472416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99b9112-9bea-49fb-a26d-1b73f3bd7c80 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:02.473421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99b9112-9bea-49fb-a26d-1b73f3bd7c80 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:02.474938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99b9112-9bea-49fb-a26d-1b73f3bd7c80 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:02.475744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99b9112-9bea-49fb-a26d-1b73f3bd7c80 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:02.837161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-537caf43-fd66-4901-b672-fadcf637dabf tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:02.837776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-537caf43-fd66-4901-b672-fadcf637dabf tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:02.840901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=10 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:02.857465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-537caf43-fd66-4901-b672-fadcf637dabf tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:02.858255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-537caf43-fd66-4901-b672-fadcf637dabf tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:02.858870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-537caf43-fd66-4901-b672-fadcf637dabf tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:02.859910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-537caf43-fd66-4901-b672-fadcf637dabf tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:02.861570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-537caf43-fd66-4901-b672-fadcf637dabf tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:02.866070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-537caf43-fd66-4901-b672-fadcf637dabf tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:03.518784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d99b9112-9bea-49fb-a26d-1b73f3bd7c80 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:03.531446 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d99b9112-9bea-49fb-a26d-1b73f3bd7c80 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 1.082998 Sep 27 10:18:03.534738 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2129/4689] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:02 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 1087 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:03.539991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-47d7054d-d843-4941-a55e-39a96799f0d0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:03.540834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-47d7054d-d843-4941-a55e-39a96799f0d0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:03.593916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47d7054d-d843-4941-a55e-39a96799f0d0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:03.595396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47d7054d-d843-4941-a55e-39a96799f0d0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:03.599938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47d7054d-d843-4941-a55e-39a96799f0d0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:03.607653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47d7054d-d843-4941-a55e-39a96799f0d0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:03.612446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47d7054d-d843-4941-a55e-39a96799f0d0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:03.618701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47d7054d-d843-4941-a55e-39a96799f0d0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:03.800759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-537caf43-fd66-4901-b672-fadcf637dabf tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:03.815260 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-537caf43-fd66-4901-b672-fadcf637dabf tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1673 microversion: 2.1 time: 0.982790 Sep 27 10:18:03.815862 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2561/4690] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:02 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1673 bytes in 984 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:04.562385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b2a577e5-e72c-4568-843e-966325236726 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:04.563705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b2a577e5-e72c-4568-843e-966325236726 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:04.575665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-47d7054d-d843-4941-a55e-39a96799f0d0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:04.601241 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-47d7054d-d843-4941-a55e-39a96799f0d0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1309 microversion: 2.1 time: 1.059369 Sep 27 10:18:04.603589 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2130/4691] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:03 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1309 bytes in 1068 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:04.616843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2a577e5-e72c-4568-843e-966325236726 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:04.617728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2a577e5-e72c-4568-843e-966325236726 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:04.619662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2a577e5-e72c-4568-843e-966325236726 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:04.621880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2a577e5-e72c-4568-843e-966325236726 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:04.623042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2a577e5-e72c-4568-843e-966325236726 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:04.628054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2a577e5-e72c-4568-843e-966325236726 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:04.840739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cbdcfd8b-8928-4a92-84a4-3e5baeeaf6ba tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:04.841818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cbdcfd8b-8928-4a92-84a4-3e5baeeaf6ba tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:04.849398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbdcfd8b-8928-4a92-84a4-3e5baeeaf6ba tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:04.853992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbdcfd8b-8928-4a92-84a4-3e5baeeaf6ba tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:04.854991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbdcfd8b-8928-4a92-84a4-3e5baeeaf6ba tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:04.855951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbdcfd8b-8928-4a92-84a4-3e5baeeaf6ba tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:04.857514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbdcfd8b-8928-4a92-84a4-3e5baeeaf6ba tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:04.858307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbdcfd8b-8928-4a92-84a4-3e5baeeaf6ba tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:05.010781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b2a577e5-e72c-4568-843e-966325236726 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:05.025506 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b2a577e5-e72c-4568-843e-966325236726 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.467410 Sep 27 10:18:05.028744 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2562/4692] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:04 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:05.458041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cbdcfd8b-8928-4a92-84a4-3e5baeeaf6ba tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:05.479899 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cbdcfd8b-8928-4a92-84a4-3e5baeeaf6ba tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1673 microversion: 2.1 time: 0.642106 Sep 27 10:18:05.480521 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2131/4693] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:04 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1673 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:05.634041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ea0793b7-2305-4ad5-b495-36fd91ef0249 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:05.635033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ea0793b7-2305-4ad5-b495-36fd91ef0249 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:05.650193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea0793b7-2305-4ad5-b495-36fd91ef0249 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:05.652915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea0793b7-2305-4ad5-b495-36fd91ef0249 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:05.653570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea0793b7-2305-4ad5-b495-36fd91ef0249 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:05.656335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea0793b7-2305-4ad5-b495-36fd91ef0249 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:05.658069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea0793b7-2305-4ad5-b495-36fd91ef0249 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:05.661589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea0793b7-2305-4ad5-b495-36fd91ef0249 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:05.944412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ea0793b7-2305-4ad5-b495-36fd91ef0249 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:05.959623 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ea0793b7-2305-4ad5-b495-36fd91ef0249 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1309 microversion: 2.1 time: 0.329397 Sep 27 10:18:05.960252 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2563/4694] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:05 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1309 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:06.053657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7bfb167c-9c31-4e5c-9099-1a51be2b5730 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:06.054292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7bfb167c-9c31-4e5c-9099-1a51be2b5730 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:06.069115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfb167c-9c31-4e5c-9099-1a51be2b5730 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:06.069621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfb167c-9c31-4e5c-9099-1a51be2b5730 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:06.070057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfb167c-9c31-4e5c-9099-1a51be2b5730 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:06.070773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfb167c-9c31-4e5c-9099-1a51be2b5730 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:06.071224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfb167c-9c31-4e5c-9099-1a51be2b5730 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:06.071647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bfb167c-9c31-4e5c-9099-1a51be2b5730 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:06.090058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:06.507448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a29630e2-575c-4b35-b7a1-953c48f243f3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:06.509639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a29630e2-575c-4b35-b7a1-953c48f243f3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:06.518328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a29630e2-575c-4b35-b7a1-953c48f243f3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:06.540954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a29630e2-575c-4b35-b7a1-953c48f243f3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:06.540954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a29630e2-575c-4b35-b7a1-953c48f243f3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:06.540954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a29630e2-575c-4b35-b7a1-953c48f243f3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:06.540954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a29630e2-575c-4b35-b7a1-953c48f243f3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:06.540954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a29630e2-575c-4b35-b7a1-953c48f243f3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:06.911616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a29630e2-575c-4b35-b7a1-953c48f243f3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '279d3662f60c4a979c22ee3e8fa92865', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e2e00c90216b4d1b80ac7892ee23550f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:06.919950 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a29630e2-575c-4b35-b7a1-953c48f243f3 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 200 len: 1605 microversion: 2.1 time: 0.416604 Sep 27 10:18:06.930859 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2564/4695] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:06 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 1605 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:06.943575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5baff2bc-04a5-4f4d-b12c-2c41e1de3644 req-76cd498c-d75c-47a9-9a40-48ec97fe6ed2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8696cba0-e2e2-4f51-858d-3b502152a6ed", "name": "network-vif-plugged", "status": "completed", "tag": "1e961036-096a-4369-b689-86c91be70173"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:06.978852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5baff2bc-04a5-4f4d-b12c-2c41e1de3644 req-76cd498c-d75c-47a9-9a40-48ec97fe6ed2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:06.991856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5baff2bc-04a5-4f4d-b12c-2c41e1de3644 req-76cd498c-d75c-47a9-9a40-48ec97fe6ed2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:06.992583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5baff2bc-04a5-4f4d-b12c-2c41e1de3644 req-76cd498c-d75c-47a9-9a40-48ec97fe6ed2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:07.008156 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5baff2bc-04a5-4f4d-b12c-2c41e1de3644 req-76cd498c-d75c-47a9-9a40-48ec97fe6ed2 service nova] Creating event network-vif-plugged:1e961036-096a-4369-b689-86c91be70173 for instance 8696cba0-e2e2-4f51-858d-3b502152a6ed on nch2-master-1 Sep 27 10:18:07.016025 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5baff2bc-04a5-4f4d-b12c-2c41e1de3644 req-76cd498c-d75c-47a9-9a40-48ec97fe6ed2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.076626 Sep 27 10:18:07.017552 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2565/4696] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:18:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:18:07.024479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e5de0439-3edf-4a0c-a874-d6c1f2f6e51f req-a2119368-b7bf-4c10-9cdc-b216a4d94f45 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "779046b2-2e5b-4946-af25-c111c590a14e", "name": "network-vif-deleted", "tag": "37f44fa9-2c33-45ba-a074-f37e0af9c6dc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:07.037377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e5de0439-3edf-4a0c-a874-d6c1f2f6e51f req-a2119368-b7bf-4c10-9cdc-b216a4d94f45 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:07.038867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e5de0439-3edf-4a0c-a874-d6c1f2f6e51f req-a2119368-b7bf-4c10-9cdc-b216a4d94f45 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:07.039498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e5de0439-3edf-4a0c-a874-d6c1f2f6e51f req-a2119368-b7bf-4c10-9cdc-b216a4d94f45 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:07.049006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7bfb167c-9c31-4e5c-9099-1a51be2b5730 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:07.056095 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e5de0439-3edf-4a0c-a874-d6c1f2f6e51f req-a2119368-b7bf-4c10-9cdc-b216a4d94f45 service nova] Creating event network-vif-deleted:37f44fa9-2c33-45ba-a074-f37e0af9c6dc for instance 779046b2-2e5b-4946-af25-c111c590a14e on nch1-master-1 Sep 27 10:18:07.062044 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e5de0439-3edf-4a0c-a874-d6c1f2f6e51f req-a2119368-b7bf-4c10-9cdc-b216a4d94f45 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042043 Sep 27 10:18:07.064220 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2566/4697] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:18:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:18:07.089571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1eaa02d9-c33e-4dfd-9ac3-088923315785 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:07.093113 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7bfb167c-9c31-4e5c-9099-1a51be2b5730 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 1.036413 Sep 27 10:18:07.094467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1eaa02d9-c33e-4dfd-9ac3-088923315785 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:07.096574 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2132/4698] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:06 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 1048 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:07.109650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eaa02d9-c33e-4dfd-9ac3-088923315785 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:07.110305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eaa02d9-c33e-4dfd-9ac3-088923315785 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:07.110986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eaa02d9-c33e-4dfd-9ac3-088923315785 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:07.111943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eaa02d9-c33e-4dfd-9ac3-088923315785 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:07.112626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eaa02d9-c33e-4dfd-9ac3-088923315785 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:07.118803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1eaa02d9-c33e-4dfd-9ac3-088923315785 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:07.424375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1eaa02d9-c33e-4dfd-9ac3-088923315785 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:07.434734 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1eaa02d9-c33e-4dfd-9ac3-088923315785 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1309 microversion: 2.1 time: 0.368957 Sep 27 10:18:07.435346 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2567/4699] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:07 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1309 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:07.959168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:07.960156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:07.978936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:07.979456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:07.980168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:07.985495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:07.986766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:07.987538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:08.044863 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] HTTP exception thrown: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Request method failure captured: Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e HTTP/1.1 Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMszABSno_2kqm3BW6SUWKFiDAujuW5Xnb7TMluE7Z34HQ5ZZ0Gk_H4nclSs5JgEczlL-fW3yspIl5udeCuxAqzrKWf09XSghDeGo6ZAMbfaXvl-0XHv0hoOQwx_Y5MP4or9tGcXTOMVnEVPWO9TJ2xA61_3oGcuZds7KIPUHECLVq_Q Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e2e00c90216b4d1b80ac7892ee23550f"}]}] Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 279d3662f60c4a979c22ee3e8fa92865 Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '779046b2-2e5b-4946-af25-c111c590a14e'} Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:08.049168 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:08.053980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:08.053980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:08.053980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:08.053980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:08.053980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:08.053980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:08.053980 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:08.056709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Returning 404 to user: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:08.062575 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f506b50a-e052-4c72-a9e2-7c05bc078a82 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 404 len: 111 microversion: 2.1 time: 0.108351 Sep 27 10:18:08.063951 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2133/4700] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:07 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 111 bytes in 111 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:08.073698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3da1ef84-c074-4ffe-a741-a8b91d71a5cb tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '.version_select at 0x7f7f624d0ee0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:08.121375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7b864092-c984-4f63-b9dc-9ff5aeed4513 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:08.121999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7b864092-c984-4f63-b9dc-9ff5aeed4513 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:08.126002 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3da1ef84-c074-4ffe-a741-a8b91d71a5cb tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1072571665" status: 202 len: 0 microversion: 2.1 time: 0.056420 Sep 27 10:18:08.127288 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2568/4701] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:18:08 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1072571665 => generated 0 bytes in 58 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:18:08.144168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b864092-c984-4f63-b9dc-9ff5aeed4513 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:08.144981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b864092-c984-4f63-b9dc-9ff5aeed4513 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:08.145588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b864092-c984-4f63-b9dc-9ff5aeed4513 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:08.146750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b864092-c984-4f63-b9dc-9ff5aeed4513 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:08.147869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b864092-c984-4f63-b9dc-9ff5aeed4513 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:08.148485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b864092-c984-4f63-b9dc-9ff5aeed4513 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:08.460183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-421af9f9-0962-4916-976d-f38a6b100c39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:08.461381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-421af9f9-0962-4916-976d-f38a6b100c39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:08.469944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-421af9f9-0962-4916-976d-f38a6b100c39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:08.470829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-421af9f9-0962-4916-976d-f38a6b100c39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:08.471654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-421af9f9-0962-4916-976d-f38a6b100c39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:08.473373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-421af9f9-0962-4916-976d-f38a6b100c39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:08.474084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-421af9f9-0962-4916-976d-f38a6b100c39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:08.474802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-421af9f9-0962-4916-976d-f38a6b100c39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:08.572814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7b864092-c984-4f63-b9dc-9ff5aeed4513 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:08.582789 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7b864092-c984-4f63-b9dc-9ff5aeed4513 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.467756 Sep 27 10:18:08.588826 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2134/4702] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:08 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:08.942530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-421af9f9-0962-4916-976d-f38a6b100c39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:08.996626 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-421af9f9-0962-4916-976d-f38a6b100c39 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1309 microversion: 2.1 time: 0.540649 Sep 27 10:18:09.002537 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2569/4703] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:08 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1309 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:09.621371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c2e049f8-486c-4f1d-b836-1dac65f696b6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:09.623916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c2e049f8-486c-4f1d-b836-1dac65f696b6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:09.633106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e049f8-486c-4f1d-b836-1dac65f696b6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:09.633601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e049f8-486c-4f1d-b836-1dac65f696b6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:09.634033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e049f8-486c-4f1d-b836-1dac65f696b6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:09.634766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e049f8-486c-4f1d-b836-1dac65f696b6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:09.635229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e049f8-486c-4f1d-b836-1dac65f696b6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:09.635657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2e049f8-486c-4f1d-b836-1dac65f696b6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:09.930836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c2e049f8-486c-4f1d-b836-1dac65f696b6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:09.939897 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c2e049f8-486c-4f1d-b836-1dac65f696b6 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.323303 Sep 27 10:18:09.945591 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2135/4704] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:09 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:10.036724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5e89514f-115a-4865-9c30-99658ec5145b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:10.037990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5e89514f-115a-4865-9c30-99658ec5145b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:10.049302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e89514f-115a-4865-9c30-99658ec5145b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:10.050366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e89514f-115a-4865-9c30-99658ec5145b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:10.051064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e89514f-115a-4865-9c30-99658ec5145b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:10.053822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e89514f-115a-4865-9c30-99658ec5145b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:10.055253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e89514f-115a-4865-9c30-99658ec5145b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:10.055910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e89514f-115a-4865-9c30-99658ec5145b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:10.091113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:10.391361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5e89514f-115a-4865-9c30-99658ec5145b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:10.403655 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5e89514f-115a-4865-9c30-99658ec5145b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1490 microversion: 2.1 time: 0.383456 Sep 27 10:18:10.405157 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2570/4705] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:10 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1490 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:10.428296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-15972259-8de5-4808-ad22-219280ba0469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'update', calling method: >, body: {"server": {"name": "tempest-\u00cd\u00f1st\u00e1\u00f1c\u00e9-545123829"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:10.431009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-15972259-8de5-4808-ad22-219280ba0469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:10.438257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15972259-8de5-4808-ad22-219280ba0469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:10.438963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15972259-8de5-4808-ad22-219280ba0469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:10.439737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15972259-8de5-4808-ad22-219280ba0469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:10.440456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15972259-8de5-4808-ad22-219280ba0469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:10.440946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15972259-8de5-4808-ad22-219280ba0469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:10.441649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15972259-8de5-4808-ad22-219280ba0469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:10.529105 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-15972259-8de5-4808-ad22-219280ba0469 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "PUT /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1074 microversion: 2.1 time: 0.105125 Sep 27 10:18:10.530569 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2136/4706] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:18:10 2022] PUT /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1074 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:10.555354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e931eae5-b0b4-428f-8f53-8d364f9084ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:10.556354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e931eae5-b0b4-428f-8f53-8d364f9084ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:10.566116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e931eae5-b0b4-428f-8f53-8d364f9084ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:10.566614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e931eae5-b0b4-428f-8f53-8d364f9084ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:10.567213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e931eae5-b0b4-428f-8f53-8d364f9084ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:10.568217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e931eae5-b0b4-428f-8f53-8d364f9084ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:10.568895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e931eae5-b0b4-428f-8f53-8d364f9084ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:10.569325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e931eae5-b0b4-428f-8f53-8d364f9084ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:10.874285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e931eae5-b0b4-428f-8f53-8d364f9084ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:10.889368 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e931eae5-b0b4-428f-8f53-8d364f9084ab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1500 microversion: 2.1 time: 0.341307 Sep 27 10:18:10.890277 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2571/4707] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:10 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1500 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:10.914144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bf587088-6768-4d23-b644-5a45cf1669b0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:10.914759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bf587088-6768-4d23-b644-5a45cf1669b0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:10.926494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf587088-6768-4d23-b644-5a45cf1669b0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:10.926991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf587088-6768-4d23-b644-5a45cf1669b0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:10.927648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf587088-6768-4d23-b644-5a45cf1669b0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:10.928606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf587088-6768-4d23-b644-5a45cf1669b0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:10.929169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf587088-6768-4d23-b644-5a45cf1669b0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:10.929597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf587088-6768-4d23-b644-5a45cf1669b0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:10.969255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dee14d9a-4bc5-45b0-a50b-03ff0afc6e6d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:10.969255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dee14d9a-4bc5-45b0-a50b-03ff0afc6e6d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:10.974925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee14d9a-4bc5-45b0-a50b-03ff0afc6e6d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:10.977618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee14d9a-4bc5-45b0-a50b-03ff0afc6e6d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:10.980068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee14d9a-4bc5-45b0-a50b-03ff0afc6e6d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:10.981695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee14d9a-4bc5-45b0-a50b-03ff0afc6e6d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:10.986082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee14d9a-4bc5-45b0-a50b-03ff0afc6e6d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:10.986742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee14d9a-4bc5-45b0-a50b-03ff0afc6e6d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:11.276417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dee14d9a-4bc5-45b0-a50b-03ff0afc6e6d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:11.296951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bf587088-6768-4d23-b644-5a45cf1669b0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:11.314537 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dee14d9a-4bc5-45b0-a50b-03ff0afc6e6d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.353325 Sep 27 10:18:11.315596 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2572/4708] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:10 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:11.339978 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bf587088-6768-4d23-b644-5a45cf1669b0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1500 microversion: 2.1 time: 0.429734 Sep 27 10:18:11.342814 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2137/4709] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:10 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1500 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:11.371547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-76695e0e-cc72-46ee-9ea9-7bd46b56113b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:11.380840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76695e0e-cc72-46ee-9ea9-7bd46b56113b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:11.381666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76695e0e-cc72-46ee-9ea9-7bd46b56113b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:11.382342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76695e0e-cc72-46ee-9ea9-7bd46b56113b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:11.452873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-76695e0e-cc72-46ee-9ea9-7bd46b56113b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Going to try to stop instance {{(pid=145968) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 10:18:11.517115 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-76695e0e-cc72-46ee-9ea9-7bd46b56113b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "POST /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed/action" status: 202 len: 0 microversion: 2.1 time: 0.150100 Sep 27 10:18:11.518427 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2573/4710] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:18:11 2022] POST /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed/action => generated 0 bytes in 153 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:18:11.528645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6932cc68-5a94-4e95-b47b-105f9ede9e2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:11.530016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6932cc68-5a94-4e95-b47b-105f9ede9e2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:11.542537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6932cc68-5a94-4e95-b47b-105f9ede9e2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:11.544398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6932cc68-5a94-4e95-b47b-105f9ede9e2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:11.545034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6932cc68-5a94-4e95-b47b-105f9ede9e2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:11.545880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6932cc68-5a94-4e95-b47b-105f9ede9e2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:11.546513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6932cc68-5a94-4e95-b47b-105f9ede9e2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:11.547810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6932cc68-5a94-4e95-b47b-105f9ede9e2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:11.869798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6932cc68-5a94-4e95-b47b-105f9ede9e2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:11.884226 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6932cc68-5a94-4e95-b47b-105f9ede9e2c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.357016 Sep 27 10:18:11.887502 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2138/4711] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:11 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:12.339629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cbdebd35-2337-4f7c-b1bb-4d8f7d97e997 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:12.340894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cbdebd35-2337-4f7c-b1bb-4d8f7d97e997 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:12.350595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdebd35-2337-4f7c-b1bb-4d8f7d97e997 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:12.351571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdebd35-2337-4f7c-b1bb-4d8f7d97e997 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:12.352232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdebd35-2337-4f7c-b1bb-4d8f7d97e997 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:12.353949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdebd35-2337-4f7c-b1bb-4d8f7d97e997 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:12.356207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdebd35-2337-4f7c-b1bb-4d8f7d97e997 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:12.358333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbdebd35-2337-4f7c-b1bb-4d8f7d97e997 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:12.707116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cbdebd35-2337-4f7c-b1bb-4d8f7d97e997 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:12.720970 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cbdebd35-2337-4f7c-b1bb-4d8f7d97e997 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.385946 Sep 27 10:18:12.733883 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2574/4712] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:12 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:12.912110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c821be8-7129-4b6b-96d0-9da254d57d69 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:12.913552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c821be8-7129-4b6b-96d0-9da254d57d69 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:12.932545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c821be8-7129-4b6b-96d0-9da254d57d69 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:12.935636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c821be8-7129-4b6b-96d0-9da254d57d69 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:12.937134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c821be8-7129-4b6b-96d0-9da254d57d69 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:12.939482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c821be8-7129-4b6b-96d0-9da254d57d69 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:12.940295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c821be8-7129-4b6b-96d0-9da254d57d69 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:12.941106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c821be8-7129-4b6b-96d0-9da254d57d69 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:13.090042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:13.090557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:13.094273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:13.114807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:13.116591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:13.119630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:13.124226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:13.125989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:13.126721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:13.152743 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] HTTP exception thrown: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Request method failure captured: Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e HTTP/1.1 Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMszABSno_2kqm3BW6SUWKFiDAujuW5Xnb7TMluE7Z34HQ5ZZ0Gk_H4nclSs5JgEczlL-fW3yspIl5udeCuxAqzrKWf09XSghDeGo6ZAMbfaXvl-0XHv0hoOQwx_Y5MP4or9tGcXTOMVnEVPWO9TJ2xA61_3oGcuZds7KIPUHECLVq_Q Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e2e00c90216b4d1b80ac7892ee23550f"}]}] Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 279d3662f60c4a979c22ee3e8fa92865 Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '779046b2-2e5b-4946-af25-c111c590a14e'} Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:13.155316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:13.159211 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:13.159211 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:13.159211 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:13.159211 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:13.159211 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:13.159211 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:13.159211 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:13.159211 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:13.159211 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:13.159211 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:13.159211 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:13.163152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Returning 404 to user: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:13.164537 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7b578439-ac32-4553-97ab-c9ea0fbf7afd tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "DELETE /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 404 len: 111 microversion: 2.1 time: 0.078706 Sep 27 10:18:13.166285 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2575/4713] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:13 2022] DELETE /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:18:13.175582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:13.176642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 779046b2-2e5b-4946-af25-c111c590a14e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:13.187932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:13.188446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:13.189166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:13.190540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:13.191211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:13.191653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:13.213750 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] HTTP exception thrown: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Request method failure captured: Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e HTTP/1.1 Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMszABSno_2kqm3BW6SUWKFiDAujuW5Xnb7TMluE7Z34HQ5ZZ0Gk_H4nclSs5JgEczlL-fW3yspIl5udeCuxAqzrKWf09XSghDeGo6ZAMbfaXvl-0XHv0hoOQwx_Y5MP4or9tGcXTOMVnEVPWO9TJ2xA61_3oGcuZds7KIPUHECLVq_Q Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e2e00c90216b4d1b80ac7892ee23550f"}]}] Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 279d3662f60c4a979c22ee3e8fa92865 Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '779046b2-2e5b-4946-af25-c111c590a14e'} Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:13.215786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:13.221038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:13.221038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:13.221038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:13.221038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:13.221038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:13.221038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. Sep 27 10:18:13.221038 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:13.221038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Returning 404 to user: Instance 779046b2-2e5b-4946-af25-c111c590a14e could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:13.221038 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d3736870-1b8d-41da-acd1-32620090a782 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e" status: 404 len: 111 microversion: 2.1 time: 0.047977 Sep 27 10:18:13.222625 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2576/4714] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:13 2022] GET /compute/v2.1/servers/779046b2-2e5b-4946-af25-c111c590a14e => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:13.763162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b98f983c-2a0d-45f5-8fe0-90be54045415 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:13.768691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b98f983c-2a0d-45f5-8fe0-90be54045415 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:13.779153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b98f983c-2a0d-45f5-8fe0-90be54045415 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:13.780201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b98f983c-2a0d-45f5-8fe0-90be54045415 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:13.781257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b98f983c-2a0d-45f5-8fe0-90be54045415 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:13.782461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b98f983c-2a0d-45f5-8fe0-90be54045415 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:13.784021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b98f983c-2a0d-45f5-8fe0-90be54045415 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:13.784753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b98f983c-2a0d-45f5-8fe0-90be54045415 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:13.937339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c821be8-7129-4b6b-96d0-9da254d57d69 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:13.955532 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c821be8-7129-4b6b-96d0-9da254d57d69 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 1.049122 Sep 27 10:18:13.960154 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2139/4715] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:12 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 1053 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:14.370567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b98f983c-2a0d-45f5-8fe0-90be54045415 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:14.382287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b98f983c-2a0d-45f5-8fe0-90be54045415 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.626426 Sep 27 10:18:14.384203 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2577/4716] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:13 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 630 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:14.523643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:14.524366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:14.528149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=75,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:14.533884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:14.538764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:14.541223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:14.542243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:14.543962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:14.545265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:14.572168 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] HTTP exception thrown: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Request method failure captured: Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 HTTP/1.1 Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMszABSno_2kqm3BW6SUWKFiDAujuW5Xnb7TMluE7Z34HQ5ZZ0Gk_H4nclSs5JgEczlL-fW3yspIl5udeCuxAqzrKWf09XSghDeGo6ZAMbfaXvl-0XHv0hoOQwx_Y5MP4or9tGcXTOMVnEVPWO9TJ2xA61_3oGcuZds7KIPUHECLVq_Q Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e2e00c90216b4d1b80ac7892ee23550f"}]}] Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 279d3662f60c4a979c22ee3e8fa92865 Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '49f9e4be-7594-4603-8f69-8765513f8b89'} Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:14.574549 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:14.578825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:18:14.578825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:14.578825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:14.578825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:14.578825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:14.578825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:14.578825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:14.578825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:14.578825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:14.578825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:18:14.578825 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:14.582190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Returning 404 to user: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:14.583336 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5972fd2b-ca34-45c8-b04f-c7fd81c36882 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "DELETE /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 404 len: 111 microversion: 2.1 time: 0.063945 Sep 27 10:18:14.585810 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2140/4717] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:14 2022] DELETE /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:14.593752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:14.594647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] [instance: 49f9e4be-7594-4603-8f69-8765513f8b89] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:14.611413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:14.611944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:14.612422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:14.618857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:14.619660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:14.622779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:14.660288 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] HTTP exception thrown: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Request method failure captured: Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 HTTP/1.1 Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMszABSno_2kqm3BW6SUWKFiDAujuW5Xnb7TMluE7Z34HQ5ZZ0Gk_H4nclSs5JgEczlL-fW3yspIl5udeCuxAqzrKWf09XSghDeGo6ZAMbfaXvl-0XHv0hoOQwx_Y5MP4or9tGcXTOMVnEVPWO9TJ2xA61_3oGcuZds7KIPUHECLVq_Q Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/e2e00c90216b4d1b80ac7892ee23550f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_e2e00c90216b4d1b80ac7892ee23550f"}]}] Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: e2e00c90216b4d1b80ac7892ee23550f Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1476802274 Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 279d3662f60c4a979c22ee3e8fa92865 Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeTestJSON-1476802274-project Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '49f9e4be-7594-4603-8f69-8765513f8b89'} Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:14.661847 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:14.670691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:14.670691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:14.670691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:14.670691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:14.670691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:14.670691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. Sep 27 10:18:14.670691 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:14.670691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] Returning 404 to user: Instance 49f9e4be-7594-4603-8f69-8765513f8b89 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:14.670691 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3774c1c7-1611-4e68-b7de-9d82532e1cd2 tempest-AttachVolumeTestJSON-1476802274 tempest-AttachVolumeTestJSON-1476802274-project] 10.222.0.26 "GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89" status: 404 len: 111 microversion: 2.1 time: 0.076875 Sep 27 10:18:14.673469 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2578/4718] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:14 2022] GET /compute/v2.1/servers/49f9e4be-7594-4603-8f69-8765513f8b89 => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:14.982318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-40981df7-fc5d-402e-a560-9447bb08c0b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:14.983755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-40981df7-fc5d-402e-a560-9447bb08c0b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:14.991448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40981df7-fc5d-402e-a560-9447bb08c0b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:14.991956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40981df7-fc5d-402e-a560-9447bb08c0b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:14.992701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40981df7-fc5d-402e-a560-9447bb08c0b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:14.993510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40981df7-fc5d-402e-a560-9447bb08c0b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:14.994194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40981df7-fc5d-402e-a560-9447bb08c0b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:14.994839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40981df7-fc5d-402e-a560-9447bb08c0b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:15.409748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fead8c03-f253-434b-8ddc-3044be5590ec tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:15.411234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fead8c03-f253-434b-8ddc-3044be5590ec tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:15.426515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fead8c03-f253-434b-8ddc-3044be5590ec tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:15.427335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fead8c03-f253-434b-8ddc-3044be5590ec tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:15.427991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fead8c03-f253-434b-8ddc-3044be5590ec tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:15.428938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fead8c03-f253-434b-8ddc-3044be5590ec tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:15.429410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fead8c03-f253-434b-8ddc-3044be5590ec tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:15.429848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fead8c03-f253-434b-8ddc-3044be5590ec tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:15.523725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-40981df7-fc5d-402e-a560-9447bb08c0b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:15.540637 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-40981df7-fc5d-402e-a560-9447bb08c0b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.561968 Sep 27 10:18:15.541785 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2141/4719] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:14 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:16.476154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fead8c03-f253-434b-8ddc-3044be5590ec tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:16.491055 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fead8c03-f253-434b-8ddc-3044be5590ec tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 1.085206 Sep 27 10:18:16.492137 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2579/4720] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:15 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 1087 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:16.568698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-db13ff90-16d3-4175-ae55-46717e457034 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:16.570265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-db13ff90-16d3-4175-ae55-46717e457034 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:16.589090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db13ff90-16d3-4175-ae55-46717e457034 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:16.592017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db13ff90-16d3-4175-ae55-46717e457034 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:16.592635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db13ff90-16d3-4175-ae55-46717e457034 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:16.594138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db13ff90-16d3-4175-ae55-46717e457034 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:16.596454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db13ff90-16d3-4175-ae55-46717e457034 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:16.597203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db13ff90-16d3-4175-ae55-46717e457034 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:17.097111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-db13ff90-16d3-4175-ae55-46717e457034 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:17.105950 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-db13ff90-16d3-4175-ae55-46717e457034 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.544142 Sep 27 10:18:17.107245 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2142/4721] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:16 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:17.516994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-337c1ca5-0ed2-4a48-9761-05a5b11befcc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:17.517610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-337c1ca5-0ed2-4a48-9761-05a5b11befcc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:17.534853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-337c1ca5-0ed2-4a48-9761-05a5b11befcc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:17.535597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-337c1ca5-0ed2-4a48-9761-05a5b11befcc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:17.535783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-337c1ca5-0ed2-4a48-9761-05a5b11befcc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:17.536516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-337c1ca5-0ed2-4a48-9761-05a5b11befcc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:17.537809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-337c1ca5-0ed2-4a48-9761-05a5b11befcc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:17.538308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-337c1ca5-0ed2-4a48-9761-05a5b11befcc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:18.129961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-29e0829a-8183-4b5c-bfbb-5dab6dcd0693 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:18.130616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-29e0829a-8183-4b5c-bfbb-5dab6dcd0693 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:18.145159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29e0829a-8183-4b5c-bfbb-5dab6dcd0693 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:18.146030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29e0829a-8183-4b5c-bfbb-5dab6dcd0693 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:18.147400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29e0829a-8183-4b5c-bfbb-5dab6dcd0693 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:18.148357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29e0829a-8183-4b5c-bfbb-5dab6dcd0693 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:18.149041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29e0829a-8183-4b5c-bfbb-5dab6dcd0693 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:18.150325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29e0829a-8183-4b5c-bfbb-5dab6dcd0693 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:18.288963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-337c1ca5-0ed2-4a48-9761-05a5b11befcc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:18.302611 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-337c1ca5-0ed2-4a48-9761-05a5b11befcc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.789534 Sep 27 10:18:18.304737 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2580/4722] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:17 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 792 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:18.454033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-29e0829a-8183-4b5c-bfbb-5dab6dcd0693 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:18.476008 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-29e0829a-8183-4b5c-bfbb-5dab6dcd0693 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.349795 Sep 27 10:18:18.478813 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2143/4723] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:18 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:18.773680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a530171a-eb43-4c0f-83c5-f8f9fd3d895b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:18.776039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a530171a-eb43-4c0f-83c5-f8f9fd3d895b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:18.782568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a530171a-eb43-4c0f-83c5-f8f9fd3d895b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:18.783448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a530171a-eb43-4c0f-83c5-f8f9fd3d895b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:18.783883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a530171a-eb43-4c0f-83c5-f8f9fd3d895b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:18.784625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a530171a-eb43-4c0f-83c5-f8f9fd3d895b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:18.785287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a530171a-eb43-4c0f-83c5-f8f9fd3d895b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:18.785736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a530171a-eb43-4c0f-83c5-f8f9fd3d895b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:18.863288 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a530171a-eb43-4c0f-83c5-f8f9fd3d895b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "POST /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436/action" status: 200 len: 24568 microversion: 2.1 time: 0.093847 Sep 27 10:18:18.873318 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2581/4724] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:18:18 2022] POST /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436/action => generated 24568 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:18:19.327925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-41edbc04-188a-4bd8-8bbe-7022a73a3cbc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:19.328547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-41edbc04-188a-4bd8-8bbe-7022a73a3cbc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:19.348911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41edbc04-188a-4bd8-8bbe-7022a73a3cbc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:19.349636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41edbc04-188a-4bd8-8bbe-7022a73a3cbc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:19.350550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41edbc04-188a-4bd8-8bbe-7022a73a3cbc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:19.351479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41edbc04-188a-4bd8-8bbe-7022a73a3cbc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:19.352112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41edbc04-188a-4bd8-8bbe-7022a73a3cbc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:19.352549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41edbc04-188a-4bd8-8bbe-7022a73a3cbc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:19.500538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e59997e6-7fce-466e-8113-a51663cb72c3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:19.501435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e59997e6-7fce-466e-8113-a51663cb72c3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:19.521024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e59997e6-7fce-466e-8113-a51663cb72c3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:19.521578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e59997e6-7fce-466e-8113-a51663cb72c3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:19.522026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e59997e6-7fce-466e-8113-a51663cb72c3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:19.522764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e59997e6-7fce-466e-8113-a51663cb72c3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:19.523227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e59997e6-7fce-466e-8113-a51663cb72c3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:19.523656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e59997e6-7fce-466e-8113-a51663cb72c3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:19.684209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-41edbc04-188a-4bd8-8bbe-7022a73a3cbc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:19.696721 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-41edbc04-188a-4bd8-8bbe-7022a73a3cbc tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.372705 Sep 27 10:18:19.698209 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2144/4725] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:19 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:20.047769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e59997e6-7fce-466e-8113-a51663cb72c3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:20.060336 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e59997e6-7fce-466e-8113-a51663cb72c3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.564219 Sep 27 10:18:20.062474 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2582/4726] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:19 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:20.728293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dd201227-dc03-434f-ad5c-e3d03b25fa3f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:20.730005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dd201227-dc03-434f-ad5c-e3d03b25fa3f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:20.747331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd201227-dc03-434f-ad5c-e3d03b25fa3f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:20.748071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd201227-dc03-434f-ad5c-e3d03b25fa3f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:20.748762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd201227-dc03-434f-ad5c-e3d03b25fa3f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:20.749964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd201227-dc03-434f-ad5c-e3d03b25fa3f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:20.750639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd201227-dc03-434f-ad5c-e3d03b25fa3f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:20.753484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd201227-dc03-434f-ad5c-e3d03b25fa3f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:21.083200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dd201227-dc03-434f-ad5c-e3d03b25fa3f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:21.088444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f971fa39-5d16-474e-8c1c-247d934a1ccc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:21.089112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f971fa39-5d16-474e-8c1c-247d934a1ccc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:21.117027 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dd201227-dc03-434f-ad5c-e3d03b25fa3f tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.379787 Sep 27 10:18:21.117742 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2145/4727] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:20 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:21.132154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f971fa39-5d16-474e-8c1c-247d934a1ccc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:21.132154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f971fa39-5d16-474e-8c1c-247d934a1ccc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:21.132154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f971fa39-5d16-474e-8c1c-247d934a1ccc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:21.132154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f971fa39-5d16-474e-8c1c-247d934a1ccc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:21.132154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f971fa39-5d16-474e-8c1c-247d934a1ccc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:21.132154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f971fa39-5d16-474e-8c1c-247d934a1ccc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:21.482684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f971fa39-5d16-474e-8c1c-247d934a1ccc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:21.515776 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f971fa39-5d16-474e-8c1c-247d934a1ccc tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.428009 Sep 27 10:18:21.517619 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2583/4728] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:21 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:22.141750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c404ff68-aaba-46cc-9a6a-b82a0efbc77b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:22.142930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c404ff68-aaba-46cc-9a6a-b82a0efbc77b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:22.155659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c404ff68-aaba-46cc-9a6a-b82a0efbc77b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:22.172783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c404ff68-aaba-46cc-9a6a-b82a0efbc77b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:22.173250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c404ff68-aaba-46cc-9a6a-b82a0efbc77b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:22.174025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c404ff68-aaba-46cc-9a6a-b82a0efbc77b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:22.174536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c404ff68-aaba-46cc-9a6a-b82a0efbc77b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:22.175158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c404ff68-aaba-46cc-9a6a-b82a0efbc77b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:22.544005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-325e4ba3-4191-4919-a0ff-f800540278df tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:22.544816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-325e4ba3-4191-4919-a0ff-f800540278df tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:22.561776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-325e4ba3-4191-4919-a0ff-f800540278df tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:22.563900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-325e4ba3-4191-4919-a0ff-f800540278df tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:22.565191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-325e4ba3-4191-4919-a0ff-f800540278df tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:22.565997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-325e4ba3-4191-4919-a0ff-f800540278df tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:22.568748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-325e4ba3-4191-4919-a0ff-f800540278df tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:22.572996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-325e4ba3-4191-4919-a0ff-f800540278df tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:22.742672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c404ff68-aaba-46cc-9a6a-b82a0efbc77b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:22.752245 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c404ff68-aaba-46cc-9a6a-b82a0efbc77b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.614217 Sep 27 10:18:22.753249 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2146/4729] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:22 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:22.989821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-325e4ba3-4191-4919-a0ff-f800540278df tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:23.006514 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-325e4ba3-4191-4919-a0ff-f800540278df tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.467368 Sep 27 10:18:23.007724 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2584/4730] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:22 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:23.775994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d4c552e2-a582-4ff3-bcd9-89d2b93ae98b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:23.776507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d4c552e2-a582-4ff3-bcd9-89d2b93ae98b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:23.792627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4c552e2-a582-4ff3-bcd9-89d2b93ae98b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:23.813003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4c552e2-a582-4ff3-bcd9-89d2b93ae98b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:23.815139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4c552e2-a582-4ff3-bcd9-89d2b93ae98b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.022s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:23.816503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4c552e2-a582-4ff3-bcd9-89d2b93ae98b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:23.818252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4c552e2-a582-4ff3-bcd9-89d2b93ae98b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:23.818689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d4c552e2-a582-4ff3-bcd9-89d2b93ae98b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:24.032926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e19c31f-44d6-49ec-81f8-6aa9172470b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:24.033549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9e19c31f-44d6-49ec-81f8-6aa9172470b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:24.041552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e19c31f-44d6-49ec-81f8-6aa9172470b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:24.042326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e19c31f-44d6-49ec-81f8-6aa9172470b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:24.042960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e19c31f-44d6-49ec-81f8-6aa9172470b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:24.043928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e19c31f-44d6-49ec-81f8-6aa9172470b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:24.044542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e19c31f-44d6-49ec-81f8-6aa9172470b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:24.045201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e19c31f-44d6-49ec-81f8-6aa9172470b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:24.331140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9e19c31f-44d6-49ec-81f8-6aa9172470b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:24.343043 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9e19c31f-44d6-49ec-81f8-6aa9172470b6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.314308 Sep 27 10:18:24.344982 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2585/4731] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:24 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:24.517206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d4c552e2-a582-4ff3-bcd9-89d2b93ae98b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:24.529019 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d4c552e2-a582-4ff3-bcd9-89d2b93ae98b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.757080 Sep 27 10:18:24.530195 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2147/4732] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:23 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:25.367659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d0c0292-a9cf-4e55-8b00-da3fb9073f42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:25.368350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2d0c0292-a9cf-4e55-8b00-da3fb9073f42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:25.385302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d0c0292-a9cf-4e55-8b00-da3fb9073f42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:25.389836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d0c0292-a9cf-4e55-8b00-da3fb9073f42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:25.395853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d0c0292-a9cf-4e55-8b00-da3fb9073f42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:25.397211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d0c0292-a9cf-4e55-8b00-da3fb9073f42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:25.405144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d0c0292-a9cf-4e55-8b00-da3fb9073f42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:25.407147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d0c0292-a9cf-4e55-8b00-da3fb9073f42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:25.555563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eeb45ee2-846a-4dc9-8bea-9dc54a8e0361 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:25.556272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eeb45ee2-846a-4dc9-8bea-9dc54a8e0361 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:25.581457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb45ee2-846a-4dc9-8bea-9dc54a8e0361 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:25.585266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb45ee2-846a-4dc9-8bea-9dc54a8e0361 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:25.586382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb45ee2-846a-4dc9-8bea-9dc54a8e0361 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:25.596509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb45ee2-846a-4dc9-8bea-9dc54a8e0361 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:25.597252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb45ee2-846a-4dc9-8bea-9dc54a8e0361 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:25.597863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb45ee2-846a-4dc9-8bea-9dc54a8e0361 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:25.882409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2d0c0292-a9cf-4e55-8b00-da3fb9073f42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:25.912437 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d0c0292-a9cf-4e55-8b00-da3fb9073f42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.548609 Sep 27 10:18:25.916390 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2586/4733] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:25 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:26.167348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eeb45ee2-846a-4dc9-8bea-9dc54a8e0361 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:26.182145 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eeb45ee2-846a-4dc9-8bea-9dc54a8e0361 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.633379 Sep 27 10:18:26.183028 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2148/4734] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:25 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:26.939753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c3725df5-0635-4383-9523-4dd453310831 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:26.940366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c3725df5-0635-4383-9523-4dd453310831 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:26.960454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3725df5-0635-4383-9523-4dd453310831 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:26.962068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3725df5-0635-4383-9523-4dd453310831 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:26.963029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3725df5-0635-4383-9523-4dd453310831 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:26.964784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3725df5-0635-4383-9523-4dd453310831 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:26.965903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3725df5-0635-4383-9523-4dd453310831 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:26.966550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c3725df5-0635-4383-9523-4dd453310831 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:27.208779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-70965c15-b3c6-41a2-8648-5f452946b2d2 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:27.209396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-70965c15-b3c6-41a2-8648-5f452946b2d2 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:27.219967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70965c15-b3c6-41a2-8648-5f452946b2d2 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:27.229403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70965c15-b3c6-41a2-8648-5f452946b2d2 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:27.230108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70965c15-b3c6-41a2-8648-5f452946b2d2 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:27.231189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70965c15-b3c6-41a2-8648-5f452946b2d2 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:27.231814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70965c15-b3c6-41a2-8648-5f452946b2d2 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:27.232416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70965c15-b3c6-41a2-8648-5f452946b2d2 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:27.287734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:27.395053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c3725df5-0635-4383-9523-4dd453310831 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:27.414274 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c3725df5-0635-4383-9523-4dd453310831 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.478964 Sep 27 10:18:27.429439 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2587/4735] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:26 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:27.803902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-70965c15-b3c6-41a2-8648-5f452946b2d2 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:27.813911 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-70965c15-b3c6-41a2-8648-5f452946b2d2 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.609930 Sep 27 10:18:27.825783 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2149/4736] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:27 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:27.847726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-44a78a95-6666-4208-9da3-b90ddcbb8ab7 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Action: 'create', calling method: .version_select at 0x7f7f63715940>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-298117471", "metadata": {"Type": "work"}}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-44a78a95-6666-4208-9da3-b90ddcbb8ab7 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Request method failure captured: Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 121 Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs3zy-ptND6J4DgIgyB1Drq1xuI0URUSCHVgPMnnq_NAHyoxRRrrUhY1ilmxmV5VLTe-mzei91ks2Ji6TnWXuhjRl39Zdeto0ibCl77wnuAHDG7Rk1gpq3s8rDrn5eZObqPXCGrj7etHwQELsNX88zOHDQkIDm5v_k2_n1cTGdp47SA Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a069fad215134b79b6b4949f412edc53"}]}] Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 426331348a8e4d4cba7022ad81ccd1f6 Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-298117471", "metadata": {"Type": "work"}}} Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-298117471', 'metadata': {'Type': 'work'}}}} Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi '#$%' Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:18:27.853316 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-44a78a95-6666-4208-9da3-b90ddcbb8ab7 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:27.858351 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-44a78a95-6666-4208-9da3-b90ddcbb8ab7 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.304286 Sep 27 10:18:27.861329 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2588/4737] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:18:27 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 312 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:18:27.871666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-17c8e5d9-f711-4a56-8b53-6f9e4a25d4b7 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Action: 'create', calling method: .version_select at 0x7f7f624c2700>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1366972933", "metadata": {"Type": "work"}}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-17c8e5d9-f711-4a56-8b53-6f9e4a25d4b7 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Request method failure captured: Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 120 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs3zy-ptND6J4DgIgyB1Drq1xuI0URUSCHVgPMnnq_NAHyoxRRrrUhY1ilmxmV5VLTe-mzei91ks2Ji6TnWXuhjRl39Zdeto0ibCl77wnuAHDG7Rk1gpq3s8rDrn5eZObqPXCGrj7etHwQELsNX88zOHDQkIDm5v_k2_n1cTGdp47SA Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a069fad215134b79b6b4949f412edc53"}]}] Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 426331348a8e4d4cba7022ad81ccd1f6 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1366972933", "metadata": {"Type": "work"}}} Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1366972933', 'metadata': {'Type': 'work'}}}} Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi '0' Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:18:27.875512 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-17c8e5d9-f711-4a56-8b53-6f9e4a25d4b7 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:27.880576 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-17c8e5d9-f711-4a56-8b53-6f9e4a25d4b7 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.010084 Sep 27 10:18:27.882989 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2150/4738] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:18:27 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 15 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:18:27.888509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a09c0d27-c598-43e3-ba2f-9d642c13faa3 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Action: 'create', calling method: .version_select at 0x7f7f63b8a940>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-375519120", "metadata": {"Type": "work"}}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a09c0d27-c598-43e3-ba2f-9d642c13faa3 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Request method failure captured: Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 118 Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs3zy-ptND6J4DgIgyB1Drq1xuI0URUSCHVgPMnnq_NAHyoxRRrrUhY1ilmxmV5VLTe-mzei91ks2Ji6TnWXuhjRl39Zdeto0ibCl77wnuAHDG7Rk1gpq3s8rDrn5eZObqPXCGrj7etHwQELsNX88zOHDQkIDm5v_k2_n1cTGdp47SA Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a069fad215134b79b6b4949f412edc53"}]}] Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 426331348a8e4d4cba7022ad81ccd1f6 Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-375519120", "metadata": {"Type": "work"}}} Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-375519120', 'metadata': {'Type': 'work'}}}} Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi '' Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:18:27.893864 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a09c0d27-c598-43e3-ba2f-9d642c13faa3 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:27.901389 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a09c0d27-c598-43e3-ba2f-9d642c13faa3 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.010280 Sep 27 10:18:27.902655 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2589/4739] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:18:27 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 20 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:18:27.907455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb490f34-28d3-4f21-ba04-10a11c75b266 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Calling method '.version_select at 0x7f7f62559d30>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:27.945780 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-fb490f34-28d3-4f21-ba04-10a11c75b266 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] HTTP exception thrown: Volume tempest-invalid-1998579921 could not be found. Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb490f34-28d3-4f21-ba04-10a11c75b266 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Request method failure captured: Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1998579921 HTTP/1.1 Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs3zy-ptND6J4DgIgyB1Drq1xuI0URUSCHVgPMnnq_NAHyoxRRrrUhY1ilmxmV5VLTe-mzei91ks2Ji6TnWXuhjRl39Zdeto0ibCl77wnuAHDG7Rk1gpq3s8rDrn5eZObqPXCGrj7etHwQELsNX88zOHDQkIDm5v_k2_n1cTGdp47SA Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a069fad215134b79b6b4949f412edc53"}]}] Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 426331348a8e4d4cba7022ad81ccd1f6 Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: exception: Volume tempest-invalid-1998579921 could not be found. Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'tempest-invalid-1998579921'} Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1998579921 could not be found. (HTTP 404) (Request-ID: req-96146f18-1539-4c44-9516-57e3c49f6a37) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 10:18:27.949052 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1998579921 could not be found. Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1998579921 could not be found. Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1998579921 could not be found. Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb490f34-28d3-4f21-ba04-10a11c75b266 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Returning 404 to user: Volume tempest-invalid-1998579921 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fb490f34-28d3-4f21-ba04-10a11c75b266 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1998579921" status: 404 len: 99 microversion: 2.1 time: 0.046915 Sep 27 10:18:27.958349 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2151/4740] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:18:27 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1998579921 => generated 99 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Sep 27 10:18:27.971465 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e84aff6d-48bc-49ea-8431-2aecade47106 tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.009272 Sep 27 10:18:27.979227 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2590/4741] 10.222.0.26 () {58 vars in 1140 bytes} [Tue Sep 27 10:18:27 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 18 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 27 10:18:27.982993 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b938eb0d-697f-431c-b28a-3b99d2ca4e9e tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003919 Sep 27 10:18:27.989028 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2152/4742] 10.222.0.26 () {58 vars in 1137 bytes} [Tue Sep 27 10:18:27 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Sep 27 10:18:27.999494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8fc9f2e3-1859-4bec-ace2-7951585b746b tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Calling method '.version_select at 0x7f7f637158b0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:28.039512 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-8fc9f2e3-1859-4bec-ace2-7951585b746b tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] HTTP exception thrown: Volume 4720d15a-4a87-4c4a-be63-5fa5d0d425ac could not be found. Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8fc9f2e3-1859-4bec-ace2-7951585b746b tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Request method failure captured: Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-volumes/4720d15a-4a87-4c4a-be63-5fa5d0d425ac HTTP/1.1 Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs3zy-ptND6J4DgIgyB1Drq1xuI0URUSCHVgPMnnq_NAHyoxRRrrUhY1ilmxmV5VLTe-mzei91ks2Ji6TnWXuhjRl39Zdeto0ibCl77wnuAHDG7Rk1gpq3s8rDrn5eZObqPXCGrj7etHwQELsNX88zOHDQkIDm5v_k2_n1cTGdp47SA Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a069fad215134b79b6b4949f412edc53"}]}] Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 426331348a8e4d4cba7022ad81ccd1f6 Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: exception: Volume 4720d15a-4a87-4c4a-be63-5fa5d0d425ac could not be found. Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '4720d15a-4a87-4c4a-be63-5fa5d0d425ac'} Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 4720d15a-4a87-4c4a-be63-5fa5d0d425ac could not be found. (HTTP 404) (Request-ID: req-2c0ac9fb-12d9-4327-95cd-9a11011a102f) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 10:18:28.042384 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 4720d15a-4a87-4c4a-be63-5fa5d0d425ac could not be found. Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 4720d15a-4a87-4c4a-be63-5fa5d0d425ac could not be found. Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 4720d15a-4a87-4c4a-be63-5fa5d0d425ac could not be found. Sep 27 10:18:28.048346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.057540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8fc9f2e3-1859-4bec-ace2-7951585b746b tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Returning 404 to user: Volume 4720d15a-4a87-4c4a-be63-5fa5d0d425ac could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:28.058689 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8fc9f2e3-1859-4bec-ace2-7951585b746b tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/4720d15a-4a87-4c4a-be63-5fa5d0d425ac" status: 404 len: 109 microversion: 2.1 time: 0.069568 Sep 27 10:18:28.059776 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2591/4743] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:18:27 2022] DELETE /compute/v2.1/os-volumes/4720d15a-4a87-4c4a-be63-5fa5d0d425ac => generated 109 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:28.069899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-30cb2d69-1f10-49be-8c83-5998f6b1915c tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Calling method '.version_select at 0x7f7f62528160>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:28.105604 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-30cb2d69-1f10-49be-8c83-5998f6b1915c tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] HTTP exception thrown: Volume 00553cf3-48fd-4d32-b67b-0729c97e7565 could not be found. Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-30cb2d69-1f10-49be-8c83-5998f6b1915c tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Request method failure captured: Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-volumes/00553cf3-48fd-4d32-b67b-0729c97e7565 HTTP/1.1 Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs3zy-ptND6J4DgIgyB1Drq1xuI0URUSCHVgPMnnq_NAHyoxRRrrUhY1ilmxmV5VLTe-mzei91ks2Ji6TnWXuhjRl39Zdeto0ibCl77wnuAHDG7Rk1gpq3s8rDrn5eZObqPXCGrj7etHwQELsNX88zOHDQkIDm5v_k2_n1cTGdp47SA Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a069fad215134b79b6b4949f412edc53"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a069fad215134b79b6b4949f412edc53"}]}] Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: a069fad215134b79b6b4949f412edc53 Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-VolumesNegativeTest-1068732303 Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 426331348a8e4d4cba7022ad81ccd1f6 Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-VolumesNegativeTest-1068732303-project Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: exception: Volume 00553cf3-48fd-4d32-b67b-0729c97e7565 could not be found. Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '00553cf3-48fd-4d32-b67b-0729c97e7565'} Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 00553cf3-48fd-4d32-b67b-0729c97e7565 could not be found. (HTTP 404) (Request-ID: req-a0e3fade-362e-4b45-b876-c0302997c141) Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 10:18:28.108062 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 00553cf3-48fd-4d32-b67b-0729c97e7565 could not be found. Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 00553cf3-48fd-4d32-b67b-0729c97e7565 could not be found. Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 00553cf3-48fd-4d32-b67b-0729c97e7565 could not be found. Sep 27 10:18:28.112999 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:28.118743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-30cb2d69-1f10-49be-8c83-5998f6b1915c tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] Returning 404 to user: Volume 00553cf3-48fd-4d32-b67b-0729c97e7565 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:28.119839 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-30cb2d69-1f10-49be-8c83-5998f6b1915c tempest-VolumesNegativeTest-1068732303 tempest-VolumesNegativeTest-1068732303-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/00553cf3-48fd-4d32-b67b-0729c97e7565" status: 404 len: 109 microversion: 2.1 time: 0.054491 Sep 27 10:18:28.121014 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2153/4744] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:18:28 2022] GET /compute/v2.1/os-volumes/00553cf3-48fd-4d32-b67b-0729c97e7565 => generated 109 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:18:28.442767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8a559d7-1e62-47ac-8769-9abef50fc06e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:28.443598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c8a559d7-1e62-47ac-8769-9abef50fc06e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:28.460186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8a559d7-1e62-47ac-8769-9abef50fc06e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:28.461994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8a559d7-1e62-47ac-8769-9abef50fc06e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:28.462461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8a559d7-1e62-47ac-8769-9abef50fc06e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:28.463789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8a559d7-1e62-47ac-8769-9abef50fc06e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:28.464059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8a559d7-1e62-47ac-8769-9abef50fc06e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:28.464503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8a559d7-1e62-47ac-8769-9abef50fc06e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:28.502146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:28.829752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c8a559d7-1e62-47ac-8769-9abef50fc06e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:28.843213 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c8a559d7-1e62-47ac-8769-9abef50fc06e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.405243 Sep 27 10:18:28.849047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb1a3df5-70ba-4a0c-b6d6-7245bb4ea60c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:28.849755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fb1a3df5-70ba-4a0c-b6d6-7245bb4ea60c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:28.857498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb1a3df5-70ba-4a0c-b6d6-7245bb4ea60c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:28.858000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb1a3df5-70ba-4a0c-b6d6-7245bb4ea60c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:28.858443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb1a3df5-70ba-4a0c-b6d6-7245bb4ea60c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:28.859179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb1a3df5-70ba-4a0c-b6d6-7245bb4ea60c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:28.859639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb1a3df5-70ba-4a0c-b6d6-7245bb4ea60c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:28.863920 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2592/4745] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:28 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:28.879801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb1a3df5-70ba-4a0c-b6d6-7245bb4ea60c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:29.443983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fb1a3df5-70ba-4a0c-b6d6-7245bb4ea60c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:29.455673 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fb1a3df5-70ba-4a0c-b6d6-7245bb4ea60c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1517 microversion: 2.1 time: 0.611941 Sep 27 10:18:29.458275 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2154/4746] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:28 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1517 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:29.871665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15d14f55-53a1-42df-b4d7-d984ff8d63c5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:29.872271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-15d14f55-53a1-42df-b4d7-d984ff8d63c5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:29.894178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d14f55-53a1-42df-b4d7-d984ff8d63c5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:29.894706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d14f55-53a1-42df-b4d7-d984ff8d63c5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:29.895162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d14f55-53a1-42df-b4d7-d984ff8d63c5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:29.905153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d14f55-53a1-42df-b4d7-d984ff8d63c5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:29.906203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d14f55-53a1-42df-b4d7-d984ff8d63c5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:29.908127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15d14f55-53a1-42df-b4d7-d984ff8d63c5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:30.482741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7fdcff68-cc50-4bd2-87ea-6886b2ec147e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:30.483361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7fdcff68-cc50-4bd2-87ea-6886b2ec147e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:30.498382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fdcff68-cc50-4bd2-87ea-6886b2ec147e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:30.500419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fdcff68-cc50-4bd2-87ea-6886b2ec147e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:30.501544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fdcff68-cc50-4bd2-87ea-6886b2ec147e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:30.503141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fdcff68-cc50-4bd2-87ea-6886b2ec147e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:30.505767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fdcff68-cc50-4bd2-87ea-6886b2ec147e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:30.506423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fdcff68-cc50-4bd2-87ea-6886b2ec147e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:30.700510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-15d14f55-53a1-42df-b4d7-d984ff8d63c5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:30.730002 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-15d14f55-53a1-42df-b4d7-d984ff8d63c5 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.847980 Sep 27 10:18:30.733189 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2593/4747] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:29 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 867 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:31.517470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7fdcff68-cc50-4bd2-87ea-6886b2ec147e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:31.536928 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7fdcff68-cc50-4bd2-87ea-6886b2ec147e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1523 microversion: 2.1 time: 1.051052 Sep 27 10:18:31.539444 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2155/4748] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:30 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1523 bytes in 1062 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:31.761090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0cce4dc-6b23-460d-8ec8-d1f728e0fd7d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:31.761701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b0cce4dc-6b23-460d-8ec8-d1f728e0fd7d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:31.776730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0cce4dc-6b23-460d-8ec8-d1f728e0fd7d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:31.777251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0cce4dc-6b23-460d-8ec8-d1f728e0fd7d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:31.777746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0cce4dc-6b23-460d-8ec8-d1f728e0fd7d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:31.778712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0cce4dc-6b23-460d-8ec8-d1f728e0fd7d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:31.779117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0cce4dc-6b23-460d-8ec8-d1f728e0fd7d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:31.779555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0cce4dc-6b23-460d-8ec8-d1f728e0fd7d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:32.564392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-61f3f0e4-1501-4b57-a282-95bcc5365eeb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:32.565554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-61f3f0e4-1501-4b57-a282-95bcc5365eeb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:32.582380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61f3f0e4-1501-4b57-a282-95bcc5365eeb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:32.583289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61f3f0e4-1501-4b57-a282-95bcc5365eeb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:32.583987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61f3f0e4-1501-4b57-a282-95bcc5365eeb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:32.584955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61f3f0e4-1501-4b57-a282-95bcc5365eeb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:32.585655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61f3f0e4-1501-4b57-a282-95bcc5365eeb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:32.586241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61f3f0e4-1501-4b57-a282-95bcc5365eeb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:32.598856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b0cce4dc-6b23-460d-8ec8-d1f728e0fd7d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:32.638291 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b0cce4dc-6b23-460d-8ec8-d1f728e0fd7d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.882655 Sep 27 10:18:32.661147 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2594/4749] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:31 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 904 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:33.667047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e6af3246-48dc-4b55-af08-5a034142ca5b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:33.667702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e6af3246-48dc-4b55-af08-5a034142ca5b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:33.696596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6af3246-48dc-4b55-af08-5a034142ca5b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:33.697664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6af3246-48dc-4b55-af08-5a034142ca5b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:33.698127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6af3246-48dc-4b55-af08-5a034142ca5b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:33.698930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6af3246-48dc-4b55-af08-5a034142ca5b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:33.702625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6af3246-48dc-4b55-af08-5a034142ca5b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:33.703161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6af3246-48dc-4b55-af08-5a034142ca5b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:33.735555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-61f3f0e4-1501-4b57-a282-95bcc5365eeb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:33.751498 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-61f3f0e4-1501-4b57-a282-95bcc5365eeb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1523 microversion: 2.1 time: 1.189680 Sep 27 10:18:33.781749 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2156/4750] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:32 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1523 bytes in 1222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:33.844428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a0dcbe21-9959-49a8-a6d0-e9fed91b3c23 req-72adab45-0f65-48d0-aac8-52afc257db72 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c45ec10-5528-48be-bc7b-4e628abd6d85", "name": "network-vif-unplugged", "status": "completed", "tag": "d2a12997-e121-466a-a120-ced0de0c1108"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:33.867543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a0dcbe21-9959-49a8-a6d0-e9fed91b3c23 req-72adab45-0f65-48d0-aac8-52afc257db72 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:33.868297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a0dcbe21-9959-49a8-a6d0-e9fed91b3c23 req-72adab45-0f65-48d0-aac8-52afc257db72 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:33.868978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a0dcbe21-9959-49a8-a6d0-e9fed91b3c23 req-72adab45-0f65-48d0-aac8-52afc257db72 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:33.881400 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a0dcbe21-9959-49a8-a6d0-e9fed91b3c23 req-72adab45-0f65-48d0-aac8-52afc257db72 service nova] Creating event network-vif-unplugged:d2a12997-e121-466a-a120-ced0de0c1108 for instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 on nch1-master-1 Sep 27 10:18:33.887911 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a0dcbe21-9959-49a8-a6d0-e9fed91b3c23 req-72adab45-0f65-48d0-aac8-52afc257db72 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.047886 Sep 27 10:18:33.889073 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2157/4751] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:18:33 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:18:34.320251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e6af3246-48dc-4b55-af08-5a034142ca5b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:34.334527 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e6af3246-48dc-4b55-af08-5a034142ca5b tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.671363 Sep 27 10:18:34.336484 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2595/4752] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:33 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 674 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:34.796224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9caa9505-9e94-4f80-aca8-ba194e888a33 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:34.797125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9caa9505-9e94-4f80-aca8-ba194e888a33 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:34.831408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9caa9505-9e94-4f80-aca8-ba194e888a33 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:34.832026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9caa9505-9e94-4f80-aca8-ba194e888a33 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:34.832485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9caa9505-9e94-4f80-aca8-ba194e888a33 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:34.833302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9caa9505-9e94-4f80-aca8-ba194e888a33 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:34.836854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9caa9505-9e94-4f80-aca8-ba194e888a33 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:34.837562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9caa9505-9e94-4f80-aca8-ba194e888a33 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:35.566231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9caa9505-9e94-4f80-aca8-ba194e888a33 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:35.607834 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9caa9505-9e94-4f80-aca8-ba194e888a33 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1523 microversion: 2.1 time: 0.817047 Sep 27 10:18:35.636072 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2158/4753] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:34 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1523 bytes in 846 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:35.668217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f78b987b-7b22-4b8e-a2ff-cbad746c5f33 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:35.669678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f78b987b-7b22-4b8e-a2ff-cbad746c5f33 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:35.706418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f78b987b-7b22-4b8e-a2ff-cbad746c5f33 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:35.707289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f78b987b-7b22-4b8e-a2ff-cbad746c5f33 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:35.709370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f78b987b-7b22-4b8e-a2ff-cbad746c5f33 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:35.712485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f78b987b-7b22-4b8e-a2ff-cbad746c5f33 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:35.714624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f78b987b-7b22-4b8e-a2ff-cbad746c5f33 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:35.715248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f78b987b-7b22-4b8e-a2ff-cbad746c5f33 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:35.753022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerTagsTestJSON-server-1441565044", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "e70ff113-10da-4ee4-b683-dd091b8722d1"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:35.860361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] validate_networks() for [('e70ff113-10da-4ee4-b683-dd091b8722d1', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:18:36.056085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f78b987b-7b22-4b8e-a2ff-cbad746c5f33 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:36.073937 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f78b987b-7b22-4b8e-a2ff-cbad746c5f33 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1510 microversion: 2.1 time: 0.433775 Sep 27 10:18:36.093860 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2159/4754] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:35 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1510 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:36.646432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-470e7e0d-9428-4ba9-a0d5-1f782d0e2e97 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:36.647436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-470e7e0d-9428-4ba9-a0d5-1f782d0e2e97 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:36.659441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-470e7e0d-9428-4ba9-a0d5-1f782d0e2e97 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:36.665512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-470e7e0d-9428-4ba9-a0d5-1f782d0e2e97 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:36.667510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-470e7e0d-9428-4ba9-a0d5-1f782d0e2e97 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:36.669958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-470e7e0d-9428-4ba9-a0d5-1f782d0e2e97 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:36.673601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-470e7e0d-9428-4ba9-a0d5-1f782d0e2e97 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:36.674924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-470e7e0d-9428-4ba9-a0d5-1f782d0e2e97 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:36.975534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-470e7e0d-9428-4ba9-a0d5-1f782d0e2e97 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:36.987103 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-470e7e0d-9428-4ba9-a0d5-1f782d0e2e97 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1523 microversion: 2.1 time: 0.345047 Sep 27 10:18:36.992180 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2160/4755] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:36 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1523 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:37.101357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f77246c3-f4bb-464e-a179-efc05609cc42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:37.103199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f77246c3-f4bb-464e-a179-efc05609cc42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:37.117898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77246c3-f4bb-464e-a179-efc05609cc42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:37.119286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77246c3-f4bb-464e-a179-efc05609cc42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:37.121315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77246c3-f4bb-464e-a179-efc05609cc42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:37.123055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77246c3-f4bb-464e-a179-efc05609cc42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:37.127024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77246c3-f4bb-464e-a179-efc05609cc42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:37.127955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77246c3-f4bb-464e-a179-efc05609cc42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:37.618300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f77246c3-f4bb-464e-a179-efc05609cc42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:37.629377 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f77246c3-f4bb-464e-a179-efc05609cc42 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1487 microversion: 2.1 time: 0.532636 Sep 27 10:18:37.643275 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2161/4756] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:37 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1487 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:37.656076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5cc57a4f-c53b-457a-826b-aa011da89dd6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Action: 'update', calling method: >, body: {"server": {"name": "tempest-\u00cd\u00f1st\u00e1\u00f1c\u00e9-1999421623"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:37.659112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5cc57a4f-c53b-457a-826b-aa011da89dd6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:37.668054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cc57a4f-c53b-457a-826b-aa011da89dd6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:37.672442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cc57a4f-c53b-457a-826b-aa011da89dd6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:37.673171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cc57a4f-c53b-457a-826b-aa011da89dd6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:37.674112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cc57a4f-c53b-457a-826b-aa011da89dd6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:37.676056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cc57a4f-c53b-457a-826b-aa011da89dd6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:37.676815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cc57a4f-c53b-457a-826b-aa011da89dd6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:37.832737 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5cc57a4f-c53b-457a-826b-aa011da89dd6 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "PUT /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1061 microversion: 2.1 time: 0.180601 Sep 27 10:18:37.846780 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2162/4757] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:18:37 2022] PUT /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1061 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:37.858560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7851d666-2cd0-43bf-8172-7b17dbaf1b54 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:37.859582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7851d666-2cd0-43bf-8172-7b17dbaf1b54 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:37.875620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7851d666-2cd0-43bf-8172-7b17dbaf1b54 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:37.889400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7851d666-2cd0-43bf-8172-7b17dbaf1b54 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:37.892412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7851d666-2cd0-43bf-8172-7b17dbaf1b54 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:37.893376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7851d666-2cd0-43bf-8172-7b17dbaf1b54 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:37.893959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7851d666-2cd0-43bf-8172-7b17dbaf1b54 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:37.896690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7851d666-2cd0-43bf-8172-7b17dbaf1b54 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:38.352503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:18:38.353010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:18:38.353422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:18:38.353923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:18:38.354347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:18:38.369048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:38.369549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:38.370768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:38.371811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:38.376239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:38.379164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:38.395477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7851d666-2cd0-43bf-8172-7b17dbaf1b54 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:38.409114 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7851d666-2cd0-43bf-8172-7b17dbaf1b54 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1488 microversion: 2.1 time: 0.554129 Sep 27 10:18:38.426158 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2163/4758] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:37 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1488 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:38.428217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Getting quotas for project cd00595fe8dc4d9594513f900172d7ff. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:18:38.436991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Getting quotas for user 6efd3d39610444b99c7bb245337c2ac3 and project cd00595fe8dc4d9594513f900172d7ff. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:18:38.443463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ae34388a-bd1d-4a00-99c3-a5aa26980680 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:38.448373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ae34388a-bd1d-4a00-99c3-a5aa26980680 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:38.463602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae34388a-bd1d-4a00-99c3-a5aa26980680 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:38.466868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae34388a-bd1d-4a00-99c3-a5aa26980680 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:38.467529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae34388a-bd1d-4a00-99c3-a5aa26980680 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:38.471870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae34388a-bd1d-4a00-99c3-a5aa26980680 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:38.472583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae34388a-bd1d-4a00-99c3-a5aa26980680 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:38.473354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae34388a-bd1d-4a00-99c3-a5aa26980680 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:38.477087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:18:38.480231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:18:38.613692 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cc60119a-3b7d-4451-86c9-658f11ef2e1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.26 time: 3.715295 Sep 27 10:18:38.628007 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2596/4759] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:18:34 2022] POST /compute/v2.1/servers => generated 380 bytes in 3730 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:18:38.646129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-17cd776c-e631-4310-a059-18d6ba7d383c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:38.651947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-17cd776c-e631-4310-a059-18d6ba7d383c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:38.688784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cd776c-e631-4310-a059-18d6ba7d383c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:38.693487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cd776c-e631-4310-a059-18d6ba7d383c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:38.694213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cd776c-e631-4310-a059-18d6ba7d383c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:38.695073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cd776c-e631-4310-a059-18d6ba7d383c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:38.697197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cd776c-e631-4310-a059-18d6ba7d383c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:38.697736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cd776c-e631-4310-a059-18d6ba7d383c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:39.030095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ae34388a-bd1d-4a00-99c3-a5aa26980680 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:39.045988 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ae34388a-bd1d-4a00-99c3-a5aa26980680 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1523 microversion: 2.1 time: 0.608083 Sep 27 10:18:39.046811 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2164/4760] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:38 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1523 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:39.055952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ffe767a9-45fe-4a0f-a4cd-c7ce04c9df1f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:39.058963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ffe767a9-45fe-4a0f-a4cd-c7ce04c9df1f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:39.438666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17cd776c-e631-4310-a059-18d6ba7d383c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:39.453952 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-17cd776c-e631-4310-a059-18d6ba7d383c tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1488 microversion: 2.1 time: 0.822843 Sep 27 10:18:39.462954 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2597/4761] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:38 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1488 bytes in 834 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:39.492312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d7fc7dc-afe4-4a7e-b1c2-606b199a3694 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:39.492851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2d7fc7dc-afe4-4a7e-b1c2-606b199a3694 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:39.506079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d7fc7dc-afe4-4a7e-b1c2-606b199a3694 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:39.506808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d7fc7dc-afe4-4a7e-b1c2-606b199a3694 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:39.508782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d7fc7dc-afe4-4a7e-b1c2-606b199a3694 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:39.510278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d7fc7dc-afe4-4a7e-b1c2-606b199a3694 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:39.510933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d7fc7dc-afe4-4a7e-b1c2-606b199a3694 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:39.511953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d7fc7dc-afe4-4a7e-b1c2-606b199a3694 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:39.588170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2d7fc7dc-afe4-4a7e-b1c2-606b199a3694 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:18:39.852700 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d7fc7dc-afe4-4a7e-b1c2-606b199a3694 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 204 len: 0 microversion: 2.1 time: 0.367260 Sep 27 10:18:39.853975 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2598/4762] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:39 2022] DELETE /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 0 bytes in 369 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:18:39.865275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b62292c3-7a74-4239-9e1a-acd4cded36c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:39.869966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b62292c3-7a74-4239-9e1a-acd4cded36c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:39.881291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ffe767a9-45fe-4a0f-a4cd-c7ce04c9df1f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:39.891900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ffe767a9-45fe-4a0f-a4cd-c7ce04c9df1f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:39.893862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ffe767a9-45fe-4a0f-a4cd-c7ce04c9df1f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:39.895063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62292c3-7a74-4239-9e1a-acd4cded36c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:39.896498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62292c3-7a74-4239-9e1a-acd4cded36c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:39.897921 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ffe767a9-45fe-4a0f-a4cd-c7ce04c9df1f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1262 microversion: 2.26 time: 0.849858 Sep 27 10:18:39.898726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62292c3-7a74-4239-9e1a-acd4cded36c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:39.901778 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2165/4763] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:39 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1262 bytes in 854 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:18:39.902811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62292c3-7a74-4239-9e1a-acd4cded36c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:39.904231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62292c3-7a74-4239-9e1a-acd4cded36c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:39.905563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b62292c3-7a74-4239-9e1a-acd4cded36c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:40.078617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3299b78-096f-4257-9aec-d4a9580b236e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:40.078617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e3299b78-096f-4257-9aec-d4a9580b236e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:40.098642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3299b78-096f-4257-9aec-d4a9580b236e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:40.099215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3299b78-096f-4257-9aec-d4a9580b236e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:40.099657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3299b78-096f-4257-9aec-d4a9580b236e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:40.104566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3299b78-096f-4257-9aec-d4a9580b236e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:40.106007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3299b78-096f-4257-9aec-d4a9580b236e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:40.110583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3299b78-096f-4257-9aec-d4a9580b236e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:40.310776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b62292c3-7a74-4239-9e1a-acd4cded36c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:40.323120 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b62292c3-7a74-4239-9e1a-acd4cded36c9 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1494 microversion: 2.1 time: 0.463372 Sep 27 10:18:40.323995 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2599/4764] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:39 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1494 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:40.914091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e3299b78-096f-4257-9aec-d4a9580b236e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:40.929307 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e3299b78-096f-4257-9aec-d4a9580b236e tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1523 microversion: 2.1 time: 0.860005 Sep 27 10:18:40.931827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:40.932454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:40.939126 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2166/4765] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:40 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1523 bytes in 870 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:40.951141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:40.951644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:40.952133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:40.953198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:40.953657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:40.954172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:41.270019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:41.291102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:41.295449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:41.300232 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ee7ae2ca-6dec-47d0-b126-53594a4243fa tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1258 microversion: 2.26 time: 0.369535 Sep 27 10:18:41.309877 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2600/4766] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:40 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1258 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:18:41.351846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c86892ec-f0d7-415b-80b7-d28186309aed tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:41.352507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c86892ec-f0d7-415b-80b7-d28186309aed tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:41.364067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c86892ec-f0d7-415b-80b7-d28186309aed tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:41.374345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c86892ec-f0d7-415b-80b7-d28186309aed tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:41.374675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c86892ec-f0d7-415b-80b7-d28186309aed tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:41.377002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c86892ec-f0d7-415b-80b7-d28186309aed tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:41.377498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c86892ec-f0d7-415b-80b7-d28186309aed tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:41.378126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c86892ec-f0d7-415b-80b7-d28186309aed tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:41.706603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c86892ec-f0d7-415b-80b7-d28186309aed tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:41.719957 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c86892ec-f0d7-415b-80b7-d28186309aed tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1494 microversion: 2.1 time: 0.372138 Sep 27 10:18:41.721259 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2167/4767] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:41 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1494 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:41.956708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a05f4b6-6acc-418e-ad3b-ddf677fae096 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:41.959578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a05f4b6-6acc-418e-ad3b-ddf677fae096 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:41.976476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a05f4b6-6acc-418e-ad3b-ddf677fae096 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:41.977786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a05f4b6-6acc-418e-ad3b-ddf677fae096 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:41.978964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a05f4b6-6acc-418e-ad3b-ddf677fae096 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:41.979926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a05f4b6-6acc-418e-ad3b-ddf677fae096 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:41.980602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a05f4b6-6acc-418e-ad3b-ddf677fae096 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:41.981249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a05f4b6-6acc-418e-ad3b-ddf677fae096 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:42.336009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:42.336755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:42.348233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:42.350037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:42.351452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:42.352781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:42.353614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:42.354315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:42.651710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8a05f4b6-6acc-418e-ad3b-ddf677fae096 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:42.662952 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a05f4b6-6acc-418e-ad3b-ddf677fae096 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1523 microversion: 2.1 time: 0.710341 Sep 27 10:18:42.667168 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2601/4768] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:41 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1523 bytes in 716 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:42.679644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-04e24be1-1c84-4620-b13f-d59f2b05dccb req-68a89fb5-3b16-4370-9728-a93f5e2857dc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c45ec10-5528-48be-bc7b-4e628abd6d85", "name": "network-vif-plugged", "status": "completed", "tag": "d2a12997-e121-466a-a120-ced0de0c1108"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:42.703916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04e24be1-1c84-4620-b13f-d59f2b05dccb req-68a89fb5-3b16-4370-9728-a93f5e2857dc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:42.709433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04e24be1-1c84-4620-b13f-d59f2b05dccb req-68a89fb5-3b16-4370-9728-a93f5e2857dc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:42.710534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04e24be1-1c84-4620-b13f-d59f2b05dccb req-68a89fb5-3b16-4370-9728-a93f5e2857dc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:42.719452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:42.735469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:42.737695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:42.739231 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b97d088d-744f-431a-8cf9-788cdff6fdc7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1320 microversion: 2.26 time: 0.407666 Sep 27 10:18:42.740793 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2168/4769] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:42 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1320 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:18:42.757180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eaeb2e36-35ae-4ee5-b179-eb3e7a7917cd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:42.758636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eaeb2e36-35ae-4ee5-b179-eb3e7a7917cd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:42.765469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eaeb2e36-35ae-4ee5-b179-eb3e7a7917cd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:42.765961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eaeb2e36-35ae-4ee5-b179-eb3e7a7917cd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:42.766396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eaeb2e36-35ae-4ee5-b179-eb3e7a7917cd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:42.767099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eaeb2e36-35ae-4ee5-b179-eb3e7a7917cd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:42.767553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eaeb2e36-35ae-4ee5-b179-eb3e7a7917cd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:42.775365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eaeb2e36-35ae-4ee5-b179-eb3e7a7917cd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:42.782904 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-04e24be1-1c84-4620-b13f-d59f2b05dccb req-68a89fb5-3b16-4370-9728-a93f5e2857dc service nova] Creating event network-vif-plugged:d2a12997-e121-466a-a120-ced0de0c1108 for instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 on nch1-master-1 Sep 27 10:18:42.789424 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-04e24be1-1c84-4620-b13f-d59f2b05dccb req-68a89fb5-3b16-4370-9728-a93f5e2857dc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120448 Sep 27 10:18:42.791761 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2602/4770] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:18:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:18:43.144391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-09b3ce50-117f-4723-b662-df89b382190e req-f5163503-877c-435d-8d79-ad43397b56be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8696cba0-e2e2-4f51-858d-3b502152a6ed", "name": "network-vif-deleted", "tag": "1e961036-096a-4369-b689-86c91be70173"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:43.156168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=64,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:43.173377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-09b3ce50-117f-4723-b662-df89b382190e req-f5163503-877c-435d-8d79-ad43397b56be service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:43.177484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-09b3ce50-117f-4723-b662-df89b382190e req-f5163503-877c-435d-8d79-ad43397b56be service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:43.178237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-09b3ce50-117f-4723-b662-df89b382190e req-f5163503-877c-435d-8d79-ad43397b56be service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:43.215459 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-09b3ce50-117f-4723-b662-df89b382190e req-f5163503-877c-435d-8d79-ad43397b56be service nova] Creating event network-vif-deleted:1e961036-096a-4369-b689-86c91be70173 for instance 8696cba0-e2e2-4f51-858d-3b502152a6ed on nch2-master-1 Sep 27 10:18:43.234990 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-09b3ce50-117f-4723-b662-df89b382190e req-f5163503-877c-435d-8d79-ad43397b56be service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080583 Sep 27 10:18:43.235125 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2603/4771] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:18:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:18:43.251704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eaeb2e36-35ae-4ee5-b179-eb3e7a7917cd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:43.267814 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eaeb2e36-35ae-4ee5-b179-eb3e7a7917cd tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 200 len: 1452 microversion: 2.1 time: 0.525872 Sep 27 10:18:43.281075 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2169/4772] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:42 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 1452 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:43.692514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da7836be-7660-444a-a265-87a49d5672c0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:43.693419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-da7836be-7660-444a-a265-87a49d5672c0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:43.705505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da7836be-7660-444a-a265-87a49d5672c0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:43.707148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da7836be-7660-444a-a265-87a49d5672c0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:43.707858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da7836be-7660-444a-a265-87a49d5672c0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:43.710490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da7836be-7660-444a-a265-87a49d5672c0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:43.711207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da7836be-7660-444a-a265-87a49d5672c0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:43.712246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da7836be-7660-444a-a265-87a49d5672c0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:43.771554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:43.772797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:43.791425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:43.800690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:43.801603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:43.816904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:43.817573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:43.818922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.128991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:44.140945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:44.142432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:44.149816 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b2d26517-d43b-4c08-a7f6-7151f7b9141d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1320 microversion: 2.26 time: 0.378374 Sep 27 10:18:44.151750 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2170/4773] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:43 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1320 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:18:44.301962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:44.303272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:44.314109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.315228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.317179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.321022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.321962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.322699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.354192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:44.357556 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed HTTP/1.1 Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '8696cba0-e2e2-4f51-858d-3b502152a6ed'} Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.359939 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.365820 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.365820 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.365820 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.365820 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:44.365820 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:44.365820 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.365820 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.368922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:44.370152 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-78d5bb58-07c1-4d7e-9830-29b8a1a81089 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 404 len: 111 microversion: 2.1 time: 0.079545 Sep 27 10:18:44.371422 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2171/4774] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:44 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:18:44.394064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:44.394577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:44.402072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.402767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.403394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.404391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.405038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.405764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.422652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-da7836be-7660-444a-a265-87a49d5672c0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:44.434992 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.437551 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da7836be-7660-444a-a265-87a49d5672c0 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1523 microversion: 2.1 time: 0.750042 Sep 27 10:18:44.439204 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2604/4775] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:43 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1523 bytes in 753 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed HTTP/1.1 Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '8696cba0-e2e2-4f51-858d-3b502152a6ed'} Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:44.446138 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:44.452039 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a8828792-8e53-44f0-8177-365958565488 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 404 len: 111 microversion: 2.1 time: 0.058227 Sep 27 10:18:44.460706 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2172/4776] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:44 2022] DELETE /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:44.515256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:44.517743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 8696cba0-e2e2-4f51-858d-3b502152a6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:44.532966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.534683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.535604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.537103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.540163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.541989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.580938 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed HTTP/1.1 Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '8696cba0-e2e2-4f51-858d-3b502152a6ed'} Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.584797 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.591114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.591114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.591114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.591114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:44.591114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:44.591114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. Sep 27 10:18:44.591114 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.614299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 8696cba0-e2e2-4f51-858d-3b502152a6ed could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:44.618557 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5770f197-ffaa-479a-928f-fc8733b9bf4e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed" status: 404 len: 111 microversion: 2.1 time: 0.153343 Sep 27 10:18:44.620991 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2605/4777] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:44 2022] GET /compute/v2.1/servers/8696cba0-e2e2-4f51-858d-3b502152a6ed => generated 111 bytes in 158 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:44.628776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:44.629413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:44.641492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.643286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.644162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.645992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.650088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.652437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.677294 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 HTTP/1.1 Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'dd547164-97da-4ba4-9cc3-b1448d2ab806'} Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:44.680608 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:18:44.688266 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.688266 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.688266 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.688266 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.688266 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.688266 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.688266 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:44.688266 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:44.688266 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:18:44.688266 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.691302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:44.695156 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9d8cf109-7561-47ca-8529-fcad087d27a3 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 404 len: 111 microversion: 2.1 time: 0.069863 Sep 27 10:18:44.696351 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2173/4778] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:44 2022] DELETE /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:44.707418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:44.708320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: dd547164-97da-4ba4-9cc3-b1448d2ab806] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:44.729668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.730330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.731278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.734263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.736045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.738390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.776026 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 HTTP/1.1 Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'dd547164-97da-4ba4-9cc3-b1448d2ab806'} Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.785948 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.791716 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.791716 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.791716 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.791716 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:44.791716 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:44.791716 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. Sep 27 10:18:44.791716 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.794485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance dd547164-97da-4ba4-9cc3-b1448d2ab806 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:44.795596 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58da477c-4997-470c-86a6-d7ee1d77e52f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806" status: 404 len: 111 microversion: 2.1 time: 0.095125 Sep 27 10:18:44.804909 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2606/4779] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:44 2022] GET /compute/v2.1/servers/dd547164-97da-4ba4-9cc3-b1448d2ab806 => generated 111 bytes in 101 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:44.811154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:44.811947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:44.830969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.831655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.835343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.838854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.839620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.857771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.902956 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 HTTP/1.1 Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '5a7e608b-b82d-43c7-a528-d4b5de120a71'} Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:44.904921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:18:44.909409 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.909409 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:44.909409 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.909409 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:44.909409 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:44.909409 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:44.909409 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:44.909409 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:44.909409 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:18:44.909409 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:44.916054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:44.923967 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fc87e5db-5cf9-473d-a9c9-b3b2700f45d1 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 404 len: 111 microversion: 2.1 time: 0.116002 Sep 27 10:18:44.926896 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2174/4780] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:44 2022] DELETE /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 111 bytes in 121 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:44.936608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:44.937566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 5a7e608b-b82d-43c7-a528-d4b5de120a71] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:44.950999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.951953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.954730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.956077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:44.957952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:44.958706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:44.998003 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 HTTP/1.1 Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '5a7e608b-b82d-43c7-a528-d4b5de120a71'} Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.003747 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.009370 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.009370 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.009370 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.009370 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:45.009370 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:45.009370 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. Sep 27 10:18:45.009370 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.013495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 5a7e608b-b82d-43c7-a528-d4b5de120a71 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:45.015179 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c6bc263a-5311-4f53-b48d-a0f8bf7e5959 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71" status: 404 len: 111 microversion: 2.1 time: 0.083465 Sep 27 10:18:45.017657 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2607/4781] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:44 2022] GET /compute/v2.1/servers/5a7e608b-b82d-43c7-a528-d4b5de120a71 => generated 111 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:45.027388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.028259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.038774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=64 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:45.043188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.044631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.046016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.047156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.049069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.052396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.095020 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed HTTP/1.1 Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'f13eb59c-b2be-4b37-addc-676d74eff6ed'} Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:45.097485 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:18:45.106467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.106467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.106467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.106467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.106467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.106467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.106467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:45.106467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:45.106467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:18:45.106467 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.111552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:45.113403 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2025721e-1f46-4a75-9e21-63dcb3cf6147 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 404 len: 111 microversion: 2.1 time: 0.090696 Sep 27 10:18:45.118509 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2175/4782] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:45 2022] DELETE /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 111 bytes in 96 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:45.123696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.124332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: f13eb59c-b2be-4b37-addc-676d74eff6ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.134750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.137462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.139616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.140582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.143726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.146134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.177667 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:18:45.178739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.179596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed HTTP/1.1 Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'f13eb59c-b2be-4b37-addc-676d74eff6ed'} Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.181569 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.187764 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.187764 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.187764 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.187764 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:45.187764 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:45.187764 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. Sep 27 10:18:45.187764 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.190468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance f13eb59c-b2be-4b37-addc-676d74eff6ed could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:45.191594 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5980551c-db12-42cb-be95-f45dd118d9b8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed" status: 404 len: 111 microversion: 2.1 time: 0.072030 Sep 27 10:18:45.193133 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2608/4783] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:45 2022] GET /compute/v2.1/servers/f13eb59c-b2be-4b37-addc-676d74eff6ed => generated 111 bytes in 74 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:45.200018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.201401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.201991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.205552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.205885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.206620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.213077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.214270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.222266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.223078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.223708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.225408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.226066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.226902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.247311 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 HTTP/1.1 Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '521b4b27-ea54-4de8-a96d-add9170f03c5'} Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:45.249593 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:18:45.254531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.254531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.254531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.254531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.254531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.254531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.254531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:45.254531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:45.254531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:18:45.254531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.257080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:45.287601 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-30b50073-2f5d-48e6-97d2-79c7c6776eab tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 404 len: 111 microversion: 2.1 time: 0.079588 Sep 27 10:18:45.288801 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2609/4784] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:45 2022] DELETE /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 111 bytes in 92 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:45.302992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.304690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 521b4b27-ea54-4de8-a96d-add9170f03c5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.314845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.317168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.319516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.320597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.321391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.322053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.350567 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 HTTP/1.1 Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '521b4b27-ea54-4de8-a96d-add9170f03c5'} Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.352461 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.356631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.356631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.356631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.356631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:45.356631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:45.356631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. Sep 27 10:18:45.356631 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.359733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 521b4b27-ea54-4de8-a96d-add9170f03c5 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:45.361008 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-75cb58f4-1f99-46e3-b9e5-01b45a5b89c0 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5" status: 404 len: 111 microversion: 2.1 time: 0.065050 Sep 27 10:18:45.362149 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2610/4785] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:45 2022] GET /compute/v2.1/servers/521b4b27-ea54-4de8-a96d-add9170f03c5 => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:45.373852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.374645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.390785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.392961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.393700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.394787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.395422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.396027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.427293 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 HTTP/1.1 Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '430df9fb-7e75-4f60-8d96-0910ec4e8243'} Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:45.429964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:18:45.434215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.434215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.434215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.434215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.434215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.434215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.434215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:45.434215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:45.434215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:18:45.434215 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.437611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:45.438519 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3b6073a4-ca9e-420c-827c-9ed1fd87404e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 404 len: 111 microversion: 2.1 time: 0.071015 Sep 27 10:18:45.443272 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2611/4786] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:45 2022] DELETE /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 111 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:45.448902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.449892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 430df9fb-7e75-4f60-8d96-0910ec4e8243] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.463660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.464510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.465351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.466350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.467228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.468092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.488697 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 HTTP/1.1 Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '430df9fb-7e75-4f60-8d96-0910ec4e8243'} Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.490964 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.495160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.495160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.495160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.495160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:45.495160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:45.495160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. Sep 27 10:18:45.495160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.495160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 430df9fb-7e75-4f60-8d96-0910ec4e8243 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:45.495893 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7b73179d-47ac-496e-8654-cf37f296fc86 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243" status: 404 len: 111 microversion: 2.1 time: 0.050553 Sep 27 10:18:45.495975 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2612/4787] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:45 2022] GET /compute/v2.1/servers/430df9fb-7e75-4f60-8d96-0910ec4e8243 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:45.510290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1573e8e0-aba6-4361-b908-5f39e4c4ac34 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.510808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1573e8e0-aba6-4361-b908-5f39e4c4ac34 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.517775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:45.529450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:45.531295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:45.532739 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5211f635-ae0c-43a0-b7b7-d351411fec49 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1320 microversion: 2.26 time: 0.359438 Sep 27 10:18:45.533919 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2176/4788] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:45 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1320 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:18:45.540100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1238572-b24b-4e61-8b81-70758904e0e4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.540945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a1238572-b24b-4e61-8b81-70758904e0e4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.546924 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a1238572-b24b-4e61-8b81-70758904e0e4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1238572-b24b-4e61-8b81-70758904e0e4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa HTTP/1.1 Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa'} Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.548875 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.553822 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.553822 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.553822 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.553822 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:45.553822 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:45.553822 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:18:45.553822 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.557537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1238572-b24b-4e61-8b81-70758904e0e4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:45.558635 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a1238572-b24b-4e61-8b81-70758904e0e4 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa" status: 404 len: 111 microversion: 2.1 time: 0.023396 Sep 27 10:18:45.561029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1573e8e0-aba6-4361-b908-5f39e4c4ac34 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.561549 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2177/4789] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:45 2022] DELETE /compute/v2.1/servers/b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa => generated 111 bytes in 27 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:45.563274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1573e8e0-aba6-4361-b908-5f39e4c4ac34 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.563274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1573e8e0-aba6-4361-b908-5f39e4c4ac34 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.566901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1573e8e0-aba6-4361-b908-5f39e4c4ac34 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.569088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1573e8e0-aba6-4361-b908-5f39e4c4ac34 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.569495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1573e8e0-aba6-4361-b908-5f39e4c4ac34 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.570835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b1cf8a60-9a81-43a1-b885-0dfbfa03014f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.571725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b1cf8a60-9a81-43a1-b885-0dfbfa03014f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.584899 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-b1cf8a60-9a81-43a1-b885-0dfbfa03014f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b1cf8a60-9a81-43a1-b885-0dfbfa03014f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa HTTP/1.1 Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa'} Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:45.587522 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:45.591657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:45.591657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:45.591657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:45.591657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. Sep 27 10:18:45.591657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:18:45.594703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b1cf8a60-9a81-43a1-b885-0dfbfa03014f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:45.595800 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b1cf8a60-9a81-43a1-b885-0dfbfa03014f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa" status: 404 len: 111 microversion: 2.1 time: 0.031254 Sep 27 10:18:45.601020 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2178/4790] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:45 2022] GET /compute/v2.1/servers/b4054fa3-5ce7-4f2e-9d78-cd79a8b5d3aa => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:18:45.606188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ad38ec9-a420-402d-a2a4-d7af0746f62d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.607004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ad38ec9-a420-402d-a2a4-d7af0746f62d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.619730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ad38ec9-a420-402d-a2a4-d7af0746f62d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.620460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ad38ec9-a420-402d-a2a4-d7af0746f62d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.621435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ad38ec9-a420-402d-a2a4-d7af0746f62d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.622416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ad38ec9-a420-402d-a2a4-d7af0746f62d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.626151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ad38ec9-a420-402d-a2a4-d7af0746f62d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.632709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ad38ec9-a420-402d-a2a4-d7af0746f62d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.721768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ad38ec9-a420-402d-a2a4-d7af0746f62d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:18:45.895771 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8ad38ec9-a420-402d-a2a4-d7af0746f62d tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 204 len: 0 microversion: 2.1 time: 0.293117 Sep 27 10:18:45.900086 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2179/4791] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:45 2022] DELETE /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 0 bytes in 295 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:18:45.910793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1573e8e0-aba6-4361-b908-5f39e4c4ac34 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:45.921862 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1573e8e0-aba6-4361-b908-5f39e4c4ac34 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1508 microversion: 2.1 time: 0.416455 Sep 27 10:18:45.928413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f3d8d2e-5a19-446b-bffc-e3eadb04e73e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.929345 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2613/4792] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:45 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1508 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:45.930408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f3d8d2e-5a19-446b-bffc-e3eadb04e73e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.936917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3d8d2e-5a19-446b-bffc-e3eadb04e73e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.937691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3d8d2e-5a19-446b-bffc-e3eadb04e73e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.938287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3d8d2e-5a19-446b-bffc-e3eadb04e73e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.939183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3d8d2e-5a19-446b-bffc-e3eadb04e73e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.939793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3d8d2e-5a19-446b-bffc-e3eadb04e73e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.940373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f3d8d2e-5a19-446b-bffc-e3eadb04e73e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.951736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e95be61a-9bb1-41d0-80aa-b53b05d1c656 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:45.953475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e95be61a-9bb1-41d0-80aa-b53b05d1c656 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:45.966450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e95be61a-9bb1-41d0-80aa-b53b05d1c656 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.966858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e95be61a-9bb1-41d0-80aa-b53b05d1c656 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.967301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e95be61a-9bb1-41d0-80aa-b53b05d1c656 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:45.968036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e95be61a-9bb1-41d0-80aa-b53b05d1c656 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:45.968479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e95be61a-9bb1-41d0-80aa-b53b05d1c656 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:45.974917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e95be61a-9bb1-41d0-80aa-b53b05d1c656 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:46.249043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e95be61a-9bb1-41d0-80aa-b53b05d1c656 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:46.251747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:46.263018 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e95be61a-9bb1-41d0-80aa-b53b05d1c656 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 200 len: 1508 microversion: 2.1 time: 0.315241 Sep 27 10:18:46.267068 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2614/4793] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:45 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 1508 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:46.293468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e8e9265a-a1b4-4f60-be53-632747220f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:46.294276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e8e9265a-a1b4-4f60-be53-632747220f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:46.306789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8e9265a-a1b4-4f60-be53-632747220f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:46.307699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8e9265a-a1b4-4f60-be53-632747220f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:46.308456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8e9265a-a1b4-4f60-be53-632747220f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:46.309436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8e9265a-a1b4-4f60-be53-632747220f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:46.310130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8e9265a-a1b4-4f60-be53-632747220f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:46.310798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e8e9265a-a1b4-4f60-be53-632747220f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:46.378469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e8e9265a-a1b4-4f60-be53-632747220f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:18:46.517457 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e8e9265a-a1b4-4f60-be53-632747220f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 204 len: 0 microversion: 2.1 time: 0.229615 Sep 27 10:18:46.518247 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2615/4794] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:18:46 2022] DELETE /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:18:46.563817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:46.564678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:46.582010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:46.582532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:46.582977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:46.583773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:46.584327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:46.588446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:46.713254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:46.727933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:46.730521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:46.731922 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7a962881-5e8e-4d68-a81d-04aa10b2875d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1320 microversion: 2.26 time: 0.176204 Sep 27 10:18:46.734269 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2616/4795] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:46 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1320 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:18:46.751188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-734825361", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "1dfd4252-9b03-463e-9f71-8ecdabb54d49"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:46.830329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] validate_networks() for [('1dfd4252-9b03-463e-9f71-8ecdabb54d49', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:18:47.233680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8f3d8d2e-5a19-446b-bffc-e3eadb04e73e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:47.242603 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f3d8d2e-5a19-446b-bffc-e3eadb04e73e tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1478 microversion: 2.1 time: 1.338074 Sep 27 10:18:47.243197 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2180/4796] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:45 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1478 bytes in 1342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:47.626959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:18:47.627534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:18:47.628527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:18:47.629001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:18:47.629437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:18:47.649628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:47.655484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:47.655943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:47.656784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:47.657250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:47.660859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:47.690961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Getting quotas for project 8070262519034021a2b6de1a1e1ec88a. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:18:47.701900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Getting quotas for user dbf876e1c23642578363f9c5995f3d86 and project 8070262519034021a2b6de1a1e1ec88a. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:18:47.733998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:18:47.745175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:18:47.759929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:47.760560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:47.778863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:47.780109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:47.780786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:47.781723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:47.784745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:47.785483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:47.832291 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e9edb5d-e999-4ca1-a968-dc91bc080597 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.086333 Sep 27 10:18:47.833142 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2617/4797] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:18:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 1088 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:18:47.859821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5a66c924-2fe0-4971-b064-4d560c9fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:47.860564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5a66c924-2fe0-4971-b064-4d560c9fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:47.950508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5a66c924-2fe0-4971-b064-4d560c9fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:47.968902 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5a66c924-2fe0-4971-b064-4d560c9fb568 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1217 microversion: 2.1 time: 0.124212 Sep 27 10:18:47.969495 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2618/4798] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:47 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1217 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:48.267554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75580c4f-0f3f-405a-bba0-942f7d76f436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:48.268433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-75580c4f-0f3f-405a-bba0-942f7d76f436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:48.286300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75580c4f-0f3f-405a-bba0-942f7d76f436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:48.287165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75580c4f-0f3f-405a-bba0-942f7d76f436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:48.287871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75580c4f-0f3f-405a-bba0-942f7d76f436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:48.288805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75580c4f-0f3f-405a-bba0-942f7d76f436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:48.289427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75580c4f-0f3f-405a-bba0-942f7d76f436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:48.290015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75580c4f-0f3f-405a-bba0-942f7d76f436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:48.862303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:48.874600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:48.879629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:48.881189 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d2cf786a-1424-4232-9b09-baf30dec0d8d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1362 microversion: 2.26 time: 1.126508 Sep 27 10:18:48.905511 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2181/4799] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:47 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1362 bytes in 1143 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:18:48.911183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-75580c4f-0f3f-405a-bba0-942f7d76f436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:48.923648 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-75580c4f-0f3f-405a-bba0-942f7d76f436 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1478 microversion: 2.1 time: 0.660793 Sep 27 10:18:48.926424 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2619/4800] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:48 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1478 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:48.993861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7acbe16f-90eb-42ba-be2f-416faf9b7eb3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:48.994480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7acbe16f-90eb-42ba-be2f-416faf9b7eb3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:49.006037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7acbe16f-90eb-42ba-be2f-416faf9b7eb3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:49.006577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7acbe16f-90eb-42ba-be2f-416faf9b7eb3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:49.013264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7acbe16f-90eb-42ba-be2f-416faf9b7eb3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:49.014069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7acbe16f-90eb-42ba-be2f-416faf9b7eb3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:49.014710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7acbe16f-90eb-42ba-be2f-416faf9b7eb3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:49.015053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7acbe16f-90eb-42ba-be2f-416faf9b7eb3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:49.365425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7acbe16f-90eb-42ba-be2f-416faf9b7eb3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:49.376511 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7acbe16f-90eb-42ba-be2f-416faf9b7eb3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1213 microversion: 2.1 time: 0.385025 Sep 27 10:18:49.394400 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2182/4801] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:48 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1213 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:49.424644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8680edec-2039-4f95-af57-24cc7eeeff2a req-41240fc0-17fa-4d2c-aa79-6a22cd1245ee service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c45ec10-5528-48be-bc7b-4e628abd6d85", "name": "network-vif-deleted", "tag": "d2a12997-e121-466a-a120-ced0de0c1108"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:49.442723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8680edec-2039-4f95-af57-24cc7eeeff2a req-41240fc0-17fa-4d2c-aa79-6a22cd1245ee service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:49.443294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8680edec-2039-4f95-af57-24cc7eeeff2a req-41240fc0-17fa-4d2c-aa79-6a22cd1245ee service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:49.443856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8680edec-2039-4f95-af57-24cc7eeeff2a req-41240fc0-17fa-4d2c-aa79-6a22cd1245ee service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:49.457872 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8680edec-2039-4f95-af57-24cc7eeeff2a req-41240fc0-17fa-4d2c-aa79-6a22cd1245ee service nova] Creating event network-vif-deleted:d2a12997-e121-466a-a120-ced0de0c1108 for instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 on nch1-master-1 Sep 27 10:18:49.468502 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8680edec-2039-4f95-af57-24cc7eeeff2a req-41240fc0-17fa-4d2c-aa79-6a22cd1245ee service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047419 Sep 27 10:18:49.473915 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2620/4802] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:18:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:18:49.913603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:49.914348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:49.930140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:49.931922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:49.932689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:49.933203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:49.934007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:49.934529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:49.935052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:49.957958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1f79b629-c89d-4d90-9da7-74662580367a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:49.959947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1f79b629-c89d-4d90-9da7-74662580367a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:49.967418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f79b629-c89d-4d90-9da7-74662580367a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:49.968142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f79b629-c89d-4d90-9da7-74662580367a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:49.968888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f79b629-c89d-4d90-9da7-74662580367a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:49.969803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f79b629-c89d-4d90-9da7-74662580367a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:49.970465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f79b629-c89d-4d90-9da7-74662580367a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:49.971105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f79b629-c89d-4d90-9da7-74662580367a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:50.383511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1f79b629-c89d-4d90-9da7-74662580367a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:50.391994 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1f79b629-c89d-4d90-9da7-74662580367a tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1478 microversion: 2.1 time: 0.442283 Sep 27 10:18:50.405106 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2621/4803] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:49 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1478 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:50.416274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3e137443-856e-4012-88da-009d5c82656a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:50.417212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3e137443-856e-4012-88da-009d5c82656a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:50.423823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e137443-856e-4012-88da-009d5c82656a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:50.424862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e137443-856e-4012-88da-009d5c82656a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:50.425550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e137443-856e-4012-88da-009d5c82656a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:50.426605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e137443-856e-4012-88da-009d5c82656a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:50.427369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e137443-856e-4012-88da-009d5c82656a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:50.428072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e137443-856e-4012-88da-009d5c82656a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:50.542655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3e137443-856e-4012-88da-009d5c82656a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:50.553490 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3e137443-856e-4012-88da-009d5c82656a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1275 microversion: 2.1 time: 0.146711 Sep 27 10:18:50.557056 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2622/4804] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:50 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1275 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:51.191004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:51.203558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:51.205958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:51.208371 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0b5fe6d4-85a4-4def-947e-e7f11f42a518 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1362 microversion: 2.26 time: 1.299447 Sep 27 10:18:51.210540 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2183/4805] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:49 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1362 bytes in 1303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:18:51.428382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-baac09d5-49d8-4e69-b9d2-9e0fd8c5b0c8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:51.430070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-baac09d5-49d8-4e69-b9d2-9e0fd8c5b0c8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:51.450596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baac09d5-49d8-4e69-b9d2-9e0fd8c5b0c8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:51.451125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baac09d5-49d8-4e69-b9d2-9e0fd8c5b0c8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:51.451570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baac09d5-49d8-4e69-b9d2-9e0fd8c5b0c8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:51.452309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baac09d5-49d8-4e69-b9d2-9e0fd8c5b0c8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:51.459488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baac09d5-49d8-4e69-b9d2-9e0fd8c5b0c8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:51.459956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baac09d5-49d8-4e69-b9d2-9e0fd8c5b0c8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:51.580931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-372577b7-458c-4e6c-ad76-a4ee9462b8bf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:51.581732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-372577b7-458c-4e6c-ad76-a4ee9462b8bf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:51.598938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-372577b7-458c-4e6c-ad76-a4ee9462b8bf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:51.599513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-372577b7-458c-4e6c-ad76-a4ee9462b8bf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:51.600159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-372577b7-458c-4e6c-ad76-a4ee9462b8bf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:51.600936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-372577b7-458c-4e6c-ad76-a4ee9462b8bf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:51.601399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-372577b7-458c-4e6c-ad76-a4ee9462b8bf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:51.601827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-372577b7-458c-4e6c-ad76-a4ee9462b8bf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:51.830731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-baac09d5-49d8-4e69-b9d2-9e0fd8c5b0c8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:51.857240 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-baac09d5-49d8-4e69-b9d2-9e0fd8c5b0c8 tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1478 microversion: 2.1 time: 0.428091 Sep 27 10:18:51.858870 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2623/4806] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:51 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1478 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:52.137205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-cc860c77-098e-4b9c-b155-99b1fb6168d9 req-7991d25d-aec6-4fac-992d-16eb545bba8e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f", "tag": "fcc2577b-5d05-4a10-a75c-efaed650824e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:52.175292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cc860c77-098e-4b9c-b155-99b1fb6168d9 req-7991d25d-aec6-4fac-992d-16eb545bba8e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:52.176095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cc860c77-098e-4b9c-b155-99b1fb6168d9 req-7991d25d-aec6-4fac-992d-16eb545bba8e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:52.176723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cc860c77-098e-4b9c-b155-99b1fb6168d9 req-7991d25d-aec6-4fac-992d-16eb545bba8e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:52.194221 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-cc860c77-098e-4b9c-b155-99b1fb6168d9 req-7991d25d-aec6-4fac-992d-16eb545bba8e service nova] Creating event network-changed:fcc2577b-5d05-4a10-a75c-efaed650824e for instance 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f on nch1-master-1 Sep 27 10:18:52.205845 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-cc860c77-098e-4b9c-b155-99b1fb6168d9 req-7991d25d-aec6-4fac-992d-16eb545bba8e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072511 Sep 27 10:18:52.207065 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2624/4807] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:18:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:18:52.240081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:52.240943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:52.261551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:52.262072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:52.262522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:52.263292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:52.263743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:52.264399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:52.559624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-372577b7-458c-4e6c-ad76-a4ee9462b8bf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:52.572227 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-372577b7-458c-4e6c-ad76-a4ee9462b8bf tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1317 microversion: 2.1 time: 0.995395 Sep 27 10:18:52.588791 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2184/4808] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:51 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1317 bytes in 1013 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:52.700284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1fd33f7a-4d7d-4354-ba97-e8c62dbc426a req-ac450e31-3f2b-429b-a511-3b9c5ac6838a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3cd54bff-350d-4106-9562-7ccfce2a106c", "name": "network-vif-deleted", "tag": "51432875-d706-4015-876d-392752eeb855"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:52.743530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1fd33f7a-4d7d-4354-ba97-e8c62dbc426a req-ac450e31-3f2b-429b-a511-3b9c5ac6838a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:52.744049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1fd33f7a-4d7d-4354-ba97-e8c62dbc426a req-ac450e31-3f2b-429b-a511-3b9c5ac6838a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:52.744489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1fd33f7a-4d7d-4354-ba97-e8c62dbc426a req-ac450e31-3f2b-429b-a511-3b9c5ac6838a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:52.786053 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1fd33f7a-4d7d-4354-ba97-e8c62dbc426a req-ac450e31-3f2b-429b-a511-3b9c5ac6838a service nova] Creating event network-vif-deleted:51432875-d706-4015-876d-392752eeb855 for instance 3cd54bff-350d-4106-9562-7ccfce2a106c on nch1-master-1 Sep 27 10:18:52.814466 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1fd33f7a-4d7d-4354-ba97-e8c62dbc426a req-ac450e31-3f2b-429b-a511-3b9c5ac6838a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.123040 Sep 27 10:18:52.815239 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2185/4809] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:18:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:18:52.896076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:52.900258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:52.954710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:52.955286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:52.955679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:52.961423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:52.962689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:52.964363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:53.087883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:53.090736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:53.091398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:53.092226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lazy-loading 'fault' on Instance uuid 3cd54bff-350d-4106-9562-7ccfce2a106c {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:18:53.353246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:53.376596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:53.378778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:53.381007 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bfbc610d-466a-4a20-a72d-190c7a446678 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1362 microversion: 2.26 time: 1.145124 Sep 27 10:18:53.381642 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2625/4810] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:52 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1362 bytes in 1147 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:18:53.599326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-89f585d1-fb44-4109-b4d8-0dfa5c4bfd1a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:53.600406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-89f585d1-fb44-4109-b4d8-0dfa5c4bfd1a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:53.622567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89f585d1-fb44-4109-b4d8-0dfa5c4bfd1a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:53.623177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89f585d1-fb44-4109-b4d8-0dfa5c4bfd1a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:53.623720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89f585d1-fb44-4109-b4d8-0dfa5c4bfd1a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:53.624548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89f585d1-fb44-4109-b4d8-0dfa5c4bfd1a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:53.625278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89f585d1-fb44-4109-b4d8-0dfa5c4bfd1a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:53.626291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89f585d1-fb44-4109-b4d8-0dfa5c4bfd1a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:53.689258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7b8599d0b40424087e52d8e35b71ec4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1d229e5b879b481082284236c9f8c446', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:53.697302 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fbe02c92-1cf2-454e-af3d-7c75b2a859ff tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 200 len: 1282 microversion: 2.1 time: 0.812617 Sep 27 10:18:53.697897 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2186/4811] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:52 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 1282 bytes in 814 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:54.253192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-89f585d1-fb44-4109-b4d8-0dfa5c4bfd1a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:54.273612 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-89f585d1-fb44-4109-b4d8-0dfa5c4bfd1a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1317 microversion: 2.1 time: 0.676412 Sep 27 10:18:54.346655 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2626/4812] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:53 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1317 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:54.408074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:54.408879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:54.422373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:54.424569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:54.425303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:54.426261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:54.426982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:54.428395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:54.727426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:54.728053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] [instance: 3cd54bff-350d-4106-9562-7ccfce2a106c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:54.741783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:54.744688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:54.747792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:54.748925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:54.750917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:54.751562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:54.789717 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] HTTP exception thrown: Instance 3cd54bff-350d-4106-9562-7ccfce2a106c could not be found. Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Request method failure captured: Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c HTTP/1.1 Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs0Nx5CSgiH-1PgMj95idy2d6l3SsJM4NS-jJ7xsDN-5GUo_1aDVwT1GRgAdl6G2KVtqsED4jhfPlkfvts7dy-dGshobmxwBfQjhBmnFhj-UkAo7CJ9yl_hgAMsBVqXUS5m12_4rOmcnK-Mw-8NkIMj2vR1unKwubPpCOF79oJJYjFY Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1d229e5b879b481082284236c9f8c446"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1d229e5b879b481082284236c9f8c446"}]}] Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersTestJSON-1366710611 Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1d229e5b879b481082284236c9f8c446 Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersTestJSON-1366710611 Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a7b8599d0b40424087e52d8e35b71ec4 Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersTestJSON-1366710611-project Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 3cd54bff-350d-4106-9562-7ccfce2a106c could not be found. Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3cd54bff-350d-4106-9562-7ccfce2a106c'} Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3cd54bff-350d-4106-9562-7ccfce2a106c could not be found. Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3cd54bff-350d-4106-9562-7ccfce2a106c could not be found. Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:18:54.794576 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:54.799696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:18:54.799696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:18:54.799696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:18:54.799696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:18:54.799696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:18:54.799696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3cd54bff-350d-4106-9562-7ccfce2a106c could not be found. Sep 27 10:18:54.799696 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:18:54.801506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] Returning 404 to user: Instance 3cd54bff-350d-4106-9562-7ccfce2a106c could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:18:54.802660 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b593f644-1b5c-4a6b-bfff-efacfb31293f tempest-ServersTestJSON-1366710611 tempest-ServersTestJSON-1366710611-project] 10.222.0.26 "GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c" status: 404 len: 111 microversion: 2.1 time: 0.081617 Sep 27 10:18:54.807278 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2627/4813] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:54 2022] GET /compute/v2.1/servers/3cd54bff-350d-4106-9562-7ccfce2a106c => generated 111 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:18:54.878639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7e19fd18-7025-41ea-ba97-3f15fe301ebb req-d0f170e1-c15a-4202-93db-090ff8bfcd68 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "176eec03-6637-4f3c-b92b-6f15e07af753", "tag": "22fb14d0-2bc4-4d8f-b5d6-c9ef641e6a5d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:54.906767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7e19fd18-7025-41ea-ba97-3f15fe301ebb req-d0f170e1-c15a-4202-93db-090ff8bfcd68 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:54.907306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7e19fd18-7025-41ea-ba97-3f15fe301ebb req-d0f170e1-c15a-4202-93db-090ff8bfcd68 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:54.907891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7e19fd18-7025-41ea-ba97-3f15fe301ebb req-d0f170e1-c15a-4202-93db-090ff8bfcd68 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:54.941589 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7e19fd18-7025-41ea-ba97-3f15fe301ebb req-d0f170e1-c15a-4202-93db-090ff8bfcd68 service nova] Creating event network-changed:22fb14d0-2bc4-4d8f-b5d6-c9ef641e6a5d for instance 176eec03-6637-4f3c-b92b-6f15e07af753 on nch2-master-1 Sep 27 10:18:54.954287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7e19fd18-7025-41ea-ba97-3f15fe301ebb req-d0f170e1-c15a-4202-93db-090ff8bfcd68 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.079466 Sep 27 10:18:54.957695 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2628/4814] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:18:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:18:55.305670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-787188a8-9f7d-4c25-9873-61648c710e7a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:55.306727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-787188a8-9f7d-4c25-9873-61648c710e7a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:55.316364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787188a8-9f7d-4c25-9873-61648c710e7a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:55.323315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787188a8-9f7d-4c25-9873-61648c710e7a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:55.324128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787188a8-9f7d-4c25-9873-61648c710e7a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:55.325075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787188a8-9f7d-4c25-9873-61648c710e7a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:55.325562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787188a8-9f7d-4c25-9873-61648c710e7a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:55.326000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-787188a8-9f7d-4c25-9873-61648c710e7a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:55.379033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:55.401062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:55.407239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:55.409045 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f6b0d887-f7ce-4372-a7c7-29718c6f5280 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1362 microversion: 2.26 time: 1.005484 Sep 27 10:18:55.410461 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2187/4815] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:54 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1362 bytes in 1007 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:18:56.441459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:56.444697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:56.466353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:56.467762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:56.468688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:56.469724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:56.471556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:56.472181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:56.486086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-787188a8-9f7d-4c25-9873-61648c710e7a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:56.500303 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-787188a8-9f7d-4c25-9873-61648c710e7a tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1317 microversion: 2.1 time: 1.200309 Sep 27 10:18:56.502090 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2629/4816] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:55 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1317 bytes in 1203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:18:56.875165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:56.901350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:56.909438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:56.914559 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f753edf5-0912-4831-ac8d-05e5c75f4e74 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1362 microversion: 2.26 time: 0.481951 Sep 27 10:18:56.915915 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2188/4817] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:56 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1362 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:18:57.524359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-24c8c726-8bac-4dc0-8bbd-34f95d77953d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:57.525137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-24c8c726-8bac-4dc0-8bbd-34f95d77953d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:57.535210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c8c726-8bac-4dc0-8bbd-34f95d77953d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:57.539259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c8c726-8bac-4dc0-8bbd-34f95d77953d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:57.540036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c8c726-8bac-4dc0-8bbd-34f95d77953d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:57.543277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c8c726-8bac-4dc0-8bbd-34f95d77953d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:57.544118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c8c726-8bac-4dc0-8bbd-34f95d77953d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:57.548839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24c8c726-8bac-4dc0-8bbd-34f95d77953d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:57.675022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:57.945962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:57.946737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:57.963322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:57.964235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:57.964952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=14 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:57.965543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:57.969274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:57.970666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:57.971209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:58.043480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-24c8c726-8bac-4dc0-8bbd-34f95d77953d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:58.078817 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-24c8c726-8bac-4dc0-8bbd-34f95d77953d tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1317 microversion: 2.1 time: 0.535913 Sep 27 10:18:58.097490 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2630/4818] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:57 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1317 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:18:58.807735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:58.822480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:58.829528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:18:58.831730 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3c16f9c4-2b86-4061-a8b2-b311d84e5a42 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1362 microversion: 2.26 time: 0.892075 Sep 27 10:18:58.834331 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2189/4819] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:18:57 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1362 bytes in 895 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:18:59.105272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ddfdbba-89b4-420a-9acb-8da4cd28dd21 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:59.105932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ddfdbba-89b4-420a-9acb-8da4cd28dd21 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:18:59.126493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ddfdbba-89b4-420a-9acb-8da4cd28dd21 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:59.127016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ddfdbba-89b4-420a-9acb-8da4cd28dd21 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:59.130018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ddfdbba-89b4-420a-9acb-8da4cd28dd21 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:59.132938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ddfdbba-89b4-420a-9acb-8da4cd28dd21 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:59.134068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ddfdbba-89b4-420a-9acb-8da4cd28dd21 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:59.134515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ddfdbba-89b4-420a-9acb-8da4cd28dd21 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:59.309435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-be501b13-6ce4-4ecf-86b3-f72791d2ab5d tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Action: 'create', calling method: >, body: {"aggregate": {"name": "tempest-aggregate-scenario-82879702", "availability_zone": "foo_zone"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:59.371524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.aggregate [None req-be501b13-6ce4-4ecf-86b3-f72791d2ab5d tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Generated uuid a2681fa3-6a9b-4858-a85d-82978fe0a9d5 for aggregate {{(pid=145968) create /opt/stack/nova/nova/objects/aggregate.py:298}} Sep 27 10:18:59.464584 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-be501b13-6ce4-4ecf-86b3-f72791d2ab5d tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates" status: 200 len: 207 microversion: 2.1 time: 0.696351 Sep 27 10:18:59.466856 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2631/4820] 10.222.0.26 () {60 vars in 1164 bytes} [Tue Sep 27 10:18:58 2022] POST /compute/v2.1/os-aggregates => generated 207 bytes in 701 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:18:59.477584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0dcd9a51-3e71-474e-a2ec-7182201cea4f req-b1720114-57d3-490d-a543-3bf67ede6dc2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f", "name": "network-vif-plugged", "status": "completed", "tag": "fcc2577b-5d05-4a10-a75c-efaed650824e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:59.498897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0dcd9a51-3e71-474e-a2ec-7182201cea4f req-b1720114-57d3-490d-a543-3bf67ede6dc2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:59.501660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0dcd9a51-3e71-474e-a2ec-7182201cea4f req-b1720114-57d3-490d-a543-3bf67ede6dc2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:59.502944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0dcd9a51-3e71-474e-a2ec-7182201cea4f req-b1720114-57d3-490d-a543-3bf67ede6dc2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:59.530939 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0dcd9a51-3e71-474e-a2ec-7182201cea4f req-b1720114-57d3-490d-a543-3bf67ede6dc2 service nova] Creating event network-vif-plugged:fcc2577b-5d05-4a10-a75c-efaed650824e for instance 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f on nch1-master-1 Sep 27 10:18:59.544505 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0dcd9a51-3e71-474e-a2ec-7182201cea4f req-b1720114-57d3-490d-a543-3bf67ede6dc2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075418 Sep 27 10:18:59.546319 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2632/4821] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:18:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:18:59.558156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2bb39a9b-b142-4034-b7df-f61d6a19f16b tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Action: 'action', calling method: >, body: {"set_metadata": {"metadata": {"meta_key": "meta_value"}}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:59.634776 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2bb39a9b-b142-4034-b7df-f61d6a19f16b tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 317 microversion: 2.1 time: 0.086149 Sep 27 10:18:59.636322 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2633/4822] 10.222.0.26 () {60 vars in 1194 bytes} [Tue Sep 27 10:18:59 2022] POST /compute/v2.1/os-aggregates/19/action => generated 317 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:18:59.649592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f053d25-3e63-4ce9-8ab1-1f3c471e49ef tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:59.660340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f053d25-3e63-4ce9-8ab1-1f3c471e49ef tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:59.661294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f053d25-3e63-4ce9-8ab1-1f3c471e49ef tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:59.662827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f053d25-3e63-4ce9-8ab1-1f3c471e49ef tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:59.666397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f053d25-3e63-4ce9-8ab1-1f3c471e49ef tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:59.669036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f053d25-3e63-4ce9-8ab1-1f3c471e49ef tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:59.669737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f053d25-3e63-4ce9-8ab1-1f3c471e49ef tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:18:59.737110 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7f053d25-3e63-4ce9-8ab1-1f3c471e49ef tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] 10.222.0.26 "GET /compute/v2.1/os-services?binary=nova-compute" status: 200 len: 380 microversion: 2.1 time: 0.092410 Sep 27 10:18:59.741279 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2634/4823] 10.222.0.26 () {58 vars in 1176 bytes} [Tue Sep 27 10:18:59 2022] GET /compute/v2.1/os-services?binary=nova-compute => generated 380 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:18:59.753206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cbe17f34-cec6-4bc2-be2e-6bb9a018c319 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:18:59.774613 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cbe17f34-cec6-4bc2-be2e-6bb9a018c319 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates" status: 200 len: 296 microversion: 2.1 time: 0.025284 Sep 27 10:18:59.776245 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2635/4824] 10.222.0.26 () {58 vars in 1143 bytes} [Tue Sep 27 10:18:59 2022] GET /compute/v2.1/os-aggregates => generated 296 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:18:59.795056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-857ce84d-5cbb-4320-a87a-94db634f8c57 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Action: 'action', calling method: >, body: {"add_host": {"host": "nch1-master-1"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:18:59.823244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=1,nova_api:INSERT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:18:59.831305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-857ce84d-5cbb-4320-a87a-94db634f8c57 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:18:59.831801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-857ce84d-5cbb-4320-a87a-94db634f8c57 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:18:59.832439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-857ce84d-5cbb-4320-a87a-94db634f8c57 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:00.017743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8ddfdbba-89b4-420a-9acb-8da4cd28dd21 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:00.036092 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8ddfdbba-89b4-420a-9acb-8da4cd28dd21 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1317 microversion: 2.1 time: 0.933503 Sep 27 10:19:00.041096 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2190/4825] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:18:59 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1317 bytes in 943 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:19:00.063545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:00.064349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:00.081210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:00.081968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:00.085730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:00.086677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:00.087230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:00.087752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:00.456475 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-857ce84d-5cbb-4320-a87a-94db634f8c57 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 308 microversion: 2.1 time: 0.671292 Sep 27 10:19:00.460715 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2636/4826] 10.222.0.26 () {60 vars in 1194 bytes} [Tue Sep 27 10:18:59 2022] POST /compute/v2.1/os-aggregates/19/action => generated 308 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:00.472383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8eac4354-65b1-4ff7-a6ba-0a47a7de6c19 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:00.491228 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8eac4354-65b1-4ff7-a6ba-0a47a7de6c19 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates/19" status: 200 len: 308 microversion: 2.1 time: 0.023757 Sep 27 10:19:00.501862 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2637/4827] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 10:19:00 2022] GET /compute/v2.1/os-aggregates/19 => generated 308 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:19:00.508254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fd4521d7-5e62-4719-89d9-18084ec00642 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Action: 'update', calling method: >, body: {"aggregate": {"name": "tempest-renamed-aggregate-scenario-1833589387", "availability_zone": "foo_zone"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:00.526119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:00.529069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:00.546528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:00.548008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:00.562642 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5fd98917-b29f-4c9d-bb35-9260cd2f3ae9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1362 microversion: 2.26 time: 0.518752 Sep 27 10:19:00.563470 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2191/4828] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:00 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1362 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:00.969226 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fd4521d7-5e62-4719-89d9-18084ec00642 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] 10.222.0.26 "PUT /compute/v2.1/os-aggregates/19" status: 200 len: 342 microversion: 2.1 time: 0.465724 Sep 27 10:19:00.969835 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2638/4829] 10.222.0.26 () {60 vars in 1173 bytes} [Tue Sep 27 10:19:00 2022] PUT /compute/v2.1/os-aggregates/19 => generated 342 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:19:00.987899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7c616f6e-15a9-457a-9257-f951a23c36c6 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Action: 'action', calling method: >, body: {"set_metadata": {"metadata": {"foo": "bar"}}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:01.081383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-64b1c305-06cf-47d2-b680-26228bf25971 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:01.082538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-64b1c305-06cf-47d2-b680-26228bf25971 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:01.102933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b1c305-06cf-47d2-b680-26228bf25971 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:01.106544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b1c305-06cf-47d2-b680-26228bf25971 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:01.109095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b1c305-06cf-47d2-b680-26228bf25971 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:01.110478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b1c305-06cf-47d2-b680-26228bf25971 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:01.113943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b1c305-06cf-47d2-b680-26228bf25971 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:01.116420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64b1c305-06cf-47d2-b680-26228bf25971 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:01.127717 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7c616f6e-15a9-457a-9257-f951a23c36c6 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 356 microversion: 2.1 time: 0.145953 Sep 27 10:19:01.133177 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2192/4830] 10.222.0.26 () {60 vars in 1194 bytes} [Tue Sep 27 10:19:00 2022] POST /compute/v2.1/os-aggregates/19/action => generated 356 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:01.141547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-76f803ab-f89f-4e97-853a-bbac4661d9a2 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:01.189103 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-76f803ab-f89f-4e97-853a-bbac4661d9a2 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] 10.222.0.26 "GET /compute/v2.1/os-aggregates/19" status: 200 len: 356 microversion: 2.1 time: 0.050826 Sep 27 10:19:01.206708 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2193/4831] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 10:19:01 2022] GET /compute/v2.1/os-aggregates/19 => generated 356 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:01.226539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-48a09cf5-2791-45d8-9fde-f755026819d3 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Action: 'action', calling method: >, body: {"remove_host": {"host": "nch1-master-1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:01.249567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48a09cf5-2791-45d8-9fde-f755026819d3 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:01.250479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48a09cf5-2791-45d8-9fde-f755026819d3 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:01.251090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48a09cf5-2791-45d8-9fde-f755026819d3 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:01.495576 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-48a09cf5-2791-45d8-9fde-f755026819d3 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] 10.222.0.26 "POST /compute/v2.1/os-aggregates/19/action" status: 200 len: 341 microversion: 2.1 time: 0.280925 Sep 27 10:19:01.499470 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2194/4832] 10.222.0.26 () {60 vars in 1194 bytes} [Tue Sep 27 10:19:01 2022] POST /compute/v2.1/os-aggregates/19/action => generated 341 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:01.508347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-31274c15-0334-455a-9c4e-0a47db3ebdd9 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:01.569985 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-31274c15-0334-455a-9c4e-0a47db3ebdd9 tempest-TestAggregatesBasicOps-1589440086 tempest-TestAggregatesBasicOps-1589440086-project-admin] 10.222.0.26 "DELETE /compute/v2.1/os-aggregates/19" status: 200 len: 0 microversion: 2.1 time: 0.065812 Sep 27 10:19:01.572300 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2195/4833] 10.222.0.26 () {58 vars in 1155 bytes} [Tue Sep 27 10:19:01 2022] DELETE /compute/v2.1/os-aggregates/19 => generated 0 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 355 bytes (2 switches on core 0) Sep 27 10:19:01.597838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:01.598469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:01.615206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:01.616028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:01.616685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:01.617629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:01.625005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:01.625503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:01.953384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-64b1c305-06cf-47d2-b680-26228bf25971 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:01.962793 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-64b1c305-06cf-47d2-b680-26228bf25971 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1511 microversion: 2.1 time: 0.900218 Sep 27 10:19:01.965706 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2639/4834] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:01 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1511 bytes in 904 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:01.991659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15de4b88-5739-4417-81b8-8fd63d797115 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:01.992289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-15de4b88-5739-4417-81b8-8fd63d797115 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:02.003509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15de4b88-5739-4417-81b8-8fd63d797115 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:02.003878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15de4b88-5739-4417-81b8-8fd63d797115 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:02.004321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15de4b88-5739-4417-81b8-8fd63d797115 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:02.005265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15de4b88-5739-4417-81b8-8fd63d797115 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:02.005778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15de4b88-5739-4417-81b8-8fd63d797115 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:02.006259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15de4b88-5739-4417-81b8-8fd63d797115 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:02.534756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-15de4b88-5739-4417-81b8-8fd63d797115 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:02.540695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:02.552712 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-15de4b88-5739-4417-81b8-8fd63d797115 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1507 microversion: 2.1 time: 0.564484 Sep 27 10:19:02.583837 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2640/4835] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:01 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1507 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:02.596327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-27a4f0d5-560d-40ae-8b44-d0420920d548 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "AUTO"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:02.631970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-27a4f0d5-560d-40ae-8b44-d0420920d548 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:02.646884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27a4f0d5-560d-40ae-8b44-d0420920d548 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:02.647725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27a4f0d5-560d-40ae-8b44-d0420920d548 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:02.649393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27a4f0d5-560d-40ae-8b44-d0420920d548 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:02.657333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27a4f0d5-560d-40ae-8b44-d0420920d548 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:02.658653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27a4f0d5-560d-40ae-8b44-d0420920d548 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:02.661155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27a4f0d5-560d-40ae-8b44-d0420920d548 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:02.931455 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-27a4f0d5-560d-40ae-8b44-d0420920d548 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "PUT /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1079 microversion: 2.1 time: 0.345728 Sep 27 10:19:02.932138 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2641/4836] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:19:02 2022] PUT /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1079 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:02.952014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b61d52bd-24df-4dfc-a543-d1689a201cef tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:02.953089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b61d52bd-24df-4dfc-a543-d1689a201cef tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:02.970242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b61d52bd-24df-4dfc-a543-d1689a201cef tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:02.970942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b61d52bd-24df-4dfc-a543-d1689a201cef tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:02.971180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b61d52bd-24df-4dfc-a543-d1689a201cef tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:02.971916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b61d52bd-24df-4dfc-a543-d1689a201cef tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:02.972385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b61d52bd-24df-4dfc-a543-d1689a201cef tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:02.976300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b61d52bd-24df-4dfc-a543-d1689a201cef tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:03.249930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:03.263863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:03.265638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:03.266795 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cdd6446c-4cdf-4274-bd1f-3465da1acaf7 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1362 microversion: 2.26 time: 1.678509 Sep 27 10:19:03.267737 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2196/4837] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:01 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1362 bytes in 1680 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:19:03.521366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b61d52bd-24df-4dfc-a543-d1689a201cef tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:03.535386 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b61d52bd-24df-4dfc-a543-d1689a201cef tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1505 microversion: 2.1 time: 0.587598 Sep 27 10:19:03.537173 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2642/4838] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:02 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1505 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:19:03.561079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9cab9a89-8979-407a-90b7-c40b5a959fd3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:03.561802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9cab9a89-8979-407a-90b7-c40b5a959fd3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:03.568695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cab9a89-8979-407a-90b7-c40b5a959fd3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:03.569691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cab9a89-8979-407a-90b7-c40b5a959fd3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:03.570796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cab9a89-8979-407a-90b7-c40b5a959fd3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:03.571767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cab9a89-8979-407a-90b7-c40b5a959fd3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:03.572527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cab9a89-8979-407a-90b7-c40b5a959fd3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:03.573197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9cab9a89-8979-407a-90b7-c40b5a959fd3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:04.297589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:04.299600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:04.325608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:04.325964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:04.326394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:04.331376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:04.331863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:04.332301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:04.404526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9cab9a89-8979-407a-90b7-c40b5a959fd3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:04.419892 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9cab9a89-8979-407a-90b7-c40b5a959fd3 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1505 microversion: 2.1 time: 0.859135 Sep 27 10:19:04.420706 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2197/4839] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:03 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1505 bytes in 864 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:04.448682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0651103-988e-4779-abfb-ad15581ae10c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "MANUAL"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:04.453278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b0651103-988e-4779-abfb-ad15581ae10c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:04.463923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0651103-988e-4779-abfb-ad15581ae10c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:04.464961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0651103-988e-4779-abfb-ad15581ae10c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:04.465739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0651103-988e-4779-abfb-ad15581ae10c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:04.466950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0651103-988e-4779-abfb-ad15581ae10c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:04.467875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0651103-988e-4779-abfb-ad15581ae10c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:04.468409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0651103-988e-4779-abfb-ad15581ae10c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:04.600648 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0651103-988e-4779-abfb-ad15581ae10c tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "PUT /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1081 microversion: 2.1 time: 0.158687 Sep 27 10:19:04.602262 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2198/4840] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:19:04 2022] PUT /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1081 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:19:04.608150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f54992ad-7561-404a-a51d-7c22eae9f357 req-d86829b6-d515-4438-ba78-9034046f4298 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "176eec03-6637-4f3c-b92b-6f15e07af753", "name": "network-vif-plugged", "status": "completed", "tag": "22fb14d0-2bc4-4d8f-b5d6-c9ef641e6a5d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:04.627411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f54992ad-7561-404a-a51d-7c22eae9f357 req-d86829b6-d515-4438-ba78-9034046f4298 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:04.630575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f54992ad-7561-404a-a51d-7c22eae9f357 req-d86829b6-d515-4438-ba78-9034046f4298 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:04.631431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f54992ad-7561-404a-a51d-7c22eae9f357 req-d86829b6-d515-4438-ba78-9034046f4298 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:04.648643 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f54992ad-7561-404a-a51d-7c22eae9f357 req-d86829b6-d515-4438-ba78-9034046f4298 service nova] Creating event network-vif-plugged:22fb14d0-2bc4-4d8f-b5d6-c9ef641e6a5d for instance 176eec03-6637-4f3c-b92b-6f15e07af753 on nch2-master-1 Sep 27 10:19:04.659047 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f54992ad-7561-404a-a51d-7c22eae9f357 req-d86829b6-d515-4438-ba78-9034046f4298 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054892 Sep 27 10:19:04.659603 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2199/4841] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:19:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:04.682979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6c76a38f-0d90-42e6-8271-e481f886d9a7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:04.690293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6c76a38f-0d90-42e6-8271-e481f886d9a7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:04.701364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c76a38f-0d90-42e6-8271-e481f886d9a7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:04.701859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c76a38f-0d90-42e6-8271-e481f886d9a7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:04.702301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c76a38f-0d90-42e6-8271-e481f886d9a7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:04.705641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c76a38f-0d90-42e6-8271-e481f886d9a7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:04.706359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c76a38f-0d90-42e6-8271-e481f886d9a7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:04.707876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c76a38f-0d90-42e6-8271-e481f886d9a7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:05.022322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:05.043030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:05.045142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:05.046353 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4c074c0-3784-4b4b-b919-9d4191d2d239 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1362 microversion: 2.26 time: 0.755003 Sep 27 10:19:05.053040 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2643/4842] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:04 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1362 bytes in 762 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:19:05.461767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6c76a38f-0d90-42e6-8271-e481f886d9a7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:05.472424 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6c76a38f-0d90-42e6-8271-e481f886d9a7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1507 microversion: 2.1 time: 0.806905 Sep 27 10:19:05.473325 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2200/4843] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:04 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1507 bytes in 809 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:05.496516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-935e1e3b-59c3-4e1f-8b45-a4b118a3d35b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:05.498010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-935e1e3b-59c3-4e1f-8b45-a4b118a3d35b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:05.508228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-935e1e3b-59c3-4e1f-8b45-a4b118a3d35b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:05.508780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-935e1e3b-59c3-4e1f-8b45-a4b118a3d35b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:05.509260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-935e1e3b-59c3-4e1f-8b45-a4b118a3d35b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:05.510549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-935e1e3b-59c3-4e1f-8b45-a4b118a3d35b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:05.511314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-935e1e3b-59c3-4e1f-8b45-a4b118a3d35b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:05.512217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-935e1e3b-59c3-4e1f-8b45-a4b118a3d35b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:05.781657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-935e1e3b-59c3-4e1f-8b45-a4b118a3d35b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:05.790576 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-935e1e3b-59c3-4e1f-8b45-a4b118a3d35b tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1507 microversion: 2.1 time: 0.298475 Sep 27 10:19:05.791589 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2644/4844] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:05 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1507 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:05.815093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ac4dcb7-972c-4229-9e74-bc37b85fcdf7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:05.815850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ac4dcb7-972c-4229-9e74-bc37b85fcdf7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:05.824050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac4dcb7-972c-4229-9e74-bc37b85fcdf7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:05.824956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac4dcb7-972c-4229-9e74-bc37b85fcdf7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:05.825635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac4dcb7-972c-4229-9e74-bc37b85fcdf7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:05.826540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac4dcb7-972c-4229-9e74-bc37b85fcdf7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:05.827156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac4dcb7-972c-4229-9e74-bc37b85fcdf7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:05.828247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8ac4dcb7-972c-4229-9e74-bc37b85fcdf7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:05.845238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e43db637-d3de-49d2-9a77-92382d790aca tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Action: 'create', calling method: .version_select at 0x7f7f680525e0>, body: {"keypair": {"name": "tempest-TestNetworkBasicOps-1625678094", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOKHQNETrWp1KDPS4beYDpi90QoEiZUINM85ExBq5dPenu3UDyZikjDDB7JqxIxWvsoJ8YaDYzCYidaEm4qnFSYXewJwv0d/mSq7RaejIgUNOfhhQP1mGdlW8WXMVmsQpg=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:05.867245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ac4dcb7-972c-4229-9e74-bc37b85fcdf7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:19:05.875483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-e43db637-d3de-49d2-9a77-92382d790aca tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:19:05.883788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-e43db637-d3de-49d2-9a77-92382d790aca tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for user 1d92f7d2ee2d47beaa4f5d43ac40cd82 and project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:19:05.927028 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e43db637-d3de-49d2-9a77-92382d790aca tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 398 microversion: 2.1 time: 0.085598 Sep 27 10:19:05.928284 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2645/4845] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:19:05 2022] POST /compute/v2.1/os-keypairs => generated 398 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:19:05.997920 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8ac4dcb7-972c-4229-9e74-bc37b85fcdf7 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 204 len: 0 microversion: 2.1 time: 0.187068 Sep 27 10:19:05.998977 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2201/4846] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:19:05 2022] DELETE /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:19:06.011083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7a8f50ff-d309-4c28-9dd1-0600e07a83db tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:06.011602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7a8f50ff-d309-4c28-9dd1-0600e07a83db tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:06.020743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a8f50ff-d309-4c28-9dd1-0600e07a83db tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:06.021475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a8f50ff-d309-4c28-9dd1-0600e07a83db tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:06.022185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a8f50ff-d309-4c28-9dd1-0600e07a83db tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:06.023556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a8f50ff-d309-4c28-9dd1-0600e07a83db tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:06.024454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a8f50ff-d309-4c28-9dd1-0600e07a83db tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:06.025101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a8f50ff-d309-4c28-9dd1-0600e07a83db tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:06.069634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7a8f50ff-d309-4c28-9dd1-0600e07a83db tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:19:06.079839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:06.080944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:06.087727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:06.088587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:06.089198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:06.090366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:06.091116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:06.091576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:06.142982 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.compute.api [None req-7a8f50ff-d309-4c28-9dd1-0600e07a83db tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Instance is already in deleting state, ignoring this request Sep 27 10:19:06.144200 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7a8f50ff-d309-4c28-9dd1-0600e07a83db tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 204 len: 0 microversion: 2.1 time: 0.139576 Sep 27 10:19:06.145178 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2646/4847] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:19:06 2022] DELETE /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:19:06.157267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-317ccfa9-beec-4648-92de-13db7aeff9eb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:06.158461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-317ccfa9-beec-4648-92de-13db7aeff9eb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:06.168384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317ccfa9-beec-4648-92de-13db7aeff9eb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:06.170022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317ccfa9-beec-4648-92de-13db7aeff9eb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:06.171198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317ccfa9-beec-4648-92de-13db7aeff9eb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:06.172182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317ccfa9-beec-4648-92de-13db7aeff9eb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:06.172935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317ccfa9-beec-4648-92de-13db7aeff9eb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:06.173385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-317ccfa9-beec-4648-92de-13db7aeff9eb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:06.385653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-317ccfa9-beec-4648-92de-13db7aeff9eb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:06.393893 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-317ccfa9-beec-4648-92de-13db7aeff9eb tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1513 microversion: 2.1 time: 0.242594 Sep 27 10:19:06.394853 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2647/4848] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:06 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1513 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:19:06.891212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:06.904240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:06.905720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:06.906981 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b2d85233-42e5-413e-8a5b-8410244f402a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1362 microversion: 2.26 time: 0.833052 Sep 27 10:19:06.907556 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2202/4849] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:06 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1362 bytes in 835 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:07.420339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b6de1033-f43f-4178-8f82-09e9433d7426 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:07.420994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b6de1033-f43f-4178-8f82-09e9433d7426 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:07.446225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6de1033-f43f-4178-8f82-09e9433d7426 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:07.447067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6de1033-f43f-4178-8f82-09e9433d7426 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:07.449347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6de1033-f43f-4178-8f82-09e9433d7426 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:07.454366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6de1033-f43f-4178-8f82-09e9433d7426 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:07.458128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6de1033-f43f-4178-8f82-09e9433d7426 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:07.459910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6de1033-f43f-4178-8f82-09e9433d7426 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:07.940934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:07.942488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:07.952697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:07.954286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:07.955642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:07.957421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:07.958491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:07.959857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:08.402994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:08.429721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:08.433975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:08.436751 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9124cf11-fb2a-4137-80ec-ab6658010de3 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1545 microversion: 2.26 time: 0.501657 Sep 27 10:19:08.438260 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2203/4850] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:07 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1545 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:08.466187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f6248aaf0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:08.494966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:08.495768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:08.498580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:08.501749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:08.517281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:08.518204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:08.518662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:08.519910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:08.521108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:08.522311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:08.528769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b6de1033-f43f-4178-8f82-09e9433d7426 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:08.542186 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b6de1033-f43f-4178-8f82-09e9433d7426 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1513 microversion: 2.1 time: 1.124688 Sep 27 10:19:08.543229 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2648/4851] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:07 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1513 bytes in 1129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:08.587875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:08.589350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:08.589808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:08.630304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:08.631925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:08.633111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:08.697151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:08.734206 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bb510db1-2ae4-4389-ae92-4fc05edf6e79 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-1333692390" status: 201 len: 0 microversion: 2.26 time: 0.272716 Sep 27 10:19:08.736774 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2204/4852] 10.222.0.26 () {62 vars in 1380 bytes} [Tue Sep 27 10:19:08 2022] PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-1333692390 => generated 0 bytes in 277 msecs (HTTP/1.1 201) 10 headers in 486 bytes (1 switches on core 0) Sep 27 10:19:08.761898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9caace05-ded6-492b-9f88-41a53a75200c tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f62bb1040>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:08.791186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9caace05-ded6-492b-9f88-41a53a75200c tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:08.793073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9caace05-ded6-492b-9f88-41a53a75200c tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:08.793994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9caace05-ded6-492b-9f88-41a53a75200c tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:08.816329 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9caace05-ded6-492b-9f88-41a53a75200c tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-1333692390" status: 204 len: 0 microversion: 2.26 time: 0.074533 Sep 27 10:19:08.818501 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2649/4853] 10.222.0.26 () {60 vars in 1361 bytes} [Tue Sep 27 10:19:08 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-1333692390 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:19:08.829152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f629081f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:08.847620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:08.848324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:08.849775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:08.851868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:08.870461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:08.871785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:08.873272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:08.877769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:08.878399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:08.878932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:08.923378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:08.924062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:08.924649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:08.986481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:09.022724 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1a90231a-5e65-4c4a-aef8-b6a7000e084f tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags" status: 204 len: 0 microversion: 2.26 time: 0.198801 Sep 27 10:19:09.026107 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2205/4854] 10.222.0.26 () {60 vars in 1295 bytes} [Tue Sep 27 10:19:08 2022] DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags => generated 0 bytes in 202 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:19:09.037265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-188bb0cf-0c20-4cf0-915a-aa74b7f82b93 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f62936310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:09.051394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-188bb0cf-0c20-4cf0-915a-aa74b7f82b93 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:09.054128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-188bb0cf-0c20-4cf0-915a-aa74b7f82b93 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:09.054586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-188bb0cf-0c20-4cf0-915a-aa74b7f82b93 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:09.118851 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-188bb0cf-0c20-4cf0-915a-aa74b7f82b93 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags" status: 200 len: 12 microversion: 2.26 time: 0.086308 Sep 27 10:19:09.128904 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2650/4855] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 10:19:09 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags => generated 12 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:09.133410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f624c20d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:09.152608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:09.154275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:09.156544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:09.157524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:09.180431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:09.183748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:09.184515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:09.201055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:09.206357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:09.207685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:09.247430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkBasicOps-server-2080058956", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "24bd33a4-77a6-49a3-90c6-bd0b209c26d3"}], "key_name": "tempest-TestNetworkBasicOps-1625678094", "security_groups": [{"name": "tempest-secgroup-smoke-1128327539"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:09.284516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:09.285308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:09.286032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:09.301506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:09.302797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:09.304269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:09.352729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=2,nova_cell1:INSERT=3,nova_cell1:SAVEPOINT=2,nova_cell1:RELEASE=1,nova_cell1:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:09.445864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:09.513913 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dcea5dba-720d-4de0-8a52-6b8464b94728 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-1004706263" status: 201 len: 0 microversion: 2.26 time: 0.383977 Sep 27 10:19:09.519062 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2206/4856] 10.222.0.26 () {62 vars in 1380 bytes} [Tue Sep 27 10:19:09 2022] PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-1004706263 => generated 0 bytes in 389 msecs (HTTP/1.1 201) 10 headers in 486 bytes (1 switches on core 0) Sep 27 10:19:09.534494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] validate_networks() for [('24bd33a4-77a6-49a3-90c6-bd0b209c26d3', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:19:09.538681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ccfe25d0-a9ef-477b-8408-dbaf48d614e7 req-c2bb590e-a670-4cb6-983a-1593a81d3964 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f", "name": "network-vif-deleted", "tag": "fcc2577b-5d05-4a10-a75c-efaed650824e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:09.589981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ccfe25d0-a9ef-477b-8408-dbaf48d614e7 req-c2bb590e-a670-4cb6-983a-1593a81d3964 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:09.592403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ccfe25d0-a9ef-477b-8408-dbaf48d614e7 req-c2bb590e-a670-4cb6-983a-1593a81d3964 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:09.593018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ccfe25d0-a9ef-477b-8408-dbaf48d614e7 req-c2bb590e-a670-4cb6-983a-1593a81d3964 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:09.649273 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ccfe25d0-a9ef-477b-8408-dbaf48d614e7 req-c2bb590e-a670-4cb6-983a-1593a81d3964 service nova] Creating event network-vif-deleted:fcc2577b-5d05-4a10-a75c-efaed650824e for instance 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f on nch1-master-1 Sep 27 10:19:09.664860 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ccfe25d0-a9ef-477b-8408-dbaf48d614e7 req-c2bb590e-a670-4cb6-983a-1593a81d3964 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.142554 Sep 27 10:19:09.666370 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2207/4857] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:19:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:09.683062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a41fd5e5-afa9-4011-b9a3-abab6d84a6fd tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f624429d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:09.690109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:09.705537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=2,nova_api:INSERT=1,nova_api:DELETE=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:09.717895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a41fd5e5-afa9-4011-b9a3-abab6d84a6fd tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:09.719151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a41fd5e5-afa9-4011-b9a3-abab6d84a6fd tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:09.720641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a41fd5e5-afa9-4011-b9a3-abab6d84a6fd tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:09.773319 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a41fd5e5-afa9-4011-b9a3-abab6d84a6fd tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags" status: 200 len: 36 microversion: 2.26 time: 0.103587 Sep 27 10:19:09.774419 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2208/4858] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 10:19:09 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags => generated 36 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:09.787014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:09.787696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:09.798713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:09.800142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:09.801046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:09.801945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:09.802714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:09.803465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:09.883847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:09.884620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:09.885816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:09.886585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lazy-loading 'fault' on Instance uuid 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:10.050347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbf876e1c23642578363f9c5995f3d86', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8070262519034021a2b6de1a1e1ec88a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:10.066270 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0551582b-8632-4e35-9660-34b57ca07d73 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 200 len: 1307 microversion: 2.1 time: 0.290620 Sep 27 10:19:10.067645 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2209/4859] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:09 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 1307 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:10.079096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f62442b80>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:10.111158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.111745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.112205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.112750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:10.135601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.145666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.146340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.147250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.147738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.153527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.223165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.223669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.224266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.291591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:10.342328 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e48d87f8-7887-4c28-88fb-6d76fd143d1a tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-1004706263" status: 204 len: 0 microversion: 2.26 time: 0.272798 Sep 27 10:19:10.347134 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2210/4860] 10.222.0.26 () {60 vars in 1364 bytes} [Tue Sep 27 10:19:10 2022] DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-1004706263 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:19:10.352757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7727fc56-19a0-4bc2-a5b9-3753159f0510 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f624429d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:10.362237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7727fc56-19a0-4bc2-a5b9-3753159f0510 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.363109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7727fc56-19a0-4bc2-a5b9-3753159f0510 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.363792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7727fc56-19a0-4bc2-a5b9-3753159f0510 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.374345 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7727fc56-19a0-4bc2-a5b9-3753159f0510 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags" status: 200 len: 12 microversion: 2.26 time: 0.025851 Sep 27 10:19:10.381252 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2211/4861] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 10:19:10 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags => generated 12 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:10.388591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f624423a0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:10.400450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.401088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.401628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.402148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:10.412391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.419539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.420465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.421594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.422078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.422509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.462517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.463406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.464342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.478061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:10.497699 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e2fac576-5d19-462f-a459-cba74914c9f5 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags" status: 204 len: 0 microversion: 2.26 time: 0.114884 Sep 27 10:19:10.499485 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2212/4862] 10.222.0.26 () {60 vars in 1295 bytes} [Tue Sep 27 10:19:10 2022] DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags => generated 0 bytes in 117 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:19:10.511966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f6251cdc0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:10.522652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.523142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.523894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.524334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:10.531626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.532422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.532862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.533584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.534029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.534482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.597719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.598211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.598646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.609899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.610494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.610937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.644408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:10.663001 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-763e4375-d8f7-471b-9cd6-917744eba317 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-1970029891" status: 201 len: 0 microversion: 2.26 time: 0.155711 Sep 27 10:19:10.663951 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2213/4863] 10.222.0.26 () {62 vars in 1380 bytes} [Tue Sep 27 10:19:10 2022] PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-1970029891 => generated 0 bytes in 158 msecs (HTTP/1.1 201) 10 headers in 486 bytes (2 switches on core 0) Sep 27 10:19:10.680458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f624c25e0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:10.692626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.693634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.695310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.696701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:10.706653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.707249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.707890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.709433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.710580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.711109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.765328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.765768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.766307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.779292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.780043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.780574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.823459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:10.843843 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-402291ee-755f-4228-879f-4ffdc036e3ec tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-761155515" status: 201 len: 0 microversion: 2.26 time: 0.170731 Sep 27 10:19:10.849181 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2214/4864] 10.222.0.26 () {62 vars in 1377 bytes} [Tue Sep 27 10:19:10 2022] PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-761155515 => generated 0 bytes in 173 msecs (HTTP/1.1 201) 10 headers in 485 bytes (1 switches on core 0) Sep 27 10:19:10.857676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f6251c9d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:10.870440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.871188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.871827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.872465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:10.883536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.884681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.885898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.886839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.888119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.888119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.953778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:10.957865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:10.962151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:10.991423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:19:10.992009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:19:10.992441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:19:10.993030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:19:10.993758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:19:11.004866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:11.019869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.023423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.024106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.025219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.034260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.034948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.051654 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ed20fcf1-be5f-4105-8f23-06ebdca89589 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags" status: 204 len: 0 microversion: 2.26 time: 0.199091 Sep 27 10:19:11.052281 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2215/4865] 10.222.0.26 () {60 vars in 1295 bytes} [Tue Sep 27 10:19:10 2022] DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags => generated 0 bytes in 200 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:19:11.056481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:19:11.064550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for user 1d92f7d2ee2d47beaa4f5d43ac40cd82 and project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:19:11.068092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-66855a92-a60c-46f4-ac30-a0a269cc67ae tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f625073a0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:11.083767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:19:11.087223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66855a92-a60c-46f4-ac30-a0a269cc67ae tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.088989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66855a92-a60c-46f4-ac30-a0a269cc67ae tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.088989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66855a92-a60c-46f4-ac30-a0a269cc67ae tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.089453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:19:11.111461 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-66855a92-a60c-46f4-ac30-a0a269cc67ae tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags" status: 200 len: 12 microversion: 2.26 time: 0.050380 Sep 27 10:19:11.112079 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2216/4866] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 10:19:11 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags => generated 12 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:11.122656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:11.123277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:11.139233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.145388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.146195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.147123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.147765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.148190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.180040 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] HTTP exception thrown: Instance 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f could not be found. Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Request method failure captured: Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f HTTP/1.1 Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs2EHDwEKpeRWo0VaGXyOLJhec1aDGB8MwTVs0JDlXy1Ylw59jLSUa8Hn-EzFHcUO80RyRuKYDTlx0rZ_YCveL6clrLOWPc2Py9ffyjc4RKiAhjKPP5J9FWsHQGWjRKl9HiUUUodqb43dy195iYxETZ_Vp50-_11Fu3Ds8hOTepkmzs Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8070262519034021a2b6de1a1e1ec88a Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8070262519034021a2b6de1a1e1ec88a"}]}] Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8070262519034021a2b6de1a1e1ec88a Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerDiskConfigTestJSON-393754532-project Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: dbf876e1c23642578363f9c5995f3d86 Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerDiskConfigTestJSON-393754532-project Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f could not be found. Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f'} Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f could not be found. Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f could not be found. Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:11.182073 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.187285 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.187285 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:11.187285 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:11.187285 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:19:11.187285 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:19:11.187285 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f could not be found. Sep 27 10:19:11.187285 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.194370 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f253b37d-11c2-4b0a-8abe-f726e36d58a9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.947672 Sep 27 10:19:11.196669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Returning 404 to user: Instance 0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:19:11.199101 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d270e6fc-b082-4139-888d-8ac1892e5dab tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f" status: 404 len: 111 microversion: 2.1 time: 0.080437 Sep 27 10:19:11.204515 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2651/4867] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:19:09 2022] POST /compute/v2.1/servers => generated 406 bytes in 1962 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:19:11.207278 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2217/4868] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:11 2022] GET /compute/v2.1/servers/0bf697e2-c1b0-41f9-8711-bf8e6bee7d4f => generated 111 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:19:11.212193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f62936310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:11.220187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:11.220755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:11.226808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.228269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.229888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.230549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:11.238244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.238847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.240839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.241543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.242197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.242347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.243758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.248874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.249431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.252392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.260229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.261555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.289080 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] HTTP exception thrown: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Request method failure captured: Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 HTTP/1.1 Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs2EHDwEKpeRWo0VaGXyOLJhec1aDGB8MwTVs0JDlXy1Ylw59jLSUa8Hn-EzFHcUO80RyRuKYDTlx0rZ_YCveL6clrLOWPc2Py9ffyjc4RKiAhjKPP5J9FWsHQGWjRKl9HiUUUodqb43dy195iYxETZ_Vp50-_11Fu3Ds8hOTepkmzs Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8070262519034021a2b6de1a1e1ec88a Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8070262519034021a2b6de1a1e1ec88a"}]}] Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8070262519034021a2b6de1a1e1ec88a Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerDiskConfigTestJSON-393754532-project Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: dbf876e1c23642578363f9c5995f3d86 Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerDiskConfigTestJSON-393754532-project Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3c45ec10-5528-48be-bc7b-4e628abd6d85'} Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:19:11.290681 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Returning 404 to user: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:19:11.295593 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d82f8173-b502-4240-af28-c450f61bd976 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 404 len: 111 microversion: 2.1 time: 0.083825 Sep 27 10:19:11.308744 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2218/4869] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:19:11 2022] DELETE /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 111 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:19:11.314265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-590cb83d-c798-46e9-a723-0dafa91e28b8 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:11.314899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-590cb83d-c798-46e9-a723-0dafa91e28b8 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:11.333118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.334806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.335633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.373086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:11.425316 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-de099d11-eebb-4c83-93a2-7cd669709540 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags" status: 204 len: 0 microversion: 2.26 time: 0.219383 Sep 27 10:19:11.430668 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2652/4870] 10.222.0.26 () {60 vars in 1295 bytes} [Tue Sep 27 10:19:11 2022] DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags => generated 0 bytes in 226 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:19:11.448603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:11.451014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: 3c45ec10-5528-48be-bc7b-4e628abd6d85] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:11.466902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.467801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.468900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.470480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.478369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.482377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.499308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-590cb83d-c798-46e9-a723-0dafa91e28b8 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:11.513027 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-590cb83d-c798-46e9-a723-0dafa91e28b8 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1249 microversion: 2.1 time: 0.200467 Sep 27 10:19:11.531922 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2219/4871] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:11 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1249 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:11.534535 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] HTTP exception thrown: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. Sep 27 10:19:11.547415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f624421f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Request method failure captured: Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 HTTP/1.1 Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs2EHDwEKpeRWo0VaGXyOLJhec1aDGB8MwTVs0JDlXy1Ylw59jLSUa8Hn-EzFHcUO80RyRuKYDTlx0rZ_YCveL6clrLOWPc2Py9ffyjc4RKiAhjKPP5J9FWsHQGWjRKl9HiUUUodqb43dy195iYxETZ_Vp50-_11Fu3Ds8hOTepkmzs Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 8070262519034021a2b6de1a1e1ec88a Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8070262519034021a2b6de1a1e1ec88a"}]}] Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 8070262519034021a2b6de1a1e1ec88a Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerDiskConfigTestJSON-393754532-project Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: dbf876e1c23642578363f9c5995f3d86 Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerDiskConfigTestJSON-393754532-project Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3c45ec10-5528-48be-bc7b-4e628abd6d85'} Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:11.547926 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.552709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.552709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:11.552709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:11.552709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:19:11.552709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:19:11.552709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. Sep 27 10:19:11.552709 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.555078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Returning 404 to user: Instance 3c45ec10-5528-48be-bc7b-4e628abd6d85 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:19:11.564924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-62f27fde-7154-4ab4-b839-8cce82497caa tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85" status: 404 len: 111 microversion: 2.1 time: 0.129543 Sep 27 10:19:11.567797 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2653/4872] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:11 2022] GET /compute/v2.1/servers/3c45ec10-5528-48be-bc7b-4e628abd6d85 => generated 111 bytes in 135 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:19:11.584515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.585037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.585487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.585994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:11.589277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:11.590946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:11.615371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.615875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.616318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.618579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.619118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.619574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.620315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.620861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.621223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.629088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.629827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.630374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.659705 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] HTTP exception thrown: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Request method failure captured: Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e HTTP/1.1 Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs2EHDwEKpeRWo0VaGXyOLJhec1aDGB8MwTVs0JDlXy1Ylw59jLSUa8Hn-EzFHcUO80RyRuKYDTlx0rZ_YCveL6clrLOWPc2Py9ffyjc4RKiAhjKPP5J9FWsHQGWjRKl9HiUUUodqb43dy195iYxETZ_Vp50-_11Fu3Ds8hOTepkmzs Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 8070262519034021a2b6de1a1e1ec88a Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8070262519034021a2b6de1a1e1ec88a"}]}] Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 8070262519034021a2b6de1a1e1ec88a Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerDiskConfigTestJSON-393754532-project Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: dbf876e1c23642578363f9c5995f3d86 Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerDiskConfigTestJSON-393754532-project Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'e5de27fc-1301-4e5a-ad28-2cd333ab9d6e'} Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:19:11.664988 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:19:11.669986 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. Sep 27 10:19:11.669986 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.669986 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:11.669986 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.669986 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.669986 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:11.669986 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:11.669986 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:19:11.669986 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:19:11.669986 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. Sep 27 10:19:11.669986 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.673931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Returning 404 to user: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:19:11.675375 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-25424bc2-6148-43d5-a9cb-8a4ff94b1f52 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 404 len: 111 microversion: 2.1 time: 0.102894 Sep 27 10:19:11.686519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.687135 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2654/4873] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:19:11 2022] DELETE /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 111 bytes in 115 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:19:11.695966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:11.696605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.697293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.701072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] [instance: e5de27fc-1301-4e5a-ad28-2cd333ab9d6e] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:11.715907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.716212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:RELEASE=3,nova_cell1:SELECT=90,nova_cell1:DELETE=3,nova_cell1:SAVEPOINT=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:11.721810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.722473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.722664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.723162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.723606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.723920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.724586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.725392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.756596 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] HTTP exception thrown: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Request method failure captured: Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e HTTP/1.1 Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs2EHDwEKpeRWo0VaGXyOLJhec1aDGB8MwTVs0JDlXy1Ylw59jLSUa8Hn-EzFHcUO80RyRuKYDTlx0rZ_YCveL6clrLOWPc2Py9ffyjc4RKiAhjKPP5J9FWsHQGWjRKl9HiUUUodqb43dy195iYxETZ_Vp50-_11Fu3Ds8hOTepkmzs Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 8070262519034021a2b6de1a1e1ec88a Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8070262519034021a2b6de1a1e1ec88a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8070262519034021a2b6de1a1e1ec88a"}]}] Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 8070262519034021a2b6de1a1e1ec88a Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-393754532 Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerDiskConfigTestJSON-393754532-project Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: dbf876e1c23642578363f9c5995f3d86 Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerDiskConfigTestJSON-393754532-project Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'e5de27fc-1301-4e5a-ad28-2cd333ab9d6e'} Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:11.763464 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.768678 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:11.768678 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:11.768678 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:11.768678 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:19:11.768678 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:19:11.768678 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. Sep 27 10:19:11.768678 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:19:11.768678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] Returning 404 to user: Instance e5de27fc-1301-4e5a-ad28-2cd333ab9d6e could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:19:11.772553 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-978e12ec-9ac7-4b25-a6a4-469d852dcb50 tempest-ServerDiskConfigTestJSON-393754532 tempest-ServerDiskConfigTestJSON-393754532-project] 10.222.0.26 "GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e" status: 404 len: 111 microversion: 2.1 time: 0.080659 Sep 27 10:19:11.773595 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2655/4874] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:11 2022] GET /compute/v2.1/servers/e5de27fc-1301-4e5a-ad28-2cd333ab9d6e => generated 111 bytes in 83 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:19:11.779703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:11.828991 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f36f4c5d-873c-4e96-a053-64b5ebc10767 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-812783766" status: 201 len: 0 microversion: 2.26 time: 0.295795 Sep 27 10:19:11.835877 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2220/4875] 10.222.0.26 () {62 vars in 1377 bytes} [Tue Sep 27 10:19:11 2022] PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-812783766 => generated 0 bytes in 303 msecs (HTTP/1.1 201) 10 headers in 485 bytes (1 switches on core 0) Sep 27 10:19:11.841434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f63797ee0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:11.863078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.867194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.867900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.868885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:11.883701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.888017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.888746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.889731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.890661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.891298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:11.959775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:11.968222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:11.973332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.001106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.001776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.002642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.056584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:12.101936 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8ad9dd01-720a-4621-b75b-d672a61697ef tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-689361011" status: 201 len: 0 microversion: 2.26 time: 0.264934 Sep 27 10:19:12.104731 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2656/4876] 10.222.0.26 () {62 vars in 1377 bytes} [Tue Sep 27 10:19:11 2022] PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags/tempest-tag-689361011 => generated 0 bytes in 267 msecs (HTTP/1.1 201) 10 headers in 485 bytes (2 switches on core 0) Sep 27 10:19:12.115280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Action: 'update_all', calling method: .version_select at 0x7f7f62442a60>, body: {"tags": ["tempest-tag-1196299893", "tempest-tag-114743565"]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:12.139089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.146960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.147388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.147888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:12.158339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.160242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.160721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.163193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.163631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.164064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.242623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.243490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.244178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.291175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:12.318032 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-753dc5c5-6917-4392-9b39-88e1f37541b6 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags" status: 200 len: 61 microversion: 2.26 time: 0.208004 Sep 27 10:19:12.325697 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2221/4877] 10.222.0.26 () {62 vars in 1312 bytes} [Tue Sep 27 10:19:12 2022] PUT /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags => generated 61 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:12.335596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b6209d02-fe5d-452a-9b21-388e8d79af98 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f62866940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:12.356588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6209d02-fe5d-452a-9b21-388e8d79af98 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.357127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6209d02-fe5d-452a-9b21-388e8d79af98 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.357577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6209d02-fe5d-452a-9b21-388e8d79af98 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.384941 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b6209d02-fe5d-452a-9b21-388e8d79af98 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags" status: 200 len: 61 microversion: 2.26 time: 0.054143 Sep 27 10:19:12.392255 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2657/4878] 10.222.0.26 () {60 vars in 1292 bytes} [Tue Sep 27 10:19:12 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags => generated 61 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:12.397494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '.version_select at 0x7f7f62442e50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:12.409331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.417220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.417520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.418031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:12.442682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.445959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.446445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.447494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.447978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.448415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.490953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.491940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.492682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.551983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2a6b5447-e413-48a7-8fec-0b05ef25eb81 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:12.553520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2a6b5447-e413-48a7-8fec-0b05ef25eb81 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:12.559278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lazy-loading 'flavor' on Instance uuid 176eec03-6637-4f3c-b92b-6f15e07af753 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:19:12.623792 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e8e42e47-59aa-4337-b010-768295b0ca22 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags" status: 204 len: 0 microversion: 2.26 time: 0.230741 Sep 27 10:19:12.629832 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2222/4879] 10.222.0.26 () {60 vars in 1295 bytes} [Tue Sep 27 10:19:12 2022] DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753/tags => generated 0 bytes in 233 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:19:12.636977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd7025fd-c193-4379-933b-4a031c230363 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:12.637497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd7025fd-c193-4379-933b-4a031c230363 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:12.649461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd7025fd-c193-4379-933b-4a031c230363 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.650287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd7025fd-c193-4379-933b-4a031c230363 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.651704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd7025fd-c193-4379-933b-4a031c230363 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.652866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd7025fd-c193-4379-933b-4a031c230363 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.653957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd7025fd-c193-4379-933b-4a031c230363 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.654410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd7025fd-c193-4379-933b-4a031c230363 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.690738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd7025fd-c193-4379-933b-4a031c230363 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:19:12.881457 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd7025fd-c193-4379-933b-4a031c230363 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 204 len: 0 microversion: 2.26 time: 0.249083 Sep 27 10:19:12.882139 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2223/4880] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:19:12 2022] DELETE /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:19:12.912338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:12.912338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:12.931102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.934049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.934780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:12.935559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:12.936089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:12.937150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:13.010360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2a6b5447-e413-48a7-8fec-0b05ef25eb81 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:13.023295 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2a6b5447-e413-48a7-8fec-0b05ef25eb81 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1249 microversion: 2.1 time: 0.479648 Sep 27 10:19:13.024428 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2658/4881] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:12 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1249 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:13.645180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:13.656003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:13.662932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:13.669671 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1551 microversion: 2.26 time: 0.778381 Sep 27 10:19:13.679982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2a6b5447-e413-48a7-8fec-0b05ef25eb81 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:13.681611 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2a6b5447-e413-48a7-8fec-0b05ef25eb81 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute//" status: 200 len: 377 microversion: - time: 0.002131 Sep 27 10:19:13.689549 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2224/4883] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:12 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1551 bytes in 792 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:13.689701 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2659/4883] 10.222.0.26 () {56 vars in 886 bytes} [Tue Sep 27 10:19:13 2022] GET /compute/ => generated 377 bytes in 10 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Sep 27 10:19:13.694707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:13.696212 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2/" status: 200 len: 379 microversion: 2.1 time: 0.004760 Sep 27 10:19:13.696631 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2225/4884] 10.222.0.26 () {54 vars in 1035 bytes} [Tue Sep 27 10:19:13 2022] GET /compute/v2/ => generated 379 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 221 bytes (1 switches on core 0) Sep 27 10:19:13.704938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2a6b5447-e413-48a7-8fec-0b05ef25eb81 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:13.706032 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2a6b5447-e413-48a7-8fec-0b05ef25eb81 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001599 Sep 27 10:19:13.719671 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2660/4885] 10.222.0.26 () {54 vars in 1041 bytes} [Tue Sep 27 10:19:13 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:13.722065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:13.723232 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-97a0f0da-8faf-4a0b-89c2-1ffefba352b4 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute//" status: 200 len: 377 microversion: - time: 0.001541 Sep 27 10:19:13.723757 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2226/4886] 10.222.0.26 () {56 vars in 886 bytes} [Tue Sep 27 10:19:13 2022] GET /compute/ => generated 377 bytes in 2 msecs (HTTP/1.1 200) 3 headers in 91 bytes (1 switches on core 0) Sep 27 10:19:14.048234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8428bf26-3c8a-48f9-abb8-5945a05319b4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:14.049422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8428bf26-3c8a-48f9-abb8-5945a05319b4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:14.061270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8428bf26-3c8a-48f9-abb8-5945a05319b4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:14.065172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8428bf26-3c8a-48f9-abb8-5945a05319b4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:14.070571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8428bf26-3c8a-48f9-abb8-5945a05319b4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:14.071749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8428bf26-3c8a-48f9-abb8-5945a05319b4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:14.073872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8428bf26-3c8a-48f9-abb8-5945a05319b4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:14.080060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8428bf26-3c8a-48f9-abb8-5945a05319b4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:14.097411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=2,nova_cell1:DELETE=1,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=1,nova_cell1:RELEASE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:14.388642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8428bf26-3c8a-48f9-abb8-5945a05319b4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:14.398843 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8428bf26-3c8a-48f9-abb8-5945a05319b4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1307 microversion: 2.1 time: 0.355134 Sep 27 10:19:14.401308 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2661/4887] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:14 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1307 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:14.703640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:14.704403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:14.711722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:14.722897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:14.723464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:14.725146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:14.725714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:14.726355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:15.409788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:15.432377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c48e5f6-468b-4e61-87ab-7b074ec14c73 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:15.433882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:15.434196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2c48e5f6-468b-4e61-87ab-7b074ec14c73 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:15.435464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:15.436755 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0e04d428-3b7b-4213-b53d-a57534f9948d tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1551 microversion: 2.26 time: 0.737646 Sep 27 10:19:15.444678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c48e5f6-468b-4e61-87ab-7b074ec14c73 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:15.445811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c48e5f6-468b-4e61-87ab-7b074ec14c73 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:15.446417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c48e5f6-468b-4e61-87ab-7b074ec14c73 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:15.447298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c48e5f6-468b-4e61-87ab-7b074ec14c73 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:15.447916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c48e5f6-468b-4e61-87ab-7b074ec14c73 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:15.448501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c48e5f6-468b-4e61-87ab-7b074ec14c73 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:15.448966 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2227/4888] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:14 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1551 bytes in 750 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:15.921060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2c48e5f6-468b-4e61-87ab-7b074ec14c73 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:15.931275 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c48e5f6-468b-4e61-87ab-7b074ec14c73 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1307 microversion: 2.1 time: 0.509017 Sep 27 10:19:15.932397 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2662/4889] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:15 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1307 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:16.469019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:16.470458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:16.527492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:16.528114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:16.528578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:16.529559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:16.530087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:16.530524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:16.540883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ec6ec838-3885-404e-b250-f717e2bf8280 req-291fd8f4-cea3-493f-b7ce-e6847ddd3d4d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "176eec03-6637-4f3c-b92b-6f15e07af753", "name": "network-vif-deleted", "tag": "22fb14d0-2bc4-4d8f-b5d6-c9ef641e6a5d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:16.585122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=5,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:16.587307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ec6ec838-3885-404e-b250-f717e2bf8280 req-291fd8f4-cea3-493f-b7ce-e6847ddd3d4d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:16.587794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ec6ec838-3885-404e-b250-f717e2bf8280 req-291fd8f4-cea3-493f-b7ce-e6847ddd3d4d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:16.588254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ec6ec838-3885-404e-b250-f717e2bf8280 req-291fd8f4-cea3-493f-b7ce-e6847ddd3d4d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:16.653676 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ec6ec838-3885-404e-b250-f717e2bf8280 req-291fd8f4-cea3-493f-b7ce-e6847ddd3d4d service nova] Creating event network-vif-deleted:22fb14d0-2bc4-4d8f-b5d6-c9ef641e6a5d for instance 176eec03-6637-4f3c-b92b-6f15e07af753 on nch2-master-1 Sep 27 10:19:16.669865 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ec6ec838-3885-404e-b250-f717e2bf8280 req-291fd8f4-cea3-493f-b7ce-e6847ddd3d4d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.132032 Sep 27 10:19:16.673418 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2663/4890] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:19:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:16.963072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:16.968625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f9ea1532-32b1-4632-8080-82260d562c13 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:16.969866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f9ea1532-32b1-4632-8080-82260d562c13 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:16.989287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9ea1532-32b1-4632-8080-82260d562c13 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:16.990350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9ea1532-32b1-4632-8080-82260d562c13 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:16.995130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:16.996104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9ea1532-32b1-4632-8080-82260d562c13 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:16.996581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '6efd3d39610444b99c7bb245337c2ac3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd00595fe8dc4d9594513f900172d7ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:16.997385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9ea1532-32b1-4632-8080-82260d562c13 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:16.997821 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4d5e1cc9-2578-4d80-9edb-7df5d12f2237 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 200 len: 1347 microversion: 2.26 time: 0.535262 Sep 27 10:19:16.998318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9ea1532-32b1-4632-8080-82260d562c13 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:16.998970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9ea1532-32b1-4632-8080-82260d562c13 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:17.005590 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2228/4891] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:16 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 1347 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:17.471325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f9ea1532-32b1-4632-8080-82260d562c13 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:17.484464 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f9ea1532-32b1-4632-8080-82260d562c13 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1375 microversion: 2.1 time: 0.526463 Sep 27 10:19:17.486543 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2664/4892] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:16 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1375 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:18.028433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:18.029068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] [instance: 176eec03-6637-4f3c-b92b-6f15e07af753] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:18.036313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:18.037220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:18.037835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:18.038699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:18.039319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:18.039736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:18.056904 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] HTTP exception thrown: Instance 176eec03-6637-4f3c-b92b-6f15e07af753 could not be found. Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Request method failure captured: Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 HTTP/1.1 Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs369Zgj9Z2hVy85GhCZNhgRVKH-NJ02gEs7e6ty-iKbebbQG4Ct1YnrU8IhHwEhh96-mUmfvPWdhhs5GohPxRxBRnikgH4veQdLZVZdM9DYDd7sca7Yv6PPA9me5GU1kwsDH_CWZmDB4G_oryHE2eOe-ey07dx7bjgt6HOxC3t-eMc Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.26 Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: cd00595fe8dc4d9594513f900172d7ff Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerTagsTestJSON-1900668796 Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/cd00595fe8dc4d9594513f900172d7ff"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/cd00595fe8dc4d9594513f900172d7ff"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/cd00595fe8dc4d9594513f900172d7ff"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_cd00595fe8dc4d9594513f900172d7ff"}]}] Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerTagsTestJSON-1900668796 Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: cd00595fe8dc4d9594513f900172d7ff Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerTagsTestJSON-1900668796 Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerTagsTestJSON-1900668796-project Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 6efd3d39610444b99c7bb245337c2ac3 Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerTagsTestJSON-1900668796-project Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 176eec03-6637-4f3c-b92b-6f15e07af753 could not be found. Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '176eec03-6637-4f3c-b92b-6f15e07af753'} Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 176eec03-6637-4f3c-b92b-6f15e07af753 could not be found. Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 176eec03-6637-4f3c-b92b-6f15e07af753 could not be found. Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:19:18.058906 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:18.063762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:19:18.063762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:19:18.063762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:19:18.063762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:19:18.063762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:19:18.063762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 176eec03-6637-4f3c-b92b-6f15e07af753 could not be found. Sep 27 10:19:18.063762 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:19:18.065383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] Returning 404 to user: Instance 176eec03-6637-4f3c-b92b-6f15e07af753 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:19:18.066438 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b41ac6dd-dfd1-49ab-bb1f-b98697dc80b9 tempest-ServerTagsTestJSON-1900668796 tempest-ServerTagsTestJSON-1900668796-project] 10.222.0.26 "GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753" status: 404 len: 111 microversion: 2.26 time: 0.041970 Sep 27 10:19:18.067811 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2229/4893] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:18 2022] GET /compute/v2.1/servers/176eec03-6637-4f3c-b92b-6f15e07af753 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:19:18.508598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5e36129-97eb-459d-aea0-b01252b8f297 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:18.514036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a5e36129-97eb-459d-aea0-b01252b8f297 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:18.529602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e36129-97eb-459d-aea0-b01252b8f297 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:18.537240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e36129-97eb-459d-aea0-b01252b8f297 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:18.540280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e36129-97eb-459d-aea0-b01252b8f297 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:18.541103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e36129-97eb-459d-aea0-b01252b8f297 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:18.541595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e36129-97eb-459d-aea0-b01252b8f297 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:18.542038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e36129-97eb-459d-aea0-b01252b8f297 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:19.553467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a5e36129-97eb-459d-aea0-b01252b8f297 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:19.567297 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a5e36129-97eb-459d-aea0-b01252b8f297 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1375 microversion: 2.1 time: 1.062879 Sep 27 10:19:19.582135 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2665/4894] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:18 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1375 bytes in 1078 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:20.533319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5fd6916c-f05c-466b-91f4-b20e6070b930 req-ed394af8-252f-48af-8a1e-356582d13624 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0998602b-d17f-47e0-8af7-bddf2587d9ad", "tag": "c6120fe6-85b3-422f-9e82-1e6a0fa5faa0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:20.549373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5fd6916c-f05c-466b-91f4-b20e6070b930 req-ed394af8-252f-48af-8a1e-356582d13624 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:20.550268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5fd6916c-f05c-466b-91f4-b20e6070b930 req-ed394af8-252f-48af-8a1e-356582d13624 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:20.551037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5fd6916c-f05c-466b-91f4-b20e6070b930 req-ed394af8-252f-48af-8a1e-356582d13624 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:20.566451 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5fd6916c-f05c-466b-91f4-b20e6070b930 req-ed394af8-252f-48af-8a1e-356582d13624 service nova] Creating event network-changed:c6120fe6-85b3-422f-9e82-1e6a0fa5faa0 for instance 0998602b-d17f-47e0-8af7-bddf2587d9ad on nch1-master-1 Sep 27 10:19:20.571878 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5fd6916c-f05c-466b-91f4-b20e6070b930 req-ed394af8-252f-48af-8a1e-356582d13624 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044528 Sep 27 10:19:20.573160 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2230/4895] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:19:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:20.596110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b42db3e-ccd3-4346-8582-3e57ca2a2700 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:20.597351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3b42db3e-ccd3-4346-8582-3e57ca2a2700 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:20.610046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b42db3e-ccd3-4346-8582-3e57ca2a2700 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:20.611694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b42db3e-ccd3-4346-8582-3e57ca2a2700 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:20.612386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b42db3e-ccd3-4346-8582-3e57ca2a2700 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:20.613475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b42db3e-ccd3-4346-8582-3e57ca2a2700 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:20.614154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b42db3e-ccd3-4346-8582-3e57ca2a2700 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:20.615032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b42db3e-ccd3-4346-8582-3e57ca2a2700 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:21.046543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:21.164984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3b42db3e-ccd3-4346-8582-3e57ca2a2700 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:21.176476 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3b42db3e-ccd3-4346-8582-3e57ca2a2700 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1375 microversion: 2.1 time: 0.585537 Sep 27 10:19:21.177946 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2666/4896] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:20 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1375 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:22.197278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:22.203623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b8a1a0c9-95b7-417f-b371-932c47d7043d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:22.204585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b8a1a0c9-95b7-417f-b371-932c47d7043d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:22.220028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8a1a0c9-95b7-417f-b371-932c47d7043d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:22.221767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8a1a0c9-95b7-417f-b371-932c47d7043d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:22.228185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8a1a0c9-95b7-417f-b371-932c47d7043d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:22.230127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8a1a0c9-95b7-417f-b371-932c47d7043d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:22.230968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8a1a0c9-95b7-417f-b371-932c47d7043d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:22.232963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8a1a0c9-95b7-417f-b371-932c47d7043d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:22.864485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b8a1a0c9-95b7-417f-b371-932c47d7043d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:22.880647 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b8a1a0c9-95b7-417f-b371-932c47d7043d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1375 microversion: 2.1 time: 0.684062 Sep 27 10:19:22.922707 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2231/4897] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:22 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1375 bytes in 727 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:23.907778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4eb0136e-5303-480a-a0aa-27d37b74802a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:23.922887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4eb0136e-5303-480a-a0aa-27d37b74802a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:23.938183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=12 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:23.956550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4eb0136e-5303-480a-a0aa-27d37b74802a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:23.961871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4eb0136e-5303-480a-a0aa-27d37b74802a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:23.962521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4eb0136e-5303-480a-a0aa-27d37b74802a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:23.965287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4eb0136e-5303-480a-a0aa-27d37b74802a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:23.968650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4eb0136e-5303-480a-a0aa-27d37b74802a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:23.969462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4eb0136e-5303-480a-a0aa-27d37b74802a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:24.809636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4eb0136e-5303-480a-a0aa-27d37b74802a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:24.820708 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4eb0136e-5303-480a-a0aa-27d37b74802a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1375 microversion: 2.1 time: 0.916752 Sep 27 10:19:24.822304 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2667/4898] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:23 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1375 bytes in 919 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:25.856337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ea81795-92d6-4214-94a8-a922ffa923ad tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:25.857215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ea81795-92d6-4214-94a8-a922ffa923ad tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:25.871606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea81795-92d6-4214-94a8-a922ffa923ad tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:25.876026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea81795-92d6-4214-94a8-a922ffa923ad tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:25.879133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea81795-92d6-4214-94a8-a922ffa923ad tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:25.884201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea81795-92d6-4214-94a8-a922ffa923ad tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:25.884981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea81795-92d6-4214-94a8-a922ffa923ad tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:25.889330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea81795-92d6-4214-94a8-a922ffa923ad tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:26.878717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2ea81795-92d6-4214-94a8-a922ffa923ad tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:26.902724 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ea81795-92d6-4214-94a8-a922ffa923ad tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1375 microversion: 2.1 time: 1.050054 Sep 27 10:19:26.911459 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2232/4899] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:25 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1375 bytes in 1060 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:27.289284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d33e4167-75f3-4863-8d3f-f28449d4d3c1 req-fc0458b4-d479-4c67-9d50-aefdde569381 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0998602b-d17f-47e0-8af7-bddf2587d9ad", "name": "network-vif-plugged", "status": "completed", "tag": "c6120fe6-85b3-422f-9e82-1e6a0fa5faa0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:27.321890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d33e4167-75f3-4863-8d3f-f28449d4d3c1 req-fc0458b4-d479-4c67-9d50-aefdde569381 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:27.322382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d33e4167-75f3-4863-8d3f-f28449d4d3c1 req-fc0458b4-d479-4c67-9d50-aefdde569381 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:27.322820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d33e4167-75f3-4863-8d3f-f28449d4d3c1 req-fc0458b4-d479-4c67-9d50-aefdde569381 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:27.343586 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d33e4167-75f3-4863-8d3f-f28449d4d3c1 req-fc0458b4-d479-4c67-9d50-aefdde569381 service nova] Creating event network-vif-plugged:c6120fe6-85b3-422f-9e82-1e6a0fa5faa0 for instance 0998602b-d17f-47e0-8af7-bddf2587d9ad on nch1-master-1 Sep 27 10:19:27.359538 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d33e4167-75f3-4863-8d3f-f28449d4d3c1 req-fc0458b4-d479-4c67-9d50-aefdde569381 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074052 Sep 27 10:19:27.360457 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2668/4900] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:19:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:27.925404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a22e42ba-301e-47cc-8200-6f395433ff41 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:27.926406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a22e42ba-301e-47cc-8200-6f395433ff41 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:27.935818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a22e42ba-301e-47cc-8200-6f395433ff41 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:27.939960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a22e42ba-301e-47cc-8200-6f395433ff41 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:27.940623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a22e42ba-301e-47cc-8200-6f395433ff41 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:27.941593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a22e42ba-301e-47cc-8200-6f395433ff41 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:27.943708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a22e42ba-301e-47cc-8200-6f395433ff41 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:27.945238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a22e42ba-301e-47cc-8200-6f395433ff41 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:28.689498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a22e42ba-301e-47cc-8200-6f395433ff41 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:28.697582 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a22e42ba-301e-47cc-8200-6f395433ff41 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1375 microversion: 2.1 time: 0.776206 Sep 27 10:19:28.707330 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2233/4901] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:27 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1375 bytes in 786 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:29.723865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-275b802f-c628-4908-b9fd-5219b34414c9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:29.725435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-275b802f-c628-4908-b9fd-5219b34414c9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:29.752456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-275b802f-c628-4908-b9fd-5219b34414c9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:29.753350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-275b802f-c628-4908-b9fd-5219b34414c9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:29.754042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-275b802f-c628-4908-b9fd-5219b34414c9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:29.755715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-275b802f-c628-4908-b9fd-5219b34414c9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:29.762437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-275b802f-c628-4908-b9fd-5219b34414c9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:29.763142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-275b802f-c628-4908-b9fd-5219b34414c9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:30.210575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-275b802f-c628-4908-b9fd-5219b34414c9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:30.222085 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-275b802f-c628-4908-b9fd-5219b34414c9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1548 microversion: 2.1 time: 0.503801 Sep 27 10:19:30.226761 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2669/4902] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:29 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1548 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:30.251829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d984e6b8-e4e9-46e2-914c-f91c1646eac3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:30.253189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d984e6b8-e4e9-46e2-914c-f91c1646eac3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:30.265361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d984e6b8-e4e9-46e2-914c-f91c1646eac3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:30.267678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d984e6b8-e4e9-46e2-914c-f91c1646eac3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:30.268373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d984e6b8-e4e9-46e2-914c-f91c1646eac3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:30.269406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d984e6b8-e4e9-46e2-914c-f91c1646eac3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:30.271618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d984e6b8-e4e9-46e2-914c-f91c1646eac3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:30.272314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d984e6b8-e4e9-46e2-914c-f91c1646eac3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:30.729201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d984e6b8-e4e9-46e2-914c-f91c1646eac3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:30.745534 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d984e6b8-e4e9-46e2-914c-f91c1646eac3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1548 microversion: 2.1 time: 0.498229 Sep 27 10:19:30.749053 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2234/4903] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:30 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1548 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:36.388717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-623b6a4a-9d2f-4509-9904-d327b242a3cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Action: 'create', calling method: .version_select at 0x7f7f638154c0>, body: {"keypair": {"name": "tempest-keypair-1644362632", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB+G04lxsNV5EZefveBO8Bn4GXE33vFQVhmQ6dSoH8bHDGjvz4powAjQqG3CPqbYQmIjkFbHi4gkMo8gIWFP7XHX0PT/rut+UkfHp1kL5LLzeU46Gk8KCjCH+K+1P4RfaQ=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:36.472551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-623b6a4a-9d2f-4509-9904-d327b242a3cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Getting quotas for project b590d2de576e41aab42f79e4daef9c63. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:19:36.497623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-623b6a4a-9d2f-4509-9904-d327b242a3cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Getting quotas for user 74878b715fc94b23bb139cb30d8d20c0 and project b590d2de576e41aab42f79e4daef9c63. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:19:36.576614 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-623b6a4a-9d2f-4509-9904-d327b242a3cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.466660 Sep 27 10:19:36.577602 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2670/4904] 10.222.0.26 () {62 vars in 1200 bytes} [Tue Sep 27 10:19:36 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 468 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 27 10:19:37.739158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerShowV257Test-server-560656323", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": "none"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:37.858612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:19:37.859131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:19:37.859774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:19:37.860266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:19:37.861388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:19:37.877163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:37.878492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:37.879133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:37.891819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:37.892854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:37.893838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:37.968613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Getting quotas for project a82b2468ace34a009ca80c159ed44bb8. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:19:37.994598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Getting quotas for user e46a8a1c539e4781afda6aadd5373c8e and project a82b2468ace34a009ca80c159ed44bb8. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:19:38.021570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:19:38.024998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:19:38.158870 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ffb607dc-ae17-40cd-bb15-585e9863cc9c tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.57 time: 0.717462 Sep 27 10:19:38.164736 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2235/4905] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:19:37 2022] POST /compute/v2.1/servers => generated 380 bytes in 722 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:19:38.175738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ce1be384-41e5-4b48-bc3c-8fd7e44ea3fc tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:38.176621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ce1be384-41e5-4b48-bc3c-8fd7e44ea3fc tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:38.666970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ce1be384-41e5-4b48-bc3c-8fd7e44ea3fc tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:38.679192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ce1be384-41e5-4b48-bc3c-8fd7e44ea3fc tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:38.686723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ce1be384-41e5-4b48-bc3c-8fd7e44ea3fc tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:38.691734 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ce1be384-41e5-4b48-bc3c-8fd7e44ea3fc tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1277 microversion: 2.57 time: 0.521506 Sep 27 10:19:38.697746 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2671/4906] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:38 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1277 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:19:39.724730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:39.726456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:39.736930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:39.737822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:39.738825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:39.741007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:39.741808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:39.742546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:39.966323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:39.968594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=92,nova_cell1:SAVEPOINT=1,nova_cell1:INSERT=3,nova_cell1:RELEASE=1,nova_cell1:DELETE=2,nova_cell1:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:39.985731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:39.987612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:39.989217 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-15ba134e-acd3-402d-a615-e02c6bb1472b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1329 microversion: 2.57 time: 0.271559 Sep 27 10:19:39.990321 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2236/4907] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:39 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1329 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:41.019773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:41.020395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:41.033728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=44 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:41.035620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:41.036398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:41.043236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:41.045057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:41.045740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:41.046392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:41.239015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:41.258195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:41.260433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:41.261690 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c63625d-bc80-40f7-a3b3-05d7d4bb1dca tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1335 microversion: 2.57 time: 0.246122 Sep 27 10:19:41.272684 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2672/4908] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:41 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1335 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:41.728810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-587f7794-2e98-4116-924d-555e6f4a1d4a tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Action: 'create', calling method: .version_select at 0x7f7f6243b430>, body: {"keypair": {"name": "tempest-TestSecurityGroupsBasicOps-1627558846", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEBf6zDPBMOvar2D5+3yEiTx5u4AyUav0Vl9kwQcHShCFpuZQj0D+82RxXNnpmTfj6/o7a3mSMcnqp4Rg3YXa+7IelAldlrD5WNtb+7A839VVb6eVr8MznejsfmQQfJUyw=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:41.735310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=89,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:41.784831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-587f7794-2e98-4116-924d-555e6f4a1d4a tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Getting quotas for project 932da48df2c44fc6a7dbd6c8daf9fc0d. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:19:41.798536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-587f7794-2e98-4116-924d-555e6f4a1d4a tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Getting quotas for user 64c9f44e57164b858421b55114acf4cf and project 932da48df2c44fc6a7dbd6c8daf9fc0d. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:19:41.867876 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-587f7794-2e98-4116-924d-555e6f4a1d4a tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 405 microversion: 2.1 time: 0.520316 Sep 27 10:19:41.868861 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2237/4909] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:19:41 2022] POST /compute/v2.1/os-keypairs => generated 405 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:42.291682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:42.292431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:42.302923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:42.306864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:42.309700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:42.310650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:42.311209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:42.313890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:43.122137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:43.139253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:43.141495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:43.143637 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4576809-a63e-42f6-ba44-d7aac0138141 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1335 microversion: 2.57 time: 0.858681 Sep 27 10:19:43.145204 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2673/4910] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:42 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1335 bytes in 861 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:44.176045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:44.176612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:44.190792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:44.192735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:44.193574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:44.196339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:44.197099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:44.197753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:44.264936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-878812840", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a24d1d04-2dec-42a9-9c22-fd56aad28b57"}], "security_groups": [{"name": "tempest-securitygroup--1373773321"}], "key_name": "tempest-keypair-1644362632", "user_data": "}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:44.504037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:44.522209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:44.525136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:44.529715 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a57b8e14-7203-48ce-a055-f59243d4b1d8 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1335 microversion: 2.57 time: 0.358085 Sep 27 10:19:44.531256 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2238/4911] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:44 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1335 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:44.662914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] validate_networks() for [('a24d1d04-2dec-42a9-9c22-fd56aad28b57', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:19:45.569562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:45.571121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:45.585731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:45.588286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:45.590273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:45.592450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:45.593369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:45.595792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:46.114379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:46.134757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:46.145080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:46.147156 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-45726933-2101-434e-bbb6-c78da033737d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1356 microversion: 2.57 time: 0.586529 Sep 27 10:19:46.148840 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2239/4912] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:45 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1356 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:46.183684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Action: 'action', calling method: >, body: {"rebuild": {"user_data": "ZWNobyAiaGVsbG8gd29ybGQi", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:46.187212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:46.196776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:46.199595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:46.201409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:46.206935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:46.208757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:46.211359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:46.404265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:19:46.407455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:19:46.409219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:19:46.409944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:19:46.410565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:19:46.565109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:46.577369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:46.579155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:46.579825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:46.595939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:46.596645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:46.597279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:46.675228 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e9d86d43-a6f4-473f-87cf-65048c6eff21 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "POST /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818/action" status: 202 len: 1137 microversion: 2.57 time: 0.501380 Sep 27 10:19:46.676520 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2240/4913] 10.222.0.26 () {62 vars in 1320 bytes} [Tue Sep 27 10:19:46 2022] POST /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818/action => generated 1137 bytes in 504 msecs (HTTP/1.1 202) 10 headers in 454 bytes (1 switches on core 0) Sep 27 10:19:46.698627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:46.700104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:46.709364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:46.710147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:46.710758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:46.711678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:46.712305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:46.712972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:46.747958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:19:46.748463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:19:46.749095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:19:46.749610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:19:46.750164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:19:46.763715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:46.764612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:46.765478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:46.766565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:46.767359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:46.768198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:46.797069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Getting quotas for project b590d2de576e41aab42f79e4daef9c63. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:19:46.806436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Getting quotas for user 74878b715fc94b23bb139cb30d8d20c0 and project b590d2de576e41aab42f79e4daef9c63. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:19:46.831698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:19:46.834729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:19:46.908222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:46.918319 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-38775c33-1024-4acb-832b-29c9dfe0729b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 2.659674 Sep 27 10:19:46.920301 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2674/4914] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:19:44 2022] POST /compute/v2.1/servers => generated 402 bytes in 2662 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:19:46.928943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:46.933590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:46.934815 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd83cdb9-1c52-43a8-b5b8-47d6e07a0119 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.240461 Sep 27 10:19:46.935810 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2241/4915] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:46 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:19:46.957280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9637ce57-e86e-4ca7-bcc3-6fde5eee610f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:46.957988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9637ce57-e86e-4ca7-bcc3-6fde5eee610f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:47.151921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9637ce57-e86e-4ca7-bcc3-6fde5eee610f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:47.179314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9637ce57-e86e-4ca7-bcc3-6fde5eee610f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:47.180813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9637ce57-e86e-4ca7-bcc3-6fde5eee610f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:47.182657 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9637ce57-e86e-4ca7-bcc3-6fde5eee610f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1281 microversion: 2.20 time: 0.251415 Sep 27 10:19:47.190118 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2675/4916] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:46 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1281 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:47.965951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:47.969740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:47.986553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:47.995252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:47.997658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:47.998128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:48.000719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:48.005007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:48.005782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:48.211604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:48.213360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:48.218581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=52,nova_api:INSERT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:48.227898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:48.228644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:48.230318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:48.231565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:48.232514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:48.234063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:48.286972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:48.300451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:48.302233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:48.303689 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5cbe1716-50f3-4666-a7ab-443cdc4de8ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.343478 Sep 27 10:19:48.307222 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2242/4917] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:47 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:48.474486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:48.484854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:48.487376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:48.488837 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-34d4c621-5f42-493b-8a38-4494d14f3a50 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1277 microversion: 2.20 time: 0.281681 Sep 27 10:19:48.491451 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2676/4918] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:48 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1277 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:19:49.336059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:49.339250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:49.358812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:49.359720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:49.360611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:49.362290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:49.363029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:49.363735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:49.491963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:49.506318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:49.507962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:49.509229 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4b0fe8df-23a8-4c50-8110-756aabac6863 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.177956 Sep 27 10:19:49.509804 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2243/4919] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:49 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:49.519241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:49.520590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:49.535482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:49.537370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:49.538026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:49.541895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:49.553868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:49.555692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:49.803241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:49.834440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:49.838859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:49.841967 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b001f5d4-f62b-4552-bf92-71089f411086 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1339 microversion: 2.20 time: 0.326688 Sep 27 10:19:49.844451 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2677/4920] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:49 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1339 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:50.432370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-InstanceActionsTestJSON-server-1729595182", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "8bef6715-5f71-415b-b67b-425f93f0b889"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:50.539780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] validate_networks() for [('8bef6715-5f71-415b-b67b-425f93f0b889', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:19:50.558573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:50.560949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:50.581904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:50.583068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:50.583990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:50.609364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:50.610350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.025s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:50.612993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:50.898592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:50.918950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:50.944158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:50.949423 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3b277303-7fe7-4d73-8468-9e81657010b3 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.394962 Sep 27 10:19:50.950934 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2678/4921] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:50 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:50.969580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:50.972310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:51.012680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:51.015323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:51.016212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:51.020616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:51.025113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:51.026410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:51.676556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:51.689537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:51.695921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:51.701235 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4c61c484-1b0d-400d-9fc6-5016efb0a8ae tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1407 microversion: 2.20 time: 0.747264 Sep 27 10:19:51.702014 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2679/4922] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:50 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1407 bytes in 750 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:51.824592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=10 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:51.993508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:51.995280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:52.013198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:52.016613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:52.019824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:52.020864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:52.022323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:52.025231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:52.122576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:19:52.123429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:19:52.124092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:19:52.124669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:19:52.125494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:19:52.137795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:52.138310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:52.139057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:52.140058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:52.140766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:52.141471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:52.199993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Getting quotas for project cb035a91b1f341ce8d8b71d39ff10a0b. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:19:52.208682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Getting quotas for user 377e47048a63456793dfc9c125f5a209 and project cb035a91b1f341ce8d8b71d39ff10a0b. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:19:52.234481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:19:52.237818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:19:52.332601 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9ac6e38-a14d-453f-8fcb-d631a223979e tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.129127 Sep 27 10:19:52.333857 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2244/4923] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:19:50 2022] POST /compute/v2.1/servers => generated 380 bytes in 2131 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:19:52.352214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e44f187e-0f98-4faf-bcc8-38454fdf15be tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:52.352890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e44f187e-0f98-4faf-bcc8-38454fdf15be tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:52.516365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e44f187e-0f98-4faf-bcc8-38454fdf15be tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:52.528738 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e44f187e-0f98-4faf-bcc8-38454fdf15be tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1217 microversion: 2.1 time: 0.187442 Sep 27 10:19:52.545449 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2245/4924] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:52 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1217 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:52.707300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:52.716501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:52.726407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:52.731336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:52.731964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:52.734563 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d400e92-1f38-44ba-bf2f-f446b60b9d84 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.755785 Sep 27 10:19:52.740612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:52.742804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:52.743419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:52.744310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:52.749237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:52.755142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:52.759825 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2680/4925] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:51 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 782 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:53.552240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cc7fe71c-78d8-48a1-ac58-4fdb100ea4d6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:53.553742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cc7fe71c-78d8-48a1-ac58-4fdb100ea4d6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:53.562196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc7fe71c-78d8-48a1-ac58-4fdb100ea4d6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:53.564718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc7fe71c-78d8-48a1-ac58-4fdb100ea4d6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:53.565657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc7fe71c-78d8-48a1-ac58-4fdb100ea4d6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:53.567287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc7fe71c-78d8-48a1-ac58-4fdb100ea4d6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:53.585232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc7fe71c-78d8-48a1-ac58-4fdb100ea4d6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:53.586402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc7fe71c-78d8-48a1-ac58-4fdb100ea4d6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:53.935217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cc7fe71c-78d8-48a1-ac58-4fdb100ea4d6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:53.969476 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cc7fe71c-78d8-48a1-ac58-4fdb100ea4d6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1221 microversion: 2.1 time: 0.421751 Sep 27 10:19:53.994659 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2681/4926] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:53 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1221 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:19:54.013395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:54.020789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:54.064526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:54.065225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:54.065685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:54.066430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:54.066899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:54.068069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:54.486955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:54.502192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:54.510870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:54.512173 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6d5678c0-a089-4a0b-afb6-014491b21400 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1407 microversion: 2.20 time: 1.785662 Sep 27 10:19:54.512782 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2246/4927] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:52 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1407 bytes in 1787 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:19:54.528974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-6cc4ec88-2e19-4f68-a4d7-9cc941cb7f74 req-ba325bdc-4f82-4208-9ea8-50d0fe6e3cc4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "691b2be7-d5b8-409c-ba22-5ab8da50b80d", "tag": "11b232b4-3236-4917-9ebb-02d7dbd3132e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:19:54.547455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6cc4ec88-2e19-4f68-a4d7-9cc941cb7f74 req-ba325bdc-4f82-4208-9ea8-50d0fe6e3cc4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:54.547946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6cc4ec88-2e19-4f68-a4d7-9cc941cb7f74 req-ba325bdc-4f82-4208-9ea8-50d0fe6e3cc4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:54.548380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6cc4ec88-2e19-4f68-a4d7-9cc941cb7f74 req-ba325bdc-4f82-4208-9ea8-50d0fe6e3cc4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:54.570084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:54.609742 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-6cc4ec88-2e19-4f68-a4d7-9cc941cb7f74 req-ba325bdc-4f82-4208-9ea8-50d0fe6e3cc4 service nova] Creating event network-changed:11b232b4-3236-4917-9ebb-02d7dbd3132e for instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d on nch1-master-1 Sep 27 10:19:54.611825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:54.614298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:54.618299 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bb880218-bc39-4721-be02-a515e28b419e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.622367 Sep 27 10:19:54.627740 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-6cc4ec88-2e19-4f68-a4d7-9cc941cb7f74 req-ba325bdc-4f82-4208-9ea8-50d0fe6e3cc4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.111306 Sep 27 10:19:54.628945 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2247/4928] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:19:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:19:54.637045 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2682/4929] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:53 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:55.007260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-16498e79-a2b6-4b6b-b3e6-ae8cb68886e2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:55.008360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-16498e79-a2b6-4b6b-b3e6-ae8cb68886e2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:55.026587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16498e79-a2b6-4b6b-b3e6-ae8cb68886e2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:55.029052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16498e79-a2b6-4b6b-b3e6-ae8cb68886e2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:55.029507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16498e79-a2b6-4b6b-b3e6-ae8cb68886e2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:55.030254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16498e79-a2b6-4b6b-b3e6-ae8cb68886e2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:55.034115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16498e79-a2b6-4b6b-b3e6-ae8cb68886e2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:55.034575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16498e79-a2b6-4b6b-b3e6-ae8cb68886e2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:55.468688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-16498e79-a2b6-4b6b-b3e6-ae8cb68886e2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:55.477660 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-16498e79-a2b6-4b6b-b3e6-ae8cb68886e2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1287 microversion: 2.1 time: 0.481241 Sep 27 10:19:55.481395 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2248/4930] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:54 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1287 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:55.541805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:55.542430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:55.555244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:55.555910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:55.557569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:55.559632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:55.561034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:55.563232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:55.652472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:55.653260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:55.659734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:55.660327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:55.660686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:55.673280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:55.680867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:55.685276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:56.163354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:56.175445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:56.181186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:56.182521 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-073e5bd3-3e4a-4f59-858c-25e56577318b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.534479 Sep 27 10:19:56.205386 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2249/4931] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:55 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:19:56.226449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:56.235213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:56.236709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:56.237883 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5941dd37-ca0c-4082-bd7e-3a73b93890e8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1407 microversion: 2.20 time: 0.701282 Sep 27 10:19:56.245068 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2683/4932] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:55 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1407 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:56.512463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bc7027f1-4d55-4a25-a69e-eb25b1fa34ed tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:56.513140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bc7027f1-4d55-4a25-a69e-eb25b1fa34ed tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:56.533334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc7027f1-4d55-4a25-a69e-eb25b1fa34ed tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:56.534484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc7027f1-4d55-4a25-a69e-eb25b1fa34ed tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:56.535038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc7027f1-4d55-4a25-a69e-eb25b1fa34ed tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:56.544632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc7027f1-4d55-4a25-a69e-eb25b1fa34ed tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:56.545571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc7027f1-4d55-4a25-a69e-eb25b1fa34ed tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:56.546362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc7027f1-4d55-4a25-a69e-eb25b1fa34ed tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:56.901996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bc7027f1-4d55-4a25-a69e-eb25b1fa34ed tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:56.925938 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bc7027f1-4d55-4a25-a69e-eb25b1fa34ed tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1275 microversion: 2.1 time: 0.418449 Sep 27 10:19:56.928153 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2250/4933] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:56 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1275 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:19:57.213190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:57.214840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:57.218981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:57.219751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=14 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:19:57.223487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:57.224401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:57.225213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:57.226166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:57.226889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:57.227577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:57.263720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:57.264116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:57.272434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:57.273184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:57.273865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:57.274793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:57.277495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:57.278222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:57.416587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:57.441981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:57.444530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:57.450416 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-03bfab18-0e37-4ab2-8d38-ac1227a9f74f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.241313 Sep 27 10:19:57.451687 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2684/4934] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:57 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:57.949396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bd666bbf-164a-4298-95ae-247023fb8dec tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:57.950025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bd666bbf-164a-4298-95ae-247023fb8dec tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:57.959271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd666bbf-164a-4298-95ae-247023fb8dec tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:57.961034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd666bbf-164a-4298-95ae-247023fb8dec tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:57.961485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd666bbf-164a-4298-95ae-247023fb8dec tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:57.962756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd666bbf-164a-4298-95ae-247023fb8dec tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:57.973503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd666bbf-164a-4298-95ae-247023fb8dec tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:57.974358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd666bbf-164a-4298-95ae-247023fb8dec tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:58.106692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bd666bbf-164a-4298-95ae-247023fb8dec tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:58.115900 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bd666bbf-164a-4298-95ae-247023fb8dec tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1275 microversion: 2.1 time: 0.170361 Sep 27 10:19:58.116512 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2685/4935] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:57 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1275 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:19:58.307438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:58.320502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:58.327210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:58.329298 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5364f270-f1a5-4337-aaed-bb7578a31576 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1407 microversion: 2.20 time: 1.069132 Sep 27 10:19:58.342386 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2251/4936] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:57 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1407 bytes in 1083 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:58.484545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:58.485339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:58.500555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:58.506898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:58.507577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:58.508576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:58.509169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:58.510662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:58.713688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:58.724636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:58.726593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:19:58.728119 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-97b965e2-fd6c-4342-b602-04b2708cff9b tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.248545 Sep 27 10:19:58.729009 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2686/4937] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:58 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:19:59.141770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb80b968-3c5d-4403-a804-1147e23293c6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:59.142502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fb80b968-3c5d-4403-a804-1147e23293c6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:59.160643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb80b968-3c5d-4403-a804-1147e23293c6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:59.178808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb80b968-3c5d-4403-a804-1147e23293c6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:59.181424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb80b968-3c5d-4403-a804-1147e23293c6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:59.182209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb80b968-3c5d-4403-a804-1147e23293c6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:59.182693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb80b968-3c5d-4403-a804-1147e23293c6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:59.183134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb80b968-3c5d-4403-a804-1147e23293c6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:59.364327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:19:59.365364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:19:59.379946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:59.390362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:59.391149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:19:59.392305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:19:59.393370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:19:59.394113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:00.344080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fb80b968-3c5d-4403-a804-1147e23293c6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:00.356717 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fb80b968-3c5d-4403-a804-1147e23293c6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1317 microversion: 2.1 time: 1.219194 Sep 27 10:20:00.365720 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2252/4938] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:19:59 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1317 bytes in 1229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:00.380886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:00.382252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:00.396166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:00.401512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:00.406233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:00.411927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:00.414147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:00.417497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:00.418798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:00.420855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:00.424602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:00.434334 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c61967f6-f264-4f81-bff0-6a221f54bb09 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1407 microversion: 2.20 time: 1.074556 Sep 27 10:20:00.436689 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2687/4939] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:19:59 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1407 bytes in 1078 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:20:00.898338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:00.914915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:00.917331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:00.920741 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d0c3c5ca-e30a-4a8d-8b6e-2e11d98582af tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.551307 Sep 27 10:20:00.922219 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2253/4940] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:00 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:01.314012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-server-tempest-TestSecurityGroupsBasicOps-1266469994-access_point-929654910", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "9eca2e80-cc66-4f10-90d8-9dc9dfa07a7f"}], "key_name": "tempest-TestSecurityGroupsBasicOps-1627558846", "security_groups": [{"name": "tempest-secgroup_access--1249605953"}, {"name": "tempest-secgroup_general--2077869872"}]}, "os:scheduler_hints": {"different_host": []}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:01.394086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1f6612d4-dbcc-49bf-9f4c-3bd76f7f2955 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:01.396061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1f6612d4-dbcc-49bf-9f4c-3bd76f7f2955 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:01.419578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f6612d4-dbcc-49bf-9f4c-3bd76f7f2955 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:01.420462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f6612d4-dbcc-49bf-9f4c-3bd76f7f2955 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:01.422520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f6612d4-dbcc-49bf-9f4c-3bd76f7f2955 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:01.424318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f6612d4-dbcc-49bf-9f4c-3bd76f7f2955 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:01.425139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f6612d4-dbcc-49bf-9f4c-3bd76f7f2955 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:01.425843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f6612d4-dbcc-49bf-9f4c-3bd76f7f2955 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:01.681493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] validate_networks() for [('9eca2e80-cc66-4f10-90d8-9dc9dfa07a7f', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:20:01.969989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1f6612d4-dbcc-49bf-9f4c-3bd76f7f2955 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:01.972948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:01.982753 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1f6612d4-dbcc-49bf-9f4c-3bd76f7f2955 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1317 microversion: 2.1 time: 0.594870 Sep 27 10:20:01.985980 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2254/4941] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:01 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1317 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:01.991074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:02.000738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:02.015038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:02.015543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:02.015985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:02.016731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:02.017195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:02.017619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:02.174973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:02.382441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:20:02.382865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:20:02.393288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:20:02.393813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:20:02.394299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:20:02.407815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:02.408509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:02.409053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:02.409874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:02.410406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:02.410922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:02.433323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Getting quotas for project 932da48df2c44fc6a7dbd6c8daf9fc0d. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:20:02.440209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Getting quotas for user 64c9f44e57164b858421b55114acf4cf and project 932da48df2c44fc6a7dbd6c8daf9fc0d. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:20:02.465016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:20:02.467485 ncdv-master-1 devstack@n-api.service[145968]: WARNING nova.utils [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Hostname tempest-server-tempest-TestSecurityGroupsBasicOps-1266469994-access_point-929654910 is longer than 63, truncate it to tempest-server-tempest-TestSecurityGroupsBasicOps-1266469994-ac Sep 27 10:20:02.468364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:20:02.499423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:02.509022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:02.511620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:02.512996 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b3efcd06-60e9-46b9-af49-6e09f7d0c1e7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1407 microversion: 2.20 time: 0.525830 Sep 27 10:20:02.514042 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2255/4942] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:01 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1407 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:02.518773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a04d0cd6-0cc3-43bd-908e-2d1318ca71fb req-46ccc6a3-838f-43e5-9eb2-708116f3982a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "691b2be7-d5b8-409c-ba22-5ab8da50b80d", "name": "network-vif-plugged", "status": "completed", "tag": "11b232b4-3236-4917-9ebb-02d7dbd3132e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:02.527644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:02.529924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a04d0cd6-0cc3-43bd-908e-2d1318ca71fb req-46ccc6a3-838f-43e5-9eb2-708116f3982a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:02.530580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a04d0cd6-0cc3-43bd-908e-2d1318ca71fb req-46ccc6a3-838f-43e5-9eb2-708116f3982a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:02.531251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a04d0cd6-0cc3-43bd-908e-2d1318ca71fb req-46ccc6a3-838f-43e5-9eb2-708116f3982a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:02.544521 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a04d0cd6-0cc3-43bd-908e-2d1318ca71fb req-46ccc6a3-838f-43e5-9eb2-708116f3982a service nova] Creating event network-vif-plugged:11b232b4-3236-4917-9ebb-02d7dbd3132e for instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d on nch1-master-1 Sep 27 10:20:02.550364 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed5b3e74-c591-46c7-9636-ceb6aa8703f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 458 microversion: 2.1 time: 1.240093 Sep 27 10:20:02.552393 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a04d0cd6-0cc3-43bd-908e-2d1318ca71fb req-46ccc6a3-838f-43e5-9eb2-708116f3982a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036883 Sep 27 10:20:02.554447 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2688/4943] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:20:01 2022] POST /compute/v2.1/servers => generated 458 bytes in 1245 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:20:02.555942 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2256/4944] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:20:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:02.565370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1d33c01f-4b30-4f4c-ae44-5d7e3633537e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:02.567486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1d33c01f-4b30-4f4c-ae44-5d7e3633537e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:02.576736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:02.582993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d33c01f-4b30-4f4c-ae44-5d7e3633537e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:02.585014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d33c01f-4b30-4f4c-ae44-5d7e3633537e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:02.585707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:02.586119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d33c01f-4b30-4f4c-ae44-5d7e3633537e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:02.587069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d33c01f-4b30-4f4c-ae44-5d7e3633537e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:02.588398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d33c01f-4b30-4f4c-ae44-5d7e3633537e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:02.589173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d33c01f-4b30-4f4c-ae44-5d7e3633537e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:02.594049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:02.594840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:02.595286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:02.596569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:02.597087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:02.597523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:02.663597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1d33c01f-4b30-4f4c-ae44-5d7e3633537e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:20:02.840534 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1d33c01f-4b30-4f4c-ae44-5d7e3633537e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 204 len: 0 microversion: 2.1 time: 0.282687 Sep 27 10:20:02.851454 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2257/4945] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:20:02 2022] DELETE /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 0 bytes in 294 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:20:02.859258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-64103b5e-5772-42b2-b9b4-d67470073652 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:02.860395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-64103b5e-5772-42b2-b9b4-d67470073652 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:02.877649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:02.903436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:02.909112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:02.911157 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0697b38d-806c-43a5-ae5f-2a528c57e9aa tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.355218 Sep 27 10:20:02.913364 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2689/4946] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:02 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:02.929434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1db62fd1-0298-4986-96ae-756af1410bba tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:02.930777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1db62fd1-0298-4986-96ae-756af1410bba tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:02.946955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db62fd1-0298-4986-96ae-756af1410bba tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:02.948765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db62fd1-0298-4986-96ae-756af1410bba tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:02.949756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db62fd1-0298-4986-96ae-756af1410bba tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:02.951079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db62fd1-0298-4986-96ae-756af1410bba tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:02.951709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db62fd1-0298-4986-96ae-756af1410bba tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:02.952306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1db62fd1-0298-4986-96ae-756af1410bba tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:03.016976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-64103b5e-5772-42b2-b9b4-d67470073652 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:03.024231 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-64103b5e-5772-42b2-b9b4-d67470073652 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1294 microversion: 2.1 time: 0.170244 Sep 27 10:20:03.038132 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2258/4947] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:02 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1294 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:03.047669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5fe693ea-9efa-4986-8aa0-58cc782f2261 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:03.049126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5fe693ea-9efa-4986-8aa0-58cc782f2261 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:03.058072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fe693ea-9efa-4986-8aa0-58cc782f2261 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:03.059051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fe693ea-9efa-4986-8aa0-58cc782f2261 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:03.059703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fe693ea-9efa-4986-8aa0-58cc782f2261 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:03.060651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fe693ea-9efa-4986-8aa0-58cc782f2261 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:03.061665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fe693ea-9efa-4986-8aa0-58cc782f2261 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:03.062381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fe693ea-9efa-4986-8aa0-58cc782f2261 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:03.255705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1db62fd1-0298-4986-96ae-756af1410bba tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:03.265483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:03.270158 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1db62fd1-0298-4986-96ae-756af1410bba tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1554 microversion: 2.1 time: 0.353348 Sep 27 10:20:03.271193 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2690/4948] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:02 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1554 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:03.436235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d5361976-e879-43dc-a3a9-83360e2433a9 req-197f5da1-e0f5-48e2-9c0d-281106e90db1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cbc301f5-3b64-4276-b482-8febf910ad81", "tag": "1d703836-e0b2-4e82-aea3-ffea6cc0b34b"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:03.477426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d5361976-e879-43dc-a3a9-83360e2433a9 req-197f5da1-e0f5-48e2-9c0d-281106e90db1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:03.479525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d5361976-e879-43dc-a3a9-83360e2433a9 req-197f5da1-e0f5-48e2-9c0d-281106e90db1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:03.481915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d5361976-e879-43dc-a3a9-83360e2433a9 req-197f5da1-e0f5-48e2-9c0d-281106e90db1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:03.515839 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d5361976-e879-43dc-a3a9-83360e2433a9 req-197f5da1-e0f5-48e2-9c0d-281106e90db1 service nova] Creating event network-changed:1d703836-e0b2-4e82-aea3-ffea6cc0b34b for instance cbc301f5-3b64-4276-b482-8febf910ad81 on nch2-master-1 Sep 27 10:20:03.533118 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d5361976-e879-43dc-a3a9-83360e2433a9 req-197f5da1-e0f5-48e2-9c0d-281106e90db1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.102213 Sep 27 10:20:03.535015 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2691/4949] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:20:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:03.569678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:03.577585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:03.608182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:03.609177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:03.613607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:03.614860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:03.616053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:03.616779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:03.700941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fe693ea-9efa-4986-8aa0-58cc782f2261 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:03.717535 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5fe693ea-9efa-4986-8aa0-58cc782f2261 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1317 microversion: 2.1 time: 0.678191 Sep 27 10:20:03.731439 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2259/4950] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:03 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1317 bytes in 692 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:03.943255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:03.944375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:03.977349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:03.977879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:03.978379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:03.979128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:03.993568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:03.995858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:04.186419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:04.204273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:04.205950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:04.209613 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fb58719c-d8ac-4a1d-8b15-2d8d640dc356 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.273615 Sep 27 10:20:04.210239 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2260/4951] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:03 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:20:04.218331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b98ec37d-7aef-4b7f-b741-bbefe749d63f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:04.218965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b98ec37d-7aef-4b7f-b741-bbefe749d63f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:04.246471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b98ec37d-7aef-4b7f-b741-bbefe749d63f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:04.246892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b98ec37d-7aef-4b7f-b741-bbefe749d63f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:04.247400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b98ec37d-7aef-4b7f-b741-bbefe749d63f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:04.248482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b98ec37d-7aef-4b7f-b741-bbefe749d63f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:04.248992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b98ec37d-7aef-4b7f-b741-bbefe749d63f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:04.249640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b98ec37d-7aef-4b7f-b741-bbefe749d63f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:04.399033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b98ec37d-7aef-4b7f-b741-bbefe749d63f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:04.413657 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b98ec37d-7aef-4b7f-b741-bbefe749d63f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1290 microversion: 2.1 time: 0.202038 Sep 27 10:20:04.416717 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2261/4952] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:04 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1290 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:04.432465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-81f7f0d2-cb15-4b7f-848c-22473e17c603 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:04.433722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-81f7f0d2-cb15-4b7f-848c-22473e17c603 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:04.452578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81f7f0d2-cb15-4b7f-848c-22473e17c603 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:04.453477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81f7f0d2-cb15-4b7f-848c-22473e17c603 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:04.455532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81f7f0d2-cb15-4b7f-848c-22473e17c603 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:04.456516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81f7f0d2-cb15-4b7f-848c-22473e17c603 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:04.460097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81f7f0d2-cb15-4b7f-848c-22473e17c603 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:04.463385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81f7f0d2-cb15-4b7f-848c-22473e17c603 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:04.590866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:04.602760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:04.604542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:04.605946 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-911feb47-0218-4042-9ebb-63d4e73b16dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1407 microversion: 2.20 time: 1.058752 Sep 27 10:20:04.611833 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2692/4953] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:03 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1407 bytes in 1065 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:04.749467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1c4f8559-71ca-4d74-8424-604154065f99 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:04.750196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1c4f8559-71ca-4d74-8424-604154065f99 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:04.764591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c4f8559-71ca-4d74-8424-604154065f99 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:04.765271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c4f8559-71ca-4d74-8424-604154065f99 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:04.765716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c4f8559-71ca-4d74-8424-604154065f99 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:04.766436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c4f8559-71ca-4d74-8424-604154065f99 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:04.766893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c4f8559-71ca-4d74-8424-604154065f99 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:04.767314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c4f8559-71ca-4d74-8424-604154065f99 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:05.117285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-81f7f0d2-cb15-4b7f-848c-22473e17c603 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:05.132247 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-81f7f0d2-cb15-4b7f-848c-22473e17c603 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1554 microversion: 2.1 time: 0.716175 Sep 27 10:20:05.142776 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2262/4954] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:04 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1554 bytes in 725 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:05.239805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:05.240580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1c4f8559-71ca-4d74-8424-604154065f99 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:05.243269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:05.250235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:05.251829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:05.251994 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1c4f8559-71ca-4d74-8424-604154065f99 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1317 microversion: 2.1 time: 0.506070 Sep 27 10:20:05.252906 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2693/4955] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:04 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1317 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:20:05.268894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:05.273049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:05.274420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:05.275252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:05.394299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:05.409699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:05.411500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:05.413130 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7217c707-3d09-414f-8e8f-8e80c477bd51 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.177866 Sep 27 10:20:05.414223 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2263/4956] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:05 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:20:05.441392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-77ed20bb-454d-478b-9b30-da4fd20b7f1b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:05.442094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-77ed20bb-454d-478b-9b30-da4fd20b7f1b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:05.449740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77ed20bb-454d-478b-9b30-da4fd20b7f1b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:05.450232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77ed20bb-454d-478b-9b30-da4fd20b7f1b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:05.450666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77ed20bb-454d-478b-9b30-da4fd20b7f1b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:05.452313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77ed20bb-454d-478b-9b30-da4fd20b7f1b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:05.453618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77ed20bb-454d-478b-9b30-da4fd20b7f1b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:05.454427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77ed20bb-454d-478b-9b30-da4fd20b7f1b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:05.591987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-77ed20bb-454d-478b-9b30-da4fd20b7f1b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:05.615764 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-77ed20bb-454d-478b-9b30-da4fd20b7f1b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1354 microversion: 2.1 time: 0.181291 Sep 27 10:20:05.617560 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2694/4957] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:05 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1354 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:05.646394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:05.654990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:05.706620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:05.713148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:05.714229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:05.719957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:05.721025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:05.725738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:05.758756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-33e53f43-9cec-400d-addd-fb346d297e9a req-36676571-9bff-425a-8516-9abc9afe249c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0998602b-d17f-47e0-8af7-bddf2587d9ad", "name": "network-vif-deleted", "tag": "c6120fe6-85b3-422f-9e82-1e6a0fa5faa0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:05.782159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-33e53f43-9cec-400d-addd-fb346d297e9a req-36676571-9bff-425a-8516-9abc9afe249c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:05.782710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-33e53f43-9cec-400d-addd-fb346d297e9a req-36676571-9bff-425a-8516-9abc9afe249c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:05.783376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-33e53f43-9cec-400d-addd-fb346d297e9a req-36676571-9bff-425a-8516-9abc9afe249c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:05.864877 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-33e53f43-9cec-400d-addd-fb346d297e9a req-36676571-9bff-425a-8516-9abc9afe249c service nova] Creating event network-vif-deleted:c6120fe6-85b3-422f-9e82-1e6a0fa5faa0 for instance 0998602b-d17f-47e0-8af7-bddf2587d9ad on nch1-master-1 Sep 27 10:20:05.877500 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-33e53f43-9cec-400d-addd-fb346d297e9a req-36676571-9bff-425a-8516-9abc9afe249c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.124661 Sep 27 10:20:05.878929 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2695/4958] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:20:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:20:06.163486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:06.164155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:06.179074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:06.179752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:06.188481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:06.191383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:06.191879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:06.194206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:06.265949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:06.266830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:06.270821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:06.273014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lazy-loading 'fault' on Instance uuid 0998602b-d17f-47e0-8af7-bddf2587d9ad {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:20:06.291105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:06.300918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:06.316008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:06.319089 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3c903bef-179b-479f-bd09-a7e9a0db90f9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1407 microversion: 2.20 time: 0.680405 Sep 27 10:20:06.340961 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2264/4959] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:05 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1407 bytes in 702 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:06.347351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-666ac694-d1a8-40b0-ba5e-a279b4936638 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:06.349825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-666ac694-d1a8-40b0-ba5e-a279b4936638 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:06.359933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-666ac694-d1a8-40b0-ba5e-a279b4936638 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:06.360827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-666ac694-d1a8-40b0-ba5e-a279b4936638 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:06.361472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-666ac694-d1a8-40b0-ba5e-a279b4936638 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:06.362611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-666ac694-d1a8-40b0-ba5e-a279b4936638 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:06.363353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-666ac694-d1a8-40b0-ba5e-a279b4936638 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:06.364619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-666ac694-d1a8-40b0-ba5e-a279b4936638 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:06.442549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:06.456725 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-49c1bd88-5059-4079-98c2-74dd931b1cdb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 200 len: 1339 microversion: 2.1 time: 0.301600 Sep 27 10:20:06.470322 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2696/4960] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:06 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 1339 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:06.478945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:06.480061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:06.492193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:06.494207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:06.494959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:06.496317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:06.497801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:06.498450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:06.597302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:06.623281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:06.627345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:06.628856 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bc066c9c-6e3c-49ef-831c-c059bb4a16cd tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.156402 Sep 27 10:20:06.629940 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2697/4961] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:06 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:06.642043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3d8a238-06de-408f-b353-2f819fa09e68 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:06.643032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3d8a238-06de-408f-b353-2f819fa09e68 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:06.652789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d8a238-06de-408f-b353-2f819fa09e68 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:06.654214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d8a238-06de-408f-b353-2f819fa09e68 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:06.655879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d8a238-06de-408f-b353-2f819fa09e68 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:06.657070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d8a238-06de-408f-b353-2f819fa09e68 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:06.657822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d8a238-06de-408f-b353-2f819fa09e68 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:06.658447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3d8a238-06de-408f-b353-2f819fa09e68 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:06.772584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b3d8a238-06de-408f-b353-2f819fa09e68 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:06.786558 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3d8a238-06de-408f-b353-2f819fa09e68 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1352 microversion: 2.1 time: 0.151417 Sep 27 10:20:06.788271 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2698/4962] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:06 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1352 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:07.081441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-666ac694-d1a8-40b0-ba5e-a279b4936638 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:07.091189 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-666ac694-d1a8-40b0-ba5e-a279b4936638 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1317 microversion: 2.1 time: 0.748986 Sep 27 10:20:07.091787 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2265/4963] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:06 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1317 bytes in 750 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:07.347494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:07.348124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:07.355306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:07.357357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:07.358036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:07.358856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:07.359315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:07.359752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:07.481666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:07.482957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 0998602b-d17f-47e0-8af7-bddf2587d9ad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:07.490677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:07.491856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:07.492533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:07.493653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:07.495859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:07.496713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:07.526288 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] HTTP exception thrown: Instance 0998602b-d17f-47e0-8af7-bddf2587d9ad could not be found. Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Request method failure captured: Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad HTTP/1.1 Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs2eRjT8bhSYLsQlvJTQ4ttNPcS9EZduJEFDCgV3jTSdS8rrt4b9DEBaRu_lxwSxOqpmgVkoN1WqHUz7hKeSvXwkoqI49Ux6j3j03a7LM9ic5cuzb0pWQizVu9lKkAw7GNxHX8WWE5NXMndH2ECVNPJifobiU_kngKBg6iv_vlyVyq0 Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: dbfb9f03e6de43a6bd5e7a6641ac8088 Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestNetworkBasicOps-437891721 Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dbfb9f03e6de43a6bd5e7a6641ac8088"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dbfb9f03e6de43a6bd5e7a6641ac8088"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/dbfb9f03e6de43a6bd5e7a6641ac8088"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_dbfb9f03e6de43a6bd5e7a6641ac8088"}]}] Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestNetworkBasicOps-437891721 Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: dbfb9f03e6de43a6bd5e7a6641ac8088 Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestNetworkBasicOps-437891721 Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestNetworkBasicOps-437891721-project Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 1d92f7d2ee2d47beaa4f5d43ac40cd82 Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestNetworkBasicOps-437891721-project Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 0998602b-d17f-47e0-8af7-bddf2587d9ad could not be found. Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '0998602b-d17f-47e0-8af7-bddf2587d9ad'} Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0998602b-d17f-47e0-8af7-bddf2587d9ad could not be found. Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0998602b-d17f-47e0-8af7-bddf2587d9ad could not be found. Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:20:07.528022 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:07.534142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:07.534142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:20:07.534142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:20:07.534142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:20:07.534142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:20:07.534142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0998602b-d17f-47e0-8af7-bddf2587d9ad could not be found. Sep 27 10:20:07.534142 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:07.534142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Returning 404 to user: Instance 0998602b-d17f-47e0-8af7-bddf2587d9ad could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:20:07.534142 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a56abdbb-8f3e-402b-ab1b-5f470ce98847 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad" status: 404 len: 111 microversion: 2.1 time: 0.056704 Sep 27 10:20:07.537194 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2266/4964] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:07 2022] GET /compute/v2.1/servers/0998602b-d17f-47e0-8af7-bddf2587d9ad => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:07.658520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:07.659215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:07.665510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:07.666000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:07.666436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:07.667147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:07.667604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:07.668029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:07.780478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:07.795218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:07.798840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:07.807337 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d01edb02-6e51-49f7-a9cb-ef88d4de5ed5 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.150209 Sep 27 10:20:07.807337 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2267/4965] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:07 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:20:07.813001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-74c2fbcb-f04a-41c2-8160-d6180bc3c28f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:07.813900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-74c2fbcb-f04a-41c2-8160-d6180bc3c28f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:07.816455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:07.836535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74c2fbcb-f04a-41c2-8160-d6180bc3c28f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:07.837287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74c2fbcb-f04a-41c2-8160-d6180bc3c28f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:07.837931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74c2fbcb-f04a-41c2-8160-d6180bc3c28f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:07.838880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74c2fbcb-f04a-41c2-8160-d6180bc3c28f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:07.840134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74c2fbcb-f04a-41c2-8160-d6180bc3c28f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:07.841673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74c2fbcb-f04a-41c2-8160-d6180bc3c28f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:07.937181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:07.950833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:07.957299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:07.959112 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8c8e25d6-8db2-4725-ba57-68c522d8f448 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1599 microversion: 2.20 time: 0.615455 Sep 27 10:20:07.963119 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2699/4966] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:07 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1599 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:07.987727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e36f487-8712-4880-b516-97ef2a4449af tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:07.988750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8e36f487-8712-4880-b516-97ef2a4449af tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:07.996600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e36f487-8712-4880-b516-97ef2a4449af tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:07.997493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e36f487-8712-4880-b516-97ef2a4449af tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:07.998237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e36f487-8712-4880-b516-97ef2a4449af tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:07.999214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e36f487-8712-4880-b516-97ef2a4449af tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:07.999886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e36f487-8712-4880-b516-97ef2a4449af tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:08.000547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e36f487-8712-4880-b516-97ef2a4449af tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:08.219740 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e36f487-8712-4880-b516-97ef2a4449af tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.236049 Sep 27 10:20:08.221000 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2700/4967] 10.222.0.26 () {60 vars in 1316 bytes} [Tue Sep 27 10:20:07 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-interface => generated 285 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 10:20:08.235562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e97cdcbe-c68f-4e75-abc0-6352eae909b8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:08.237241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e97cdcbe-c68f-4e75-abc0-6352eae909b8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:08.250927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e97cdcbe-c68f-4e75-abc0-6352eae909b8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:08.261330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e97cdcbe-c68f-4e75-abc0-6352eae909b8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:08.261885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e97cdcbe-c68f-4e75-abc0-6352eae909b8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:08.263940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e97cdcbe-c68f-4e75-abc0-6352eae909b8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:08.264473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e97cdcbe-c68f-4e75-abc0-6352eae909b8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:08.269071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e97cdcbe-c68f-4e75-abc0-6352eae909b8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:08.271956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-74c2fbcb-f04a-41c2-8160-d6180bc3c28f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:08.280716 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-74c2fbcb-f04a-41c2-8160-d6180bc3c28f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1472 microversion: 2.1 time: 0.471744 Sep 27 10:20:08.281777 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2268/4968] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:07 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1472 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:08.333698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c2d15567-851d-4e34-a6f8-a83edacf15d0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '.version_select at 0x7f7f6252adc0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:08.359176 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c2d15567-851d-4e34-a6f8-a83edacf15d0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-1625678094" status: 202 len: 0 microversion: 2.1 time: 0.030510 Sep 27 10:20:08.360193 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2269/4969] 10.222.0.26 () {58 vars in 1257 bytes} [Tue Sep 27 10:20:08 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-1625678094 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:20:08.709330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e97cdcbe-c68f-4e75-abc0-6352eae909b8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:08.718404 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e97cdcbe-c68f-4e75-abc0-6352eae909b8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1317 microversion: 2.1 time: 0.495947 Sep 27 10:20:08.721370 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2701/4970] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:08 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1317 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:08.804308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-094e4049-2129-4c1b-82db-d08a53b65409 req-977c815c-d4f6-4924-9baf-c30393e704df service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3bba7aa2-0a3a-42f0-b00c-93ac58368945", "tag": "47866525-9b21-41d1-bec7-5a5dc5d53fad"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:08.829227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-094e4049-2129-4c1b-82db-d08a53b65409 req-977c815c-d4f6-4924-9baf-c30393e704df service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:08.830061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-094e4049-2129-4c1b-82db-d08a53b65409 req-977c815c-d4f6-4924-9baf-c30393e704df service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:08.830683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-094e4049-2129-4c1b-82db-d08a53b65409 req-977c815c-d4f6-4924-9baf-c30393e704df service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:08.837950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:08.838573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:08.851536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:08.851811 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-094e4049-2129-4c1b-82db-d08a53b65409 req-977c815c-d4f6-4924-9baf-c30393e704df service nova] Creating event network-changed:47866525-9b21-41d1-bec7-5a5dc5d53fad for instance 3bba7aa2-0a3a-42f0-b00c-93ac58368945 on nch1-master-1 Sep 27 10:20:08.853372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:08.854565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:08.857783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:08.858716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:08.859374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:08.859509 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-094e4049-2129-4c1b-82db-d08a53b65409 req-977c815c-d4f6-4924-9baf-c30393e704df service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065193 Sep 27 10:20:08.864512 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2270/4971] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:20:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:08.979228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:09.006220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:09.007753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:09.016921 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-51b283fd-66ee-44b7-a958-4e3d04855de6 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.175714 Sep 27 10:20:09.017673 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2702/4972] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:08 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:20:09.305509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-18911259-5c86-4aaa-bde0-c551db18b738 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:09.307042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-18911259-5c86-4aaa-bde0-c551db18b738 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:09.319378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18911259-5c86-4aaa-bde0-c551db18b738 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:09.319897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18911259-5c86-4aaa-bde0-c551db18b738 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:09.320321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18911259-5c86-4aaa-bde0-c551db18b738 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:09.321489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18911259-5c86-4aaa-bde0-c551db18b738 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:09.322191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18911259-5c86-4aaa-bde0-c551db18b738 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:09.322863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18911259-5c86-4aaa-bde0-c551db18b738 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:09.758720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ddca08f0-525c-4c56-be69-77dbcf6c84ea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:09.768689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ddca08f0-525c-4c56-be69-77dbcf6c84ea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:09.803777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddca08f0-525c-4c56-be69-77dbcf6c84ea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:09.806826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddca08f0-525c-4c56-be69-77dbcf6c84ea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:09.807861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddca08f0-525c-4c56-be69-77dbcf6c84ea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:09.810297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddca08f0-525c-4c56-be69-77dbcf6c84ea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:09.812954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddca08f0-525c-4c56-be69-77dbcf6c84ea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:09.813581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddca08f0-525c-4c56-be69-77dbcf6c84ea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:09.838961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-18911259-5c86-4aaa-bde0-c551db18b738 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:09.848686 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-18911259-5c86-4aaa-bde0-c551db18b738 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1472 microversion: 2.1 time: 0.547669 Sep 27 10:20:09.849978 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2271/4973] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:09 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1472 bytes in 550 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:10.046350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:10.047350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:10.065899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:10.066841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:10.067468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:10.069402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:10.070085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:10.070771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:10.209244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ddca08f0-525c-4c56-be69-77dbcf6c84ea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:10.223861 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ddca08f0-525c-4c56-be69-77dbcf6c84ea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1317 microversion: 2.1 time: 0.484226 Sep 27 10:20:10.225291 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2703/4974] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:09 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1317 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:10.247023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:10.268226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:10.276275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:10.278344 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32ac0222-c48d-4e7b-b822-fa72f9ea850d tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.235928 Sep 27 10:20:10.279444 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2272/4975] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:10 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:10.733301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-36f2f151-17ed-451e-802b-f49ebdb916d0 req-bb5129e9-5c77-466b-a6cb-be1f77d33a88 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cbc301f5-3b64-4276-b482-8febf910ad81", "name": "network-vif-plugged", "status": "completed", "tag": "1d703836-e0b2-4e82-aea3-ffea6cc0b34b"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:10.746802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-36f2f151-17ed-451e-802b-f49ebdb916d0 req-bb5129e9-5c77-466b-a6cb-be1f77d33a88 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:10.747301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-36f2f151-17ed-451e-802b-f49ebdb916d0 req-bb5129e9-5c77-466b-a6cb-be1f77d33a88 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:10.747744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-36f2f151-17ed-451e-802b-f49ebdb916d0 req-bb5129e9-5c77-466b-a6cb-be1f77d33a88 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:10.779823 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-36f2f151-17ed-451e-802b-f49ebdb916d0 req-bb5129e9-5c77-466b-a6cb-be1f77d33a88 service nova] Creating event network-vif-plugged:1d703836-e0b2-4e82-aea3-ffea6cc0b34b for instance cbc301f5-3b64-4276-b482-8febf910ad81 on nch2-master-1 Sep 27 10:20:10.787252 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-36f2f151-17ed-451e-802b-f49ebdb916d0 req-bb5129e9-5c77-466b-a6cb-be1f77d33a88 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057570 Sep 27 10:20:10.788205 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2704/4976] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:20:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:10.876957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6fbb7aa2-5c27-4685-8c19-aeaafbf09eb2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:10.878373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6fbb7aa2-5c27-4685-8c19-aeaafbf09eb2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:10.898014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fbb7aa2-5c27-4685-8c19-aeaafbf09eb2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:10.898516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fbb7aa2-5c27-4685-8c19-aeaafbf09eb2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:10.898971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fbb7aa2-5c27-4685-8c19-aeaafbf09eb2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:10.899690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fbb7aa2-5c27-4685-8c19-aeaafbf09eb2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:10.900143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fbb7aa2-5c27-4685-8c19-aeaafbf09eb2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:10.900766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fbb7aa2-5c27-4685-8c19-aeaafbf09eb2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:11.250714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a834124-443f-4c24-8c6d-4aa6cf25c8b6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:11.251651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3a834124-443f-4c24-8c6d-4aa6cf25c8b6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:11.263147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a834124-443f-4c24-8c6d-4aa6cf25c8b6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:11.266745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a834124-443f-4c24-8c6d-4aa6cf25c8b6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:11.267340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a834124-443f-4c24-8c6d-4aa6cf25c8b6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:11.268199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a834124-443f-4c24-8c6d-4aa6cf25c8b6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:11.270423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a834124-443f-4c24-8c6d-4aa6cf25c8b6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:11.271026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a834124-443f-4c24-8c6d-4aa6cf25c8b6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:11.407128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6fbb7aa2-5c27-4685-8c19-aeaafbf09eb2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:11.409947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:11.420796 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6fbb7aa2-5c27-4685-8c19-aeaafbf09eb2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1472 microversion: 2.1 time: 0.550182 Sep 27 10:20:11.437326 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2273/4977] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:10 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1472 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:11.447746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:11.450968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:11.467269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:11.468371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:11.469072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:11.472067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:11.475803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:11.477067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:11.597252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3a834124-443f-4c24-8c6d-4aa6cf25c8b6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:11.627961 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3a834124-443f-4c24-8c6d-4aa6cf25c8b6 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1317 microversion: 2.1 time: 0.379782 Sep 27 10:20:11.633842 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2705/4978] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:11 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1317 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:11.654418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-9f1e771d-d022-4e57-94ca-1b149429e5a9 req-a78bc6b9-82da-431b-8012-4c9cd0853c1c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "691b2be7-d5b8-409c-ba22-5ab8da50b80d", "tag": "11b232b4-3236-4917-9ebb-02d7dbd3132e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:11.666755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:11.679736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:11.683242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:11.688192 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1ba8d3d8-5993-4c27-adb4-e0fc35725c1e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.247192 Sep 27 10:20:11.689173 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2274/4979] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:11 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:11.702199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f1e771d-d022-4e57-94ca-1b149429e5a9 req-a78bc6b9-82da-431b-8012-4c9cd0853c1c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:11.702729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f1e771d-d022-4e57-94ca-1b149429e5a9 req-a78bc6b9-82da-431b-8012-4c9cd0853c1c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:11.705373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f1e771d-d022-4e57-94ca-1b149429e5a9 req-a78bc6b9-82da-431b-8012-4c9cd0853c1c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:11.761653 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-9f1e771d-d022-4e57-94ca-1b149429e5a9 req-a78bc6b9-82da-431b-8012-4c9cd0853c1c service nova] Creating event network-changed:11b232b4-3236-4917-9ebb-02d7dbd3132e for instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d on nch1-master-1 Sep 27 10:20:11.775129 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-9f1e771d-d022-4e57-94ca-1b149429e5a9 req-a78bc6b9-82da-431b-8012-4c9cd0853c1c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.132627 Sep 27 10:20:11.781884 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2706/4980] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:20:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:12.452989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9e2eebc-b37e-41c4-a5d0-f918a9078af2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:12.454626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9e2eebc-b37e-41c4-a5d0-f918a9078af2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:12.486867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9e2eebc-b37e-41c4-a5d0-f918a9078af2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:12.488239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9e2eebc-b37e-41c4-a5d0-f918a9078af2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:12.493725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9e2eebc-b37e-41c4-a5d0-f918a9078af2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:12.495154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9e2eebc-b37e-41c4-a5d0-f918a9078af2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:12.495843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9e2eebc-b37e-41c4-a5d0-f918a9078af2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:12.497226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9e2eebc-b37e-41c4-a5d0-f918a9078af2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:12.610303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dd9d46f6-e7fe-48d2-951d-11c1aecaf859 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:12.610303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dd9d46f6-e7fe-48d2-951d-11c1aecaf859 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:12.613589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:12.615210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:12.625624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd9d46f6-e7fe-48d2-951d-11c1aecaf859 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:12.628048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd9d46f6-e7fe-48d2-951d-11c1aecaf859 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:12.628600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd9d46f6-e7fe-48d2-951d-11c1aecaf859 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:12.633642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd9d46f6-e7fe-48d2-951d-11c1aecaf859 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:12.634415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd9d46f6-e7fe-48d2-951d-11c1aecaf859 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:12.634956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd9d46f6-e7fe-48d2-951d-11c1aecaf859 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:12.676670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dd9d46f6-e7fe-48d2-951d-11c1aecaf859 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:20:12.840718 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dd9d46f6-e7fe-48d2-951d-11c1aecaf859 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 204 len: 0 microversion: 2.1 time: 0.235423 Sep 27 10:20:12.846915 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2707/4981] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:20:12 2022] DELETE /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:20:12.867166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-31dd624d-38ed-4f6a-af81-1fe7be822d7e req-b1ceb485-270e-415f-bc0e-589e41d48943 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f0d9e453-3f14-4cd2-b7a7-31a47762c436", "tag": "e1e18a0a-7b42-40a1-b724-3c904a45faa3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:12.897573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-31dd624d-38ed-4f6a-af81-1fe7be822d7e req-b1ceb485-270e-415f-bc0e-589e41d48943 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:12.900576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-31dd624d-38ed-4f6a-af81-1fe7be822d7e req-b1ceb485-270e-415f-bc0e-589e41d48943 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:12.901079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-31dd624d-38ed-4f6a-af81-1fe7be822d7e req-b1ceb485-270e-415f-bc0e-589e41d48943 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:12.929609 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-31dd624d-38ed-4f6a-af81-1fe7be822d7e req-b1ceb485-270e-415f-bc0e-589e41d48943 service nova] Creating event network-changed:e1e18a0a-7b42-40a1-b724-3c904a45faa3 for instance f0d9e453-3f14-4cd2-b7a7-31a47762c436 on nch2-master-1 Sep 27 10:20:12.937719 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-31dd624d-38ed-4f6a-af81-1fe7be822d7e req-b1ceb485-270e-415f-bc0e-589e41d48943 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086751 Sep 27 10:20:12.938893 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2708/4982] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:20:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:12.949156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-baedcab4-b7ec-43d6-9c6b-5cced4378ac1 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:12.950599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-baedcab4-b7ec-43d6-9c6b-5cced4378ac1 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:12.968066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baedcab4-b7ec-43d6-9c6b-5cced4378ac1 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:12.968775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baedcab4-b7ec-43d6-9c6b-5cced4378ac1 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:12.969418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baedcab4-b7ec-43d6-9c6b-5cced4378ac1 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:12.970561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baedcab4-b7ec-43d6-9c6b-5cced4378ac1 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:12.971193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baedcab4-b7ec-43d6-9c6b-5cced4378ac1 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:12.971781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-baedcab4-b7ec-43d6-9c6b-5cced4378ac1 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:12.981135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:13.025046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:13.301967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-baedcab4-b7ec-43d6-9c6b-5cced4378ac1 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:13.315647 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-baedcab4-b7ec-43d6-9c6b-5cced4378ac1 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1507 microversion: 2.1 time: 0.374831 Sep 27 10:20:13.323656 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2709/4983] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:12 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1507 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:13.333397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:13.335686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:13.338454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c9e2eebc-b37e-41c4-a5d0-f918a9078af2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:13.355697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:13.356202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:13.356641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:13.357399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:13.357978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:13.359448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:13.362372 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9e2eebc-b37e-41c4-a5d0-f918a9078af2 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1472 microversion: 2.1 time: 0.915890 Sep 27 10:20:13.380353 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2275/4984] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:12 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1472 bytes in 934 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:13.385414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9a28434-5eb9-4671-a033-f2bced8b7a0b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:13.391252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9a28434-5eb9-4671-a033-f2bced8b7a0b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:13.416983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9a28434-5eb9-4671-a033-f2bced8b7a0b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:13.417866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9a28434-5eb9-4671-a033-f2bced8b7a0b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:13.418607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9a28434-5eb9-4671-a033-f2bced8b7a0b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:13.419626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9a28434-5eb9-4671-a033-f2bced8b7a0b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:13.420312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9a28434-5eb9-4671-a033-f2bced8b7a0b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:13.423168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9a28434-5eb9-4671-a033-f2bced8b7a0b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:13.513913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:13.528687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:13.533071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:13.534371 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4cf34a55-d98e-4510-b677-ce4c0ae25afb tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.208237 Sep 27 10:20:13.537379 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2710/4985] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:13 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:13.553006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eafdf37e-fc0a-4417-bdda-673810452cea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:13.553558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eafdf37e-fc0a-4417-bdda-673810452cea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:13.571234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eafdf37e-fc0a-4417-bdda-673810452cea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:13.572040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eafdf37e-fc0a-4417-bdda-673810452cea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:13.572744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eafdf37e-fc0a-4417-bdda-673810452cea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:13.573805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eafdf37e-fc0a-4417-bdda-673810452cea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:13.574653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eafdf37e-fc0a-4417-bdda-673810452cea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:13.575286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eafdf37e-fc0a-4417-bdda-673810452cea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:13.640833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-eafdf37e-fc0a-4417-bdda-673810452cea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-instance-actions:events failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:13.641858 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eafdf37e-fc0a-4417-bdda-673810452cea tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81/os-instance-actions/req-c9ac6e38-a14d-453f-8fcb-d631a223979e" status: 200 len: 313 microversion: 2.1 time: 0.103271 Sep 27 10:20:13.642470 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2711/4986] 10.222.0.26 () {58 vars in 1419 bytes} [Tue Sep 27 10:20:13 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81/os-instance-actions/req-c9ac6e38-a14d-453f-8fcb-d631a223979e => generated 313 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:13.657725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d811433b-70aa-45dd-ba0e-daf7bae22d45 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:13.669724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d811433b-70aa-45dd-ba0e-daf7bae22d45 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:13.676270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d811433b-70aa-45dd-ba0e-daf7bae22d45 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:13.677731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d811433b-70aa-45dd-ba0e-daf7bae22d45 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:13.678371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d811433b-70aa-45dd-ba0e-daf7bae22d45 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:13.680600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d811433b-70aa-45dd-ba0e-daf7bae22d45 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:13.686536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d811433b-70aa-45dd-ba0e-daf7bae22d45 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:13.687069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d811433b-70aa-45dd-ba0e-daf7bae22d45 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:13.845371 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d811433b-70aa-45dd-ba0e-daf7bae22d45 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "POST /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81/action" status: 202 len: 0 microversion: 2.1 time: 0.192019 Sep 27 10:20:13.846382 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2712/4987] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:20:13 2022] POST /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81/action => generated 0 bytes in 195 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:20:13.865437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f1c46cd-cfc6-4928-82f5-2ab5af0548fc tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:13.866468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9f1c46cd-cfc6-4928-82f5-2ab5af0548fc tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:13.890423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f1c46cd-cfc6-4928-82f5-2ab5af0548fc tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:13.891157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f1c46cd-cfc6-4928-82f5-2ab5af0548fc tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:13.892098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f1c46cd-cfc6-4928-82f5-2ab5af0548fc tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:13.893041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f1c46cd-cfc6-4928-82f5-2ab5af0548fc tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:13.893681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f1c46cd-cfc6-4928-82f5-2ab5af0548fc tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:13.894283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f1c46cd-cfc6-4928-82f5-2ab5af0548fc tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:13.967891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c9a28434-5eb9-4671-a033-f2bced8b7a0b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:13.981526 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9a28434-5eb9-4671-a033-f2bced8b7a0b tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1670 microversion: 2.1 time: 0.599833 Sep 27 10:20:13.982419 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2276/4988] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:13 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1670 bytes in 602 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:14.221097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9f1c46cd-cfc6-4928-82f5-2ab5af0548fc tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:14.231828 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9f1c46cd-cfc6-4928-82f5-2ab5af0548fc tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1509 microversion: 2.1 time: 0.380030 Sep 27 10:20:14.232972 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2713/4989] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:13 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1509 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:14.389721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25c68566-2f18-479a-974c-5d70a9151633 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:14.390346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-25c68566-2f18-479a-974c-5d70a9151633 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:14.398352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25c68566-2f18-479a-974c-5d70a9151633 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:14.399577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25c68566-2f18-479a-974c-5d70a9151633 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:14.400211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25c68566-2f18-479a-974c-5d70a9151633 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:14.401163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25c68566-2f18-479a-974c-5d70a9151633 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:14.401787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25c68566-2f18-479a-974c-5d70a9151633 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:14.402386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25c68566-2f18-479a-974c-5d70a9151633 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:14.567036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:14.567791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:14.569908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:14.578833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:14.579926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:14.580563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:14.581472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:14.582133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:14.582728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:14.718760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-25c68566-2f18-479a-974c-5d70a9151633 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:14.732252 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-25c68566-2f18-479a-974c-5d70a9151633 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1472 microversion: 2.1 time: 0.344552 Sep 27 10:20:14.733672 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2277/4990] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:14 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1472 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:14.737191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:14.746629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:14.748233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:14.749600 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-12794be9-ffcb-4d88-befd-d513f1cb6837 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1365 microversion: 2.57 time: 0.186485 Sep 27 10:20:14.750858 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2714/4991] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:14 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1365 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:15.008470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-35535d34-aa76-478e-b470-be3bedeaff40 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:15.009680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-35535d34-aa76-478e-b470-be3bedeaff40 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:15.016351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35535d34-aa76-478e-b470-be3bedeaff40 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:15.017188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35535d34-aa76-478e-b470-be3bedeaff40 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:15.017835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35535d34-aa76-478e-b470-be3bedeaff40 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:15.018737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35535d34-aa76-478e-b470-be3bedeaff40 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:15.019371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35535d34-aa76-478e-b470-be3bedeaff40 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:15.019970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35535d34-aa76-478e-b470-be3bedeaff40 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:15.260400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c7733d62-7af6-4466-a4be-cbf47a6b80a8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:15.261609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c7733d62-7af6-4466-a4be-cbf47a6b80a8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:15.273442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7733d62-7af6-4466-a4be-cbf47a6b80a8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:15.274127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7733d62-7af6-4466-a4be-cbf47a6b80a8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:15.275196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7733d62-7af6-4466-a4be-cbf47a6b80a8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:15.277722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7733d62-7af6-4466-a4be-cbf47a6b80a8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:15.278461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7733d62-7af6-4466-a4be-cbf47a6b80a8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:15.279144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7733d62-7af6-4466-a4be-cbf47a6b80a8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:15.554843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c7733d62-7af6-4466-a4be-cbf47a6b80a8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:15.569889 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c7733d62-7af6-4466-a4be-cbf47a6b80a8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1509 microversion: 2.1 time: 0.314411 Sep 27 10:20:15.571149 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2715/4992] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:15 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1509 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:15.590053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-795f6ee2-0ba8-4437-8ddf-6403da9a7272 req-651dd1a8-9fa3-4a17-8112-54b275534078 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3bba7aa2-0a3a-42f0-b00c-93ac58368945", "name": "network-vif-plugged", "status": "completed", "tag": "47866525-9b21-41d1-bec7-5a5dc5d53fad"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:15.621940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-795f6ee2-0ba8-4437-8ddf-6403da9a7272 req-651dd1a8-9fa3-4a17-8112-54b275534078 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:15.622435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-795f6ee2-0ba8-4437-8ddf-6403da9a7272 req-651dd1a8-9fa3-4a17-8112-54b275534078 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:15.622873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-795f6ee2-0ba8-4437-8ddf-6403da9a7272 req-651dd1a8-9fa3-4a17-8112-54b275534078 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:15.646835 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-795f6ee2-0ba8-4437-8ddf-6403da9a7272 req-651dd1a8-9fa3-4a17-8112-54b275534078 service nova] Creating event network-vif-plugged:47866525-9b21-41d1-bec7-5a5dc5d53fad for instance 3bba7aa2-0a3a-42f0-b00c-93ac58368945 on nch1-master-1 Sep 27 10:20:15.655836 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-795f6ee2-0ba8-4437-8ddf-6403da9a7272 req-651dd1a8-9fa3-4a17-8112-54b275534078 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082801 Sep 27 10:20:15.657784 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2716/4993] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:20:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:15.766529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e5a995b0-98f2-42d5-b468-1128f7515b4f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:15.767278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e5a995b0-98f2-42d5-b468-1128f7515b4f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:15.781616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a995b0-98f2-42d5-b468-1128f7515b4f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:15.784062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a995b0-98f2-42d5-b468-1128f7515b4f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:15.785222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a995b0-98f2-42d5-b468-1128f7515b4f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:15.786428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a995b0-98f2-42d5-b468-1128f7515b4f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:15.786975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a995b0-98f2-42d5-b468-1128f7515b4f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:15.787468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a995b0-98f2-42d5-b468-1128f7515b4f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:15.985817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-35535d34-aa76-478e-b470-be3bedeaff40 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:15.998782 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-35535d34-aa76-478e-b470-be3bedeaff40 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1670 microversion: 2.1 time: 0.994358 Sep 27 10:20:16.003139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e5a995b0-98f2-42d5-b468-1128f7515b4f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:16.013873 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2278/4994] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:15 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1670 bytes in 1010 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:16.014389 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e5a995b0-98f2-42d5-b468-1128f7515b4f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1472 microversion: 2.1 time: 0.261762 Sep 27 10:20:16.023845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:16.025525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:16.031962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:16.032836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:16.033529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:16.033681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2717/4995] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:15 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1472 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:20:16.034578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:16.035048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:16.035484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:16.244207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:16.257787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:16.259468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:16.262491 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-da3f611e-b38b-4cfb-8b7e-1db244365d74 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1371 microversion: 2.57 time: 0.246458 Sep 27 10:20:16.269977 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2279/4996] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:16 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1371 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:16.597175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5ab69594-5a43-40aa-beb1-e398fd0d3568 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:16.597883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5ab69594-5a43-40aa-beb1-e398fd0d3568 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:16.610198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ab69594-5a43-40aa-beb1-e398fd0d3568 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:16.612685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ab69594-5a43-40aa-beb1-e398fd0d3568 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:16.613979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ab69594-5a43-40aa-beb1-e398fd0d3568 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:16.614984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ab69594-5a43-40aa-beb1-e398fd0d3568 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:16.617328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ab69594-5a43-40aa-beb1-e398fd0d3568 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:16.617770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ab69594-5a43-40aa-beb1-e398fd0d3568 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:16.803800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5ab69594-5a43-40aa-beb1-e398fd0d3568 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:16.819648 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5ab69594-5a43-40aa-beb1-e398fd0d3568 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1509 microversion: 2.1 time: 0.225774 Sep 27 10:20:16.821118 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2718/4997] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:16 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1509 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:17.028046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4fd83147-85c9-4ba4-b2a6-514107eb54fa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:17.029115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4fd83147-85c9-4ba4-b2a6-514107eb54fa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:17.038840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd83147-85c9-4ba4-b2a6-514107eb54fa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:17.039609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd83147-85c9-4ba4-b2a6-514107eb54fa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:17.040281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd83147-85c9-4ba4-b2a6-514107eb54fa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:17.041280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd83147-85c9-4ba4-b2a6-514107eb54fa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:17.042022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd83147-85c9-4ba4-b2a6-514107eb54fa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:17.042664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4fd83147-85c9-4ba4-b2a6-514107eb54fa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:17.047608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-35cd6a71-4b7d-4661-8911-f42f2b930f22 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:17.049606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-35cd6a71-4b7d-4661-8911-f42f2b930f22 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:17.066511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35cd6a71-4b7d-4661-8911-f42f2b930f22 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:17.067282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35cd6a71-4b7d-4661-8911-f42f2b930f22 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:17.068124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35cd6a71-4b7d-4661-8911-f42f2b930f22 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:17.069868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35cd6a71-4b7d-4661-8911-f42f2b930f22 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:17.070867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35cd6a71-4b7d-4661-8911-f42f2b930f22 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:17.071612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35cd6a71-4b7d-4661-8911-f42f2b930f22 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:17.140300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4fd83147-85c9-4ba4-b2a6-514107eb54fa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:17.154099 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4fd83147-85c9-4ba4-b2a6-514107eb54fa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1603 microversion: 2.1 time: 0.131288 Sep 27 10:20:17.155847 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2280/4998] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:17 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1603 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:17.166182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a0a301ba-32f4-496f-86d8-0f5d87c2bd84 req-a9332cc2-9d11-4748-a4f2-5a1991aeada6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f0d9e453-3f14-4cd2-b7a7-31a47762c436", "name": "network-vif-deleted", "tag": "e1e18a0a-7b42-40a1-b724-3c904a45faa3"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:17.178912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a0a301ba-32f4-496f-86d8-0f5d87c2bd84 req-a9332cc2-9d11-4748-a4f2-5a1991aeada6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:17.179648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a0a301ba-32f4-496f-86d8-0f5d87c2bd84 req-a9332cc2-9d11-4748-a4f2-5a1991aeada6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:17.180033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a0a301ba-32f4-496f-86d8-0f5d87c2bd84 req-a9332cc2-9d11-4748-a4f2-5a1991aeada6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:17.196935 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a0a301ba-32f4-496f-86d8-0f5d87c2bd84 req-a9332cc2-9d11-4748-a4f2-5a1991aeada6 service nova] Creating event network-vif-deleted:e1e18a0a-7b42-40a1-b724-3c904a45faa3 for instance f0d9e453-3f14-4cd2-b7a7-31a47762c436 on nch2-master-1 Sep 27 10:20:17.221316 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a0a301ba-32f4-496f-86d8-0f5d87c2bd84 req-a9332cc2-9d11-4748-a4f2-5a1991aeada6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064119 Sep 27 10:20:17.224875 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2281/4999] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:20:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:17.294433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:17.295629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:17.296793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-35cd6a71-4b7d-4661-8911-f42f2b930f22 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:17.309277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:17.311359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:17.312569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:17.313639 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-35cd6a71-4b7d-4661-8911-f42f2b930f22 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1472 microversion: 2.1 time: 0.274874 Sep 27 10:20:17.313963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:17.315340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:17.315878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:17.316810 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2719/5000] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:17 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1472 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:17.463431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:17.478820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:17.480513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:17.483598 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5b109aa9-df23-4be9-99fd-73d87e8c48c9 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1371 microversion: 2.57 time: 0.196154 Sep 27 10:20:17.484174 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2282/5001] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:17 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1371 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:20:17.845160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9085509-5925-4003-bf41-81fffa572287 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:17.845800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b9085509-5925-4003-bf41-81fffa572287 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:17.853102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9085509-5925-4003-bf41-81fffa572287 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:17.853594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9085509-5925-4003-bf41-81fffa572287 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:17.854028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9085509-5925-4003-bf41-81fffa572287 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:17.854736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9085509-5925-4003-bf41-81fffa572287 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:17.855185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9085509-5925-4003-bf41-81fffa572287 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:17.855622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9085509-5925-4003-bf41-81fffa572287 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:18.149783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b9085509-5925-4003-bf41-81fffa572287 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:18.160036 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b9085509-5925-4003-bf41-81fffa572287 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1509 microversion: 2.1 time: 0.318978 Sep 27 10:20:18.161416 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2720/5002] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:17 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1509 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:20:18.182140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5f744c02-9844-4b7a-bcb8-f58d98f9063f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:18.182812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5f744c02-9844-4b7a-bcb8-f58d98f9063f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:18.191637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f744c02-9844-4b7a-bcb8-f58d98f9063f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:18.192826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f744c02-9844-4b7a-bcb8-f58d98f9063f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:18.193461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f744c02-9844-4b7a-bcb8-f58d98f9063f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:18.194576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f744c02-9844-4b7a-bcb8-f58d98f9063f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:18.195324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f744c02-9844-4b7a-bcb8-f58d98f9063f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:18.195966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f744c02-9844-4b7a-bcb8-f58d98f9063f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:18.294749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5f744c02-9844-4b7a-bcb8-f58d98f9063f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:18.303104 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5f744c02-9844-4b7a-bcb8-f58d98f9063f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1603 microversion: 2.1 time: 0.125514 Sep 27 10:20:18.303956 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2283/5003] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:18 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1603 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:18.341417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7dc3c68f-5838-4aec-a881-844701c4cb1c tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:18.342108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7dc3c68f-5838-4aec-a881-844701c4cb1c tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:18.360536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc3c68f-5838-4aec-a881-844701c4cb1c tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:18.361062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc3c68f-5838-4aec-a881-844701c4cb1c tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:18.361494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc3c68f-5838-4aec-a881-844701c4cb1c tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:18.362790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc3c68f-5838-4aec-a881-844701c4cb1c tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:18.363418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc3c68f-5838-4aec-a881-844701c4cb1c tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:18.364010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc3c68f-5838-4aec-a881-844701c4cb1c tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:18.514169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:18.515132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:18.526422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:18.528141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:18.529792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:18.531543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:18.533499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:18.534522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:18.682658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:18.695728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:18.697542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7dc3c68f-5838-4aec-a881-844701c4cb1c tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:18.699292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:18.700951 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-324f2ab5-d729-4ce0-a0db-e583573fae5a tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1371 microversion: 2.57 time: 0.190638 Sep 27 10:20:18.702059 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2284/5004] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:18 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1371 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:18.706476 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7dc3c68f-5838-4aec-a881-844701c4cb1c tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1644 microversion: 2.1 time: 0.368967 Sep 27 10:20:18.707063 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2721/5005] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:18 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1644 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:18.735352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a2d6f994-1471-4a46-ac29-41e3d1ccc37b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:18.736265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a2d6f994-1471-4a46-ac29-41e3d1ccc37b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:18.763097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2d6f994-1471-4a46-ac29-41e3d1ccc37b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:18.765111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2d6f994-1471-4a46-ac29-41e3d1ccc37b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:18.765549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2d6f994-1471-4a46-ac29-41e3d1ccc37b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:18.766287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2d6f994-1471-4a46-ac29-41e3d1ccc37b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:18.766760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2d6f994-1471-4a46-ac29-41e3d1ccc37b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:18.767174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2d6f994-1471-4a46-ac29-41e3d1ccc37b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:19.185084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-36ca64ed-5b72-4a74-9b2d-599f31626a48 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:19.185862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-36ca64ed-5b72-4a74-9b2d-599f31626a48 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:19.199591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36ca64ed-5b72-4a74-9b2d-599f31626a48 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:19.200897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36ca64ed-5b72-4a74-9b2d-599f31626a48 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:19.202977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36ca64ed-5b72-4a74-9b2d-599f31626a48 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:19.209996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36ca64ed-5b72-4a74-9b2d-599f31626a48 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:19.212237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36ca64ed-5b72-4a74-9b2d-599f31626a48 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:19.212868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-36ca64ed-5b72-4a74-9b2d-599f31626a48 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:19.218838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a2d6f994-1471-4a46-ac29-41e3d1ccc37b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:19.232179 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a2d6f994-1471-4a46-ac29-41e3d1ccc37b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1644 microversion: 2.1 time: 0.502636 Sep 27 10:20:19.233937 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2285/5006] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:18 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1644 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:19.259063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f4b29d03-05a4-436c-8ffd-6ea60a39ee44 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:19.259800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f4b29d03-05a4-436c-8ffd-6ea60a39ee44 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:19.267080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b29d03-05a4-436c-8ffd-6ea60a39ee44 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:19.267844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b29d03-05a4-436c-8ffd-6ea60a39ee44 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:19.268297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b29d03-05a4-436c-8ffd-6ea60a39ee44 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:19.269296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b29d03-05a4-436c-8ffd-6ea60a39ee44 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:19.269783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b29d03-05a4-436c-8ffd-6ea60a39ee44 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:19.270254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4b29d03-05a4-436c-8ffd-6ea60a39ee44 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:19.481810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-36ca64ed-5b72-4a74-9b2d-599f31626a48 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:19.495059 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-36ca64ed-5b72-4a74-9b2d-599f31626a48 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1509 microversion: 2.1 time: 0.314605 Sep 27 10:20:19.498883 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2722/5007] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:19 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1509 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:20:19.505035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:19.514661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:19.525702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:19.529350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:19.531914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:19.534328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:19.535068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:19.539788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:19.635225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:19.635793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:19.637022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:19.637022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lazy-loading 'fault' on Instance uuid f0d9e453-3f14-4cd2-b7a7-31a47762c436 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:20:19.743453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:19.754540 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a0ee0a6d-a629-4602-9cd0-f1f898e687fb tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 200 len: 1337 microversion: 2.1 time: 0.254361 Sep 27 10:20:19.755692 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2723/5008] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:19 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 1337 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:19.761192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:19.761811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:19.769258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:19.769946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:19.770382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:19.771301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:19.771929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:19.772522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:19.856193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:19.872312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:19.874541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:19.876198 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-05b6b504-ca64-4d12-b72e-e245f2337c97 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1371 microversion: 2.57 time: 0.119068 Sep 27 10:20:19.890140 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2724/5009] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:19 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1371 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:20.239816 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f4b29d03-05a4-436c-8ffd-6ea60a39ee44 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1795 microversion: 2.1 time: 0.985328 Sep 27 10:20:20.243114 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2286/5010] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:19 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1795 bytes in 989 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:20:20.526289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-01758df3-e657-45dd-a527-73175c13910a tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:20.527897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-01758df3-e657-45dd-a527-73175c13910a tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:20.538356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01758df3-e657-45dd-a527-73175c13910a tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:20.539407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01758df3-e657-45dd-a527-73175c13910a tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:20.540196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01758df3-e657-45dd-a527-73175c13910a tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:20.541943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01758df3-e657-45dd-a527-73175c13910a tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:20.542687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01758df3-e657-45dd-a527-73175c13910a tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:20.543467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01758df3-e657-45dd-a527-73175c13910a tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:20.550339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:20.783216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:20.784017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: f0d9e453-3f14-4cd2-b7a7-31a47762c436] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:20.793991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:20.799859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:20.800546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:20.802618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:20.803845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:20.805004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:20.859400 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] HTTP exception thrown: Instance f0d9e453-3f14-4cd2-b7a7-31a47762c436 could not be found. Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Request method failure captured: Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 HTTP/1.1 Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs2eRjT8bhSYLsQlvJTQ4ttNPcS9EZduJEFDCgV3jTSdS8rrt4b9DEBaRu_lxwSxOqpmgVkoN1WqHUz7hKeSvXwkoqI49Ux6j3j03a7LM9ic5cuzb0pWQizVu9lKkAw7GNxHX8WWE5NXMndH2ECVNPJifobiU_kngKBg6iv_vlyVyq0 Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: dbfb9f03e6de43a6bd5e7a6641ac8088 Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestNetworkBasicOps-437891721 Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dbfb9f03e6de43a6bd5e7a6641ac8088"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dbfb9f03e6de43a6bd5e7a6641ac8088"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/dbfb9f03e6de43a6bd5e7a6641ac8088"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_dbfb9f03e6de43a6bd5e7a6641ac8088"}]}] Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestNetworkBasicOps-437891721 Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: dbfb9f03e6de43a6bd5e7a6641ac8088 Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestNetworkBasicOps-437891721 Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestNetworkBasicOps-437891721-project Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 1d92f7d2ee2d47beaa4f5d43ac40cd82 Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestNetworkBasicOps-437891721-project Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance f0d9e453-3f14-4cd2-b7a7-31a47762c436 could not be found. Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'f0d9e453-3f14-4cd2-b7a7-31a47762c436'} Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f0d9e453-3f14-4cd2-b7a7-31a47762c436 could not be found. Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f0d9e453-3f14-4cd2-b7a7-31a47762c436 could not be found. Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:20:20.879320 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:20.885736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:20.885736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:20:20.885736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:20:20.885736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:20:20.885736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:20:20.885736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f0d9e453-3f14-4cd2-b7a7-31a47762c436 could not be found. Sep 27 10:20:20.885736 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:20.885736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Returning 404 to user: Instance f0d9e453-3f14-4cd2-b7a7-31a47762c436 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:20:20.885736 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-90e7e5e3-0933-4036-a0a0-8ab4fc42e119 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436" status: 404 len: 111 microversion: 2.1 time: 0.103405 Sep 27 10:20:20.895930 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2287/5011] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:20 2022] GET /compute/v2.1/servers/f0d9e453-3f14-4cd2-b7a7-31a47762c436 => generated 111 bytes in 117 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:20.896477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-01758df3-e657-45dd-a527-73175c13910a tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:20.907648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:20.911318 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-01758df3-e657-45dd-a527-73175c13910a tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1514 microversion: 2.1 time: 0.389760 Sep 27 10:20:20.911551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:20.914331 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2725/5012] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:20 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1514 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:20.940832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:20.941360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:20.946959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:20.947844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:20.948335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:20.950283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:21.086618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:21.106054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:21.107569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:21.112821 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-48dabe42-cbbb-4d10-a773-6cadcd83e06f tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1356 microversion: 2.57 time: 0.209069 Sep 27 10:20:21.115244 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2288/5013] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:20 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1356 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:21.148084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9cda5406-fb8e-4057-90d6-f4e121e7d9ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:21.148926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9cda5406-fb8e-4057-90d6-f4e121e7d9ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:21.156022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cda5406-fb8e-4057-90d6-f4e121e7d9ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:21.157204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cda5406-fb8e-4057-90d6-f4e121e7d9ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:21.157894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cda5406-fb8e-4057-90d6-f4e121e7d9ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:21.158865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cda5406-fb8e-4057-90d6-f4e121e7d9ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:21.159545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cda5406-fb8e-4057-90d6-f4e121e7d9ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:21.160442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cda5406-fb8e-4057-90d6-f4e121e7d9ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:21.198265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9cda5406-fb8e-4057-90d6-f4e121e7d9ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:20:21.351597 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9cda5406-fb8e-4057-90d6-f4e121e7d9ac tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 204 len: 0 microversion: 2.57 time: 0.209591 Sep 27 10:20:21.352274 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2726/5014] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:20:21 2022] DELETE /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:20:21.361580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:21.362868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:21.374821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:21.375436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:21.376791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:21.377813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:21.378457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:21.378965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:21.533167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:21.555481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:21.557682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:21.559356 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9125518e-51fe-490a-8357-bb9b80eba5ed tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1362 microversion: 2.57 time: 0.202210 Sep 27 10:20:21.561804 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2289/5015] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:21 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1362 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:21.814134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3db86059-f745-4b85-a912-5b01877f86b0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '.version_select at 0x7f7f6c22d310>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:21.854128 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3db86059-f745-4b85-a912-5b01877f86b0 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-762744049" status: 202 len: 0 microversion: 2.1 time: 0.044670 Sep 27 10:20:21.856562 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2727/5016] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 10:20:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-762744049 => generated 0 bytes in 48 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:20:21.939849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6359b93c-de10-4f63-bb82-2ba363a560c5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:21.940773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6359b93c-de10-4f63-bb82-2ba363a560c5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:21.958828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6359b93c-de10-4f63-bb82-2ba363a560c5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:21.959533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6359b93c-de10-4f63-bb82-2ba363a560c5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:21.960240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6359b93c-de10-4f63-bb82-2ba363a560c5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:21.961516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6359b93c-de10-4f63-bb82-2ba363a560c5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:21.962151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6359b93c-de10-4f63-bb82-2ba363a560c5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:21.962744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6359b93c-de10-4f63-bb82-2ba363a560c5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:22.295821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6359b93c-de10-4f63-bb82-2ba363a560c5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:22.305033 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6359b93c-de10-4f63-bb82-2ba363a560c5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1507 microversion: 2.1 time: 0.369311 Sep 27 10:20:22.305652 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2290/5017] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:21 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1507 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:22.330482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e82ff21-2eae-4331-ace6-07e1af3fc994 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '.version_select at 0x7f7f6c201820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:22.331860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2e82ff21-2eae-4331-ace6-07e1af3fc994 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:22.342883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e82ff21-2eae-4331-ace6-07e1af3fc994 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:22.343655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e82ff21-2eae-4331-ace6-07e1af3fc994 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:22.344283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e82ff21-2eae-4331-ace6-07e1af3fc994 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:22.345835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e82ff21-2eae-4331-ace6-07e1af3fc994 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:22.346468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e82ff21-2eae-4331-ace6-07e1af3fc994 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:22.348353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e82ff21-2eae-4331-ace6-07e1af3fc994 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:22.395964 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2e82ff21-2eae-4331-ace6-07e1af3fc994 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81/os-instance-actions" status: 200 len: 611 microversion: 2.1 time: 0.071100 Sep 27 10:20:22.396684 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2728/5018] 10.222.0.26 () {58 vars in 1296 bytes} [Tue Sep 27 10:20:22 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81/os-instance-actions => generated 611 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:22.411768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-753a668d-fc5c-4b35-b682-d766e42efc2d tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:22.413100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-753a668d-fc5c-4b35-b682-d766e42efc2d tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:22.419530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753a668d-fc5c-4b35-b682-d766e42efc2d tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:22.420309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753a668d-fc5c-4b35-b682-d766e42efc2d tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:22.421057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753a668d-fc5c-4b35-b682-d766e42efc2d tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:22.422028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753a668d-fc5c-4b35-b682-d766e42efc2d tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:22.422735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753a668d-fc5c-4b35-b682-d766e42efc2d tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:22.423426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-753a668d-fc5c-4b35-b682-d766e42efc2d tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:22.466533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-753a668d-fc5c-4b35-b682-d766e42efc2d tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:20:22.470010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:22.588067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:22.589226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:22.598358 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-753a668d-fc5c-4b35-b682-d766e42efc2d tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "DELETE /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 204 len: 0 microversion: 2.1 time: 0.192062 Sep 27 10:20:22.601566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:22.602316 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2291/5019] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:20:22 2022] DELETE /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:20:22.603227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:22.604052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:22.609940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:22.610861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:22.611549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:22.617153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2305ada9-89f7-460c-bcee-aa51446ed8b2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:22.618209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2305ada9-89f7-460c-bcee-aa51446ed8b2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:22.633079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2305ada9-89f7-460c-bcee-aa51446ed8b2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:22.634150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2305ada9-89f7-460c-bcee-aa51446ed8b2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:22.634816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2305ada9-89f7-460c-bcee-aa51446ed8b2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:22.636543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2305ada9-89f7-460c-bcee-aa51446ed8b2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:22.637483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2305ada9-89f7-460c-bcee-aa51446ed8b2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:22.638137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2305ada9-89f7-460c-bcee-aa51446ed8b2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:22.722401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:22.746604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:22.748425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:22.750595 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d732a2c-be39-4301-8f4e-6b2aa7b08134 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1362 microversion: 2.57 time: 0.167637 Sep 27 10:20:22.776714 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2729/5020] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:22 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1362 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:23.037945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2305ada9-89f7-460c-bcee-aa51446ed8b2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:23.058964 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2305ada9-89f7-460c-bcee-aa51446ed8b2 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1513 microversion: 2.1 time: 0.452236 Sep 27 10:20:23.060612 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2292/5021] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:22 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1513 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:23.783200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:23.783820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:23.796183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:23.799520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:23.800788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:23.802136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:23.803777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:23.804623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:23.912570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:23.926616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:23.928490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:23.930499 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-73b9df3e-1ba5-4ea8-b9c1-10ccc36c1443 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1362 microversion: 2.57 time: 0.152214 Sep 27 10:20:23.931792 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2730/5022] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:23 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1362 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:20:24.083293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6506ded6-e09b-4259-9468-24415e599aa5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:24.084586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6506ded6-e09b-4259-9468-24415e599aa5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:24.101085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6506ded6-e09b-4259-9468-24415e599aa5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:24.101771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6506ded6-e09b-4259-9468-24415e599aa5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:24.102482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6506ded6-e09b-4259-9468-24415e599aa5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:24.103251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6506ded6-e09b-4259-9468-24415e599aa5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:24.103720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6506ded6-e09b-4259-9468-24415e599aa5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:24.104350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6506ded6-e09b-4259-9468-24415e599aa5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:24.437574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6506ded6-e09b-4259-9468-24415e599aa5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:24.456476 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6506ded6-e09b-4259-9468-24415e599aa5 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1513 microversion: 2.1 time: 0.377313 Sep 27 10:20:24.457506 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2293/5023] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:24 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1513 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:24.960121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:24.961401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:24.969070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:24.969901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:24.970513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:24.971405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:24.972190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:24.972833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:25.014313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:25.015280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:25.016053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:25.016712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lazy-loading 'fault' on Instance uuid 7da921f3-7aab-4378-bc78-960cfb9be818 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:20:25.113883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:25.151113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:25.158404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e46a8a1c539e4781afda6aadd5373c8e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a82b2468ace34a009ca80c159ed44bb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:25.165095 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5cb89011-0bd5-4231-9fb8-7a215e3e596e tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 200 len: 1367 microversion: 2.57 time: 0.205711 Sep 27 10:20:25.168685 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2731/5024] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:24 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 1367 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:25.199989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e7187f8b-3dfd-4866-8e32-f1c214caab7a req-6d22ab54-1374-4b2e-a8e9-2f57bce0ae47 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3bba7aa2-0a3a-42f0-b00c-93ac58368945", "tag": "47866525-9b21-41d1-bec7-5a5dc5d53fad"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:25.237309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e7187f8b-3dfd-4866-8e32-f1c214caab7a req-6d22ab54-1374-4b2e-a8e9-2f57bce0ae47 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:25.238276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e7187f8b-3dfd-4866-8e32-f1c214caab7a req-6d22ab54-1374-4b2e-a8e9-2f57bce0ae47 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:25.238962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e7187f8b-3dfd-4866-8e32-f1c214caab7a req-6d22ab54-1374-4b2e-a8e9-2f57bce0ae47 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:25.279524 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e7187f8b-3dfd-4866-8e32-f1c214caab7a req-6d22ab54-1374-4b2e-a8e9-2f57bce0ae47 service nova] Creating event network-changed:47866525-9b21-41d1-bec7-5a5dc5d53fad for instance 3bba7aa2-0a3a-42f0-b00c-93ac58368945 on nch1-master-1 Sep 27 10:20:25.293015 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e7187f8b-3dfd-4866-8e32-f1c214caab7a req-6d22ab54-1374-4b2e-a8e9-2f57bce0ae47 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.101083 Sep 27 10:20:25.297202 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2294/5025] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:20:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:25.485131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ac84df93-c8dd-4c5f-993b-6d5ce0b57f06 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:25.485752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ac84df93-c8dd-4c5f-993b-6d5ce0b57f06 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:25.497294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac84df93-c8dd-4c5f-993b-6d5ce0b57f06 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:25.498509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac84df93-c8dd-4c5f-993b-6d5ce0b57f06 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:25.502110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac84df93-c8dd-4c5f-993b-6d5ce0b57f06 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:25.503651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac84df93-c8dd-4c5f-993b-6d5ce0b57f06 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:25.510040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac84df93-c8dd-4c5f-993b-6d5ce0b57f06 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:25.512020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac84df93-c8dd-4c5f-993b-6d5ce0b57f06 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:26.021744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ac84df93-c8dd-4c5f-993b-6d5ce0b57f06 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:26.034041 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ac84df93-c8dd-4c5f-993b-6d5ce0b57f06 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1513 microversion: 2.1 time: 0.553120 Sep 27 10:20:26.037979 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2732/5026] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:25 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1513 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:26.199812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:26.200529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] [instance: 7da921f3-7aab-4378-bc78-960cfb9be818] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:26.217086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:26.217592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:26.218035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:26.218758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:26.223522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:26.226650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:26.264211 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] HTTP exception thrown: Instance 7da921f3-7aab-4378-bc78-960cfb9be818 could not be found. Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Request method failure captured: Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 HTTP/1.1 Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs453w0vSfc1VU-Zpxz5PT9Awx0a4keORFB4JpEYEYL-P0Fzwvm0v6HWZayUMUJJ6XbYVnCRGQ-E8C2AZMSQ0q4dzCEdrZE3LbY_MJfI5vRumHJXJB-ybr2WGGaIa58lifD33SCVa7DBPXOg3KqCit7yPmAAR4md8Ob7u3cAnanKBgI Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.57 Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: a82b2468ace34a009ca80c159ed44bb8 Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerShowV257Test-748035604 Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a82b2468ace34a009ca80c159ed44bb8"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a82b2468ace34a009ca80c159ed44bb8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a82b2468ace34a009ca80c159ed44bb8"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a82b2468ace34a009ca80c159ed44bb8"}]}] Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerShowV257Test-748035604 Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: a82b2468ace34a009ca80c159ed44bb8 Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerShowV257Test-748035604 Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerShowV257Test-748035604-project Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: e46a8a1c539e4781afda6aadd5373c8e Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerShowV257Test-748035604-project Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 7da921f3-7aab-4378-bc78-960cfb9be818 could not be found. Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '7da921f3-7aab-4378-bc78-960cfb9be818'} Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7da921f3-7aab-4378-bc78-960cfb9be818 could not be found. Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7da921f3-7aab-4378-bc78-960cfb9be818 could not be found. Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:20:26.267477 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:26.272209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:26.272209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:20:26.272209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:20:26.272209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:20:26.272209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:20:26.272209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7da921f3-7aab-4378-bc78-960cfb9be818 could not be found. Sep 27 10:20:26.272209 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:26.274605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] Returning 404 to user: Instance 7da921f3-7aab-4378-bc78-960cfb9be818 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:20:26.275700 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d9ae123d-fe70-4c47-9aea-5efa7cb14d63 tempest-ServerShowV257Test-748035604 tempest-ServerShowV257Test-748035604-project] 10.222.0.26 "GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818" status: 404 len: 111 microversion: 2.57 time: 0.083944 Sep 27 10:20:26.276479 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2295/5027] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:26 2022] GET /compute/v2.1/servers/7da921f3-7aab-4378-bc78-960cfb9be818 => generated 111 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 27 10:20:26.965958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-48d5b070-d678-4f06-9652-db9860aabc2a req-ed49796e-57f5-4aa3-9e01-45d4eda10008 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cbc301f5-3b64-4276-b482-8febf910ad81", "name": "network-vif-deleted", "tag": "1d703836-e0b2-4e82-aea3-ffea6cc0b34b"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:27.013689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-48d5b070-d678-4f06-9652-db9860aabc2a req-ed49796e-57f5-4aa3-9e01-45d4eda10008 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:27.014402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-48d5b070-d678-4f06-9652-db9860aabc2a req-ed49796e-57f5-4aa3-9e01-45d4eda10008 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:27.015346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-48d5b070-d678-4f06-9652-db9860aabc2a req-ed49796e-57f5-4aa3-9e01-45d4eda10008 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:27.054535 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-48d5b070-d678-4f06-9652-db9860aabc2a req-ed49796e-57f5-4aa3-9e01-45d4eda10008 service nova] Creating event network-vif-deleted:1d703836-e0b2-4e82-aea3-ffea6cc0b34b for instance cbc301f5-3b64-4276-b482-8febf910ad81 on nch2-master-1 Sep 27 10:20:27.064147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d8253aee-b11b-4382-b605-069a1a6c2ab8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:27.065108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d8253aee-b11b-4382-b605-069a1a6c2ab8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:27.073482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8253aee-b11b-4382-b605-069a1a6c2ab8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:27.080936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8253aee-b11b-4382-b605-069a1a6c2ab8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:27.081645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8253aee-b11b-4382-b605-069a1a6c2ab8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:27.083148 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-48d5b070-d678-4f06-9652-db9860aabc2a req-ed49796e-57f5-4aa3-9e01-45d4eda10008 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.123121 Sep 27 10:20:27.085228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8253aee-b11b-4382-b605-069a1a6c2ab8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:27.087089 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2733/5028] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:20:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:27.087800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8253aee-b11b-4382-b605-069a1a6c2ab8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:27.092556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8253aee-b11b-4382-b605-069a1a6c2ab8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:27.411982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d8253aee-b11b-4382-b605-069a1a6c2ab8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '377e47048a63456793dfc9c125f5a209', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cb035a91b1f341ce8d8b71d39ff10a0b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:27.433716 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d8253aee-b11b-4382-b605-069a1a6c2ab8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 200 len: 1302 microversion: 2.1 time: 0.371345 Sep 27 10:20:27.436249 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2296/5029] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:27 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 1302 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:28.468513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:28.469746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] [instance: cbc301f5-3b64-4276-b482-8febf910ad81] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:28.488790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:28.489339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:28.491064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:28.493851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:28.494540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:28.496149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:28.529722 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] HTTP exception thrown: Instance cbc301f5-3b64-4276-b482-8febf910ad81 could not be found. Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Request method failure captured: Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 HTTP/1.1 Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs5FLITTX-uV7zdSGV44W9j0wnefBK-u6b_TC-pJGdERUXPexi_4BCWnNQ0p-iqBptJ1WszxJ9wZ2UAYk1THZ0UFUi3PFZVeIlxcVXl02gkZMJcNAB2LoVH53sWH9aZGuNnFmUYtHifTSFwIXTP6UrUrtXQPmNVlsIbA4M0KhVFVcyU Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: cb035a91b1f341ce8d8b71d39ff10a0b Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-InstanceActionsTestJSON-1323704877 Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/cb035a91b1f341ce8d8b71d39ff10a0b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/cb035a91b1f341ce8d8b71d39ff10a0b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/cb035a91b1f341ce8d8b71d39ff10a0b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_cb035a91b1f341ce8d8b71d39ff10a0b"}]}] Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-InstanceActionsTestJSON-1323704877 Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: cb035a91b1f341ce8d8b71d39ff10a0b Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-InstanceActionsTestJSON-1323704877 Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-InstanceActionsTestJSON-1323704877-project Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 377e47048a63456793dfc9c125f5a209 Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-InstanceActionsTestJSON-1323704877-project Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance cbc301f5-3b64-4276-b482-8febf910ad81 could not be found. Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'cbc301f5-3b64-4276-b482-8febf910ad81'} Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cbc301f5-3b64-4276-b482-8febf910ad81 could not be found. Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cbc301f5-3b64-4276-b482-8febf910ad81 could not be found. Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:20:28.531430 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:20:28.537941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:28.537941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:20:28.537941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:20:28.537941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:20:28.537941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:20:28.537941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cbc301f5-3b64-4276-b482-8febf910ad81 could not be found. Sep 27 10:20:28.537941 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:20:28.537941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] Returning 404 to user: Instance cbc301f5-3b64-4276-b482-8febf910ad81 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:20:28.537941 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-80ac80e7-b4dd-49e9-8702-fb7a1ee3d1c8 tempest-InstanceActionsTestJSON-1323704877 tempest-InstanceActionsTestJSON-1323704877-project] 10.222.0.26 "GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81" status: 404 len: 111 microversion: 2.1 time: 0.072341 Sep 27 10:20:28.537941 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2734/5030] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:28 2022] GET /compute/v2.1/servers/cbc301f5-3b64-4276-b482-8febf910ad81 => generated 111 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:30.486403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75423c69-ad4e-4114-ad7f-6a1d951fad49 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] Action: 'create', calling method: .version_select at 0x7f7f6251cd30>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-67048335", "metadata": {"Type": "work"}}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:31.287890 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-75423c69-ad4e-4114-ad7f-6a1d951fad49 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.982301 Sep 27 10:20:31.289135 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2297/5031] 10.222.0.26 () {60 vars in 1156 bytes} [Tue Sep 27 10:20:30 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 985 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 27 10:20:31.304409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62b72bc1-104d-49eb-a6d3-48a5fd2358bc tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] Calling method '.version_select at 0x7f7f6c2d98b0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:31.386323 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-62b72bc1-104d-49eb-a6d3-48a5fd2358bc tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59" status: 200 len: 344 microversion: 2.1 time: 0.086505 Sep 27 10:20:31.387283 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2735/5032] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:20:31 2022] GET /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:32.402890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7e0d756b-e4fe-4f71-8d91-f7fe8b5b3a01 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] Calling method '.version_select at 0x7f7f625afe50>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:32.463592 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7e0d756b-e4fe-4f71-8d91-f7fe8b5b3a01 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59" status: 200 len: 345 microversion: 2.1 time: 0.065196 Sep 27 10:20:32.475490 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2298/5033] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:20:32 2022] GET /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59 => generated 345 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:32.483090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-87f0c43f-b433-4006-98fb-243a1732697a tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] Calling method '.version_select at 0x7f7f630a6550>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:32.541989 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-87f0c43f-b433-4006-98fb-243a1732697a tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59" status: 200 len: 345 microversion: 2.1 time: 0.064510 Sep 27 10:20:32.542817 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2736/5034] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:20:32 2022] GET /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59 => generated 345 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:32.566132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] Calling method '.version_select at 0x7f7f625ffd30>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:32.656455 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-78e6b53f-a753-44c8-b1c2-46bd9c43db29 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59" status: 202 len: 0 microversion: 2.1 time: 0.096244 Sep 27 10:20:32.658519 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2299/5035] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:20:32 2022] DELETE /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59 => generated 0 bytes in 103 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:20:32.668896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-812e62c5-8f15-41d9-b0f3-095064f363c8 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] Calling method '.version_select at 0x7f7f6c2d9ca0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:32.733520 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-812e62c5-8f15-41d9-b0f3-095064f363c8 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59" status: 200 len: 344 microversion: 2.1 time: 0.068889 Sep 27 10:20:32.734860 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2737/5036] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:20:32 2022] GET /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59 => generated 344 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:33.750327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-88977882-084d-40fa-b679-e8d54d098cd5 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] Calling method '.version_select at 0x7f7f624a7670>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:33.818729 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-88977882-084d-40fa-b679-e8d54d098cd5 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] HTTP exception thrown: Volume 61fd1543-5e89-47fc-a9e5-641aa5252e59 could not be found. Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-88977882-084d-40fa-b679-e8d54d098cd5 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] Request method failure captured: Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59 HTTP/1.1 Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs5uAYsrh_wVA01rYe1EKIvaB6sTOSHycqz15dG7Jk9vmsWcksIjus912E3TrkuHcfj8TuxlGAMqo_Xqn821c-y_mSqlvcK8X2uj4GOMXpeDLJyLm8AchwiCJsOPFAfl75USF-ZQUmiCHr49mvVPnMuhQINj_2D2RwVfwLaWFl-BCPU Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 4f80f3a9dcfe43c6868d300dd1495253 Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-VolumesGetTestJSON-1505950546 Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4f80f3a9dcfe43c6868d300dd1495253"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4f80f3a9dcfe43c6868d300dd1495253"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/4f80f3a9dcfe43c6868d300dd1495253"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_4f80f3a9dcfe43c6868d300dd1495253"}]}] Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-VolumesGetTestJSON-1505950546 Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 4f80f3a9dcfe43c6868d300dd1495253 Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-VolumesGetTestJSON-1505950546 Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-VolumesGetTestJSON-1505950546-project Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 4c6f77ddc6a245db9d9b236145c7647f Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-VolumesGetTestJSON-1505950546-project Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: exception: Volume 61fd1543-5e89-47fc-a9e5-641aa5252e59 could not be found. Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '61fd1543-5e89-47fc-a9e5-641aa5252e59'} Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 61fd1543-5e89-47fc-a9e5-641aa5252e59 could not be found. (HTTP 404) (Request-ID: req-0ee333ae-19b9-4d10-a401-9f8e2d5b4ecd) Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 10:20:33.821173 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 61fd1543-5e89-47fc-a9e5-641aa5252e59 could not be found. Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 61fd1543-5e89-47fc-a9e5-641aa5252e59 could not be found. Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 61fd1543-5e89-47fc-a9e5-641aa5252e59 could not be found. Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-88977882-084d-40fa-b679-e8d54d098cd5 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] Returning 404 to user: Volume 61fd1543-5e89-47fc-a9e5-641aa5252e59 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-88977882-084d-40fa-b679-e8d54d098cd5 tempest-VolumesGetTestJSON-1505950546 tempest-VolumesGetTestJSON-1505950546-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59" status: 404 len: 109 microversion: 2.1 time: 0.077353 Sep 27 10:20:33.827131 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2300/5037] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:20:33 2022] GET /compute/v2.1/os-volumes/61fd1543-5e89-47fc-a9e5-641aa5252e59 => generated 109 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:20:44.447484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4a396a2-c9af-4b93-be47-9345f5214b2a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Action: 'create', calling method: .version_select at 0x7f7f6c201dc0>, body: {"keypair": {"name": "tempest-TestNetworkBasicOps-677549886", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIALg2vmJhRcBlSCmmj16cvLarpEp2bqSsqKPCDSE2v5yjhCCeD7WK22q69DqTEHaP68NNPuKTf83mxE9td84mJJgzO2uQcEudsw78CZ64sa3SFKKLvjtBNIv+oq/7NHzQ=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:44.463251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:44.463993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=54,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:44.493821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d4a396a2-c9af-4b93-be47-9345f5214b2a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:20:44.514466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d4a396a2-c9af-4b93-be47-9345f5214b2a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for user 1d92f7d2ee2d47beaa4f5d43ac40cd82 and project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:20:44.560831 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4a396a2-c9af-4b93-be47-9345f5214b2a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.117055 Sep 27 10:20:44.561579 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2738/5038] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:20:44 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:20:45.084058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=63,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:45.087579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=31,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:45.097168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ffc83364-12f7-489e-8ce0-74c046f3b377 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:45.099195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ffc83364-12f7-489e-8ce0-74c046f3b377 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:45.138806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffc83364-12f7-489e-8ce0-74c046f3b377 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:45.139470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffc83364-12f7-489e-8ce0-74c046f3b377 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:45.139750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffc83364-12f7-489e-8ce0-74c046f3b377 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:45.141695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffc83364-12f7-489e-8ce0-74c046f3b377 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:45.143765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffc83364-12f7-489e-8ce0-74c046f3b377 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:45.144315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffc83364-12f7-489e-8ce0-74c046f3b377 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:45.206087 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ffc83364-12f7-489e-8ce0-74c046f3b377 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/ips" status: 200 len: 151 microversion: 2.20 time: 0.125124 Sep 27 10:20:45.215872 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2301/5039] 10.222.0.26 () {60 vars in 1289 bytes} [Tue Sep 27 10:20:45 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/ips => generated 151 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:20:47.314631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8216f6c6-754d-4da0-b1ad-2cc53e4f5567 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:47.315291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8216f6c6-754d-4da0-b1ad-2cc53e4f5567 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:47.339672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8216f6c6-754d-4da0-b1ad-2cc53e4f5567 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:47.341824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8216f6c6-754d-4da0-b1ad-2cc53e4f5567 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:47.342299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8216f6c6-754d-4da0-b1ad-2cc53e4f5567 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:47.348024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8216f6c6-754d-4da0-b1ad-2cc53e4f5567 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:47.348689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8216f6c6-754d-4da0-b1ad-2cc53e4f5567 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:47.349410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8216f6c6-754d-4da0-b1ad-2cc53e4f5567 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:47.434397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-8216f6c6-754d-4da0-b1ad-2cc53e4f5567 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lazy-loading 'flavor' on Instance uuid 691b2be7-d5b8-409c-ba22-5ab8da50b80d {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:20:47.883316 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8216f6c6-754d-4da0-b1ad-2cc53e4f5567 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "POST /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/action" status: 202 len: 0 microversion: 2.20 time: 0.578711 Sep 27 10:20:47.887407 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2739/5040] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:20:47 2022] POST /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/action => generated 0 bytes in 582 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:20:47.895606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:47.897525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:47.909884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:47.910492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:47.910945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:47.912448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:47.913643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:47.914478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:48.464167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:48.475427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:48.485951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:48.497176 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f92e359-db51-4011-94c9-7ffb63b66706 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1723 microversion: 2.20 time: 0.607608 Sep 27 10:20:48.504576 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2302/5041] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:47 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1723 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:49.541180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:49.542004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:49.556725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:49.557205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:49.560122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:49.561041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:49.561551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:49.566914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:49.799527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkBasicOps-server-564488713", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "6101d46f-0c39-4389-a47f-08fb91a14f03"}], "key_name": "tempest-TestNetworkBasicOps-677549886", "security_groups": [{"name": "tempest-secgroup-smoke-137834933"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:50.111926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:50.125357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:50.132569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:50.134243 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a3530d3f-b952-471a-a539-2b195a61e426 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1723 microversion: 2.20 time: 0.602691 Sep 27 10:20:50.140134 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2740/5042] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:49 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1723 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:50.225400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] validate_networks() for [('6101d46f-0c39-4389-a47f-08fb91a14f03', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:20:51.170390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:51.171113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:51.199153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:51.199679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:51.200169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:51.201064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:51.201544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:51.201985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:51.663516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:20:51.663516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:20:51.663516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:20:51.663516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:20:51.663516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:20:51.675141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:51.675742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:51.677249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:51.678328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:51.679121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:51.679981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:51.705575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:20:51.712335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Getting quotas for user 1d92f7d2ee2d47beaa4f5d43ac40cd82 and project dbfb9f03e6de43a6bd5e7a6641ac8088. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:20:51.740605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:20:51.743925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:20:51.831218 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a87b16aa-f48e-431d-8828-19ece567b741 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 2.037068 Sep 27 10:20:51.836382 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2303/5043] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:20:49 2022] POST /compute/v2.1/servers => generated 405 bytes in 2043 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:20:51.855763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c1c6b558-f15c-48f6-b63c-461ae835b42e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:51.857636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c1c6b558-f15c-48f6-b63c-461ae835b42e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:51.908490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:51.921961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:51.925317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:51.926964 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-086c8722-354c-4312-8219-0ce9fc923082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1723 microversion: 2.20 time: 0.760573 Sep 27 10:20:51.932435 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2741/5044] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:51 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1723 bytes in 767 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:52.011739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c1c6b558-f15c-48f6-b63c-461ae835b42e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:52.036554 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c1c6b558-f15c-48f6-b63c-461ae835b42e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1247 microversion: 2.1 time: 0.192238 Sep 27 10:20:52.040077 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2304/5045] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:51 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1247 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:52.971826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:52.972474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:52.991649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:52.993630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:52.994317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:52.998954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:52.999665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:53.000388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:53.061988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ad3141e2-3a20-4738-8883-7049f297a90a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:53.063208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ad3141e2-3a20-4738-8883-7049f297a90a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:53.087031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad3141e2-3a20-4738-8883-7049f297a90a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:53.087537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad3141e2-3a20-4738-8883-7049f297a90a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:53.087985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad3141e2-3a20-4738-8883-7049f297a90a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:53.088733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad3141e2-3a20-4738-8883-7049f297a90a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:53.089200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad3141e2-3a20-4738-8883-7049f297a90a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:53.089632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad3141e2-3a20-4738-8883-7049f297a90a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:53.256504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ad3141e2-3a20-4738-8883-7049f297a90a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:53.274056 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ad3141e2-3a20-4738-8883-7049f297a90a tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1299 microversion: 2.1 time: 0.213892 Sep 27 10:20:53.276317 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2305/5046] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:53 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1299 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:20:53.534692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:53.561142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:53.566949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:53.571942 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0ec82e37-b386-4d08-b882-22f647f7d985 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1723 microversion: 2.20 time: 0.612032 Sep 27 10:20:53.573018 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2742/5047] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:52 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1723 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:54.311016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-be3f6dfb-7eb6-4188-b6a5-2bc904b736aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:54.312381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-be3f6dfb-7eb6-4188-b6a5-2bc904b736aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:54.328784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be3f6dfb-7eb6-4188-b6a5-2bc904b736aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:54.329208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be3f6dfb-7eb6-4188-b6a5-2bc904b736aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:54.330657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be3f6dfb-7eb6-4188-b6a5-2bc904b736aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:54.332680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be3f6dfb-7eb6-4188-b6a5-2bc904b736aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:54.333431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be3f6dfb-7eb6-4188-b6a5-2bc904b736aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:54.334042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be3f6dfb-7eb6-4188-b6a5-2bc904b736aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:54.600848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:54.602202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:54.609180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:20:54.629382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:54.631572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:54.633990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:54.635389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:54.638004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:54.638902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:54.741120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-be3f6dfb-7eb6-4188-b6a5-2bc904b736aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:54.775653 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-be3f6dfb-7eb6-4188-b6a5-2bc904b736aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1305 microversion: 2.1 time: 0.469759 Sep 27 10:20:54.788697 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2306/5048] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:54 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1305 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:55.337357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:55.354138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:55.355634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:55.356807 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af6d81f9-e2b9-42fe-9152-1f49c37b5e1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1723 microversion: 2.20 time: 0.761109 Sep 27 10:20:55.364523 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2743/5049] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:54 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1723 bytes in 769 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:55.809235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-52adbfdb-e362-4ccb-a230-117c9e96a477 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:55.811060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-52adbfdb-e362-4ccb-a230-117c9e96a477 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:55.829011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52adbfdb-e362-4ccb-a230-117c9e96a477 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:55.831414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52adbfdb-e362-4ccb-a230-117c9e96a477 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:55.833984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52adbfdb-e362-4ccb-a230-117c9e96a477 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:55.835939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52adbfdb-e362-4ccb-a230-117c9e96a477 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:55.839671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52adbfdb-e362-4ccb-a230-117c9e96a477 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:55.840526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-52adbfdb-e362-4ccb-a230-117c9e96a477 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:56.026190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-52adbfdb-e362-4ccb-a230-117c9e96a477 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:56.055526 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-52adbfdb-e362-4ccb-a230-117c9e96a477 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1305 microversion: 2.1 time: 0.253261 Sep 27 10:20:56.056514 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2307/5050] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:55 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1305 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:56.391748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:56.392351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:56.403845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:56.404367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:56.404800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:56.405526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:56.407821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:56.408448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:56.851712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:56.870260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:56.871708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:56.884397 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a09a2316-8f5a-4ab1-8511-8387e2985e28 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1723 microversion: 2.20 time: 0.496483 Sep 27 10:20:56.887965 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2744/5051] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:56 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1723 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:20:57.084416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f3587477-da48-4cf5-b859-1d3a9e8e9bb6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:57.084989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f3587477-da48-4cf5-b859-1d3a9e8e9bb6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:57.103385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3587477-da48-4cf5-b859-1d3a9e8e9bb6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:57.103910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3587477-da48-4cf5-b859-1d3a9e8e9bb6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:57.104363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3587477-da48-4cf5-b859-1d3a9e8e9bb6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:57.105335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3587477-da48-4cf5-b859-1d3a9e8e9bb6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:57.105649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3587477-da48-4cf5-b859-1d3a9e8e9bb6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:57.106086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3587477-da48-4cf5-b859-1d3a9e8e9bb6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:57.918982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:57.919607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:57.932535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:57.933082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:57.933535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:57.934278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:57.934749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:57.935189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:58.080814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f3587477-da48-4cf5-b859-1d3a9e8e9bb6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:58.093480 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f3587477-da48-4cf5-b859-1d3a9e8e9bb6 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1372 microversion: 2.1 time: 1.013755 Sep 27 10:20:58.113173 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2308/5052] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:57 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1372 bytes in 1034 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:20:58.513423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:58.526626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:58.534881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:58.536087 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-991ef3ad-2636-47b2-ab7a-6f331120cef0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1723 microversion: 2.20 time: 0.621179 Sep 27 10:20:58.537356 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2745/5053] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:57 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1723 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:20:58.860286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-27caaf62-e94e-4b6c-92fa-ddab08dc7d67 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'create', calling method: .version_select at 0x7f7f625ffee0>, body: {"keypair": {"name": "tempest-keypair-1650140436", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBN2B4veJfUbyeUskE17OvhbdFsN/YTzZaw3EyR1LJMwFUw5k1ovmvvI6+dXOdOjp0/p4foFrXCOU72OJOF5R0LR8wdF514at6SvStbXXFFLiB4DoxhX3X72wK+iogQkoEA=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:20:58.896186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-27caaf62-e94e-4b6c-92fa-ddab08dc7d67 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for project 10375bd836f84383b39f2df73f53800e. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:20:58.918375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-27caaf62-e94e-4b6c-92fa-ddab08dc7d67 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for user 97c769dbba794531b0a9db8e764ff097 and project 10375bd836f84383b39f2df73f53800e. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:20:58.972299 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-27caaf62-e94e-4b6c-92fa-ddab08dc7d67 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.697566 Sep 27 10:20:58.973333 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2309/5054] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:20:58 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 699 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:20:59.122953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f9e9309-a39d-454c-ba75-994e9091cbe1 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:59.123903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8f9e9309-a39d-454c-ba75-994e9091cbe1 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:59.141583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f9e9309-a39d-454c-ba75-994e9091cbe1 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:59.142515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f9e9309-a39d-454c-ba75-994e9091cbe1 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:59.153454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f9e9309-a39d-454c-ba75-994e9091cbe1 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:59.154610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f9e9309-a39d-454c-ba75-994e9091cbe1 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:59.155275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f9e9309-a39d-454c-ba75-994e9091cbe1 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:59.155920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f9e9309-a39d-454c-ba75-994e9091cbe1 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:59.575528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:20:59.577417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:20:59.603718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:59.613476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:59.618558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:59.620179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:20:59.625501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:20:59.631630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:20:59.894724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8f9e9309-a39d-454c-ba75-994e9091cbe1 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:20:59.919146 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8f9e9309-a39d-454c-ba75-994e9091cbe1 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1372 microversion: 2.1 time: 0.786454 Sep 27 10:20:59.920878 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2746/5055] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:20:59 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1372 bytes in 804 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:00.124894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d909fd15-bbf1-4bd2-9e71-69da53284132 req-e24cc483-5cd8-4403-a9be-56ae1822cc2b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5f899983-46ec-426e-b8bd-418768296d1a", "tag": "506eabfb-3d01-4318-9a51-6b79cfefff6c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:00.160587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d909fd15-bbf1-4bd2-9e71-69da53284132 req-e24cc483-5cd8-4403-a9be-56ae1822cc2b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:00.161413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d909fd15-bbf1-4bd2-9e71-69da53284132 req-e24cc483-5cd8-4403-a9be-56ae1822cc2b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:00.162678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d909fd15-bbf1-4bd2-9e71-69da53284132 req-e24cc483-5cd8-4403-a9be-56ae1822cc2b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:00.191740 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d909fd15-bbf1-4bd2-9e71-69da53284132 req-e24cc483-5cd8-4403-a9be-56ae1822cc2b service nova] Creating event network-changed:506eabfb-3d01-4318-9a51-6b79cfefff6c for instance 5f899983-46ec-426e-b8bd-418768296d1a on nch2-master-1 Sep 27 10:21:00.207271 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d909fd15-bbf1-4bd2-9e71-69da53284132 req-e24cc483-5cd8-4403-a9be-56ae1822cc2b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.085969 Sep 27 10:21:00.209256 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2747/5056] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:21:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:21:00.213377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:00.228532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:00.230159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:00.231405 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b743968f-0ce1-4e75-934b-bc8694c3a0f1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1744 microversion: 2.20 time: 0.663831 Sep 27 10:21:00.237464 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2310/5057] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:20:59 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1744 bytes in 670 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:00.950486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3df49e18-87ab-46aa-902e-ee25345e11f9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:00.952015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3df49e18-87ab-46aa-902e-ee25345e11f9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:00.963599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3df49e18-87ab-46aa-902e-ee25345e11f9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:00.965124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3df49e18-87ab-46aa-902e-ee25345e11f9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:00.966262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3df49e18-87ab-46aa-902e-ee25345e11f9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:00.968557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3df49e18-87ab-46aa-902e-ee25345e11f9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:00.969487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3df49e18-87ab-46aa-902e-ee25345e11f9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:00.970181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3df49e18-87ab-46aa-902e-ee25345e11f9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:01.290034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:01.292473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:01.328874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:01.329386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:01.329839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:01.330914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:01.331376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:01.331810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:01.693787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:02.336949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:02.354497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:02.361148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:02.367381 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3010e648-3c0c-495a-b1fd-554fabdf5c13 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 1.105786 Sep 27 10:21:02.370988 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2311/5058] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:01 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 1110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:02.381077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ab2745c0-43fb-4cb5-81af-57154609733b req-1354267e-6e50-4a47-bb29-edab978fb106 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "691b2be7-d5b8-409c-ba22-5ab8da50b80d", "name": "network-vif-plugged", "status": "completed", "tag": "11b232b4-3236-4917-9ebb-02d7dbd3132e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:02.411814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ab2745c0-43fb-4cb5-81af-57154609733b req-1354267e-6e50-4a47-bb29-edab978fb106 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:02.412626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ab2745c0-43fb-4cb5-81af-57154609733b req-1354267e-6e50-4a47-bb29-edab978fb106 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:02.417559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ab2745c0-43fb-4cb5-81af-57154609733b req-1354267e-6e50-4a47-bb29-edab978fb106 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:02.455684 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ab2745c0-43fb-4cb5-81af-57154609733b req-1354267e-6e50-4a47-bb29-edab978fb106 service nova] Creating event network-vif-plugged:11b232b4-3236-4917-9ebb-02d7dbd3132e for instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d on nch1-master-1 Sep 27 10:21:02.472996 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ab2745c0-43fb-4cb5-81af-57154609733b req-1354267e-6e50-4a47-bb29-edab978fb106 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100111 Sep 27 10:21:02.473580 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2312/5059] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:21:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:21:02.630378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3df49e18-87ab-46aa-902e-ee25345e11f9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:02.652933 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3df49e18-87ab-46aa-902e-ee25345e11f9 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1372 microversion: 2.1 time: 1.708048 Sep 27 10:21:02.657791 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2748/5060] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:00 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1372 bytes in 1714 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:03.399943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:03.401348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:03.425004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:03.425515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:03.425972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:03.434120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:03.434759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:03.435218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:03.681555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-10288f9b-4ba5-4592-82d7-59e0434adc51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:03.682191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-10288f9b-4ba5-4592-82d7-59e0434adc51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:03.694217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10288f9b-4ba5-4592-82d7-59e0434adc51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:03.694720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10288f9b-4ba5-4592-82d7-59e0434adc51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:03.695164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10288f9b-4ba5-4592-82d7-59e0434adc51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:03.695900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10288f9b-4ba5-4592-82d7-59e0434adc51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:03.696365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10288f9b-4ba5-4592-82d7-59e0434adc51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:03.712224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10288f9b-4ba5-4592-82d7-59e0434adc51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:04.302469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-10288f9b-4ba5-4592-82d7-59e0434adc51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:04.316992 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-10288f9b-4ba5-4592-82d7-59e0434adc51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1372 microversion: 2.1 time: 0.637001 Sep 27 10:21:04.318220 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2749/5061] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:03 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1372 bytes in 642 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:05.061988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:05.068588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:05.076730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:05.085826 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4da4f4b3-4b7f-4e72-9165-6f3408c7dc74 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 1.687058 Sep 27 10:21:05.089574 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2313/5062] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:03 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 1695 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:05.346796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e998e58-4a1d-4702-9c60-842009f93b94 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:05.348031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2e998e58-4a1d-4702-9c60-842009f93b94 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:05.362591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e998e58-4a1d-4702-9c60-842009f93b94 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:05.363294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e998e58-4a1d-4702-9c60-842009f93b94 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:05.364613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e998e58-4a1d-4702-9c60-842009f93b94 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:05.365733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e998e58-4a1d-4702-9c60-842009f93b94 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:05.366387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e998e58-4a1d-4702-9c60-842009f93b94 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:05.372431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e998e58-4a1d-4702-9c60-842009f93b94 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:06.116725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:06.117344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:06.137772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:06.143477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:06.144286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:06.147253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:06.151004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:06.152319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:07.033819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2e998e58-4a1d-4702-9c60-842009f93b94 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:07.047381 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2e998e58-4a1d-4702-9c60-842009f93b94 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1372 microversion: 2.1 time: 1.705610 Sep 27 10:21:07.073802 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2750/5063] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:05 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1372 bytes in 1733 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:07.655030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:07.677688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:07.683396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:07.685871 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8088590f-a6ef-47fc-b30d-8fae2e1e495c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 1.574531 Sep 27 10:21:07.690327 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2314/5064] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:06 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 1579 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:07.774968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-497443647", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "247e3d7d-5912-42d7-9027-ebcb26003cfc"}], "security_groups": [{"name": "tempest-securitygroup--631962135"}], "key_name": "tempest-keypair-1650140436", "user_data": "}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:08.007698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] validate_networks() for [('247e3d7d-5912-42d7-9027-ebcb26003cfc', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:21:08.084570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f3e2098c-cb6a-4e2f-99df-fbca87508c3d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:08.087365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f3e2098c-cb6a-4e2f-99df-fbca87508c3d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:08.117029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e2098c-cb6a-4e2f-99df-fbca87508c3d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:08.119104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e2098c-cb6a-4e2f-99df-fbca87508c3d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:08.122506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e2098c-cb6a-4e2f-99df-fbca87508c3d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:08.125055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e2098c-cb6a-4e2f-99df-fbca87508c3d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:08.127341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e2098c-cb6a-4e2f-99df-fbca87508c3d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:08.131912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3e2098c-cb6a-4e2f-99df-fbca87508c3d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:08.939855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=12 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:09.061199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:21:09.062625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:21:09.063325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:21:09.063959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:21:09.064580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:21:09.086066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:09.090302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:09.097613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:09.098901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:09.103516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:09.104293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:09.173330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for project 10375bd836f84383b39f2df73f53800e. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:21:09.199995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for user 97c769dbba794531b0a9db8e764ff097 and project 10375bd836f84383b39f2df73f53800e. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:21:09.240755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:21:09.249214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:21:09.347909 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6b44ee93-d3e4-4bd4-a6d0-5dab852690a9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 1.577300 Sep 27 10:21:09.351036 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2751/5065] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:21:07 2022] POST /compute/v2.1/servers => generated 408 bytes in 1581 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:21:09.365466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-2093ba4e-e1d8-403d-b1b9-63c6a7e6bb6b req-1cc4224f-66d3-4d53-bf48-2cf7e11c895a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5f899983-46ec-426e-b8bd-418768296d1a", "name": "network-vif-plugged", "status": "completed", "tag": "506eabfb-3d01-4318-9a51-6b79cfefff6c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:09.398418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2093ba4e-e1d8-403d-b1b9-63c6a7e6bb6b req-1cc4224f-66d3-4d53-bf48-2cf7e11c895a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:09.400839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2093ba4e-e1d8-403d-b1b9-63c6a7e6bb6b req-1cc4224f-66d3-4d53-bf48-2cf7e11c895a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:09.403604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2093ba4e-e1d8-403d-b1b9-63c6a7e6bb6b req-1cc4224f-66d3-4d53-bf48-2cf7e11c895a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:09.436368 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-2093ba4e-e1d8-403d-b1b9-63c6a7e6bb6b req-1cc4224f-66d3-4d53-bf48-2cf7e11c895a service nova] Creating event network-vif-plugged:506eabfb-3d01-4318-9a51-6b79cfefff6c for instance 5f899983-46ec-426e-b8bd-418768296d1a on nch2-master-1 Sep 27 10:21:09.448739 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-2093ba4e-e1d8-403d-b1b9-63c6a7e6bb6b req-1cc4224f-66d3-4d53-bf48-2cf7e11c895a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.096345 Sep 27 10:21:09.452434 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2752/5066] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:21:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:21:09.462633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:09.463801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:09.472152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:09.484124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:09.484803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:09.485830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:09.488412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:09.490594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:09.660797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f3e2098c-cb6a-4e2f-99df-fbca87508c3d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:09.673268 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f3e2098c-cb6a-4e2f-99df-fbca87508c3d tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1372 microversion: 2.1 time: 1.591200 Sep 27 10:21:09.676675 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2315/5067] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:08 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1372 bytes in 1597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:09.717709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d69b0041-18ca-4ede-8a3b-4834cbfb7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:09.718411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d69b0041-18ca-4ede-8a3b-4834cbfb7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:09.959834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:09.972283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:09.976041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:09.988076 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4435f465-90ca-4ddd-9e0c-745fcd2594d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 0.525539 Sep 27 10:21:09.992684 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2753/5068] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:09 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:10.408210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d69b0041-18ca-4ede-8a3b-4834cbfb7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:10.420680 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d69b0041-18ca-4ede-8a3b-4834cbfb7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1238 microversion: 2.1 time: 0.742736 Sep 27 10:21:10.426435 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2316/5069] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:09 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1238 bytes in 749 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:10.698220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9477fc2-b973-44ac-a6f9-f1a413d48a51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:10.699809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b9477fc2-b973-44ac-a6f9-f1a413d48a51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:10.721069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9477fc2-b973-44ac-a6f9-f1a413d48a51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:10.722504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9477fc2-b973-44ac-a6f9-f1a413d48a51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:10.725298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9477fc2-b973-44ac-a6f9-f1a413d48a51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:10.731621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9477fc2-b973-44ac-a6f9-f1a413d48a51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:10.734841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9477fc2-b973-44ac-a6f9-f1a413d48a51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:10.735641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9477fc2-b973-44ac-a6f9-f1a413d48a51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:11.018356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:11.019288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:11.036375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:11.037352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:11.039569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:11.041784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:11.042982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:11.043435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:11.428898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b9477fc2-b973-44ac-a6f9-f1a413d48a51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:11.457036 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b9477fc2-b973-44ac-a6f9-f1a413d48a51 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1372 microversion: 2.1 time: 0.762336 Sep 27 10:21:11.458058 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2754/5070] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:10 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1372 bytes in 767 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:11.470008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9ae44419-e9e3-490e-9861-bce1bde2f2e8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:11.475836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ae44419-e9e3-490e-9861-bce1bde2f2e8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:11.528095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ae44419-e9e3-490e-9861-bce1bde2f2e8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:11.530180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ae44419-e9e3-490e-9861-bce1bde2f2e8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:11.531371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ae44419-e9e3-490e-9861-bce1bde2f2e8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:11.538460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ae44419-e9e3-490e-9861-bce1bde2f2e8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:11.539232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ae44419-e9e3-490e-9861-bce1bde2f2e8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:11.543223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ae44419-e9e3-490e-9861-bce1bde2f2e8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:11.680474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:11.953578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ae44419-e9e3-490e-9861-bce1bde2f2e8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:11.985197 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9ae44419-e9e3-490e-9861-bce1bde2f2e8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1290 microversion: 2.1 time: 0.525116 Sep 27 10:21:11.986244 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2755/5071] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:11 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1290 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:12.441660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:12.463729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:12.470478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:12.477959 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-779703dd-0db2-430a-96fe-d3c24e8bc724 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 1.464698 Sep 27 10:21:12.481366 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2317/5072] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:11 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 1469 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:21:12.500931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f028d597-28f2-4309-a458-9fb6de5a85aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:12.501644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f028d597-28f2-4309-a458-9fb6de5a85aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:12.511225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f028d597-28f2-4309-a458-9fb6de5a85aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:12.513537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f028d597-28f2-4309-a458-9fb6de5a85aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:12.518751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f028d597-28f2-4309-a458-9fb6de5a85aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:12.525177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f028d597-28f2-4309-a458-9fb6de5a85aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:12.526931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f028d597-28f2-4309-a458-9fb6de5a85aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:12.528254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f028d597-28f2-4309-a458-9fb6de5a85aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:13.016182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d3a65089-dff0-4139-aa97-712028d4aa2a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:13.018066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d3a65089-dff0-4139-aa97-712028d4aa2a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:13.028976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3a65089-dff0-4139-aa97-712028d4aa2a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:13.029761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3a65089-dff0-4139-aa97-712028d4aa2a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:13.029917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3a65089-dff0-4139-aa97-712028d4aa2a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:13.032999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3a65089-dff0-4139-aa97-712028d4aa2a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:13.035101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3a65089-dff0-4139-aa97-712028d4aa2a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:13.035462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3a65089-dff0-4139-aa97-712028d4aa2a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:13.079544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f028d597-28f2-4309-a458-9fb6de5a85aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:13.100523 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f028d597-28f2-4309-a458-9fb6de5a85aa tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1543 microversion: 2.1 time: 0.611552 Sep 27 10:21:13.101266 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2756/5073] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:12 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1543 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:13.128207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f262ab5c-5615-4bb9-9312-421078fd3c33 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:13.129398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f262ab5c-5615-4bb9-9312-421078fd3c33 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:13.144909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f262ab5c-5615-4bb9-9312-421078fd3c33 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:13.148476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f262ab5c-5615-4bb9-9312-421078fd3c33 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:13.152147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f262ab5c-5615-4bb9-9312-421078fd3c33 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:13.159933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f262ab5c-5615-4bb9-9312-421078fd3c33 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:13.161446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f262ab5c-5615-4bb9-9312-421078fd3c33 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:13.164151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f262ab5c-5615-4bb9-9312-421078fd3c33 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:13.439082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d3a65089-dff0-4139-aa97-712028d4aa2a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:13.452421 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d3a65089-dff0-4139-aa97-712028d4aa2a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1296 microversion: 2.1 time: 0.440102 Sep 27 10:21:13.455882 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2318/5074] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:13 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1296 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:13.509654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:13.510727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:13.519662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:13.523193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:13.523862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:13.528108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:13.529809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:13.530968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:13.656687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f262ab5c-5615-4bb9-9312-421078fd3c33 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:13.681346 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f262ab5c-5615-4bb9-9312-421078fd3c33 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1543 microversion: 2.1 time: 0.557837 Sep 27 10:21:13.695553 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2757/5075] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:13 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1543 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:14.478901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-453efe39-b2a0-4347-81ca-793617f15843 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:14.479698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-453efe39-b2a0-4347-81ca-793617f15843 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:14.499602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-453efe39-b2a0-4347-81ca-793617f15843 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:14.500172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-453efe39-b2a0-4347-81ca-793617f15843 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:14.500622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-453efe39-b2a0-4347-81ca-793617f15843 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:14.506581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-453efe39-b2a0-4347-81ca-793617f15843 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:14.513467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-453efe39-b2a0-4347-81ca-793617f15843 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:14.514377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-453efe39-b2a0-4347-81ca-793617f15843 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:14.720249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-453efe39-b2a0-4347-81ca-793617f15843 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:14.729608 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-453efe39-b2a0-4347-81ca-793617f15843 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1296 microversion: 2.1 time: 0.254805 Sep 27 10:21:14.735498 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2758/5076] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:14 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1296 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:21:15.113606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:15.143477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:15.148019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:15.157280 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e22d5370-a013-4183-b519-ebe81498d3f2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 1.646681 Sep 27 10:21:15.168942 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2319/5077] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:13 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 1665 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:15.761684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d75068da-22fc-4df4-a3d5-a630e9ff76fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:15.764694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d75068da-22fc-4df4-a3d5-a630e9ff76fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:15.769177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=65,nova_api:INSERT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:15.784010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75068da-22fc-4df4-a3d5-a630e9ff76fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:15.784724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75068da-22fc-4df4-a3d5-a630e9ff76fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:15.785430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75068da-22fc-4df4-a3d5-a630e9ff76fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:15.787972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75068da-22fc-4df4-a3d5-a630e9ff76fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:15.793178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75068da-22fc-4df4-a3d5-a630e9ff76fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:15.794170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75068da-22fc-4df4-a3d5-a630e9ff76fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:16.025167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d75068da-22fc-4df4-a3d5-a630e9ff76fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:16.035981 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d75068da-22fc-4df4-a3d5-a630e9ff76fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1296 microversion: 2.1 time: 0.279563 Sep 27 10:21:16.043871 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2759/5078] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:15 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1296 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:16.191368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:16.192010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:16.219316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:16.220449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:16.220449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:16.221002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:16.221467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:16.221900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:16.949962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:16.959136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:16.960644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:16.968871 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-97b89152-43a5-4e02-aada-400849c6e186 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 0.774806 Sep 27 10:21:16.980879 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2320/5079] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:16 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 794 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:17.071360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d656fcf-4fd0-4818-9750-e9a89b3a8c80 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:17.072822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d656fcf-4fd0-4818-9750-e9a89b3a8c80 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:17.090792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d656fcf-4fd0-4818-9750-e9a89b3a8c80 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:17.094212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d656fcf-4fd0-4818-9750-e9a89b3a8c80 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:17.097207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d656fcf-4fd0-4818-9750-e9a89b3a8c80 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:17.097779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-server-smoke-262940321", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "465eb0c2-ba81-45cd-8471-eb7325008a85"}], "key_name": "tempest-TestSecurityGroupsBasicOps-1627558846", "security_groups": []}, "os:scheduler_hints": {"different_host": ["3bba7aa2-0a3a-42f0-b00c-93ac58368945"]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:17.101127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d656fcf-4fd0-4818-9750-e9a89b3a8c80 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:17.105721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d656fcf-4fd0-4818-9750-e9a89b3a8c80 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:17.106290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d656fcf-4fd0-4818-9750-e9a89b3a8c80 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:17.211128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] validate_networks() for [('465eb0c2-ba81-45cd-8471-eb7325008a85', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:21:17.658140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1d656fcf-4fd0-4818-9750-e9a89b3a8c80 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:17.666642 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d656fcf-4fd0-4818-9750-e9a89b3a8c80 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1363 microversion: 2.1 time: 0.600331 Sep 27 10:21:17.687166 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2760/5080] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:17 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1363 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:18.006426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:18.007046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:18.025486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:18.026003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:18.026459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:18.027205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:18.027670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:18.028101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:18.561775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:18.573401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:18.581441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:18.582950 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e22cbbcd-2064-439c-8966-842449aca9e1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 0.580849 Sep 27 10:21:18.590401 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2761/5081] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:18 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:18.695302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-574b37b5-f884-4e1f-aafa-6e71b924c0e7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:18.696063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-574b37b5-f884-4e1f-aafa-6e71b924c0e7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:18.701327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:21:18.701959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:21:18.702438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:21:18.703106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:21:18.703932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:21:18.707666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=71,nova_api:INSERT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:18.709365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-574b37b5-f884-4e1f-aafa-6e71b924c0e7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:18.712922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-574b37b5-f884-4e1f-aafa-6e71b924c0e7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:18.713757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-574b37b5-f884-4e1f-aafa-6e71b924c0e7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:18.717990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-574b37b5-f884-4e1f-aafa-6e71b924c0e7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:18.719963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-574b37b5-f884-4e1f-aafa-6e71b924c0e7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:18.720586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-574b37b5-f884-4e1f-aafa-6e71b924c0e7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:18.737291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:18.738310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:18.743308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:18.744682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:18.745776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:18.746433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:18.759793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=93 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:18.772959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Getting quotas for project 932da48df2c44fc6a7dbd6c8daf9fc0d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:21:18.781522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Getting quotas for user 64c9f44e57164b858421b55114acf4cf and project 932da48df2c44fc6a7dbd6c8daf9fc0d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:21:18.800012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:21:18.802926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:21:18.900770 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0b2e0b97-1315-48f2-bc0b-20e4fc283f9d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 361 microversion: 2.1 time: 1.807224 Sep 27 10:21:18.903499 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2321/5082] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:21:17 2022] POST /compute/v2.1/servers => generated 361 bytes in 1811 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:21:18.940062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-54ca454a-efd0-4a89-877d-8237055153d5 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:18.944163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-54ca454a-efd0-4a89-877d-8237055153d5 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:19.092303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-54ca454a-efd0-4a89-877d-8237055153d5 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:19.106486 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-54ca454a-efd0-4a89-877d-8237055153d5 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1241 microversion: 2.1 time: 0.195261 Sep 27 10:21:19.110606 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2322/5083] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:18 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1241 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:19.137671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:19.219722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:19.269400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1b257c16-04b7-432e-9361-9e0caaa91781 req-11cbf5e3-76c8-475d-aa9e-0041666d8c39 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5ac6dabf-1315-4e05-ab6c-22211dc2440b", "tag": "b894a455-9bb5-443a-84fe-e04c8d0c3e28"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:19.322341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1b257c16-04b7-432e-9361-9e0caaa91781 req-11cbf5e3-76c8-475d-aa9e-0041666d8c39 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:19.322895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1b257c16-04b7-432e-9361-9e0caaa91781 req-11cbf5e3-76c8-475d-aa9e-0041666d8c39 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:19.325389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1b257c16-04b7-432e-9361-9e0caaa91781 req-11cbf5e3-76c8-475d-aa9e-0041666d8c39 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:19.373202 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1b257c16-04b7-432e-9361-9e0caaa91781 req-11cbf5e3-76c8-475d-aa9e-0041666d8c39 service nova] Creating event network-changed:b894a455-9bb5-443a-84fe-e04c8d0c3e28 for instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b on nch1-master-1 Sep 27 10:21:19.385290 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1b257c16-04b7-432e-9361-9e0caaa91781 req-11cbf5e3-76c8-475d-aa9e-0041666d8c39 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.122219 Sep 27 10:21:19.385916 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2323/5084] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:21:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:21:19.630897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:19.631723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:19.650777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:19.651349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:19.651815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:19.652563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:19.660034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:19.661611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:20.070295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:20.084402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:20.085907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:20.087139 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1d4e911e-eb3d-4c10-9d68-18befe50d84c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 0.467739 Sep 27 10:21:20.101289 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2324/5085] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:19 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:20.133980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2759ccd1-d9e4-468c-b8e1-58860bef56d6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:20.135780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2759ccd1-d9e4-468c-b8e1-58860bef56d6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:20.165304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2759ccd1-d9e4-468c-b8e1-58860bef56d6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:20.166926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2759ccd1-d9e4-468c-b8e1-58860bef56d6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:20.171835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2759ccd1-d9e4-468c-b8e1-58860bef56d6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:20.172960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2759ccd1-d9e4-468c-b8e1-58860bef56d6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:20.176453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2759ccd1-d9e4-468c-b8e1-58860bef56d6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:20.180765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2759ccd1-d9e4-468c-b8e1-58860bef56d6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:20.293652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2759ccd1-d9e4-468c-b8e1-58860bef56d6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:20.309201 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2759ccd1-d9e4-468c-b8e1-58860bef56d6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1245 microversion: 2.1 time: 0.179701 Sep 27 10:21:20.310779 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2325/5086] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:20 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1245 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:21:20.375413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-574b37b5-f884-4e1f-aafa-6e71b924c0e7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:20.388934 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-574b37b5-f884-4e1f-aafa-6e71b924c0e7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1363 microversion: 2.1 time: 1.697986 Sep 27 10:21:20.394677 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2762/5087] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:18 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1363 bytes in 1704 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:21.135650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:21.136436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:21.167672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:21.168312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:21.170861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:21.172292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:21.175480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:21.176294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:21.336984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4b321eb6-e0e6-4ca4-b02e-2bd23dc0dcb6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:21.337646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4b321eb6-e0e6-4ca4-b02e-2bd23dc0dcb6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:21.365921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b321eb6-e0e6-4ca4-b02e-2bd23dc0dcb6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:21.372273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b321eb6-e0e6-4ca4-b02e-2bd23dc0dcb6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:21.375918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b321eb6-e0e6-4ca4-b02e-2bd23dc0dcb6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:21.380058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b321eb6-e0e6-4ca4-b02e-2bd23dc0dcb6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:21.388337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b321eb6-e0e6-4ca4-b02e-2bd23dc0dcb6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:21.388850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b321eb6-e0e6-4ca4-b02e-2bd23dc0dcb6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:22.015930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4b321eb6-e0e6-4ca4-b02e-2bd23dc0dcb6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:22.051319 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4b321eb6-e0e6-4ca4-b02e-2bd23dc0dcb6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1301 microversion: 2.1 time: 0.718863 Sep 27 10:21:22.069081 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2763/5088] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:21 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1301 bytes in 737 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:22.087903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-b7bc9190-87dd-4a6a-835d-e907fd926489 req-4b78c11f-1385-4024-9386-3674ef343762 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5f899983-46ec-426e-b8bd-418768296d1a", "tag": "506eabfb-3d01-4318-9a51-6b79cfefff6c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:22.126527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b7bc9190-87dd-4a6a-835d-e907fd926489 req-4b78c11f-1385-4024-9386-3674ef343762 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:22.127043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b7bc9190-87dd-4a6a-835d-e907fd926489 req-4b78c11f-1385-4024-9386-3674ef343762 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:22.127508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b7bc9190-87dd-4a6a-835d-e907fd926489 req-4b78c11f-1385-4024-9386-3674ef343762 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:22.163257 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b7bc9190-87dd-4a6a-835d-e907fd926489 req-4b78c11f-1385-4024-9386-3674ef343762 service nova] Creating event network-changed:506eabfb-3d01-4318-9a51-6b79cfefff6c for instance 5f899983-46ec-426e-b8bd-418768296d1a on nch2-master-1 Sep 27 10:21:22.210436 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-b7bc9190-87dd-4a6a-835d-e907fd926489 req-4b78c11f-1385-4024-9386-3674ef343762 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.127281 Sep 27 10:21:22.212475 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2764/5089] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:21:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:21:22.221095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-50ddd1f1-cb60-4c94-aae2-7f7da55811ea tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:22.222831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-50ddd1f1-cb60-4c94-aae2-7f7da55811ea tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:22.235443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50ddd1f1-cb60-4c94-aae2-7f7da55811ea tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:22.238767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50ddd1f1-cb60-4c94-aae2-7f7da55811ea tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:22.239796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50ddd1f1-cb60-4c94-aae2-7f7da55811ea tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:22.240948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50ddd1f1-cb60-4c94-aae2-7f7da55811ea tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:22.241726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50ddd1f1-cb60-4c94-aae2-7f7da55811ea tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:22.242896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50ddd1f1-cb60-4c94-aae2-7f7da55811ea tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:22.835001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-50ddd1f1-cb60-4c94-aae2-7f7da55811ea tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:22.843543 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-50ddd1f1-cb60-4c94-aae2-7f7da55811ea tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1363 microversion: 2.1 time: 0.628789 Sep 27 10:21:22.849624 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2765/5090] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:22 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1363 bytes in 636 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:23.079403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e44bbcb-58ad-4af8-80f3-aabd80f22331 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:23.080027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8e44bbcb-58ad-4af8-80f3-aabd80f22331 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:23.099992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e44bbcb-58ad-4af8-80f3-aabd80f22331 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:23.100499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e44bbcb-58ad-4af8-80f3-aabd80f22331 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:23.100976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e44bbcb-58ad-4af8-80f3-aabd80f22331 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:23.101717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e44bbcb-58ad-4af8-80f3-aabd80f22331 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:23.105362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e44bbcb-58ad-4af8-80f3-aabd80f22331 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:23.105878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e44bbcb-58ad-4af8-80f3-aabd80f22331 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:23.110326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:23.130700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:23.145182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:23.147302 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5975a15e-1d67-4f40-ad6b-c4e37179859e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 2.035386 Sep 27 10:21:23.149750 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2326/5091] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:21 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 2038 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:23.239471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8e44bbcb-58ad-4af8-80f3-aabd80f22331 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:23.248465 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e44bbcb-58ad-4af8-80f3-aabd80f22331 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.172884 Sep 27 10:21:23.269930 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2766/5092] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:23 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:21:23.875318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1e5d2890-a53e-47d2-8ed1-a332b1e89576 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:23.875946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1e5d2890-a53e-47d2-8ed1-a332b1e89576 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:23.890420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e5d2890-a53e-47d2-8ed1-a332b1e89576 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:23.890929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e5d2890-a53e-47d2-8ed1-a332b1e89576 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:23.892962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e5d2890-a53e-47d2-8ed1-a332b1e89576 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:23.896825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e5d2890-a53e-47d2-8ed1-a332b1e89576 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:23.897967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e5d2890-a53e-47d2-8ed1-a332b1e89576 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:23.904952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1e5d2890-a53e-47d2-8ed1-a332b1e89576 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:24.175763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:24.176385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:24.194458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:24.194956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:24.197702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:24.199616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:24.200297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:24.200978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:25.266857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1e5d2890-a53e-47d2-8ed1-a332b1e89576 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:25.278250 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1e5d2890-a53e-47d2-8ed1-a332b1e89576 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1363 microversion: 2.1 time: 1.407116 Sep 27 10:21:25.290534 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2327/5093] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:23 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1363 bytes in 1417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:25.316754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c2a8e472-994c-47d1-b790-6b7727ad9775 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:25.317851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c2a8e472-994c-47d1-b790-6b7727ad9775 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:25.344323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:25.351735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a8e472-994c-47d1-b790-6b7727ad9775 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:25.362951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:25.368178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a8e472-994c-47d1-b790-6b7727ad9775 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:25.369141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:25.369778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a8e472-994c-47d1-b790-6b7727ad9775 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:25.373342 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6ca975cd-5a1f-4de2-9317-a803ddd0f621 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 1.201784 Sep 27 10:21:25.374307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a8e472-994c-47d1-b790-6b7727ad9775 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:25.375042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a8e472-994c-47d1-b790-6b7727ad9775 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:25.376002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2a8e472-994c-47d1-b790-6b7727ad9775 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:25.377928 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2767/5094] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:24 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 1207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:25.606969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c2a8e472-994c-47d1-b790-6b7727ad9775 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:25.620401 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c2a8e472-994c-47d1-b790-6b7727ad9775 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.326618 Sep 27 10:21:25.622121 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2328/5095] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:25 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:25.829867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8af34225-553f-4765-97ba-06a83ba90b66 req-f04213d1-fbc5-4df2-a639-7993498894b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ac6dabf-1315-4e05-ab6c-22211dc2440b", "name": "network-vif-plugged", "status": "completed", "tag": "b894a455-9bb5-443a-84fe-e04c8d0c3e28"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:25.856326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8af34225-553f-4765-97ba-06a83ba90b66 req-f04213d1-fbc5-4df2-a639-7993498894b6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:25.857120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8af34225-553f-4765-97ba-06a83ba90b66 req-f04213d1-fbc5-4df2-a639-7993498894b6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:25.859138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8af34225-553f-4765-97ba-06a83ba90b66 req-f04213d1-fbc5-4df2-a639-7993498894b6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:25.882836 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8af34225-553f-4765-97ba-06a83ba90b66 req-f04213d1-fbc5-4df2-a639-7993498894b6 service nova] Creating event network-vif-plugged:b894a455-9bb5-443a-84fe-e04c8d0c3e28 for instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b on nch1-master-1 Sep 27 10:21:25.902955 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8af34225-553f-4765-97ba-06a83ba90b66 req-f04213d1-fbc5-4df2-a639-7993498894b6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075732 Sep 27 10:21:25.904029 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2768/5096] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:21:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:21:26.310687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-435aa6c2-efa4-4631-a1ff-c4eb67120d0f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:26.311413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-435aa6c2-efa4-4631-a1ff-c4eb67120d0f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:26.343268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-435aa6c2-efa4-4631-a1ff-c4eb67120d0f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:26.343895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-435aa6c2-efa4-4631-a1ff-c4eb67120d0f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:26.345008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-435aa6c2-efa4-4631-a1ff-c4eb67120d0f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:26.347874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-435aa6c2-efa4-4631-a1ff-c4eb67120d0f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:26.349989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-435aa6c2-efa4-4631-a1ff-c4eb67120d0f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:26.352312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-435aa6c2-efa4-4631-a1ff-c4eb67120d0f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:26.408203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:26.410707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:26.456024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:26.459639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:26.463570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:26.468428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:26.471456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:26.472355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:26.952946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:26.980408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:26.985787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:26.988170 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3bad5592-a669-4030-b42c-9dede67a0e85 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 0.586812 Sep 27 10:21:26.991454 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2769/5097] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:26 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 591 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:21:27.008479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2b7f6c90-fcce-4fe4-ac67-776d2e606d98 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:27.009699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2b7f6c90-fcce-4fe4-ac67-776d2e606d98 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:27.035291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7f6c90-fcce-4fe4-ac67-776d2e606d98 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:27.036483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7f6c90-fcce-4fe4-ac67-776d2e606d98 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:27.037319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7f6c90-fcce-4fe4-ac67-776d2e606d98 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:27.039784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7f6c90-fcce-4fe4-ac67-776d2e606d98 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:27.040527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7f6c90-fcce-4fe4-ac67-776d2e606d98 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:27.042024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b7f6c90-fcce-4fe4-ac67-776d2e606d98 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:27.067578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-435aa6c2-efa4-4631-a1ff-c4eb67120d0f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:27.083504 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-435aa6c2-efa4-4631-a1ff-c4eb67120d0f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1363 microversion: 2.1 time: 0.775871 Sep 27 10:21:27.103828 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2329/5098] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:26 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1363 bytes in 798 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:27.295899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2b7f6c90-fcce-4fe4-ac67-776d2e606d98 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:27.307373 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2b7f6c90-fcce-4fe4-ac67-776d2e606d98 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.313710 Sep 27 10:21:27.308072 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2770/5099] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:26 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:28.029370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:28.030793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:28.058097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:28.063897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:28.064430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:28.073372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:28.073889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:28.074342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:28.108471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f3535d9a-82dc-4116-b56d-d25088e33118 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:28.110075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f3535d9a-82dc-4116-b56d-d25088e33118 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:28.125450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3535d9a-82dc-4116-b56d-d25088e33118 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:28.126365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3535d9a-82dc-4116-b56d-d25088e33118 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:28.127086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3535d9a-82dc-4116-b56d-d25088e33118 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:28.128041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3535d9a-82dc-4116-b56d-d25088e33118 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:28.131944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3535d9a-82dc-4116-b56d-d25088e33118 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:28.132646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f3535d9a-82dc-4116-b56d-d25088e33118 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:28.591268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f3535d9a-82dc-4116-b56d-d25088e33118 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:28.604635 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f3535d9a-82dc-4116-b56d-d25088e33118 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1551 microversion: 2.1 time: 0.499844 Sep 27 10:21:28.605874 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2771/5100] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:28 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1551 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:21:28.611712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6db0ab48-8355-4ac0-aa87-bfac3b089a9e tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:28.612568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6db0ab48-8355-4ac0-aa87-bfac3b089a9e tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:28.620245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db0ab48-8355-4ac0-aa87-bfac3b089a9e tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:28.622796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db0ab48-8355-4ac0-aa87-bfac3b089a9e tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:28.623439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db0ab48-8355-4ac0-aa87-bfac3b089a9e tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:28.624504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db0ab48-8355-4ac0-aa87-bfac3b089a9e tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:28.625853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db0ab48-8355-4ac0-aa87-bfac3b089a9e tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:28.625853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db0ab48-8355-4ac0-aa87-bfac3b089a9e tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:28.761065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:28.793848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6db0ab48-8355-4ac0-aa87-bfac3b089a9e tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:28.811772 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6db0ab48-8355-4ac0-aa87-bfac3b089a9e tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.201125 Sep 27 10:21:28.814518 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2772/5101] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:28 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:28.828385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d3ea854-a10f-40bb-98dc-0e6c51d71db0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:28.829718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d3ea854-a10f-40bb-98dc-0e6c51d71db0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:28.861331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d3ea854-a10f-40bb-98dc-0e6c51d71db0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:28.862748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d3ea854-a10f-40bb-98dc-0e6c51d71db0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:28.867302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d3ea854-a10f-40bb-98dc-0e6c51d71db0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:28.868768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d3ea854-a10f-40bb-98dc-0e6c51d71db0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:28.873550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d3ea854-a10f-40bb-98dc-0e6c51d71db0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:28.874819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d3ea854-a10f-40bb-98dc-0e6c51d71db0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:28.917476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:28.960778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:28.988712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:28.996001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:28.999415 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5192259b-f334-494c-9c79-ad35a6c68f1b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 0.981348 Sep 27 10:21:29.001548 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2330/5102] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:28 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 984 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:29.030524 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d3ea854-a10f-40bb-98dc-0e6c51d71db0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.210188 Sep 27 10:21:29.040203 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2773/5103] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:21:28 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/os-interface => generated 285 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:21:29.685081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-65cf4485-4065-48ce-bc95-eff4981981cd req-2c0effb0-9954-493a-afe4-a230ee9a4136 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3021488b-e2e0-4444-a89c-80de9e14a49f", "tag": "91cd8263-a144-429a-97a4-1338746dd22c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:29.751416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=10 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:29.761835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-65cf4485-4065-48ce-bc95-eff4981981cd req-2c0effb0-9954-493a-afe4-a230ee9a4136 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:29.763899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-65cf4485-4065-48ce-bc95-eff4981981cd req-2c0effb0-9954-493a-afe4-a230ee9a4136 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:29.767549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-65cf4485-4065-48ce-bc95-eff4981981cd req-2c0effb0-9954-493a-afe4-a230ee9a4136 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:29.806773 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-65cf4485-4065-48ce-bc95-eff4981981cd req-2c0effb0-9954-493a-afe4-a230ee9a4136 service nova] Creating event network-changed:91cd8263-a144-429a-97a4-1338746dd22c for instance 3021488b-e2e0-4444-a89c-80de9e14a49f on nch2-master-1 Sep 27 10:21:29.826917 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-65cf4485-4065-48ce-bc95-eff4981981cd req-2c0effb0-9954-493a-afe4-a230ee9a4136 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.147312 Sep 27 10:21:29.836074 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2331/5104] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:21:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:21:29.840766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ea43f1e1-833d-4d48-9a7f-b3ff6d815159 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:29.841959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ea43f1e1-833d-4d48-9a7f-b3ff6d815159 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:29.864550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea43f1e1-833d-4d48-9a7f-b3ff6d815159 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:29.875627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea43f1e1-833d-4d48-9a7f-b3ff6d815159 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:29.876443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea43f1e1-833d-4d48-9a7f-b3ff6d815159 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:29.877527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea43f1e1-833d-4d48-9a7f-b3ff6d815159 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:29.878235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea43f1e1-833d-4d48-9a7f-b3ff6d815159 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:29.878843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea43f1e1-833d-4d48-9a7f-b3ff6d815159 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:30.037943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:30.038743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:30.075147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:30.079889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:30.082099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:30.084266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:30.086257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:30.088837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:30.606740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:30.618054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ea43f1e1-833d-4d48-9a7f-b3ff6d815159 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:30.620154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:30.627352 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ea43f1e1-833d-4d48-9a7f-b3ff6d815159 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.792374 Sep 27 10:21:30.629204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:30.634626 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c23fb459-faf1-4886-8ca2-4b05a26b54dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 0.607169 Sep 27 10:21:30.637822 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2332/5105] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:30 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 611 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:30.656036 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2774/5106] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:29 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 821 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:31.669537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:31.671282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8d0d6b5c-63ea-4767-81b4-157215a58a08 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:31.672389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8d0d6b5c-63ea-4767-81b4-157215a58a08 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:31.673241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:31.709255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0d6b5c-63ea-4767-81b4-157215a58a08 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:31.711402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0d6b5c-63ea-4767-81b4-157215a58a08 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:31.713821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0d6b5c-63ea-4767-81b4-157215a58a08 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:31.714874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0d6b5c-63ea-4767-81b4-157215a58a08 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:31.717352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:31.718220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:31.718923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0d6b5c-63ea-4767-81b4-157215a58a08 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:31.719694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:31.720310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8d0d6b5c-63ea-4767-81b4-157215a58a08 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:31.721295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:31.725955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:31.729126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:32.055631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8d0d6b5c-63ea-4767-81b4-157215a58a08 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:32.068159 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8d0d6b5c-63ea-4767-81b4-157215a58a08 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.404267 Sep 27 10:21:32.077531 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2775/5107] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:31 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:32.501910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:32.518275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:32.519744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:32.520936 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8d34aa15-c54a-46fb-bacc-b67ab01bba36 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1739 microversion: 2.20 time: 0.861409 Sep 27 10:21:32.533352 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2333/5108] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:31 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1739 bytes in 866 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:33.099548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-59a5ce5f-520b-440c-845d-1f392341fbe6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:33.100949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-59a5ce5f-520b-440c-845d-1f392341fbe6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:33.114456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a5ce5f-520b-440c-845d-1f392341fbe6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:33.114973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a5ce5f-520b-440c-845d-1f392341fbe6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:33.115487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a5ce5f-520b-440c-845d-1f392341fbe6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:33.116235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a5ce5f-520b-440c-845d-1f392341fbe6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:33.120519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a5ce5f-520b-440c-845d-1f392341fbe6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:33.121329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59a5ce5f-520b-440c-845d-1f392341fbe6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:33.558209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:33.558828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:33.593669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:33.596556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:33.597828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:33.598270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59a5ce5f-520b-440c-845d-1f392341fbe6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:33.603803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:33.604341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:33.609215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:33.615799 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-59a5ce5f-520b-440c-845d-1f392341fbe6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.519065 Sep 27 10:21:33.618514 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2776/5109] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:33 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 523 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:21:33.638660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-04b8fe8e-8cf2-4104-8127-4843825e7097 req-83b59ccb-f4ae-4f2d-9e1c-1f1e6b91a9c1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5ac6dabf-1315-4e05-ab6c-22211dc2440b", "tag": "b894a455-9bb5-443a-84fe-e04c8d0c3e28"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:33.674502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04b8fe8e-8cf2-4104-8127-4843825e7097 req-83b59ccb-f4ae-4f2d-9e1c-1f1e6b91a9c1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:33.677562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04b8fe8e-8cf2-4104-8127-4843825e7097 req-83b59ccb-f4ae-4f2d-9e1c-1f1e6b91a9c1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:33.678613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04b8fe8e-8cf2-4104-8127-4843825e7097 req-83b59ccb-f4ae-4f2d-9e1c-1f1e6b91a9c1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:33.711080 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-04b8fe8e-8cf2-4104-8127-4843825e7097 req-83b59ccb-f4ae-4f2d-9e1c-1f1e6b91a9c1 service nova] Creating event network-changed:b894a455-9bb5-443a-84fe-e04c8d0c3e28 for instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b on nch1-master-1 Sep 27 10:21:33.728097 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-04b8fe8e-8cf2-4104-8127-4843825e7097 req-83b59ccb-f4ae-4f2d-9e1c-1f1e6b91a9c1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.104249 Sep 27 10:21:33.731697 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2777/5110] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:21:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:21:34.377371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:34.398131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:34.400397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:34.404241 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-42bc509e-436e-45e8-89a1-b6a198aad9f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1721 microversion: 2.20 time: 0.855954 Sep 27 10:21:34.406963 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2334/5111] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:33 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1721 bytes in 860 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:34.644251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-213f06ec-66c5-4d4b-9c50-f69ace9c40b6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:34.645181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-213f06ec-66c5-4d4b-9c50-f69ace9c40b6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:34.669223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-213f06ec-66c5-4d4b-9c50-f69ace9c40b6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:34.669721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-213f06ec-66c5-4d4b-9c50-f69ace9c40b6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:34.670165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-213f06ec-66c5-4d4b-9c50-f69ace9c40b6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:34.670894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-213f06ec-66c5-4d4b-9c50-f69ace9c40b6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:34.671349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-213f06ec-66c5-4d4b-9c50-f69ace9c40b6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:34.676362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-213f06ec-66c5-4d4b-9c50-f69ace9c40b6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:34.885879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-213f06ec-66c5-4d4b-9c50-f69ace9c40b6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:34.905908 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-213f06ec-66c5-4d4b-9c50-f69ace9c40b6 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.267362 Sep 27 10:21:34.909934 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2778/5112] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:34 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:35.436165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:35.436946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:35.453521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:35.455495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:35.456972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:35.458743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:35.461823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:35.462416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:35.942197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-64856e8f-753b-4261-9dc3-979d0de2e5e7 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:35.944526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-64856e8f-753b-4261-9dc3-979d0de2e5e7 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:35.959453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64856e8f-753b-4261-9dc3-979d0de2e5e7 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:35.962547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64856e8f-753b-4261-9dc3-979d0de2e5e7 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:35.964190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64856e8f-753b-4261-9dc3-979d0de2e5e7 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:35.966778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64856e8f-753b-4261-9dc3-979d0de2e5e7 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:35.967745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64856e8f-753b-4261-9dc3-979d0de2e5e7 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:35.968362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64856e8f-753b-4261-9dc3-979d0de2e5e7 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:35.986768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:36.003930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:36.014054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:36.020566 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-75c19830-1943-45c0-906c-0532534df1a0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1724 microversion: 2.20 time: 0.587230 Sep 27 10:21:36.032826 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2335/5113] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:35 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1724 bytes in 602 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:36.083485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95786c4f-edd3-42b8-874a-6960f6a65ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "05adfdbe-11b6-4e2e-9ace-45f0a99dba1d"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:36.096261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-95786c4f-edd3-42b8-874a-6960f6a65ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:36.127980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95786c4f-edd3-42b8-874a-6960f6a65ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:36.131757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95786c4f-edd3-42b8-874a-6960f6a65ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:36.133475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95786c4f-edd3-42b8-874a-6960f6a65ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:36.138875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95786c4f-edd3-42b8-874a-6960f6a65ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:36.139600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95786c4f-edd3-42b8-874a-6960f6a65ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:36.142474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95786c4f-edd3-42b8-874a-6960f6a65ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:36.777218 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-95786c4f-edd3-42b8-874a-6960f6a65ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "POST /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.717111 Sep 27 10:21:36.778334 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2336/5114] 10.222.0.26 () {62 vars in 1364 bytes} [Tue Sep 27 10:21:36 2022] POST /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-volume_attachments => generated 188 bytes in 723 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:21:36.796450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a9b727c2-5aee-4f3c-bf1b-123aae69704c req-e887db63-b5f3-4b01-8c4d-7df9ffcaebf5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "691b2be7-d5b8-409c-ba22-5ab8da50b80d", "name": "network-vif-unplugged", "status": "completed", "tag": "11b232b4-3236-4917-9ebb-02d7dbd3132e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:36.813741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-64856e8f-753b-4261-9dc3-979d0de2e5e7 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:36.826023 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-64856e8f-753b-4261-9dc3-979d0de2e5e7 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.884939 Sep 27 10:21:36.827390 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2779/5115] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:35 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 890 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:21:36.831312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a9b727c2-5aee-4f3c-bf1b-123aae69704c req-e887db63-b5f3-4b01-8c4d-7df9ffcaebf5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:36.832057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a9b727c2-5aee-4f3c-bf1b-123aae69704c req-e887db63-b5f3-4b01-8c4d-7df9ffcaebf5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:36.832739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a9b727c2-5aee-4f3c-bf1b-123aae69704c req-e887db63-b5f3-4b01-8c4d-7df9ffcaebf5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:36.873465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.server_external_events [req-a9b727c2-5aee-4f3c-bf1b-123aae69704c req-e887db63-b5f3-4b01-8c4d-7df9ffcaebf5 service nova] Unable to find a host for instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d. Dropping event network-vif-unplugged {{(pid=145969) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 27 10:21:36.877595 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a9b727c2-5aee-4f3c-bf1b-123aae69704c req-e887db63-b5f3-4b01-8c4d-7df9ffcaebf5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.096104 Sep 27 10:21:36.878259 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2337/5116] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:21:36 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 99 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Sep 27 10:21:37.039684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-012d7a88-7071-4cf2-95a6-6d66547d80a9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:37.040428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-012d7a88-7071-4cf2-95a6-6d66547d80a9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:37.062064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012d7a88-7071-4cf2-95a6-6d66547d80a9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:37.064006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012d7a88-7071-4cf2-95a6-6d66547d80a9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:37.065255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012d7a88-7071-4cf2-95a6-6d66547d80a9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:37.067370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012d7a88-7071-4cf2-95a6-6d66547d80a9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:37.067996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012d7a88-7071-4cf2-95a6-6d66547d80a9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:37.068524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-012d7a88-7071-4cf2-95a6-6d66547d80a9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:37.245354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-012d7a88-7071-4cf2-95a6-6d66547d80a9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=145968) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 27 10:21:37.396078 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-012d7a88-7071-4cf2-95a6-6d66547d80a9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "POST /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/action" status: 202 len: 0 microversion: 2.20 time: 0.355906 Sep 27 10:21:37.397095 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2780/5117] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:21:37 2022] POST /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/action => generated 0 bytes in 363 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:21:37.407301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:37.408084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:37.425007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:37.430133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:37.430850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:37.431896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:37.434034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:37.435378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:37.747885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:37.751066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=3,nova_cell1:UPDATE=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:37.763875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:37.765448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:37.767319 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fbb7d18b-9583-497c-8bb9-defd8cb4f887 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1750 microversion: 2.20 time: 0.365240 Sep 27 10:21:37.768356 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2338/5118] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:37 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1750 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:37.855025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f13af368-470e-458f-901d-87bac00da0cf tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:37.856039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f13af368-470e-458f-901d-87bac00da0cf tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:37.862733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f13af368-470e-458f-901d-87bac00da0cf tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:37.863516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f13af368-470e-458f-901d-87bac00da0cf tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:37.864482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f13af368-470e-458f-901d-87bac00da0cf tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:37.865412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f13af368-470e-458f-901d-87bac00da0cf tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:37.865851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f13af368-470e-458f-901d-87bac00da0cf tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:37.866311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f13af368-470e-458f-901d-87bac00da0cf tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:37.976428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f13af368-470e-458f-901d-87bac00da0cf tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:37.995106 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f13af368-470e-458f-901d-87bac00da0cf tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.144109 Sep 27 10:21:37.996520 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2781/5119] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:37 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:38.795379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:38.796931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:38.806913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:38.807656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:38.808406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:38.809430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:38.810136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:38.811188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:38.993241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-cf50b89f-c23d-45c9-b8a2-4014b208d800 req-8ca671e7-af1e-4930-9c7a-f0ce473e5e4e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3021488b-e2e0-4444-a89c-80de9e14a49f", "name": "network-vif-plugged", "status": "completed", "tag": "91cd8263-a144-429a-97a4-1338746dd22c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:39.014601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cf50b89f-c23d-45c9-b8a2-4014b208d800 req-8ca671e7-af1e-4930-9c7a-f0ce473e5e4e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:39.017689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cf50b89f-c23d-45c9-b8a2-4014b208d800 req-8ca671e7-af1e-4930-9c7a-f0ce473e5e4e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:39.018848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-cf50b89f-c23d-45c9-b8a2-4014b208d800 req-8ca671e7-af1e-4930-9c7a-f0ce473e5e4e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:39.025012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:39.036354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:39.038326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:39.039888 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fec94deb-a713-452d-89a5-9ab3cccf56fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1750 microversion: 2.20 time: 0.248902 Sep 27 10:21:39.047902 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-cf50b89f-c23d-45c9-b8a2-4014b208d800 req-8ca671e7-af1e-4930-9c7a-f0ce473e5e4e service nova] Creating event network-vif-plugged:91cd8263-a144-429a-97a4-1338746dd22c for instance 3021488b-e2e0-4444-a89c-80de9e14a49f on nch2-master-1 Sep 27 10:21:39.049228 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2339/5120] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:38 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1750 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:39.057158 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-cf50b89f-c23d-45c9-b8a2-4014b208d800 req-8ca671e7-af1e-4930-9c7a-f0ce473e5e4e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068462 Sep 27 10:21:39.061379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-db96fe80-e77f-47f9-b8a3-186a4ba26273 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:39.062201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-db96fe80-e77f-47f9-b8a3-186a4ba26273 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:39.065097 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2782/5121] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:21:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:21:39.074871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db96fe80-e77f-47f9-b8a3-186a4ba26273 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:39.075376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db96fe80-e77f-47f9-b8a3-186a4ba26273 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:39.075819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db96fe80-e77f-47f9-b8a3-186a4ba26273 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:39.076531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db96fe80-e77f-47f9-b8a3-186a4ba26273 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:39.077183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db96fe80-e77f-47f9-b8a3-186a4ba26273 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:39.077776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db96fe80-e77f-47f9-b8a3-186a4ba26273 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:39.304411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-db96fe80-e77f-47f9-b8a3-186a4ba26273 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:39.319611 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-db96fe80-e77f-47f9-b8a3-186a4ba26273 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.266964 Sep 27 10:21:39.322618 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2340/5122] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:39 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:40.077990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:40.079087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:40.098075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:40.099910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:40.101197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:40.104551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:40.105626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:40.107261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:40.355474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-baa5e754-f904-46a1-98d8-75da1b55538f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:40.356870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-baa5e754-f904-46a1-98d8-75da1b55538f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:40.369930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baa5e754-f904-46a1-98d8-75da1b55538f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:40.372226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baa5e754-f904-46a1-98d8-75da1b55538f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:40.372994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baa5e754-f904-46a1-98d8-75da1b55538f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:40.375266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baa5e754-f904-46a1-98d8-75da1b55538f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:40.375940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baa5e754-f904-46a1-98d8-75da1b55538f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:40.378557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baa5e754-f904-46a1-98d8-75da1b55538f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:40.532461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-baa5e754-f904-46a1-98d8-75da1b55538f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:40.541027 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-baa5e754-f904-46a1-98d8-75da1b55538f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1299 microversion: 2.1 time: 0.189750 Sep 27 10:21:40.548965 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2341/5123] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:40 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1299 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:21:40.847921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:40.869011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:40.874206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:40.878115 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-596731e2-e2dc-47d6-b8ef-65511081058d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1752 microversion: 2.20 time: 0.805127 Sep 27 10:21:40.879514 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2783/5124] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:40 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1752 bytes in 807 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:41.572938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-85120df7-7cbd-4213-8e8e-8d9871d3250f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:41.574305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-85120df7-7cbd-4213-8e8e-8d9871d3250f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:41.589785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85120df7-7cbd-4213-8e8e-8d9871d3250f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:41.592064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85120df7-7cbd-4213-8e8e-8d9871d3250f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:41.593295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85120df7-7cbd-4213-8e8e-8d9871d3250f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:41.596127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85120df7-7cbd-4213-8e8e-8d9871d3250f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:41.598333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85120df7-7cbd-4213-8e8e-8d9871d3250f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:41.601892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85120df7-7cbd-4213-8e8e-8d9871d3250f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:41.807150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-85120df7-7cbd-4213-8e8e-8d9871d3250f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:41.819658 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-85120df7-7cbd-4213-8e8e-8d9871d3250f tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1471 microversion: 2.1 time: 0.250675 Sep 27 10:21:41.833309 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2342/5125] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:41 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1471 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:41.852165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ccea6ac7-1c47-4b76-91af-dd30ea5d2e5d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:41.852860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ccea6ac7-1c47-4b76-91af-dd30ea5d2e5d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:41.869174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccea6ac7-1c47-4b76-91af-dd30ea5d2e5d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:41.870778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccea6ac7-1c47-4b76-91af-dd30ea5d2e5d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:41.872545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccea6ac7-1c47-4b76-91af-dd30ea5d2e5d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:41.874412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccea6ac7-1c47-4b76-91af-dd30ea5d2e5d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:41.874892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccea6ac7-1c47-4b76-91af-dd30ea5d2e5d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:41.875343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccea6ac7-1c47-4b76-91af-dd30ea5d2e5d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:41.918170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:41.919007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:41.943825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:41.944793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:41.945502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:41.948370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:41.949149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:41.950118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:42.012915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ccea6ac7-1c47-4b76-91af-dd30ea5d2e5d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:42.021920 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ccea6ac7-1c47-4b76-91af-dd30ea5d2e5d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1471 microversion: 2.1 time: 0.175534 Sep 27 10:21:42.022812 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2784/5126] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:41 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1471 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:42.043559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5f758672-8e36-4d95-9f8c-a4133cc3afa3 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:42.044450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5f758672-8e36-4d95-9f8c-a4133cc3afa3 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:42.051223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f758672-8e36-4d95-9f8c-a4133cc3afa3 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:42.052176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f758672-8e36-4d95-9f8c-a4133cc3afa3 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:42.052885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f758672-8e36-4d95-9f8c-a4133cc3afa3 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:42.053811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f758672-8e36-4d95-9f8c-a4133cc3afa3 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:42.054476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f758672-8e36-4d95-9f8c-a4133cc3afa3 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:42.055132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f758672-8e36-4d95-9f8c-a4133cc3afa3 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:42.179250 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5f758672-8e36-4d95-9f8c-a4133cc3afa3 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1622 microversion: 2.1 time: 0.138977 Sep 27 10:21:42.185068 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2785/5127] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:42 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1622 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:21:42.216062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f893ec5-37e2-44c4-8fa2-b62bc7154c81 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:42.217033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f893ec5-37e2-44c4-8fa2-b62bc7154c81 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:42.225043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f893ec5-37e2-44c4-8fa2-b62bc7154c81 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:42.225696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f893ec5-37e2-44c4-8fa2-b62bc7154c81 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:42.226147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f893ec5-37e2-44c4-8fa2-b62bc7154c81 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:42.226876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f893ec5-37e2-44c4-8fa2-b62bc7154c81 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:42.227337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f893ec5-37e2-44c4-8fa2-b62bc7154c81 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:42.227765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f893ec5-37e2-44c4-8fa2-b62bc7154c81 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:42.303482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:42.342197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:42.345378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:42.348171 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9cecdc5-dbaa-4d09-986d-b16286d20323 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.433632 Sep 27 10:21:42.352524 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2343/5128] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:41 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:21:42.524865 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f893ec5-37e2-44c4-8fa2-b62bc7154c81 tempest-TestSecurityGroupsBasicOps-849122065 tempest-TestSecurityGroupsBasicOps-849122065-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1913 microversion: 2.1 time: 0.312723 Sep 27 10:21:42.544010 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2786/5129] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:42 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1913 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:21:42.641864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-92c18601-dea6-40f5-accb-50f9bad6b99b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:42.642438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-92c18601-dea6-40f5-accb-50f9bad6b99b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:42.652523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c18601-dea6-40f5-accb-50f9bad6b99b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:42.652915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c18601-dea6-40f5-accb-50f9bad6b99b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:42.653367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c18601-dea6-40f5-accb-50f9bad6b99b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:42.654739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c18601-dea6-40f5-accb-50f9bad6b99b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:42.654739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c18601-dea6-40f5-accb-50f9bad6b99b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:42.655089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-92c18601-dea6-40f5-accb-50f9bad6b99b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:42.686431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-92c18601-dea6-40f5-accb-50f9bad6b99b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:21:42.805498 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-92c18601-dea6-40f5-accb-50f9bad6b99b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 204 len: 0 microversion: 2.1 time: 0.166962 Sep 27 10:21:42.806475 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2344/5130] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:21:42 2022] DELETE /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:21:42.816605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e87d2614-d934-4821-aae4-983d55fb18f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:42.819282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e87d2614-d934-4821-aae4-983d55fb18f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:42.828241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e87d2614-d934-4821-aae4-983d55fb18f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:42.831150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e87d2614-d934-4821-aae4-983d55fb18f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:42.831967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e87d2614-d934-4821-aae4-983d55fb18f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:42.835441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e87d2614-d934-4821-aae4-983d55fb18f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:42.836449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e87d2614-d934-4821-aae4-983d55fb18f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:42.837818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e87d2614-d934-4821-aae4-983d55fb18f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:43.022683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e87d2614-d934-4821-aae4-983d55fb18f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:43.040035 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e87d2614-d934-4821-aae4-983d55fb18f3 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1477 microversion: 2.1 time: 0.228591 Sep 27 10:21:43.066553 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2787/5131] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:42 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1477 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:43.378818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:43.379984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:43.395548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:43.396395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:43.397126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:43.398120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:43.398815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:43.399391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:43.738144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:43.748275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:43.750252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:43.751740 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2101d913-8ad0-4934-a4ae-a99b461b9c1c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.377380 Sep 27 10:21:43.776157 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2345/5132] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:43 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:44.071212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fef09fb3-1064-4457-a77a-241042c3f5da tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:44.072221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fef09fb3-1064-4457-a77a-241042c3f5da tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:44.082773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef09fb3-1064-4457-a77a-241042c3f5da tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:44.083297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef09fb3-1064-4457-a77a-241042c3f5da tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:44.087501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef09fb3-1064-4457-a77a-241042c3f5da tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:44.088353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef09fb3-1064-4457-a77a-241042c3f5da tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:44.088887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef09fb3-1064-4457-a77a-241042c3f5da tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:44.089333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef09fb3-1064-4457-a77a-241042c3f5da tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:44.358651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fef09fb3-1064-4457-a77a-241042c3f5da tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:44.380754 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fef09fb3-1064-4457-a77a-241042c3f5da tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1477 microversion: 2.1 time: 0.314429 Sep 27 10:21:44.381371 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2788/5133] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:44 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1477 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:44.785587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:44.787749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:44.834133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:44.837607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:44.839437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:44.845544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:44.846403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:44.853826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:45.412957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5cff1dfe-c5cf-4f25-a256-da2be878dc7b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:45.416176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5cff1dfe-c5cf-4f25-a256-da2be878dc7b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:45.432268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cff1dfe-c5cf-4f25-a256-da2be878dc7b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:45.435334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cff1dfe-c5cf-4f25-a256-da2be878dc7b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:45.437128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cff1dfe-c5cf-4f25-a256-da2be878dc7b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:45.438177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cff1dfe-c5cf-4f25-a256-da2be878dc7b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:45.439773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cff1dfe-c5cf-4f25-a256-da2be878dc7b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:45.440631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5cff1dfe-c5cf-4f25-a256-da2be878dc7b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:45.493141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:45.516482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:45.518769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:45.520019 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-09953061-4a9f-4a10-a9e4-f9fae16b3ef4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.741097 Sep 27 10:21:45.529523 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2346/5134] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:44 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:45.912290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5cff1dfe-c5cf-4f25-a256-da2be878dc7b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:45.935880 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5cff1dfe-c5cf-4f25-a256-da2be878dc7b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 200 len: 1477 microversion: 2.1 time: 0.530712 Sep 27 10:21:45.968796 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2789/5135] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:45 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 1477 bytes in 565 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:21:45.989582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-59cd8653-7001-41aa-a388-7dafcbbf7fbc req-98b36464-d35c-48ba-9a61-d8cc4a5b8a5e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3021488b-e2e0-4444-a89c-80de9e14a49f", "name": "network-vif-deleted", "tag": "91cd8263-a144-429a-97a4-1338746dd22c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:46.035367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-59cd8653-7001-41aa-a388-7dafcbbf7fbc req-98b36464-d35c-48ba-9a61-d8cc4a5b8a5e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:46.042176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-59cd8653-7001-41aa-a388-7dafcbbf7fbc req-98b36464-d35c-48ba-9a61-d8cc4a5b8a5e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:46.042832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-59cd8653-7001-41aa-a388-7dafcbbf7fbc req-98b36464-d35c-48ba-9a61-d8cc4a5b8a5e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:46.095205 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-59cd8653-7001-41aa-a388-7dafcbbf7fbc req-98b36464-d35c-48ba-9a61-d8cc4a5b8a5e service nova] Creating event network-vif-deleted:91cd8263-a144-429a-97a4-1338746dd22c for instance 3021488b-e2e0-4444-a89c-80de9e14a49f on nch2-master-1 Sep 27 10:21:46.115540 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-59cd8653-7001-41aa-a388-7dafcbbf7fbc req-98b36464-d35c-48ba-9a61-d8cc4a5b8a5e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.131725 Sep 27 10:21:46.124504 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2347/5136] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:21:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:21:46.558434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:46.559198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:46.575204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=66,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:46.593198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:46.636744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:46.637358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.044s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:46.638149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:46.638635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:46.646897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:46.787693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-67fc22a6-7cc7-4c16-a4d1-07bcab8e82da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:21:46.790662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-67fc22a6-7cc7-4c16-a4d1-07bcab8e82da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:46.812930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67fc22a6-7cc7-4c16-a4d1-07bcab8e82da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:46.813499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67fc22a6-7cc7-4c16-a4d1-07bcab8e82da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:46.827489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67fc22a6-7cc7-4c16-a4d1-07bcab8e82da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:46.828415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67fc22a6-7cc7-4c16-a4d1-07bcab8e82da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:46.832253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67fc22a6-7cc7-4c16-a4d1-07bcab8e82da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:46.837418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67fc22a6-7cc7-4c16-a4d1-07bcab8e82da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:47.025189 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-67fc22a6-7cc7-4c16-a4d1-07bcab8e82da tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "POST /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a/action" status: 200 len: 24572 microversion: 2.1 time: 0.242695 Sep 27 10:21:47.036641 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2348/5137] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:21:46 2022] POST /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a/action => generated 24572 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:21:47.046135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:47.047446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3021488b-e2e0-4444-a89c-80de9e14a49f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:47.091067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:47.093853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:47.094339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:47.096268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:47.096791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:47.097237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:47.199405 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] HTTP exception thrown: Instance 3021488b-e2e0-4444-a89c-80de9e14a49f could not be found. Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Request method failure captured: Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f HTTP/1.1 Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs45wmzMX1kH6hs-WINHehGwJp24IAps6gEihr8jUnXkw1iMT5w3_sa-m0wAqepdVmR7KfDPwkSmkZc5HOcjz38sAVQmlzOXZNmpkGf-wnJdpb0UViMt8A9DOneb8hDEVHYOIZwvWuo-zyRYxNcF7Kvy5ZxqPUMkszYwi1n7Sdfjpro Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 932da48df2c44fc6a7dbd6c8daf9fc0d Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestSecurityGroupsBasicOps-1266469994 Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/932da48df2c44fc6a7dbd6c8daf9fc0d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/932da48df2c44fc6a7dbd6c8daf9fc0d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/932da48df2c44fc6a7dbd6c8daf9fc0d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_932da48df2c44fc6a7dbd6c8daf9fc0d"}]}] Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestSecurityGroupsBasicOps-1266469994 Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 932da48df2c44fc6a7dbd6c8daf9fc0d Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestSecurityGroupsBasicOps-1266469994 Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestSecurityGroupsBasicOps-1266469994-project Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 64c9f44e57164b858421b55114acf4cf Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestSecurityGroupsBasicOps-1266469994-project Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 3021488b-e2e0-4444-a89c-80de9e14a49f could not be found. Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3021488b-e2e0-4444-a89c-80de9e14a49f'} Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3021488b-e2e0-4444-a89c-80de9e14a49f could not be found. Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3021488b-e2e0-4444-a89c-80de9e14a49f could not be found. Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:21:47.201382 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:21:47.210804 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:21:47.210804 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:21:47.210804 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:21:47.210804 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:21:47.210804 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:21:47.210804 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:21:47.210804 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3021488b-e2e0-4444-a89c-80de9e14a49f could not be found. Sep 27 10:21:47.210804 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:21:47.210804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Returning 404 to user: Instance 3021488b-e2e0-4444-a89c-80de9e14a49f could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:21:47.210804 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-87593b88-9591-4732-b933-a98eeced6416 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f" status: 404 len: 111 microversion: 2.1 time: 0.168006 Sep 27 10:21:47.210804 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2349/5138] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:21:47 2022] GET /compute/v2.1/servers/3021488b-e2e0-4444-a89c-80de9e14a49f => generated 111 bytes in 173 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:21:47.681378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:47.685746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:47.698848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:47.702008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:47.707247 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3150d04e-4c83-49dd-a405-14777c36ea6c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 1.152384 Sep 27 10:21:47.708533 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2790/5139] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:46 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 1157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:48.747800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:48.770406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:48.793235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=74,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:21:48.825335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:48.826222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:48.829127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:48.830655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:48.833340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:48.834268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:50.545978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:50.577353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:50.585311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:50.590180 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-63907f3e-b6f8-4018-a06a-bdab0c17d57e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 1.856853 Sep 27 10:21:50.613140 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2350/5140] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:48 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 1882 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:51.624273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:51.625011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:51.689386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:51.689902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:51.690357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:51.691114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:51.691570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:51.691999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:52.418555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:52.432940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:52.438795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:52.440055 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5f5d7a4f-299b-42ea-b5de-2335da3e95ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.826565 Sep 27 10:21:52.441552 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2791/5141] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:51 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 829 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:53.472761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:53.473842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:53.486886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:53.487899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:53.488787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:53.490154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:53.490942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:53.491672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:54.246121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:54.267373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:54.269679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:54.271947 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a7587b63-1dd9-41f1-a7f6-1539089ce529 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.804672 Sep 27 10:21:54.276536 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2351/5142] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:53 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 810 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:55.310796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:55.312568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:55.325001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:55.329074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:55.331878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:55.333717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:55.335902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:55.338551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:56.032615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:56.046068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:56.047522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:56.060982 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9ce5fed2-c17c-401a-b24e-cc55b629eb94 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.743060 Sep 27 10:21:56.061683 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2792/5143] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:55 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 757 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:57.092322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:57.093724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:57.126990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:57.133699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:57.135156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:57.138520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:57.142680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:57.144886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:57.964723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:57.983055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:57.984739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:57.986006 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dbf8e7aa-ff29-4bd3-bae1-dedb53002f27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.900143 Sep 27 10:21:57.986563 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2352/5144] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:57 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 901 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:21:59.012523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:21:59.013173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:21:59.028462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:59.033189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:59.033681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:59.034440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:21:59.034913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:21:59.035347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:21:59.568538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:59.590521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:59.591974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:21:59.593342 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2868fe44-b479-4af8-80cd-60cb5b7c4c8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.585592 Sep 27 10:21:59.593911 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2793/5145] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:21:59 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:22:00.631750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:00.632384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:00.641718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:00.642440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:00.643206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:00.644242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:00.644914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:00.645740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:00.948359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:00.966439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:00.968014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:00.969291 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-da1bdce3-feb5-489c-b46c-43881f6b38de tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.344320 Sep 27 10:22:00.970473 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2353/5146] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:00 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:22:02.000929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:02.001543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:02.018337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:02.023861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:02.024823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:02.033777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:02.034426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:02.037043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:03.189144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:03.225829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:03.227596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:03.236878 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5df9a82c-0c40-4dda-9a83-9b953866aae8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 1.233118 Sep 27 10:22:03.238168 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2794/5147] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:01 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 1243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:22:03.572463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3094be86-48ff-4f03-b7a8-10dbccd621ae tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:03.573126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3094be86-48ff-4f03-b7a8-10dbccd621ae tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:03.583543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3094be86-48ff-4f03-b7a8-10dbccd621ae tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:03.593541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3094be86-48ff-4f03-b7a8-10dbccd621ae tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:03.594256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3094be86-48ff-4f03-b7a8-10dbccd621ae tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:03.594575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ec6eb74f-9957-480e-8daa-89d7c7ddd7a6 req-5a418d92-d588-4e5f-89ed-0371e0e44baa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3bba7aa2-0a3a-42f0-b00c-93ac58368945", "tag": "47866525-9b21-41d1-bec7-5a5dc5d53fad"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:03.595353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3094be86-48ff-4f03-b7a8-10dbccd621ae tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:03.595965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3094be86-48ff-4f03-b7a8-10dbccd621ae tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:03.596757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3094be86-48ff-4f03-b7a8-10dbccd621ae tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:03.628701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ec6eb74f-9957-480e-8daa-89d7c7ddd7a6 req-5a418d92-d588-4e5f-89ed-0371e0e44baa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:03.629477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ec6eb74f-9957-480e-8daa-89d7c7ddd7a6 req-5a418d92-d588-4e5f-89ed-0371e0e44baa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:03.631099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ec6eb74f-9957-480e-8daa-89d7c7ddd7a6 req-5a418d92-d588-4e5f-89ed-0371e0e44baa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:03.667420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3094be86-48ff-4f03-b7a8-10dbccd621ae tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:22:03.683260 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ec6eb74f-9957-480e-8daa-89d7c7ddd7a6 req-5a418d92-d588-4e5f-89ed-0371e0e44baa service nova] Creating event network-changed:47866525-9b21-41d1-bec7-5a5dc5d53fad for instance 3bba7aa2-0a3a-42f0-b00c-93ac58368945 on nch1-master-1 Sep 27 10:22:03.689354 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ec6eb74f-9957-480e-8daa-89d7c7ddd7a6 req-5a418d92-d588-4e5f-89ed-0371e0e44baa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.099872 Sep 27 10:22:03.691409 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2795/5148] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:22:03.790991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:03.912307 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3094be86-48ff-4f03-b7a8-10dbccd621ae tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 204 len: 0 microversion: 2.1 time: 0.339982 Sep 27 10:22:03.917436 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2354/5149] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:22:03 2022] DELETE /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:22:03.926090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-41c48ec8-5e77-4334-bdea-9cb10d26b74b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:03.927112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-41c48ec8-5e77-4334-bdea-9cb10d26b74b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:03.934119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41c48ec8-5e77-4334-bdea-9cb10d26b74b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:03.934817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41c48ec8-5e77-4334-bdea-9cb10d26b74b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:03.935403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41c48ec8-5e77-4334-bdea-9cb10d26b74b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:03.936301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41c48ec8-5e77-4334-bdea-9cb10d26b74b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:03.937082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41c48ec8-5e77-4334-bdea-9cb10d26b74b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:03.937710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-41c48ec8-5e77-4334-bdea-9cb10d26b74b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:04.276203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:04.277327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:04.292001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:04.293034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:04.293868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:04.294894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:04.295600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:04.296175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:04.496121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-41c48ec8-5e77-4334-bdea-9cb10d26b74b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:04.520379 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-41c48ec8-5e77-4334-bdea-9cb10d26b74b tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1768 microversion: 2.1 time: 0.598148 Sep 27 10:22:04.522378 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2796/5150] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:03 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1768 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:04.673076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:04.688477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:04.690219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:04.691491 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-91d2d23c-c466-4b3d-8baa-ea51d909bec4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.423024 Sep 27 10:22:04.693787 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2355/5151] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:04 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:22:05.547117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba7927e3-ea71-4c47-bf42-7dd5f4a7d0c4 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:05.547848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba7927e3-ea71-4c47-bf42-7dd5f4a7d0c4 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:05.558220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7927e3-ea71-4c47-bf42-7dd5f4a7d0c4 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:05.569422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7927e3-ea71-4c47-bf42-7dd5f4a7d0c4 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:05.569983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7927e3-ea71-4c47-bf42-7dd5f4a7d0c4 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:05.570956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7927e3-ea71-4c47-bf42-7dd5f4a7d0c4 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:05.571629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7927e3-ea71-4c47-bf42-7dd5f4a7d0c4 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:05.572228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba7927e3-ea71-4c47-bf42-7dd5f4a7d0c4 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:05.727134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:05.728907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:05.737562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:05.738324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:05.738989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:05.740068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:05.740766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:05.741336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:06.125951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba7927e3-ea71-4c47-bf42-7dd5f4a7d0c4 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:06.151587 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba7927e3-ea71-4c47-bf42-7dd5f4a7d0c4 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1768 microversion: 2.1 time: 0.608554 Sep 27 10:22:06.152810 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2797/5152] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:05 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1768 bytes in 611 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:06.300373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:06.318476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:06.324086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:06.326039 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9edc3853-f919-4985-be77-ddb8815a395b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.603056 Sep 27 10:22:06.343213 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2356/5153] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:05 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:22:06.358907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-110b94eb-2f8d-49bb-8209-60b8a50fa0fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:06.360720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-110b94eb-2f8d-49bb-8209-60b8a50fa0fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:06.382170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-110b94eb-2f8d-49bb-8209-60b8a50fa0fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:06.387309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-110b94eb-2f8d-49bb-8209-60b8a50fa0fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:06.388128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-110b94eb-2f8d-49bb-8209-60b8a50fa0fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:06.389584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-110b94eb-2f8d-49bb-8209-60b8a50fa0fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:06.392450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-110b94eb-2f8d-49bb-8209-60b8a50fa0fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:06.394846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-110b94eb-2f8d-49bb-8209-60b8a50fa0fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:06.844926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-529c5361-0e57-4b5a-9d6c-d4f9dd040493 req-6ff31f67-b6fd-46b9-9ba2-dfde3c39b904 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "691b2be7-d5b8-409c-ba22-5ab8da50b80d", "name": "network-vif-plugged", "status": "completed", "tag": "11b232b4-3236-4917-9ebb-02d7dbd3132e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:06.875866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-529c5361-0e57-4b5a-9d6c-d4f9dd040493 req-6ff31f67-b6fd-46b9-9ba2-dfde3c39b904 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:06.879789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-529c5361-0e57-4b5a-9d6c-d4f9dd040493 req-6ff31f67-b6fd-46b9-9ba2-dfde3c39b904 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:06.881555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-529c5361-0e57-4b5a-9d6c-d4f9dd040493 req-6ff31f67-b6fd-46b9-9ba2-dfde3c39b904 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:06.934021 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-529c5361-0e57-4b5a-9d6c-d4f9dd040493 req-6ff31f67-b6fd-46b9-9ba2-dfde3c39b904 service nova] Creating event network-vif-plugged:11b232b4-3236-4917-9ebb-02d7dbd3132e for instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d on nch1-master-1 Sep 27 10:22:06.953367 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-529c5361-0e57-4b5a-9d6c-d4f9dd040493 req-6ff31f67-b6fd-46b9-9ba2-dfde3c39b904 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.116240 Sep 27 10:22:06.977290 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2357/5154] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:22:07.183585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-178da60c-dedf-4dc0-b2e1-eaa960f03d8d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:07.184310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-178da60c-dedf-4dc0-b2e1-eaa960f03d8d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:07.195701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-178da60c-dedf-4dc0-b2e1-eaa960f03d8d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:07.196193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-178da60c-dedf-4dc0-b2e1-eaa960f03d8d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:07.202048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-178da60c-dedf-4dc0-b2e1-eaa960f03d8d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:07.202949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-178da60c-dedf-4dc0-b2e1-eaa960f03d8d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:07.203434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-178da60c-dedf-4dc0-b2e1-eaa960f03d8d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:07.209077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-178da60c-dedf-4dc0-b2e1-eaa960f03d8d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:07.629617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-178da60c-dedf-4dc0-b2e1-eaa960f03d8d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:07.652155 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-178da60c-dedf-4dc0-b2e1-eaa960f03d8d tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1650 microversion: 2.1 time: 0.473440 Sep 27 10:22:07.656697 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2358/5155] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:07 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1650 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:22:07.671845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-110b94eb-2f8d-49bb-8209-60b8a50fa0fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:07.687336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:07.688530 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-110b94eb-2f8d-49bb-8209-60b8a50fa0fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 1.332107 Sep 27 10:22:07.689461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:07.690410 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2798/5156] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:06 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 1338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:07.719554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9a3f4af5-e690-4614-9a06-c06938bc3fcc tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:07.722552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9a3f4af5-e690-4614-9a06-c06938bc3fcc tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:07.727151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:07.729763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:07.731838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:07.734727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:07.734727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:07.736389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:07.740639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a3f4af5-e690-4614-9a06-c06938bc3fcc tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:07.743633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a3f4af5-e690-4614-9a06-c06938bc3fcc tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:07.745410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a3f4af5-e690-4614-9a06-c06938bc3fcc tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:07.746247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a3f4af5-e690-4614-9a06-c06938bc3fcc tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:07.746787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a3f4af5-e690-4614-9a06-c06938bc3fcc tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:07.747863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a3f4af5-e690-4614-9a06-c06938bc3fcc tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:08.074591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:08.091537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:08.094018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:08.098077 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d090ae29-f723-487d-b07f-8bb7aaa7c963 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.439255 Sep 27 10:22:08.099033 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2359/5157] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:07 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:22:08.109206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5afca9a1-6a6f-4691-9533-8e94751bcb28 req-baef9158-1f38-4365-97cd-b714a6e06f58 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3bba7aa2-0a3a-42f0-b00c-93ac58368945", "name": "network-vif-deleted", "tag": "47866525-9b21-41d1-bec7-5a5dc5d53fad"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:08.130024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5afca9a1-6a6f-4691-9533-8e94751bcb28 req-baef9158-1f38-4365-97cd-b714a6e06f58 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:08.130692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5afca9a1-6a6f-4691-9533-8e94751bcb28 req-baef9158-1f38-4365-97cd-b714a6e06f58 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:08.131237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5afca9a1-6a6f-4691-9533-8e94751bcb28 req-baef9158-1f38-4365-97cd-b714a6e06f58 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:08.151408 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5afca9a1-6a6f-4691-9533-8e94751bcb28 req-baef9158-1f38-4365-97cd-b714a6e06f58 service nova] Creating event network-vif-deleted:47866525-9b21-41d1-bec7-5a5dc5d53fad for instance 3bba7aa2-0a3a-42f0-b00c-93ac58368945 on nch1-master-1 Sep 27 10:22:08.159325 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5afca9a1-6a6f-4691-9533-8e94751bcb28 req-baef9158-1f38-4365-97cd-b714a6e06f58 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057822 Sep 27 10:22:08.160098 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2360/5158] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:22:08.260177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9a3f4af5-e690-4614-9a06-c06938bc3fcc tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:08.278847 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9a3f4af5-e690-4614-9a06-c06938bc3fcc tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.563978 Sep 27 10:22:08.283147 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2799/5159] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:07 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:08.312755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0fa23929-683a-4b4f-9261-d4dd8a14e08c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:08.316462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0fa23929-683a-4b4f-9261-d4dd8a14e08c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:08.336619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fa23929-683a-4b4f-9261-d4dd8a14e08c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:08.337244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fa23929-683a-4b4f-9261-d4dd8a14e08c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:08.337707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fa23929-683a-4b4f-9261-d4dd8a14e08c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:08.338447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fa23929-683a-4b4f-9261-d4dd8a14e08c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:08.338909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fa23929-683a-4b4f-9261-d4dd8a14e08c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:08.339338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fa23929-683a-4b4f-9261-d4dd8a14e08c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:08.677988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:08.679200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:08.686666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:08.687162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:08.687601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:08.688634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:08.696390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:08.696913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:08.758660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0fa23929-683a-4b4f-9261-d4dd8a14e08c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:08.768017 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0fa23929-683a-4b4f-9261-d4dd8a14e08c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.459713 Sep 27 10:22:08.787281 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2361/5160] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:08 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 480 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:08.791132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:08.797830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e8a2be4b-d275-4c38-863e-ffae90fe138b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-1-1290501729"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:08.798795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:08.799552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:08.800202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lazy-loading 'fault' on Instance uuid 3bba7aa2-0a3a-42f0-b00c-93ac58368945 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:22:08.808212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e8a2be4b-d275-4c38-863e-ffae90fe138b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:08.823428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8a2be4b-d275-4c38-863e-ffae90fe138b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:08.824228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8a2be4b-d275-4c38-863e-ffae90fe138b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:08.825093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8a2be4b-d275-4c38-863e-ffae90fe138b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:08.826009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8a2be4b-d275-4c38-863e-ffae90fe138b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:08.826736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8a2be4b-d275-4c38-863e-ffae90fe138b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:08.827335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8a2be4b-d275-4c38-863e-ffae90fe138b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:09.017553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64c9f44e57164b858421b55114acf4cf', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '932da48df2c44fc6a7dbd6c8daf9fc0d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:09.046543 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b9efcc9d-df5f-4619-8d17-dfc7a469f480 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 200 len: 1384 microversion: 2.1 time: 0.370614 Sep 27 10:22:09.047161 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2800/5161] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:08 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 1384 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:09.127066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:09.134842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:09.149277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:09.153138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:09.154725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:09.157367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:09.158084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:09.158814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:09.310073 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e8a2be4b-d275-4c38-863e-ffae90fe138b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.520734 Sep 27 10:22:09.315950 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2362/5162] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:22:08 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 528 msecs (HTTP/1.1 202) 10 headers in 456 bytes (1 switches on core 0) Sep 27 10:22:09.811406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:09.827283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:09.834891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:09.837821 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bc85c7ef-799a-4714-a66b-799f300385d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1808 microversion: 2.20 time: 0.713656 Sep 27 10:22:09.841140 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2801/5163] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:09 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1808 bytes in 719 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:22:10.073886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:10.075027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] [instance: 3bba7aa2-0a3a-42f0-b00c-93ac58368945] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:10.084956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:10.085719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:10.086361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:10.087304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:10.088206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:10.088872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:10.107082 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] HTTP exception thrown: Instance 3bba7aa2-0a3a-42f0-b00c-93ac58368945 could not be found. Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Request method failure captured: Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 HTTP/1.1 Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs45wmzMX1kH6hs-WINHehGwJp24IAps6gEihr8jUnXkw1iMT5w3_sa-m0wAqepdVmR7KfDPwkSmkZc5HOcjz38sAVQmlzOXZNmpkGf-wnJdpb0UViMt8A9DOneb8hDEVHYOIZwvWuo-zyRYxNcF7Kvy5ZxqPUMkszYwi1n7Sdfjpro Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 932da48df2c44fc6a7dbd6c8daf9fc0d Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestSecurityGroupsBasicOps-1266469994 Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/932da48df2c44fc6a7dbd6c8daf9fc0d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/932da48df2c44fc6a7dbd6c8daf9fc0d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/932da48df2c44fc6a7dbd6c8daf9fc0d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_932da48df2c44fc6a7dbd6c8daf9fc0d"}]}] Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestSecurityGroupsBasicOps-1266469994 Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 932da48df2c44fc6a7dbd6c8daf9fc0d Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestSecurityGroupsBasicOps-1266469994 Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestSecurityGroupsBasicOps-1266469994-project Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 64c9f44e57164b858421b55114acf4cf Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestSecurityGroupsBasicOps-1266469994-project Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 3bba7aa2-0a3a-42f0-b00c-93ac58368945 could not be found. Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3bba7aa2-0a3a-42f0-b00c-93ac58368945'} Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3bba7aa2-0a3a-42f0-b00c-93ac58368945 could not be found. Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3bba7aa2-0a3a-42f0-b00c-93ac58368945 could not be found. Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:22:10.110207 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:22:10.113348 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:22:10.113348 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:10.113348 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:22:10.113348 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:22:10.113348 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:22:10.113348 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:22:10.113348 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3bba7aa2-0a3a-42f0-b00c-93ac58368945 could not be found. Sep 27 10:22:10.113348 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:22:10.115497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Returning 404 to user: Instance 3bba7aa2-0a3a-42f0-b00c-93ac58368945 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:22:10.116581 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2727604d-1ade-458a-a932-f06db9479b97 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945" status: 404 len: 111 microversion: 2.1 time: 0.048598 Sep 27 10:22:10.117724 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2363/5164] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:10 2022] GET /compute/v2.1/servers/3bba7aa2-0a3a-42f0-b00c-93ac58368945 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:22:10.868318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:10.869818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:10.877345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:10.878167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:10.878837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:10.879652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:10.880343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:10.881000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:11.221706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:11.237354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:11.239031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:11.240278 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-19f79e97-e5fb-4a5a-bf62-c2c84a216b8e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1795 microversion: 2.20 time: 0.376095 Sep 27 10:22:11.248152 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2802/5165] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:10 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1795 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:22:14.771895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b7be25cb-2bf6-4ce3-9d0a-0fbb1302a9e5 req-650100a4-eb7f-4c4c-a6fb-be2373085014 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ac6dabf-1315-4e05-ab6c-22211dc2440b", "name": "network-vif-plugged", "status": "completed", "tag": "b894a455-9bb5-443a-84fe-e04c8d0c3e28"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:14.798876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b7be25cb-2bf6-4ce3-9d0a-0fbb1302a9e5 req-650100a4-eb7f-4c4c-a6fb-be2373085014 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:14.799385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b7be25cb-2bf6-4ce3-9d0a-0fbb1302a9e5 req-650100a4-eb7f-4c4c-a6fb-be2373085014 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:14.799825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b7be25cb-2bf6-4ce3-9d0a-0fbb1302a9e5 req-650100a4-eb7f-4c4c-a6fb-be2373085014 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:14.821417 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b7be25cb-2bf6-4ce3-9d0a-0fbb1302a9e5 req-650100a4-eb7f-4c4c-a6fb-be2373085014 service nova] Creating event network-vif-plugged:b894a455-9bb5-443a-84fe-e04c8d0c3e28 for instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b on nch1-master-1 Sep 27 10:22:14.832085 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b7be25cb-2bf6-4ce3-9d0a-0fbb1302a9e5 req-650100a4-eb7f-4c4c-a6fb-be2373085014 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067272 Sep 27 10:22:14.834143 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2364/5166] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:22:17.214838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2f24bb5f-7770-4566-a57a-9eacb5a0ba81 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] Calling method '.version_select at 0x7f7f62615790>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:17.245554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=28 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:17.264526 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2f24bb5f-7770-4566-a57a-9eacb5a0ba81 tempest-TestSecurityGroupsBasicOps-1266469994 tempest-TestSecurityGroupsBasicOps-1266469994-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestSecurityGroupsBasicOps-1627558846" status: 202 len: 0 microversion: 2.1 time: 0.056702 Sep 27 10:22:17.265326 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2803/5167] 10.222.0.26 () {58 vars in 1278 bytes} [Tue Sep 27 10:22:17 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestSecurityGroupsBasicOps-1627558846 => generated 0 bytes in 59 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:22:18.301442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-16661272-50f0-4913-bbd5-c5c1669f383e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:18.305411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-16661272-50f0-4913-bbd5-c5c1669f383e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:18.321757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16661272-50f0-4913-bbd5-c5c1669f383e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:18.322240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16661272-50f0-4913-bbd5-c5c1669f383e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:18.322681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16661272-50f0-4913-bbd5-c5c1669f383e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:18.325321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16661272-50f0-4913-bbd5-c5c1669f383e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:18.328720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16661272-50f0-4913-bbd5-c5c1669f383e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:18.335785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16661272-50f0-4913-bbd5-c5c1669f383e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:18.468105 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-16661272-50f0-4913-bbd5-c5c1669f383e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "POST /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a/action" status: 200 len: 32643 microversion: 2.1 time: 0.170767 Sep 27 10:22:18.469364 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2365/5168] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:22:18 2022] POST /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a/action => generated 32643 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:22:19.113936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-241856656", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:19.215383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] validate_networks() for None {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:22:21.130456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:22:21.131540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:22:21.132177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:22:21.132790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:22:21.133496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:22:21.151573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:21.152331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:21.153129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:21.154325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:21.156686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:21.160184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:21.207617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Getting quotas for project ab5aec6b80a447bab8eb4f1adf06e61f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:22:21.222190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Getting quotas for user b652b17bea8942c4b4a30034c6c1f44c and project ab5aec6b80a447bab8eb4f1adf06e61f. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:22:21.259733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:22:21.263367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:22:21.347414 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3ea23e6c-bdd8-468b-b999-8d536e8597d5 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.238187 Sep 27 10:22:21.358466 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2804/5169] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:22:19 2022] POST /compute/v2.1/servers => generated 380 bytes in 2250 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:22:21.363931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d0891d21-f5f8-4c9c-b2ac-5c4563efd87d tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:21.364800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d0891d21-f5f8-4c9c-b2ac-5c4563efd87d tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:21.376289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=37,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:21.498031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0891d21-f5f8-4c9c-b2ac-5c4563efd87d tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:21.514626 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d0891d21-f5f8-4c9c-b2ac-5c4563efd87d tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1214 microversion: 2.1 time: 0.154845 Sep 27 10:22:21.523460 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2366/5170] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:21 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1214 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:22.578101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-76bf2250-13fb-4055-8acf-0cffb4576c27 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:22.578621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-76bf2250-13fb-4055-8acf-0cffb4576c27 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:22.591061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bc70e33d-e6af-4f4f-b842-daa7487f5902 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:22.592649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bc70e33d-e6af-4f4f-b842-daa7487f5902 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:22.624767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76bf2250-13fb-4055-8acf-0cffb4576c27 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:22.625661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76bf2250-13fb-4055-8acf-0cffb4576c27 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:22.626333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76bf2250-13fb-4055-8acf-0cffb4576c27 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:22.627428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76bf2250-13fb-4055-8acf-0cffb4576c27 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:22.628306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76bf2250-13fb-4055-8acf-0cffb4576c27 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:22.630456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76bf2250-13fb-4055-8acf-0cffb4576c27 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:22.636602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=73 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:22.662154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-76bf2250-13fb-4055-8acf-0cffb4576c27 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:22:22.722639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bc70e33d-e6af-4f4f-b842-daa7487f5902 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:22.743168 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bc70e33d-e6af-4f4f-b842-daa7487f5902 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1214 microversion: 2.1 time: 0.161083 Sep 27 10:22:22.743662 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2367/5171] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:22 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1214 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:22:22.789703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-3021888a-4885-40be-8faf-6bdf34f23039 req-fc6d87e1-09e7-435d-8f26-dbb4f5272109 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5f899983-46ec-426e-b8bd-418768296d1a", "tag": "506eabfb-3d01-4318-9a51-6b79cfefff6c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:22.816048 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-76bf2250-13fb-4055-8acf-0cffb4576c27 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "DELETE /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 204 len: 0 microversion: 2.1 time: 0.241429 Sep 27 10:22:22.820898 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2805/5172] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:22:22 2022] DELETE /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:22:22.824940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3021888a-4885-40be-8faf-6bdf34f23039 req-fc6d87e1-09e7-435d-8f26-dbb4f5272109 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:22.825737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3021888a-4885-40be-8faf-6bdf34f23039 req-fc6d87e1-09e7-435d-8f26-dbb4f5272109 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:22.826888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3021888a-4885-40be-8faf-6bdf34f23039 req-fc6d87e1-09e7-435d-8f26-dbb4f5272109 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:22.827911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a2339aab-b2a3-4127-b019-73f6b0707bed tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:22.829473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a2339aab-b2a3-4127-b019-73f6b0707bed tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:22.844232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2339aab-b2a3-4127-b019-73f6b0707bed tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:22.848770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2339aab-b2a3-4127-b019-73f6b0707bed tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:22.850776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2339aab-b2a3-4127-b019-73f6b0707bed tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:22.852895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2339aab-b2a3-4127-b019-73f6b0707bed tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:22.853593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2339aab-b2a3-4127-b019-73f6b0707bed tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:22.854219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2339aab-b2a3-4127-b019-73f6b0707bed tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:22.871737 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-3021888a-4885-40be-8faf-6bdf34f23039 req-fc6d87e1-09e7-435d-8f26-dbb4f5272109 service nova] Creating event network-changed:506eabfb-3d01-4318-9a51-6b79cfefff6c for instance 5f899983-46ec-426e-b8bd-418768296d1a on nch2-master-1 Sep 27 10:22:22.892958 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-3021888a-4885-40be-8faf-6bdf34f23039 req-fc6d87e1-09e7-435d-8f26-dbb4f5272109 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.125951 Sep 27 10:22:22.893602 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2368/5173] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:22:23.429008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a2339aab-b2a3-4127-b019-73f6b0707bed tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:23.444346 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a2339aab-b2a3-4127-b019-73f6b0707bed tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1667 microversion: 2.1 time: 0.620906 Sep 27 10:22:23.447721 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2806/5174] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:22 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1667 bytes in 626 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:22:23.774408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b7835903-340c-4a3b-a953-e9fd80a1bb20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:23.777077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b7835903-340c-4a3b-a953-e9fd80a1bb20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:23.797565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7835903-340c-4a3b-a953-e9fd80a1bb20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:23.799649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7835903-340c-4a3b-a953-e9fd80a1bb20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:23.802953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7835903-340c-4a3b-a953-e9fd80a1bb20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:23.804143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7835903-340c-4a3b-a953-e9fd80a1bb20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:23.805944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7835903-340c-4a3b-a953-e9fd80a1bb20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:23.806579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7835903-340c-4a3b-a953-e9fd80a1bb20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:24.124106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b7835903-340c-4a3b-a953-e9fd80a1bb20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:24.185856 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b7835903-340c-4a3b-a953-e9fd80a1bb20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1218 microversion: 2.1 time: 0.406348 Sep 27 10:22:24.192133 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2369/5175] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:23 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1218 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:24.472540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d7b0723e-ce7d-46f8-8f2e-1ae3cf26239f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:24.473879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d7b0723e-ce7d-46f8-8f2e-1ae3cf26239f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:24.484313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b0723e-ce7d-46f8-8f2e-1ae3cf26239f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:24.486456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b0723e-ce7d-46f8-8f2e-1ae3cf26239f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:24.487058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b0723e-ce7d-46f8-8f2e-1ae3cf26239f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:24.489738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b0723e-ce7d-46f8-8f2e-1ae3cf26239f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:24.494044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b0723e-ce7d-46f8-8f2e-1ae3cf26239f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:24.494675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b0723e-ce7d-46f8-8f2e-1ae3cf26239f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:24.889513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d7b0723e-ce7d-46f8-8f2e-1ae3cf26239f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:24.901278 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d7b0723e-ce7d-46f8-8f2e-1ae3cf26239f tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1667 microversion: 2.1 time: 0.433276 Sep 27 10:22:24.901871 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2807/5176] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:24 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1667 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:25.214015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-457e0b4b-6554-4abd-b0f4-5599887b3f4f tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:25.215337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-457e0b4b-6554-4abd-b0f4-5599887b3f4f tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:25.236484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-457e0b4b-6554-4abd-b0f4-5599887b3f4f tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:25.243139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-457e0b4b-6554-4abd-b0f4-5599887b3f4f tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:25.244838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-457e0b4b-6554-4abd-b0f4-5599887b3f4f tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:25.250572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-457e0b4b-6554-4abd-b0f4-5599887b3f4f tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:25.252534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-457e0b4b-6554-4abd-b0f4-5599887b3f4f tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:25.254276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-457e0b4b-6554-4abd-b0f4-5599887b3f4f tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:25.633550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-457e0b4b-6554-4abd-b0f4-5599887b3f4f tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:25.654770 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-457e0b4b-6554-4abd-b0f4-5599887b3f4f tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1274 microversion: 2.1 time: 0.446220 Sep 27 10:22:25.660131 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2370/5177] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:25 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1274 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:25.929553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4091cf2-6514-466a-a505-490289bcedf3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:25.930375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4091cf2-6514-466a-a505-490289bcedf3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:25.951180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4091cf2-6514-466a-a505-490289bcedf3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:25.957102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4091cf2-6514-466a-a505-490289bcedf3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:25.957921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4091cf2-6514-466a-a505-490289bcedf3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:25.960646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4091cf2-6514-466a-a505-490289bcedf3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:25.961499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4091cf2-6514-466a-a505-490289bcedf3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:25.962594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4091cf2-6514-466a-a505-490289bcedf3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:26.532788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4091cf2-6514-466a-a505-490289bcedf3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:26.545111 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4091cf2-6514-466a-a505-490289bcedf3 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1667 microversion: 2.1 time: 0.620172 Sep 27 10:22:26.547464 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2808/5178] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:25 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1667 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:26.679695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eb39d266-f875-4317-9b18-1c0520e9bbd9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:26.680737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eb39d266-f875-4317-9b18-1c0520e9bbd9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:26.695499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb39d266-f875-4317-9b18-1c0520e9bbd9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:26.697207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb39d266-f875-4317-9b18-1c0520e9bbd9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:26.698922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb39d266-f875-4317-9b18-1c0520e9bbd9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:26.699864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb39d266-f875-4317-9b18-1c0520e9bbd9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:26.700480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb39d266-f875-4317-9b18-1c0520e9bbd9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:26.701602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb39d266-f875-4317-9b18-1c0520e9bbd9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:26.861194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eb39d266-f875-4317-9b18-1c0520e9bbd9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:26.870196 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eb39d266-f875-4317-9b18-1c0520e9bbd9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1272 microversion: 2.1 time: 0.194445 Sep 27 10:22:26.873799 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2371/5179] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:26 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1272 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:22:27.573293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fc1eda8f-a4ab-4d29-b525-3f85e0f3c658 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:27.573973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fc1eda8f-a4ab-4d29-b525-3f85e0f3c658 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:27.592435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1eda8f-a4ab-4d29-b525-3f85e0f3c658 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:27.593228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1eda8f-a4ab-4d29-b525-3f85e0f3c658 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:27.593808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1eda8f-a4ab-4d29-b525-3f85e0f3c658 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:27.594659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1eda8f-a4ab-4d29-b525-3f85e0f3c658 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:27.595463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1eda8f-a4ab-4d29-b525-3f85e0f3c658 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:27.596081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc1eda8f-a4ab-4d29-b525-3f85e0f3c658 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:27.802418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc1eda8f-a4ab-4d29-b525-3f85e0f3c658 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1d92f7d2ee2d47beaa4f5d43ac40cd82', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dbfb9f03e6de43a6bd5e7a6641ac8088', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:27.815419 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fc1eda8f-a4ab-4d29-b525-3f85e0f3c658 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 200 len: 1600 microversion: 2.1 time: 0.245974 Sep 27 10:22:27.816362 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2809/5180] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:27 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 1600 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:27.902790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-50aa6001-048d-40e9-b0f4-d02d53683444 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:27.903751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-50aa6001-048d-40e9-b0f4-d02d53683444 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:27.923925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50aa6001-048d-40e9-b0f4-d02d53683444 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:27.924592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50aa6001-048d-40e9-b0f4-d02d53683444 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:27.925160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50aa6001-048d-40e9-b0f4-d02d53683444 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:27.925814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50aa6001-048d-40e9-b0f4-d02d53683444 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:27.930181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50aa6001-048d-40e9-b0f4-d02d53683444 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:27.931064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50aa6001-048d-40e9-b0f4-d02d53683444 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:28.297985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-47f06cc6-0ff8-4b49-b87c-86b5bc7c0550 req-e9f108e7-4ac6-4104-8f1c-98e8dba233de service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5f899983-46ec-426e-b8bd-418768296d1a", "name": "network-vif-deleted", "tag": "506eabfb-3d01-4318-9a51-6b79cfefff6c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:28.318930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-47f06cc6-0ff8-4b49-b87c-86b5bc7c0550 req-e9f108e7-4ac6-4104-8f1c-98e8dba233de service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:28.321864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-47f06cc6-0ff8-4b49-b87c-86b5bc7c0550 req-e9f108e7-4ac6-4104-8f1c-98e8dba233de service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:28.322518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-47f06cc6-0ff8-4b49-b87c-86b5bc7c0550 req-e9f108e7-4ac6-4104-8f1c-98e8dba233de service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:28.350091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-47f06cc6-0ff8-4b49-b87c-86b5bc7c0550 req-e9f108e7-4ac6-4104-8f1c-98e8dba233de service nova] Creating event network-vif-deleted:506eabfb-3d01-4318-9a51-6b79cfefff6c for instance 5f899983-46ec-426e-b8bd-418768296d1a on nch2-master-1 Sep 27 10:22:28.370604 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-47f06cc6-0ff8-4b49-b87c-86b5bc7c0550 req-e9f108e7-4ac6-4104-8f1c-98e8dba233de service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.420383 Sep 27 10:22:28.371546 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2810/5181] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:22:28.418085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-50aa6001-048d-40e9-b0f4-d02d53683444 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:28.442781 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-50aa6001-048d-40e9-b0f4-d02d53683444 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1272 microversion: 2.1 time: 0.548386 Sep 27 10:22:28.447657 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2372/5182] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:27 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1272 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:28.883863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:28.885376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] [instance: 5f899983-46ec-426e-b8bd-418768296d1a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:28.904324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:28.905992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:28.907224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:28.907962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:28.908433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:28.908887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:28.948298 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] HTTP exception thrown: Instance 5f899983-46ec-426e-b8bd-418768296d1a could not be found. Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Request method failure captured: Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a HTTP/1.1 Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs2eRjT8bhSYLsQlvJTQ4ttNPcS9EZduJEFDCgV3jTSdS8rrt4b9DEBaRu_lxwSxOqpmgVkoN1WqHUz7hKeSvXwkoqI49Ux6j3j03a7LM9ic5cuzb0pWQizVu9lKkAw7GNxHX8WWE5NXMndH2ECVNPJifobiU_kngKBg6iv_vlyVyq0 Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: dbfb9f03e6de43a6bd5e7a6641ac8088 Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestNetworkBasicOps-437891721 Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dbfb9f03e6de43a6bd5e7a6641ac8088"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dbfb9f03e6de43a6bd5e7a6641ac8088"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/dbfb9f03e6de43a6bd5e7a6641ac8088"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_dbfb9f03e6de43a6bd5e7a6641ac8088"}]}] Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestNetworkBasicOps-437891721 Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: dbfb9f03e6de43a6bd5e7a6641ac8088 Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestNetworkBasicOps-437891721 Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestNetworkBasicOps-437891721-project Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 1d92f7d2ee2d47beaa4f5d43ac40cd82 Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestNetworkBasicOps-437891721-project Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 5f899983-46ec-426e-b8bd-418768296d1a could not be found. Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '5f899983-46ec-426e-b8bd-418768296d1a'} Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5f899983-46ec-426e-b8bd-418768296d1a could not be found. Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5f899983-46ec-426e-b8bd-418768296d1a could not be found. Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:22:28.954476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:22:28.959341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:28.959341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:22:28.959341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:22:28.959341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:22:28.959341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:22:28.959341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5f899983-46ec-426e-b8bd-418768296d1a could not be found. Sep 27 10:22:28.959341 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:22:28.963064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Returning 404 to user: Instance 5f899983-46ec-426e-b8bd-418768296d1a could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:22:28.964109 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-99ea35e7-3711-4f67-98f1-d5bf0ed6228e tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a" status: 404 len: 111 microversion: 2.1 time: 0.086751 Sep 27 10:22:28.972696 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2811/5183] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:28 2022] GET /compute/v2.1/servers/5f899983-46ec-426e-b8bd-418768296d1a => generated 111 bytes in 93 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:22:29.474541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-045df05a-c1c2-44fa-9d08-491e502ee2a2 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:29.476754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-045df05a-c1c2-44fa-9d08-491e502ee2a2 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:29.491852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045df05a-c1c2-44fa-9d08-491e502ee2a2 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:29.497495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045df05a-c1c2-44fa-9d08-491e502ee2a2 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:29.498122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045df05a-c1c2-44fa-9d08-491e502ee2a2 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:29.501078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045df05a-c1c2-44fa-9d08-491e502ee2a2 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:29.501783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045df05a-c1c2-44fa-9d08-491e502ee2a2 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:29.502378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045df05a-c1c2-44fa-9d08-491e502ee2a2 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:29.998666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e85cd2d0-64a6-41b4-9800-efd841a4e3c4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] Calling method '.version_select at 0x7f7f6c2d9820>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:30.041546 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e85cd2d0-64a6-41b4-9800-efd841a4e3c4 tempest-TestNetworkBasicOps-437891721 tempest-TestNetworkBasicOps-437891721-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-677549886" status: 202 len: 0 microversion: 2.1 time: 0.046735 Sep 27 10:22:30.042790 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2812/5184] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 10:22:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkBasicOps-677549886 => generated 0 bytes in 49 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:22:30.256307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-045df05a-c1c2-44fa-9d08-491e502ee2a2 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:30.270152 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-045df05a-c1c2-44fa-9d08-491e502ee2a2 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1314 microversion: 2.1 time: 0.802477 Sep 27 10:22:30.273300 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2373/5185] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:29 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1314 bytes in 806 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:31.295797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:31.296508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:31.302829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ec036b6e-94e5-4546-b99c-83e30be61590 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:31.305177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ec036b6e-94e5-4546-b99c-83e30be61590 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:31.325992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec036b6e-94e5-4546-b99c-83e30be61590 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:31.326758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec036b6e-94e5-4546-b99c-83e30be61590 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:31.328551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec036b6e-94e5-4546-b99c-83e30be61590 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:31.330631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec036b6e-94e5-4546-b99c-83e30be61590 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:31.334598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec036b6e-94e5-4546-b99c-83e30be61590 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:31.335360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec036b6e-94e5-4546-b99c-83e30be61590 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:32.641805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ec036b6e-94e5-4546-b99c-83e30be61590 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:32.665867 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ec036b6e-94e5-4546-b99c-83e30be61590 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1314 microversion: 2.1 time: 1.371813 Sep 27 10:22:32.674534 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2813/5186] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:31 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1314 bytes in 1381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:32.932384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-6f15b337-fbdf-4710-bb03-e91bf5b38049 req-a9171058-7e03-43a2-a9b9-c168bb3ecada service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a", "tag": "583d8591-4753-421d-b0db-6174d4df0a77"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:32.954610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:32.963576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6f15b337-fbdf-4710-bb03-e91bf5b38049 req-a9171058-7e03-43a2-a9b9-c168bb3ecada service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:32.964290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6f15b337-fbdf-4710-bb03-e91bf5b38049 req-a9171058-7e03-43a2-a9b9-c168bb3ecada service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:32.964769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6f15b337-fbdf-4710-bb03-e91bf5b38049 req-a9171058-7e03-43a2-a9b9-c168bb3ecada service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:32.997069 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-6f15b337-fbdf-4710-bb03-e91bf5b38049 req-a9171058-7e03-43a2-a9b9-c168bb3ecada service nova] Creating event network-changed:583d8591-4753-421d-b0db-6174d4df0a77 for instance 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a on nch2-master-1 Sep 27 10:22:33.006589 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-6f15b337-fbdf-4710-bb03-e91bf5b38049 req-a9171058-7e03-43a2-a9b9-c168bb3ecada service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.078104 Sep 27 10:22:33.011746 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2374/5187] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:22:33.690248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bd82e817-d37a-4cda-bc91-44519e772976 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:33.691412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bd82e817-d37a-4cda-bc91-44519e772976 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:33.702923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd82e817-d37a-4cda-bc91-44519e772976 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:33.703429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd82e817-d37a-4cda-bc91-44519e772976 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:33.703875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd82e817-d37a-4cda-bc91-44519e772976 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:33.704605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd82e817-d37a-4cda-bc91-44519e772976 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:33.718453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd82e817-d37a-4cda-bc91-44519e772976 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:33.721633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd82e817-d37a-4cda-bc91-44519e772976 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:36.138768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bd82e817-d37a-4cda-bc91-44519e772976 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:36.156579 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bd82e817-d37a-4cda-bc91-44519e772976 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1314 microversion: 2.1 time: 2.465017 Sep 27 10:22:36.157684 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2814/5188] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:33 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1314 bytes in 2472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:37.187120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3860f446-11b7-4155-893a-2df880d75700 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:37.187855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3860f446-11b7-4155-893a-2df880d75700 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:37.211097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3860f446-11b7-4155-893a-2df880d75700 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:37.211530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3860f446-11b7-4155-893a-2df880d75700 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:37.211981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3860f446-11b7-4155-893a-2df880d75700 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:37.212727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3860f446-11b7-4155-893a-2df880d75700 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:37.214055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3860f446-11b7-4155-893a-2df880d75700 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:37.214816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3860f446-11b7-4155-893a-2df880d75700 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:37.269131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:37.958743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3860f446-11b7-4155-893a-2df880d75700 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:37.973684 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3860f446-11b7-4155-893a-2df880d75700 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1314 microversion: 2.1 time: 0.791774 Sep 27 10:22:37.981660 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2375/5189] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:37 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1314 bytes in 801 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:38.677037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cc5c36fb-1efb-4811-8d21-e492faaf10ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:38.678785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cc5c36fb-1efb-4811-8d21-e492faaf10ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:38.689814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc5c36fb-1efb-4811-8d21-e492faaf10ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:38.690595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc5c36fb-1efb-4811-8d21-e492faaf10ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:38.691366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc5c36fb-1efb-4811-8d21-e492faaf10ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:38.692591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc5c36fb-1efb-4811-8d21-e492faaf10ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:38.693916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc5c36fb-1efb-4811-8d21-e492faaf10ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:38.694579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc5c36fb-1efb-4811-8d21-e492faaf10ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:39.003069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-53e37ad3-1e77-4c41-a773-f570a7857546 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:39.005752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-53e37ad3-1e77-4c41-a773-f570a7857546 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:39.014867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53e37ad3-1e77-4c41-a773-f570a7857546 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:39.016427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53e37ad3-1e77-4c41-a773-f570a7857546 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:39.017160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53e37ad3-1e77-4c41-a773-f570a7857546 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:39.018216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53e37ad3-1e77-4c41-a773-f570a7857546 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:39.019497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53e37ad3-1e77-4c41-a773-f570a7857546 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:39.020556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53e37ad3-1e77-4c41-a773-f570a7857546 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:39.410898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-53e37ad3-1e77-4c41-a773-f570a7857546 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:39.425725 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-53e37ad3-1e77-4c41-a773-f570a7857546 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1314 microversion: 2.1 time: 0.426855 Sep 27 10:22:39.433670 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2376/5190] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:38 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1314 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:39.559778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cc5c36fb-1efb-4811-8d21-e492faaf10ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:39.568893 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cc5c36fb-1efb-4811-8d21-e492faaf10ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.898156 Sep 27 10:22:39.581620 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2815/5191] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:38 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 911 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:40.454860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-66b8e7e4-97e9-495a-9160-36b646df29c9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:40.455487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-66b8e7e4-97e9-495a-9160-36b646df29c9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:40.494653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b8e7e4-97e9-495a-9160-36b646df29c9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:40.505777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b8e7e4-97e9-495a-9160-36b646df29c9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:40.506413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b8e7e4-97e9-495a-9160-36b646df29c9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:40.507326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b8e7e4-97e9-495a-9160-36b646df29c9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:40.508452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b8e7e4-97e9-495a-9160-36b646df29c9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:40.510361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-66b8e7e4-97e9-495a-9160-36b646df29c9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:40.605704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f1bf2192-a67b-4165-939f-5010d067a803 req-497cf846-acbb-43ce-8dcc-731ba7d6061c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a", "name": "network-vif-plugged", "status": "completed", "tag": "583d8591-4753-421d-b0db-6174d4df0a77"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:40.632474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f1bf2192-a67b-4165-939f-5010d067a803 req-497cf846-acbb-43ce-8dcc-731ba7d6061c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:40.636529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f1bf2192-a67b-4165-939f-5010d067a803 req-497cf846-acbb-43ce-8dcc-731ba7d6061c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:40.641286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f1bf2192-a67b-4165-939f-5010d067a803 req-497cf846-acbb-43ce-8dcc-731ba7d6061c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:40.671340 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f1bf2192-a67b-4165-939f-5010d067a803 req-497cf846-acbb-43ce-8dcc-731ba7d6061c service nova] Creating event network-vif-plugged:583d8591-4753-421d-b0db-6174d4df0a77 for instance 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a on nch2-master-1 Sep 27 10:22:40.685951 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f1bf2192-a67b-4165-939f-5010d067a803 req-497cf846-acbb-43ce-8dcc-731ba7d6061c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.337392 Sep 27 10:22:40.689419 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2377/5192] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:22:40.701166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2865aaf8-9ac5-4398-8427-e0e67206f92f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:40.705647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2865aaf8-9ac5-4398-8427-e0e67206f92f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:40.722523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2865aaf8-9ac5-4398-8427-e0e67206f92f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:40.729947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2865aaf8-9ac5-4398-8427-e0e67206f92f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:40.730576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2865aaf8-9ac5-4398-8427-e0e67206f92f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:40.731854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2865aaf8-9ac5-4398-8427-e0e67206f92f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:40.733152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2865aaf8-9ac5-4398-8427-e0e67206f92f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:40.735297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2865aaf8-9ac5-4398-8427-e0e67206f92f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:41.202201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-66b8e7e4-97e9-495a-9160-36b646df29c9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:41.212053 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-66b8e7e4-97e9-495a-9160-36b646df29c9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1314 microversion: 2.1 time: 0.762291 Sep 27 10:22:41.213421 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2816/5193] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:40 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1314 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:41.373986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2865aaf8-9ac5-4398-8427-e0e67206f92f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:41.390676 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2865aaf8-9ac5-4398-8427-e0e67206f92f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.699670 Sep 27 10:22:41.408338 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2378/5194] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:40 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 718 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:41.422478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f90d592-46d0-4ade-9a87-cc68a59a6156 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-2-544183907"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:41.425668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8f90d592-46d0-4ade-9a87-cc68a59a6156 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:41.433345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f90d592-46d0-4ade-9a87-cc68a59a6156 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:41.433822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f90d592-46d0-4ade-9a87-cc68a59a6156 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:41.434267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f90d592-46d0-4ade-9a87-cc68a59a6156 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:41.435313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f90d592-46d0-4ade-9a87-cc68a59a6156 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:41.435772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f90d592-46d0-4ade-9a87-cc68a59a6156 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:41.436238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f90d592-46d0-4ade-9a87-cc68a59a6156 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:41.780218 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8f90d592-46d0-4ade-9a87-cc68a59a6156 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.359383 Sep 27 10:22:41.808436 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2817/5195] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:22:41 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 390 msecs (HTTP/1.1 202) 10 headers in 456 bytes (1 switches on core 0) Sep 27 10:22:42.239034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a5eb9ead-5087-4995-be24-113a4fc08420 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:42.241085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a5eb9ead-5087-4995-be24-113a4fc08420 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:42.248417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5eb9ead-5087-4995-be24-113a4fc08420 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:42.249220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5eb9ead-5087-4995-be24-113a4fc08420 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:42.249859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5eb9ead-5087-4995-be24-113a4fc08420 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:42.250792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5eb9ead-5087-4995-be24-113a4fc08420 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:42.251563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5eb9ead-5087-4995-be24-113a4fc08420 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:42.252189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5eb9ead-5087-4995-be24-113a4fc08420 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:42.616458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a5eb9ead-5087-4995-be24-113a4fc08420 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:42.630100 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a5eb9ead-5087-4995-be24-113a4fc08420 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1314 microversion: 2.1 time: 0.395010 Sep 27 10:22:42.631842 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2379/5196] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:42 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1314 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:42.738352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6330caaa-6157-4b9e-9269-ba268e9aaa8d req-3291e6b7-8e60-4215-a1bc-fc490b9e304a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ac6dabf-1315-4e05-ab6c-22211dc2440b", "name": "network-vif-unplugged", "status": "completed", "tag": "b894a455-9bb5-443a-84fe-e04c8d0c3e28"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:42.751263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6330caaa-6157-4b9e-9269-ba268e9aaa8d req-3291e6b7-8e60-4215-a1bc-fc490b9e304a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:42.752713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6330caaa-6157-4b9e-9269-ba268e9aaa8d req-3291e6b7-8e60-4215-a1bc-fc490b9e304a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:42.753347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6330caaa-6157-4b9e-9269-ba268e9aaa8d req-3291e6b7-8e60-4215-a1bc-fc490b9e304a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:42.767548 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6330caaa-6157-4b9e-9269-ba268e9aaa8d req-3291e6b7-8e60-4215-a1bc-fc490b9e304a service nova] Creating event network-vif-unplugged:b894a455-9bb5-443a-84fe-e04c8d0c3e28 for instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b on nch1-master-1 Sep 27 10:22:42.773548 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6330caaa-6157-4b9e-9269-ba268e9aaa8d req-3291e6b7-8e60-4215-a1bc-fc490b9e304a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.039180 Sep 27 10:22:42.775637 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2818/5197] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:42 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:22:43.657120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d0e6dec6-e43a-494f-9757-b4c329fb6089 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:43.658476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d0e6dec6-e43a-494f-9757-b4c329fb6089 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:43.674557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0e6dec6-e43a-494f-9757-b4c329fb6089 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:43.675225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0e6dec6-e43a-494f-9757-b4c329fb6089 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:43.675656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0e6dec6-e43a-494f-9757-b4c329fb6089 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:43.676457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0e6dec6-e43a-494f-9757-b4c329fb6089 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:43.676933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0e6dec6-e43a-494f-9757-b4c329fb6089 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:43.677623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0e6dec6-e43a-494f-9757-b4c329fb6089 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:44.378067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0e6dec6-e43a-494f-9757-b4c329fb6089 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:44.397740 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d0e6dec6-e43a-494f-9757-b4c329fb6089 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1464 microversion: 2.1 time: 0.744732 Sep 27 10:22:44.402496 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2380/5198] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:43 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1464 bytes in 749 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:44.473025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-153fc590-9820-42d2-9d34-dbc695ea7822 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:44.473866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-153fc590-9820-42d2-9d34-dbc695ea7822 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:44.499844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153fc590-9820-42d2-9d34-dbc695ea7822 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:44.500929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153fc590-9820-42d2-9d34-dbc695ea7822 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:44.502336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153fc590-9820-42d2-9d34-dbc695ea7822 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:44.503285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153fc590-9820-42d2-9d34-dbc695ea7822 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:44.504096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153fc590-9820-42d2-9d34-dbc695ea7822 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:44.504830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-153fc590-9820-42d2-9d34-dbc695ea7822 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:44.575626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-153fc590-9820-42d2-9d34-dbc695ea7822 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:22:44.809735 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-153fc590-9820-42d2-9d34-dbc695ea7822 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 204 len: 0 microversion: 2.1 time: 0.342440 Sep 27 10:22:44.812540 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2819/5199] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:22:44 2022] DELETE /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:22:44.829089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dd112ee5-f68e-45ad-bb92-adfb182ed8a6 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:44.829749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dd112ee5-f68e-45ad-bb92-adfb182ed8a6 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:44.844079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd112ee5-f68e-45ad-bb92-adfb182ed8a6 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:44.844585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd112ee5-f68e-45ad-bb92-adfb182ed8a6 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:44.845053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd112ee5-f68e-45ad-bb92-adfb182ed8a6 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:44.850136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd112ee5-f68e-45ad-bb92-adfb182ed8a6 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:44.850676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd112ee5-f68e-45ad-bb92-adfb182ed8a6 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:44.851183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd112ee5-f68e-45ad-bb92-adfb182ed8a6 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:45.039427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d9486d10-86f4-4ea7-ade9-ded56eccf2c6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:45.040642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d9486d10-86f4-4ea7-ade9-ded56eccf2c6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:45.053050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9486d10-86f4-4ea7-ade9-ded56eccf2c6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:45.054886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9486d10-86f4-4ea7-ade9-ded56eccf2c6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:45.055513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9486d10-86f4-4ea7-ade9-ded56eccf2c6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:45.056426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9486d10-86f4-4ea7-ade9-ded56eccf2c6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:45.062506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9486d10-86f4-4ea7-ade9-ded56eccf2c6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:45.064283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9486d10-86f4-4ea7-ade9-ded56eccf2c6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:45.103314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:45.128439 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d9486d10-86f4-4ea7-ade9-ded56eccf2c6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-volume_attachments/05adfdbe-11b6-4e2e-9ace-45f0a99dba1d" status: 200 len: 194 microversion: 2.20 time: 0.094812 Sep 27 10:22:45.131274 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2820/5200] 10.222.0.26 () {60 vars in 1454 bytes} [Tue Sep 27 10:22:45 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-volume_attachments/05adfdbe-11b6-4e2e-9ace-45f0a99dba1d => generated 194 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:22:45.236622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:45.237199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:45.251128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:45.251804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:45.252257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:45.257455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:45.257952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:45.258394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:45.386933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lazy-loading 'services' on Instance uuid 691b2be7-d5b8-409c-ba22-5ab8da50b80d {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:22:45.517088 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ebd5d143-a595-41fd-bd2f-cc122eb02bd7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "DELETE /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-volume_attachments/05adfdbe-11b6-4e2e-9ace-45f0a99dba1d" status: 202 len: 0 microversion: 2.20 time: 0.287499 Sep 27 10:22:45.518725 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2821/5201] 10.222.0.26 () {60 vars in 1457 bytes} [Tue Sep 27 10:22:45 2022] DELETE /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-volume_attachments/05adfdbe-11b6-4e2e-9ace-45f0a99dba1d => generated 0 bytes in 291 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:22:45.542670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af50dfc7-47ce-4197-84ea-e2a45b9ab3fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:45.547692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af50dfc7-47ce-4197-84ea-e2a45b9ab3fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:45.558972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af50dfc7-47ce-4197-84ea-e2a45b9ab3fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:45.559781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af50dfc7-47ce-4197-84ea-e2a45b9ab3fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:45.560602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af50dfc7-47ce-4197-84ea-e2a45b9ab3fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:45.561983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af50dfc7-47ce-4197-84ea-e2a45b9ab3fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:45.562709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af50dfc7-47ce-4197-84ea-e2a45b9ab3fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:45.563357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af50dfc7-47ce-4197-84ea-e2a45b9ab3fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:45.667892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dd112ee5-f68e-45ad-bb92-adfb182ed8a6 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:45.692887 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dd112ee5-f68e-45ad-bb92-adfb182ed8a6 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1470 microversion: 2.1 time: 0.874305 Sep 27 10:22:45.695977 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2381/5202] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:44 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1470 bytes in 879 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:45.701982 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af50dfc7-47ce-4197-84ea-e2a45b9ab3fa tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.173691 Sep 27 10:22:45.703419 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2822/5203] 10.222.0.26 () {60 vars in 1343 bytes} [Tue Sep 27 10:22:45 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-volume_attachments => generated 197 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:22:46.716552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8dc15d88-1de1-44b3-8842-95d20b41ac60 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:46.723150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-efc275d1-2dd4-4650-b6ef-13c23741fe20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:46.723868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-efc275d1-2dd4-4650-b6ef-13c23741fe20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:46.727213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8dc15d88-1de1-44b3-8842-95d20b41ac60 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:46.745804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc275d1-2dd4-4650-b6ef-13c23741fe20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:46.752483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc275d1-2dd4-4650-b6ef-13c23741fe20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:46.753180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc275d1-2dd4-4650-b6ef-13c23741fe20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:46.756953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc275d1-2dd4-4650-b6ef-13c23741fe20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:46.757896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc275d1-2dd4-4650-b6ef-13c23741fe20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:46.758477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-efc275d1-2dd4-4650-b6ef-13c23741fe20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:46.758979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dc15d88-1de1-44b3-8842-95d20b41ac60 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:46.766420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dc15d88-1de1-44b3-8842-95d20b41ac60 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:46.766871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dc15d88-1de1-44b3-8842-95d20b41ac60 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:46.767636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dc15d88-1de1-44b3-8842-95d20b41ac60 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:46.768083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dc15d88-1de1-44b3-8842-95d20b41ac60 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:46.768504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8dc15d88-1de1-44b3-8842-95d20b41ac60 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:46.883120 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8dc15d88-1de1-44b3-8842-95d20b41ac60 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.171242 Sep 27 10:22:46.887635 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2382/5204] 10.222.0.26 () {60 vars in 1343 bytes} [Tue Sep 27 10:22:46 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d/os-volume_attachments => generated 25 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:46.999201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2788284d-797a-4e05-a86f-35a869271648 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:47.000064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2788284d-797a-4e05-a86f-35a869271648 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:47.018565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2788284d-797a-4e05-a86f-35a869271648 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:47.019134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2788284d-797a-4e05-a86f-35a869271648 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:47.019518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2788284d-797a-4e05-a86f-35a869271648 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:47.020537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2788284d-797a-4e05-a86f-35a869271648 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:47.021495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2788284d-797a-4e05-a86f-35a869271648 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:47.022091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2788284d-797a-4e05-a86f-35a869271648 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:47.099797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2788284d-797a-4e05-a86f-35a869271648 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:22:47.319229 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2788284d-797a-4e05-a86f-35a869271648 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "DELETE /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 204 len: 0 microversion: 2.20 time: 0.324204 Sep 27 10:22:47.321994 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2383/5205] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:22:46 2022] DELETE /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Sep 27 10:22:47.330521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:47.331705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:47.342676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:47.344289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:47.345713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:47.347000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:47.350516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:47.351363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:47.605725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-efc275d1-2dd4-4650-b6ef-13c23741fe20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:47.619276 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-efc275d1-2dd4-4650-b6ef-13c23741fe20 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1470 microversion: 2.1 time: 0.898578 Sep 27 10:22:47.622038 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2823/5206] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:46 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1470 bytes in 903 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:48.653300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8349742e-0fb5-42b8-b79a-bec148aa5701 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:48.653966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8349742e-0fb5-42b8-b79a-bec148aa5701 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:48.664706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=65,nova_api:DELETE=2,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:48.670406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8349742e-0fb5-42b8-b79a-bec148aa5701 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:48.671230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8349742e-0fb5-42b8-b79a-bec148aa5701 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:48.671903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8349742e-0fb5-42b8-b79a-bec148aa5701 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:48.674871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8349742e-0fb5-42b8-b79a-bec148aa5701 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:48.679363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8349742e-0fb5-42b8-b79a-bec148aa5701 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:48.679363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8349742e-0fb5-42b8-b79a-bec148aa5701 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:49.219149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8349742e-0fb5-42b8-b79a-bec148aa5701 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:49.230216 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8349742e-0fb5-42b8-b79a-bec148aa5701 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1470 microversion: 2.1 time: 0.581026 Sep 27 10:22:49.233112 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2824/5207] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:48 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1470 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:22:49.283962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:49.302738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:49.304436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:49.305701 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-131abdac-15e3-4c43-a939-6130e896a06d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1723 microversion: 2.20 time: 1.980427 Sep 27 10:22:49.334760 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2384/5208] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:47 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1723 bytes in 2002 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:22:50.081552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5a34ed25-e979-46df-890b-326115747e03 req-32306736-974f-4cdc-9608-caefc6a1a069 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a", "name": "network-vif-deleted", "tag": "583d8591-4753-421d-b0db-6174d4df0a77"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:50.115219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5a34ed25-e979-46df-890b-326115747e03 req-32306736-974f-4cdc-9608-caefc6a1a069 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:50.116127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5a34ed25-e979-46df-890b-326115747e03 req-32306736-974f-4cdc-9608-caefc6a1a069 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:50.118259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5a34ed25-e979-46df-890b-326115747e03 req-32306736-974f-4cdc-9608-caefc6a1a069 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:50.159288 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5a34ed25-e979-46df-890b-326115747e03 req-32306736-974f-4cdc-9608-caefc6a1a069 service nova] Creating event network-vif-deleted:583d8591-4753-421d-b0db-6174d4df0a77 for instance 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a on nch2-master-1 Sep 27 10:22:50.193780 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5a34ed25-e979-46df-890b-326115747e03 req-32306736-974f-4cdc-9608-caefc6a1a069 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.114038 Sep 27 10:22:50.199814 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2825/5209] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:22:50.263905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:50.264434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:50.279325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:50.279840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:50.280294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:50.304118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:50.309071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.024s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:50.310761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:50.355487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:50.356391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:50.375456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:50.377326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:50.379059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:50.382366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:50.383661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:50.384121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:50.464200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:50.464729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:50.465394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:50.468296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lazy-loading 'fault' on Instance uuid 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:22:50.692735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b652b17bea8942c4b4a30034c6c1f44c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ab5aec6b80a447bab8eb4f1adf06e61f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:50.705159 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6c7c5c13-7c3e-4612-b553-13c01c865fb9 tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 200 len: 1304 microversion: 2.1 time: 0.445202 Sep 27 10:22:50.725882 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2385/5210] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:50 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 1304 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:22:51.504264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:51.517874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:51.528476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:51.538752 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a09283f8-3793-4695-a18b-652b3e8dede2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1723 microversion: 2.20 time: 1.189964 Sep 27 10:22:51.542045 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2826/5211] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:50 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1723 bytes in 1196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:22:51.736335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:51.737163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] [instance: 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:51.742379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:22:51.755039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:51.759915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:51.762674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:51.765223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:51.769333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:51.770685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:51.822555 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] HTTP exception thrown: Instance 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a could not be found. Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Request method failure captured: Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a HTTP/1.1 Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs7YeTwSppOkXvQUrYlhV-xjTRoAzNLHwLm_i3sXyRjWDb84IttcgfG3_PtD9H_4P_Fm1ZL1z3L4qqqhCeAtj_-wxpEgyaISpInS4GwEDr14DZjbw6eOtS5Zt0Nj0c0nSxNxmpX0JvdPHVIXelpEK2oXSPFpYbF1OTBnQr5dMduLf-0 Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: ab5aec6b80a447bab8eb4f1adf06e61f Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-VolumesNegativeTest-72830281 Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ab5aec6b80a447bab8eb4f1adf06e61f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ab5aec6b80a447bab8eb4f1adf06e61f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ab5aec6b80a447bab8eb4f1adf06e61f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ab5aec6b80a447bab8eb4f1adf06e61f"}]}] Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-VolumesNegativeTest-72830281 Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: ab5aec6b80a447bab8eb4f1adf06e61f Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-VolumesNegativeTest-72830281 Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-VolumesNegativeTest-72830281-project Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b652b17bea8942c4b4a30034c6c1f44c Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-VolumesNegativeTest-72830281-project Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a could not be found. Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a'} Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a could not be found. Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a could not be found. Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:22:51.824917 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:22:51.828415 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:51.828415 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:22:51.828415 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:22:51.828415 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:22:51.828415 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:22:51.828415 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a could not be found. Sep 27 10:22:51.828415 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:22:51.833479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] Returning 404 to user: Instance 1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:22:51.834651 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-be9b660f-f826-423c-bdcb-64858ab2476e tempest-VolumesNegativeTest-72830281 tempest-VolumesNegativeTest-72830281-project] 10.222.0.26 "GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a" status: 404 len: 111 microversion: 2.1 time: 0.102631 Sep 27 10:22:51.853331 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2386/5212] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:22:51 2022] GET /compute/v2.1/servers/1d0ac9c2-d861-4b2a-b3ad-7431f053ab5a => generated 111 bytes in 122 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:22:52.573488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:52.574565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:52.602359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:52.603626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:52.607234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:52.611491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:52.614578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:52.616412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:53.112832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7247fdf7-a489-4315-a26b-9fd91a3d0d48 req-659c132a-8aee-4f04-a856-e6b2fe9165f8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ac6dabf-1315-4e05-ab6c-22211dc2440b", "name": "network-vif-plugged", "status": "completed", "tag": "b894a455-9bb5-443a-84fe-e04c8d0c3e28"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:53.139407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7247fdf7-a489-4315-a26b-9fd91a3d0d48 req-659c132a-8aee-4f04-a856-e6b2fe9165f8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:53.150734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7247fdf7-a489-4315-a26b-9fd91a3d0d48 req-659c132a-8aee-4f04-a856-e6b2fe9165f8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:53.151379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7247fdf7-a489-4315-a26b-9fd91a3d0d48 req-659c132a-8aee-4f04-a856-e6b2fe9165f8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:53.261116 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7247fdf7-a489-4315-a26b-9fd91a3d0d48 req-659c132a-8aee-4f04-a856-e6b2fe9165f8 service nova] Creating event network-vif-plugged:b894a455-9bb5-443a-84fe-e04c8d0c3e28 for instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b on nch1-master-1 Sep 27 10:22:53.262996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:53.280073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:53.301625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:53.306296 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b0529b58-0a3d-42c3-ad54-632c5b4fdf86 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1655 microversion: 2.20 time: 0.738481 Sep 27 10:22:53.306914 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2827/5213] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:52 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1655 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:22:53.311274 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7247fdf7-a489-4315-a26b-9fd91a3d0d48 req-659c132a-8aee-4f04-a856-e6b2fe9165f8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.191897 Sep 27 10:22:53.314151 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2387/5214] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:22:54.336229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:54.337393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:54.366541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:54.367056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:54.372389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:54.373274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:54.384252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:54.385107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:54.423448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-aee68269-590f-4ade-bf90-2e62921c8c27 req-529a2bc0-ca13-4ba6-9f97-5e03789f1dc9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "691b2be7-d5b8-409c-ba22-5ab8da50b80d", "name": "network-vif-deleted", "tag": "11b232b4-3236-4917-9ebb-02d7dbd3132e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:22:54.442046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-aee68269-590f-4ade-bf90-2e62921c8c27 req-529a2bc0-ca13-4ba6-9f97-5e03789f1dc9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:54.442543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-aee68269-590f-4ade-bf90-2e62921c8c27 req-529a2bc0-ca13-4ba6-9f97-5e03789f1dc9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:54.442988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-aee68269-590f-4ade-bf90-2e62921c8c27 req-529a2bc0-ca13-4ba6-9f97-5e03789f1dc9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:54.477394 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-aee68269-590f-4ade-bf90-2e62921c8c27 req-529a2bc0-ca13-4ba6-9f97-5e03789f1dc9 service nova] Creating event network-vif-deleted:11b232b4-3236-4917-9ebb-02d7dbd3132e for instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d on nch1-master-1 Sep 27 10:22:54.491400 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-aee68269-590f-4ade-bf90-2e62921c8c27 req-529a2bc0-ca13-4ba6-9f97-5e03789f1dc9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.534814 Sep 27 10:22:54.494815 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2828/5215] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:22:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:22:54.526669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:54.529585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:54.530062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:54.530535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lazy-loading 'fault' on Instance uuid 691b2be7-d5b8-409c-ba22-5ab8da50b80d {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:22:55.145518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:55.158380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:55.161782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:22:55.169273 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a1dd5d6c-f7e7-4e3e-88f9-be88fbde0c64 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 200 len: 1371 microversion: 2.20 time: 0.834801 Sep 27 10:22:55.171496 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2388/5216] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:54 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 1371 bytes in 841 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:22:56.201290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:56.202513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:22:56.232022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:56.232317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:56.235751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:56.238532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:22:56.239446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:22:56.240053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:22:56.293116 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] HTTP exception thrown: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Request method failure captured: Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d HTTP/1.1 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs43MkA6YH_ixExFA1qRpWwXmXGbF-X_Bicstp25yt5cwZaJ_7_F6w6FNYvUTVn_zcFt-0zaviYlvixPIic7TZLOz3r-vbaQPyBInmtKjHDdi7-muE6EYrBHzESgBlFeUkDC1UgwRkklVZ1DXjJLATMTWHchIJRaap3LucgH2a8cfv4 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b590d2de576e41aab42f79e4daef9c63"}]}] Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 74878b715fc94b23bb139cb30d8d20c0 Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '691b2be7-d5b8-409c-ba22-5ab8da50b80d'} Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:22:56.294827 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:22:56.298600 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:22:56.298600 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:22:56.298600 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:22:56.298600 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:22:56.298600 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:22:56.298600 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:22:56.298600 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:22:56.298600 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:22:56.303679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Returning 404 to user: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:22:56.306915 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f75c9f65-fcdb-4f9f-893e-5122baffd7eb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 404 len: 111 microversion: 2.20 time: 0.110600 Sep 27 10:22:56.310178 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2829/5217] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:22:56 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 111 bytes in 114 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:22:56.320560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0c96e382-fb8d-40e9-8bdf-81a8a8dbd2f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '.version_select at 0x7f7f629d83a0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:22:56.419884 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0c96e382-fb8d-40e9-8bdf-81a8a8dbd2f6 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1644362632" status: 204 len: 0 microversion: 2.20 time: 0.103280 Sep 27 10:22:56.424804 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2389/5218] 10.222.0.26 () {60 vars in 1262 bytes} [Tue Sep 27 10:22:56 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1644362632 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:23:00.908139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-674a6e10-ef29-42a3-9a49-4b989aefa6f8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Action: 'create', calling method: .version_select at 0x7f7f62425310>, body: {"keypair": {"name": "tempest-keypair-80707728", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNycrrnjRn2wkiaQ2/nHkdUXXN/EzJMZ+k2QVMuMiNZLKYpO64c6r8R5rhKOHTO+CHeN61jNSByFyU04eoFRXBV45aqLkfAKVlT+rxyCmRM4z2RWkuVClB12UMpIMfBhNA=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:00.939412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-674a6e10-ef29-42a3-9a49-4b989aefa6f8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Getting quotas for project b590d2de576e41aab42f79e4daef9c63. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:23:00.948936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-674a6e10-ef29-42a3-9a49-4b989aefa6f8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Getting quotas for user 74878b715fc94b23bb139cb30d8d20c0 and project b590d2de576e41aab42f79e4daef9c63. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:23:00.999386 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-674a6e10-ef29-42a3-9a49-4b989aefa6f8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 399 microversion: 2.20 time: 0.101137 Sep 27 10:23:01.008512 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2830/5219] 10.222.0.26 () {62 vars in 1200 bytes} [Tue Sep 27 10:23:00 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 111 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 27 10:23:07.276207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=83,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:07.278509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=10,nova_api:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:07.657011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2d0b9b89-4061-4249-8306-be4e0b43d2cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Action: 'create', calling method: .version_select at 0x7f7f6271f670>, body: {"keypair": {"name": "tempest-TestServerBasicOps-829950563", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNJJ8AYeCp48nTr79G2OribhAaDBhvH36Zev3O6rJCy3+I9gfvlitxt5326YYO8+6OT2Fp19e2NuzH6LleCqxy2Lqn66/2P6ijy8xWaqnEPXBoAd9KdN7m9x6JBEEcDKyg=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:07.696676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-2d0b9b89-4061-4249-8306-be4e0b43d2cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Getting quotas for project 4195812e6f274bbba026e71b7fc4e231. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:23:07.710290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-2d0b9b89-4061-4249-8306-be4e0b43d2cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Getting quotas for user dc394c36e4bb436fabbecada1990b41e and project 4195812e6f274bbba026e71b7fc4e231. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:23:07.780249 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2d0b9b89-4061-4249-8306-be4e0b43d2cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.512617 Sep 27 10:23:07.781391 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2390/5220] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:23:07 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:23:08.482148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=50,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:08.493650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1417353190", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a24d1d04-2dec-42a9-9c22-fd56aad28b57"}], "security_groups": [{"name": "tempest-securitygroup--1245017972"}], "key_name": "tempest-keypair-80707728", "user_data": "}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:08.752602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] validate_networks() for [('a24d1d04-2dec-42a9-9c22-fd56aad28b57', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:23:10.768008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:23:10.769789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:23:10.770560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:23:10.771238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:23:10.772428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:23:10.796554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:10.798963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:10.800122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:10.803206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:10.803838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:10.806902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:10.860978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Getting quotas for project b590d2de576e41aab42f79e4daef9c63. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:23:10.876226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Getting quotas for user 74878b715fc94b23bb139cb30d8d20c0 and project b590d2de576e41aab42f79e4daef9c63. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:23:10.913883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:23:10.916487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:23:11.056049 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-37cda84f-917c-4ed1-9097-27a6ea6a677f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 2.576298 Sep 27 10:23:11.060369 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2831/5221] 10.222.0.26 () {62 vars in 1188 bytes} [Tue Sep 27 10:23:08 2022] POST /compute/v2.1/servers => generated 402 bytes in 2581 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 27 10:23:11.083946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d0135708-a0e4-433b-accc-7b1dd2479ea7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:11.084565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d0135708-a0e4-433b-accc-7b1dd2479ea7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:11.352402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0135708-a0e4-433b-accc-7b1dd2479ea7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:11.364357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0135708-a0e4-433b-accc-7b1dd2479ea7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:11.365859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0135708-a0e4-433b-accc-7b1dd2479ea7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:11.372261 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d0135708-a0e4-433b-accc-7b1dd2479ea7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1280 microversion: 2.20 time: 0.298827 Sep 27 10:23:11.375245 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2391/5222] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:11 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1280 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:23:12.228729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerBasicOps-server-311147561", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "key_name": "tempest-TestServerBasicOps-829950563", "security_groups": [{"name": "tempest-secgroup-smoke-769200584"}], "config_drive": true, "metadata": {"meta1": "data1", "meta2": "data2", "metaN": "dataN"}, "networks": [{"uuid": "21c4998b-302d-44d4-93b4-b5ba726a3ed7"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:12.401776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:12.403219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:12.422461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:12.422762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:12.423202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:12.423946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:12.424414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:12.451688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:12.822991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:12.834387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:12.835854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:12.853522 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea3cebe5-6135-49fd-8827-3f19efd1ed98 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1276 microversion: 2.20 time: 0.439317 Sep 27 10:23:12.855442 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2392/5223] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:12 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1276 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:23:13.746982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] validate_networks() for [('21c4998b-302d-44d4-93b4-b5ba726a3ed7', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:23:13.884134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:13.884903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:13.892974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:13.893486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:13.900598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:13.901504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:13.902003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:13.902453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:14.513946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:14.529832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:14.551344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:14.556914 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-98f9c7d8-5858-4f65-9d4c-4b6322f1180b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1338 microversion: 2.20 time: 0.676889 Sep 27 10:23:14.558789 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2393/5224] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:13 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1338 bytes in 680 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:23:14.670826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Getting quotas for project 4195812e6f274bbba026e71b7fc4e231. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:23:14.692592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Getting quotas for user dc394c36e4bb436fabbecada1990b41e and project 4195812e6f274bbba026e71b7fc4e231. Resources: dict_keys(['metadata_items']) {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:23:14.759719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:23:14.760347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:23:14.760645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:23:14.761096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:23:14.761522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:23:14.814905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:14.816319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:14.821137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:14.837707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:14.840409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:14.843884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:14.900057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Getting quotas for project 4195812e6f274bbba026e71b7fc4e231. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:23:14.913824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Getting quotas for user dc394c36e4bb436fabbecada1990b41e and project 4195812e6f274bbba026e71b7fc4e231. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:23:14.949017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:23:14.953557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:23:15.080056 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ff680f6e-22ce-4fe8-9e10-3f38c654ec39 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 2.855213 Sep 27 10:23:15.088382 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2832/5225] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:23:12 2022] POST /compute/v2.1/servers => generated 405 bytes in 2864 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:23:15.097029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-99bf87b8-d97c-40b4-813a-e31437f379d2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:15.097668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-99bf87b8-d97c-40b4-813a-e31437f379d2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:15.590463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:15.591454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:15.628979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:15.630046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:15.630702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:15.632158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:15.632920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:15.641681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:15.790160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:15.801330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:15.803919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:15.806448 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-34379deb-1d07-438d-945a-7c80ff34908f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1338 microversion: 2.20 time: 0.221441 Sep 27 10:23:15.807290 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2833/5226] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:15 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1338 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:23:16.084137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-99bf87b8-d97c-40b4-813a-e31437f379d2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:16.096060 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-99bf87b8-d97c-40b4-813a-e31437f379d2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1301 microversion: 2.1 time: 1.003464 Sep 27 10:23:16.109602 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2394/5227] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:15 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1301 bytes in 1006 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:16.674603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0ca3a95-b0b4-4e4a-9827-e6140e6ec348 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:16.675433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f0ca3a95-b0b4-4e4a-9827-e6140e6ec348 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:16.691985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ca3a95-b0b4-4e4a-9827-e6140e6ec348 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:16.701891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ca3a95-b0b4-4e4a-9827-e6140e6ec348 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:16.702504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ca3a95-b0b4-4e4a-9827-e6140e6ec348 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:16.705565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ca3a95-b0b4-4e4a-9827-e6140e6ec348 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:16.706048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ca3a95-b0b4-4e4a-9827-e6140e6ec348 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:16.706524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0ca3a95-b0b4-4e4a-9827-e6140e6ec348 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:16.840221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:16.841231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:16.861388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:16.862113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:16.862778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:16.864279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:16.865244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:16.865903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:17.094977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f0ca3a95-b0b4-4e4a-9827-e6140e6ec348 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:17.125368 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f0ca3a95-b0b4-4e4a-9827-e6140e6ec348 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.455727 Sep 27 10:23:17.128772 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2834/5228] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:16 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:17.152884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-340945cf-a9a2-417e-94ba-c925baf3bf9b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:17.155567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-340945cf-a9a2-417e-94ba-c925baf3bf9b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:17.175433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:17.189570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:17.190407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-340945cf-a9a2-417e-94ba-c925baf3bf9b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:17.191169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-340945cf-a9a2-417e-94ba-c925baf3bf9b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:17.193808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:17.197095 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-413f707a-5853-4d35-9271-3af673fd26b5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1338 microversion: 2.20 time: 0.366263 Sep 27 10:23:17.209103 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2395/5229] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:16 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1338 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:23:17.209328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-340945cf-a9a2-417e-94ba-c925baf3bf9b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:17.209328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-340945cf-a9a2-417e-94ba-c925baf3bf9b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:17.211495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-340945cf-a9a2-417e-94ba-c925baf3bf9b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:17.213895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-340945cf-a9a2-417e-94ba-c925baf3bf9b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:17.226467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-43f86be8-5671-4c06-8eca-c70b297557e9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"createBackup": {"backup_type": "daily", "rotation": 2, "name": "tempest-backup-3-1549343132"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:17.238970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-43f86be8-5671-4c06-8eca-c70b297557e9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:17.270197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f86be8-5671-4c06-8eca-c70b297557e9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:17.270936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f86be8-5671-4c06-8eca-c70b297557e9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:17.271845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f86be8-5671-4c06-8eca-c70b297557e9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:17.272858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f86be8-5671-4c06-8eca-c70b297557e9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:17.273515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f86be8-5671-4c06-8eca-c70b297557e9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:17.274253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43f86be8-5671-4c06-8eca-c70b297557e9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:17.558554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-340945cf-a9a2-417e-94ba-c925baf3bf9b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:17.574738 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-340945cf-a9a2-417e-94ba-c925baf3bf9b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1361 microversion: 2.1 time: 0.438258 Sep 27 10:23:17.579847 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2835/5230] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:17 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1361 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:17.722172 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-43f86be8-5671-4c06-8eca-c70b297557e9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.520745 Sep 27 10:23:17.726106 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2396/5231] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:23:17 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 522 msecs (HTTP/1.1 202) 10 headers in 456 bytes (1 switches on core 0) Sep 27 10:23:17.731764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3e056a3-ddb5-4979-b0de-d5ee536d0afb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:17.732836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d3e056a3-ddb5-4979-b0de-d5ee536d0afb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:17.748606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3e056a3-ddb5-4979-b0de-d5ee536d0afb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:17.749651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3e056a3-ddb5-4979-b0de-d5ee536d0afb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:17.750314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3e056a3-ddb5-4979-b0de-d5ee536d0afb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:17.759239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3e056a3-ddb5-4979-b0de-d5ee536d0afb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:17.760759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3e056a3-ddb5-4979-b0de-d5ee536d0afb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:17.761886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3e056a3-ddb5-4979-b0de-d5ee536d0afb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:18.225356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:18.226770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:18.248076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:18.248869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:18.249568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:18.255099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:18.256232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:18.256927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:18.383879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:18.402448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:18.404679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:18.406171 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d6f88dae-e496-448c-a3de-0a4e26e0e2da tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1338 microversion: 2.20 time: 0.186219 Sep 27 10:23:18.407290 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2397/5232] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:18 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1338 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:23:18.427126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d3e056a3-ddb5-4979-b0de-d5ee536d0afb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:18.438635 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d3e056a3-ddb5-4979-b0de-d5ee536d0afb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.710730 Sep 27 10:23:18.443756 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2836/5233] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:17 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 717 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:18.603623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-266c1c10-4912-49a6-877b-c640b963dec9 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:18.604886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-266c1c10-4912-49a6-877b-c640b963dec9 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:18.617993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266c1c10-4912-49a6-877b-c640b963dec9 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:18.618729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266c1c10-4912-49a6-877b-c640b963dec9 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:18.620157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266c1c10-4912-49a6-877b-c640b963dec9 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:18.622682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266c1c10-4912-49a6-877b-c640b963dec9 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:18.623480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266c1c10-4912-49a6-877b-c640b963dec9 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:18.624066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-266c1c10-4912-49a6-877b-c640b963dec9 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:18.814562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-266c1c10-4912-49a6-877b-c640b963dec9 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:18.835756 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-266c1c10-4912-49a6-877b-c640b963dec9 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1359 microversion: 2.1 time: 0.239402 Sep 27 10:23:18.837030 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2398/5234] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:18 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1359 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:19.434552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:19.435160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:19.438292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:19.453891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:19.454573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:19.455177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:19.456064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:19.456697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:19.457272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:19.467259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-227dbe22-7a23-4c5c-855f-4563b6f350c9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:19.467971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-227dbe22-7a23-4c5c-855f-4563b6f350c9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:19.479957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-227dbe22-7a23-4c5c-855f-4563b6f350c9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:19.481832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-227dbe22-7a23-4c5c-855f-4563b6f350c9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:19.482579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-227dbe22-7a23-4c5c-855f-4563b6f350c9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:19.483469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-227dbe22-7a23-4c5c-855f-4563b6f350c9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:19.490407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-227dbe22-7a23-4c5c-855f-4563b6f350c9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:19.491319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-227dbe22-7a23-4c5c-855f-4563b6f350c9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:20.153731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-227dbe22-7a23-4c5c-855f-4563b6f350c9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:20.170786 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-227dbe22-7a23-4c5c-855f-4563b6f350c9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1687 microversion: 2.1 time: 0.707311 Sep 27 10:23:20.172032 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2399/5235] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:19 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1687 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:23:20.178748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f0fd0e2b-0919-4a4e-9baa-ed293cdec546 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:20.180405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f0fd0e2b-0919-4a4e-9baa-ed293cdec546 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:20.193361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fd0e2b-0919-4a4e-9baa-ed293cdec546 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:20.194509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fd0e2b-0919-4a4e-9baa-ed293cdec546 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:20.195159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fd0e2b-0919-4a4e-9baa-ed293cdec546 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:20.196566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fd0e2b-0919-4a4e-9baa-ed293cdec546 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:20.197298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fd0e2b-0919-4a4e-9baa-ed293cdec546 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:20.197997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0fd0e2b-0919-4a4e-9baa-ed293cdec546 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:20.335109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f0fd0e2b-0919-4a4e-9baa-ed293cdec546 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:20.393254 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f0fd0e2b-0919-4a4e-9baa-ed293cdec546 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1359 microversion: 2.1 time: 0.214807 Sep 27 10:23:20.415242 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2400/5236] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:20 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1359 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:20.439583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:20.474720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:20.478880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:20.481002 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4adc572a-ff48-4317-a4a6-45fd8f098feb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1406 microversion: 2.20 time: 1.050151 Sep 27 10:23:20.483142 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2837/5237] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:19 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1406 bytes in 1053 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:23:20.826269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b7250cc4-4a7d-4f7d-a7a1-5e00af4b3400 req-798819f6-2a17-46c4-afcf-ced493a99fbf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ac6dabf-1315-4e05-ab6c-22211dc2440b", "name": "network-vif-unplugged", "status": "completed", "tag": "b894a455-9bb5-443a-84fe-e04c8d0c3e28"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:20.845307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b7250cc4-4a7d-4f7d-a7a1-5e00af4b3400 req-798819f6-2a17-46c4-afcf-ced493a99fbf service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:20.860853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b7250cc4-4a7d-4f7d-a7a1-5e00af4b3400 req-798819f6-2a17-46c4-afcf-ced493a99fbf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:20.862172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b7250cc4-4a7d-4f7d-a7a1-5e00af4b3400 req-798819f6-2a17-46c4-afcf-ced493a99fbf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:20.909884 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b7250cc4-4a7d-4f7d-a7a1-5e00af4b3400 req-798819f6-2a17-46c4-afcf-ced493a99fbf service nova] Creating event network-vif-unplugged:b894a455-9bb5-443a-84fe-e04c8d0c3e28 for instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b on nch1-master-1 Sep 27 10:23:20.921353 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b7250cc4-4a7d-4f7d-a7a1-5e00af4b3400 req-798819f6-2a17-46c4-afcf-ced493a99fbf service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.106218 Sep 27 10:23:20.922829 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2401/5238] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:23:20 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:23:21.199122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0d7d84f9-28d7-49fa-9c9a-692906c59e4a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:21.199750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0d7d84f9-28d7-49fa-9c9a-692906c59e4a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:21.215540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d7d84f9-28d7-49fa-9c9a-692906c59e4a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:21.216349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d7d84f9-28d7-49fa-9c9a-692906c59e4a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:21.217011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d7d84f9-28d7-49fa-9c9a-692906c59e4a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:21.217890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d7d84f9-28d7-49fa-9c9a-692906c59e4a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:21.225571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d7d84f9-28d7-49fa-9c9a-692906c59e4a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:21.225957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d7d84f9-28d7-49fa-9c9a-692906c59e4a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:21.422667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-412db40c-7a29-433b-9707-31a7df45f9fd tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:21.423385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-412db40c-7a29-433b-9707-31a7df45f9fd tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:21.430385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-412db40c-7a29-433b-9707-31a7df45f9fd tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:21.431164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-412db40c-7a29-433b-9707-31a7df45f9fd tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:21.431798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-412db40c-7a29-433b-9707-31a7df45f9fd tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:21.433125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-412db40c-7a29-433b-9707-31a7df45f9fd tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:21.433779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-412db40c-7a29-433b-9707-31a7df45f9fd tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:21.434393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-412db40c-7a29-433b-9707-31a7df45f9fd tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:21.826283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-412db40c-7a29-433b-9707-31a7df45f9fd tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:21.842018 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-412db40c-7a29-433b-9707-31a7df45f9fd tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1426 microversion: 2.1 time: 0.423544 Sep 27 10:23:21.842965 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2402/5239] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:21 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1426 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:21.854564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:21.855211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:21.885502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:21.886331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:21.886974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:21.887715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:21.888363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:21.888830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:21.996944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0d7d84f9-28d7-49fa-9c9a-692906c59e4a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:22.010526 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0d7d84f9-28d7-49fa-9c9a-692906c59e4a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1687 microversion: 2.1 time: 0.815188 Sep 27 10:23:22.025305 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2838/5240] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:21 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1687 bytes in 831 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:22.230175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-804fe997-bcf8-493d-9954-245445f9471f req-06226aca-662b-4803-af68-4044ce9b76ec service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77037d39-844c-4761-88cc-899710443401", "tag": "6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:22.250887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-804fe997-bcf8-493d-9954-245445f9471f req-06226aca-662b-4803-af68-4044ce9b76ec service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:22.253673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-804fe997-bcf8-493d-9954-245445f9471f req-06226aca-662b-4803-af68-4044ce9b76ec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:22.254217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-804fe997-bcf8-493d-9954-245445f9471f req-06226aca-662b-4803-af68-4044ce9b76ec service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:22.305591 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-804fe997-bcf8-493d-9954-245445f9471f req-06226aca-662b-4803-af68-4044ce9b76ec service nova] Creating event network-changed:6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1 for instance 77037d39-844c-4761-88cc-899710443401 on nch2-master-1 Sep 27 10:23:22.316277 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-804fe997-bcf8-493d-9954-245445f9471f req-06226aca-662b-4803-af68-4044ce9b76ec service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089315 Sep 27 10:23:22.323504 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2839/5241] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:23:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:23:22.688576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:22.699711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:22.710357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:22.711654 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8fa6f60d-8560-49a4-a93e-181da083ad21 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1406 microversion: 2.20 time: 0.867620 Sep 27 10:23:22.720083 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2403/5242] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:21 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1406 bytes in 876 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:23:22.869304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f55f4d69-c15e-4262-9250-9b2d582dfb70 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:22.870017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f55f4d69-c15e-4262-9250-9b2d582dfb70 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:22.885883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f55f4d69-c15e-4262-9250-9b2d582dfb70 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:22.886380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f55f4d69-c15e-4262-9250-9b2d582dfb70 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:22.886830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f55f4d69-c15e-4262-9250-9b2d582dfb70 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:22.887546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f55f4d69-c15e-4262-9250-9b2d582dfb70 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:22.888007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f55f4d69-c15e-4262-9250-9b2d582dfb70 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:22.888434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f55f4d69-c15e-4262-9250-9b2d582dfb70 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:23.038165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-24cd108b-115f-4b18-a23e-6927a8d4d54c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:23.039397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-24cd108b-115f-4b18-a23e-6927a8d4d54c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:23.055831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24cd108b-115f-4b18-a23e-6927a8d4d54c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:23.056798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24cd108b-115f-4b18-a23e-6927a8d4d54c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:23.057448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24cd108b-115f-4b18-a23e-6927a8d4d54c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:23.058367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24cd108b-115f-4b18-a23e-6927a8d4d54c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:23.058979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24cd108b-115f-4b18-a23e-6927a8d4d54c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:23.059556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24cd108b-115f-4b18-a23e-6927a8d4d54c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:23.341039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f55f4d69-c15e-4262-9250-9b2d582dfb70 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:23.349663 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f55f4d69-c15e-4262-9250-9b2d582dfb70 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1426 microversion: 2.1 time: 0.485348 Sep 27 10:23:23.361020 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2840/5243] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:22 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1426 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:23.665342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-24cd108b-115f-4b18-a23e-6927a8d4d54c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:23.670874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-b55893f8-c143-40a6-9490-e28fe615f970 req-ef248038-33a3-4756-9c85-5287256f7998 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e246452-0210-449b-b9b7-f1f42ccaa2eb", "tag": "14864aa7-71a6-4bce-944b-e859f11985e0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:23.692787 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-24cd108b-115f-4b18-a23e-6927a8d4d54c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.659202 Sep 27 10:23:23.693397 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2404/5244] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:23 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:23.720652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b55893f8-c143-40a6-9490-e28fe615f970 req-ef248038-33a3-4756-9c85-5287256f7998 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:23.740241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b55893f8-c143-40a6-9490-e28fe615f970 req-ef248038-33a3-4756-9c85-5287256f7998 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:23.740734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b55893f8-c143-40a6-9490-e28fe615f970 req-ef248038-33a3-4756-9c85-5287256f7998 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:23.747953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:23.748575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:23.807999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:23.808503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:23.808965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:23.809686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:23.810148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:23.810574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:23.833646 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b55893f8-c143-40a6-9490-e28fe615f970 req-ef248038-33a3-4756-9c85-5287256f7998 service nova] Creating event network-changed:14864aa7-71a6-4bce-944b-e859f11985e0 for instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb on nch1-master-1 Sep 27 10:23:23.855237 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-b55893f8-c143-40a6-9490-e28fe615f970 req-ef248038-33a3-4756-9c85-5287256f7998 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.193187 Sep 27 10:23:23.859423 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2841/5245] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:23:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:23:24.188313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:24.199536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:24.213376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:24.221323 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-37c02e34-c2ca-47f4-b4ed-c509b1ad63cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1406 microversion: 2.20 time: 0.488162 Sep 27 10:23:24.226744 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2405/5246] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:23 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1406 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:23:24.375819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-df23dfd1-d6e2-4a70-8fed-ed48a1e4be03 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:24.376540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-df23dfd1-d6e2-4a70-8fed-ed48a1e4be03 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:24.400616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df23dfd1-d6e2-4a70-8fed-ed48a1e4be03 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:24.401163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df23dfd1-d6e2-4a70-8fed-ed48a1e4be03 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:24.401615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df23dfd1-d6e2-4a70-8fed-ed48a1e4be03 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:24.402355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df23dfd1-d6e2-4a70-8fed-ed48a1e4be03 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:24.402820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df23dfd1-d6e2-4a70-8fed-ed48a1e4be03 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:24.403257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df23dfd1-d6e2-4a70-8fed-ed48a1e4be03 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:24.721662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ceb4531d-57cc-45d0-9920-de3c7db1760b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:24.722722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ceb4531d-57cc-45d0-9920-de3c7db1760b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:24.736489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceb4531d-57cc-45d0-9920-de3c7db1760b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:24.737329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceb4531d-57cc-45d0-9920-de3c7db1760b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:24.738724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceb4531d-57cc-45d0-9920-de3c7db1760b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:24.741843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceb4531d-57cc-45d0-9920-de3c7db1760b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:24.746613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceb4531d-57cc-45d0-9920-de3c7db1760b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:24.747347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ceb4531d-57cc-45d0-9920-de3c7db1760b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:24.878944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:24.931363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=20 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:25.332694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ceb4531d-57cc-45d0-9920-de3c7db1760b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:25.343726 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ceb4531d-57cc-45d0-9920-de3c7db1760b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.626264 Sep 27 10:23:25.361915 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2406/5247] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:24 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:25.377743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:25.378366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:25.407207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:25.407756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:25.408209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:25.408995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:25.409459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:25.409891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:25.561942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-df23dfd1-d6e2-4a70-8fed-ed48a1e4be03 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:25.578146 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-df23dfd1-d6e2-4a70-8fed-ed48a1e4be03 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1426 microversion: 2.1 time: 1.207766 Sep 27 10:23:25.586077 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2842/5248] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:24 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1426 bytes in 1216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:26.090347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:26.387128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e316e6a8-3914-4088-9fd6-1f5eac303c8d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:26.387910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e316e6a8-3914-4088-9fd6-1f5eac303c8d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:26.401095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e316e6a8-3914-4088-9fd6-1f5eac303c8d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:26.402725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e316e6a8-3914-4088-9fd6-1f5eac303c8d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:26.403294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e316e6a8-3914-4088-9fd6-1f5eac303c8d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:26.405641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e316e6a8-3914-4088-9fd6-1f5eac303c8d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:26.407578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e316e6a8-3914-4088-9fd6-1f5eac303c8d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:26.409765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e316e6a8-3914-4088-9fd6-1f5eac303c8d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:26.477951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:26.490765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:26.501006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:26.503970 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0dbbd7f-a283-4db5-9366-2f1991c728c1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1406 microversion: 2.20 time: 1.140950 Sep 27 10:23:26.506973 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2407/5249] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:25 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1406 bytes in 1144 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:23:26.622258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-edab61d1-739e-4d2c-b1c2-c4af4536e2b0 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:26.623972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-edab61d1-739e-4d2c-b1c2-c4af4536e2b0 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:26.635785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edab61d1-739e-4d2c-b1c2-c4af4536e2b0 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:26.636207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edab61d1-739e-4d2c-b1c2-c4af4536e2b0 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:26.638973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edab61d1-739e-4d2c-b1c2-c4af4536e2b0 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:26.639744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edab61d1-739e-4d2c-b1c2-c4af4536e2b0 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:26.640222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edab61d1-739e-4d2c-b1c2-c4af4536e2b0 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:26.640777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edab61d1-739e-4d2c-b1c2-c4af4536e2b0 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:26.912471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e316e6a8-3914-4088-9fd6-1f5eac303c8d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:26.925592 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e316e6a8-3914-4088-9fd6-1f5eac303c8d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.542478 Sep 27 10:23:26.951512 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2843/5250] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:26 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:27.251521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-edab61d1-739e-4d2c-b1c2-c4af4536e2b0 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:27.256741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:27.274147 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-edab61d1-739e-4d2c-b1c2-c4af4536e2b0 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1426 microversion: 2.1 time: 0.658696 Sep 27 10:23:27.278857 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2408/5251] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:26 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1426 bytes in 664 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:27.533803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:27.534350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:27.559274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:27.559783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:27.560227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:27.561022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:27.566800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:27.567493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:27.961760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-feb2e36c-7f23-4e90-a892-436d14ef42a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:27.962374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-feb2e36c-7f23-4e90-a892-436d14ef42a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:27.980556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feb2e36c-7f23-4e90-a892-436d14ef42a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:27.987466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feb2e36c-7f23-4e90-a892-436d14ef42a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:27.987964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feb2e36c-7f23-4e90-a892-436d14ef42a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:27.994714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feb2e36c-7f23-4e90-a892-436d14ef42a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:27.998169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feb2e36c-7f23-4e90-a892-436d14ef42a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:27.998655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feb2e36c-7f23-4e90-a892-436d14ef42a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:28.017791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:28.036802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:28.042563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:28.046826 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-55403cbf-f4d5-4917-b9a1-c2ad6fc1792e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1406 microversion: 2.20 time: 0.517097 Sep 27 10:23:28.054478 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2844/5252] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:27 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1406 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:23:28.307578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-24080cad-004c-425b-b731-70cf147237b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:28.308320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-24080cad-004c-425b-b731-70cf147237b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:28.318018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24080cad-004c-425b-b731-70cf147237b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:28.320529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24080cad-004c-425b-b731-70cf147237b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:28.322680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24080cad-004c-425b-b731-70cf147237b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:28.326171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24080cad-004c-425b-b731-70cf147237b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:28.328939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24080cad-004c-425b-b731-70cf147237b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:28.331367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-24080cad-004c-425b-b731-70cf147237b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:28.386036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-feb2e36c-7f23-4e90-a892-436d14ef42a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:28.394281 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-feb2e36c-7f23-4e90-a892-436d14ef42a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.438863 Sep 27 10:23:28.395916 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2409/5253] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:27 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:29.077933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:29.078559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:29.096841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:29.103508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:29.107287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:29.108242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:29.118470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:29.122066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:30.022460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:30.041746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:30.047390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:30.050082 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aacc9d1c-fcee-4b36-9cb0-988236084272 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1406 microversion: 2.20 time: 0.976346 Sep 27 10:23:30.053866 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2410/5254] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:29 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1406 bytes in 980 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:23:30.075151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-9e2d0daf-4f47-430a-a37e-70aeca6f9929 req-418c2fc7-b954-4d7c-b64e-f0fcbc16153b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ac6dabf-1315-4e05-ab6c-22211dc2440b", "name": "network-vif-plugged", "status": "completed", "tag": "b894a455-9bb5-443a-84fe-e04c8d0c3e28"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:30.089555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9e2d0daf-4f47-430a-a37e-70aeca6f9929 req-418c2fc7-b954-4d7c-b64e-f0fcbc16153b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:30.090700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9e2d0daf-4f47-430a-a37e-70aeca6f9929 req-418c2fc7-b954-4d7c-b64e-f0fcbc16153b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:30.090700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9e2d0daf-4f47-430a-a37e-70aeca6f9929 req-418c2fc7-b954-4d7c-b64e-f0fcbc16153b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:30.118865 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-9e2d0daf-4f47-430a-a37e-70aeca6f9929 req-418c2fc7-b954-4d7c-b64e-f0fcbc16153b service nova] Creating event network-vif-plugged:b894a455-9bb5-443a-84fe-e04c8d0c3e28 for instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b on nch1-master-1 Sep 27 10:23:30.125690 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-9e2d0daf-4f47-430a-a37e-70aeca6f9929 req-418c2fc7-b954-4d7c-b64e-f0fcbc16153b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070090 Sep 27 10:23:30.135570 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2411/5255] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:23:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:23:30.141771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1b80b8ed-8b81-472e-ade2-455f24fb411a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:30.143430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1b80b8ed-8b81-472e-ade2-455f24fb411a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:30.158216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b80b8ed-8b81-472e-ade2-455f24fb411a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:30.158742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b80b8ed-8b81-472e-ade2-455f24fb411a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:30.159172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b80b8ed-8b81-472e-ade2-455f24fb411a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:30.160033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b80b8ed-8b81-472e-ade2-455f24fb411a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:30.160573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b80b8ed-8b81-472e-ade2-455f24fb411a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:30.162105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b80b8ed-8b81-472e-ade2-455f24fb411a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:30.373441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-24080cad-004c-425b-b731-70cf147237b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:30.386524 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-24080cad-004c-425b-b731-70cf147237b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1426 microversion: 2.1 time: 2.083660 Sep 27 10:23:30.392284 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2845/5256] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:28 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1426 bytes in 2090 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:30.580393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1b80b8ed-8b81-472e-ade2-455f24fb411a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:30.592867 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1b80b8ed-8b81-472e-ade2-455f24fb411a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.453909 Sep 27 10:23:30.609641 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2412/5257] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:30 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:31.080951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:31.082131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:31.102317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:31.104360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:31.106352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:31.107347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:31.109101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:31.109815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:31.417698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5b07f673-297e-4a5b-be0d-bd560ac2e995 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:31.418337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5b07f673-297e-4a5b-be0d-bd560ac2e995 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:31.430497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b07f673-297e-4a5b-be0d-bd560ac2e995 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:31.431012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b07f673-297e-4a5b-be0d-bd560ac2e995 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:31.431463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b07f673-297e-4a5b-be0d-bd560ac2e995 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:31.432205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b07f673-297e-4a5b-be0d-bd560ac2e995 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:31.432691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b07f673-297e-4a5b-be0d-bd560ac2e995 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:31.446992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b07f673-297e-4a5b-be0d-bd560ac2e995 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:32.038392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b07f673-297e-4a5b-be0d-bd560ac2e995 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:32.047324 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5b07f673-297e-4a5b-be0d-bd560ac2e995 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1426 microversion: 2.1 time: 0.633739 Sep 27 10:23:32.048273 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2413/5258] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:31 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1426 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:32.054813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fa8bedad-da09-4c83-9abf-2a81fb07d553 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:32.055729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa8bedad-da09-4c83-9abf-2a81fb07d553 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:32.062891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8bedad-da09-4c83-9abf-2a81fb07d553 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:32.063586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8bedad-da09-4c83-9abf-2a81fb07d553 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:32.064206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8bedad-da09-4c83-9abf-2a81fb07d553 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:32.065255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8bedad-da09-4c83-9abf-2a81fb07d553 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:32.065934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8bedad-da09-4c83-9abf-2a81fb07d553 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:32.066575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8bedad-da09-4c83-9abf-2a81fb07d553 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:32.469078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fa8bedad-da09-4c83-9abf-2a81fb07d553 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:32.495675 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fa8bedad-da09-4c83-9abf-2a81fb07d553 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.445416 Sep 27 10:23:32.500767 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2414/5259] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:32 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:33.092238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f1d147f7-ad0e-45a8-a520-c20ba4144b17 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:33.096374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f1d147f7-ad0e-45a8-a520-c20ba4144b17 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:33.122266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d147f7-ad0e-45a8-a520-c20ba4144b17 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:33.125775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d147f7-ad0e-45a8-a520-c20ba4144b17 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:33.126717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d147f7-ad0e-45a8-a520-c20ba4144b17 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:33.128186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d147f7-ad0e-45a8-a520-c20ba4144b17 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:33.132335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d147f7-ad0e-45a8-a520-c20ba4144b17 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:33.133598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1d147f7-ad0e-45a8-a520-c20ba4144b17 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:33.513587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f1d147f7-ad0e-45a8-a520-c20ba4144b17 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:33.527823 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f1d147f7-ad0e-45a8-a520-c20ba4144b17 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1426 microversion: 2.1 time: 0.453817 Sep 27 10:23:33.543894 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2415/5260] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:33 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1426 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:33.549049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7d6dabbd-da89-49e2-96f6-57ce6171cf28 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:33.549677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7d6dabbd-da89-49e2-96f6-57ce6171cf28 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:33.568319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d6dabbd-da89-49e2-96f6-57ce6171cf28 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:33.569600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d6dabbd-da89-49e2-96f6-57ce6171cf28 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:33.571688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d6dabbd-da89-49e2-96f6-57ce6171cf28 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:33.572700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d6dabbd-da89-49e2-96f6-57ce6171cf28 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:33.577460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d6dabbd-da89-49e2-96f6-57ce6171cf28 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:33.578202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d6dabbd-da89-49e2-96f6-57ce6171cf28 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:33.673185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:33.699877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:33.712428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:33.712428 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e03c156e-d8c4-407d-b219-c4816b5cb8f4 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1406 microversion: 2.20 time: 2.630666 Sep 27 10:23:33.734071 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2846/5261] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:31 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1406 bytes in 2658 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:23:33.875208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7d6dabbd-da89-49e2-96f6-57ce6171cf28 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:33.900492 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7d6dabbd-da89-49e2-96f6-57ce6171cf28 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.353760 Sep 27 10:23:33.905227 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2416/5262] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:33 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:34.367560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d3b0f3bd-7ea5-43a3-bb33-2e9a1e4b2a96 req-655d17e7-2ea6-4e71-a5ae-d3c6125aeb33 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77037d39-844c-4761-88cc-899710443401", "name": "network-vif-plugged", "status": "completed", "tag": "6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:34.405361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d3b0f3bd-7ea5-43a3-bb33-2e9a1e4b2a96 req-655d17e7-2ea6-4e71-a5ae-d3c6125aeb33 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:34.405842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d3b0f3bd-7ea5-43a3-bb33-2e9a1e4b2a96 req-655d17e7-2ea6-4e71-a5ae-d3c6125aeb33 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:34.412143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d3b0f3bd-7ea5-43a3-bb33-2e9a1e4b2a96 req-655d17e7-2ea6-4e71-a5ae-d3c6125aeb33 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:34.453398 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d3b0f3bd-7ea5-43a3-bb33-2e9a1e4b2a96 req-655d17e7-2ea6-4e71-a5ae-d3c6125aeb33 service nova] Creating event network-vif-plugged:6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1 for instance 77037d39-844c-4761-88cc-899710443401 on nch2-master-1 Sep 27 10:23:34.469643 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d3b0f3bd-7ea5-43a3-bb33-2e9a1e4b2a96 req-655d17e7-2ea6-4e71-a5ae-d3c6125aeb33 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.107283 Sep 27 10:23:34.470236 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2847/5263] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:23:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:23:34.550727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-98ef481a-5513-47fd-b562-1f75fcb13fc2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:34.551482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-98ef481a-5513-47fd-b562-1f75fcb13fc2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:34.564003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98ef481a-5513-47fd-b562-1f75fcb13fc2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:34.574253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98ef481a-5513-47fd-b562-1f75fcb13fc2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:34.575066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98ef481a-5513-47fd-b562-1f75fcb13fc2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:34.577972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98ef481a-5513-47fd-b562-1f75fcb13fc2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:34.581974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98ef481a-5513-47fd-b562-1f75fcb13fc2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:34.583674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-98ef481a-5513-47fd-b562-1f75fcb13fc2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:34.738024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:34.738757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:34.752507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:34.757820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:34.758670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:34.759679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:34.760354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:34.772713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:35.311860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-98ef481a-5513-47fd-b562-1f75fcb13fc2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:35.325951 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-98ef481a-5513-47fd-b562-1f75fcb13fc2 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1426 microversion: 2.1 time: 0.779071 Sep 27 10:23:35.347762 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2417/5264] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:34 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1426 bytes in 801 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:35.360538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1d7165a3-473e-489b-bfd1-2d5ce8456b0b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:35.361367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1d7165a3-473e-489b-bfd1-2d5ce8456b0b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:35.382031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d7165a3-473e-489b-bfd1-2d5ce8456b0b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:35.382724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d7165a3-473e-489b-bfd1-2d5ce8456b0b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:35.383870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d7165a3-473e-489b-bfd1-2d5ce8456b0b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:35.384892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d7165a3-473e-489b-bfd1-2d5ce8456b0b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:35.385574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d7165a3-473e-489b-bfd1-2d5ce8456b0b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:35.386236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d7165a3-473e-489b-bfd1-2d5ce8456b0b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:35.897485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1d7165a3-473e-489b-bfd1-2d5ce8456b0b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:35.910792 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1d7165a3-473e-489b-bfd1-2d5ce8456b0b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.560530 Sep 27 10:23:35.911930 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2418/5265] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:35 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:36.124602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:36.146512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:36.147837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:36.158338 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-81483b34-e4fe-470a-bee4-f52b69076fb5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1406 microversion: 2.20 time: 1.424414 Sep 27 10:23:36.161775 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2848/5266] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:34 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1406 bytes in 1428 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:23:36.355711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-37c8204f-b492-4d70-b889-a9177e0eaaaa tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:36.365140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-37c8204f-b492-4d70-b889-a9177e0eaaaa tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:36.378834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c8204f-b492-4d70-b889-a9177e0eaaaa tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:36.382549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c8204f-b492-4d70-b889-a9177e0eaaaa tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:36.383248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c8204f-b492-4d70-b889-a9177e0eaaaa tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:36.384190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c8204f-b492-4d70-b889-a9177e0eaaaa tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:36.386655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c8204f-b492-4d70-b889-a9177e0eaaaa tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:36.388628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37c8204f-b492-4d70-b889-a9177e0eaaaa tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:36.496372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d8cf8153-5d6d-46fc-b07e-8f9617306ec6 req-2f940b76-0e32-402a-bcda-97ee25c181a3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3e246452-0210-449b-b9b7-f1f42ccaa2eb", "name": "network-vif-plugged", "status": "completed", "tag": "14864aa7-71a6-4bce-944b-e859f11985e0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:36.541067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d8cf8153-5d6d-46fc-b07e-8f9617306ec6 req-2f940b76-0e32-402a-bcda-97ee25c181a3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:36.541839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d8cf8153-5d6d-46fc-b07e-8f9617306ec6 req-2f940b76-0e32-402a-bcda-97ee25c181a3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:36.543463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d8cf8153-5d6d-46fc-b07e-8f9617306ec6 req-2f940b76-0e32-402a-bcda-97ee25c181a3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:36.561263 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d8cf8153-5d6d-46fc-b07e-8f9617306ec6 req-2f940b76-0e32-402a-bcda-97ee25c181a3 service nova] Creating event network-vif-plugged:14864aa7-71a6-4bce-944b-e859f11985e0 for instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb on nch1-master-1 Sep 27 10:23:36.567507 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d8cf8153-5d6d-46fc-b07e-8f9617306ec6 req-2f940b76-0e32-402a-bcda-97ee25c181a3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.078534 Sep 27 10:23:36.569066 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2849/5267] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:23:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:23:36.940247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23b8a37f-17d7-4b93-9b6a-697f5764c091 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:36.942735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-23b8a37f-17d7-4b93-9b6a-697f5764c091 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:36.962281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23b8a37f-17d7-4b93-9b6a-697f5764c091 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:36.963060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23b8a37f-17d7-4b93-9b6a-697f5764c091 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:36.963723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23b8a37f-17d7-4b93-9b6a-697f5764c091 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:36.964787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23b8a37f-17d7-4b93-9b6a-697f5764c091 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:36.965443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23b8a37f-17d7-4b93-9b6a-697f5764c091 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:36.966084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23b8a37f-17d7-4b93-9b6a-697f5764c091 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:37.042472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-37c8204f-b492-4d70-b889-a9177e0eaaaa tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:37.068336 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-37c8204f-b492-4d70-b889-a9177e0eaaaa tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1426 microversion: 2.1 time: 0.717076 Sep 27 10:23:37.079639 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2419/5268] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:36 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1426 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:37.188550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:37.189534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:37.202878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:37.204770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:37.205923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:37.206798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:37.209233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:37.210744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:37.653143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-23b8a37f-17d7-4b93-9b6a-697f5764c091 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:37.717614 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-23b8a37f-17d7-4b93-9b6a-697f5764c091 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.776668 Sep 27 10:23:37.720522 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2850/5269] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:36 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 786 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:23:37.765170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:37.784304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:37.790610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:37.792226 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a2ed2147-6d7f-4068-a279-be7b1db0d3b9 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1598 microversion: 2.20 time: 0.608634 Sep 27 10:23:37.794952 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2420/5270] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:23:37 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1598 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:23:37.837791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f71e622-8914-4382-90e9-d526eee848b0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:37.839432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f71e622-8914-4382-90e9-d526eee848b0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:37.877438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f71e622-8914-4382-90e9-d526eee848b0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:37.883020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f71e622-8914-4382-90e9-d526eee848b0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:37.883849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f71e622-8914-4382-90e9-d526eee848b0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:37.884876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f71e622-8914-4382-90e9-d526eee848b0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:37.885683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f71e622-8914-4382-90e9-d526eee848b0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:37.886303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f71e622-8914-4382-90e9-d526eee848b0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:37.935900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:38.104395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ad41b131-d6ec-4d2a-a598-4bb323da3b83 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:38.105254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ad41b131-d6ec-4d2a-a598-4bb323da3b83 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:38.123976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=76,nova_api:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:38.127300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad41b131-d6ec-4d2a-a598-4bb323da3b83 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:38.129635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad41b131-d6ec-4d2a-a598-4bb323da3b83 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:38.135043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad41b131-d6ec-4d2a-a598-4bb323da3b83 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:38.139896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad41b131-d6ec-4d2a-a598-4bb323da3b83 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:38.141585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad41b131-d6ec-4d2a-a598-4bb323da3b83 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:38.143918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad41b131-d6ec-4d2a-a598-4bb323da3b83 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:38.179586 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f71e622-8914-4382-90e9-d526eee848b0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.349294 Sep 27 10:23:38.188812 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2851/5271] 10.222.0.26 () {60 vars in 1316 bytes} [Tue Sep 27 10:23:37 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/os-interface => generated 285 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:23:38.750952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae5654d2-5b3a-4760-8ce7-b27d4c812341 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:38.753148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ae5654d2-5b3a-4760-8ce7-b27d4c812341 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:38.783188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae5654d2-5b3a-4760-8ce7-b27d4c812341 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:38.784575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae5654d2-5b3a-4760-8ce7-b27d4c812341 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:38.788301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae5654d2-5b3a-4760-8ce7-b27d4c812341 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:38.789502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae5654d2-5b3a-4760-8ce7-b27d4c812341 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:38.794307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae5654d2-5b3a-4760-8ce7-b27d4c812341 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:38.798294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae5654d2-5b3a-4760-8ce7-b27d4c812341 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:39.260389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ae5654d2-5b3a-4760-8ce7-b27d4c812341 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:39.275164 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ae5654d2-5b3a-4760-8ce7-b27d4c812341 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.531676 Sep 27 10:23:39.275770 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2852/5272] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:38 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:39.373434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ad41b131-d6ec-4d2a-a598-4bb323da3b83 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:39.388610 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ad41b131-d6ec-4d2a-a598-4bb323da3b83 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1426 microversion: 2.1 time: 1.290076 Sep 27 10:23:39.401944 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2421/5273] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:38 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1426 bytes in 1304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:40.300017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9e8adae4-e345-4956-8319-aa70453c11cb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:40.301997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9e8adae4-e345-4956-8319-aa70453c11cb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:40.320582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e8adae4-e345-4956-8319-aa70453c11cb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:40.322121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e8adae4-e345-4956-8319-aa70453c11cb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:40.322831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e8adae4-e345-4956-8319-aa70453c11cb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:40.323792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e8adae4-e345-4956-8319-aa70453c11cb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:40.324409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e8adae4-e345-4956-8319-aa70453c11cb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:40.325079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9e8adae4-e345-4956-8319-aa70453c11cb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:40.427397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-45d862c6-1d68-4174-bf9f-6b18ee1663f3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:40.428684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-45d862c6-1d68-4174-bf9f-6b18ee1663f3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:40.445206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45d862c6-1d68-4174-bf9f-6b18ee1663f3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:40.446308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45d862c6-1d68-4174-bf9f-6b18ee1663f3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:40.448920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45d862c6-1d68-4174-bf9f-6b18ee1663f3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:40.450927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45d862c6-1d68-4174-bf9f-6b18ee1663f3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:40.451610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45d862c6-1d68-4174-bf9f-6b18ee1663f3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:40.453838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45d862c6-1d68-4174-bf9f-6b18ee1663f3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:40.839015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-45d862c6-1d68-4174-bf9f-6b18ee1663f3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:40.869328 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-45d862c6-1d68-4174-bf9f-6b18ee1663f3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1605 microversion: 2.1 time: 0.458829 Sep 27 10:23:40.869957 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2422/5274] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:40 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1605 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:40.892291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-96ad06c7-54b4-4e98-864a-c920a871eea3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:40.893295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-96ad06c7-54b4-4e98-864a-c920a871eea3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:40.900206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96ad06c7-54b4-4e98-864a-c920a871eea3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:40.901118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96ad06c7-54b4-4e98-864a-c920a871eea3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:40.902931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96ad06c7-54b4-4e98-864a-c920a871eea3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:40.905432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96ad06c7-54b4-4e98-864a-c920a871eea3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:40.906205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96ad06c7-54b4-4e98-864a-c920a871eea3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:40.906842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96ad06c7-54b4-4e98-864a-c920a871eea3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:40.980595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9e8adae4-e345-4956-8319-aa70453c11cb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:40.989235 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9e8adae4-e345-4956-8319-aa70453c11cb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.693467 Sep 27 10:23:40.989885 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2853/5275] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:40 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:41.575368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-96ad06c7-54b4-4e98-864a-c920a871eea3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:41.591517 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-96ad06c7-54b4-4e98-864a-c920a871eea3 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1605 microversion: 2.1 time: 0.702909 Sep 27 10:23:41.592413 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2423/5276] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:40 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1605 bytes in 705 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:41.619134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3f61f5fd-1e4c-4d7b-967c-a2d9eb979f6f tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:41.620319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3f61f5fd-1e4c-4d7b-967c-a2d9eb979f6f tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:41.628065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f61f5fd-1e4c-4d7b-967c-a2d9eb979f6f tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:41.635065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f61f5fd-1e4c-4d7b-967c-a2d9eb979f6f tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:41.637913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f61f5fd-1e4c-4d7b-967c-a2d9eb979f6f tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:41.638924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f61f5fd-1e4c-4d7b-967c-a2d9eb979f6f tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:41.641755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f61f5fd-1e4c-4d7b-967c-a2d9eb979f6f tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:41.645108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f61f5fd-1e4c-4d7b-967c-a2d9eb979f6f tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:42.019588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce9166d2-26f1-444a-b213-3289f3c78b14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:42.020679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ce9166d2-26f1-444a-b213-3289f3c78b14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:42.034202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce9166d2-26f1-444a-b213-3289f3c78b14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:42.035303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce9166d2-26f1-444a-b213-3289f3c78b14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:42.037486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce9166d2-26f1-444a-b213-3289f3c78b14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:42.038506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce9166d2-26f1-444a-b213-3289f3c78b14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:42.044372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce9166d2-26f1-444a-b213-3289f3c78b14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:42.045241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce9166d2-26f1-444a-b213-3289f3c78b14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:42.371671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3f61f5fd-1e4c-4d7b-967c-a2d9eb979f6f tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:42.386255 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3f61f5fd-1e4c-4d7b-967c-a2d9eb979f6f tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1605 microversion: 2.1 time: 0.771529 Sep 27 10:23:42.389327 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2854/5277] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:41 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1605 bytes in 775 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:42.399136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-4554430d-3f44-4e00-9a78-64122211d2d3 req-5723f9e3-267d-468e-b4fa-0312cd3ffac7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77037d39-844c-4761-88cc-899710443401", "tag": "6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:42.432412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4554430d-3f44-4e00-9a78-64122211d2d3 req-5723f9e3-267d-468e-b4fa-0312cd3ffac7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:42.432939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4554430d-3f44-4e00-9a78-64122211d2d3 req-5723f9e3-267d-468e-b4fa-0312cd3ffac7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:42.435582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4554430d-3f44-4e00-9a78-64122211d2d3 req-5723f9e3-267d-468e-b4fa-0312cd3ffac7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:42.483509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ce9166d2-26f1-444a-b213-3289f3c78b14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:42.495736 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-4554430d-3f44-4e00-9a78-64122211d2d3 req-5723f9e3-267d-468e-b4fa-0312cd3ffac7 service nova] Creating event network-changed:6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1 for instance 77037d39-844c-4761-88cc-899710443401 on nch2-master-1 Sep 27 10:23:42.505982 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ce9166d2-26f1-444a-b213-3289f3c78b14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.481564 Sep 27 10:23:42.506620 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2424/5278] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:42 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:42.533390 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-4554430d-3f44-4e00-9a78-64122211d2d3 req-5723f9e3-267d-468e-b4fa-0312cd3ffac7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.142480 Sep 27 10:23:42.545383 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2855/5279] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:23:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:23:43.543186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e3679ec-a56c-45f7-8667-a65792aa4009 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:43.544096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4e3679ec-a56c-45f7-8667-a65792aa4009 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:43.559856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3679ec-a56c-45f7-8667-a65792aa4009 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:43.560966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3679ec-a56c-45f7-8667-a65792aa4009 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:43.563433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3679ec-a56c-45f7-8667-a65792aa4009 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:43.564451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3679ec-a56c-45f7-8667-a65792aa4009 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:43.565029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3679ec-a56c-45f7-8667-a65792aa4009 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:43.568891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e3679ec-a56c-45f7-8667-a65792aa4009 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:44.238576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4e3679ec-a56c-45f7-8667-a65792aa4009 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:44.249289 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4e3679ec-a56c-45f7-8667-a65792aa4009 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.718035 Sep 27 10:23:44.260238 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2425/5280] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:43 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 730 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:45.287689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-83856acf-90b5-419f-bf7d-1e371ccfd67c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:45.289120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-83856acf-90b5-419f-bf7d-1e371ccfd67c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:45.306793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83856acf-90b5-419f-bf7d-1e371ccfd67c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:45.315279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83856acf-90b5-419f-bf7d-1e371ccfd67c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:45.315988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83856acf-90b5-419f-bf7d-1e371ccfd67c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:45.317001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83856acf-90b5-419f-bf7d-1e371ccfd67c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:45.319485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83856acf-90b5-419f-bf7d-1e371ccfd67c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:45.320116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-83856acf-90b5-419f-bf7d-1e371ccfd67c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:46.129696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-83856acf-90b5-419f-bf7d-1e371ccfd67c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:46.163243 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-83856acf-90b5-419f-bf7d-1e371ccfd67c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.879406 Sep 27 10:23:46.166233 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2856/5281] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:45 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 886 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:47.195235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6f62e4aa-2018-4732-9b43-6bc0955d90cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:47.196534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6f62e4aa-2018-4732-9b43-6bc0955d90cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:47.212429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f62e4aa-2018-4732-9b43-6bc0955d90cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:47.217745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f62e4aa-2018-4732-9b43-6bc0955d90cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:47.220366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f62e4aa-2018-4732-9b43-6bc0955d90cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:47.221805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f62e4aa-2018-4732-9b43-6bc0955d90cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:47.225005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f62e4aa-2018-4732-9b43-6bc0955d90cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:47.230953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f62e4aa-2018-4732-9b43-6bc0955d90cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:47.237575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:47.698403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6f62e4aa-2018-4732-9b43-6bc0955d90cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:47.710134 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6f62e4aa-2018-4732-9b43-6bc0955d90cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1682 microversion: 2.1 time: 0.523190 Sep 27 10:23:47.710754 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2426/5282] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:47 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1682 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:48.736316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2f660d97-17ed-4c95-b3c3-1ce2d22afc78 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:48.737080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2f660d97-17ed-4c95-b3c3-1ce2d22afc78 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:48.749607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f660d97-17ed-4c95-b3c3-1ce2d22afc78 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:48.753898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f660d97-17ed-4c95-b3c3-1ce2d22afc78 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:48.754484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f660d97-17ed-4c95-b3c3-1ce2d22afc78 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:48.757617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f660d97-17ed-4c95-b3c3-1ce2d22afc78 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:48.760062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f660d97-17ed-4c95-b3c3-1ce2d22afc78 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:48.765032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f660d97-17ed-4c95-b3c3-1ce2d22afc78 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:49.830389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2f660d97-17ed-4c95-b3c3-1ce2d22afc78 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:49.842610 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2f660d97-17ed-4c95-b3c3-1ce2d22afc78 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 1.111668 Sep 27 10:23:49.847009 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2857/5283] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:48 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 1117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:50.034405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c8776606-0bd8-414e-82cf-1d58d2bbd75e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:50.035827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c8776606-0bd8-414e-82cf-1d58d2bbd75e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:50.045034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8776606-0bd8-414e-82cf-1d58d2bbd75e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:50.045945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8776606-0bd8-414e-82cf-1d58d2bbd75e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:50.046528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8776606-0bd8-414e-82cf-1d58d2bbd75e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:50.047386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8776606-0bd8-414e-82cf-1d58d2bbd75e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:50.048137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8776606-0bd8-414e-82cf-1d58d2bbd75e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:50.048728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8776606-0bd8-414e-82cf-1d58d2bbd75e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:50.358393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-359eb6bc-8fe0-49ad-a6af-1e2083bdadfa req-b119a3da-4096-4a3e-93ae-f24c2c799b89 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3e246452-0210-449b-b9b7-f1f42ccaa2eb", "tag": "14864aa7-71a6-4bce-944b-e859f11985e0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:50.369786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=42 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:23:50.375715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-359eb6bc-8fe0-49ad-a6af-1e2083bdadfa req-b119a3da-4096-4a3e-93ae-f24c2c799b89 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:50.376209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-359eb6bc-8fe0-49ad-a6af-1e2083bdadfa req-b119a3da-4096-4a3e-93ae-f24c2c799b89 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:50.376698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-359eb6bc-8fe0-49ad-a6af-1e2083bdadfa req-b119a3da-4096-4a3e-93ae-f24c2c799b89 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:50.408615 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-359eb6bc-8fe0-49ad-a6af-1e2083bdadfa req-b119a3da-4096-4a3e-93ae-f24c2c799b89 service nova] Creating event network-changed:14864aa7-71a6-4bce-944b-e859f11985e0 for instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb on nch1-master-1 Sep 27 10:23:50.429297 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-359eb6bc-8fe0-49ad-a6af-1e2083bdadfa req-b119a3da-4096-4a3e-93ae-f24c2c799b89 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.076167 Sep 27 10:23:50.433182 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2858/5284] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:23:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:23:50.538634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c8776606-0bd8-414e-82cf-1d58d2bbd75e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:50.555702 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c8776606-0bd8-414e-82cf-1d58d2bbd75e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.525705 Sep 27 10:23:50.582261 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2427/5285] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:50 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:52.523272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ccfe04c5-18b9-421e-967c-8d74ce97a339 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:52.524546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ccfe04c5-18b9-421e-967c-8d74ce97a339 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:52.542972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe04c5-18b9-421e-967c-8d74ce97a339 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:52.544190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe04c5-18b9-421e-967c-8d74ce97a339 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:52.546943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe04c5-18b9-421e-967c-8d74ce97a339 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:52.547714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe04c5-18b9-421e-967c-8d74ce97a339 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:52.548191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe04c5-18b9-421e-967c-8d74ce97a339 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:52.555708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ccfe04c5-18b9-421e-967c-8d74ce97a339 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:52.742116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ccfe04c5-18b9-421e-967c-8d74ce97a339 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:52.756048 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ccfe04c5-18b9-421e-967c-8d74ce97a339 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.240507 Sep 27 10:23:52.768690 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2859/5286] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:52 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:52.778914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa09a5d9-6c46-403d-988a-d3aa440ca683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:52.779544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aa09a5d9-6c46-403d-988a-d3aa440ca683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:52.786577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa09a5d9-6c46-403d-988a-d3aa440ca683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:52.787046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa09a5d9-6c46-403d-988a-d3aa440ca683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:52.787491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa09a5d9-6c46-403d-988a-d3aa440ca683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:52.788275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa09a5d9-6c46-403d-988a-d3aa440ca683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:52.788691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa09a5d9-6c46-403d-988a-d3aa440ca683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:52.789131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa09a5d9-6c46-403d-988a-d3aa440ca683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:53.811284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aa09a5d9-6c46-403d-988a-d3aa440ca683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:53.826104 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aa09a5d9-6c46-403d-988a-d3aa440ca683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 1.051135 Sep 27 10:23:53.829057 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2428/5287] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:52 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 1055 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:53.858549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c01bfab-6e77-4063-a66b-826774cdf43e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:53.866758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-494f27f4-f0b1-4eef-aebd-dea052e1406c req-a31e2909-0498-41d3-9a26-7ea4954b9ef4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ac6dabf-1315-4e05-ab6c-22211dc2440b", "name": "network-vif-unplugged", "status": "completed", "tag": "b894a455-9bb5-443a-84fe-e04c8d0c3e28"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:53.867408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5c01bfab-6e77-4063-a66b-826774cdf43e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:53.886285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c01bfab-6e77-4063-a66b-826774cdf43e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:53.892875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c01bfab-6e77-4063-a66b-826774cdf43e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:53.893817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c01bfab-6e77-4063-a66b-826774cdf43e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:53.895846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c01bfab-6e77-4063-a66b-826774cdf43e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:53.896432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c01bfab-6e77-4063-a66b-826774cdf43e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:53.897426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c01bfab-6e77-4063-a66b-826774cdf43e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:53.899722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-494f27f4-f0b1-4eef-aebd-dea052e1406c req-a31e2909-0498-41d3-9a26-7ea4954b9ef4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:53.900239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-494f27f4-f0b1-4eef-aebd-dea052e1406c req-a31e2909-0498-41d3-9a26-7ea4954b9ef4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:53.900709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-494f27f4-f0b1-4eef-aebd-dea052e1406c req-a31e2909-0498-41d3-9a26-7ea4954b9ef4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:53.924594 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-494f27f4-f0b1-4eef-aebd-dea052e1406c req-a31e2909-0498-41d3-9a26-7ea4954b9ef4 service nova] Creating event network-vif-unplugged:b894a455-9bb5-443a-84fe-e04c8d0c3e28 for instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b on nch1-master-1 Sep 27 10:23:53.932969 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-494f27f4-f0b1-4eef-aebd-dea052e1406c req-a31e2909-0498-41d3-9a26-7ea4954b9ef4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.071672 Sep 27 10:23:53.933597 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2429/5288] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:23:53 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:23:54.344378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c01bfab-6e77-4063-a66b-826774cdf43e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:54.360645 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c01bfab-6e77-4063-a66b-826774cdf43e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.506206 Sep 27 10:23:54.361285 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2860/5289] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:53 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:54.400067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-34a34fb4-f183-476c-b59f-bf06734341d0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:54.403026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-34a34fb4-f183-476c-b59f-bf06734341d0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:54.423936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34a34fb4-f183-476c-b59f-bf06734341d0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:54.424972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34a34fb4-f183-476c-b59f-bf06734341d0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:54.426062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34a34fb4-f183-476c-b59f-bf06734341d0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:54.428231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34a34fb4-f183-476c-b59f-bf06734341d0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:54.429360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34a34fb4-f183-476c-b59f-bf06734341d0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:54.430537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34a34fb4-f183-476c-b59f-bf06734341d0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:54.639347 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-34a34fb4-f183-476c-b59f-bf06734341d0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.253310 Sep 27 10:23:54.647708 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2430/5290] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:23:54 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 265 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:23:54.650001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fc9cfa64-c63a-4cae-936f-9bdfe49c008a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:54.650877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fc9cfa64-c63a-4cae-936f-9bdfe49c008a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:54.662013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9cfa64-c63a-4cae-936f-9bdfe49c008a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:54.662524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9cfa64-c63a-4cae-936f-9bdfe49c008a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:54.668728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9cfa64-c63a-4cae-936f-9bdfe49c008a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:54.671381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9cfa64-c63a-4cae-936f-9bdfe49c008a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:54.671888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9cfa64-c63a-4cae-936f-9bdfe49c008a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:54.672317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9cfa64-c63a-4cae-936f-9bdfe49c008a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:56.469552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc9cfa64-c63a-4cae-936f-9bdfe49c008a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:56.479189 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fc9cfa64-c63a-4cae-936f-9bdfe49c008a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1671 microversion: 2.1 time: 1.832604 Sep 27 10:23:56.502287 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2861/5291] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:54 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1671 bytes in 1857 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:57.507930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f945a50b-a091-40c2-8551-6c57bc35e970 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:57.509392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f945a50b-a091-40c2-8551-6c57bc35e970 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:57.532541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945a50b-a091-40c2-8551-6c57bc35e970 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:57.533138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945a50b-a091-40c2-8551-6c57bc35e970 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:57.539249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945a50b-a091-40c2-8551-6c57bc35e970 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:57.540635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945a50b-a091-40c2-8551-6c57bc35e970 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:57.542426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945a50b-a091-40c2-8551-6c57bc35e970 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:57.554597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f945a50b-a091-40c2-8551-6c57bc35e970 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:57.855673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f945a50b-a091-40c2-8551-6c57bc35e970 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:57.878078 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f945a50b-a091-40c2-8551-6c57bc35e970 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1671 microversion: 2.1 time: 0.375955 Sep 27 10:23:57.879857 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2431/5292] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:57 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1671 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:58.908647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c440b3da-2b16-431f-907f-831ec4e7c014 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:58.910033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c440b3da-2b16-431f-907f-831ec4e7c014 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:58.923966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c440b3da-2b16-431f-907f-831ec4e7c014 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:58.924896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c440b3da-2b16-431f-907f-831ec4e7c014 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:58.926715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c440b3da-2b16-431f-907f-831ec4e7c014 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:58.947193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c440b3da-2b16-431f-907f-831ec4e7c014 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:58.948354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c440b3da-2b16-431f-907f-831ec4e7c014 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:58.951272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c440b3da-2b16-431f-907f-831ec4e7c014 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:59.506164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c440b3da-2b16-431f-907f-831ec4e7c014 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:23:59.516387 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c440b3da-2b16-431f-907f-831ec4e7c014 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.612674 Sep 27 10:23:59.534128 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2862/5293] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:58 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 631 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:23:59.547183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-de3e8429-3100-4ab2-b7c4-dfbd35272b22 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {"length": 3}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:23:59.549875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-de3e8429-3100-4ab2-b7c4-dfbd35272b22 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:59.574597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de3e8429-3100-4ab2-b7c4-dfbd35272b22 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:59.575116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de3e8429-3100-4ab2-b7c4-dfbd35272b22 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:59.576272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de3e8429-3100-4ab2-b7c4-dfbd35272b22 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:59.576850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de3e8429-3100-4ab2-b7c4-dfbd35272b22 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:59.577377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de3e8429-3100-4ab2-b7c4-dfbd35272b22 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:59.577805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de3e8429-3100-4ab2-b7c4-dfbd35272b22 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:59.689781 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-de3e8429-3100-4ab2-b7c4-dfbd35272b22 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 200 len: 322 microversion: 2.1 time: 0.147461 Sep 27 10:23:59.691305 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2432/5294] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:23:59 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 322 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:23:59.706586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92ff0c6a-556b-435a-acc1-7aa1f94b9db6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:23:59.707558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-92ff0c6a-556b-435a-acc1-7aa1f94b9db6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:23:59.727066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92ff0c6a-556b-435a-acc1-7aa1f94b9db6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:59.729002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92ff0c6a-556b-435a-acc1-7aa1f94b9db6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:59.729746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92ff0c6a-556b-435a-acc1-7aa1f94b9db6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:23:59.733654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92ff0c6a-556b-435a-acc1-7aa1f94b9db6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:23:59.734335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92ff0c6a-556b-435a-acc1-7aa1f94b9db6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:23:59.734977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92ff0c6a-556b-435a-acc1-7aa1f94b9db6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:00.087282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-92ff0c6a-556b-435a-acc1-7aa1f94b9db6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:00.103213 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-92ff0c6a-556b-435a-acc1-7aa1f94b9db6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.405069 Sep 27 10:24:00.126363 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2863/5295] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:23:59 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:00.131511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5b70bccd-137e-47b7-98ff-112abd6693b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:00.132232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5b70bccd-137e-47b7-98ff-112abd6693b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:00.158071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b70bccd-137e-47b7-98ff-112abd6693b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:00.158575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b70bccd-137e-47b7-98ff-112abd6693b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:00.159033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b70bccd-137e-47b7-98ff-112abd6693b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:00.159774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b70bccd-137e-47b7-98ff-112abd6693b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:00.160245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b70bccd-137e-47b7-98ff-112abd6693b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:00.160781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b70bccd-137e-47b7-98ff-112abd6693b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:00.943530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b70bccd-137e-47b7-98ff-112abd6693b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:00.959141 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5b70bccd-137e-47b7-98ff-112abd6693b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.831481 Sep 27 10:24:00.960448 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2433/5296] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:00 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 835 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:00.997477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-868439cb-7f3d-4851-a689-175bbd701f9a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:00.999869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-868439cb-7f3d-4851-a689-175bbd701f9a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:01.027495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-868439cb-7f3d-4851-a689-175bbd701f9a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:01.029788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-868439cb-7f3d-4851-a689-175bbd701f9a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:01.032171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-868439cb-7f3d-4851-a689-175bbd701f9a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:01.033030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-868439cb-7f3d-4851-a689-175bbd701f9a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:01.033496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-868439cb-7f3d-4851-a689-175bbd701f9a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:01.034678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-868439cb-7f3d-4851-a689-175bbd701f9a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:01.322327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-868439cb-7f3d-4851-a689-175bbd701f9a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:01.336841 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-868439cb-7f3d-4851-a689-175bbd701f9a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.353221 Sep 27 10:24:01.337672 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2864/5297] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:00 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:01.365383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e6f00148-461a-45fa-8e8d-71d4e45c0f49 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:01.365995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e6f00148-461a-45fa-8e8d-71d4e45c0f49 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:01.380527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6f00148-461a-45fa-8e8d-71d4e45c0f49 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:01.381041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6f00148-461a-45fa-8e8d-71d4e45c0f49 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:01.381496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6f00148-461a-45fa-8e8d-71d4e45c0f49 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:01.388031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6f00148-461a-45fa-8e8d-71d4e45c0f49 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:01.388535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6f00148-461a-45fa-8e8d-71d4e45c0f49 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:01.389020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e6f00148-461a-45fa-8e8d-71d4e45c0f49 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:01.992199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e6f00148-461a-45fa-8e8d-71d4e45c0f49 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:02.005632 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e6f00148-461a-45fa-8e8d-71d4e45c0f49 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.643643 Sep 27 10:24:02.006787 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2434/5298] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:01 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:02.206032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-1045059904", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "247e3d7d-5912-42d7-9027-ebcb26003cfc"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:02.351074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] validate_networks() for [('247e3d7d-5912-42d7-9027-ebcb26003cfc', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:24:03.928201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:24:03.929933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:24:03.930595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:24:03.932852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:24:03.933933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:24:03.955685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:03.961074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:03.963689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:03.967419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:03.968821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:03.971226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:04.094516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for project 10375bd836f84383b39f2df73f53800e. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:24:04.112188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for user 97c769dbba794531b0a9db8e764ff097 and project 10375bd836f84383b39f2df73f53800e. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:24:04.154327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:24:04.160033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:24:04.333838 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c1ea3902-e539-4926-a334-3393884d1482 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.129982 Sep 27 10:24:04.338724 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2865/5299] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:24:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 2137 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:24:04.349711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-55916093-e36b-4428-abe7-5c1c00e31745 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:04.350593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-55916093-e36b-4428-abe7-5c1c00e31745 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:04.715186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-55916093-e36b-4428-abe7-5c1c00e31745 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:04.740039 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-55916093-e36b-4428-abe7-5c1c00e31745 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1215 microversion: 2.1 time: 0.394776 Sep 27 10:24:04.741165 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2435/5300] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:04 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1215 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:05.764611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bb3c405d-db6c-4f37-8f5d-c36359d87573 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:05.765390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bb3c405d-db6c-4f37-8f5d-c36359d87573 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:05.782130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3c405d-db6c-4f37-8f5d-c36359d87573 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:05.782593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3c405d-db6c-4f37-8f5d-c36359d87573 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:05.785813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3c405d-db6c-4f37-8f5d-c36359d87573 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:05.785813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3c405d-db6c-4f37-8f5d-c36359d87573 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:05.786202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3c405d-db6c-4f37-8f5d-c36359d87573 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:05.786641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb3c405d-db6c-4f37-8f5d-c36359d87573 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:06.641581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb3c405d-db6c-4f37-8f5d-c36359d87573 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:06.654110 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bb3c405d-db6c-4f37-8f5d-c36359d87573 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1211 microversion: 2.1 time: 0.893957 Sep 27 10:24:06.655393 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2866/5301] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:05 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1211 bytes in 896 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:07.680885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ee6a27df-9fc0-4740-a5a8-6b8517cecfd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:07.681515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ee6a27df-9fc0-4740-a5a8-6b8517cecfd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:07.705599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6a27df-9fc0-4740-a5a8-6b8517cecfd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:07.706431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6a27df-9fc0-4740-a5a8-6b8517cecfd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:07.706565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6a27df-9fc0-4740-a5a8-6b8517cecfd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:07.707784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6a27df-9fc0-4740-a5a8-6b8517cecfd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:07.709030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6a27df-9fc0-4740-a5a8-6b8517cecfd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:07.709623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6a27df-9fc0-4740-a5a8-6b8517cecfd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:07.892895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ee6a27df-9fc0-4740-a5a8-6b8517cecfd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:07.901139 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ee6a27df-9fc0-4740-a5a8-6b8517cecfd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1273 microversion: 2.1 time: 0.224476 Sep 27 10:24:07.901733 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2436/5302] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:07 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1273 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:24:08.927344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b5f3fcc-703f-45ee-99b4-834f22567c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:08.928042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7b5f3fcc-703f-45ee-99b4-834f22567c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:08.944294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b5f3fcc-703f-45ee-99b4-834f22567c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:08.946292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b5f3fcc-703f-45ee-99b4-834f22567c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:08.946985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b5f3fcc-703f-45ee-99b4-834f22567c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:08.949387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b5f3fcc-703f-45ee-99b4-834f22567c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:08.951191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b5f3fcc-703f-45ee-99b4-834f22567c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:08.952684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7b5f3fcc-703f-45ee-99b4-834f22567c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:09.019456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=79 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:09.317446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7b5f3fcc-703f-45ee-99b4-834f22567c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:09.325836 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7b5f3fcc-703f-45ee-99b4-834f22567c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1315 microversion: 2.1 time: 0.403935 Sep 27 10:24:09.326337 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2867/5303] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:08 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1315 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:10.350518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d13ce15e-6bfa-4b54-932b-ebdfd0d2c6ff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:10.351364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d13ce15e-6bfa-4b54-932b-ebdfd0d2c6ff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:10.356096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=34 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:10.371522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d13ce15e-6bfa-4b54-932b-ebdfd0d2c6ff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:10.371522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d13ce15e-6bfa-4b54-932b-ebdfd0d2c6ff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:10.371522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d13ce15e-6bfa-4b54-932b-ebdfd0d2c6ff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:10.371522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d13ce15e-6bfa-4b54-932b-ebdfd0d2c6ff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:10.371522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d13ce15e-6bfa-4b54-932b-ebdfd0d2c6ff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:10.371522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d13ce15e-6bfa-4b54-932b-ebdfd0d2c6ff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:11.153810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d13ce15e-6bfa-4b54-932b-ebdfd0d2c6ff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:11.170635 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d13ce15e-6bfa-4b54-932b-ebdfd0d2c6ff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1315 microversion: 2.1 time: 0.824517 Sep 27 10:24:11.171257 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2437/5304] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:10 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1315 bytes in 826 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:11.893494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f2242850-d993-4ec6-ada5-a632a8dca9e2 req-4e8b4e24-7043-4a5f-9e4e-57ba5bacd30c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "39e807ee-fe6b-4812-bc48-b47085b1b89f", "tag": "7a2a97a6-af5d-4a2c-a121-7127ea436b1f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:11.916631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f2242850-d993-4ec6-ada5-a632a8dca9e2 req-4e8b4e24-7043-4a5f-9e4e-57ba5bacd30c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:11.917454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f2242850-d993-4ec6-ada5-a632a8dca9e2 req-4e8b4e24-7043-4a5f-9e4e-57ba5bacd30c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:11.917850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f2242850-d993-4ec6-ada5-a632a8dca9e2 req-4e8b4e24-7043-4a5f-9e4e-57ba5bacd30c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:11.947716 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f2242850-d993-4ec6-ada5-a632a8dca9e2 req-4e8b4e24-7043-4a5f-9e4e-57ba5bacd30c service nova] Creating event network-changed:7a2a97a6-af5d-4a2c-a121-7127ea436b1f for instance 39e807ee-fe6b-4812-bc48-b47085b1b89f on nch2-master-1 Sep 27 10:24:11.963829 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f2242850-d993-4ec6-ada5-a632a8dca9e2 req-4e8b4e24-7043-4a5f-9e4e-57ba5bacd30c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.073992 Sep 27 10:24:11.964548 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2868/5305] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:24:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:24:12.209083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-353dc8e2-9274-41f8-9b32-4eb79fc85840 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:12.210353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-353dc8e2-9274-41f8-9b32-4eb79fc85840 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:12.218771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-353dc8e2-9274-41f8-9b32-4eb79fc85840 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:12.221312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-353dc8e2-9274-41f8-9b32-4eb79fc85840 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:12.222976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-353dc8e2-9274-41f8-9b32-4eb79fc85840 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:12.224189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-353dc8e2-9274-41f8-9b32-4eb79fc85840 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:12.226460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-353dc8e2-9274-41f8-9b32-4eb79fc85840 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:12.227156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-353dc8e2-9274-41f8-9b32-4eb79fc85840 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:13.034500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-353dc8e2-9274-41f8-9b32-4eb79fc85840 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:13.044264 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-353dc8e2-9274-41f8-9b32-4eb79fc85840 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1315 microversion: 2.1 time: 0.848397 Sep 27 10:24:13.044764 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2438/5306] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:12 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1315 bytes in 850 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:14.066579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:14.074995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dd7f772e-cff7-4c34-bbc0-bf35e7c270c6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:14.076251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dd7f772e-cff7-4c34-bbc0-bf35e7c270c6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:14.086487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd7f772e-cff7-4c34-bbc0-bf35e7c270c6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:14.087377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd7f772e-cff7-4c34-bbc0-bf35e7c270c6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:14.088038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd7f772e-cff7-4c34-bbc0-bf35e7c270c6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:14.090163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd7f772e-cff7-4c34-bbc0-bf35e7c270c6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:14.091777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd7f772e-cff7-4c34-bbc0-bf35e7c270c6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:14.093586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd7f772e-cff7-4c34-bbc0-bf35e7c270c6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:14.143124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:14.400077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dd7f772e-cff7-4c34-bbc0-bf35e7c270c6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:14.411724 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dd7f772e-cff7-4c34-bbc0-bf35e7c270c6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1315 microversion: 2.1 time: 0.347828 Sep 27 10:24:14.430297 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2869/5307] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:14 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1315 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:15.442327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d710f1f-50c1-49d4-a093-1ef08998a463 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:15.457408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4d710f1f-50c1-49d4-a093-1ef08998a463 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:15.478051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:15.483531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d710f1f-50c1-49d4-a093-1ef08998a463 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:15.484308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d710f1f-50c1-49d4-a093-1ef08998a463 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:15.485951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d710f1f-50c1-49d4-a093-1ef08998a463 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:15.487621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d710f1f-50c1-49d4-a093-1ef08998a463 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:15.489918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d710f1f-50c1-49d4-a093-1ef08998a463 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:15.490595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d710f1f-50c1-49d4-a093-1ef08998a463 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:15.827481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4d710f1f-50c1-49d4-a093-1ef08998a463 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:15.842408 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4d710f1f-50c1-49d4-a093-1ef08998a463 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1315 microversion: 2.1 time: 0.408004 Sep 27 10:24:15.843068 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2439/5308] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:15 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1315 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:16.865518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15fdaa41-227f-47f5-be02-1d391a533ba6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:16.866103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-15fdaa41-227f-47f5-be02-1d391a533ba6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:16.876402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15fdaa41-227f-47f5-be02-1d391a533ba6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:16.878729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15fdaa41-227f-47f5-be02-1d391a533ba6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:16.880442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15fdaa41-227f-47f5-be02-1d391a533ba6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:16.881439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15fdaa41-227f-47f5-be02-1d391a533ba6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:16.883229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15fdaa41-227f-47f5-be02-1d391a533ba6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:16.886814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15fdaa41-227f-47f5-be02-1d391a533ba6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:17.203923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-15fdaa41-227f-47f5-be02-1d391a533ba6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:17.220971 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-15fdaa41-227f-47f5-be02-1d391a533ba6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1315 microversion: 2.1 time: 0.359480 Sep 27 10:24:17.222929 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2870/5309] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:16 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1315 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:18.248097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0af6b2c-2878-47f4-a418-601da0211501 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:18.248950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b0af6b2c-2878-47f4-a418-601da0211501 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:18.297189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-47461fd4-429f-4fd6-a603-cf1ff700b2b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:18.297741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-47461fd4-429f-4fd6-a603-cf1ff700b2b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:18.303148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0af6b2c-2878-47f4-a418-601da0211501 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:18.303654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0af6b2c-2878-47f4-a418-601da0211501 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:18.304098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0af6b2c-2878-47f4-a418-601da0211501 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:18.319005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0af6b2c-2878-47f4-a418-601da0211501 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:18.319465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0af6b2c-2878-47f4-a418-601da0211501 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:18.319926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0af6b2c-2878-47f4-a418-601da0211501 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:18.324333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47461fd4-429f-4fd6-a603-cf1ff700b2b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:18.327175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47461fd4-429f-4fd6-a603-cf1ff700b2b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:18.327767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47461fd4-429f-4fd6-a603-cf1ff700b2b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:18.345169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47461fd4-429f-4fd6-a603-cf1ff700b2b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:18.345822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47461fd4-429f-4fd6-a603-cf1ff700b2b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:18.346276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47461fd4-429f-4fd6-a603-cf1ff700b2b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:18.350281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=75,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:18.499075 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-47461fd4-429f-4fd6-a603-cf1ff700b2b8 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb/os-server-password" status: 200 len: 116 microversion: 2.1 time: 0.202832 Sep 27 10:24:18.501540 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2871/5310] 10.222.0.26 () {58 vars in 1293 bytes} [Tue Sep 27 10:24:18 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb/os-server-password => generated 116 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:24:18.668168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b0af6b2c-2878-47f4-a418-601da0211501 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:18.677964 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0af6b2c-2878-47f4-a418-601da0211501 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1315 microversion: 2.1 time: 0.433879 Sep 27 10:24:18.678572 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2440/5311] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:18 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1315 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:19.716217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-03d3bdf2-dbb1-42a6-ba57-3ef96b1c0b6b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:19.717143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-03d3bdf2-dbb1-42a6-ba57-3ef96b1c0b6b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:19.737002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03d3bdf2-dbb1-42a6-ba57-3ef96b1c0b6b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:19.737701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03d3bdf2-dbb1-42a6-ba57-3ef96b1c0b6b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:19.738382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03d3bdf2-dbb1-42a6-ba57-3ef96b1c0b6b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:19.749261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03d3bdf2-dbb1-42a6-ba57-3ef96b1c0b6b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:19.751446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03d3bdf2-dbb1-42a6-ba57-3ef96b1c0b6b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:19.753523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-03d3bdf2-dbb1-42a6-ba57-3ef96b1c0b6b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:20.276325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-03d3bdf2-dbb1-42a6-ba57-3ef96b1c0b6b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:20.304276 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-03d3bdf2-dbb1-42a6-ba57-3ef96b1c0b6b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1315 microversion: 2.1 time: 0.600716 Sep 27 10:24:20.309978 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2872/5312] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:19 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1315 bytes in 607 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:21.159410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1613708265", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"uuid": "ad7a2197-c1e6-4e39-9521-89f6d7499b41", "source_type": "snapshot", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}], "networks": [{"uuid": "1e417b59-62bb-4007-a78a-c15c153efd33"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:21.341837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bcc04601-38a9-486e-8c29-512ee1384268 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:21.342539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] validate_networks() for [('1e417b59-62bb-4007-a78a-c15c153efd33', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:24:21.342718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bcc04601-38a9-486e-8c29-512ee1384268 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:21.349227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:21.368912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc04601-38a9-486e-8c29-512ee1384268 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:21.370152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc04601-38a9-486e-8c29-512ee1384268 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:21.370844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc04601-38a9-486e-8c29-512ee1384268 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:21.371763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc04601-38a9-486e-8c29-512ee1384268 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:21.372443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc04601-38a9-486e-8c29-512ee1384268 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:21.372855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcc04601-38a9-486e-8c29-512ee1384268 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:21.648327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bcc04601-38a9-486e-8c29-512ee1384268 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:21.659760 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bcc04601-38a9-486e-8c29-512ee1384268 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1315 microversion: 2.1 time: 0.327509 Sep 27 10:24:21.673701 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2873/5313] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:21 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1315 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:22.110260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-de4c0649-c85f-46ca-a59d-a28f06c32b85 req-02bc2936-9a13-4b04-8324-ddc1707f9fea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39e807ee-fe6b-4812-bc48-b47085b1b89f", "name": "network-vif-plugged", "status": "completed", "tag": "7a2a97a6-af5d-4a2c-a121-7127ea436b1f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:22.128619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-de4c0649-c85f-46ca-a59d-a28f06c32b85 req-02bc2936-9a13-4b04-8324-ddc1707f9fea service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:22.133277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-de4c0649-c85f-46ca-a59d-a28f06c32b85 req-02bc2936-9a13-4b04-8324-ddc1707f9fea service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:22.133993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-de4c0649-c85f-46ca-a59d-a28f06c32b85 req-02bc2936-9a13-4b04-8324-ddc1707f9fea service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:22.159031 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-de4c0649-c85f-46ca-a59d-a28f06c32b85 req-02bc2936-9a13-4b04-8324-ddc1707f9fea service nova] Creating event network-vif-plugged:7a2a97a6-af5d-4a2c-a121-7127ea436b1f for instance 39e807ee-fe6b-4812-bc48-b47085b1b89f on nch2-master-1 Sep 27 10:24:22.172941 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-de4c0649-c85f-46ca-a59d-a28f06c32b85 req-02bc2936-9a13-4b04-8324-ddc1707f9fea service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067024 Sep 27 10:24:22.173824 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2874/5314] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:24:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:24:22.296636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:24:22.297217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:24:22.298026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:24:22.298623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:24:22.299536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:24:22.320811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:22.322519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:22.323282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:22.324265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:22.325568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:22.326216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:22.359442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:24:22.368443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for user 178956f6c8154a06a0710b295aff3086 and project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:24:22.389095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:24:22.391598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id='ad7a2197-c1e6-4e39-9521-89f6d7499b41',source_type='snapshot',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:24:22.528523 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5d994685-523d-4e0e-98c5-ffcfbe3ab4be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.373816 Sep 27 10:24:22.537204 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2441/5315] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:24:21 2022] POST /compute/v2.1/servers => generated 380 bytes in 1375 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:24:22.545633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c094ae12-2779-4dfb-86bc-736c86c67b6a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:22.548150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c094ae12-2779-4dfb-86bc-736c86c67b6a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:22.682735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c094ae12-2779-4dfb-86bc-736c86c67b6a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:22.687499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a5e6ac66-65f8-4210-8e1f-6ec3f617ffee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:22.688131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a5e6ac66-65f8-4210-8e1f-6ec3f617ffee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:22.706965 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c094ae12-2779-4dfb-86bc-736c86c67b6a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1057 microversion: 2.1 time: 0.168345 Sep 27 10:24:22.712852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5e6ac66-65f8-4210-8e1f-6ec3f617ffee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:22.713320 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2875/5316] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:22 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1057 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:22.714196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5e6ac66-65f8-4210-8e1f-6ec3f617ffee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:22.717632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5e6ac66-65f8-4210-8e1f-6ec3f617ffee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:22.718587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5e6ac66-65f8-4210-8e1f-6ec3f617ffee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:22.720404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5e6ac66-65f8-4210-8e1f-6ec3f617ffee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:22.721695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5e6ac66-65f8-4210-8e1f-6ec3f617ffee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:23.214175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a5e6ac66-65f8-4210-8e1f-6ec3f617ffee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:23.237534 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a5e6ac66-65f8-4210-8e1f-6ec3f617ffee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1315 microversion: 2.1 time: 0.555682 Sep 27 10:24:23.239227 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2442/5317] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:22 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1315 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:23.730638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ea430bb7-b3d2-4623-8860-8924759c17ae tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:23.732959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ea430bb7-b3d2-4623-8860-8924759c17ae tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:23.746575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea430bb7-b3d2-4623-8860-8924759c17ae tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:23.751313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea430bb7-b3d2-4623-8860-8924759c17ae tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:23.751975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea430bb7-b3d2-4623-8860-8924759c17ae tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:23.752947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea430bb7-b3d2-4623-8860-8924759c17ae tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:23.756344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea430bb7-b3d2-4623-8860-8924759c17ae tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:23.756842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea430bb7-b3d2-4623-8860-8924759c17ae tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:23.891294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ea430bb7-b3d2-4623-8860-8924759c17ae tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:23.909290 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ea430bb7-b3d2-4623-8860-8924759c17ae tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1053 microversion: 2.1 time: 0.183340 Sep 27 10:24:23.913151 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2876/5318] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:23 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1053 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:24.265111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f0083ead-b019-4dd5-8a61-bcf41446f307 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:24.265736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f0083ead-b019-4dd5-8a61-bcf41446f307 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:24.285911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0083ead-b019-4dd5-8a61-bcf41446f307 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:24.286879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0083ead-b019-4dd5-8a61-bcf41446f307 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:24.287969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0083ead-b019-4dd5-8a61-bcf41446f307 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:24.292988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0083ead-b019-4dd5-8a61-bcf41446f307 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:24.294015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0083ead-b019-4dd5-8a61-bcf41446f307 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:24.294899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0083ead-b019-4dd5-8a61-bcf41446f307 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:24.645978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f0083ead-b019-4dd5-8a61-bcf41446f307 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:24.657322 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f0083ead-b019-4dd5-8a61-bcf41446f307 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1502 microversion: 2.1 time: 0.399311 Sep 27 10:24:24.657943 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2443/5319] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:24 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1502 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:24.687420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed8b0ec2-48ec-449c-b41b-9222d42211bd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:24.696118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed8b0ec2-48ec-449c-b41b-9222d42211bd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:24.697007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed8b0ec2-48ec-449c-b41b-9222d42211bd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:24.697688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed8b0ec2-48ec-449c-b41b-9222d42211bd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:24.737062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed8b0ec2-48ec-449c-b41b-9222d42211bd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Going to try to stop instance {{(pid=145968) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 10:24:24.812164 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed8b0ec2-48ec-449c-b41b-9222d42211bd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f/action" status: 202 len: 0 microversion: 2.1 time: 0.134778 Sep 27 10:24:24.826528 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2877/5320] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:24:24 2022] POST /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f/action => generated 0 bytes in 150 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:24:24.838458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8853629d-b377-403a-a691-acf527ee7ab6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:24.839324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8853629d-b377-403a-a691-acf527ee7ab6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:24.850544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8853629d-b377-403a-a691-acf527ee7ab6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:24.851455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8853629d-b377-403a-a691-acf527ee7ab6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:24.852139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8853629d-b377-403a-a691-acf527ee7ab6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:24.853200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8853629d-b377-403a-a691-acf527ee7ab6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:24.853881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8853629d-b377-403a-a691-acf527ee7ab6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:24.854995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8853629d-b377-403a-a691-acf527ee7ab6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:24.938142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dc8da691-3d64-48ff-a1ab-cc1f05380c26 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:24.939206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dc8da691-3d64-48ff-a1ab-cc1f05380c26 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:24.948448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc8da691-3d64-48ff-a1ab-cc1f05380c26 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:24.949266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc8da691-3d64-48ff-a1ab-cc1f05380c26 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:24.950018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc8da691-3d64-48ff-a1ab-cc1f05380c26 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:24.951146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc8da691-3d64-48ff-a1ab-cc1f05380c26 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:24.951831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc8da691-3d64-48ff-a1ab-cc1f05380c26 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:24.952473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc8da691-3d64-48ff-a1ab-cc1f05380c26 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:25.122949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dc8da691-3d64-48ff-a1ab-cc1f05380c26 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:25.156903 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dc8da691-3d64-48ff-a1ab-cc1f05380c26 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1127 microversion: 2.1 time: 0.224669 Sep 27 10:24:25.158645 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2878/5321] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:24 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1127 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:24:25.326271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8853629d-b377-403a-a691-acf527ee7ab6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:25.338810 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8853629d-b377-403a-a691-acf527ee7ab6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.505614 Sep 27 10:24:25.347071 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2444/5322] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:24 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:26.192187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c4945163-7b32-42af-85f3-515800c5d786 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:26.193650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c4945163-7b32-42af-85f3-515800c5d786 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:26.207699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4945163-7b32-42af-85f3-515800c5d786 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:26.211031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4945163-7b32-42af-85f3-515800c5d786 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:26.211696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4945163-7b32-42af-85f3-515800c5d786 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:26.213841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4945163-7b32-42af-85f3-515800c5d786 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:26.218432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4945163-7b32-42af-85f3-515800c5d786 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:26.219142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c4945163-7b32-42af-85f3-515800c5d786 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:26.365361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea09d005-0da9-4786-8242-0bc390eded6c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:26.371005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ea09d005-0da9-4786-8242-0bc390eded6c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:26.391361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea09d005-0da9-4786-8242-0bc390eded6c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:26.409058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea09d005-0da9-4786-8242-0bc390eded6c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:26.409654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea09d005-0da9-4786-8242-0bc390eded6c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.018s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:26.410478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea09d005-0da9-4786-8242-0bc390eded6c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:26.410983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea09d005-0da9-4786-8242-0bc390eded6c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:26.411412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea09d005-0da9-4786-8242-0bc390eded6c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:26.792162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ea09d005-0da9-4786-8242-0bc390eded6c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:26.824737 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea09d005-0da9-4786-8242-0bc390eded6c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.463829 Sep 27 10:24:26.845387 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2445/5323] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:26 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:27.115019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c4945163-7b32-42af-85f3-515800c5d786 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:27.133053 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c4945163-7b32-42af-85f3-515800c5d786 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1169 microversion: 2.1 time: 0.949247 Sep 27 10:24:27.134377 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2879/5324] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:26 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1169 bytes in 953 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:27.853232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4eefed7e-c614-4119-b546-5f41d0ab7aaa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:27.854480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4eefed7e-c614-4119-b546-5f41d0ab7aaa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:27.869991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4eefed7e-c614-4119-b546-5f41d0ab7aaa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:27.874893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4eefed7e-c614-4119-b546-5f41d0ab7aaa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:27.880700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4eefed7e-c614-4119-b546-5f41d0ab7aaa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:27.881961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4eefed7e-c614-4119-b546-5f41d0ab7aaa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:27.883378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4eefed7e-c614-4119-b546-5f41d0ab7aaa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:27.884369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4eefed7e-c614-4119-b546-5f41d0ab7aaa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:28.002886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-df476b31-8be3-490b-90a9-6016f484d195 req-8c92efa9-1c49-49d1-8db6-76064af19a4a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a1aa79e2-46eb-4599-b206-1cb7a8e7c373", "tag": "9f10a9b6-ce87-4f11-8dd5-d6950a4e270e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:28.024985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-df476b31-8be3-490b-90a9-6016f484d195 req-8c92efa9-1c49-49d1-8db6-76064af19a4a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:28.027970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-df476b31-8be3-490b-90a9-6016f484d195 req-8c92efa9-1c49-49d1-8db6-76064af19a4a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:28.028573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-df476b31-8be3-490b-90a9-6016f484d195 req-8c92efa9-1c49-49d1-8db6-76064af19a4a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:28.067297 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-df476b31-8be3-490b-90a9-6016f484d195 req-8c92efa9-1c49-49d1-8db6-76064af19a4a service nova] Creating event network-changed:9f10a9b6-ce87-4f11-8dd5-d6950a4e270e for instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 on nch2-master-1 Sep 27 10:24:28.093281 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-df476b31-8be3-490b-90a9-6016f484d195 req-8c92efa9-1c49-49d1-8db6-76064af19a4a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.094564 Sep 27 10:24:28.096024 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2880/5325] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:24:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:24:28.160132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-252e9d7f-6534-4ed2-b0c6-2c0f47c3d50c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:28.160955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-252e9d7f-6534-4ed2-b0c6-2c0f47c3d50c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:28.176790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-252e9d7f-6534-4ed2-b0c6-2c0f47c3d50c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:28.177682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-252e9d7f-6534-4ed2-b0c6-2c0f47c3d50c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:28.178326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-252e9d7f-6534-4ed2-b0c6-2c0f47c3d50c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:28.179260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-252e9d7f-6534-4ed2-b0c6-2c0f47c3d50c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:28.180385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-252e9d7f-6534-4ed2-b0c6-2c0f47c3d50c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:28.181054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-252e9d7f-6534-4ed2-b0c6-2c0f47c3d50c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:28.311897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4eefed7e-c614-4119-b546-5f41d0ab7aaa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:28.321020 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4eefed7e-c614-4119-b546-5f41d0ab7aaa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.472733 Sep 27 10:24:28.321634 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2446/5326] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:27 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:28.512338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-252e9d7f-6534-4ed2-b0c6-2c0f47c3d50c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:28.534961 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-252e9d7f-6534-4ed2-b0c6-2c0f47c3d50c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1169 microversion: 2.1 time: 0.379026 Sep 27 10:24:28.553874 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2881/5327] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:28 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1169 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:29.345031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d0f3ef4d-1b44-4334-a272-6c6276e0c47c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:29.345664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d0f3ef4d-1b44-4334-a272-6c6276e0c47c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:29.366156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0f3ef4d-1b44-4334-a272-6c6276e0c47c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:29.370532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0f3ef4d-1b44-4334-a272-6c6276e0c47c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:29.371220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0f3ef4d-1b44-4334-a272-6c6276e0c47c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:29.372274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0f3ef4d-1b44-4334-a272-6c6276e0c47c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:29.375796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0f3ef4d-1b44-4334-a272-6c6276e0c47c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:29.376264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0f3ef4d-1b44-4334-a272-6c6276e0c47c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:29.562637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f2b608d-4879-41a2-b70b-6877a2fd705f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:29.563429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9f2b608d-4879-41a2-b70b-6877a2fd705f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:29.573946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f2b608d-4879-41a2-b70b-6877a2fd705f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:29.574461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f2b608d-4879-41a2-b70b-6877a2fd705f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:29.574916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f2b608d-4879-41a2-b70b-6877a2fd705f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:29.575759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f2b608d-4879-41a2-b70b-6877a2fd705f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:29.576099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f2b608d-4879-41a2-b70b-6877a2fd705f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:29.587019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f2b608d-4879-41a2-b70b-6877a2fd705f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:30.182329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9f2b608d-4879-41a2-b70b-6877a2fd705f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:30.198398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0f3ef4d-1b44-4334-a272-6c6276e0c47c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:30.210823 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d0f3ef4d-1b44-4334-a272-6c6276e0c47c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.869800 Sep 27 10:24:30.211457 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2447/5328] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:29 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 871 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:30.222590 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9f2b608d-4879-41a2-b70b-6877a2fd705f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1169 microversion: 2.1 time: 0.663545 Sep 27 10:24:30.239265 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2882/5329] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:29 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1169 bytes in 679 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:31.239060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c0c7358d-42e7-416d-b8f3-0ded9b820a34 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:31.239907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c0c7358d-42e7-416d-b8f3-0ded9b820a34 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:31.250241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e450da69-3862-4f0b-bb55-8215f9ea7933 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:31.250858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e450da69-3862-4f0b-bb55-8215f9ea7933 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:31.252255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c7358d-42e7-416d-b8f3-0ded9b820a34 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:31.253007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c7358d-42e7-416d-b8f3-0ded9b820a34 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:31.253691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c7358d-42e7-416d-b8f3-0ded9b820a34 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:31.255567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c7358d-42e7-416d-b8f3-0ded9b820a34 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:31.262687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e450da69-3862-4f0b-bb55-8215f9ea7933 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:31.266781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e450da69-3862-4f0b-bb55-8215f9ea7933 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:31.267246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e450da69-3862-4f0b-bb55-8215f9ea7933 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:31.268022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e450da69-3862-4f0b-bb55-8215f9ea7933 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:31.268491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e450da69-3862-4f0b-bb55-8215f9ea7933 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:31.275529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c7358d-42e7-416d-b8f3-0ded9b820a34 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:31.275750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e450da69-3862-4f0b-bb55-8215f9ea7933 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:31.277753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0c7358d-42e7-416d-b8f3-0ded9b820a34 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:31.653016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c0c7358d-42e7-416d-b8f3-0ded9b820a34 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:31.674674 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c0c7358d-42e7-416d-b8f3-0ded9b820a34 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.440364 Sep 27 10:24:31.679046 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2448/5330] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:31 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:31.724530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e450da69-3862-4f0b-bb55-8215f9ea7933 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:31.744798 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e450da69-3862-4f0b-bb55-8215f9ea7933 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1215 microversion: 2.1 time: 0.500830 Sep 27 10:24:31.761416 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2883/5331] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:31 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1215 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:24:32.706356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5f18bb54-1080-4003-b216-330873f3a4c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:32.707618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5f18bb54-1080-4003-b216-330873f3a4c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:32.711263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:32.712278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:32.727054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f18bb54-1080-4003-b216-330873f3a4c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:32.727750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f18bb54-1080-4003-b216-330873f3a4c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:32.728373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f18bb54-1080-4003-b216-330873f3a4c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:32.729159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f18bb54-1080-4003-b216-330873f3a4c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:32.729924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f18bb54-1080-4003-b216-330873f3a4c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:32.731168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f18bb54-1080-4003-b216-330873f3a4c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:32.771946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2bb97775-017b-44e3-92a8-acc51f02cf47 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:32.772884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2bb97775-017b-44e3-92a8-acc51f02cf47 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:32.783571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:32.787274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bb97775-017b-44e3-92a8-acc51f02cf47 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:32.787777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bb97775-017b-44e3-92a8-acc51f02cf47 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:32.790458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bb97775-017b-44e3-92a8-acc51f02cf47 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:32.791172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bb97775-017b-44e3-92a8-acc51f02cf47 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:32.792831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bb97775-017b-44e3-92a8-acc51f02cf47 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:32.795251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2bb97775-017b-44e3-92a8-acc51f02cf47 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:33.034836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5f18bb54-1080-4003-b216-330873f3a4c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:33.048258 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5f18bb54-1080-4003-b216-330873f3a4c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.345806 Sep 27 10:24:33.049581 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2449/5332] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:32 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:33.139796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2bb97775-017b-44e3-92a8-acc51f02cf47 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:33.151244 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2bb97775-017b-44e3-92a8-acc51f02cf47 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1203 microversion: 2.1 time: 0.385012 Sep 27 10:24:33.169030 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2884/5333] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:32 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1203 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:34.073452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-df4a775f-7cfd-48be-a921-9c3a8961be1e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:34.077400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-df4a775f-7cfd-48be-a921-9c3a8961be1e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:34.105157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df4a775f-7cfd-48be-a921-9c3a8961be1e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:34.105872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df4a775f-7cfd-48be-a921-9c3a8961be1e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:34.109237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df4a775f-7cfd-48be-a921-9c3a8961be1e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:34.110001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df4a775f-7cfd-48be-a921-9c3a8961be1e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:34.110525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df4a775f-7cfd-48be-a921-9c3a8961be1e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:34.111086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df4a775f-7cfd-48be-a921-9c3a8961be1e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:34.176397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-581b93d6-2e01-4f1b-829d-5563287c85b6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:34.178453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-581b93d6-2e01-4f1b-829d-5563287c85b6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:34.189723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-581b93d6-2e01-4f1b-829d-5563287c85b6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:34.189723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-581b93d6-2e01-4f1b-829d-5563287c85b6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:34.189723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-581b93d6-2e01-4f1b-829d-5563287c85b6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:34.204720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-581b93d6-2e01-4f1b-829d-5563287c85b6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:34.205191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-581b93d6-2e01-4f1b-829d-5563287c85b6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:34.209554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-581b93d6-2e01-4f1b-829d-5563287c85b6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:34.544319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-df4a775f-7cfd-48be-a921-9c3a8961be1e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:34.554559 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-df4a775f-7cfd-48be-a921-9c3a8961be1e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.485833 Sep 27 10:24:34.560931 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2450/5334] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:34 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:34.615460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-581b93d6-2e01-4f1b-829d-5563287c85b6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:34.648867 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-581b93d6-2e01-4f1b-829d-5563287c85b6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1203 microversion: 2.1 time: 0.476563 Sep 27 10:24:34.650000 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2885/5335] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:34 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1203 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:35.583651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-91a650f9-29fc-4054-8458-dacd15e4334b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:35.584617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-91a650f9-29fc-4054-8458-dacd15e4334b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:35.597727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91a650f9-29fc-4054-8458-dacd15e4334b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:35.598290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91a650f9-29fc-4054-8458-dacd15e4334b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:35.598739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91a650f9-29fc-4054-8458-dacd15e4334b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:35.599475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91a650f9-29fc-4054-8458-dacd15e4334b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:35.599958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91a650f9-29fc-4054-8458-dacd15e4334b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:35.600373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91a650f9-29fc-4054-8458-dacd15e4334b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:35.677186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1cf96314-9728-4562-a622-6472bb43c02b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:35.678751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1cf96314-9728-4562-a622-6472bb43c02b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:35.690155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cf96314-9728-4562-a622-6472bb43c02b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:35.695637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cf96314-9728-4562-a622-6472bb43c02b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:35.702174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cf96314-9728-4562-a622-6472bb43c02b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:35.703769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cf96314-9728-4562-a622-6472bb43c02b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:35.704737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cf96314-9728-4562-a622-6472bb43c02b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:35.705371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cf96314-9728-4562-a622-6472bb43c02b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:35.718407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:35.924514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1cf96314-9728-4562-a622-6472bb43c02b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:35.938459 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1cf96314-9728-4562-a622-6472bb43c02b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1203 microversion: 2.1 time: 0.265806 Sep 27 10:24:35.940305 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2886/5336] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:35 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1203 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:24:36.392270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91a650f9-29fc-4054-8458-dacd15e4334b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:36.406303 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-91a650f9-29fc-4054-8458-dacd15e4334b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.827077 Sep 27 10:24:36.425160 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2451/5337] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:35 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 847 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:36.969205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f50e8b44-a4fc-4351-8afc-4ba55229dc10 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:36.971089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f50e8b44-a4fc-4351-8afc-4ba55229dc10 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:36.987340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f50e8b44-a4fc-4351-8afc-4ba55229dc10 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:36.989203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f50e8b44-a4fc-4351-8afc-4ba55229dc10 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:36.989867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f50e8b44-a4fc-4351-8afc-4ba55229dc10 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:36.991738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f50e8b44-a4fc-4351-8afc-4ba55229dc10 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:36.992696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f50e8b44-a4fc-4351-8afc-4ba55229dc10 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:36.993431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f50e8b44-a4fc-4351-8afc-4ba55229dc10 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:37.221292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f50e8b44-a4fc-4351-8afc-4ba55229dc10 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:37.232287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f50e8b44-a4fc-4351-8afc-4ba55229dc10 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1203 microversion: 2.1 time: 0.269174 Sep 27 10:24:37.233059 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2887/5338] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:36 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1203 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:37.435524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dc2fd2c7-a4b0-46aa-88ee-e73594e21032 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:37.436227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dc2fd2c7-a4b0-46aa-88ee-e73594e21032 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:37.444494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2fd2c7-a4b0-46aa-88ee-e73594e21032 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:37.447408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2fd2c7-a4b0-46aa-88ee-e73594e21032 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:37.447892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2fd2c7-a4b0-46aa-88ee-e73594e21032 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:37.448607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2fd2c7-a4b0-46aa-88ee-e73594e21032 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:37.449270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2fd2c7-a4b0-46aa-88ee-e73594e21032 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:37.449859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2fd2c7-a4b0-46aa-88ee-e73594e21032 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:37.845828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dc2fd2c7-a4b0-46aa-88ee-e73594e21032 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:37.859245 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dc2fd2c7-a4b0-46aa-88ee-e73594e21032 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.431172 Sep 27 10:24:37.860380 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2452/5339] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:37 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:38.262639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8fa71636-32f3-4f51-a52c-d9439ef5afc3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:38.266551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8fa71636-32f3-4f51-a52c-d9439ef5afc3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:38.294108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fa71636-32f3-4f51-a52c-d9439ef5afc3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:38.295472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fa71636-32f3-4f51-a52c-d9439ef5afc3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:38.296745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fa71636-32f3-4f51-a52c-d9439ef5afc3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:38.298073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fa71636-32f3-4f51-a52c-d9439ef5afc3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:38.298845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fa71636-32f3-4f51-a52c-d9439ef5afc3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:38.299812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fa71636-32f3-4f51-a52c-d9439ef5afc3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:38.336057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-0cd6abe8-5b17-40d2-9e25-ded8383f9128 req-bdb1431d-3d12-4923-8daa-b54a2b9db8fb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1aa79e2-46eb-4599-b206-1cb7a8e7c373", "name": "network-vif-plugged", "status": "completed", "tag": "9f10a9b6-ce87-4f11-8dd5-d6950a4e270e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:38.359036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0cd6abe8-5b17-40d2-9e25-ded8383f9128 req-bdb1431d-3d12-4923-8daa-b54a2b9db8fb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:38.359797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0cd6abe8-5b17-40d2-9e25-ded8383f9128 req-bdb1431d-3d12-4923-8daa-b54a2b9db8fb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:38.360493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-0cd6abe8-5b17-40d2-9e25-ded8383f9128 req-bdb1431d-3d12-4923-8daa-b54a2b9db8fb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:38.399211 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-0cd6abe8-5b17-40d2-9e25-ded8383f9128 req-bdb1431d-3d12-4923-8daa-b54a2b9db8fb service nova] Creating event network-vif-plugged:9f10a9b6-ce87-4f11-8dd5-d6950a4e270e for instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 on nch2-master-1 Sep 27 10:24:38.409145 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-0cd6abe8-5b17-40d2-9e25-ded8383f9128 req-bdb1431d-3d12-4923-8daa-b54a2b9db8fb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077782 Sep 27 10:24:38.410658 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2453/5340] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:24:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:24:38.671889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8fa71636-32f3-4f51-a52c-d9439ef5afc3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:38.682669 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8fa71636-32f3-4f51-a52c-d9439ef5afc3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1203 microversion: 2.1 time: 0.429102 Sep 27 10:24:38.683295 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2888/5341] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:38 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1203 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:38.887779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f7783803-3c95-4aac-b88e-9bc20c2afeda tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:38.888581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f7783803-3c95-4aac-b88e-9bc20c2afeda tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:38.907771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7783803-3c95-4aac-b88e-9bc20c2afeda tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:38.908282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7783803-3c95-4aac-b88e-9bc20c2afeda tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:38.908753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7783803-3c95-4aac-b88e-9bc20c2afeda tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:38.909493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7783803-3c95-4aac-b88e-9bc20c2afeda tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:38.909961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7783803-3c95-4aac-b88e-9bc20c2afeda tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:38.913486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7783803-3c95-4aac-b88e-9bc20c2afeda tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:38.928267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1780440030", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "ecc1560a-8770-41ea-af0f-93cdb9ee5b8d"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:39.030561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] validate_networks() for [('ecc1560a-8770-41ea-af0f-93cdb9ee5b8d', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:24:39.097863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f7783803-3c95-4aac-b88e-9bc20c2afeda tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:39.107028 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f7783803-3c95-4aac-b88e-9bc20c2afeda tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.224731 Sep 27 10:24:39.107679 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2454/5342] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:38 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:39.534591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:24:39.535184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:24:39.535605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:24:39.536066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:24:39.536452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:24:39.560078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:39.560578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:39.561075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:39.561843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:39.562301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:39.562810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:39.590888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Getting quotas for project 712c1c4c858b4b788a7703345716504b. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:24:39.597368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Getting quotas for user 584c395757a54d9b9a64dd399d8557b1 and project 712c1c4c858b4b788a7703345716504b. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:24:39.634954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:24:39.638462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:24:39.707059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75a7b44a-2258-4e12-a137-8c3111b34fdb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:39.708116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-75a7b44a-2258-4e12-a137-8c3111b34fdb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:39.713129 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-63fecc98-ce72-4724-9dff-d81e4891cff8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.790910 Sep 27 10:24:39.717209 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2889/5343] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:24:38 2022] POST /compute/v2.1/servers => generated 380 bytes in 796 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:24:39.725533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7b44a-2258-4e12-a137-8c3111b34fdb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:39.728571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7b44a-2258-4e12-a137-8c3111b34fdb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:39.729180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7b44a-2258-4e12-a137-8c3111b34fdb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:39.730036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7b44a-2258-4e12-a137-8c3111b34fdb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:39.730181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bb0ed814-5cfa-44fd-b57d-084f8b90ed95 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:39.731707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bb0ed814-5cfa-44fd-b57d-084f8b90ed95 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:39.732085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7b44a-2258-4e12-a137-8c3111b34fdb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:39.733016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a7b44a-2258-4e12-a137-8c3111b34fdb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:39.821435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb0ed814-5cfa-44fd-b57d-084f8b90ed95 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:39.829621 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bb0ed814-5cfa-44fd-b57d-084f8b90ed95 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1219 microversion: 2.1 time: 0.105156 Sep 27 10:24:39.830947 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2890/5344] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:39 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1219 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:24:40.044335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-75a7b44a-2258-4e12-a137-8c3111b34fdb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:40.063878 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-75a7b44a-2258-4e12-a137-8c3111b34fdb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1203 microversion: 2.1 time: 0.359456 Sep 27 10:24:40.064533 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2455/5345] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:39 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1203 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:24:40.130478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-78277323-fdba-4f79-900c-4fd065445c43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:40.132946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-78277323-fdba-4f79-900c-4fd065445c43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:40.148174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78277323-fdba-4f79-900c-4fd065445c43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:40.149379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78277323-fdba-4f79-900c-4fd065445c43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:40.150041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78277323-fdba-4f79-900c-4fd065445c43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:40.151211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78277323-fdba-4f79-900c-4fd065445c43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:40.152338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78277323-fdba-4f79-900c-4fd065445c43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:40.153004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78277323-fdba-4f79-900c-4fd065445c43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:40.385820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-78277323-fdba-4f79-900c-4fd065445c43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:40.395867 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-78277323-fdba-4f79-900c-4fd065445c43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.268801 Sep 27 10:24:40.412521 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2891/5346] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:40 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:40.856115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5dce7b50-4c23-4d22-ad57-a3e6f84c4153 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:40.857276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5dce7b50-4c23-4d22-ad57-a3e6f84c4153 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:40.860386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=49,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:40.867146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dce7b50-4c23-4d22-ad57-a3e6f84c4153 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:40.867842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dce7b50-4c23-4d22-ad57-a3e6f84c4153 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:40.868551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dce7b50-4c23-4d22-ad57-a3e6f84c4153 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:40.869441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dce7b50-4c23-4d22-ad57-a3e6f84c4153 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:40.870183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dce7b50-4c23-4d22-ad57-a3e6f84c4153 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:40.870681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dce7b50-4c23-4d22-ad57-a3e6f84c4153 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:40.961814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5dce7b50-4c23-4d22-ad57-a3e6f84c4153 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:40.977923 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5dce7b50-4c23-4d22-ad57-a3e6f84c4153 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1271 microversion: 2.1 time: 0.125860 Sep 27 10:24:40.978838 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2456/5347] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:40 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1271 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:41.087254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3e8213b-26a5-416c-b98c-faea1bae7c60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:41.087851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3e8213b-26a5-416c-b98c-faea1bae7c60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:41.094981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e8213b-26a5-416c-b98c-faea1bae7c60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:41.095478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e8213b-26a5-416c-b98c-faea1bae7c60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:41.095922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e8213b-26a5-416c-b98c-faea1bae7c60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:41.096740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e8213b-26a5-416c-b98c-faea1bae7c60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:41.097156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e8213b-26a5-416c-b98c-faea1bae7c60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:41.097591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e8213b-26a5-416c-b98c-faea1bae7c60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:41.252367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b3e8213b-26a5-416c-b98c-faea1bae7c60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:41.262959 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3e8213b-26a5-416c-b98c-faea1bae7c60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1390 microversion: 2.1 time: 0.179907 Sep 27 10:24:41.264141 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2892/5348] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:41 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1390 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:41.287990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3055901f-034b-4f50-9483-bd057d9ae7be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:41.289318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3055901f-034b-4f50-9483-bd057d9ae7be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:41.296333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3055901f-034b-4f50-9483-bd057d9ae7be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:41.296863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3055901f-034b-4f50-9483-bd057d9ae7be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:41.297629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3055901f-034b-4f50-9483-bd057d9ae7be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:41.298639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3055901f-034b-4f50-9483-bd057d9ae7be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:41.299262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3055901f-034b-4f50-9483-bd057d9ae7be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:41.299687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3055901f-034b-4f50-9483-bd057d9ae7be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:41.421133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-19312f05-9a69-4eb7-a46e-35bb0f9198aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:41.422337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-19312f05-9a69-4eb7-a46e-35bb0f9198aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:41.437556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19312f05-9a69-4eb7-a46e-35bb0f9198aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:41.438433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19312f05-9a69-4eb7-a46e-35bb0f9198aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:41.439418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19312f05-9a69-4eb7-a46e-35bb0f9198aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:41.443166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19312f05-9a69-4eb7-a46e-35bb0f9198aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:41.443739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19312f05-9a69-4eb7-a46e-35bb0f9198aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:41.445328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19312f05-9a69-4eb7-a46e-35bb0f9198aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:41.634460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3055901f-034b-4f50-9483-bd057d9ae7be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:41.651994 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3055901f-034b-4f50-9483-bd057d9ae7be tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1390 microversion: 2.1 time: 0.369051 Sep 27 10:24:41.652877 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2457/5349] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:41 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1390 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:41.678705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-67d85cef-ff26-4029-9de0-50a8978ae245 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:41.679320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-67d85cef-ff26-4029-9de0-50a8978ae245 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:41.685722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67d85cef-ff26-4029-9de0-50a8978ae245 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:41.686179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67d85cef-ff26-4029-9de0-50a8978ae245 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:41.686609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67d85cef-ff26-4029-9de0-50a8978ae245 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:41.687538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67d85cef-ff26-4029-9de0-50a8978ae245 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:41.688185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67d85cef-ff26-4029-9de0-50a8978ae245 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:41.688611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67d85cef-ff26-4029-9de0-50a8978ae245 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:41.694617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:41.895193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-67d85cef-ff26-4029-9de0-50a8978ae245 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:41.904297 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-67d85cef-ff26-4029-9de0-50a8978ae245 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1390 microversion: 2.1 time: 0.229906 Sep 27 10:24:41.905777 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2458/5350] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:41 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1390 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:41.971075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-19312f05-9a69-4eb7-a46e-35bb0f9198aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:41.980108 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-19312f05-9a69-4eb7-a46e-35bb0f9198aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.565198 Sep 27 10:24:41.983243 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2893/5351] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:41 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:42.006184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-120fb583-7795-4cea-aca1-ea353df16ae8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:42.006876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-120fb583-7795-4cea-aca1-ea353df16ae8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:42.034128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120fb583-7795-4cea-aca1-ea353df16ae8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:42.035552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:42.035990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120fb583-7795-4cea-aca1-ea353df16ae8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:42.036130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:42.036892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120fb583-7795-4cea-aca1-ea353df16ae8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:42.038332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120fb583-7795-4cea-aca1-ea353df16ae8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:42.039287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120fb583-7795-4cea-aca1-ea353df16ae8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:42.039914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120fb583-7795-4cea-aca1-ea353df16ae8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:42.042431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:42.042938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:42.043507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:42.044099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:42.044565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:42.045227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:42.089995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:24:42.148716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-120fb583-7795-4cea-aca1-ea353df16ae8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:42.158242 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-120fb583-7795-4cea-aca1-ea353df16ae8 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1277 microversion: 2.1 time: 0.156240 Sep 27 10:24:42.163667 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2459/5352] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:42 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1277 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:42.205346 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0f7a75c1-71e6-4bd1-88a4-df44ed600c0c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 204 len: 0 microversion: 2.1 time: 0.173301 Sep 27 10:24:42.210266 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2894/5353] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:24:42 2022] DELETE /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:24:42.216010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-15224655-7ebd-499b-83cb-3e29058344a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:42.216958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-15224655-7ebd-499b-83cb-3e29058344a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:42.224032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15224655-7ebd-499b-83cb-3e29058344a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:42.224832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15224655-7ebd-499b-83cb-3e29058344a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:42.225470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15224655-7ebd-499b-83cb-3e29058344a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:42.226222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15224655-7ebd-499b-83cb-3e29058344a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:42.226687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15224655-7ebd-499b-83cb-3e29058344a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:42.227134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15224655-7ebd-499b-83cb-3e29058344a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:42.405472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-15224655-7ebd-499b-83cb-3e29058344a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:42.415948 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-15224655-7ebd-499b-83cb-3e29058344a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1396 microversion: 2.1 time: 0.204463 Sep 27 10:24:42.417129 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2460/5354] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:42 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1396 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:43.009340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7380e955-b595-421f-b9a3-1db6c1efb6ad tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:43.010207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7380e955-b595-421f-b9a3-1db6c1efb6ad tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:43.018223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7380e955-b595-421f-b9a3-1db6c1efb6ad tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:43.018986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7380e955-b595-421f-b9a3-1db6c1efb6ad tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:43.019637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7380e955-b595-421f-b9a3-1db6c1efb6ad tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:43.020788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7380e955-b595-421f-b9a3-1db6c1efb6ad tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:43.022073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7380e955-b595-421f-b9a3-1db6c1efb6ad tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:43.022718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7380e955-b595-421f-b9a3-1db6c1efb6ad tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:43.182908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d8827744-7d7c-46d5-89a7-2faa106e05be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:43.183800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d8827744-7d7c-46d5-89a7-2faa106e05be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:43.191319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8827744-7d7c-46d5-89a7-2faa106e05be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:43.191881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8827744-7d7c-46d5-89a7-2faa106e05be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:43.193001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8827744-7d7c-46d5-89a7-2faa106e05be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:43.193737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8827744-7d7c-46d5-89a7-2faa106e05be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:43.194191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8827744-7d7c-46d5-89a7-2faa106e05be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:43.194624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8827744-7d7c-46d5-89a7-2faa106e05be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:43.270014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d8827744-7d7c-46d5-89a7-2faa106e05be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:43.279522 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d8827744-7d7c-46d5-89a7-2faa106e05be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1277 microversion: 2.1 time: 0.100485 Sep 27 10:24:43.295810 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2461/5355] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:43 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1277 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:43.355407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7380e955-b595-421f-b9a3-1db6c1efb6ad tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:43.364186 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7380e955-b595-421f-b9a3-1db6c1efb6ad tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.359831 Sep 27 10:24:43.364977 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2895/5356] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:43 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:43.444260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5517738-c7bd-46f4-9dcf-868b7859a422 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:43.445575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5517738-c7bd-46f4-9dcf-868b7859a422 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:43.451809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5517738-c7bd-46f4-9dcf-868b7859a422 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:43.452539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5517738-c7bd-46f4-9dcf-868b7859a422 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:43.452991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5517738-c7bd-46f4-9dcf-868b7859a422 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:43.453913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5517738-c7bd-46f4-9dcf-868b7859a422 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:43.454379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5517738-c7bd-46f4-9dcf-868b7859a422 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:43.454814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5517738-c7bd-46f4-9dcf-868b7859a422 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:43.675662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5517738-c7bd-46f4-9dcf-868b7859a422 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:43.692130 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5517738-c7bd-46f4-9dcf-868b7859a422 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1396 microversion: 2.1 time: 0.251341 Sep 27 10:24:43.693971 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2462/5357] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:43 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1396 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:44.304135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bdab4398-77a0-493b-a0c6-3c3112da0f1e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:44.304984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bdab4398-77a0-493b-a0c6-3c3112da0f1e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:44.312258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdab4398-77a0-493b-a0c6-3c3112da0f1e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:44.313039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdab4398-77a0-493b-a0c6-3c3112da0f1e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:44.313637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdab4398-77a0-493b-a0c6-3c3112da0f1e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:44.314514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdab4398-77a0-493b-a0c6-3c3112da0f1e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:44.315131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdab4398-77a0-493b-a0c6-3c3112da0f1e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:44.315713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bdab4398-77a0-493b-a0c6-3c3112da0f1e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:44.387043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-93a3a44f-fb3e-4ee3-a355-ea7e42954f04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:44.388405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-93a3a44f-fb3e-4ee3-a355-ea7e42954f04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:44.396063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93a3a44f-fb3e-4ee3-a355-ea7e42954f04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:44.396565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93a3a44f-fb3e-4ee3-a355-ea7e42954f04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:44.397041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93a3a44f-fb3e-4ee3-a355-ea7e42954f04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:44.397773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93a3a44f-fb3e-4ee3-a355-ea7e42954f04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:44.398238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93a3a44f-fb3e-4ee3-a355-ea7e42954f04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:44.398672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-93a3a44f-fb3e-4ee3-a355-ea7e42954f04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:44.572492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bdab4398-77a0-493b-a0c6-3c3112da0f1e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:44.588628 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bdab4398-77a0-493b-a0c6-3c3112da0f1e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1319 microversion: 2.1 time: 0.288043 Sep 27 10:24:44.589286 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2896/5358] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:44 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1319 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:24:44.718441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c905ea5c-7259-498d-bb18-ccf2efbbac3f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:44.719808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c905ea5c-7259-498d-bb18-ccf2efbbac3f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:44.728873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c905ea5c-7259-498d-bb18-ccf2efbbac3f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:44.729855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c905ea5c-7259-498d-bb18-ccf2efbbac3f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:44.730523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c905ea5c-7259-498d-bb18-ccf2efbbac3f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:44.731434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c905ea5c-7259-498d-bb18-ccf2efbbac3f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:44.732506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c905ea5c-7259-498d-bb18-ccf2efbbac3f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:44.733121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c905ea5c-7259-498d-bb18-ccf2efbbac3f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:44.773084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-93a3a44f-fb3e-4ee3-a355-ea7e42954f04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:44.787226 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-93a3a44f-fb3e-4ee3-a355-ea7e42954f04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.402625 Sep 27 10:24:44.789113 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2463/5359] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:44 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:44.955354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c905ea5c-7259-498d-bb18-ccf2efbbac3f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:44.977750 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c905ea5c-7259-498d-bb18-ccf2efbbac3f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1396 microversion: 2.1 time: 0.263804 Sep 27 10:24:44.993838 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2897/5360] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:44 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1396 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:45.102226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7d6be3a0-1723-4674-98f6-c26f3f0a7fda req-6f761a4c-74a3-4c57-ae0d-3e473149f837 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ac45a77-b872-4aab-93f2-25556deb0d3f", "tag": "52b80ca4-891c-420a-a226-c0f942315728"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:45.116495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7d6be3a0-1723-4674-98f6-c26f3f0a7fda req-6f761a4c-74a3-4c57-ae0d-3e473149f837 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:45.118232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7d6be3a0-1723-4674-98f6-c26f3f0a7fda req-6f761a4c-74a3-4c57-ae0d-3e473149f837 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:45.119284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7d6be3a0-1723-4674-98f6-c26f3f0a7fda req-6f761a4c-74a3-4c57-ae0d-3e473149f837 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:45.151393 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7d6be3a0-1723-4674-98f6-c26f3f0a7fda req-6f761a4c-74a3-4c57-ae0d-3e473149f837 service nova] Creating event network-changed:52b80ca4-891c-420a-a226-c0f942315728 for instance 7ac45a77-b872-4aab-93f2-25556deb0d3f on nch1-master-1 Sep 27 10:24:45.158768 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7d6be3a0-1723-4674-98f6-c26f3f0a7fda req-6f761a4c-74a3-4c57-ae0d-3e473149f837 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071209 Sep 27 10:24:45.159742 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2464/5361] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:24:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:24:45.374553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0856ae4e-22ca-4724-a962-c544c9a53826 req-eb7b51b7-d2e0-432e-acd6-59ff94d4586c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a1aa79e2-46eb-4599-b206-1cb7a8e7c373", "name": "network-vif-deleted", "tag": "9f10a9b6-ce87-4f11-8dd5-d6950a4e270e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:45.401141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0856ae4e-22ca-4724-a962-c544c9a53826 req-eb7b51b7-d2e0-432e-acd6-59ff94d4586c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:45.401972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0856ae4e-22ca-4724-a962-c544c9a53826 req-eb7b51b7-d2e0-432e-acd6-59ff94d4586c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:45.402686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0856ae4e-22ca-4724-a962-c544c9a53826 req-eb7b51b7-d2e0-432e-acd6-59ff94d4586c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:45.451752 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0856ae4e-22ca-4724-a962-c544c9a53826 req-eb7b51b7-d2e0-432e-acd6-59ff94d4586c service nova] Creating event network-vif-deleted:9f10a9b6-ce87-4f11-8dd5-d6950a4e270e for instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 on nch2-master-1 Sep 27 10:24:45.482988 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0856ae4e-22ca-4724-a962-c544c9a53826 req-eb7b51b7-d2e0-432e-acd6-59ff94d4586c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.112579 Sep 27 10:24:45.484998 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2898/5362] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:24:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:24:45.635988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e03e1a4a-f316-404b-8f20-502b79a55b8e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:45.637030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e03e1a4a-f316-404b-8f20-502b79a55b8e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:45.654836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e03e1a4a-f316-404b-8f20-502b79a55b8e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:45.655731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e03e1a4a-f316-404b-8f20-502b79a55b8e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:45.656784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e03e1a4a-f316-404b-8f20-502b79a55b8e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:45.658396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e03e1a4a-f316-404b-8f20-502b79a55b8e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:45.659062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e03e1a4a-f316-404b-8f20-502b79a55b8e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:45.659739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e03e1a4a-f316-404b-8f20-502b79a55b8e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:45.813256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad5d53fb-77bc-42a4-8c5d-b35420df1b6a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:45.821659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ad5d53fb-77bc-42a4-8c5d-b35420df1b6a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:45.844494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad5d53fb-77bc-42a4-8c5d-b35420df1b6a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:45.845509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad5d53fb-77bc-42a4-8c5d-b35420df1b6a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:45.850722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad5d53fb-77bc-42a4-8c5d-b35420df1b6a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:45.851709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad5d53fb-77bc-42a4-8c5d-b35420df1b6a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:45.852345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad5d53fb-77bc-42a4-8c5d-b35420df1b6a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:45.853003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad5d53fb-77bc-42a4-8c5d-b35420df1b6a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:46.103059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ad5d53fb-77bc-42a4-8c5d-b35420df1b6a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:46.113192 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ad5d53fb-77bc-42a4-8c5d-b35420df1b6a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.305296 Sep 27 10:24:46.132929 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2899/5363] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:45 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:46.139346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:46.140268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:46.147778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:46.148529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:46.149720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:46.150905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:46.151535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:46.152127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:46.193633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:46.194369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:46.194971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:46.195616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lazy-loading 'fault' on Instance uuid a1aa79e2-46eb-4599-b206-1cb7a8e7c373 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:24:46.199702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e03e1a4a-f316-404b-8f20-502b79a55b8e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:46.231212 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e03e1a4a-f316-404b-8f20-502b79a55b8e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1319 microversion: 2.1 time: 0.608079 Sep 27 10:24:46.241897 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2465/5364] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:45 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1319 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:46.247974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:46.267398 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-346ba255-f855-49a9-bf9d-59bfa5828043 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 200 len: 1193 microversion: 2.1 time: 0.132557 Sep 27 10:24:46.268386 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2900/5365] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:46 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 1193 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:47.136689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5deb8ae6-8fde-43e5-84b9-b99f92cc9234 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:47.137848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5deb8ae6-8fde-43e5-84b9-b99f92cc9234 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:47.145411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5deb8ae6-8fde-43e5-84b9-b99f92cc9234 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:47.146251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5deb8ae6-8fde-43e5-84b9-b99f92cc9234 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:47.146793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5deb8ae6-8fde-43e5-84b9-b99f92cc9234 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:47.147719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5deb8ae6-8fde-43e5-84b9-b99f92cc9234 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:47.148355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5deb8ae6-8fde-43e5-84b9-b99f92cc9234 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:47.149119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5deb8ae6-8fde-43e5-84b9-b99f92cc9234 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:47.256790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-edae4191-16d5-47f2-8800-2b7698243189 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:47.258413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-edae4191-16d5-47f2-8800-2b7698243189 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:47.266157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edae4191-16d5-47f2-8800-2b7698243189 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:47.266579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edae4191-16d5-47f2-8800-2b7698243189 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:47.267009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edae4191-16d5-47f2-8800-2b7698243189 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:47.267888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edae4191-16d5-47f2-8800-2b7698243189 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:47.268497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edae4191-16d5-47f2-8800-2b7698243189 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:47.269126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edae4191-16d5-47f2-8800-2b7698243189 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:47.343383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5deb8ae6-8fde-43e5-84b9-b99f92cc9234 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:47.351989 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5deb8ae6-8fde-43e5-84b9-b99f92cc9234 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.219226 Sep 27 10:24:47.353135 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2466/5366] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:47 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:47.360322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:47.360973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:47.374263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:47.374977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:47.375601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:47.377254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:47.377722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:47.378320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:47.392975 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 HTTP/1.1 Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'a1aa79e2-46eb-4599-b206-1cb7a8e7c373'} Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:24:47.394873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.403873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:24:47.403873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:24:47.403873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:24:47.403873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:24:47.403873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:24:47.403873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.403873 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.403873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:24:47.403873 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7dc8ea86-8dca-41ce-ac5c-bc076d25f9e4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 404 len: 111 microversion: 2.1 time: 0.042392 Sep 27 10:24:47.403873 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2467/5367] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:47 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:24:47.432028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:47.432614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:47.439359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:47.439942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:47.440520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:47.441352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:47.442277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:47.442972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:47.465234 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 HTTP/1.1 Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'a1aa79e2-46eb-4599-b206-1cb7a8e7c373'} Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:24:47.467003 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:24:47.471447 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.471447 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.471447 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:24:47.471447 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.471447 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:24:47.471447 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:24:47.471447 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:24:47.471447 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:24:47.471447 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:24:47.471447 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.471447 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.473518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:24:47.474757 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-75b0db15-7d11-477b-9f82-bf5552d3dbc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 404 len: 111 microversion: 2.1 time: 0.046814 Sep 27 10:24:47.475901 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2468/5368] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:24:47 2022] DELETE /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:24:47.485344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:47.486210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a1aa79e2-46eb-4599-b206-1cb7a8e7c373] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:47.494667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:47.495390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:47.496029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:47.497126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:47.497773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:47.498437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:47.520894 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 HTTP/1.1 Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'a1aa79e2-46eb-4599-b206-1cb7a8e7c373'} Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:24:47.524020 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.529096 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:24:47.529096 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:24:47.529096 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:24:47.529096 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:24:47.529096 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:24:47.529096 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. Sep 27 10:24:47.529096 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:24:47.531161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance a1aa79e2-46eb-4599-b206-1cb7a8e7c373 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:24:47.532750 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af25406b-0ff2-4ecf-af0a-9573ae68a17f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373" status: 404 len: 111 microversion: 2.1 time: 0.052182 Sep 27 10:24:47.534296 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2469/5369] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:47 2022] GET /compute/v2.1/servers/a1aa79e2-46eb-4599-b206-1cb7a8e7c373 => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:24:47.661023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-edae4191-16d5-47f2-8800-2b7698243189 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:47.675228 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-edae4191-16d5-47f2-8800-2b7698243189 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1319 microversion: 2.1 time: 0.422269 Sep 27 10:24:47.681025 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2901/5370] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:47 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1319 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:24:48.378485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c1429a16-50bd-44c2-b630-cb9ea09f3ed1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:48.378906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c1429a16-50bd-44c2-b630-cb9ea09f3ed1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:48.389584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1429a16-50bd-44c2-b630-cb9ea09f3ed1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:48.390322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1429a16-50bd-44c2-b630-cb9ea09f3ed1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:48.390729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1429a16-50bd-44c2-b630-cb9ea09f3ed1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:48.391545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1429a16-50bd-44c2-b630-cb9ea09f3ed1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:48.392032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1429a16-50bd-44c2-b630-cb9ea09f3ed1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:48.392853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1429a16-50bd-44c2-b630-cb9ea09f3ed1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:48.896197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9c59cceb-40c1-4b59-b128-c742ba9d44f3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:48.896730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9c59cceb-40c1-4b59-b128-c742ba9d44f3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:48.906989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c59cceb-40c1-4b59-b128-c742ba9d44f3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:48.907683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c59cceb-40c1-4b59-b128-c742ba9d44f3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:48.909763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c59cceb-40c1-4b59-b128-c742ba9d44f3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:48.910786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c59cceb-40c1-4b59-b128-c742ba9d44f3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:48.912798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c59cceb-40c1-4b59-b128-c742ba9d44f3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:48.915534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c59cceb-40c1-4b59-b128-c742ba9d44f3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:48.955380 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9c59cceb-40c1-4b59-b128-c742ba9d44f3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/ips" status: 200 len: 151 microversion: 2.20 time: 0.352215 Sep 27 10:24:48.965507 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2902/5371] 10.222.0.26 () {60 vars in 1289 bytes} [Tue Sep 27 10:24:48 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/ips => generated 151 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:24:48.979780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f1f40af9-5135-4d9a-a82c-c3b4273bc949 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:48.981498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f1f40af9-5135-4d9a-a82c-c3b4273bc949 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:48.990813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f40af9-5135-4d9a-a82c-c3b4273bc949 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:48.991453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f40af9-5135-4d9a-a82c-c3b4273bc949 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:48.992055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f40af9-5135-4d9a-a82c-c3b4273bc949 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:48.993601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f40af9-5135-4d9a-a82c-c3b4273bc949 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:48.993759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f40af9-5135-4d9a-a82c-c3b4273bc949 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:48.994287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1f40af9-5135-4d9a-a82c-c3b4273bc949 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:49.020360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:49.073644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c1429a16-50bd-44c2-b630-cb9ea09f3ed1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:49.096273 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c1429a16-50bd-44c2-b630-cb9ea09f3ed1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.722941 Sep 27 10:24:49.099119 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2470/5372] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:48 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 725 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:49.578666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:49.774738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f1f40af9-5135-4d9a-a82c-c3b4273bc949 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:49.786639 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f1f40af9-5135-4d9a-a82c-c3b4273bc949 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1319 microversion: 2.1 time: 0.818100 Sep 27 10:24:49.788221 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2903/5373] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:48 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1319 bytes in 822 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:50.123984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9ec3ca44-c7bc-4ac7-abb3-92cef7a1d802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:50.126488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9ec3ca44-c7bc-4ac7-abb3-92cef7a1d802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:50.136390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ec3ca44-c7bc-4ac7-abb3-92cef7a1d802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:50.137255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ec3ca44-c7bc-4ac7-abb3-92cef7a1d802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:50.137797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ec3ca44-c7bc-4ac7-abb3-92cef7a1d802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:50.151309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ec3ca44-c7bc-4ac7-abb3-92cef7a1d802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:50.152191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ec3ca44-c7bc-4ac7-abb3-92cef7a1d802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:50.152997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ec3ca44-c7bc-4ac7-abb3-92cef7a1d802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:50.375599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9ec3ca44-c7bc-4ac7-abb3-92cef7a1d802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:50.398939 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9ec3ca44-c7bc-4ac7-abb3-92cef7a1d802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.279245 Sep 27 10:24:50.404152 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2471/5374] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:50 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:50.811068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9206763f-0033-494e-9dc2-87ad618eb110 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:50.812884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9206763f-0033-494e-9dc2-87ad618eb110 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:50.816231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:50.822764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9206763f-0033-494e-9dc2-87ad618eb110 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:50.823399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9206763f-0033-494e-9dc2-87ad618eb110 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:50.828159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9206763f-0033-494e-9dc2-87ad618eb110 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:50.829452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9206763f-0033-494e-9dc2-87ad618eb110 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:50.830009 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9206763f-0033-494e-9dc2-87ad618eb110 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:50.830515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9206763f-0033-494e-9dc2-87ad618eb110 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:51.063354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9206763f-0033-494e-9dc2-87ad618eb110 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:51.081156 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9206763f-0033-494e-9dc2-87ad618eb110 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1319 microversion: 2.1 time: 0.272964 Sep 27 10:24:51.095422 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2904/5375] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:50 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1319 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:51.307819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5570a3f3-ed72-40ed-8a10-03aa8f95e5ea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:51.308439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5570a3f3-ed72-40ed-8a10-03aa8f95e5ea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:51.315482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5570a3f3-ed72-40ed-8a10-03aa8f95e5ea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:51.316257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5570a3f3-ed72-40ed-8a10-03aa8f95e5ea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:51.317108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5570a3f3-ed72-40ed-8a10-03aa8f95e5ea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:51.319211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5570a3f3-ed72-40ed-8a10-03aa8f95e5ea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:51.319830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5570a3f3-ed72-40ed-8a10-03aa8f95e5ea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:51.320462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5570a3f3-ed72-40ed-8a10-03aa8f95e5ea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:51.351708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-5570a3f3-ed72-40ed-8a10-03aa8f95e5ea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lazy-loading 'flavor' on Instance uuid 77037d39-844c-4761-88cc-899710443401 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:24:51.359846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-01bd6d10-1117-4b3a-b4c0-5c4f99fede85 req-e34ab5ec-81a4-4abc-b6aa-6d1f9d9651d9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ac45a77-b872-4aab-93f2-25556deb0d3f", "name": "network-vif-plugged", "status": "completed", "tag": "52b80ca4-891c-420a-a226-c0f942315728"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:51.376528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-01bd6d10-1117-4b3a-b4c0-5c4f99fede85 req-e34ab5ec-81a4-4abc-b6aa-6d1f9d9651d9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:51.377480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-01bd6d10-1117-4b3a-b4c0-5c4f99fede85 req-e34ab5ec-81a4-4abc-b6aa-6d1f9d9651d9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:51.378776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-01bd6d10-1117-4b3a-b4c0-5c4f99fede85 req-e34ab5ec-81a4-4abc-b6aa-6d1f9d9651d9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:51.398880 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-01bd6d10-1117-4b3a-b4c0-5c4f99fede85 req-e34ab5ec-81a4-4abc-b6aa-6d1f9d9651d9 service nova] Creating event network-vif-plugged:52b80ca4-891c-420a-a226-c0f942315728 for instance 7ac45a77-b872-4aab-93f2-25556deb0d3f on nch1-master-1 Sep 27 10:24:51.405850 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-01bd6d10-1117-4b3a-b4c0-5c4f99fede85 req-e34ab5ec-81a4-4abc-b6aa-6d1f9d9651d9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052529 Sep 27 10:24:51.408727 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2905/5376] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:24:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:24:51.455783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4cd1b2a9-c44e-407d-82ef-9d409ebff01a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:51.461181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4cd1b2a9-c44e-407d-82ef-9d409ebff01a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:51.469665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=88,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:51.479789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd1b2a9-c44e-407d-82ef-9d409ebff01a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:51.480299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd1b2a9-c44e-407d-82ef-9d409ebff01a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:51.480806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd1b2a9-c44e-407d-82ef-9d409ebff01a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:51.481620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd1b2a9-c44e-407d-82ef-9d409ebff01a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:51.488273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd1b2a9-c44e-407d-82ef-9d409ebff01a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:51.489481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4cd1b2a9-c44e-407d-82ef-9d409ebff01a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:51.814877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4cd1b2a9-c44e-407d-82ef-9d409ebff01a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:51.830808 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4cd1b2a9-c44e-407d-82ef-9d409ebff01a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.390863 Sep 27 10:24:51.831410 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2906/5377] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:51 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:51.866668 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5570a3f3-ed72-40ed-8a10-03aa8f95e5ea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "POST /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/action" status: 202 len: 0 microversion: 2.20 time: 0.562454 Sep 27 10:24:51.869469 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2472/5378] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:24:51 2022] POST /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/action => generated 0 bytes in 565 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:24:51.877382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:51.878798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:51.887836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:51.888878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:51.889509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:51.890481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:51.891279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:51.891804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:52.069988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:52.079241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:52.080885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:52.082357 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f8e07fca-ac13-40a3-b047-84323658cf27 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1722 microversion: 2.20 time: 0.209867 Sep 27 10:24:52.083523 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2907/5379] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:24:51 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1722 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:24:52.122256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a64bc5b9-cb6b-465d-9c49-27aa03ffd011 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:52.123193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a64bc5b9-cb6b-465d-9c49-27aa03ffd011 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:52.133683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a64bc5b9-cb6b-465d-9c49-27aa03ffd011 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:52.134183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a64bc5b9-cb6b-465d-9c49-27aa03ffd011 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:52.146580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a64bc5b9-cb6b-465d-9c49-27aa03ffd011 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:52.147683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a64bc5b9-cb6b-465d-9c49-27aa03ffd011 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:52.148176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a64bc5b9-cb6b-465d-9c49-27aa03ffd011 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:52.148614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a64bc5b9-cb6b-465d-9c49-27aa03ffd011 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:52.535275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a64bc5b9-cb6b-465d-9c49-27aa03ffd011 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:52.544465 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a64bc5b9-cb6b-465d-9c49-27aa03ffd011 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1319 microversion: 2.1 time: 0.427953 Sep 27 10:24:52.547317 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2473/5380] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:52 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1319 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:52.858337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e0eb332-c3c9-4bef-9423-b1b454e0b20c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:52.859228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8e0eb332-c3c9-4bef-9423-b1b454e0b20c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:52.893931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e0eb332-c3c9-4bef-9423-b1b454e0b20c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:52.895643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e0eb332-c3c9-4bef-9423-b1b454e0b20c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:52.896179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e0eb332-c3c9-4bef-9423-b1b454e0b20c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:52.898850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e0eb332-c3c9-4bef-9423-b1b454e0b20c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:52.899640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e0eb332-c3c9-4bef-9423-b1b454e0b20c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:52.901494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e0eb332-c3c9-4bef-9423-b1b454e0b20c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:53.110436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:53.111376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:53.118791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:53.119298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:53.119739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:53.128213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:53.128839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:53.129459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:53.185043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8e0eb332-c3c9-4bef-9423-b1b454e0b20c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:53.193872 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e0eb332-c3c9-4bef-9423-b1b454e0b20c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.340361 Sep 27 10:24:53.194716 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2908/5381] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:52 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:53.389480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:53.398924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:53.400765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:53.402453 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2177ec62-ecf9-4052-9fcb-2b3474e7dc97 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1722 microversion: 2.20 time: 0.296269 Sep 27 10:24:53.403036 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2474/5382] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:24:53 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1722 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:24:53.571613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67c70b08-3792-46ee-8085-16e50ac8934e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:53.573026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67c70b08-3792-46ee-8085-16e50ac8934e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:53.580360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c70b08-3792-46ee-8085-16e50ac8934e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:53.580886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c70b08-3792-46ee-8085-16e50ac8934e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:53.582212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c70b08-3792-46ee-8085-16e50ac8934e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:53.583281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c70b08-3792-46ee-8085-16e50ac8934e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:53.584325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c70b08-3792-46ee-8085-16e50ac8934e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:53.585359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67c70b08-3792-46ee-8085-16e50ac8934e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:53.752282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-67c70b08-3792-46ee-8085-16e50ac8934e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:53.767152 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67c70b08-3792-46ee-8085-16e50ac8934e tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1507 microversion: 2.1 time: 0.202839 Sep 27 10:24:53.768071 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2909/5383] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:53 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1507 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:54.230036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-704932c3-7273-4a5d-9d92-4186577a786e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:54.231188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-704932c3-7273-4a5d-9d92-4186577a786e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:54.248889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704932c3-7273-4a5d-9d92-4186577a786e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:54.249675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704932c3-7273-4a5d-9d92-4186577a786e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:54.250331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704932c3-7273-4a5d-9d92-4186577a786e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:54.251275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704932c3-7273-4a5d-9d92-4186577a786e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:54.252041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704932c3-7273-4a5d-9d92-4186577a786e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:54.252706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-704932c3-7273-4a5d-9d92-4186577a786e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:54.277334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:24:54.437080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:54.437709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:54.445484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:54.446172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:54.446805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:54.447523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:54.448136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:54.448764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:54.483541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-704932c3-7273-4a5d-9d92-4186577a786e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:54.495500 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-704932c3-7273-4a5d-9d92-4186577a786e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1512 microversion: 2.1 time: 0.271481 Sep 27 10:24:54.496559 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2475/5384] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:54 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1512 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:54.861681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:54.871244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:54.874187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:54.876085 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4305c470-e544-431e-8028-27e33125d140 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1722 microversion: 2.20 time: 0.451015 Sep 27 10:24:54.877418 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2910/5385] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:24:54 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1722 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:24:55.398908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-volume-backed-server-1616516968", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"uuid": "c2bfd580-e62a-4f33-a8ea-3e0437752d94", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}], "networks": [{"uuid": "1e417b59-62bb-4007-a78a-c15c153efd33"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:55.519322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af76e04d-119d-4e5d-9165-3f75fae59e3c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:55.520770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af76e04d-119d-4e5d-9165-3f75fae59e3c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:55.542756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af76e04d-119d-4e5d-9165-3f75fae59e3c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:55.544320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af76e04d-119d-4e5d-9165-3f75fae59e3c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:55.547490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af76e04d-119d-4e5d-9165-3f75fae59e3c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:55.548833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af76e04d-119d-4e5d-9165-3f75fae59e3c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:55.552941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] validate_networks() for [('1e417b59-62bb-4007-a78a-c15c153efd33', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:24:55.561852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af76e04d-119d-4e5d-9165-3f75fae59e3c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:55.562550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af76e04d-119d-4e5d-9165-3f75fae59e3c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:55.779787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af76e04d-119d-4e5d-9165-3f75fae59e3c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:55.791610 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af76e04d-119d-4e5d-9165-3f75fae59e3c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1489 microversion: 2.1 time: 0.276777 Sep 27 10:24:55.792327 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2911/5386] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:55 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1489 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:55.822898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6b8b0973-cfc5-454c-8f7e-96057316b1b7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {"length": 3}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:55.826599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6b8b0973-cfc5-454c-8f7e-96057316b1b7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:55.839358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b8b0973-cfc5-454c-8f7e-96057316b1b7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:55.840231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b8b0973-cfc5-454c-8f7e-96057316b1b7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:55.841345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b8b0973-cfc5-454c-8f7e-96057316b1b7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:55.842339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b8b0973-cfc5-454c-8f7e-96057316b1b7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:55.843543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b8b0973-cfc5-454c-8f7e-96057316b1b7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:55.844199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b8b0973-cfc5-454c-8f7e-96057316b1b7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:55.918614 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6b8b0973-cfc5-454c-8f7e-96057316b1b7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 200 len: 122 microversion: 2.1 time: 0.099584 Sep 27 10:24:55.926111 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2912/5387] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:24:55 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 122 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:24:55.931151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:55.932259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:55.941266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:55.942297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:55.942938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:55.944529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:55.945875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:55.946557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:56.211681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:56.223307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:56.226004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:56.228864 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e471e949-89c7-4d95-8223-b5173fce2035 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1722 microversion: 2.20 time: 0.304120 Sep 27 10:24:56.229716 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2913/5388] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:24:55 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1722 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:24:56.254066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0704f4e7-4f9d-4a78-90b7-bd1143fb8796 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:56.254949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0704f4e7-4f9d-4a78-90b7-bd1143fb8796 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:56.272371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0704f4e7-4f9d-4a78-90b7-bd1143fb8796 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:56.274403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0704f4e7-4f9d-4a78-90b7-bd1143fb8796 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:56.275046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0704f4e7-4f9d-4a78-90b7-bd1143fb8796 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:56.276075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0704f4e7-4f9d-4a78-90b7-bd1143fb8796 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:56.276866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0704f4e7-4f9d-4a78-90b7-bd1143fb8796 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:56.277502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0704f4e7-4f9d-4a78-90b7-bd1143fb8796 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:56.518465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0704f4e7-4f9d-4a78-90b7-bd1143fb8796 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:56.528936 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0704f4e7-4f9d-4a78-90b7-bd1143fb8796 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.297549 Sep 27 10:24:56.529561 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2914/5389] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:56 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:56.559159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ff0ff6ff-f4fc-41fe-900b-c20689bfafe6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:56.560142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ff0ff6ff-f4fc-41fe-900b-c20689bfafe6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:56.577729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff0ff6ff-f4fc-41fe-900b-c20689bfafe6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:56.578246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff0ff6ff-f4fc-41fe-900b-c20689bfafe6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:56.578686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff0ff6ff-f4fc-41fe-900b-c20689bfafe6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:56.579448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff0ff6ff-f4fc-41fe-900b-c20689bfafe6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:56.579906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff0ff6ff-f4fc-41fe-900b-c20689bfafe6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:56.580333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff0ff6ff-f4fc-41fe-900b-c20689bfafe6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:56.750202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:24:56.750733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:24:56.751106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:24:56.751521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:24:56.751961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:24:56.762034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:56.762543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:56.762976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:56.763758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:56.764333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:56.764640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:56.786934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:24:56.792988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for user 178956f6c8154a06a0710b295aff3086 and project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:24:56.849551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ff0ff6ff-f4fc-41fe-900b-c20689bfafe6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:56.858413 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ff0ff6ff-f4fc-41fe-900b-c20689bfafe6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.303998 Sep 27 10:24:56.859302 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2915/5390] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:56 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:24:56.865224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bb6d12db-088d-42d0-8688-6ee42f079aa2 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "1962f1c2-cd1b-44c0-ab12-4f391cd8acf5", "device": "/dev/sdb"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:56.868157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bb6d12db-088d-42d0-8688-6ee42f079aa2 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:56.875358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6d12db-088d-42d0-8688-6ee42f079aa2 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:56.876138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6d12db-088d-42d0-8688-6ee42f079aa2 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:56.876754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6d12db-088d-42d0-8688-6ee42f079aa2 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:56.877796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6d12db-088d-42d0-8688-6ee42f079aa2 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:56.878478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6d12db-088d-42d0-8688-6ee42f079aa2 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:56.879415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bb6d12db-088d-42d0-8688-6ee42f079aa2 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:56.934337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:24:56.936499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c2bfd580-e62a-4f33-a8ea-3e0437752d94',volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:24:57.195002 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c18a1228-907a-40f3-bdc0-e9242c388ae2 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.801301 Sep 27 10:24:57.199164 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2476/5391] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:24:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 1806 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:24:57.212486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-efa09e76-ae8f-463f-b327-689627a37172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:57.215437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-efa09e76-ae8f-463f-b327-689627a37172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:57.230947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efa09e76-ae8f-463f-b327-689627a37172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:57.231458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efa09e76-ae8f-463f-b327-689627a37172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:57.232819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efa09e76-ae8f-463f-b327-689627a37172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:57.233957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efa09e76-ae8f-463f-b327-689627a37172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:57.234433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efa09e76-ae8f-463f-b327-689627a37172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:57.235164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-efa09e76-ae8f-463f-b327-689627a37172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:57.295126 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bb6d12db-088d-42d0-8688-6ee42f079aa2 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "POST /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.433488 Sep 27 10:24:57.296682 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2916/5392] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:24:56 2022] POST /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f/os-volume_attachments => generated 194 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:24:57.311024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d07bbcc2-7b91-47e5-8014-e86c1eed5264 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:57.314252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d07bbcc2-7b91-47e5-8014-e86c1eed5264 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:57.451197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d07bbcc2-7b91-47e5-8014-e86c1eed5264 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:57.486760 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d07bbcc2-7b91-47e5-8014-e86c1eed5264 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1071 microversion: 2.1 time: 0.184438 Sep 27 10:24:57.497825 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2917/5393] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:57 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1071 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:57.519992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:57.523887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:57.566143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:57.566955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:57.567792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:57.568866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:57.569541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:57.570354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:57.621004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-efa09e76-ae8f-463f-b327-689627a37172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:57.641247 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-efa09e76-ae8f-463f-b327-689627a37172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.440775 Sep 27 10:24:57.642729 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2477/5394] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:57 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:57.680219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0d15085d-30c5-4217-9895-8297f01f0225 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:57.682041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0d15085d-30c5-4217-9895-8297f01f0225 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:57.694800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d15085d-30c5-4217-9895-8297f01f0225 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:57.700853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d15085d-30c5-4217-9895-8297f01f0225 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:57.701769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d15085d-30c5-4217-9895-8297f01f0225 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:57.702921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d15085d-30c5-4217-9895-8297f01f0225 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:57.704639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d15085d-30c5-4217-9895-8297f01f0225 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:57.705385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d15085d-30c5-4217-9895-8297f01f0225 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:57.949725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:57.966076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:57.971450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:57.980521 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f1855907-f873-4b15-8e87-436ab06c8a01 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1722 microversion: 2.20 time: 0.478618 Sep 27 10:24:57.981906 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2918/5395] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:24:57 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1722 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:24:58.224793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0d15085d-30c5-4217-9895-8297f01f0225 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:58.235210 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0d15085d-30c5-4217-9895-8297f01f0225 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.570832 Sep 27 10:24:58.264571 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2478/5396] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:57 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 601 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:24:58.412851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-328509946", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "247e3d7d-5912-42d7-9027-ebcb26003cfc"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:24:58.500838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] validate_networks() for [('247e3d7d-5912-42d7-9027-ebcb26003cfc', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:24:58.523099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4dd7446c-ca52-45fb-9a6f-7028f178221a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:58.525490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4dd7446c-ca52-45fb-9a6f-7028f178221a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:58.546898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd7446c-ca52-45fb-9a6f-7028f178221a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:58.550094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd7446c-ca52-45fb-9a6f-7028f178221a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:58.552365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd7446c-ca52-45fb-9a6f-7028f178221a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:58.554009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd7446c-ca52-45fb-9a6f-7028f178221a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:58.558805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd7446c-ca52-45fb-9a6f-7028f178221a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:58.559463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dd7446c-ca52-45fb-9a6f-7028f178221a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:58.725842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4dd7446c-ca52-45fb-9a6f-7028f178221a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:58.735276 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4dd7446c-ca52-45fb-9a6f-7028f178221a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1169 microversion: 2.1 time: 0.217840 Sep 27 10:24:58.736361 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2479/5397] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:58 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1169 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:24:59.011171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:59.018964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:59.040857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:59.049056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:59.052200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:59.053152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:59.053718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:59.055027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:59.319822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:24:59.320994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:24:59.321701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:24:59.322299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:24:59.322896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:24:59.344212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:59.347650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:59.348522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:59.350734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:59.354004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:59.354587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:59.403155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for project 10375bd836f84383b39f2df73f53800e. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:24:59.409127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for user 97c769dbba794531b0a9db8e764ff097 and project 10375bd836f84383b39f2df73f53800e. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:24:59.431676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:24:59.434631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:24:59.510436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:59.538911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:59.564426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:59.571591 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e39c877a-6945-4412-a651-60f6bcee3b71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1722 microversion: 2.20 time: 0.563709 Sep 27 10:24:59.572214 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2480/5398] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:24:59 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1722 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:24:59.600278 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e5a167c6-1811-4f6f-98d7-0b9eb40e496e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.191847 Sep 27 10:24:59.623829 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2919/5399] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:24:58 2022] POST /compute/v2.1/servers => generated 380 bytes in 1215 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:24:59.635044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a8bb5308-7061-41d2-ba01-0db7c98cebe0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:59.635689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a8bb5308-7061-41d2-ba01-0db7c98cebe0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:59.777397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-584f4064-7354-4b95-90c8-09ecb979d07d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:24:59.778129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-584f4064-7354-4b95-90c8-09ecb979d07d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:24:59.792331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-584f4064-7354-4b95-90c8-09ecb979d07d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:59.792935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-584f4064-7354-4b95-90c8-09ecb979d07d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:59.793388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-584f4064-7354-4b95-90c8-09ecb979d07d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:59.794125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-584f4064-7354-4b95-90c8-09ecb979d07d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:24:59.794586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-584f4064-7354-4b95-90c8-09ecb979d07d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:24:59.795016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-584f4064-7354-4b95-90c8-09ecb979d07d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:24:59.846524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a8bb5308-7061-41d2-ba01-0db7c98cebe0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:59.855662 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a8bb5308-7061-41d2-ba01-0db7c98cebe0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1214 microversion: 2.1 time: 0.229465 Sep 27 10:24:59.872173 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2481/5400] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:59 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1214 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:24:59.926921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-584f4064-7354-4b95-90c8-09ecb979d07d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:24:59.940899 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-584f4064-7354-4b95-90c8-09ecb979d07d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1187 microversion: 2.1 time: 0.167417 Sep 27 10:24:59.943496 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2920/5401] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:24:59 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1187 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:00.605123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:00.614684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:00.640404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:00.643855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:00.647642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:00.651428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:00.652263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:00.652940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:00.886055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-12183f65-a56e-4a59-b9da-9f3ffddfa518 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:00.886790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12183f65-a56e-4a59-b9da-9f3ffddfa518 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:00.896519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12183f65-a56e-4a59-b9da-9f3ffddfa518 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:00.897347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12183f65-a56e-4a59-b9da-9f3ffddfa518 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:00.898171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12183f65-a56e-4a59-b9da-9f3ffddfa518 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:00.899244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12183f65-a56e-4a59-b9da-9f3ffddfa518 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:00.899929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12183f65-a56e-4a59-b9da-9f3ffddfa518 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:00.901095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12183f65-a56e-4a59-b9da-9f3ffddfa518 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:00.924026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:01.012151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12183f65-a56e-4a59-b9da-9f3ffddfa518 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:01.040867 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-12183f65-a56e-4a59-b9da-9f3ffddfa518 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1266 microversion: 2.1 time: 0.159535 Sep 27 10:25:01.043186 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2921/5402] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:00 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1266 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:01.050274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33e55f7f-3de9-4edf-b7d4-47910a729cfa tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:01.050894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33e55f7f-3de9-4edf-b7d4-47910a729cfa tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:01.064394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33e55f7f-3de9-4edf-b7d4-47910a729cfa tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:01.064933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33e55f7f-3de9-4edf-b7d4-47910a729cfa tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:01.065383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33e55f7f-3de9-4edf-b7d4-47910a729cfa tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:01.066107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33e55f7f-3de9-4edf-b7d4-47910a729cfa tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:01.066814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33e55f7f-3de9-4edf-b7d4-47910a729cfa tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:01.067475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33e55f7f-3de9-4edf-b7d4-47910a729cfa tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:01.198418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33e55f7f-3de9-4edf-b7d4-47910a729cfa tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:01.209245 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33e55f7f-3de9-4edf-b7d4-47910a729cfa tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1187 microversion: 2.1 time: 0.164480 Sep 27 10:25:01.210413 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2922/5403] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:01 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1187 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:01.511679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:01.523915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:01.528477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:01.530834 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-88566ed3-3184-468b-b175-6e348f65e24d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1722 microversion: 2.20 time: 0.932887 Sep 27 10:25:01.537574 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2482/5404] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:00 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1722 bytes in 940 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:02.075745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cac0d482-467b-48e7-b50b-52ad0e9ca91d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:02.077032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cac0d482-467b-48e7-b50b-52ad0e9ca91d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:02.098601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cac0d482-467b-48e7-b50b-52ad0e9ca91d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:02.099247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cac0d482-467b-48e7-b50b-52ad0e9ca91d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:02.102866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cac0d482-467b-48e7-b50b-52ad0e9ca91d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:02.103937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cac0d482-467b-48e7-b50b-52ad0e9ca91d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:02.109169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cac0d482-467b-48e7-b50b-52ad0e9ca91d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:02.109641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cac0d482-467b-48e7-b50b-52ad0e9ca91d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:02.237411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-767f1be4-56b8-42c8-9f76-0a4922da0862 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:02.238233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-767f1be4-56b8-42c8-9f76-0a4922da0862 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:02.252387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-767f1be4-56b8-42c8-9f76-0a4922da0862 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:02.253768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-767f1be4-56b8-42c8-9f76-0a4922da0862 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:02.254859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-767f1be4-56b8-42c8-9f76-0a4922da0862 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:02.256251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-767f1be4-56b8-42c8-9f76-0a4922da0862 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:02.257026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-767f1be4-56b8-42c8-9f76-0a4922da0862 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:02.257819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-767f1be4-56b8-42c8-9f76-0a4922da0862 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:02.385912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cac0d482-467b-48e7-b50b-52ad0e9ca91d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:02.397980 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cac0d482-467b-48e7-b50b-52ad0e9ca91d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1272 microversion: 2.1 time: 0.332056 Sep 27 10:25:02.398816 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2923/5405] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:02 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1272 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:02.567408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:02.568774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:02.594785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:02.597153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:02.598108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:02.601741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:02.602714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:02.606733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:02.710966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-767f1be4-56b8-42c8-9f76-0a4922da0862 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:02.720079 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-767f1be4-56b8-42c8-9f76-0a4922da0862 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.489685 Sep 27 10:25:02.724937 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2483/5406] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:02 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:03.275107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:03.284102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:03.286742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:03.290146 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef4fb34c-2be4-47d0-bfd8-57b1b798ec17 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1722 microversion: 2.20 time: 0.727635 Sep 27 10:25:03.291431 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2924/5407] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:02 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1722 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:25:03.425308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d5136e4c-6c9a-40ea-8c8e-ecc0892b981f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:03.427315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d5136e4c-6c9a-40ea-8c8e-ecc0892b981f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:03.438226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5136e4c-6c9a-40ea-8c8e-ecc0892b981f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:03.438986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5136e4c-6c9a-40ea-8c8e-ecc0892b981f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:03.439581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5136e4c-6c9a-40ea-8c8e-ecc0892b981f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:03.440729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5136e4c-6c9a-40ea-8c8e-ecc0892b981f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:03.441525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5136e4c-6c9a-40ea-8c8e-ecc0892b981f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:03.448291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5136e4c-6c9a-40ea-8c8e-ecc0892b981f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:03.744061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1a479a84-e09e-4118-9662-318f404bc8ec tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:03.744635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1a479a84-e09e-4118-9662-318f404bc8ec tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:03.764708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a479a84-e09e-4118-9662-318f404bc8ec tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:03.765138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a479a84-e09e-4118-9662-318f404bc8ec tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:03.765576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a479a84-e09e-4118-9662-318f404bc8ec tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:03.766662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a479a84-e09e-4118-9662-318f404bc8ec tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:03.767656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a479a84-e09e-4118-9662-318f404bc8ec tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:03.768227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a479a84-e09e-4118-9662-318f404bc8ec tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:03.786289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d5136e4c-6c9a-40ea-8c8e-ecc0892b981f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:03.798191 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d5136e4c-6c9a-40ea-8c8e-ecc0892b981f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1314 microversion: 2.1 time: 0.379424 Sep 27 10:25:03.810963 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2484/5408] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:03 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1314 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:04.118920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesBackupsTest-instance-1569918355", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:04.235261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:25:04.291350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1a479a84-e09e-4118-9662-318f404bc8ec tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:04.300889 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1a479a84-e09e-4118-9662-318f404bc8ec tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.560902 Sep 27 10:25:04.312313 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2925/5409] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:03 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:04.327826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:04.330394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:04.346069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:04.346578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:04.347023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:04.347751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:04.348319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:04.348762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:04.683927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:04.695358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:04.697203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:04.698991 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c1bb8ba-5641-4da2-9b98-3362a0c274dc tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1743 microversion: 2.20 time: 0.384350 Sep 27 10:25:04.700293 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2926/5410] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:04 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1743 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:04.825303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5e0f6f3-7975-4b3f-ace3-d5c9c8d84560 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:04.825932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a5e0f6f3-7975-4b3f-ace3-d5c9c8d84560 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:04.840430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e0f6f3-7975-4b3f-ace3-d5c9c8d84560 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:04.841040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e0f6f3-7975-4b3f-ace3-d5c9c8d84560 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:04.841488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e0f6f3-7975-4b3f-ace3-d5c9c8d84560 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:04.842220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e0f6f3-7975-4b3f-ace3-d5c9c8d84560 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:04.842688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e0f6f3-7975-4b3f-ace3-d5c9c8d84560 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:04.843119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5e0f6f3-7975-4b3f-ace3-d5c9c8d84560 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:05.174514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a5e0f6f3-7975-4b3f-ace3-d5c9c8d84560 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:05.191871 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a5e0f6f3-7975-4b3f-ace3-d5c9c8d84560 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1314 microversion: 2.1 time: 0.370071 Sep 27 10:25:05.197816 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2927/5411] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:04 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1314 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:05.205690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e3af1565-25a1-4177-87c1-eb38de2f36e0 req-b701889e-1a05-4ef2-b647-d2c534df6ad4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc", "tag": "ad654d98-8103-4e52-a2b9-095d297e78f7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:05.220877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e3af1565-25a1-4177-87c1-eb38de2f36e0 req-b701889e-1a05-4ef2-b647-d2c534df6ad4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:05.221370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e3af1565-25a1-4177-87c1-eb38de2f36e0 req-b701889e-1a05-4ef2-b647-d2c534df6ad4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:05.221934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e3af1565-25a1-4177-87c1-eb38de2f36e0 req-b701889e-1a05-4ef2-b647-d2c534df6ad4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:05.235987 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e3af1565-25a1-4177-87c1-eb38de2f36e0 req-b701889e-1a05-4ef2-b647-d2c534df6ad4 service nova] Creating event network-changed:ad654d98-8103-4e52-a2b9-095d297e78f7 for instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc on nch2-master-1 Sep 27 10:25:05.241441 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e3af1565-25a1-4177-87c1-eb38de2f36e0 req-b701889e-1a05-4ef2-b647-d2c534df6ad4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042301 Sep 27 10:25:05.242571 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2928/5412] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:05.256139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f02bbed0-7ca9-44cc-87b6-e9b96cdb4db4 req-042967a6-0b26-496e-a9b3-ca410229b028 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "43bd3356-0fc6-4589-8247-b3394edbaa99", "tag": "4e47c3e5-afc9-4f0c-abc1-59a3d3f0b51c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:05.273430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f02bbed0-7ca9-44cc-87b6-e9b96cdb4db4 req-042967a6-0b26-496e-a9b3-ca410229b028 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:05.274128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f02bbed0-7ca9-44cc-87b6-e9b96cdb4db4 req-042967a6-0b26-496e-a9b3-ca410229b028 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:05.274768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f02bbed0-7ca9-44cc-87b6-e9b96cdb4db4 req-042967a6-0b26-496e-a9b3-ca410229b028 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:05.289356 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f02bbed0-7ca9-44cc-87b6-e9b96cdb4db4 req-042967a6-0b26-496e-a9b3-ca410229b028 service nova] Creating event network-changed:4e47c3e5-afc9-4f0c-abc1-59a3d3f0b51c for instance 43bd3356-0fc6-4589-8247-b3394edbaa99 on nch1-master-1 Sep 27 10:25:05.300706 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f02bbed0-7ca9-44cc-87b6-e9b96cdb4db4 req-042967a6-0b26-496e-a9b3-ca410229b028 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056394 Sep 27 10:25:05.302271 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2929/5413] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:05.340135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4553af2-b22f-4088-a0f7-22a8915a638f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:05.340921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4553af2-b22f-4088-a0f7-22a8915a638f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:05.348261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4553af2-b22f-4088-a0f7-22a8915a638f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:05.348910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4553af2-b22f-4088-a0f7-22a8915a638f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:05.349566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4553af2-b22f-4088-a0f7-22a8915a638f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:05.350289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4553af2-b22f-4088-a0f7-22a8915a638f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:05.350743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4553af2-b22f-4088-a0f7-22a8915a638f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:05.351343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4553af2-b22f-4088-a0f7-22a8915a638f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:05.439239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:25:05.440385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:25:05.440911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:25:05.441316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:25:05.442434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:25:05.464768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:05.465263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:05.465696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:05.466969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:05.467431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:05.467858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:05.490473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Getting quotas for project 430555edbc87497db7d7f79f2b891c04. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:25:05.498099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Getting quotas for user fbcfed06aa6b4177a4287064884125b2 and project 430555edbc87497db7d7f79f2b891c04. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:25:05.513786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:25:05.516581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:25:05.576467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4553af2-b22f-4088-a0f7-22a8915a638f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:05.585808 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3ddcb95f-67ef-4012-8ed6-ab7654f8cf44 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.474581 Sep 27 10:25:05.594243 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2485/5414] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:25:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 1485 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:25:05.603712 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4553af2-b22f-4088-a0f7-22a8915a638f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.269024 Sep 27 10:25:05.619541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-01139c9c-acb7-4bd5-a3b5-df5de55a850e tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:05.620402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-01139c9c-acb7-4bd5-a3b5-df5de55a850e tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:05.624853 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2930/5415] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:05 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:05.728546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:05.729957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:05.748542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:05.749350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:05.750532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:05.751476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:05.753192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:05.753909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:05.768539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-01139c9c-acb7-4bd5-a3b5-df5de55a850e tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:05.793434 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-01139c9c-acb7-4bd5-a3b5-df5de55a850e tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1214 microversion: 2.1 time: 0.195346 Sep 27 10:25:05.794088 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2486/5416] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:05 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1214 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:06.070896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:06.080362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:06.084150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:06.090789 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d6db0ef7-3c99-4eef-bbb0-1d1ed3de55a7 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1743 microversion: 2.20 time: 0.366328 Sep 27 10:25:06.091642 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2931/5417] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:05 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1743 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:06.216631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-de419227-87e6-477f-8798-d3b70254d173 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:06.217328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-de419227-87e6-477f-8798-d3b70254d173 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:06.231547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de419227-87e6-477f-8798-d3b70254d173 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:06.232376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de419227-87e6-477f-8798-d3b70254d173 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:06.233214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de419227-87e6-477f-8798-d3b70254d173 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:06.234179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de419227-87e6-477f-8798-d3b70254d173 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:06.234802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de419227-87e6-477f-8798-d3b70254d173 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:06.236619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de419227-87e6-477f-8798-d3b70254d173 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:06.341846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-db85d801-dc55-44f0-ba8a-92f7eac3a56a req-f6cec50c-4235-45e9-ba37-a5646a8342e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77037d39-844c-4761-88cc-899710443401", "name": "network-vif-plugged", "status": "completed", "tag": "6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:06.371281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-db85d801-dc55-44f0-ba8a-92f7eac3a56a req-f6cec50c-4235-45e9-ba37-a5646a8342e7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:06.372712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-db85d801-dc55-44f0-ba8a-92f7eac3a56a req-f6cec50c-4235-45e9-ba37-a5646a8342e7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:06.373685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-db85d801-dc55-44f0-ba8a-92f7eac3a56a req-f6cec50c-4235-45e9-ba37-a5646a8342e7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:06.401692 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-db85d801-dc55-44f0-ba8a-92f7eac3a56a req-f6cec50c-4235-45e9-ba37-a5646a8342e7 service nova] Creating event network-vif-plugged:6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1 for instance 77037d39-844c-4761-88cc-899710443401 on nch2-master-1 Sep 27 10:25:06.414823 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-db85d801-dc55-44f0-ba8a-92f7eac3a56a req-f6cec50c-4235-45e9-ba37-a5646a8342e7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077043 Sep 27 10:25:06.415521 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2932/5418] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:06.503878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-de419227-87e6-477f-8798-d3b70254d173 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:06.513046 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-de419227-87e6-477f-8798-d3b70254d173 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1314 microversion: 2.1 time: 0.301611 Sep 27 10:25:06.513747 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2487/5419] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:06 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1314 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:06.634710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb7299c8-5b7e-4809-a68b-21251e9fa208 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:06.635577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cb7299c8-5b7e-4809-a68b-21251e9fa208 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:06.642143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb7299c8-5b7e-4809-a68b-21251e9fa208 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:06.642640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb7299c8-5b7e-4809-a68b-21251e9fa208 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:06.643077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb7299c8-5b7e-4809-a68b-21251e9fa208 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:06.643888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb7299c8-5b7e-4809-a68b-21251e9fa208 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:06.644341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb7299c8-5b7e-4809-a68b-21251e9fa208 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:06.644859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb7299c8-5b7e-4809-a68b-21251e9fa208 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:06.820195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-15d5cf0b-222a-4738-95cb-9776f1c6b63f tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:06.821155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-15d5cf0b-222a-4738-95cb-9776f1c6b63f tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:06.831848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15d5cf0b-222a-4738-95cb-9776f1c6b63f tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:06.833367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15d5cf0b-222a-4738-95cb-9776f1c6b63f tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:06.834065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15d5cf0b-222a-4738-95cb-9776f1c6b63f tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:06.835405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15d5cf0b-222a-4738-95cb-9776f1c6b63f tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:06.836128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15d5cf0b-222a-4738-95cb-9776f1c6b63f tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:06.836814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15d5cf0b-222a-4738-95cb-9776f1c6b63f tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:06.867047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb7299c8-5b7e-4809-a68b-21251e9fa208 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:06.877830 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cb7299c8-5b7e-4809-a68b-21251e9fa208 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.247312 Sep 27 10:25:06.884055 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2933/5420] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:06 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:06.924521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-15d5cf0b-222a-4738-95cb-9776f1c6b63f tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:06.936345 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-15d5cf0b-222a-4738-95cb-9776f1c6b63f tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1218 microversion: 2.1 time: 0.120608 Sep 27 10:25:06.938916 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2488/5421] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:06 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1218 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:07.128245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:07.133717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:07.149391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:07.161123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:07.161686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:07.162633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:07.163104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:07.163527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:07.452038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:07.466352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:07.467829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:07.470571 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-42865e30-f011-40e6-ae42-ab972ebc1a16 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1743 microversion: 2.20 time: 0.357239 Sep 27 10:25:07.471308 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2934/5422] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:07 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1743 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:07.536696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-37102cd4-ce1d-40dc-8dce-0132d5d6a4f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:07.537506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-37102cd4-ce1d-40dc-8dce-0132d5d6a4f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:07.558378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37102cd4-ce1d-40dc-8dce-0132d5d6a4f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:07.558875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37102cd4-ce1d-40dc-8dce-0132d5d6a4f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:07.559313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37102cd4-ce1d-40dc-8dce-0132d5d6a4f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:07.560252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37102cd4-ce1d-40dc-8dce-0132d5d6a4f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:07.560724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37102cd4-ce1d-40dc-8dce-0132d5d6a4f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:07.561308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-37102cd4-ce1d-40dc-8dce-0132d5d6a4f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:07.778508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-37102cd4-ce1d-40dc-8dce-0132d5d6a4f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:07.788029 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-37102cd4-ce1d-40dc-8dce-0132d5d6a4f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1314 microversion: 2.1 time: 0.256039 Sep 27 10:25:07.788984 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2489/5423] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:07 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1314 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:07.907526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9fb427dd-4eab-478f-b144-c7182bd69928 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:07.908176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9fb427dd-4eab-478f-b144-c7182bd69928 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:07.916969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fb427dd-4eab-478f-b144-c7182bd69928 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:07.917806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fb427dd-4eab-478f-b144-c7182bd69928 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:07.918266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fb427dd-4eab-478f-b144-c7182bd69928 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:07.919151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fb427dd-4eab-478f-b144-c7182bd69928 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:07.919668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fb427dd-4eab-478f-b144-c7182bd69928 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:07.920280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9fb427dd-4eab-478f-b144-c7182bd69928 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:07.968651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4da3850a-40b7-4007-bc99-f77130284cf4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:07.969500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4da3850a-40b7-4007-bc99-f77130284cf4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:07.971716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:07.983606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da3850a-40b7-4007-bc99-f77130284cf4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:07.984359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da3850a-40b7-4007-bc99-f77130284cf4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:07.985036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da3850a-40b7-4007-bc99-f77130284cf4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:07.985765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da3850a-40b7-4007-bc99-f77130284cf4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:07.986224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da3850a-40b7-4007-bc99-f77130284cf4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:07.986650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4da3850a-40b7-4007-bc99-f77130284cf4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:08.098530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4da3850a-40b7-4007-bc99-f77130284cf4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:08.111773 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4da3850a-40b7-4007-bc99-f77130284cf4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1218 microversion: 2.1 time: 0.154922 Sep 27 10:25:08.112386 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2490/5424] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:07 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1218 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:08.133755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9fb427dd-4eab-478f-b144-c7182bd69928 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:08.143185 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9fb427dd-4eab-478f-b144-c7182bd69928 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.239514 Sep 27 10:25:08.174355 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2935/5425] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:07 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:08.496791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:08.497913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:08.505230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:08.505911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:08.506357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:08.507088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:08.507702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:08.508134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:08.805152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:08.812324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c881448-22a3-46a3-bf1d-c41d23831ed4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:08.817595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2c881448-22a3-46a3-bf1d-c41d23831ed4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:08.829079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:08.830541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:08.834067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c881448-22a3-46a3-bf1d-c41d23831ed4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:08.834572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c881448-22a3-46a3-bf1d-c41d23831ed4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:08.835342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c881448-22a3-46a3-bf1d-c41d23831ed4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:08.836062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c881448-22a3-46a3-bf1d-c41d23831ed4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:08.836524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c881448-22a3-46a3-bf1d-c41d23831ed4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:08.837005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c881448-22a3-46a3-bf1d-c41d23831ed4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:08.840436 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d5c8232e-d25d-44d7-ad47-feee497ea93e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1743 microversion: 2.20 time: 0.339873 Sep 27 10:25:08.841172 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2491/5426] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:08 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1743 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:09.053276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2c881448-22a3-46a3-bf1d-c41d23831ed4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:09.063204 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c881448-22a3-46a3-bf1d-c41d23831ed4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1314 microversion: 2.1 time: 0.255590 Sep 27 10:25:09.063803 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2936/5427] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:08 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1314 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:09.142025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa088564-8410-47d0-8425-361a74153bbb tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:09.142599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aa088564-8410-47d0-8425-361a74153bbb tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:09.156020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa088564-8410-47d0-8425-361a74153bbb tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:09.156769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa088564-8410-47d0-8425-361a74153bbb tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:09.157371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa088564-8410-47d0-8425-361a74153bbb tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:09.158883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa088564-8410-47d0-8425-361a74153bbb tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:09.159519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa088564-8410-47d0-8425-361a74153bbb tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:09.160112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa088564-8410-47d0-8425-361a74153bbb tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:09.174256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-207c1321-28e2-4ea2-b857-62c7e9c49065 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:09.178681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-207c1321-28e2-4ea2-b857-62c7e9c49065 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:09.192444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-207c1321-28e2-4ea2-b857-62c7e9c49065 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:09.194549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-207c1321-28e2-4ea2-b857-62c7e9c49065 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:09.195269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-207c1321-28e2-4ea2-b857-62c7e9c49065 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:09.196286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-207c1321-28e2-4ea2-b857-62c7e9c49065 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:09.197532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-207c1321-28e2-4ea2-b857-62c7e9c49065 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:09.202936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-207c1321-28e2-4ea2-b857-62c7e9c49065 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:09.373005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:09.394369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aa088564-8410-47d0-8425-361a74153bbb tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:09.402635 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aa088564-8410-47d0-8425-361a74153bbb tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1274 microversion: 2.1 time: 0.270675 Sep 27 10:25:09.402898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-207c1321-28e2-4ea2-b857-62c7e9c49065 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:09.421196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:09.421609 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2492/5428] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:09 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1274 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:09.432149 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-207c1321-28e2-4ea2-b857-62c7e9c49065 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.263975 Sep 27 10:25:09.433002 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2937/5429] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:09 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:09.735073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-3c2d2fc0-4c70-4076-87ff-6ad8308a40c9 req-07d97c71-0491-40ff-aa11-1428a9ac6f31 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43bd3356-0fc6-4589-8247-b3394edbaa99", "name": "network-vif-plugged", "status": "completed", "tag": "4e47c3e5-afc9-4f0c-abc1-59a3d3f0b51c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:09.746447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3c2d2fc0-4c70-4076-87ff-6ad8308a40c9 req-07d97c71-0491-40ff-aa11-1428a9ac6f31 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:09.747005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3c2d2fc0-4c70-4076-87ff-6ad8308a40c9 req-07d97c71-0491-40ff-aa11-1428a9ac6f31 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:09.747484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3c2d2fc0-4c70-4076-87ff-6ad8308a40c9 req-07d97c71-0491-40ff-aa11-1428a9ac6f31 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:09.760430 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-3c2d2fc0-4c70-4076-87ff-6ad8308a40c9 req-07d97c71-0491-40ff-aa11-1428a9ac6f31 service nova] Creating event network-vif-plugged:4e47c3e5-afc9-4f0c-abc1-59a3d3f0b51c for instance 43bd3356-0fc6-4589-8247-b3394edbaa99 on nch1-master-1 Sep 27 10:25:09.767297 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-3c2d2fc0-4c70-4076-87ff-6ad8308a40c9 req-07d97c71-0491-40ff-aa11-1428a9ac6f31 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036425 Sep 27 10:25:09.768953 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2493/5430] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:09.868459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:09.869931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:09.879223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:09.880147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:09.880601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:09.881369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:09.882015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:09.882458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:10.087962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-733e701d-a256-4014-9235-efd39ef64943 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:10.089154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-733e701d-a256-4014-9235-efd39ef64943 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:10.096340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733e701d-a256-4014-9235-efd39ef64943 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:10.097268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733e701d-a256-4014-9235-efd39ef64943 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:10.097824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733e701d-a256-4014-9235-efd39ef64943 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:10.099009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733e701d-a256-4014-9235-efd39ef64943 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:10.099556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733e701d-a256-4014-9235-efd39ef64943 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:10.100059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-733e701d-a256-4014-9235-efd39ef64943 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:10.156506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:10.173466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:10.177034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:10.178604 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f915ff67-14da-4875-86b0-74cf4efee353 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1743 microversion: 2.20 time: 0.316011 Sep 27 10:25:10.179411 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2938/5431] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:09 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1743 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:10.314615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-733e701d-a256-4014-9235-efd39ef64943 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:10.323998 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-733e701d-a256-4014-9235-efd39ef64943 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1314 microversion: 2.1 time: 0.241321 Sep 27 10:25:10.324703 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2494/5432] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:10 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1314 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:10.426545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-93a16420-8e3e-4ff5-bcfb-bd556ef6a931 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:10.427206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-93a16420-8e3e-4ff5-bcfb-bd556ef6a931 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:10.434640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93a16420-8e3e-4ff5-bcfb-bd556ef6a931 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:10.435352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93a16420-8e3e-4ff5-bcfb-bd556ef6a931 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:10.435978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93a16420-8e3e-4ff5-bcfb-bd556ef6a931 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:10.436902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93a16420-8e3e-4ff5-bcfb-bd556ef6a931 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:10.437522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93a16420-8e3e-4ff5-bcfb-bd556ef6a931 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:10.438108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93a16420-8e3e-4ff5-bcfb-bd556ef6a931 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:10.456969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0f52706c-e24b-4ff7-b8aa-1c6e4f019c84 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:10.458611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0f52706c-e24b-4ff7-b8aa-1c6e4f019c84 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:10.467950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f52706c-e24b-4ff7-b8aa-1c6e4f019c84 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:10.471672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f52706c-e24b-4ff7-b8aa-1c6e4f019c84 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:10.473965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f52706c-e24b-4ff7-b8aa-1c6e4f019c84 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:10.477126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f52706c-e24b-4ff7-b8aa-1c6e4f019c84 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:10.477903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f52706c-e24b-4ff7-b8aa-1c6e4f019c84 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:10.492693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0f52706c-e24b-4ff7-b8aa-1c6e4f019c84 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:10.590372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-93a16420-8e3e-4ff5-bcfb-bd556ef6a931 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:10.620642 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-93a16420-8e3e-4ff5-bcfb-bd556ef6a931 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1272 microversion: 2.1 time: 0.198235 Sep 27 10:25:10.621314 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2939/5433] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:10 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1272 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:10.767023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0f52706c-e24b-4ff7-b8aa-1c6e4f019c84 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:10.778900 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0f52706c-e24b-4ff7-b8aa-1c6e4f019c84 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.327000 Sep 27 10:25:10.779931 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2495/5434] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:10 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:11.205500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:11.207120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:11.226769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:11.228492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:11.229203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:11.237535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:11.239311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:11.240212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:11.246735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:11.348806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e8c1f17c-b34c-4aae-acf8-2809eb5832e2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:11.350225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e8c1f17c-b34c-4aae-acf8-2809eb5832e2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:11.364642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8c1f17c-b34c-4aae-acf8-2809eb5832e2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:11.374150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8c1f17c-b34c-4aae-acf8-2809eb5832e2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:11.374679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8c1f17c-b34c-4aae-acf8-2809eb5832e2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:11.375624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8c1f17c-b34c-4aae-acf8-2809eb5832e2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:11.376151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8c1f17c-b34c-4aae-acf8-2809eb5832e2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:11.376534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8c1f17c-b34c-4aae-acf8-2809eb5832e2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:11.401642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:11.594366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:11.615707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:11.628887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:11.633974 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c864c6b-e87d-4ea9-9b9f-dd1ede63190c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.433124 Sep 27 10:25:11.634813 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2940/5435] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:11 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:11.646867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ddc61e29-b78c-40f2-ae52-ef9ccff0f284 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:11.647902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ddc61e29-b78c-40f2-ae52-ef9ccff0f284 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:11.661049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddc61e29-b78c-40f2-ae52-ef9ccff0f284 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:11.662137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddc61e29-b78c-40f2-ae52-ef9ccff0f284 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:11.665136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddc61e29-b78c-40f2-ae52-ef9ccff0f284 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:11.669623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddc61e29-b78c-40f2-ae52-ef9ccff0f284 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:11.670378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddc61e29-b78c-40f2-ae52-ef9ccff0f284 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:11.671043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddc61e29-b78c-40f2-ae52-ef9ccff0f284 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:11.793630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ddc61e29-b78c-40f2-ae52-ef9ccff0f284 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:11.817582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e8c1f17c-b34c-4aae-acf8-2809eb5832e2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:11.817784 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ddc61e29-b78c-40f2-ae52-ef9ccff0f284 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1272 microversion: 2.1 time: 0.170119 Sep 27 10:25:11.823406 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2941/5436] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:11 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1272 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:11.830223 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e8c1f17c-b34c-4aae-acf8-2809eb5832e2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1501 microversion: 2.1 time: 0.486378 Sep 27 10:25:11.837574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-521168d9-cf55-42fd-b3bd-51233dbdfa2e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:11.838544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-521168d9-cf55-42fd-b3bd-51233dbdfa2e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:11.850510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521168d9-cf55-42fd-b3bd-51233dbdfa2e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:11.851016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521168d9-cf55-42fd-b3bd-51233dbdfa2e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:11.851457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521168d9-cf55-42fd-b3bd-51233dbdfa2e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:11.852178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521168d9-cf55-42fd-b3bd-51233dbdfa2e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:11.852638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521168d9-cf55-42fd-b3bd-51233dbdfa2e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:11.855009 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2496/5437] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:11 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1501 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:11.861232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-521168d9-cf55-42fd-b3bd-51233dbdfa2e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:11.867968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0ca2833c-8c56-4670-aacd-19768848c543 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:11.870729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0ca2833c-8c56-4670-aacd-19768848c543 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:11.881642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ca2833c-8c56-4670-aacd-19768848c543 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:11.884885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ca2833c-8c56-4670-aacd-19768848c543 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:11.889864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ca2833c-8c56-4670-aacd-19768848c543 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:11.891000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ca2833c-8c56-4670-aacd-19768848c543 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:11.892136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ca2833c-8c56-4670-aacd-19768848c543 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:11.893935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ca2833c-8c56-4670-aacd-19768848c543 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:11.996268 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0ca2833c-8c56-4670-aacd-19768848c543 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action" status: 200 len: 14 microversion: 2.1 time: 0.131621 Sep 27 10:25:12.003139 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2497/5438] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:11 2022] POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action => generated 14 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 27 10:25:12.381765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-521168d9-cf55-42fd-b3bd-51233dbdfa2e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:12.395429 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-521168d9-cf55-42fd-b3bd-51233dbdfa2e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.567127 Sep 27 10:25:12.414125 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2942/5439] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:11 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:12.669605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:12.671539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:12.705693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:12.708722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:12.709467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:12.712508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:12.713315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:12.713971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:12.852924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a272eeaa-e888-44c2-bf8d-934d0c54dcab tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:12.854851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a272eeaa-e888-44c2-bf8d-934d0c54dcab tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:12.871670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a272eeaa-e888-44c2-bf8d-934d0c54dcab tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:12.893597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a272eeaa-e888-44c2-bf8d-934d0c54dcab tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:12.894620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a272eeaa-e888-44c2-bf8d-934d0c54dcab tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:12.898696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a272eeaa-e888-44c2-bf8d-934d0c54dcab tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:12.900798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a272eeaa-e888-44c2-bf8d-934d0c54dcab tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:12.902992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a272eeaa-e888-44c2-bf8d-934d0c54dcab tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:13.340176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:13.341518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a272eeaa-e888-44c2-bf8d-934d0c54dcab tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:13.352829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:13.357266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:13.358473 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-06c62cd3-fbdf-4bb8-944a-652068c37db8 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.695418 Sep 27 10:25:13.359054 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2498/5440] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:12 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 697 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:13.361362 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a272eeaa-e888-44c2-bf8d-934d0c54dcab tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1314 microversion: 2.1 time: 0.523144 Sep 27 10:25:13.361985 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2943/5441] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:12 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1314 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:13.376986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-26878958-a7fc-44e4-9a9e-c8b2513d54b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:13.379667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-26878958-a7fc-44e4-9a9e-c8b2513d54b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:13.402347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26878958-a7fc-44e4-9a9e-c8b2513d54b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:13.403759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26878958-a7fc-44e4-9a9e-c8b2513d54b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:13.409659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26878958-a7fc-44e4-9a9e-c8b2513d54b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:13.412870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26878958-a7fc-44e4-9a9e-c8b2513d54b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:13.416926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26878958-a7fc-44e4-9a9e-c8b2513d54b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:13.417658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26878958-a7fc-44e4-9a9e-c8b2513d54b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:13.421751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5c081b9d-6e83-4f66-ba43-557842303b1d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:13.422377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5c081b9d-6e83-4f66-ba43-557842303b1d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:13.446568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c081b9d-6e83-4f66-ba43-557842303b1d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:13.448214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c081b9d-6e83-4f66-ba43-557842303b1d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:13.450720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c081b9d-6e83-4f66-ba43-557842303b1d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:13.458556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c081b9d-6e83-4f66-ba43-557842303b1d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:13.459534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c081b9d-6e83-4f66-ba43-557842303b1d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:13.464548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c081b9d-6e83-4f66-ba43-557842303b1d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:13.500696 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-26878958-a7fc-44e4-9a9e-c8b2513d54b3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action" status: 200 len: 14 microversion: 2.1 time: 0.137381 Sep 27 10:25:13.504553 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2944/5442] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:13 2022] POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action => generated 14 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:25:13.842445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5c081b9d-6e83-4f66-ba43-557842303b1d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:13.858986 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5c081b9d-6e83-4f66-ba43-557842303b1d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.441061 Sep 27 10:25:13.861778 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2499/5443] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:13 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:13.925844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-50167248-537f-4ccb-9e6b-511657417c69 req-25f0a711-4f93-463b-8528-76c6564cfee9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3db53022-2562-4d3c-a0e1-9d9162d5e1ac", "tag": "64db7780-37ee-4c1d-95f1-58bf82537877"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:13.971989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-50167248-537f-4ccb-9e6b-511657417c69 req-25f0a711-4f93-463b-8528-76c6564cfee9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:13.974112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-50167248-537f-4ccb-9e6b-511657417c69 req-25f0a711-4f93-463b-8528-76c6564cfee9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:13.976505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-50167248-537f-4ccb-9e6b-511657417c69 req-25f0a711-4f93-463b-8528-76c6564cfee9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:14.003030 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-50167248-537f-4ccb-9e6b-511657417c69 req-25f0a711-4f93-463b-8528-76c6564cfee9 service nova] Creating event network-changed:64db7780-37ee-4c1d-95f1-58bf82537877 for instance 3db53022-2562-4d3c-a0e1-9d9162d5e1ac on nch2-master-1 Sep 27 10:25:14.012136 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-50167248-537f-4ccb-9e6b-511657417c69 req-25f0a711-4f93-463b-8528-76c6564cfee9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.096954 Sep 27 10:25:14.013289 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2945/5444] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:14.392627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:14.393678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:14.395442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-20b51f9b-5a2e-4ad3-b61c-d022dfd4cae4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:14.396163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-20b51f9b-5a2e-4ad3-b61c-d022dfd4cae4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:14.403276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:14.404033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:14.404760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:14.405705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:14.406629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:14.407282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:14.414501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20b51f9b-5a2e-4ad3-b61c-d022dfd4cae4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:14.418252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20b51f9b-5a2e-4ad3-b61c-d022dfd4cae4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:14.418968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20b51f9b-5a2e-4ad3-b61c-d022dfd4cae4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:14.419895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20b51f9b-5a2e-4ad3-b61c-d022dfd4cae4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:14.422557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20b51f9b-5a2e-4ad3-b61c-d022dfd4cae4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:14.423217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-20b51f9b-5a2e-4ad3-b61c-d022dfd4cae4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:14.769006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-20b51f9b-5a2e-4ad3-b61c-d022dfd4cae4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:14.782911 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-20b51f9b-5a2e-4ad3-b61c-d022dfd4cae4 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1314 microversion: 2.1 time: 0.396757 Sep 27 10:25:14.783521 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2946/5445] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:14 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1314 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:14.791093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cbc9743b-255a-4475-94f2-ee1d0eb21fee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:14.796090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cbc9743b-255a-4475-94f2-ee1d0eb21fee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:14.817397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:14.821022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc9743b-255a-4475-94f2-ee1d0eb21fee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:14.821540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc9743b-255a-4475-94f2-ee1d0eb21fee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:14.822180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc9743b-255a-4475-94f2-ee1d0eb21fee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:14.823085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc9743b-255a-4475-94f2-ee1d0eb21fee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:14.825282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc9743b-255a-4475-94f2-ee1d0eb21fee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:14.825965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc9743b-255a-4475-94f2-ee1d0eb21fee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:14.830374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:14.833262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:14.836024 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7a46c1ea-31ab-419f-9a4b-28212ce8a4fb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.452969 Sep 27 10:25:14.837867 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2500/5446] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:14 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:14.887800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-40a61fa6-671f-46aa-866f-ce07daebd4c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:14.889714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-40a61fa6-671f-46aa-866f-ce07daebd4c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:14.897722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a61fa6-671f-46aa-866f-ce07daebd4c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:14.898622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a61fa6-671f-46aa-866f-ce07daebd4c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:14.899310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a61fa6-671f-46aa-866f-ce07daebd4c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:14.900296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a61fa6-671f-46aa-866f-ce07daebd4c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:14.901029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a61fa6-671f-46aa-866f-ce07daebd4c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:14.901882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40a61fa6-671f-46aa-866f-ce07daebd4c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:14.907098 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cbc9743b-255a-4475-94f2-ee1d0eb21fee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action" status: 200 len: 14 microversion: 2.1 time: 0.122079 Sep 27 10:25:14.908286 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2947/5447] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:14 2022] POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action => generated 14 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:25:15.159628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-40a61fa6-671f-46aa-866f-ce07daebd4c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:15.169080 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-40a61fa6-671f-46aa-866f-ce07daebd4c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.285457 Sep 27 10:25:15.173323 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2501/5448] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:14 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:15.381026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-abca626a-a0c9-4b76-b2b5-e67db037015b req-360d03ca-5ee0-43b6-a034-d36f1fdbb86d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc", "name": "network-vif-plugged", "status": "completed", "tag": "ad654d98-8103-4e52-a2b9-095d297e78f7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:15.391439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:15.414610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-abca626a-a0c9-4b76-b2b5-e67db037015b req-360d03ca-5ee0-43b6-a034-d36f1fdbb86d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:15.416140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-abca626a-a0c9-4b76-b2b5-e67db037015b req-360d03ca-5ee0-43b6-a034-d36f1fdbb86d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:15.418001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-abca626a-a0c9-4b76-b2b5-e67db037015b req-360d03ca-5ee0-43b6-a034-d36f1fdbb86d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:15.458670 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-abca626a-a0c9-4b76-b2b5-e67db037015b req-360d03ca-5ee0-43b6-a034-d36f1fdbb86d service nova] Creating event network-vif-plugged:ad654d98-8103-4e52-a2b9-095d297e78f7 for instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc on nch2-master-1 Sep 27 10:25:15.476885 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-abca626a-a0c9-4b76-b2b5-e67db037015b req-360d03ca-5ee0-43b6-a034-d36f1fdbb86d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.095622 Sep 27 10:25:15.478381 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2948/5449] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:15.809167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-45569908-3f83-420a-ba87-a666c4629a37 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:15.810433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-45569908-3f83-420a-ba87-a666c4629a37 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:15.814038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:15.821978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:15.832442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45569908-3f83-420a-ba87-a666c4629a37 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:15.833408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45569908-3f83-420a-ba87-a666c4629a37 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:15.836994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45569908-3f83-420a-ba87-a666c4629a37 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:15.838030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45569908-3f83-420a-ba87-a666c4629a37 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:15.838741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45569908-3f83-420a-ba87-a666c4629a37 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:15.839388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45569908-3f83-420a-ba87-a666c4629a37 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:15.867188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:15.867985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:15.887738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:15.888241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:15.892203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:15.893420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:15.897405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:15.898417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:16.080495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-45569908-3f83-420a-ba87-a666c4629a37 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:16.091598 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-45569908-3f83-420a-ba87-a666c4629a37 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1314 microversion: 2.1 time: 0.286407 Sep 27 10:25:16.104722 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2502/5450] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:15 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1314 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:16.104722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af81a1b0-5c40-47b8-8de6-728e48df1ce3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:16.104722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-af81a1b0-5c40-47b8-8de6-728e48df1ce3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:16.109121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af81a1b0-5c40-47b8-8de6-728e48df1ce3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:16.110104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af81a1b0-5c40-47b8-8de6-728e48df1ce3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:16.110804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af81a1b0-5c40-47b8-8de6-728e48df1ce3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:16.111973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af81a1b0-5c40-47b8-8de6-728e48df1ce3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:16.112716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af81a1b0-5c40-47b8-8de6-728e48df1ce3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:16.113401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af81a1b0-5c40-47b8-8de6-728e48df1ce3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:16.171244 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af81a1b0-5c40-47b8-8de6-728e48df1ce3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action" status: 200 len: 14 microversion: 2.1 time: 0.076218 Sep 27 10:25:16.189154 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2503/5451] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:16 2022] POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action => generated 14 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:25:16.203552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa9bda95-d9f6-484f-a2dd-da2c3d18e1b5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:16.204277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aa9bda95-d9f6-484f-a2dd-da2c3d18e1b5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:16.216508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa9bda95-d9f6-484f-a2dd-da2c3d18e1b5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:16.217350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa9bda95-d9f6-484f-a2dd-da2c3d18e1b5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:16.218087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa9bda95-d9f6-484f-a2dd-da2c3d18e1b5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:16.219008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa9bda95-d9f6-484f-a2dd-da2c3d18e1b5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:16.219460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa9bda95-d9f6-484f-a2dd-da2c3d18e1b5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:16.220264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa9bda95-d9f6-484f-a2dd-da2c3d18e1b5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:16.312863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:16.325023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:16.326971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:16.328396 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-59516324-4372-48ef-b3ed-fd6149cd2c83 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.469716 Sep 27 10:25:16.337325 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2949/5452] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:15 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:16.511632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aa9bda95-d9f6-484f-a2dd-da2c3d18e1b5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:16.522549 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aa9bda95-d9f6-484f-a2dd-da2c3d18e1b5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.323530 Sep 27 10:25:16.537441 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2504/5453] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:16 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:17.129151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6fd66b4f-e347-4fbf-b056-ed1bb68df210 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:17.132315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6fd66b4f-e347-4fbf-b056-ed1bb68df210 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:17.166952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fd66b4f-e347-4fbf-b056-ed1bb68df210 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:17.174355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fd66b4f-e347-4fbf-b056-ed1bb68df210 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:17.175481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fd66b4f-e347-4fbf-b056-ed1bb68df210 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:17.178099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fd66b4f-e347-4fbf-b056-ed1bb68df210 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:17.188072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fa048ace-1304-4482-a983-82b1e7cd9c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:17.193300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa048ace-1304-4482-a983-82b1e7cd9c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:17.193541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fd66b4f-e347-4fbf-b056-ed1bb68df210 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:17.193957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fd66b4f-e347-4fbf-b056-ed1bb68df210 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:17.212689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa048ace-1304-4482-a983-82b1e7cd9c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:17.216257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa048ace-1304-4482-a983-82b1e7cd9c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:17.216952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa048ace-1304-4482-a983-82b1e7cd9c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:17.217878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa048ace-1304-4482-a983-82b1e7cd9c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:17.218659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa048ace-1304-4482-a983-82b1e7cd9c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:17.219873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa048ace-1304-4482-a983-82b1e7cd9c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:17.303848 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fa048ace-1304-4482-a983-82b1e7cd9c14 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action" status: 200 len: 14 microversion: 2.1 time: 0.119240 Sep 27 10:25:17.305795 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2505/5454] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:17 2022] POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action => generated 14 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:25:17.356749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:17.357505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:17.372489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:17.374137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:17.374587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:17.380452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:17.381131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:17.381592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:17.591517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6fd66b4f-e347-4fbf-b056-ed1bb68df210 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:17.613330 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6fd66b4f-e347-4fbf-b056-ed1bb68df210 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1314 microversion: 2.1 time: 0.497077 Sep 27 10:25:17.639386 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2950/5455] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:17 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1314 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:17.665273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92a4eecf-8ec1-4988-a603-e6b9da65d83a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:17.665896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-92a4eecf-8ec1-4988-a603-e6b9da65d83a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:17.684362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92a4eecf-8ec1-4988-a603-e6b9da65d83a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:17.684910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92a4eecf-8ec1-4988-a603-e6b9da65d83a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:17.685355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92a4eecf-8ec1-4988-a603-e6b9da65d83a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:17.686074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92a4eecf-8ec1-4988-a603-e6b9da65d83a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:17.696086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92a4eecf-8ec1-4988-a603-e6b9da65d83a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:17.697950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92a4eecf-8ec1-4988-a603-e6b9da65d83a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:17.954563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-92a4eecf-8ec1-4988-a603-e6b9da65d83a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:17.965482 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-92a4eecf-8ec1-4988-a603-e6b9da65d83a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.305986 Sep 27 10:25:17.966233 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2951/5456] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:17 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:18.273556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:18.283816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:18.285370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:18.294350 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-53b8669f-4168-4120-8048-3f8e0999c4cb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.942630 Sep 27 10:25:18.295702 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2506/5457] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:17 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 945 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:18.315815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-172fb642-068b-4608-bb4d-59657064f756 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:18.319203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-172fb642-068b-4608-bb4d-59657064f756 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:18.331342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172fb642-068b-4608-bb4d-59657064f756 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:18.333459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172fb642-068b-4608-bb4d-59657064f756 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:18.335999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172fb642-068b-4608-bb4d-59657064f756 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:18.337165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172fb642-068b-4608-bb4d-59657064f756 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:18.338493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172fb642-068b-4608-bb4d-59657064f756 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:18.339342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172fb642-068b-4608-bb4d-59657064f756 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:18.434764 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-172fb642-068b-4608-bb4d-59657064f756 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action" status: 200 len: 14 microversion: 2.1 time: 0.122625 Sep 27 10:25:18.437171 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2952/5458] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:18 2022] POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action => generated 14 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:25:18.644545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c0d7ba67-c6a4-4a2a-813d-9253062cd121 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:18.645732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c0d7ba67-c6a4-4a2a-813d-9253062cd121 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:18.658735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d7ba67-c6a4-4a2a-813d-9253062cd121 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:18.659300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d7ba67-c6a4-4a2a-813d-9253062cd121 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:18.659889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d7ba67-c6a4-4a2a-813d-9253062cd121 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:18.660922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d7ba67-c6a4-4a2a-813d-9253062cd121 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:18.666986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d7ba67-c6a4-4a2a-813d-9253062cd121 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:18.667959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d7ba67-c6a4-4a2a-813d-9253062cd121 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:18.995816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-56a27e2c-aad8-485e-85b6-0787b9bb3663 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:18.997104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-56a27e2c-aad8-485e-85b6-0787b9bb3663 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:19.014224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56a27e2c-aad8-485e-85b6-0787b9bb3663 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:19.015099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56a27e2c-aad8-485e-85b6-0787b9bb3663 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:19.016318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56a27e2c-aad8-485e-85b6-0787b9bb3663 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:19.017353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56a27e2c-aad8-485e-85b6-0787b9bb3663 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:19.018697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56a27e2c-aad8-485e-85b6-0787b9bb3663 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:19.019396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-56a27e2c-aad8-485e-85b6-0787b9bb3663 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:19.123789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c0d7ba67-c6a4-4a2a-813d-9253062cd121 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:19.136229 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c0d7ba67-c6a4-4a2a-813d-9253062cd121 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1314 microversion: 2.1 time: 0.497412 Sep 27 10:25:19.136703 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2507/5459] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:18 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1314 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:19.323677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:19.325861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:19.335987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:19.339006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:19.347196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:19.348830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:19.349632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:19.350297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:19.810482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-56a27e2c-aad8-485e-85b6-0787b9bb3663 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:19.825565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-56a27e2c-aad8-485e-85b6-0787b9bb3663 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1217 microversion: 2.1 time: 0.838617 Sep 27 10:25:19.846181 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2953/5460] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:18 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1217 bytes in 861 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:19.853483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f4939119-8a2c-4b73-95ec-49cf4bd3cd41 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:19.856075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f4939119-8a2c-4b73-95ec-49cf4bd3cd41 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:19.864890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4939119-8a2c-4b73-95ec-49cf4bd3cd41 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:19.865651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4939119-8a2c-4b73-95ec-49cf4bd3cd41 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:19.866462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4939119-8a2c-4b73-95ec-49cf4bd3cd41 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:19.867648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4939119-8a2c-4b73-95ec-49cf4bd3cd41 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:19.868282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4939119-8a2c-4b73-95ec-49cf4bd3cd41 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:19.868906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f4939119-8a2c-4b73-95ec-49cf4bd3cd41 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:19.939626 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f4939119-8a2c-4b73-95ec-49cf4bd3cd41 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action" status: 200 len: 14 microversion: 2.1 time: 0.091672 Sep 27 10:25:19.943187 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2954/5461] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:19 2022] POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action => generated 14 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:25:19.986831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:20.004697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:20.007459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:20.009744 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f2424395-3f99-42ca-a4f0-61ebb847c289 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.690703 Sep 27 10:25:20.027965 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2508/5462] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:19 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:20.165425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a17fd6d9-4d33-448c-bd3e-838778994dda tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:20.166393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a17fd6d9-4d33-448c-bd3e-838778994dda tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:20.184130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a17fd6d9-4d33-448c-bd3e-838778994dda tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:20.187601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a17fd6d9-4d33-448c-bd3e-838778994dda tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:20.188336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a17fd6d9-4d33-448c-bd3e-838778994dda tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:20.189762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a17fd6d9-4d33-448c-bd3e-838778994dda tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:20.191697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a17fd6d9-4d33-448c-bd3e-838778994dda tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:20.192357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a17fd6d9-4d33-448c-bd3e-838778994dda tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:20.402580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a17fd6d9-4d33-448c-bd3e-838778994dda tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:20.414208 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a17fd6d9-4d33-448c-bd3e-838778994dda tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1314 microversion: 2.1 time: 0.253552 Sep 27 10:25:20.416417 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2955/5463] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:20 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1314 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:20.855957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-87421f4a-fa3f-4712-9aa8-be803497abbf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:20.857869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-87421f4a-fa3f-4712-9aa8-be803497abbf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:20.869364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87421f4a-fa3f-4712-9aa8-be803497abbf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:20.870127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87421f4a-fa3f-4712-9aa8-be803497abbf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:20.870727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87421f4a-fa3f-4712-9aa8-be803497abbf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:20.872036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87421f4a-fa3f-4712-9aa8-be803497abbf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:20.872883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87421f4a-fa3f-4712-9aa8-be803497abbf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:20.873759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-87421f4a-fa3f-4712-9aa8-be803497abbf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:20.954685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-35a47e53-4987-4981-b3b7-c33d9239be05 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:20.957232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-35a47e53-4987-4981-b3b7-c33d9239be05 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:20.976280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a47e53-4987-4981-b3b7-c33d9239be05 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:20.977245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a47e53-4987-4981-b3b7-c33d9239be05 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:20.978057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a47e53-4987-4981-b3b7-c33d9239be05 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:20.979035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a47e53-4987-4981-b3b7-c33d9239be05 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:20.979661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a47e53-4987-4981-b3b7-c33d9239be05 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:20.980272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35a47e53-4987-4981-b3b7-c33d9239be05 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:21.058432 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-35a47e53-4987-4981-b3b7-c33d9239be05 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action" status: 200 len: 14 microversion: 2.1 time: 0.107929 Sep 27 10:25:21.059378 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2956/5464] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:20 2022] POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action => generated 14 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:25:21.064960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:21.065978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:21.073501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:21.074330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:21.074965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:21.075888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:21.076546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:21.077225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:21.162866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-87421f4a-fa3f-4712-9aa8-be803497abbf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:21.178441 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-87421f4a-fa3f-4712-9aa8-be803497abbf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1404 microversion: 2.1 time: 0.327948 Sep 27 10:25:21.181331 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2509/5465] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:20 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1404 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:21.207888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-051b3d8e-a850-4518-b659-97a8f3173a79 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:21.208889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-051b3d8e-a850-4518-b659-97a8f3173a79 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:21.221681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b3d8e-a850-4518-b659-97a8f3173a79 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:21.222826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b3d8e-a850-4518-b659-97a8f3173a79 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:21.225036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b3d8e-a850-4518-b659-97a8f3173a79 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:21.226059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b3d8e-a850-4518-b659-97a8f3173a79 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:21.232480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b3d8e-a850-4518-b659-97a8f3173a79 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:21.237346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-051b3d8e-a850-4518-b659-97a8f3173a79 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:21.352377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:21.369546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:21.378159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:21.386567 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-47607292-a835-48e9-9882-e01f8099a7dd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.320856 Sep 27 10:25:21.387787 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2957/5466] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:21 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:21.420615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-82ba381e-5877-4faa-ac1a-f06b57100bcd req-941b942b-51bd-462b-b51c-fbd8c4486734 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3db53022-2562-4d3c-a0e1-9d9162d5e1ac", "name": "network-vif-plugged", "status": "completed", "tag": "64db7780-37ee-4c1d-95f1-58bf82537877"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:21.447984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-82ba381e-5877-4faa-ac1a-f06b57100bcd req-941b942b-51bd-462b-b51c-fbd8c4486734 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:21.448989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-82ba381e-5877-4faa-ac1a-f06b57100bcd req-941b942b-51bd-462b-b51c-fbd8c4486734 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:21.449608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-82ba381e-5877-4faa-ac1a-f06b57100bcd req-941b942b-51bd-462b-b51c-fbd8c4486734 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:21.495275 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-82ba381e-5877-4faa-ac1a-f06b57100bcd req-941b942b-51bd-462b-b51c-fbd8c4486734 service nova] Creating event network-vif-plugged:64db7780-37ee-4c1d-95f1-58bf82537877 for instance 3db53022-2562-4d3c-a0e1-9d9162d5e1ac on nch2-master-1 Sep 27 10:25:21.509020 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-82ba381e-5877-4faa-ac1a-f06b57100bcd req-941b942b-51bd-462b-b51c-fbd8c4486734 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093472 Sep 27 10:25:21.511023 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2958/5467] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:21.519291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-51888076-b7fb-4e9e-8028-81ca5904e7e5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:21.520825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-51888076-b7fb-4e9e-8028-81ca5904e7e5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:21.530134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51888076-b7fb-4e9e-8028-81ca5904e7e5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:21.530859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51888076-b7fb-4e9e-8028-81ca5904e7e5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:21.531933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51888076-b7fb-4e9e-8028-81ca5904e7e5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:21.533041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51888076-b7fb-4e9e-8028-81ca5904e7e5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:21.533716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51888076-b7fb-4e9e-8028-81ca5904e7e5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:21.534480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51888076-b7fb-4e9e-8028-81ca5904e7e5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:21.579239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-051b3d8e-a850-4518-b659-97a8f3173a79 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:21.588621 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-051b3d8e-a850-4518-b659-97a8f3173a79 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1404 microversion: 2.1 time: 0.384870 Sep 27 10:25:21.610105 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2510/5468] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:21 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1404 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:21.622470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-TestVolumeBootPatternsnapshot-1867614797"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:21.624982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:21.639121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:21.639862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:21.640318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:21.643269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:21.643884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:21.644337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:21.802284 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.compute.api [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Attempting to quiesce instance before volume snapshot. Sep 27 10:25:21.851374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-51888076-b7fb-4e9e-8028-81ca5904e7e5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:21.860090 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-51888076-b7fb-4e9e-8028-81ca5904e7e5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1314 microversion: 2.1 time: 0.347374 Sep 27 10:25:21.863767 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2959/5469] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:21 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1314 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:22.071617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a3a76dde-d8b3-474e-98b5-70dde2bd970f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:22.073547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a3a76dde-d8b3-474e-98b5-70dde2bd970f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:22.093085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a76dde-d8b3-474e-98b5-70dde2bd970f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:22.093655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a76dde-d8b3-474e-98b5-70dde2bd970f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:22.094099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a76dde-d8b3-474e-98b5-70dde2bd970f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:22.095033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a76dde-d8b3-474e-98b5-70dde2bd970f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:22.095506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a76dde-d8b3-474e-98b5-70dde2bd970f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:22.095938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3a76dde-d8b3-474e-98b5-70dde2bd970f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:22.125207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:22.156406 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a3a76dde-d8b3-474e-98b5-70dde2bd970f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action" status: 200 len: 14 microversion: 2.1 time: 0.089607 Sep 27 10:25:22.157840 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2960/5470] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:22 2022] POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action => generated 14 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 27 10:25:22.231728 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.compute.api [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Skipping quiescing instance because the operation is not supported by the underlying compute driver. Sep 27 10:25:22.332897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Creating snapshot from volume c2bfd580-e62a-4f33-a8ea-3e0437752d94. {{(pid=145969) snapshot_instance /opt/stack/nova/nova/compute/api.py:3509}} Sep 27 10:25:22.419938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:22.423869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:22.446412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:22.449887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:22.450788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:22.454574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:22.455292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:22.458411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:22.616631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:22.754370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:22.771123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:22.774468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:22.777483 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0890ecf6-3066-412b-b59e-379753becdb1 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.362355 Sep 27 10:25:22.779385 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2961/5471] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:22 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:22.899540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d0f09b16-9c17-4cba-899f-cc898472d175 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:22.900193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d0f09b16-9c17-4cba-899f-cc898472d175 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:22.914768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f09b16-9c17-4cba-899f-cc898472d175 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:22.915265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f09b16-9c17-4cba-899f-cc898472d175 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:22.934789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f09b16-9c17-4cba-899f-cc898472d175 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:22.935802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f09b16-9c17-4cba-899f-cc898472d175 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:22.937694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f09b16-9c17-4cba-899f-cc898472d175 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:22.938235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f09b16-9c17-4cba-899f-cc898472d175 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:23.338177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d0f09b16-9c17-4cba-899f-cc898472d175 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:23.349832 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d0f09b16-9c17-4cba-899f-cc898472d175 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1314 microversion: 2.1 time: 0.465470 Sep 27 10:25:23.350469 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2962/5472] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:22 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1314 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:23.356437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b26aba47-3b11-482b-aca1-ad7956ae6c6d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:23.358792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b26aba47-3b11-482b-aca1-ad7956ae6c6d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:23.365772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b26aba47-3b11-482b-aca1-ad7956ae6c6d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:23.366534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b26aba47-3b11-482b-aca1-ad7956ae6c6d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:23.367211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b26aba47-3b11-482b-aca1-ad7956ae6c6d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:23.368141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b26aba47-3b11-482b-aca1-ad7956ae6c6d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:23.368828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b26aba47-3b11-482b-aca1-ad7956ae6c6d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:23.369501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b26aba47-3b11-482b-aca1-ad7956ae6c6d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:23.429956 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b26aba47-3b11-482b-aca1-ad7956ae6c6d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action" status: 200 len: 14 microversion: 2.1 time: 0.078180 Sep 27 10:25:23.430967 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2963/5473] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:23 2022] POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action => generated 14 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:25:23.633291 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3df12c14-9082-4eac-bab9-fd9236c872eb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "POST /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc/action" status: 202 len: 0 microversion: 2.1 time: 2.014757 Sep 27 10:25:23.634776 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2511/5474] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:21 2022] POST /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc/action => generated 0 bytes in 2017 msecs (HTTP/1.1 202) 10 headers in 450 bytes (2 switches on core 0) Sep 27 10:25:23.809322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:23.810414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:23.838505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:23.840206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:23.842579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:23.844176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:23.845408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:23.846035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:24.271388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:24.284707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:24.287795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:24.294589 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9ebe169d-b353-458b-8b68-105072505543 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.491667 Sep 27 10:25:24.310059 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2964/5475] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:23 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:24.381975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-39a2d498-23ec-43e5-b5cd-eed715823bc5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:24.383229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-39a2d498-23ec-43e5-b5cd-eed715823bc5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:24.398876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a2d498-23ec-43e5-b5cd-eed715823bc5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:24.406038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a2d498-23ec-43e5-b5cd-eed715823bc5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:24.407583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a2d498-23ec-43e5-b5cd-eed715823bc5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:24.408537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a2d498-23ec-43e5-b5cd-eed715823bc5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:24.409212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a2d498-23ec-43e5-b5cd-eed715823bc5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:24.409890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39a2d498-23ec-43e5-b5cd-eed715823bc5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:24.441923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a8d6fc4e-b11c-48e0-a99a-69e953d4058e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:24.444167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a8d6fc4e-b11c-48e0-a99a-69e953d4058e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:24.458826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d6fc4e-b11c-48e0-a99a-69e953d4058e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:24.459254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d6fc4e-b11c-48e0-a99a-69e953d4058e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:24.459702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d6fc4e-b11c-48e0-a99a-69e953d4058e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:24.460826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d6fc4e-b11c-48e0-a99a-69e953d4058e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:24.462077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d6fc4e-b11c-48e0-a99a-69e953d4058e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:24.462700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d6fc4e-b11c-48e0-a99a-69e953d4058e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:24.520286 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a8d6fc4e-b11c-48e0-a99a-69e953d4058e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action" status: 200 len: 8207 microversion: 2.1 time: 0.081741 Sep 27 10:25:24.521615 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2965/5476] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:24 2022] POST /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99/action => generated 8207 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:24.534053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-071d4da4-e97a-49c5-a785-39815ccef98b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:24.545606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-071d4da4-e97a-49c5-a785-39815ccef98b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:24.559260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-071d4da4-e97a-49c5-a785-39815ccef98b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:24.560086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-071d4da4-e97a-49c5-a785-39815ccef98b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:24.562257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-071d4da4-e97a-49c5-a785-39815ccef98b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:24.563198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-071d4da4-e97a-49c5-a785-39815ccef98b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:24.563890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-071d4da4-e97a-49c5-a785-39815ccef98b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:24.566657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-071d4da4-e97a-49c5-a785-39815ccef98b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:24.744605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-39a2d498-23ec-43e5-b5cd-eed715823bc5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:24.764210 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-39a2d498-23ec-43e5-b5cd-eed715823bc5 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1464 microversion: 2.1 time: 0.388525 Sep 27 10:25:24.793071 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2512/5477] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:24 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1464 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:24.807163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "bb8280cc-ed98-4857-a90d-aef277d7bdaf", "device": "/dev/sdb"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:24.809736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:24.828894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:24.833557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-071d4da4-e97a-49c5-a785-39815ccef98b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:24.834378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:24.834980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:24.836208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:24.836852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:24.838631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:24.848627 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-071d4da4-e97a-49c5-a785-39815ccef98b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.320986 Sep 27 10:25:24.851843 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2966/5478] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:24 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:24.881975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f64a1a50-cab2-48fa-a60b-4e21f19ab711 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:24.882851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f64a1a50-cab2-48fa-a60b-4e21f19ab711 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:24.897160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f64a1a50-cab2-48fa-a60b-4e21f19ab711 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:24.898011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f64a1a50-cab2-48fa-a60b-4e21f19ab711 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:24.898843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f64a1a50-cab2-48fa-a60b-4e21f19ab711 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:24.902982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f64a1a50-cab2-48fa-a60b-4e21f19ab711 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:24.905678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f64a1a50-cab2-48fa-a60b-4e21f19ab711 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:24.906338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f64a1a50-cab2-48fa-a60b-4e21f19ab711 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:25.159694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f64a1a50-cab2-48fa-a60b-4e21f19ab711 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:25.171930 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f64a1a50-cab2-48fa-a60b-4e21f19ab711 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.294146 Sep 27 10:25:25.173069 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2967/5479] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:24 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:25.185491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-image-snapshot-server-2109778300", "imageRef": "8a47e4d9-f03a-448a-9323-d125c043a18d", "flavorRef": "452", "networks": [{"uuid": "1e417b59-62bb-4007-a78a-c15c153efd33"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:25.276336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] validate_networks() for [('1e417b59-62bb-4007-a78a-c15c153efd33', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:25:25.374353 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b4fb9469-7813-4bf4-bd5a-a0c31dc6eac1 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "POST /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.572146 Sep 27 10:25:25.378151 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2513/5480] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:25:24 2022] POST /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac/os-volume_attachments => generated 194 bytes in 578 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:25.397753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7b745f20-c2cc-4cc1-aca9-43f65735f528 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:25.398820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7b745f20-c2cc-4cc1-aca9-43f65735f528 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:25.409203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b745f20-c2cc-4cc1-aca9-43f65735f528 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:25.410106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b745f20-c2cc-4cc1-aca9-43f65735f528 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:25.411583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b745f20-c2cc-4cc1-aca9-43f65735f528 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:25.414000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b745f20-c2cc-4cc1-aca9-43f65735f528 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:25.414726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b745f20-c2cc-4cc1-aca9-43f65735f528 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:25.415368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b745f20-c2cc-4cc1-aca9-43f65735f528 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:25.804832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7b745f20-c2cc-4cc1-aca9-43f65735f528 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:25.820454 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7b745f20-c2cc-4cc1-aca9-43f65735f528 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.440552 Sep 27 10:25:25.821091 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2514/5481] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:25 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:25.833543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:25.837954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:25.861134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:25.861659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:25.862127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:25.865347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:25.865863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:25.877460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:26.127303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:26.147769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:26.150678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:26.168746 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-323de48c-cb6b-40d9-826f-c2ed383ccffd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.340899 Sep 27 10:25:26.169953 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2515/5482] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:25 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:26.176083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-48b99057-385b-4905-b8de-023ccda6b19a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:26.176966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-48b99057-385b-4905-b8de-023ccda6b19a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:26.198103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48b99057-385b-4905-b8de-023ccda6b19a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:26.198719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48b99057-385b-4905-b8de-023ccda6b19a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:26.199191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48b99057-385b-4905-b8de-023ccda6b19a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:26.208071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48b99057-385b-4905-b8de-023ccda6b19a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:26.208594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48b99057-385b-4905-b8de-023ccda6b19a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:26.209086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48b99057-385b-4905-b8de-023ccda6b19a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:26.408484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:25:26.409149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:25:26.409748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:25:26.410159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:25:26.410714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:25:26.425717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:26.426160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:26.426866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:26.427603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:26.428062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:26.428495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:26.474565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:25:26.486834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for user 178956f6c8154a06a0710b295aff3086 and project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:25:26.530367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:25:26.533030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name='/dev/sda',device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id='b3c99c48-92e3-4590-aff4-92ad11f3158c',source_type='snapshot',tag=None,updated_at=,uuid=,volume_id=None,volume_size=1,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:25:26.578283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-48b99057-385b-4905-b8de-023ccda6b19a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:26.594060 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-48b99057-385b-4905-b8de-023ccda6b19a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.422785 Sep 27 10:25:26.595907 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2516/5483] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:26 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:26.626924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6cdec49a-5019-4aec-8b0c-29c55a20db7d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"lock": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:26.627925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6cdec49a-5019-4aec-8b0c-29c55a20db7d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:26.648299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cdec49a-5019-4aec-8b0c-29c55a20db7d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:26.649993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cdec49a-5019-4aec-8b0c-29c55a20db7d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:26.652212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cdec49a-5019-4aec-8b0c-29c55a20db7d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:26.653890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cdec49a-5019-4aec-8b0c-29c55a20db7d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:26.655847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cdec49a-5019-4aec-8b0c-29c55a20db7d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:26.657061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cdec49a-5019-4aec-8b0c-29c55a20db7d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:26.818443 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-29d3a756-7d4b-48a6-ac76-65f58ea00e60 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.643928 Sep 27 10:25:26.825731 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2968/5484] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:25:25 2022] POST /compute/v2.1/servers => generated 380 bytes in 1652 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:25:26.831486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6ddddea4-3b78-432f-8743-c74351a2f78b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:26.843316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6ddddea4-3b78-432f-8743-c74351a2f78b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:26.849906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6cdec49a-5019-4aec-8b0c-29c55a20db7d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Locking {{(pid=145969) lock /opt/stack/nova/nova/compute/api.py:4845}} Sep 27 10:25:26.949976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6ddddea4-3b78-432f-8743-c74351a2f78b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:26.958582 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6ddddea4-3b78-432f-8743-c74351a2f78b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1230 microversion: 2.1 time: 0.131399 Sep 27 10:25:26.977601 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2969/5485] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:26 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1230 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:27.014066 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6cdec49a-5019-4aec-8b0c-29c55a20db7d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.391468 Sep 27 10:25:27.015801 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2517/5486] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:26 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 395 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:25:27.024170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-358fc92b-c729-4c72-b27c-bd6e0e7ba885 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:27.025189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-358fc92b-c729-4c72-b27c-bd6e0e7ba885 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:27.038715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-358fc92b-c729-4c72-b27c-bd6e0e7ba885 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:27.039306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-358fc92b-c729-4c72-b27c-bd6e0e7ba885 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:27.041063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-358fc92b-c729-4c72-b27c-bd6e0e7ba885 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:27.041809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-358fc92b-c729-4c72-b27c-bd6e0e7ba885 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:27.045418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-358fc92b-c729-4c72-b27c-bd6e0e7ba885 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:27.045938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-358fc92b-c729-4c72-b27c-bd6e0e7ba885 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:27.211485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:27.211485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:27.225190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:27.225190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:27.225190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:27.225190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:27.225190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:27.225190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:27.423361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-358fc92b-c729-4c72-b27c-bd6e0e7ba885 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:27.450964 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-358fc92b-c729-4c72-b27c-bd6e0e7ba885 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.430537 Sep 27 10:25:27.451557 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2970/5487] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:27 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:27.487798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15b3f73e-fe8e-4894-b865-eef97cf2babf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:27.496027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15b3f73e-fe8e-4894-b865-eef97cf2babf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:27.498682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15b3f73e-fe8e-4894-b865-eef97cf2babf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:27.502797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15b3f73e-fe8e-4894-b865-eef97cf2babf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:27.573125 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-15b3f73e-fe8e-4894-b865-eef97cf2babf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] HTTP exception thrown: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b is locked Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15b3f73e-fe8e-4894-b865-eef97cf2babf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Request method failure captured: Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action HTTP/1.1 Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 15 Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs6KVh3H5X7rX3evxN11ucsVlmHnbL2MCObHKl1pQO1q40XS_7dvNdOvYhoxemUBx1Ms1xpNth_S22Rp1rV9hfITX6TEFNP6CCnYhe72LSUTyMJphfTwYhmOPX9p8o6YFOanCD6UQJX6ltkdsgee7CkTpD8vTeyh3fvEfCOyNJIfUWA Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_10375bd836f84383b39f2df73f53800e"}]}] Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 97c769dbba794531b0a9db8e764ff097 Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: {"os-stop": {}} Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b is locked Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '5ac6dabf-1315-4e05-ab6c-22211dc2440b', 'body': {'os-stop': {}}} Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1449, in _stop_server Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.stop(context, instance) Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 240, in inner Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceIsLocked(instance_uuid=instance.uuid) Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceIsLocked: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b is locked Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1451, in _stop_server Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=e.format_message()) Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b is locked Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b is locked Sep 27 10:25:27.582531 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:27.589753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15b3f73e-fe8e-4894-b865-eef97cf2babf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Returning 409 to user: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b is locked {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:25:27.593146 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-15b3f73e-fe8e-4894-b865-eef97cf2babf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 409 len: 107 microversion: 2.1 time: 0.115795 Sep 27 10:25:27.598478 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2971/5488] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:27 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 107 bytes in 120 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Sep 27 10:25:27.611498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5e57d817-e79d-4e72-9cc2-71aecf8dbd5e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"unlock": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:27.612398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5e57d817-e79d-4e72-9cc2-71aecf8dbd5e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:27.626029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e57d817-e79d-4e72-9cc2-71aecf8dbd5e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:27.626756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e57d817-e79d-4e72-9cc2-71aecf8dbd5e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:27.627541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e57d817-e79d-4e72-9cc2-71aecf8dbd5e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:27.628437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e57d817-e79d-4e72-9cc2-71aecf8dbd5e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:27.629119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e57d817-e79d-4e72-9cc2-71aecf8dbd5e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:27.630751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e57d817-e79d-4e72-9cc2-71aecf8dbd5e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:27.668581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:27.685198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:27.689143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:27.693126 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-444eab43-038a-4d0d-abc7-e6a22cf6f62c tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.493934 Sep 27 10:25:27.695135 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2518/5489] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:27 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:27.802703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5e57d817-e79d-4e72-9cc2-71aecf8dbd5e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Unlocking {{(pid=145968) unlock /opt/stack/nova/nova/compute/api.py:4874}} Sep 27 10:25:27.929770 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5e57d817-e79d-4e72-9cc2-71aecf8dbd5e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.324043 Sep 27 10:25:27.935129 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2972/5490] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:27 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 335 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:25:27.940314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e73a4ad4-4aab-4280-965a-856a0f00da8a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:27.950626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e73a4ad4-4aab-4280-965a-856a0f00da8a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:27.951550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e73a4ad4-4aab-4280-965a-856a0f00da8a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:27.952775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e73a4ad4-4aab-4280-965a-856a0f00da8a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:27.985907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aeb46253-7180-4409-93f7-89762aade645 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:27.986628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aeb46253-7180-4409-93f7-89762aade645 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:27.997955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aeb46253-7180-4409-93f7-89762aade645 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:27.998461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aeb46253-7180-4409-93f7-89762aade645 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:27.998896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aeb46253-7180-4409-93f7-89762aade645 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:28.000206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aeb46253-7180-4409-93f7-89762aade645 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:28.000695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aeb46253-7180-4409-93f7-89762aade645 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:28.001479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aeb46253-7180-4409-93f7-89762aade645 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:28.003064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e73a4ad4-4aab-4280-965a-856a0f00da8a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Going to try to stop instance {{(pid=145969) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 10:25:28.109813 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e73a4ad4-4aab-4280-965a-856a0f00da8a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.174365 Sep 27 10:25:28.111082 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2519/5491] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:27 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 177 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:25:28.126842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-44ab42a7-b9bf-477f-988c-74759f89d6ee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:28.127959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-44ab42a7-b9bf-477f-988c-74759f89d6ee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:28.130269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aeb46253-7180-4409-93f7-89762aade645 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:28.153984 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aeb46253-7180-4409-93f7-89762aade645 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1234 microversion: 2.1 time: 0.172259 Sep 27 10:25:28.154833 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2973/5492] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:27 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1234 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:28.171601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44ab42a7-b9bf-477f-988c-74759f89d6ee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:28.175553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44ab42a7-b9bf-477f-988c-74759f89d6ee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:28.178812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44ab42a7-b9bf-477f-988c-74759f89d6ee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:28.179812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44ab42a7-b9bf-477f-988c-74759f89d6ee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:28.181344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44ab42a7-b9bf-477f-988c-74759f89d6ee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:28.182058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44ab42a7-b9bf-477f-988c-74759f89d6ee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:28.425449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-44ab42a7-b9bf-477f-988c-74759f89d6ee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:28.448611 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-44ab42a7-b9bf-477f-988c-74759f89d6ee tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.331228 Sep 27 10:25:28.468692 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2520/5493] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:28 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:28.727473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:28.728967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:28.739890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:28.740395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:28.740895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:28.741623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:28.742089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:28.742519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:28.993549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:29.003753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:29.006015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:29.007246 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f7e1a919-e67b-41d0-b6b2-e33f907935be tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1738 microversion: 2.20 time: 0.288540 Sep 27 10:25:29.007823 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2974/5494] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:28 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1738 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:29.182240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-69d83b8d-2ba2-4ecc-bc8b-97caec153b7e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:29.182872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-69d83b8d-2ba2-4ecc-bc8b-97caec153b7e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:29.189520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d83b8d-2ba2-4ecc-bc8b-97caec153b7e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:29.190157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d83b8d-2ba2-4ecc-bc8b-97caec153b7e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:29.190848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d83b8d-2ba2-4ecc-bc8b-97caec153b7e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:29.191644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d83b8d-2ba2-4ecc-bc8b-97caec153b7e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:29.193091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d83b8d-2ba2-4ecc-bc8b-97caec153b7e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:29.193755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69d83b8d-2ba2-4ecc-bc8b-97caec153b7e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:29.284284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-69d83b8d-2ba2-4ecc-bc8b-97caec153b7e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:29.296177 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-69d83b8d-2ba2-4ecc-bc8b-97caec153b7e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1290 microversion: 2.1 time: 0.117865 Sep 27 10:25:29.309935 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2521/5495] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:29 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1290 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:29.474537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aae94e0b-dfca-467b-a915-aaaed450baaf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:29.475387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aae94e0b-dfca-467b-a915-aaaed450baaf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:29.485443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aae94e0b-dfca-467b-a915-aaaed450baaf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:29.485953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aae94e0b-dfca-467b-a915-aaaed450baaf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:29.486397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aae94e0b-dfca-467b-a915-aaaed450baaf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:29.489375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aae94e0b-dfca-467b-a915-aaaed450baaf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:29.491515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aae94e0b-dfca-467b-a915-aaaed450baaf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:29.492037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aae94e0b-dfca-467b-a915-aaaed450baaf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:29.774951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aae94e0b-dfca-467b-a915-aaaed450baaf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:29.786491 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aae94e0b-dfca-467b-a915-aaaed450baaf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.316889 Sep 27 10:25:29.791401 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2975/5496] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:29 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:30.036682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:30.037515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:30.054969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:30.055396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:30.055873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:30.058812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:30.059369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:30.059837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:30.323788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e1fbea5-9a99-4f17-a4f8-e7c0bb53dacd tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:30.324686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7e1fbea5-9a99-4f17-a4f8-e7c0bb53dacd tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:30.342165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e1fbea5-9a99-4f17-a4f8-e7c0bb53dacd tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:30.343231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e1fbea5-9a99-4f17-a4f8-e7c0bb53dacd tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:30.344539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e1fbea5-9a99-4f17-a4f8-e7c0bb53dacd tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:30.348406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e1fbea5-9a99-4f17-a4f8-e7c0bb53dacd tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:30.359434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e1fbea5-9a99-4f17-a4f8-e7c0bb53dacd tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:30.361154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e1fbea5-9a99-4f17-a4f8-e7c0bb53dacd tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:30.544566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7e1fbea5-9a99-4f17-a4f8-e7c0bb53dacd tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:30.558466 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7e1fbea5-9a99-4f17-a4f8-e7c0bb53dacd tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1300 microversion: 2.1 time: 0.236699 Sep 27 10:25:30.569262 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2976/5497] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:30 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1300 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:30.583782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:30.597073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:30.603600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:30.605224 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e20b49f4-f798-45c6-a5f4-292f03708010 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1720 microversion: 2.20 time: 0.574153 Sep 27 10:25:30.607425 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2522/5498] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:30 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1720 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:30.816080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dd8cb9d1-6f19-408a-9cc2-ee03da857fa6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:30.817597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dd8cb9d1-6f19-408a-9cc2-ee03da857fa6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:30.834876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd8cb9d1-6f19-408a-9cc2-ee03da857fa6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:30.836185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd8cb9d1-6f19-408a-9cc2-ee03da857fa6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:30.837577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd8cb9d1-6f19-408a-9cc2-ee03da857fa6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:30.839483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd8cb9d1-6f19-408a-9cc2-ee03da857fa6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:30.844086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd8cb9d1-6f19-408a-9cc2-ee03da857fa6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:30.846814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd8cb9d1-6f19-408a-9cc2-ee03da857fa6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:30.894814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:31.340087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dd8cb9d1-6f19-408a-9cc2-ee03da857fa6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:31.356356 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dd8cb9d1-6f19-408a-9cc2-ee03da857fa6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.544768 Sep 27 10:25:31.372678 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2977/5499] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:30 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:31.586360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-15349821-f6ee-4284-925a-90b7a925789f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:31.587229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-15349821-f6ee-4284-925a-90b7a925789f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:31.619298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15349821-f6ee-4284-925a-90b7a925789f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:31.620263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15349821-f6ee-4284-925a-90b7a925789f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.021s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:31.621015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15349821-f6ee-4284-925a-90b7a925789f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:31.622688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15349821-f6ee-4284-925a-90b7a925789f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:31.624569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15349821-f6ee-4284-925a-90b7a925789f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:31.625224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15349821-f6ee-4284-925a-90b7a925789f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:31.638089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:31.638850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:31.650612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:31.651119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:31.651599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:31.655630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:31.656327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:31.656977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:31.662686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:UPDATE=9,nova_cell1:SELECT=87,nova_cell1:INSERT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:31.944168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:31.962288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:31.964063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:31.965518 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a243775c-f8f3-42ee-981f-e97f9eabee71 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1720 microversion: 2.20 time: 0.331583 Sep 27 10:25:31.969551 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2978/5500] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:31 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1720 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:25:32.102851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-15349821-f6ee-4284-925a-90b7a925789f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:32.117270 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-15349821-f6ee-4284-925a-90b7a925789f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1342 microversion: 2.1 time: 0.536229 Sep 27 10:25:32.118150 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2523/5501] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:31 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1342 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:32.379899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b593a752-1918-431d-9c5f-da4e77e60099 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:32.380577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b593a752-1918-431d-9c5f-da4e77e60099 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:32.387829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593a752-1918-431d-9c5f-da4e77e60099 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:32.388306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593a752-1918-431d-9c5f-da4e77e60099 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:32.389010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593a752-1918-431d-9c5f-da4e77e60099 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:32.389950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593a752-1918-431d-9c5f-da4e77e60099 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:32.390777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593a752-1918-431d-9c5f-da4e77e60099 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:32.391362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b593a752-1918-431d-9c5f-da4e77e60099 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:32.999497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:33.000440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:33.008382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:33.009748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:33.010607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:33.011367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:33.011855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:33.012283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:33.104750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b593a752-1918-431d-9c5f-da4e77e60099 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:33.125524 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b593a752-1918-431d-9c5f-da4e77e60099 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.741843 Sep 27 10:25:33.127615 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2979/5502] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:32 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 752 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:33.135310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-38238543-420b-4854-893b-e31a0db99a2f req-4e40fc8b-ddae-4e7f-a6a7-199e5677dd3a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "04dc4859-cd17-4187-8019-e2bcf346a7df", "tag": "e72ee7df-caa1-44c6-88f1-3281aefb5154"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:33.149878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-38238543-420b-4854-893b-e31a0db99a2f req-4e40fc8b-ddae-4e7f-a6a7-199e5677dd3a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:33.150761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-38238543-420b-4854-893b-e31a0db99a2f req-4e40fc8b-ddae-4e7f-a6a7-199e5677dd3a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:33.151483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-38238543-420b-4854-893b-e31a0db99a2f req-4e40fc8b-ddae-4e7f-a6a7-199e5677dd3a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:33.177141 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-38238543-420b-4854-893b-e31a0db99a2f req-4e40fc8b-ddae-4e7f-a6a7-199e5677dd3a service nova] Creating event network-changed:e72ee7df-caa1-44c6-88f1-3281aefb5154 for instance 04dc4859-cd17-4187-8019-e2bcf346a7df on nch1-master-1 Sep 27 10:25:33.185197 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-38238543-420b-4854-893b-e31a0db99a2f req-4e40fc8b-ddae-4e7f-a6a7-199e5677dd3a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054398 Sep 27 10:25:33.186802 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2980/5503] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:33.193142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6cab8de7-bea9-41e2-919e-73ae886838ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:33.194337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6cab8de7-bea9-41e2-919e-73ae886838ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:33.196583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:33.203307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6cab8de7-bea9-41e2-919e-73ae886838ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:33.206450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6cab8de7-bea9-41e2-919e-73ae886838ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:33.207318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6cab8de7-bea9-41e2-919e-73ae886838ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:33.208317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6cab8de7-bea9-41e2-919e-73ae886838ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:33.208763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:33.209627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6cab8de7-bea9-41e2-919e-73ae886838ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:33.210917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6cab8de7-bea9-41e2-919e-73ae886838ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:33.211031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:33.212343 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8355fc16-3a38-400e-8cdc-752f7aeda221 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1720 microversion: 2.20 time: 0.217163 Sep 27 10:25:33.228810 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2524/5504] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:32 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1720 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:33.240220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d0570637-6264-4305-8706-1c707262341d req-8d54e27f-120e-4f09-8ea9-749569f8959a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77037d39-844c-4761-88cc-899710443401", "name": "network-vif-unplugged", "status": "completed", "tag": "6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:33.252837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d0570637-6264-4305-8706-1c707262341d req-8d54e27f-120e-4f09-8ea9-749569f8959a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:33.253666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d0570637-6264-4305-8706-1c707262341d req-8d54e27f-120e-4f09-8ea9-749569f8959a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:33.254529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d0570637-6264-4305-8706-1c707262341d req-8d54e27f-120e-4f09-8ea9-749569f8959a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:33.274851 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d0570637-6264-4305-8706-1c707262341d req-8d54e27f-120e-4f09-8ea9-749569f8959a service nova] Creating event network-vif-unplugged:6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1 for instance 77037d39-844c-4761-88cc-899710443401 on nch2-master-1 Sep 27 10:25:33.282636 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d0570637-6264-4305-8706-1c707262341d req-8d54e27f-120e-4f09-8ea9-749569f8959a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.052428 Sep 27 10:25:33.283678 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2525/5505] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:33 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:33.588332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6cab8de7-bea9-41e2-919e-73ae886838ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:33.596119 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6cab8de7-bea9-41e2-919e-73ae886838ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1342 microversion: 2.1 time: 0.407840 Sep 27 10:25:33.596713 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2981/5506] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:33 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1342 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:34.153706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-281d8e88-5fcb-4f10-ba2e-1cad56bac411 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:34.154539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-281d8e88-5fcb-4f10-ba2e-1cad56bac411 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:34.167260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-281d8e88-5fcb-4f10-ba2e-1cad56bac411 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:34.170485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-281d8e88-5fcb-4f10-ba2e-1cad56bac411 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:34.173282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-281d8e88-5fcb-4f10-ba2e-1cad56bac411 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:34.174322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-281d8e88-5fcb-4f10-ba2e-1cad56bac411 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:34.177362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-281d8e88-5fcb-4f10-ba2e-1cad56bac411 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:34.178101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-281d8e88-5fcb-4f10-ba2e-1cad56bac411 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:34.247549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:34.248170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:34.259775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:34.261321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:34.262705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:34.263505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:34.263907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:34.266828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:34.517515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-281d8e88-5fcb-4f10-ba2e-1cad56bac411 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:34.526679 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-281d8e88-5fcb-4f10-ba2e-1cad56bac411 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.377520 Sep 27 10:25:34.531215 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2526/5507] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:34 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:34.627829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1357885d-8a33-4258-a429-b9cd7bb62ff9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:34.628471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1357885d-8a33-4258-a429-b9cd7bb62ff9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:34.638540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1357885d-8a33-4258-a429-b9cd7bb62ff9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:34.641316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1357885d-8a33-4258-a429-b9cd7bb62ff9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:34.641952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1357885d-8a33-4258-a429-b9cd7bb62ff9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:34.642849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1357885d-8a33-4258-a429-b9cd7bb62ff9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:34.645210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1357885d-8a33-4258-a429-b9cd7bb62ff9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:34.645841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1357885d-8a33-4258-a429-b9cd7bb62ff9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:34.869137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1357885d-8a33-4258-a429-b9cd7bb62ff9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:34.878767 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1357885d-8a33-4258-a429-b9cd7bb62ff9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1342 microversion: 2.1 time: 0.256589 Sep 27 10:25:34.888698 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2527/5508] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:34 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1342 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:34.936241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:34.946954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:34.948697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:34.950265 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-619c060c-068b-4311-8504-d61b1ea03082 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1723 microversion: 2.20 time: 0.710282 Sep 27 10:25:34.951217 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2982/5509] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:34 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1723 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:34.977127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2083e1cb-c441-468e-81f2-8bd1f1c5be91 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d558400b-8728-4133-8e2d-d76be2e106d8"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:34.980115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2083e1cb-c441-468e-81f2-8bd1f1c5be91 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:34.987161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2083e1cb-c441-468e-81f2-8bd1f1c5be91 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:34.987664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2083e1cb-c441-468e-81f2-8bd1f1c5be91 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:34.988193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2083e1cb-c441-468e-81f2-8bd1f1c5be91 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:34.997103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2083e1cb-c441-468e-81f2-8bd1f1c5be91 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:34.997919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2083e1cb-c441-468e-81f2-8bd1f1c5be91 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:34.998565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2083e1cb-c441-468e-81f2-8bd1f1c5be91 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:35.420602 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2083e1cb-c441-468e-81f2-8bd1f1c5be91 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "POST /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.447827 Sep 27 10:25:35.421718 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2528/5510] 10.222.0.26 () {62 vars in 1364 bytes} [Tue Sep 27 10:25:34 2022] POST /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/os-volume_attachments => generated 188 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 10:25:35.554217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-71d824a1-74a9-441a-902a-e21b6b7e767e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:35.555140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-71d824a1-74a9-441a-902a-e21b6b7e767e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:35.560181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4aa9143b-fcef-4b76-91bc-67178b911c6f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:35.560773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4aa9143b-fcef-4b76-91bc-67178b911c6f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:35.573588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aa9143b-fcef-4b76-91bc-67178b911c6f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:35.574269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aa9143b-fcef-4b76-91bc-67178b911c6f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:35.574903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aa9143b-fcef-4b76-91bc-67178b911c6f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:35.575924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aa9143b-fcef-4b76-91bc-67178b911c6f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:35.576560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aa9143b-fcef-4b76-91bc-67178b911c6f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:35.577204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aa9143b-fcef-4b76-91bc-67178b911c6f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:35.578250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d824a1-74a9-441a-902a-e21b6b7e767e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:35.578748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d824a1-74a9-441a-902a-e21b6b7e767e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:35.581209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d824a1-74a9-441a-902a-e21b6b7e767e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:35.581953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d824a1-74a9-441a-902a-e21b6b7e767e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:35.582455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d824a1-74a9-441a-902a-e21b6b7e767e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:35.582886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71d824a1-74a9-441a-902a-e21b6b7e767e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:35.900017 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4aa9143b-fcef-4b76-91bc-67178b911c6f tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "DELETE /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/os-volume_attachments/d558400b-8728-4133-8e2d-d76be2e106d8" status: 202 len: 0 microversion: 2.20 time: 0.343328 Sep 27 10:25:35.900708 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2529/5511] 10.222.0.26 () {60 vars in 1457 bytes} [Tue Sep 27 10:25:35 2022] DELETE /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/os-volume_attachments/d558400b-8728-4133-8e2d-d76be2e106d8 => generated 0 bytes in 345 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:25:35.905998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6b15ac5d-9696-4e38-9ca1-72b8e9a17b9d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:35.906783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6b15ac5d-9696-4e38-9ca1-72b8e9a17b9d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:35.914240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b15ac5d-9696-4e38-9ca1-72b8e9a17b9d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:35.914935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b15ac5d-9696-4e38-9ca1-72b8e9a17b9d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:35.915387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b15ac5d-9696-4e38-9ca1-72b8e9a17b9d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:35.916117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b15ac5d-9696-4e38-9ca1-72b8e9a17b9d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:35.916583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b15ac5d-9696-4e38-9ca1-72b8e9a17b9d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:35.917049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b15ac5d-9696-4e38-9ca1-72b8e9a17b9d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:35.963892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-71d824a1-74a9-441a-902a-e21b6b7e767e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:35.971864 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-71d824a1-74a9-441a-902a-e21b6b7e767e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.421666 Sep 27 10:25:35.972461 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2983/5512] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:35 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:35.981942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e365d5d6-7da0-4c0d-8f38-da73d53a88d5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:35.982948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e365d5d6-7da0-4c0d-8f38-da73d53a88d5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:35.992958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e365d5d6-7da0-4c0d-8f38-da73d53a88d5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:35.993611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e365d5d6-7da0-4c0d-8f38-da73d53a88d5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:35.994107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e365d5d6-7da0-4c0d-8f38-da73d53a88d5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:35.994899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e365d5d6-7da0-4c0d-8f38-da73d53a88d5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:35.995447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e365d5d6-7da0-4c0d-8f38-da73d53a88d5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:35.996006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e365d5d6-7da0-4c0d-8f38-da73d53a88d5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:36.032965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e365d5d6-7da0-4c0d-8f38-da73d53a88d5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=145968) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 27 10:25:36.082846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6b15ac5d-9696-4e38-9ca1-72b8e9a17b9d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:36.092286 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6b15ac5d-9696-4e38-9ca1-72b8e9a17b9d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1376 microversion: 2.1 time: 0.189175 Sep 27 10:25:36.093425 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2530/5513] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:35 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1376 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:36.136397 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e365d5d6-7da0-4c0d-8f38-da73d53a88d5 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "POST /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/action" status: 202 len: 0 microversion: 2.20 time: 0.158221 Sep 27 10:25:36.137915 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2984/5514] 10.222.0.26 () {62 vars in 1319 bytes} [Tue Sep 27 10:25:35 2022] POST /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/action => generated 0 bytes in 160 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:25:36.149217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:36.149851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:36.157196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:36.158216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:36.158918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:36.159969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:36.160891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:36.161666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:36.344246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:36.362152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:36.365110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:36.366782 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-66b8ca4b-f00f-49e2-b0dc-482db10da302 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1671 microversion: 2.20 time: 0.222472 Sep 27 10:25:36.367438 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2531/5515] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:36 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1671 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:37.003173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d791c4c-32b5-42df-8fc2-95747c0793ae tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:37.006306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2d791c4c-32b5-42df-8fc2-95747c0793ae tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:37.008546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:37.010538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:37.016320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d791c4c-32b5-42df-8fc2-95747c0793ae tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:37.017053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d791c4c-32b5-42df-8fc2-95747c0793ae tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:37.017500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d791c4c-32b5-42df-8fc2-95747c0793ae tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:37.023787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d791c4c-32b5-42df-8fc2-95747c0793ae tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:37.023787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d791c4c-32b5-42df-8fc2-95747c0793ae tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:37.023787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d791c4c-32b5-42df-8fc2-95747c0793ae tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:37.117266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9729a21d-bf04-4649-84af-bea142d26ef4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:37.118006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9729a21d-bf04-4649-84af-bea142d26ef4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:37.128014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9729a21d-bf04-4649-84af-bea142d26ef4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:37.131358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9729a21d-bf04-4649-84af-bea142d26ef4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:37.131952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9729a21d-bf04-4649-84af-bea142d26ef4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:37.132869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9729a21d-bf04-4649-84af-bea142d26ef4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:37.135110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9729a21d-bf04-4649-84af-bea142d26ef4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:37.136343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9729a21d-bf04-4649-84af-bea142d26ef4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:37.181176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2d791c4c-32b5-42df-8fc2-95747c0793ae tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:37.191310 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d791c4c-32b5-42df-8fc2-95747c0793ae tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.195350 Sep 27 10:25:37.205514 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2985/5516] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:36 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:37.401010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:37.401648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:37.416147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:37.417418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:37.417418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:37.418505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:37.418665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:37.419283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:37.491694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9729a21d-bf04-4649-84af-bea142d26ef4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:37.502352 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9729a21d-bf04-4649-84af-bea142d26ef4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1376 microversion: 2.1 time: 0.390103 Sep 27 10:25:37.503293 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2532/5517] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:37 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1376 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:37.621884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:37.633986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:37.635874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:37.637430 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9a5bce79-6d58-44ca-b0e8-5ce11276d681 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1671 microversion: 2.20 time: 0.241577 Sep 27 10:25:37.638491 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2986/5518] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:37 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1671 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:37.857327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-de12a93c-ad0a-45a9-b74b-5221780756ff tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:37.858937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-de12a93c-ad0a-45a9-b74b-5221780756ff tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:37.880216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de12a93c-ad0a-45a9-b74b-5221780756ff tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:37.880959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de12a93c-ad0a-45a9-b74b-5221780756ff tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:37.883139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de12a93c-ad0a-45a9-b74b-5221780756ff tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:37.884051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de12a93c-ad0a-45a9-b74b-5221780756ff tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:37.884709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de12a93c-ad0a-45a9-b74b-5221780756ff tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:37.885505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de12a93c-ad0a-45a9-b74b-5221780756ff tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:37.993200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-de12a93c-ad0a-45a9-b74b-5221780756ff tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lazy-loading 'services' on Instance uuid 3db53022-2562-4d3c-a0e1-9d9162d5e1ac {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:25:38.088350 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-de12a93c-ad0a-45a9-b74b-5221780756ff tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac/os-volume_attachments/bb8280cc-ed98-4857-a90d-aef277d7bdaf" status: 202 len: 0 microversion: 2.1 time: 0.244023 Sep 27 10:25:38.089036 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2533/5519] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:25:37 2022] DELETE /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac/os-volume_attachments/bb8280cc-ed98-4857-a90d-aef277d7bdaf => generated 0 bytes in 249 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:25:38.217057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ca6f805e-c714-44ac-84db-3eccc64269fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:38.218482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ca6f805e-c714-44ac-84db-3eccc64269fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:38.229670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:38.232512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca6f805e-c714-44ac-84db-3eccc64269fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:38.233326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca6f805e-c714-44ac-84db-3eccc64269fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:38.234026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca6f805e-c714-44ac-84db-3eccc64269fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:38.235423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca6f805e-c714-44ac-84db-3eccc64269fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:38.237024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca6f805e-c714-44ac-84db-3eccc64269fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:38.238574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ca6f805e-c714-44ac-84db-3eccc64269fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:38.389103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ca6f805e-c714-44ac-84db-3eccc64269fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:38.397988 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ca6f805e-c714-44ac-84db-3eccc64269fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.185213 Sep 27 10:25:38.398703 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2987/5520] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:38 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:38.533205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fc9fe29c-c329-4530-a218-f8cd1877961d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:38.534550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fc9fe29c-c329-4530-a218-f8cd1877961d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:38.542259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=88 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:38.544524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9fe29c-c329-4530-a218-f8cd1877961d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:38.545444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9fe29c-c329-4530-a218-f8cd1877961d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:38.546338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9fe29c-c329-4530-a218-f8cd1877961d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:38.547272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9fe29c-c329-4530-a218-f8cd1877961d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:38.547891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9fe29c-c329-4530-a218-f8cd1877961d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:38.548551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fc9fe29c-c329-4530-a218-f8cd1877961d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:38.667168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:38.667674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:38.677979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:38.678486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:38.678930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:38.679657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:38.682823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:38.687738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:38.968714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fc9fe29c-c329-4530-a218-f8cd1877961d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:38.972325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=86,nova_cell1:INSERT=6,nova_cell1:UPDATE=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:38.985815 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fc9fe29c-c329-4530-a218-f8cd1877961d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1376 microversion: 2.1 time: 0.460287 Sep 27 10:25:38.986491 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2534/5521] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:38 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1376 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:39.023302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:39.037350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:39.039338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:39.041230 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69e233be-541b-49d9-9a99-8bf60ed1fafd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.378494 Sep 27 10:25:39.044713 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2988/5522] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:38 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:25:39.255219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cfb466e2-6ec0-42de-a7c5-3abb05848745 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:39.255869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cfb466e2-6ec0-42de-a7c5-3abb05848745 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:39.266548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfb466e2-6ec0-42de-a7c5-3abb05848745 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:39.267302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfb466e2-6ec0-42de-a7c5-3abb05848745 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:39.267932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfb466e2-6ec0-42de-a7c5-3abb05848745 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:39.268754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfb466e2-6ec0-42de-a7c5-3abb05848745 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:39.269237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfb466e2-6ec0-42de-a7c5-3abb05848745 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:39.271160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfb466e2-6ec0-42de-a7c5-3abb05848745 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:39.325370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cfb466e2-6ec0-42de-a7c5-3abb05848745 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:25:39.422706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9575a2b-88d9-4dc6-8e75-4ec6fd487fb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:39.424079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c9575a2b-88d9-4dc6-8e75-4ec6fd487fb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:39.433184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9575a2b-88d9-4dc6-8e75-4ec6fd487fb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:39.434109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9575a2b-88d9-4dc6-8e75-4ec6fd487fb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:39.434898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9575a2b-88d9-4dc6-8e75-4ec6fd487fb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:39.435983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9575a2b-88d9-4dc6-8e75-4ec6fd487fb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:39.436675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9575a2b-88d9-4dc6-8e75-4ec6fd487fb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:39.437922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9575a2b-88d9-4dc6-8e75-4ec6fd487fb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:39.488585 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cfb466e2-6ec0-42de-a7c5-3abb05848745 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 204 len: 0 microversion: 2.1 time: 0.238203 Sep 27 10:25:39.490354 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2535/5523] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:25:39 2022] DELETE /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 0 bytes in 241 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:25:39.502464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2764d931-7092-458a-8751-f20ace312b3d tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:39.503865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2764d931-7092-458a-8751-f20ace312b3d tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:39.541529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2764d931-7092-458a-8751-f20ace312b3d tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:39.542275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2764d931-7092-458a-8751-f20ace312b3d tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:39.546779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2764d931-7092-458a-8751-f20ace312b3d tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:39.547805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2764d931-7092-458a-8751-f20ace312b3d tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:39.548681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2764d931-7092-458a-8751-f20ace312b3d tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:39.549169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2764d931-7092-458a-8751-f20ace312b3d tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:39.679944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9575a2b-88d9-4dc6-8e75-4ec6fd487fb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:39.698755 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9575a2b-88d9-4dc6-8e75-4ec6fd487fb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1656 microversion: 2.1 time: 0.280506 Sep 27 10:25:39.700209 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2989/5524] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:39 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1656 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:39.732122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f1a3ad41-5f4d-432e-8064-b7d1937806d2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:39.740940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1a3ad41-5f4d-432e-8064-b7d1937806d2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:39.741904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1a3ad41-5f4d-432e-8064-b7d1937806d2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:39.742927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1a3ad41-5f4d-432e-8064-b7d1937806d2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:39.799559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f1a3ad41-5f4d-432e-8064-b7d1937806d2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Going to try to start instance {{(pid=145968) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 27 10:25:39.922075 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f1a3ad41-5f4d-432e-8064-b7d1937806d2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.179081 Sep 27 10:25:39.922782 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2990/5525] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:39 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 198 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:25:39.942682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2764d931-7092-458a-8751-f20ace312b3d tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:39.967661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9222e31c-a23d-4d34-b526-b731caafa14f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:39.970216 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2764d931-7092-458a-8751-f20ace312b3d tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1470 microversion: 2.1 time: 0.473571 Sep 27 10:25:39.970887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9222e31c-a23d-4d34-b526-b731caafa14f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:39.976976 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2536/5526] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:39 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1470 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:40.003262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9222e31c-a23d-4d34-b526-b731caafa14f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:40.006195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9222e31c-a23d-4d34-b526-b731caafa14f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:40.006899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9222e31c-a23d-4d34-b526-b731caafa14f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:40.009022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9222e31c-a23d-4d34-b526-b731caafa14f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:40.009575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9222e31c-a23d-4d34-b526-b731caafa14f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:40.011974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9222e31c-a23d-4d34-b526-b731caafa14f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:40.016077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-439555f0-8ef7-41fd-be0f-c1577f987048 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:40.016750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-439555f0-8ef7-41fd-be0f-c1577f987048 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:40.035079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-439555f0-8ef7-41fd-be0f-c1577f987048 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:40.040120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-439555f0-8ef7-41fd-be0f-c1577f987048 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:40.040973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-439555f0-8ef7-41fd-be0f-c1577f987048 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:40.044401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-439555f0-8ef7-41fd-be0f-c1577f987048 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:40.047229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-439555f0-8ef7-41fd-be0f-c1577f987048 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:40.049207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-439555f0-8ef7-41fd-be0f-c1577f987048 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:40.406912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-439555f0-8ef7-41fd-be0f-c1577f987048 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:40.421834 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-439555f0-8ef7-41fd-be0f-c1577f987048 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1376 microversion: 2.1 time: 0.407586 Sep 27 10:25:40.423472 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2537/5527] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:40 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1376 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:40.435110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:40.439322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:40.456612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:40.457068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:40.460892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:40.461874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:40.462553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:40.464005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:40.804320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9222e31c-a23d-4d34-b526-b731caafa14f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:40.814529 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9222e31c-a23d-4d34-b526-b731caafa14f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1665 microversion: 2.1 time: 0.883972 Sep 27 10:25:40.816154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:40.824303 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2991/5528] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:39 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1665 bytes in 896 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:40.825841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:40.827312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:40.828517 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c71f1a5-96e9-4fae-9046-308ac56e6873 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.398118 Sep 27 10:25:40.830321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-22f4b5cd-3476-43b5-920a-84365d410e2a req-cc0a6631-3b69-4452-9a87-31602bc27fef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77037d39-844c-4761-88cc-899710443401", "tag": "6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:40.848698 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2538/5529] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:40 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:40.856855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-22f4b5cd-3476-43b5-920a-84365d410e2a req-cc0a6631-3b69-4452-9a87-31602bc27fef service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:40.857623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-22f4b5cd-3476-43b5-920a-84365d410e2a req-cc0a6631-3b69-4452-9a87-31602bc27fef service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:40.859394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-22f4b5cd-3476-43b5-920a-84365d410e2a req-cc0a6631-3b69-4452-9a87-31602bc27fef service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:40.874461 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-22f4b5cd-3476-43b5-920a-84365d410e2a req-cc0a6631-3b69-4452-9a87-31602bc27fef service nova] Creating event network-changed:6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1 for instance 77037d39-844c-4761-88cc-899710443401 on nch1-master-1 Sep 27 10:25:40.884493 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-22f4b5cd-3476-43b5-920a-84365d410e2a req-cc0a6631-3b69-4452-9a87-31602bc27fef service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.058490 Sep 27 10:25:40.885592 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2992/5530] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:40 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:41.004409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58f5d7ae-45b9-45ba-87d7-0a5df007bca3 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:41.005561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-58f5d7ae-45b9-45ba-87d7-0a5df007bca3 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:41.016374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5d7ae-45b9-45ba-87d7-0a5df007bca3 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:41.017093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5d7ae-45b9-45ba-87d7-0a5df007bca3 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:41.017701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5d7ae-45b9-45ba-87d7-0a5df007bca3 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:41.018632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5d7ae-45b9-45ba-87d7-0a5df007bca3 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:41.019269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5d7ae-45b9-45ba-87d7-0a5df007bca3 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:41.019944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5d7ae-45b9-45ba-87d7-0a5df007bca3 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:41.449085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3e4514d4-fc74-4b34-8ee4-f6b693c0527c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:41.450086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3e4514d4-fc74-4b34-8ee4-f6b693c0527c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:41.463036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4514d4-fc74-4b34-8ee4-f6b693c0527c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:41.463616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4514d4-fc74-4b34-8ee4-f6b693c0527c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:41.464938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4514d4-fc74-4b34-8ee4-f6b693c0527c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:41.466093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4514d4-fc74-4b34-8ee4-f6b693c0527c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:41.466551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4514d4-fc74-4b34-8ee4-f6b693c0527c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:41.466725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e4514d4-fc74-4b34-8ee4-f6b693c0527c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:41.516553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:41.962316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-58f5d7ae-45b9-45ba-87d7-0a5df007bca3 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:41.975645 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-58f5d7ae-45b9-45ba-87d7-0a5df007bca3 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1470 microversion: 2.1 time: 0.973910 Sep 27 10:25:41.977483 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2539/5531] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:40 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1470 bytes in 979 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:41.987334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-4f92830a-aef6-4371-8850-8cd8c9e4f666 req-0b68c61d-04e4-4a1b-88f0-fc762f66dba3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "04dc4859-cd17-4187-8019-e2bcf346a7df", "name": "network-vif-plugged", "status": "completed", "tag": "e72ee7df-caa1-44c6-88f1-3281aefb5154"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:42.014183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4f92830a-aef6-4371-8850-8cd8c9e4f666 req-0b68c61d-04e4-4a1b-88f0-fc762f66dba3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:42.016734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4f92830a-aef6-4371-8850-8cd8c9e4f666 req-0b68c61d-04e4-4a1b-88f0-fc762f66dba3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:42.017548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4f92830a-aef6-4371-8850-8cd8c9e4f666 req-0b68c61d-04e4-4a1b-88f0-fc762f66dba3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:42.049015 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-4f92830a-aef6-4371-8850-8cd8c9e4f666 req-0b68c61d-04e4-4a1b-88f0-fc762f66dba3 service nova] Creating event network-vif-plugged:e72ee7df-caa1-44c6-88f1-3281aefb5154 for instance 04dc4859-cd17-4187-8019-e2bcf346a7df on nch1-master-1 Sep 27 10:25:42.062617 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-4f92830a-aef6-4371-8850-8cd8c9e4f666 req-0b68c61d-04e4-4a1b-88f0-fc762f66dba3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082768 Sep 27 10:25:42.066796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3e4514d4-fc74-4b34-8ee4-f6b693c0527c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:42.067948 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2540/5532] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:42.075644 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3e4514d4-fc74-4b34-8ee4-f6b693c0527c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1376 microversion: 2.1 time: 0.632703 Sep 27 10:25:42.076929 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2993/5533] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:41 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1376 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:42.082844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:42.083799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:42.088355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80fbe786-e510-4505-bf45-d84b234edb93 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:42.089119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-80fbe786-e510-4505-bf45-d84b234edb93 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:42.095832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fbe786-e510-4505-bf45-d84b234edb93 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:42.096539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fbe786-e510-4505-bf45-d84b234edb93 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:42.097229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fbe786-e510-4505-bf45-d84b234edb93 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:42.099157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:42.099994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:42.100754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fbe786-e510-4505-bf45-d84b234edb93 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:42.101097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:42.101728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fbe786-e510-4505-bf45-d84b234edb93 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:42.102156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:42.102394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80fbe786-e510-4505-bf45-d84b234edb93 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:42.102842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:42.103418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:42.439440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:42.475460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:42.477680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:42.480352 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-19be864d-096c-48d0-bf5e-a6832c133f92 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.402149 Sep 27 10:25:42.482116 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2994/5534] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:42 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:42.547376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-80fbe786-e510-4505-bf45-d84b234edb93 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:42.557748 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-80fbe786-e510-4505-bf45-d84b234edb93 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1665 microversion: 2.1 time: 0.488206 Sep 27 10:25:42.571482 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2541/5535] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:42 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1665 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:43.001174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-658e5bcb-2d16-4b43-a072-26fd80786052 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:43.001805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-658e5bcb-2d16-4b43-a072-26fd80786052 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:43.013717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658e5bcb-2d16-4b43-a072-26fd80786052 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:43.015027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658e5bcb-2d16-4b43-a072-26fd80786052 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:43.015477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658e5bcb-2d16-4b43-a072-26fd80786052 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:43.018817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658e5bcb-2d16-4b43-a072-26fd80786052 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:43.019432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658e5bcb-2d16-4b43-a072-26fd80786052 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:43.019918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-658e5bcb-2d16-4b43-a072-26fd80786052 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:43.100912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-961182d8-b728-4da6-b163-d99a3fb2d957 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:43.102285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-961182d8-b728-4da6-b163-d99a3fb2d957 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:43.118257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-961182d8-b728-4da6-b163-d99a3fb2d957 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:43.118783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-961182d8-b728-4da6-b163-d99a3fb2d957 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:43.119250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-961182d8-b728-4da6-b163-d99a3fb2d957 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:43.129043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-961182d8-b728-4da6-b163-d99a3fb2d957 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:43.130008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-961182d8-b728-4da6-b163-d99a3fb2d957 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:43.135589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-961182d8-b728-4da6-b163-d99a3fb2d957 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:43.256523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-658e5bcb-2d16-4b43-a072-26fd80786052 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcfed06aa6b4177a4287064884125b2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '430555edbc87497db7d7f79f2b891c04', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:43.269807 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-658e5bcb-2d16-4b43-a072-26fd80786052 tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 200 len: 1428 microversion: 2.1 time: 0.272846 Sep 27 10:25:43.273376 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2995/5536] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:42 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 1428 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:43.283199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7fef61d8-9230-4b67-8e84-f8df57fc6e4f req-fad3ba5f-8470-4533-ba6c-8adcd39b58e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3db53022-2562-4d3c-a0e1-9d9162d5e1ac", "name": "network-vif-deleted", "tag": "64db7780-37ee-4c1d-95f1-58bf82537877"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:43.307815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7fef61d8-9230-4b67-8e84-f8df57fc6e4f req-fad3ba5f-8470-4533-ba6c-8adcd39b58e7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:43.308452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7fef61d8-9230-4b67-8e84-f8df57fc6e4f req-fad3ba5f-8470-4533-ba6c-8adcd39b58e7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:43.323478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7fef61d8-9230-4b67-8e84-f8df57fc6e4f req-fad3ba5f-8470-4533-ba6c-8adcd39b58e7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:43.372292 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7fef61d8-9230-4b67-8e84-f8df57fc6e4f req-fad3ba5f-8470-4533-ba6c-8adcd39b58e7 service nova] Creating event network-vif-deleted:64db7780-37ee-4c1d-95f1-58bf82537877 for instance 3db53022-2562-4d3c-a0e1-9d9162d5e1ac on nch2-master-1 Sep 27 10:25:43.383620 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7fef61d8-9230-4b67-8e84-f8df57fc6e4f req-fad3ba5f-8470-4533-ba6c-8adcd39b58e7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.108603 Sep 27 10:25:43.386980 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2996/5537] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:43.444151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-961182d8-b728-4da6-b163-d99a3fb2d957 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:43.453445 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-961182d8-b728-4da6-b163-d99a3fb2d957 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1376 microversion: 2.1 time: 0.357570 Sep 27 10:25:43.465688 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2542/5538] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:43 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1376 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:43.512318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:43.513339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:43.530767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:43.532854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:43.536953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:43.539070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:43.540144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:43.544261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:43.589463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-954f8cd7-6174-40b8-a1a1-927d86f69498 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:43.591023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-954f8cd7-6174-40b8-a1a1-927d86f69498 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:43.612000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-954f8cd7-6174-40b8-a1a1-927d86f69498 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:43.612916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-954f8cd7-6174-40b8-a1a1-927d86f69498 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:43.614785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-954f8cd7-6174-40b8-a1a1-927d86f69498 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:43.616930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-954f8cd7-6174-40b8-a1a1-927d86f69498 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:43.618376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-954f8cd7-6174-40b8-a1a1-927d86f69498 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:43.621319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-954f8cd7-6174-40b8-a1a1-927d86f69498 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:43.877118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:43.887324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:43.896300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:43.897877 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d8f771ff-d411-4c4f-8470-57d225ac6b7d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.391147 Sep 27 10:25:43.898268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-954f8cd7-6174-40b8-a1a1-927d86f69498 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:43.901779 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2997/5539] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:43 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:43.911122 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-954f8cd7-6174-40b8-a1a1-927d86f69498 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.330534 Sep 27 10:25:43.916534 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2543/5540] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:43 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:43.947602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-503c01c4-8dbe-490e-9b97-2ec490f8aeb8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:43.948390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-503c01c4-8dbe-490e-9b97-2ec490f8aeb8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:43.961360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-503c01c4-8dbe-490e-9b97-2ec490f8aeb8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:43.962325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-503c01c4-8dbe-490e-9b97-2ec490f8aeb8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:43.965809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-503c01c4-8dbe-490e-9b97-2ec490f8aeb8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:43.968072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-503c01c4-8dbe-490e-9b97-2ec490f8aeb8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:43.968938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-503c01c4-8dbe-490e-9b97-2ec490f8aeb8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:43.969865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-503c01c4-8dbe-490e-9b97-2ec490f8aeb8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:44.208097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-503c01c4-8dbe-490e-9b97-2ec490f8aeb8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:44.232446 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-503c01c4-8dbe-490e-9b97-2ec490f8aeb8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.290192 Sep 27 10:25:44.245856 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2998/5541] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:43 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:44.265260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-936d3dd7-5e1f-47bb-8129-7a2ce2090ecb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"unlock": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:44.266779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-936d3dd7-5e1f-47bb-8129-7a2ce2090ecb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:44.276232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-936d3dd7-5e1f-47bb-8129-7a2ce2090ecb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:44.278414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-936d3dd7-5e1f-47bb-8129-7a2ce2090ecb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:44.280195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-936d3dd7-5e1f-47bb-8129-7a2ce2090ecb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:44.282074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-936d3dd7-5e1f-47bb-8129-7a2ce2090ecb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:44.282775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-936d3dd7-5e1f-47bb-8129-7a2ce2090ecb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:44.283440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-936d3dd7-5e1f-47bb-8129-7a2ce2090ecb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:44.309207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:44.314180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] [instance: 3db53022-2562-4d3c-a0e1-9d9162d5e1ac] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:44.330633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:44.331160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:44.336253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:44.337202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:44.337695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:44.340534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:44.371659 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] HTTP exception thrown: Instance 3db53022-2562-4d3c-a0e1-9d9162d5e1ac could not be found. Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Request method failure captured: Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac HTTP/1.1 Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs9999Tee_wWvTDKC07uG4ZmQzTikO42Nl4ox1451f4ukdpamS-lo1UAI1uQvWLG9tPN1ElMgUB34bow1Y73ii27eqAAkT8EZOMxG1omOp8CzNJfqkNWRkduV20dVcQ7xQux0vmjV1xzWljPGxcJaSWrXGQOUr5Xz27kp-WUUhXRWM0 Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 430555edbc87497db7d7f79f2b891c04 Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesBackupsTest-353079186 Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/430555edbc87497db7d7f79f2b891c04"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/430555edbc87497db7d7f79f2b891c04"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/430555edbc87497db7d7f79f2b891c04"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_430555edbc87497db7d7f79f2b891c04"}]}] Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesBackupsTest-353079186 Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 430555edbc87497db7d7f79f2b891c04 Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesBackupsTest-353079186 Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesBackupsTest-353079186-project Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: fbcfed06aa6b4177a4287064884125b2 Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesBackupsTest-353079186-project Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 3db53022-2562-4d3c-a0e1-9d9162d5e1ac could not be found. Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3db53022-2562-4d3c-a0e1-9d9162d5e1ac'} Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3db53022-2562-4d3c-a0e1-9d9162d5e1ac could not be found. Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3db53022-2562-4d3c-a0e1-9d9162d5e1ac could not be found. Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:25:44.376241 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:44.382378 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:25:44.382378 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:25:44.382378 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:25:44.382378 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:25:44.382378 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:25:44.382378 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3db53022-2562-4d3c-a0e1-9d9162d5e1ac could not be found. Sep 27 10:25:44.382378 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:44.384512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] Returning 404 to user: Instance 3db53022-2562-4d3c-a0e1-9d9162d5e1ac could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:25:44.389995 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d3ab9269-ec67-4ab4-9777-2f7e187fd30b tempest-VolumesBackupsTest-353079186 tempest-VolumesBackupsTest-353079186-project] 10.222.0.26 "GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac" status: 404 len: 111 microversion: 2.1 time: 0.091704 Sep 27 10:25:44.391186 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 2999/5542] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:44 2022] GET /compute/v2.1/servers/3db53022-2562-4d3c-a0e1-9d9162d5e1ac => generated 111 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:25:44.396046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-936d3dd7-5e1f-47bb-8129-7a2ce2090ecb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Unlocking {{(pid=145969) unlock /opt/stack/nova/nova/compute/api.py:4874}} Sep 27 10:25:44.492269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-96f7cc98-b3f7-4b6e-82af-6d93a338b6f1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:44.494905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-96f7cc98-b3f7-4b6e-82af-6d93a338b6f1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:44.512795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96f7cc98-b3f7-4b6e-82af-6d93a338b6f1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:44.513303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96f7cc98-b3f7-4b6e-82af-6d93a338b6f1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:44.516326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96f7cc98-b3f7-4b6e-82af-6d93a338b6f1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:44.518786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96f7cc98-b3f7-4b6e-82af-6d93a338b6f1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:44.520160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96f7cc98-b3f7-4b6e-82af-6d93a338b6f1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:44.521941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96f7cc98-b3f7-4b6e-82af-6d93a338b6f1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:44.539159 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-936d3dd7-5e1f-47bb-8129-7a2ce2090ecb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.278864 Sep 27 10:25:44.539831 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2544/5543] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:44 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 287 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:25:44.557339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9f4a22e-52b0-4de8-a2e4-446ff1b64657 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:44.558249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9f4a22e-52b0-4de8-a2e4-446ff1b64657 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:44.577761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9f4a22e-52b0-4de8-a2e4-446ff1b64657 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:44.580905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9f4a22e-52b0-4de8-a2e4-446ff1b64657 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:44.585300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9f4a22e-52b0-4de8-a2e4-446ff1b64657 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:44.597310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9f4a22e-52b0-4de8-a2e4-446ff1b64657 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:44.598558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9f4a22e-52b0-4de8-a2e4-446ff1b64657 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:44.623018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9f4a22e-52b0-4de8-a2e4-446ff1b64657 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.024s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:44.954055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-96f7cc98-b3f7-4b6e-82af-6d93a338b6f1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:44.967069 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-96f7cc98-b3f7-4b6e-82af-6d93a338b6f1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1376 microversion: 2.1 time: 0.482011 Sep 27 10:25:44.977905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c9f4a22e-52b0-4de8-a2e4-446ff1b64657 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:44.979703 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3000/5544] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:44 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1376 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:44.979703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:44.979703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:44.992960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:44.993842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:44.994602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:44.995818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:44.996559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:44.996985 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9f4a22e-52b0-4de8-a2e4-446ff1b64657 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.444080 Sep 27 10:25:44.997597 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2545/5545] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:44 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:44.998240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:45.030877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d50e8779-afac-45ad-9be2-3379953d5dd7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:45.031464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d50e8779-afac-45ad-9be2-3379953d5dd7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:45.043757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50e8779-afac-45ad-9be2-3379953d5dd7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:45.044234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50e8779-afac-45ad-9be2-3379953d5dd7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:45.044744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50e8779-afac-45ad-9be2-3379953d5dd7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:45.045452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50e8779-afac-45ad-9be2-3379953d5dd7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:45.045927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50e8779-afac-45ad-9be2-3379953d5dd7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:45.046370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d50e8779-afac-45ad-9be2-3379953d5dd7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:45.344838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d50e8779-afac-45ad-9be2-3379953d5dd7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:45.356470 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d50e8779-afac-45ad-9be2-3379953d5dd7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.329592 Sep 27 10:25:45.357766 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2546/5546] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:45 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:45.384003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73c54ac2-7180-4c56-a552-b34062072f88 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:45.385151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-73c54ac2-7180-4c56-a552-b34062072f88 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:45.398039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73c54ac2-7180-4c56-a552-b34062072f88 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:45.398306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73c54ac2-7180-4c56-a552-b34062072f88 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:45.398822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73c54ac2-7180-4c56-a552-b34062072f88 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:45.399630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73c54ac2-7180-4c56-a552-b34062072f88 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:45.400190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73c54ac2-7180-4c56-a552-b34062072f88 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:45.400703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73c54ac2-7180-4c56-a552-b34062072f88 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:45.457866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:45.478213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:45.487680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:45.491204 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-29c9a47e-f203-425c-9c6c-f2d472d0ee8b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.520035 Sep 27 10:25:45.504043 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3001/5547] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:44 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:45.708327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-73c54ac2-7180-4c56-a552-b34062072f88 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:45.723950 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-73c54ac2-7180-4c56-a552-b34062072f88 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.347644 Sep 27 10:25:45.748833 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2547/5548] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:45 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:45.761636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c1168fd8-4a15-4f83-994d-6bcf87a3a22e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:45.762474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c1168fd8-4a15-4f83-994d-6bcf87a3a22e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:45.779879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1168fd8-4a15-4f83-994d-6bcf87a3a22e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:45.780755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1168fd8-4a15-4f83-994d-6bcf87a3a22e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:45.781378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1168fd8-4a15-4f83-994d-6bcf87a3a22e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:45.782385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1168fd8-4a15-4f83-994d-6bcf87a3a22e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:45.783073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1168fd8-4a15-4f83-994d-6bcf87a3a22e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:45.783675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1168fd8-4a15-4f83-994d-6bcf87a3a22e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:45.929393 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c1168fd8-4a15-4f83-994d-6bcf87a3a22e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.171288 Sep 27 10:25:45.943302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-38db06b8-064b-455d-81fa-f11797067167 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:45.944490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-38db06b8-064b-455d-81fa-f11797067167 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:45.948038 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3002/5549] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:45 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 191 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:25:45.953584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38db06b8-064b-455d-81fa-f11797067167 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:45.954354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38db06b8-064b-455d-81fa-f11797067167 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:45.954702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38db06b8-064b-455d-81fa-f11797067167 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:45.955438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38db06b8-064b-455d-81fa-f11797067167 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:45.956203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38db06b8-064b-455d-81fa-f11797067167 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:45.956851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38db06b8-064b-455d-81fa-f11797067167 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:45.961389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=5,nova_cell1:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:46.009733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9490f480-6fb1-48ec-973b-42e649ed3c7f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:46.011249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9490f480-6fb1-48ec-973b-42e649ed3c7f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:46.018341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9490f480-6fb1-48ec-973b-42e649ed3c7f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:46.019057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9490f480-6fb1-48ec-973b-42e649ed3c7f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:46.019890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9490f480-6fb1-48ec-973b-42e649ed3c7f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:46.021570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9490f480-6fb1-48ec-973b-42e649ed3c7f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:46.022198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9490f480-6fb1-48ec-973b-42e649ed3c7f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:46.022795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9490f480-6fb1-48ec-973b-42e649ed3c7f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:46.408015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-38db06b8-064b-455d-81fa-f11797067167 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:46.419704 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-38db06b8-064b-455d-81fa-f11797067167 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1674 microversion: 2.1 time: 0.481993 Sep 27 10:25:46.421073 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2548/5550] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:45 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1674 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:46.459854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9490f480-6fb1-48ec-973b-42e649ed3c7f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:46.471717 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9490f480-6fb1-48ec-973b-42e649ed3c7f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1376 microversion: 2.1 time: 0.474368 Sep 27 10:25:46.473285 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3003/5551] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:45 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1376 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:46.524438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:46.525242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:46.542381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:46.544933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:46.545735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:46.549345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:46.551120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:46.552463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:46.936514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:46.953978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:46.955927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:46.957538 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3df85ee6-6c38-41d9-92f5-fda898cdd9d2 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.437567 Sep 27 10:25:46.979605 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2549/5552] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:46 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:47.447392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2a35cb29-e67b-4223-8ffc-70f742a0f937 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:47.448125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2a35cb29-e67b-4223-8ffc-70f742a0f937 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:47.466894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a35cb29-e67b-4223-8ffc-70f742a0f937 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:47.467785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a35cb29-e67b-4223-8ffc-70f742a0f937 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:47.468606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a35cb29-e67b-4223-8ffc-70f742a0f937 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:47.469936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a35cb29-e67b-4223-8ffc-70f742a0f937 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:47.470737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a35cb29-e67b-4223-8ffc-70f742a0f937 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:47.471314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a35cb29-e67b-4223-8ffc-70f742a0f937 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:47.502856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6770d5b7-e2ad-4df1-8149-43ec43fbd37b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:47.504730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6770d5b7-e2ad-4df1-8149-43ec43fbd37b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:47.522700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6770d5b7-e2ad-4df1-8149-43ec43fbd37b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:47.532531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6770d5b7-e2ad-4df1-8149-43ec43fbd37b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:47.532531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6770d5b7-e2ad-4df1-8149-43ec43fbd37b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:47.532531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6770d5b7-e2ad-4df1-8149-43ec43fbd37b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:47.532531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6770d5b7-e2ad-4df1-8149-43ec43fbd37b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:47.532531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6770d5b7-e2ad-4df1-8149-43ec43fbd37b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:47.964030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2a35cb29-e67b-4223-8ffc-70f742a0f937 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:47.977300 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2a35cb29-e67b-4223-8ffc-70f742a0f937 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1654 microversion: 2.1 time: 0.534138 Sep 27 10:25:47.978167 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3004/5553] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:47 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1654 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:47.996706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:47.997678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:48.003276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6770d5b7-e2ad-4df1-8149-43ec43fbd37b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:48.009679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:48.021383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:48.022310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:48.023115 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6770d5b7-e2ad-4df1-8149-43ec43fbd37b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1563 microversion: 2.1 time: 0.528598 Sep 27 10:25:48.023557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:48.024128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:48.024552 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2550/5554] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:47 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1563 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:25:48.024926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:48.033395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f78ad1d-d48c-495b-a316-aa8a64d42172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:48.034925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f78ad1d-d48c-495b-a316-aa8a64d42172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:48.043566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f78ad1d-d48c-495b-a316-aa8a64d42172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:48.044617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f78ad1d-d48c-495b-a316-aa8a64d42172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:48.046661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f78ad1d-d48c-495b-a316-aa8a64d42172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:48.048158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f78ad1d-d48c-495b-a316-aa8a64d42172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:48.050667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f78ad1d-d48c-495b-a316-aa8a64d42172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:48.054017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f78ad1d-d48c-495b-a316-aa8a64d42172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:48.214672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:48.230787 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f78ad1d-d48c-495b-a316-aa8a64d42172 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.201113 Sep 27 10:25:48.233282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:48.235996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:48.237738 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4286658e-9aa4-40bb-a917-6754b5ff1b34 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.246806 Sep 27 10:25:48.238247 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2551/5555] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:25:48 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 208 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:25:48.240240 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3005/5556] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:47 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:25:48.250033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eb2210ea-a0c6-4954-85a3-6babbdcff8f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:48.251367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eb2210ea-a0c6-4954-85a3-6babbdcff8f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:48.259731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dbe910cd-2c77-40d7-af4f-c5c6bc027ea8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:48.262028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dbe910cd-2c77-40d7-af4f-c5c6bc027ea8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:48.269223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2210ea-a0c6-4954-85a3-6babbdcff8f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:48.270954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2210ea-a0c6-4954-85a3-6babbdcff8f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:48.271770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2210ea-a0c6-4954-85a3-6babbdcff8f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:48.275299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2210ea-a0c6-4954-85a3-6babbdcff8f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:48.276336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2210ea-a0c6-4954-85a3-6babbdcff8f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:48.277219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb2210ea-a0c6-4954-85a3-6babbdcff8f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:48.286217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbe910cd-2c77-40d7-af4f-c5c6bc027ea8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:48.286217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbe910cd-2c77-40d7-af4f-c5c6bc027ea8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:48.286217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbe910cd-2c77-40d7-af4f-c5c6bc027ea8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:48.290971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbe910cd-2c77-40d7-af4f-c5c6bc027ea8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:48.291790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbe910cd-2c77-40d7-af4f-c5c6bc027ea8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:48.294436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbe910cd-2c77-40d7-af4f-c5c6bc027ea8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:48.699567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-eb2210ea-a0c6-4954-85a3-6babbdcff8f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:48.726448 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eb2210ea-a0c6-4954-85a3-6babbdcff8f4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1661 microversion: 2.1 time: 0.478387 Sep 27 10:25:48.727371 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3006/5557] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:48 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1661 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:48.762256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dbe910cd-2c77-40d7-af4f-c5c6bc027ea8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:48.772843 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dbe910cd-2c77-40d7-af4f-c5c6bc027ea8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1563 microversion: 2.1 time: 0.530658 Sep 27 10:25:48.774212 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2552/5558] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:48 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1563 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:48.919481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-336c664f-90da-467d-951a-2db0b0a4bf97 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:48.920228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-336c664f-90da-467d-951a-2db0b0a4bf97 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:48.933873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336c664f-90da-467d-951a-2db0b0a4bf97 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:48.935520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336c664f-90da-467d-951a-2db0b0a4bf97 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:48.937728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336c664f-90da-467d-951a-2db0b0a4bf97 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:48.938853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336c664f-90da-467d-951a-2db0b0a4bf97 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:48.940044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336c664f-90da-467d-951a-2db0b0a4bf97 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:48.940611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-336c664f-90da-467d-951a-2db0b0a4bf97 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:48.993223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-336c664f-90da-467d-951a-2db0b0a4bf97 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:25:49.213688 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-336c664f-90da-467d-951a-2db0b0a4bf97 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "DELETE /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 204 len: 0 microversion: 2.1 time: 0.297025 Sep 27 10:25:49.215157 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3007/5559] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:25:48 2022] DELETE /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 0 bytes in 300 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:25:49.224071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6dec799a-d4c4-4c17-b37f-60dda1a87a7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:49.224745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6dec799a-d4c4-4c17-b37f-60dda1a87a7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:49.234059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dec799a-d4c4-4c17-b37f-60dda1a87a7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:49.234597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dec799a-d4c4-4c17-b37f-60dda1a87a7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:49.235033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dec799a-d4c4-4c17-b37f-60dda1a87a7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:49.236219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dec799a-d4c4-4c17-b37f-60dda1a87a7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:49.236640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dec799a-d4c4-4c17-b37f-60dda1a87a7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:49.236968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6dec799a-d4c4-4c17-b37f-60dda1a87a7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:49.271097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:49.271764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:49.295402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:49.296043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:49.296502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:49.297401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:49.297873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:49.298308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:49.603423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:49.611114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:49.620934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:49.622956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:49.629791 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aafe4a64-8167-4597-8259-084b17385c43 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.363781 Sep 27 10:25:49.630877 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3008/5560] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:49 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:49.662273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6dec799a-d4c4-4c17-b37f-60dda1a87a7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:49.674322 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6dec799a-d4c4-4c17-b37f-60dda1a87a7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1569 microversion: 2.1 time: 0.451584 Sep 27 10:25:49.675448 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2553/5561] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:49 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1569 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:49.757547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e73fb2e0-48fd-4212-a784-be97a1b09da1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:49.758860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e73fb2e0-48fd-4212-a784-be97a1b09da1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:49.766715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73fb2e0-48fd-4212-a784-be97a1b09da1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:49.767213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73fb2e0-48fd-4212-a784-be97a1b09da1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:49.767658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73fb2e0-48fd-4212-a784-be97a1b09da1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:49.768582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73fb2e0-48fd-4212-a784-be97a1b09da1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:49.769090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73fb2e0-48fd-4212-a784-be97a1b09da1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:49.769524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73fb2e0-48fd-4212-a784-be97a1b09da1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:49.933252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e73fb2e0-48fd-4212-a784-be97a1b09da1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:49.949188 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e73fb2e0-48fd-4212-a784-be97a1b09da1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.197675 Sep 27 10:25:49.950460 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3009/5562] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:49 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:49.975479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7b76236e-2ec1-480f-a6ee-6f5ac4a45107 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:49.976765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7b76236e-2ec1-480f-a6ee-6f5ac4a45107 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:49.983610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b76236e-2ec1-480f-a6ee-6f5ac4a45107 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:49.984102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b76236e-2ec1-480f-a6ee-6f5ac4a45107 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:49.984795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b76236e-2ec1-480f-a6ee-6f5ac4a45107 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:49.985531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b76236e-2ec1-480f-a6ee-6f5ac4a45107 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:49.987226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b76236e-2ec1-480f-a6ee-6f5ac4a45107 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:49.987607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b76236e-2ec1-480f-a6ee-6f5ac4a45107 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:50.219223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7b76236e-2ec1-480f-a6ee-6f5ac4a45107 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:50.229810 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7b76236e-2ec1-480f-a6ee-6f5ac4a45107 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.258339 Sep 27 10:25:50.230994 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2554/5563] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:49 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:50.260043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dc01511b-32aa-43d2-8add-3a2157de5a0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:50.260995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dc01511b-32aa-43d2-8add-3a2157de5a0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:50.282143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01511b-32aa-43d2-8add-3a2157de5a0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:50.282905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01511b-32aa-43d2-8add-3a2157de5a0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:50.283354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01511b-32aa-43d2-8add-3a2157de5a0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:50.284223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01511b-32aa-43d2-8add-3a2157de5a0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:50.285013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01511b-32aa-43d2-8add-3a2157de5a0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:50.285884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dc01511b-32aa-43d2-8add-3a2157de5a0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:50.467115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dc01511b-32aa-43d2-8add-3a2157de5a0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:50.479369 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dc01511b-32aa-43d2-8add-3a2157de5a0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.225770 Sep 27 10:25:50.480131 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3010/5564] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:50 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:50.503057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-118fae52-3459-4204-a39b-e6aa01627683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:50.503302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-118fae52-3459-4204-a39b-e6aa01627683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:50.509978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-118fae52-3459-4204-a39b-e6aa01627683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:50.510480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-118fae52-3459-4204-a39b-e6aa01627683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:50.511107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-118fae52-3459-4204-a39b-e6aa01627683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:50.511874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-118fae52-3459-4204-a39b-e6aa01627683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:50.512478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-118fae52-3459-4204-a39b-e6aa01627683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:50.512936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-118fae52-3459-4204-a39b-e6aa01627683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:50.667931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:50.669185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:50.677420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:50.678605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:50.678965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:50.679886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:50.680537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:50.681171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:50.711744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-118fae52-3459-4204-a39b-e6aa01627683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:50.733532 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-118fae52-3459-4204-a39b-e6aa01627683 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.234742 Sep 27 10:25:50.737144 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2555/5565] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:50 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:50.742344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-816451ba-e948-4f32-8f8c-b4c3f423a10f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:50.742980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-816451ba-e948-4f32-8f8c-b4c3f423a10f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:50.773839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-816451ba-e948-4f32-8f8c-b4c3f423a10f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:50.774364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-816451ba-e948-4f32-8f8c-b4c3f423a10f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:50.774832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-816451ba-e948-4f32-8f8c-b4c3f423a10f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:50.775585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-816451ba-e948-4f32-8f8c-b4c3f423a10f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:50.776069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-816451ba-e948-4f32-8f8c-b4c3f423a10f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:50.776521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-816451ba-e948-4f32-8f8c-b4c3f423a10f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:51.066211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:51.081718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:51.084642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:51.091004 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-17cbaa1a-d89d-4487-b63e-234087434bbb tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.428161 Sep 27 10:25:51.093822 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3011/5566] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:50 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:51.106371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d1351a9-1fce-4028-b428-ba887186c6c8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:51.108097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9d1351a9-1fce-4028-b428-ba887186c6c8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:51.126845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d1351a9-1fce-4028-b428-ba887186c6c8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:51.128427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d1351a9-1fce-4028-b428-ba887186c6c8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:51.131154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d1351a9-1fce-4028-b428-ba887186c6c8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:51.138037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d1351a9-1fce-4028-b428-ba887186c6c8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:51.138764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d1351a9-1fce-4028-b428-ba887186c6c8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:51.140988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d1351a9-1fce-4028-b428-ba887186c6c8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:51.147369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-816451ba-e948-4f32-8f8c-b4c3f423a10f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:51.155762 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-816451ba-e948-4f32-8f8c-b4c3f423a10f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1569 microversion: 2.1 time: 0.417450 Sep 27 10:25:51.156366 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2556/5567] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:50 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1569 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:51.441168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9d1351a9-1fce-4028-b428-ba887186c6c8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:51.452439 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d1351a9-1fce-4028-b428-ba887186c6c8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.356169 Sep 27 10:25:51.453874 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3012/5568] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:51 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:51.480392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8323abbe-3014-4d6f-ac6f-6170dadcf6e6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:51.481352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8323abbe-3014-4d6f-ac6f-6170dadcf6e6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:51.488414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8323abbe-3014-4d6f-ac6f-6170dadcf6e6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:51.489676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8323abbe-3014-4d6f-ac6f-6170dadcf6e6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:51.491113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8323abbe-3014-4d6f-ac6f-6170dadcf6e6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:51.492168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8323abbe-3014-4d6f-ac6f-6170dadcf6e6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:51.492892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8323abbe-3014-4d6f-ac6f-6170dadcf6e6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:51.493565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8323abbe-3014-4d6f-ac6f-6170dadcf6e6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:51.707505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8323abbe-3014-4d6f-ac6f-6170dadcf6e6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:51.718107 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8323abbe-3014-4d6f-ac6f-6170dadcf6e6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.241474 Sep 27 10:25:51.737784 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2557/5569] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:51 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:52.127915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:52.129653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:52.142547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:52.143400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:52.144250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:52.145993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:52.146676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:52.147738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:52.182526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f6aeda4e-648f-4d88-9ea2-a12bd7e48f21 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:52.183798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f6aeda4e-648f-4d88-9ea2-a12bd7e48f21 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:52.191655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6aeda4e-648f-4d88-9ea2-a12bd7e48f21 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:52.193422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6aeda4e-648f-4d88-9ea2-a12bd7e48f21 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:52.194114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6aeda4e-648f-4d88-9ea2-a12bd7e48f21 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:52.195089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6aeda4e-648f-4d88-9ea2-a12bd7e48f21 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:52.195780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6aeda4e-648f-4d88-9ea2-a12bd7e48f21 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:52.196451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6aeda4e-648f-4d88-9ea2-a12bd7e48f21 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:52.433894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:52.448525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:52.453075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:52.456924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b7262712-a98b-4867-bb8f-2cef9b2f61ec tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.333644 Sep 27 10:25:52.458961 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3013/5570] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:52 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:52.545753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6aeda4e-648f-4d88-9ea2-a12bd7e48f21 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:52.556199 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f6aeda4e-648f-4d88-9ea2-a12bd7e48f21 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1569 microversion: 2.1 time: 0.377686 Sep 27 10:25:52.557450 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2558/5571] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:52 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1569 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:53.489156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:53.489845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:53.497012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:53.497491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:53.497932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:53.498633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:53.499089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:53.499514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:53.580021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7017429f-55a3-48ee-8ebd-98a35776e943 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:53.580787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7017429f-55a3-48ee-8ebd-98a35776e943 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:53.589076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7017429f-55a3-48ee-8ebd-98a35776e943 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:53.589681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7017429f-55a3-48ee-8ebd-98a35776e943 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:53.590223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7017429f-55a3-48ee-8ebd-98a35776e943 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:53.591023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7017429f-55a3-48ee-8ebd-98a35776e943 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:53.591567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7017429f-55a3-48ee-8ebd-98a35776e943 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:53.592084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7017429f-55a3-48ee-8ebd-98a35776e943 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:53.663232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:53.673809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:53.675242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:53.676679 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9109b2b0-ee78-41e0-ad2a-36dd35699f4e tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.191605 Sep 27 10:25:53.677263 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3014/5572] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:53 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:53.765132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7017429f-55a3-48ee-8ebd-98a35776e943 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:53.780391 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7017429f-55a3-48ee-8ebd-98a35776e943 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1569 microversion: 2.1 time: 0.204226 Sep 27 10:25:53.781963 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2559/5573] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:53 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1569 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:54.587779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a3543865-cd75-43e3-937c-102891d319e8 req-c281d255-b6d4-4e95-8b54-5dd403c25592 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "04dc4859-cd17-4187-8019-e2bcf346a7df", "name": "network-vif-deleted", "tag": "e72ee7df-caa1-44c6-88f1-3281aefb5154"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:54.616931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a3543865-cd75-43e3-937c-102891d319e8 req-c281d255-b6d4-4e95-8b54-5dd403c25592 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:54.618575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a3543865-cd75-43e3-937c-102891d319e8 req-c281d255-b6d4-4e95-8b54-5dd403c25592 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:54.619037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a3543865-cd75-43e3-937c-102891d319e8 req-c281d255-b6d4-4e95-8b54-5dd403c25592 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:54.633114 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a3543865-cd75-43e3-937c-102891d319e8 req-c281d255-b6d4-4e95-8b54-5dd403c25592 service nova] Creating event network-vif-deleted:e72ee7df-caa1-44c6-88f1-3281aefb5154 for instance 04dc4859-cd17-4187-8019-e2bcf346a7df on nch1-master-1 Sep 27 10:25:54.639755 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a3543865-cd75-43e3-937c-102891d319e8 req-c281d255-b6d4-4e95-8b54-5dd403c25592 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056275 Sep 27 10:25:54.640644 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3015/5574] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:25:54.702749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:54.703769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:54.712700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:54.714081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:54.714784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:54.715789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:54.716484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:54.717176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:54.812522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-77734538-2a2f-442a-8e39-b51d77d1f00e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:54.815109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-77734538-2a2f-442a-8e39-b51d77d1f00e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:54.827018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77734538-2a2f-442a-8e39-b51d77d1f00e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:54.832205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77734538-2a2f-442a-8e39-b51d77d1f00e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:54.832711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77734538-2a2f-442a-8e39-b51d77d1f00e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:54.834478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77734538-2a2f-442a-8e39-b51d77d1f00e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:54.834971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77734538-2a2f-442a-8e39-b51d77d1f00e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:54.835451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77734538-2a2f-442a-8e39-b51d77d1f00e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:54.915241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-77734538-2a2f-442a-8e39-b51d77d1f00e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:54.926326 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-77734538-2a2f-442a-8e39-b51d77d1f00e tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1527 microversion: 2.1 time: 0.117682 Sep 27 10:25:54.926924 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3016/5575] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:54 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1527 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:54.953037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:54.965068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:54.967351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:54.968948 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-74110614-123c-4f9a-ae20-7adee9af120b tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.270158 Sep 27 10:25:54.970153 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2560/5576] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:54 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:25:55.956334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:55.957865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:55.966367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:55.968891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:55.970291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:55.971629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:55.972365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:55.973780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:56.027262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:56.027884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:56.040084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:56.040603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:56.042109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:56.044943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:56.052427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:56.053985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:56.054705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:56.056548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:56.059084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:56.059856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lazy-loading 'fault' on Instance uuid 04dc4859-cd17-4187-8019-e2bcf346a7df {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:25:56.121597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:56.136154 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-789cfca2-acc7-45bd-98bc-4c0c2bef0ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 200 len: 1366 microversion: 2.1 time: 0.184428 Sep 27 10:25:56.137860 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3017/5577] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:55 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 1366 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:25:56.329408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:56.340700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:56.342556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:56.344037 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c6dbc11f-ac8b-486a-bbc2-6c15f0f21eea tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.327277 Sep 27 10:25:56.344762 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2561/5578] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:56 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:57.165413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:57.166031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:57.175194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:57.177282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:57.178210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:57.178954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:57.179418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:57.179839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:57.195107 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df HTTP/1.1 Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '04dc4859-cd17-4187-8019-e2bcf346a7df'} Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:25:57.197092 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:57.201922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:25:57.201922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:25:57.201922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:25:57.201922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:25:57.201922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:25:57.201922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:25:57.201922 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:25:57.201922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:25:57.201922 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d106483-f206-4e7d-a3b5-7172ed955ea6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 404 len: 111 microversion: 2.1 time: 0.037494 Sep 27 10:25:57.201922 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3018/5579] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:25:57 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:25:57.381246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:57.382124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:57.394178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:57.395155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:57.395923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:57.396788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:57.397485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:57.398040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:57.682141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:57.715033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:57.717283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:57.718576 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6216b88e-ec62-491e-9e2e-f365393e7fab tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.348205 Sep 27 10:25:57.719169 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2562/5580] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:57 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:25:58.070457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-593528fe-1a3a-4c24-92e3-6c965469c42f req-fafbfd8b-7b9f-44ad-b205-febdfad837c1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77037d39-844c-4761-88cc-899710443401", "name": "network-vif-plugged", "status": "completed", "tag": "6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:25:58.081306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-593528fe-1a3a-4c24-92e3-6c965469c42f req-fafbfd8b-7b9f-44ad-b205-febdfad837c1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:58.082187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-593528fe-1a3a-4c24-92e3-6c965469c42f req-fafbfd8b-7b9f-44ad-b205-febdfad837c1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:58.082834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-593528fe-1a3a-4c24-92e3-6c965469c42f req-fafbfd8b-7b9f-44ad-b205-febdfad837c1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:58.096583 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-593528fe-1a3a-4c24-92e3-6c965469c42f req-fafbfd8b-7b9f-44ad-b205-febdfad837c1 service nova] Creating event network-vif-plugged:6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1 for instance 77037d39-844c-4761-88cc-899710443401 on nch1-master-1 Sep 27 10:25:58.126252 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-593528fe-1a3a-4c24-92e3-6c965469c42f req-fafbfd8b-7b9f-44ad-b205-febdfad837c1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059793 Sep 27 10:25:58.127640 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3019/5581] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:25:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:25:58.752881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:25:58.753656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:25:58.762859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:58.763553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:58.764113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:58.764986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:25:58.765568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:25:58.766143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:25:59.084088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:59.091890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:25:59.095187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:59.097261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:25:59.099161 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bbda5771-ad93-4759-b117-4c89ced13920 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.350585 Sep 27 10:25:59.100283 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2563/5582] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:25:58 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:26:00.150261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:00.151065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:00.160080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:00.160937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:00.162557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:00.163487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:00.164129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:00.164749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:00.359712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:00.371691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:00.373260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:00.375219 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-246c9924-c259-47af-a95d-18d5a2a9d20d tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1729 microversion: 2.20 time: 0.249424 Sep 27 10:26:00.375772 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3020/5583] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:26:00 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1729 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:26:01.403998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:01.404633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:01.413652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:01.414172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:01.414818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:01.415587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:01.416062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:01.416508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:01.646976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:01.659577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:01.663557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:01.666412 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d5b9cc57-4b57-4f9f-a3aa-d475041e87ce tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1716 microversion: 2.20 time: 0.265369 Sep 27 10:26:01.683450 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2564/5584] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:26:01 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1716 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:26:18.165674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=73,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:18.166556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=62 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:18.387711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-11611feb-cd33-46c6-af2f-8a9324049907 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:26:18.390200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-11611feb-cd33-46c6-af2f-8a9324049907 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:18.397575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11611feb-cd33-46c6-af2f-8a9324049907 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:18.398075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11611feb-cd33-46c6-af2f-8a9324049907 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:18.398757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11611feb-cd33-46c6-af2f-8a9324049907 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:18.399679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11611feb-cd33-46c6-af2f-8a9324049907 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:18.400327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11611feb-cd33-46c6-af2f-8a9324049907 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:18.401000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11611feb-cd33-46c6-af2f-8a9324049907 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:18.510432 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-11611feb-cd33-46c6-af2f-8a9324049907 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.353184 Sep 27 10:26:18.511966 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3021/5585] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:26:18 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 356 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:26:18.523017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-629635cb-4689-4e9b-83d0-444b1b02606a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:18.524016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-629635cb-4689-4e9b-83d0-444b1b02606a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:18.532276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=66,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:18.534438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=7 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:18.534956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-629635cb-4689-4e9b-83d0-444b1b02606a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:18.536004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-629635cb-4689-4e9b-83d0-444b1b02606a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:18.536792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-629635cb-4689-4e9b-83d0-444b1b02606a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:18.537787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-629635cb-4689-4e9b-83d0-444b1b02606a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:18.538850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-629635cb-4689-4e9b-83d0-444b1b02606a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:18.539505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-629635cb-4689-4e9b-83d0-444b1b02606a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:18.783908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-629635cb-4689-4e9b-83d0-444b1b02606a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:18.796267 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-629635cb-4689-4e9b-83d0-444b1b02606a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1671 microversion: 2.1 time: 0.277015 Sep 27 10:26:18.797829 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2565/5586] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:18 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1671 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:19.822114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7c0bc668-b9df-4a6e-8aed-98c6d3c8680b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:19.823431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7c0bc668-b9df-4a6e-8aed-98c6d3c8680b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:19.833148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c0bc668-b9df-4a6e-8aed-98c6d3c8680b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:19.833869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c0bc668-b9df-4a6e-8aed-98c6d3c8680b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:19.835085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c0bc668-b9df-4a6e-8aed-98c6d3c8680b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:19.836015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c0bc668-b9df-4a6e-8aed-98c6d3c8680b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:19.837112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c0bc668-b9df-4a6e-8aed-98c6d3c8680b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:19.837744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7c0bc668-b9df-4a6e-8aed-98c6d3c8680b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:20.021034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7c0bc668-b9df-4a6e-8aed-98c6d3c8680b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:20.029875 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7c0bc668-b9df-4a6e-8aed-98c6d3c8680b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1676 microversion: 2.1 time: 0.214243 Sep 27 10:26:20.046541 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3022/5587] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:19 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1676 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:21.076249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-02663d74-d269-4bf0-b940-80faed5c2c0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:21.077444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-02663d74-d269-4bf0-b940-80faed5c2c0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:21.085455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02663d74-d269-4bf0-b940-80faed5c2c0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:21.086159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02663d74-d269-4bf0-b940-80faed5c2c0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:21.087240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02663d74-d269-4bf0-b940-80faed5c2c0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:21.088717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02663d74-d269-4bf0-b940-80faed5c2c0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:21.089388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02663d74-d269-4bf0-b940-80faed5c2c0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:21.090021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02663d74-d269-4bf0-b940-80faed5c2c0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:21.321833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-02663d74-d269-4bf0-b940-80faed5c2c0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:21.330962 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-02663d74-d269-4bf0-b940-80faed5c2c0c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1676 microversion: 2.1 time: 0.279431 Sep 27 10:26:21.331568 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2566/5588] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:21 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1676 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:22.355328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bf63863e-94b8-4008-97b3-51b2819db5fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:22.355983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bf63863e-94b8-4008-97b3-51b2819db5fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:22.365545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf63863e-94b8-4008-97b3-51b2819db5fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:22.366192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf63863e-94b8-4008-97b3-51b2819db5fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:22.366725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf63863e-94b8-4008-97b3-51b2819db5fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:22.367463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf63863e-94b8-4008-97b3-51b2819db5fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:22.368013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf63863e-94b8-4008-97b3-51b2819db5fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:22.368536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bf63863e-94b8-4008-97b3-51b2819db5fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:22.582052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bf63863e-94b8-4008-97b3-51b2819db5fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:22.593877 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bf63863e-94b8-4008-97b3-51b2819db5fa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1676 microversion: 2.1 time: 0.242278 Sep 27 10:26:22.595206 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3023/5589] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:22 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1676 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:23.621268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f407694e-9649-401e-b9ca-28c34ddeebc0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:23.622258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f407694e-9649-401e-b9ca-28c34ddeebc0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:23.642201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f407694e-9649-401e-b9ca-28c34ddeebc0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:23.642825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f407694e-9649-401e-b9ca-28c34ddeebc0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:23.643764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f407694e-9649-401e-b9ca-28c34ddeebc0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:23.644614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f407694e-9649-401e-b9ca-28c34ddeebc0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:23.645329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f407694e-9649-401e-b9ca-28c34ddeebc0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:23.645962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f407694e-9649-401e-b9ca-28c34ddeebc0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:23.853601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f407694e-9649-401e-b9ca-28c34ddeebc0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:23.864539 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f407694e-9649-401e-b9ca-28c34ddeebc0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1676 microversion: 2.1 time: 0.248485 Sep 27 10:26:23.866811 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2567/5590] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:23 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1676 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:24.889409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-37ca5627-3963-4fcc-af22-4d42e8c79a63 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:24.890169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-37ca5627-3963-4fcc-af22-4d42e8c79a63 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:24.899043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37ca5627-3963-4fcc-af22-4d42e8c79a63 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:24.899732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37ca5627-3963-4fcc-af22-4d42e8c79a63 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:24.900539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37ca5627-3963-4fcc-af22-4d42e8c79a63 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:24.901349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37ca5627-3963-4fcc-af22-4d42e8c79a63 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:24.901821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37ca5627-3963-4fcc-af22-4d42e8c79a63 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:24.902590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37ca5627-3963-4fcc-af22-4d42e8c79a63 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:25.052053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-37ca5627-3963-4fcc-af22-4d42e8c79a63 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:25.066865 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-37ca5627-3963-4fcc-af22-4d42e8c79a63 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1676 microversion: 2.1 time: 0.181959 Sep 27 10:26:25.067460 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3024/5591] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:24 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1676 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:26.096171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9f6ed7e9-00d5-4b13-b36e-681d50d08c3e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:26.096859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9f6ed7e9-00d5-4b13-b36e-681d50d08c3e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:26.113520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f6ed7e9-00d5-4b13-b36e-681d50d08c3e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:26.114063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f6ed7e9-00d5-4b13-b36e-681d50d08c3e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:26.114969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f6ed7e9-00d5-4b13-b36e-681d50d08c3e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:26.116500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f6ed7e9-00d5-4b13-b36e-681d50d08c3e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:26.117744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f6ed7e9-00d5-4b13-b36e-681d50d08c3e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:26.118350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9f6ed7e9-00d5-4b13-b36e-681d50d08c3e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:26.307026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9f6ed7e9-00d5-4b13-b36e-681d50d08c3e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:26.318818 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9f6ed7e9-00d5-4b13-b36e-681d50d08c3e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.226063 Sep 27 10:26:26.320017 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2568/5592] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:26 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:29.760625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e152c28f-e7e1-4955-9ceb-4d92833cb03d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:29.761258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e152c28f-e7e1-4955-9ceb-4d92833cb03d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:29.772989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e152c28f-e7e1-4955-9ceb-4d92833cb03d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:29.773476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e152c28f-e7e1-4955-9ceb-4d92833cb03d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:29.773908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e152c28f-e7e1-4955-9ceb-4d92833cb03d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:29.774615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e152c28f-e7e1-4955-9ceb-4d92833cb03d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:29.778433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e152c28f-e7e1-4955-9ceb-4d92833cb03d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:29.778875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e152c28f-e7e1-4955-9ceb-4d92833cb03d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:29.808973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e152c28f-e7e1-4955-9ceb-4d92833cb03d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:26:29.935022 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e152c28f-e7e1-4955-9ceb-4d92833cb03d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 204 len: 0 microversion: 2.1 time: 0.177698 Sep 27 10:26:29.936434 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3025/5593] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:26:29 2022] DELETE /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 0 bytes in 180 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:26:29.948494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4b979805-5373-4076-aa09-5c51f958c0a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:29.950963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4b979805-5373-4076-aa09-5c51f958c0a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:29.961706 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 10:26:29.962299 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 10:26:29 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc (ip 10.222.0.26) !!! Sep 27 10:26:29.969734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b979805-5373-4076-aa09-5c51f958c0a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:29.970245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b979805-5373-4076-aa09-5c51f958c0a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:29.970903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b979805-5373-4076-aa09-5c51f958c0a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:29.971836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b979805-5373-4076-aa09-5c51f958c0a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:29.972467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b979805-5373-4076-aa09-5c51f958c0a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:29.973090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b979805-5373-4076-aa09-5c51f958c0a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:30.147861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4b979805-5373-4076-aa09-5c51f958c0a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:30.168445 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4b979805-5373-4076-aa09-5c51f958c0a5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1410 microversion: 2.1 time: 0.225968 Sep 27 10:26:30.170001 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2569/5594] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:29 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1410 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:31.193076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-595fa9e7-6fda-4aa1-aaf6-e627d0a3a4ad tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:31.194733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-595fa9e7-6fda-4aa1-aaf6-e627d0a3a4ad tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:31.207190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-595fa9e7-6fda-4aa1-aaf6-e627d0a3a4ad tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:31.208503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-595fa9e7-6fda-4aa1-aaf6-e627d0a3a4ad tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:31.209199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-595fa9e7-6fda-4aa1-aaf6-e627d0a3a4ad tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:31.210103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-595fa9e7-6fda-4aa1-aaf6-e627d0a3a4ad tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:31.210786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-595fa9e7-6fda-4aa1-aaf6-e627d0a3a4ad tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:31.211843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-595fa9e7-6fda-4aa1-aaf6-e627d0a3a4ad tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:31.496233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-595fa9e7-6fda-4aa1-aaf6-e627d0a3a4ad tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:31.510229 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-595fa9e7-6fda-4aa1-aaf6-e627d0a3a4ad tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1410 microversion: 2.1 time: 0.320816 Sep 27 10:26:31.510854 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3026/5595] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:31 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1410 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:32.529612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-64b22404-c824-43cf-8eef-85b126bc2a45 req-84e0b16b-7347-4e41-b1b3-6064ce055eb3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc", "name": "network-vif-deleted", "tag": "ad654d98-8103-4e52-a2b9-095d297e78f7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:26:32.538851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e5737bd4-5243-4c1f-9e72-40600482cf7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:32.540984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e5737bd4-5243-4c1f-9e72-40600482cf7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:32.551481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-64b22404-c824-43cf-8eef-85b126bc2a45 req-84e0b16b-7347-4e41-b1b3-6064ce055eb3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:32.552422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-64b22404-c824-43cf-8eef-85b126bc2a45 req-84e0b16b-7347-4e41-b1b3-6064ce055eb3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:32.564973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5737bd4-5243-4c1f-9e72-40600482cf7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:32.565682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5737bd4-5243-4c1f-9e72-40600482cf7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:32.566588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5737bd4-5243-4c1f-9e72-40600482cf7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:32.567562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5737bd4-5243-4c1f-9e72-40600482cf7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:32.568335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5737bd4-5243-4c1f-9e72-40600482cf7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:32.569419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-64b22404-c824-43cf-8eef-85b126bc2a45 req-84e0b16b-7347-4e41-b1b3-6064ce055eb3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:32.569942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5737bd4-5243-4c1f-9e72-40600482cf7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:32.593922 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-64b22404-c824-43cf-8eef-85b126bc2a45 req-84e0b16b-7347-4e41-b1b3-6064ce055eb3 service nova] Creating event network-vif-deleted:ad654d98-8103-4e52-a2b9-095d297e78f7 for instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc on nch2-master-1 Sep 27 10:26:32.603019 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-64b22404-c824-43cf-8eef-85b126bc2a45 req-84e0b16b-7347-4e41-b1b3-6064ce055eb3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079658 Sep 27 10:26:32.604126 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2570/5596] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:26:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:26:33.098380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e5737bd4-5243-4c1f-9e72-40600482cf7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:33.115658 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e5737bd4-5243-4c1f-9e72-40600482cf7c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 200 len: 1368 microversion: 2.1 time: 0.581623 Sep 27 10:26:33.115793 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3027/5597] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:32 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 1368 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:26:34.141109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:34.142164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:34.160427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:34.161142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:34.161829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:34.170069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:34.171313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:34.172874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:34.244933 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc HTTP/1.1 Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc'} Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:34.250727 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:34.256800 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:34.256800 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:34.256800 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:34.256800 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:26:34.256800 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:26:34.256800 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:34.256800 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:34.256800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:26:34.256800 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dbb97d8c-9508-4aab-ba6a-2ab6a98f91a4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 404 len: 111 microversion: 2.1 time: 0.117770 Sep 27 10:26:34.256800 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2571/5598] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:34 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 111 bytes in 121 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:26:34.345758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-46f1bd58-da23-4893-93f6-89d46d939fcf tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:34.349163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-46f1bd58-da23-4893-93f6-89d46d939fcf tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:34.358422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46f1bd58-da23-4893-93f6-89d46d939fcf tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:34.359218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46f1bd58-da23-4893-93f6-89d46d939fcf tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:34.359942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46f1bd58-da23-4893-93f6-89d46d939fcf tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:34.361021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46f1bd58-da23-4893-93f6-89d46d939fcf tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:34.361776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46f1bd58-da23-4893-93f6-89d46d939fcf tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:34.362388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46f1bd58-da23-4893-93f6-89d46d939fcf tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:34.402112 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-46f1bd58-da23-4893-93f6-89d46d939fcf tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.061252 Sep 27 10:26:34.402776 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3028/5599] 10.222.0.26 () {60 vars in 1343 bytes} [Tue Sep 27 10:26:34 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401/os-volume_attachments => generated 25 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:34.519562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-68e24c3d-674c-41f6-b4bc-58473ebd73d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:34.520683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-68e24c3d-674c-41f6-b4bc-58473ebd73d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:34.529465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68e24c3d-674c-41f6-b4bc-58473ebd73d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:34.529972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68e24c3d-674c-41f6-b4bc-58473ebd73d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:34.530421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68e24c3d-674c-41f6-b4bc-58473ebd73d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:34.531144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68e24c3d-674c-41f6-b4bc-58473ebd73d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:34.531610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68e24c3d-674c-41f6-b4bc-58473ebd73d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:34.532048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68e24c3d-674c-41f6-b4bc-58473ebd73d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:34.585253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-68e24c3d-674c-41f6-b4bc-58473ebd73d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:26:34.731299 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-68e24c3d-674c-41f6-b4bc-58473ebd73d3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "DELETE /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 204 len: 0 microversion: 2.20 time: 0.218343 Sep 27 10:26:34.736279 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2572/5600] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:26:34 2022] DELETE /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:26:34.750216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:34.750881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:34.761173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:34.761681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:34.766803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:34.769511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:34.770021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:34.771515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:34.935081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:34.945187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:34.947450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:34.949091 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-298e2ef6-45ec-41ec-8f05-da24cd85e862 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1722 microversion: 2.20 time: 0.208808 Sep 27 10:26:34.949669 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3029/5601] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:26:34 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1722 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 27 10:26:35.378311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:35.378979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:35.394968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:35.396081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:35.397189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:35.398218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:35.399141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:35.399828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:35.418537 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df HTTP/1.1 Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '04dc4859-cd17-4187-8019-e2bcf346a7df'} Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:26:35.421521 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f96e19d3-ff9f-4f94-ac3e-3f85ee7910c1 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "DELETE /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 404 len: 111 microversion: 2.1 time: 0.049516 Sep 27 10:26:35.426319 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2573/5602] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:26:35 2022] DELETE /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:26:35.432508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:35.433251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 04dc4859-cd17-4187-8019-e2bcf346a7df] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:35.442534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:35.443717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:35.444212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:35.445179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:35.445668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:35.446241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:35.463151 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df HTTP/1.1 Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '04dc4859-cd17-4187-8019-e2bcf346a7df'} Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:35.464892 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:35.469989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:35.469989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:35.469989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:35.469989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:26:35.469989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:26:35.469989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. Sep 27 10:26:35.469989 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:35.469989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance 04dc4859-cd17-4187-8019-e2bcf346a7df could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:26:35.471920 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4343f2e5-1c2e-428d-8b10-80989c5700b0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df" status: 404 len: 111 microversion: 2.1 time: 0.037641 Sep 27 10:26:35.472559 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3030/5603] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:35 2022] GET /compute/v2.1/servers/04dc4859-cd17-4187-8019-e2bcf346a7df => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:26:35.985680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:35.986316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:35.992768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:35.993501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:35.994774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:35.995995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:35.996673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:35.997297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:36.191484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:36.191484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:36.199858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:36.199858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:36.199858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:36.200412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:36.201336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:36.202481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:36.237114 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc HTTP/1.1 Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc'} Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:26:36.241077 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:26:36.245145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:36.245145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:36.245145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:36.245145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:36.245145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:36.245145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:36.245145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:36.245145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:26:36.245145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:26:36.245145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:36.245145 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:36.247737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:26:36.261559 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b5c40270-ce7f-4441-b599-d57ed5a553de tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 404 len: 111 microversion: 2.1 time: 0.065731 Sep 27 10:26:36.266168 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3031/5604] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:26:36 2022] DELETE /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 111 bytes in 82 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:26:36.271919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:36.272687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:36.280283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:36.280966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:36.281414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:36.282144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:36.282617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:36.283044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:36.300560 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc HTTP/1.1 Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc'} Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:36.302722 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:36.307144 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:36.307144 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:36.307144 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:36.307144 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:26:36.307144 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:26:36.307144 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. Sep 27 10:26:36.307144 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:36.309377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:26:36.310973 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6e57e4eb-4375-4a1c-8bec-4c2ce9d6359c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc" status: 404 len: 111 microversion: 2.1 time: 0.043856 Sep 27 10:26:36.311746 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3032/5605] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:36 2022] GET /compute/v2.1/servers/a2dc6bde-f7b4-48e5-b59a-e30b85fcb4bc => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:26:36.409068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f4906673-2ddc-4471-8e6c-df601829415f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Action: 'create', calling method: .version_select at 0x7f7f623fb430>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-362079488", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLg/2uP/YfeEnpmvFp7TAMotoEG45k4vGIPPnchqJbVkLo0Vf8cOI8q9jT54yhZcN3DsqN50TklstaGoDMbFTVYwZKPM0ppMlEeq0uPOP76OJoBS2StmC6udnpsegloNGw=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:26:36.434423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f4906673-2ddc-4471-8e6c-df601829415f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:26:36.443162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f4906673-2ddc-4471-8e6c-df601829415f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for user 178956f6c8154a06a0710b295aff3086 and project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:26:36.481454 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f4906673-2ddc-4471-8e6c-df601829415f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.076713 Sep 27 10:26:36.483296 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3033/5606] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:26:36 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:26:36.679447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:36.689571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:36.691196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:36.692684 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f6731332-51c8-4c16-b55b-708c4753d1c0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1722 microversion: 2.20 time: 0.711841 Sep 27 10:26:36.693483 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2574/5607] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:26:35 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1722 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:26:37.485205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5e54afcf-b744-4d1c-980e-895e4b8cb8db req-f1d4a350-76a2-4bee-af3e-306ee1c03ecd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77037d39-844c-4761-88cc-899710443401", "name": "network-vif-deleted", "tag": "6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:26:37.508100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5e54afcf-b744-4d1c-980e-895e4b8cb8db req-f1d4a350-76a2-4bee-af3e-306ee1c03ecd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:37.508592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5e54afcf-b744-4d1c-980e-895e4b8cb8db req-f1d4a350-76a2-4bee-af3e-306ee1c03ecd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:37.510129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5e54afcf-b744-4d1c-980e-895e4b8cb8db req-f1d4a350-76a2-4bee-af3e-306ee1c03ecd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:37.542520 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5e54afcf-b744-4d1c-980e-895e4b8cb8db req-f1d4a350-76a2-4bee-af3e-306ee1c03ecd service nova] Creating event network-vif-deleted:6fa6d7a3-1f69-4964-8759-8f7bd10d5fd1 for instance 77037d39-844c-4761-88cc-899710443401 on nch1-master-1 Sep 27 10:26:37.551714 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5e54afcf-b744-4d1c-980e-895e4b8cb8db req-f1d4a350-76a2-4bee-af3e-306ee1c03ecd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070489 Sep 27 10:26:37.554032 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3034/5608] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:26:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:26:37.721879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:37.723216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:37.734158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:37.734657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:37.737390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:37.739268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:37.740189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:37.740937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:37.788702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:37.789326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:37.789784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:37.790246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lazy-loading 'fault' on Instance uuid 77037d39-844c-4761-88cc-899710443401 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:26:37.938389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:37.954897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:37.957873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '74878b715fc94b23bb139cb30d8d20c0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b590d2de576e41aab42f79e4daef9c63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:37.960160 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8a9877d4-e68b-421b-8140-9ec1aeaf1ff3 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 200 len: 1370 microversion: 2.20 time: 0.242692 Sep 27 10:26:37.962489 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2575/5609] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:26:37 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 1370 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 27 10:26:38.996081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:38.999832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:39.019903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:39.020569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:39.021131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:39.021959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:39.022422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:39.022930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:39.058941 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] HTTP exception thrown: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Request method failure captured: Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 HTTP/1.1 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs43MkA6YH_ixExFA1qRpWwXmXGbF-X_Bicstp25yt5cwZaJ_7_F6w6FNYvUTVn_zcFt-0zaviYlvixPIic7TZLOz3r-vbaQPyBInmtKjHDdi7-muE6EYrBHzESgBlFeUkDC1UgwRkklVZ1DXjJLATMTWHchIJRaap3LucgH2a8cfv4 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b590d2de576e41aab42f79e4daef9c63"}]}] Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 74878b715fc94b23bb139cb30d8d20c0 Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '77037d39-844c-4761-88cc-899710443401'} Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:39.065406 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:39.069717 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:39.069717 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:39.069717 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:39.069717 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:39.069717 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:26:39.069717 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:26:39.069717 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:39.069717 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:39.069717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Returning 404 to user: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:26:39.069717 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-61f63f33-b822-494e-ac3e-9935dd59a0cd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 404 len: 111 microversion: 2.20 time: 0.081464 Sep 27 10:26:39.070364 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3035/5610] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:26:38 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 111 bytes in 83 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:26:39.083584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3a455107-0804-43a5-8dbb-e74d8ad9ddfd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '.version_select at 0x7f7f636078b0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:39.136129 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3a455107-0804-43a5-8dbb-e74d8ad9ddfd tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-80707728" status: 204 len: 0 microversion: 2.20 time: 0.059202 Sep 27 10:26:39.140827 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2576/5611] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:26:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-80707728 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 27 10:26:40.006891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:40.008057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:40.019777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:40.020419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:40.021157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:40.022166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:40.022729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:40.023486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:40.769601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lazy-loading 'services' on Instance uuid 7ac45a77-b872-4aab-93f2-25556deb0d3f {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:26:40.904865 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c4ce39b-fc09-49cc-8bb6-919f4d63e2c6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f/os-volume_attachments/1962f1c2-cd1b-44c0-ab12-4f391cd8acf5" status: 202 len: 0 microversion: 2.1 time: 0.903209 Sep 27 10:26:40.910638 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3036/5612] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:26:39 2022] DELETE /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f/os-volume_attachments/1962f1c2-cd1b-44c0-ab12-4f391cd8acf5 => generated 0 bytes in 909 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:26:41.527879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:41.529478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:41.537256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:41.538258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:41.539127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:41.540277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:41.541103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:41.541720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:41.561024 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] HTTP exception thrown: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Request method failure captured: Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 HTTP/1.1 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs43MkA6YH_ixExFA1qRpWwXmXGbF-X_Bicstp25yt5cwZaJ_7_F6w6FNYvUTVn_zcFt-0zaviYlvixPIic7TZLOz3r-vbaQPyBInmtKjHDdi7-muE6EYrBHzESgBlFeUkDC1UgwRkklVZ1DXjJLATMTWHchIJRaap3LucgH2a8cfv4 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b590d2de576e41aab42f79e4daef9c63"}]}] Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 74878b715fc94b23bb139cb30d8d20c0 Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '77037d39-844c-4761-88cc-899710443401'} Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:26:41.562657 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:26:41.568621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:41.568621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:41.568621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:41.568621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:41.568621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:41.568621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:41.568621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:41.568621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:26:41.568621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:26:41.568621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:41.568621 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:26:41.571024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Returning 404 to user: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:26:41.572198 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f2faa215-de39-42ad-82c3-282b554aabb0 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "DELETE /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 404 len: 111 microversion: 2.20 time: 0.050539 Sep 27 10:26:41.573088 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2577/5613] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:26:41 2022] DELETE /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:26:41.581648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:41.582912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 77037d39-844c-4761-88cc-899710443401] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:41.596640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:41.597179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:41.597616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:41.619012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:41.619553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:41.622124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:41.648198 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] HTTP exception thrown: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Request method failure captured: Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 HTTP/1.1 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs43MkA6YH_ixExFA1qRpWwXmXGbF-X_Bicstp25yt5cwZaJ_7_F6w6FNYvUTVn_zcFt-0zaviYlvixPIic7TZLOz3r-vbaQPyBInmtKjHDdi7-muE6EYrBHzESgBlFeUkDC1UgwRkklVZ1DXjJLATMTWHchIJRaap3LucgH2a8cfv4 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b590d2de576e41aab42f79e4daef9c63"}]}] Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 74878b715fc94b23bb139cb30d8d20c0 Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '77037d39-844c-4761-88cc-899710443401'} Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:41.650123 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:41.654828 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:41.654828 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:41.654828 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:41.654828 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:41.654828 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:26:41.654828 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:26:41.654828 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. Sep 27 10:26:41.654828 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:41.654828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Returning 404 to user: Instance 77037d39-844c-4761-88cc-899710443401 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:26:41.654828 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2c3bf182-51dc-40f2-bf5b-ade2a18e2586 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401" status: 404 len: 111 microversion: 2.20 time: 0.074363 Sep 27 10:26:41.657366 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3037/5614] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:26:41 2022] GET /compute/v2.1/servers/77037d39-844c-4761-88cc-899710443401 => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:26:41.970502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1361466202", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-362079488", "security_groups": [{"name": "tempest-secgroup-smoke-883528541"}], "block_device_mapping_v2": [{"uuid": "8489f678-48b9-47ca-8f5e-397c453365e4", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "1e417b59-62bb-4007-a78a-c15c153efd33"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:26:42.126498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef863f8f-5bc7-45a7-8bf7-b504120e56be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:42.127021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ef863f8f-5bc7-45a7-8bf7-b504120e56be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:42.137270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef863f8f-5bc7-45a7-8bf7-b504120e56be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:42.139418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef863f8f-5bc7-45a7-8bf7-b504120e56be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:42.139879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef863f8f-5bc7-45a7-8bf7-b504120e56be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:42.141907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef863f8f-5bc7-45a7-8bf7-b504120e56be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:42.142971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef863f8f-5bc7-45a7-8bf7-b504120e56be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:42.143574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef863f8f-5bc7-45a7-8bf7-b504120e56be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:42.199807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ef863f8f-5bc7-45a7-8bf7-b504120e56be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:26:42.223537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:42.232268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] validate_networks() for [('1e417b59-62bb-4007-a78a-c15c153efd33', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:26:42.393076 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef863f8f-5bc7-45a7-8bf7-b504120e56be tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 204 len: 0 microversion: 2.1 time: 0.272689 Sep 27 10:26:42.394206 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3038/5615] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:26:42 2022] DELETE /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:26:42.406502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c21f630c-40fb-4866-a53d-c1fda5c70dc6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:42.407839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c21f630c-40fb-4866-a53d-c1fda5c70dc6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:42.417747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21f630c-40fb-4866-a53d-c1fda5c70dc6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:42.418702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21f630c-40fb-4866-a53d-c1fda5c70dc6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:42.419369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21f630c-40fb-4866-a53d-c1fda5c70dc6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:42.420249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21f630c-40fb-4866-a53d-c1fda5c70dc6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:42.420813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21f630c-40fb-4866-a53d-c1fda5c70dc6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:42.421345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c21f630c-40fb-4866-a53d-c1fda5c70dc6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:42.687093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c21f630c-40fb-4866-a53d-c1fda5c70dc6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:42.707935 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c21f630c-40fb-4866-a53d-c1fda5c70dc6 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1513 microversion: 2.1 time: 0.308388 Sep 27 10:26:42.711952 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3039/5616] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:42 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1513 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:42.940508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:42.941173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:42.948138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:42.948913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:42.949698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:42.950658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:42.951353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:42.951991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:42.970682 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] HTTP exception thrown: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Request method failure captured: Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d HTTP/1.1 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs43MkA6YH_ixExFA1qRpWwXmXGbF-X_Bicstp25yt5cwZaJ_7_F6w6FNYvUTVn_zcFt-0zaviYlvixPIic7TZLOz3r-vbaQPyBInmtKjHDdi7-muE6EYrBHzESgBlFeUkDC1UgwRkklVZ1DXjJLATMTWHchIJRaap3LucgH2a8cfv4 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b590d2de576e41aab42f79e4daef9c63"}]}] Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 74878b715fc94b23bb139cb30d8d20c0 Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '691b2be7-d5b8-409c-ba22-5ab8da50b80d'} Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:26:42.972885 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:26:42.979230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:26:42.979230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:42.979230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:42.979230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:42.979230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:42.979230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:42.979230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:42.979230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:26:42.979230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:26:42.979230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:26:42.979230 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:42.981715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Returning 404 to user: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:26:42.982653 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-52880d1d-db6d-45ae-a1f4-8442c7f3ed82 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "DELETE /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 404 len: 111 microversion: 2.20 time: 0.046345 Sep 27 10:26:42.985456 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3040/5617] 10.222.0.26 () {60 vars in 1280 bytes} [Tue Sep 27 10:26:42 2022] DELETE /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:26:43.042560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:43.043202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] [instance: 691b2be7-d5b8-409c-ba22-5ab8da50b80d] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:43.059350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:43.060112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:43.063851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:43.078380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:43.079076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:43.081093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:43.105849 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] HTTP exception thrown: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Request method failure captured: Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d HTTP/1.1 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs43MkA6YH_ixExFA1qRpWwXmXGbF-X_Bicstp25yt5cwZaJ_7_F6w6FNYvUTVn_zcFt-0zaviYlvixPIic7TZLOz3r-vbaQPyBInmtKjHDdi7-muE6EYrBHzESgBlFeUkDC1UgwRkklVZ1DXjJLATMTWHchIJRaap3LucgH2a8cfv4 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Openstack-Nova-Api-Version: 2.20 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b590d2de576e41aab42f79e4daef9c63"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b590d2de576e41aab42f79e4daef9c63"}]}] Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b590d2de576e41aab42f79e4daef9c63 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1912240963 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 74878b715fc94b23bb139cb30d8d20c0 Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1912240963-project Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '691b2be7-d5b8-409c-ba22-5ab8da50b80d'} Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:43.123310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:43.130050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:43.130050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:43.130050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:43.130050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:43.130050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:26:43.130050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:26:43.130050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. Sep 27 10:26:43.130050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:43.130050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] Returning 404 to user: Instance 691b2be7-d5b8-409c-ba22-5ab8da50b80d could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:26:43.130050 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a00dcd8a-1f81-4d83-a0b2-03cf793cb757 tempest-AttachVolumeShelveTestJSON-1912240963 tempest-AttachVolumeShelveTestJSON-1912240963-project] 10.222.0.26 "GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d" status: 404 len: 111 microversion: 2.20 time: 0.096846 Sep 27 10:26:43.130050 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3041/5618] 10.222.0.26 () {60 vars in 1277 bytes} [Tue Sep 27 10:26:43 2022] GET /compute/v2.1/servers/691b2be7-d5b8-409c-ba22-5ab8da50b80d => generated 111 bytes in 99 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:26:43.742535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-98cec47c-ceda-4b2f-b865-8e2cdc0d137b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:43.760105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-98cec47c-ceda-4b2f-b865-8e2cdc0d137b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:43.772783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98cec47c-ceda-4b2f-b865-8e2cdc0d137b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:43.777463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98cec47c-ceda-4b2f-b865-8e2cdc0d137b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:43.781059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98cec47c-ceda-4b2f-b865-8e2cdc0d137b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:43.782092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98cec47c-ceda-4b2f-b865-8e2cdc0d137b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:43.782765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98cec47c-ceda-4b2f-b865-8e2cdc0d137b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:43.783916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-98cec47c-ceda-4b2f-b865-8e2cdc0d137b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:43.884193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:26:43.884742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:26:43.885329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:26:43.886046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:26:43.887236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:26:43.908831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:43.909697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:43.910361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:43.911345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:43.912154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:43.919510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:43.956107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:26:43.965188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for user 178956f6c8154a06a0710b295aff3086 and project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:26:44.062650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:26:44.065489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='8489f678-48b9-47ca-8f5e-397c453365e4',volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:26:44.128715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-98cec47c-ceda-4b2f-b865-8e2cdc0d137b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:44.137216 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-98cec47c-ceda-4b2f-b865-8e2cdc0d137b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1513 microversion: 2.1 time: 0.409425 Sep 27 10:26:44.137964 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3042/5619] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:43 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1513 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:26:44.260081 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-18d07a7c-b409-4540-a6d3-709bb588c326 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 2.294145 Sep 27 10:26:44.261156 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2578/5620] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:26:41 2022] POST /compute/v2.1/servers => generated 405 bytes in 2296 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:26:44.273748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b368a1df-c3aa-4ba6-85ba-afaf8cc540c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:44.275470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b368a1df-c3aa-4ba6-85ba-afaf8cc540c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:44.903494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b368a1df-c3aa-4ba6-85ba-afaf8cc540c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:44.913812 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b368a1df-c3aa-4ba6-85ba-afaf8cc540c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1094 microversion: 2.1 time: 0.645290 Sep 27 10:26:44.915023 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3043/5621] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:44 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1094 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:45.144078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b1bc0e80-ebff-4986-bf4b-de7692392634 req-81409943-b47a-4f57-b6b2-8900b841ab57 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ac45a77-b872-4aab-93f2-25556deb0d3f", "name": "network-vif-deleted", "tag": "52b80ca4-891c-420a-a226-c0f942315728"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:26:45.163574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-04be49ac-e0be-4670-91b8-78c4afc0b7de tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:45.165224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-04be49ac-e0be-4670-91b8-78c4afc0b7de tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:45.182836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b1bc0e80-ebff-4986-bf4b-de7692392634 req-81409943-b47a-4f57-b6b2-8900b841ab57 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:45.183336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b1bc0e80-ebff-4986-bf4b-de7692392634 req-81409943-b47a-4f57-b6b2-8900b841ab57 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:45.183780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b1bc0e80-ebff-4986-bf4b-de7692392634 req-81409943-b47a-4f57-b6b2-8900b841ab57 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:45.185638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04be49ac-e0be-4670-91b8-78c4afc0b7de tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:45.186371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04be49ac-e0be-4670-91b8-78c4afc0b7de tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:45.186981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04be49ac-e0be-4670-91b8-78c4afc0b7de tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:45.188081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04be49ac-e0be-4670-91b8-78c4afc0b7de tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:45.194493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04be49ac-e0be-4670-91b8-78c4afc0b7de tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:45.195277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04be49ac-e0be-4670-91b8-78c4afc0b7de tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:45.198220 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b1bc0e80-ebff-4986-bf4b-de7692392634 req-81409943-b47a-4f57-b6b2-8900b841ab57 service nova] Creating event network-vif-deleted:52b80ca4-891c-420a-a226-c0f942315728 for instance 7ac45a77-b872-4aab-93f2-25556deb0d3f on nch1-master-1 Sep 27 10:26:45.203767 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b1bc0e80-ebff-4986-bf4b-de7692392634 req-81409943-b47a-4f57-b6b2-8900b841ab57 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063901 Sep 27 10:26:45.210313 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2579/5622] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:26:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:26:45.335961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-04be49ac-e0be-4670-91b8-78c4afc0b7de tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:45.348853 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-04be49ac-e0be-4670-91b8-78c4afc0b7de tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 200 len: 1471 microversion: 2.1 time: 0.189869 Sep 27 10:26:45.349714 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3044/5623] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:45 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 1471 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:45.941695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d6468da5-6f00-4740-8559-34b99a176fed tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:45.943098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d6468da5-6f00-4740-8559-34b99a176fed tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:45.955838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6468da5-6f00-4740-8559-34b99a176fed tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:45.956352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6468da5-6f00-4740-8559-34b99a176fed tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:45.961265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6468da5-6f00-4740-8559-34b99a176fed tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:45.964072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6468da5-6f00-4740-8559-34b99a176fed tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:45.964782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6468da5-6f00-4740-8559-34b99a176fed tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:45.967799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6468da5-6f00-4740-8559-34b99a176fed tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:46.132447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d6468da5-6f00-4740-8559-34b99a176fed tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:46.142648 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d6468da5-6f00-4740-8559-34b99a176fed tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1200 microversion: 2.1 time: 0.206785 Sep 27 10:26:46.143276 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2580/5624] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:45 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1200 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:46.373884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:46.374491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 7ac45a77-b872-4aab-93f2-25556deb0d3f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:46.381657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:46.382145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:46.382580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:46.383286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:46.383958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:46.384385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:46.398955 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] HTTP exception thrown: Instance 7ac45a77-b872-4aab-93f2-25556deb0d3f could not be found. Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Request method failure captured: Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f HTTP/1.1 Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs9MjI_0KcnHJXSGLHBgvPUwQslys5FrzbZdDzR4LHS5P20obfvDE55uJC2bWHinkoUYLZCvPzz-e9beg48lQaNqfrNLKsX3K3Ey8lJ8snaERSFExgnBV6ls2CawRaQyxkeK5OON4LxIbGkC2jNntAUgLFOke0b-3XrL-UkjH5k7RRo Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 712c1c4c858b4b788a7703345716504b Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1705236581 Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/712c1c4c858b4b788a7703345716504b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/712c1c4c858b4b788a7703345716504b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/712c1c4c858b4b788a7703345716504b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_712c1c4c858b4b788a7703345716504b"}]}] Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesSnapshotTestJSON-1705236581 Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 712c1c4c858b4b788a7703345716504b Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1705236581 Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesSnapshotTestJSON-1705236581-project Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 584c395757a54d9b9a64dd399d8557b1 Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesSnapshotTestJSON-1705236581-project Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 7ac45a77-b872-4aab-93f2-25556deb0d3f could not be found. Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '7ac45a77-b872-4aab-93f2-25556deb0d3f'} Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7ac45a77-b872-4aab-93f2-25556deb0d3f could not be found. Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7ac45a77-b872-4aab-93f2-25556deb0d3f could not be found. Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:26:46.400508 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:46.404644 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:26:46.404644 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:26:46.404644 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:26:46.404644 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:26:46.404644 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:26:46.404644 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7ac45a77-b872-4aab-93f2-25556deb0d3f could not be found. Sep 27 10:26:46.404644 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:26:46.406239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Returning 404 to user: Instance 7ac45a77-b872-4aab-93f2-25556deb0d3f could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:26:46.407109 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7e01ff02-e868-43e9-a8da-37c6921c7bfa tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f" status: 404 len: 111 microversion: 2.1 time: 0.037162 Sep 27 10:26:46.407677 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3045/5625] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:46 2022] GET /compute/v2.1/servers/7ac45a77-b872-4aab-93f2-25556deb0d3f => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:26:47.169945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2a3ba8c3-4105-437d-aee0-cbe9848254f9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:47.170676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2a3ba8c3-4105-437d-aee0-cbe9848254f9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:47.184130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3ba8c3-4105-437d-aee0-cbe9848254f9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:47.184955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3ba8c3-4105-437d-aee0-cbe9848254f9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:47.185581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3ba8c3-4105-437d-aee0-cbe9848254f9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:47.186330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3ba8c3-4105-437d-aee0-cbe9848254f9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:47.186958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3ba8c3-4105-437d-aee0-cbe9848254f9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:47.187547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3ba8c3-4105-437d-aee0-cbe9848254f9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:47.353679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2a3ba8c3-4105-437d-aee0-cbe9848254f9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:47.364478 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2a3ba8c3-4105-437d-aee0-cbe9848254f9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1210 microversion: 2.1 time: 0.199631 Sep 27 10:26:47.367419 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2581/5626] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:47 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1210 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:48.396529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-959c7bba-ffdc-4b91-b130-a92ab2cc20c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:48.398863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-959c7bba-ffdc-4b91-b130-a92ab2cc20c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:48.413778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=72,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:48.416506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-959c7bba-ffdc-4b91-b130-a92ab2cc20c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:48.417472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-959c7bba-ffdc-4b91-b130-a92ab2cc20c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:48.419113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-959c7bba-ffdc-4b91-b130-a92ab2cc20c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:48.421517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-959c7bba-ffdc-4b91-b130-a92ab2cc20c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:48.424572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-959c7bba-ffdc-4b91-b130-a92ab2cc20c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:48.425300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-959c7bba-ffdc-4b91-b130-a92ab2cc20c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:48.864445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-959c7bba-ffdc-4b91-b130-a92ab2cc20c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:48.877629 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-959c7bba-ffdc-4b91-b130-a92ab2cc20c9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1265 microversion: 2.1 time: 0.490010 Sep 27 10:26:48.878529 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3046/5627] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:48 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1265 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:49.692557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a23c9ca5-98b9-47c9-bbae-a727d9b8c725 req-2a12a0fc-cd45-4aa1-a592-47954c5b7e64 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "82b31a6f-a898-4796-a3ad-1628306fbe7b", "tag": "f840e3d8-9c94-4d2d-b759-279dceffb800"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:26:49.717714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=56,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:49.736491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a23c9ca5-98b9-47c9-bbae-a727d9b8c725 req-2a12a0fc-cd45-4aa1-a592-47954c5b7e64 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:49.740156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a23c9ca5-98b9-47c9-bbae-a727d9b8c725 req-2a12a0fc-cd45-4aa1-a592-47954c5b7e64 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:49.745669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a23c9ca5-98b9-47c9-bbae-a727d9b8c725 req-2a12a0fc-cd45-4aa1-a592-47954c5b7e64 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:49.754105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=83,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:49.809040 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a23c9ca5-98b9-47c9-bbae-a727d9b8c725 req-2a12a0fc-cd45-4aa1-a592-47954c5b7e64 service nova] Creating event network-changed:f840e3d8-9c94-4d2d-b759-279dceffb800 for instance 82b31a6f-a898-4796-a3ad-1628306fbe7b on nch2-master-1 Sep 27 10:26:49.832831 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a23c9ca5-98b9-47c9-bbae-a727d9b8c725 req-2a12a0fc-cd45-4aa1-a592-47954c5b7e64 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.149491 Sep 27 10:26:49.834310 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2582/5628] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:26:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:26:49.901313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0c33663-fb99-40c5-9268-044ecfb21db8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:49.901970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f0c33663-fb99-40c5-9268-044ecfb21db8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:49.910591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0c33663-fb99-40c5-9268-044ecfb21db8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:49.911378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0c33663-fb99-40c5-9268-044ecfb21db8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:49.912615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0c33663-fb99-40c5-9268-044ecfb21db8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:49.914497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0c33663-fb99-40c5-9268-044ecfb21db8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:49.916070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0c33663-fb99-40c5-9268-044ecfb21db8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:49.917306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0c33663-fb99-40c5-9268-044ecfb21db8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:50.484130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f0c33663-fb99-40c5-9268-044ecfb21db8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:50.532007 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f0c33663-fb99-40c5-9268-044ecfb21db8 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1265 microversion: 2.1 time: 0.632939 Sep 27 10:26:50.535150 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3047/5629] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:49 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1265 bytes in 638 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:51.558674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2a8ef223-548c-47e8-b1fd-b0e513e5b921 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:51.559423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2a8ef223-548c-47e8-b1fd-b0e513e5b921 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:51.571291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a8ef223-548c-47e8-b1fd-b0e513e5b921 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:51.571803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a8ef223-548c-47e8-b1fd-b0e513e5b921 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:51.575094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a8ef223-548c-47e8-b1fd-b0e513e5b921 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:51.576614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a8ef223-548c-47e8-b1fd-b0e513e5b921 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:51.578909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a8ef223-548c-47e8-b1fd-b0e513e5b921 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:51.579530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a8ef223-548c-47e8-b1fd-b0e513e5b921 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:51.814393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-242505968", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "ecc1560a-8770-41ea-af0f-93cdb9ee5b8d"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:26:51.928216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] validate_networks() for [('ecc1560a-8770-41ea-af0f-93cdb9ee5b8d', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:26:52.700612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:26:52.704003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:26:52.704713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:26:52.707132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:26:52.707812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:26:52.739689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:52.740192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:52.740773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:52.741744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:52.742208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:52.742636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:52.765350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Getting quotas for project 712c1c4c858b4b788a7703345716504b. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:26:52.772733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Getting quotas for user 584c395757a54d9b9a64dd399d8557b1 and project 712c1c4c858b4b788a7703345716504b. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:26:52.777384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2a8ef223-548c-47e8-b1fd-b0e513e5b921 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:52.787027 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2a8ef223-548c-47e8-b1fd-b0e513e5b921 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1265 microversion: 2.1 time: 1.235051 Sep 27 10:26:52.795992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:26:52.799434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:26:52.805634 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2583/5630] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:51 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1265 bytes in 1255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:52.918648 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9cd8e129-befe-427c-8de5-bf50e8b07413 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.106151 Sep 27 10:26:52.921552 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3048/5631] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:26:51 2022] POST /compute/v2.1/servers => generated 380 bytes in 1112 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:26:52.932327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5af1ff21-2d09-4607-8ad2-bf6c0beee838 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:52.932996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5af1ff21-2d09-4607-8ad2-bf6c0beee838 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:53.089356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5af1ff21-2d09-4607-8ad2-bf6c0beee838 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:53.101367 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5af1ff21-2d09-4607-8ad2-bf6c0beee838 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1218 microversion: 2.1 time: 0.172300 Sep 27 10:26:53.102538 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2584/5632] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:52 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1218 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:53.828218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9ffa1cb7-4b02-47a8-b611-4e12d8f27f53 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:53.828939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ffa1cb7-4b02-47a8-b611-4e12d8f27f53 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:53.843224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ffa1cb7-4b02-47a8-b611-4e12d8f27f53 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:53.844159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ffa1cb7-4b02-47a8-b611-4e12d8f27f53 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:53.845016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ffa1cb7-4b02-47a8-b611-4e12d8f27f53 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:53.847262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ffa1cb7-4b02-47a8-b611-4e12d8f27f53 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:53.847960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ffa1cb7-4b02-47a8-b611-4e12d8f27f53 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:53.850789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ffa1cb7-4b02-47a8-b611-4e12d8f27f53 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:54.124693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-db2a86a2-abc0-41d4-bc8e-2fdaa33a05a1 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:54.126089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-db2a86a2-abc0-41d4-bc8e-2fdaa33a05a1 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:54.130200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:26:54.149238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db2a86a2-abc0-41d4-bc8e-2fdaa33a05a1 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:54.149756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db2a86a2-abc0-41d4-bc8e-2fdaa33a05a1 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:54.150555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db2a86a2-abc0-41d4-bc8e-2fdaa33a05a1 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:54.151604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db2a86a2-abc0-41d4-bc8e-2fdaa33a05a1 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:54.152437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db2a86a2-abc0-41d4-bc8e-2fdaa33a05a1 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:54.153883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-db2a86a2-abc0-41d4-bc8e-2fdaa33a05a1 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:54.330324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-db2a86a2-abc0-41d4-bc8e-2fdaa33a05a1 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:54.341704 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-db2a86a2-abc0-41d4-bc8e-2fdaa33a05a1 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1270 microversion: 2.1 time: 0.220782 Sep 27 10:26:54.346075 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2585/5633] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:54 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1270 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:26:54.927829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ffa1cb7-4b02-47a8-b611-4e12d8f27f53 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:54.940485 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9ffa1cb7-4b02-47a8-b611-4e12d8f27f53 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1265 microversion: 2.1 time: 1.116441 Sep 27 10:26:54.942079 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3049/5634] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:53 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1265 bytes in 1119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:55.371614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c1283bd7-7927-4b9e-a38c-ed601ce74e64 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:55.372255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c1283bd7-7927-4b9e-a38c-ed601ce74e64 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:55.389152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1283bd7-7927-4b9e-a38c-ed601ce74e64 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:55.389679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1283bd7-7927-4b9e-a38c-ed601ce74e64 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:55.390686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1283bd7-7927-4b9e-a38c-ed601ce74e64 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:55.391465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1283bd7-7927-4b9e-a38c-ed601ce74e64 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:55.391952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1283bd7-7927-4b9e-a38c-ed601ce74e64 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:55.392396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1283bd7-7927-4b9e-a38c-ed601ce74e64 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:55.598745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c1283bd7-7927-4b9e-a38c-ed601ce74e64 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:55.616017 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c1283bd7-7927-4b9e-a38c-ed601ce74e64 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1276 microversion: 2.1 time: 0.248389 Sep 27 10:26:55.617056 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2586/5635] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:55 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1276 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:55.964018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9df9753a-cf75-4d14-b871-1f31a09e83b3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:55.964644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9df9753a-cf75-4d14-b871-1f31a09e83b3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:55.974614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9df9753a-cf75-4d14-b871-1f31a09e83b3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:55.975088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9df9753a-cf75-4d14-b871-1f31a09e83b3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:55.975528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9df9753a-cf75-4d14-b871-1f31a09e83b3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:55.976275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9df9753a-cf75-4d14-b871-1f31a09e83b3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:55.976755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9df9753a-cf75-4d14-b871-1f31a09e83b3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:55.977541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9df9753a-cf75-4d14-b871-1f31a09e83b3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:56.295890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9df9753a-cf75-4d14-b871-1f31a09e83b3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:56.313213 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9df9753a-cf75-4d14-b871-1f31a09e83b3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1265 microversion: 2.1 time: 0.352242 Sep 27 10:26:56.315078 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3050/5636] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:55 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1265 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:56.640843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f51d705a-c5ee-4cfc-83f9-e013dd87a7e7 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:56.641521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f51d705a-c5ee-4cfc-83f9-e013dd87a7e7 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:56.655705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f51d705a-c5ee-4cfc-83f9-e013dd87a7e7 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:56.656242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f51d705a-c5ee-4cfc-83f9-e013dd87a7e7 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:56.656725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f51d705a-c5ee-4cfc-83f9-e013dd87a7e7 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:56.657545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f51d705a-c5ee-4cfc-83f9-e013dd87a7e7 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:56.658386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f51d705a-c5ee-4cfc-83f9-e013dd87a7e7 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:56.658519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f51d705a-c5ee-4cfc-83f9-e013dd87a7e7 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:56.949976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f51d705a-c5ee-4cfc-83f9-e013dd87a7e7 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:56.961209 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f51d705a-c5ee-4cfc-83f9-e013dd87a7e7 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1318 microversion: 2.1 time: 0.325107 Sep 27 10:26:56.962760 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2587/5637] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:56 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1318 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:26:57.341312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f60d274c-aa68-48be-964c-4c9c5390dafb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:57.342868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f60d274c-aa68-48be-964c-4c9c5390dafb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:57.357336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f60d274c-aa68-48be-964c-4c9c5390dafb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:57.359769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f60d274c-aa68-48be-964c-4c9c5390dafb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:57.361207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f60d274c-aa68-48be-964c-4c9c5390dafb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:57.371076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f60d274c-aa68-48be-964c-4c9c5390dafb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:57.371738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f60d274c-aa68-48be-964c-4c9c5390dafb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:57.372360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f60d274c-aa68-48be-964c-4c9c5390dafb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:57.987791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c3ed5290-f38a-4c38-9b1e-f972b70ab068 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:57.988437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c3ed5290-f38a-4c38-9b1e-f972b70ab068 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:58.016076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c3ed5290-f38a-4c38-9b1e-f972b70ab068 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:58.018557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c3ed5290-f38a-4c38-9b1e-f972b70ab068 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:58.019366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c3ed5290-f38a-4c38-9b1e-f972b70ab068 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:58.020542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c3ed5290-f38a-4c38-9b1e-f972b70ab068 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:58.021378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c3ed5290-f38a-4c38-9b1e-f972b70ab068 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:58.022100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c3ed5290-f38a-4c38-9b1e-f972b70ab068 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:58.100532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f60d274c-aa68-48be-964c-4c9c5390dafb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:58.110910 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f60d274c-aa68-48be-964c-4c9c5390dafb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1265 microversion: 2.1 time: 0.773830 Sep 27 10:26:58.114934 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3051/5638] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:57 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1265 bytes in 779 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:26:58.439581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b7c5905f-c888-41db-b065-b52b269450ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:58.440214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b7c5905f-c888-41db-b065-b52b269450ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:58.463622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7c5905f-c888-41db-b065-b52b269450ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:58.464509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7c5905f-c888-41db-b065-b52b269450ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:58.465193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7c5905f-c888-41db-b065-b52b269450ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:58.466152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7c5905f-c888-41db-b065-b52b269450ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:58.466808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7c5905f-c888-41db-b065-b52b269450ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:58.467503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7c5905f-c888-41db-b065-b52b269450ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:58.757994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c3ed5290-f38a-4c38-9b1e-f972b70ab068 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:58.769684 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c3ed5290-f38a-4c38-9b1e-f972b70ab068 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1318 microversion: 2.1 time: 0.783319 Sep 27 10:26:58.780934 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2588/5639] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:57 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1318 bytes in 791 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:26:59.149922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-41555164-370a-4e8c-8141-a0feab89dc90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:59.150599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-41555164-370a-4e8c-8141-a0feab89dc90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:59.164764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41555164-370a-4e8c-8141-a0feab89dc90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:59.175400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41555164-370a-4e8c-8141-a0feab89dc90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:59.177647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41555164-370a-4e8c-8141-a0feab89dc90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:59.178936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41555164-370a-4e8c-8141-a0feab89dc90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:59.179891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41555164-370a-4e8c-8141-a0feab89dc90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:59.180417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41555164-370a-4e8c-8141-a0feab89dc90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:59.707362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-41555164-370a-4e8c-8141-a0feab89dc90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:26:59.723777 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-41555164-370a-4e8c-8141-a0feab89dc90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1265 microversion: 2.1 time: 0.580121 Sep 27 10:26:59.742333 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2589/5640] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:59 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1265 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:26:59.812456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e784a09e-8d5f-4e4b-8554-c616b3ce84db tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:26:59.813236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e784a09e-8d5f-4e4b-8554-c616b3ce84db tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:26:59.849884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e784a09e-8d5f-4e4b-8554-c616b3ce84db tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:59.851785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e784a09e-8d5f-4e4b-8554-c616b3ce84db tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:59.852269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e784a09e-8d5f-4e4b-8554-c616b3ce84db tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:26:59.860143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e784a09e-8d5f-4e4b-8554-c616b3ce84db tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:26:59.860694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e784a09e-8d5f-4e4b-8554-c616b3ce84db tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:26:59.861179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e784a09e-8d5f-4e4b-8554-c616b3ce84db tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:00.012350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b7c5905f-c888-41db-b065-b52b269450ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:00.029856 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b7c5905f-c888-41db-b065-b52b269450ed tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 1.586039 Sep 27 10:27:00.046007 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3052/5641] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:58 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 1596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:00.059718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-21a2fd05-eb73-471f-9b3f-06a1ce31bbce tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:00.060523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-21a2fd05-eb73-471f-9b3f-06a1ce31bbce tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:00.076891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a2fd05-eb73-471f-9b3f-06a1ce31bbce tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:00.081279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a2fd05-eb73-471f-9b3f-06a1ce31bbce tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:00.081956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a2fd05-eb73-471f-9b3f-06a1ce31bbce tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:00.082905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a2fd05-eb73-471f-9b3f-06a1ce31bbce tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:00.083580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a2fd05-eb73-471f-9b3f-06a1ce31bbce tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:00.084113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-21a2fd05-eb73-471f-9b3f-06a1ce31bbce tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:00.398519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e784a09e-8d5f-4e4b-8554-c616b3ce84db tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:00.411333 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e784a09e-8d5f-4e4b-8554-c616b3ce84db tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1318 microversion: 2.1 time: 0.619069 Sep 27 10:27:00.412063 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2590/5642] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:26:59 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1318 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:00.636540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-fd34cd48-79cc-4979-b009-31a51554a583 req-dd299cf3-6622-4d85-8bba-a07ac4b768f3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9d4f13c2-3c04-4984-a293-df05378d29e5", "tag": "2c5322b1-7e1a-4d84-9b57-45c942d8b0e0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:00.664519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fd34cd48-79cc-4979-b009-31a51554a583 req-dd299cf3-6622-4d85-8bba-a07ac4b768f3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:00.665436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fd34cd48-79cc-4979-b009-31a51554a583 req-dd299cf3-6622-4d85-8bba-a07ac4b768f3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:00.667536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fd34cd48-79cc-4979-b009-31a51554a583 req-dd299cf3-6622-4d85-8bba-a07ac4b768f3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:00.705408 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-fd34cd48-79cc-4979-b009-31a51554a583 req-dd299cf3-6622-4d85-8bba-a07ac4b768f3 service nova] Creating event network-changed:2c5322b1-7e1a-4d84-9b57-45c942d8b0e0 for instance 9d4f13c2-3c04-4984-a293-df05378d29e5 on nch1-master-1 Sep 27 10:27:00.713710 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-fd34cd48-79cc-4979-b009-31a51554a583 req-dd299cf3-6622-4d85-8bba-a07ac4b768f3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.082537 Sep 27 10:27:00.716770 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2591/5643] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:00.726536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e5a7fe2c-9f1d-4a5d-a792-d998c0545cc9 req-660f61ba-171b-427d-b392-86b5024497a3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "82b31a6f-a898-4796-a3ad-1628306fbe7b", "name": "network-vif-plugged", "status": "completed", "tag": "f840e3d8-9c94-4d2d-b759-279dceffb800"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:00.750647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e5a7fe2c-9f1d-4a5d-a792-d998c0545cc9 req-660f61ba-171b-427d-b392-86b5024497a3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:00.751144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e5a7fe2c-9f1d-4a5d-a792-d998c0545cc9 req-660f61ba-171b-427d-b392-86b5024497a3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:00.753750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e5a7fe2c-9f1d-4a5d-a792-d998c0545cc9 req-660f61ba-171b-427d-b392-86b5024497a3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:00.786050 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e5a7fe2c-9f1d-4a5d-a792-d998c0545cc9 req-660f61ba-171b-427d-b392-86b5024497a3 service nova] Creating event network-vif-plugged:f840e3d8-9c94-4d2d-b759-279dceffb800 for instance 82b31a6f-a898-4796-a3ad-1628306fbe7b on nch2-master-1 Sep 27 10:27:00.813905 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e5a7fe2c-9f1d-4a5d-a792-d998c0545cc9 req-660f61ba-171b-427d-b392-86b5024497a3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075863 Sep 27 10:27:00.815433 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2592/5644] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:00.832542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f9405f1f-8cd0-42ba-83fc-f69c8341a83f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:00.852157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f9405f1f-8cd0-42ba-83fc-f69c8341a83f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:00.898552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9405f1f-8cd0-42ba-83fc-f69c8341a83f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:00.900418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9405f1f-8cd0-42ba-83fc-f69c8341a83f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:00.901226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9405f1f-8cd0-42ba-83fc-f69c8341a83f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:00.902688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9405f1f-8cd0-42ba-83fc-f69c8341a83f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:00.903401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9405f1f-8cd0-42ba-83fc-f69c8341a83f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:00.904368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9405f1f-8cd0-42ba-83fc-f69c8341a83f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:00.964387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-21a2fd05-eb73-471f-9b3f-06a1ce31bbce tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:00.976074 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-21a2fd05-eb73-471f-9b3f-06a1ce31bbce tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.919836 Sep 27 10:27:00.977145 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3053/5645] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:00 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 922 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:01.004350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bffb3198-ff4f-4849-ba09-2e704ed1226b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:01.005326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bffb3198-ff4f-4849-ba09-2e704ed1226b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:01.015804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bffb3198-ff4f-4849-ba09-2e704ed1226b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:01.017109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bffb3198-ff4f-4849-ba09-2e704ed1226b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:01.017720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bffb3198-ff4f-4849-ba09-2e704ed1226b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:01.022546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bffb3198-ff4f-4849-ba09-2e704ed1226b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:01.023644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bffb3198-ff4f-4849-ba09-2e704ed1226b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:01.024275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bffb3198-ff4f-4849-ba09-2e704ed1226b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:01.579093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bffb3198-ff4f-4849-ba09-2e704ed1226b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:01.591865 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bffb3198-ff4f-4849-ba09-2e704ed1226b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.591763 Sep 27 10:27:01.592722 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3054/5646] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:00 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:01.635671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a594f87d-5569-40db-aac6-860dec41db8b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:01.636300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a594f87d-5569-40db-aac6-860dec41db8b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:01.648266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a594f87d-5569-40db-aac6-860dec41db8b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:01.649168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a594f87d-5569-40db-aac6-860dec41db8b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:01.649867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a594f87d-5569-40db-aac6-860dec41db8b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:01.651064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a594f87d-5569-40db-aac6-860dec41db8b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:01.651750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a594f87d-5569-40db-aac6-860dec41db8b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:01.658398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a594f87d-5569-40db-aac6-860dec41db8b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:02.103032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f9405f1f-8cd0-42ba-83fc-f69c8341a83f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:02.118238 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f9405f1f-8cd0-42ba-83fc-f69c8341a83f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1265 microversion: 2.1 time: 1.297235 Sep 27 10:27:02.119366 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2593/5647] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:00 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1265 bytes in 1303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:02.136069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8d90fe89-3e1d-4970-b052-864e88dc5efa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:02.136785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8d90fe89-3e1d-4970-b052-864e88dc5efa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:02.143489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d90fe89-3e1d-4970-b052-864e88dc5efa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:02.149633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d90fe89-3e1d-4970-b052-864e88dc5efa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:02.151881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d90fe89-3e1d-4970-b052-864e88dc5efa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:02.156318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d90fe89-3e1d-4970-b052-864e88dc5efa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:02.158214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d90fe89-3e1d-4970-b052-864e88dc5efa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:02.161048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8d90fe89-3e1d-4970-b052-864e88dc5efa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:02.196942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a594f87d-5569-40db-aac6-860dec41db8b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:02.205018 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a594f87d-5569-40db-aac6-860dec41db8b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1318 microversion: 2.1 time: 0.611040 Sep 27 10:27:02.205616 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3055/5648] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:01 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1318 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:02.624774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8d90fe89-3e1d-4970-b052-864e88dc5efa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:02.634138 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8d90fe89-3e1d-4970-b052-864e88dc5efa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.514248 Sep 27 10:27:02.654158 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2594/5649] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:02 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:02.842471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-1204941566", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "247e3d7d-5912-42d7-9027-ebcb26003cfc"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:02.851625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:02.852310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=12 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:02.934664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] validate_networks() for [('247e3d7d-5912-42d7-9027-ebcb26003cfc', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:27:03.144373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6c5cfe96-c293-4879-a1bb-1da82559abe0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:03.145680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6c5cfe96-c293-4879-a1bb-1da82559abe0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:03.154233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:03.167010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c5cfe96-c293-4879-a1bb-1da82559abe0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:03.167670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c5cfe96-c293-4879-a1bb-1da82559abe0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:03.168137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c5cfe96-c293-4879-a1bb-1da82559abe0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:03.169136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c5cfe96-c293-4879-a1bb-1da82559abe0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:03.173448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c5cfe96-c293-4879-a1bb-1da82559abe0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:03.174331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c5cfe96-c293-4879-a1bb-1da82559abe0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:03.514260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6c5cfe96-c293-4879-a1bb-1da82559abe0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:03.531076 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6c5cfe96-c293-4879-a1bb-1da82559abe0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1265 microversion: 2.1 time: 0.391099 Sep 27 10:27:03.532988 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2595/5650] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:03 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1265 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:03.545171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1d2510f8-e511-4c79-a28b-2e309ec4ac44 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:03.546029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1d2510f8-e511-4c79-a28b-2e309ec4ac44 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:03.560799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d2510f8-e511-4c79-a28b-2e309ec4ac44 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:03.561324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d2510f8-e511-4c79-a28b-2e309ec4ac44 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:03.561780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d2510f8-e511-4c79-a28b-2e309ec4ac44 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:03.562520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d2510f8-e511-4c79-a28b-2e309ec4ac44 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:03.562995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d2510f8-e511-4c79-a28b-2e309ec4ac44 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:03.563442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d2510f8-e511-4c79-a28b-2e309ec4ac44 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:03.934054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1d2510f8-e511-4c79-a28b-2e309ec4ac44 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:03.947332 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1d2510f8-e511-4c79-a28b-2e309ec4ac44 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1318 microversion: 2.1 time: 0.413007 Sep 27 10:27:03.957708 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2596/5651] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:03 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1318 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:04.047561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:27:04.048110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:27:04.048689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:27:04.049396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:27:04.050815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:27:04.090031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:04.090921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:04.096069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:04.101456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:04.102136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:04.102744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:04.184510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for project 10375bd836f84383b39f2df73f53800e. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:27:04.202275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for user 97c769dbba794531b0a9db8e764ff097 and project 10375bd836f84383b39f2df73f53800e. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:27:04.243294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:27:04.254969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:27:04.446903 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2dfa1530-390e-4711-88ad-2ebb9563e229 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.608581 Sep 27 10:27:04.450850 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3056/5652] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:27:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 1613 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:27:04.474111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-08281d3c-31e9-424e-888b-f3f243a9abcd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:04.475121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-08281d3c-31e9-424e-888b-f3f243a9abcd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:04.482628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e596d133-dc99-4834-ac2b-1019f83e744d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:04.487477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e596d133-dc99-4834-ac2b-1019f83e744d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:04.487994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e596d133-dc99-4834-ac2b-1019f83e744d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:04.488450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e596d133-dc99-4834-ac2b-1019f83e744d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:04.489225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e596d133-dc99-4834-ac2b-1019f83e744d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:04.490428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e596d133-dc99-4834-ac2b-1019f83e744d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:04.490674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e596d133-dc99-4834-ac2b-1019f83e744d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:04.586117 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e596d133-dc99-4834-ac2b-1019f83e744d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/os-availability-zone/detail" status: 200 len: 613 microversion: 2.1 time: 0.442725 Sep 27 10:27:04.586719 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2597/5653] 10.222.0.26 () {58 vars in 1185 bytes} [Tue Sep 27 10:27:04 2022] GET /compute/v2.1/os-availability-zone/detail => generated 613 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:27:04.597154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3dac11b8-db79-49c6-a339-3dbf3883c54b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:04.599978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3dac11b8-db79-49c6-a339-3dbf3883c54b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:04.607094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dac11b8-db79-49c6-a339-3dbf3883c54b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:04.620013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dac11b8-db79-49c6-a339-3dbf3883c54b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:04.621189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dac11b8-db79-49c6-a339-3dbf3883c54b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:04.631049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dac11b8-db79-49c6-a339-3dbf3883c54b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:04.632677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dac11b8-db79-49c6-a339-3dbf3883c54b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:04.633876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3dac11b8-db79-49c6-a339-3dbf3883c54b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:04.638488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-08281d3c-31e9-424e-888b-f3f243a9abcd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:04.649399 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-08281d3c-31e9-424e-888b-f3f243a9abcd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1215 microversion: 2.1 time: 0.178126 Sep 27 10:27:04.657918 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3057/5654] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:04 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1215 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:04.883648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerMultinode-server-963342743", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "availability_zone": "nova:nch1-master-1", "networks": [{"uuid": "5dd82b97-2333-4934-8cc7-c9806488a076"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:05.101152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] validate_networks() for [('5dd82b97-2333-4934-8cc7-c9806488a076', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:27:05.522944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3dac11b8-db79-49c6-a339-3dbf3883c54b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:05.533463 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3dac11b8-db79-49c6-a339-3dbf3883c54b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1452 microversion: 2.1 time: 0.940015 Sep 27 10:27:05.534182 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2598/5655] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:04 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1452 bytes in 946 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:05.553981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0236cd3-daa0-4af6-89ae-0afbe08e84a5 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:05.565348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b0236cd3-daa0-4af6-89ae-0afbe08e84a5 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:05.582424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0236cd3-daa0-4af6-89ae-0afbe08e84a5 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:05.583172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0236cd3-daa0-4af6-89ae-0afbe08e84a5 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:05.583917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0236cd3-daa0-4af6-89ae-0afbe08e84a5 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:05.585301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0236cd3-daa0-4af6-89ae-0afbe08e84a5 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:05.586815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0236cd3-daa0-4af6-89ae-0afbe08e84a5 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:05.587727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0236cd3-daa0-4af6-89ae-0afbe08e84a5 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:06.153119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b0236cd3-daa0-4af6-89ae-0afbe08e84a5 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:06.171148 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0236cd3-daa0-4af6-89ae-0afbe08e84a5 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1318 microversion: 2.1 time: 0.619030 Sep 27 10:27:06.174292 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2599/5656] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:05 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1318 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:06.189127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ad26fc01-3e7c-41e9-b140-7504c3b99122 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:06.190804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ad26fc01-3e7c-41e9-b140-7504c3b99122 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:06.210526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad26fc01-3e7c-41e9-b140-7504c3b99122 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:06.210526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad26fc01-3e7c-41e9-b140-7504c3b99122 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:06.210526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad26fc01-3e7c-41e9-b140-7504c3b99122 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:06.210526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad26fc01-3e7c-41e9-b140-7504c3b99122 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:06.210526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad26fc01-3e7c-41e9-b140-7504c3b99122 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:06.210526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad26fc01-3e7c-41e9-b140-7504c3b99122 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:06.236462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:27:06.237718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:27:06.238394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:27:06.238972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:27:06.239642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:27:06.270412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:06.273302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:06.274779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:06.275805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:06.281606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:06.282302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:06.354754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Getting quotas for project 0a60557528994b829cd5131340a497b7. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:27:06.362229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Getting quotas for user a0e47aca97434f09846b580091b06829 and project 0a60557528994b829cd5131340a497b7. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:27:06.385572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:27:06.390235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:27:06.535747 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e370416-321e-4eeb-89ff-717c8971e574 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.656884 Sep 27 10:27:06.561908 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3058/5657] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:27:04 2022] POST /compute/v2.1/servers => generated 380 bytes in 1685 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:27:06.573240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-84155ff6-0494-418f-b484-4e961a52f9c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:06.576513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-84155ff6-0494-418f-b484-4e961a52f9c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:06.593199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84155ff6-0494-418f-b484-4e961a52f9c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:06.594435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84155ff6-0494-418f-b484-4e961a52f9c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:06.596461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84155ff6-0494-418f-b484-4e961a52f9c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:06.597834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84155ff6-0494-418f-b484-4e961a52f9c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:06.598984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84155ff6-0494-418f-b484-4e961a52f9c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:06.601306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-84155ff6-0494-418f-b484-4e961a52f9c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:06.726897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ad26fc01-3e7c-41e9-b140-7504c3b99122 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:06.728402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-84155ff6-0494-418f-b484-4e961a52f9c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:06.742132 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-84155ff6-0494-418f-b484-4e961a52f9c7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1267 microversion: 2.1 time: 0.178867 Sep 27 10:27:06.744265 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3059/5658] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:06 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1267 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:06.757446 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ad26fc01-3e7c-41e9-b140-7504c3b99122 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1452 microversion: 2.1 time: 0.581756 Sep 27 10:27:06.758529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-76b4b96f-2d67-40e4-95d1-7f51fa4c64b6 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:06.759370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-76b4b96f-2d67-40e4-95d1-7f51fa4c64b6 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:06.765393 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2600/5659] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:06 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1452 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:07.004015 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-76b4b96f-2d67-40e4-95d1-7f51fa4c64b6 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1328 microversion: 2.1 time: 0.256308 Sep 27 10:27:07.004607 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3060/5660] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:06 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1328 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:07.200067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6621cf2f-caaf-4b43-af60-c4e75c1bec17 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:07.203084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6621cf2f-caaf-4b43-af60-c4e75c1bec17 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:07.220451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6621cf2f-caaf-4b43-af60-c4e75c1bec17 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:07.223176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6621cf2f-caaf-4b43-af60-c4e75c1bec17 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:07.226876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6621cf2f-caaf-4b43-af60-c4e75c1bec17 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:07.229066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6621cf2f-caaf-4b43-af60-c4e75c1bec17 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:07.232006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6621cf2f-caaf-4b43-af60-c4e75c1bec17 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:07.232764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6621cf2f-caaf-4b43-af60-c4e75c1bec17 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:07.243165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:07.505077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d81587d9-ca86-4ae0-93ba-d62f7bf4bc07 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: .version_select at 0x7f7f62658af0>, body: {"keypair": {"name": "tempest-keypair-1250635480", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBkWzzJK6P0PG4l1fAn9xuGmxNE7I6NwxqeNd3CC25yPWqH8p5HrHcpCPfoEDsro03pZ6fBPR9pGSLrj66ZsFskHGzWvWBID1FMZv7YQtygb0CdFqFaQtEojca3BZrkWqw=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:07.551892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-d81587d9-ca86-4ae0-93ba-d62f7bf4bc07 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for project 90c0fbb4a6d84767b920564584715f9d. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:27:07.570066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-d81587d9-ca86-4ae0-93ba-d62f7bf4bc07 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for user 64090c8fd3394980ad5409adcde4986f and project 90c0fbb4a6d84767b920564584715f9d. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:27:07.629023 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d81587d9-ca86-4ae0-93ba-d62f7bf4bc07 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.455257 Sep 27 10:27:07.637478 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2601/5661] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:27:07 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:07.771473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea55f37c-10a7-4ea6-ba3e-45d5189dafdf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:07.772969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ea55f37c-10a7-4ea6-ba3e-45d5189dafdf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:07.785354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea55f37c-10a7-4ea6-ba3e-45d5189dafdf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:07.787785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea55f37c-10a7-4ea6-ba3e-45d5189dafdf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:07.788614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea55f37c-10a7-4ea6-ba3e-45d5189dafdf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:07.790544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea55f37c-10a7-4ea6-ba3e-45d5189dafdf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:07.791457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea55f37c-10a7-4ea6-ba3e-45d5189dafdf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:07.792444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea55f37c-10a7-4ea6-ba3e-45d5189dafdf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:07.941145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ea55f37c-10a7-4ea6-ba3e-45d5189dafdf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:07.966935 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea55f37c-10a7-4ea6-ba3e-45d5189dafdf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1273 microversion: 2.1 time: 0.201596 Sep 27 10:27:07.968830 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2602/5662] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:07 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1273 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:07.993489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7a740634-c66e-4f3f-ac27-5eb04dcb2be6 req-7e27a18e-a731-42ad-ad2d-c0cf2f30060f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9d4f13c2-3c04-4984-a293-df05378d29e5", "name": "network-vif-plugged", "status": "completed", "tag": "2c5322b1-7e1a-4d84-9b57-45c942d8b0e0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:08.009334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7a740634-c66e-4f3f-ac27-5eb04dcb2be6 req-7e27a18e-a731-42ad-ad2d-c0cf2f30060f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:08.018831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7a740634-c66e-4f3f-ac27-5eb04dcb2be6 req-7e27a18e-a731-42ad-ad2d-c0cf2f30060f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:08.020454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7a740634-c66e-4f3f-ac27-5eb04dcb2be6 req-7e27a18e-a731-42ad-ad2d-c0cf2f30060f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:08.058881 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7a740634-c66e-4f3f-ac27-5eb04dcb2be6 req-7e27a18e-a731-42ad-ad2d-c0cf2f30060f service nova] Creating event network-vif-plugged:2c5322b1-7e1a-4d84-9b57-45c942d8b0e0 for instance 9d4f13c2-3c04-4984-a293-df05378d29e5 on nch1-master-1 Sep 27 10:27:08.083036 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7a740634-c66e-4f3f-ac27-5eb04dcb2be6 req-7e27a18e-a731-42ad-ad2d-c0cf2f30060f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093695 Sep 27 10:27:08.086257 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2603/5663] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:27:08.111953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d6f66c9b-7bfb-4871-be8d-b072e9e82e8a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:08.118498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d6f66c9b-7bfb-4871-be8d-b072e9e82e8a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:08.151910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f66c9b-7bfb-4871-be8d-b072e9e82e8a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:08.153770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f66c9b-7bfb-4871-be8d-b072e9e82e8a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:08.154355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f66c9b-7bfb-4871-be8d-b072e9e82e8a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:08.155926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f66c9b-7bfb-4871-be8d-b072e9e82e8a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:08.156522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f66c9b-7bfb-4871-be8d-b072e9e82e8a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:08.157236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d6f66c9b-7bfb-4871-be8d-b072e9e82e8a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:08.203501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6621cf2f-caaf-4b43-af60-c4e75c1bec17 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:08.221263 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6621cf2f-caaf-4b43-af60-c4e75c1bec17 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1318 microversion: 2.1 time: 1.023650 Sep 27 10:27:08.222990 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3061/5664] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:07 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1318 bytes in 1027 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:08.244774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:08.552903 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d6f66c9b-7bfb-4871-be8d-b072e9e82e8a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1415 microversion: 2.1 time: 0.463692 Sep 27 10:27:08.561563 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2604/5665] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:08 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1415 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:08.997003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9dbe0926-451b-4ae9-aa21-b6599539282c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:08.997645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9dbe0926-451b-4ae9-aa21-b6599539282c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:09.008275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dbe0926-451b-4ae9-aa21-b6599539282c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:09.008693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dbe0926-451b-4ae9-aa21-b6599539282c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:09.009120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dbe0926-451b-4ae9-aa21-b6599539282c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:09.009845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dbe0926-451b-4ae9-aa21-b6599539282c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:09.010293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dbe0926-451b-4ae9-aa21-b6599539282c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:09.010715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9dbe0926-451b-4ae9-aa21-b6599539282c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:09.247604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b2d07361-32d5-43ff-bfcc-c23ec1e7c421 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:09.249022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b2d07361-32d5-43ff-bfcc-c23ec1e7c421 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:09.258767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d07361-32d5-43ff-bfcc-c23ec1e7c421 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:09.262758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d07361-32d5-43ff-bfcc-c23ec1e7c421 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:09.263555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d07361-32d5-43ff-bfcc-c23ec1e7c421 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:09.264673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d07361-32d5-43ff-bfcc-c23ec1e7c421 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:09.267538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d07361-32d5-43ff-bfcc-c23ec1e7c421 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:09.268483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b2d07361-32d5-43ff-bfcc-c23ec1e7c421 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:09.465916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9dbe0926-451b-4ae9-aa21-b6599539282c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:09.480345 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9dbe0926-451b-4ae9-aa21-b6599539282c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1315 microversion: 2.1 time: 0.487957 Sep 27 10:27:09.481668 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3062/5666] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:08 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1315 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:09.597352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3cc2e98-8e39-4151-b865-02ec24cf6708 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:09.598012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e3cc2e98-8e39-4151-b865-02ec24cf6708 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:09.613551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3cc2e98-8e39-4151-b865-02ec24cf6708 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:09.617618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3cc2e98-8e39-4151-b865-02ec24cf6708 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:09.619094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3cc2e98-8e39-4151-b865-02ec24cf6708 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:09.621180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3cc2e98-8e39-4151-b865-02ec24cf6708 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:09.622126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3cc2e98-8e39-4151-b865-02ec24cf6708 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:09.622589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3cc2e98-8e39-4151-b865-02ec24cf6708 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:09.759164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b2d07361-32d5-43ff-bfcc-c23ec1e7c421 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:09.774996 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b2d07361-32d5-43ff-bfcc-c23ec1e7c421 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1318 microversion: 2.1 time: 0.531439 Sep 27 10:27:09.775786 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2605/5667] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:09 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1318 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:09.814472 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e3cc2e98-8e39-4151-b865-02ec24cf6708 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1421 microversion: 2.1 time: 0.227658 Sep 27 10:27:09.829794 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3063/5668] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:09 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1421 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:10.432023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ad7c8481-d1e5-449c-be8f-6356996d45d9 req-fa56a5c2-6166-4c42-a4b4-0aae07e642ed service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "af5aa123-db45-4556-bd67-47687f84ba0f", "tag": "9482af94-f7d5-4e78-b8bd-2be5c1dcae94"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:10.459521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ad7c8481-d1e5-449c-be8f-6356996d45d9 req-fa56a5c2-6166-4c42-a4b4-0aae07e642ed service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:10.461196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ad7c8481-d1e5-449c-be8f-6356996d45d9 req-fa56a5c2-6166-4c42-a4b4-0aae07e642ed service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:10.461679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ad7c8481-d1e5-449c-be8f-6356996d45d9 req-fa56a5c2-6166-4c42-a4b4-0aae07e642ed service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:10.505215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e39672e3-db31-4668-9d5a-37283af03b29 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:10.506579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e39672e3-db31-4668-9d5a-37283af03b29 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:10.516394 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ad7c8481-d1e5-449c-be8f-6356996d45d9 req-fa56a5c2-6166-4c42-a4b4-0aae07e642ed service nova] Creating event network-changed:9482af94-f7d5-4e78-b8bd-2be5c1dcae94 for instance af5aa123-db45-4556-bd67-47687f84ba0f on nch2-master-1 Sep 27 10:27:10.521070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e39672e3-db31-4668-9d5a-37283af03b29 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:10.521993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e39672e3-db31-4668-9d5a-37283af03b29 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:10.522686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e39672e3-db31-4668-9d5a-37283af03b29 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:10.523700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e39672e3-db31-4668-9d5a-37283af03b29 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:10.524401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e39672e3-db31-4668-9d5a-37283af03b29 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:10.525063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e39672e3-db31-4668-9d5a-37283af03b29 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:10.547025 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ad7c8481-d1e5-449c-be8f-6356996d45d9 req-fa56a5c2-6166-4c42-a4b4-0aae07e642ed service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.117537 Sep 27 10:27:10.548757 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2606/5669] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:10.803885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4653f81e-f8c3-45d5-b0b6-504e26f6b325 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:10.804690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4653f81e-f8c3-45d5-b0b6-504e26f6b325 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:10.818250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4653f81e-f8c3-45d5-b0b6-504e26f6b325 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:10.818769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4653f81e-f8c3-45d5-b0b6-504e26f6b325 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:10.819230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4653f81e-f8c3-45d5-b0b6-504e26f6b325 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:10.819977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4653f81e-f8c3-45d5-b0b6-504e26f6b325 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:10.820458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4653f81e-f8c3-45d5-b0b6-504e26f6b325 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:10.820933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4653f81e-f8c3-45d5-b0b6-504e26f6b325 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:11.090401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4653f81e-f8c3-45d5-b0b6-504e26f6b325 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:11.101311 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4653f81e-f8c3-45d5-b0b6-504e26f6b325 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1506 microversion: 2.1 time: 0.300990 Sep 27 10:27:11.113598 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2607/5670] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:10 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1506 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:11.126564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3e8f621c-aef5-4e3b-bf30-12dbd0914de0 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:11.127486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e8f621c-aef5-4e3b-bf30-12dbd0914de0 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:11.135586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e8f621c-aef5-4e3b-bf30-12dbd0914de0 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:11.136390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e8f621c-aef5-4e3b-bf30-12dbd0914de0 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:11.137657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e8f621c-aef5-4e3b-bf30-12dbd0914de0 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:11.138482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e8f621c-aef5-4e3b-bf30-12dbd0914de0 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:11.139411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e8f621c-aef5-4e3b-bf30-12dbd0914de0 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:11.140088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e8f621c-aef5-4e3b-bf30-12dbd0914de0 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:11.221307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e39672e3-db31-4668-9d5a-37283af03b29 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:11.235942 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e39672e3-db31-4668-9d5a-37283af03b29 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1315 microversion: 2.1 time: 0.735899 Sep 27 10:27:11.236208 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3e8f621c-aef5-4e3b-bf30-12dbd0914de0 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1421 microversion: 2.1 time: 0.119373 Sep 27 10:27:11.252286 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2608/5671] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:11 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1421 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:11.252286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6bfb6226-7d6a-4222-8133-a490e895d05b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "cd03ad33-8f4c-4084-8b36-72d1fe84c526", "device": "/dev/sdb"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:11.254001 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3064/5672] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:10 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1315 bytes in 755 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:11.257710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6bfb6226-7d6a-4222-8133-a490e895d05b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:11.273545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bfb6226-7d6a-4222-8133-a490e895d05b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:11.274422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bfb6226-7d6a-4222-8133-a490e895d05b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:11.276444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bfb6226-7d6a-4222-8133-a490e895d05b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:11.277453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bfb6226-7d6a-4222-8133-a490e895d05b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:11.281338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bfb6226-7d6a-4222-8133-a490e895d05b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:11.281875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bfb6226-7d6a-4222-8133-a490e895d05b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:11.610072 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6bfb6226-7d6a-4222-8133-a490e895d05b tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "POST /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.368907 Sep 27 10:27:11.610817 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2609/5673] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:27:11 2022] POST /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5/os-volume_attachments => generated 194 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:12.263672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e6ea2bbf-ea53-4cf4-8d9c-f5ac6493c9a9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:12.264298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e6ea2bbf-ea53-4cf4-8d9c-f5ac6493c9a9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:12.281131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ea2bbf-ea53-4cf4-8d9c-f5ac6493c9a9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:12.281820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ea2bbf-ea53-4cf4-8d9c-f5ac6493c9a9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:12.282245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ea2bbf-ea53-4cf4-8d9c-f5ac6493c9a9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:12.283025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ea2bbf-ea53-4cf4-8d9c-f5ac6493c9a9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:12.283488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ea2bbf-ea53-4cf4-8d9c-f5ac6493c9a9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:12.283923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6ea2bbf-ea53-4cf4-8d9c-f5ac6493c9a9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:12.288770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-adab1805-3a40-4d87-8167-695a4488f2d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:12.289607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-adab1805-3a40-4d87-8167-695a4488f2d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:12.319775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-adab1805-3a40-4d87-8167-695a4488f2d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:12.324800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-adab1805-3a40-4d87-8167-695a4488f2d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:12.325544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-adab1805-3a40-4d87-8167-695a4488f2d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:12.326314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-adab1805-3a40-4d87-8167-695a4488f2d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:12.328991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-adab1805-3a40-4d87-8167-695a4488f2d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:12.337542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-adab1805-3a40-4d87-8167-695a4488f2d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:12.622772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-adab1805-3a40-4d87-8167-695a4488f2d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:12.648456 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-adab1805-3a40-4d87-8167-695a4488f2d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1315 microversion: 2.1 time: 0.370254 Sep 27 10:27:12.650965 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2610/5674] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:12 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1315 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:13.071800 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e6ea2bbf-ea53-4cf4-8d9c-f5ac6493c9a9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1463 microversion: 2.1 time: 0.807806 Sep 27 10:27:13.072760 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3065/5675] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:12 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1463 bytes in 814 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:13.676300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c85c4cdc-3072-45d3-9db8-dd9dbc2c776c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:13.678823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c85c4cdc-3072-45d3-9db8-dd9dbc2c776c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:13.698487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c85c4cdc-3072-45d3-9db8-dd9dbc2c776c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:13.699236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c85c4cdc-3072-45d3-9db8-dd9dbc2c776c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:13.700305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c85c4cdc-3072-45d3-9db8-dd9dbc2c776c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:13.701302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c85c4cdc-3072-45d3-9db8-dd9dbc2c776c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:13.701953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c85c4cdc-3072-45d3-9db8-dd9dbc2c776c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:13.703812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c85c4cdc-3072-45d3-9db8-dd9dbc2c776c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:13.874447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1353813867", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "6a85e0d6-e759-43a8-9abb-8e12abcca6c4"}], "security_groups": [{"name": "tempest-securitygroup--1716784668"}], "key_name": "tempest-keypair-1250635480", "user_data": "}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:14.090379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] validate_networks() for [('6a85e0d6-e759-43a8-9abb-8e12abcca6c4', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:27:14.525091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:14.666147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c85c4cdc-3072-45d3-9db8-dd9dbc2c776c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:14.711021 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c85c4cdc-3072-45d3-9db8-dd9dbc2c776c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1315 microversion: 2.1 time: 1.042289 Sep 27 10:27:14.735093 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2611/5676] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:13 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1315 bytes in 1068 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:14.744376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9121afa6-2300-4cdc-a80c-e6178808031b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:14.745687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9121afa6-2300-4cdc-a80c-e6178808031b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:14.759468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9121afa6-2300-4cdc-a80c-e6178808031b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:14.760251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9121afa6-2300-4cdc-a80c-e6178808031b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:14.762700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9121afa6-2300-4cdc-a80c-e6178808031b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:14.765519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9121afa6-2300-4cdc-a80c-e6178808031b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:14.768610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9121afa6-2300-4cdc-a80c-e6178808031b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:14.770282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9121afa6-2300-4cdc-a80c-e6178808031b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:15.089893 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9121afa6-2300-4cdc-a80c-e6178808031b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1463 microversion: 2.1 time: 0.352923 Sep 27 10:27:15.091812 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2612/5677] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:14 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1463 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:15.099166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-de337bce-5b7c-446d-9d12-b74d33ac851d req-d77665fd-78eb-494a-9471-9286154d0c64 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "82b31a6f-a898-4796-a3ad-1628306fbe7b", "tag": "f840e3d8-9c94-4d2d-b759-279dceffb800"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:15.117641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-de337bce-5b7c-446d-9d12-b74d33ac851d req-d77665fd-78eb-494a-9471-9286154d0c64 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:15.118844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-de337bce-5b7c-446d-9d12-b74d33ac851d req-d77665fd-78eb-494a-9471-9286154d0c64 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:15.119690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-de337bce-5b7c-446d-9d12-b74d33ac851d req-d77665fd-78eb-494a-9471-9286154d0c64 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:15.141339 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-de337bce-5b7c-446d-9d12-b74d33ac851d req-d77665fd-78eb-494a-9471-9286154d0c64 service nova] Creating event network-changed:f840e3d8-9c94-4d2d-b759-279dceffb800 for instance 82b31a6f-a898-4796-a3ad-1628306fbe7b on nch2-master-1 Sep 27 10:27:15.149852 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-de337bce-5b7c-446d-9d12-b74d33ac851d req-d77665fd-78eb-494a-9471-9286154d0c64 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056181 Sep 27 10:27:15.150952 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2613/5678] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:15.170547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:27:15.171066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:27:15.172238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:27:15.172839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:27:15.173359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:27:15.201285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:15.201988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:15.202751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:15.203654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:15.204120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:15.204720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:15.236479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for project 90c0fbb4a6d84767b920564584715f9d. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:27:15.260261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for user 64090c8fd3394980ad5409adcde4986f and project 90c0fbb4a6d84767b920564584715f9d. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:27:15.294072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:15.346924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:27:15.350154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:27:15.473944 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f56abb07-8f14-4c8f-b4f4-e3fe90783c59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.602287 Sep 27 10:27:15.479246 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3066/5679] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:27:13 2022] POST /compute/v2.1/servers => generated 402 bytes in 1611 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:27:15.494136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-572fde47-7ce5-4221-a671-fe687c5f28ba tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:15.495421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-572fde47-7ce5-4221-a671-fe687c5f28ba tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:15.730453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-572fde47-7ce5-4221-a671-fe687c5f28ba tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:15.741816 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-572fde47-7ce5-4221-a671-fe687c5f28ba tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1242 microversion: 2.1 time: 0.253361 Sep 27 10:27:15.743875 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2614/5680] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:15 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1242 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:15.749715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2dbb42d2-cb36-4dc9-bce4-5b39c999821f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:15.750732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2dbb42d2-cb36-4dc9-bce4-5b39c999821f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:15.767801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbb42d2-cb36-4dc9-bce4-5b39c999821f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:15.768336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbb42d2-cb36-4dc9-bce4-5b39c999821f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:15.769404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbb42d2-cb36-4dc9-bce4-5b39c999821f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:15.770409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbb42d2-cb36-4dc9-bce4-5b39c999821f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:15.773388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbb42d2-cb36-4dc9-bce4-5b39c999821f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:15.776466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbb42d2-cb36-4dc9-bce4-5b39c999821f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:15.981347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2dbb42d2-cb36-4dc9-bce4-5b39c999821f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:15.991933 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2dbb42d2-cb36-4dc9-bce4-5b39c999821f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1315 microversion: 2.1 time: 0.251101 Sep 27 10:27:16.005384 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3067/5681] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:15 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1315 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:16.114888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-96213b76-0a73-448f-acd4-33b91d301535 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:16.115804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-96213b76-0a73-448f-acd4-33b91d301535 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:16.125179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96213b76-0a73-448f-acd4-33b91d301535 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:16.126723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96213b76-0a73-448f-acd4-33b91d301535 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:16.127351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96213b76-0a73-448f-acd4-33b91d301535 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:16.131021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96213b76-0a73-448f-acd4-33b91d301535 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:16.131986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96213b76-0a73-448f-acd4-33b91d301535 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:16.134382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96213b76-0a73-448f-acd4-33b91d301535 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:16.399076 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-96213b76-0a73-448f-acd4-33b91d301535 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1463 microversion: 2.1 time: 0.288222 Sep 27 10:27:16.399692 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2615/5682] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:16 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1463 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:16.766346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-50a455c7-fbe5-4499-a0db-9425baad6f85 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:16.767120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-50a455c7-fbe5-4499-a0db-9425baad6f85 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:16.773701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50a455c7-fbe5-4499-a0db-9425baad6f85 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:16.774336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50a455c7-fbe5-4499-a0db-9425baad6f85 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:16.774876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50a455c7-fbe5-4499-a0db-9425baad6f85 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:16.775845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50a455c7-fbe5-4499-a0db-9425baad6f85 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:16.776413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50a455c7-fbe5-4499-a0db-9425baad6f85 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:16.777215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50a455c7-fbe5-4499-a0db-9425baad6f85 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:16.854204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-50a455c7-fbe5-4499-a0db-9425baad6f85 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:16.863853 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-50a455c7-fbe5-4499-a0db-9425baad6f85 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1294 microversion: 2.1 time: 0.100903 Sep 27 10:27:16.881323 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3068/5683] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:16 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1294 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:17.017008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b7212f15-5a9a-42b3-8216-5f22fe7a7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:17.017605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b7212f15-5a9a-42b3-8216-5f22fe7a7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:17.025602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7212f15-5a9a-42b3-8216-5f22fe7a7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:17.026258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7212f15-5a9a-42b3-8216-5f22fe7a7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:17.026944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7212f15-5a9a-42b3-8216-5f22fe7a7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:17.027847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7212f15-5a9a-42b3-8216-5f22fe7a7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:17.028324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7212f15-5a9a-42b3-8216-5f22fe7a7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:17.028826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7212f15-5a9a-42b3-8216-5f22fe7a7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:17.169931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b7212f15-5a9a-42b3-8216-5f22fe7a7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:17.172133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a9ed66fa-bb8c-4cc3-a703-47facc8a6fe4 req-7c32f689-3a77-47d8-b4b8-246fc22830fb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cee68c7e-0857-45fb-b5b8-e98bd46532dc", "tag": "04a989a1-7997-4f2b-98c4-d619153897cc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:17.181465 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b7212f15-5a9a-42b3-8216-5f22fe7a7ff9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1315 microversion: 2.1 time: 0.169499 Sep 27 10:27:17.182071 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2616/5684] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:17 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1315 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:17.194536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a9ed66fa-bb8c-4cc3-a703-47facc8a6fe4 req-7c32f689-3a77-47d8-b4b8-246fc22830fb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:17.195337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a9ed66fa-bb8c-4cc3-a703-47facc8a6fe4 req-7c32f689-3a77-47d8-b4b8-246fc22830fb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:17.195810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a9ed66fa-bb8c-4cc3-a703-47facc8a6fe4 req-7c32f689-3a77-47d8-b4b8-246fc22830fb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:17.207908 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a9ed66fa-bb8c-4cc3-a703-47facc8a6fe4 req-7c32f689-3a77-47d8-b4b8-246fc22830fb service nova] Creating event network-changed:04a989a1-7997-4f2b-98c4-d619153897cc for instance cee68c7e-0857-45fb-b5b8-e98bd46532dc on nch1-master-1 Sep 27 10:27:17.214191 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a9ed66fa-bb8c-4cc3-a703-47facc8a6fe4 req-7c32f689-3a77-47d8-b4b8-246fc22830fb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046615 Sep 27 10:27:17.214794 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3069/5685] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:17.422765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-07cad681-897c-40ba-bf57-e00d75e6ce15 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:17.423873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-07cad681-897c-40ba-bf57-e00d75e6ce15 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:17.445690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad681-897c-40ba-bf57-e00d75e6ce15 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:17.446454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad681-897c-40ba-bf57-e00d75e6ce15 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:17.447072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad681-897c-40ba-bf57-e00d75e6ce15 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:17.448234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad681-897c-40ba-bf57-e00d75e6ce15 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:17.449303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad681-897c-40ba-bf57-e00d75e6ce15 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:17.451900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07cad681-897c-40ba-bf57-e00d75e6ce15 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:17.666336 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-07cad681-897c-40ba-bf57-e00d75e6ce15 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1463 microversion: 2.1 time: 0.247591 Sep 27 10:27:17.667216 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2617/5686] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:17 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1463 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:17.890571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1697a89a-0bb9-4339-8059-247620cc2fe5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:17.891385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1697a89a-0bb9-4339-8059-247620cc2fe5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:17.898948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1697a89a-0bb9-4339-8059-247620cc2fe5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:17.899583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1697a89a-0bb9-4339-8059-247620cc2fe5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:17.900298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1697a89a-0bb9-4339-8059-247620cc2fe5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:17.901281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1697a89a-0bb9-4339-8059-247620cc2fe5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:17.901985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1697a89a-0bb9-4339-8059-247620cc2fe5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:17.902630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1697a89a-0bb9-4339-8059-247620cc2fe5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:17.986783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1697a89a-0bb9-4339-8059-247620cc2fe5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:17.997383 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1697a89a-0bb9-4339-8059-247620cc2fe5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1300 microversion: 2.1 time: 0.111451 Sep 27 10:27:18.000984 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3070/5687] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:17 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1300 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:18.205449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-824b8df6-672a-4d77-9931-0aac8f9f9a39 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:18.206560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-824b8df6-672a-4d77-9931-0aac8f9f9a39 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:18.222601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8df6-672a-4d77-9931-0aac8f9f9a39 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:18.222601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8df6-672a-4d77-9931-0aac8f9f9a39 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:18.222601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8df6-672a-4d77-9931-0aac8f9f9a39 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:18.223418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8df6-672a-4d77-9931-0aac8f9f9a39 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:18.224323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8df6-672a-4d77-9931-0aac8f9f9a39 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:18.224992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8df6-672a-4d77-9931-0aac8f9f9a39 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:18.590074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-824b8df6-672a-4d77-9931-0aac8f9f9a39 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:18.601705 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-824b8df6-672a-4d77-9931-0aac8f9f9a39 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1315 microversion: 2.1 time: 0.400572 Sep 27 10:27:18.603573 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2618/5688] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:18 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1315 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:18.697536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-99dc80a9-8643-41a6-9185-c786325e6a22 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:18.698926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-99dc80a9-8643-41a6-9185-c786325e6a22 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:18.709247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dc80a9-8643-41a6-9185-c786325e6a22 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:18.710674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dc80a9-8643-41a6-9185-c786325e6a22 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:18.711338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dc80a9-8643-41a6-9185-c786325e6a22 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:18.712983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dc80a9-8643-41a6-9185-c786325e6a22 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:18.713647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dc80a9-8643-41a6-9185-c786325e6a22 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:18.714900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-99dc80a9-8643-41a6-9185-c786325e6a22 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:18.815228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-01cc5080-7058-4409-9d95-eb2d0f32166e req-99292d2e-17c6-48c4-94e3-9abe582f2460 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "af5aa123-db45-4556-bd67-47687f84ba0f", "name": "network-vif-plugged", "status": "completed", "tag": "9482af94-f7d5-4e78-b8bd-2be5c1dcae94"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:18.835945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-01cc5080-7058-4409-9d95-eb2d0f32166e req-99292d2e-17c6-48c4-94e3-9abe582f2460 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:18.838966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-01cc5080-7058-4409-9d95-eb2d0f32166e req-99292d2e-17c6-48c4-94e3-9abe582f2460 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:18.839602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-01cc5080-7058-4409-9d95-eb2d0f32166e req-99292d2e-17c6-48c4-94e3-9abe582f2460 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:18.866374 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-01cc5080-7058-4409-9d95-eb2d0f32166e req-99292d2e-17c6-48c4-94e3-9abe582f2460 service nova] Creating event network-vif-plugged:9482af94-f7d5-4e78-b8bd-2be5c1dcae94 for instance af5aa123-db45-4556-bd67-47687f84ba0f on nch2-master-1 Sep 27 10:27:18.883061 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-01cc5080-7058-4409-9d95-eb2d0f32166e req-99292d2e-17c6-48c4-94e3-9abe582f2460 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071150 Sep 27 10:27:18.884269 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2619/5689] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:27:18.920852 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-99dc80a9-8643-41a6-9185-c786325e6a22 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1463 microversion: 2.1 time: 0.234258 Sep 27 10:27:18.924162 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3071/5690] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:18 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1463 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:19.024419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a95196cb-0239-47c9-9323-8c77ca8f040b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:19.025509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a95196cb-0239-47c9-9323-8c77ca8f040b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:19.032508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a95196cb-0239-47c9-9323-8c77ca8f040b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:19.033348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a95196cb-0239-47c9-9323-8c77ca8f040b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:19.033982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a95196cb-0239-47c9-9323-8c77ca8f040b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:19.034892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a95196cb-0239-47c9-9323-8c77ca8f040b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:19.035570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a95196cb-0239-47c9-9323-8c77ca8f040b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:19.036385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a95196cb-0239-47c9-9323-8c77ca8f040b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:19.299719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a95196cb-0239-47c9-9323-8c77ca8f040b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:19.310342 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a95196cb-0239-47c9-9323-8c77ca8f040b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1368 microversion: 2.1 time: 0.290748 Sep 27 10:27:19.310932 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2620/5691] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:19 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1368 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:19.469240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d166c18b-593f-450d-96d7-b61f84ff6ab7 req-3486fe85-c06a-4577-b454-2cab4679fad0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "403bfd67-4e7f-4918-baf9-4d14086a4598", "tag": "64716a19-8703-40a7-b759-cc1ba1192e49"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:19.478752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d166c18b-593f-450d-96d7-b61f84ff6ab7 req-3486fe85-c06a-4577-b454-2cab4679fad0 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:19.479234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d166c18b-593f-450d-96d7-b61f84ff6ab7 req-3486fe85-c06a-4577-b454-2cab4679fad0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:19.479660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d166c18b-593f-450d-96d7-b61f84ff6ab7 req-3486fe85-c06a-4577-b454-2cab4679fad0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:19.490734 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d166c18b-593f-450d-96d7-b61f84ff6ab7 req-3486fe85-c06a-4577-b454-2cab4679fad0 service nova] Creating event network-changed:64716a19-8703-40a7-b759-cc1ba1192e49 for instance 403bfd67-4e7f-4918-baf9-4d14086a4598 on nch2-master-1 Sep 27 10:27:19.495501 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d166c18b-593f-450d-96d7-b61f84ff6ab7 req-3486fe85-c06a-4577-b454-2cab4679fad0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.029933 Sep 27 10:27:19.496865 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3072/5692] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:19.627121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a09cb75-5b3c-49fa-a3c6-ad5bfcc197a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:19.627942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a09cb75-5b3c-49fa-a3c6-ad5bfcc197a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:19.635687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a09cb75-5b3c-49fa-a3c6-ad5bfcc197a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:19.636213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a09cb75-5b3c-49fa-a3c6-ad5bfcc197a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:19.636644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a09cb75-5b3c-49fa-a3c6-ad5bfcc197a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:19.637403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a09cb75-5b3c-49fa-a3c6-ad5bfcc197a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:19.637889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a09cb75-5b3c-49fa-a3c6-ad5bfcc197a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:19.638323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a09cb75-5b3c-49fa-a3c6-ad5bfcc197a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:19.841803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a09cb75-5b3c-49fa-a3c6-ad5bfcc197a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:19.850185 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a09cb75-5b3c-49fa-a3c6-ad5bfcc197a5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1315 microversion: 2.1 time: 0.227587 Sep 27 10:27:19.850959 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2621/5693] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:19 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1315 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:19.948997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6a507e6d-8dfd-48c1-928a-d2ed67e38bcd tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:19.950528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6a507e6d-8dfd-48c1-928a-d2ed67e38bcd tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:19.958388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a507e6d-8dfd-48c1-928a-d2ed67e38bcd tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:19.959108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a507e6d-8dfd-48c1-928a-d2ed67e38bcd tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:19.959595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a507e6d-8dfd-48c1-928a-d2ed67e38bcd tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:19.960344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a507e6d-8dfd-48c1-928a-d2ed67e38bcd tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:19.960886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a507e6d-8dfd-48c1-928a-d2ed67e38bcd tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:19.961287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6a507e6d-8dfd-48c1-928a-d2ed67e38bcd tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:20.104577 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6a507e6d-8dfd-48c1-928a-d2ed67e38bcd tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1463 microversion: 2.1 time: 0.164861 Sep 27 10:27:20.105265 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3073/5694] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:19 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1463 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:20.336187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-47214e6e-3bdf-4ec4-bc45-41074b624b60 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:20.336767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-47214e6e-3bdf-4ec4-bc45-41074b624b60 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:20.344916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47214e6e-3bdf-4ec4-bc45-41074b624b60 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:20.345576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47214e6e-3bdf-4ec4-bc45-41074b624b60 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:20.346082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47214e6e-3bdf-4ec4-bc45-41074b624b60 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:20.346849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47214e6e-3bdf-4ec4-bc45-41074b624b60 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:20.347429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47214e6e-3bdf-4ec4-bc45-41074b624b60 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:20.347979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-47214e6e-3bdf-4ec4-bc45-41074b624b60 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:20.366426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:20.548778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-47214e6e-3bdf-4ec4-bc45-41074b624b60 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:20.557435 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-47214e6e-3bdf-4ec4-bc45-41074b624b60 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1368 microversion: 2.1 time: 0.226546 Sep 27 10:27:20.558079 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2622/5695] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:20 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1368 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:20.874517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-810b4160-a779-4df0-8556-6052387e1289 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:20.875139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-810b4160-a779-4df0-8556-6052387e1289 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:20.884336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-810b4160-a779-4df0-8556-6052387e1289 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:20.885184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-810b4160-a779-4df0-8556-6052387e1289 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:20.885538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-810b4160-a779-4df0-8556-6052387e1289 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:20.886369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-810b4160-a779-4df0-8556-6052387e1289 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:20.886835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-810b4160-a779-4df0-8556-6052387e1289 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:20.887263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-810b4160-a779-4df0-8556-6052387e1289 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:20.902440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-c99b2e65-2e5c-456a-bd80-16bcf7e9d87b req-3de0fe48-cb7b-477a-acc8-bd62783fd430 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cee68c7e-0857-45fb-b5b8-e98bd46532dc", "name": "network-vif-plugged", "status": "completed", "tag": "04a989a1-7997-4f2b-98c4-d619153897cc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:20.913892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c99b2e65-2e5c-456a-bd80-16bcf7e9d87b req-3de0fe48-cb7b-477a-acc8-bd62783fd430 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:20.914636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c99b2e65-2e5c-456a-bd80-16bcf7e9d87b req-3de0fe48-cb7b-477a-acc8-bd62783fd430 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:20.915341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c99b2e65-2e5c-456a-bd80-16bcf7e9d87b req-3de0fe48-cb7b-477a-acc8-bd62783fd430 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:20.929769 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c99b2e65-2e5c-456a-bd80-16bcf7e9d87b req-3de0fe48-cb7b-477a-acc8-bd62783fd430 service nova] Creating event network-vif-plugged:04a989a1-7997-4f2b-98c4-d619153897cc for instance cee68c7e-0857-45fb-b5b8-e98bd46532dc on nch1-master-1 Sep 27 10:27:20.935113 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-c99b2e65-2e5c-456a-bd80-16bcf7e9d87b req-3de0fe48-cb7b-477a-acc8-bd62783fd430 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036579 Sep 27 10:27:20.936455 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2623/5696] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:21.075249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-810b4160-a779-4df0-8556-6052387e1289 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:21.085658 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-810b4160-a779-4df0-8556-6052387e1289 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1502 microversion: 2.1 time: 0.215271 Sep 27 10:27:21.086610 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3074/5697] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:20 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1502 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:21.111421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dc2a501c-b819-453b-b8f2-26389e954dd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "default"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:21.112060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dc2a501c-b819-453b-b8f2-26389e954dd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:21.114440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:21.120335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2a501c-b819-453b-b8f2-26389e954dd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:21.120871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2a501c-b819-453b-b8f2-26389e954dd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:21.121320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2a501c-b819-453b-b8f2-26389e954dd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:21.122043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2a501c-b819-453b-b8f2-26389e954dd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:21.122512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2a501c-b819-453b-b8f2-26389e954dd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:21.122948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc2a501c-b819-453b-b8f2-26389e954dd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:21.129279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1a0e3682-a9a8-4f0f-a85b-224a812072a3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:21.129973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1a0e3682-a9a8-4f0f-a85b-224a812072a3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:21.136962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0e3682-a9a8-4f0f-a85b-224a812072a3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:21.137452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0e3682-a9a8-4f0f-a85b-224a812072a3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:21.138075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0e3682-a9a8-4f0f-a85b-224a812072a3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:21.138963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0e3682-a9a8-4f0f-a85b-224a812072a3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:21.139422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0e3682-a9a8-4f0f-a85b-224a812072a3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:21.139848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a0e3682-a9a8-4f0f-a85b-224a812072a3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:21.283910 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1a0e3682-a9a8-4f0f-a85b-224a812072a3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1463 microversion: 2.1 time: 0.158462 Sep 27 10:27:21.284497 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3075/5698] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:21 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1463 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:21.334556 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.network.security_group_api [None req-dc2a501c-b819-453b-b8f2-26389e954dd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Removing security group 6d5a2612-8152-43d5-99e8-3212d8964d03 from port 9482af94-f7d5-4e78-b8bd-2be5c1dcae94 Sep 27 10:27:21.582141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0b68b12e-ec6c-48a2-9196-095afacda761 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:21.582669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0b68b12e-ec6c-48a2-9196-095afacda761 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:21.590044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b68b12e-ec6c-48a2-9196-095afacda761 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:21.590532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b68b12e-ec6c-48a2-9196-095afacda761 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:21.590970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b68b12e-ec6c-48a2-9196-095afacda761 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:21.591668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b68b12e-ec6c-48a2-9196-095afacda761 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:21.592124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b68b12e-ec6c-48a2-9196-095afacda761 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:21.592535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0b68b12e-ec6c-48a2-9196-095afacda761 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:21.687169 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dc2a501c-b819-453b-b8f2-26389e954dd6 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f/action" status: 202 len: 0 microversion: 2.1 time: 0.580419 Sep 27 10:27:21.688219 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2624/5699] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:27:21 2022] POST /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f/action => generated 0 bytes in 582 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:27:21.697462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a2c6dd6-2d41-4e89-ae96-ab081dc80910 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:21.698197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a2c6dd6-2d41-4e89-ae96-ab081dc80910 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:21.706406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a2c6dd6-2d41-4e89-ae96-ab081dc80910 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:21.707088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a2c6dd6-2d41-4e89-ae96-ab081dc80910 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:21.707711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a2c6dd6-2d41-4e89-ae96-ab081dc80910 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:21.709046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a2c6dd6-2d41-4e89-ae96-ab081dc80910 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:21.709711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a2c6dd6-2d41-4e89-ae96-ab081dc80910 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:21.710407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a2c6dd6-2d41-4e89-ae96-ab081dc80910 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:21.808338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0b68b12e-ec6c-48a2-9196-095afacda761 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:21.817503 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0b68b12e-ec6c-48a2-9196-095afacda761 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1368 microversion: 2.1 time: 0.239417 Sep 27 10:27:21.820635 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3076/5700] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:21 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1368 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:21.835471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-fcdb042b-64f9-48e0-aa08-f2eade0b11d8 req-fccd9070-e6f7-4dae-a7d5-8c9edbfae22c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "af5aa123-db45-4556-bd67-47687f84ba0f", "tag": "9482af94-f7d5-4e78-b8bd-2be5c1dcae94"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:21.837080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a2c6dd6-2d41-4e89-ae96-ab081dc80910 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:21.849363 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a2c6dd6-2d41-4e89-ae96-ab081dc80910 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1460 microversion: 2.1 time: 0.156923 Sep 27 10:27:21.850440 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2625/5701] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:21 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1460 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:21.851070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-fcdb042b-64f9-48e0-aa08-f2eade0b11d8 req-fccd9070-e6f7-4dae-a7d5-8c9edbfae22c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:21.851917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-fcdb042b-64f9-48e0-aa08-f2eade0b11d8 req-fccd9070-e6f7-4dae-a7d5-8c9edbfae22c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:21.852691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-fcdb042b-64f9-48e0-aa08-f2eade0b11d8 req-fccd9070-e6f7-4dae-a7d5-8c9edbfae22c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:21.874638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-10b22e64-80bb-420f-b44e-cfc15f286109 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:21.875370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-10b22e64-80bb-420f-b44e-cfc15f286109 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:21.880479 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-fcdb042b-64f9-48e0-aa08-f2eade0b11d8 req-fccd9070-e6f7-4dae-a7d5-8c9edbfae22c service nova] Creating event network-changed:9482af94-f7d5-4e78-b8bd-2be5c1dcae94 for instance af5aa123-db45-4556-bd67-47687f84ba0f on nch2-master-1 Sep 27 10:27:21.882256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b22e64-80bb-420f-b44e-cfc15f286109 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:21.882828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b22e64-80bb-420f-b44e-cfc15f286109 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:21.883630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b22e64-80bb-420f-b44e-cfc15f286109 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:21.884568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b22e64-80bb-420f-b44e-cfc15f286109 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:21.885253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b22e64-80bb-420f-b44e-cfc15f286109 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:21.885900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-10b22e64-80bb-420f-b44e-cfc15f286109 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:21.888565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-fcdb042b-64f9-48e0-aa08-f2eade0b11d8 req-fccd9070-e6f7-4dae-a7d5-8c9edbfae22c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065219 Sep 27 10:27:21.890536 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3077/5702] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:22.057459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-10b22e64-80bb-420f-b44e-cfc15f286109 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:22.067727 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-10b22e64-80bb-420f-b44e-cfc15f286109 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.197403 Sep 27 10:27:22.068360 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2626/5703] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:21 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:22.095801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-60157225-1d5f-4f13-9238-5d87bd712c97 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:22.096532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-60157225-1d5f-4f13-9238-5d87bd712c97 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:22.104072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60157225-1d5f-4f13-9238-5d87bd712c97 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.104564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60157225-1d5f-4f13-9238-5d87bd712c97 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.105034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60157225-1d5f-4f13-9238-5d87bd712c97 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:22.105748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60157225-1d5f-4f13-9238-5d87bd712c97 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.106214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60157225-1d5f-4f13-9238-5d87bd712c97 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.106645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60157225-1d5f-4f13-9238-5d87bd712c97 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:22.259292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-60157225-1d5f-4f13-9238-5d87bd712c97 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:22.271404 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-60157225-1d5f-4f13-9238-5d87bd712c97 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.180922 Sep 27 10:27:22.285910 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3078/5704] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:22 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:22.294192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca9b392b-20cf-4d65-8d6f-fe21b57dd46e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:22.294916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ca9b392b-20cf-4d65-8d6f-fe21b57dd46e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:22.310010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca9b392b-20cf-4d65-8d6f-fe21b57dd46e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.313547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca9b392b-20cf-4d65-8d6f-fe21b57dd46e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.314165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca9b392b-20cf-4d65-8d6f-fe21b57dd46e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:22.315088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca9b392b-20cf-4d65-8d6f-fe21b57dd46e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.315572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca9b392b-20cf-4d65-8d6f-fe21b57dd46e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.316118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca9b392b-20cf-4d65-8d6f-fe21b57dd46e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:22.321965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-96a2a0e8-0709-40d6-a041-797558458912 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:22.322845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-96a2a0e8-0709-40d6-a041-797558458912 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:22.336119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96a2a0e8-0709-40d6-a041-797558458912 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.336674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96a2a0e8-0709-40d6-a041-797558458912 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.337174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96a2a0e8-0709-40d6-a041-797558458912 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:22.338259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96a2a0e8-0709-40d6-a041-797558458912 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.338777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96a2a0e8-0709-40d6-a041-797558458912 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.339238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96a2a0e8-0709-40d6-a041-797558458912 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:22.473535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ca9b392b-20cf-4d65-8d6f-fe21b57dd46e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:22.484467 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ca9b392b-20cf-4d65-8d6f-fe21b57dd46e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.194172 Sep 27 10:27:22.485333 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2627/5705] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:22 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:22.488547 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-96a2a0e8-0709-40d6-a041-797558458912 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1647 microversion: 2.1 time: 0.180970 Sep 27 10:27:22.489104 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3079/5706] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:22 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1647 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:22.507440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-00723d26-2b27-43bb-8428-b3ebba03bec9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:22.508087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-00723d26-2b27-43bb-8428-b3ebba03bec9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:22.511534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d884be74-1482-4a7f-bcf7-1d9ddd59cc58 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:22.512219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d884be74-1482-4a7f-bcf7-1d9ddd59cc58 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:22.514768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00723d26-2b27-43bb-8428-b3ebba03bec9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.515280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00723d26-2b27-43bb-8428-b3ebba03bec9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.515739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00723d26-2b27-43bb-8428-b3ebba03bec9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:22.516453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00723d26-2b27-43bb-8428-b3ebba03bec9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.516958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00723d26-2b27-43bb-8428-b3ebba03bec9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.517402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00723d26-2b27-43bb-8428-b3ebba03bec9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:22.518534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d884be74-1482-4a7f-bcf7-1d9ddd59cc58 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.519022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d884be74-1482-4a7f-bcf7-1d9ddd59cc58 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.519597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d884be74-1482-4a7f-bcf7-1d9ddd59cc58 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:22.521048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d884be74-1482-4a7f-bcf7-1d9ddd59cc58 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.521660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d884be74-1482-4a7f-bcf7-1d9ddd59cc58 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.522100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d884be74-1482-4a7f-bcf7-1d9ddd59cc58 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:22.679693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-00723d26-2b27-43bb-8428-b3ebba03bec9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:22.689520 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-00723d26-2b27-43bb-8428-b3ebba03bec9 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.186379 Sep 27 10:27:22.690866 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2628/5707] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:22 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:22.811716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-507287347", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "247e3d7d-5912-42d7-9027-ebcb26003cfc"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:22.831784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:22.872462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] validate_networks() for [('247e3d7d-5912-42d7-9027-ebcb26003cfc', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:27:22.949056 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d884be74-1482-4a7f-bcf7-1d9ddd59cc58 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1647 microversion: 2.1 time: 0.441666 Sep 27 10:27:22.949644 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3080/5708] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:22 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1647 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:22.956355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69a8fd27-2c19-4414-97ea-5af1ba2ff9e1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:22.957116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69a8fd27-2c19-4414-97ea-5af1ba2ff9e1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:22.964086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69a8fd27-2c19-4414-97ea-5af1ba2ff9e1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.964718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69a8fd27-2c19-4414-97ea-5af1ba2ff9e1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.965235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69a8fd27-2c19-4414-97ea-5af1ba2ff9e1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:22.966032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69a8fd27-2c19-4414-97ea-5af1ba2ff9e1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:22.966564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69a8fd27-2c19-4414-97ea-5af1ba2ff9e1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:22.967069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69a8fd27-2c19-4414-97ea-5af1ba2ff9e1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:23.129710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69a8fd27-2c19-4414-97ea-5af1ba2ff9e1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:23.137875 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69a8fd27-2c19-4414-97ea-5af1ba2ff9e1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1368 microversion: 2.1 time: 0.187025 Sep 27 10:27:23.138455 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3081/5709] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:22 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1368 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:23.143764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f6a04341-e571-4e41-8677-802cfe79801a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:23.144553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f6a04341-e571-4e41-8677-802cfe79801a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:23.150757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6a04341-e571-4e41-8677-802cfe79801a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:23.151428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6a04341-e571-4e41-8677-802cfe79801a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:23.152024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6a04341-e571-4e41-8677-802cfe79801a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:23.153258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6a04341-e571-4e41-8677-802cfe79801a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:23.154140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6a04341-e571-4e41-8677-802cfe79801a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:23.154731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6a04341-e571-4e41-8677-802cfe79801a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:23.302947 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f6a04341-e571-4e41-8677-802cfe79801a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1647 microversion: 2.1 time: 0.162857 Sep 27 10:27:23.313655 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3082/5710] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:23 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1647 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:23.344138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:27:23.344978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:27:23.345598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:27:23.346085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:27:23.346606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:27:23.363576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:23.364259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:23.365191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:23.366046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:23.366725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:23.367345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:23.389558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for project 10375bd836f84383b39f2df73f53800e. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:27:23.396454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Getting quotas for user 97c769dbba794531b0a9db8e764ff097 and project 10375bd836f84383b39f2df73f53800e. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:27:23.412143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:27:23.413431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:27:23.446294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerMultinode-server-254370645", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "availability_zone": "nova:nch2-master-1", "networks": [{"uuid": "5dd82b97-2333-4934-8cc7-c9806488a076"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:23.475373 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-82b3fb2b-5222-41bf-8990-8f3c57c49090 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.667464 Sep 27 10:27:23.476829 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2629/5711] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:27:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 670 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:27:23.488219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-96a0d793-57f9-4e6e-b2a3-203affcbb87e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:23.489311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-96a0d793-57f9-4e6e-b2a3-203affcbb87e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:23.525176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] validate_networks() for [('5dd82b97-2333-4934-8cc7-c9806488a076', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:27:23.559762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-96a0d793-57f9-4e6e-b2a3-203affcbb87e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:23.569009 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-96a0d793-57f9-4e6e-b2a3-203affcbb87e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1214 microversion: 2.1 time: 0.084106 Sep 27 10:27:23.569718 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2630/5712] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:23 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1214 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:23.984250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:27:23.985049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:27:23.985748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:27:23.986348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:27:23.986772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:27:23.999607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:24.000107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:24.000537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:24.001782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:24.002411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:24.003038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:24.025999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Getting quotas for project 0a60557528994b829cd5131340a497b7. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:27:24.033383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Getting quotas for user a0e47aca97434f09846b580091b06829 and project 0a60557528994b829cd5131340a497b7. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:27:24.049538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:27:24.052159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:27:24.125357 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4bed24c4-c01d-4721-89aa-3560ce8ba33b tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.683546 Sep 27 10:27:24.126437 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3083/5713] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:27:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 686 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:27:24.139890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7a8f4ba7-d8e1-4f70-917c-0127215ad1f9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:24.141228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7a8f4ba7-d8e1-4f70-917c-0127215ad1f9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:24.161880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-de323aa9-a93a-445c-a0e0-1d00d95967a7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:24.163889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-de323aa9-a93a-445c-a0e0-1d00d95967a7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:24.174816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de323aa9-a93a-445c-a0e0-1d00d95967a7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:24.175946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de323aa9-a93a-445c-a0e0-1d00d95967a7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:24.184896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de323aa9-a93a-445c-a0e0-1d00d95967a7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:24.187558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de323aa9-a93a-445c-a0e0-1d00d95967a7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:24.188060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de323aa9-a93a-445c-a0e0-1d00d95967a7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:24.194731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-de323aa9-a93a-445c-a0e0-1d00d95967a7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:24.397656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-de323aa9-a93a-445c-a0e0-1d00d95967a7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:24.412344 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-de323aa9-a93a-445c-a0e0-1d00d95967a7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1368 microversion: 2.1 time: 0.254217 Sep 27 10:27:24.415083 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3084/5714] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:24 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1368 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:24.593118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef8ef208-3972-4096-833a-d31193f89ce0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:24.593750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ef8ef208-3972-4096-833a-d31193f89ce0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:24.608568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8ef208-3972-4096-833a-d31193f89ce0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:24.609286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8ef208-3972-4096-833a-d31193f89ce0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:24.609834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8ef208-3972-4096-833a-d31193f89ce0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:24.611592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8ef208-3972-4096-833a-d31193f89ce0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:24.612481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8ef208-3972-4096-833a-d31193f89ce0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:24.613069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8ef208-3972-4096-833a-d31193f89ce0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:24.668724 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7a8f4ba7-d8e1-4f70-917c-0127215ad1f9 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1328 microversion: 2.1 time: 0.533493 Sep 27 10:27:24.669651 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2631/5715] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:24 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1328 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:24.702309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ef8ef208-3972-4096-833a-d31193f89ce0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:24.712141 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef8ef208-3972-4096-833a-d31193f89ce0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1266 microversion: 2.1 time: 0.123027 Sep 27 10:27:24.714077 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3085/5716] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:24 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1266 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:25.440040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-82a4c58e-e93b-4307-aa5d-fe552fec3ad5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:25.441727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-82a4c58e-e93b-4307-aa5d-fe552fec3ad5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:25.453483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82a4c58e-e93b-4307-aa5d-fe552fec3ad5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:25.454209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82a4c58e-e93b-4307-aa5d-fe552fec3ad5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:25.454948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82a4c58e-e93b-4307-aa5d-fe552fec3ad5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:25.455858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82a4c58e-e93b-4307-aa5d-fe552fec3ad5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:25.456493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82a4c58e-e93b-4307-aa5d-fe552fec3ad5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:25.457275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82a4c58e-e93b-4307-aa5d-fe552fec3ad5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:25.635944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-82a4c58e-e93b-4307-aa5d-fe552fec3ad5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:25.646861 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-82a4c58e-e93b-4307-aa5d-fe552fec3ad5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1368 microversion: 2.1 time: 0.211078 Sep 27 10:27:25.648255 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2632/5717] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:25 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1368 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:25.696259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-209db1c0-bfce-4867-8197-f0fd45f640fc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:25.696922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-209db1c0-bfce-4867-8197-f0fd45f640fc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:25.710208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209db1c0-bfce-4867-8197-f0fd45f640fc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:25.710947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209db1c0-bfce-4867-8197-f0fd45f640fc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:25.711665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209db1c0-bfce-4867-8197-f0fd45f640fc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:25.713133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209db1c0-bfce-4867-8197-f0fd45f640fc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:25.713985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209db1c0-bfce-4867-8197-f0fd45f640fc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:25.714602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209db1c0-bfce-4867-8197-f0fd45f640fc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:25.737531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0132fb0-1034-4dd7-9330-6740a57d3f8f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:25.738215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b0132fb0-1034-4dd7-9330-6740a57d3f8f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:25.749841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0132fb0-1034-4dd7-9330-6740a57d3f8f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:25.750711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0132fb0-1034-4dd7-9330-6740a57d3f8f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:25.751448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0132fb0-1034-4dd7-9330-6740a57d3f8f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:25.752471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0132fb0-1034-4dd7-9330-6740a57d3f8f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:25.753043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0132fb0-1034-4dd7-9330-6740a57d3f8f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:25.753566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b0132fb0-1034-4dd7-9330-6740a57d3f8f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:25.809177 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-209db1c0-bfce-4867-8197-f0fd45f640fc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1415 microversion: 2.1 time: 0.117201 Sep 27 10:27:25.813632 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3086/5718] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:25 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1415 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:25.854911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b0132fb0-1034-4dd7-9330-6740a57d3f8f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:25.864872 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0132fb0-1034-4dd7-9330-6740a57d3f8f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1272 microversion: 2.1 time: 0.131639 Sep 27 10:27:25.865825 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2633/5719] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:25 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1272 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:26.671581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e827346c-1020-470a-952f-171b2fde71c7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:26.672526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e827346c-1020-470a-952f-171b2fde71c7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:26.680117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e827346c-1020-470a-952f-171b2fde71c7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:26.682657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e827346c-1020-470a-952f-171b2fde71c7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:26.683665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e827346c-1020-470a-952f-171b2fde71c7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:26.684915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e827346c-1020-470a-952f-171b2fde71c7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:26.685897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e827346c-1020-470a-952f-171b2fde71c7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:26.686622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e827346c-1020-470a-952f-171b2fde71c7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:26.841295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8efea015-13b3-4179-8a4c-11aacc50354a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:26.842864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8efea015-13b3-4179-8a4c-11aacc50354a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:26.861472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8efea015-13b3-4179-8a4c-11aacc50354a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:26.865121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8efea015-13b3-4179-8a4c-11aacc50354a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:26.865880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8efea015-13b3-4179-8a4c-11aacc50354a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:26.866852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8efea015-13b3-4179-8a4c-11aacc50354a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:26.874385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8efea015-13b3-4179-8a4c-11aacc50354a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:26.875117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8efea015-13b3-4179-8a4c-11aacc50354a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:27.254576 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8efea015-13b3-4179-8a4c-11aacc50354a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1463 microversion: 2.1 time: 0.420540 Sep 27 10:27:27.255878 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2634/5720] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:26 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1463 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:27.261123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-df044c03-940c-40a1-9a32-a1ea97d7c6aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:27.262121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-df044c03-940c-40a1-9a32-a1ea97d7c6aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:27.267028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e827346c-1020-470a-952f-171b2fde71c7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:27.269908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df044c03-940c-40a1-9a32-a1ea97d7c6aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:27.271523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df044c03-940c-40a1-9a32-a1ea97d7c6aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:27.273721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df044c03-940c-40a1-9a32-a1ea97d7c6aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:27.274843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df044c03-940c-40a1-9a32-a1ea97d7c6aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:27.275583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df044c03-940c-40a1-9a32-a1ea97d7c6aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:27.276553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df044c03-940c-40a1-9a32-a1ea97d7c6aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:27.281812 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e827346c-1020-470a-952f-171b2fde71c7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1368 microversion: 2.1 time: 0.614331 Sep 27 10:27:27.282407 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3087/5721] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:26 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1368 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:27.305749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c9fbb124-3430-48d7-a0ed-c0a2c047d126 req-41673d49-93af-4977-aa2e-f4487c8d905e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "403bfd67-4e7f-4918-baf9-4d14086a4598", "name": "network-vif-plugged", "status": "completed", "tag": "64716a19-8703-40a7-b759-cc1ba1192e49"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:27.321786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9fbb124-3430-48d7-a0ed-c0a2c047d126 req-41673d49-93af-4977-aa2e-f4487c8d905e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:27.322359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9fbb124-3430-48d7-a0ed-c0a2c047d126 req-41673d49-93af-4977-aa2e-f4487c8d905e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:27.323204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9fbb124-3430-48d7-a0ed-c0a2c047d126 req-41673d49-93af-4977-aa2e-f4487c8d905e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:27.341259 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c9fbb124-3430-48d7-a0ed-c0a2c047d126 req-41673d49-93af-4977-aa2e-f4487c8d905e service nova] Creating event network-vif-plugged:64716a19-8703-40a7-b759-cc1ba1192e49 for instance 403bfd67-4e7f-4918-baf9-4d14086a4598 on nch2-master-1 Sep 27 10:27:27.349251 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c9fbb124-3430-48d7-a0ed-c0a2c047d126 req-41673d49-93af-4977-aa2e-f4487c8d905e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064558 Sep 27 10:27:27.350328 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3088/5722] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:27.563352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-df044c03-940c-40a1-9a32-a1ea97d7c6aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:27.573460 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-df044c03-940c-40a1-9a32-a1ea97d7c6aa tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1314 microversion: 2.1 time: 0.316070 Sep 27 10:27:27.574944 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2635/5723] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:27 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1314 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:27.681598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-44583adc-5185-4512-b0e0-15aee7c6000d req-262a3f53-5467-431b-be60-e82f378732d2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "966d80e8-453c-467c-a8da-844a6ef094f5", "tag": "325e5b71-36a4-4466-bece-550e0138fb06"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:27.701160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-44583adc-5185-4512-b0e0-15aee7c6000d req-262a3f53-5467-431b-be60-e82f378732d2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:27.702042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-44583adc-5185-4512-b0e0-15aee7c6000d req-262a3f53-5467-431b-be60-e82f378732d2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:27.704237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-44583adc-5185-4512-b0e0-15aee7c6000d req-262a3f53-5467-431b-be60-e82f378732d2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:27.720939 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-44583adc-5185-4512-b0e0-15aee7c6000d req-262a3f53-5467-431b-be60-e82f378732d2 service nova] Creating event network-changed:325e5b71-36a4-4466-bece-550e0138fb06 for instance 966d80e8-453c-467c-a8da-844a6ef094f5 on nch1-master-1 Sep 27 10:27:27.727785 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-44583adc-5185-4512-b0e0-15aee7c6000d req-262a3f53-5467-431b-be60-e82f378732d2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051416 Sep 27 10:27:27.729513 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3089/5724] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:27:27.965638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-bac74754-5207-40b6-945d-66e69fd5a7c1 req-7d55bc23-2799-46fb-9e96-34344cf4af89 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b0700470-1689-4cf3-a999-f57299963859", "tag": "c98d6279-3918-42b1-940b-b8bcff3dcf6b"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:27.978875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bac74754-5207-40b6-945d-66e69fd5a7c1 req-7d55bc23-2799-46fb-9e96-34344cf4af89 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:27.979388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bac74754-5207-40b6-945d-66e69fd5a7c1 req-7d55bc23-2799-46fb-9e96-34344cf4af89 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:27.979832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bac74754-5207-40b6-945d-66e69fd5a7c1 req-7d55bc23-2799-46fb-9e96-34344cf4af89 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:28.006285 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-bac74754-5207-40b6-945d-66e69fd5a7c1 req-7d55bc23-2799-46fb-9e96-34344cf4af89 service nova] Creating event network-changed:c98d6279-3918-42b1-940b-b8bcff3dcf6b for instance b0700470-1689-4cf3-a999-f57299963859 on nch2-master-1 Sep 27 10:27:28.012146 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-bac74754-5207-40b6-945d-66e69fd5a7c1 req-7d55bc23-2799-46fb-9e96-34344cf4af89 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052151 Sep 27 10:27:28.013409 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2636/5725] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:28.281087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e4424035-eeb2-4dd9-804f-cc487d8f38a7 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:28.282397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e4424035-eeb2-4dd9-804f-cc487d8f38a7 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:28.289776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4424035-eeb2-4dd9-804f-cc487d8f38a7 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:28.290494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4424035-eeb2-4dd9-804f-cc487d8f38a7 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:28.291117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4424035-eeb2-4dd9-804f-cc487d8f38a7 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:28.291857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4424035-eeb2-4dd9-804f-cc487d8f38a7 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:28.292324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4424035-eeb2-4dd9-804f-cc487d8f38a7 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:28.292789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4424035-eeb2-4dd9-804f-cc487d8f38a7 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:28.315843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4b882691-5d78-4803-9184-e820c1e9859e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:28.316637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4b882691-5d78-4803-9184-e820c1e9859e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:28.324549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b882691-5d78-4803-9184-e820c1e9859e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:28.325180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b882691-5d78-4803-9184-e820c1e9859e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:28.325709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b882691-5d78-4803-9184-e820c1e9859e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:28.327253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b882691-5d78-4803-9184-e820c1e9859e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:28.328247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b882691-5d78-4803-9184-e820c1e9859e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:28.328724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b882691-5d78-4803-9184-e820c1e9859e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:28.437197 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e4424035-eeb2-4dd9-804f-cc487d8f38a7 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1463 microversion: 2.1 time: 0.160683 Sep 27 10:27:28.437699 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3090/5726] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:28 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1463 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:28.492994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4b882691-5d78-4803-9184-e820c1e9859e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:28.501978 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4b882691-5d78-4803-9184-e820c1e9859e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1368 microversion: 2.1 time: 0.191396 Sep 27 10:27:28.502469 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2637/5727] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:28 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1368 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:28.599013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d1e74d6-5ba2-41ad-bd2e-c4ee20fb65be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:28.600020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2d1e74d6-5ba2-41ad-bd2e-c4ee20fb65be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:28.607689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d1e74d6-5ba2-41ad-bd2e-c4ee20fb65be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:28.608255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d1e74d6-5ba2-41ad-bd2e-c4ee20fb65be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:28.608740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d1e74d6-5ba2-41ad-bd2e-c4ee20fb65be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:28.609865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d1e74d6-5ba2-41ad-bd2e-c4ee20fb65be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:28.610017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d1e74d6-5ba2-41ad-bd2e-c4ee20fb65be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:28.610330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d1e74d6-5ba2-41ad-bd2e-c4ee20fb65be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:28.769640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2d1e74d6-5ba2-41ad-bd2e-c4ee20fb65be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:28.778622 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d1e74d6-5ba2-41ad-bd2e-c4ee20fb65be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1314 microversion: 2.1 time: 0.184193 Sep 27 10:27:28.779547 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3091/5728] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:28 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1314 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:29.460919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d5e3abd9-2039-425e-b77d-cc3d3ab51bbf tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:29.461742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d5e3abd9-2039-425e-b77d-cc3d3ab51bbf tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:29.469579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5e3abd9-2039-425e-b77d-cc3d3ab51bbf tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:29.470310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5e3abd9-2039-425e-b77d-cc3d3ab51bbf tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:29.470959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5e3abd9-2039-425e-b77d-cc3d3ab51bbf tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:29.471886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5e3abd9-2039-425e-b77d-cc3d3ab51bbf tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:29.473043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5e3abd9-2039-425e-b77d-cc3d3ab51bbf tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:29.473657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5e3abd9-2039-425e-b77d-cc3d3ab51bbf tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:29.527652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fab2621f-3148-4360-9597-94fa0e737d83 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:29.528312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fab2621f-3148-4360-9597-94fa0e737d83 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:29.538328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fab2621f-3148-4360-9597-94fa0e737d83 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:29.538848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fab2621f-3148-4360-9597-94fa0e737d83 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:29.539646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fab2621f-3148-4360-9597-94fa0e737d83 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:29.540638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fab2621f-3148-4360-9597-94fa0e737d83 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:29.541383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fab2621f-3148-4360-9597-94fa0e737d83 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:29.541983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fab2621f-3148-4360-9597-94fa0e737d83 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:29.753902 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d5e3abd9-2039-425e-b77d-cc3d3ab51bbf tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1463 microversion: 2.1 time: 0.296677 Sep 27 10:27:29.761703 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2638/5729] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:29 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1463 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:29.804345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a19d86a-95d6-4269-a6bf-e79313bef095 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:29.805028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a19d86a-95d6-4269-a6bf-e79313bef095 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:29.812937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a19d86a-95d6-4269-a6bf-e79313bef095 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:29.814112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a19d86a-95d6-4269-a6bf-e79313bef095 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:29.814814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a19d86a-95d6-4269-a6bf-e79313bef095 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:29.815985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a19d86a-95d6-4269-a6bf-e79313bef095 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:29.817011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a19d86a-95d6-4269-a6bf-e79313bef095 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:29.817788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a19d86a-95d6-4269-a6bf-e79313bef095 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:29.999922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5a19d86a-95d6-4269-a6bf-e79313bef095 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:30.013743 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a19d86a-95d6-4269-a6bf-e79313bef095 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1314 microversion: 2.1 time: 0.214272 Sep 27 10:27:30.015314 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2639/5730] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:29 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1314 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:30.093166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fab2621f-3148-4360-9597-94fa0e737d83 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:30.102604 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fab2621f-3148-4360-9597-94fa0e737d83 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1558 microversion: 2.1 time: 0.580709 Sep 27 10:27:30.103694 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3092/5731] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:29 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1558 bytes in 583 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:30.127382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-041ce9f3-1fc1-486b-a610-207ba2d9dd58 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:30.127943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-041ce9f3-1fc1-486b-a610-207ba2d9dd58 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:30.134602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-041ce9f3-1fc1-486b-a610-207ba2d9dd58 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:30.135107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-041ce9f3-1fc1-486b-a610-207ba2d9dd58 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:30.135560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-041ce9f3-1fc1-486b-a610-207ba2d9dd58 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:30.136275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-041ce9f3-1fc1-486b-a610-207ba2d9dd58 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:30.136772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-041ce9f3-1fc1-486b-a610-207ba2d9dd58 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:30.137219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-041ce9f3-1fc1-486b-a610-207ba2d9dd58 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:30.224632 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-041ce9f3-1fc1-486b-a610-207ba2d9dd58 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.101477 Sep 27 10:27:30.229905 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2640/5732] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:27:30 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/os-interface => generated 284 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:30.777716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-db1b0f57-057e-4f46-9094-c6a3bdb7c18e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:30.778346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-db1b0f57-057e-4f46-9094-c6a3bdb7c18e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:30.785670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db1b0f57-057e-4f46-9094-c6a3bdb7c18e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:30.786572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db1b0f57-057e-4f46-9094-c6a3bdb7c18e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:30.787674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db1b0f57-057e-4f46-9094-c6a3bdb7c18e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:30.788519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db1b0f57-057e-4f46-9094-c6a3bdb7c18e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:30.789299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db1b0f57-057e-4f46-9094-c6a3bdb7c18e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:30.789976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-db1b0f57-057e-4f46-9094-c6a3bdb7c18e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:30.975361 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-db1b0f57-057e-4f46-9094-c6a3bdb7c18e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1463 microversion: 2.1 time: 0.201325 Sep 27 10:27:30.978180 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3093/5733] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:30 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1463 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:31.045047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-69710243-30df-444d-868c-5f25b757a0a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:31.046030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-69710243-30df-444d-868c-5f25b757a0a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:31.064107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69710243-30df-444d-868c-5f25b757a0a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:31.065321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69710243-30df-444d-868c-5f25b757a0a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:31.065828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69710243-30df-444d-868c-5f25b757a0a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:31.066708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69710243-30df-444d-868c-5f25b757a0a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:31.067215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69710243-30df-444d-868c-5f25b757a0a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:31.069002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-69710243-30df-444d-868c-5f25b757a0a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:31.322574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-69710243-30df-444d-868c-5f25b757a0a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:31.333660 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-69710243-30df-444d-868c-5f25b757a0a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1314 microversion: 2.1 time: 0.297576 Sep 27 10:27:31.335004 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2641/5734] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:31 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1314 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:31.711196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-94e53609-ba92-43a2-9f0d-98d75e786a32 req-42620dbf-61d9-48fa-99c5-2d56664d11ba service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "403bfd67-4e7f-4918-baf9-4d14086a4598", "tag": "64716a19-8703-40a7-b759-cc1ba1192e49"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:31.722643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-94e53609-ba92-43a2-9f0d-98d75e786a32 req-42620dbf-61d9-48fa-99c5-2d56664d11ba service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:31.723208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-94e53609-ba92-43a2-9f0d-98d75e786a32 req-42620dbf-61d9-48fa-99c5-2d56664d11ba service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:31.724087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-94e53609-ba92-43a2-9f0d-98d75e786a32 req-42620dbf-61d9-48fa-99c5-2d56664d11ba service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:31.738220 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-94e53609-ba92-43a2-9f0d-98d75e786a32 req-42620dbf-61d9-48fa-99c5-2d56664d11ba service nova] Creating event network-changed:64716a19-8703-40a7-b759-cc1ba1192e49 for instance 403bfd67-4e7f-4918-baf9-4d14086a4598 on nch2-master-1 Sep 27 10:27:31.743684 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-94e53609-ba92-43a2-9f0d-98d75e786a32 req-42620dbf-61d9-48fa-99c5-2d56664d11ba service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.226782 Sep 27 10:27:31.744259 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3094/5735] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:27:31.972021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-85f3a881-8e53-460e-bc79-68ced621c15a req-f4a3e702-6023-477c-970e-02fe2e7c9870 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "966d80e8-453c-467c-a8da-844a6ef094f5", "name": "network-vif-plugged", "status": "completed", "tag": "325e5b71-36a4-4466-bece-550e0138fb06"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:31.983284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-85f3a881-8e53-460e-bc79-68ced621c15a req-f4a3e702-6023-477c-970e-02fe2e7c9870 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:31.984156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-85f3a881-8e53-460e-bc79-68ced621c15a req-f4a3e702-6023-477c-970e-02fe2e7c9870 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:31.984899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-85f3a881-8e53-460e-bc79-68ced621c15a req-f4a3e702-6023-477c-970e-02fe2e7c9870 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:32.001845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d5243097-b4c2-4d97-8dd3-cd6b245550bc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:32.002640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d5243097-b4c2-4d97-8dd3-cd6b245550bc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:32.005375 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-85f3a881-8e53-460e-bc79-68ced621c15a req-f4a3e702-6023-477c-970e-02fe2e7c9870 service nova] Creating event network-vif-plugged:325e5b71-36a4-4466-bece-550e0138fb06 for instance 966d80e8-453c-467c-a8da-844a6ef094f5 on nch1-master-1 Sep 27 10:27:32.010851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5243097-b4c2-4d97-8dd3-cd6b245550bc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:32.011340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5243097-b4c2-4d97-8dd3-cd6b245550bc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:32.012000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5243097-b4c2-4d97-8dd3-cd6b245550bc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:32.012976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5243097-b4c2-4d97-8dd3-cd6b245550bc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:32.013642 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-85f3a881-8e53-460e-bc79-68ced621c15a req-f4a3e702-6023-477c-970e-02fe2e7c9870 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046080 Sep 27 10:27:32.014189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5243097-b4c2-4d97-8dd3-cd6b245550bc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:32.014404 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2642/5736] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:32.015038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5243097-b4c2-4d97-8dd3-cd6b245550bc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:32.274692 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d5243097-b4c2-4d97-8dd3-cd6b245550bc tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1463 microversion: 2.1 time: 0.278070 Sep 27 10:27:32.277095 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3095/5737] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:31 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1463 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:32.366065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ddf0a6d-9cd5-4851-a800-d50ae84f4e43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:32.367738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ddf0a6d-9cd5-4851-a800-d50ae84f4e43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:32.375674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ddf0a6d-9cd5-4851-a800-d50ae84f4e43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:32.377727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ddf0a6d-9cd5-4851-a800-d50ae84f4e43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:32.378423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ddf0a6d-9cd5-4851-a800-d50ae84f4e43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:32.379888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ddf0a6d-9cd5-4851-a800-d50ae84f4e43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:32.380782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ddf0a6d-9cd5-4851-a800-d50ae84f4e43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:32.381749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ddf0a6d-9cd5-4851-a800-d50ae84f4e43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:32.660497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2ddf0a6d-9cd5-4851-a800-d50ae84f4e43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:32.671136 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ddf0a6d-9cd5-4851-a800-d50ae84f4e43 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1314 microversion: 2.1 time: 0.314079 Sep 27 10:27:32.685167 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2643/5738] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:32 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1314 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:33.301270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e73a5d96-2018-41b6-a46c-e9366469c209 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:33.302158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e73a5d96-2018-41b6-a46c-e9366469c209 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:33.310018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73a5d96-2018-41b6-a46c-e9366469c209 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:33.312597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73a5d96-2018-41b6-a46c-e9366469c209 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:33.313086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73a5d96-2018-41b6-a46c-e9366469c209 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:33.313973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73a5d96-2018-41b6-a46c-e9366469c209 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:33.314468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73a5d96-2018-41b6-a46c-e9366469c209 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:33.314934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e73a5d96-2018-41b6-a46c-e9366469c209 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:33.497843 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e73a5d96-2018-41b6-a46c-e9366469c209 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1463 microversion: 2.1 time: 0.202163 Sep 27 10:27:33.498670 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3096/5739] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:33 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1463 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:33.699537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b4ec93f-c1cc-4ddb-a2ac-7ba78af471c2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:33.700278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0b4ec93f-c1cc-4ddb-a2ac-7ba78af471c2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:33.703413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:33.711158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4ec93f-c1cc-4ddb-a2ac-7ba78af471c2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:33.712056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4ec93f-c1cc-4ddb-a2ac-7ba78af471c2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:33.713139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4ec93f-c1cc-4ddb-a2ac-7ba78af471c2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:33.714739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4ec93f-c1cc-4ddb-a2ac-7ba78af471c2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:33.715449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4ec93f-c1cc-4ddb-a2ac-7ba78af471c2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:33.716109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4ec93f-c1cc-4ddb-a2ac-7ba78af471c2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:33.736002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:33.903164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0b4ec93f-c1cc-4ddb-a2ac-7ba78af471c2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:33.913165 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0b4ec93f-c1cc-4ddb-a2ac-7ba78af471c2 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1501 microversion: 2.1 time: 0.218239 Sep 27 10:27:33.914003 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2644/5740] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:33 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1501 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:33.937976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a8d23de7-c521-49cb-a196-dcde7d883d04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:33.938540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a8d23de7-c521-49cb-a196-dcde7d883d04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:33.944808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d23de7-c521-49cb-a196-dcde7d883d04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:33.945536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d23de7-c521-49cb-a196-dcde7d883d04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:33.945985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d23de7-c521-49cb-a196-dcde7d883d04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:33.946713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d23de7-c521-49cb-a196-dcde7d883d04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:33.947179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d23de7-c521-49cb-a196-dcde7d883d04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:33.947616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8d23de7-c521-49cb-a196-dcde7d883d04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:34.016307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:34.041511 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a8d23de7-c521-49cb-a196-dcde7d883d04 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5/action" status: 202 len: 0 microversion: 2.1 time: 0.107954 Sep 27 10:27:34.042249 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3097/5741] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:27:33 2022] POST /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5/action => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:27:34.052458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7522cce0-16c8-4271-9f5f-6e5b6f438390 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:34.053544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7522cce0-16c8-4271-9f5f-6e5b6f438390 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:34.061712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7522cce0-16c8-4271-9f5f-6e5b6f438390 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:34.062248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7522cce0-16c8-4271-9f5f-6e5b6f438390 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:34.062901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7522cce0-16c8-4271-9f5f-6e5b6f438390 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:34.064196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7522cce0-16c8-4271-9f5f-6e5b6f438390 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:34.065377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7522cce0-16c8-4271-9f5f-6e5b6f438390 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:34.066152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7522cce0-16c8-4271-9f5f-6e5b6f438390 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:34.229073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7522cce0-16c8-4271-9f5f-6e5b6f438390 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:34.247538 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7522cce0-16c8-4271-9f5f-6e5b6f438390 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1506 microversion: 2.1 time: 0.199603 Sep 27 10:27:34.264394 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2645/5742] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:34 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1506 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:34.523928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=28 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:34.529021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-167cf0ed-1903-4a98-9793-a26a3872286d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:34.529742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-167cf0ed-1903-4a98-9793-a26a3872286d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:34.536589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167cf0ed-1903-4a98-9793-a26a3872286d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:34.537283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167cf0ed-1903-4a98-9793-a26a3872286d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:34.537731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167cf0ed-1903-4a98-9793-a26a3872286d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:34.538458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167cf0ed-1903-4a98-9793-a26a3872286d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:34.538924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167cf0ed-1903-4a98-9793-a26a3872286d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:34.539356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-167cf0ed-1903-4a98-9793-a26a3872286d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:34.775674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:34.781667 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-167cf0ed-1903-4a98-9793-a26a3872286d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1463 microversion: 2.1 time: 0.258515 Sep 27 10:27:34.798789 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3098/5743] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:34 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1463 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:35.093443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-abccf9f2-05f7-4ac9-a2c3-d401011eb8ff req-5ed43d36-6987-4c74-a416-ca58b1ba4ee7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0700470-1689-4cf3-a999-f57299963859", "name": "network-vif-plugged", "status": "completed", "tag": "c98d6279-3918-42b1-940b-b8bcff3dcf6b"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:35.100317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:35.109361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abccf9f2-05f7-4ac9-a2c3-d401011eb8ff req-5ed43d36-6987-4c74-a416-ca58b1ba4ee7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:35.109853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abccf9f2-05f7-4ac9-a2c3-d401011eb8ff req-5ed43d36-6987-4c74-a416-ca58b1ba4ee7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:35.110296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-abccf9f2-05f7-4ac9-a2c3-d401011eb8ff req-5ed43d36-6987-4c74-a416-ca58b1ba4ee7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:35.123253 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-abccf9f2-05f7-4ac9-a2c3-d401011eb8ff req-5ed43d36-6987-4c74-a416-ca58b1ba4ee7 service nova] Creating event network-vif-plugged:c98d6279-3918-42b1-940b-b8bcff3dcf6b for instance b0700470-1689-4cf3-a999-f57299963859 on nch2-master-1 Sep 27 10:27:35.129283 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-abccf9f2-05f7-4ac9-a2c3-d401011eb8ff req-5ed43d36-6987-4c74-a416-ca58b1ba4ee7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039198 Sep 27 10:27:35.129672 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2646/5744] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:35.271384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-556928f6-6acf-4704-a04b-2f20cf9c0c65 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:35.272367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-556928f6-6acf-4704-a04b-2f20cf9c0c65 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:35.287368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556928f6-6acf-4704-a04b-2f20cf9c0c65 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:35.288186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556928f6-6acf-4704-a04b-2f20cf9c0c65 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:35.288636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556928f6-6acf-4704-a04b-2f20cf9c0c65 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:35.289393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556928f6-6acf-4704-a04b-2f20cf9c0c65 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:35.289859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556928f6-6acf-4704-a04b-2f20cf9c0c65 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:35.290485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-556928f6-6acf-4704-a04b-2f20cf9c0c65 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:35.440406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-556928f6-6acf-4704-a04b-2f20cf9c0c65 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:35.448518 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-556928f6-6acf-4704-a04b-2f20cf9c0c65 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1486 microversion: 2.1 time: 0.181668 Sep 27 10:27:35.449123 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3099/5745] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:35 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1486 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:35.473124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b267d438-e3da-48a0-8f2c-2865a462d5d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:35.473795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b267d438-e3da-48a0-8f2c-2865a462d5d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:35.480756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b267d438-e3da-48a0-8f2c-2865a462d5d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:35.481267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b267d438-e3da-48a0-8f2c-2865a462d5d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:35.481719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b267d438-e3da-48a0-8f2c-2865a462d5d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:35.482446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b267d438-e3da-48a0-8f2c-2865a462d5d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:35.482921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b267d438-e3da-48a0-8f2c-2865a462d5d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:35.483388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b267d438-e3da-48a0-8f2c-2865a462d5d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:35.506897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-b267d438-e3da-48a0-8f2c-2865a462d5d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lazy-loading 'flavor' on Instance uuid 966d80e8-453c-467c-a8da-844a6ef094f5 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:27:35.728068 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b267d438-e3da-48a0-8f2c-2865a462d5d7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5/action" status: 202 len: 0 microversion: 2.1 time: 0.258975 Sep 27 10:27:35.728720 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2647/5746] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:27:35 2022] POST /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5/action => generated 0 bytes in 261 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:27:35.739057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-77d2d527-fc6f-4d36-a8a6-c5dccd90e1be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:35.740056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-77d2d527-fc6f-4d36-a8a6-c5dccd90e1be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:35.748904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77d2d527-fc6f-4d36-a8a6-c5dccd90e1be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:35.749641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77d2d527-fc6f-4d36-a8a6-c5dccd90e1be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:35.750658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77d2d527-fc6f-4d36-a8a6-c5dccd90e1be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:35.752175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77d2d527-fc6f-4d36-a8a6-c5dccd90e1be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:35.752396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77d2d527-fc6f-4d36-a8a6-c5dccd90e1be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:35.753302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77d2d527-fc6f-4d36-a8a6-c5dccd90e1be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:35.808810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1738bebc-fe6f-402e-9df3-cfe23b2e994a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:35.809558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1738bebc-fe6f-402e-9df3-cfe23b2e994a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:35.816008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1738bebc-fe6f-402e-9df3-cfe23b2e994a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:35.816898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1738bebc-fe6f-402e-9df3-cfe23b2e994a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:35.817339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1738bebc-fe6f-402e-9df3-cfe23b2e994a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:35.818275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1738bebc-fe6f-402e-9df3-cfe23b2e994a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:35.819132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1738bebc-fe6f-402e-9df3-cfe23b2e994a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:35.819579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1738bebc-fe6f-402e-9df3-cfe23b2e994a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:35.905128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-77d2d527-fc6f-4d36-a8a6-c5dccd90e1be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:35.917539 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-77d2d527-fc6f-4d36-a8a6-c5dccd90e1be tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1492 microversion: 2.1 time: 0.183805 Sep 27 10:27:35.918423 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3100/5747] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:35 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1492 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:35.973464 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1738bebc-fe6f-402e-9df3-cfe23b2e994a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1463 microversion: 2.1 time: 0.168727 Sep 27 10:27:35.974204 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2648/5748] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:35 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1463 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:36.944516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-46b567ff-2205-4c29-890b-4bc7489c5992 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:36.945499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-46b567ff-2205-4c29-890b-4bc7489c5992 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:36.953319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46b567ff-2205-4c29-890b-4bc7489c5992 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:36.953865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46b567ff-2205-4c29-890b-4bc7489c5992 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:36.954371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46b567ff-2205-4c29-890b-4bc7489c5992 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:36.955193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46b567ff-2205-4c29-890b-4bc7489c5992 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:36.955841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46b567ff-2205-4c29-890b-4bc7489c5992 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:36.956497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46b567ff-2205-4c29-890b-4bc7489c5992 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:36.997446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a3479aa-5459-4167-beb9-3107e5e27e14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:36.998196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a3479aa-5459-4167-beb9-3107e5e27e14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:37.014163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3479aa-5459-4167-beb9-3107e5e27e14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:37.015288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3479aa-5459-4167-beb9-3107e5e27e14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:37.015919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3479aa-5459-4167-beb9-3107e5e27e14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:37.016773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3479aa-5459-4167-beb9-3107e5e27e14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:37.017517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3479aa-5459-4167-beb9-3107e5e27e14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:37.018244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a3479aa-5459-4167-beb9-3107e5e27e14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:37.134102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-46b567ff-2205-4c29-890b-4bc7489c5992 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:37.143696 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-46b567ff-2205-4c29-890b-4bc7489c5992 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1513 microversion: 2.1 time: 0.203999 Sep 27 10:27:37.144804 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3101/5749] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:36 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1513 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:37.175746 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a3479aa-5459-4167-beb9-3107e5e27e14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1648 microversion: 2.1 time: 0.182970 Sep 27 10:27:37.176502 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2649/5750] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:36 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1648 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:37.200526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f21a1e20-965a-4644-a997-b88adb46b04e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:37.201209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f21a1e20-965a-4644-a997-b88adb46b04e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:37.208202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21a1e20-965a-4644-a997-b88adb46b04e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:37.208831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21a1e20-965a-4644-a997-b88adb46b04e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:37.209397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21a1e20-965a-4644-a997-b88adb46b04e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:37.210112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21a1e20-965a-4644-a997-b88adb46b04e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:37.210577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21a1e20-965a-4644-a997-b88adb46b04e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:37.211038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f21a1e20-965a-4644-a997-b88adb46b04e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:37.360252 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f21a1e20-965a-4644-a997-b88adb46b04e tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1648 microversion: 2.1 time: 0.163792 Sep 27 10:27:37.361428 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3102/5751] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:37 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1648 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:37.385449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-78f665b0-9bca-4387-bd5e-92e16ac5370d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:37.386366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-78f665b0-9bca-4387-bd5e-92e16ac5370d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:37.393575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f665b0-9bca-4387-bd5e-92e16ac5370d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:37.394126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f665b0-9bca-4387-bd5e-92e16ac5370d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:37.394656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f665b0-9bca-4387-bd5e-92e16ac5370d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:37.395460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f665b0-9bca-4387-bd5e-92e16ac5370d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:37.395918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f665b0-9bca-4387-bd5e-92e16ac5370d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:37.396589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78f665b0-9bca-4387-bd5e-92e16ac5370d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:37.550036 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-78f665b0-9bca-4387-bd5e-92e16ac5370d tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1648 microversion: 2.1 time: 0.169441 Sep 27 10:27:37.562944 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2650/5752] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:37 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1648 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:37.576736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-57b816d3-32b8-4b3e-9708-14b5c3db332a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:37.577275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-57b816d3-32b8-4b3e-9708-14b5c3db332a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:37.583545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57b816d3-32b8-4b3e-9708-14b5c3db332a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:37.584527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57b816d3-32b8-4b3e-9708-14b5c3db332a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:37.584527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57b816d3-32b8-4b3e-9708-14b5c3db332a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:37.585253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57b816d3-32b8-4b3e-9708-14b5c3db332a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:37.585756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57b816d3-32b8-4b3e-9708-14b5c3db332a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:37.586151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57b816d3-32b8-4b3e-9708-14b5c3db332a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:37.616257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-57b816d3-32b8-4b3e-9708-14b5c3db332a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:27:37.701790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:37.713037 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-57b816d3-32b8-4b3e-9708-14b5c3db332a tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "DELETE /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 204 len: 0 microversion: 2.1 time: 0.139553 Sep 27 10:27:37.714067 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3103/5753] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:27:37 2022] DELETE /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:27:37.723230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-078d4f69-8bc9-4ff4-91e6-b129934ae63c tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:37.724094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-078d4f69-8bc9-4ff4-91e6-b129934ae63c tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:37.731286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-078d4f69-8bc9-4ff4-91e6-b129934ae63c tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:37.732038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-078d4f69-8bc9-4ff4-91e6-b129934ae63c tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:37.732735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-078d4f69-8bc9-4ff4-91e6-b129934ae63c tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:37.733646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-078d4f69-8bc9-4ff4-91e6-b129934ae63c tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:37.734259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-078d4f69-8bc9-4ff4-91e6-b129934ae63c tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:37.734770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-078d4f69-8bc9-4ff4-91e6-b129934ae63c tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:37.929875 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-078d4f69-8bc9-4ff4-91e6-b129934ae63c tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1654 microversion: 2.1 time: 0.210627 Sep 27 10:27:37.942356 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2651/5754] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:37 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1654 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:38.175560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-985a7e8a-081f-4036-9189-59d119967f44 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:38.177562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-985a7e8a-081f-4036-9189-59d119967f44 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:38.187264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985a7e8a-081f-4036-9189-59d119967f44 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:38.187926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985a7e8a-081f-4036-9189-59d119967f44 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:38.188468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985a7e8a-081f-4036-9189-59d119967f44 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:38.189512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985a7e8a-081f-4036-9189-59d119967f44 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:38.190095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985a7e8a-081f-4036-9189-59d119967f44 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:38.190647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-985a7e8a-081f-4036-9189-59d119967f44 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:38.344876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-985a7e8a-081f-4036-9189-59d119967f44 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:38.355183 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-985a7e8a-081f-4036-9189-59d119967f44 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1513 microversion: 2.1 time: 0.184863 Sep 27 10:27:38.356261 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3104/5755] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:38 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1513 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:38.956805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-914d2d71-90dc-49a8-90eb-9fff159779f1 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:38.957459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-914d2d71-90dc-49a8-90eb-9fff159779f1 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:38.967221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914d2d71-90dc-49a8-90eb-9fff159779f1 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:38.967813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914d2d71-90dc-49a8-90eb-9fff159779f1 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:38.968286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914d2d71-90dc-49a8-90eb-9fff159779f1 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:38.969147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914d2d71-90dc-49a8-90eb-9fff159779f1 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:38.969840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914d2d71-90dc-49a8-90eb-9fff159779f1 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:38.970334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914d2d71-90dc-49a8-90eb-9fff159779f1 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:39.149397 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-914d2d71-90dc-49a8-90eb-9fff159779f1 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1654 microversion: 2.1 time: 0.197032 Sep 27 10:27:39.151156 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2652/5756] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:38 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1654 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:39.383386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-76f5cc50-a071-43fe-bd79-c7e4d80561bf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:39.384721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-76f5cc50-a071-43fe-bd79-c7e4d80561bf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:39.393658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76f5cc50-a071-43fe-bd79-c7e4d80561bf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:39.394389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76f5cc50-a071-43fe-bd79-c7e4d80561bf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:39.395159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76f5cc50-a071-43fe-bd79-c7e4d80561bf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:39.396472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76f5cc50-a071-43fe-bd79-c7e4d80561bf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:39.397158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76f5cc50-a071-43fe-bd79-c7e4d80561bf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:39.398261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76f5cc50-a071-43fe-bd79-c7e4d80561bf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:39.576577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-76f5cc50-a071-43fe-bd79-c7e4d80561bf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:39.586642 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-76f5cc50-a071-43fe-bd79-c7e4d80561bf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.210721 Sep 27 10:27:39.587604 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3105/5757] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:39 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:40.127723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-41701851-123b-424b-987c-dc54e866f5eb req-810e3b16-39e9-45b1-90ef-9678440df813 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0700470-1689-4cf3-a999-f57299963859", "name": "network-vif-deleted", "tag": "c98d6279-3918-42b1-940b-b8bcff3dcf6b"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:40.142376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-41701851-123b-424b-987c-dc54e866f5eb req-810e3b16-39e9-45b1-90ef-9678440df813 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:40.143321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-41701851-123b-424b-987c-dc54e866f5eb req-810e3b16-39e9-45b1-90ef-9678440df813 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:40.144334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-41701851-123b-424b-987c-dc54e866f5eb req-810e3b16-39e9-45b1-90ef-9678440df813 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:40.166841 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-41701851-123b-424b-987c-dc54e866f5eb req-810e3b16-39e9-45b1-90ef-9678440df813 service nova] Creating event network-vif-deleted:c98d6279-3918-42b1-940b-b8bcff3dcf6b for instance b0700470-1689-4cf3-a999-f57299963859 on nch2-master-1 Sep 27 10:27:40.181526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4765e163-2256-4f6b-a657-fa5e13c811e3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:40.182537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4765e163-2256-4f6b-a657-fa5e13c811e3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:40.206924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4765e163-2256-4f6b-a657-fa5e13c811e3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:40.210988 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-41701851-123b-424b-987c-dc54e866f5eb req-810e3b16-39e9-45b1-90ef-9678440df813 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.088538 Sep 27 10:27:40.212382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4765e163-2256-4f6b-a657-fa5e13c811e3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:40.213273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4765e163-2256-4f6b-a657-fa5e13c811e3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:40.213993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4765e163-2256-4f6b-a657-fa5e13c811e3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:40.217023 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2653/5758] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:40.217559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4765e163-2256-4f6b-a657-fa5e13c811e3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:40.218053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4765e163-2256-4f6b-a657-fa5e13c811e3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:40.318192 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4765e163-2256-4f6b-a657-fa5e13c811e3 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1612 microversion: 2.1 time: 0.147624 Sep 27 10:27:40.322866 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3106/5759] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:40 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1612 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:40.618129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c05a1b5d-d694-454e-b159-5420f439318a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:40.618788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c05a1b5d-d694-454e-b159-5420f439318a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:40.664077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c05a1b5d-d694-454e-b159-5420f439318a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:40.664636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c05a1b5d-d694-454e-b159-5420f439318a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:40.665117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c05a1b5d-d694-454e-b159-5420f439318a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:40.665943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c05a1b5d-d694-454e-b159-5420f439318a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:40.666346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c05a1b5d-d694-454e-b159-5420f439318a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:40.666866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c05a1b5d-d694-454e-b159-5420f439318a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:40.728547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-737c200b-5219-48ec-ba59-97e77bfae2ae req-7335a494-4fcb-46a8-a2e6-293c1c32b88f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "966d80e8-453c-467c-a8da-844a6ef094f5", "name": "network-vif-plugged", "status": "completed", "tag": "325e5b71-36a4-4466-bece-550e0138fb06"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:40.752465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-737c200b-5219-48ec-ba59-97e77bfae2ae req-7335a494-4fcb-46a8-a2e6-293c1c32b88f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:40.753299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-737c200b-5219-48ec-ba59-97e77bfae2ae req-7335a494-4fcb-46a8-a2e6-293c1c32b88f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:40.753842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-737c200b-5219-48ec-ba59-97e77bfae2ae req-7335a494-4fcb-46a8-a2e6-293c1c32b88f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:40.773824 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-737c200b-5219-48ec-ba59-97e77bfae2ae req-7335a494-4fcb-46a8-a2e6-293c1c32b88f service nova] Creating event network-vif-plugged:325e5b71-36a4-4466-bece-550e0138fb06 for instance 966d80e8-453c-467c-a8da-844a6ef094f5 on nch1-master-1 Sep 27 10:27:40.781586 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-737c200b-5219-48ec-ba59-97e77bfae2ae req-7335a494-4fcb-46a8-a2e6-293c1c32b88f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.434068 Sep 27 10:27:40.782521 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2654/5760] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:27:40.965453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c05a1b5d-d694-454e-b159-5420f439318a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:40.976265 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c05a1b5d-d694-454e-b159-5420f439318a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.367888 Sep 27 10:27:40.976265 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3107/5761] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:40 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:41.355104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:41.356774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:41.365479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:41.366391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:41.366851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:41.367754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:41.368247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:41.368859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:41.424383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:41.425229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:41.425938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:41.426592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lazy-loading 'fault' on Instance uuid b0700470-1689-4cf3-a999-f57299963859 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:27:41.552512 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1d70f930-d612-4c1b-8b68-f0dae1a6c7e5 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 200 len: 1453 microversion: 2.1 time: 0.209291 Sep 27 10:27:41.553173 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2655/5762] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:41 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 1453 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:42.005746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9a965f0-f3f2-4c3b-80dc-c9d866834261 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:42.007604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c9a965f0-f3f2-4c3b-80dc-c9d866834261 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:42.016564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a965f0-f3f2-4c3b-80dc-c9d866834261 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:42.017391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a965f0-f3f2-4c3b-80dc-c9d866834261 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:42.019138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a965f0-f3f2-4c3b-80dc-c9d866834261 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:42.020889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a965f0-f3f2-4c3b-80dc-c9d866834261 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:42.021616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a965f0-f3f2-4c3b-80dc-c9d866834261 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:42.022147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a965f0-f3f2-4c3b-80dc-c9d866834261 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:42.307775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9a965f0-f3f2-4c3b-80dc-c9d866834261 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:42.317491 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9a965f0-f3f2-4c3b-80dc-c9d866834261 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.321739 Sep 27 10:27:42.318492 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3108/5763] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:41 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:42.593402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:42.595478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: b0700470-1689-4cf3-a999-f57299963859] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:42.612377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:42.613549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:42.614523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:42.616158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:42.617593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:42.618689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:42.654802 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] HTTP exception thrown: Instance b0700470-1689-4cf3-a999-f57299963859 could not be found. Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Request method failure captured: Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 HTTP/1.1 Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs_3k6dG7eZRa2SrjYxWlrbDDEB5A_ISd3nBDmapgIyIQK5_6XJZMzMKbtnZ1JZpR-49kDu0VUW-7OGmw2JwXNXa7p-i7a1KYYxi3-oqjUbEhm2gG2cRsp97c3WwASc7WN41rB6rjSNowNa3dltleEJTEhqc-t1wC-mkSYcZZFmqZNU Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 0a60557528994b829cd5131340a497b7 Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestServerMultinode-180876763 Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0a60557528994b829cd5131340a497b7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0a60557528994b829cd5131340a497b7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0a60557528994b829cd5131340a497b7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0a60557528994b829cd5131340a497b7"}]}] Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestServerMultinode-180876763 Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 0a60557528994b829cd5131340a497b7 Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestServerMultinode-180876763 Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestServerMultinode-180876763-project-admin Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: a0e47aca97434f09846b580091b06829 Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestServerMultinode-180876763-project-admin Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance b0700470-1689-4cf3-a999-f57299963859 could not be found. Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'b0700470-1689-4cf3-a999-f57299963859'} Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b0700470-1689-4cf3-a999-f57299963859 could not be found. Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b0700470-1689-4cf3-a999-f57299963859 could not be found. Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:27:42.658977 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:27:42.671180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:27:42.671180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:27:42.671180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:27:42.671180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:27:42.671180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:27:42.671180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b0700470-1689-4cf3-a999-f57299963859 could not be found. Sep 27 10:27:42.671180 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:27:42.673777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Returning 404 to user: Instance b0700470-1689-4cf3-a999-f57299963859 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:27:42.674441 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a9370595-71f5-4d86-bf90-24af11d8d583 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859" status: 404 len: 111 microversion: 2.1 time: 0.092601 Sep 27 10:27:42.680181 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2656/5764] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:42 2022] GET /compute/v2.1/servers/b0700470-1689-4cf3-a999-f57299963859 => generated 111 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:27:42.686275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b021b248-ba88-416d-9a46-7bf836cd9b79 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:42.687038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b021b248-ba88-416d-9a46-7bf836cd9b79 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:42.693791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b021b248-ba88-416d-9a46-7bf836cd9b79 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:42.694704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b021b248-ba88-416d-9a46-7bf836cd9b79 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:42.695491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b021b248-ba88-416d-9a46-7bf836cd9b79 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:42.696409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b021b248-ba88-416d-9a46-7bf836cd9b79 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:42.697088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b021b248-ba88-416d-9a46-7bf836cd9b79 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:42.697520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b021b248-ba88-416d-9a46-7bf836cd9b79 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:42.739951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b021b248-ba88-416d-9a46-7bf836cd9b79 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:27:42.862113 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b021b248-ba88-416d-9a46-7bf836cd9b79 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "DELETE /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 204 len: 0 microversion: 2.1 time: 0.179606 Sep 27 10:27:42.863054 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3109/5765] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:27:42 2022] DELETE /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:27:42.873453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9dea7336-44ce-48b2-a2ae-8d5b32256a26 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:42.874427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9dea7336-44ce-48b2-a2ae-8d5b32256a26 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:42.883794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dea7336-44ce-48b2-a2ae-8d5b32256a26 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:42.884451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dea7336-44ce-48b2-a2ae-8d5b32256a26 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:42.885177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dea7336-44ce-48b2-a2ae-8d5b32256a26 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:42.886300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dea7336-44ce-48b2-a2ae-8d5b32256a26 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:42.886943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dea7336-44ce-48b2-a2ae-8d5b32256a26 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:42.887546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9dea7336-44ce-48b2-a2ae-8d5b32256a26 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:43.111110 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9dea7336-44ce-48b2-a2ae-8d5b32256a26 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1653 microversion: 2.1 time: 0.242363 Sep 27 10:27:43.122735 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2657/5766] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:42 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1653 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:43.345321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2b3efaae-14cf-45dc-91bc-58d8c9d1a1b5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:43.347447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2b3efaae-14cf-45dc-91bc-58d8c9d1a1b5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:43.366232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b3efaae-14cf-45dc-91bc-58d8c9d1a1b5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:43.367302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b3efaae-14cf-45dc-91bc-58d8c9d1a1b5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:43.368036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b3efaae-14cf-45dc-91bc-58d8c9d1a1b5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:43.369023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b3efaae-14cf-45dc-91bc-58d8c9d1a1b5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:43.369719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b3efaae-14cf-45dc-91bc-58d8c9d1a1b5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:43.370362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b3efaae-14cf-45dc-91bc-58d8c9d1a1b5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:43.560247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2b3efaae-14cf-45dc-91bc-58d8c9d1a1b5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:43.572441 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2b3efaae-14cf-45dc-91bc-58d8c9d1a1b5 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.232154 Sep 27 10:27:43.574557 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3110/5767] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:43 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:44.138854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ee6fee3a-de71-4eb8-bdbf-ed24e60f2975 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:44.140450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ee6fee3a-de71-4eb8-bdbf-ed24e60f2975 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:44.154705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6fee3a-de71-4eb8-bdbf-ed24e60f2975 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:44.155571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6fee3a-de71-4eb8-bdbf-ed24e60f2975 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:44.156651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6fee3a-de71-4eb8-bdbf-ed24e60f2975 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:44.157680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6fee3a-de71-4eb8-bdbf-ed24e60f2975 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:44.158693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6fee3a-de71-4eb8-bdbf-ed24e60f2975 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:44.159326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee6fee3a-de71-4eb8-bdbf-ed24e60f2975 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:44.348301 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ee6fee3a-de71-4eb8-bdbf-ed24e60f2975 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1653 microversion: 2.1 time: 0.214034 Sep 27 10:27:44.349571 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2658/5768] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:44 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1653 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:44.600886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7400627a-27af-446d-83fa-d599f1d1a285 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:44.601926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7400627a-27af-446d-83fa-d599f1d1a285 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:44.610305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7400627a-27af-446d-83fa-d599f1d1a285 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:44.611048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7400627a-27af-446d-83fa-d599f1d1a285 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:44.612029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7400627a-27af-446d-83fa-d599f1d1a285 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:44.613101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7400627a-27af-446d-83fa-d599f1d1a285 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:44.613912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7400627a-27af-446d-83fa-d599f1d1a285 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:44.614502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7400627a-27af-446d-83fa-d599f1d1a285 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:44.883964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7400627a-27af-446d-83fa-d599f1d1a285 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:44.893810 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7400627a-27af-446d-83fa-d599f1d1a285 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.297477 Sep 27 10:27:44.898830 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3111/5769] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:44 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:45.009730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1659a307-505b-444a-bbcd-a8e8ed38fd66 req-a15c697b-5475-4f92-b49d-68436139973d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cee68c7e-0857-45fb-b5b8-e98bd46532dc", "name": "network-vif-deleted", "tag": "04a989a1-7997-4f2b-98c4-d619153897cc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:45.027915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1659a307-505b-444a-bbcd-a8e8ed38fd66 req-a15c697b-5475-4f92-b49d-68436139973d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:45.028390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1659a307-505b-444a-bbcd-a8e8ed38fd66 req-a15c697b-5475-4f92-b49d-68436139973d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:45.031555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1659a307-505b-444a-bbcd-a8e8ed38fd66 req-a15c697b-5475-4f92-b49d-68436139973d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:45.053284 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1659a307-505b-444a-bbcd-a8e8ed38fd66 req-a15c697b-5475-4f92-b49d-68436139973d service nova] Creating event network-vif-deleted:04a989a1-7997-4f2b-98c4-d619153897cc for instance cee68c7e-0857-45fb-b5b8-e98bd46532dc on nch1-master-1 Sep 27 10:27:45.065808 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1659a307-505b-444a-bbcd-a8e8ed38fd66 req-a15c697b-5475-4f92-b49d-68436139973d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064160 Sep 27 10:27:45.066944 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2659/5770] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:27:45.377212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:45.378121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:45.392150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:45.392837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:45.393829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:45.395871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:45.399063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:45.399763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:45.405141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:45.466089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:45.467049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:45.467741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:45.468322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lazy-loading 'fault' on Instance uuid cee68c7e-0857-45fb-b5b8-e98bd46532dc {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:27:45.542141 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-48a30e7e-0c0b-46b0-93e5-08038d9ccd37 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 200 len: 1453 microversion: 2.1 time: 0.170353 Sep 27 10:27:45.543310 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3112/5771] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:45 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 1453 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:45.927852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8c3ca903-634d-449f-bf95-772ad042e536 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:45.929577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8c3ca903-634d-449f-bf95-772ad042e536 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:45.942076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c3ca903-634d-449f-bf95-772ad042e536 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:45.946067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c3ca903-634d-449f-bf95-772ad042e536 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:45.947549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c3ca903-634d-449f-bf95-772ad042e536 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:45.947549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c3ca903-634d-449f-bf95-772ad042e536 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:45.948254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c3ca903-634d-449f-bf95-772ad042e536 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:45.948937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8c3ca903-634d-449f-bf95-772ad042e536 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:46.151229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8c3ca903-634d-449f-bf95-772ad042e536 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:46.162622 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8c3ca903-634d-449f-bf95-772ad042e536 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.242223 Sep 27 10:27:46.164112 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2660/5772] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:45 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:46.576956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:46.579432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] [instance: cee68c7e-0857-45fb-b5b8-e98bd46532dc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:46.588256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:46.589068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:46.590078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:46.591133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:46.592864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:46.593672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:46.631721 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] HTTP exception thrown: Instance cee68c7e-0857-45fb-b5b8-e98bd46532dc could not be found. Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Request method failure captured: Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc HTTP/1.1 Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs_3k6dG7eZRa2SrjYxWlrbDDEB5A_ISd3nBDmapgIyIQK5_6XJZMzMKbtnZ1JZpR-49kDu0VUW-7OGmw2JwXNXa7p-i7a1KYYxi3-oqjUbEhm2gG2cRsp97c3WwASc7WN41rB6rjSNowNa3dltleEJTEhqc-t1wC-mkSYcZZFmqZNU Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 0a60557528994b829cd5131340a497b7 Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestServerMultinode-180876763 Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0a60557528994b829cd5131340a497b7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0a60557528994b829cd5131340a497b7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0a60557528994b829cd5131340a497b7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0a60557528994b829cd5131340a497b7"}]}] Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestServerMultinode-180876763 Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 0a60557528994b829cd5131340a497b7 Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestServerMultinode-180876763 Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestServerMultinode-180876763-project-admin Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: a0e47aca97434f09846b580091b06829 Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestServerMultinode-180876763-project-admin Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance cee68c7e-0857-45fb-b5b8-e98bd46532dc could not be found. Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'cee68c7e-0857-45fb-b5b8-e98bd46532dc'} Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cee68c7e-0857-45fb-b5b8-e98bd46532dc could not be found. Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cee68c7e-0857-45fb-b5b8-e98bd46532dc could not be found. Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:27:46.641553 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:27:46.646140 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:27:46.646140 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:27:46.646140 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:27:46.646140 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:27:46.646140 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:27:46.646140 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cee68c7e-0857-45fb-b5b8-e98bd46532dc could not be found. Sep 27 10:27:46.646140 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:27:46.647562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] Returning 404 to user: Instance cee68c7e-0857-45fb-b5b8-e98bd46532dc could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:27:46.648732 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2e12d3b3-1851-43e4-a407-f3ecfa714b14 tempest-TestServerMultinode-180876763 tempest-TestServerMultinode-180876763-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc" status: 404 len: 111 microversion: 2.1 time: 0.076959 Sep 27 10:27:46.649783 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3113/5773] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:46 2022] GET /compute/v2.1/servers/cee68c7e-0857-45fb-b5b8-e98bd46532dc => generated 111 bytes in 78 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:27:47.188927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1bd1b40a-a7c4-4ab3-9149-84a814d393f1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:47.190461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1bd1b40a-a7c4-4ab3-9149-84a814d393f1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:47.199593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd1b40a-a7c4-4ab3-9149-84a814d393f1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:47.201738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd1b40a-a7c4-4ab3-9149-84a814d393f1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:47.202351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd1b40a-a7c4-4ab3-9149-84a814d393f1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:47.203252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd1b40a-a7c4-4ab3-9149-84a814d393f1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:47.203827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd1b40a-a7c4-4ab3-9149-84a814d393f1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:47.204417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd1b40a-a7c4-4ab3-9149-84a814d393f1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:47.497162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1bd1b40a-a7c4-4ab3-9149-84a814d393f1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:47.512062 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1bd1b40a-a7c4-4ab3-9149-84a814d393f1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.326965 Sep 27 10:27:47.512638 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2661/5774] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:47 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:27:48.538706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d1a6a4c0-492a-479a-83a8-aba3cf41e82e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:48.540030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d1a6a4c0-492a-479a-83a8-aba3cf41e82e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:48.558821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a6a4c0-492a-479a-83a8-aba3cf41e82e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:48.566591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a6a4c0-492a-479a-83a8-aba3cf41e82e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:48.567063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a6a4c0-492a-479a-83a8-aba3cf41e82e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:48.567938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a6a4c0-492a-479a-83a8-aba3cf41e82e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:48.568602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a6a4c0-492a-479a-83a8-aba3cf41e82e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:48.569220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d1a6a4c0-492a-479a-83a8-aba3cf41e82e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:48.839329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d1a6a4c0-492a-479a-83a8-aba3cf41e82e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:48.848210 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d1a6a4c0-492a-479a-83a8-aba3cf41e82e tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.315312 Sep 27 10:27:48.849139 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3114/5775] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:48 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:49.876579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-74556c95-b03c-47ef-b226-6dc6c26041fd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:49.877293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-74556c95-b03c-47ef-b226-6dc6c26041fd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:49.908350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74556c95-b03c-47ef-b226-6dc6c26041fd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:49.909371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74556c95-b03c-47ef-b226-6dc6c26041fd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:49.910888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74556c95-b03c-47ef-b226-6dc6c26041fd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:49.913355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74556c95-b03c-47ef-b226-6dc6c26041fd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:49.923878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74556c95-b03c-47ef-b226-6dc6c26041fd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:49.924528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74556c95-b03c-47ef-b226-6dc6c26041fd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:49.930881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:50.149559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-74556c95-b03c-47ef-b226-6dc6c26041fd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:50.160177 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-74556c95-b03c-47ef-b226-6dc6c26041fd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.290147 Sep 27 10:27:50.176908 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2662/5776] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:49 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:51.188533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-894af4b9-95c9-4660-9a3c-040a3905253f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:51.189162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-894af4b9-95c9-4660-9a3c-040a3905253f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:51.217077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-894af4b9-95c9-4660-9a3c-040a3905253f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:51.220216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-894af4b9-95c9-4660-9a3c-040a3905253f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:51.221038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-894af4b9-95c9-4660-9a3c-040a3905253f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:51.222227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-894af4b9-95c9-4660-9a3c-040a3905253f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:51.223083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-894af4b9-95c9-4660-9a3c-040a3905253f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:51.223695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-894af4b9-95c9-4660-9a3c-040a3905253f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:51.489335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-894af4b9-95c9-4660-9a3c-040a3905253f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:51.498387 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-894af4b9-95c9-4660-9a3c-040a3905253f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.314003 Sep 27 10:27:51.517781 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3115/5777] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:51 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:52.536331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8804a47f-e3bc-4d9d-8aee-ab40ef2d3836 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:52.537975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8804a47f-e3bc-4d9d-8aee-ab40ef2d3836 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:52.548529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:27:52.551209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8804a47f-e3bc-4d9d-8aee-ab40ef2d3836 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:52.552070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8804a47f-e3bc-4d9d-8aee-ab40ef2d3836 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:52.552883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8804a47f-e3bc-4d9d-8aee-ab40ef2d3836 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:52.554509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8804a47f-e3bc-4d9d-8aee-ab40ef2d3836 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:52.555530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8804a47f-e3bc-4d9d-8aee-ab40ef2d3836 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:52.556220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8804a47f-e3bc-4d9d-8aee-ab40ef2d3836 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:52.884911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8804a47f-e3bc-4d9d-8aee-ab40ef2d3836 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:52.900508 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8804a47f-e3bc-4d9d-8aee-ab40ef2d3836 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.371615 Sep 27 10:27:52.901443 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2663/5778] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:52 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:53.924150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e4106a4-13b0-4864-9e7d-3a88a60f0aff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:53.924834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2e4106a4-13b0-4864-9e7d-3a88a60f0aff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:53.934516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e4106a4-13b0-4864-9e7d-3a88a60f0aff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:53.935022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e4106a4-13b0-4864-9e7d-3a88a60f0aff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:53.935901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e4106a4-13b0-4864-9e7d-3a88a60f0aff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:53.936603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e4106a4-13b0-4864-9e7d-3a88a60f0aff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:53.937270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e4106a4-13b0-4864-9e7d-3a88a60f0aff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:53.937703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e4106a4-13b0-4864-9e7d-3a88a60f0aff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:54.372427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2e4106a4-13b0-4864-9e7d-3a88a60f0aff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:54.385186 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2e4106a4-13b0-4864-9e7d-3a88a60f0aff tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1505 microversion: 2.1 time: 0.460775 Sep 27 10:27:54.385875 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3116/5779] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:53 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1505 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:55.409477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-634be89e-40b5-4d91-ad04-755c469daf52 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:55.410322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-634be89e-40b5-4d91-ad04-755c469daf52 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:55.419335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-634be89e-40b5-4d91-ad04-755c469daf52 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:55.419899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-634be89e-40b5-4d91-ad04-755c469daf52 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:55.420693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-634be89e-40b5-4d91-ad04-755c469daf52 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:55.421572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-634be89e-40b5-4d91-ad04-755c469daf52 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:55.422244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-634be89e-40b5-4d91-ad04-755c469daf52 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:55.422948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-634be89e-40b5-4d91-ad04-755c469daf52 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:55.632264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-634be89e-40b5-4d91-ad04-755c469daf52 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:55.645157 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-634be89e-40b5-4d91-ad04-755c469daf52 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1505 microversion: 2.1 time: 0.238796 Sep 27 10:27:55.660888 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2664/5780] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:55 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1505 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:56.671016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-35b55ae1-59b3-4766-8b1e-ddef203376cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:56.671621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-35b55ae1-59b3-4766-8b1e-ddef203376cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:56.678849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35b55ae1-59b3-4766-8b1e-ddef203376cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:56.679599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35b55ae1-59b3-4766-8b1e-ddef203376cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:56.680226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35b55ae1-59b3-4766-8b1e-ddef203376cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:56.681214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35b55ae1-59b3-4766-8b1e-ddef203376cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:56.681850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35b55ae1-59b3-4766-8b1e-ddef203376cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:56.682499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35b55ae1-59b3-4766-8b1e-ddef203376cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:56.856347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-35b55ae1-59b3-4766-8b1e-ddef203376cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:56.870024 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-35b55ae1-59b3-4766-8b1e-ddef203376cf tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 200 len: 1508 microversion: 2.1 time: 0.203325 Sep 27 10:27:56.870672 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3117/5781] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:56 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 1508 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:56.893660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2116ecee-c18c-4609-9e70-b203c4de02db tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:56.894932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2116ecee-c18c-4609-9e70-b203c4de02db tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:56.902243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2116ecee-c18c-4609-9e70-b203c4de02db tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:56.902965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2116ecee-c18c-4609-9e70-b203c4de02db tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:56.903593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2116ecee-c18c-4609-9e70-b203c4de02db tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:56.904324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2116ecee-c18c-4609-9e70-b203c4de02db tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:56.904830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2116ecee-c18c-4609-9e70-b203c4de02db tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:56.905280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2116ecee-c18c-4609-9e70-b203c4de02db tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:57.077142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2116ecee-c18c-4609-9e70-b203c4de02db tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:57.089800 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2116ecee-c18c-4609-9e70-b203c4de02db tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.200721 Sep 27 10:27:57.094003 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2665/5782] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:56 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:57.125049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b1a23a89-f338-4b5a-a50e-a4b1c37cb82a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:57.125693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b1a23a89-f338-4b5a-a50e-a4b1c37cb82a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:57.132178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a23a89-f338-4b5a-a50e-a4b1c37cb82a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:57.132765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a23a89-f338-4b5a-a50e-a4b1c37cb82a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:57.133272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a23a89-f338-4b5a-a50e-a4b1c37cb82a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:57.134083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a23a89-f338-4b5a-a50e-a4b1c37cb82a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:57.134609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a23a89-f338-4b5a-a50e-a4b1c37cb82a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:57.135102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b1a23a89-f338-4b5a-a50e-a4b1c37cb82a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:57.279972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b1a23a89-f338-4b5a-a50e-a4b1c37cb82a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:57.289652 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b1a23a89-f338-4b5a-a50e-a4b1c37cb82a tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.168562 Sep 27 10:27:57.290382 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3118/5783] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:57 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:57.313075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5604c08d-ab4d-4c59-b032-c6bf4f3857f7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:57.313822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5604c08d-ab4d-4c59-b032-c6bf4f3857f7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:57.321058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5604c08d-ab4d-4c59-b032-c6bf4f3857f7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:57.321697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5604c08d-ab4d-4c59-b032-c6bf4f3857f7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:57.322399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5604c08d-ab4d-4c59-b032-c6bf4f3857f7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:57.323176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5604c08d-ab4d-4c59-b032-c6bf4f3857f7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:57.323708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5604c08d-ab4d-4c59-b032-c6bf4f3857f7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:57.324133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5604c08d-ab4d-4c59-b032-c6bf4f3857f7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:57.469726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5604c08d-ab4d-4c59-b032-c6bf4f3857f7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:57.478892 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5604c08d-ab4d-4c59-b032-c6bf4f3857f7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.169889 Sep 27 10:27:57.479489 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2666/5784] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:57 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:57.506998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-87c0d427-dfd4-446d-a6d2-3a673d77b102 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:57.507724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-87c0d427-dfd4-446d-a6d2-3a673d77b102 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:57.515117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c0d427-dfd4-446d-a6d2-3a673d77b102 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:57.515860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c0d427-dfd4-446d-a6d2-3a673d77b102 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:57.516487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c0d427-dfd4-446d-a6d2-3a673d77b102 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:57.517632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c0d427-dfd4-446d-a6d2-3a673d77b102 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:57.518288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c0d427-dfd4-446d-a6d2-3a673d77b102 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:57.518897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-87c0d427-dfd4-446d-a6d2-3a673d77b102 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:57.700353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-87c0d427-dfd4-446d-a6d2-3a673d77b102 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:57.709329 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-87c0d427-dfd4-446d-a6d2-3a673d77b102 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.206683 Sep 27 10:27:57.720920 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3119/5785] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:57 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:57.733176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f48d970a-d254-4bc6-b102-c57db90c2cd3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:57.740337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f48d970a-d254-4bc6-b102-c57db90c2cd3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:57.741077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f48d970a-d254-4bc6-b102-c57db90c2cd3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:57.743084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f48d970a-d254-4bc6-b102-c57db90c2cd3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:57.769462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f48d970a-d254-4bc6-b102-c57db90c2cd3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Going to try to stop instance {{(pid=145969) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 10:27:57.818283 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f48d970a-d254-4bc6-b102-c57db90c2cd3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.088996 Sep 27 10:27:57.819091 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2667/5786] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:27:57 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:27:57.827686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e5a02076-e9d3-4998-9ebe-88a4ca20bffb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:57.828306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e5a02076-e9d3-4998-9ebe-88a4ca20bffb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:57.834729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a02076-e9d3-4998-9ebe-88a4ca20bffb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:57.835263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a02076-e9d3-4998-9ebe-88a4ca20bffb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:57.835844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a02076-e9d3-4998-9ebe-88a4ca20bffb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:57.836584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a02076-e9d3-4998-9ebe-88a4ca20bffb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:57.837077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a02076-e9d3-4998-9ebe-88a4ca20bffb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:57.837504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e5a02076-e9d3-4998-9ebe-88a4ca20bffb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:57.991115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e5a02076-e9d3-4998-9ebe-88a4ca20bffb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:58.005632 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e5a02076-e9d3-4998-9ebe-88a4ca20bffb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.181758 Sep 27 10:27:58.007155 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3120/5787] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:57 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:27:58.039831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-bc13a001-57d1-4fb9-b31b-35884496c1cb req-8cd6064e-4ecc-4eac-b52a-e4221eab693b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "966d80e8-453c-467c-a8da-844a6ef094f5", "name": "network-vif-unplugged", "status": "completed", "tag": "325e5b71-36a4-4466-bece-550e0138fb06"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:27:58.050334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bc13a001-57d1-4fb9-b31b-35884496c1cb req-8cd6064e-4ecc-4eac-b52a-e4221eab693b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:58.051755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bc13a001-57d1-4fb9-b31b-35884496c1cb req-8cd6064e-4ecc-4eac-b52a-e4221eab693b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:58.052297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bc13a001-57d1-4fb9-b31b-35884496c1cb req-8cd6064e-4ecc-4eac-b52a-e4221eab693b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:58.066582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.server_external_events [req-bc13a001-57d1-4fb9-b31b-35884496c1cb req-8cd6064e-4ecc-4eac-b52a-e4221eab693b service nova] Unable to find a host for instance 966d80e8-453c-467c-a8da-844a6ef094f5. Dropping event network-vif-unplugged {{(pid=145969) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 27 10:27:58.067685 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-bc13a001-57d1-4fb9-b31b-35884496c1cb req-8cd6064e-4ecc-4eac-b52a-e4221eab693b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.033446 Sep 27 10:27:58.068747 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2668/5788] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:27:58 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 35 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Sep 27 10:27:59.029497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3af0d17d-b77f-4362-9b49-19e0897416dd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:27:59.030100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3af0d17d-b77f-4362-9b49-19e0897416dd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:27:59.038627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3af0d17d-b77f-4362-9b49-19e0897416dd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:59.039291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3af0d17d-b77f-4362-9b49-19e0897416dd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:59.039611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3af0d17d-b77f-4362-9b49-19e0897416dd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:59.040404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3af0d17d-b77f-4362-9b49-19e0897416dd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:27:59.041221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3af0d17d-b77f-4362-9b49-19e0897416dd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:27:59.041863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3af0d17d-b77f-4362-9b49-19e0897416dd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:27:59.192260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3af0d17d-b77f-4362-9b49-19e0897416dd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:27:59.200557 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3af0d17d-b77f-4362-9b49-19e0897416dd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.174984 Sep 27 10:27:59.201189 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3121/5789] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:27:59 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:00.224352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fcee0680-fe4a-48bc-a7dd-dbcc2fb74f4b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:00.225279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fcee0680-fe4a-48bc-a7dd-dbcc2fb74f4b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:00.233227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcee0680-fe4a-48bc-a7dd-dbcc2fb74f4b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:00.233799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcee0680-fe4a-48bc-a7dd-dbcc2fb74f4b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:00.234317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcee0680-fe4a-48bc-a7dd-dbcc2fb74f4b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:00.235157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcee0680-fe4a-48bc-a7dd-dbcc2fb74f4b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:00.235749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcee0680-fe4a-48bc-a7dd-dbcc2fb74f4b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:00.236320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcee0680-fe4a-48bc-a7dd-dbcc2fb74f4b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:00.387137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fcee0680-fe4a-48bc-a7dd-dbcc2fb74f4b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:00.395177 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fcee0680-fe4a-48bc-a7dd-dbcc2fb74f4b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.175290 Sep 27 10:28:00.395704 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2669/5790] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:00 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:01.419864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2f68528a-e3f5-42ad-b786-daec4cf00ce1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:01.420816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2f68528a-e3f5-42ad-b786-daec4cf00ce1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:01.434819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f68528a-e3f5-42ad-b786-daec4cf00ce1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:01.435486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f68528a-e3f5-42ad-b786-daec4cf00ce1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:01.435942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f68528a-e3f5-42ad-b786-daec4cf00ce1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:01.436692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f68528a-e3f5-42ad-b786-daec4cf00ce1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:01.437188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f68528a-e3f5-42ad-b786-daec4cf00ce1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:01.437664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2f68528a-e3f5-42ad-b786-daec4cf00ce1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:01.587282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2f68528a-e3f5-42ad-b786-daec4cf00ce1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:01.596238 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2f68528a-e3f5-42ad-b786-daec4cf00ce1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.180674 Sep 27 10:28:01.597078 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3122/5791] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:01 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:02.621402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-feca6cae-43c4-4e08-b5c4-740b8d5ea2b8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:02.622259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-feca6cae-43c4-4e08-b5c4-740b8d5ea2b8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:02.630514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feca6cae-43c4-4e08-b5c4-740b8d5ea2b8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:02.631322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feca6cae-43c4-4e08-b5c4-740b8d5ea2b8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:02.631922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feca6cae-43c4-4e08-b5c4-740b8d5ea2b8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:02.632710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feca6cae-43c4-4e08-b5c4-740b8d5ea2b8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:02.633252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feca6cae-43c4-4e08-b5c4-740b8d5ea2b8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:02.633726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-feca6cae-43c4-4e08-b5c4-740b8d5ea2b8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:02.857022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-feca6cae-43c4-4e08-b5c4-740b8d5ea2b8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:02.865805 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-feca6cae-43c4-4e08-b5c4-740b8d5ea2b8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.250136 Sep 27 10:28:02.866934 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2670/5792] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:02 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:03.891252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0be9f831-b65c-4246-afbe-268d114536b1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:03.891977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0be9f831-b65c-4246-afbe-268d114536b1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:03.903089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be9f831-b65c-4246-afbe-268d114536b1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:03.903089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be9f831-b65c-4246-afbe-268d114536b1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:03.903089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be9f831-b65c-4246-afbe-268d114536b1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:03.903089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be9f831-b65c-4246-afbe-268d114536b1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:03.903089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be9f831-b65c-4246-afbe-268d114536b1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:03.903089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0be9f831-b65c-4246-afbe-268d114536b1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:04.050356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0be9f831-b65c-4246-afbe-268d114536b1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:04.058381 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0be9f831-b65c-4246-afbe-268d114536b1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.172569 Sep 27 10:28:04.068828 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3123/5793] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:03 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:04.404358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1905ca81-4d53-4d02-9843-92753865d9fd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'create', calling method: .version_select at 0x7f7f62438820>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-1304200436", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFqVLmZpARChVER23BHZKo0D+u2NX+UxR1p7+n+haG8ofUq2htehLcEwuRx2EXUBf9B+Lsqd0zW2hFnav7sf3zqjSrDQP0Ieo+G/jwC3w0awohq2DXVj1LZ/bsc7il1i4w=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:04.420310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-1905ca81-4d53-4d02-9843-92753865d9fd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for project 86e71212d96f46a78680afaacd8709ca. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:04.426338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-1905ca81-4d53-4d02-9843-92753865d9fd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for user 0bd8a19d0ab44011b5430f26757a0efe and project 86e71212d96f46a78680afaacd8709ca. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:04.453732 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1905ca81-4d53-4d02-9843-92753865d9fd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.233011 Sep 27 10:28:04.454315 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2671/5794] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:28:04 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:05.082567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ac9a9826-1ce9-4409-a0ef-b84173eb6fca tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:05.083300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ac9a9826-1ce9-4409-a0ef-b84173eb6fca tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:05.091077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac9a9826-1ce9-4409-a0ef-b84173eb6fca tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:05.091655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac9a9826-1ce9-4409-a0ef-b84173eb6fca tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:05.092205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac9a9826-1ce9-4409-a0ef-b84173eb6fca tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:05.093067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac9a9826-1ce9-4409-a0ef-b84173eb6fca tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:05.093812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac9a9826-1ce9-4409-a0ef-b84173eb6fca tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:05.094324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac9a9826-1ce9-4409-a0ef-b84173eb6fca tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:05.249060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ac9a9826-1ce9-4409-a0ef-b84173eb6fca tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:05.257616 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ac9a9826-1ce9-4409-a0ef-b84173eb6fca tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.179199 Sep 27 10:28:05.258211 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3124/5795] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:05 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:06.282976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f2aebb12-3c8e-4ee7-a568-b5178ccdf7a7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:06.283762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f2aebb12-3c8e-4ee7-a568-b5178ccdf7a7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:06.290230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2aebb12-3c8e-4ee7-a568-b5178ccdf7a7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:06.290806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2aebb12-3c8e-4ee7-a568-b5178ccdf7a7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:06.291273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2aebb12-3c8e-4ee7-a568-b5178ccdf7a7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:06.291996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2aebb12-3c8e-4ee7-a568-b5178ccdf7a7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:06.292475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2aebb12-3c8e-4ee7-a568-b5178ccdf7a7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:06.292941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f2aebb12-3c8e-4ee7-a568-b5178ccdf7a7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:06.526956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f2aebb12-3c8e-4ee7-a568-b5178ccdf7a7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:06.535758 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f2aebb12-3c8e-4ee7-a568-b5178ccdf7a7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.257804 Sep 27 10:28:06.536411 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2672/5796] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:06 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:07.560396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75f05956-f435-4d75-8f99-21d75e97decd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:07.561139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-75f05956-f435-4d75-8f99-21d75e97decd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:07.568465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f05956-f435-4d75-8f99-21d75e97decd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:07.569025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f05956-f435-4d75-8f99-21d75e97decd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:07.569480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f05956-f435-4d75-8f99-21d75e97decd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:07.570201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f05956-f435-4d75-8f99-21d75e97decd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:07.570669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f05956-f435-4d75-8f99-21d75e97decd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:07.571105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75f05956-f435-4d75-8f99-21d75e97decd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:07.718373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-75f05956-f435-4d75-8f99-21d75e97decd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:07.727503 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-75f05956-f435-4d75-8f99-21d75e97decd tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1679 microversion: 2.1 time: 0.171123 Sep 27 10:28:07.739261 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3125/5797] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:07 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1679 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:08.756082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1087af6d-584c-4c91-8d73-4e1d749b8ae4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:08.760336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1087af6d-584c-4c91-8d73-4e1d749b8ae4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:08.769249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1087af6d-584c-4c91-8d73-4e1d749b8ae4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:08.770213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1087af6d-584c-4c91-8d73-4e1d749b8ae4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:08.771620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1087af6d-584c-4c91-8d73-4e1d749b8ae4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:08.773126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1087af6d-584c-4c91-8d73-4e1d749b8ae4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:08.774822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1087af6d-584c-4c91-8d73-4e1d749b8ae4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:08.775505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1087af6d-584c-4c91-8d73-4e1d749b8ae4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:09.060974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1087af6d-584c-4c91-8d73-4e1d749b8ae4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:09.074690 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1087af6d-584c-4c91-8d73-4e1d749b8ae4 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1656 microversion: 2.1 time: 0.324106 Sep 27 10:28:09.075633 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2673/5798] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:08 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1656 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:09.099292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3d53b5b2-9c8e-4721-a656-062925aa4166 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:09.102757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=53,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:09.111425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d53b5b2-9c8e-4721-a656-062925aa4166 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:09.111915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d53b5b2-9c8e-4721-a656-062925aa4166 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:09.117222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3d53b5b2-9c8e-4721-a656-062925aa4166 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:09.148884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3d53b5b2-9c8e-4721-a656-062925aa4166 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Going to try to start instance {{(pid=145968) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 27 10:28:09.213802 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3d53b5b2-9c8e-4721-a656-062925aa4166 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.118478 Sep 27 10:28:09.214422 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3126/5799] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:28:09 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 120 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:28:09.225048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8950b3cf-64e2-4a35-bcc5-e651379984a1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:09.225694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8950b3cf-64e2-4a35-bcc5-e651379984a1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:09.233315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8950b3cf-64e2-4a35-bcc5-e651379984a1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:09.234230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8950b3cf-64e2-4a35-bcc5-e651379984a1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:09.234629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8950b3cf-64e2-4a35-bcc5-e651379984a1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:09.235485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8950b3cf-64e2-4a35-bcc5-e651379984a1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:09.235974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8950b3cf-64e2-4a35-bcc5-e651379984a1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:09.236551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8950b3cf-64e2-4a35-bcc5-e651379984a1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:09.701365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8950b3cf-64e2-4a35-bcc5-e651379984a1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:09.718726 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8950b3cf-64e2-4a35-bcc5-e651379984a1 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1665 microversion: 2.1 time: 0.498294 Sep 27 10:28:09.721901 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2674/5800] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:09 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1665 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:10.743960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7644c769-6fd8-4cd2-8ce3-e8714aa15188 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:10.744597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7644c769-6fd8-4cd2-8ce3-e8714aa15188 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:10.763426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7644c769-6fd8-4cd2-8ce3-e8714aa15188 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:10.765646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7644c769-6fd8-4cd2-8ce3-e8714aa15188 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:10.766259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7644c769-6fd8-4cd2-8ce3-e8714aa15188 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:10.767042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7644c769-6fd8-4cd2-8ce3-e8714aa15188 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:10.767523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7644c769-6fd8-4cd2-8ce3-e8714aa15188 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:10.768036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7644c769-6fd8-4cd2-8ce3-e8714aa15188 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:10.995525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7644c769-6fd8-4cd2-8ce3-e8714aa15188 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:11.005557 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7644c769-6fd8-4cd2-8ce3-e8714aa15188 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.266381 Sep 27 10:28:11.029223 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3127/5801] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:10 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:11.040864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2aaccb56-6d97-4632-aca3-1c1208236b51 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:11.041497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2aaccb56-6d97-4632-aca3-1c1208236b51 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:11.053788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2aaccb56-6d97-4632-aca3-1c1208236b51 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:11.054174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2aaccb56-6d97-4632-aca3-1c1208236b51 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:11.054625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2aaccb56-6d97-4632-aca3-1c1208236b51 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:11.055349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2aaccb56-6d97-4632-aca3-1c1208236b51 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:11.055816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2aaccb56-6d97-4632-aca3-1c1208236b51 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:11.056251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2aaccb56-6d97-4632-aca3-1c1208236b51 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:11.368491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2aaccb56-6d97-4632-aca3-1c1208236b51 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:11.381086 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2aaccb56-6d97-4632-aca3-1c1208236b51 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.344473 Sep 27 10:28:11.387177 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2675/5802] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:11 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:11.410686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d286e465-7297-499b-ad5b-2992a1179742 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:11.411459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d286e465-7297-499b-ad5b-2992a1179742 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:11.420050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d286e465-7297-499b-ad5b-2992a1179742 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:11.420770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d286e465-7297-499b-ad5b-2992a1179742 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:11.422861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d286e465-7297-499b-ad5b-2992a1179742 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:11.423709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d286e465-7297-499b-ad5b-2992a1179742 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:11.424270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d286e465-7297-499b-ad5b-2992a1179742 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:11.424851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d286e465-7297-499b-ad5b-2992a1179742 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:11.442586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:11.446905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1880558214", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "96e94855-5dbb-41a3-90e2-f0691bcc1822"}], "key_name": "tempest-TestNetworkAdvancedServerOps-1304200436", "security_groups": [{"name": "tempest-secgroup-smoke-461819255"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:11.577735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] validate_networks() for [('96e94855-5dbb-41a3-90e2-f0691bcc1822', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:28:11.595114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d286e465-7297-499b-ad5b-2992a1179742 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:11.605924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d286e465-7297-499b-ad5b-2992a1179742 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.201147 Sep 27 10:28:11.615739 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3128/5803] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:11 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:11.637667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-353cbacc-acd3-4415-8844-08ec97da12a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:11.639117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-353cbacc-acd3-4415-8844-08ec97da12a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:11.645721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353cbacc-acd3-4415-8844-08ec97da12a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:11.646425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353cbacc-acd3-4415-8844-08ec97da12a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:11.647215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353cbacc-acd3-4415-8844-08ec97da12a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:11.648181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353cbacc-acd3-4415-8844-08ec97da12a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:11.648899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353cbacc-acd3-4415-8844-08ec97da12a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:11.649646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-353cbacc-acd3-4415-8844-08ec97da12a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:11.797189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-353cbacc-acd3-4415-8844-08ec97da12a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:11.805706 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-353cbacc-acd3-4415-8844-08ec97da12a3 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.172055 Sep 27 10:28:11.806415 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3129/5804] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:11 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:11.828381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-11600524-cbd3-4695-bc07-a8f12b892bcb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:11.829473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-11600524-cbd3-4695-bc07-a8f12b892bcb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:11.837991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11600524-cbd3-4695-bc07-a8f12b892bcb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:11.838991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11600524-cbd3-4695-bc07-a8f12b892bcb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:11.839474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11600524-cbd3-4695-bc07-a8f12b892bcb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:11.840529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11600524-cbd3-4695-bc07-a8f12b892bcb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:11.841237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11600524-cbd3-4695-bc07-a8f12b892bcb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:11.841730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11600524-cbd3-4695-bc07-a8f12b892bcb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:11.987769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-11600524-cbd3-4695-bc07-a8f12b892bcb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:11.998931 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-11600524-cbd3-4695-bc07-a8f12b892bcb tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.173968 Sep 27 10:28:12.000214 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3130/5805] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:11 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:12.021812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b090acd-efae-49a3-b9cb-c8e4b6fa198b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:12.022607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3b090acd-efae-49a3-b9cb-c8e4b6fa198b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:12.024469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:28:12.024950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:28:12.026177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:28:12.026636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:28:12.027067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:28:12.028763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b090acd-efae-49a3-b9cb-c8e4b6fa198b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:12.029516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b090acd-efae-49a3-b9cb-c8e4b6fa198b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:12.029974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b090acd-efae-49a3-b9cb-c8e4b6fa198b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:12.030868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b090acd-efae-49a3-b9cb-c8e4b6fa198b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:12.031504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b090acd-efae-49a3-b9cb-c8e4b6fa198b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:12.031928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b090acd-efae-49a3-b9cb-c8e4b6fa198b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:12.036826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:12.037339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:12.037790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:12.038545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:12.039014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:12.039454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:12.058321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-3b090acd-efae-49a3-b9cb-c8e4b6fa198b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lazy-loading 'flavor' on Instance uuid 5ac6dabf-1315-4e05-ab6c-22211dc2440b {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:28:12.058895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for project 86e71212d96f46a78680afaacd8709ca. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:12.064406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for user 0bd8a19d0ab44011b5430f26757a0efe and project 86e71212d96f46a78680afaacd8709ca. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:12.077601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:28:12.080247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:28:12.133406 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2bfb30bb-3fdf-499d-9bc5-7f1edefed2ae tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.690775 Sep 27 10:28:12.134104 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3b090acd-efae-49a3-b9cb-c8e4b6fa198b tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.115761 Sep 27 10:28:12.134844 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2676/5806] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:28:11 2022] POST /compute/v2.1/servers => generated 405 bytes in 693 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:28:12.141534 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3131/5807] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:28:12 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 124 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:28:12.151647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-10f9e7fb-4741-4cdc-a633-322f9f275e3b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:12.159288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d7c22139-6b0c-464c-b8d2-32c033936154 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:12.160044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d7c22139-6b0c-464c-b8d2-32c033936154 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:12.164729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-10f9e7fb-4741-4cdc-a633-322f9f275e3b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:12.168888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7c22139-6b0c-464c-b8d2-32c033936154 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:12.169484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7c22139-6b0c-464c-b8d2-32c033936154 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:12.170018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7c22139-6b0c-464c-b8d2-32c033936154 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:12.170896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7c22139-6b0c-464c-b8d2-32c033936154 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:12.171454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7c22139-6b0c-464c-b8d2-32c033936154 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:12.171981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7c22139-6b0c-464c-b8d2-32c033936154 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:12.228913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-10f9e7fb-4741-4cdc-a633-322f9f275e3b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:12.238529 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-10f9e7fb-4741-4cdc-a633-322f9f275e3b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1267 microversion: 2.1 time: 0.090719 Sep 27 10:28:12.252680 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2677/5808] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:12 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1267 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:12.359594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d7c22139-6b0c-464c-b8d2-32c033936154 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:12.367803 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d7c22139-6b0c-464c-b8d2-32c033936154 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1677 microversion: 2.1 time: 0.221276 Sep 27 10:28:12.368372 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3132/5809] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:12 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1677 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:13.262788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7a795227-4e12-4f63-aff8-c9fea8a148e0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:13.263504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7a795227-4e12-4f63-aff8-c9fea8a148e0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:13.272175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a795227-4e12-4f63-aff8-c9fea8a148e0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:13.272785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a795227-4e12-4f63-aff8-c9fea8a148e0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:13.273228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a795227-4e12-4f63-aff8-c9fea8a148e0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:13.273953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a795227-4e12-4f63-aff8-c9fea8a148e0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:13.274402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a795227-4e12-4f63-aff8-c9fea8a148e0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:13.274834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a795227-4e12-4f63-aff8-c9fea8a148e0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:13.367795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a795227-4e12-4f63-aff8-c9fea8a148e0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:13.382768 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7a795227-4e12-4f63-aff8-c9fea8a148e0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1319 microversion: 2.1 time: 0.124124 Sep 27 10:28:13.383245 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2678/5810] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:13 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1319 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:13.398795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7ea17b37-e3ad-414c-b5b9-5532aa1ede46 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:13.399419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7ea17b37-e3ad-414c-b5b9-5532aa1ede46 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:13.406966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea17b37-e3ad-414c-b5b9-5532aa1ede46 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:13.407464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea17b37-e3ad-414c-b5b9-5532aa1ede46 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:13.407956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea17b37-e3ad-414c-b5b9-5532aa1ede46 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:13.408680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea17b37-e3ad-414c-b5b9-5532aa1ede46 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:13.409203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea17b37-e3ad-414c-b5b9-5532aa1ede46 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:13.409629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ea17b37-e3ad-414c-b5b9-5532aa1ede46 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:13.634407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7ea17b37-e3ad-414c-b5b9-5532aa1ede46 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:13.643288 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7ea17b37-e3ad-414c-b5b9-5532aa1ede46 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1660 microversion: 2.1 time: 0.251516 Sep 27 10:28:13.657200 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3133/5811] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:13 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1660 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:13.667923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-722ad442-a410-4d69-aa72-8f0f8ee1dc4f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:13.668906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-722ad442-a410-4d69-aa72-8f0f8ee1dc4f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:13.677078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-722ad442-a410-4d69-aa72-8f0f8ee1dc4f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:13.677839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-722ad442-a410-4d69-aa72-8f0f8ee1dc4f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:13.678379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-722ad442-a410-4d69-aa72-8f0f8ee1dc4f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:13.679330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-722ad442-a410-4d69-aa72-8f0f8ee1dc4f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:13.679790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-722ad442-a410-4d69-aa72-8f0f8ee1dc4f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:13.680380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-722ad442-a410-4d69-aa72-8f0f8ee1dc4f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:13.771870 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-722ad442-a410-4d69-aa72-8f0f8ee1dc4f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action" status: 202 len: 0 microversion: 2.1 time: 0.108138 Sep 27 10:28:13.775957 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2679/5812] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:28:13 2022] POST /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b/action => generated 0 bytes in 110 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:28:13.782283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4e5e6cca-1fbd-4434-b704-0d37d10d1e37 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:13.782894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4e5e6cca-1fbd-4434-b704-0d37d10d1e37 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:13.789386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e5e6cca-1fbd-4434-b704-0d37d10d1e37 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:13.790040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e5e6cca-1fbd-4434-b704-0d37d10d1e37 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:13.790566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e5e6cca-1fbd-4434-b704-0d37d10d1e37 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:13.791337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e5e6cca-1fbd-4434-b704-0d37d10d1e37 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:13.791804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e5e6cca-1fbd-4434-b704-0d37d10d1e37 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:13.792239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e5e6cca-1fbd-4434-b704-0d37d10d1e37 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:13.962634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4e5e6cca-1fbd-4434-b704-0d37d10d1e37 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:13.989170 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4e5e6cca-1fbd-4434-b704-0d37d10d1e37 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1666 microversion: 2.1 time: 0.205678 Sep 27 10:28:14.001677 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3134/5813] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:13 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1666 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:14.410037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9ff50171-8ec8-4a4d-96d2-f0b20482ccea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:14.412340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9ff50171-8ec8-4a4d-96d2-f0b20482ccea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:14.422894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff50171-8ec8-4a4d-96d2-f0b20482ccea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:14.423830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff50171-8ec8-4a4d-96d2-f0b20482ccea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:14.424786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff50171-8ec8-4a4d-96d2-f0b20482ccea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:14.426017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff50171-8ec8-4a4d-96d2-f0b20482ccea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:14.426733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff50171-8ec8-4a4d-96d2-f0b20482ccea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:14.427398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ff50171-8ec8-4a4d-96d2-f0b20482ccea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:14.432777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:14.686301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9ff50171-8ec8-4a4d-96d2-f0b20482ccea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:14.695994 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9ff50171-8ec8-4a4d-96d2-f0b20482ccea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1392 microversion: 2.1 time: 0.292668 Sep 27 10:28:14.696956 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2680/5814] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:14 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1392 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:15.014145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-89c094c9-2501-40f8-9143-17bf9208ec31 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:15.016108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-89c094c9-2501-40f8-9143-17bf9208ec31 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:15.023717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89c094c9-2501-40f8-9143-17bf9208ec31 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:15.024193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89c094c9-2501-40f8-9143-17bf9208ec31 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:15.024844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89c094c9-2501-40f8-9143-17bf9208ec31 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:15.025697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89c094c9-2501-40f8-9143-17bf9208ec31 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:15.026172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89c094c9-2501-40f8-9143-17bf9208ec31 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:15.026835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89c094c9-2501-40f8-9143-17bf9208ec31 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:15.172592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-89c094c9-2501-40f8-9143-17bf9208ec31 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:15.184156 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-89c094c9-2501-40f8-9143-17bf9208ec31 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1666 microversion: 2.1 time: 0.174539 Sep 27 10:28:15.186991 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3135/5815] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:15 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1666 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:15.312500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-116bcd21-0bc6-42a3-9569-34d98c9404f2 req-481d813c-daa0-48db-921f-fc9c9ad58064 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "39429342-b89d-443d-9645-17ec8e2869f3", "tag": "b41f6e87-e1ce-49b6-a47a-5129ca582abc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:15.324516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-116bcd21-0bc6-42a3-9569-34d98c9404f2 req-481d813c-daa0-48db-921f-fc9c9ad58064 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:15.325304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-116bcd21-0bc6-42a3-9569-34d98c9404f2 req-481d813c-daa0-48db-921f-fc9c9ad58064 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:15.325994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-116bcd21-0bc6-42a3-9569-34d98c9404f2 req-481d813c-daa0-48db-921f-fc9c9ad58064 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:15.343966 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-116bcd21-0bc6-42a3-9569-34d98c9404f2 req-481d813c-daa0-48db-921f-fc9c9ad58064 service nova] Creating event network-changed:b41f6e87-e1ce-49b6-a47a-5129ca582abc for instance 39429342-b89d-443d-9645-17ec8e2869f3 on nch2-master-1 Sep 27 10:28:15.349309 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-116bcd21-0bc6-42a3-9569-34d98c9404f2 req-481d813c-daa0-48db-921f-fc9c9ad58064 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042064 Sep 27 10:28:15.350445 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2681/5816] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:15.720334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-edb7812a-039f-47bf-aa02-af6f82fb330c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:15.721024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-edb7812a-039f-47bf-aa02-af6f82fb330c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:15.727366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edb7812a-039f-47bf-aa02-af6f82fb330c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:15.727859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edb7812a-039f-47bf-aa02-af6f82fb330c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:15.728308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edb7812a-039f-47bf-aa02-af6f82fb330c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:15.729067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edb7812a-039f-47bf-aa02-af6f82fb330c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:15.729530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edb7812a-039f-47bf-aa02-af6f82fb330c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:15.729962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-edb7812a-039f-47bf-aa02-af6f82fb330c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:15.872932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-edb7812a-039f-47bf-aa02-af6f82fb330c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:15.881015 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-edb7812a-039f-47bf-aa02-af6f82fb330c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1392 microversion: 2.1 time: 0.165026 Sep 27 10:28:15.881608 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3136/5817] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:15 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1392 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:16.216548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3bbf4652-30e6-473d-9a9d-d6720ffe3a17 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:16.217742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3bbf4652-30e6-473d-9a9d-d6720ffe3a17 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:16.224630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bbf4652-30e6-473d-9a9d-d6720ffe3a17 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:16.225192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bbf4652-30e6-473d-9a9d-d6720ffe3a17 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:16.225643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bbf4652-30e6-473d-9a9d-d6720ffe3a17 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:16.226358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bbf4652-30e6-473d-9a9d-d6720ffe3a17 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:16.226830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bbf4652-30e6-473d-9a9d-d6720ffe3a17 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:16.227294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3bbf4652-30e6-473d-9a9d-d6720ffe3a17 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:16.364101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3bbf4652-30e6-473d-9a9d-d6720ffe3a17 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:16.376640 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3bbf4652-30e6-473d-9a9d-d6720ffe3a17 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.168876 Sep 27 10:28:16.377621 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2682/5818] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:16 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:16.403367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-43e41974-40c5-476f-b68c-39660874437d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:16.403998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-43e41974-40c5-476f-b68c-39660874437d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:16.411382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43e41974-40c5-476f-b68c-39660874437d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:16.411873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43e41974-40c5-476f-b68c-39660874437d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:16.412319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43e41974-40c5-476f-b68c-39660874437d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:16.413072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43e41974-40c5-476f-b68c-39660874437d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:16.413537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43e41974-40c5-476f-b68c-39660874437d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:16.413974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-43e41974-40c5-476f-b68c-39660874437d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:16.556208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-43e41974-40c5-476f-b68c-39660874437d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:16.565485 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-43e41974-40c5-476f-b68c-39660874437d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1669 microversion: 2.1 time: 0.166196 Sep 27 10:28:16.566073 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3137/5819] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:16 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1669 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:16.593427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:16.593947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:16.600694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:16.601216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:16.601677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:16.602407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:16.602886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:16.603331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:16.630993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:28:16.637584 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.compute.api [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Working on deleting snapshot 4ea72bfa-9899-46b6-a2b4-0a7b55c1b7a5 from shelved instance... Sep 27 10:28:16.904758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f9402796-a533-47fa-8ca6-43fd81c8d191 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:16.905474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f9402796-a533-47fa-8ca6-43fd81c8d191 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:16.912449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9402796-a533-47fa-8ca6-43fd81c8d191 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:16.913289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9402796-a533-47fa-8ca6-43fd81c8d191 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:16.913707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9402796-a533-47fa-8ca6-43fd81c8d191 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:16.914466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9402796-a533-47fa-8ca6-43fd81c8d191 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:16.914954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9402796-a533-47fa-8ca6-43fd81c8d191 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:16.916388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9402796-a533-47fa-8ca6-43fd81c8d191 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:17.079856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f9402796-a533-47fa-8ca6-43fd81c8d191 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:17.091505 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f9402796-a533-47fa-8ca6-43fd81c8d191 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1392 microversion: 2.1 time: 0.191670 Sep 27 10:28:17.092105 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3138/5820] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:16 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1392 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:17.269159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Doing local delete in cell ed1088da-caae-4426-8048-9cba90e7755c(cell1) {{(pid=145969) _delete /opt/stack/nova/nova/compute/api.py:2546}} Sep 27 10:28:17.269803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:17.270294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:17.270738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:17.271171 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.compute.api [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] instance is in SHELVED_OFFLOADED state, cleanup the instance's info from database. Sep 27 10:28:17.274930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] deallocate_for_instance() {{(pid=145969) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1780}} Sep 27 10:28:17.776212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-4190b822-6b0e-444f-b22e-3bffc55435a9 req-0e2cde58-ca4e-4f49-bd7c-dd5c9074caa2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "966d80e8-453c-467c-a8da-844a6ef094f5", "name": "network-vif-deleted", "tag": "325e5b71-36a4-4466-bece-550e0138fb06"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:17.785526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Updating instance_info_cache with network_info: [] {{(pid=145969) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}} Sep 27 10:28:17.788577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4190b822-6b0e-444f-b22e-3bffc55435a9 req-0e2cde58-ca4e-4f49-bd7c-dd5c9074caa2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:17.789499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4190b822-6b0e-444f-b22e-3bffc55435a9 req-0e2cde58-ca4e-4f49-bd7c-dd5c9074caa2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:17.789937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4190b822-6b0e-444f-b22e-3bffc55435a9 req-0e2cde58-ca4e-4f49-bd7c-dd5c9074caa2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:17.809203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.compute.server_external_events [req-4190b822-6b0e-444f-b22e-3bffc55435a9 req-0e2cde58-ca4e-4f49-bd7c-dd5c9074caa2 service nova] Unable to find a host for instance 966d80e8-453c-467c-a8da-844a6ef094f5. Dropping event network-vif-deleted {{(pid=145968) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 27 10:28:17.810151 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-4190b822-6b0e-444f-b22e-3bffc55435a9 req-0e2cde58-ca4e-4f49-bd7c-dd5c9074caa2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 207 len: 180 microversion: 2.1 time: 0.037852 Sep 27 10:28:17.811328 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3139/5821] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:17 2022] POST /compute/v2.1/os-server-external-events => generated 180 bytes in 39 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Sep 27 10:28:18.117825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eefb4c5b-f1c9-42cc-b62a-99de29357255 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:18.119249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eefb4c5b-f1c9-42cc-b62a-99de29357255 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:18.123195 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e8501200-1e74-41b7-9683-54bbc2e21983 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "DELETE /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 204 len: 0 microversion: 2.1 time: 1.534059 Sep 27 10:28:18.125120 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2683/5822] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:28:16 2022] DELETE /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 0 bytes in 1537 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:28:18.131295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefb4c5b-f1c9-42cc-b62a-99de29357255 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:18.132803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefb4c5b-f1c9-42cc-b62a-99de29357255 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:18.133458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefb4c5b-f1c9-42cc-b62a-99de29357255 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:18.134818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:18.135245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 966d80e8-453c-467c-a8da-844a6ef094f5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:18.135458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefb4c5b-f1c9-42cc-b62a-99de29357255 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:18.136124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefb4c5b-f1c9-42cc-b62a-99de29357255 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:18.136822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eefb4c5b-f1c9-42cc-b62a-99de29357255 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:18.143060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:18.143791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:18.144451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:18.145468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:18.145911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:18.146721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:18.162515 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] HTTP exception thrown: Instance 966d80e8-453c-467c-a8da-844a6ef094f5 could not be found. Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Request method failure captured: Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 HTTP/1.1 Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs6KVh3H5X7rX3evxN11ucsVlmHnbL2MCObHKl1pQO1q40XS_7dvNdOvYhoxemUBx1Ms1xpNth_S22Rp1rV9hfITX6TEFNP6CCnYhe72LSUTyMJphfTwYhmOPX9p8o6YFOanCD6UQJX6ltkdsgee7CkTpD8vTeyh3fvEfCOyNJIfUWA Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_10375bd836f84383b39f2df73f53800e"}]}] Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 97c769dbba794531b0a9db8e764ff097 Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 966d80e8-453c-467c-a8da-844a6ef094f5 could not be found. Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '966d80e8-453c-467c-a8da-844a6ef094f5'} Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 966d80e8-453c-467c-a8da-844a6ef094f5 could not be found. Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 966d80e8-453c-467c-a8da-844a6ef094f5 could not be found. Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:18.164465 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:18.172392 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:18.172392 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:18.172392 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:18.172392 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:28:18.172392 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:28:18.172392 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 966d80e8-453c-467c-a8da-844a6ef094f5 could not be found. Sep 27 10:28:18.172392 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:18.172392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Returning 404 to user: Instance 966d80e8-453c-467c-a8da-844a6ef094f5 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:28:18.172392 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-616ce720-06c4-4481-ae7b-33caceeac5a0 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5" status: 404 len: 111 microversion: 2.1 time: 0.036023 Sep 27 10:28:18.172392 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2684/5823] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:18 2022] GET /compute/v2.1/servers/966d80e8-453c-467c-a8da-844a6ef094f5 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:28:18.180226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1b8dc1de-4a6b-44e0-aa99-9488322d9ea8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:18.180763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1b8dc1de-4a6b-44e0-aa99-9488322d9ea8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:18.188151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b8dc1de-4a6b-44e0-aa99-9488322d9ea8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:18.188651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b8dc1de-4a6b-44e0-aa99-9488322d9ea8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:18.189156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b8dc1de-4a6b-44e0-aa99-9488322d9ea8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:18.189886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b8dc1de-4a6b-44e0-aa99-9488322d9ea8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:18.190358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b8dc1de-4a6b-44e0-aa99-9488322d9ea8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:18.190799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b8dc1de-4a6b-44e0-aa99-9488322d9ea8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:18.222176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1b8dc1de-4a6b-44e0-aa99-9488322d9ea8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:28:18.291252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-eefb4c5b-f1c9-42cc-b62a-99de29357255 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:18.301596 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eefb4c5b-f1c9-42cc-b62a-99de29357255 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1392 microversion: 2.1 time: 0.189258 Sep 27 10:28:18.302778 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3140/5824] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:18 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1392 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:18.311953 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1b8dc1de-4a6b-44e0-aa99-9488322d9ea8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "DELETE /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 204 len: 0 microversion: 2.1 time: 0.135081 Sep 27 10:28:18.312983 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2685/5825] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:28:18 2022] DELETE /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:28:18.324851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ecfa6a37-c8a1-4bbc-9b61-5d89a7dff422 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:18.325487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ecfa6a37-c8a1-4bbc-9b61-5d89a7dff422 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:18.331869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecfa6a37-c8a1-4bbc-9b61-5d89a7dff422 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:18.332360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecfa6a37-c8a1-4bbc-9b61-5d89a7dff422 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:18.332834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecfa6a37-c8a1-4bbc-9b61-5d89a7dff422 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:18.333547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecfa6a37-c8a1-4bbc-9b61-5d89a7dff422 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:18.334018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecfa6a37-c8a1-4bbc-9b61-5d89a7dff422 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:18.334449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecfa6a37-c8a1-4bbc-9b61-5d89a7dff422 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:18.419238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ecfa6a37-c8a1-4bbc-9b61-5d89a7dff422 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:18.427812 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ecfa6a37-c8a1-4bbc-9b61-5d89a7dff422 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1466 microversion: 2.1 time: 0.109480 Sep 27 10:28:18.428393 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3141/5826] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:18 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1466 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:19.326900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cbf95dfc-e99a-4ade-a04f-59966eda5873 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:19.327588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cbf95dfc-e99a-4ade-a04f-59966eda5873 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:19.334805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbf95dfc-e99a-4ade-a04f-59966eda5873 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:19.335311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbf95dfc-e99a-4ade-a04f-59966eda5873 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:19.335765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbf95dfc-e99a-4ade-a04f-59966eda5873 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:19.336488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbf95dfc-e99a-4ade-a04f-59966eda5873 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:19.336986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbf95dfc-e99a-4ade-a04f-59966eda5873 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:19.337428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbf95dfc-e99a-4ade-a04f-59966eda5873 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:19.450486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aa132cd3-a3ee-4171-886a-27c68609a458 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:19.450872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aa132cd3-a3ee-4171-886a-27c68609a458 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:19.460364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa132cd3-a3ee-4171-886a-27c68609a458 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:19.460894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa132cd3-a3ee-4171-886a-27c68609a458 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:19.461339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa132cd3-a3ee-4171-886a-27c68609a458 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:19.462059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa132cd3-a3ee-4171-886a-27c68609a458 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:19.462521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa132cd3-a3ee-4171-886a-27c68609a458 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:19.462950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa132cd3-a3ee-4171-886a-27c68609a458 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:19.551146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cbf95dfc-e99a-4ade-a04f-59966eda5873 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:19.553530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aa132cd3-a3ee-4171-886a-27c68609a458 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:19.559741 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cbf95dfc-e99a-4ade-a04f-59966eda5873 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1392 microversion: 2.1 time: 0.237196 Sep 27 10:28:19.560889 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2686/5827] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:19 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1392 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:19.562909 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aa132cd3-a3ee-4171-886a-27c68609a458 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1466 microversion: 2.1 time: 0.116065 Sep 27 10:28:19.563492 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3142/5828] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:19 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1466 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:20.461779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-07efb2ed-201c-43a4-9295-391dd00ab73f req-732b449d-35e4-485b-bda8-05443df378fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "af5aa123-db45-4556-bd67-47687f84ba0f", "name": "network-vif-deleted", "tag": "9482af94-f7d5-4e78-b8bd-2be5c1dcae94"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:20.474849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-07efb2ed-201c-43a4-9295-391dd00ab73f req-732b449d-35e4-485b-bda8-05443df378fd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:20.475639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-07efb2ed-201c-43a4-9295-391dd00ab73f req-732b449d-35e4-485b-bda8-05443df378fd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:20.476342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-07efb2ed-201c-43a4-9295-391dd00ab73f req-732b449d-35e4-485b-bda8-05443df378fd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:20.490567 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-07efb2ed-201c-43a4-9295-391dd00ab73f req-732b449d-35e4-485b-bda8-05443df378fd service nova] Creating event network-vif-deleted:9482af94-f7d5-4e78-b8bd-2be5c1dcae94 for instance af5aa123-db45-4556-bd67-47687f84ba0f on nch2-master-1 Sep 27 10:28:20.497883 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-07efb2ed-201c-43a4-9295-391dd00ab73f req-732b449d-35e4-485b-bda8-05443df378fd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039244 Sep 27 10:28:20.498340 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2687/5829] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:20.586655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9411f01a-7b6c-4d22-b2b0-6c875c5d62c0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:20.587286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9411f01a-7b6c-4d22-b2b0-6c875c5d62c0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:20.587737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:20.588887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:20.595455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9411f01a-7b6c-4d22-b2b0-6c875c5d62c0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:20.595993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9411f01a-7b6c-4d22-b2b0-6c875c5d62c0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:20.596154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:20.596609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:20.596801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9411f01a-7b6c-4d22-b2b0-6c875c5d62c0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:20.597097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:20.597519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9411f01a-7b6c-4d22-b2b0-6c875c5d62c0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:20.597813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:20.597989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9411f01a-7b6c-4d22-b2b0-6c875c5d62c0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:20.598278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:20.598427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9411f01a-7b6c-4d22-b2b0-6c875c5d62c0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:20.598732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:20.640724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:20.641209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:20.641685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:20.642154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lazy-loading 'fault' on Instance uuid af5aa123-db45-4556-bd67-47687f84ba0f {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:28:20.690999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:20.700007 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-16d054b3-ed63-4dff-af9c-bc15f5370311 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 200 len: 1305 microversion: 2.1 time: 0.117659 Sep 27 10:28:20.712709 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2688/5830] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:20 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 1305 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:20.760731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9411f01a-7b6c-4d22-b2b0-6c875c5d62c0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:20.764324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:20.771292 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9411f01a-7b6c-4d22-b2b0-6c875c5d62c0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1392 microversion: 2.1 time: 0.191104 Sep 27 10:28:20.771764 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3143/5831] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:20 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1392 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:21.297678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-cbd070d6-d72d-4c2f-85f0-196a5f4daf71 req-f6d71c91-8c89-4210-b3d6-4f03979fa7cb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39429342-b89d-443d-9645-17ec8e2869f3", "name": "network-vif-plugged", "status": "completed", "tag": "b41f6e87-e1ce-49b6-a47a-5129ca582abc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:21.307781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cbd070d6-d72d-4c2f-85f0-196a5f4daf71 req-f6d71c91-8c89-4210-b3d6-4f03979fa7cb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:21.308281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cbd070d6-d72d-4c2f-85f0-196a5f4daf71 req-f6d71c91-8c89-4210-b3d6-4f03979fa7cb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:21.308754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cbd070d6-d72d-4c2f-85f0-196a5f4daf71 req-f6d71c91-8c89-4210-b3d6-4f03979fa7cb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:21.321002 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-cbd070d6-d72d-4c2f-85f0-196a5f4daf71 req-f6d71c91-8c89-4210-b3d6-4f03979fa7cb service nova] Creating event network-vif-plugged:b41f6e87-e1ce-49b6-a47a-5129ca582abc for instance 39429342-b89d-443d-9645-17ec8e2869f3 on nch2-master-1 Sep 27 10:28:21.325915 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-cbd070d6-d72d-4c2f-85f0-196a5f4daf71 req-f6d71c91-8c89-4210-b3d6-4f03979fa7cb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032394 Sep 27 10:28:21.326515 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2689/5832] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:21.731070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:21.731975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: af5aa123-db45-4556-bd67-47687f84ba0f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:21.742058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:21.742898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:21.743636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:21.745036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:21.745349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:21.745749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:21.764023 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] HTTP exception thrown: Instance af5aa123-db45-4556-bd67-47687f84ba0f could not be found. Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Request method failure captured: Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f HTTP/1.1 Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs6KVh3H5X7rX3evxN11ucsVlmHnbL2MCObHKl1pQO1q40XS_7dvNdOvYhoxemUBx1Ms1xpNth_S22Rp1rV9hfITX6TEFNP6CCnYhe72LSUTyMJphfTwYhmOPX9p8o6YFOanCD6UQJX6ltkdsgee7CkTpD8vTeyh3fvEfCOyNJIfUWA Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_10375bd836f84383b39f2df73f53800e"}]}] Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 97c769dbba794531b0a9db8e764ff097 Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance af5aa123-db45-4556-bd67-47687f84ba0f could not be found. Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'af5aa123-db45-4556-bd67-47687f84ba0f'} Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance af5aa123-db45-4556-bd67-47687f84ba0f could not be found. Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance af5aa123-db45-4556-bd67-47687f84ba0f could not be found. Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:21.765911 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:21.771388 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:21.771388 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:21.771388 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:21.771388 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:28:21.771388 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:28:21.771388 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance af5aa123-db45-4556-bd67-47687f84ba0f could not be found. Sep 27 10:28:21.771388 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:21.771388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Returning 404 to user: Instance af5aa123-db45-4556-bd67-47687f84ba0f could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:28:21.771388 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-051cfdf5-2cd0-4965-aebf-835847d8ad11 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f" status: 404 len: 111 microversion: 2.1 time: 0.043808 Sep 27 10:28:21.771388 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3144/5833] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:21 2022] GET /compute/v2.1/servers/af5aa123-db45-4556-bd67-47687f84ba0f => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:28:21.777684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-29b1afa9-88ce-442e-968c-6c374602140d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:21.778288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-29b1afa9-88ce-442e-968c-6c374602140d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:21.785565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b1afa9-88ce-442e-968c-6c374602140d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:21.786062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b1afa9-88ce-442e-968c-6c374602140d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:21.786512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b1afa9-88ce-442e-968c-6c374602140d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:21.787219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b1afa9-88ce-442e-968c-6c374602140d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:21.787687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b1afa9-88ce-442e-968c-6c374602140d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:21.788223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b1afa9-88ce-442e-968c-6c374602140d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:21.794600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8fed12f4-38b6-4f7c-b4ff-c36280ad0a02 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:21.795218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8fed12f4-38b6-4f7c-b4ff-c36280ad0a02 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:21.801866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fed12f4-38b6-4f7c-b4ff-c36280ad0a02 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:21.802348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fed12f4-38b6-4f7c-b4ff-c36280ad0a02 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:21.802773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fed12f4-38b6-4f7c-b4ff-c36280ad0a02 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:21.803481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fed12f4-38b6-4f7c-b4ff-c36280ad0a02 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:21.803934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fed12f4-38b6-4f7c-b4ff-c36280ad0a02 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:21.804509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8fed12f4-38b6-4f7c-b4ff-c36280ad0a02 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:21.815296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-29b1afa9-88ce-442e-968c-6c374602140d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:28:21.829799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=78,nova_cell1:UPDATE=17,nova_cell1:SAVEPOINT=1,nova_cell1:RELEASE=1,nova_cell1:DELETE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:21.886641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=4,nova_api:UPDATE=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:21.909014 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-29b1afa9-88ce-442e-968c-6c374602140d tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "DELETE /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 204 len: 0 microversion: 2.1 time: 0.135821 Sep 27 10:28:21.913063 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2690/5834] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:28:21 2022] DELETE /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:28:21.921839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f84d9d0f-39c1-40d5-ad2c-894456fb3004 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:21.922753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f84d9d0f-39c1-40d5-ad2c-894456fb3004 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:21.928916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f84d9d0f-39c1-40d5-ad2c-894456fb3004 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:21.929507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f84d9d0f-39c1-40d5-ad2c-894456fb3004 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:21.929856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f84d9d0f-39c1-40d5-ad2c-894456fb3004 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:21.930576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f84d9d0f-39c1-40d5-ad2c-894456fb3004 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:21.931063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f84d9d0f-39c1-40d5-ad2c-894456fb3004 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:21.931497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f84d9d0f-39c1-40d5-ad2c-894456fb3004 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:21.959810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8fed12f4-38b6-4f7c-b4ff-c36280ad0a02 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:21.968772 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8fed12f4-38b6-4f7c-b4ff-c36280ad0a02 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1392 microversion: 2.1 time: 0.177734 Sep 27 10:28:21.969460 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3145/5835] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:21 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1392 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:22.049805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:22.072506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f84d9d0f-39c1-40d5-ad2c-894456fb3004 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:22.080620 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f84d9d0f-39c1-40d5-ad2c-894456fb3004 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1507 microversion: 2.1 time: 0.166310 Sep 27 10:28:22.081223 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2691/5836] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:21 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1507 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:22.213967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5682337-6e6d-4081-b8de-88a81185423b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:22.214478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b5682337-6e6d-4081-b8de-88a81185423b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:22.221087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5682337-6e6d-4081-b8de-88a81185423b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:22.221570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5682337-6e6d-4081-b8de-88a81185423b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:22.222008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5682337-6e6d-4081-b8de-88a81185423b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:22.222710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5682337-6e6d-4081-b8de-88a81185423b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:22.223161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5682337-6e6d-4081-b8de-88a81185423b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:22.223568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b5682337-6e6d-4081-b8de-88a81185423b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:22.249815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b5682337-6e6d-4081-b8de-88a81185423b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:28:22.336253 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b5682337-6e6d-4081-b8de-88a81185423b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 204 len: 0 microversion: 2.1 time: 0.252423 Sep 27 10:28:22.336876 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3146/5837] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:28:22 2022] DELETE /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 0 bytes in 254 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:28:22.347090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6b83d4d8-4705-451e-b0a0-4886e4307100 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:22.347717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6b83d4d8-4705-451e-b0a0-4886e4307100 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:22.349898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=12 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:22.356341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b83d4d8-4705-451e-b0a0-4886e4307100 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:22.356884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b83d4d8-4705-451e-b0a0-4886e4307100 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:22.357339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b83d4d8-4705-451e-b0a0-4886e4307100 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:22.358071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b83d4d8-4705-451e-b0a0-4886e4307100 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:22.358545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b83d4d8-4705-451e-b0a0-4886e4307100 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:22.359044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6b83d4d8-4705-451e-b0a0-4886e4307100 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:22.518532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6b83d4d8-4705-451e-b0a0-4886e4307100 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:22.526555 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6b83d4d8-4705-451e-b0a0-4886e4307100 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1729 microversion: 2.1 time: 0.183443 Sep 27 10:28:22.527140 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2692/5838] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:22 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1729 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:22.992314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-113d33cd-ebe5-466b-8043-81affc16110a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:22.993066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-113d33cd-ebe5-466b-8043-81affc16110a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:23.000048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113d33cd-ebe5-466b-8043-81affc16110a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:23.000681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113d33cd-ebe5-466b-8043-81affc16110a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:23.001266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113d33cd-ebe5-466b-8043-81affc16110a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:23.001970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113d33cd-ebe5-466b-8043-81affc16110a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:23.002428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113d33cd-ebe5-466b-8043-81affc16110a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:23.002852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-113d33cd-ebe5-466b-8043-81affc16110a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:23.105348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d9ddf2db-96e6-4f7b-9b24-1a9f50490923 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:23.105901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d9ddf2db-96e6-4f7b-9b24-1a9f50490923 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:23.112772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ddf2db-96e6-4f7b-9b24-1a9f50490923 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:23.113274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ddf2db-96e6-4f7b-9b24-1a9f50490923 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:23.113726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ddf2db-96e6-4f7b-9b24-1a9f50490923 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:23.114477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ddf2db-96e6-4f7b-9b24-1a9f50490923 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:23.115688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ddf2db-96e6-4f7b-9b24-1a9f50490923 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:23.116144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d9ddf2db-96e6-4f7b-9b24-1a9f50490923 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:23.147935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-113d33cd-ebe5-466b-8043-81affc16110a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:23.156238 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-113d33cd-ebe5-466b-8043-81affc16110a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1563 microversion: 2.1 time: 0.168355 Sep 27 10:28:23.156860 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3147/5839] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:22 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1563 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:23.179888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b3aea22-6fe5-4dc2-b632-c29c95cd361d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:23.180566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3b3aea22-6fe5-4dc2-b632-c29c95cd361d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:23.186959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b3aea22-6fe5-4dc2-b632-c29c95cd361d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:23.187452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b3aea22-6fe5-4dc2-b632-c29c95cd361d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:23.187935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b3aea22-6fe5-4dc2-b632-c29c95cd361d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:23.188825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b3aea22-6fe5-4dc2-b632-c29c95cd361d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:23.189307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b3aea22-6fe5-4dc2-b632-c29c95cd361d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:23.189723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b3aea22-6fe5-4dc2-b632-c29c95cd361d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:23.259812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d9ddf2db-96e6-4f7b-9b24-1a9f50490923 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:23.268493 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d9ddf2db-96e6-4f7b-9b24-1a9f50490923 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1507 microversion: 2.1 time: 0.167174 Sep 27 10:28:23.269182 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2693/5840] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:23 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1507 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:23.328125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3b3aea22-6fe5-4dc2-b632-c29c95cd361d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:23.340906 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3b3aea22-6fe5-4dc2-b632-c29c95cd361d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1563 microversion: 2.1 time: 0.164663 Sep 27 10:28:23.341800 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3148/5841] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:23 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1563 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:23.550446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6283f72e-33d5-4df5-bea8-cf60a147c8cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:23.551072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6283f72e-33d5-4df5-bea8-cf60a147c8cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:23.558096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6283f72e-33d5-4df5-bea8-cf60a147c8cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:23.558587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6283f72e-33d5-4df5-bea8-cf60a147c8cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:23.559038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6283f72e-33d5-4df5-bea8-cf60a147c8cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:23.559745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6283f72e-33d5-4df5-bea8-cf60a147c8cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:23.560210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6283f72e-33d5-4df5-bea8-cf60a147c8cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:23.560646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6283f72e-33d5-4df5-bea8-cf60a147c8cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:23.790365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6283f72e-33d5-4df5-bea8-cf60a147c8cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:23.801483 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6283f72e-33d5-4df5-bea8-cf60a147c8cc tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1729 microversion: 2.1 time: 0.255001 Sep 27 10:28:23.802256 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2694/5842] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:23 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1729 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:24.291477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-55148482-b0fe-4fad-9f46-0c6f12ab4eb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:24.292068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-55148482-b0fe-4fad-9f46-0c6f12ab4eb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:24.299457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55148482-b0fe-4fad-9f46-0c6f12ab4eb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:24.299952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55148482-b0fe-4fad-9f46-0c6f12ab4eb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:24.300423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55148482-b0fe-4fad-9f46-0c6f12ab4eb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:24.301171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55148482-b0fe-4fad-9f46-0c6f12ab4eb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:24.301632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55148482-b0fe-4fad-9f46-0c6f12ab4eb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:24.302060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-55148482-b0fe-4fad-9f46-0c6f12ab4eb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:24.447857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-55148482-b0fe-4fad-9f46-0c6f12ab4eb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:24.457444 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-55148482-b0fe-4fad-9f46-0c6f12ab4eb7 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1507 microversion: 2.1 time: 0.170286 Sep 27 10:28:24.458026 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3149/5843] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:24 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1507 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:24.827184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b3695cff-0044-4498-b3da-59cff8fdddb4 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:24.827815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b3695cff-0044-4498-b3da-59cff8fdddb4 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:24.835771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3695cff-0044-4498-b3da-59cff8fdddb4 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:24.836268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3695cff-0044-4498-b3da-59cff8fdddb4 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:24.836714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3695cff-0044-4498-b3da-59cff8fdddb4 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:24.837676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3695cff-0044-4498-b3da-59cff8fdddb4 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:24.838480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3695cff-0044-4498-b3da-59cff8fdddb4 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:24.839426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b3695cff-0044-4498-b3da-59cff8fdddb4 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:25.083783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7eeea2fd-6f3f-4640-b436-df136678cee8 req-39ffdefb-54e0-42cd-9c87-46afbad617ee service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43bd3356-0fc6-4589-8247-b3394edbaa99", "name": "network-vif-deleted", "tag": "4e47c3e5-afc9-4f0c-abc1-59a3d3f0b51c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:25.097364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7eeea2fd-6f3f-4640-b436-df136678cee8 req-39ffdefb-54e0-42cd-9c87-46afbad617ee service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:25.097861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7eeea2fd-6f3f-4640-b436-df136678cee8 req-39ffdefb-54e0-42cd-9c87-46afbad617ee service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:25.098309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7eeea2fd-6f3f-4640-b436-df136678cee8 req-39ffdefb-54e0-42cd-9c87-46afbad617ee service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:25.111753 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7eeea2fd-6f3f-4640-b436-df136678cee8 req-39ffdefb-54e0-42cd-9c87-46afbad617ee service nova] Creating event network-vif-deleted:4e47c3e5-afc9-4f0c-abc1-59a3d3f0b51c for instance 43bd3356-0fc6-4589-8247-b3394edbaa99 on nch1-master-1 Sep 27 10:28:25.117240 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7eeea2fd-6f3f-4640-b436-df136678cee8 req-39ffdefb-54e0-42cd-9c87-46afbad617ee service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.126057 Sep 27 10:28:25.118582 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3150/5844] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:28:25.483210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:25.483959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:25.490838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:25.491348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:25.491818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:25.492532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:25.493045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:25.493482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:25.527481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:25.527985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:25.528431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:25.528920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lazy-loading 'fault' on Instance uuid 43bd3356-0fc6-4589-8247-b3394edbaa99 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:28:25.573287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:25.582329 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8eded60d-d067-4981-b72e-e497912e5c03 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 200 len: 1304 microversion: 2.1 time: 0.104482 Sep 27 10:28:25.583210 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3151/5845] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:25 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 1304 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:25.719367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b3695cff-0044-4498-b3da-59cff8fdddb4 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:25.727880 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b3695cff-0044-4498-b3da-59cff8fdddb4 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1729 microversion: 2.1 time: 0.905096 Sep 27 10:28:25.741062 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2695/5846] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:24 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1729 bytes in 919 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:26.356454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b31a4852-8d30-4b94-8a0c-85635770c0d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:26.357656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b31a4852-8d30-4b94-8a0c-85635770c0d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:26.362313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ced3b715-8858-4dbd-a3df-5e132b71cc0a req-780a6e7d-85e2-4a9c-ac1d-61a1e959a4d6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "39429342-b89d-443d-9645-17ec8e2869f3", "tag": "b41f6e87-e1ce-49b6-a47a-5129ca582abc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:26.365906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31a4852-8d30-4b94-8a0c-85635770c0d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:26.366721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31a4852-8d30-4b94-8a0c-85635770c0d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:26.367349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31a4852-8d30-4b94-8a0c-85635770c0d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:26.368265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31a4852-8d30-4b94-8a0c-85635770c0d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:26.368923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31a4852-8d30-4b94-8a0c-85635770c0d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:26.369522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31a4852-8d30-4b94-8a0c-85635770c0d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:26.375257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ced3b715-8858-4dbd-a3df-5e132b71cc0a req-780a6e7d-85e2-4a9c-ac1d-61a1e959a4d6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:26.376987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ced3b715-8858-4dbd-a3df-5e132b71cc0a req-780a6e7d-85e2-4a9c-ac1d-61a1e959a4d6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:26.377915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ced3b715-8858-4dbd-a3df-5e132b71cc0a req-780a6e7d-85e2-4a9c-ac1d-61a1e959a4d6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:26.392258 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ced3b715-8858-4dbd-a3df-5e132b71cc0a req-780a6e7d-85e2-4a9c-ac1d-61a1e959a4d6 service nova] Creating event network-changed:b41f6e87-e1ce-49b6-a47a-5129ca582abc for instance 39429342-b89d-443d-9645-17ec8e2869f3 on nch2-master-1 Sep 27 10:28:26.398606 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ced3b715-8858-4dbd-a3df-5e132b71cc0a req-780a6e7d-85e2-4a9c-ac1d-61a1e959a4d6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040998 Sep 27 10:28:26.399882 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2696/5847] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:26.522969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b31a4852-8d30-4b94-8a0c-85635770c0d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:26.553007 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b31a4852-8d30-4b94-8a0c-85635770c0d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1563 microversion: 2.1 time: 0.201122 Sep 27 10:28:26.554681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3152/5848] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:26 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1563 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:26.627608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:26.630088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 43bd3356-0fc6-4589-8247-b3394edbaa99] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:26.642990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:26.643777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:26.645923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:26.661318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:26.662323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:26.663543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:26.689146 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] HTTP exception thrown: Instance 43bd3356-0fc6-4589-8247-b3394edbaa99 could not be found. Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Request method failure captured: Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 HTTP/1.1 Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs6KVh3H5X7rX3evxN11ucsVlmHnbL2MCObHKl1pQO1q40XS_7dvNdOvYhoxemUBx1Ms1xpNth_S22Rp1rV9hfITX6TEFNP6CCnYhe72LSUTyMJphfTwYhmOPX9p8o6YFOanCD6UQJX6ltkdsgee7CkTpD8vTeyh3fvEfCOyNJIfUWA Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_10375bd836f84383b39f2df73f53800e"}]}] Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 97c769dbba794531b0a9db8e764ff097 Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 43bd3356-0fc6-4589-8247-b3394edbaa99 could not be found. Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '43bd3356-0fc6-4589-8247-b3394edbaa99'} Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43bd3356-0fc6-4589-8247-b3394edbaa99 could not be found. Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43bd3356-0fc6-4589-8247-b3394edbaa99 could not be found. Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:26.694088 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:26.704238 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:26.704238 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:26.704238 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:26.704238 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:28:26.704238 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:28:26.704238 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43bd3356-0fc6-4589-8247-b3394edbaa99 could not be found. Sep 27 10:28:26.704238 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:26.707961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Returning 404 to user: Instance 43bd3356-0fc6-4589-8247-b3394edbaa99 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:28:26.711841 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-744f9c38-c22a-47b6-b017-67e69c1bd297 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99" status: 404 len: 111 microversion: 2.1 time: 0.094721 Sep 27 10:28:26.715978 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2697/5849] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:26 2022] GET /compute/v2.1/servers/43bd3356-0fc6-4589-8247-b3394edbaa99 => generated 111 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:28:26.721738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b8380f33-8568-4ceb-9c54-a0f3af18155f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:26.722874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b8380f33-8568-4ceb-9c54-a0f3af18155f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:26.744763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8380f33-8568-4ceb-9c54-a0f3af18155f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:26.748329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8380f33-8568-4ceb-9c54-a0f3af18155f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:26.749147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8380f33-8568-4ceb-9c54-a0f3af18155f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:26.750139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8380f33-8568-4ceb-9c54-a0f3af18155f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:26.751012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8380f33-8568-4ceb-9c54-a0f3af18155f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:26.752038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8380f33-8568-4ceb-9c54-a0f3af18155f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:26.752315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eb86d1c0-9d15-4549-b08f-2811b7efa28e tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:26.753086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eb86d1c0-9d15-4549-b08f-2811b7efa28e tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:26.765470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb86d1c0-9d15-4549-b08f-2811b7efa28e tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:26.766225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb86d1c0-9d15-4549-b08f-2811b7efa28e tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:26.766767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb86d1c0-9d15-4549-b08f-2811b7efa28e tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:26.769787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb86d1c0-9d15-4549-b08f-2811b7efa28e tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:26.769787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb86d1c0-9d15-4549-b08f-2811b7efa28e tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:26.772238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb86d1c0-9d15-4549-b08f-2811b7efa28e tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:26.825804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b8380f33-8568-4ceb-9c54-a0f3af18155f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:28:26.958483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eb86d1c0-9d15-4549-b08f-2811b7efa28e tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dc394c36e4bb436fabbecada1990b41e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4195812e6f274bbba026e71b7fc4e231', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:26.973860 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eb86d1c0-9d15-4549-b08f-2811b7efa28e tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 200 len: 1382 microversion: 2.1 time: 0.226274 Sep 27 10:28:26.975662 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2698/5850] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:26 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 1382 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:26.982034 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b8380f33-8568-4ceb-9c54-a0f3af18155f tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "DELETE /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 204 len: 0 microversion: 2.1 time: 0.264362 Sep 27 10:28:26.982645 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3153/5851] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:28:26 2022] DELETE /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:28:26.992422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c934846d-c42f-47bb-baad-a39eff85f0f8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:26.993204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c934846d-c42f-47bb-baad-a39eff85f0f8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:27.007631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934846d-c42f-47bb-baad-a39eff85f0f8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:27.012069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934846d-c42f-47bb-baad-a39eff85f0f8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:27.012840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934846d-c42f-47bb-baad-a39eff85f0f8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:27.015617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934846d-c42f-47bb-baad-a39eff85f0f8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:27.016363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934846d-c42f-47bb-baad-a39eff85f0f8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:27.017326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934846d-c42f-47bb-baad-a39eff85f0f8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:27.182936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c934846d-c42f-47bb-baad-a39eff85f0f8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:27.193267 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c934846d-c42f-47bb-baad-a39eff85f0f8 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1495 microversion: 2.1 time: 0.205740 Sep 27 10:28:27.194178 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3154/5852] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:26 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1495 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:27.999526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:28.000320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:28.007847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:28.008359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:28.008850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:28.009574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:28.010045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:28.010466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:28.024403 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] HTTP exception thrown: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Request method failure captured: Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb HTTP/1.1 Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs7zmHSrX3lTm0BL0CpFaoZzGgSbr4FN9HQvJ4Zk0ISvKloeNcvR1q8xIZrrj54HuiFmulYMlOdcXaFnzbmW6Rx4zTE_kYb_2a0xfNNh32W289dj5UywSrq2KXqNye2dsfDgwjPLB-DmLjTYMNgX0Lh8Un0wrs3zHNfXrFELzzbajAI Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 4195812e6f274bbba026e71b7fc4e231 Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestServerBasicOps-1856727398 Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4195812e6f274bbba026e71b7fc4e231"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4195812e6f274bbba026e71b7fc4e231"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/4195812e6f274bbba026e71b7fc4e231"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_4195812e6f274bbba026e71b7fc4e231"}]}] Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestServerBasicOps-1856727398 Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 4195812e6f274bbba026e71b7fc4e231 Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestServerBasicOps-1856727398 Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestServerBasicOps-1856727398-project Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: dc394c36e4bb436fabbecada1990b41e Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestServerBasicOps-1856727398-project Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3e246452-0210-449b-b9b7-f1f42ccaa2eb'} Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:28.026038 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.030963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:28.030963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:28.030963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:28.030963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:28:28.030963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:28:28.030963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.030963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.030963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Returning 404 to user: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:28:28.030963 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-95f4e73f-ff84-40b0-a7b8-dcad8793bd3b tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 404 len: 111 microversion: 2.1 time: 0.031994 Sep 27 10:28:28.030963 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2699/5853] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:27 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:28:28.216054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13cb7f85-187a-46f7-a066-343225bf7802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:28.216753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-13cb7f85-187a-46f7-a066-343225bf7802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:28.226190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13cb7f85-187a-46f7-a066-343225bf7802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:28.226672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13cb7f85-187a-46f7-a066-343225bf7802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:28.227108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13cb7f85-187a-46f7-a066-343225bf7802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:28.227812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13cb7f85-187a-46f7-a066-343225bf7802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:28.228263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13cb7f85-187a-46f7-a066-343225bf7802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:28.228696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13cb7f85-187a-46f7-a066-343225bf7802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:28.365440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-13cb7f85-187a-46f7-a066-343225bf7802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:28.373733 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13cb7f85-187a-46f7-a066-343225bf7802 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1495 microversion: 2.1 time: 0.162108 Sep 27 10:28:28.374418 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3155/5854] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:28 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1495 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:28.435962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-eff6af8f-e5c8-4a54-a7d1-8f784d3bd044 req-8e17b0be-4546-46d0-b0c0-3db19662d46c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3e246452-0210-449b-b9b7-f1f42ccaa2eb", "name": "network-vif-deleted", "tag": "14864aa7-71a6-4bce-944b-e859f11985e0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:28.446098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-eff6af8f-e5c8-4a54-a7d1-8f784d3bd044 req-8e17b0be-4546-46d0-b0c0-3db19662d46c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:28.446595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-eff6af8f-e5c8-4a54-a7d1-8f784d3bd044 req-8e17b0be-4546-46d0-b0c0-3db19662d46c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:28.447031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-eff6af8f-e5c8-4a54-a7d1-8f784d3bd044 req-8e17b0be-4546-46d0-b0c0-3db19662d46c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:28.459020 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-eff6af8f-e5c8-4a54-a7d1-8f784d3bd044 req-8e17b0be-4546-46d0-b0c0-3db19662d46c service nova] Creating event network-vif-deleted:14864aa7-71a6-4bce-944b-e859f11985e0 for instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb on nch1-master-1 Sep 27 10:28:28.464554 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-eff6af8f-e5c8-4a54-a7d1-8f784d3bd044 req-8e17b0be-4546-46d0-b0c0-3db19662d46c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031982 Sep 27 10:28:28.465192 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2700/5855] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:28.927807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:28.928249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:28.934858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:28.935350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:28.935795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:28.936507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:28.937006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:28.937425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:28.951553 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] HTTP exception thrown: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Request method failure captured: Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb HTTP/1.1 Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs7zmHSrX3lTm0BL0CpFaoZzGgSbr4FN9HQvJ4Zk0ISvKloeNcvR1q8xIZrrj54HuiFmulYMlOdcXaFnzbmW6Rx4zTE_kYb_2a0xfNNh32W289dj5UywSrq2KXqNye2dsfDgwjPLB-DmLjTYMNgX0Lh8Un0wrs3zHNfXrFELzzbajAI Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 4195812e6f274bbba026e71b7fc4e231 Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestServerBasicOps-1856727398 Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4195812e6f274bbba026e71b7fc4e231"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4195812e6f274bbba026e71b7fc4e231"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/4195812e6f274bbba026e71b7fc4e231"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_4195812e6f274bbba026e71b7fc4e231"}]}] Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestServerBasicOps-1856727398 Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 4195812e6f274bbba026e71b7fc4e231 Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestServerBasicOps-1856727398 Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestServerBasicOps-1856727398-project Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: dc394c36e4bb436fabbecada1990b41e Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestServerBasicOps-1856727398-project Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3e246452-0210-449b-b9b7-f1f42ccaa2eb'} Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:28:28.953484 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Returning 404 to user: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-05dbb50a-dc58-4404-9df0-545640852453 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 404 len: 111 microversion: 2.1 time: 0.031548 Sep 27 10:28:28.961837 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3156/5856] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:28:28 2022] DELETE /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:28:28.967069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:28.967869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] [instance: 3e246452-0210-449b-b9b7-f1f42ccaa2eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:28.975460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:28.976037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:28.976494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:28.977243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:28.977691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:28.978151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:28.992994 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] HTTP exception thrown: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Request method failure captured: Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb HTTP/1.1 Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs7zmHSrX3lTm0BL0CpFaoZzGgSbr4FN9HQvJ4Zk0ISvKloeNcvR1q8xIZrrj54HuiFmulYMlOdcXaFnzbmW6Rx4zTE_kYb_2a0xfNNh32W289dj5UywSrq2KXqNye2dsfDgwjPLB-DmLjTYMNgX0Lh8Un0wrs3zHNfXrFELzzbajAI Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 4195812e6f274bbba026e71b7fc4e231 Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestServerBasicOps-1856727398 Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4195812e6f274bbba026e71b7fc4e231"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/4195812e6f274bbba026e71b7fc4e231"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/4195812e6f274bbba026e71b7fc4e231"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_4195812e6f274bbba026e71b7fc4e231"}]}] Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestServerBasicOps-1856727398 Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 4195812e6f274bbba026e71b7fc4e231 Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestServerBasicOps-1856727398 Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestServerBasicOps-1856727398-project Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: dc394c36e4bb436fabbecada1990b41e Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestServerBasicOps-1856727398-project Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3e246452-0210-449b-b9b7-f1f42ccaa2eb'} Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:28.994661 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:29.002813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:29.002813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:29.002813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:29.002813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:28:29.002813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:28:29.002813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. Sep 27 10:28:29.002813 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:29.002813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Returning 404 to user: Instance 3e246452-0210-449b-b9b7-f1f42ccaa2eb could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:28:29.002813 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ce30b430-90ff-425f-9f08-d2673c25ad62 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb" status: 404 len: 111 microversion: 2.1 time: 0.034643 Sep 27 10:28:29.002813 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2701/5857] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:28 2022] GET /compute/v2.1/servers/3e246452-0210-449b-b9b7-f1f42ccaa2eb => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:28:29.193555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4f60439e-4551-4058-8f5e-db3b847de089 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] Calling method '.version_select at 0x7f7f626fb700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:29.218016 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4f60439e-4551-4058-8f5e-db3b847de089 tempest-TestServerBasicOps-1856727398 tempest-TestServerBasicOps-1856727398-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestServerBasicOps-829950563" status: 202 len: 0 microversion: 2.1 time: 0.030425 Sep 27 10:28:29.218969 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3157/5858] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:28:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestServerBasicOps-829950563 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:28:29.396620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:29.397273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:29.404546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:29.405107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:29.405554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:29.406357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:29.407021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:29.407450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:29.444040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:29.444435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:29.444907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:29.445369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lazy-loading 'fault' on Instance uuid 39e807ee-fe6b-4812-bc48-b47085b1b89f {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:28:29.491567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:29.500633 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f2067a5-b97b-4d55-a227-19e153666439 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 200 len: 1305 microversion: 2.1 time: 0.108192 Sep 27 10:28:29.501449 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2702/5859] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:29 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 1305 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:30.484187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7c29e9c1-ff64-45b5-bbb8-ecf78465544e req-2a2e0003-d909-4db0-926b-0c1ead3fdbc1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39e807ee-fe6b-4812-bc48-b47085b1b89f", "name": "network-vif-deleted", "tag": "7a2a97a6-af5d-4a2c-a121-7127ea436b1f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:30.495842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7c29e9c1-ff64-45b5-bbb8-ecf78465544e req-2a2e0003-d909-4db0-926b-0c1ead3fdbc1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:30.496333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7c29e9c1-ff64-45b5-bbb8-ecf78465544e req-2a2e0003-d909-4db0-926b-0c1ead3fdbc1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:30.496809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7c29e9c1-ff64-45b5-bbb8-ecf78465544e req-2a2e0003-d909-4db0-926b-0c1ead3fdbc1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:30.508458 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7c29e9c1-ff64-45b5-bbb8-ecf78465544e req-2a2e0003-d909-4db0-926b-0c1ead3fdbc1 service nova] Creating event network-vif-deleted:7a2a97a6-af5d-4a2c-a121-7127ea436b1f for instance 39e807ee-fe6b-4812-bc48-b47085b1b89f on nch2-master-1 Sep 27 10:28:30.513438 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7c29e9c1-ff64-45b5-bbb8-ecf78465544e req-2a2e0003-d909-4db0-926b-0c1ead3fdbc1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033094 Sep 27 10:28:30.514012 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3158/5860] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:30.524173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:30.524816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 39e807ee-fe6b-4812-bc48-b47085b1b89f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:30.532382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:30.532915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:30.533366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:30.534099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:30.534545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:30.534979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:30.550949 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] HTTP exception thrown: Instance 39e807ee-fe6b-4812-bc48-b47085b1b89f could not be found. Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Request method failure captured: Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f HTTP/1.1 Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs6KVh3H5X7rX3evxN11ucsVlmHnbL2MCObHKl1pQO1q40XS_7dvNdOvYhoxemUBx1Ms1xpNth_S22Rp1rV9hfITX6TEFNP6CCnYhe72LSUTyMJphfTwYhmOPX9p8o6YFOanCD6UQJX6ltkdsgee7CkTpD8vTeyh3fvEfCOyNJIfUWA Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_10375bd836f84383b39f2df73f53800e"}]}] Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 97c769dbba794531b0a9db8e764ff097 Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 39e807ee-fe6b-4812-bc48-b47085b1b89f could not be found. Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '39e807ee-fe6b-4812-bc48-b47085b1b89f'} Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 39e807ee-fe6b-4812-bc48-b47085b1b89f could not be found. Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 39e807ee-fe6b-4812-bc48-b47085b1b89f could not be found. Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:30.552434 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:30.557010 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:30.557010 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:30.557010 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:30.557010 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:28:30.557010 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:28:30.557010 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 39e807ee-fe6b-4812-bc48-b47085b1b89f could not be found. Sep 27 10:28:30.557010 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:30.557010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Returning 404 to user: Instance 39e807ee-fe6b-4812-bc48-b47085b1b89f could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:28:30.557010 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4d45fb8e-ec50-49e1-aa45-7f4570b14686 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f" status: 404 len: 111 microversion: 2.1 time: 0.034044 Sep 27 10:28:30.557010 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2703/5861] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:30 2022] GET /compute/v2.1/servers/39e807ee-fe6b-4812-bc48-b47085b1b89f => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:28:30.563870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d23fc9c-b3d9-4f7f-bce4-7bfba489653c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:30.564524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d23fc9c-b3d9-4f7f-bce4-7bfba489653c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:30.571031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d23fc9c-b3d9-4f7f-bce4-7bfba489653c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:30.571525 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d23fc9c-b3d9-4f7f-bce4-7bfba489653c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:30.571971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d23fc9c-b3d9-4f7f-bce4-7bfba489653c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:30.572733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d23fc9c-b3d9-4f7f-bce4-7bfba489653c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:30.573208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d23fc9c-b3d9-4f7f-bce4-7bfba489653c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:30.573644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d23fc9c-b3d9-4f7f-bce4-7bfba489653c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:30.599773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d23fc9c-b3d9-4f7f-bce4-7bfba489653c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:28:30.627949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:30.685537 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d23fc9c-b3d9-4f7f-bce4-7bfba489653c tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "DELETE /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 204 len: 0 microversion: 2.1 time: 0.125899 Sep 27 10:28:30.686320 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3159/5862] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:28:30 2022] DELETE /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:28:30.695455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fcb15459-16b9-468f-a944-784811a74303 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:30.696088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fcb15459-16b9-468f-a944-784811a74303 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:30.702836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcb15459-16b9-468f-a944-784811a74303 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:30.703668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcb15459-16b9-468f-a944-784811a74303 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:30.704371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcb15459-16b9-468f-a944-784811a74303 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:30.705146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcb15459-16b9-468f-a944-784811a74303 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:30.705674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcb15459-16b9-468f-a944-784811a74303 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:30.706165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fcb15459-16b9-468f-a944-784811a74303 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:30.860353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fcb15459-16b9-468f-a944-784811a74303 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:30.873339 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fcb15459-16b9-468f-a944-784811a74303 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1675 microversion: 2.1 time: 0.181261 Sep 27 10:28:30.879419 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2704/5863] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:30 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1675 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:31.899447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d5bb59e4-3091-4c76-bd26-9432189b2367 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:31.900166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d5bb59e4-3091-4c76-bd26-9432189b2367 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:31.907535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5bb59e4-3091-4c76-bd26-9432189b2367 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:31.908324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5bb59e4-3091-4c76-bd26-9432189b2367 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:31.908825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5bb59e4-3091-4c76-bd26-9432189b2367 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:31.909589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5bb59e4-3091-4c76-bd26-9432189b2367 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:31.910089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5bb59e4-3091-4c76-bd26-9432189b2367 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:31.910673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5bb59e4-3091-4c76-bd26-9432189b2367 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:32.069660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d5bb59e4-3091-4c76-bd26-9432189b2367 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:32.079276 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d5bb59e4-3091-4c76-bd26-9432189b2367 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1675 microversion: 2.1 time: 0.184905 Sep 27 10:28:32.079996 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3160/5864] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:31 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1675 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:33.104355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2bc0d44b-a0b9-4f3f-9130-3208ea43bf26 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:33.105300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2bc0d44b-a0b9-4f3f-9130-3208ea43bf26 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:33.112566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bc0d44b-a0b9-4f3f-9130-3208ea43bf26 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:33.113116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bc0d44b-a0b9-4f3f-9130-3208ea43bf26 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:33.113578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bc0d44b-a0b9-4f3f-9130-3208ea43bf26 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:33.114303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bc0d44b-a0b9-4f3f-9130-3208ea43bf26 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:33.114829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bc0d44b-a0b9-4f3f-9130-3208ea43bf26 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:33.115276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2bc0d44b-a0b9-4f3f-9130-3208ea43bf26 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:33.477604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2bc0d44b-a0b9-4f3f-9130-3208ea43bf26 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:33.486599 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2bc0d44b-a0b9-4f3f-9130-3208ea43bf26 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1675 microversion: 2.1 time: 0.386899 Sep 27 10:28:33.487484 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2705/5865] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:33 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1675 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:34.066335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f9a29db9-df38-40a0-a134-801940591f55 req-cca70adc-5f66-4765-b60c-72acd894896e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5ac6dabf-1315-4e05-ab6c-22211dc2440b", "name": "network-vif-deleted", "tag": "b894a455-9bb5-443a-84fe-e04c8d0c3e28"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:34.085442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f9a29db9-df38-40a0-a134-801940591f55 req-cca70adc-5f66-4765-b60c-72acd894896e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:34.086123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f9a29db9-df38-40a0-a134-801940591f55 req-cca70adc-5f66-4765-b60c-72acd894896e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:34.086730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f9a29db9-df38-40a0-a134-801940591f55 req-cca70adc-5f66-4765-b60c-72acd894896e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:34.103834 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f9a29db9-df38-40a0-a134-801940591f55 req-cca70adc-5f66-4765-b60c-72acd894896e service nova] Creating event network-vif-deleted:b894a455-9bb5-443a-84fe-e04c8d0c3e28 for instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b on nch1-master-1 Sep 27 10:28:34.109828 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f9a29db9-df38-40a0-a134-801940591f55 req-cca70adc-5f66-4765-b60c-72acd894896e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047600 Sep 27 10:28:34.111371 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3161/5866] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:34.517597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:34.519522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:34.531531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:34.532306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:34.532787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:34.533829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:34.534470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:34.535077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:34.573662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:34.574169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:34.574615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:34.575080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lazy-loading 'fault' on Instance uuid 5ac6dabf-1315-4e05-ab6c-22211dc2440b {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:28:34.650945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '97c769dbba794531b0a9db8e764ff097', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '10375bd836f84383b39f2df73f53800e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:34.660721 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-50939c14-d57e-4fab-9116-aadcf8b140ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 200 len: 1328 microversion: 2.1 time: 0.153314 Sep 27 10:28:34.662522 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2706/5867] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:34 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 1328 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:35.691712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:35.692421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] [instance: 5ac6dabf-1315-4e05-ab6c-22211dc2440b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:35.699180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:35.700079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:35.700514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:35.701269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:35.701730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:35.702164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:35.721971 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] HTTP exception thrown: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b could not be found. Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Request method failure captured: Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b HTTP/1.1 Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs6KVh3H5X7rX3evxN11ucsVlmHnbL2MCObHKl1pQO1q40XS_7dvNdOvYhoxemUBx1Ms1xpNth_S22Rp1rV9hfITX6TEFNP6CCnYhe72LSUTyMJphfTwYhmOPX9p8o6YFOanCD6UQJX6ltkdsgee7CkTpD8vTeyh3fvEfCOyNJIfUWA Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/10375bd836f84383b39f2df73f53800e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_10375bd836f84383b39f2df73f53800e"}]}] Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 10375bd836f84383b39f2df73f53800e Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerActionsTestJSON-1470599362 Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 97c769dbba794531b0a9db8e764ff097 Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerActionsTestJSON-1470599362-project Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b could not be found. Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '5ac6dabf-1315-4e05-ab6c-22211dc2440b'} Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b could not be found. Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b could not be found. Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:35.723710 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:35.730390 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:35.730390 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:35.730390 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:35.730390 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:28:35.730390 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:28:35.730390 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b could not be found. Sep 27 10:28:35.730390 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:28:35.730390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Returning 404 to user: Instance 5ac6dabf-1315-4e05-ab6c-22211dc2440b could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:28:35.730390 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3e846d85-f813-4921-9415-1255666b3605 tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b" status: 404 len: 111 microversion: 2.1 time: 0.039869 Sep 27 10:28:35.730390 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3162/5868] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:35 2022] GET /compute/v2.1/servers/5ac6dabf-1315-4e05-ab6c-22211dc2440b => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:28:35.734727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-72645cdb-87af-4922-adfd-4d24da8389ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] Calling method '.version_select at 0x7f7f62392940>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:35.760130 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-72645cdb-87af-4922-adfd-4d24da8389ec tempest-ServerActionsTestJSON-1470599362 tempest-ServerActionsTestJSON-1470599362-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1650140436" status: 202 len: 0 microversion: 2.1 time: 0.029179 Sep 27 10:28:35.760990 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2707/5869] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:28:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1650140436 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:28:45.696358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=91,nova_api:UPDATE=3,nova_api:DELETE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:45.707116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:UPDATE=1,nova_cell1:SELECT=14,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:45.728203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5452c0a1-1fcd-499a-8fed-f9ae7cda6c9b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Action: 'create', calling method: .version_select at 0x7f7f630f9160>, body: {"server_group": {"name": "tempest-ServerGroupTestJSON-Server-Group-1985137543", "policies": ["affinity"]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:45.748822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5452c0a1-1fcd-499a-8fed-f9ae7cda6c9b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:45.754659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5452c0a1-1fcd-499a-8fed-f9ae7cda6c9b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:45.805468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5452c0a1-1fcd-499a-8fed-f9ae7cda6c9b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:45.810713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-5452c0a1-1fcd-499a-8fed-f9ae7cda6c9b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:45.823607 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5452c0a1-1fcd-499a-8fed-f9ae7cda6c9b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "POST /compute/v2.1/os-server-groups" status: 200 len: 184 microversion: 2.1 time: 0.240352 Sep 27 10:28:45.828917 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3163/5870] 10.222.0.26 () {60 vars in 1174 bytes} [Tue Sep 27 10:28:45 2022] POST /compute/v2.1/os-server-groups => generated 184 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:45.835434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-20130e12-e5fc-403f-a358-0377086d5480 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Action: 'create', calling method: .version_select at 0x7f7f6031e0d0>, body: {"server_group": {"name": "tempest-server-group-334319484", "policies": ["affinity"]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:45.845442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=83,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:45.846564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:45.871400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-20130e12-e5fc-403f-a358-0377086d5480 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:45.878971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-20130e12-e5fc-403f-a358-0377086d5480 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:45.936534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-20130e12-e5fc-403f-a358-0377086d5480 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:45.944399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-20130e12-e5fc-403f-a358-0377086d5480 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:45.962785 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-20130e12-e5fc-403f-a358-0377086d5480 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "POST /compute/v2.1/os-server-groups" status: 200 len: 163 microversion: 2.1 time: 0.131284 Sep 27 10:28:45.963412 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2708/5871] 10.222.0.26 () {60 vars in 1173 bytes} [Tue Sep 27 10:28:45 2022] POST /compute/v2.1/os-server-groups => generated 163 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:28:45.975152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8cec738a-e83a-42d0-a1cc-37ea14a4f983 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Action: 'create', calling method: .version_select at 0x7f7f626fbaf0>, body: {"server_group": {"name": "tempest-server-group-334319484", "policies": ["affinity"]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:45.993812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8cec738a-e83a-42d0-a1cc-37ea14a4f983 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:45.999374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8cec738a-e83a-42d0-a1cc-37ea14a4f983 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:46.044505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8cec738a-e83a-42d0-a1cc-37ea14a4f983 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:46.050147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8cec738a-e83a-42d0-a1cc-37ea14a4f983 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:46.064248 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8cec738a-e83a-42d0-a1cc-37ea14a4f983 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "POST /compute/v2.1/os-server-groups" status: 200 len: 163 microversion: 2.1 time: 0.093224 Sep 27 10:28:46.065012 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3164/5872] 10.222.0.26 () {60 vars in 1173 bytes} [Tue Sep 27 10:28:45 2022] POST /compute/v2.1/os-server-groups => generated 163 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:46.075770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2118ae3e-3a9e-4767-8ee0-a4b343ddbf75 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:46.103631 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2118ae3e-3a9e-4767-8ee0-a4b343ddbf75 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/955305b7-bf42-4111-8186-5f9420d5eaef" status: 204 len: 0 microversion: 2.1 time: 0.031544 Sep 27 10:28:46.104205 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2709/5873] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:28:46 2022] DELETE /compute/v2.1/os-server-groups/955305b7-bf42-4111-8186-5f9420d5eaef => generated 0 bytes in 33 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:28:46.112923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1af74da7-bdd0-48a6-960a-a359ccd7f41a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:46.130192 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1af74da7-bdd0-48a6-960a-a359ccd7f41a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/os-server-groups" status: 200 len: 334 microversion: 2.1 time: 0.021329 Sep 27 10:28:46.130685 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3165/5874] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 10:28:46 2022] GET /compute/v2.1/os-server-groups => generated 334 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:46.141888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f7649dc7-9e2d-46a1-a1c2-4d4b4748e159 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:46.172403 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f7649dc7-9e2d-46a1-a1c2-4d4b4748e159 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/fc138075-c913-4390-a240-d4e6c7dbdc69" status: 204 len: 0 microversion: 2.1 time: 0.034371 Sep 27 10:28:46.173315 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2710/5875] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:28:46 2022] DELETE /compute/v2.1/os-server-groups/fc138075-c913-4390-a240-d4e6c7dbdc69 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:28:46.182090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b19d2977-fe3c-4ffc-9fd7-8443bfad28da tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:46.194646 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b19d2977-fe3c-4ffc-9fd7-8443bfad28da tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.016952 Sep 27 10:28:46.195674 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3166/5876] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 10:28:46 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:46.208942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ba03a0cd-521c-4de2-9b98-a28babada775 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Action: 'create', calling method: .version_select at 0x7f7f6036aca0>, body: {"server_group": {"name": "tempest-server-group-903297600", "policies": ["affinity"]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:46.228208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ba03a0cd-521c-4de2-9b98-a28babada775 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:46.233883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ba03a0cd-521c-4de2-9b98-a28babada775 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:46.279212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ba03a0cd-521c-4de2-9b98-a28babada775 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:46.285675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ba03a0cd-521c-4de2-9b98-a28babada775 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:46.299763 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ba03a0cd-521c-4de2-9b98-a28babada775 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "POST /compute/v2.1/os-server-groups" status: 200 len: 163 microversion: 2.1 time: 0.094938 Sep 27 10:28:46.300381 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2711/5877] 10.222.0.26 () {60 vars in 1173 bytes} [Tue Sep 27 10:28:46 2022] POST /compute/v2.1/os-server-groups => generated 163 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:46.311458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-10429b1f-00df-4f0c-b0f4-f1ef9c042e8c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:46.345149 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-10429b1f-00df-4f0c-b0f4-f1ef9c042e8c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/483d15c5-e552-4c69-ba11-66f1aab314bb" status: 204 len: 0 microversion: 2.1 time: 0.037434 Sep 27 10:28:46.346960 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3167/5878] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:28:46 2022] DELETE /compute/v2.1/os-server-groups/483d15c5-e552-4c69-ba11-66f1aab314bb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:28:46.356876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-242bb1c2-4d2e-48df-a3b6-5ac47f5c7d47 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:46.371777 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-242bb1c2-4d2e-48df-a3b6-5ac47f5c7d47 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.019675 Sep 27 10:28:46.372526 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2712/5879] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 10:28:46 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:46.385237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d417b65f-df89-487c-85a3-6f79430939b6 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Action: 'create', calling method: .version_select at 0x7f7f630c9040>, body: {"server_group": {"name": "tempest-server-group-565293448", "policies": ["anti-affinity"]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:46.405642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d417b65f-df89-487c-85a3-6f79430939b6 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:46.414308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d417b65f-df89-487c-85a3-6f79430939b6 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:46.461924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d417b65f-df89-487c-85a3-6f79430939b6 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:46.467483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d417b65f-df89-487c-85a3-6f79430939b6 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_groups'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:46.480917 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d417b65f-df89-487c-85a3-6f79430939b6 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "POST /compute/v2.1/os-server-groups" status: 200 len: 168 microversion: 2.1 time: 0.099531 Sep 27 10:28:46.481533 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3168/5880] 10.222.0.26 () {60 vars in 1173 bytes} [Tue Sep 27 10:28:46 2022] POST /compute/v2.1/os-server-groups => generated 168 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:46.493736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d4ba0618-96db-47a5-951c-572585332aaf tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:46.520831 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d4ba0618-96db-47a5-951c-572585332aaf tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/67432b7d-d3ef-4407-bafe-d67a2ac0868b" status: 204 len: 0 microversion: 2.1 time: 0.031502 Sep 27 10:28:46.521432 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2713/5881] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:28:46 2022] DELETE /compute/v2.1/os-server-groups/67432b7d-d3ef-4407-bafe-d67a2ac0868b => generated 0 bytes in 33 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:28:46.529848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8d0edae9-998f-4095-b58d-3bbfcab595e4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:46.541524 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8d0edae9-998f-4095-b58d-3bbfcab595e4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.015608 Sep 27 10:28:46.542352 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3169/5882] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 10:28:46 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 17 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:46.599013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerGroupTestJSON-server-1046370153", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "a9a75669-cb0c-441b-a9fb-a8eaab59c5a2"}]}, "os:scheduler_hints": {"group": "218a0d89-d6a7-4adb-9e51-3cb6ed379476"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:46.658053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] validate_networks() for [('a9a75669-cb0c-441b-a9fb-a8eaab59c5a2', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:28:46.916904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a8da68fe-8f6d-43ba-9cdf-cfdcd6938ea0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:46.918384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a8da68fe-8f6d-43ba-9cdf-cfdcd6938ea0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:46.927392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8da68fe-8f6d-43ba-9cdf-cfdcd6938ea0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:46.927915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8da68fe-8f6d-43ba-9cdf-cfdcd6938ea0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:46.928511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8da68fe-8f6d-43ba-9cdf-cfdcd6938ea0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:46.929254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8da68fe-8f6d-43ba-9cdf-cfdcd6938ea0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:46.929901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8da68fe-8f6d-43ba-9cdf-cfdcd6938ea0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:46.930327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8da68fe-8f6d-43ba-9cdf-cfdcd6938ea0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:46.958830 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a8da68fe-8f6d-43ba-9cdf-cfdcd6938ea0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/ips" status: 200 len: 150 microversion: 2.1 time: 0.046429 Sep 27 10:28:46.967875 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3170/5883] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:28:46 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/ips => generated 150 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:47.082373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:28:47.083377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:28:47.084047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:28:47.084646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:28:47.085268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:28:47.101209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:47.102051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:47.102673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:47.103807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:47.104453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:47.105100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:47.131718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:47.138115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:47.153091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:28:47.155693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:28:47.198740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=7,nova_api:DELETE=9 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:47.210365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:47.217490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:47.253882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:47.259183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Getting quotas for user 91b11c3c34f8402390e4ee61b6c1bc8f and project 2c5bb6bd7e434b25992308dca2593154. Resources: {'server_group_members'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:47.317394 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a0f1b78d-a3d4-49cd-aa8b-d9cc1430bbd4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.722312 Sep 27 10:28:47.320815 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2714/5884] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:28:46 2022] POST /compute/v2.1/servers => generated 380 bytes in 726 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:28:47.332876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-138f8a7d-2882-4a0b-b7fa-75141a263b2e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:47.333881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-138f8a7d-2882-4a0b-b7fa-75141a263b2e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:47.348177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=6,nova_api:DELETE=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:47.408577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-138f8a7d-2882-4a0b-b7fa-75141a263b2e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:47.419251 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-138f8a7d-2882-4a0b-b7fa-75141a263b2e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1213 microversion: 2.1 time: 0.090978 Sep 27 10:28:47.420149 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3171/5885] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:47 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1213 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:48.429404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "64e8066b-82ca-4278-ac3b-4ce6d3460710"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:48.431817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:48.439899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:48.440689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:48.441326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:48.442202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:48.442820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:48.443415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:48.457985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-46589bd3-7c3c-480a-b11d-3b25c9e50bf7 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:48.458611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-46589bd3-7c3c-480a-b11d-3b25c9e50bf7 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:48.466715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46589bd3-7c3c-480a-b11d-3b25c9e50bf7 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:48.467216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46589bd3-7c3c-480a-b11d-3b25c9e50bf7 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:48.467663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46589bd3-7c3c-480a-b11d-3b25c9e50bf7 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:48.468387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46589bd3-7c3c-480a-b11d-3b25c9e50bf7 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:48.468891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46589bd3-7c3c-480a-b11d-3b25c9e50bf7 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:48.469328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46589bd3-7c3c-480a-b11d-3b25c9e50bf7 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:48.539692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-46589bd3-7c3c-480a-b11d-3b25c9e50bf7 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:48.548711 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-46589bd3-7c3c-480a-b11d-3b25c9e50bf7 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1265 microversion: 2.1 time: 0.094560 Sep 27 10:28:48.549607 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3172/5886] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:48 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1265 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:48.813348 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-edd5fb06-cd7a-43f9-b1f8-988cc589d5a3 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.387869 Sep 27 10:28:48.814172 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2715/5887] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:28:48 2022] POST /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/os-volume_attachments => generated 194 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:49.571631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e30e9cf1-8b7f-432b-afd0-7a62f27f814e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:49.574129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e30e9cf1-8b7f-432b-afd0-7a62f27f814e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:49.582297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e30e9cf1-8b7f-432b-afd0-7a62f27f814e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:49.584643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e30e9cf1-8b7f-432b-afd0-7a62f27f814e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:49.585310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e30e9cf1-8b7f-432b-afd0-7a62f27f814e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:49.587994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e30e9cf1-8b7f-432b-afd0-7a62f27f814e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:49.588641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e30e9cf1-8b7f-432b-afd0-7a62f27f814e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:49.589605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e30e9cf1-8b7f-432b-afd0-7a62f27f814e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:49.843203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e30e9cf1-8b7f-432b-afd0-7a62f27f814e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:49.854810 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e30e9cf1-8b7f-432b-afd0-7a62f27f814e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1313 microversion: 2.1 time: 0.286068 Sep 27 10:28:49.870928 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3173/5888] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:49 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1313 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:50.294807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b4ece302-85e9-44e0-82fb-ea9ebb7aef58 req-a0861b1a-944d-4ad1-b0dd-eb0edc3c2604 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "87a295a8-60b5-4213-bdb8-a41e36e3adb5", "tag": "32374fc0-593b-45d3-8f7f-5d997a06028f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:50.308746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b4ece302-85e9-44e0-82fb-ea9ebb7aef58 req-a0861b1a-944d-4ad1-b0dd-eb0edc3c2604 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:50.309393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b4ece302-85e9-44e0-82fb-ea9ebb7aef58 req-a0861b1a-944d-4ad1-b0dd-eb0edc3c2604 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:50.310121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b4ece302-85e9-44e0-82fb-ea9ebb7aef58 req-a0861b1a-944d-4ad1-b0dd-eb0edc3c2604 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:50.324008 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b4ece302-85e9-44e0-82fb-ea9ebb7aef58 req-a0861b1a-944d-4ad1-b0dd-eb0edc3c2604 service nova] Creating event network-changed:32374fc0-593b-45d3-8f7f-5d997a06028f for instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 on nch1-master-1 Sep 27 10:28:50.329639 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b4ece302-85e9-44e0-82fb-ea9ebb7aef58 req-a0861b1a-944d-4ad1-b0dd-eb0edc3c2604 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040869 Sep 27 10:28:50.330348 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2716/5889] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:50.882537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3bf1d65f-2d56-4ab0-9805-9f34a0c7808f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:50.883163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3bf1d65f-2d56-4ab0-9805-9f34a0c7808f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:50.891361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf1d65f-2d56-4ab0-9805-9f34a0c7808f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:50.891798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf1d65f-2d56-4ab0-9805-9f34a0c7808f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:50.892883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf1d65f-2d56-4ab0-9805-9f34a0c7808f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:50.893724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf1d65f-2d56-4ab0-9805-9f34a0c7808f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:50.894074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf1d65f-2d56-4ab0-9805-9f34a0c7808f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:50.894517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bf1d65f-2d56-4ab0-9805-9f34a0c7808f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:51.055428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3bf1d65f-2d56-4ab0-9805-9f34a0c7808f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:51.066198 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3bf1d65f-2d56-4ab0-9805-9f34a0c7808f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1313 microversion: 2.1 time: 0.187973 Sep 27 10:28:51.070644 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3174/5890] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:50 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1313 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:51.955622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:51.956402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:51.963074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:51.963576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:51.964030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:51.965119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:51.965591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:51.966032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:52.049790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lazy-loading 'services' on Instance uuid 9d4f13c2-3c04-4984-a293-df05378d29e5 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:28:52.090587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1fa24fa6-dc74-4f4d-803c-56f4aa98da38 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:52.091153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1fa24fa6-dc74-4f4d-803c-56f4aa98da38 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:52.099580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fa24fa6-dc74-4f4d-803c-56f4aa98da38 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:52.100413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fa24fa6-dc74-4f4d-803c-56f4aa98da38 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:52.101527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fa24fa6-dc74-4f4d-803c-56f4aa98da38 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:52.102475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fa24fa6-dc74-4f4d-803c-56f4aa98da38 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:52.103485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fa24fa6-dc74-4f4d-803c-56f4aa98da38 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:52.104033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fa24fa6-dc74-4f4d-803c-56f4aa98da38 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:52.145896 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9e5ac4bd-15c4-4e95-ab60-031050f63f7c tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5/os-volume_attachments/cd03ad33-8f4c-4084-8b36-72d1fe84c526" status: 202 len: 0 microversion: 2.1 time: 0.196018 Sep 27 10:28:52.146732 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2717/5891] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:28:51 2022] DELETE /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5/os-volume_attachments/cd03ad33-8f4c-4084-8b36-72d1fe84c526 => generated 0 bytes in 198 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:28:52.238518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-24b171e2-d6b9-4dd9-8615-8e9198a2de6c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: .version_select at 0x7f7f62392dc0>, body: {"keypair": {"name": "tempest-keypair-995570364", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ9JfbkZy47okHwvtPCFQpJV2vDxALfvfPlu9YcvGbSFb30KCMOXImiC3pTd99RfQUDJX/vhWlwuSqWF4TIO1Uo/7AzC8W3Fg47LGO3sscZ3f7QR0eM2KvY0RRTUZmlXcw=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:52.262966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-24b171e2-d6b9-4dd9-8615-8e9198a2de6c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for project 90c0fbb4a6d84767b920564584715f9d. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:52.268363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-24b171e2-d6b9-4dd9-8615-8e9198a2de6c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for user 64090c8fd3394980ad5409adcde4986f and project 90c0fbb4a6d84767b920564584715f9d. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:52.290833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1fa24fa6-dc74-4f4d-803c-56f4aa98da38 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:52.294037 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-24b171e2-d6b9-4dd9-8615-8e9198a2de6c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.072711 Sep 27 10:28:52.294827 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2718/5892] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:28:52 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:28:52.301594 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1fa24fa6-dc74-4f4d-803c-56f4aa98da38 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1313 microversion: 2.1 time: 0.215383 Sep 27 10:28:52.302183 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3175/5893] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:52 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1313 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:53.312556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f0b72559-5403-434d-a2ad-99bb444bada9 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:53.314030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f0b72559-5403-434d-a2ad-99bb444bada9 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:53.325296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b72559-5403-434d-a2ad-99bb444bada9 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:53.326242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b72559-5403-434d-a2ad-99bb444bada9 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:53.326928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5177a04d-fcbc-4490-89ca-6bbda6db682b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:53.327642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b72559-5403-434d-a2ad-99bb444bada9 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:53.328074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b72559-5403-434d-a2ad-99bb444bada9 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:53.328650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b72559-5403-434d-a2ad-99bb444bada9 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:53.329297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0b72559-5403-434d-a2ad-99bb444bada9 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:53.329791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5177a04d-fcbc-4490-89ca-6bbda6db682b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:53.341192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5177a04d-fcbc-4490-89ca-6bbda6db682b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:53.341774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5177a04d-fcbc-4490-89ca-6bbda6db682b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:53.342371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5177a04d-fcbc-4490-89ca-6bbda6db682b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:53.343466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5177a04d-fcbc-4490-89ca-6bbda6db682b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:53.368058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f0b72559-5403-434d-a2ad-99bb444bada9 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:28:53.389279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5177a04d-fcbc-4490-89ca-6bbda6db682b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.045s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:53.389988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5177a04d-fcbc-4490-89ca-6bbda6db682b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:53.464499 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f0b72559-5403-434d-a2ad-99bb444bada9 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 204 len: 0 microversion: 2.1 time: 0.156626 Sep 27 10:28:53.473848 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2719/5894] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:28:53 2022] DELETE /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:28:53.484066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-30cb347a-8b18-4753-a592-5ffa01164a76 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:53.484816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-30cb347a-8b18-4753-a592-5ffa01164a76 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:53.491210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30cb347a-8b18-4753-a592-5ffa01164a76 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:53.492428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30cb347a-8b18-4753-a592-5ffa01164a76 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:53.492921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30cb347a-8b18-4753-a592-5ffa01164a76 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:53.493643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30cb347a-8b18-4753-a592-5ffa01164a76 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:53.494111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30cb347a-8b18-4753-a592-5ffa01164a76 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:53.494549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30cb347a-8b18-4753-a592-5ffa01164a76 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:53.550577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5177a04d-fcbc-4490-89ca-6bbda6db682b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:53.559088 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5177a04d-fcbc-4490-89ca-6bbda6db682b tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1313 microversion: 2.1 time: 0.238496 Sep 27 10:28:53.559940 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3176/5895] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:53 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1313 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:53.650961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-30cb347a-8b18-4753-a592-5ffa01164a76 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:53.659191 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-30cb347a-8b18-4753-a592-5ffa01164a76 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1512 microversion: 2.1 time: 0.180143 Sep 27 10:28:53.659768 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2720/5896] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:53 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1512 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:53.711315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1309045928", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "6a85e0d6-e759-43a8-9abb-8e12abcca6c4"}], "security_groups": [{"name": "tempest-securitygroup--44707128"}], "key_name": "tempest-keypair-995570364", "user_data": "}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:53.818141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] validate_networks() for [('6a85e0d6-e759-43a8-9abb-8e12abcca6c4', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:28:54.559620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:28:54.560198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:28:54.560532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:28:54.560962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:28:54.561379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:28:54.571782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:54.572264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:54.572733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:54.573472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:54.573913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:54.574339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:54.582934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f7c6cf0d-adf2-4b0a-b0d1-5780d396d49c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:54.584507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f7c6cf0d-adf2-4b0a-b0d1-5780d396d49c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:54.592597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7c6cf0d-adf2-4b0a-b0d1-5780d396d49c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:54.593142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for project 90c0fbb4a6d84767b920564584715f9d. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:28:54.593391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7c6cf0d-adf2-4b0a-b0d1-5780d396d49c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:54.593835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7c6cf0d-adf2-4b0a-b0d1-5780d396d49c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:54.594559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7c6cf0d-adf2-4b0a-b0d1-5780d396d49c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:54.595394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7c6cf0d-adf2-4b0a-b0d1-5780d396d49c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:54.596257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f7c6cf0d-adf2-4b0a-b0d1-5780d396d49c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:54.600126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for user 64090c8fd3394980ad5409adcde4986f and project 90c0fbb4a6d84767b920564584715f9d. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:28:54.618605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:28:54.621193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:28:54.673053 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b54eea5e-7abd-4f23-a02d-d336747337f9 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.965923 Sep 27 10:28:54.680870 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3177/5897] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:28:53 2022] POST /compute/v2.1/servers => generated 400 bytes in 974 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:28:54.685909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-afb78eb5-b999-4d28-b850-970189dff273 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:54.686531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-afb78eb5-b999-4d28-b850-970189dff273 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:54.741891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f7c6cf0d-adf2-4b0a-b0d1-5780d396d49c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:54.749106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-afb78eb5-b999-4d28-b850-970189dff273 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:54.753781 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f7c6cf0d-adf2-4b0a-b0d1-5780d396d49c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1313 microversion: 2.1 time: 0.174793 Sep 27 10:28:54.754695 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2721/5898] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:54 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1313 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:54.756421 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-afb78eb5-b999-4d28-b850-970189dff273 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1241 microversion: 2.1 time: 0.074410 Sep 27 10:28:54.757169 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3178/5899] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:54 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1241 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:54.759387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b54c5df7-62e6-4fff-991d-c5e24fe5853f tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:54.760152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b54c5df7-62e6-4fff-991d-c5e24fe5853f tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:54.766362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54c5df7-62e6-4fff-991d-c5e24fe5853f tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:54.766974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54c5df7-62e6-4fff-991d-c5e24fe5853f tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:54.767550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54c5df7-62e6-4fff-991d-c5e24fe5853f tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:54.768373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54c5df7-62e6-4fff-991d-c5e24fe5853f tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:54.768884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54c5df7-62e6-4fff-991d-c5e24fe5853f tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:54.769330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54c5df7-62e6-4fff-991d-c5e24fe5853f tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:54.916376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b54c5df7-62e6-4fff-991d-c5e24fe5853f tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:54.925254 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b54c5df7-62e6-4fff-991d-c5e24fe5853f tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1512 microversion: 2.1 time: 0.169700 Sep 27 10:28:54.925840 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2722/5900] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:54 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1512 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:55.454490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-447ac22b-af50-4a80-bc40-46d9c096a5f5 req-4c15f956-28b8-4f07-badd-2226dc1f6e75 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9d4f13c2-3c04-4984-a293-df05378d29e5", "name": "network-vif-deleted", "tag": "2c5322b1-7e1a-4d84-9b57-45c942d8b0e0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:55.476019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-447ac22b-af50-4a80-bc40-46d9c096a5f5 req-4c15f956-28b8-4f07-badd-2226dc1f6e75 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:55.476506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-447ac22b-af50-4a80-bc40-46d9c096a5f5 req-4c15f956-28b8-4f07-badd-2226dc1f6e75 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:55.477155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-447ac22b-af50-4a80-bc40-46d9c096a5f5 req-4c15f956-28b8-4f07-badd-2226dc1f6e75 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:55.494065 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-447ac22b-af50-4a80-bc40-46d9c096a5f5 req-4c15f956-28b8-4f07-badd-2226dc1f6e75 service nova] Creating event network-vif-deleted:2c5322b1-7e1a-4d84-9b57-45c942d8b0e0 for instance 9d4f13c2-3c04-4984-a293-df05378d29e5 on nch1-master-1 Sep 27 10:28:55.500374 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-447ac22b-af50-4a80-bc40-46d9c096a5f5 req-4c15f956-28b8-4f07-badd-2226dc1f6e75 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049373 Sep 27 10:28:55.501275 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3179/5901] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:55.786644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32cccfc1-7358-4803-baf6-713c3254610d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:55.787252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32cccfc1-7358-4803-baf6-713c3254610d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:55.790265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cf947c99-d253-46dc-bb86-88b6471191c6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:55.791025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cf947c99-d253-46dc-bb86-88b6471191c6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:55.794920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32cccfc1-7358-4803-baf6-713c3254610d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:55.795428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32cccfc1-7358-4803-baf6-713c3254610d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:55.795883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32cccfc1-7358-4803-baf6-713c3254610d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:55.796640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32cccfc1-7358-4803-baf6-713c3254610d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:55.797147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32cccfc1-7358-4803-baf6-713c3254610d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:55.797588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32cccfc1-7358-4803-baf6-713c3254610d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:55.798144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf947c99-d253-46dc-bb86-88b6471191c6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:55.798780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf947c99-d253-46dc-bb86-88b6471191c6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:55.799280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf947c99-d253-46dc-bb86-88b6471191c6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:55.800056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf947c99-d253-46dc-bb86-88b6471191c6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:55.800760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf947c99-d253-46dc-bb86-88b6471191c6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:55.801282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf947c99-d253-46dc-bb86-88b6471191c6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:55.875763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cf947c99-d253-46dc-bb86-88b6471191c6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:55.887780 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cf947c99-d253-46dc-bb86-88b6471191c6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1293 microversion: 2.1 time: 0.100802 Sep 27 10:28:55.889096 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3180/5902] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:55 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1293 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:55.951925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:55.952557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:55.961752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:55.962253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:55.962699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:55.963429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:55.963889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:55.967188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:56.012651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:56.013697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:56.014323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:56.014996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lazy-loading 'fault' on Instance uuid 9d4f13c2-3c04-4984-a293-df05378d29e5 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:28:56.062530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '584c395757a54d9b9a64dd399d8557b1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '712c1c4c858b4b788a7703345716504b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:56.073951 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6242da78-b6a4-4e8d-b588-4da4d8d2da34 tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 200 len: 1308 microversion: 2.1 time: 0.127647 Sep 27 10:28:56.091631 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3181/5903] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:55 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 1308 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:28:56.250163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-63619ed9-5897-4769-aa2d-2493fe276487 req-d1ca77b3-a9bb-4ec5-8354-8b408befd7cf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "87a295a8-60b5-4213-bdb8-a41e36e3adb5", "name": "network-vif-plugged", "status": "completed", "tag": "32374fc0-593b-45d3-8f7f-5d997a06028f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:56.264698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-63619ed9-5897-4769-aa2d-2493fe276487 req-d1ca77b3-a9bb-4ec5-8354-8b408befd7cf service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:56.266842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-63619ed9-5897-4769-aa2d-2493fe276487 req-d1ca77b3-a9bb-4ec5-8354-8b408befd7cf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:56.267697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-63619ed9-5897-4769-aa2d-2493fe276487 req-d1ca77b3-a9bb-4ec5-8354-8b408befd7cf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:56.281819 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-63619ed9-5897-4769-aa2d-2493fe276487 req-d1ca77b3-a9bb-4ec5-8354-8b408befd7cf service nova] Creating event network-vif-plugged:32374fc0-593b-45d3-8f7f-5d997a06028f for instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 on nch1-master-1 Sep 27 10:28:56.287371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-32cccfc1-7358-4803-baf6-713c3254610d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:56.288357 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-63619ed9-5897-4769-aa2d-2493fe276487 req-d1ca77b3-a9bb-4ec5-8354-8b408befd7cf service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042475 Sep 27 10:28:56.289581 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3182/5904] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:56.297004 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32cccfc1-7358-4803-baf6-713c3254610d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1313 microversion: 2.1 time: 0.514791 Sep 27 10:28:56.301534 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2723/5905] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:55 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1313 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:56.914300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6fc09cbc-7e75-46b9-9524-2b34a8d81720 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:56.915397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6fc09cbc-7e75-46b9-9524-2b34a8d81720 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:56.922070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fc09cbc-7e75-46b9-9524-2b34a8d81720 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:56.922653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fc09cbc-7e75-46b9-9524-2b34a8d81720 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:56.923176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fc09cbc-7e75-46b9-9524-2b34a8d81720 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:56.924020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fc09cbc-7e75-46b9-9524-2b34a8d81720 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:56.924698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fc09cbc-7e75-46b9-9524-2b34a8d81720 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:56.925206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6fc09cbc-7e75-46b9-9524-2b34a8d81720 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:57.075800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6fc09cbc-7e75-46b9-9524-2b34a8d81720 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:57.085224 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6fc09cbc-7e75-46b9-9524-2b34a8d81720 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1365 microversion: 2.1 time: 0.176205 Sep 27 10:28:57.086071 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3183/5906] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:56 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1365 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:57.099665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:57.100424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] [instance: 9d4f13c2-3c04-4984-a293-df05378d29e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:57.108017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:57.108586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:57.109379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:57.109785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:57.110250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:57.110683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:57.115378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:57.126191 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] HTTP exception thrown: Instance 9d4f13c2-3c04-4984-a293-df05378d29e5 could not be found. Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Request method failure captured: Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 HTTP/1.1 Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs9MjI_0KcnHJXSGLHBgvPUwQslys5FrzbZdDzR4LHS5P20obfvDE55uJC2bWHinkoUYLZCvPzz-e9beg48lQaNqfrNLKsX3K3Ey8lJ8snaERSFExgnBV6ls2CawRaQyxkeK5OON4LxIbGkC2jNntAUgLFOke0b-3XrL-UkjH5k7RRo Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 712c1c4c858b4b788a7703345716504b Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1705236581 Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/712c1c4c858b4b788a7703345716504b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/712c1c4c858b4b788a7703345716504b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/712c1c4c858b4b788a7703345716504b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_712c1c4c858b4b788a7703345716504b"}]}] Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-VolumesSnapshotTestJSON-1705236581 Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 712c1c4c858b4b788a7703345716504b Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1705236581 Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-VolumesSnapshotTestJSON-1705236581-project Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 584c395757a54d9b9a64dd399d8557b1 Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-VolumesSnapshotTestJSON-1705236581-project Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 9d4f13c2-3c04-4984-a293-df05378d29e5 could not be found. Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '9d4f13c2-3c04-4984-a293-df05378d29e5'} Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9d4f13c2-3c04-4984-a293-df05378d29e5 could not be found. Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9d4f13c2-3c04-4984-a293-df05378d29e5 could not be found. Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:28:57.127955 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:57.134460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:28:57.134460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:28:57.134460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:28:57.134460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:28:57.134460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:28:57.134460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9d4f13c2-3c04-4984-a293-df05378d29e5 could not be found. Sep 27 10:28:57.134460 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:28:57.134460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] Returning 404 to user: Instance 9d4f13c2-3c04-4984-a293-df05378d29e5 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:28:57.134460 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3a79af04-8c2e-4c96-8755-9dba925855ac tempest-VolumesSnapshotTestJSON-1705236581 tempest-VolumesSnapshotTestJSON-1705236581-project] 10.222.0.26 "GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5" status: 404 len: 111 microversion: 2.1 time: 0.034464 Sep 27 10:28:57.134460 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2724/5907] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:57 2022] GET /compute/v2.1/servers/9d4f13c2-3c04-4984-a293-df05378d29e5 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:28:57.324622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9c672c08-92d7-42d3-98d7-f9a51cea7db2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:57.325370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9c672c08-92d7-42d3-98d7-f9a51cea7db2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:57.331856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c672c08-92d7-42d3-98d7-f9a51cea7db2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:57.332754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c672c08-92d7-42d3-98d7-f9a51cea7db2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:57.333199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c672c08-92d7-42d3-98d7-f9a51cea7db2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:57.333911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c672c08-92d7-42d3-98d7-f9a51cea7db2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:57.334548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c672c08-92d7-42d3-98d7-f9a51cea7db2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:57.334976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c672c08-92d7-42d3-98d7-f9a51cea7db2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:57.524487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9c672c08-92d7-42d3-98d7-f9a51cea7db2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:57.533221 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9c672c08-92d7-42d3-98d7-f9a51cea7db2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1313 microversion: 2.1 time: 0.213461 Sep 27 10:28:57.545140 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3184/5908] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:57 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1313 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:57.833361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-cfc695c7-e85c-4617-86a0-d108cb4c5f71 req-37335f90-0c40-4f9d-b64f-397083b34f5a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d5319326-fc6f-4d93-85ae-98a8b422dc14", "tag": "ea0cb073-0c12-471c-b08c-f88ce79e8500"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:28:57.847122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cfc695c7-e85c-4617-86a0-d108cb4c5f71 req-37335f90-0c40-4f9d-b64f-397083b34f5a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:57.847979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cfc695c7-e85c-4617-86a0-d108cb4c5f71 req-37335f90-0c40-4f9d-b64f-397083b34f5a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:57.848435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cfc695c7-e85c-4617-86a0-d108cb4c5f71 req-37335f90-0c40-4f9d-b64f-397083b34f5a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:57.860109 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-cfc695c7-e85c-4617-86a0-d108cb4c5f71 req-37335f90-0c40-4f9d-b64f-397083b34f5a service nova] Creating event network-changed:ea0cb073-0c12-471c-b08c-f88ce79e8500 for instance d5319326-fc6f-4d93-85ae-98a8b422dc14 on nch2-master-1 Sep 27 10:28:57.865225 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-cfc695c7-e85c-4617-86a0-d108cb4c5f71 req-37335f90-0c40-4f9d-b64f-397083b34f5a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036721 Sep 27 10:28:57.866197 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2725/5909] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:28:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:58.111665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-315a26a3-aeaa-4628-945a-9ca170a61d54 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:58.112567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-315a26a3-aeaa-4628-945a-9ca170a61d54 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:58.119788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-315a26a3-aeaa-4628-945a-9ca170a61d54 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:58.120285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-315a26a3-aeaa-4628-945a-9ca170a61d54 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:58.120726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-315a26a3-aeaa-4628-945a-9ca170a61d54 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:58.121455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-315a26a3-aeaa-4628-945a-9ca170a61d54 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:58.121919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-315a26a3-aeaa-4628-945a-9ca170a61d54 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:58.122407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-315a26a3-aeaa-4628-945a-9ca170a61d54 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:58.268867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-315a26a3-aeaa-4628-945a-9ca170a61d54 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:58.277206 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-315a26a3-aeaa-4628-945a-9ca170a61d54 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1365 microversion: 2.1 time: 0.169627 Sep 27 10:28:58.277767 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3185/5910] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:58 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1365 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:58.559531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5b48377-7b6c-4ad1-8783-2a7f58fd903d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:58.561191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5b48377-7b6c-4ad1-8783-2a7f58fd903d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:58.570895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5b48377-7b6c-4ad1-8783-2a7f58fd903d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:58.571459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5b48377-7b6c-4ad1-8783-2a7f58fd903d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:58.572465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5b48377-7b6c-4ad1-8783-2a7f58fd903d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:58.573412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5b48377-7b6c-4ad1-8783-2a7f58fd903d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:58.573933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5b48377-7b6c-4ad1-8783-2a7f58fd903d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:58.574428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5b48377-7b6c-4ad1-8783-2a7f58fd903d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:58.723563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5b48377-7b6c-4ad1-8783-2a7f58fd903d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:58.731866 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5b48377-7b6c-4ad1-8783-2a7f58fd903d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1498 microversion: 2.1 time: 0.178366 Sep 27 10:28:58.732520 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2726/5911] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:58 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1498 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:58.755081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6df09d08-3bba-4d0e-91bb-6048b9de9d98 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:58.770227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.compute.server_groups [None req-6df09d08-3bba-4d0e-91bb-6048b9de9d98 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Querying cell ed1088da-caae-4426-8048-9cba90e7755c(cell1) for 1 instances {{(pid=145968) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Sep 27 10:28:58.770984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6df09d08-3bba-4d0e-91bb-6048b9de9d98 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:58.771463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6df09d08-3bba-4d0e-91bb-6048b9de9d98 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:58.771898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6df09d08-3bba-4d0e-91bb-6048b9de9d98 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:58.790063 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6df09d08-3bba-4d0e-91bb-6048b9de9d98 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/os-server-groups/218a0d89-d6a7-4adb-9e51-3cb6ed379476" status: 200 len: 222 microversion: 2.1 time: 0.038884 Sep 27 10:28:58.790710 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3186/5912] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:28:58 2022] GET /compute/v2.1/os-server-groups/218a0d89-d6a7-4adb-9e51-3cb6ed379476 => generated 222 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:28:58.803696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-41e3bb53-6aee-4e3f-92f3-07be951b1390 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:58.804216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-41e3bb53-6aee-4e3f-92f3-07be951b1390 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:58.810612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41e3bb53-6aee-4e3f-92f3-07be951b1390 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:58.811117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41e3bb53-6aee-4e3f-92f3-07be951b1390 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:58.811572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41e3bb53-6aee-4e3f-92f3-07be951b1390 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:58.812356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41e3bb53-6aee-4e3f-92f3-07be951b1390 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:58.812864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41e3bb53-6aee-4e3f-92f3-07be951b1390 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:58.813309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41e3bb53-6aee-4e3f-92f3-07be951b1390 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:58.838901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-41e3bb53-6aee-4e3f-92f3-07be951b1390 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:28:58.919310 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-41e3bb53-6aee-4e3f-92f3-07be951b1390 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "DELETE /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 204 len: 0 microversion: 2.1 time: 0.119322 Sep 27 10:28:58.919996 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2727/5913] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:28:58 2022] DELETE /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:28:58.929552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d43c1b46-9731-4b25-a40c-9cf3e1c98965 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:58.930294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d43c1b46-9731-4b25-a40c-9cf3e1c98965 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:58.936716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43c1b46-9731-4b25-a40c-9cf3e1c98965 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:58.937222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43c1b46-9731-4b25-a40c-9cf3e1c98965 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:58.937680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43c1b46-9731-4b25-a40c-9cf3e1c98965 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:58.938378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43c1b46-9731-4b25-a40c-9cf3e1c98965 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:58.938844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43c1b46-9731-4b25-a40c-9cf3e1c98965 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:58.939571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d43c1b46-9731-4b25-a40c-9cf3e1c98965 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:59.080578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d43c1b46-9731-4b25-a40c-9cf3e1c98965 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:59.088900 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d43c1b46-9731-4b25-a40c-9cf3e1c98965 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1504 microversion: 2.1 time: 0.163291 Sep 27 10:28:59.089852 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3187/5914] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:58 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1504 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:28:59.302609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2962eca7-c1b5-4a57-b76f-9d88d6ba6916 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:28:59.303555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2962eca7-c1b5-4a57-b76f-9d88d6ba6916 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:28:59.311923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2962eca7-c1b5-4a57-b76f-9d88d6ba6916 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:59.312584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2962eca7-c1b5-4a57-b76f-9d88d6ba6916 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:59.313263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2962eca7-c1b5-4a57-b76f-9d88d6ba6916 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:59.314348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2962eca7-c1b5-4a57-b76f-9d88d6ba6916 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:28:59.315107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2962eca7-c1b5-4a57-b76f-9d88d6ba6916 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:28:59.315546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2962eca7-c1b5-4a57-b76f-9d88d6ba6916 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:28:59.464076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2962eca7-c1b5-4a57-b76f-9d88d6ba6916 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:28:59.469696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=4,nova_cell1:INSERT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:28:59.474899 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2962eca7-c1b5-4a57-b76f-9d88d6ba6916 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1365 microversion: 2.1 time: 0.178190 Sep 27 10:28:59.475461 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2728/5915] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:28:59 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1365 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:00.116707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5febede6-da78-476a-be75-748946021985 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:00.117523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5febede6-da78-476a-be75-748946021985 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:00.124732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5febede6-da78-476a-be75-748946021985 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:00.125370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5febede6-da78-476a-be75-748946021985 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:00.125815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5febede6-da78-476a-be75-748946021985 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:00.126644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5febede6-da78-476a-be75-748946021985 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:00.127798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5febede6-da78-476a-be75-748946021985 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:00.128236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5febede6-da78-476a-be75-748946021985 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:00.278479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5febede6-da78-476a-be75-748946021985 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:00.287181 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5febede6-da78-476a-be75-748946021985 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1504 microversion: 2.1 time: 0.177771 Sep 27 10:29:00.288351 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3188/5916] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:00 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1504 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:00.499610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b4786dba-9683-4f01-8e60-9ccde2604e3b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:00.500243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b4786dba-9683-4f01-8e60-9ccde2604e3b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:00.513256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4786dba-9683-4f01-8e60-9ccde2604e3b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:00.513764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4786dba-9683-4f01-8e60-9ccde2604e3b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:00.514297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4786dba-9683-4f01-8e60-9ccde2604e3b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:00.514941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4786dba-9683-4f01-8e60-9ccde2604e3b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:00.515539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4786dba-9683-4f01-8e60-9ccde2604e3b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:00.515977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4786dba-9683-4f01-8e60-9ccde2604e3b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:00.717528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b4786dba-9683-4f01-8e60-9ccde2604e3b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:00.726125 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b4786dba-9683-4f01-8e60-9ccde2604e3b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1365 microversion: 2.1 time: 0.231051 Sep 27 10:29:00.726700 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2729/5917] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:00 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1365 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:00.828181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f69888b3-e183-422a-8e6d-6874de86d642 req-0df0a7f7-765b-446e-9c68-8015ebcc4d94 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "87a295a8-60b5-4213-bdb8-a41e36e3adb5", "name": "network-vif-deleted", "tag": "32374fc0-593b-45d3-8f7f-5d997a06028f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:00.838829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f69888b3-e183-422a-8e6d-6874de86d642 req-0df0a7f7-765b-446e-9c68-8015ebcc4d94 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:00.839843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f69888b3-e183-422a-8e6d-6874de86d642 req-0df0a7f7-765b-446e-9c68-8015ebcc4d94 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:00.840219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f69888b3-e183-422a-8e6d-6874de86d642 req-0df0a7f7-765b-446e-9c68-8015ebcc4d94 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:00.853754 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f69888b3-e183-422a-8e6d-6874de86d642 req-0df0a7f7-765b-446e-9c68-8015ebcc4d94 service nova] Creating event network-vif-deleted:32374fc0-593b-45d3-8f7f-5d997a06028f for instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 on nch1-master-1 Sep 27 10:29:00.860567 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f69888b3-e183-422a-8e6d-6874de86d642 req-0df0a7f7-765b-446e-9c68-8015ebcc4d94 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035977 Sep 27 10:29:00.861819 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3189/5918] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:01.311441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:01.312345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:01.320011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:01.321849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:01.322575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:01.324018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:01.325012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:01.325936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:01.369451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:01.370328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:01.371261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:01.371961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lazy-loading 'fault' on Instance uuid 87a295a8-60b5-4213-bdb8-a41e36e3adb5 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:29:01.429964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '91b11c3c34f8402390e4ee61b6c1bc8f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2c5bb6bd7e434b25992308dca2593154', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:01.440943 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-76b2dafc-4829-4a55-9043-5c11553e80c4 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 200 len: 1303 microversion: 2.1 time: 0.133406 Sep 27 10:29:01.442722 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2730/5919] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:01 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 1303 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:01.750221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c96721f-4952-44fc-8014-08c112422399 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:01.750941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5c96721f-4952-44fc-8014-08c112422399 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:01.757493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c96721f-4952-44fc-8014-08c112422399 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:01.757990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c96721f-4952-44fc-8014-08c112422399 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:01.758437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c96721f-4952-44fc-8014-08c112422399 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:01.759173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c96721f-4952-44fc-8014-08c112422399 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:01.759662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c96721f-4952-44fc-8014-08c112422399 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:01.760059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c96721f-4952-44fc-8014-08c112422399 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:01.915064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c96721f-4952-44fc-8014-08c112422399 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:01.926065 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c96721f-4952-44fc-8014-08c112422399 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1365 microversion: 2.1 time: 0.181140 Sep 27 10:29:01.926690 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3190/5920] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:01 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1365 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:01.988026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f94ba56d-01e1-43c8-8095-3dab9c39c92b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:01.990400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f94ba56d-01e1-43c8-8095-3dab9c39c92b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:01.997952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ba56d-01e1-43c8-8095-3dab9c39c92b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:01.998450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ba56d-01e1-43c8-8095-3dab9c39c92b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:01.998915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ba56d-01e1-43c8-8095-3dab9c39c92b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:01.999641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ba56d-01e1-43c8-8095-3dab9c39c92b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:02.000122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ba56d-01e1-43c8-8095-3dab9c39c92b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:02.000564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f94ba56d-01e1-43c8-8095-3dab9c39c92b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:02.070965 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f94ba56d-01e1-43c8-8095-3dab9c39c92b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3/action" status: 200 len: 24568 microversion: 2.1 time: 0.087455 Sep 27 10:29:02.072152 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2731/5921] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:29:01 2022] POST /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3/action => generated 24568 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:29:02.467159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:02.467919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:02.475347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:02.475937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:02.476468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:02.477298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:02.477871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:02.478372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:02.493040 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] HTTP exception thrown: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Request method failure captured: Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 HTTP/1.1 Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtBdTRUW8grABlUet5agHJGRJyv7SkZCIPF4IlSLQq-Eaolewq10d1sFPnbu62zO7VCt_oiLbdboT9b5bSx1YJ0MshEY-25DBpQoNR4RQoaySCoWejuHdff4ww-NWhmXmNIPwsXuZGikf8RfBWXazQqVpu1Fb5QdId32BUyTW6w3mTk Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2c5bb6bd7e434b25992308dca2593154"}]}] Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 91b11c3c34f8402390e4ee61b6c1bc8f Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '87a295a8-60b5-4213-bdb8-a41e36e3adb5'} Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.494705 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.503003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.503003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.503003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.503003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:29:02.503003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:29:02.503003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.503003 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.503003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Returning 404 to user: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:29:02.503003 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-acf02ffd-1ab3-44f5-9d6f-930c53d8e92a tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 404 len: 111 microversion: 2.1 time: 0.033283 Sep 27 10:29:02.503003 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3191/5922] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:02 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:29:02.509697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c68b0228-20de-4878-91c7-316b7cd12d10 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:02.515361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=34 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:02.530317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.compute.server_groups [None req-c68b0228-20de-4878-91c7-316b7cd12d10 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Querying cell ed1088da-caae-4426-8048-9cba90e7755c(cell1) for 1 instances {{(pid=145969) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Sep 27 10:29:02.531186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c68b0228-20de-4878-91c7-316b7cd12d10 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:02.531904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c68b0228-20de-4878-91c7-316b7cd12d10 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:02.532450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c68b0228-20de-4878-91c7-316b7cd12d10 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:02.543131 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c68b0228-20de-4878-91c7-316b7cd12d10 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/os-server-groups" status: 200 len: 187 microversion: 2.1 time: 0.037621 Sep 27 10:29:02.543808 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2732/5923] 10.222.0.26 () {58 vars in 1152 bytes} [Tue Sep 27 10:29:02 2022] GET /compute/v2.1/os-server-groups => generated 187 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:02.555583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-00a84950-5e00-4380-a73a-14b25a1bf2fd tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:02.568751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.compute.server_groups [None req-00a84950-5e00-4380-a73a-14b25a1bf2fd tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Querying cell ed1088da-caae-4426-8048-9cba90e7755c(cell1) for 1 instances {{(pid=145968) _get_not_deleted /opt/stack/nova/nova/api/openstack/compute/server_groups.py:69}} Sep 27 10:29:02.569476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00a84950-5e00-4380-a73a-14b25a1bf2fd tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:02.569957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00a84950-5e00-4380-a73a-14b25a1bf2fd tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:02.570396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00a84950-5e00-4380-a73a-14b25a1bf2fd tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:02.574861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:02.581648 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-00a84950-5e00-4380-a73a-14b25a1bf2fd tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/os-server-groups/218a0d89-d6a7-4adb-9e51-3cb6ed379476" status: 200 len: 184 microversion: 2.1 time: 0.029923 Sep 27 10:29:02.582305 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3192/5924] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:29:02 2022] GET /compute/v2.1/os-server-groups/218a0d89-d6a7-4adb-9e51-3cb6ed379476 => generated 184 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:02.594588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:02.595523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:02.602789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:02.603292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:02.603747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:02.604467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:02.604991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:02.605433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:02.620113 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] HTTP exception thrown: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Request method failure captured: Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 HTTP/1.1 Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtBdTRUW8grABlUet5agHJGRJyv7SkZCIPF4IlSLQq-Eaolewq10d1sFPnbu62zO7VCt_oiLbdboT9b5bSx1YJ0MshEY-25DBpQoNR4RQoaySCoWejuHdff4ww-NWhmXmNIPwsXuZGikf8RfBWXazQqVpu1Fb5QdId32BUyTW6w3mTk Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2c5bb6bd7e434b25992308dca2593154"}]}] Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 91b11c3c34f8402390e4ee61b6c1bc8f Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '87a295a8-60b5-4213-bdb8-a41e36e3adb5'} Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:29:02.621852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Returning 404 to user: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9aa0b3e9-c8a4-4a96-8080-8457dd06980c tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "DELETE /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 404 len: 111 microversion: 2.1 time: 0.032917 Sep 27 10:29:02.627504 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2733/5925] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:29:02 2022] DELETE /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:29:02.637868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:02.638516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] [instance: 87a295a8-60b5-4213-bdb8-a41e36e3adb5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:02.645254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:02.645741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:02.646179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:02.646879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:02.647333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:02.647760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:02.661789 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] HTTP exception thrown: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Request method failure captured: Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 HTTP/1.1 Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtBdTRUW8grABlUet5agHJGRJyv7SkZCIPF4IlSLQq-Eaolewq10d1sFPnbu62zO7VCt_oiLbdboT9b5bSx1YJ0MshEY-25DBpQoNR4RQoaySCoWejuHdff4ww-NWhmXmNIPwsXuZGikf8RfBWXazQqVpu1Fb5QdId32BUyTW6w3mTk Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2c5bb6bd7e434b25992308dca2593154"}]}] Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 91b11c3c34f8402390e4ee61b6c1bc8f Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '87a295a8-60b5-4213-bdb8-a41e36e3adb5'} Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.663559 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.669510 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.669510 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.669510 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.669510 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:29:02.669510 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:29:02.669510 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. Sep 27 10:29:02.669510 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.669510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Returning 404 to user: Instance 87a295a8-60b5-4213-bdb8-a41e36e3adb5 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:29:02.669510 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-34ed2d36-d93d-49c1-ad06-3ac7f256b76d tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5" status: 404 len: 111 microversion: 2.1 time: 0.031839 Sep 27 10:29:02.669510 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3193/5926] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:02 2022] GET /compute/v2.1/servers/87a295a8-60b5-4213-bdb8-a41e36e3adb5 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d577fd2-3092-4ff5-ac8a-0b542ed4350f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-4d577fd2-3092-4ff5-ac8a-0b542ed4350f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] HTTP exception thrown: Instance group 67432b7d-d3ef-4407-bafe-d67a2ac0868b could not be found. Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d577fd2-3092-4ff5-ac8a-0b542ed4350f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Request method failure captured: Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/os-server-groups/67432b7d-d3ef-4407-bafe-d67a2ac0868b HTTP/1.1 Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtBdTRUW8grABlUet5agHJGRJyv7SkZCIPF4IlSLQq-Eaolewq10d1sFPnbu62zO7VCt_oiLbdboT9b5bSx1YJ0MshEY-25DBpQoNR4RQoaySCoWejuHdff4ww-NWhmXmNIPwsXuZGikf8RfBWXazQqVpu1Fb5QdId32BUyTW6w3mTk Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2c5bb6bd7e434b25992308dca2593154"}]}] Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 91b11c3c34f8402390e4ee61b6c1bc8f Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance group 67432b7d-d3ef-4407-bafe-d67a2ac0868b could not be found. Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '67432b7d-d3ef-4407-bafe-d67a2ac0868b'} Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group 67432b7d-d3ef-4407-bafe-d67a2ac0868b could not be found. Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:29:02.685235 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group 67432b7d-d3ef-4407-bafe-d67a2ac0868b could not be found. Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group 67432b7d-d3ef-4407-bafe-d67a2ac0868b could not be found. Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d577fd2-3092-4ff5-ac8a-0b542ed4350f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Returning 404 to user: Instance group 67432b7d-d3ef-4407-bafe-d67a2ac0868b could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4d577fd2-3092-4ff5-ac8a-0b542ed4350f tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/67432b7d-d3ef-4407-bafe-d67a2ac0868b" status: 404 len: 117 microversion: 2.1 time: 0.015000 Sep 27 10:29:02.696030 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2734/5927] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:29:02 2022] DELETE /compute/v2.1/os-server-groups/67432b7d-d3ef-4407-bafe-d67a2ac0868b => generated 117 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:29:02.702409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90be9b04-b7ec-48d6-835a-c35d49c7b8aa tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:02.710918 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-90be9b04-b7ec-48d6-835a-c35d49c7b8aa tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] HTTP exception thrown: Instance group 483d15c5-e552-4c69-ba11-66f1aab314bb could not be found. Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90be9b04-b7ec-48d6-835a-c35d49c7b8aa tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Request method failure captured: Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-server-groups/483d15c5-e552-4c69-ba11-66f1aab314bb HTTP/1.1 Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtBdTRUW8grABlUet5agHJGRJyv7SkZCIPF4IlSLQq-Eaolewq10d1sFPnbu62zO7VCt_oiLbdboT9b5bSx1YJ0MshEY-25DBpQoNR4RQoaySCoWejuHdff4ww-NWhmXmNIPwsXuZGikf8RfBWXazQqVpu1Fb5QdId32BUyTW6w3mTk Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2c5bb6bd7e434b25992308dca2593154"}]}] Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 91b11c3c34f8402390e4ee61b6c1bc8f Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance group 483d15c5-e552-4c69-ba11-66f1aab314bb could not be found. Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '483d15c5-e552-4c69-ba11-66f1aab314bb'} Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group 483d15c5-e552-4c69-ba11-66f1aab314bb could not be found. Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group 483d15c5-e552-4c69-ba11-66f1aab314bb could not be found. Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.725449 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.732244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.732244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.732244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:29:02.732244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:29:02.732244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group 483d15c5-e552-4c69-ba11-66f1aab314bb could not be found. Sep 27 10:29:02.732244 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.732244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90be9b04-b7ec-48d6-835a-c35d49c7b8aa tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Returning 404 to user: Instance group 483d15c5-e552-4c69-ba11-66f1aab314bb could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:29:02.732244 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-90be9b04-b7ec-48d6-835a-c35d49c7b8aa tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/483d15c5-e552-4c69-ba11-66f1aab314bb" status: 404 len: 117 microversion: 2.1 time: 0.030795 Sep 27 10:29:02.732244 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3194/5928] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:29:02 2022] DELETE /compute/v2.1/os-server-groups/483d15c5-e552-4c69-ba11-66f1aab314bb => generated 117 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:29:02.736770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-738cae5a-09f8-4c7c-b8bb-06336f2b82a2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:02.746622 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-738cae5a-09f8-4c7c-b8bb-06336f2b82a2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] HTTP exception thrown: Instance group fc138075-c913-4390-a240-d4e6c7dbdc69 could not be found. Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-738cae5a-09f8-4c7c-b8bb-06336f2b82a2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Request method failure captured: Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/os-server-groups/fc138075-c913-4390-a240-d4e6c7dbdc69 HTTP/1.1 Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtBdTRUW8grABlUet5agHJGRJyv7SkZCIPF4IlSLQq-Eaolewq10d1sFPnbu62zO7VCt_oiLbdboT9b5bSx1YJ0MshEY-25DBpQoNR4RQoaySCoWejuHdff4ww-NWhmXmNIPwsXuZGikf8RfBWXazQqVpu1Fb5QdId32BUyTW6w3mTk Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2c5bb6bd7e434b25992308dca2593154"}]}] Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 91b11c3c34f8402390e4ee61b6c1bc8f Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance group fc138075-c913-4390-a240-d4e6c7dbdc69 could not be found. Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'fc138075-c913-4390-a240-d4e6c7dbdc69'} Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group fc138075-c913-4390-a240-d4e6c7dbdc69 could not be found. Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group fc138075-c913-4390-a240-d4e6c7dbdc69 could not be found. Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.748360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.753639 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.753639 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.753639 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:29:02.753639 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:29:02.753639 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group fc138075-c913-4390-a240-d4e6c7dbdc69 could not be found. Sep 27 10:29:02.753639 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.755157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-738cae5a-09f8-4c7c-b8bb-06336f2b82a2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Returning 404 to user: Instance group fc138075-c913-4390-a240-d4e6c7dbdc69 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:29:02.756248 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-738cae5a-09f8-4c7c-b8bb-06336f2b82a2 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/fc138075-c913-4390-a240-d4e6c7dbdc69" status: 404 len: 117 microversion: 2.1 time: 0.023486 Sep 27 10:29:02.759851 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2735/5929] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:29:02 2022] DELETE /compute/v2.1/os-server-groups/fc138075-c913-4390-a240-d4e6c7dbdc69 => generated 117 bytes in 25 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:29:02.765947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e6fa4873-1935-4854-9fbb-b3e9407a071e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:02.776424 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-e6fa4873-1935-4854-9fbb-b3e9407a071e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] HTTP exception thrown: Instance group 955305b7-bf42-4111-8186-5f9420d5eaef could not be found. Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e6fa4873-1935-4854-9fbb-b3e9407a071e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Request method failure captured: Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-server-groups/955305b7-bf42-4111-8186-5f9420d5eaef HTTP/1.1 Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtBdTRUW8grABlUet5agHJGRJyv7SkZCIPF4IlSLQq-Eaolewq10d1sFPnbu62zO7VCt_oiLbdboT9b5bSx1YJ0MshEY-25DBpQoNR4RQoaySCoWejuHdff4ww-NWhmXmNIPwsXuZGikf8RfBWXazQqVpu1Fb5QdId32BUyTW6w3mTk Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2c5bb6bd7e434b25992308dca2593154"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2c5bb6bd7e434b25992308dca2593154"}]}] Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 2c5bb6bd7e434b25992308dca2593154 Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerGroupTestJSON-1216714544 Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 91b11c3c34f8402390e4ee61b6c1bc8f Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerGroupTestJSON-1216714544-project Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance group 955305b7-bf42-4111-8186-5f9420d5eaef could not be found. Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '955305b7-bf42-4111-8186-5f9420d5eaef'} Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 139, in delete Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi sg = objects.InstanceGroup.get_by_uuid(context, id) Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 373, in get_by_uuid Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi db_group = cls._get_from_db_by_uuid(context, uuid) Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/instance_group.py", line 228, in _get_from_db_by_uuid Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceGroupNotFound(group_uuid=uuid) Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceGroupNotFound: Instance group 955305b7-bf42-4111-8186-5f9420d5eaef could not be found. Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/server_groups.py", line 141, in delete Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance group 955305b7-bf42-4111-8186-5f9420d5eaef could not be found. Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.778204 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:02.787136 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:02.787136 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:02.787136 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:29:02.787136 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:29:02.787136 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance group 955305b7-bf42-4111-8186-5f9420d5eaef could not be found. Sep 27 10:29:02.787136 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:02.787136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e6fa4873-1935-4854-9fbb-b3e9407a071e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Returning 404 to user: Instance group 955305b7-bf42-4111-8186-5f9420d5eaef could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:29:02.787136 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e6fa4873-1935-4854-9fbb-b3e9407a071e tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/955305b7-bf42-4111-8186-5f9420d5eaef" status: 404 len: 117 microversion: 2.1 time: 0.018596 Sep 27 10:29:02.787136 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3195/5930] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:29:02 2022] DELETE /compute/v2.1/os-server-groups/955305b7-bf42-4111-8186-5f9420d5eaef => generated 117 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:29:02.789433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-953fd9ae-08ed-451c-bdd0-69e478be4093 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:02.818377 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-953fd9ae-08ed-451c-bdd0-69e478be4093 tempest-ServerGroupTestJSON-1216714544 tempest-ServerGroupTestJSON-1216714544-project] 10.222.0.26 "DELETE /compute/v2.1/os-server-groups/218a0d89-d6a7-4adb-9e51-3cb6ed379476" status: 204 len: 0 microversion: 2.1 time: 0.033408 Sep 27 10:29:02.819515 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2736/5931] 10.222.0.26 () {58 vars in 1266 bytes} [Tue Sep 27 10:29:02 2022] DELETE /compute/v2.1/os-server-groups/218a0d89-d6a7-4adb-9e51-3cb6ed379476 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:29:02.958883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1e9ddabd-fa82-46c0-9ec2-cd0dcd912652 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:02.959727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1e9ddabd-fa82-46c0-9ec2-cd0dcd912652 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:02.967891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e9ddabd-fa82-46c0-9ec2-cd0dcd912652 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:02.968799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e9ddabd-fa82-46c0-9ec2-cd0dcd912652 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:02.969414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e9ddabd-fa82-46c0-9ec2-cd0dcd912652 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:02.970399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e9ddabd-fa82-46c0-9ec2-cd0dcd912652 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:02.971179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e9ddabd-fa82-46c0-9ec2-cd0dcd912652 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:02.971827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e9ddabd-fa82-46c0-9ec2-cd0dcd912652 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:03.110900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1e9ddabd-fa82-46c0-9ec2-cd0dcd912652 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:03.119012 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1e9ddabd-fa82-46c0-9ec2-cd0dcd912652 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1365 microversion: 2.1 time: 0.164806 Sep 27 10:29:03.119591 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3196/5932] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:02 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1365 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:29:03.504496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-79f658ce-ecc3-4760-b4d0-e44bfaccd38e req-56bd72a4-1aa3-4e0c-97f7-197c5a392ead service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d5319326-fc6f-4d93-85ae-98a8b422dc14", "name": "network-vif-plugged", "status": "completed", "tag": "ea0cb073-0c12-471c-b08c-f88ce79e8500"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:03.511767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=2,nova_api:UPDATE=2,nova_api:DELETE=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:03.517107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-79f658ce-ecc3-4760-b4d0-e44bfaccd38e req-56bd72a4-1aa3-4e0c-97f7-197c5a392ead service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:03.518207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-79f658ce-ecc3-4760-b4d0-e44bfaccd38e req-56bd72a4-1aa3-4e0c-97f7-197c5a392ead service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:03.518932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-79f658ce-ecc3-4760-b4d0-e44bfaccd38e req-56bd72a4-1aa3-4e0c-97f7-197c5a392ead service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:03.532293 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-79f658ce-ecc3-4760-b4d0-e44bfaccd38e req-56bd72a4-1aa3-4e0c-97f7-197c5a392ead service nova] Creating event network-vif-plugged:ea0cb073-0c12-471c-b08c-f88ce79e8500 for instance d5319326-fc6f-4d93-85ae-98a8b422dc14 on nch2-master-1 Sep 27 10:29:03.537962 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-79f658ce-ecc3-4760-b4d0-e44bfaccd38e req-56bd72a4-1aa3-4e0c-97f7-197c5a392ead service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039371 Sep 27 10:29:03.538808 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2737/5933] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:04.142644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c18b49dc-daf1-480e-9a1d-26149770bc9e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:04.143306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c18b49dc-daf1-480e-9a1d-26149770bc9e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:04.150644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18b49dc-daf1-480e-9a1d-26149770bc9e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:04.150968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18b49dc-daf1-480e-9a1d-26149770bc9e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:04.151435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18b49dc-daf1-480e-9a1d-26149770bc9e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:04.152127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18b49dc-daf1-480e-9a1d-26149770bc9e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:04.152609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18b49dc-daf1-480e-9a1d-26149770bc9e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:04.153090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c18b49dc-daf1-480e-9a1d-26149770bc9e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:04.305576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c18b49dc-daf1-480e-9a1d-26149770bc9e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:04.317083 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c18b49dc-daf1-480e-9a1d-26149770bc9e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1365 microversion: 2.1 time: 0.178434 Sep 27 10:29:04.318299 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3197/5934] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:04 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1365 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:05.344300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-313906e0-e6bf-4570-a975-5791e21490a6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:05.345846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-313906e0-e6bf-4570-a975-5791e21490a6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:05.355769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-313906e0-e6bf-4570-a975-5791e21490a6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:05.356648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-313906e0-e6bf-4570-a975-5791e21490a6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:05.357414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-313906e0-e6bf-4570-a975-5791e21490a6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:05.358455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-313906e0-e6bf-4570-a975-5791e21490a6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:05.359261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-313906e0-e6bf-4570-a975-5791e21490a6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:05.359974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-313906e0-e6bf-4570-a975-5791e21490a6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:05.744359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-313906e0-e6bf-4570-a975-5791e21490a6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:05.756163 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-313906e0-e6bf-4570-a975-5791e21490a6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1365 microversion: 2.1 time: 0.417090 Sep 27 10:29:05.769293 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2738/5935] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:05 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1365 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:29:06.785486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-223f2211-193c-4869-b5fd-9c92916edf8a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:06.786125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-223f2211-193c-4869-b5fd-9c92916edf8a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:06.792197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:06.792986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=34 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:06.804337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223f2211-193c-4869-b5fd-9c92916edf8a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:06.804777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223f2211-193c-4869-b5fd-9c92916edf8a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:06.805230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223f2211-193c-4869-b5fd-9c92916edf8a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:06.810115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223f2211-193c-4869-b5fd-9c92916edf8a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:06.810924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223f2211-193c-4869-b5fd-9c92916edf8a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:06.813117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-223f2211-193c-4869-b5fd-9c92916edf8a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:07.008721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-223f2211-193c-4869-b5fd-9c92916edf8a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:07.020703 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-223f2211-193c-4869-b5fd-9c92916edf8a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1555 microversion: 2.1 time: 0.241982 Sep 27 10:29:07.024700 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3198/5936] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:06 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1555 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:07.051106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9b3a0b8-69e2-43fc-8e28-bb7752c7c4ff tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:07.051641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9b3a0b8-69e2-43fc-8e28-bb7752c7c4ff tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:07.062352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9b3a0b8-69e2-43fc-8e28-bb7752c7c4ff tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:07.062859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9b3a0b8-69e2-43fc-8e28-bb7752c7c4ff tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:07.063323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9b3a0b8-69e2-43fc-8e28-bb7752c7c4ff tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:07.064133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9b3a0b8-69e2-43fc-8e28-bb7752c7c4ff tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:07.066353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9b3a0b8-69e2-43fc-8e28-bb7752c7c4ff tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:07.066814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9b3a0b8-69e2-43fc-8e28-bb7752c7c4ff tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:07.167557 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9b3a0b8-69e2-43fc-8e28-bb7752c7c4ff tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.121419 Sep 27 10:29:07.168751 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2739/5937] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:29:07 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14/os-interface => generated 284 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:08.567917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-1d2a98e1-110d-4fa9-95a0-e7f59721536d req-6d641e10-f159-42ef-bb19-d147797a20e2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d5319326-fc6f-4d93-85ae-98a8b422dc14", "tag": "ea0cb073-0c12-471c-b08c-f88ce79e8500"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:08.580592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1d2a98e1-110d-4fa9-95a0-e7f59721536d req-6d641e10-f159-42ef-bb19-d147797a20e2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:08.581355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1d2a98e1-110d-4fa9-95a0-e7f59721536d req-6d641e10-f159-42ef-bb19-d147797a20e2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:08.581980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1d2a98e1-110d-4fa9-95a0-e7f59721536d req-6d641e10-f159-42ef-bb19-d147797a20e2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:08.596741 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-1d2a98e1-110d-4fa9-95a0-e7f59721536d req-6d641e10-f159-42ef-bb19-d147797a20e2 service nova] Creating event network-changed:ea0cb073-0c12-471c-b08c-f88ce79e8500 for instance d5319326-fc6f-4d93-85ae-98a8b422dc14 on nch2-master-1 Sep 27 10:29:08.603666 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-1d2a98e1-110d-4fa9-95a0-e7f59721536d req-6d641e10-f159-42ef-bb19-d147797a20e2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041029 Sep 27 10:29:08.612353 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3199/5938] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:13.415473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-532969140", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:13.483116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] validate_networks() for [('01047f88-8e96-44cb-a0d6-1b5491078154', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:29:14.158736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:29:14.159180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:29:14.159677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:29:14.160104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:29:14.160548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:29:14.170543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:14.171051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:14.171504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:14.172259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:14.172792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:14.173239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:14.191345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Getting quotas for project 40247edd5a444fd8b1c937c13d39393a. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:29:14.196909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Getting quotas for user 6771df45b4ee4426b2f76a09eab2998a and project 40247edd5a444fd8b1c937c13d39393a. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:29:14.210110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:29:14.212536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:29:14.264494 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5e2116e0-9dfe-4bf3-b5a9-14bcfc4cfa70 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.034176 Sep 27 10:29:14.265204 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2740/5939] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:29:13 2022] POST /compute/v2.1/servers => generated 383 bytes in 1036 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:29:14.278601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5da67551-1e42-425a-a5da-f2c29a9fc2e5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:14.279282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5da67551-1e42-425a-a5da-f2c29a9fc2e5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:14.343753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5da67551-1e42-425a-a5da-f2c29a9fc2e5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:14.351253 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5da67551-1e42-425a-a5da-f2c29a9fc2e5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1221 microversion: 2.1 time: 0.078340 Sep 27 10:29:14.351843 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3200/5940] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:14 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1221 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:14.419260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1890186576", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:14.793984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] validate_networks() for [('01047f88-8e96-44cb-a0d6-1b5491078154', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:29:15.284881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:29:15.285558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:29:15.286046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:29:15.286582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:29:15.287047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:29:15.297454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:15.298196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:15.298709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:15.299491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:15.299962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:15.301075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:15.319061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Getting quotas for project 40247edd5a444fd8b1c937c13d39393a. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:29:15.324406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Getting quotas for user 6771df45b4ee4426b2f76a09eab2998a and project 40247edd5a444fd8b1c937c13d39393a. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:29:15.355417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:29:15.357704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:29:15.419216 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-665023c0-cd98-417f-a2a5-75de2ffbc7d2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.004097 Sep 27 10:29:15.420813 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2741/5941] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:29:14 2022] POST /compute/v2.1/servers => generated 383 bytes in 1006 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:29:15.432449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f06149a0-3dc7-453e-876a-b129c06d7874 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:15.433301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f06149a0-3dc7-453e-876a-b129c06d7874 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:15.537039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f06149a0-3dc7-453e-876a-b129c06d7874 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:15.545863 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f06149a0-3dc7-453e-876a-b129c06d7874 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1222 microversion: 2.1 time: 0.117549 Sep 27 10:29:15.551879 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3201/5942] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:15 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1222 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:16.572778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ff9bc131-cfa3-4526-8d6a-937c15d44a50 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:16.579996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ff9bc131-cfa3-4526-8d6a-937c15d44a50 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:16.591545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff9bc131-cfa3-4526-8d6a-937c15d44a50 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:16.594910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff9bc131-cfa3-4526-8d6a-937c15d44a50 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:16.595936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff9bc131-cfa3-4526-8d6a-937c15d44a50 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:16.597157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff9bc131-cfa3-4526-8d6a-937c15d44a50 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:16.598340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff9bc131-cfa3-4526-8d6a-937c15d44a50 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:16.614584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff9bc131-cfa3-4526-8d6a-937c15d44a50 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:16.712208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ff9bc131-cfa3-4526-8d6a-937c15d44a50 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:16.726726 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ff9bc131-cfa3-4526-8d6a-937c15d44a50 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1274 microversion: 2.1 time: 0.158355 Sep 27 10:29:16.729138 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2742/5943] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:16 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1274 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:17.504978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0e19c9c1-57c4-4ba6-a995-4cf5b388fd33 req-1072cdeb-7ebd-4e9e-ba95-c262616ae8dc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "15833e30-ec8c-4756-b9df-5ca3061b4ed9", "tag": "77c15aca-e4ea-4c69-9903-2fba830c8336"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:17.512276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:17.526115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0e19c9c1-57c4-4ba6-a995-4cf5b388fd33 req-1072cdeb-7ebd-4e9e-ba95-c262616ae8dc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:17.526462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0e19c9c1-57c4-4ba6-a995-4cf5b388fd33 req-1072cdeb-7ebd-4e9e-ba95-c262616ae8dc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:17.526906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0e19c9c1-57c4-4ba6-a995-4cf5b388fd33 req-1072cdeb-7ebd-4e9e-ba95-c262616ae8dc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:17.548741 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0e19c9c1-57c4-4ba6-a995-4cf5b388fd33 req-1072cdeb-7ebd-4e9e-ba95-c262616ae8dc service nova] Creating event network-changed:77c15aca-e4ea-4c69-9903-2fba830c8336 for instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 on nch1-master-1 Sep 27 10:29:17.555945 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0e19c9c1-57c4-4ba6-a995-4cf5b388fd33 req-1072cdeb-7ebd-4e9e-ba95-c262616ae8dc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055273 Sep 27 10:29:17.561850 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3202/5944] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:17.752611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d8afd15-7227-4a59-80a4-65a10b13d9f2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:17.753956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4d8afd15-7227-4a59-80a4-65a10b13d9f2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:17.778041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8afd15-7227-4a59-80a4-65a10b13d9f2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:17.783435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8afd15-7227-4a59-80a4-65a10b13d9f2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:17.784227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8afd15-7227-4a59-80a4-65a10b13d9f2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:17.785214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8afd15-7227-4a59-80a4-65a10b13d9f2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:17.785726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8afd15-7227-4a59-80a4-65a10b13d9f2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:17.786169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8afd15-7227-4a59-80a4-65a10b13d9f2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:18.113471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4d8afd15-7227-4a59-80a4-65a10b13d9f2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:18.129005 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4d8afd15-7227-4a59-80a4-65a10b13d9f2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1322 microversion: 2.1 time: 0.380722 Sep 27 10:29:18.130169 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2743/5945] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:17 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1322 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:18.513868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-041d5d46-1bc2-4c98-abcf-286c51f90c3c req-424b25df-fcb5-4688-8d79-99b7455bad58 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6537d086-a852-4b27-a032-807cecd97b86", "tag": "d9fe8fff-a04f-429f-9369-933e6efad0ea"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:18.524988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-041d5d46-1bc2-4c98-abcf-286c51f90c3c req-424b25df-fcb5-4688-8d79-99b7455bad58 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:18.525469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-041d5d46-1bc2-4c98-abcf-286c51f90c3c req-424b25df-fcb5-4688-8d79-99b7455bad58 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:18.525905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-041d5d46-1bc2-4c98-abcf-286c51f90c3c req-424b25df-fcb5-4688-8d79-99b7455bad58 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:18.537793 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-041d5d46-1bc2-4c98-abcf-286c51f90c3c req-424b25df-fcb5-4688-8d79-99b7455bad58 service nova] Creating event network-changed:d9fe8fff-a04f-429f-9369-933e6efad0ea for instance 6537d086-a852-4b27-a032-807cecd97b86 on nch1-master-1 Sep 27 10:29:18.542924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-041d5d46-1bc2-4c98-abcf-286c51f90c3c req-424b25df-fcb5-4688-8d79-99b7455bad58 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033267 Sep 27 10:29:18.543524 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3203/5946] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:19.154705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f4d6560e-3a5d-466a-8458-1a914c48c68d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:19.155323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f4d6560e-3a5d-466a-8458-1a914c48c68d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:19.162287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4d6560e-3a5d-466a-8458-1a914c48c68d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:19.162797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4d6560e-3a5d-466a-8458-1a914c48c68d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:19.163246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4d6560e-3a5d-466a-8458-1a914c48c68d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:19.163965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4d6560e-3a5d-466a-8458-1a914c48c68d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:19.164614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4d6560e-3a5d-466a-8458-1a914c48c68d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:19.165243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4d6560e-3a5d-466a-8458-1a914c48c68d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:19.573669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f4d6560e-3a5d-466a-8458-1a914c48c68d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:19.582198 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f4d6560e-3a5d-466a-8458-1a914c48c68d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1322 microversion: 2.1 time: 0.431319 Sep 27 10:29:19.583081 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2744/5947] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:19 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1322 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:20.610800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33ff2f9d-5bd3-4f4b-a375-b2e012c2fdd3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:20.611843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33ff2f9d-5bd3-4f4b-a375-b2e012c2fdd3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:20.619473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ff2f9d-5bd3-4f4b-a375-b2e012c2fdd3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:20.619975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ff2f9d-5bd3-4f4b-a375-b2e012c2fdd3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:20.620412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ff2f9d-5bd3-4f4b-a375-b2e012c2fdd3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:20.621118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ff2f9d-5bd3-4f4b-a375-b2e012c2fdd3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:20.621751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ff2f9d-5bd3-4f4b-a375-b2e012c2fdd3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:20.622372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33ff2f9d-5bd3-4f4b-a375-b2e012c2fdd3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:20.775726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33ff2f9d-5bd3-4f4b-a375-b2e012c2fdd3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:20.785069 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33ff2f9d-5bd3-4f4b-a375-b2e012c2fdd3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1322 microversion: 2.1 time: 0.178434 Sep 27 10:29:20.785885 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3204/5948] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:20 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1322 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:21.817812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b81c6f4-c9a8-49ca-b521-03f6604786fc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:21.818484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0b81c6f4-c9a8-49ca-b521-03f6604786fc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:21.829296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b81c6f4-c9a8-49ca-b521-03f6604786fc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:21.829994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b81c6f4-c9a8-49ca-b521-03f6604786fc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:21.830858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b81c6f4-c9a8-49ca-b521-03f6604786fc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:21.831975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b81c6f4-c9a8-49ca-b521-03f6604786fc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:21.832648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b81c6f4-c9a8-49ca-b521-03f6604786fc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:21.833362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b81c6f4-c9a8-49ca-b521-03f6604786fc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:21.975445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0b81c6f4-c9a8-49ca-b521-03f6604786fc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:21.987284 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0b81c6f4-c9a8-49ca-b521-03f6604786fc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1322 microversion: 2.1 time: 0.175814 Sep 27 10:29:21.988044 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2745/5949] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:21 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1322 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:23.015517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3edca73-c3a4-425a-8d49-ed2e6a519ac4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:23.016147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e3edca73-c3a4-425a-8d49-ed2e6a519ac4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:23.024528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3edca73-c3a4-425a-8d49-ed2e6a519ac4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:23.025067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3edca73-c3a4-425a-8d49-ed2e6a519ac4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:23.025497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3edca73-c3a4-425a-8d49-ed2e6a519ac4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:23.026252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3edca73-c3a4-425a-8d49-ed2e6a519ac4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:23.026713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3edca73-c3a4-425a-8d49-ed2e6a519ac4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:23.027149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3edca73-c3a4-425a-8d49-ed2e6a519ac4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:23.169778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e3edca73-c3a4-425a-8d49-ed2e6a519ac4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:23.178383 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e3edca73-c3a4-425a-8d49-ed2e6a519ac4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1322 microversion: 2.1 time: 0.167009 Sep 27 10:29:23.178965 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3205/5950] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:23 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1322 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:24.202251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-35a8920f-4e43-40bb-a389-b073ce64c6a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:24.202820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-35a8920f-4e43-40bb-a389-b073ce64c6a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:24.209753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35a8920f-4e43-40bb-a389-b073ce64c6a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:24.210235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35a8920f-4e43-40bb-a389-b073ce64c6a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:24.210712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35a8920f-4e43-40bb-a389-b073ce64c6a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:24.211439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35a8920f-4e43-40bb-a389-b073ce64c6a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:24.211908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35a8920f-4e43-40bb-a389-b073ce64c6a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:24.212330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35a8920f-4e43-40bb-a389-b073ce64c6a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:24.350910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-35a8920f-4e43-40bb-a389-b073ce64c6a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:24.359019 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-35a8920f-4e43-40bb-a389-b073ce64c6a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1322 microversion: 2.1 time: 0.160829 Sep 27 10:29:24.359674 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2746/5951] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:24 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:24.489511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-1a7ed2eb-8de2-4977-a5d5-0ae9cd6801db req-208aeeee-d89c-40ae-af14-8b723813cccd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "15833e30-ec8c-4756-b9df-5ca3061b4ed9", "name": "network-vif-plugged", "status": "completed", "tag": "77c15aca-e4ea-4c69-9903-2fba830c8336"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:24.500278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1a7ed2eb-8de2-4977-a5d5-0ae9cd6801db req-208aeeee-d89c-40ae-af14-8b723813cccd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:24.500795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1a7ed2eb-8de2-4977-a5d5-0ae9cd6801db req-208aeeee-d89c-40ae-af14-8b723813cccd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:24.501250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1a7ed2eb-8de2-4977-a5d5-0ae9cd6801db req-208aeeee-d89c-40ae-af14-8b723813cccd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:24.516468 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-1a7ed2eb-8de2-4977-a5d5-0ae9cd6801db req-208aeeee-d89c-40ae-af14-8b723813cccd service nova] Creating event network-vif-plugged:77c15aca-e4ea-4c69-9903-2fba830c8336 for instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 on nch1-master-1 Sep 27 10:29:24.521716 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-1a7ed2eb-8de2-4977-a5d5-0ae9cd6801db req-208aeeee-d89c-40ae-af14-8b723813cccd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036736 Sep 27 10:29:24.522536 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3206/5952] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:25.385108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-751cc6bf-a2be-41a9-8ba6-3d80c4710028 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:25.385979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-751cc6bf-a2be-41a9-8ba6-3d80c4710028 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:25.390073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:25.391029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=12 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:25.398476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-751cc6bf-a2be-41a9-8ba6-3d80c4710028 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:25.398981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-751cc6bf-a2be-41a9-8ba6-3d80c4710028 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:25.399536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-751cc6bf-a2be-41a9-8ba6-3d80c4710028 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:25.400271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-751cc6bf-a2be-41a9-8ba6-3d80c4710028 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:25.400919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-751cc6bf-a2be-41a9-8ba6-3d80c4710028 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:25.401364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-751cc6bf-a2be-41a9-8ba6-3d80c4710028 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:25.548426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-751cc6bf-a2be-41a9-8ba6-3d80c4710028 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:25.556818 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-751cc6bf-a2be-41a9-8ba6-3d80c4710028 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1322 microversion: 2.1 time: 0.177418 Sep 27 10:29:25.557393 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2747/5953] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:25 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1322 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:25.968192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:26.207660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:26.208264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:26.214618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:26.215474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:26.215923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:26.216633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:26.217140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:26.217599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:26.244430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:29:26.334064 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-65fcaa50-40cc-41cf-8cda-ae4814ed98c0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "DELETE /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 204 len: 0 microversion: 2.1 time: 0.371707 Sep 27 10:29:26.339503 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3207/5954] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:29:25 2022] DELETE /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:29:26.345349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3693e491-2fc4-42e5-a9be-8cca38d67864 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:26.346045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3693e491-2fc4-42e5-a9be-8cca38d67864 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:26.353778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3693e491-2fc4-42e5-a9be-8cca38d67864 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:26.354292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3693e491-2fc4-42e5-a9be-8cca38d67864 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:26.354749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3693e491-2fc4-42e5-a9be-8cca38d67864 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:26.355479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3693e491-2fc4-42e5-a9be-8cca38d67864 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:26.355975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3693e491-2fc4-42e5-a9be-8cca38d67864 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:26.356421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3693e491-2fc4-42e5-a9be-8cca38d67864 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:26.496459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3693e491-2fc4-42e5-a9be-8cca38d67864 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:26.505302 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3693e491-2fc4-42e5-a9be-8cca38d67864 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1577 microversion: 2.1 time: 0.163960 Sep 27 10:29:26.505853 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2748/5955] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:26 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1577 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:26.550911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-1f61ad9a-d274-4649-909f-7ea70a8eb3ad req-9dddee80-fe83-4d29-b6a2-8843aa04e5d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6537d086-a852-4b27-a032-807cecd97b86", "name": "network-vif-plugged", "status": "completed", "tag": "d9fe8fff-a04f-429f-9369-933e6efad0ea"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:26.565849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1f61ad9a-d274-4649-909f-7ea70a8eb3ad req-9dddee80-fe83-4d29-b6a2-8843aa04e5d8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:26.566427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1f61ad9a-d274-4649-909f-7ea70a8eb3ad req-9dddee80-fe83-4d29-b6a2-8843aa04e5d8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:26.567083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1f61ad9a-d274-4649-909f-7ea70a8eb3ad req-9dddee80-fe83-4d29-b6a2-8843aa04e5d8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:26.579393 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-1f61ad9a-d274-4649-909f-7ea70a8eb3ad req-9dddee80-fe83-4d29-b6a2-8843aa04e5d8 service nova] Creating event network-vif-plugged:d9fe8fff-a04f-429f-9369-933e6efad0ea for instance 6537d086-a852-4b27-a032-807cecd97b86 on nch1-master-1 Sep 27 10:29:26.579730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c486d161-2e7f-479c-a966-4c054ce9628f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:26.580353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c486d161-2e7f-479c-a966-4c054ce9628f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:26.584626 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-1f61ad9a-d274-4649-909f-7ea70a8eb3ad req-9dddee80-fe83-4d29-b6a2-8843aa04e5d8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040042 Sep 27 10:29:26.585430 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3208/5956] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:26.587790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486d161-2e7f-479c-a966-4c054ce9628f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:26.588521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486d161-2e7f-479c-a966-4c054ce9628f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:26.589267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486d161-2e7f-479c-a966-4c054ce9628f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:26.590377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486d161-2e7f-479c-a966-4c054ce9628f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:26.590998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486d161-2e7f-479c-a966-4c054ce9628f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:26.591633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486d161-2e7f-479c-a966-4c054ce9628f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:26.789402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c486d161-2e7f-479c-a966-4c054ce9628f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:26.797334 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c486d161-2e7f-479c-a966-4c054ce9628f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1322 microversion: 2.1 time: 0.221608 Sep 27 10:29:26.798193 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2749/5957] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:26 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1322 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:27.534130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-50bdde52-4083-487e-be6b-a1767f9c2a9f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:27.535393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-50bdde52-4083-487e-be6b-a1767f9c2a9f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:27.547512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50bdde52-4083-487e-be6b-a1767f9c2a9f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:27.548329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50bdde52-4083-487e-be6b-a1767f9c2a9f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:27.549113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50bdde52-4083-487e-be6b-a1767f9c2a9f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:27.550408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50bdde52-4083-487e-be6b-a1767f9c2a9f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:27.551016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50bdde52-4083-487e-be6b-a1767f9c2a9f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:27.551446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-50bdde52-4083-487e-be6b-a1767f9c2a9f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:27.700229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-50bdde52-4083-487e-be6b-a1767f9c2a9f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:27.709007 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-50bdde52-4083-487e-be6b-a1767f9c2a9f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1577 microversion: 2.1 time: 0.182446 Sep 27 10:29:27.709625 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3209/5958] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:27 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1577 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:27.820991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-15ba915c-a03e-4724-baa8-dc74aabfd691 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:27.821757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-15ba915c-a03e-4724-baa8-dc74aabfd691 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:27.829081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba915c-a03e-4724-baa8-dc74aabfd691 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:27.829596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba915c-a03e-4724-baa8-dc74aabfd691 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:27.830054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba915c-a03e-4724-baa8-dc74aabfd691 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:27.830802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba915c-a03e-4724-baa8-dc74aabfd691 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:27.831353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba915c-a03e-4724-baa8-dc74aabfd691 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:27.831893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-15ba915c-a03e-4724-baa8-dc74aabfd691 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:27.977995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-15ba915c-a03e-4724-baa8-dc74aabfd691 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:27.986189 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-15ba915c-a03e-4724-baa8-dc74aabfd691 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1322 microversion: 2.1 time: 0.169793 Sep 27 10:29:27.986763 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2750/5959] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:27 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1322 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:28.733351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-07574ef0-79de-41b3-a397-ab748e957aeb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:28.733983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-07574ef0-79de-41b3-a397-ab748e957aeb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:28.741311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07574ef0-79de-41b3-a397-ab748e957aeb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:28.741799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07574ef0-79de-41b3-a397-ab748e957aeb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:28.742216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07574ef0-79de-41b3-a397-ab748e957aeb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:28.742932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07574ef0-79de-41b3-a397-ab748e957aeb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:28.743358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07574ef0-79de-41b3-a397-ab748e957aeb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:28.743784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-07574ef0-79de-41b3-a397-ab748e957aeb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:28.879939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-07574ef0-79de-41b3-a397-ab748e957aeb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:28.888925 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-07574ef0-79de-41b3-a397-ab748e957aeb tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1577 microversion: 2.1 time: 0.159392 Sep 27 10:29:28.889747 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3210/5960] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:28 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1577 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:29.009540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c8f5c44b-ea6f-44d0-af21-49ee6944bd5e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:29.010178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c8f5c44b-ea6f-44d0-af21-49ee6944bd5e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:29.017562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f5c44b-ea6f-44d0-af21-49ee6944bd5e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:29.018199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f5c44b-ea6f-44d0-af21-49ee6944bd5e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:29.018631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f5c44b-ea6f-44d0-af21-49ee6944bd5e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:29.020117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f5c44b-ea6f-44d0-af21-49ee6944bd5e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:29.020606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f5c44b-ea6f-44d0-af21-49ee6944bd5e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:29.021098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f5c44b-ea6f-44d0-af21-49ee6944bd5e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:29.249816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c8f5c44b-ea6f-44d0-af21-49ee6944bd5e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:29.258547 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c8f5c44b-ea6f-44d0-af21-49ee6944bd5e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1517 microversion: 2.1 time: 0.253242 Sep 27 10:29:29.259121 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2751/5961] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:29 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1517 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:29.281527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cfcd9fc5-3cd8-4cb4-a320-6f6c0b4dcb91 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:29.283954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cfcd9fc5-3cd8-4cb4-a320-6f6c0b4dcb91 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:29.291233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfcd9fc5-3cd8-4cb4-a320-6f6c0b4dcb91 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:29.291727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfcd9fc5-3cd8-4cb4-a320-6f6c0b4dcb91 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:29.292171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfcd9fc5-3cd8-4cb4-a320-6f6c0b4dcb91 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:29.292906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfcd9fc5-3cd8-4cb4-a320-6f6c0b4dcb91 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:29.293452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfcd9fc5-3cd8-4cb4-a320-6f6c0b4dcb91 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:29.293890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cfcd9fc5-3cd8-4cb4-a320-6f6c0b4dcb91 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:29.318221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-cfcd9fc5-3cd8-4cb4-a320-6f6c0b4dcb91 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lazy-loading 'flavor' on Instance uuid 6537d086-a852-4b27-a032-807cecd97b86 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:29:29.403984 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cfcd9fc5-3cd8-4cb4-a320-6f6c0b4dcb91 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86/action" status: 200 len: 32 microversion: 2.1 time: 0.126568 Sep 27 10:29:29.404897 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3211/5962] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:29:29 2022] POST /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86/action => generated 32 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:29:29.415761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b1124aa0-7eff-4f19-9a00-a23628bbf84b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:29.416402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b1124aa0-7eff-4f19-9a00-a23628bbf84b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:29.424516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1124aa0-7eff-4f19-9a00-a23628bbf84b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:29.425238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1124aa0-7eff-4f19-9a00-a23628bbf84b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:29.426128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1124aa0-7eff-4f19-9a00-a23628bbf84b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:29.426623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1124aa0-7eff-4f19-9a00-a23628bbf84b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:29.427072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1124aa0-7eff-4f19-9a00-a23628bbf84b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:29.427529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1124aa0-7eff-4f19-9a00-a23628bbf84b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:29.432268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:29.478706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7c19dd20-7aa3-43e1-9de9-a2109be77cd3 req-43f699af-8685-41bd-b550-e7037563fef0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "82b31a6f-a898-4796-a3ad-1628306fbe7b", "name": "network-vif-deleted", "tag": "f840e3d8-9c94-4d2d-b759-279dceffb800"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:29.493731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7c19dd20-7aa3-43e1-9de9-a2109be77cd3 req-43f699af-8685-41bd-b550-e7037563fef0 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:29.494577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7c19dd20-7aa3-43e1-9de9-a2109be77cd3 req-43f699af-8685-41bd-b550-e7037563fef0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:29.496599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7c19dd20-7aa3-43e1-9de9-a2109be77cd3 req-43f699af-8685-41bd-b550-e7037563fef0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:29.518910 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7c19dd20-7aa3-43e1-9de9-a2109be77cd3 req-43f699af-8685-41bd-b550-e7037563fef0 service nova] Creating event network-vif-deleted:f840e3d8-9c94-4d2d-b759-279dceffb800 for instance 82b31a6f-a898-4796-a3ad-1628306fbe7b on nch2-master-1 Sep 27 10:29:29.525510 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7c19dd20-7aa3-43e1-9de9-a2109be77cd3 req-43f699af-8685-41bd-b550-e7037563fef0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051656 Sep 27 10:29:29.526391 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3212/5963] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:29:29.725641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b1124aa0-7eff-4f19-9a00-a23628bbf84b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:29.733557 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b1124aa0-7eff-4f19-9a00-a23628bbf84b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.321776 Sep 27 10:29:29.734131 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2752/5964] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:29 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:29.918199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:29.927160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:29.934573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:29.935361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:29.937371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:29.938333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:29.938972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:29.939579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:29.977461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:29.978193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:29.978628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:29.979273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lazy-loading 'fault' on Instance uuid 82b31a6f-a898-4796-a3ad-1628306fbe7b {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:29:30.032237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:30.042612 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c73800f9-1043-4739-b79f-b81eee879188 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 200 len: 1230 microversion: 2.1 time: 0.133164 Sep 27 10:29:30.043200 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3213/5965] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:29 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 1230 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:30.756642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-66d88689-18f8-4a4b-87cd-7f2c0b4593dc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:30.757251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-66d88689-18f8-4a4b-87cd-7f2c0b4593dc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:30.764617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66d88689-18f8-4a4b-87cd-7f2c0b4593dc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:30.765168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66d88689-18f8-4a4b-87cd-7f2c0b4593dc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:30.765632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66d88689-18f8-4a4b-87cd-7f2c0b4593dc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:30.766367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66d88689-18f8-4a4b-87cd-7f2c0b4593dc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:30.766882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66d88689-18f8-4a4b-87cd-7f2c0b4593dc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:30.767341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-66d88689-18f8-4a4b-87cd-7f2c0b4593dc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:30.914815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-66d88689-18f8-4a4b-87cd-7f2c0b4593dc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:30.922952 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-66d88689-18f8-4a4b-87cd-7f2c0b4593dc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.170679 Sep 27 10:29:30.923536 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2753/5966] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:30 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:31.069568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:31.070253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:31.078471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:31.079056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:31.079579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:31.080408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:31.081070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:31.081589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:31.086654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:31.098818 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b HTTP/1.1 Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '82b31a6f-a898-4796-a3ad-1628306fbe7b'} Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:31.100463 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:31.106245 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:31.106245 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:31.106245 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:31.106245 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:29:31.106245 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:29:31.106245 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:29:31.106245 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:31.106245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:29:31.106245 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3bdc4b6a-db4c-4127-b081-30d7f3859c58 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 404 len: 111 microversion: 2.1 time: 0.037961 Sep 27 10:29:31.106245 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3214/5967] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:31 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:29:31.153021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1063125783", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-362079488", "security_groups": [{"name": "tempest-secgroup-smoke-883528541"}], "block_device_mapping_v2": [{"uuid": "8489f678-48b9-47ca-8f5e-397c453365e4", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "1e417b59-62bb-4007-a78a-c15c153efd33"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:31.280422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] validate_networks() for [('1e417b59-62bb-4007-a78a-c15c153efd33', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:29:31.780726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:29:31.781376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:29:31.781788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:29:31.782337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:29:31.782923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:29:31.793421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:31.793931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:31.794392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:31.795154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:31.795643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:31.796079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:31.813831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:29:31.819650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for user 178956f6c8154a06a0710b295aff3086 and project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:29:31.882292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:29:31.884443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='8489f678-48b9-47ca-8f5e-397c453365e4',volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:29:31.946824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0317b809-02c7-47ca-b58e-1643cb5774a3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:31.947440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0317b809-02c7-47ca-b58e-1643cb5774a3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:31.954180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0317b809-02c7-47ca-b58e-1643cb5774a3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:31.954674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0317b809-02c7-47ca-b58e-1643cb5774a3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:31.955143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0317b809-02c7-47ca-b58e-1643cb5774a3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:31.955845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0317b809-02c7-47ca-b58e-1643cb5774a3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:31.956302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0317b809-02c7-47ca-b58e-1643cb5774a3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:31.956755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0317b809-02c7-47ca-b58e-1643cb5774a3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:32.044959 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c4ad5cd3-1db0-4990-91d2-7e6c45265c05 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.895810 Sep 27 10:29:32.045937 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2754/5968] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:29:31 2022] POST /compute/v2.1/servers => generated 405 bytes in 898 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:29:32.057526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0cbd4c0e-933d-41de-a748-632d9198f81d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:32.058336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0cbd4c0e-933d-41de-a748-632d9198f81d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:32.118486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0317b809-02c7-47ca-b58e-1643cb5774a3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:32.120383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0cbd4c0e-933d-41de-a748-632d9198f81d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:32.127232 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0317b809-02c7-47ca-b58e-1643cb5774a3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.185104 Sep 27 10:29:32.127670 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0cbd4c0e-933d-41de-a748-632d9198f81d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1094 microversion: 2.1 time: 0.074285 Sep 27 10:29:32.128252 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3215/5969] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:31 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:29:32.128587 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2755/5970] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:32 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1094 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:33.150979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d43a2fe-5933-425d-b89a-c5a0dad9934c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:33.151576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c20af8c1-8b5f-4eef-8ea7-dfba18a156ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:33.151957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4d43a2fe-5933-425d-b89a-c5a0dad9934c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:33.152102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c20af8c1-8b5f-4eef-8ea7-dfba18a156ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:33.159020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d43a2fe-5933-425d-b89a-c5a0dad9934c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:33.159611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d43a2fe-5933-425d-b89a-c5a0dad9934c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:33.159798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c20af8c1-8b5f-4eef-8ea7-dfba18a156ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:33.160161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c20af8c1-8b5f-4eef-8ea7-dfba18a156ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:33.160327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d43a2fe-5933-425d-b89a-c5a0dad9934c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:33.160710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c20af8c1-8b5f-4eef-8ea7-dfba18a156ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:33.161212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d43a2fe-5933-425d-b89a-c5a0dad9934c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:33.161725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c20af8c1-8b5f-4eef-8ea7-dfba18a156ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:33.161851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d43a2fe-5933-425d-b89a-c5a0dad9934c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:33.162186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c20af8c1-8b5f-4eef-8ea7-dfba18a156ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:33.162363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d43a2fe-5933-425d-b89a-c5a0dad9934c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:33.162635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c20af8c1-8b5f-4eef-8ea7-dfba18a156ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:33.227555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c20af8c1-8b5f-4eef-8ea7-dfba18a156ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:33.240888 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c20af8c1-8b5f-4eef-8ea7-dfba18a156ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1192 microversion: 2.1 time: 0.091170 Sep 27 10:29:33.240888 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2756/5971] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:33 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1192 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:33.304441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4d43a2fe-5933-425d-b89a-c5a0dad9934c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:33.313074 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4d43a2fe-5933-425d-b89a-c5a0dad9934c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.166066 Sep 27 10:29:33.316744 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3216/5972] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:33 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:34.261957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0225cdd1-0d09-4c89-bb9d-3577b4e37e3c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:34.262700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0225cdd1-0d09-4c89-bb9d-3577b4e37e3c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:34.272688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0225cdd1-0d09-4c89-bb9d-3577b4e37e3c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:34.273320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0225cdd1-0d09-4c89-bb9d-3577b4e37e3c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:34.273777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0225cdd1-0d09-4c89-bb9d-3577b4e37e3c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:34.274648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0225cdd1-0d09-4c89-bb9d-3577b4e37e3c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:34.275212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0225cdd1-0d09-4c89-bb9d-3577b4e37e3c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:34.275664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0225cdd1-0d09-4c89-bb9d-3577b4e37e3c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:34.342505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a7c31403-7b77-41ae-ac6a-06d0a5a8efbf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:34.343139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a7c31403-7b77-41ae-ac6a-06d0a5a8efbf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:34.356729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c31403-7b77-41ae-ac6a-06d0a5a8efbf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:34.357511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c31403-7b77-41ae-ac6a-06d0a5a8efbf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:34.357983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c31403-7b77-41ae-ac6a-06d0a5a8efbf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:34.358815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c31403-7b77-41ae-ac6a-06d0a5a8efbf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:34.359334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c31403-7b77-41ae-ac6a-06d0a5a8efbf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:34.359782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7c31403-7b77-41ae-ac6a-06d0a5a8efbf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:34.537962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a7c31403-7b77-41ae-ac6a-06d0a5a8efbf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:34.547229 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a7c31403-7b77-41ae-ac6a-06d0a5a8efbf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.215566 Sep 27 10:29:34.547819 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3217/5973] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:34 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:34.672709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0225cdd1-0d09-4c89-bb9d-3577b4e37e3c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:34.681683 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0225cdd1-0d09-4c89-bb9d-3577b4e37e3c tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1277 microversion: 2.1 time: 0.424561 Sep 27 10:29:34.682295 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2757/5974] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:34 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1277 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:35.258967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-9dad4559-6842-4c28-a06f-fe5e3f6ef165 req-de01e08e-eb60-4150-9334-f2d5ce5c107d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9f14e82f-c065-4b4e-bb03-c94b57bc21e7", "tag": "86766b79-cf12-4620-813b-644b95940fbc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:35.271028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9dad4559-6842-4c28-a06f-fe5e3f6ef165 req-de01e08e-eb60-4150-9334-f2d5ce5c107d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:35.271647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9dad4559-6842-4c28-a06f-fe5e3f6ef165 req-de01e08e-eb60-4150-9334-f2d5ce5c107d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:35.272389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9dad4559-6842-4c28-a06f-fe5e3f6ef165 req-de01e08e-eb60-4150-9334-f2d5ce5c107d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:35.285409 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-9dad4559-6842-4c28-a06f-fe5e3f6ef165 req-de01e08e-eb60-4150-9334-f2d5ce5c107d service nova] Creating event network-changed:86766b79-cf12-4620-813b-644b95940fbc for instance 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 on nch2-master-1 Sep 27 10:29:35.291452 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-9dad4559-6842-4c28-a06f-fe5e3f6ef165 req-de01e08e-eb60-4150-9334-f2d5ce5c107d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036698 Sep 27 10:29:35.292381 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3218/5975] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:35.571283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4a82aa55-a578-4130-acac-0c1d95dfdc4c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:35.571887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4a82aa55-a578-4130-acac-0c1d95dfdc4c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:35.574717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=78,nova_api:INSERT=9 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:35.579993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a82aa55-a578-4130-acac-0c1d95dfdc4c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:35.580501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a82aa55-a578-4130-acac-0c1d95dfdc4c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:35.580989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a82aa55-a578-4130-acac-0c1d95dfdc4c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:35.581711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a82aa55-a578-4130-acac-0c1d95dfdc4c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:35.582181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a82aa55-a578-4130-acac-0c1d95dfdc4c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:35.582805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a82aa55-a578-4130-acac-0c1d95dfdc4c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:35.708332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cbfafdc8-5d8f-4bc9-83ea-33f2b6864e40 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:35.709865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cbfafdc8-5d8f-4bc9-83ea-33f2b6864e40 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:35.717165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbfafdc8-5d8f-4bc9-83ea-33f2b6864e40 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:35.717849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbfafdc8-5d8f-4bc9-83ea-33f2b6864e40 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:35.718470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbfafdc8-5d8f-4bc9-83ea-33f2b6864e40 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:35.719310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbfafdc8-5d8f-4bc9-83ea-33f2b6864e40 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:35.719814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbfafdc8-5d8f-4bc9-83ea-33f2b6864e40 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:35.720236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbfafdc8-5d8f-4bc9-83ea-33f2b6864e40 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:35.813015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4a82aa55-a578-4130-acac-0c1d95dfdc4c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:35.821846 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4a82aa55-a578-4130-acac-0c1d95dfdc4c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.254713 Sep 27 10:29:35.822438 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2758/5976] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:35 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:35.876212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cbfafdc8-5d8f-4bc9-83ea-33f2b6864e40 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:35.885020 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cbfafdc8-5d8f-4bc9-83ea-33f2b6864e40 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1265 microversion: 2.1 time: 0.182853 Sep 27 10:29:35.885528 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3219/5977] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:35 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1265 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:36.852035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-20e4f6ea-cd4e-480e-8baf-e605119f2d7b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:36.852815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-20e4f6ea-cd4e-480e-8baf-e605119f2d7b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:36.861733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e4f6ea-cd4e-480e-8baf-e605119f2d7b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:36.862411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e4f6ea-cd4e-480e-8baf-e605119f2d7b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:36.862880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e4f6ea-cd4e-480e-8baf-e605119f2d7b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:36.863716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e4f6ea-cd4e-480e-8baf-e605119f2d7b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:36.864314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e4f6ea-cd4e-480e-8baf-e605119f2d7b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:36.864881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20e4f6ea-cd4e-480e-8baf-e605119f2d7b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:36.907744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c0e2c985-d4c9-4f7b-b35b-61e1ba5dfe8f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:36.908337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c0e2c985-d4c9-4f7b-b35b-61e1ba5dfe8f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:36.915936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0e2c985-d4c9-4f7b-b35b-61e1ba5dfe8f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:36.916428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0e2c985-d4c9-4f7b-b35b-61e1ba5dfe8f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:36.916927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0e2c985-d4c9-4f7b-b35b-61e1ba5dfe8f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:36.917612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0e2c985-d4c9-4f7b-b35b-61e1ba5dfe8f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:36.918074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0e2c985-d4c9-4f7b-b35b-61e1ba5dfe8f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:36.918502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c0e2c985-d4c9-4f7b-b35b-61e1ba5dfe8f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:37.004819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-20e4f6ea-cd4e-480e-8baf-e605119f2d7b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:37.012780 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-20e4f6ea-cd4e-480e-8baf-e605119f2d7b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.166321 Sep 27 10:29:37.013330 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2759/5978] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:36 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:37.059287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c0e2c985-d4c9-4f7b-b35b-61e1ba5dfe8f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:37.067845 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c0e2c985-d4c9-4f7b-b35b-61e1ba5dfe8f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1265 microversion: 2.1 time: 0.164173 Sep 27 10:29:37.068406 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3220/5979] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:36 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1265 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:38.039171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7706f321-9d12-4f4f-b313-651b2b9b4236 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:38.039917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7706f321-9d12-4f4f-b313-651b2b9b4236 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:38.050770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7706f321-9d12-4f4f-b313-651b2b9b4236 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:38.051607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7706f321-9d12-4f4f-b313-651b2b9b4236 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:38.052201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7706f321-9d12-4f4f-b313-651b2b9b4236 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:38.052862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7706f321-9d12-4f4f-b313-651b2b9b4236 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:38.053349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7706f321-9d12-4f4f-b313-651b2b9b4236 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:38.053802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7706f321-9d12-4f4f-b313-651b2b9b4236 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:38.091279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4f4d564a-b8ca-4458-9d06-2d2272091b11 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:38.092112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4f4d564a-b8ca-4458-9d06-2d2272091b11 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:38.100002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f4d564a-b8ca-4458-9d06-2d2272091b11 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:38.100953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f4d564a-b8ca-4458-9d06-2d2272091b11 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:38.101457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f4d564a-b8ca-4458-9d06-2d2272091b11 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:38.102246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f4d564a-b8ca-4458-9d06-2d2272091b11 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:38.102758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f4d564a-b8ca-4458-9d06-2d2272091b11 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:38.103190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f4d564a-b8ca-4458-9d06-2d2272091b11 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:38.190619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7706f321-9d12-4f4f-b313-651b2b9b4236 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:38.199382 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7706f321-9d12-4f4f-b313-651b2b9b4236 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.166574 Sep 27 10:29:38.200084 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2760/5980] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:38 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:38.242352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4f4d564a-b8ca-4458-9d06-2d2272091b11 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:38.253300 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4f4d564a-b8ca-4458-9d06-2d2272091b11 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1265 microversion: 2.1 time: 0.166238 Sep 27 10:29:38.254258 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3221/5981] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:38 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1265 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:39.226175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e8e0e385-458e-44e6-8edf-da4dd68894d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:39.226758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e8e0e385-458e-44e6-8edf-da4dd68894d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:39.234158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e0e385-458e-44e6-8edf-da4dd68894d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:39.234670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e0e385-458e-44e6-8edf-da4dd68894d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:39.235131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e0e385-458e-44e6-8edf-da4dd68894d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:39.235880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e0e385-458e-44e6-8edf-da4dd68894d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:39.236364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e0e385-458e-44e6-8edf-da4dd68894d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:39.236877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e8e0e385-458e-44e6-8edf-da4dd68894d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:39.280345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7112d0f6-4e26-4b11-8dd5-e082bcdd017a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:39.280956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7112d0f6-4e26-4b11-8dd5-e082bcdd017a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:39.289127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7112d0f6-4e26-4b11-8dd5-e082bcdd017a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:39.289599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7112d0f6-4e26-4b11-8dd5-e082bcdd017a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:39.290036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7112d0f6-4e26-4b11-8dd5-e082bcdd017a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:39.290742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7112d0f6-4e26-4b11-8dd5-e082bcdd017a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:39.291199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7112d0f6-4e26-4b11-8dd5-e082bcdd017a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:39.291622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7112d0f6-4e26-4b11-8dd5-e082bcdd017a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:39.386001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e8e0e385-458e-44e6-8edf-da4dd68894d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:39.394486 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e8e0e385-458e-44e6-8edf-da4dd68894d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.173961 Sep 27 10:29:39.400277 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2761/5982] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:39 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:39.433334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7112d0f6-4e26-4b11-8dd5-e082bcdd017a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:39.441772 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7112d0f6-4e26-4b11-8dd5-e082bcdd017a tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1265 microversion: 2.1 time: 0.168041 Sep 27 10:29:39.442351 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3222/5983] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:39 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1265 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:40.425844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9579914a-664f-48d4-aaf0-c7c2202c407d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:40.426517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9579914a-664f-48d4-aaf0-c7c2202c407d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:40.433923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9579914a-664f-48d4-aaf0-c7c2202c407d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:40.434451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9579914a-664f-48d4-aaf0-c7c2202c407d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:40.435021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9579914a-664f-48d4-aaf0-c7c2202c407d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:40.435920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9579914a-664f-48d4-aaf0-c7c2202c407d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:40.436454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9579914a-664f-48d4-aaf0-c7c2202c407d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:40.436922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9579914a-664f-48d4-aaf0-c7c2202c407d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:40.465050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-209ddcd0-b509-463d-a86b-e91d33b85f94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:40.465991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-209ddcd0-b509-463d-a86b-e91d33b85f94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:40.473257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209ddcd0-b509-463d-a86b-e91d33b85f94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:40.473753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209ddcd0-b509-463d-a86b-e91d33b85f94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:40.474387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209ddcd0-b509-463d-a86b-e91d33b85f94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:40.475109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209ddcd0-b509-463d-a86b-e91d33b85f94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:40.475571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209ddcd0-b509-463d-a86b-e91d33b85f94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:40.476004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-209ddcd0-b509-463d-a86b-e91d33b85f94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:40.570873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9579914a-664f-48d4-aaf0-c7c2202c407d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:40.579362 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9579914a-664f-48d4-aaf0-c7c2202c407d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.158558 Sep 27 10:29:40.580016 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2762/5984] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:40 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:40.615736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-209ddcd0-b509-463d-a86b-e91d33b85f94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:40.623886 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-209ddcd0-b509-463d-a86b-e91d33b85f94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1265 microversion: 2.1 time: 0.162840 Sep 27 10:29:40.624506 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3223/5985] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:40 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1265 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:29:41.153667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-17e9ff2f-014d-4c5e-97bb-bc93ee14ce25 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:41.154203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-17e9ff2f-014d-4c5e-97bb-bc93ee14ce25 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:41.160792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17e9ff2f-014d-4c5e-97bb-bc93ee14ce25 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:41.161301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17e9ff2f-014d-4c5e-97bb-bc93ee14ce25 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:41.161867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17e9ff2f-014d-4c5e-97bb-bc93ee14ce25 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:41.162592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17e9ff2f-014d-4c5e-97bb-bc93ee14ce25 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:41.163063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17e9ff2f-014d-4c5e-97bb-bc93ee14ce25 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:41.163510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17e9ff2f-014d-4c5e-97bb-bc93ee14ce25 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:41.412993 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-17e9ff2f-014d-4c5e-97bb-bc93ee14ce25 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1832 microversion: 2.1 time: 0.263435 Sep 27 10:29:41.413897 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2763/5986] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:41 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1832 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:41.435661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-89be73a1-bec4-41d3-8cc1-b8920d354106 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:41.436334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-89be73a1-bec4-41d3-8cc1-b8920d354106 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:41.442978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89be73a1-bec4-41d3-8cc1-b8920d354106 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:41.443441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89be73a1-bec4-41d3-8cc1-b8920d354106 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:41.443922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89be73a1-bec4-41d3-8cc1-b8920d354106 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:41.444691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89be73a1-bec4-41d3-8cc1-b8920d354106 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:41.445165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89be73a1-bec4-41d3-8cc1-b8920d354106 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:41.445571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-89be73a1-bec4-41d3-8cc1-b8920d354106 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:41.592021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-89be73a1-bec4-41d3-8cc1-b8920d354106 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:41.600924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-89be73a1-bec4-41d3-8cc1-b8920d354106 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1681 microversion: 2.1 time: 0.169132 Sep 27 10:29:41.602170 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3224/5987] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:41 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1681 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:41.603668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8b5c0154-0d06-45f9-ad7e-573a55bcae98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:41.604307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8b5c0154-0d06-45f9-ad7e-573a55bcae98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:41.614513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b5c0154-0d06-45f9-ad7e-573a55bcae98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:41.615039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b5c0154-0d06-45f9-ad7e-573a55bcae98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:41.623054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b5c0154-0d06-45f9-ad7e-573a55bcae98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:41.623941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b5c0154-0d06-45f9-ad7e-573a55bcae98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:41.624451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b5c0154-0d06-45f9-ad7e-573a55bcae98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:41.624998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b5c0154-0d06-45f9-ad7e-573a55bcae98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:41.648688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4be190a8-fe4b-49f3-a0ab-acb99bd4b71d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:41.649558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4be190a8-fe4b-49f3-a0ab-acb99bd4b71d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:41.661492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4be190a8-fe4b-49f3-a0ab-acb99bd4b71d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:41.663440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4be190a8-fe4b-49f3-a0ab-acb99bd4b71d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:41.663903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4be190a8-fe4b-49f3-a0ab-acb99bd4b71d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:41.665134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4be190a8-fe4b-49f3-a0ab-acb99bd4b71d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:41.665667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4be190a8-fe4b-49f3-a0ab-acb99bd4b71d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:41.666185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4be190a8-fe4b-49f3-a0ab-acb99bd4b71d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:41.779249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8b5c0154-0d06-45f9-ad7e-573a55bcae98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:41.790415 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8b5c0154-0d06-45f9-ad7e-573a55bcae98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.190766 Sep 27 10:29:41.805759 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2764/5988] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:41 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:41.815898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e86993cb-cb00-4e31-8b5e-36b0252a9ea8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:41.818688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e86993cb-cb00-4e31-8b5e-36b0252a9ea8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:41.820699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:41.826106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e86993cb-cb00-4e31-8b5e-36b0252a9ea8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:41.826926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e86993cb-cb00-4e31-8b5e-36b0252a9ea8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:41.833053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e86993cb-cb00-4e31-8b5e-36b0252a9ea8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:41.833053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e86993cb-cb00-4e31-8b5e-36b0252a9ea8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:41.833053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e86993cb-cb00-4e31-8b5e-36b0252a9ea8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:41.833053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e86993cb-cb00-4e31-8b5e-36b0252a9ea8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:41.857141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4be190a8-fe4b-49f3-a0ab-acb99bd4b71d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:41.865885 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4be190a8-fe4b-49f3-a0ab-acb99bd4b71d tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1265 microversion: 2.1 time: 0.222880 Sep 27 10:29:41.866484 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3225/5989] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:41 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1265 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:41.871313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-92443138-41a9-499d-9410-c2900e5ae33f req-63f7ce5c-4d20-4964-bb05-8f093772e939 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9f14e82f-c065-4b4e-bb03-c94b57bc21e7", "name": "network-vif-plugged", "status": "completed", "tag": "86766b79-cf12-4620-813b-644b95940fbc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:41.881390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-92443138-41a9-499d-9410-c2900e5ae33f req-63f7ce5c-4d20-4964-bb05-8f093772e939 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:41.881879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-92443138-41a9-499d-9410-c2900e5ae33f req-63f7ce5c-4d20-4964-bb05-8f093772e939 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:41.882308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-92443138-41a9-499d-9410-c2900e5ae33f req-63f7ce5c-4d20-4964-bb05-8f093772e939 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:41.893467 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-92443138-41a9-499d-9410-c2900e5ae33f req-63f7ce5c-4d20-4964-bb05-8f093772e939 service nova] Creating event network-vif-plugged:86766b79-cf12-4620-813b-644b95940fbc for instance 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 on nch2-master-1 Sep 27 10:29:41.898529 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-92443138-41a9-499d-9410-c2900e5ae33f req-63f7ce5c-4d20-4964-bb05-8f093772e939 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030929 Sep 27 10:29:41.899158 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3226/5990] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:41.936471 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e86993cb-cb00-4e31-8b5e-36b0252a9ea8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3/action" status: 200 len: 32644 microversion: 2.1 time: 0.129236 Sep 27 10:29:41.937112 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2765/5991] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:29:41 2022] POST /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3/action => generated 32644 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:29:42.107067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:42.107749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:42.114554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:42.115047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:42.115494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:42.116205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:42.116697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:42.117119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:42.212915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '5436a87ddec64a47bde28e63c0a3a368', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd4cd350726034f9985727755c395c181', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:42.213845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] flavor_id is None. Assuming migration. {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4196}} Sep 27 10:29:42.214499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:29:42.405081 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a0866a8f-ade4-4b15-b7be-123b0ef3d0da tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3/action" status: 202 len: 0 microversion: 2.1 time: 0.302248 Sep 27 10:29:42.405859 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3227/5992] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:29:42 2022] POST /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3/action => generated 0 bytes in 304 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:29:42.415143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-51924d25-37ce-4131-89d5-5815b62c6d86 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:42.415974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-51924d25-37ce-4131-89d5-5815b62c6d86 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:42.419600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:42.424487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51924d25-37ce-4131-89d5-5815b62c6d86 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:42.425029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51924d25-37ce-4131-89d5-5815b62c6d86 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:42.425482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51924d25-37ce-4131-89d5-5815b62c6d86 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:42.426518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51924d25-37ce-4131-89d5-5815b62c6d86 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:42.427300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51924d25-37ce-4131-89d5-5815b62c6d86 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:42.427598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-51924d25-37ce-4131-89d5-5815b62c6d86 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:42.572698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-51924d25-37ce-4131-89d5-5815b62c6d86 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:42.580784 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-51924d25-37ce-4131-89d5-5815b62c6d86 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1690 microversion: 2.1 time: 0.170114 Sep 27 10:29:42.581371 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2766/5993] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:42 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1690 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:42.818206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-730554f4-d12b-42b7-ab24-3cfd872545de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:42.818892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-730554f4-d12b-42b7-ab24-3cfd872545de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:42.826976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-730554f4-d12b-42b7-ab24-3cfd872545de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:42.827471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-730554f4-d12b-42b7-ab24-3cfd872545de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:42.827915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-730554f4-d12b-42b7-ab24-3cfd872545de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:42.828624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-730554f4-d12b-42b7-ab24-3cfd872545de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:42.829114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-730554f4-d12b-42b7-ab24-3cfd872545de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:42.829538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-730554f4-d12b-42b7-ab24-3cfd872545de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:42.889566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d0ec0fb8-e7af-4387-922d-238b51d475d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:42.890670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d0ec0fb8-e7af-4387-922d-238b51d475d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:42.902507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ec0fb8-e7af-4387-922d-238b51d475d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:42.903027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ec0fb8-e7af-4387-922d-238b51d475d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:42.903472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ec0fb8-e7af-4387-922d-238b51d475d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:42.904438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ec0fb8-e7af-4387-922d-238b51d475d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:42.904966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ec0fb8-e7af-4387-922d-238b51d475d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:42.906219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d0ec0fb8-e7af-4387-922d-238b51d475d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:42.968453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-730554f4-d12b-42b7-ab24-3cfd872545de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:42.978909 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-730554f4-d12b-42b7-ab24-3cfd872545de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.164873 Sep 27 10:29:42.979506 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3228/5994] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:42 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:43.061848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d0ec0fb8-e7af-4387-922d-238b51d475d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:43.070273 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d0ec0fb8-e7af-4387-922d-238b51d475d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1265 microversion: 2.1 time: 0.184755 Sep 27 10:29:43.071220 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2767/5995] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:42 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1265 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:43.605638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3945a9e-3f80-4053-ae0c-dd248c843dc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:43.606268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3945a9e-3f80-4053-ae0c-dd248c843dc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:43.613134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3945a9e-3f80-4053-ae0c-dd248c843dc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:43.613620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3945a9e-3f80-4053-ae0c-dd248c843dc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:43.614099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3945a9e-3f80-4053-ae0c-dd248c843dc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:43.614846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3945a9e-3f80-4053-ae0c-dd248c843dc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:43.615316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3945a9e-3f80-4053-ae0c-dd248c843dc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:43.615751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3945a9e-3f80-4053-ae0c-dd248c843dc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:43.753777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b3945a9e-3f80-4053-ae0c-dd248c843dc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:43.763561 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3945a9e-3f80-4053-ae0c-dd248c843dc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1690 microversion: 2.1 time: 0.161773 Sep 27 10:29:43.764219 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3229/5996] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:43 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1690 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:44.003089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7a3d0a20-2874-4c95-a4e2-c633965bfc99 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:44.003809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7a3d0a20-2874-4c95-a4e2-c633965bfc99 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:44.011157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3d0a20-2874-4c95-a4e2-c633965bfc99 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:44.011660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3d0a20-2874-4c95-a4e2-c633965bfc99 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:44.012120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3d0a20-2874-4c95-a4e2-c633965bfc99 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:44.013032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3d0a20-2874-4c95-a4e2-c633965bfc99 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:44.013514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3d0a20-2874-4c95-a4e2-c633965bfc99 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:44.013965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a3d0a20-2874-4c95-a4e2-c633965bfc99 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:44.093946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-16b5c88c-0ac4-4f61-8753-1df7c68b9e5b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:44.094684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-16b5c88c-0ac4-4f61-8753-1df7c68b9e5b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:44.102080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16b5c88c-0ac4-4f61-8753-1df7c68b9e5b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:44.102573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16b5c88c-0ac4-4f61-8753-1df7c68b9e5b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:44.102995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16b5c88c-0ac4-4f61-8753-1df7c68b9e5b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:44.103730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16b5c88c-0ac4-4f61-8753-1df7c68b9e5b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:44.104192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16b5c88c-0ac4-4f61-8753-1df7c68b9e5b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:44.104618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16b5c88c-0ac4-4f61-8753-1df7c68b9e5b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:44.239915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a3d0a20-2874-4c95-a4e2-c633965bfc99 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:44.249050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-16b5c88c-0ac4-4f61-8753-1df7c68b9e5b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:44.255022 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7a3d0a20-2874-4c95-a4e2-c633965bfc99 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.255863 Sep 27 10:29:44.255619 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2768/5997] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:43 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:44.268204 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-16b5c88c-0ac4-4f61-8753-1df7c68b9e5b tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1453 microversion: 2.1 time: 0.171931 Sep 27 10:29:44.268204 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3230/5998] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:44 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1453 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:44.288200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5fde0b26-bd8c-443d-833f-cb53e61052d5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:44.289152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5fde0b26-bd8c-443d-833f-cb53e61052d5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:44.295647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fde0b26-bd8c-443d-833f-cb53e61052d5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:44.296209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fde0b26-bd8c-443d-833f-cb53e61052d5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:44.296698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fde0b26-bd8c-443d-833f-cb53e61052d5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:44.297433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fde0b26-bd8c-443d-833f-cb53e61052d5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:44.297908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fde0b26-bd8c-443d-833f-cb53e61052d5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:44.298354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5fde0b26-bd8c-443d-833f-cb53e61052d5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:44.554789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5fde0b26-bd8c-443d-833f-cb53e61052d5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:44.564584 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5fde0b26-bd8c-443d-833f-cb53e61052d5 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1453 microversion: 2.1 time: 0.283950 Sep 27 10:29:44.565219 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2769/5999] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:44 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1453 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:44.787864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6969f7d7-8066-4fd1-bc58-ecbd0cb65a99 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:44.788493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6969f7d7-8066-4fd1-bc58-ecbd0cb65a99 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:44.795067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6969f7d7-8066-4fd1-bc58-ecbd0cb65a99 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:44.795578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6969f7d7-8066-4fd1-bc58-ecbd0cb65a99 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:44.796024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6969f7d7-8066-4fd1-bc58-ecbd0cb65a99 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:44.796835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6969f7d7-8066-4fd1-bc58-ecbd0cb65a99 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:44.797302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6969f7d7-8066-4fd1-bc58-ecbd0cb65a99 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:44.797742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6969f7d7-8066-4fd1-bc58-ecbd0cb65a99 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:44.802433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:45.033964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6969f7d7-8066-4fd1-bc58-ecbd0cb65a99 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:45.043005 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6969f7d7-8066-4fd1-bc58-ecbd0cb65a99 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1695 microversion: 2.1 time: 0.259255 Sep 27 10:29:45.043624 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3231/6000] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:44 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1695 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:29:45.284423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-824b8828-d1a5-4c0f-aace-8943c9600d98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:45.285908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-824b8828-d1a5-4c0f-aace-8943c9600d98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:45.293835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8828-d1a5-4c0f-aace-8943c9600d98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:45.294671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8828-d1a5-4c0f-aace-8943c9600d98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:45.295598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8828-d1a5-4c0f-aace-8943c9600d98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:45.296684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8828-d1a5-4c0f-aace-8943c9600d98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:45.297377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8828-d1a5-4c0f-aace-8943c9600d98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:45.298026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-824b8828-d1a5-4c0f-aace-8943c9600d98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:45.455935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-824b8828-d1a5-4c0f-aace-8943c9600d98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:45.460001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:45.465549 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-824b8828-d1a5-4c0f-aace-8943c9600d98 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.185298 Sep 27 10:29:45.466136 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2770/6001] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:45 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:46.068844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c47c5b04-c5b6-4557-8f46-fb5fffc56c49 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:46.069708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c47c5b04-c5b6-4557-8f46-fb5fffc56c49 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:46.078460 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47c5b04-c5b6-4557-8f46-fb5fffc56c49 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:46.079074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47c5b04-c5b6-4557-8f46-fb5fffc56c49 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:46.079625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47c5b04-c5b6-4557-8f46-fb5fffc56c49 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:46.080420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47c5b04-c5b6-4557-8f46-fb5fffc56c49 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:46.081144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47c5b04-c5b6-4557-8f46-fb5fffc56c49 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:46.081605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c47c5b04-c5b6-4557-8f46-fb5fffc56c49 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:46.327728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c47c5b04-c5b6-4557-8f46-fb5fffc56c49 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:46.336469 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c47c5b04-c5b6-4557-8f46-fb5fffc56c49 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1695 microversion: 2.1 time: 0.272062 Sep 27 10:29:46.337112 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3232/6002] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:46 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1695 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:46.490844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5bcdc619-87e5-494e-a9c1-cf87a777dbf2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:46.491672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5bcdc619-87e5-494e-a9c1-cf87a777dbf2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:46.499202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcdc619-87e5-494e-a9c1-cf87a777dbf2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:46.499724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcdc619-87e5-494e-a9c1-cf87a777dbf2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:46.500181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcdc619-87e5-494e-a9c1-cf87a777dbf2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:46.500944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcdc619-87e5-494e-a9c1-cf87a777dbf2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:46.501424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcdc619-87e5-494e-a9c1-cf87a777dbf2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:46.502225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5bcdc619-87e5-494e-a9c1-cf87a777dbf2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:46.558920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f430b5a1-1d51-41c3-8e25-da36135f6a05 req-002f6415-2a12-438a-a51b-a8952feee556 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9f14e82f-c065-4b4e-bb03-c94b57bc21e7", "tag": "86766b79-cf12-4620-813b-644b95940fbc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:46.569168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f430b5a1-1d51-41c3-8e25-da36135f6a05 req-002f6415-2a12-438a-a51b-a8952feee556 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:46.569745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f430b5a1-1d51-41c3-8e25-da36135f6a05 req-002f6415-2a12-438a-a51b-a8952feee556 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:46.570249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f430b5a1-1d51-41c3-8e25-da36135f6a05 req-002f6415-2a12-438a-a51b-a8952feee556 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:46.582007 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f430b5a1-1d51-41c3-8e25-da36135f6a05 req-002f6415-2a12-438a-a51b-a8952feee556 service nova] Creating event network-changed:86766b79-cf12-4620-813b-644b95940fbc for instance 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 on nch2-master-1 Sep 27 10:29:46.588000 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f430b5a1-1d51-41c3-8e25-da36135f6a05 req-002f6415-2a12-438a-a51b-a8952feee556 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032923 Sep 27 10:29:46.588871 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3233/6003] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:46.645889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5bcdc619-87e5-494e-a9c1-cf87a777dbf2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:46.654928 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5bcdc619-87e5-494e-a9c1-cf87a777dbf2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.169045 Sep 27 10:29:46.669172 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2771/6004] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:46 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:47.363844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3ebfb05-60ee-47dc-9cd6-2f766baa37ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:47.365360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e3ebfb05-60ee-47dc-9cd6-2f766baa37ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:47.372163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3ebfb05-60ee-47dc-9cd6-2f766baa37ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:47.372683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3ebfb05-60ee-47dc-9cd6-2f766baa37ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:47.373134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3ebfb05-60ee-47dc-9cd6-2f766baa37ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:47.373856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3ebfb05-60ee-47dc-9cd6-2f766baa37ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:47.374316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3ebfb05-60ee-47dc-9cd6-2f766baa37ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:47.374724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3ebfb05-60ee-47dc-9cd6-2f766baa37ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:47.521521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e3ebfb05-60ee-47dc-9cd6-2f766baa37ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:47.530065 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e3ebfb05-60ee-47dc-9cd6-2f766baa37ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1695 microversion: 2.1 time: 0.172068 Sep 27 10:29:47.530655 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3234/6005] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:47 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1695 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:47.679375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0cc548af-2fbf-4467-8c10-ebef50b913b7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:47.680248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0cc548af-2fbf-4467-8c10-ebef50b913b7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:47.687850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cc548af-2fbf-4467-8c10-ebef50b913b7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:47.688359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cc548af-2fbf-4467-8c10-ebef50b913b7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:47.689022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cc548af-2fbf-4467-8c10-ebef50b913b7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:47.689763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cc548af-2fbf-4467-8c10-ebef50b913b7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:47.690278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cc548af-2fbf-4467-8c10-ebef50b913b7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:47.690822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cc548af-2fbf-4467-8c10-ebef50b913b7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:47.828258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0cc548af-2fbf-4467-8c10-ebef50b913b7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:47.836984 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0cc548af-2fbf-4467-8c10-ebef50b913b7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.162014 Sep 27 10:29:47.837582 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2772/6006] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:47 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:29:48.555537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-12ec581b-d1b6-4561-beca-31a2421f0002 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:48.556207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12ec581b-d1b6-4561-beca-31a2421f0002 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:48.559131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=75,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:29:48.565225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12ec581b-d1b6-4561-beca-31a2421f0002 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:48.565723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12ec581b-d1b6-4561-beca-31a2421f0002 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:48.566188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12ec581b-d1b6-4561-beca-31a2421f0002 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:48.566887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12ec581b-d1b6-4561-beca-31a2421f0002 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:48.567344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12ec581b-d1b6-4561-beca-31a2421f0002 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:48.567760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12ec581b-d1b6-4561-beca-31a2421f0002 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:48.711641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12ec581b-d1b6-4561-beca-31a2421f0002 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:48.723925 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-12ec581b-d1b6-4561-beca-31a2421f0002 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1695 microversion: 2.1 time: 0.172292 Sep 27 10:29:48.724823 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3235/6007] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:48 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1695 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:48.864957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a26a7d45-cf69-413f-80a1-7b3d98b85395 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:48.865709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a26a7d45-cf69-413f-80a1-7b3d98b85395 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:48.876424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a26a7d45-cf69-413f-80a1-7b3d98b85395 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:48.877268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a26a7d45-cf69-413f-80a1-7b3d98b85395 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:48.877911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a26a7d45-cf69-413f-80a1-7b3d98b85395 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:48.879143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a26a7d45-cf69-413f-80a1-7b3d98b85395 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:48.879876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a26a7d45-cf69-413f-80a1-7b3d98b85395 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:48.880490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a26a7d45-cf69-413f-80a1-7b3d98b85395 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:49.018620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a26a7d45-cf69-413f-80a1-7b3d98b85395 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:49.027099 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a26a7d45-cf69-413f-80a1-7b3d98b85395 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.168602 Sep 27 10:29:49.027699 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2773/6008] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:48 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:49.750309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dd073450-d610-4282-a103-d6df99983e0f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:49.750898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dd073450-d610-4282-a103-d6df99983e0f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:49.760115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd073450-d610-4282-a103-d6df99983e0f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:49.760614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd073450-d610-4282-a103-d6df99983e0f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:49.761088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd073450-d610-4282-a103-d6df99983e0f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:49.761793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd073450-d610-4282-a103-d6df99983e0f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:49.762248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd073450-d610-4282-a103-d6df99983e0f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:49.762679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd073450-d610-4282-a103-d6df99983e0f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:49.910044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dd073450-d610-4282-a103-d6df99983e0f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:49.918926 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dd073450-d610-4282-a103-d6df99983e0f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1695 microversion: 2.1 time: 0.174788 Sep 27 10:29:49.920443 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3236/6009] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:49 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1695 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:50.049653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a9a19b59-3970-4e81-910e-013db794a96b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:50.050299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a9a19b59-3970-4e81-910e-013db794a96b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:50.058172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a19b59-3970-4e81-910e-013db794a96b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:50.058679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a19b59-3970-4e81-910e-013db794a96b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:50.059129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a19b59-3970-4e81-910e-013db794a96b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:50.059850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a19b59-3970-4e81-910e-013db794a96b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:50.060321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a19b59-3970-4e81-910e-013db794a96b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:50.060797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9a19b59-3970-4e81-910e-013db794a96b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:50.206643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a9a19b59-3970-4e81-910e-013db794a96b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:50.214982 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a9a19b59-3970-4e81-910e-013db794a96b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.169312 Sep 27 10:29:50.215907 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2774/6010] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:50 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:50.943708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a9476ba0-0c1b-45c9-8b6e-e380de28b224 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:50.944384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a9476ba0-0c1b-45c9-8b6e-e380de28b224 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:50.951875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9476ba0-0c1b-45c9-8b6e-e380de28b224 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:50.952375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9476ba0-0c1b-45c9-8b6e-e380de28b224 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:50.952843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9476ba0-0c1b-45c9-8b6e-e380de28b224 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:50.953590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9476ba0-0c1b-45c9-8b6e-e380de28b224 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:50.954053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9476ba0-0c1b-45c9-8b6e-e380de28b224 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:50.954577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9476ba0-0c1b-45c9-8b6e-e380de28b224 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:51.106604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a9476ba0-0c1b-45c9-8b6e-e380de28b224 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:51.114631 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a9476ba0-0c1b-45c9-8b6e-e380de28b224 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1695 microversion: 2.1 time: 0.174939 Sep 27 10:29:51.115265 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3237/6011] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:50 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1695 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:51.245189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5dbf6834-bf18-421c-bc3f-a354a7fd085d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:51.246018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5dbf6834-bf18-421c-bc3f-a354a7fd085d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:51.256122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dbf6834-bf18-421c-bc3f-a354a7fd085d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:51.256966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dbf6834-bf18-421c-bc3f-a354a7fd085d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:51.257432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dbf6834-bf18-421c-bc3f-a354a7fd085d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:51.258156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dbf6834-bf18-421c-bc3f-a354a7fd085d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:51.258626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dbf6834-bf18-421c-bc3f-a354a7fd085d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:51.259071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dbf6834-bf18-421c-bc3f-a354a7fd085d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:51.407158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5dbf6834-bf18-421c-bc3f-a354a7fd085d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:51.419464 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5dbf6834-bf18-421c-bc3f-a354a7fd085d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.180080 Sep 27 10:29:51.420269 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2775/6012] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:51 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:52.140056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3603f47a-189e-4348-b632-7ecf2c05f974 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:52.140800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3603f47a-189e-4348-b632-7ecf2c05f974 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:52.149521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3603f47a-189e-4348-b632-7ecf2c05f974 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:52.149998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3603f47a-189e-4348-b632-7ecf2c05f974 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:52.150448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3603f47a-189e-4348-b632-7ecf2c05f974 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:52.151172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3603f47a-189e-4348-b632-7ecf2c05f974 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:52.151639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3603f47a-189e-4348-b632-7ecf2c05f974 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:52.152078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3603f47a-189e-4348-b632-7ecf2c05f974 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:52.295430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3603f47a-189e-4348-b632-7ecf2c05f974 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:52.304056 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3603f47a-189e-4348-b632-7ecf2c05f974 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1695 microversion: 2.1 time: 0.169500 Sep 27 10:29:52.304716 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3238/6013] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:52 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1695 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:52.447630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ec49546-042b-4795-ae54-94ae6478e37d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:52.448210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ec49546-042b-4795-ae54-94ae6478e37d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:52.455755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec49546-042b-4795-ae54-94ae6478e37d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:52.456268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec49546-042b-4795-ae54-94ae6478e37d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:52.456753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec49546-042b-4795-ae54-94ae6478e37d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:52.457530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec49546-042b-4795-ae54-94ae6478e37d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:52.458007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec49546-042b-4795-ae54-94ae6478e37d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:52.458460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ec49546-042b-4795-ae54-94ae6478e37d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:52.599406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2ec49546-042b-4795-ae54-94ae6478e37d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:52.608849 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ec49546-042b-4795-ae54-94ae6478e37d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.165419 Sep 27 10:29:52.609718 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2776/6014] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:52 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:53.331208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af8cc2bd-c32c-4f1b-acd1-d5f95bd34329 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:53.332612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af8cc2bd-c32c-4f1b-acd1-d5f95bd34329 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:53.344049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af8cc2bd-c32c-4f1b-acd1-d5f95bd34329 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:53.344866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af8cc2bd-c32c-4f1b-acd1-d5f95bd34329 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:53.345598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af8cc2bd-c32c-4f1b-acd1-d5f95bd34329 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:53.346809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af8cc2bd-c32c-4f1b-acd1-d5f95bd34329 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:53.347566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af8cc2bd-c32c-4f1b-acd1-d5f95bd34329 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:53.348281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af8cc2bd-c32c-4f1b-acd1-d5f95bd34329 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:53.492446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af8cc2bd-c32c-4f1b-acd1-d5f95bd34329 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:53.501509 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af8cc2bd-c32c-4f1b-acd1-d5f95bd34329 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1695 microversion: 2.1 time: 0.176990 Sep 27 10:29:53.502198 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3239/6015] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:53 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1695 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:53.631835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd2c8f79-6786-4b37-9311-41682b20ec5c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:53.632459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd2c8f79-6786-4b37-9311-41682b20ec5c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:53.640431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd2c8f79-6786-4b37-9311-41682b20ec5c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:53.640962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd2c8f79-6786-4b37-9311-41682b20ec5c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:53.641414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd2c8f79-6786-4b37-9311-41682b20ec5c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:53.642133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd2c8f79-6786-4b37-9311-41682b20ec5c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:53.642606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd2c8f79-6786-4b37-9311-41682b20ec5c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:53.643047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd2c8f79-6786-4b37-9311-41682b20ec5c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:53.781558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd2c8f79-6786-4b37-9311-41682b20ec5c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:53.792642 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd2c8f79-6786-4b37-9311-41682b20ec5c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.164499 Sep 27 10:29:53.793267 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2777/6016] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:53 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:54.525632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33fdedff-a89a-4c4e-96b6-528bdead64a0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:54.526272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33fdedff-a89a-4c4e-96b6-528bdead64a0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:54.534426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33fdedff-a89a-4c4e-96b6-528bdead64a0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:54.534954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33fdedff-a89a-4c4e-96b6-528bdead64a0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:54.535395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33fdedff-a89a-4c4e-96b6-528bdead64a0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:54.536177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33fdedff-a89a-4c4e-96b6-528bdead64a0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:54.536632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33fdedff-a89a-4c4e-96b6-528bdead64a0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:54.537143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33fdedff-a89a-4c4e-96b6-528bdead64a0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:54.688287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33fdedff-a89a-4c4e-96b6-528bdead64a0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:54.697113 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33fdedff-a89a-4c4e-96b6-528bdead64a0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1695 microversion: 2.1 time: 0.175680 Sep 27 10:29:54.697737 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3240/6017] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:54 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1695 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:54.817345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e492433f-ebc4-4af9-8792-d24406be6b78 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:54.817945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e492433f-ebc4-4af9-8792-d24406be6b78 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:54.825551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e492433f-ebc4-4af9-8792-d24406be6b78 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:54.826071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e492433f-ebc4-4af9-8792-d24406be6b78 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:54.826530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e492433f-ebc4-4af9-8792-d24406be6b78 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:54.827265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e492433f-ebc4-4af9-8792-d24406be6b78 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:54.827739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e492433f-ebc4-4af9-8792-d24406be6b78 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:54.828209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e492433f-ebc4-4af9-8792-d24406be6b78 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:54.977166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e492433f-ebc4-4af9-8792-d24406be6b78 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:54.985501 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e492433f-ebc4-4af9-8792-d24406be6b78 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.173384 Sep 27 10:29:54.986092 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2778/6018] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:54 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:55.722689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f9aa008d-f5b5-4962-8cb8-fa51154378bd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:55.723355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f9aa008d-f5b5-4962-8cb8-fa51154378bd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:55.730803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9aa008d-f5b5-4962-8cb8-fa51154378bd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:55.731298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9aa008d-f5b5-4962-8cb8-fa51154378bd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:55.731741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9aa008d-f5b5-4962-8cb8-fa51154378bd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:55.732544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9aa008d-f5b5-4962-8cb8-fa51154378bd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:55.733091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9aa008d-f5b5-4962-8cb8-fa51154378bd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:55.733537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9aa008d-f5b5-4962-8cb8-fa51154378bd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:55.879686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f9aa008d-f5b5-4962-8cb8-fa51154378bd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:55.887674 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f9aa008d-f5b5-4962-8cb8-fa51154378bd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1695 microversion: 2.1 time: 0.169780 Sep 27 10:29:55.888261 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3241/6019] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:55 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1695 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:56.009502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-29bc8ac8-e343-49d0-ba9d-53fadb648a66 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:56.010141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-29bc8ac8-e343-49d0-ba9d-53fadb648a66 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:56.018074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29bc8ac8-e343-49d0-ba9d-53fadb648a66 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:56.018684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29bc8ac8-e343-49d0-ba9d-53fadb648a66 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:56.019336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29bc8ac8-e343-49d0-ba9d-53fadb648a66 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:56.020184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29bc8ac8-e343-49d0-ba9d-53fadb648a66 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:56.020690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29bc8ac8-e343-49d0-ba9d-53fadb648a66 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:56.021172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29bc8ac8-e343-49d0-ba9d-53fadb648a66 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:56.163529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-29bc8ac8-e343-49d0-ba9d-53fadb648a66 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:56.171927 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-29bc8ac8-e343-49d0-ba9d-53fadb648a66 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.166788 Sep 27 10:29:56.172502 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2779/6020] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:56 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:29:56.914390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-26a59446-ba75-4e49-88c8-9fcee1c1e87e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:56.915261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-26a59446-ba75-4e49-88c8-9fcee1c1e87e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:56.925101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a59446-ba75-4e49-88c8-9fcee1c1e87e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:56.925775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a59446-ba75-4e49-88c8-9fcee1c1e87e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:56.926538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a59446-ba75-4e49-88c8-9fcee1c1e87e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:56.927260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a59446-ba75-4e49-88c8-9fcee1c1e87e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:56.927708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a59446-ba75-4e49-88c8-9fcee1c1e87e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:56.928136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26a59446-ba75-4e49-88c8-9fcee1c1e87e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:57.068179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-26a59446-ba75-4e49-88c8-9fcee1c1e87e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:57.076999 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-26a59446-ba75-4e49-88c8-9fcee1c1e87e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1695 microversion: 2.1 time: 0.168382 Sep 27 10:29:57.077793 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3242/6021] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:56 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1695 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:57.197599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca2bfa83-8603-475d-9da7-9e62f38e30ef tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:57.198222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ca2bfa83-8603-475d-9da7-9e62f38e30ef tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:57.206606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca2bfa83-8603-475d-9da7-9e62f38e30ef tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:57.207136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca2bfa83-8603-475d-9da7-9e62f38e30ef tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:57.207612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca2bfa83-8603-475d-9da7-9e62f38e30ef tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:57.208596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca2bfa83-8603-475d-9da7-9e62f38e30ef tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:57.209467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca2bfa83-8603-475d-9da7-9e62f38e30ef tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:57.210099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca2bfa83-8603-475d-9da7-9e62f38e30ef tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:57.409612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ca2bfa83-8603-475d-9da7-9e62f38e30ef tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:57.418527 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ca2bfa83-8603-475d-9da7-9e62f38e30ef tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1523 microversion: 2.1 time: 0.227148 Sep 27 10:29:57.432724 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2780/6022] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:57 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1523 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:57.695574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3443d3e3-a998-42a0-9276-a9a17c585bd0 req-501d12b2-52ed-4620-b5cf-6d942c282fda service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39429342-b89d-443d-9645-17ec8e2869f3", "name": "network-vif-unplugged", "status": "completed", "tag": "b41f6e87-e1ce-49b6-a47a-5129ca582abc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:57.705559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3443d3e3-a998-42a0-9276-a9a17c585bd0 req-501d12b2-52ed-4620-b5cf-6d942c282fda service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:57.706018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3443d3e3-a998-42a0-9276-a9a17c585bd0 req-501d12b2-52ed-4620-b5cf-6d942c282fda service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:57.706453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3443d3e3-a998-42a0-9276-a9a17c585bd0 req-501d12b2-52ed-4620-b5cf-6d942c282fda service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:57.719134 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3443d3e3-a998-42a0-9276-a9a17c585bd0 req-501d12b2-52ed-4620-b5cf-6d942c282fda service nova] Creating event network-vif-unplugged:b41f6e87-e1ce-49b6-a47a-5129ca582abc for instance 39429342-b89d-443d-9645-17ec8e2869f3 on nch1-master-1 Sep 27 10:29:57.724850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-3443d3e3-a998-42a0-9276-a9a17c585bd0 req-501d12b2-52ed-4620-b5cf-6d942c282fda service nova] Instance 39429342-b89d-443d-9645-17ec8e2869f3 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:29:57.735701 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3443d3e3-a998-42a0-9276-a9a17c585bd0 req-501d12b2-52ed-4620-b5cf-6d942c282fda service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.044099 Sep 27 10:29:57.736350 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3243/6023] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:57 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:29:58.102027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-437a5387-fd29-4542-a3f2-77caa7f84f93 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:58.102651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-437a5387-fd29-4542-a3f2-77caa7f84f93 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:58.108940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437a5387-fd29-4542-a3f2-77caa7f84f93 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:58.109427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437a5387-fd29-4542-a3f2-77caa7f84f93 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:58.109882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437a5387-fd29-4542-a3f2-77caa7f84f93 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:58.110603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437a5387-fd29-4542-a3f2-77caa7f84f93 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:58.111073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437a5387-fd29-4542-a3f2-77caa7f84f93 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:58.111509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-437a5387-fd29-4542-a3f2-77caa7f84f93 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:58.359861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-437a5387-fd29-4542-a3f2-77caa7f84f93 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:58.368486 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-437a5387-fd29-4542-a3f2-77caa7f84f93 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1694 microversion: 2.1 time: 0.270481 Sep 27 10:29:58.369134 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2781/6024] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:58 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1694 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:29:58.444913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30d58754-fd3a-49fc-a4f5-3d5a83714f47 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:58.445768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-30d58754-fd3a-49fc-a4f5-3d5a83714f47 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:58.452211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d58754-fd3a-49fc-a4f5-3d5a83714f47 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:58.452702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d58754-fd3a-49fc-a4f5-3d5a83714f47 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:58.453177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d58754-fd3a-49fc-a4f5-3d5a83714f47 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:58.453960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d58754-fd3a-49fc-a4f5-3d5a83714f47 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:58.454361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d58754-fd3a-49fc-a4f5-3d5a83714f47 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:58.454790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30d58754-fd3a-49fc-a4f5-3d5a83714f47 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:58.587428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-30d58754-fd3a-49fc-a4f5-3d5a83714f47 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:58.595986 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-30d58754-fd3a-49fc-a4f5-3d5a83714f47 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1503 microversion: 2.1 time: 0.156377 Sep 27 10:29:58.596584 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3244/6025] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:58 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1503 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:58.620054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b54e5d04-ef15-4520-8bc9-3b05d2459b3c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:58.620872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b54e5d04-ef15-4520-8bc9-3b05d2459b3c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:58.627357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54e5d04-ef15-4520-8bc9-3b05d2459b3c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:58.627959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54e5d04-ef15-4520-8bc9-3b05d2459b3c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:58.628427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54e5d04-ef15-4520-8bc9-3b05d2459b3c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:58.629218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54e5d04-ef15-4520-8bc9-3b05d2459b3c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:58.629830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54e5d04-ef15-4520-8bc9-3b05d2459b3c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:58.630278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54e5d04-ef15-4520-8bc9-3b05d2459b3c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:58.844644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b54e5d04-ef15-4520-8bc9-3b05d2459b3c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:58.854841 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b54e5d04-ef15-4520-8bc9-3b05d2459b3c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1515 microversion: 2.1 time: 0.239644 Sep 27 10:29:58.855452 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2782/6026] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:58 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1515 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:58.880698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a79046dd-60fe-46b4-859e-038be0c1c475 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'action', calling method: >, body: {"rescue": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:58.883808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a79046dd-60fe-46b4-859e-038be0c1c475 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: abcc0758-4fcf-4259-8cf7-d8712cde8283] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:58.889516 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a79046dd-60fe-46b4-859e-038be0c1c475 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] HTTP exception thrown: Instance abcc0758-4fcf-4259-8cf7-d8712cde8283 could not be found. Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a79046dd-60fe-46b4-859e-038be0c1c475 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Request method failure captured: Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/abcc0758-4fcf-4259-8cf7-d8712cde8283/action HTTP/1.1 Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 14 Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtB5lZw26WfhuIIxDLfEGtJx8GdKac9mUecPJWt47RWTVgFEQNt2CRnQRU62Kxhrd_5bXczg0Q9QWsT4c85-tEEw3L_ZoXg7ttqo2fTxJvLnAMIld1Q9kkM9ItLXE6S_aDTtz0h96WsfSIhdls5sa9AFKLBsHqf09n27PObWbpVe7TQ Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_40247edd5a444fd8b1c937c13d39393a"}]}] Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6771df45b4ee4426b2f76a09eab2998a Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: {"rescue": {}} Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance abcc0758-4fcf-4259-8cf7-d8712cde8283 could not be found. Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'abcc0758-4fcf-4259-8cf7-d8712cde8283', 'body': {'rescue': {}}} Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance abcc0758-4fcf-4259-8cf7-d8712cde8283 could not be found. Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/rescue.py", line 53, in _rescue Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, id) Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance abcc0758-4fcf-4259-8cf7-d8712cde8283 could not be found. Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:58.902050 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:29:58.908012 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:29:58.908012 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:29:58.908012 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:29:58.908012 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:29:58.908012 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance abcc0758-4fcf-4259-8cf7-d8712cde8283 could not be found. Sep 27 10:29:58.908012 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:29:58.908012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a79046dd-60fe-46b4-859e-038be0c1c475 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Returning 404 to user: Instance abcc0758-4fcf-4259-8cf7-d8712cde8283 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:29:58.908012 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a79046dd-60fe-46b4-859e-038be0c1c475 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/abcc0758-4fcf-4259-8cf7-d8712cde8283/action" status: 404 len: 111 microversion: 2.1 time: 0.027397 Sep 27 10:29:58.908012 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3245/6027] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:29:58 2022] POST /compute/v2.1/servers/abcc0758-4fcf-4259-8cf7-d8712cde8283/action => generated 111 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:29:58.915722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2f5dd8f7-ec4a-4632-8e72-1340f0282473 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:58.916254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2f5dd8f7-ec4a-4632-8e72-1340f0282473 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:58.922787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f5dd8f7-ec4a-4632-8e72-1340f0282473 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:58.923543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f5dd8f7-ec4a-4632-8e72-1340f0282473 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:58.924021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f5dd8f7-ec4a-4632-8e72-1340f0282473 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:58.924803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f5dd8f7-ec4a-4632-8e72-1340f0282473 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:58.925262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f5dd8f7-ec4a-4632-8e72-1340f0282473 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:58.925703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f5dd8f7-ec4a-4632-8e72-1340f0282473 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:58.998220 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2f5dd8f7-ec4a-4632-8e72-1340f0282473 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/action" status: 202 len: 0 microversion: 2.1 time: 0.086841 Sep 27 10:29:58.998851 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2783/6028] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:29:58 2022] POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/action => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:29:59.008692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d0e4fce-2bde-437f-b495-b209c823020b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:59.009618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5d0e4fce-2bde-437f-b495-b209c823020b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:59.018590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d0e4fce-2bde-437f-b495-b209c823020b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:59.019332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d0e4fce-2bde-437f-b495-b209c823020b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:59.019775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d0e4fce-2bde-437f-b495-b209c823020b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:59.020564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d0e4fce-2bde-437f-b495-b209c823020b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:59.021360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d0e4fce-2bde-437f-b495-b209c823020b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:59.021849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d0e4fce-2bde-437f-b495-b209c823020b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:59.169625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5d0e4fce-2bde-437f-b495-b209c823020b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:59.178419 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5d0e4fce-2bde-437f-b495-b209c823020b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1520 microversion: 2.1 time: 0.174993 Sep 27 10:29:59.179010 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3246/6029] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:59 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1520 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:29:59.395030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ac4f5cdd-f7f3-4f6a-8511-027e55620bca tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:29:59.395866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ac4f5cdd-f7f3-4f6a-8511-027e55620bca tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:29:59.402673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac4f5cdd-f7f3-4f6a-8511-027e55620bca tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:59.403186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac4f5cdd-f7f3-4f6a-8511-027e55620bca tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:59.403654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac4f5cdd-f7f3-4f6a-8511-027e55620bca tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:59.404402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac4f5cdd-f7f3-4f6a-8511-027e55620bca tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:59.404925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac4f5cdd-f7f3-4f6a-8511-027e55620bca tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:59.405377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac4f5cdd-f7f3-4f6a-8511-027e55620bca tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:59.621802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ac4f5cdd-f7f3-4f6a-8511-027e55620bca tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:29:59.630166 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ac4f5cdd-f7f3-4f6a-8511-027e55620bca tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1692 microversion: 2.1 time: 0.241328 Sep 27 10:29:59.630758 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2784/6030] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:29:59 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1692 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:29:59.793761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-255182ea-bd8f-4e54-92b8-f8702f4dceb3 req-592432be-9c7a-4a41-97f9-7c995e76dfe6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "39429342-b89d-443d-9645-17ec8e2869f3", "tag": "b41f6e87-e1ce-49b6-a47a-5129ca582abc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:29:59.807475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-255182ea-bd8f-4e54-92b8-f8702f4dceb3 req-592432be-9c7a-4a41-97f9-7c995e76dfe6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:29:59.807956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-255182ea-bd8f-4e54-92b8-f8702f4dceb3 req-592432be-9c7a-4a41-97f9-7c995e76dfe6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:29:59.808378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-255182ea-bd8f-4e54-92b8-f8702f4dceb3 req-592432be-9c7a-4a41-97f9-7c995e76dfe6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:29:59.820902 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-255182ea-bd8f-4e54-92b8-f8702f4dceb3 req-592432be-9c7a-4a41-97f9-7c995e76dfe6 service nova] Creating event network-changed:b41f6e87-e1ce-49b6-a47a-5129ca582abc for instance 39429342-b89d-443d-9645-17ec8e2869f3 on nch1-master-1 Sep 27 10:29:59.826451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-255182ea-bd8f-4e54-92b8-f8702f4dceb3 req-592432be-9c7a-4a41-97f9-7c995e76dfe6 service nova] Instance 39429342-b89d-443d-9645-17ec8e2869f3 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:29:59.837452 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-255182ea-bd8f-4e54-92b8-f8702f4dceb3 req-592432be-9c7a-4a41-97f9-7c995e76dfe6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050078 Sep 27 10:29:59.838096 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3247/6031] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:29:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:00.201991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f012d71d-b225-41a8-aca4-51789f3de913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:00.202524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f012d71d-b225-41a8-aca4-51789f3de913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:00.209942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f012d71d-b225-41a8-aca4-51789f3de913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:00.210458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f012d71d-b225-41a8-aca4-51789f3de913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:00.210909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f012d71d-b225-41a8-aca4-51789f3de913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:00.211638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f012d71d-b225-41a8-aca4-51789f3de913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:00.212134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f012d71d-b225-41a8-aca4-51789f3de913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:00.212574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f012d71d-b225-41a8-aca4-51789f3de913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:00.538663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f012d71d-b225-41a8-aca4-51789f3de913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:00.548247 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f012d71d-b225-41a8-aca4-51789f3de913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1520 microversion: 2.1 time: 0.350279 Sep 27 10:30:00.561335 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2785/6032] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:00 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1520 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:00.657055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f45f3812-123a-4c09-b83e-35c74e459482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:00.657858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f45f3812-123a-4c09-b83e-35c74e459482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:00.664620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f45f3812-123a-4c09-b83e-35c74e459482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:00.665148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f45f3812-123a-4c09-b83e-35c74e459482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:00.665585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f45f3812-123a-4c09-b83e-35c74e459482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:00.666324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f45f3812-123a-4c09-b83e-35c74e459482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:00.666783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f45f3812-123a-4c09-b83e-35c74e459482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:00.667211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f45f3812-123a-4c09-b83e-35c74e459482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:00.804413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f45f3812-123a-4c09-b83e-35c74e459482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:00.813281 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f45f3812-123a-4c09-b83e-35c74e459482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1689 microversion: 2.1 time: 0.160988 Sep 27 10:30:00.813868 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3248/6033] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:00 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1689 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:00.837192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d18a7566-823f-435d-9d15-3ff82616db96 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:00.837753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d18a7566-823f-435d-9d15-3ff82616db96 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:00.844202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d18a7566-823f-435d-9d15-3ff82616db96 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:00.844701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d18a7566-823f-435d-9d15-3ff82616db96 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:00.845167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d18a7566-823f-435d-9d15-3ff82616db96 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:00.845902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d18a7566-823f-435d-9d15-3ff82616db96 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:00.846373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d18a7566-823f-435d-9d15-3ff82616db96 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:00.846813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d18a7566-823f-435d-9d15-3ff82616db96 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:00.866193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:00.920746 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d18a7566-823f-435d-9d15-3ff82616db96 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3/action" status: 204 len: 0 microversion: 2.1 time: 0.087649 Sep 27 10:30:00.921424 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2786/6034] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:30:00 2022] POST /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3/action => generated 0 bytes in 89 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:30:00.930150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6532db88-aa8a-4368-a8ef-139b9f0747ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:00.930821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6532db88-aa8a-4368-a8ef-139b9f0747ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:00.938149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6532db88-aa8a-4368-a8ef-139b9f0747ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:00.938709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6532db88-aa8a-4368-a8ef-139b9f0747ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:00.939194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6532db88-aa8a-4368-a8ef-139b9f0747ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:00.939944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6532db88-aa8a-4368-a8ef-139b9f0747ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:00.940390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6532db88-aa8a-4368-a8ef-139b9f0747ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:00.941014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6532db88-aa8a-4368-a8ef-139b9f0747ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:01.085450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6532db88-aa8a-4368-a8ef-139b9f0747ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:01.093797 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6532db88-aa8a-4368-a8ef-139b9f0747ef tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1689 microversion: 2.1 time: 0.167920 Sep 27 10:30:01.094386 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3249/6035] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:00 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1689 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:01.574349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-111a2538-06b9-4538-a582-c30d05b999bf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:01.575150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-111a2538-06b9-4538-a582-c30d05b999bf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:01.583279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-111a2538-06b9-4538-a582-c30d05b999bf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:01.583863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-111a2538-06b9-4538-a582-c30d05b999bf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:01.584321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-111a2538-06b9-4538-a582-c30d05b999bf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:01.585732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-111a2538-06b9-4538-a582-c30d05b999bf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:01.586204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-111a2538-06b9-4538-a582-c30d05b999bf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:01.586644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-111a2538-06b9-4538-a582-c30d05b999bf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:01.828108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-111a2538-06b9-4538-a582-c30d05b999bf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:01.836960 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-111a2538-06b9-4538-a582-c30d05b999bf tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1500 microversion: 2.1 time: 0.266855 Sep 27 10:30:01.837577 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2787/6036] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:01 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1500 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:01.865034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'action', calling method: >, body: {"rescue": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:01.867657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:01.874883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:01.875378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:01.875823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:01.876545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:01.877045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:01.877479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:01.881538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:01.907186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lazy-loading 'flavor' on Instance uuid 15833e30-ec8c-4756-b9df-5ca3061b4ed9 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:30:01.920568 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] HTTP exception thrown: Cannot 'rescue' instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 while it is in vm_state paused Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Request method failure captured: Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/action HTTP/1.1 Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 14 Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtB5lZw26WfhuIIxDLfEGtJx8GdKac9mUecPJWt47RWTVgFEQNt2CRnQRU62Kxhrd_5bXczg0Q9QWsT4c85-tEEw3L_ZoXg7ttqo2fTxJvLnAMIld1Q9kkM9ItLXE6S_aDTtz0h96WsfSIhdls5sa9AFKLBsHqf09n27PObWbpVe7TQ Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_40247edd5a444fd8b1c937c13d39393a"}]}] Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6771df45b4ee4426b2f76a09eab2998a Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: {"rescue": {}} Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: exception: Cannot 'rescue' instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 while it is in vm_state paused Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '15833e30-ec8c-4756-b9df-5ca3061b4ed9', 'body': {'rescue': {}}} Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/rescue.py", line 62, in _rescue Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.rescue(context, instance, Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 278, in inner Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 in vm_state paused. Cannot rescue while the instance is in this state. Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/rescue.py", line 72, in _rescue Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state( Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'rescue' instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 while it is in vm_state paused Sep 27 10:30:01.922350 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'rescue' instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 while it is in vm_state paused Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Returning 409 to user: Cannot 'rescue' instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 while it is in vm_state paused {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e0779dec-2f4a-4be3-88b2-b6f51a507669 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/action" status: 409 len: 144 microversion: 2.1 time: 0.063405 Sep 27 10:30:01.927285 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3250/6037] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:30:01 2022] POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/action => generated 144 bytes in 65 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:30:01.934218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a6c21fcd-3417-4212-85fc-e8df9e262df9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:01.934909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a6c21fcd-3417-4212-85fc-e8df9e262df9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:01.941523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6c21fcd-3417-4212-85fc-e8df9e262df9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:01.942020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6c21fcd-3417-4212-85fc-e8df9e262df9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:01.942514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6c21fcd-3417-4212-85fc-e8df9e262df9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:01.943249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6c21fcd-3417-4212-85fc-e8df9e262df9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:01.943730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6c21fcd-3417-4212-85fc-e8df9e262df9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:01.944184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a6c21fcd-3417-4212-85fc-e8df9e262df9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:02.017340 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a6c21fcd-3417-4212-85fc-e8df9e262df9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/action" status: 202 len: 0 microversion: 2.1 time: 0.086994 Sep 27 10:30:02.017988 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2788/6038] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:30:01 2022] POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/action => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:30:02.026403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1393fbf3-4d32-40f4-afb0-147c83718dc1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:02.027030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1393fbf3-4d32-40f4-afb0-147c83718dc1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:02.033393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1393fbf3-4d32-40f4-afb0-147c83718dc1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:02.034192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1393fbf3-4d32-40f4-afb0-147c83718dc1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:02.034683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1393fbf3-4d32-40f4-afb0-147c83718dc1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:02.035360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1393fbf3-4d32-40f4-afb0-147c83718dc1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:02.036015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1393fbf3-4d32-40f4-afb0-147c83718dc1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:02.036489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1393fbf3-4d32-40f4-afb0-147c83718dc1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:02.118774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-09b5db28-ce45-42ec-acae-2d28166fb034 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:02.119613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-09b5db28-ce45-42ec-acae-2d28166fb034 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:02.126667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09b5db28-ce45-42ec-acae-2d28166fb034 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:02.127221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09b5db28-ce45-42ec-acae-2d28166fb034 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:02.127650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09b5db28-ce45-42ec-acae-2d28166fb034 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:02.128384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09b5db28-ce45-42ec-acae-2d28166fb034 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:02.129049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09b5db28-ce45-42ec-acae-2d28166fb034 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:02.129320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09b5db28-ce45-42ec-acae-2d28166fb034 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:02.220647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1393fbf3-4d32-40f4-afb0-147c83718dc1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:02.233976 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1393fbf3-4d32-40f4-afb0-147c83718dc1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1507 microversion: 2.1 time: 0.211218 Sep 27 10:30:02.235651 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3251/6039] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:02 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1507 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:02.335432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-09b5db28-ce45-42ec-acae-2d28166fb034 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:02.344884 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-09b5db28-ce45-42ec-acae-2d28166fb034 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1689 microversion: 2.1 time: 0.231869 Sep 27 10:30:02.345492 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2789/6040] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:02 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1689 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:30:03.262475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2843c193-07ba-4d1d-9755-f7555fb36bfd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:03.263229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2843c193-07ba-4d1d-9755-f7555fb36bfd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:03.271332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2843c193-07ba-4d1d-9755-f7555fb36bfd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:03.271827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2843c193-07ba-4d1d-9755-f7555fb36bfd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:03.272264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2843c193-07ba-4d1d-9755-f7555fb36bfd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:03.273015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2843c193-07ba-4d1d-9755-f7555fb36bfd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:03.273474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2843c193-07ba-4d1d-9755-f7555fb36bfd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:03.273901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2843c193-07ba-4d1d-9755-f7555fb36bfd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:03.370884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c531ddf6-8977-458c-aad6-7c8bf1177ce7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:03.371519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c531ddf6-8977-458c-aad6-7c8bf1177ce7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:03.379100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c531ddf6-8977-458c-aad6-7c8bf1177ce7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:03.379666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c531ddf6-8977-458c-aad6-7c8bf1177ce7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:03.380122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c531ddf6-8977-458c-aad6-7c8bf1177ce7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:03.380909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c531ddf6-8977-458c-aad6-7c8bf1177ce7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:03.381388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c531ddf6-8977-458c-aad6-7c8bf1177ce7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:03.381833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c531ddf6-8977-458c-aad6-7c8bf1177ce7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:03.430157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2843c193-07ba-4d1d-9755-f7555fb36bfd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:03.439044 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2843c193-07ba-4d1d-9755-f7555fb36bfd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1515 microversion: 2.1 time: 0.180921 Sep 27 10:30:03.439620 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3252/6041] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:03 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1515 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:03.526381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c531ddf6-8977-458c-aad6-7c8bf1177ce7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:03.536825 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c531ddf6-8977-458c-aad6-7c8bf1177ce7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1681 microversion: 2.1 time: 0.169983 Sep 27 10:30:03.537786 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2790/6042] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:03 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1681 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:04.612375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-844e28e1-d6b9-4b91-99ff-da173b136b61 req-e4ec081c-90e1-4cda-ae3f-3753a7ae1249 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39429342-b89d-443d-9645-17ec8e2869f3", "name": "network-vif-plugged", "status": "completed", "tag": "b41f6e87-e1ce-49b6-a47a-5129ca582abc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:04.624407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-844e28e1-d6b9-4b91-99ff-da173b136b61 req-e4ec081c-90e1-4cda-ae3f-3753a7ae1249 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:04.625008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-844e28e1-d6b9-4b91-99ff-da173b136b61 req-e4ec081c-90e1-4cda-ae3f-3753a7ae1249 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:04.625457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-844e28e1-d6b9-4b91-99ff-da173b136b61 req-e4ec081c-90e1-4cda-ae3f-3753a7ae1249 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:04.637728 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-844e28e1-d6b9-4b91-99ff-da173b136b61 req-e4ec081c-90e1-4cda-ae3f-3753a7ae1249 service nova] Creating event network-vif-plugged:b41f6e87-e1ce-49b6-a47a-5129ca582abc for instance 39429342-b89d-443d-9645-17ec8e2869f3 on nch1-master-1 Sep 27 10:30:04.642926 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-844e28e1-d6b9-4b91-99ff-da173b136b61 req-e4ec081c-90e1-4cda-ae3f-3753a7ae1249 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035328 Sep 27 10:30:04.643550 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3253/6043] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:04.860565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-963c76d9-16f2-4b6e-82d5-d7461ab234a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:04.862997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-963c76d9-16f2-4b6e-82d5-d7461ab234a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:04.870206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-963c76d9-16f2-4b6e-82d5-d7461ab234a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:04.870720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-963c76d9-16f2-4b6e-82d5-d7461ab234a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:04.871184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-963c76d9-16f2-4b6e-82d5-d7461ab234a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:04.871911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-963c76d9-16f2-4b6e-82d5-d7461ab234a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:04.872393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-963c76d9-16f2-4b6e-82d5-d7461ab234a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:04.872951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-963c76d9-16f2-4b6e-82d5-d7461ab234a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:04.897158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-963c76d9-16f2-4b6e-82d5-d7461ab234a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lazy-loading 'flavor' on Instance uuid 15833e30-ec8c-4756-b9df-5ca3061b4ed9 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:30:04.976814 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-963c76d9-16f2-4b6e-82d5-d7461ab234a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/action" status: 200 len: 32 microversion: 2.1 time: 0.120323 Sep 27 10:30:04.977438 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2791/6044] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:30:04 2022] POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/action => generated 32 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:30:04.987697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-02512eb2-9ec1-4a1e-a3f7-202c424aec81 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:04.988411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-02512eb2-9ec1-4a1e-a3f7-202c424aec81 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:04.996408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02512eb2-9ec1-4a1e-a3f7-202c424aec81 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:04.996408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02512eb2-9ec1-4a1e-a3f7-202c424aec81 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:04.996918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02512eb2-9ec1-4a1e-a3f7-202c424aec81 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:04.997806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02512eb2-9ec1-4a1e-a3f7-202c424aec81 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:04.998296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02512eb2-9ec1-4a1e-a3f7-202c424aec81 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:04.998718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-02512eb2-9ec1-4a1e-a3f7-202c424aec81 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:05.150005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-02512eb2-9ec1-4a1e-a3f7-202c424aec81 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:05.160432 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-02512eb2-9ec1-4a1e-a3f7-202c424aec81 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.177014 Sep 27 10:30:05.163076 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3254/6045] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:04 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:06.186170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-daa16c36-c01d-4341-b23e-23f4bd268513 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:06.186997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-daa16c36-c01d-4341-b23e-23f4bd268513 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:06.189247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=68 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:06.196004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daa16c36-c01d-4341-b23e-23f4bd268513 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:06.197107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daa16c36-c01d-4341-b23e-23f4bd268513 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:06.197729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daa16c36-c01d-4341-b23e-23f4bd268513 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:06.198557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daa16c36-c01d-4341-b23e-23f4bd268513 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:06.199243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daa16c36-c01d-4341-b23e-23f4bd268513 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:06.199782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daa16c36-c01d-4341-b23e-23f4bd268513 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:06.357065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-daa16c36-c01d-4341-b23e-23f4bd268513 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:06.369752 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-daa16c36-c01d-4341-b23e-23f4bd268513 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.189028 Sep 27 10:30:06.370813 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2792/6046] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:06 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:07.395852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2ae5a469-111c-4e42-9aa5-3ae9dae160e8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:07.396491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2ae5a469-111c-4e42-9aa5-3ae9dae160e8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:07.407408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5a469-111c-4e42-9aa5-3ae9dae160e8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:07.408042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5a469-111c-4e42-9aa5-3ae9dae160e8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:07.408589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5a469-111c-4e42-9aa5-3ae9dae160e8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:07.409490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5a469-111c-4e42-9aa5-3ae9dae160e8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:07.410060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5a469-111c-4e42-9aa5-3ae9dae160e8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:07.410558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5a469-111c-4e42-9aa5-3ae9dae160e8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:07.559368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2ae5a469-111c-4e42-9aa5-3ae9dae160e8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:07.574392 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2ae5a469-111c-4e42-9aa5-3ae9dae160e8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.183868 Sep 27 10:30:07.575154 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3255/6047] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:07 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:08.602607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3e36cd29-96a9-4781-974a-d65209ea82b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:08.603433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e36cd29-96a9-4781-974a-d65209ea82b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:08.612436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e36cd29-96a9-4781-974a-d65209ea82b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:08.613110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e36cd29-96a9-4781-974a-d65209ea82b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:08.613660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e36cd29-96a9-4781-974a-d65209ea82b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:08.614393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e36cd29-96a9-4781-974a-d65209ea82b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:08.614866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e36cd29-96a9-4781-974a-d65209ea82b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:08.615407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e36cd29-96a9-4781-974a-d65209ea82b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:08.752504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3e36cd29-96a9-4781-974a-d65209ea82b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:08.761546 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3e36cd29-96a9-4781-974a-d65209ea82b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.162897 Sep 27 10:30:08.762142 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2793/6048] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:08 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:09.785942 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8732572d-49db-4084-9488-2f68c2cf9550 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:09.786618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8732572d-49db-4084-9488-2f68c2cf9550 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:09.795312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8732572d-49db-4084-9488-2f68c2cf9550 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:09.795950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8732572d-49db-4084-9488-2f68c2cf9550 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:09.796481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8732572d-49db-4084-9488-2f68c2cf9550 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:09.797374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8732572d-49db-4084-9488-2f68c2cf9550 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:09.797958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8732572d-49db-4084-9488-2f68c2cf9550 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:09.798468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8732572d-49db-4084-9488-2f68c2cf9550 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:09.949579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8732572d-49db-4084-9488-2f68c2cf9550 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:09.958255 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8732572d-49db-4084-9488-2f68c2cf9550 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.176712 Sep 27 10:30:09.958904 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3256/6049] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:09 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:10.984149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1567fbdf-76e7-4672-ada6-ed4088e0803d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:10.984909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1567fbdf-76e7-4672-ada6-ed4088e0803d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:10.992825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1567fbdf-76e7-4672-ada6-ed4088e0803d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:10.993349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1567fbdf-76e7-4672-ada6-ed4088e0803d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:10.993805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1567fbdf-76e7-4672-ada6-ed4088e0803d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:10.994524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1567fbdf-76e7-4672-ada6-ed4088e0803d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:10.994997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1567fbdf-76e7-4672-ada6-ed4088e0803d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:10.995535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1567fbdf-76e7-4672-ada6-ed4088e0803d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:11.136907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1567fbdf-76e7-4672-ada6-ed4088e0803d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:11.149463 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1567fbdf-76e7-4672-ada6-ed4088e0803d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.169528 Sep 27 10:30:11.150079 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2794/6050] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:10 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:12.176813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-46499470-c9a2-4b21-ae0e-9335ee8979e4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:12.177474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-46499470-c9a2-4b21-ae0e-9335ee8979e4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:12.185801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46499470-c9a2-4b21-ae0e-9335ee8979e4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:12.186333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46499470-c9a2-4b21-ae0e-9335ee8979e4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:12.186879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46499470-c9a2-4b21-ae0e-9335ee8979e4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:12.187819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46499470-c9a2-4b21-ae0e-9335ee8979e4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:12.188329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46499470-c9a2-4b21-ae0e-9335ee8979e4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:12.188979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-46499470-c9a2-4b21-ae0e-9335ee8979e4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:12.332261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-46499470-c9a2-4b21-ae0e-9335ee8979e4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:12.341264 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-46499470-c9a2-4b21-ae0e-9335ee8979e4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.168552 Sep 27 10:30:12.341844 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3257/6051] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:12 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:13.366814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c486310b-f3a8-4694-b501-8f7ba2bb55af tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:13.367428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c486310b-f3a8-4694-b501-8f7ba2bb55af tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:13.377956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486310b-f3a8-4694-b501-8f7ba2bb55af tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:13.378613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486310b-f3a8-4694-b501-8f7ba2bb55af tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:13.379199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486310b-f3a8-4694-b501-8f7ba2bb55af tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:13.380099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486310b-f3a8-4694-b501-8f7ba2bb55af tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:13.380644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486310b-f3a8-4694-b501-8f7ba2bb55af tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:13.381199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c486310b-f3a8-4694-b501-8f7ba2bb55af tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:13.525343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c486310b-f3a8-4694-b501-8f7ba2bb55af tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:13.534315 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c486310b-f3a8-4694-b501-8f7ba2bb55af tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.173342 Sep 27 10:30:13.534904 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2795/6052] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:13 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:14.562193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5afa1517-16d2-47fe-9da1-ca33cfedf9eb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:14.563211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5afa1517-16d2-47fe-9da1-ca33cfedf9eb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:14.573608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5afa1517-16d2-47fe-9da1-ca33cfedf9eb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:14.574107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5afa1517-16d2-47fe-9da1-ca33cfedf9eb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:14.574558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5afa1517-16d2-47fe-9da1-ca33cfedf9eb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:14.575275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5afa1517-16d2-47fe-9da1-ca33cfedf9eb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:14.575743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5afa1517-16d2-47fe-9da1-ca33cfedf9eb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:14.576178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5afa1517-16d2-47fe-9da1-ca33cfedf9eb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:14.724378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5afa1517-16d2-47fe-9da1-ca33cfedf9eb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:14.732969 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5afa1517-16d2-47fe-9da1-ca33cfedf9eb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.177414 Sep 27 10:30:14.733553 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3258/6053] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:14 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:15.758236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e8a5a36-cdb3-4330-bbe9-6a22bebe15fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:15.758867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6e8a5a36-cdb3-4330-bbe9-6a22bebe15fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:15.766868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e8a5a36-cdb3-4330-bbe9-6a22bebe15fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:15.767377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e8a5a36-cdb3-4330-bbe9-6a22bebe15fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:15.767818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e8a5a36-cdb3-4330-bbe9-6a22bebe15fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:15.768547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e8a5a36-cdb3-4330-bbe9-6a22bebe15fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:15.769050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e8a5a36-cdb3-4330-bbe9-6a22bebe15fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:15.769498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e8a5a36-cdb3-4330-bbe9-6a22bebe15fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:15.914790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6e8a5a36-cdb3-4330-bbe9-6a22bebe15fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:15.922764 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6e8a5a36-cdb3-4330-bbe9-6a22bebe15fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.169194 Sep 27 10:30:15.923337 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2796/6054] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:15 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:16.946118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7bbf1f24-f1a0-423d-b791-159ba5a6e946 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:16.946742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7bbf1f24-f1a0-423d-b791-159ba5a6e946 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:16.954818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bbf1f24-f1a0-423d-b791-159ba5a6e946 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:16.955313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bbf1f24-f1a0-423d-b791-159ba5a6e946 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:16.955756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bbf1f24-f1a0-423d-b791-159ba5a6e946 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:16.956469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bbf1f24-f1a0-423d-b791-159ba5a6e946 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:16.956987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bbf1f24-f1a0-423d-b791-159ba5a6e946 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:16.957405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bbf1f24-f1a0-423d-b791-159ba5a6e946 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:17.098739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7bbf1f24-f1a0-423d-b791-159ba5a6e946 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:17.107227 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7bbf1f24-f1a0-423d-b791-159ba5a6e946 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.165108 Sep 27 10:30:17.107801 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3259/6055] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:16 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:18.135243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8be71a15-f61b-419b-8e67-acac24b2556b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:18.136345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8be71a15-f61b-419b-8e67-acac24b2556b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:18.148282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8be71a15-f61b-419b-8e67-acac24b2556b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:18.149203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8be71a15-f61b-419b-8e67-acac24b2556b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:18.149976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8be71a15-f61b-419b-8e67-acac24b2556b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:18.151232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8be71a15-f61b-419b-8e67-acac24b2556b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:18.152035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8be71a15-f61b-419b-8e67-acac24b2556b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:18.152482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8be71a15-f61b-419b-8e67-acac24b2556b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:18.302231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8be71a15-f61b-419b-8e67-acac24b2556b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:18.310617 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8be71a15-f61b-419b-8e67-acac24b2556b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.182164 Sep 27 10:30:18.311197 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2797/6056] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:18 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:19.334245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c2f28ec-1b46-4861-ae7e-e921182618ed tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:19.334874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5c2f28ec-1b46-4861-ae7e-e921182618ed tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:19.337748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=50 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:19.344179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c2f28ec-1b46-4861-ae7e-e921182618ed tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:19.344731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c2f28ec-1b46-4861-ae7e-e921182618ed tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:19.345168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c2f28ec-1b46-4861-ae7e-e921182618ed tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:19.345886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c2f28ec-1b46-4861-ae7e-e921182618ed tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:19.346328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c2f28ec-1b46-4861-ae7e-e921182618ed tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:19.346782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c2f28ec-1b46-4861-ae7e-e921182618ed tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:19.493159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c2f28ec-1b46-4861-ae7e-e921182618ed tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:19.503025 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c2f28ec-1b46-4861-ae7e-e921182618ed tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.172836 Sep 27 10:30:19.503587 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3260/6057] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:19 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:20.527915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-de2c22ad-39a9-47c0-aad9-899b586efc03 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:20.528801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-de2c22ad-39a9-47c0-aad9-899b586efc03 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:20.538715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de2c22ad-39a9-47c0-aad9-899b586efc03 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:20.539379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de2c22ad-39a9-47c0-aad9-899b586efc03 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:20.539982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de2c22ad-39a9-47c0-aad9-899b586efc03 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:20.540985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de2c22ad-39a9-47c0-aad9-899b586efc03 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:20.541617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de2c22ad-39a9-47c0-aad9-899b586efc03 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:20.542209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de2c22ad-39a9-47c0-aad9-899b586efc03 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:20.683785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-de2c22ad-39a9-47c0-aad9-899b586efc03 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:20.695531 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-de2c22ad-39a9-47c0-aad9-899b586efc03 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.172746 Sep 27 10:30:20.696117 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2798/6058] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:20 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:21.720022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-affb7392-28a0-4f90-98e0-c4be2d097e55 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:21.720768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-affb7392-28a0-4f90-98e0-c4be2d097e55 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:21.728092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-affb7392-28a0-4f90-98e0-c4be2d097e55 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:21.728608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-affb7392-28a0-4f90-98e0-c4be2d097e55 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:21.729147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-affb7392-28a0-4f90-98e0-c4be2d097e55 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:21.729855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-affb7392-28a0-4f90-98e0-c4be2d097e55 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:21.730325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-affb7392-28a0-4f90-98e0-c4be2d097e55 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:21.730762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-affb7392-28a0-4f90-98e0-c4be2d097e55 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:21.870656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-affb7392-28a0-4f90-98e0-c4be2d097e55 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:21.882960 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-affb7392-28a0-4f90-98e0-c4be2d097e55 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.167570 Sep 27 10:30:21.883722 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3261/6059] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:21 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:22.907827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0adf4b87-a961-4863-9198-adb738381655 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:22.908426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0adf4b87-a961-4863-9198-adb738381655 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:22.915141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0adf4b87-a961-4863-9198-adb738381655 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:22.915672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0adf4b87-a961-4863-9198-adb738381655 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:22.916122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0adf4b87-a961-4863-9198-adb738381655 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:22.916884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0adf4b87-a961-4863-9198-adb738381655 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:22.917335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0adf4b87-a961-4863-9198-adb738381655 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:22.917803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0adf4b87-a961-4863-9198-adb738381655 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:22.920383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:23.061165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0adf4b87-a961-4863-9198-adb738381655 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:23.069368 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0adf4b87-a961-4863-9198-adb738381655 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.166044 Sep 27 10:30:23.069953 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2799/6060] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:22 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:23.174093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5805c76c-68c2-4855-89b7-566811359764 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:23.174652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5805c76c-68c2-4855-89b7-566811359764 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:23.182117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5805c76c-68c2-4855-89b7-566811359764 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:23.182613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5805c76c-68c2-4855-89b7-566811359764 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:23.183054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5805c76c-68c2-4855-89b7-566811359764 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:23.183802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5805c76c-68c2-4855-89b7-566811359764 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:23.184258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5805c76c-68c2-4855-89b7-566811359764 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:23.184708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5805c76c-68c2-4855-89b7-566811359764 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:23.209708 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5805c76c-68c2-4855-89b7-566811359764 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14/ips" status: 200 len: 149 microversion: 2.1 time: 0.039740 Sep 27 10:30:23.210290 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3262/6061] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:30:23 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14/ips => generated 149 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:23.222855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "64e8066b-82ca-4278-ac3b-4ce6d3460710"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:23.225326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:23.231840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:23.232351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:23.232849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:23.233613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:23.234089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:23.234532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:23.318967 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Invalid volume: volume 64e8066b-82ca-4278-ac3b-4ce6d3460710 is already attached to instances: 403bfd67-4e7f-4918-baf9-4d14086a4598 Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14/os-volume_attachments HTTP/1.1 Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 74 Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: {"volumeAttachment": {"volumeId": "64e8066b-82ca-4278-ac3b-4ce6d3460710"}} Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: exception: Invalid volume: volume 64e8066b-82ca-4278-ac3b-4ce6d3460710 is already attached to instances: 403bfd67-4e7f-4918-baf9-4d14086a4598 Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'server_id': 'd5319326-fc6f-4d93-85ae-98a8b422dc14', 'body': {'volumeAttachment': {'volumeId': '64e8066b-82ca-4278-ac3b-4ce6d3460710'}}} Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 64e8066b-82ca-4278-ac3b-4ce6d3460710 is already attached to instances: 403bfd67-4e7f-4918-baf9-4d14086a4598 Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:30:23.320805 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 64e8066b-82ca-4278-ac3b-4ce6d3460710 is already attached to instances: 403bfd67-4e7f-4918-baf9-4d14086a4598 Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 64e8066b-82ca-4278-ac3b-4ce6d3460710 is already attached to instances: 403bfd67-4e7f-4918-baf9-4d14086a4598 Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 400 to user: Invalid volume: volume 64e8066b-82ca-4278-ac3b-4ce6d3460710 is already attached to instances: 403bfd67-4e7f-4918-baf9-4d14086a4598 {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-21c3b20c-df30-45fb-af67-6751e5917dcb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.103640 Sep 27 10:30:23.325834 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2800/6062] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:30:23 2022] POST /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14/os-volume_attachments => generated 174 bytes in 105 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:30:23.331444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f359964-ca9f-425d-88a7-681c169a1d9f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:23.331979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7f359964-ca9f-425d-88a7-681c169a1d9f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:23.338377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f359964-ca9f-425d-88a7-681c169a1d9f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:23.338892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f359964-ca9f-425d-88a7-681c169a1d9f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:23.339430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f359964-ca9f-425d-88a7-681c169a1d9f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:23.340140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f359964-ca9f-425d-88a7-681c169a1d9f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:23.340594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f359964-ca9f-425d-88a7-681c169a1d9f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:23.341400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7f359964-ca9f-425d-88a7-681c169a1d9f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:23.367467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7f359964-ca9f-425d-88a7-681c169a1d9f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:30:23.452542 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7f359964-ca9f-425d-88a7-681c169a1d9f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 204 len: 0 microversion: 2.1 time: 0.124700 Sep 27 10:30:23.453292 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3263/6063] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:30:23 2022] DELETE /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:30:23.462227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5dd3d07-1e8f-40bc-bd13-e487b144f000 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:23.462870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5dd3d07-1e8f-40bc-bd13-e487b144f000 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:23.470045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5dd3d07-1e8f-40bc-bd13-e487b144f000 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:23.470558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5dd3d07-1e8f-40bc-bd13-e487b144f000 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:23.471059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5dd3d07-1e8f-40bc-bd13-e487b144f000 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:23.471787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5dd3d07-1e8f-40bc-bd13-e487b144f000 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:23.472264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5dd3d07-1e8f-40bc-bd13-e487b144f000 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:23.472738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5dd3d07-1e8f-40bc-bd13-e487b144f000 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:23.608104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5dd3d07-1e8f-40bc-bd13-e487b144f000 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:23.617095 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5dd3d07-1e8f-40bc-bd13-e487b144f000 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1679 microversion: 2.1 time: 0.159065 Sep 27 10:30:23.617665 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2801/6064] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:23 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1679 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:24.098697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bd33c6f2-ab2c-4cb6-8c82-5dc03370c6de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:24.099434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bd33c6f2-ab2c-4cb6-8c82-5dc03370c6de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:24.106387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd33c6f2-ab2c-4cb6-8c82-5dc03370c6de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:24.106902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd33c6f2-ab2c-4cb6-8c82-5dc03370c6de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:24.107363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd33c6f2-ab2c-4cb6-8c82-5dc03370c6de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:24.108110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd33c6f2-ab2c-4cb6-8c82-5dc03370c6de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:24.108595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd33c6f2-ab2c-4cb6-8c82-5dc03370c6de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:24.109077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd33c6f2-ab2c-4cb6-8c82-5dc03370c6de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:24.247402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bd33c6f2-ab2c-4cb6-8c82-5dc03370c6de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:24.257640 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bd33c6f2-ab2c-4cb6-8c82-5dc03370c6de tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1501 microversion: 2.1 time: 0.163993 Sep 27 10:30:24.258212 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3264/6065] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:24 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1501 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:24.280435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c4b02a63-754a-4f44-ab9f-69541e872afc"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:24.283063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:24.290494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:24.290994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:24.291453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:24.292166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:24.292678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:24.293116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:24.317538 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] HTTP exception thrown: Cannot 'attach_volume' instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 while it is in vm_state rescued Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Request method failure captured: Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/os-volume_attachments HTTP/1.1 Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 74 Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtB5lZw26WfhuIIxDLfEGtJx8GdKac9mUecPJWt47RWTVgFEQNt2CRnQRU62Kxhrd_5bXczg0Q9QWsT4c85-tEEw3L_ZoXg7ttqo2fTxJvLnAMIld1Q9kkM9ItLXE6S_aDTtz0h96WsfSIhdls5sa9AFKLBsHqf09n27PObWbpVe7TQ Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_40247edd5a444fd8b1c937c13d39393a"}]}] Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 6771df45b4ee4426b2f76a09eab2998a Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: {"volumeAttachment": {"volumeId": "c4b02a63-754a-4f44-ab9f-69541e872afc"}} Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: exception: Cannot 'attach_volume' instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 while it is in vm_state rescued Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'server_id': '15833e30-ec8c-4756-b9df-5ca3061b4ed9', 'body': {'volumeAttachment': {'volumeId': 'c4b02a63-754a-4f44-ab9f-69541e872afc'}}} Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 in vm_state rescued. Cannot attach_volume while the instance is in this state. Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Sep 27 10:30:24.319165 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 while it is in vm_state rescued Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 while it is in vm_state rescued Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Returning 409 to user: Cannot 'attach_volume' instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 while it is in vm_state rescued {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d3e04dbd-63e6-462a-b003-fc2a9312bd45 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.044367 Sep 27 10:30:24.323792 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2802/6066] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:30:24 2022] POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/os-volume_attachments => generated 152 bytes in 45 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:30:24.329804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7df23d4e-a10b-48ef-bab0-00e1d8f808d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:24.330347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7df23d4e-a10b-48ef-bab0-00e1d8f808d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:24.336776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df23d4e-a10b-48ef-bab0-00e1d8f808d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:24.337259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df23d4e-a10b-48ef-bab0-00e1d8f808d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:24.337699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df23d4e-a10b-48ef-bab0-00e1d8f808d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:24.338468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df23d4e-a10b-48ef-bab0-00e1d8f808d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:24.338933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df23d4e-a10b-48ef-bab0-00e1d8f808d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:24.339361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7df23d4e-a10b-48ef-bab0-00e1d8f808d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:24.382734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:24.412370 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7df23d4e-a10b-48ef-bab0-00e1d8f808d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/action" status: 202 len: 0 microversion: 2.1 time: 0.086383 Sep 27 10:30:24.413068 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3265/6067] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:30:24 2022] POST /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9/action => generated 0 bytes in 88 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:30:24.421556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dc766425-6dbd-42a1-b8be-078f708fffb5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:24.422203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dc766425-6dbd-42a1-b8be-078f708fffb5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:24.428444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc766425-6dbd-42a1-b8be-078f708fffb5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:24.429000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc766425-6dbd-42a1-b8be-078f708fffb5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:24.429492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc766425-6dbd-42a1-b8be-078f708fffb5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:24.430259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc766425-6dbd-42a1-b8be-078f708fffb5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:24.430757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc766425-6dbd-42a1-b8be-078f708fffb5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:24.431258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc766425-6dbd-42a1-b8be-078f708fffb5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:24.642690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29ae9a96-1cdc-4cba-93e7-54e3c949454b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:24.643343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-29ae9a96-1cdc-4cba-93e7-54e3c949454b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:24.650165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ae9a96-1cdc-4cba-93e7-54e3c949454b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:24.650334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dc766425-6dbd-42a1-b8be-078f708fffb5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:24.650657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ae9a96-1cdc-4cba-93e7-54e3c949454b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:24.651093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ae9a96-1cdc-4cba-93e7-54e3c949454b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:24.651824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ae9a96-1cdc-4cba-93e7-54e3c949454b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:24.652308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ae9a96-1cdc-4cba-93e7-54e3c949454b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:24.652738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29ae9a96-1cdc-4cba-93e7-54e3c949454b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:24.658977 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dc766425-6dbd-42a1-b8be-078f708fffb5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1509 microversion: 2.1 time: 0.241434 Sep 27 10:30:24.659775 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2803/6068] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:24 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1509 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:24.792270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29ae9a96-1cdc-4cba-93e7-54e3c949454b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:24.800532 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-29ae9a96-1cdc-4cba-93e7-54e3c949454b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1679 microversion: 2.1 time: 0.162965 Sep 27 10:30:24.801207 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3266/6069] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:24 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1679 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:25.683309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3cad2349-97ca-4be6-8b10-d12378ebe0b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:25.684055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3cad2349-97ca-4be6-8b10-d12378ebe0b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:25.691415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cad2349-97ca-4be6-8b10-d12378ebe0b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:25.691940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cad2349-97ca-4be6-8b10-d12378ebe0b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:25.692396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cad2349-97ca-4be6-8b10-d12378ebe0b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:25.693155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cad2349-97ca-4be6-8b10-d12378ebe0b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:25.693636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cad2349-97ca-4be6-8b10-d12378ebe0b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:25.694088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3cad2349-97ca-4be6-8b10-d12378ebe0b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:25.823807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4f97fb4-567f-4715-bca9-857d281366aa tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:25.824461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4f97fb4-567f-4715-bca9-857d281366aa tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:25.832011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4f97fb4-567f-4715-bca9-857d281366aa tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:25.832480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4f97fb4-567f-4715-bca9-857d281366aa tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:25.832968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4f97fb4-567f-4715-bca9-857d281366aa tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:25.834635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4f97fb4-567f-4715-bca9-857d281366aa tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:25.836441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4f97fb4-567f-4715-bca9-857d281366aa tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:25.837464 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4f97fb4-567f-4715-bca9-857d281366aa tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:25.914710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3cad2349-97ca-4be6-8b10-d12378ebe0b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:25.923775 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3cad2349-97ca-4be6-8b10-d12378ebe0b2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1509 microversion: 2.1 time: 0.244898 Sep 27 10:30:25.924689 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2804/6070] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:25 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1509 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:25.981744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4f97fb4-567f-4715-bca9-857d281366aa tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:25.990057 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4f97fb4-567f-4715-bca9-857d281366aa tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1679 microversion: 2.1 time: 0.170220 Sep 27 10:30:25.990721 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3267/6071] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:25 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1679 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:26.486986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-89bc7c76-51d9-4b1d-b1ea-5be66c2678dd req-ae0ea952-7f96-46d3-80cd-2446c5940049 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d5319326-fc6f-4d93-85ae-98a8b422dc14", "name": "network-vif-deleted", "tag": "ea0cb073-0c12-471c-b08c-f88ce79e8500"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:26.497480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-89bc7c76-51d9-4b1d-b1ea-5be66c2678dd req-ae0ea952-7f96-46d3-80cd-2446c5940049 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:26.498067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-89bc7c76-51d9-4b1d-b1ea-5be66c2678dd req-ae0ea952-7f96-46d3-80cd-2446c5940049 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:26.498586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-89bc7c76-51d9-4b1d-b1ea-5be66c2678dd req-ae0ea952-7f96-46d3-80cd-2446c5940049 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:26.511134 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-89bc7c76-51d9-4b1d-b1ea-5be66c2678dd req-ae0ea952-7f96-46d3-80cd-2446c5940049 service nova] Creating event network-vif-deleted:ea0cb073-0c12-471c-b08c-f88ce79e8500 for instance d5319326-fc6f-4d93-85ae-98a8b422dc14 on nch2-master-1 Sep 27 10:30:26.517250 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-89bc7c76-51d9-4b1d-b1ea-5be66c2678dd req-ae0ea952-7f96-46d3-80cd-2446c5940049 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033711 Sep 27 10:30:26.518649 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2805/6072] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:26.947758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-01241586-aa4f-4ff7-91a7-29a9dfef5754 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:26.948377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-01241586-aa4f-4ff7-91a7-29a9dfef5754 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:26.955526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01241586-aa4f-4ff7-91a7-29a9dfef5754 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:26.955955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01241586-aa4f-4ff7-91a7-29a9dfef5754 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:26.956383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01241586-aa4f-4ff7-91a7-29a9dfef5754 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:26.957131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01241586-aa4f-4ff7-91a7-29a9dfef5754 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:26.957585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01241586-aa4f-4ff7-91a7-29a9dfef5754 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:26.958011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01241586-aa4f-4ff7-91a7-29a9dfef5754 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:27.015417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:27.016164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:27.022521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:27.023022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:27.023442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:27.024179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:27.024652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:27.025117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:27.065756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:27.066255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:27.066899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:27.067560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lazy-loading 'fault' on Instance uuid d5319326-fc6f-4d93-85ae-98a8b422dc14 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:30:27.104357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-01241586-aa4f-4ff7-91a7-29a9dfef5754 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:27.118792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:27.121635 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-01241586-aa4f-4ff7-91a7-29a9dfef5754 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1515 microversion: 2.1 time: 0.177812 Sep 27 10:30:27.122646 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3268/6073] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:26 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1515 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:27.149231 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fb24a865-3099-43af-9838-e302a3497fae tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 200 len: 1331 microversion: 2.1 time: 0.138891 Sep 27 10:30:27.165066 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2806/6074] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:27 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 1331 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:30:27.798435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13edd1fa-e477-4437-a5bf-a383daff233f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:27.800912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-13edd1fa-e477-4437-a5bf-a383daff233f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:27.807570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13edd1fa-e477-4437-a5bf-a383daff233f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:27.808065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13edd1fa-e477-4437-a5bf-a383daff233f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:27.808519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13edd1fa-e477-4437-a5bf-a383daff233f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:27.809327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13edd1fa-e477-4437-a5bf-a383daff233f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:27.809793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13edd1fa-e477-4437-a5bf-a383daff233f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:27.810208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13edd1fa-e477-4437-a5bf-a383daff233f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:27.864124 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13edd1fa-e477-4437-a5bf-a383daff233f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3/action" status: 200 len: 24571 microversion: 2.1 time: 0.069592 Sep 27 10:30:27.864833 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3269/6075] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:30:27 2022] POST /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3/action => generated 24571 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:30:28.176898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:28.177603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:28.188578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:28.189355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:28.190038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:28.191162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:28.191750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:28.192396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:28.210466 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 HTTP/1.1 Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'd5319326-fc6f-4d93-85ae-98a8b422dc14'} Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:28.212189 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:28.218242 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:28.218242 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:28.218242 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:28.218242 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:30:28.218242 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:30:28.218242 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:30:28.218242 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:28.218242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:30:28.218242 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5015f59-8e78-4d58-b0c4-46efcc9f80d7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 404 len: 111 microversion: 2.1 time: 0.042502 Sep 27 10:30:28.218242 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2807/6076] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:28 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:30:28.223657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-89a8558f-1217-45de-afc3-533fb8bb87c5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '.version_select at 0x7f7f6c184f70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:28.249003 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-89a8558f-1217-45de-afc3-533fb8bb87c5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-995570364" status: 202 len: 0 microversion: 2.1 time: 0.029499 Sep 27 10:30:28.249662 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3270/6077] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:30:28 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-995570364 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:30:28.525315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7c1cd6e6-0dc2-4d39-a524-d567f007ceff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'create', calling method: .version_select at 0x7f7f635f88b0>, body: {"keypair": {"name": "tempest-keypair-1650579745", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGzxFRr/mCqf4WlD5qMIxH1R5lSlMg4P6UKL12DthoW4wQ/jdH+tMJoE/xa0zYQkupzmL+bpscrIFmHHgIy4QRmmUsjVCo5nhxfBee63P+PHu0R5bUQsCo3c6gEcmQ+xew=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:28.542401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7c1cd6e6-0dc2-4d39-a524-d567f007ceff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Getting quotas for project 40247edd5a444fd8b1c937c13d39393a. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:30:28.547987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7c1cd6e6-0dc2-4d39-a524-d567f007ceff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Getting quotas for user 6771df45b4ee4426b2f76a09eab2998a and project 40247edd5a444fd8b1c937c13d39393a. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:30:28.575666 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7c1cd6e6-0dc2-4d39-a524-d567f007ceff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.054229 Sep 27 10:30:28.576444 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2808/6078] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:30:28 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:29.009935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-74ee7503-b943-44ba-97c1-16ce9da8462d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:29.011228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-74ee7503-b943-44ba-97c1-16ce9da8462d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:29.019626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74ee7503-b943-44ba-97c1-16ce9da8462d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:29.020206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74ee7503-b943-44ba-97c1-16ce9da8462d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:29.020989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74ee7503-b943-44ba-97c1-16ce9da8462d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:29.021872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74ee7503-b943-44ba-97c1-16ce9da8462d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:29.022469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74ee7503-b943-44ba-97c1-16ce9da8462d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:29.023059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74ee7503-b943-44ba-97c1-16ce9da8462d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:29.121536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-74ee7503-b943-44ba-97c1-16ce9da8462d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lazy-loading 'services' on Instance uuid 403bfd67-4e7f-4918-baf9-4d14086a4598 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:30:29.205546 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-74ee7503-b943-44ba-97c1-16ce9da8462d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/os-volume_attachments/64e8066b-82ca-4278-ac3b-4ce6d3460710" status: 202 len: 0 microversion: 2.1 time: 0.199825 Sep 27 10:30:29.206255 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3271/6079] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:30:29 2022] DELETE /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/os-volume_attachments/64e8066b-82ca-4278-ac3b-4ce6d3460710 => generated 0 bytes in 202 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:30:29.216642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7373198-ff30-4393-8b9f-325256287db4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:29.220487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7373198-ff30-4393-8b9f-325256287db4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:29.228103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7373198-ff30-4393-8b9f-325256287db4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:29.228616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7373198-ff30-4393-8b9f-325256287db4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:29.229309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7373198-ff30-4393-8b9f-325256287db4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:29.230228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7373198-ff30-4393-8b9f-325256287db4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:29.230875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7373198-ff30-4393-8b9f-325256287db4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:29.231384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7373198-ff30-4393-8b9f-325256287db4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:29.267596 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c7373198-ff30-4393-8b9f-325256287db4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056231 Sep 27 10:30:29.268486 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2809/6080] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:30:29 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:30:30.094419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1559588025", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "01047f88-8e96-44cb-a0d6-1b5491078154"}], "security_groups": [{"name": "tempest-securitygroup--1901042034"}], "key_name": "tempest-keypair-1650579745", "user_data": "}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:30.218153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] validate_networks() for [('01047f88-8e96-44cb-a0d6-1b5491078154', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:30:30.282966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9be7858a-87c8-45be-b5ce-c6d9308006ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:30.285510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9be7858a-87c8-45be-b5ce-c6d9308006ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:30.294277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9be7858a-87c8-45be-b5ce-c6d9308006ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:30.295012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9be7858a-87c8-45be-b5ce-c6d9308006ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:30.295552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9be7858a-87c8-45be-b5ce-c6d9308006ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:30.296498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9be7858a-87c8-45be-b5ce-c6d9308006ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:30.297081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9be7858a-87c8-45be-b5ce-c6d9308006ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:30.297649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9be7858a-87c8-45be-b5ce-c6d9308006ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:30.333233 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9be7858a-87c8-45be-b5ce-c6d9308006ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.056285 Sep 27 10:30:30.333747 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2810/6081] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:30:30 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598/os-volume_attachments => generated 25 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 27 10:30:30.409121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-286133b4-7046-4afe-8ae4-1afdb132c96b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:30.410343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-286133b4-7046-4afe-8ae4-1afdb132c96b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:30.417450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-286133b4-7046-4afe-8ae4-1afdb132c96b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:30.417951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-286133b4-7046-4afe-8ae4-1afdb132c96b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:30.418685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-286133b4-7046-4afe-8ae4-1afdb132c96b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:30.419638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-286133b4-7046-4afe-8ae4-1afdb132c96b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:30.420189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-286133b4-7046-4afe-8ae4-1afdb132c96b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:30.420630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-286133b4-7046-4afe-8ae4-1afdb132c96b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:30.449894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-286133b4-7046-4afe-8ae4-1afdb132c96b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:30:30.545365 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-286133b4-7046-4afe-8ae4-1afdb132c96b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 204 len: 0 microversion: 2.1 time: 0.139512 Sep 27 10:30:30.546434 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2811/6082] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:30:30 2022] DELETE /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:30:30.556418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3a1edf05-3eee-487e-b733-052ed9ef8762 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:30.557415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3a1edf05-3eee-487e-b733-052ed9ef8762 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:30.565223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a1edf05-3eee-487e-b733-052ed9ef8762 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:30.565811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a1edf05-3eee-487e-b733-052ed9ef8762 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:30.566339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a1edf05-3eee-487e-b733-052ed9ef8762 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:30.567122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a1edf05-3eee-487e-b733-052ed9ef8762 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:30.567665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a1edf05-3eee-487e-b733-052ed9ef8762 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:30.568124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a1edf05-3eee-487e-b733-052ed9ef8762 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:30.677000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:30:30.677482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:30:30.677962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:30:30.678430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:30:30.679028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:30:30.689848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:30.690480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:30.691032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:30.691913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:30.692609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:30.693135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:30.713537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Getting quotas for project 40247edd5a444fd8b1c937c13d39393a. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:30:30.719958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Getting quotas for user 6771df45b4ee4426b2f76a09eab2998a and project 40247edd5a444fd8b1c937c13d39393a. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:30:30.723811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3a1edf05-3eee-487e-b733-052ed9ef8762 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:30.732784 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3a1edf05-3eee-487e-b733-052ed9ef8762 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1683 microversion: 2.1 time: 0.181141 Sep 27 10:30:30.733410 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2812/6083] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:30 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1683 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:30.735333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:30:30.738001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:30:30.790884 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c697801d-420c-4a57-9988-00f57b093503 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 0.702401 Sep 27 10:30:30.791624 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3272/6084] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:30:30 2022] POST /compute/v2.1/servers => generated 409 bytes in 704 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:30:30.804328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-00773573-615a-4366-977a-86300f9a929d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:30.805020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-00773573-615a-4366-977a-86300f9a929d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:30.864122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-00773573-615a-4366-977a-86300f9a929d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:30.872047 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-00773573-615a-4366-977a-86300f9a929d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1246 microversion: 2.1 time: 0.072227 Sep 27 10:30:30.872779 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2813/6085] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:30 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1246 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:31.760858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d05ba5c-b7cb-4f3e-86b6-9adc7c639f59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:31.761507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5d05ba5c-b7cb-4f3e-86b6-9adc7c639f59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:31.768889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d05ba5c-b7cb-4f3e-86b6-9adc7c639f59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:31.769484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d05ba5c-b7cb-4f3e-86b6-9adc7c639f59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:31.770000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d05ba5c-b7cb-4f3e-86b6-9adc7c639f59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:31.770808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d05ba5c-b7cb-4f3e-86b6-9adc7c639f59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:31.771318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d05ba5c-b7cb-4f3e-86b6-9adc7c639f59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:31.771825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d05ba5c-b7cb-4f3e-86b6-9adc7c639f59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:31.897750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-62a97738-082e-47c7-8170-51d59703534b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:31.899171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-62a97738-082e-47c7-8170-51d59703534b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:31.910055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62a97738-082e-47c7-8170-51d59703534b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:31.911017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62a97738-082e-47c7-8170-51d59703534b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:31.911680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62a97738-082e-47c7-8170-51d59703534b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:31.912677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62a97738-082e-47c7-8170-51d59703534b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:31.914252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62a97738-082e-47c7-8170-51d59703534b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:31.915023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62a97738-082e-47c7-8170-51d59703534b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:31.929677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5d05ba5c-b7cb-4f3e-86b6-9adc7c639f59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:31.939964 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5d05ba5c-b7cb-4f3e-86b6-9adc7c639f59 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1683 microversion: 2.1 time: 0.183608 Sep 27 10:30:31.940933 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3273/6086] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:31 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1683 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:31.988694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-62a97738-082e-47c7-8170-51d59703534b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:31.997375 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-62a97738-082e-47c7-8170-51d59703534b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1242 microversion: 2.1 time: 0.105594 Sep 27 10:30:31.997965 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2814/6087] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:31 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1242 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:30:32.966817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0bc05643-f2f1-44f7-aff3-13a9196c5a68 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:32.967569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0bc05643-f2f1-44f7-aff3-13a9196c5a68 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:32.976783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0bc05643-f2f1-44f7-aff3-13a9196c5a68 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:32.978020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0bc05643-f2f1-44f7-aff3-13a9196c5a68 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:32.978350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0bc05643-f2f1-44f7-aff3-13a9196c5a68 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:32.979393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0bc05643-f2f1-44f7-aff3-13a9196c5a68 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:32.980277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0bc05643-f2f1-44f7-aff3-13a9196c5a68 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:32.980410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0bc05643-f2f1-44f7-aff3-13a9196c5a68 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:33.022361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-acabde4a-88f6-4f84-92de-8624bba4653b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:33.023714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-acabde4a-88f6-4f84-92de-8624bba4653b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:33.036542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acabde4a-88f6-4f84-92de-8624bba4653b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:33.038060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acabde4a-88f6-4f84-92de-8624bba4653b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:33.038707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acabde4a-88f6-4f84-92de-8624bba4653b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:33.039828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acabde4a-88f6-4f84-92de-8624bba4653b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:33.040504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acabde4a-88f6-4f84-92de-8624bba4653b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:33.041162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-acabde4a-88f6-4f84-92de-8624bba4653b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:33.149011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-acabde4a-88f6-4f84-92de-8624bba4653b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:33.150308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0bc05643-f2f1-44f7-aff3-13a9196c5a68 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:33.158542 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-acabde4a-88f6-4f84-92de-8624bba4653b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1304 microversion: 2.1 time: 0.141068 Sep 27 10:30:33.159796 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2815/6088] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:33 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1304 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:30:33.162869 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0bc05643-f2f1-44f7-aff3-13a9196c5a68 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1683 microversion: 2.1 time: 0.200991 Sep 27 10:30:33.163690 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3274/6089] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:32 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1683 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:34.033035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-5abf135e-db3c-4c40-838f-4d0f277b7a55 req-68cc253c-6a3c-42e2-84dd-22830f3ed0f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "403bfd67-4e7f-4918-baf9-4d14086a4598", "name": "network-vif-deleted", "tag": "64716a19-8703-40a7-b759-cc1ba1192e49"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:34.055170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5abf135e-db3c-4c40-838f-4d0f277b7a55 req-68cc253c-6a3c-42e2-84dd-22830f3ed0f4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:34.055714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5abf135e-db3c-4c40-838f-4d0f277b7a55 req-68cc253c-6a3c-42e2-84dd-22830f3ed0f4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:34.057219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-5abf135e-db3c-4c40-838f-4d0f277b7a55 req-68cc253c-6a3c-42e2-84dd-22830f3ed0f4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:34.099923 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-5abf135e-db3c-4c40-838f-4d0f277b7a55 req-68cc253c-6a3c-42e2-84dd-22830f3ed0f4 service nova] Creating event network-vif-deleted:64716a19-8703-40a7-b759-cc1ba1192e49 for instance 403bfd67-4e7f-4918-baf9-4d14086a4598 on nch2-master-1 Sep 27 10:30:34.114432 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-5abf135e-db3c-4c40-838f-4d0f277b7a55 req-68cc253c-6a3c-42e2-84dd-22830f3ed0f4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087939 Sep 27 10:30:34.116164 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2816/6090] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:34.191677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-635aefd4-8e62-4e62-8e9b-9003baae66ff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:34.192430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-635aefd4-8e62-4e62-8e9b-9003baae66ff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:34.194312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8ce73f4f-6634-43ba-af04-d8bbe85859ae tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:34.194994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8ce73f4f-6634-43ba-af04-d8bbe85859ae tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:34.217890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ce73f4f-6634-43ba-af04-d8bbe85859ae tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:34.218383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ce73f4f-6634-43ba-af04-d8bbe85859ae tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:34.226331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-635aefd4-8e62-4e62-8e9b-9003baae66ff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:34.226927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-635aefd4-8e62-4e62-8e9b-9003baae66ff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:34.227463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ce73f4f-6634-43ba-af04-d8bbe85859ae tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:34.228349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-635aefd4-8e62-4e62-8e9b-9003baae66ff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:34.230557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ce73f4f-6634-43ba-af04-d8bbe85859ae tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:34.231059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ce73f4f-6634-43ba-af04-d8bbe85859ae tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:34.231507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ce73f4f-6634-43ba-af04-d8bbe85859ae tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:34.236310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-635aefd4-8e62-4e62-8e9b-9003baae66ff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:34.237013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-635aefd4-8e62-4e62-8e9b-9003baae66ff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:34.241721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-635aefd4-8e62-4e62-8e9b-9003baae66ff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:34.519380 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8ce73f4f-6634-43ba-af04-d8bbe85859ae tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1832 microversion: 2.1 time: 0.343652 Sep 27 10:30:34.520807 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3275/6091] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:34 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1832 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:34.526797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:34.527610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:34.534562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:34.535327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:34.536788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:34.537819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:34.538691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:34.539337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:34.582858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-635aefd4-8e62-4e62-8e9b-9003baae66ff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:34.606559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:34.607066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:34.607513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:34.607981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lazy-loading 'fault' on Instance uuid 403bfd67-4e7f-4918-baf9-4d14086a4598 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:30:34.615033 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-635aefd4-8e62-4e62-8e9b-9003baae66ff tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1372 microversion: 2.1 time: 0.419738 Sep 27 10:30:34.615532 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2817/6092] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:34 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1372 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:34.692434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:34.706460 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7097106c-0c48-4794-8097-d8f20bf67058 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 200 len: 1332 microversion: 2.1 time: 0.183546 Sep 27 10:30:34.708518 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3276/6093] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:34 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 1332 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:35.641825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ff8a954-006a-437d-bb7a-e47a1bc47e07 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:35.643902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ff8a954-006a-437d-bb7a-e47a1bc47e07 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:35.654608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ff8a954-006a-437d-bb7a-e47a1bc47e07 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:35.655579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ff8a954-006a-437d-bb7a-e47a1bc47e07 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:35.657128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ff8a954-006a-437d-bb7a-e47a1bc47e07 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:35.659396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ff8a954-006a-437d-bb7a-e47a1bc47e07 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:35.661683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ff8a954-006a-437d-bb7a-e47a1bc47e07 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:35.662469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ff8a954-006a-437d-bb7a-e47a1bc47e07 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:35.667467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:35.736440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:35.737175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:35.760264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:35.761024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:35.762082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:35.765750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:35.768149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:35.769656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:35.801842 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 HTTP/1.1 Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '403bfd67-4e7f-4918-baf9-4d14086a4598'} Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:35.804240 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:30:35.813354 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:35.813354 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:35.813354 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:35.813354 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:30:35.813354 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:30:35.813354 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:30:35.813354 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:30:35.813354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:30:35.813354 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3f54ef72-cd32-494d-af62-ae44ea50a72f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 404 len: 111 microversion: 2.1 time: 0.074344 Sep 27 10:30:35.813354 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3277/6094] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:35 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 111 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:30:35.824351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-73508302-8878-4c65-b2a8-3ac2f5dbb8cc tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '.version_select at 0x7f7f63763430>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:35.868394 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-73508302-8878-4c65-b2a8-3ac2f5dbb8cc tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1250635480" status: 202 len: 0 microversion: 2.1 time: 0.054912 Sep 27 10:30:35.870166 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3278/6095] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:30:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1250635480 => generated 0 bytes in 58 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:30:36.137914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9f58f1a7-e5ae-49c2-8dbe-bb712f14d076 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:36.138719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9f58f1a7-e5ae-49c2-8dbe-bb712f14d076 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:36.156262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f58f1a7-e5ae-49c2-8dbe-bb712f14d076 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:36.157116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f58f1a7-e5ae-49c2-8dbe-bb712f14d076 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:36.157744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f58f1a7-e5ae-49c2-8dbe-bb712f14d076 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:36.158779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f58f1a7-e5ae-49c2-8dbe-bb712f14d076 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:36.159414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f58f1a7-e5ae-49c2-8dbe-bb712f14d076 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:36.160492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9f58f1a7-e5ae-49c2-8dbe-bb712f14d076 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:36.192249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9f58f1a7-e5ae-49c2-8dbe-bb712f14d076 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:30:36.251840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2ff8a954-006a-437d-bb7a-e47a1bc47e07 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:36.270056 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ff8a954-006a-437d-bb7a-e47a1bc47e07 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1372 microversion: 2.1 time: 0.631879 Sep 27 10:30:36.281290 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2818/6096] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:35 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1372 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:36.301660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d512bc0e-88c2-450c-8240-f0f16dc5e82e req-00220324-9505-494a-92b1-55ec96f7acdb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "39429342-b89d-443d-9645-17ec8e2869f3", "tag": "b41f6e87-e1ce-49b6-a47a-5129ca582abc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:36.311067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:36.319117 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9f58f1a7-e5ae-49c2-8dbe-bb712f14d076 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "DELETE /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 204 len: 0 microversion: 2.1 time: 0.185859 Sep 27 10:30:36.320603 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3279/6097] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:30:36 2022] DELETE /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:30:36.321435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d512bc0e-88c2-450c-8240-f0f16dc5e82e req-00220324-9505-494a-92b1-55ec96f7acdb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:36.322140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d512bc0e-88c2-450c-8240-f0f16dc5e82e req-00220324-9505-494a-92b1-55ec96f7acdb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:36.322766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d512bc0e-88c2-450c-8240-f0f16dc5e82e req-00220324-9505-494a-92b1-55ec96f7acdb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:36.329471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e333f45b-602b-462d-954f-cf660b131e1a req-f5445d05-5c7b-4372-9e99-56c60b388973 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dc9ddb88-bfb7-40d7-b615-9755d1e824d5", "tag": "b77efabf-15d1-4744-b2d6-473b94c9fb51"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:36.346725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e333f45b-602b-462d-954f-cf660b131e1a req-f5445d05-5c7b-4372-9e99-56c60b388973 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:36.347124 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d512bc0e-88c2-450c-8240-f0f16dc5e82e req-00220324-9505-494a-92b1-55ec96f7acdb service nova] Creating event network-changed:b41f6e87-e1ce-49b6-a47a-5129ca582abc for instance 39429342-b89d-443d-9645-17ec8e2869f3 on nch1-master-1 Sep 27 10:30:36.348023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e333f45b-602b-462d-954f-cf660b131e1a req-f5445d05-5c7b-4372-9e99-56c60b388973 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:36.349938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e333f45b-602b-462d-954f-cf660b131e1a req-f5445d05-5c7b-4372-9e99-56c60b388973 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:36.356965 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d512bc0e-88c2-450c-8240-f0f16dc5e82e req-00220324-9505-494a-92b1-55ec96f7acdb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.073298 Sep 27 10:30:36.357990 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2819/6098] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:36.366764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b894a6f3-1d57-45f4-888f-be3d92813663 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:36.368085 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e333f45b-602b-462d-954f-cf660b131e1a req-f5445d05-5c7b-4372-9e99-56c60b388973 service nova] Creating event network-changed:b77efabf-15d1-4744-b2d6-473b94c9fb51 for instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 on nch2-master-1 Sep 27 10:30:36.369070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b894a6f3-1d57-45f4-888f-be3d92813663 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:36.374986 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e333f45b-602b-462d-954f-cf660b131e1a req-f5445d05-5c7b-4372-9e99-56c60b388973 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051740 Sep 27 10:30:36.376969 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3280/6099] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:36.378753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b894a6f3-1d57-45f4-888f-be3d92813663 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:36.379530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b894a6f3-1d57-45f4-888f-be3d92813663 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:36.380183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b894a6f3-1d57-45f4-888f-be3d92813663 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:36.381000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b894a6f3-1d57-45f4-888f-be3d92813663 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:36.381459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b894a6f3-1d57-45f4-888f-be3d92813663 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:36.381911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b894a6f3-1d57-45f4-888f-be3d92813663 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:36.677627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b894a6f3-1d57-45f4-888f-be3d92813663 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:36.692390 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b894a6f3-1d57-45f4-888f-be3d92813663 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1687 microversion: 2.1 time: 0.332818 Sep 27 10:30:36.694027 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2820/6100] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:36 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1687 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:36.993182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06c63913-ac84-4b02-b72a-02c4c1cb99f7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: .version_select at 0x7f7f63ac34c0>, body: {"keypair": {"name": "tempest-keypair-1709046796", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK87bJ4jlqp2lSN23Qm9uqEoHFOwsTD229c7ZMH5KsTdxmTMYMgi5hrA4LK4tcE8agz+6aWPry8dnXTJUqPvnvFvHCRAIHWJzCMGP0OHKkOxKxYKw8tqa7eqbUN3HLL7ig=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:37.010847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-06c63913-ac84-4b02-b72a-02c4c1cb99f7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for project 90c0fbb4a6d84767b920564584715f9d. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:30:37.017591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-06c63913-ac84-4b02-b72a-02c4c1cb99f7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for user 64090c8fd3394980ad5409adcde4986f and project 90c0fbb4a6d84767b920564584715f9d. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:30:37.047687 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-06c63913-ac84-4b02-b72a-02c4c1cb99f7 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.059133 Sep 27 10:30:37.048509 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3281/6101] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:30:36 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:37.295808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a9093750-43ba-4824-b0e1-8303f00367f4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:37.297137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a9093750-43ba-4824-b0e1-8303f00367f4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:37.306244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9093750-43ba-4824-b0e1-8303f00367f4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:37.307198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9093750-43ba-4824-b0e1-8303f00367f4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:37.308134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9093750-43ba-4824-b0e1-8303f00367f4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:37.309302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9093750-43ba-4824-b0e1-8303f00367f4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:37.309846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9093750-43ba-4824-b0e1-8303f00367f4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:37.310982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9093750-43ba-4824-b0e1-8303f00367f4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:37.548059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a9093750-43ba-4824-b0e1-8303f00367f4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:37.557566 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a9093750-43ba-4824-b0e1-8303f00367f4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1372 microversion: 2.1 time: 0.266604 Sep 27 10:30:37.558728 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2821/6102] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:37 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1372 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:37.719026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-14312379-96ac-41e3-804b-6f1712c53015 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:37.719764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-14312379-96ac-41e3-804b-6f1712c53015 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:37.727835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14312379-96ac-41e3-804b-6f1712c53015 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:37.728808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14312379-96ac-41e3-804b-6f1712c53015 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:37.729444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14312379-96ac-41e3-804b-6f1712c53015 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:37.730494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14312379-96ac-41e3-804b-6f1712c53015 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:37.731133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14312379-96ac-41e3-804b-6f1712c53015 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:37.731731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14312379-96ac-41e3-804b-6f1712c53015 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:38.107459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-14312379-96ac-41e3-804b-6f1712c53015 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:38.117396 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-14312379-96ac-41e3-804b-6f1712c53015 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 200 len: 1687 microversion: 2.1 time: 0.403016 Sep 27 10:30:38.118296 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3282/6103] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:37 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 1687 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:38.397214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-4c99d88d-0041-4bef-88e1-a8882f31c617 req-73de5bbd-2689-423c-919c-4cac29660621 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39429342-b89d-443d-9645-17ec8e2869f3", "name": "network-vif-deleted", "tag": "b41f6e87-e1ce-49b6-a47a-5129ca582abc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:38.417122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4c99d88d-0041-4bef-88e1-a8882f31c617 req-73de5bbd-2689-423c-919c-4cac29660621 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:38.418215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4c99d88d-0041-4bef-88e1-a8882f31c617 req-73de5bbd-2689-423c-919c-4cac29660621 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:38.419138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4c99d88d-0041-4bef-88e1-a8882f31c617 req-73de5bbd-2689-423c-919c-4cac29660621 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:38.447787 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-4c99d88d-0041-4bef-88e1-a8882f31c617 req-73de5bbd-2689-423c-919c-4cac29660621 service nova] Creating event network-vif-deleted:b41f6e87-e1ce-49b6-a47a-5129ca582abc for instance 39429342-b89d-443d-9645-17ec8e2869f3 on nch1-master-1 Sep 27 10:30:38.464344 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-4c99d88d-0041-4bef-88e1-a8882f31c617 req-73de5bbd-2689-423c-919c-4cac29660621 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073624 Sep 27 10:30:38.466158 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2822/6104] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:38.582289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-94a3e641-23bd-4c07-b1da-fc68340a44f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:38.583334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-94a3e641-23bd-4c07-b1da-fc68340a44f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:38.591583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94a3e641-23bd-4c07-b1da-fc68340a44f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:38.593350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94a3e641-23bd-4c07-b1da-fc68340a44f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:38.594468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94a3e641-23bd-4c07-b1da-fc68340a44f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:38.595782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94a3e641-23bd-4c07-b1da-fc68340a44f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:38.596540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94a3e641-23bd-4c07-b1da-fc68340a44f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:38.597560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94a3e641-23bd-4c07-b1da-fc68340a44f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:38.883955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-94a3e641-23bd-4c07-b1da-fc68340a44f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:38.892510 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-94a3e641-23bd-4c07-b1da-fc68340a44f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1372 microversion: 2.1 time: 0.314777 Sep 27 10:30:38.893384 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3283/6105] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:38 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1372 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:39.144088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:39.144850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 39429342-b89d-443d-9645-17ec8e2869f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:39.165714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-524496501", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "6a85e0d6-e759-43a8-9abb-8e12abcca6c4"}], "security_groups": [{"name": "tempest-securitygroup--579846872"}], "key_name": "tempest-keypair-1709046796", "user_data": "}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:39.166477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:39.168696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:39.169184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:39.170612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:39.171140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:39.172355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:39.193133 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] HTTP exception thrown: Instance 39429342-b89d-443d-9645-17ec8e2869f3 could not be found. Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Request method failure captured: Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 HTTP/1.1 Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtAz-I5R9pkEmM8ENwgEXlrfwhpK5QJgpteQwRSpBNpzmPCQ6_jXCbvGwjb68dGwzfz0hV_obx0eVwNZLFOTRd61-pMOQhq1CSGhYJOTDYOHjRjmEKJs84qfkSC88em1BS9SfLSGMC-OPias42e-aueDGLNj17wWnJe-vn54z9Dzb2o Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 86e71212d96f46a78680afaacd8709ca Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_86e71212d96f46a78680afaacd8709ca"}]}] Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 86e71212d96f46a78680afaacd8709ca Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestNetworkAdvancedServerOps-1611618676-project Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 0bd8a19d0ab44011b5430f26757a0efe Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1611618676-project Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 39429342-b89d-443d-9645-17ec8e2869f3 could not be found. Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '39429342-b89d-443d-9645-17ec8e2869f3'} Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 39429342-b89d-443d-9645-17ec8e2869f3 could not be found. Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 39429342-b89d-443d-9645-17ec8e2869f3 could not be found. Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:39.194852 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:30:39.200344 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:39.200344 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:30:39.200344 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:30:39.200344 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:30:39.200344 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:30:39.200344 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:30:39.200344 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 39429342-b89d-443d-9645-17ec8e2869f3 could not be found. Sep 27 10:30:39.200344 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:30:39.200344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Returning 404 to user: Instance 39429342-b89d-443d-9645-17ec8e2869f3 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:30:39.200344 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e269e5e4-3629-448d-a0a5-0685bef3d4fa tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3" status: 404 len: 111 microversion: 2.1 time: 0.058037 Sep 27 10:30:39.200344 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2823/6106] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:39 2022] GET /compute/v2.1/servers/39429342-b89d-443d-9645-17ec8e2869f3 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:30:39.354799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] validate_networks() for [('6a85e0d6-e759-43a8-9abb-8e12abcca6c4', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:30:39.915812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9c6110e1-2317-4099-ba08-a4b113ea584d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:39.916716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9c6110e1-2317-4099-ba08-a4b113ea584d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:39.924543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c6110e1-2317-4099-ba08-a4b113ea584d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:39.925285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c6110e1-2317-4099-ba08-a4b113ea584d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:39.925840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c6110e1-2317-4099-ba08-a4b113ea584d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:39.926682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c6110e1-2317-4099-ba08-a4b113ea584d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:39.927261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c6110e1-2317-4099-ba08-a4b113ea584d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:39.927819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9c6110e1-2317-4099-ba08-a4b113ea584d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:40.075901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9c6110e1-2317-4099-ba08-a4b113ea584d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:40.084134 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9c6110e1-2317-4099-ba08-a4b113ea584d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1372 microversion: 2.1 time: 0.172333 Sep 27 10:30:40.085048 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2824/6107] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:39 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1372 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:40.089449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:30:40.090020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:30:40.091795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:30:40.092315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:30:40.092938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:30:40.104222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:40.104905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:40.105456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:40.106302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:40.106912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:40.107453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:40.124914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for project 90c0fbb4a6d84767b920564584715f9d. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:30:40.126348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=2,nova_api:DELETE=2,nova_api:INSERT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:40.131962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for user 64090c8fd3394980ad5409adcde4986f and project 90c0fbb4a6d84767b920564584715f9d. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:30:40.147062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:30:40.149564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:30:40.197371 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dee2e1c8-0903-4801-a1de-52325de9f1df tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.036522 Sep 27 10:30:40.204333 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3284/6108] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:30:39 2022] POST /compute/v2.1/servers => generated 401 bytes in 1038 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 10:30:40.210822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-40697828-845e-485e-926d-2d62001ac1ad tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:40.211453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-40697828-845e-485e-926d-2d62001ac1ad tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:40.273386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-40697828-845e-485e-926d-2d62001ac1ad tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:40.281554 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-40697828-845e-485e-926d-2d62001ac1ad tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1241 microversion: 2.1 time: 0.075017 Sep 27 10:30:40.282151 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2825/6109] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:40 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1241 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:41.108357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-974c2a6c-ceec-44ca-a1ec-11e01815cce9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:41.109120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-974c2a6c-ceec-44ca-a1ec-11e01815cce9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:41.118680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-974c2a6c-ceec-44ca-a1ec-11e01815cce9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:41.119254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-974c2a6c-ceec-44ca-a1ec-11e01815cce9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:41.119735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-974c2a6c-ceec-44ca-a1ec-11e01815cce9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:41.120463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-974c2a6c-ceec-44ca-a1ec-11e01815cce9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:41.120977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-974c2a6c-ceec-44ca-a1ec-11e01815cce9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:41.121418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-974c2a6c-ceec-44ca-a1ec-11e01815cce9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:41.125846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:UPDATE=2,nova_cell1:SELECT=95,nova_cell1:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:41.305958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-43e6d87e-0e42-4cf8-90f1-ee54c1e3bf7e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:41.306615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-43e6d87e-0e42-4cf8-90f1-ee54c1e3bf7e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:41.314678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43e6d87e-0e42-4cf8-90f1-ee54c1e3bf7e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:41.315808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43e6d87e-0e42-4cf8-90f1-ee54c1e3bf7e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:41.316541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43e6d87e-0e42-4cf8-90f1-ee54c1e3bf7e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:41.317539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43e6d87e-0e42-4cf8-90f1-ee54c1e3bf7e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:41.317998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43e6d87e-0e42-4cf8-90f1-ee54c1e3bf7e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:41.318606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-43e6d87e-0e42-4cf8-90f1-ee54c1e3bf7e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:41.346858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-974c2a6c-ceec-44ca-a1ec-11e01815cce9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:41.356159 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-974c2a6c-ceec-44ca-a1ec-11e01815cce9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1372 microversion: 2.1 time: 0.251736 Sep 27 10:30:41.369388 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3285/6110] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:41 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1372 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:41.395217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-43e6d87e-0e42-4cf8-90f1-ee54c1e3bf7e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:41.403814 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-43e6d87e-0e42-4cf8-90f1-ee54c1e3bf7e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1293 microversion: 2.1 time: 0.102875 Sep 27 10:30:41.404398 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2826/6111] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:41 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1293 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:42.382510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8c1a811b-5f8a-40cd-adad-94affdaa8ecb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:42.383253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8c1a811b-5f8a-40cd-adad-94affdaa8ecb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:42.391378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c1a811b-5f8a-40cd-adad-94affdaa8ecb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:42.392236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c1a811b-5f8a-40cd-adad-94affdaa8ecb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:42.392997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c1a811b-5f8a-40cd-adad-94affdaa8ecb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:42.393968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c1a811b-5f8a-40cd-adad-94affdaa8ecb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:42.394683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c1a811b-5f8a-40cd-adad-94affdaa8ecb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:42.395469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c1a811b-5f8a-40cd-adad-94affdaa8ecb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:42.428682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa405181-c45b-4448-b3a6-00e0e0f58ff8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:42.430064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aa405181-c45b-4448-b3a6-00e0e0f58ff8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:42.442711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa405181-c45b-4448-b3a6-00e0e0f58ff8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:42.443584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa405181-c45b-4448-b3a6-00e0e0f58ff8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:42.445552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa405181-c45b-4448-b3a6-00e0e0f58ff8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:42.448541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa405181-c45b-4448-b3a6-00e0e0f58ff8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:42.450711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa405181-c45b-4448-b3a6-00e0e0f58ff8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:42.451944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa405181-c45b-4448-b3a6-00e0e0f58ff8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:42.524583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aa405181-c45b-4448-b3a6-00e0e0f58ff8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:42.534043 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aa405181-c45b-4448-b3a6-00e0e0f58ff8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1299 microversion: 2.1 time: 0.109126 Sep 27 10:30:42.549472 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2827/6112] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:42 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1299 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:30:42.761138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8c1a811b-5f8a-40cd-adad-94affdaa8ecb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:42.789893 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8c1a811b-5f8a-40cd-adad-94affdaa8ecb tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1372 microversion: 2.1 time: 0.411511 Sep 27 10:30:42.808741 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3286/6113] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:42 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1372 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:43.559350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-56bf7439-6f49-4fa7-b913-ad7ef4b9a8e0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:43.560167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-56bf7439-6f49-4fa7-b913-ad7ef4b9a8e0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:43.568620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56bf7439-6f49-4fa7-b913-ad7ef4b9a8e0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:43.569185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56bf7439-6f49-4fa7-b913-ad7ef4b9a8e0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:43.569686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56bf7439-6f49-4fa7-b913-ad7ef4b9a8e0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:43.570511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56bf7439-6f49-4fa7-b913-ad7ef4b9a8e0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:43.571023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56bf7439-6f49-4fa7-b913-ad7ef4b9a8e0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:43.571475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56bf7439-6f49-4fa7-b913-ad7ef4b9a8e0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:43.812993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4e2d67a6-b345-4942-98ac-a250e79c82f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:43.813770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4e2d67a6-b345-4942-98ac-a250e79c82f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:43.821357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e2d67a6-b345-4942-98ac-a250e79c82f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:43.821849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e2d67a6-b345-4942-98ac-a250e79c82f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:43.822576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e2d67a6-b345-4942-98ac-a250e79c82f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:43.823467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e2d67a6-b345-4942-98ac-a250e79c82f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:43.824087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e2d67a6-b345-4942-98ac-a250e79c82f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:43.824537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4e2d67a6-b345-4942-98ac-a250e79c82f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:43.898267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-56bf7439-6f49-4fa7-b913-ad7ef4b9a8e0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:43.907158 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-56bf7439-6f49-4fa7-b913-ad7ef4b9a8e0 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1366 microversion: 2.1 time: 0.352143 Sep 27 10:30:43.919200 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2828/6114] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:43 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1366 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:44.049484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4e2d67a6-b345-4942-98ac-a250e79c82f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:44.065429 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4e2d67a6-b345-4942-98ac-a250e79c82f6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1372 microversion: 2.1 time: 0.255671 Sep 27 10:30:44.067598 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3287/6115] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:43 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1372 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:44.109882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-463431bc-519b-4ac5-b0aa-d911a985c671 req-0e86c21d-9dab-4631-ae01-942a51f93b4b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dc9ddb88-bfb7-40d7-b615-9755d1e824d5", "name": "network-vif-plugged", "status": "completed", "tag": "b77efabf-15d1-4744-b2d6-473b94c9fb51"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:44.130910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-463431bc-519b-4ac5-b0aa-d911a985c671 req-0e86c21d-9dab-4631-ae01-942a51f93b4b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:44.134417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-463431bc-519b-4ac5-b0aa-d911a985c671 req-0e86c21d-9dab-4631-ae01-942a51f93b4b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:44.135992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-463431bc-519b-4ac5-b0aa-d911a985c671 req-0e86c21d-9dab-4631-ae01-942a51f93b4b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:44.157994 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-463431bc-519b-4ac5-b0aa-d911a985c671 req-0e86c21d-9dab-4631-ae01-942a51f93b4b service nova] Creating event network-vif-plugged:b77efabf-15d1-4744-b2d6-473b94c9fb51 for instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 on nch2-master-1 Sep 27 10:30:44.164000 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-463431bc-519b-4ac5-b0aa-d911a985c671 req-0e86c21d-9dab-4631-ae01-942a51f93b4b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.060360 Sep 27 10:30:44.165297 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2829/6116] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:30:44.242147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e98d4a2c-f3d2-42d1-85ee-0c9e3e61f181 req-af262e57-48aa-450a-9aa6-fbd6b76c3fba service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "15bd0cb2-4701-4992-9f64-895716df0533", "tag": "cccf9085-27fb-4577-ad7e-7d726fb318a7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:44.267778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e98d4a2c-f3d2-42d1-85ee-0c9e3e61f181 req-af262e57-48aa-450a-9aa6-fbd6b76c3fba service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:44.268910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e98d4a2c-f3d2-42d1-85ee-0c9e3e61f181 req-af262e57-48aa-450a-9aa6-fbd6b76c3fba service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:44.269469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e98d4a2c-f3d2-42d1-85ee-0c9e3e61f181 req-af262e57-48aa-450a-9aa6-fbd6b76c3fba service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:44.285909 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e98d4a2c-f3d2-42d1-85ee-0c9e3e61f181 req-af262e57-48aa-450a-9aa6-fbd6b76c3fba service nova] Creating event network-changed:cccf9085-27fb-4577-ad7e-7d726fb318a7 for instance 15bd0cb2-4701-4992-9f64-895716df0533 on nch1-master-1 Sep 27 10:30:44.292397 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e98d4a2c-f3d2-42d1-85ee-0c9e3e61f181 req-af262e57-48aa-450a-9aa6-fbd6b76c3fba service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056716 Sep 27 10:30:44.293349 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3288/6117] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:44.545894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-efd270ff-5f03-4bc8-a608-c3a49fedea2c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '.version_select at 0x7f7f6377b8b0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:44.571548 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-efd270ff-5f03-4bc8-a608-c3a49fedea2c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1304200436" status: 202 len: 0 microversion: 2.1 time: 0.030219 Sep 27 10:30:44.572276 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2830/6118] 10.222.0.26 () {58 vars in 1284 bytes} [Tue Sep 27 10:30:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1304200436 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:30:44.613182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef9bf94c-f783-4394-b276-888325b19ba7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'create', calling method: .version_select at 0x7f7f6c063d30>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-1344904965", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJTKsEbzl7h/Dcki1QgIkhutDQxcSIb+rikO9aziJre6O4VVYigzDDOlVk5YjrBMRUFKSTVe0IR0zQpRRVVKi/qgwD+Hm88JB+mVJkgaioeVbXQS9b+Z7hyJ6tWlZ3usBw=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:44.634826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-ef9bf94c-f783-4394-b276-888325b19ba7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for project 86e71212d96f46a78680afaacd8709ca. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:30:44.640145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-ef9bf94c-f783-4394-b276-888325b19ba7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for user 0bd8a19d0ab44011b5430f26757a0efe and project 86e71212d96f46a78680afaacd8709ca. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:30:44.666734 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef9bf94c-f783-4394-b276-888325b19ba7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.060298 Sep 27 10:30:44.667679 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3289/6119] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:30:44 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:44.933708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23ebde55-3f75-4cd5-af00-55f44143abdb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:44.934497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-23ebde55-3f75-4cd5-af00-55f44143abdb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:44.943351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ebde55-3f75-4cd5-af00-55f44143abdb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:44.944361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ebde55-3f75-4cd5-af00-55f44143abdb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:44.945511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ebde55-3f75-4cd5-af00-55f44143abdb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:44.946704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ebde55-3f75-4cd5-af00-55f44143abdb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:44.947092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ebde55-3f75-4cd5-af00-55f44143abdb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:44.947697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23ebde55-3f75-4cd5-af00-55f44143abdb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:45.090724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5fd15ca6-278d-4bbd-8281-29769f6400ba tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:45.091779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5fd15ca6-278d-4bbd-8281-29769f6400ba tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:45.101048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fd15ca6-278d-4bbd-8281-29769f6400ba tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:45.101534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fd15ca6-278d-4bbd-8281-29769f6400ba tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:45.101970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fd15ca6-278d-4bbd-8281-29769f6400ba tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:45.102662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fd15ca6-278d-4bbd-8281-29769f6400ba tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:45.103419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fd15ca6-278d-4bbd-8281-29769f6400ba tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:45.104285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fd15ca6-278d-4bbd-8281-29769f6400ba tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:45.146731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-23ebde55-3f75-4cd5-af00-55f44143abdb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:45.156323 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-23ebde55-3f75-4cd5-af00-55f44143abdb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1366 microversion: 2.1 time: 0.227208 Sep 27 10:30:45.157268 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2831/6120] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:44 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1366 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:45.276548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5fd15ca6-278d-4bbd-8281-29769f6400ba tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:45.286483 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5fd15ca6-278d-4bbd-8281-29769f6400ba tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1372 microversion: 2.1 time: 0.199749 Sep 27 10:30:45.287315 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3290/6121] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:45 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1372 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:46.181474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e7313c6e-2ae4-4e2b-9e2e-a8eea404043a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:46.183325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e7313c6e-2ae4-4e2b-9e2e-a8eea404043a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:46.192628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7313c6e-2ae4-4e2b-9e2e-a8eea404043a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:46.193710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7313c6e-2ae4-4e2b-9e2e-a8eea404043a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:46.194355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7313c6e-2ae4-4e2b-9e2e-a8eea404043a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:46.195618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7313c6e-2ae4-4e2b-9e2e-a8eea404043a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:46.196270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7313c6e-2ae4-4e2b-9e2e-a8eea404043a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:46.197190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e7313c6e-2ae4-4e2b-9e2e-a8eea404043a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:46.311250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-df3c91ea-139b-492d-8031-055c40b47e6b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:46.312137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-df3c91ea-139b-492d-8031-055c40b47e6b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:46.320103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df3c91ea-139b-492d-8031-055c40b47e6b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:46.320982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df3c91ea-139b-492d-8031-055c40b47e6b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:46.321671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df3c91ea-139b-492d-8031-055c40b47e6b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:46.322416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df3c91ea-139b-492d-8031-055c40b47e6b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:46.322860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df3c91ea-139b-492d-8031-055c40b47e6b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:46.323512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df3c91ea-139b-492d-8031-055c40b47e6b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:46.482088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-df3c91ea-139b-492d-8031-055c40b47e6b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:46.490664 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-df3c91ea-139b-492d-8031-055c40b47e6b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1566 microversion: 2.1 time: 0.183649 Sep 27 10:30:46.491986 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3291/6122] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:46 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1566 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:30:46.516096 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-548b611c-0f05-4661-afed-f78ed47a0809 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:46.517087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-548b611c-0f05-4661-afed-f78ed47a0809 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:46.524996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-548b611c-0f05-4661-afed-f78ed47a0809 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:46.525506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-548b611c-0f05-4661-afed-f78ed47a0809 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:46.525956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-548b611c-0f05-4661-afed-f78ed47a0809 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:46.526698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-548b611c-0f05-4661-afed-f78ed47a0809 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:46.527273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-548b611c-0f05-4661-afed-f78ed47a0809 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:46.527841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-548b611c-0f05-4661-afed-f78ed47a0809 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:46.531664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e7313c6e-2ae4-4e2b-9e2e-a8eea404043a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:46.542037 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e7313c6e-2ae4-4e2b-9e2e-a8eea404043a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1366 microversion: 2.1 time: 0.365461 Sep 27 10:30:46.543344 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2832/6123] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:46 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1366 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:46.620808 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-548b611c-0f05-4661-afed-f78ed47a0809 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.109600 Sep 27 10:30:46.621610 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3292/6124] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:30:46 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-interface => generated 284 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:47.569612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cfc6ac44-859c-41ca-aa3a-e5c9ef536b5b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:47.570907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cfc6ac44-859c-41ca-aa3a-e5c9ef536b5b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:47.583356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfc6ac44-859c-41ca-aa3a-e5c9ef536b5b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:47.584474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfc6ac44-859c-41ca-aa3a-e5c9ef536b5b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:47.585018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfc6ac44-859c-41ca-aa3a-e5c9ef536b5b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:47.586414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfc6ac44-859c-41ca-aa3a-e5c9ef536b5b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:47.587152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfc6ac44-859c-41ca-aa3a-e5c9ef536b5b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:47.587948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cfc6ac44-859c-41ca-aa3a-e5c9ef536b5b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:47.837275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cfc6ac44-859c-41ca-aa3a-e5c9ef536b5b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:47.847500 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cfc6ac44-859c-41ca-aa3a-e5c9ef536b5b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1366 microversion: 2.1 time: 0.283772 Sep 27 10:30:47.848724 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2833/6125] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:47 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1366 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:48.065455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e54effc1-1072-4db0-b9f4-2b29c96c1d97 req-316020cc-e273-4122-9b51-1c976f615b3c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dc9ddb88-bfb7-40d7-b615-9755d1e824d5", "tag": "b77efabf-15d1-4744-b2d6-473b94c9fb51"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:48.076874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e54effc1-1072-4db0-b9f4-2b29c96c1d97 req-316020cc-e273-4122-9b51-1c976f615b3c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:48.077495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e54effc1-1072-4db0-b9f4-2b29c96c1d97 req-316020cc-e273-4122-9b51-1c976f615b3c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:48.077993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e54effc1-1072-4db0-b9f4-2b29c96c1d97 req-316020cc-e273-4122-9b51-1c976f615b3c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:48.092769 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e54effc1-1072-4db0-b9f4-2b29c96c1d97 req-316020cc-e273-4122-9b51-1c976f615b3c service nova] Creating event network-changed:b77efabf-15d1-4744-b2d6-473b94c9fb51 for instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 on nch2-master-1 Sep 27 10:30:48.099919 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e54effc1-1072-4db0-b9f4-2b29c96c1d97 req-316020cc-e273-4122-9b51-1c976f615b3c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040239 Sep 27 10:30:48.101122 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3293/6126] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:48.893615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b8519a0-8ee0-4bbc-a6c0-c71b816ba67e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:48.897587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0b8519a0-8ee0-4bbc-a6c0-c71b816ba67e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:48.922445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b8519a0-8ee0-4bbc-a6c0-c71b816ba67e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:48.924651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b8519a0-8ee0-4bbc-a6c0-c71b816ba67e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:48.927764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b8519a0-8ee0-4bbc-a6c0-c71b816ba67e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:48.929565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b8519a0-8ee0-4bbc-a6c0-c71b816ba67e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:48.933601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b8519a0-8ee0-4bbc-a6c0-c71b816ba67e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:48.936863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b8519a0-8ee0-4bbc-a6c0-c71b816ba67e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:49.142056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0b8519a0-8ee0-4bbc-a6c0-c71b816ba67e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:49.153001 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0b8519a0-8ee0-4bbc-a6c0-c71b816ba67e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1366 microversion: 2.1 time: 0.283036 Sep 27 10:30:49.154060 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2834/6127] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:48 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1366 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:50.176639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:50.182308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-14406706-c6a4-4af5-bb58-74694641e1b4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:50.183311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-14406706-c6a4-4af5-bb58-74694641e1b4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:50.190673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14406706-c6a4-4af5-bb58-74694641e1b4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:50.191169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14406706-c6a4-4af5-bb58-74694641e1b4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:50.191774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14406706-c6a4-4af5-bb58-74694641e1b4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:50.192843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14406706-c6a4-4af5-bb58-74694641e1b4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:50.193314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14406706-c6a4-4af5-bb58-74694641e1b4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:50.193747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14406706-c6a4-4af5-bb58-74694641e1b4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:50.476408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-14406706-c6a4-4af5-bb58-74694641e1b4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:50.485390 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-14406706-c6a4-4af5-bb58-74694641e1b4 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1366 microversion: 2.1 time: 0.310834 Sep 27 10:30:50.486573 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3294/6128] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:50 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1366 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:50.999011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e8edb322-7805-4582-9cf9-c3eb39e1f183 req-09f48ecd-428a-4555-a4b9-1b22b49a33ad service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "15bd0cb2-4701-4992-9f64-895716df0533", "name": "network-vif-plugged", "status": "completed", "tag": "cccf9085-27fb-4577-ad7e-7d726fb318a7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:51.023338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:51.028749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e8edb322-7805-4582-9cf9-c3eb39e1f183 req-09f48ecd-428a-4555-a4b9-1b22b49a33ad service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:51.029752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e8edb322-7805-4582-9cf9-c3eb39e1f183 req-09f48ecd-428a-4555-a4b9-1b22b49a33ad service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:51.030455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e8edb322-7805-4582-9cf9-c3eb39e1f183 req-09f48ecd-428a-4555-a4b9-1b22b49a33ad service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:51.051023 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e8edb322-7805-4582-9cf9-c3eb39e1f183 req-09f48ecd-428a-4555-a4b9-1b22b49a33ad service nova] Creating event network-vif-plugged:cccf9085-27fb-4577-ad7e-7d726fb318a7 for instance 15bd0cb2-4701-4992-9f64-895716df0533 on nch1-master-1 Sep 27 10:30:51.063344 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e8edb322-7805-4582-9cf9-c3eb39e1f183 req-09f48ecd-428a-4555-a4b9-1b22b49a33ad service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075915 Sep 27 10:30:51.065083 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2835/6129] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:51.511738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0df544cb-de17-4129-93bd-72ea4250daa8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:51.513149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0df544cb-de17-4129-93bd-72ea4250daa8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:51.522496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0df544cb-de17-4129-93bd-72ea4250daa8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:51.525318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0df544cb-de17-4129-93bd-72ea4250daa8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:51.526140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0df544cb-de17-4129-93bd-72ea4250daa8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:51.527247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0df544cb-de17-4129-93bd-72ea4250daa8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:51.527885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0df544cb-de17-4129-93bd-72ea4250daa8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:51.528491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0df544cb-de17-4129-93bd-72ea4250daa8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:52.093302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0df544cb-de17-4129-93bd-72ea4250daa8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:52.102643 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0df544cb-de17-4129-93bd-72ea4250daa8 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1366 microversion: 2.1 time: 0.594767 Sep 27 10:30:52.110322 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3295/6130] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:51 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1366 bytes in 603 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:52.367420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-669899251", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "68bf2d15-4180-4570-a3b1-f45adcb1ba98"}], "key_name": "tempest-TestNetworkAdvancedServerOps-1344904965", "security_groups": [{"name": "tempest-secgroup-smoke-1617477252"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:52.508259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] validate_networks() for [('68bf2d15-4180-4570-a3b1-f45adcb1ba98', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:30:52.909936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:30:52.910435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:30:52.910934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:30:52.911380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:30:52.911794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:30:52.921248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:52.921841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:52.922374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:52.923199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:52.923794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:52.924264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:52.942957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for project 86e71212d96f46a78680afaacd8709ca. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:30:52.948704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for user 0bd8a19d0ab44011b5430f26757a0efe and project 86e71212d96f46a78680afaacd8709ca. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:30:52.963269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:30:52.965774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:30:53.023480 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bf432a4f-d759-4a68-b3f5-f9b46181bcb2 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.660489 Sep 27 10:30:53.024325 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2836/6131] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:30:52 2022] POST /compute/v2.1/servers => generated 406 bytes in 662 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:30:53.042651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8ed68de9-9399-464e-a51f-dbaf524e1796 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:53.043278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8ed68de9-9399-464e-a51f-dbaf524e1796 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:53.126444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8105e983-f402-4fbf-afcb-4c41dfa006ac tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:53.127327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8105e983-f402-4fbf-afcb-4c41dfa006ac tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:53.132831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8ed68de9-9399-464e-a51f-dbaf524e1796 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:53.137135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8105e983-f402-4fbf-afcb-4c41dfa006ac tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:53.138561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8105e983-f402-4fbf-afcb-4c41dfa006ac tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:53.139028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8105e983-f402-4fbf-afcb-4c41dfa006ac tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:53.139810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8105e983-f402-4fbf-afcb-4c41dfa006ac tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:53.140264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8105e983-f402-4fbf-afcb-4c41dfa006ac tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:53.140743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8105e983-f402-4fbf-afcb-4c41dfa006ac tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:53.145412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=21 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:53.159040 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8ed68de9-9399-464e-a51f-dbaf524e1796 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1266 microversion: 2.1 time: 0.126954 Sep 27 10:30:53.180281 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3296/6132] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:53 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1266 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:53.322385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8105e983-f402-4fbf-afcb-4c41dfa006ac tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:53.333550 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8105e983-f402-4fbf-afcb-4c41dfa006ac tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1557 microversion: 2.1 time: 0.211800 Sep 27 10:30:53.350945 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2837/6133] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:53 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1557 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:30:53.358914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e1bc1b3c-1275-40e6-b7f2-aa036870eaaf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:53.359462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e1bc1b3c-1275-40e6-b7f2-aa036870eaaf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:53.366940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc1b3c-1275-40e6-b7f2-aa036870eaaf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:53.367758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc1b3c-1275-40e6-b7f2-aa036870eaaf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:53.368445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc1b3c-1275-40e6-b7f2-aa036870eaaf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:53.369564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc1b3c-1275-40e6-b7f2-aa036870eaaf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:53.370163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc1b3c-1275-40e6-b7f2-aa036870eaaf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:53.370597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e1bc1b3c-1275-40e6-b7f2-aa036870eaaf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:53.460585 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e1bc1b3c-1275-40e6-b7f2-aa036870eaaf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.105829 Sep 27 10:30:53.461312 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3297/6134] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:30:53 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-interface => generated 285 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:54.187590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1750d6cd-a588-48f4-87e5-92492c9e58f1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:54.188230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1750d6cd-a588-48f4-87e5-92492c9e58f1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:54.197664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1750d6cd-a588-48f4-87e5-92492c9e58f1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:54.198520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1750d6cd-a588-48f4-87e5-92492c9e58f1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:54.198977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1750d6cd-a588-48f4-87e5-92492c9e58f1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:54.199915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1750d6cd-a588-48f4-87e5-92492c9e58f1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:54.200396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1750d6cd-a588-48f4-87e5-92492c9e58f1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:54.200877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1750d6cd-a588-48f4-87e5-92492c9e58f1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:54.283124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1750d6cd-a588-48f4-87e5-92492c9e58f1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:54.292571 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1750d6cd-a588-48f4-87e5-92492c9e58f1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1318 microversion: 2.1 time: 0.108610 Sep 27 10:30:54.311217 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2838/6135] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:54 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1318 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:54.678727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3208c050-2d24-4cfc-82e2-f45536252fa0 req-2cda9351-633f-46bc-a791-058c8a5b44a3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "15bd0cb2-4701-4992-9f64-895716df0533", "tag": "cccf9085-27fb-4577-ad7e-7d726fb318a7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:54.690619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3208c050-2d24-4cfc-82e2-f45536252fa0 req-2cda9351-633f-46bc-a791-058c8a5b44a3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:54.691104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3208c050-2d24-4cfc-82e2-f45536252fa0 req-2cda9351-633f-46bc-a791-058c8a5b44a3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:54.691541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3208c050-2d24-4cfc-82e2-f45536252fa0 req-2cda9351-633f-46bc-a791-058c8a5b44a3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:54.704730 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3208c050-2d24-4cfc-82e2-f45536252fa0 req-2cda9351-633f-46bc-a791-058c8a5b44a3 service nova] Creating event network-changed:cccf9085-27fb-4577-ad7e-7d726fb318a7 for instance 15bd0cb2-4701-4992-9f64-895716df0533 on nch1-master-1 Sep 27 10:30:54.710904 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3208c050-2d24-4cfc-82e2-f45536252fa0 req-2cda9351-633f-46bc-a791-058c8a5b44a3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037254 Sep 27 10:30:54.712175 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3298/6136] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:55.321373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-74118ed9-326a-4e48-986f-abc6f45b7154 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:55.323271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-74118ed9-326a-4e48-986f-abc6f45b7154 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:55.336312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74118ed9-326a-4e48-986f-abc6f45b7154 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:55.337240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74118ed9-326a-4e48-986f-abc6f45b7154 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:55.337900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74118ed9-326a-4e48-986f-abc6f45b7154 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:55.338984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74118ed9-326a-4e48-986f-abc6f45b7154 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:55.339668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74118ed9-326a-4e48-986f-abc6f45b7154 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:55.341510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-74118ed9-326a-4e48-986f-abc6f45b7154 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:55.905821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-74118ed9-326a-4e48-986f-abc6f45b7154 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:55.914323 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-74118ed9-326a-4e48-986f-abc6f45b7154 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1392 microversion: 2.1 time: 0.599372 Sep 27 10:30:55.914905 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2839/6137] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:55 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1392 bytes in 602 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:56.736892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a51ea5b9-dbcd-4ee2-9357-cebee87dc234 req-ce8fa894-1086-4e67-9f96-11b34f94fb85 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0565ee6c-196a-4ca9-981f-24e019cd89ed", "tag": "dde5d266-f09e-421c-b470-aacdbb240d15"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:30:56.747111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a51ea5b9-dbcd-4ee2-9357-cebee87dc234 req-ce8fa894-1086-4e67-9f96-11b34f94fb85 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:56.747602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a51ea5b9-dbcd-4ee2-9357-cebee87dc234 req-ce8fa894-1086-4e67-9f96-11b34f94fb85 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:56.748042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a51ea5b9-dbcd-4ee2-9357-cebee87dc234 req-ce8fa894-1086-4e67-9f96-11b34f94fb85 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:56.759634 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a51ea5b9-dbcd-4ee2-9357-cebee87dc234 req-ce8fa894-1086-4e67-9f96-11b34f94fb85 service nova] Creating event network-changed:dde5d266-f09e-421c-b470-aacdbb240d15 for instance 0565ee6c-196a-4ca9-981f-24e019cd89ed on nch2-master-1 Sep 27 10:30:56.764827 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a51ea5b9-dbcd-4ee2-9357-cebee87dc234 req-ce8fa894-1086-4e67-9f96-11b34f94fb85 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035035 Sep 27 10:30:56.766007 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3299/6138] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:30:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:30:56.938390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d7bf9b06-e9e2-4acd-b13e-8412c28cd794 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:56.939153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d7bf9b06-e9e2-4acd-b13e-8412c28cd794 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:56.946539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7bf9b06-e9e2-4acd-b13e-8412c28cd794 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:56.947045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7bf9b06-e9e2-4acd-b13e-8412c28cd794 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:56.947497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7bf9b06-e9e2-4acd-b13e-8412c28cd794 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:56.948221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7bf9b06-e9e2-4acd-b13e-8412c28cd794 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:56.948709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7bf9b06-e9e2-4acd-b13e-8412c28cd794 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:56.949187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7bf9b06-e9e2-4acd-b13e-8412c28cd794 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:57.128311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d7bf9b06-e9e2-4acd-b13e-8412c28cd794 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:57.130801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:30:57.137757 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d7bf9b06-e9e2-4acd-b13e-8412c28cd794 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1392 microversion: 2.1 time: 0.203702 Sep 27 10:30:57.138342 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2840/6139] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:56 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1392 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:58.162284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2e7a99a7-075a-46a6-a2da-8dd41af1ee71 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:58.163188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2e7a99a7-075a-46a6-a2da-8dd41af1ee71 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:58.171801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e7a99a7-075a-46a6-a2da-8dd41af1ee71 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:58.172290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e7a99a7-075a-46a6-a2da-8dd41af1ee71 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:58.172786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e7a99a7-075a-46a6-a2da-8dd41af1ee71 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:58.173488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e7a99a7-075a-46a6-a2da-8dd41af1ee71 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:58.174023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e7a99a7-075a-46a6-a2da-8dd41af1ee71 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:58.174482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2e7a99a7-075a-46a6-a2da-8dd41af1ee71 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:58.311317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2e7a99a7-075a-46a6-a2da-8dd41af1ee71 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:58.324026 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2e7a99a7-075a-46a6-a2da-8dd41af1ee71 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1392 microversion: 2.1 time: 0.166072 Sep 27 10:30:58.324582 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3300/6140] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:58 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1392 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:30:59.350389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7a5c64f9-a52f-4f82-b3e2-95a281c5381b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:30:59.351038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7a5c64f9-a52f-4f82-b3e2-95a281c5381b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:30:59.358563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a5c64f9-a52f-4f82-b3e2-95a281c5381b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:59.359071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a5c64f9-a52f-4f82-b3e2-95a281c5381b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:59.359556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a5c64f9-a52f-4f82-b3e2-95a281c5381b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:59.360281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a5c64f9-a52f-4f82-b3e2-95a281c5381b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:30:59.360781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a5c64f9-a52f-4f82-b3e2-95a281c5381b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:30:59.361251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a5c64f9-a52f-4f82-b3e2-95a281c5381b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:30:59.589642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a5c64f9-a52f-4f82-b3e2-95a281c5381b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:30:59.599555 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7a5c64f9-a52f-4f82-b3e2-95a281c5381b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1392 microversion: 2.1 time: 0.253714 Sep 27 10:30:59.600747 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2841/6141] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:30:59 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1392 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:00.623418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f7783a16-1786-4808-a826-dd7468583b4a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:00.624011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f7783a16-1786-4808-a826-dd7468583b4a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:00.631540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7783a16-1786-4808-a826-dd7468583b4a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:00.632028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7783a16-1786-4808-a826-dd7468583b4a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:00.632482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7783a16-1786-4808-a826-dd7468583b4a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:00.633221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7783a16-1786-4808-a826-dd7468583b4a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:00.633683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7783a16-1786-4808-a826-dd7468583b4a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:00.634125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7783a16-1786-4808-a826-dd7468583b4a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:00.771146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f7783a16-1786-4808-a826-dd7468583b4a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:00.779675 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f7783a16-1786-4808-a826-dd7468583b4a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1392 microversion: 2.1 time: 0.160389 Sep 27 10:31:00.780255 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3301/6142] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:00 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1392 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:01.814579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-721bf50a-9349-4f38-8737-1cf4fdc3c3de tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:01.815572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-721bf50a-9349-4f38-8737-1cf4fdc3c3de tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:01.823038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721bf50a-9349-4f38-8737-1cf4fdc3c3de tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:01.823549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721bf50a-9349-4f38-8737-1cf4fdc3c3de tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:01.824007 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721bf50a-9349-4f38-8737-1cf4fdc3c3de tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:01.824764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721bf50a-9349-4f38-8737-1cf4fdc3c3de tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:01.825245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721bf50a-9349-4f38-8737-1cf4fdc3c3de tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:01.825708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-721bf50a-9349-4f38-8737-1cf4fdc3c3de tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:01.962745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-721bf50a-9349-4f38-8737-1cf4fdc3c3de tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:01.971238 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-721bf50a-9349-4f38-8737-1cf4fdc3c3de tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1392 microversion: 2.1 time: 0.163551 Sep 27 10:31:01.971878 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2842/6143] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:01 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1392 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:02.994858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-68713f50-8166-452c-affe-626542aa8e6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:02.995470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-68713f50-8166-452c-affe-626542aa8e6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:03.002639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68713f50-8166-452c-affe-626542aa8e6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:03.003124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68713f50-8166-452c-affe-626542aa8e6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:03.003562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68713f50-8166-452c-affe-626542aa8e6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:03.004259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68713f50-8166-452c-affe-626542aa8e6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:03.004786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68713f50-8166-452c-affe-626542aa8e6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:03.005218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-68713f50-8166-452c-affe-626542aa8e6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:03.147032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-68713f50-8166-452c-affe-626542aa8e6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:03.155606 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-68713f50-8166-452c-affe-626542aa8e6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1392 microversion: 2.1 time: 0.164751 Sep 27 10:31:03.156477 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3302/6144] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:02 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1392 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:04.155646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b926c59c-6c8b-41fd-9b2e-0cb2d1b6a71e req-4bd342a9-4c6a-4010-862a-5b67ce9218e1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0565ee6c-196a-4ca9-981f-24e019cd89ed", "name": "network-vif-plugged", "status": "completed", "tag": "dde5d266-f09e-421c-b470-aacdbb240d15"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:04.164464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:04.165419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:04.175891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b926c59c-6c8b-41fd-9b2e-0cb2d1b6a71e req-4bd342a9-4c6a-4010-862a-5b67ce9218e1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:04.182496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-79f7924b-0f8c-4501-b597-223cc223be4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:04.183919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-79f7924b-0f8c-4501-b597-223cc223be4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:04.186277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:04.187858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b926c59c-6c8b-41fd-9b2e-0cb2d1b6a71e req-4bd342a9-4c6a-4010-862a-5b67ce9218e1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:04.188458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b926c59c-6c8b-41fd-9b2e-0cb2d1b6a71e req-4bd342a9-4c6a-4010-862a-5b67ce9218e1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:04.200286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79f7924b-0f8c-4501-b597-223cc223be4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:04.202543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79f7924b-0f8c-4501-b597-223cc223be4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:04.203134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79f7924b-0f8c-4501-b597-223cc223be4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:04.206232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79f7924b-0f8c-4501-b597-223cc223be4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:04.209016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79f7924b-0f8c-4501-b597-223cc223be4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:04.209542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79f7924b-0f8c-4501-b597-223cc223be4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:04.218457 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b926c59c-6c8b-41fd-9b2e-0cb2d1b6a71e req-4bd342a9-4c6a-4010-862a-5b67ce9218e1 service nova] Creating event network-vif-plugged:dde5d266-f09e-421c-b470-aacdbb240d15 for instance 0565ee6c-196a-4ca9-981f-24e019cd89ed on nch2-master-1 Sep 27 10:31:04.232832 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b926c59c-6c8b-41fd-9b2e-0cb2d1b6a71e req-4bd342a9-4c6a-4010-862a-5b67ce9218e1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081902 Sep 27 10:31:04.233745 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2843/6145] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:31:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:31:04.370610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-79f7924b-0f8c-4501-b597-223cc223be4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:04.382797 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-79f7924b-0f8c-4501-b597-223cc223be4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1392 microversion: 2.1 time: 0.204877 Sep 27 10:31:04.383622 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3303/6146] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:04 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1392 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:31:05.407209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-580ab17e-315c-4ebd-a80e-6dd0538cff6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:05.407955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-580ab17e-315c-4ebd-a80e-6dd0538cff6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:05.415686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-580ab17e-315c-4ebd-a80e-6dd0538cff6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:05.416184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-580ab17e-315c-4ebd-a80e-6dd0538cff6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:05.416639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-580ab17e-315c-4ebd-a80e-6dd0538cff6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:05.417390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-580ab17e-315c-4ebd-a80e-6dd0538cff6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:05.417859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-580ab17e-315c-4ebd-a80e-6dd0538cff6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:05.418299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-580ab17e-315c-4ebd-a80e-6dd0538cff6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:05.573928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-580ab17e-315c-4ebd-a80e-6dd0538cff6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:05.582328 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-580ab17e-315c-4ebd-a80e-6dd0538cff6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1392 microversion: 2.1 time: 0.179527 Sep 27 10:31:05.582986 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2844/6147] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:05 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1392 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:06.609445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3009a29a-c306-4c09-8626-1c18b09e7ea0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:06.609445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3009a29a-c306-4c09-8626-1c18b09e7ea0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:06.615126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3009a29a-c306-4c09-8626-1c18b09e7ea0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:06.615641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3009a29a-c306-4c09-8626-1c18b09e7ea0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:06.616077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3009a29a-c306-4c09-8626-1c18b09e7ea0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:06.616812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3009a29a-c306-4c09-8626-1c18b09e7ea0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:06.617267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3009a29a-c306-4c09-8626-1c18b09e7ea0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:06.617721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3009a29a-c306-4c09-8626-1c18b09e7ea0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:06.760824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3009a29a-c306-4c09-8626-1c18b09e7ea0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:06.771877 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3009a29a-c306-4c09-8626-1c18b09e7ea0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1564 microversion: 2.1 time: 0.170435 Sep 27 10:31:06.772614 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3304/6148] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:06 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1564 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:06.798675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-83a8560a-0957-455b-aa82-dbc740c952d6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:06.799343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-83a8560a-0957-455b-aa82-dbc740c952d6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:06.802342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=68,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:06.807996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83a8560a-0957-455b-aa82-dbc740c952d6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:06.808499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83a8560a-0957-455b-aa82-dbc740c952d6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:06.809047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83a8560a-0957-455b-aa82-dbc740c952d6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:06.809717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83a8560a-0957-455b-aa82-dbc740c952d6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:06.810195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83a8560a-0957-455b-aa82-dbc740c952d6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:06.810624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83a8560a-0957-455b-aa82-dbc740c952d6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:06.955623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-83a8560a-0957-455b-aa82-dbc740c952d6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:06.964098 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-83a8560a-0957-455b-aa82-dbc740c952d6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1564 microversion: 2.1 time: 0.169447 Sep 27 10:31:06.964705 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2845/6149] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:06 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1564 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:08.669122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4d2ed633-a1b7-4cb4-a1d5-73428025fc41 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:08.669824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4d2ed633-a1b7-4cb4-a1d5-73428025fc41 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:08.671739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-815ec793-6474-4f0d-b4a6-9d8460c0c52e req-834993d2-1782-4e57-8c0b-7d0924c535aa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0565ee6c-196a-4ca9-981f-24e019cd89ed", "tag": "dde5d266-f09e-421c-b470-aacdbb240d15"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:08.678161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d2ed633-a1b7-4cb4-a1d5-73428025fc41 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:08.678668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d2ed633-a1b7-4cb4-a1d5-73428025fc41 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:08.679902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d2ed633-a1b7-4cb4-a1d5-73428025fc41 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:08.680284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d2ed633-a1b7-4cb4-a1d5-73428025fc41 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:08.680922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d2ed633-a1b7-4cb4-a1d5-73428025fc41 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:08.681381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4d2ed633-a1b7-4cb4-a1d5-73428025fc41 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:08.684575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-815ec793-6474-4f0d-b4a6-9d8460c0c52e req-834993d2-1782-4e57-8c0b-7d0924c535aa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:08.684858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-815ec793-6474-4f0d-b4a6-9d8460c0c52e req-834993d2-1782-4e57-8c0b-7d0924c535aa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:08.685304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-815ec793-6474-4f0d-b4a6-9d8460c0c52e req-834993d2-1782-4e57-8c0b-7d0924c535aa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:08.697446 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-815ec793-6474-4f0d-b4a6-9d8460c0c52e req-834993d2-1782-4e57-8c0b-7d0924c535aa service nova] Creating event network-changed:dde5d266-f09e-421c-b470-aacdbb240d15 for instance 0565ee6c-196a-4ca9-981f-24e019cd89ed on nch2-master-1 Sep 27 10:31:08.703345 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-815ec793-6474-4f0d-b4a6-9d8460c0c52e req-834993d2-1782-4e57-8c0b-7d0924c535aa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035394 Sep 27 10:31:08.703904 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2846/6150] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:31:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:31:08.828023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4d2ed633-a1b7-4cb4-a1d5-73428025fc41 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:08.836411 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4d2ed633-a1b7-4cb4-a1d5-73428025fc41 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1564 microversion: 2.1 time: 0.171454 Sep 27 10:31:08.840744 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3305/6151] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:08 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1564 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:30.305369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-86804526", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-362079488", "security_groups": [{"name": "tempest-secgroup-smoke-883528541"}], "block_device_mapping_v2": [{"uuid": "c2c6c4b3-ba4e-4994-8674-d39521abb223", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "1e417b59-62bb-4007-a78a-c15c153efd33"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:30.316021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:30.316743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=29 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:30.427637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] validate_networks() for [('1e417b59-62bb-4007-a78a-c15c153efd33', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:31:30.460377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1cc7dea6-264a-40dd-a0c3-5cd5d69a1e42 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:30.461030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1cc7dea6-264a-40dd-a0c3-5cd5d69a1e42 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:30.464899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=59,nova_api:INSERT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:30.465584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=92 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:30.473341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc7dea6-264a-40dd-a0c3-5cd5d69a1e42 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:30.473837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc7dea6-264a-40dd-a0c3-5cd5d69a1e42 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:30.474295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc7dea6-264a-40dd-a0c3-5cd5d69a1e42 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:30.475012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc7dea6-264a-40dd-a0c3-5cd5d69a1e42 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:30.475469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc7dea6-264a-40dd-a0c3-5cd5d69a1e42 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:30.475896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1cc7dea6-264a-40dd-a0c3-5cd5d69a1e42 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:30.501836 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1cc7dea6-264a-40dd-a0c3-5cd5d69a1e42 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/ips" status: 200 len: 150 microversion: 2.1 time: 0.045529 Sep 27 10:31:30.502764 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3306/6152] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:31:30 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/ips => generated 150 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:31:30.835872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:31:30.837332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:31:30.838348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:31:30.839124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:31:30.839967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:31:30.851157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:30.851665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:30.852761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:30.854347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:30.854841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:30.855285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:30.873510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:31:30.878876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Getting quotas for user 178956f6c8154a06a0710b295aff3086 and project 64f8fe93425d4e97b1ae5661172f4846. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:31:30.939998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:31:30.942703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c2c6c4b3-ba4e-4994-8674-d39521abb223',volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:31:31.083362 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-973b0829-4c87-4370-a265-2e1549ef7e54 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.783942 Sep 27 10:31:31.083995 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2847/6153] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:31:30 2022] POST /compute/v2.1/servers => generated 405 bytes in 786 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:31:31.095853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb9a81d3-671b-45a4-9a85-618348525d65 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:31.096486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cb9a81d3-671b-45a4-9a85-618348525d65 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:31.155867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb9a81d3-671b-45a4-9a85-618348525d65 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:31.163173 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cb9a81d3-671b-45a4-9a85-618348525d65 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1092 microversion: 2.1 time: 0.071687 Sep 27 10:31:31.163743 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3307/6154] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:31 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1092 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:31.878786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4f4a8bff-8624-4fb4-b665-2f8273aaf6ea"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:31.882012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:31.890475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:31.890977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:31.891441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:31.892229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:31.892738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:31.893270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:32.186043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3ce3c4d-d556-4715-96dc-4fca408b1860 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:32.186616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3ce3c4d-d556-4715-96dc-4fca408b1860 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:32.195171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3ce3c4d-d556-4715-96dc-4fca408b1860 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:32.195979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3ce3c4d-d556-4715-96dc-4fca408b1860 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:32.196595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3ce3c4d-d556-4715-96dc-4fca408b1860 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:32.197595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3ce3c4d-d556-4715-96dc-4fca408b1860 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:32.198206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3ce3c4d-d556-4715-96dc-4fca408b1860 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:32.198601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3ce3c4d-d556-4715-96dc-4fca408b1860 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:32.323221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b3ce3c4d-d556-4715-96dc-4fca408b1860 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:32.329818 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-603bee2c-b2ad-43ac-853f-b0c67f6ed317 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.456464 Sep 27 10:31:32.335793 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2848/6155] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:31:31 2022] POST /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-volume_attachments => generated 194 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:31:32.336001 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3ce3c4d-d556-4715-96dc-4fca408b1860 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1190 microversion: 2.1 time: 0.151408 Sep 27 10:31:32.336001 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3308/6156] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:32 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1190 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:33.359427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ef331277-5144-4ac7-bac5-5579e0194940 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:33.361170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ef331277-5144-4ac7-bac5-5579e0194940 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:33.371381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef331277-5144-4ac7-bac5-5579e0194940 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:33.372304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef331277-5144-4ac7-bac5-5579e0194940 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:33.373033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef331277-5144-4ac7-bac5-5579e0194940 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:33.374096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef331277-5144-4ac7-bac5-5579e0194940 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:33.374811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef331277-5144-4ac7-bac5-5579e0194940 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:33.375570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef331277-5144-4ac7-bac5-5579e0194940 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:33.570423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ef331277-5144-4ac7-bac5-5579e0194940 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:33.579523 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ef331277-5144-4ac7-bac5-5579e0194940 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1275 microversion: 2.1 time: 0.224617 Sep 27 10:31:33.580735 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2849/6157] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:33 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1275 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:33.854538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-9d35316f-4904-4c49-bf8a-f7002ca7221e req-c7a70e32-80ef-4779-8d6d-ffee413a287c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a43e72b9-7453-4f8c-a959-8a5a38d3e9f3", "tag": "0f2d0535-c751-4347-b21f-ee095b753310"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:33.871635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9d35316f-4904-4c49-bf8a-f7002ca7221e req-c7a70e32-80ef-4779-8d6d-ffee413a287c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:33.872400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9d35316f-4904-4c49-bf8a-f7002ca7221e req-c7a70e32-80ef-4779-8d6d-ffee413a287c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:33.872877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9d35316f-4904-4c49-bf8a-f7002ca7221e req-c7a70e32-80ef-4779-8d6d-ffee413a287c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:33.886780 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-9d35316f-4904-4c49-bf8a-f7002ca7221e req-c7a70e32-80ef-4779-8d6d-ffee413a287c service nova] Creating event network-changed:0f2d0535-c751-4347-b21f-ee095b753310 for instance a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 on nch1-master-1 Sep 27 10:31:33.892975 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-9d35316f-4904-4c49-bf8a-f7002ca7221e req-c7a70e32-80ef-4779-8d6d-ffee413a287c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044480 Sep 27 10:31:33.893901 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3309/6158] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:31:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:31:34.604563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fa25d44c-ca20-4bda-8a8d-c0a8c0dd92ce tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:34.605373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa25d44c-ca20-4bda-8a8d-c0a8c0dd92ce tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:34.613113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa25d44c-ca20-4bda-8a8d-c0a8c0dd92ce tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:34.613620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa25d44c-ca20-4bda-8a8d-c0a8c0dd92ce tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:34.614079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa25d44c-ca20-4bda-8a8d-c0a8c0dd92ce tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:34.614809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa25d44c-ca20-4bda-8a8d-c0a8c0dd92ce tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:34.615290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa25d44c-ca20-4bda-8a8d-c0a8c0dd92ce tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:34.615767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa25d44c-ca20-4bda-8a8d-c0a8c0dd92ce tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:34.762063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fa25d44c-ca20-4bda-8a8d-c0a8c0dd92ce tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:34.770181 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fa25d44c-ca20-4bda-8a8d-c0a8c0dd92ce tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1263 microversion: 2.1 time: 0.169971 Sep 27 10:31:34.770772 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2850/6159] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:34 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1263 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:35.714140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4f4a8bff-8624-4fb4-b665-2f8273aaf6ea"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:35.718218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:35.728100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:35.728676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:35.729927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:35.730726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:35.731268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:35.731633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:35.793814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b951e11e-92c8-4c88-8120-c864cb81c355 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:35.794402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b951e11e-92c8-4c88-8120-c864cb81c355 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:35.802025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b951e11e-92c8-4c88-8120-c864cb81c355 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:35.802690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b951e11e-92c8-4c88-8120-c864cb81c355 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:35.803237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b951e11e-92c8-4c88-8120-c864cb81c355 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:35.803987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b951e11e-92c8-4c88-8120-c864cb81c355 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:35.804455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b951e11e-92c8-4c88-8120-c864cb81c355 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:35.804918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b951e11e-92c8-4c88-8120-c864cb81c355 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:35.816911 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Invalid volume: volume 4f4a8bff-8624-4fb4-b665-2f8273aaf6ea is already attached to instances: 15bd0cb2-4701-4992-9f64-895716df0533 Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-volume_attachments HTTP/1.1 Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 74 Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: {"volumeAttachment": {"volumeId": "4f4a8bff-8624-4fb4-b665-2f8273aaf6ea"}} Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid volume: volume 4f4a8bff-8624-4fb4-b665-2f8273aaf6ea is already attached to instances: 15bd0cb2-4701-4992-9f64-895716df0533 Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'server_id': '15bd0cb2-4701-4992-9f64-895716df0533', 'body': {'volumeAttachment': {'volumeId': '4f4a8bff-8624-4fb4-b665-2f8273aaf6ea'}}} Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 4f4a8bff-8624-4fb4-b665-2f8273aaf6ea is already attached to instances: 15bd0cb2-4701-4992-9f64-895716df0533 Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:31:35.818816 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 4f4a8bff-8624-4fb4-b665-2f8273aaf6ea is already attached to instances: 15bd0cb2-4701-4992-9f64-895716df0533 Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 4f4a8bff-8624-4fb4-b665-2f8273aaf6ea is already attached to instances: 15bd0cb2-4701-4992-9f64-895716df0533 Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 400 to user: Invalid volume: volume 4f4a8bff-8624-4fb4-b665-2f8273aaf6ea is already attached to instances: 15bd0cb2-4701-4992-9f64-895716df0533 {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e076eedb-a561-409f-acfd-a94cf076b02d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.111794 Sep 27 10:31:35.828105 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3310/6160] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:31:35 2022] POST /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-volume_attachments => generated 174 bytes in 116 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:31:35.898441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3c624d48-a4e0-4b84-852b-f0f554689d4a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:35.899239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3c624d48-a4e0-4b84-852b-f0f554689d4a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:35.905820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c624d48-a4e0-4b84-852b-f0f554689d4a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:35.906306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c624d48-a4e0-4b84-852b-f0f554689d4a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:35.906738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c624d48-a4e0-4b84-852b-f0f554689d4a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:35.907436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c624d48-a4e0-4b84-852b-f0f554689d4a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:35.907885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c624d48-a4e0-4b84-852b-f0f554689d4a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:35.908327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c624d48-a4e0-4b84-852b-f0f554689d4a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:35.952443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b951e11e-92c8-4c88-8120-c864cb81c355 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:35.960631 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b951e11e-92c8-4c88-8120-c864cb81c355 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1263 microversion: 2.1 time: 0.171146 Sep 27 10:31:35.961264 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2851/6161] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:35 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1263 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:31:35.997928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-3c624d48-a4e0-4b84-852b-f0f554689d4a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lazy-loading 'services' on Instance uuid 15bd0cb2-4701-4992-9f64-895716df0533 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:31:36.082265 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3c624d48-a4e0-4b84-852b-f0f554689d4a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-volume_attachments/4f4a8bff-8624-4fb4-b665-2f8273aaf6ea" status: 202 len: 0 microversion: 2.1 time: 0.187156 Sep 27 10:31:36.083132 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3311/6162] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:31:35 2022] DELETE /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-volume_attachments/4f4a8bff-8624-4fb4-b665-2f8273aaf6ea => generated 0 bytes in 189 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:31:36.094115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a04167f-af6c-4fda-baa3-0c029bf5fee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:36.096523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a04167f-af6c-4fda-baa3-0c029bf5fee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:36.103218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a04167f-af6c-4fda-baa3-0c029bf5fee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:36.103719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a04167f-af6c-4fda-baa3-0c029bf5fee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:36.104179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a04167f-af6c-4fda-baa3-0c029bf5fee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:36.104949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a04167f-af6c-4fda-baa3-0c029bf5fee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:36.105429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a04167f-af6c-4fda-baa3-0c029bf5fee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:36.105884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a04167f-af6c-4fda-baa3-0c029bf5fee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:36.136686 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a04167f-af6c-4fda-baa3-0c029bf5fee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046993 Sep 27 10:31:36.137303 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2852/6163] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:31:36 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:31:36.986580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b4e7bec7-8b45-41c8-92e6-e262dc5065d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:36.987381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b4e7bec7-8b45-41c8-92e6-e262dc5065d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:37.004457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4e7bec7-8b45-41c8-92e6-e262dc5065d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:37.004996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4e7bec7-8b45-41c8-92e6-e262dc5065d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:37.005428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4e7bec7-8b45-41c8-92e6-e262dc5065d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:37.006114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4e7bec7-8b45-41c8-92e6-e262dc5065d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:37.006802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4e7bec7-8b45-41c8-92e6-e262dc5065d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:37.007233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b4e7bec7-8b45-41c8-92e6-e262dc5065d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:37.149933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-225df52d-5aba-4d34-8e19-cc48ed1de7eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:37.152205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-225df52d-5aba-4d34-8e19-cc48ed1de7eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:37.157378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b4e7bec7-8b45-41c8-92e6-e262dc5065d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:37.160198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-225df52d-5aba-4d34-8e19-cc48ed1de7eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:37.160734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-225df52d-5aba-4d34-8e19-cc48ed1de7eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:37.161383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-225df52d-5aba-4d34-8e19-cc48ed1de7eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:37.162205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-225df52d-5aba-4d34-8e19-cc48ed1de7eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:37.162684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-225df52d-5aba-4d34-8e19-cc48ed1de7eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:37.163130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-225df52d-5aba-4d34-8e19-cc48ed1de7eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:37.165842 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b4e7bec7-8b45-41c8-92e6-e262dc5065d9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1263 microversion: 2.1 time: 0.184366 Sep 27 10:31:37.166584 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3312/6164] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:36 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1263 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:37.199787 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-225df52d-5aba-4d34-8e19-cc48ed1de7eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053955 Sep 27 10:31:37.200369 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2853/6165] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:31:37 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 27 10:31:37.267565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-42d1577a-8db5-457b-91dc-afda6fa9a3bf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:37.268068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-42d1577a-8db5-457b-91dc-afda6fa9a3bf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:37.274317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42d1577a-8db5-457b-91dc-afda6fa9a3bf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:37.275289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42d1577a-8db5-457b-91dc-afda6fa9a3bf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:37.275969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42d1577a-8db5-457b-91dc-afda6fa9a3bf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:37.276407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42d1577a-8db5-457b-91dc-afda6fa9a3bf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:37.276926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42d1577a-8db5-457b-91dc-afda6fa9a3bf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:37.277353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42d1577a-8db5-457b-91dc-afda6fa9a3bf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:37.303455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-42d1577a-8db5-457b-91dc-afda6fa9a3bf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:31:37.388988 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-42d1577a-8db5-457b-91dc-afda6fa9a3bf tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 204 len: 0 microversion: 2.1 time: 0.125071 Sep 27 10:31:37.389657 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3313/6166] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:31:37 2022] DELETE /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:31:37.398288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f63966a5-da0c-4ee8-8023-43a43fd04554 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:37.398987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f63966a5-da0c-4ee8-8023-43a43fd04554 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:37.405320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63966a5-da0c-4ee8-8023-43a43fd04554 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:37.405821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63966a5-da0c-4ee8-8023-43a43fd04554 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:37.406300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63966a5-da0c-4ee8-8023-43a43fd04554 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:37.407019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63966a5-da0c-4ee8-8023-43a43fd04554 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:37.407487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63966a5-da0c-4ee8-8023-43a43fd04554 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:37.408201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63966a5-da0c-4ee8-8023-43a43fd04554 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:37.654717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f63966a5-da0c-4ee8-8023-43a43fd04554 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:37.666856 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f63966a5-da0c-4ee8-8023-43a43fd04554 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1681 microversion: 2.1 time: 0.272537 Sep 27 10:31:37.668120 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2854/6167] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:37 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1681 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:38.189806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a1303a35-7040-461a-9b53-b509d2c0143f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:38.190470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a1303a35-7040-461a-9b53-b509d2c0143f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:38.197695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1303a35-7040-461a-9b53-b509d2c0143f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:38.198184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1303a35-7040-461a-9b53-b509d2c0143f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:38.198614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1303a35-7040-461a-9b53-b509d2c0143f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:38.199307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1303a35-7040-461a-9b53-b509d2c0143f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:38.199757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1303a35-7040-461a-9b53-b509d2c0143f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:38.200181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1303a35-7040-461a-9b53-b509d2c0143f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:38.339568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a1303a35-7040-461a-9b53-b509d2c0143f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:38.348687 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a1303a35-7040-461a-9b53-b509d2c0143f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1263 microversion: 2.1 time: 0.163218 Sep 27 10:31:38.349291 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3314/6168] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:38 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1263 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:38.696159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-948cc826-25c8-452e-b7e5-b78e0734e958 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:38.696853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-948cc826-25c8-452e-b7e5-b78e0734e958 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:38.704907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-948cc826-25c8-452e-b7e5-b78e0734e958 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:38.705414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-948cc826-25c8-452e-b7e5-b78e0734e958 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:38.705870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-948cc826-25c8-452e-b7e5-b78e0734e958 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:38.706600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-948cc826-25c8-452e-b7e5-b78e0734e958 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:38.707069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-948cc826-25c8-452e-b7e5-b78e0734e958 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:38.707512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-948cc826-25c8-452e-b7e5-b78e0734e958 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:38.851844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-948cc826-25c8-452e-b7e5-b78e0734e958 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:38.860547 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-948cc826-25c8-452e-b7e5-b78e0734e958 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1681 microversion: 2.1 time: 0.168364 Sep 27 10:31:38.861225 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2855/6169] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:38 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1681 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:39.371821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d6f9de44-4583-478e-9cd6-c5ec8f9644ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:39.372406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d6f9de44-4583-478e-9cd6-c5ec8f9644ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:39.379271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6f9de44-4583-478e-9cd6-c5ec8f9644ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:39.379760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6f9de44-4583-478e-9cd6-c5ec8f9644ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:39.380200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6f9de44-4583-478e-9cd6-c5ec8f9644ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:39.380964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6f9de44-4583-478e-9cd6-c5ec8f9644ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:39.381420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6f9de44-4583-478e-9cd6-c5ec8f9644ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:39.381849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d6f9de44-4583-478e-9cd6-c5ec8f9644ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:39.637632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d6f9de44-4583-478e-9cd6-c5ec8f9644ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:39.646382 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d6f9de44-4583-478e-9cd6-c5ec8f9644ea tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1263 microversion: 2.1 time: 0.278625 Sep 27 10:31:39.647177 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3315/6170] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:39 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1263 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:39.883884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cf2c3465-2875-41d1-9056-01f48b1cf38d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:39.884521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cf2c3465-2875-41d1-9056-01f48b1cf38d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:39.892619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf2c3465-2875-41d1-9056-01f48b1cf38d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:39.893157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf2c3465-2875-41d1-9056-01f48b1cf38d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:39.893638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf2c3465-2875-41d1-9056-01f48b1cf38d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:39.894353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf2c3465-2875-41d1-9056-01f48b1cf38d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:39.894823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf2c3465-2875-41d1-9056-01f48b1cf38d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:39.895268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf2c3465-2875-41d1-9056-01f48b1cf38d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:40.035277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cf2c3465-2875-41d1-9056-01f48b1cf38d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:40.043879 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cf2c3465-2875-41d1-9056-01f48b1cf38d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 200 len: 1681 microversion: 2.1 time: 0.164007 Sep 27 10:31:40.044460 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2856/6171] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:39 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 1681 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:40.352307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0e08d0ce-5ccf-4c40-93fa-6014c55e0b9a req-c1a0bf5f-a01c-4526-8010-ca3ee42eb7fe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "15bd0cb2-4701-4992-9f64-895716df0533", "name": "network-vif-deleted", "tag": "cccf9085-27fb-4577-ad7e-7d726fb318a7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:40.362138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0e08d0ce-5ccf-4c40-93fa-6014c55e0b9a req-c1a0bf5f-a01c-4526-8010-ca3ee42eb7fe service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:40.362623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0e08d0ce-5ccf-4c40-93fa-6014c55e0b9a req-c1a0bf5f-a01c-4526-8010-ca3ee42eb7fe service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:40.363093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0e08d0ce-5ccf-4c40-93fa-6014c55e0b9a req-c1a0bf5f-a01c-4526-8010-ca3ee42eb7fe service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:40.375569 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0e08d0ce-5ccf-4c40-93fa-6014c55e0b9a req-c1a0bf5f-a01c-4526-8010-ca3ee42eb7fe service nova] Creating event network-vif-deleted:cccf9085-27fb-4577-ad7e-7d726fb318a7 for instance 15bd0cb2-4701-4992-9f64-895716df0533 on nch1-master-1 Sep 27 10:31:40.382287 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0e08d0ce-5ccf-4c40-93fa-6014c55e0b9a req-c1a0bf5f-a01c-4526-8010-ca3ee42eb7fe service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033659 Sep 27 10:31:40.382910 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3316/6172] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:31:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:31:40.670694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d5cb5122-30d3-4b66-a089-225ee49a6569 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:40.671662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d5cb5122-30d3-4b66-a089-225ee49a6569 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:40.677896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5cb5122-30d3-4b66-a089-225ee49a6569 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:40.678352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5cb5122-30d3-4b66-a089-225ee49a6569 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:40.678810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5cb5122-30d3-4b66-a089-225ee49a6569 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:40.679729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5cb5122-30d3-4b66-a089-225ee49a6569 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:40.680014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5cb5122-30d3-4b66-a089-225ee49a6569 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:40.680504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5cb5122-30d3-4b66-a089-225ee49a6569 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:40.824837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d5cb5122-30d3-4b66-a089-225ee49a6569 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:40.833303 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d5cb5122-30d3-4b66-a089-225ee49a6569 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1263 microversion: 2.1 time: 0.166747 Sep 27 10:31:40.833996 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2857/6173] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:40 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1263 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:41.068179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:41.069075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:41.076062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:41.076682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:41.077227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:41.078022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:41.078628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:41.079106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:41.095777 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 HTTP/1.1 Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '15bd0cb2-4701-4992-9f64-895716df0533'} Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:31:41.097558 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:31:41.103178 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:31:41.103178 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:31:41.103178 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:31:41.103178 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:31:41.103178 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:31:41.103178 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:31:41.103178 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:31:41.103178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:31:41.103178 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-22f3568a-4904-42a0-88ec-c65c11038b4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 404 len: 111 microversion: 2.1 time: 0.035387 Sep 27 10:31:41.103178 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3317/6174] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:41 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:31:41.109012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25ac9d8b-e10c-428c-acfa-8698c181b18c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '.version_select at 0x7f7f625c80d0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:41.114268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:41.115249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:41.136758 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-25ac9d8b-e10c-428c-acfa-8698c181b18c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1709046796" status: 202 len: 0 microversion: 2.1 time: 0.032202 Sep 27 10:31:41.137450 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2858/6175] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:31:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1709046796 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:31:41.261776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-1e61b0ff-645d-4997-ad5d-a2e7bca42d5a req-81a4afec-f558-4673-8aa7-1eee1cacdbd1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a43e72b9-7453-4f8c-a959-8a5a38d3e9f3", "name": "network-vif-plugged", "status": "completed", "tag": "0f2d0535-c751-4347-b21f-ee095b753310"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:41.268617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:41.276389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1e61b0ff-645d-4997-ad5d-a2e7bca42d5a req-81a4afec-f558-4673-8aa7-1eee1cacdbd1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:41.277793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1e61b0ff-645d-4997-ad5d-a2e7bca42d5a req-81a4afec-f558-4673-8aa7-1eee1cacdbd1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:41.278490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1e61b0ff-645d-4997-ad5d-a2e7bca42d5a req-81a4afec-f558-4673-8aa7-1eee1cacdbd1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:41.303615 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-1e61b0ff-645d-4997-ad5d-a2e7bca42d5a req-81a4afec-f558-4673-8aa7-1eee1cacdbd1 service nova] Creating event network-vif-plugged:0f2d0535-c751-4347-b21f-ee095b753310 for instance a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 on nch1-master-1 Sep 27 10:31:41.312073 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-1e61b0ff-645d-4997-ad5d-a2e7bca42d5a req-81a4afec-f558-4673-8aa7-1eee1cacdbd1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053965 Sep 27 10:31:41.313871 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3318/6176] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:31:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:31:41.856903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c0fdb187-49c2-4077-ae07-278a7ec43eef tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:41.857536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c0fdb187-49c2-4077-ae07-278a7ec43eef tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:41.864785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0fdb187-49c2-4077-ae07-278a7ec43eef tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:41.865301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0fdb187-49c2-4077-ae07-278a7ec43eef tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:41.865742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0fdb187-49c2-4077-ae07-278a7ec43eef tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:41.866465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0fdb187-49c2-4077-ae07-278a7ec43eef tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:41.866939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0fdb187-49c2-4077-ae07-278a7ec43eef tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:41.867378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0fdb187-49c2-4077-ae07-278a7ec43eef tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:41.893446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cf38109c-f668-4e31-a1aa-7b68c661e71b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: .version_select at 0x7f7f63763940>, body: {"keypair": {"name": "tempest-keypair-1852512552", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBD41IrsVhDF9R5fq76wjJSiFwARrYQEZRZX+ZPl6wMFxFrpfquFsBlveP13W9ELXN0RLT6/9WOOy5VKYeL4iobV4eonuNXaIfEICy5PXJgPN/ppPVruJhho5FKvDk7B6g=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:41.911230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-cf38109c-f668-4e31-a1aa-7b68c661e71b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for project 90c0fbb4a6d84767b920564584715f9d. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:31:41.917687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-cf38109c-f668-4e31-a1aa-7b68c661e71b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for user 64090c8fd3394980ad5409adcde4986f and project 90c0fbb4a6d84767b920564584715f9d. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:31:41.942918 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cf38109c-f668-4e31-a1aa-7b68c661e71b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.053847 Sep 27 10:31:41.943538 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3319/6177] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:31:41 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:31:42.041607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c0fdb187-49c2-4077-ae07-278a7ec43eef tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:42.049913 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c0fdb187-49c2-4077-ae07-278a7ec43eef tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1263 microversion: 2.1 time: 0.197348 Sep 27 10:31:42.050479 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2859/6178] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:41 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1263 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:43.075119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f78f1a7e-084e-4ec7-bb4c-acd216c39d94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:43.076380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f78f1a7e-084e-4ec7-bb4c-acd216c39d94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:43.083730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f78f1a7e-084e-4ec7-bb4c-acd216c39d94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:43.084349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f78f1a7e-084e-4ec7-bb4c-acd216c39d94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:43.084841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f78f1a7e-084e-4ec7-bb4c-acd216c39d94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:43.085561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f78f1a7e-084e-4ec7-bb4c-acd216c39d94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:43.086004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f78f1a7e-084e-4ec7-bb4c-acd216c39d94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:43.086622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f78f1a7e-084e-4ec7-bb4c-acd216c39d94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:43.284878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f78f1a7e-084e-4ec7-bb4c-acd216c39d94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:43.293338 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f78f1a7e-084e-4ec7-bb4c-acd216c39d94 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1450 microversion: 2.1 time: 0.223569 Sep 27 10:31:43.293937 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3320/6179] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:43 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1450 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:31:43.314518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1827520593", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "6a85e0d6-e759-43a8-9abb-8e12abcca6c4"}], "security_groups": [{"name": "tempest-securitygroup--59257037"}], "key_name": "tempest-keypair-1852512552", "user_data": "}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:43.316803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8924958d-d9cb-4abe-b2b8-66f93115060f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:43.317520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8924958d-d9cb-4abe-b2b8-66f93115060f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:43.324272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8924958d-d9cb-4abe-b2b8-66f93115060f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:43.325882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8924958d-d9cb-4abe-b2b8-66f93115060f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:43.326328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8924958d-d9cb-4abe-b2b8-66f93115060f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:43.327501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8924958d-d9cb-4abe-b2b8-66f93115060f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:43.327961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8924958d-d9cb-4abe-b2b8-66f93115060f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:43.328390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8924958d-d9cb-4abe-b2b8-66f93115060f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:43.425288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] validate_networks() for [('6a85e0d6-e759-43a8-9abb-8e12abcca6c4', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:31:43.478468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8924958d-d9cb-4abe-b2b8-66f93115060f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:43.487297 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8924958d-d9cb-4abe-b2b8-66f93115060f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1450 microversion: 2.1 time: 0.174739 Sep 27 10:31:43.487878 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3321/6180] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:43 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1450 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:43.857039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:31:43.857484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:31:43.858700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:31:43.859192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:31:43.859625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:31:43.868615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:43.869164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:43.869596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:43.870337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:43.870946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:43.871375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:43.888909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for project 90c0fbb4a6d84767b920564584715f9d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:31:43.894226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Getting quotas for user 64090c8fd3394980ad5409adcde4986f and project 90c0fbb4a6d84767b920564584715f9d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:31:43.913711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:31:43.915799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:31:43.964704 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e5e4ee65-c20e-496d-bff5-691d4bd1b255 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.654563 Sep 27 10:31:43.965551 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2860/6181] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:31:43 2022] POST /compute/v2.1/servers => generated 400 bytes in 656 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:31:43.977008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b5859ccc-6726-41e4-9651-6686abf37722 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:43.977626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b5859ccc-6726-41e4-9651-6686abf37722 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:44.065187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b5859ccc-6726-41e4-9651-6686abf37722 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:44.074370 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b5859ccc-6726-41e4-9651-6686abf37722 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1242 microversion: 2.1 time: 0.101215 Sep 27 10:31:44.075581 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3322/6182] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:43 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1242 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:44.178690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6887cac0-b20c-4305-a9ae-e0311e34d03f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:44.181325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6887cac0-b20c-4305-a9ae-e0311e34d03f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:44.188144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6887cac0-b20c-4305-a9ae-e0311e34d03f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:44.189066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6887cac0-b20c-4305-a9ae-e0311e34d03f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:44.189752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6887cac0-b20c-4305-a9ae-e0311e34d03f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:44.190686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6887cac0-b20c-4305-a9ae-e0311e34d03f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:44.191168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6887cac0-b20c-4305-a9ae-e0311e34d03f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:44.191790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6887cac0-b20c-4305-a9ae-e0311e34d03f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:44.263809 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6887cac0-b20c-4305-a9ae-e0311e34d03f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed/action" status: 200 len: 24562 microversion: 2.1 time: 0.088824 Sep 27 10:31:44.264500 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2861/6183] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:31:44 2022] POST /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed/action => generated 24562 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:31:45.098709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d0f88715-e425-4328-8131-366611d4ce12 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:45.100909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d0f88715-e425-4328-8131-366611d4ce12 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:45.109159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f88715-e425-4328-8131-366611d4ce12 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:45.109945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f88715-e425-4328-8131-366611d4ce12 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:45.110508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f88715-e425-4328-8131-366611d4ce12 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:45.111267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f88715-e425-4328-8131-366611d4ce12 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:45.111736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f88715-e425-4328-8131-366611d4ce12 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:45.112159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d0f88715-e425-4328-8131-366611d4ce12 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:45.185167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d0f88715-e425-4328-8131-366611d4ce12 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:45.198275 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d0f88715-e425-4328-8131-366611d4ce12 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1294 microversion: 2.1 time: 0.103350 Sep 27 10:31:45.221562 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3323/6184] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:45 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1294 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:45.294055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b60cf8b7-947f-4968-be7b-83f895ed1b86 req-8429992d-95a1-43ad-9493-54608ab4e8b5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a43e72b9-7453-4f8c-a959-8a5a38d3e9f3", "tag": "0f2d0535-c751-4347-b21f-ee095b753310"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:45.307202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b60cf8b7-947f-4968-be7b-83f895ed1b86 req-8429992d-95a1-43ad-9493-54608ab4e8b5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:45.307983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b60cf8b7-947f-4968-be7b-83f895ed1b86 req-8429992d-95a1-43ad-9493-54608ab4e8b5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:45.309041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b60cf8b7-947f-4968-be7b-83f895ed1b86 req-8429992d-95a1-43ad-9493-54608ab4e8b5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:45.325678 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b60cf8b7-947f-4968-be7b-83f895ed1b86 req-8429992d-95a1-43ad-9493-54608ab4e8b5 service nova] Creating event network-changed:0f2d0535-c751-4347-b21f-ee095b753310 for instance a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 on nch1-master-1 Sep 27 10:31:45.332544 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b60cf8b7-947f-4968-be7b-83f895ed1b86 req-8429992d-95a1-43ad-9493-54608ab4e8b5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041565 Sep 27 10:31:45.336749 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2862/6185] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:31:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:31:46.229860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-087538bd-93ae-4f55-ab3d-6b51c60d648b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:46.230645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-087538bd-93ae-4f55-ab3d-6b51c60d648b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:46.237611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087538bd-93ae-4f55-ab3d-6b51c60d648b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:46.238097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087538bd-93ae-4f55-ab3d-6b51c60d648b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:46.241682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087538bd-93ae-4f55-ab3d-6b51c60d648b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:46.242450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087538bd-93ae-4f55-ab3d-6b51c60d648b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:46.242925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087538bd-93ae-4f55-ab3d-6b51c60d648b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:46.243359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-087538bd-93ae-4f55-ab3d-6b51c60d648b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:46.410276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-087538bd-93ae-4f55-ab3d-6b51c60d648b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:46.418850 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-087538bd-93ae-4f55-ab3d-6b51c60d648b tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1366 microversion: 2.1 time: 0.193559 Sep 27 10:31:46.419548 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3324/6186] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:46 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1366 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:46.853848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-77660891-4ef3-4eeb-bd9b-c9e65a2b3ded req-48b484f8-4701-4486-a199-4c17c0ddf84f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e73482b8-b537-4e85-9442-e35475931895", "tag": "fe5983cb-efc0-4737-88bd-44a4e18e8c6f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:46.865665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-77660891-4ef3-4eeb-bd9b-c9e65a2b3ded req-48b484f8-4701-4486-a199-4c17c0ddf84f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:46.866341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-77660891-4ef3-4eeb-bd9b-c9e65a2b3ded req-48b484f8-4701-4486-a199-4c17c0ddf84f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:46.866969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-77660891-4ef3-4eeb-bd9b-c9e65a2b3ded req-48b484f8-4701-4486-a199-4c17c0ddf84f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:46.880463 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-77660891-4ef3-4eeb-bd9b-c9e65a2b3ded req-48b484f8-4701-4486-a199-4c17c0ddf84f service nova] Creating event network-changed:fe5983cb-efc0-4737-88bd-44a4e18e8c6f for instance e73482b8-b537-4e85-9442-e35475931895 on nch2-master-1 Sep 27 10:31:46.885726 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-77660891-4ef3-4eeb-bd9b-c9e65a2b3ded req-48b484f8-4701-4486-a199-4c17c0ddf84f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037255 Sep 27 10:31:46.886542 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2863/6187] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:31:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:31:47.443469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-947a8f93-b1f1-4b8c-99bb-8fd43d793851 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:47.444054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-947a8f93-b1f1-4b8c-99bb-8fd43d793851 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:47.452334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-947a8f93-b1f1-4b8c-99bb-8fd43d793851 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:47.452867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-947a8f93-b1f1-4b8c-99bb-8fd43d793851 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:47.453309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-947a8f93-b1f1-4b8c-99bb-8fd43d793851 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:47.454025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-947a8f93-b1f1-4b8c-99bb-8fd43d793851 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:47.454491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-947a8f93-b1f1-4b8c-99bb-8fd43d793851 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:47.454920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-947a8f93-b1f1-4b8c-99bb-8fd43d793851 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:47.611348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-947a8f93-b1f1-4b8c-99bb-8fd43d793851 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:47.619957 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-947a8f93-b1f1-4b8c-99bb-8fd43d793851 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1366 microversion: 2.1 time: 0.181063 Sep 27 10:31:47.620537 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3325/6188] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:47 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1366 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:48.654907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-356a5f7f-08a4-456d-9454-f3c7ed171d1c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:48.655622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-356a5f7f-08a4-456d-9454-f3c7ed171d1c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:48.662569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-356a5f7f-08a4-456d-9454-f3c7ed171d1c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:48.663075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-356a5f7f-08a4-456d-9454-f3c7ed171d1c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:48.663536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-356a5f7f-08a4-456d-9454-f3c7ed171d1c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:48.664258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-356a5f7f-08a4-456d-9454-f3c7ed171d1c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:48.664766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-356a5f7f-08a4-456d-9454-f3c7ed171d1c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:48.665213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-356a5f7f-08a4-456d-9454-f3c7ed171d1c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:48.825255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-356a5f7f-08a4-456d-9454-f3c7ed171d1c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:48.833729 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-356a5f7f-08a4-456d-9454-f3c7ed171d1c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1366 microversion: 2.1 time: 0.184337 Sep 27 10:31:48.834317 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2864/6189] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:48 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1366 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:49.861444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-475f1b68-7ff0-4c51-9fc0-da8de0c769e6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:49.862277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-475f1b68-7ff0-4c51-9fc0-da8de0c769e6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:49.869052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475f1b68-7ff0-4c51-9fc0-da8de0c769e6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:49.869578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475f1b68-7ff0-4c51-9fc0-da8de0c769e6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:49.870043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475f1b68-7ff0-4c51-9fc0-da8de0c769e6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:49.870802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475f1b68-7ff0-4c51-9fc0-da8de0c769e6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:49.871496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475f1b68-7ff0-4c51-9fc0-da8de0c769e6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:49.871775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-475f1b68-7ff0-4c51-9fc0-da8de0c769e6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:49.876369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:50.017424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-475f1b68-7ff0-4c51-9fc0-da8de0c769e6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:50.025426 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-475f1b68-7ff0-4c51-9fc0-da8de0c769e6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1366 microversion: 2.1 time: 0.170181 Sep 27 10:31:50.026087 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3326/6190] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:49 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1366 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:51.049385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-78d85f3f-18a3-4ee2-a13c-ba7d26f7ce4c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:51.050101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-78d85f3f-18a3-4ee2-a13c-ba7d26f7ce4c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:51.057623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d85f3f-18a3-4ee2-a13c-ba7d26f7ce4c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:51.058132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d85f3f-18a3-4ee2-a13c-ba7d26f7ce4c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:51.058632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d85f3f-18a3-4ee2-a13c-ba7d26f7ce4c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:51.059430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d85f3f-18a3-4ee2-a13c-ba7d26f7ce4c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:51.059918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d85f3f-18a3-4ee2-a13c-ba7d26f7ce4c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:51.060385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78d85f3f-18a3-4ee2-a13c-ba7d26f7ce4c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:51.213154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-78d85f3f-18a3-4ee2-a13c-ba7d26f7ce4c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:51.222480 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-78d85f3f-18a3-4ee2-a13c-ba7d26f7ce4c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1366 microversion: 2.1 time: 0.177346 Sep 27 10:31:51.223102 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2865/6191] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:51 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1366 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:52.247628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-33653052-8dd4-4d2f-9af7-5bc18a3d943d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:52.248283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-33653052-8dd4-4d2f-9af7-5bc18a3d943d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:52.256199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33653052-8dd4-4d2f-9af7-5bc18a3d943d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:52.256682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33653052-8dd4-4d2f-9af7-5bc18a3d943d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:52.257142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33653052-8dd4-4d2f-9af7-5bc18a3d943d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:52.257848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33653052-8dd4-4d2f-9af7-5bc18a3d943d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:52.258302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33653052-8dd4-4d2f-9af7-5bc18a3d943d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:52.258732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-33653052-8dd4-4d2f-9af7-5bc18a3d943d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:52.397471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-33653052-8dd4-4d2f-9af7-5bc18a3d943d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:52.405619 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-33653052-8dd4-4d2f-9af7-5bc18a3d943d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1366 microversion: 2.1 time: 0.162012 Sep 27 10:31:52.406212 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3327/6192] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:52 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:53.429283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1482a90d-4107-45db-86b6-514c8e850246 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:53.430052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1482a90d-4107-45db-86b6-514c8e850246 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:53.437310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1482a90d-4107-45db-86b6-514c8e850246 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:53.437815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1482a90d-4107-45db-86b6-514c8e850246 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:53.438254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1482a90d-4107-45db-86b6-514c8e850246 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:53.439008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1482a90d-4107-45db-86b6-514c8e850246 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:53.439451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1482a90d-4107-45db-86b6-514c8e850246 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:53.440896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1482a90d-4107-45db-86b6-514c8e850246 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:53.444413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:53.588800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1482a90d-4107-45db-86b6-514c8e850246 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:53.599164 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1482a90d-4107-45db-86b6-514c8e850246 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1366 microversion: 2.1 time: 0.173940 Sep 27 10:31:53.599813 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2866/6193] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:53 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:54.358279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-56d14cd3-f980-427b-914b-447bfdc76c04 req-753e5323-5ff0-4fae-acf4-e2080b0ce8f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e73482b8-b537-4e85-9442-e35475931895", "name": "network-vif-plugged", "status": "completed", "tag": "fe5983cb-efc0-4737-88bd-44a4e18e8c6f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:54.365316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:54.373748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-56d14cd3-f980-427b-914b-447bfdc76c04 req-753e5323-5ff0-4fae-acf4-e2080b0ce8f7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:54.374734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-56d14cd3-f980-427b-914b-447bfdc76c04 req-753e5323-5ff0-4fae-acf4-e2080b0ce8f7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:54.375497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-56d14cd3-f980-427b-914b-447bfdc76c04 req-753e5323-5ff0-4fae-acf4-e2080b0ce8f7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:54.392806 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-56d14cd3-f980-427b-914b-447bfdc76c04 req-753e5323-5ff0-4fae-acf4-e2080b0ce8f7 service nova] Creating event network-vif-plugged:fe5983cb-efc0-4737-88bd-44a4e18e8c6f for instance e73482b8-b537-4e85-9442-e35475931895 on nch2-master-1 Sep 27 10:31:54.399372 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-56d14cd3-f980-427b-914b-447bfdc76c04 req-753e5323-5ff0-4fae-acf4-e2080b0ce8f7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045250 Sep 27 10:31:54.400041 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3328/6194] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:31:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:31:54.627434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5384ad55-fa77-4a71-9a4b-d1a6436c20e5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:54.628717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5384ad55-fa77-4a71-9a4b-d1a6436c20e5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:54.630699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:54.631402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:31:54.639298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5384ad55-fa77-4a71-9a4b-d1a6436c20e5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:54.639824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5384ad55-fa77-4a71-9a4b-d1a6436c20e5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:54.640303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5384ad55-fa77-4a71-9a4b-d1a6436c20e5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:54.641062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5384ad55-fa77-4a71-9a4b-d1a6436c20e5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:54.641536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5384ad55-fa77-4a71-9a4b-d1a6436c20e5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:54.641985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5384ad55-fa77-4a71-9a4b-d1a6436c20e5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:54.780377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5384ad55-fa77-4a71-9a4b-d1a6436c20e5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:54.788555 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5384ad55-fa77-4a71-9a4b-d1a6436c20e5 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1366 microversion: 2.1 time: 0.165326 Sep 27 10:31:54.789165 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2867/6195] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:54 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1366 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:55.815398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-970b068b-0857-4c05-b971-bb3a65fd8e89 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:55.816093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-970b068b-0857-4c05-b971-bb3a65fd8e89 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:55.823061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970b068b-0857-4c05-b971-bb3a65fd8e89 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:55.823550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970b068b-0857-4c05-b971-bb3a65fd8e89 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:55.823992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970b068b-0857-4c05-b971-bb3a65fd8e89 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:55.824698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970b068b-0857-4c05-b971-bb3a65fd8e89 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:55.825400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970b068b-0857-4c05-b971-bb3a65fd8e89 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:55.825644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-970b068b-0857-4c05-b971-bb3a65fd8e89 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:55.966730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-970b068b-0857-4c05-b971-bb3a65fd8e89 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:55.975487 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-970b068b-0857-4c05-b971-bb3a65fd8e89 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1366 microversion: 2.1 time: 0.165408 Sep 27 10:31:55.976076 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3329/6196] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:55 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1366 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:56.999228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-55bce3f0-8760-4dc3-a982-8bbe8ce95715 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:56.999870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-55bce3f0-8760-4dc3-a982-8bbe8ce95715 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:57.007545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55bce3f0-8760-4dc3-a982-8bbe8ce95715 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:57.008060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55bce3f0-8760-4dc3-a982-8bbe8ce95715 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:57.008524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55bce3f0-8760-4dc3-a982-8bbe8ce95715 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:57.009287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55bce3f0-8760-4dc3-a982-8bbe8ce95715 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:57.009769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55bce3f0-8760-4dc3-a982-8bbe8ce95715 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:57.010218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55bce3f0-8760-4dc3-a982-8bbe8ce95715 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:57.149369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-55bce3f0-8760-4dc3-a982-8bbe8ce95715 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:31:57.162287 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-55bce3f0-8760-4dc3-a982-8bbe8ce95715 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1557 microversion: 2.1 time: 0.167083 Sep 27 10:31:57.163201 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2868/6197] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:31:56 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1557 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:31:57.190258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-97bd6c70-113b-42a4-86db-59fe9309b2fc tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:31:57.190818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-97bd6c70-113b-42a4-86db-59fe9309b2fc tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:31:57.200371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bd6c70-113b-42a4-86db-59fe9309b2fc tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:57.201067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bd6c70-113b-42a4-86db-59fe9309b2fc tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:57.201680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bd6c70-113b-42a4-86db-59fe9309b2fc tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:57.202551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bd6c70-113b-42a4-86db-59fe9309b2fc tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:57.203004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bd6c70-113b-42a4-86db-59fe9309b2fc tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:57.203434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-97bd6c70-113b-42a4-86db-59fe9309b2fc tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:57.282772 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-97bd6c70-113b-42a4-86db-59fe9309b2fc tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.098397 Sep 27 10:31:57.283385 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3330/6198] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:31:57 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/os-interface => generated 285 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:31:58.501816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f63c52fd-b2ff-45e0-a61f-2552a0fb3a3b req-215c8daa-6b47-4bf8-b613-17b7ade1400b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e73482b8-b537-4e85-9442-e35475931895", "tag": "fe5983cb-efc0-4737-88bd-44a4e18e8c6f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:31:58.512806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f63c52fd-b2ff-45e0-a61f-2552a0fb3a3b req-215c8daa-6b47-4bf8-b613-17b7ade1400b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:31:58.513321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f63c52fd-b2ff-45e0-a61f-2552a0fb3a3b req-215c8daa-6b47-4bf8-b613-17b7ade1400b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:31:58.513771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f63c52fd-b2ff-45e0-a61f-2552a0fb3a3b req-215c8daa-6b47-4bf8-b613-17b7ade1400b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:31:58.525442 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f63c52fd-b2ff-45e0-a61f-2552a0fb3a3b req-215c8daa-6b47-4bf8-b613-17b7ade1400b service nova] Creating event network-changed:fe5983cb-efc0-4737-88bd-44a4e18e8c6f for instance e73482b8-b537-4e85-9442-e35475931895 on nch2-master-1 Sep 27 10:31:58.531244 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f63c52fd-b2ff-45e0-a61f-2552a0fb3a3b req-215c8daa-6b47-4bf8-b613-17b7ade1400b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033158 Sep 27 10:31:58.532275 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2869/6199] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:31:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:32:03.830293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:03.833522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:03.835561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=69,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:03.841333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:03.841803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:03.842235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:03.843252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:03.843709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:03.844132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:04.142437 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58b32c9e-5a04-4f4e-b6aa-09d80c7caee8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.319673 Sep 27 10:32:04.143056 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3331/6200] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:32:03 2022] POST /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-volume_attachments => generated 194 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:32:07.469752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3a2ceb28-c639-44c6-aa4e-1048992ed8ec tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:07.472123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3a2ceb28-c639-44c6-aa4e-1048992ed8ec tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:07.474307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=74,nova_api:INSERT=6,nova_api:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:07.479911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a2ceb28-c639-44c6-aa4e-1048992ed8ec tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:07.480469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a2ceb28-c639-44c6-aa4e-1048992ed8ec tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:07.481296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a2ceb28-c639-44c6-aa4e-1048992ed8ec tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:07.482030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a2ceb28-c639-44c6-aa4e-1048992ed8ec tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:07.482504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a2ceb28-c639-44c6-aa4e-1048992ed8ec tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:07.482945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a2ceb28-c639-44c6-aa4e-1048992ed8ec tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:07.510054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-3a2ceb28-c639-44c6-aa4e-1048992ed8ec tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lazy-loading 'flavor' on Instance uuid dc9ddb88-bfb7-40d7-b615-9755d1e824d5 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:32:07.656292 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3a2ceb28-c639-44c6-aa4e-1048992ed8ec tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/action" status: 200 len: 32 microversion: 2.1 time: 0.193243 Sep 27 10:32:07.656986 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2870/6201] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:32:07 2022] POST /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/action => generated 32 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:32:07.671629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa4984bf-8e62-4f3e-8f62-285fbb1ef7c9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:07.673121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fa4984bf-8e62-4f3e-8f62-285fbb1ef7c9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:07.680295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4984bf-8e62-4f3e-8f62-285fbb1ef7c9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:07.680826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4984bf-8e62-4f3e-8f62-285fbb1ef7c9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:07.681413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4984bf-8e62-4f3e-8f62-285fbb1ef7c9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:07.682057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4984bf-8e62-4f3e-8f62-285fbb1ef7c9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:07.682572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4984bf-8e62-4f3e-8f62-285fbb1ef7c9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:07.682985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4984bf-8e62-4f3e-8f62-285fbb1ef7c9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:07.829265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fa4984bf-8e62-4f3e-8f62-285fbb1ef7c9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:07.837778 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fa4984bf-8e62-4f3e-8f62-285fbb1ef7c9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.174004 Sep 27 10:32:07.838595 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3332/6202] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:07 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:08.863675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7de29f8-47be-447d-a6ae-4a85ea3fc2c0 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:08.864324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7de29f8-47be-447d-a6ae-4a85ea3fc2c0 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:08.874298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7de29f8-47be-447d-a6ae-4a85ea3fc2c0 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:08.874809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7de29f8-47be-447d-a6ae-4a85ea3fc2c0 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:08.875267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7de29f8-47be-447d-a6ae-4a85ea3fc2c0 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:08.876032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7de29f8-47be-447d-a6ae-4a85ea3fc2c0 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:08.876512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7de29f8-47be-447d-a6ae-4a85ea3fc2c0 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:08.876995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7de29f8-47be-447d-a6ae-4a85ea3fc2c0 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:09.023576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c7de29f8-47be-447d-a6ae-4a85ea3fc2c0 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:09.034036 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c7de29f8-47be-447d-a6ae-4a85ea3fc2c0 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.175024 Sep 27 10:32:09.034607 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2871/6203] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:08 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:10.058903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58b466e7-ea57-46cf-9c44-7b8370063974 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:10.059529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58b466e7-ea57-46cf-9c44-7b8370063974 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:10.066561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b466e7-ea57-46cf-9c44-7b8370063974 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:10.067055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b466e7-ea57-46cf-9c44-7b8370063974 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:10.067493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b466e7-ea57-46cf-9c44-7b8370063974 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:10.068235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b466e7-ea57-46cf-9c44-7b8370063974 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:10.068796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b466e7-ea57-46cf-9c44-7b8370063974 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:10.069222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58b466e7-ea57-46cf-9c44-7b8370063974 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:10.219855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58b466e7-ea57-46cf-9c44-7b8370063974 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:10.231820 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58b466e7-ea57-46cf-9c44-7b8370063974 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.177038 Sep 27 10:32:10.232464 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3333/6204] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:10 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:11.257627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4955a3d6-a55a-466a-b665-39af78c970f1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:11.258381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4955a3d6-a55a-466a-b665-39af78c970f1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:11.266355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4955a3d6-a55a-466a-b665-39af78c970f1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:11.266941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4955a3d6-a55a-466a-b665-39af78c970f1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:11.267439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4955a3d6-a55a-466a-b665-39af78c970f1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:11.268424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4955a3d6-a55a-466a-b665-39af78c970f1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:11.269128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4955a3d6-a55a-466a-b665-39af78c970f1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:11.269685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4955a3d6-a55a-466a-b665-39af78c970f1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:11.417878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4955a3d6-a55a-466a-b665-39af78c970f1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:11.427046 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4955a3d6-a55a-466a-b665-39af78c970f1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.173850 Sep 27 10:32:11.428002 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2872/6205] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:11 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:12.456350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0f153abf-ff97-429a-9f5f-3bfff28eeffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:12.456969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0f153abf-ff97-429a-9f5f-3bfff28eeffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:12.467380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f153abf-ff97-429a-9f5f-3bfff28eeffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:12.468007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f153abf-ff97-429a-9f5f-3bfff28eeffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:12.469059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f153abf-ff97-429a-9f5f-3bfff28eeffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:12.470015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f153abf-ff97-429a-9f5f-3bfff28eeffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:12.470646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f153abf-ff97-429a-9f5f-3bfff28eeffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:12.471191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f153abf-ff97-429a-9f5f-3bfff28eeffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:12.618449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0f153abf-ff97-429a-9f5f-3bfff28eeffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:12.627320 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0f153abf-ff97-429a-9f5f-3bfff28eeffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.177107 Sep 27 10:32:12.627902 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3334/6206] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:12 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:13.654582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f4684ad3-9981-41cf-b743-43ddd4de3955 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:13.655468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f4684ad3-9981-41cf-b743-43ddd4de3955 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:13.666032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4684ad3-9981-41cf-b743-43ddd4de3955 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:13.666710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4684ad3-9981-41cf-b743-43ddd4de3955 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:13.667463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4684ad3-9981-41cf-b743-43ddd4de3955 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:13.668516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4684ad3-9981-41cf-b743-43ddd4de3955 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:13.669226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4684ad3-9981-41cf-b743-43ddd4de3955 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:13.669824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f4684ad3-9981-41cf-b743-43ddd4de3955 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:13.809444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f4684ad3-9981-41cf-b743-43ddd4de3955 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:13.818092 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f4684ad3-9981-41cf-b743-43ddd4de3955 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.170227 Sep 27 10:32:13.818669 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2873/6207] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:13 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:14.846288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5258c49e-bfb2-469d-965c-5381fee1b755 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:14.847316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5258c49e-bfb2-469d-965c-5381fee1b755 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:14.856554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5258c49e-bfb2-469d-965c-5381fee1b755 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:14.857081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5258c49e-bfb2-469d-965c-5381fee1b755 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:14.857511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5258c49e-bfb2-469d-965c-5381fee1b755 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:14.858219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5258c49e-bfb2-469d-965c-5381fee1b755 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:14.858671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5258c49e-bfb2-469d-965c-5381fee1b755 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:14.859098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5258c49e-bfb2-469d-965c-5381fee1b755 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:15.008165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5258c49e-bfb2-469d-965c-5381fee1b755 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:15.016956 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5258c49e-bfb2-469d-965c-5381fee1b755 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.177685 Sep 27 10:32:15.017540 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3335/6208] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:14 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:16.040334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb5d5670-c3be-4903-a9c2-937b0fcde0a2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:16.041166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fb5d5670-c3be-4903-a9c2-937b0fcde0a2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:16.048768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5d5670-c3be-4903-a9c2-937b0fcde0a2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:16.049274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5d5670-c3be-4903-a9c2-937b0fcde0a2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:16.049733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5d5670-c3be-4903-a9c2-937b0fcde0a2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:16.050450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5d5670-c3be-4903-a9c2-937b0fcde0a2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:16.050922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5d5670-c3be-4903-a9c2-937b0fcde0a2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:16.051363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb5d5670-c3be-4903-a9c2-937b0fcde0a2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:16.188915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fb5d5670-c3be-4903-a9c2-937b0fcde0a2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:16.198337 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fb5d5670-c3be-4903-a9c2-937b0fcde0a2 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.162270 Sep 27 10:32:16.198964 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2874/6209] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:16 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:17.226716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-08ab42f3-0c06-4f9c-a5fb-2b12c4b1a4d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:17.227747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-08ab42f3-0c06-4f9c-a5fb-2b12c4b1a4d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:17.239781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08ab42f3-0c06-4f9c-a5fb-2b12c4b1a4d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:17.240505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08ab42f3-0c06-4f9c-a5fb-2b12c4b1a4d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:17.241327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08ab42f3-0c06-4f9c-a5fb-2b12c4b1a4d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:17.242500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08ab42f3-0c06-4f9c-a5fb-2b12c4b1a4d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:17.243278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08ab42f3-0c06-4f9c-a5fb-2b12c4b1a4d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:17.244317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08ab42f3-0c06-4f9c-a5fb-2b12c4b1a4d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:17.389565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-08ab42f3-0c06-4f9c-a5fb-2b12c4b1a4d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:17.402309 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-08ab42f3-0c06-4f9c-a5fb-2b12c4b1a4d5 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.182289 Sep 27 10:32:17.402874 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3336/6210] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:17 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:18.439006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dd4d0e5f-ec9e-4e1b-84cc-d607af0a552d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:18.440020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dd4d0e5f-ec9e-4e1b-84cc-d607af0a552d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:18.450248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd4d0e5f-ec9e-4e1b-84cc-d607af0a552d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:18.450750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd4d0e5f-ec9e-4e1b-84cc-d607af0a552d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:18.451201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd4d0e5f-ec9e-4e1b-84cc-d607af0a552d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:18.451917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd4d0e5f-ec9e-4e1b-84cc-d607af0a552d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:18.452388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd4d0e5f-ec9e-4e1b-84cc-d607af0a552d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:18.452870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd4d0e5f-ec9e-4e1b-84cc-d607af0a552d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:18.585390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dd4d0e5f-ec9e-4e1b-84cc-d607af0a552d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:18.594446 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dd4d0e5f-ec9e-4e1b-84cc-d607af0a552d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.162736 Sep 27 10:32:18.595021 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2875/6211] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:18 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:19.621441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8f391dad-bdc1-465f-a9c8-8f20b3e4000e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:19.622055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8f391dad-bdc1-465f-a9c8-8f20b3e4000e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:19.629655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f391dad-bdc1-465f-a9c8-8f20b3e4000e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:19.630150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f391dad-bdc1-465f-a9c8-8f20b3e4000e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:19.630589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f391dad-bdc1-465f-a9c8-8f20b3e4000e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:19.631301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f391dad-bdc1-465f-a9c8-8f20b3e4000e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:19.631764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f391dad-bdc1-465f-a9c8-8f20b3e4000e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:19.632195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8f391dad-bdc1-465f-a9c8-8f20b3e4000e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:19.781362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8f391dad-bdc1-465f-a9c8-8f20b3e4000e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:19.791701 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8f391dad-bdc1-465f-a9c8-8f20b3e4000e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.175638 Sep 27 10:32:19.792459 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3337/6212] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:19 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:20.818446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7fb26f2e-9d72-4bc8-942b-1a44a1217ee4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:20.819073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7fb26f2e-9d72-4bc8-942b-1a44a1217ee4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:20.828092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fb26f2e-9d72-4bc8-942b-1a44a1217ee4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:20.828623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fb26f2e-9d72-4bc8-942b-1a44a1217ee4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:20.829098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fb26f2e-9d72-4bc8-942b-1a44a1217ee4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:20.829871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fb26f2e-9d72-4bc8-942b-1a44a1217ee4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:20.830388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fb26f2e-9d72-4bc8-942b-1a44a1217ee4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:20.830832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fb26f2e-9d72-4bc8-942b-1a44a1217ee4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:20.964919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7fb26f2e-9d72-4bc8-942b-1a44a1217ee4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:20.974150 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7fb26f2e-9d72-4bc8-942b-1a44a1217ee4 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.161988 Sep 27 10:32:20.974733 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2876/6213] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:20 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:22.001937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0f744017-2f96-4004-bd69-ff0f6dea671e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:22.003429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0f744017-2f96-4004-bd69-ff0f6dea671e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:22.015207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f744017-2f96-4004-bd69-ff0f6dea671e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:22.015710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f744017-2f96-4004-bd69-ff0f6dea671e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:22.016150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f744017-2f96-4004-bd69-ff0f6dea671e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:22.016915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f744017-2f96-4004-bd69-ff0f6dea671e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:22.017351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f744017-2f96-4004-bd69-ff0f6dea671e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:22.017784 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0f744017-2f96-4004-bd69-ff0f6dea671e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:22.020442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=70,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:22.159244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0f744017-2f96-4004-bd69-ff0f6dea671e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:22.167792 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0f744017-2f96-4004-bd69-ff0f6dea671e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.172696 Sep 27 10:32:22.168400 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3338/6214] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:21 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:22.984876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8b8ed91d-badc-44cf-b8c3-89125c766aab tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:22.985630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8b8ed91d-badc-44cf-b8c3-89125c766aab tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:22.992493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b8ed91d-badc-44cf-b8c3-89125c766aab tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:22.993049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b8ed91d-badc-44cf-b8c3-89125c766aab tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:22.993514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b8ed91d-badc-44cf-b8c3-89125c766aab tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:22.994236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b8ed91d-badc-44cf-b8c3-89125c766aab tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:22.994716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b8ed91d-badc-44cf-b8c3-89125c766aab tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:22.995153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b8ed91d-badc-44cf-b8c3-89125c766aab tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:23.133168 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8b8ed91d-badc-44cf-b8c3-89125c766aab tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1834 microversion: 2.1 time: 0.153905 Sep 27 10:32:23.133791 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2877/6215] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:22 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1834 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:23.158250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e7ecbad-110c-4d81-9047-a3f6896c46ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:23.158846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e7ecbad-110c-4d81-9047-a3f6896c46ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:23.166057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7ecbad-110c-4d81-9047-a3f6896c46ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:23.166554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7ecbad-110c-4d81-9047-a3f6896c46ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:23.167000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7ecbad-110c-4d81-9047-a3f6896c46ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:23.167708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7ecbad-110c-4d81-9047-a3f6896c46ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:23.168180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7ecbad-110c-4d81-9047-a3f6896c46ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:23.168608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e7ecbad-110c-4d81-9047-a3f6896c46ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:23.191912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6596bed3-ba8b-426e-a15f-d026e6d8aaac tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:23.192556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6596bed3-ba8b-426e-a15f-d026e6d8aaac tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:23.198797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6596bed3-ba8b-426e-a15f-d026e6d8aaac tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:23.199305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6596bed3-ba8b-426e-a15f-d026e6d8aaac tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:23.199761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6596bed3-ba8b-426e-a15f-d026e6d8aaac tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:23.200492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6596bed3-ba8b-426e-a15f-d026e6d8aaac tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:23.201009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6596bed3-ba8b-426e-a15f-d026e6d8aaac tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:23.201461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6596bed3-ba8b-426e-a15f-d026e6d8aaac tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:23.306359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0e7ecbad-110c-4d81-9047-a3f6896c46ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:23.314675 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e7ecbad-110c-4d81-9047-a3f6896c46ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1683 microversion: 2.1 time: 0.160766 Sep 27 10:32:23.315309 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3339/6216] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:23 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1683 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:23.336758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6596bed3-ba8b-426e-a15f-d026e6d8aaac tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:23.346354 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6596bed3-ba8b-426e-a15f-d026e6d8aaac tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.158507 Sep 27 10:32:23.346944 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2878/6217] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:23 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:23.421745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-93e16c2b-919b-40b7-bd6e-6cbccd514096 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:23.425372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-93e16c2b-919b-40b7-bd6e-6cbccd514096 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:23.435434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93e16c2b-919b-40b7-bd6e-6cbccd514096 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:23.436127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93e16c2b-919b-40b7-bd6e-6cbccd514096 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:23.436901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93e16c2b-919b-40b7-bd6e-6cbccd514096 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:23.437611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93e16c2b-919b-40b7-bd6e-6cbccd514096 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:23.438071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93e16c2b-919b-40b7-bd6e-6cbccd514096 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:23.438496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93e16c2b-919b-40b7-bd6e-6cbccd514096 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:23.512013 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-93e16c2b-919b-40b7-bd6e-6cbccd514096 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed/action" status: 200 len: 32634 microversion: 2.1 time: 0.096461 Sep 27 10:32:23.512962 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3340/6218] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:32:23 2022] POST /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed/action => generated 32634 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:32:23.693668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Action: 'action', calling method: >, body: {"migrate": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:23.694461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:23.702578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:23.703088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:23.703560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:23.704402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:23.704915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:23.705363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:23.707973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=82,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:23.793173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '5436a87ddec64a47bde28e63c0a3a368', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd4cd350726034f9985727755c395c181', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:23.793796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] flavor_id is None. Assuming migration. {{(pid=145969) resize /opt/stack/nova/nova/compute/api.py:4196}} Sep 27 10:32:23.794261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Old instance type m1.heat2, new instance type m1.heat2 {{(pid=145969) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:32:23.960606 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cc720729-adc4-4c43-be1b-b58ca8006282 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed/action" status: 202 len: 0 microversion: 2.1 time: 0.272270 Sep 27 10:32:23.961260 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2879/6219] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:32:23 2022] POST /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed/action => generated 0 bytes in 274 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:32:23.970992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a4e00469-e9f9-45a2-b0ed-6bb039bb2bfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:23.971698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a4e00469-e9f9-45a2-b0ed-6bb039bb2bfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:23.978918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4e00469-e9f9-45a2-b0ed-6bb039bb2bfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:23.979406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4e00469-e9f9-45a2-b0ed-6bb039bb2bfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:23.979843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4e00469-e9f9-45a2-b0ed-6bb039bb2bfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:23.980547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4e00469-e9f9-45a2-b0ed-6bb039bb2bfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:23.981035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4e00469-e9f9-45a2-b0ed-6bb039bb2bfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:23.981454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a4e00469-e9f9-45a2-b0ed-6bb039bb2bfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:24.152019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a4e00469-e9f9-45a2-b0ed-6bb039bb2bfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:24.160490 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a4e00469-e9f9-45a2-b0ed-6bb039bb2bfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1692 microversion: 2.1 time: 0.194615 Sep 27 10:32:24.161152 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3341/6220] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:23 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1692 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:24.370769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c73023a7-4885-456b-b138-d34c112a64ab tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:24.371399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c73023a7-4885-456b-b138-d34c112a64ab tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:24.379009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c73023a7-4885-456b-b138-d34c112a64ab tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:24.379511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c73023a7-4885-456b-b138-d34c112a64ab tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:24.379966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c73023a7-4885-456b-b138-d34c112a64ab tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:24.380675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c73023a7-4885-456b-b138-d34c112a64ab tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:24.381157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c73023a7-4885-456b-b138-d34c112a64ab tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:24.381599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c73023a7-4885-456b-b138-d34c112a64ab tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:24.518258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c73023a7-4885-456b-b138-d34c112a64ab tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:24.527965 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c73023a7-4885-456b-b138-d34c112a64ab tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.162323 Sep 27 10:32:24.528550 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2880/6221] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:24 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:25.184230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b6a6cc25-45d3-4036-b45b-bc8f6050927e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:25.184880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b6a6cc25-45d3-4036-b45b-bc8f6050927e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:25.192969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a6cc25-45d3-4036-b45b-bc8f6050927e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:25.193781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a6cc25-45d3-4036-b45b-bc8f6050927e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:25.193921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a6cc25-45d3-4036-b45b-bc8f6050927e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:25.194799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a6cc25-45d3-4036-b45b-bc8f6050927e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:25.195507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a6cc25-45d3-4036-b45b-bc8f6050927e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:25.196171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6a6cc25-45d3-4036-b45b-bc8f6050927e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:25.337762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b6a6cc25-45d3-4036-b45b-bc8f6050927e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:25.347974 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b6a6cc25-45d3-4036-b45b-bc8f6050927e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1692 microversion: 2.1 time: 0.167690 Sep 27 10:32:25.348619 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3342/6222] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:25 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1692 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:25.552100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c22667c5-dfb5-452d-89e0-69231351dfd6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:25.553404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c22667c5-dfb5-452d-89e0-69231351dfd6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:25.560890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22667c5-dfb5-452d-89e0-69231351dfd6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:25.561443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22667c5-dfb5-452d-89e0-69231351dfd6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:25.561954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22667c5-dfb5-452d-89e0-69231351dfd6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:25.562817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22667c5-dfb5-452d-89e0-69231351dfd6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:25.563393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22667c5-dfb5-452d-89e0-69231351dfd6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:25.563839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22667c5-dfb5-452d-89e0-69231351dfd6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:25.775468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c22667c5-dfb5-452d-89e0-69231351dfd6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:25.785682 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c22667c5-dfb5-452d-89e0-69231351dfd6 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.238416 Sep 27 10:32:25.786268 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2881/6223] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:25 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:26.380400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6ea084c3-0908-4e27-964d-d7ed088137c6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:26.381357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6ea084c3-0908-4e27-964d-d7ed088137c6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:26.391905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea084c3-0908-4e27-964d-d7ed088137c6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:26.392423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea084c3-0908-4e27-964d-d7ed088137c6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:26.392898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea084c3-0908-4e27-964d-d7ed088137c6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:26.393613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea084c3-0908-4e27-964d-d7ed088137c6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:26.394070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea084c3-0908-4e27-964d-d7ed088137c6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:26.394502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ea084c3-0908-4e27-964d-d7ed088137c6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:26.540875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6ea084c3-0908-4e27-964d-d7ed088137c6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:26.549807 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6ea084c3-0908-4e27-964d-d7ed088137c6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1692 microversion: 2.1 time: 0.176684 Sep 27 10:32:26.550430 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3343/6224] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:26 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1692 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:26.809827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cf83e55a-7b0b-448c-a4c9-5a0c3c2f2315 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:26.811287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cf83e55a-7b0b-448c-a4c9-5a0c3c2f2315 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:26.819317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf83e55a-7b0b-448c-a4c9-5a0c3c2f2315 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:26.819986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf83e55a-7b0b-448c-a4c9-5a0c3c2f2315 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:26.820512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf83e55a-7b0b-448c-a4c9-5a0c3c2f2315 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:26.821369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf83e55a-7b0b-448c-a4c9-5a0c3c2f2315 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:26.821905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf83e55a-7b0b-448c-a4c9-5a0c3c2f2315 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:26.822416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf83e55a-7b0b-448c-a4c9-5a0c3c2f2315 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:26.963338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cf83e55a-7b0b-448c-a4c9-5a0c3c2f2315 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:26.972522 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cf83e55a-7b0b-448c-a4c9-5a0c3c2f2315 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1737 microversion: 2.1 time: 0.166875 Sep 27 10:32:26.973076 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2882/6225] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:26 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:27.577690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c53ebdb4-4864-417d-9585-51950cc6f266 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:27.578884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c53ebdb4-4864-417d-9585-51950cc6f266 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:27.590917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c53ebdb4-4864-417d-9585-51950cc6f266 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:27.591768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c53ebdb4-4864-417d-9585-51950cc6f266 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:27.592551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c53ebdb4-4864-417d-9585-51950cc6f266 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:27.594127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c53ebdb4-4864-417d-9585-51950cc6f266 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:27.594594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c53ebdb4-4864-417d-9585-51950cc6f266 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:27.595019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c53ebdb4-4864-417d-9585-51950cc6f266 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:27.745102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c53ebdb4-4864-417d-9585-51950cc6f266 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:27.753321 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c53ebdb4-4864-417d-9585-51950cc6f266 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.182298 Sep 27 10:32:27.753896 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3344/6226] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:27 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:28.000087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3e75d2c-db72-4216-ba91-d14d70bae7fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:28.001041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e3e75d2c-db72-4216-ba91-d14d70bae7fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:28.008566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3e75d2c-db72-4216-ba91-d14d70bae7fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:28.009185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3e75d2c-db72-4216-ba91-d14d70bae7fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:28.009645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3e75d2c-db72-4216-ba91-d14d70bae7fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:28.010372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3e75d2c-db72-4216-ba91-d14d70bae7fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:28.010852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3e75d2c-db72-4216-ba91-d14d70bae7fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:28.011300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3e75d2c-db72-4216-ba91-d14d70bae7fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:28.155673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e3e75d2c-db72-4216-ba91-d14d70bae7fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:28.164580 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e3e75d2c-db72-4216-ba91-d14d70bae7fd tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1717 microversion: 2.1 time: 0.171492 Sep 27 10:32:28.165221 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2883/6227] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:27 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1717 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:28.229708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:28.230212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:28.236907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:28.237414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:28.237846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:28.238542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:28.238998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:28.239430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:28.338911 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] HTTP exception thrown: Cannot 'detach_volume' instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 while it is in vm_state rescued Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Request method failure captured: Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-volume_attachments/a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3 HTTP/1.1 Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtB5lZw26WfhuIIxDLfEGtJx8GdKac9mUecPJWt47RWTVgFEQNt2CRnQRU62Kxhrd_5bXczg0Q9QWsT4c85-tEEw3L_ZoXg7ttqo2fTxJvLnAMIld1Q9kkM9ItLXE6S_aDTtz0h96WsfSIhdls5sa9AFKLBsHqf09n27PObWbpVe7TQ Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_40247edd5a444fd8b1c937c13d39393a"}]}] Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6771df45b4ee4426b2f76a09eab2998a Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: exception: Cannot 'detach_volume' instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 while it is in vm_state rescued Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'server_id': 'dc9ddb88-bfb7-40d7-b615-9755d1e824d5', 'id': 'a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3'} Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 in vm_state rescued. Cannot detach_volume while the instance is in this state. Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 while it is in vm_state rescued Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:32:28.340690 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:32:28.346791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:32:28.346791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:32:28.346791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:32:28.346791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:32:28.346791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 while it is in vm_state rescued Sep 27 10:32:28.346791 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:32:28.346791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Returning 409 to user: Cannot 'detach_volume' instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 while it is in vm_state rescued {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:32:28.346791 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f6258e07-75e1-4935-be0e-6832cd445ffa tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "DELETE /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-volume_attachments/a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3" status: 409 len: 152 microversion: 2.1 time: 0.116727 Sep 27 10:32:28.346791 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3345/6228] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:32:28 2022] DELETE /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-volume_attachments/a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3 => generated 152 bytes in 118 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:32:28.352032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a7c3b12b-20e1-4b8c-b60f-6b1a3a4ccf73 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:28.352584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a7c3b12b-20e1-4b8c-b60f-6b1a3a4ccf73 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:28.359083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7c3b12b-20e1-4b8c-b60f-6b1a3a4ccf73 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:28.360010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7c3b12b-20e1-4b8c-b60f-6b1a3a4ccf73 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:28.360491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7c3b12b-20e1-4b8c-b60f-6b1a3a4ccf73 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:28.361241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7c3b12b-20e1-4b8c-b60f-6b1a3a4ccf73 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:28.361743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7c3b12b-20e1-4b8c-b60f-6b1a3a4ccf73 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:28.362179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7c3b12b-20e1-4b8c-b60f-6b1a3a4ccf73 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:28.431982 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a7c3b12b-20e1-4b8c-b60f-6b1a3a4ccf73 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/action" status: 202 len: 0 microversion: 2.1 time: 0.083814 Sep 27 10:32:28.432686 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2884/6229] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:32:28 2022] POST /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/action => generated 0 bytes in 85 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:32:28.441261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d2de7c00-cfff-4eb2-9dfb-3ba47dfff0a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:28.441880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d2de7c00-cfff-4eb2-9dfb-3ba47dfff0a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:28.448763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2de7c00-cfff-4eb2-9dfb-3ba47dfff0a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:28.449358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2de7c00-cfff-4eb2-9dfb-3ba47dfff0a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:28.449872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2de7c00-cfff-4eb2-9dfb-3ba47dfff0a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:28.450908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2de7c00-cfff-4eb2-9dfb-3ba47dfff0a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:28.451522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2de7c00-cfff-4eb2-9dfb-3ba47dfff0a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:28.451991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d2de7c00-cfff-4eb2-9dfb-3ba47dfff0a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:28.595881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d2de7c00-cfff-4eb2-9dfb-3ba47dfff0a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:28.605129 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d2de7c00-cfff-4eb2-9dfb-3ba47dfff0a7 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1725 microversion: 2.1 time: 0.168084 Sep 27 10:32:28.605717 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3346/6230] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:28 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1725 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:28.779602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-83f48b3a-584f-4152-9546-183cdf5e4347 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:28.780568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-83f48b3a-584f-4152-9546-183cdf5e4347 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:28.787646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f48b3a-584f-4152-9546-183cdf5e4347 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:28.788133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f48b3a-584f-4152-9546-183cdf5e4347 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:28.788578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f48b3a-584f-4152-9546-183cdf5e4347 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:28.789838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f48b3a-584f-4152-9546-183cdf5e4347 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:28.789838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f48b3a-584f-4152-9546-183cdf5e4347 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:28.790434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f48b3a-584f-4152-9546-183cdf5e4347 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:29.049039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-83f48b3a-584f-4152-9546-183cdf5e4347 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:29.058191 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-83f48b3a-584f-4152-9546-183cdf5e4347 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.284304 Sep 27 10:32:29.058776 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2885/6231] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:28 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:29.628838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6a49c75-5a9c-4699-b0fc-55bbe6f89a8d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:29.629472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c6a49c75-5a9c-4699-b0fc-55bbe6f89a8d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:29.636974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6a49c75-5a9c-4699-b0fc-55bbe6f89a8d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:29.637469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6a49c75-5a9c-4699-b0fc-55bbe6f89a8d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:29.637892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6a49c75-5a9c-4699-b0fc-55bbe6f89a8d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:29.638624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6a49c75-5a9c-4699-b0fc-55bbe6f89a8d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:29.639078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6a49c75-5a9c-4699-b0fc-55bbe6f89a8d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:29.639505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6a49c75-5a9c-4699-b0fc-55bbe6f89a8d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:29.790796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c6a49c75-5a9c-4699-b0fc-55bbe6f89a8d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:29.801181 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c6a49c75-5a9c-4699-b0fc-55bbe6f89a8d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1725 microversion: 2.1 time: 0.176612 Sep 27 10:32:29.801848 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3347/6232] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:29 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1725 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:32:30.082679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-27f394fc-ad63-4113-89d8-6a98fffaad44 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:30.083394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-27f394fc-ad63-4113-89d8-6a98fffaad44 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:30.091109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27f394fc-ad63-4113-89d8-6a98fffaad44 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:30.091626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27f394fc-ad63-4113-89d8-6a98fffaad44 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:30.092117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27f394fc-ad63-4113-89d8-6a98fffaad44 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:30.092878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27f394fc-ad63-4113-89d8-6a98fffaad44 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:30.093744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27f394fc-ad63-4113-89d8-6a98fffaad44 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:30.094266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27f394fc-ad63-4113-89d8-6a98fffaad44 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:30.246800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-27f394fc-ad63-4113-89d8-6a98fffaad44 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:30.255416 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-27f394fc-ad63-4113-89d8-6a98fffaad44 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.177268 Sep 27 10:32:30.255999 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2886/6233] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:30 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:32:30.826891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fcdb9d57-eecd-4c64-9cd4-d6b9829b0e62 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:30.827528 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fcdb9d57-eecd-4c64-9cd4-d6b9829b0e62 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:30.835556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcdb9d57-eecd-4c64-9cd4-d6b9829b0e62 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:30.836050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcdb9d57-eecd-4c64-9cd4-d6b9829b0e62 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:30.836485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcdb9d57-eecd-4c64-9cd4-d6b9829b0e62 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:30.837437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcdb9d57-eecd-4c64-9cd4-d6b9829b0e62 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:30.838122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcdb9d57-eecd-4c64-9cd4-d6b9829b0e62 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:30.838533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcdb9d57-eecd-4c64-9cd4-d6b9829b0e62 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:30.977899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fcdb9d57-eecd-4c64-9cd4-d6b9829b0e62 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:30.987423 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fcdb9d57-eecd-4c64-9cd4-d6b9829b0e62 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1725 microversion: 2.1 time: 0.164717 Sep 27 10:32:30.988322 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3348/6234] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:30 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1725 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:31.282940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-29d57e51-2c9e-4eda-8d82-f5f234c97d1a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:31.283869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-29d57e51-2c9e-4eda-8d82-f5f234c97d1a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:31.294181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d57e51-2c9e-4eda-8d82-f5f234c97d1a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:31.294928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d57e51-2c9e-4eda-8d82-f5f234c97d1a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:31.295587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d57e51-2c9e-4eda-8d82-f5f234c97d1a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:31.296526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d57e51-2c9e-4eda-8d82-f5f234c97d1a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:31.297250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d57e51-2c9e-4eda-8d82-f5f234c97d1a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:31.297879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d57e51-2c9e-4eda-8d82-f5f234c97d1a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:31.456084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-29d57e51-2c9e-4eda-8d82-f5f234c97d1a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:31.465505 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-29d57e51-2c9e-4eda-8d82-f5f234c97d1a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.189055 Sep 27 10:32:31.466097 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2887/6235] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:31 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:32.012585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e08c3b0-5600-4143-9889-3343bce66f8a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:32.013242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8e08c3b0-5600-4143-9889-3343bce66f8a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:32.022907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e08c3b0-5600-4143-9889-3343bce66f8a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:32.023465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e08c3b0-5600-4143-9889-3343bce66f8a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:32.024008 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e08c3b0-5600-4143-9889-3343bce66f8a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:32.025048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e08c3b0-5600-4143-9889-3343bce66f8a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:32.025538 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e08c3b0-5600-4143-9889-3343bce66f8a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:32.025980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e08c3b0-5600-4143-9889-3343bce66f8a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:32.169691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8e08c3b0-5600-4143-9889-3343bce66f8a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:32.178382 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e08c3b0-5600-4143-9889-3343bce66f8a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1731 microversion: 2.1 time: 0.169960 Sep 27 10:32:32.178952 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3349/6236] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:32 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1731 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:32.201322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd4cc42f-6b3c-49dc-9612-3f227f35339b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:32.201836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cd4cc42f-6b3c-49dc-9612-3f227f35339b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:32.208182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4cc42f-6b3c-49dc-9612-3f227f35339b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:32.209251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4cc42f-6b3c-49dc-9612-3f227f35339b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:32.209729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4cc42f-6b3c-49dc-9612-3f227f35339b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:32.210451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4cc42f-6b3c-49dc-9612-3f227f35339b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:32.210924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4cc42f-6b3c-49dc-9612-3f227f35339b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:32.211375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4cc42f-6b3c-49dc-9612-3f227f35339b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:32.307639 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd4cc42f-6b3c-49dc-9612-3f227f35339b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.110242 Sep 27 10:32:32.308567 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2888/6237] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:32:32 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-interface => generated 284 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:32:32.491903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-32c37092-ae0d-4667-b493-8cbacce67963 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:32.492641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-32c37092-ae0d-4667-b493-8cbacce67963 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:32.499937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c37092-ae0d-4667-b493-8cbacce67963 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:32.500693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c37092-ae0d-4667-b493-8cbacce67963 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:32.501225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c37092-ae0d-4667-b493-8cbacce67963 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:32.502056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c37092-ae0d-4667-b493-8cbacce67963 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:32.502637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c37092-ae0d-4667-b493-8cbacce67963 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:32.503133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-32c37092-ae0d-4667-b493-8cbacce67963 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:32.782843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-32c37092-ae0d-4667-b493-8cbacce67963 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:32.792241 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-32c37092-ae0d-4667-b493-8cbacce67963 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.304880 Sep 27 10:32:32.792896 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3350/6238] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:32 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:33.372032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7d725f7f-d34a-4a92-9eee-e513ef5f120c req-9ee6060d-485b-43fc-a058-964910a4f0c0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dc9ddb88-bfb7-40d7-b615-9755d1e824d5", "tag": "b77efabf-15d1-4744-b2d6-473b94c9fb51"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:33.386730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7d725f7f-d34a-4a92-9eee-e513ef5f120c req-9ee6060d-485b-43fc-a058-964910a4f0c0 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:33.387698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7d725f7f-d34a-4a92-9eee-e513ef5f120c req-9ee6060d-485b-43fc-a058-964910a4f0c0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:33.388424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7d725f7f-d34a-4a92-9eee-e513ef5f120c req-9ee6060d-485b-43fc-a058-964910a4f0c0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:33.402945 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7d725f7f-d34a-4a92-9eee-e513ef5f120c req-9ee6060d-485b-43fc-a058-964910a4f0c0 service nova] Creating event network-changed:b77efabf-15d1-4744-b2d6-473b94c9fb51 for instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 on nch2-master-1 Sep 27 10:32:33.408619 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7d725f7f-d34a-4a92-9eee-e513ef5f120c req-9ee6060d-485b-43fc-a058-964910a4f0c0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041832 Sep 27 10:32:33.409354 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2889/6239] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:32:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:32:33.818672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e7127a87-b2a7-4a45-8db5-951c1ca4811c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:33.819419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e7127a87-b2a7-4a45-8db5-951c1ca4811c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:33.826629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7127a87-b2a7-4a45-8db5-951c1ca4811c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:33.827124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7127a87-b2a7-4a45-8db5-951c1ca4811c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:33.827585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7127a87-b2a7-4a45-8db5-951c1ca4811c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:33.828339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7127a87-b2a7-4a45-8db5-951c1ca4811c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:33.828863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7127a87-b2a7-4a45-8db5-951c1ca4811c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:33.829301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e7127a87-b2a7-4a45-8db5-951c1ca4811c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:33.993876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e7127a87-b2a7-4a45-8db5-951c1ca4811c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:34.004210 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e7127a87-b2a7-4a45-8db5-951c1ca4811c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.190361 Sep 27 10:32:34.004865 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3351/6240] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:33 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:35.030248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-91798c6c-7b57-4f60-9623-a10c2b0087a7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:35.030997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-91798c6c-7b57-4f60-9623-a10c2b0087a7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:35.038003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91798c6c-7b57-4f60-9623-a10c2b0087a7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:35.038535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91798c6c-7b57-4f60-9623-a10c2b0087a7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:35.039075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91798c6c-7b57-4f60-9623-a10c2b0087a7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:35.039835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91798c6c-7b57-4f60-9623-a10c2b0087a7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:35.040381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91798c6c-7b57-4f60-9623-a10c2b0087a7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:35.040865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-91798c6c-7b57-4f60-9623-a10c2b0087a7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:35.198713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-91798c6c-7b57-4f60-9623-a10c2b0087a7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:35.207793 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-91798c6c-7b57-4f60-9623-a10c2b0087a7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.183475 Sep 27 10:32:35.208410 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2890/6241] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:35 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:35.424395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e00d565e-d567-411a-bada-94f708324e4a req-c3856089-ceb5-4a47-8f1b-0b97e32a8c72 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dc9ddb88-bfb7-40d7-b615-9755d1e824d5", "tag": "b77efabf-15d1-4744-b2d6-473b94c9fb51"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:35.430696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=42 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:35.437811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e00d565e-d567-411a-bada-94f708324e4a req-c3856089-ceb5-4a47-8f1b-0b97e32a8c72 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:35.438370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e00d565e-d567-411a-bada-94f708324e4a req-c3856089-ceb5-4a47-8f1b-0b97e32a8c72 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:35.438839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e00d565e-d567-411a-bada-94f708324e4a req-c3856089-ceb5-4a47-8f1b-0b97e32a8c72 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:35.450865 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e00d565e-d567-411a-bada-94f708324e4a req-c3856089-ceb5-4a47-8f1b-0b97e32a8c72 service nova] Creating event network-changed:b77efabf-15d1-4744-b2d6-473b94c9fb51 for instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 on nch2-master-1 Sep 27 10:32:35.456408 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e00d565e-d567-411a-bada-94f708324e4a req-c3856089-ceb5-4a47-8f1b-0b97e32a8c72 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035985 Sep 27 10:32:35.457255 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3352/6242] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:32:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:32:36.236423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58a98270-a8d5-440b-a0ef-f96f591900f6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:36.237042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-58a98270-a8d5-440b-a0ef-f96f591900f6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:36.245047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58a98270-a8d5-440b-a0ef-f96f591900f6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:36.245695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58a98270-a8d5-440b-a0ef-f96f591900f6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:36.246466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58a98270-a8d5-440b-a0ef-f96f591900f6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:36.247428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58a98270-a8d5-440b-a0ef-f96f591900f6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:36.248026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58a98270-a8d5-440b-a0ef-f96f591900f6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:36.248532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58a98270-a8d5-440b-a0ef-f96f591900f6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:36.403178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-58a98270-a8d5-440b-a0ef-f96f591900f6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:36.411566 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-58a98270-a8d5-440b-a0ef-f96f591900f6 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.180476 Sep 27 10:32:36.412856 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2891/6243] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:36 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:37.441497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b23fc488-f79e-4520-b262-8e25ee4a0fd5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:37.442180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b23fc488-f79e-4520-b262-8e25ee4a0fd5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:37.449180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b23fc488-f79e-4520-b262-8e25ee4a0fd5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:37.449669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b23fc488-f79e-4520-b262-8e25ee4a0fd5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:37.450229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b23fc488-f79e-4520-b262-8e25ee4a0fd5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:37.450941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b23fc488-f79e-4520-b262-8e25ee4a0fd5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:37.451399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b23fc488-f79e-4520-b262-8e25ee4a0fd5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:37.451830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b23fc488-f79e-4520-b262-8e25ee4a0fd5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:37.601172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b23fc488-f79e-4520-b262-8e25ee4a0fd5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:37.609745 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b23fc488-f79e-4520-b262-8e25ee4a0fd5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.174249 Sep 27 10:32:37.610468 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3353/6244] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:37 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:38.633923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ddd659b7-bdbf-446e-9c37-e21221668de9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:38.634605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ddd659b7-bdbf-446e-9c37-e21221668de9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:38.637916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=46 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:38.643919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddd659b7-bdbf-446e-9c37-e21221668de9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:38.644427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddd659b7-bdbf-446e-9c37-e21221668de9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:38.644913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddd659b7-bdbf-446e-9c37-e21221668de9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:38.645648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddd659b7-bdbf-446e-9c37-e21221668de9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:38.646154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddd659b7-bdbf-446e-9c37-e21221668de9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:38.646603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddd659b7-bdbf-446e-9c37-e21221668de9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:38.786748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ddd659b7-bdbf-446e-9c37-e21221668de9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:38.795757 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ddd659b7-bdbf-446e-9c37-e21221668de9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.165142 Sep 27 10:32:38.796715 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2892/6245] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:38 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:39.822554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4ed01b02-2ab1-4ebc-be37-13ba8cf1a788 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:39.823351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4ed01b02-2ab1-4ebc-be37-13ba8cf1a788 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:39.834138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed01b02-2ab1-4ebc-be37-13ba8cf1a788 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:39.834851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed01b02-2ab1-4ebc-be37-13ba8cf1a788 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:39.835505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed01b02-2ab1-4ebc-be37-13ba8cf1a788 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:39.836501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed01b02-2ab1-4ebc-be37-13ba8cf1a788 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:39.837260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed01b02-2ab1-4ebc-be37-13ba8cf1a788 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:39.837887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ed01b02-2ab1-4ebc-be37-13ba8cf1a788 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:39.981508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4ed01b02-2ab1-4ebc-be37-13ba8cf1a788 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:39.990669 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4ed01b02-2ab1-4ebc-be37-13ba8cf1a788 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.173958 Sep 27 10:32:39.991324 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3354/6246] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:39 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:41.018037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4aabe35f-435c-417b-a5ea-2e6061930c4e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:41.018640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4aabe35f-435c-417b-a5ea-2e6061930c4e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:41.026177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aabe35f-435c-417b-a5ea-2e6061930c4e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:41.026710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aabe35f-435c-417b-a5ea-2e6061930c4e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:41.027158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aabe35f-435c-417b-a5ea-2e6061930c4e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:41.027881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aabe35f-435c-417b-a5ea-2e6061930c4e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:41.028361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aabe35f-435c-417b-a5ea-2e6061930c4e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:41.028855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4aabe35f-435c-417b-a5ea-2e6061930c4e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:41.173849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4aabe35f-435c-417b-a5ea-2e6061930c4e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:41.182137 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4aabe35f-435c-417b-a5ea-2e6061930c4e tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.169459 Sep 27 10:32:41.182692 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2893/6247] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:41 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:42.206820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8b46e443-66e1-4741-8eb1-ccb1a572fd80 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:42.207415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8b46e443-66e1-4741-8eb1-ccb1a572fd80 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:42.214709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b46e443-66e1-4741-8eb1-ccb1a572fd80 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:42.215198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b46e443-66e1-4741-8eb1-ccb1a572fd80 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:42.215638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b46e443-66e1-4741-8eb1-ccb1a572fd80 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:42.216346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b46e443-66e1-4741-8eb1-ccb1a572fd80 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:42.216842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b46e443-66e1-4741-8eb1-ccb1a572fd80 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:42.217274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b46e443-66e1-4741-8eb1-ccb1a572fd80 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:42.372876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8b46e443-66e1-4741-8eb1-ccb1a572fd80 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:42.375215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=100 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:42.382659 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8b46e443-66e1-4741-8eb1-ccb1a572fd80 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.180302 Sep 27 10:32:42.383493 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3355/6248] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:42 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:43.407864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-14fc0ef0-d45a-4bed-a41f-1f809e2ca041 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:43.408508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-14fc0ef0-d45a-4bed-a41f-1f809e2ca041 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:43.416124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14fc0ef0-d45a-4bed-a41f-1f809e2ca041 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:43.416681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14fc0ef0-d45a-4bed-a41f-1f809e2ca041 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:43.417197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14fc0ef0-d45a-4bed-a41f-1f809e2ca041 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:43.417982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14fc0ef0-d45a-4bed-a41f-1f809e2ca041 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:43.418453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14fc0ef0-d45a-4bed-a41f-1f809e2ca041 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:43.418896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-14fc0ef0-d45a-4bed-a41f-1f809e2ca041 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:43.421714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:32:43.570786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-14fc0ef0-d45a-4bed-a41f-1f809e2ca041 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:43.579058 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-14fc0ef0-d45a-4bed-a41f-1f809e2ca041 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1697 microversion: 2.1 time: 0.175622 Sep 27 10:32:43.579680 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2894/6249] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:43 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1697 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:44.605925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d950723e-af50-44f8-be56-0480776b21dd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:44.607029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d950723e-af50-44f8-be56-0480776b21dd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:44.615868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d950723e-af50-44f8-be56-0480776b21dd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:44.616703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d950723e-af50-44f8-be56-0480776b21dd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:44.617625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d950723e-af50-44f8-be56-0480776b21dd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:44.618537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d950723e-af50-44f8-be56-0480776b21dd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:44.619336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d950723e-af50-44f8-be56-0480776b21dd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:44.619849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d950723e-af50-44f8-be56-0480776b21dd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:45.123697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d950723e-af50-44f8-be56-0480776b21dd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:45.132394 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d950723e-af50-44f8-be56-0480776b21dd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1696 microversion: 2.1 time: 0.531804 Sep 27 10:32:45.133586 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3356/6250] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:44 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1696 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:45.467594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-46d375b4-9efd-4e2d-a13d-c31279f54460 req-75c38ce6-c51c-49e8-9553-690760e1bc59 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0565ee6c-196a-4ca9-981f-24e019cd89ed", "name": "network-vif-unplugged", "status": "completed", "tag": "dde5d266-f09e-421c-b470-aacdbb240d15"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:45.479346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-46d375b4-9efd-4e2d-a13d-c31279f54460 req-75c38ce6-c51c-49e8-9553-690760e1bc59 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:45.479865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-46d375b4-9efd-4e2d-a13d-c31279f54460 req-75c38ce6-c51c-49e8-9553-690760e1bc59 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:45.480452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-46d375b4-9efd-4e2d-a13d-c31279f54460 req-75c38ce6-c51c-49e8-9553-690760e1bc59 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:45.495265 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-46d375b4-9efd-4e2d-a13d-c31279f54460 req-75c38ce6-c51c-49e8-9553-690760e1bc59 service nova] Creating event network-vif-unplugged:dde5d266-f09e-421c-b470-aacdbb240d15 for instance 0565ee6c-196a-4ca9-981f-24e019cd89ed on nch1-master-1 Sep 27 10:32:45.506971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-46d375b4-9efd-4e2d-a13d-c31279f54460 req-75c38ce6-c51c-49e8-9553-690760e1bc59 service nova] Instance 0565ee6c-196a-4ca9-981f-24e019cd89ed is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:32:45.519890 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-46d375b4-9efd-4e2d-a13d-c31279f54460 req-75c38ce6-c51c-49e8-9553-690760e1bc59 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.056443 Sep 27 10:32:45.521434 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2895/6251] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:32:45 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:32:46.158394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-089a03a0-3967-430e-b0c0-900f7920183a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:46.159059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-089a03a0-3967-430e-b0c0-900f7920183a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:46.167812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-089a03a0-3967-430e-b0c0-900f7920183a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:46.168316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-089a03a0-3967-430e-b0c0-900f7920183a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:46.168792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-089a03a0-3967-430e-b0c0-900f7920183a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:46.169519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-089a03a0-3967-430e-b0c0-900f7920183a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:46.169981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-089a03a0-3967-430e-b0c0-900f7920183a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:46.170651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-089a03a0-3967-430e-b0c0-900f7920183a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:46.420594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-089a03a0-3967-430e-b0c0-900f7920183a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:46.429983 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-089a03a0-3967-430e-b0c0-900f7920183a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1696 microversion: 2.1 time: 0.275686 Sep 27 10:32:46.430992 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3357/6252] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:46 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1696 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:47.462098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-55e3a98d-7652-4fe6-9d14-efb1487f0178 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:47.463353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-55e3a98d-7652-4fe6-9d14-efb1487f0178 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:47.474422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e3a98d-7652-4fe6-9d14-efb1487f0178 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:47.475262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e3a98d-7652-4fe6-9d14-efb1487f0178 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:47.476030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e3a98d-7652-4fe6-9d14-efb1487f0178 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:47.477397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e3a98d-7652-4fe6-9d14-efb1487f0178 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:47.477890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e3a98d-7652-4fe6-9d14-efb1487f0178 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:47.478340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-55e3a98d-7652-4fe6-9d14-efb1487f0178 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:47.545245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-407344af-cb2a-4708-8cbf-ac1316efe4b8 req-8b62f34d-d10c-4fa0-aad3-048bbabef54f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0565ee6c-196a-4ca9-981f-24e019cd89ed", "tag": "dde5d266-f09e-421c-b470-aacdbb240d15"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:47.556678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-407344af-cb2a-4708-8cbf-ac1316efe4b8 req-8b62f34d-d10c-4fa0-aad3-048bbabef54f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:47.557180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-407344af-cb2a-4708-8cbf-ac1316efe4b8 req-8b62f34d-d10c-4fa0-aad3-048bbabef54f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:47.557596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-407344af-cb2a-4708-8cbf-ac1316efe4b8 req-8b62f34d-d10c-4fa0-aad3-048bbabef54f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:47.571009 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-407344af-cb2a-4708-8cbf-ac1316efe4b8 req-8b62f34d-d10c-4fa0-aad3-048bbabef54f service nova] Creating event network-changed:dde5d266-f09e-421c-b470-aacdbb240d15 for instance 0565ee6c-196a-4ca9-981f-24e019cd89ed on nch1-master-1 Sep 27 10:32:47.577039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-407344af-cb2a-4708-8cbf-ac1316efe4b8 req-8b62f34d-d10c-4fa0-aad3-048bbabef54f service nova] Instance 0565ee6c-196a-4ca9-981f-24e019cd89ed is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:32:47.589463 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-407344af-cb2a-4708-8cbf-ac1316efe4b8 req-8b62f34d-d10c-4fa0-aad3-048bbabef54f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048787 Sep 27 10:32:47.590259 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3358/6253] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:32:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:32:47.626638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-55e3a98d-7652-4fe6-9d14-efb1487f0178 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:47.638362 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-55e3a98d-7652-4fe6-9d14-efb1487f0178 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1694 microversion: 2.1 time: 0.184409 Sep 27 10:32:47.639208 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2896/6254] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:47 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1694 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:48.665731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-22eec2ec-5fac-4efd-96c0-c2d76ce31c7a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:48.666445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-22eec2ec-5fac-4efd-96c0-c2d76ce31c7a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:48.673526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22eec2ec-5fac-4efd-96c0-c2d76ce31c7a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:48.674016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22eec2ec-5fac-4efd-96c0-c2d76ce31c7a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:48.674458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22eec2ec-5fac-4efd-96c0-c2d76ce31c7a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:48.675154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22eec2ec-5fac-4efd-96c0-c2d76ce31c7a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:48.675615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22eec2ec-5fac-4efd-96c0-c2d76ce31c7a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:48.676048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22eec2ec-5fac-4efd-96c0-c2d76ce31c7a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:48.818445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-22eec2ec-5fac-4efd-96c0-c2d76ce31c7a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:48.827202 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-22eec2ec-5fac-4efd-96c0-c2d76ce31c7a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1691 microversion: 2.1 time: 0.167295 Sep 27 10:32:48.827793 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3359/6255] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:48 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1691 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:48.850774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-62420acf-9df0-4c42-badb-63a590827256 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:48.851340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-62420acf-9df0-4c42-badb-63a590827256 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:48.859521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62420acf-9df0-4c42-badb-63a590827256 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:48.860059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62420acf-9df0-4c42-badb-63a590827256 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:48.860527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62420acf-9df0-4c42-badb-63a590827256 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:48.861294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62420acf-9df0-4c42-badb-63a590827256 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:48.861886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62420acf-9df0-4c42-badb-63a590827256 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:48.862341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62420acf-9df0-4c42-badb-63a590827256 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:48.995141 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-62420acf-9df0-4c42-badb-63a590827256 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed/action" status: 202 len: 0 microversion: 2.1 time: 0.148120 Sep 27 10:32:48.996033 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2897/6256] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:32:48 2022] POST /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed/action => generated 0 bytes in 150 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:32:49.004279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2540a562-cb2a-420e-a6de-58e0a6aec5d3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:49.004925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2540a562-cb2a-420e-a6de-58e0a6aec5d3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:49.011457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2540a562-cb2a-420e-a6de-58e0a6aec5d3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:49.011946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2540a562-cb2a-420e-a6de-58e0a6aec5d3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:49.012382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2540a562-cb2a-420e-a6de-58e0a6aec5d3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:49.013123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2540a562-cb2a-420e-a6de-58e0a6aec5d3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:49.013587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2540a562-cb2a-420e-a6de-58e0a6aec5d3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:49.014005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2540a562-cb2a-420e-a6de-58e0a6aec5d3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:49.264523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2540a562-cb2a-420e-a6de-58e0a6aec5d3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:49.272824 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2540a562-cb2a-420e-a6de-58e0a6aec5d3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1690 microversion: 2.1 time: 0.272573 Sep 27 10:32:49.273409 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3360/6257] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:48 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1690 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:50.302348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-12604c2b-ff23-4984-9270-f9399abfa81c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:50.303077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-12604c2b-ff23-4984-9270-f9399abfa81c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:50.313976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12604c2b-ff23-4984-9270-f9399abfa81c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:50.314694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12604c2b-ff23-4984-9270-f9399abfa81c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:50.315199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12604c2b-ff23-4984-9270-f9399abfa81c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:50.315946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12604c2b-ff23-4984-9270-f9399abfa81c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:50.316474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12604c2b-ff23-4984-9270-f9399abfa81c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:50.317234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-12604c2b-ff23-4984-9270-f9399abfa81c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:50.477221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-12604c2b-ff23-4984-9270-f9399abfa81c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:50.485557 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-12604c2b-ff23-4984-9270-f9399abfa81c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1690 microversion: 2.1 time: 0.191048 Sep 27 10:32:50.486147 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2898/6258] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:50 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1690 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:51.510367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2dbdb670-33fb-4cca-be05-a0693a2d6636 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:51.510974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2dbdb670-33fb-4cca-be05-a0693a2d6636 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:51.518122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbdb670-33fb-4cca-be05-a0693a2d6636 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:51.518536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbdb670-33fb-4cca-be05-a0693a2d6636 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:51.518973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbdb670-33fb-4cca-be05-a0693a2d6636 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:51.519667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbdb670-33fb-4cca-be05-a0693a2d6636 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:51.520127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbdb670-33fb-4cca-be05-a0693a2d6636 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:51.520550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2dbdb670-33fb-4cca-be05-a0693a2d6636 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:51.679838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2dbdb670-33fb-4cca-be05-a0693a2d6636 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:51.681106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-42342c01-c30a-43dc-8f61-67cdefcd9a6e req-4b608ae0-23a2-438d-b55f-fa0ab4a4b6cf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0565ee6c-196a-4ca9-981f-24e019cd89ed", "name": "network-vif-plugged", "status": "completed", "tag": "dde5d266-f09e-421c-b470-aacdbb240d15"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:51.689383 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2dbdb670-33fb-4cca-be05-a0693a2d6636 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1690 microversion: 2.1 time: 0.183017 Sep 27 10:32:51.689964 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3361/6259] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:51 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1690 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:51.694022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-42342c01-c30a-43dc-8f61-67cdefcd9a6e req-4b608ae0-23a2-438d-b55f-fa0ab4a4b6cf service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:51.701511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-42342c01-c30a-43dc-8f61-67cdefcd9a6e req-4b608ae0-23a2-438d-b55f-fa0ab4a4b6cf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:51.701980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-42342c01-c30a-43dc-8f61-67cdefcd9a6e req-4b608ae0-23a2-438d-b55f-fa0ab4a4b6cf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:51.717754 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-42342c01-c30a-43dc-8f61-67cdefcd9a6e req-4b608ae0-23a2-438d-b55f-fa0ab4a4b6cf service nova] Creating event network-vif-plugged:dde5d266-f09e-421c-b470-aacdbb240d15 for instance 0565ee6c-196a-4ca9-981f-24e019cd89ed on nch1-master-1 Sep 27 10:32:51.725547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-42342c01-c30a-43dc-8f61-67cdefcd9a6e req-4b608ae0-23a2-438d-b55f-fa0ab4a4b6cf service nova] Instance 0565ee6c-196a-4ca9-981f-24e019cd89ed is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:32:51.737225 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-42342c01-c30a-43dc-8f61-67cdefcd9a6e req-4b608ae0-23a2-438d-b55f-fa0ab4a4b6cf service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.150727 Sep 27 10:32:51.737844 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2899/6260] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:32:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:32:52.545599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-95a8c353-11ef-4b65-92f3-a0533f95327e req-3a8d151d-0508-47a5-a62a-a98329ea38c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0565ee6c-196a-4ca9-981f-24e019cd89ed", "name": "network-vif-unplugged", "status": "completed", "tag": "dde5d266-f09e-421c-b470-aacdbb240d15"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:52.555503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-95a8c353-11ef-4b65-92f3-a0533f95327e req-3a8d151d-0508-47a5-a62a-a98329ea38c5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:52.555989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-95a8c353-11ef-4b65-92f3-a0533f95327e req-3a8d151d-0508-47a5-a62a-a98329ea38c5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:52.556507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-95a8c353-11ef-4b65-92f3-a0533f95327e req-3a8d151d-0508-47a5-a62a-a98329ea38c5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:52.569236 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-95a8c353-11ef-4b65-92f3-a0533f95327e req-3a8d151d-0508-47a5-a62a-a98329ea38c5 service nova] Creating event network-vif-unplugged:dde5d266-f09e-421c-b470-aacdbb240d15 for instance 0565ee6c-196a-4ca9-981f-24e019cd89ed on nch2-master-1 Sep 27 10:32:52.575125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-95a8c353-11ef-4b65-92f3-a0533f95327e req-3a8d151d-0508-47a5-a62a-a98329ea38c5 service nova] Instance 0565ee6c-196a-4ca9-981f-24e019cd89ed is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:32:52.586353 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-95a8c353-11ef-4b65-92f3-a0533f95327e req-3a8d151d-0508-47a5-a62a-a98329ea38c5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.112834 Sep 27 10:32:52.587166 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3362/6261] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:32:52 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:32:52.715383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cdb52183-f4d0-45a9-8a59-7650cdc0d0e7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:52.716028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cdb52183-f4d0-45a9-8a59-7650cdc0d0e7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:52.723886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb52183-f4d0-45a9-8a59-7650cdc0d0e7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:52.724390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb52183-f4d0-45a9-8a59-7650cdc0d0e7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:52.724881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb52183-f4d0-45a9-8a59-7650cdc0d0e7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:52.725614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb52183-f4d0-45a9-8a59-7650cdc0d0e7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:52.726093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb52183-f4d0-45a9-8a59-7650cdc0d0e7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:52.726542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cdb52183-f4d0-45a9-8a59-7650cdc0d0e7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:52.945732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cdb52183-f4d0-45a9-8a59-7650cdc0d0e7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:52.955193 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cdb52183-f4d0-45a9-8a59-7650cdc0d0e7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1690 microversion: 2.1 time: 0.244492 Sep 27 10:32:52.955760 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2900/6262] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:52 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1690 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:53.982859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8355981-cd76-4c40-9762-9b06608921f5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:53.983578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c8355981-cd76-4c40-9762-9b06608921f5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:53.991110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8355981-cd76-4c40-9762-9b06608921f5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:53.991590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8355981-cd76-4c40-9762-9b06608921f5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:53.992030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8355981-cd76-4c40-9762-9b06608921f5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:53.992757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8355981-cd76-4c40-9762-9b06608921f5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:53.993218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8355981-cd76-4c40-9762-9b06608921f5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:53.993647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8355981-cd76-4c40-9762-9b06608921f5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:54.126903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c8355981-cd76-4c40-9762-9b06608921f5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:54.135561 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c8355981-cd76-4c40-9762-9b06608921f5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1690 microversion: 2.1 time: 0.159501 Sep 27 10:32:54.136136 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3363/6263] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:53 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1690 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:54.602520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-308b1f82-11d9-4deb-bff7-ceec1c445c95 req-522a66c8-4037-4777-a0ce-b95b9da66f9c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0565ee6c-196a-4ca9-981f-24e019cd89ed", "tag": "dde5d266-f09e-421c-b470-aacdbb240d15"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:32:54.613654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-308b1f82-11d9-4deb-bff7-ceec1c445c95 req-522a66c8-4037-4777-a0ce-b95b9da66f9c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:54.614303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-308b1f82-11d9-4deb-bff7-ceec1c445c95 req-522a66c8-4037-4777-a0ce-b95b9da66f9c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:54.614856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-308b1f82-11d9-4deb-bff7-ceec1c445c95 req-522a66c8-4037-4777-a0ce-b95b9da66f9c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:54.628008 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-308b1f82-11d9-4deb-bff7-ceec1c445c95 req-522a66c8-4037-4777-a0ce-b95b9da66f9c service nova] Creating event network-changed:dde5d266-f09e-421c-b470-aacdbb240d15 for instance 0565ee6c-196a-4ca9-981f-24e019cd89ed on nch2-master-1 Sep 27 10:32:54.633951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-308b1f82-11d9-4deb-bff7-ceec1c445c95 req-522a66c8-4037-4777-a0ce-b95b9da66f9c service nova] Instance 0565ee6c-196a-4ca9-981f-24e019cd89ed is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:32:54.644758 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-308b1f82-11d9-4deb-bff7-ceec1c445c95 req-522a66c8-4037-4777-a0ce-b95b9da66f9c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046489 Sep 27 10:32:54.645347 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2901/6264] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:32:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:32:55.159988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed045e15-e012-426b-9faa-99fb9ba03790 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:55.160563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed045e15-e012-426b-9faa-99fb9ba03790 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:55.168229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed045e15-e012-426b-9faa-99fb9ba03790 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:55.168762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed045e15-e012-426b-9faa-99fb9ba03790 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:55.169206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed045e15-e012-426b-9faa-99fb9ba03790 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:55.169940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed045e15-e012-426b-9faa-99fb9ba03790 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:55.170407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed045e15-e012-426b-9faa-99fb9ba03790 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:55.170852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed045e15-e012-426b-9faa-99fb9ba03790 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:55.463969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ed045e15-e012-426b-9faa-99fb9ba03790 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:55.474029 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed045e15-e012-426b-9faa-99fb9ba03790 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1690 microversion: 2.1 time: 0.318182 Sep 27 10:32:55.474688 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3364/6265] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:55 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1690 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:32:56.504584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1a115111-9eb8-461f-bf88-80cea0eef4ed tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:56.505828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1a115111-9eb8-461f-bf88-80cea0eef4ed tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:56.514423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a115111-9eb8-461f-bf88-80cea0eef4ed tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:56.514958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a115111-9eb8-461f-bf88-80cea0eef4ed tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:56.515412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a115111-9eb8-461f-bf88-80cea0eef4ed tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:56.516133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a115111-9eb8-461f-bf88-80cea0eef4ed tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:56.516611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a115111-9eb8-461f-bf88-80cea0eef4ed tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:56.517098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a115111-9eb8-461f-bf88-80cea0eef4ed tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:56.658132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1a115111-9eb8-461f-bf88-80cea0eef4ed tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:56.666384 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1a115111-9eb8-461f-bf88-80cea0eef4ed tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1690 microversion: 2.1 time: 0.170433 Sep 27 10:32:56.666954 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2902/6266] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:56 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1690 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:32:57.692705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ecf1106e-3cac-44af-9c26-871cc1e711e5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:32:57.693451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ecf1106e-3cac-44af-9c26-871cc1e711e5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:32:57.702399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecf1106e-3cac-44af-9c26-871cc1e711e5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:57.703040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecf1106e-3cac-44af-9c26-871cc1e711e5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:57.703582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecf1106e-3cac-44af-9c26-871cc1e711e5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:57.704494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecf1106e-3cac-44af-9c26-871cc1e711e5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:32:57.705148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecf1106e-3cac-44af-9c26-871cc1e711e5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:32:57.705678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecf1106e-3cac-44af-9c26-871cc1e711e5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:32:57.842334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ecf1106e-3cac-44af-9c26-871cc1e711e5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:32:57.851010 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ecf1106e-3cac-44af-9c26-871cc1e711e5 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1683 microversion: 2.1 time: 0.163459 Sep 27 10:32:57.851606 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3365/6267] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:32:57 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1683 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:00.831719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7c60d1da-5c11-4222-a8c9-a6f3d3aa4893 req-ca0af58f-9573-41d5-9540-4ca1c95db7fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0565ee6c-196a-4ca9-981f-24e019cd89ed", "name": "network-vif-plugged", "status": "completed", "tag": "dde5d266-f09e-421c-b470-aacdbb240d15"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:00.842780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7c60d1da-5c11-4222-a8c9-a6f3d3aa4893 req-ca0af58f-9573-41d5-9540-4ca1c95db7fd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:00.843274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7c60d1da-5c11-4222-a8c9-a6f3d3aa4893 req-ca0af58f-9573-41d5-9540-4ca1c95db7fd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:00.843779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7c60d1da-5c11-4222-a8c9-a6f3d3aa4893 req-ca0af58f-9573-41d5-9540-4ca1c95db7fd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:00.856338 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7c60d1da-5c11-4222-a8c9-a6f3d3aa4893 req-ca0af58f-9573-41d5-9540-4ca1c95db7fd service nova] Creating event network-vif-plugged:dde5d266-f09e-421c-b470-aacdbb240d15 for instance 0565ee6c-196a-4ca9-981f-24e019cd89ed on nch2-master-1 Sep 27 10:33:00.861454 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7c60d1da-5c11-4222-a8c9-a6f3d3aa4893 req-ca0af58f-9573-41d5-9540-4ca1c95db7fd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033486 Sep 27 10:33:00.861991 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2903/6268] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:33:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:13.528376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=27 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:13.529219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=57 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:13.587904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-049dc1f2-ecb6-48af-a437-e8f346f28283 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:13.588992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-049dc1f2-ecb6-48af-a437-e8f346f28283 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:13.595803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-049dc1f2-ecb6-48af-a437-e8f346f28283 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:13.596298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-049dc1f2-ecb6-48af-a437-e8f346f28283 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:13.596775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-049dc1f2-ecb6-48af-a437-e8f346f28283 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:13.597490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-049dc1f2-ecb6-48af-a437-e8f346f28283 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:13.597951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-049dc1f2-ecb6-48af-a437-e8f346f28283 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:13.598380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-049dc1f2-ecb6-48af-a437-e8f346f28283 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:13.629522 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-049dc1f2-ecb6-48af-a437-e8f346f28283 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/ips" status: 200 len: 151 microversion: 2.1 time: 0.108934 Sep 27 10:33:13.630090 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3366/6269] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:33:13 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/ips => generated 151 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:15.001844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=27 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:15.002628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=57,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:15.007808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3db5609-ad35-4c28-ace8-3b6541f6b176 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "80fa90a7-dc37-41b3-9de3-7d7c37cc72e1"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:15.010850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e3db5609-ad35-4c28-ace8-3b6541f6b176 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:15.018597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3db5609-ad35-4c28-ace8-3b6541f6b176 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:15.019311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3db5609-ad35-4c28-ace8-3b6541f6b176 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:15.019771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3db5609-ad35-4c28-ace8-3b6541f6b176 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:15.020498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3db5609-ad35-4c28-ace8-3b6541f6b176 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:15.021016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3db5609-ad35-4c28-ace8-3b6541f6b176 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:15.021483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3db5609-ad35-4c28-ace8-3b6541f6b176 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:15.299205 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e3db5609-ad35-4c28-ace8-3b6541f6b176 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "POST /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.299048 Sep 27 10:33:15.299829 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2904/6270] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:33:14 2022] POST /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/os-volume_attachments => generated 194 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:18.722371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:18.722969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:18.730910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:18.731408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:18.731834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:18.732577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:18.733078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:18.733516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:18.822643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lazy-loading 'services' on Instance uuid e73482b8-b537-4e85-9442-e35475931895 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:33:18.904723 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a7ced177-8d2c-46cf-b116-ffd4e4bca8c2 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/os-volume_attachments/80fa90a7-dc37-41b3-9de3-7d7c37cc72e1" status: 202 len: 0 microversion: 2.1 time: 0.186249 Sep 27 10:33:18.905326 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3367/6271] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:33:18 2022] DELETE /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/os-volume_attachments/80fa90a7-dc37-41b3-9de3-7d7c37cc72e1 => generated 0 bytes in 188 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:33:18.914641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-891f70b4-8104-4655-ad7d-e18cce4cee18 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:18.917371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-891f70b4-8104-4655-ad7d-e18cce4cee18 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:18.924985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-891f70b4-8104-4655-ad7d-e18cce4cee18 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:18.925549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-891f70b4-8104-4655-ad7d-e18cce4cee18 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:18.926034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-891f70b4-8104-4655-ad7d-e18cce4cee18 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:18.926805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-891f70b4-8104-4655-ad7d-e18cce4cee18 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:18.927354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-891f70b4-8104-4655-ad7d-e18cce4cee18 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:18.927790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-891f70b4-8104-4655-ad7d-e18cce4cee18 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:18.959480 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-891f70b4-8104-4655-ad7d-e18cce4cee18 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049153 Sep 27 10:33:18.960937 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2905/6272] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:33:18 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:19.973658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-109d902e-001d-4c35-ab24-75561598cd87 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:19.976094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-109d902e-001d-4c35-ab24-75561598cd87 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:19.983341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-109d902e-001d-4c35-ab24-75561598cd87 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:19.983833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-109d902e-001d-4c35-ab24-75561598cd87 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:19.984277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-109d902e-001d-4c35-ab24-75561598cd87 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:19.985026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-109d902e-001d-4c35-ab24-75561598cd87 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:19.985492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-109d902e-001d-4c35-ab24-75561598cd87 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:19.985935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-109d902e-001d-4c35-ab24-75561598cd87 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:20.016512 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-109d902e-001d-4c35-ab24-75561598cd87 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048213 Sep 27 10:33:20.017113 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3368/6273] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:33:19 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:33:20.083100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4ee5e028-5768-41e8-a09c-05f9360f0503 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:20.083651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4ee5e028-5768-41e8-a09c-05f9360f0503 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:20.090967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ee5e028-5768-41e8-a09c-05f9360f0503 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:20.091476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ee5e028-5768-41e8-a09c-05f9360f0503 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:20.091931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ee5e028-5768-41e8-a09c-05f9360f0503 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:20.093057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ee5e028-5768-41e8-a09c-05f9360f0503 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:20.093527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ee5e028-5768-41e8-a09c-05f9360f0503 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:20.093972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4ee5e028-5768-41e8-a09c-05f9360f0503 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:20.120484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4ee5e028-5768-41e8-a09c-05f9360f0503 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:33:20.208050 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4ee5e028-5768-41e8-a09c-05f9360f0503 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 204 len: 0 microversion: 2.1 time: 0.128538 Sep 27 10:33:20.208617 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2906/6274] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:33:20 2022] DELETE /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:33:20.217462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-999fe7a3-8d55-49b6-b49b-ba28b35f2e8c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:20.218090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-999fe7a3-8d55-49b6-b49b-ba28b35f2e8c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:20.224478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-999fe7a3-8d55-49b6-b49b-ba28b35f2e8c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:20.225022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-999fe7a3-8d55-49b6-b49b-ba28b35f2e8c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:20.225471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-999fe7a3-8d55-49b6-b49b-ba28b35f2e8c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:20.226178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-999fe7a3-8d55-49b6-b49b-ba28b35f2e8c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:20.226642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-999fe7a3-8d55-49b6-b49b-ba28b35f2e8c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:20.227080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-999fe7a3-8d55-49b6-b49b-ba28b35f2e8c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:20.361367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-999fe7a3-8d55-49b6-b49b-ba28b35f2e8c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:20.369612 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-999fe7a3-8d55-49b6-b49b-ba28b35f2e8c tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1682 microversion: 2.1 time: 0.156508 Sep 27 10:33:20.370209 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3369/6275] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:20 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1682 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:21.399343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3f43d3a4-41fb-4ed1-8435-ac4fb4923240 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:21.400522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3f43d3a4-41fb-4ed1-8435-ac4fb4923240 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:21.412436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f43d3a4-41fb-4ed1-8435-ac4fb4923240 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:21.413319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f43d3a4-41fb-4ed1-8435-ac4fb4923240 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:21.414079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f43d3a4-41fb-4ed1-8435-ac4fb4923240 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:21.415259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f43d3a4-41fb-4ed1-8435-ac4fb4923240 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:21.416051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f43d3a4-41fb-4ed1-8435-ac4fb4923240 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:21.416928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3f43d3a4-41fb-4ed1-8435-ac4fb4923240 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:21.564903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3f43d3a4-41fb-4ed1-8435-ac4fb4923240 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:21.573662 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3f43d3a4-41fb-4ed1-8435-ac4fb4923240 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1682 microversion: 2.1 time: 0.181756 Sep 27 10:33:21.574261 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2907/6276] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:21 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1682 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:22.597221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8535e1f7-c9ca-4b7f-b65a-6df312e251b6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:22.597866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8535e1f7-c9ca-4b7f-b65a-6df312e251b6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:22.606128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8535e1f7-c9ca-4b7f-b65a-6df312e251b6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:22.606630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8535e1f7-c9ca-4b7f-b65a-6df312e251b6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:22.607103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8535e1f7-c9ca-4b7f-b65a-6df312e251b6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:22.607827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8535e1f7-c9ca-4b7f-b65a-6df312e251b6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:22.608284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8535e1f7-c9ca-4b7f-b65a-6df312e251b6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:22.608704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8535e1f7-c9ca-4b7f-b65a-6df312e251b6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:22.743918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8535e1f7-c9ca-4b7f-b65a-6df312e251b6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:22.752125 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8535e1f7-c9ca-4b7f-b65a-6df312e251b6 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1682 microversion: 2.1 time: 0.158894 Sep 27 10:33:22.752720 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3370/6277] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:22 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1682 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:23.130599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e0de538f-c639-47b6-bbb5-9c374233e3b9 req-ef7d364e-6282-4e30-9bba-31564607fdcc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e73482b8-b537-4e85-9442-e35475931895", "name": "network-vif-deleted", "tag": "fe5983cb-efc0-4737-88bd-44a4e18e8c6f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:23.141136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e0de538f-c639-47b6-bbb5-9c374233e3b9 req-ef7d364e-6282-4e30-9bba-31564607fdcc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:23.141627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e0de538f-c639-47b6-bbb5-9c374233e3b9 req-ef7d364e-6282-4e30-9bba-31564607fdcc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:23.142264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e0de538f-c639-47b6-bbb5-9c374233e3b9 req-ef7d364e-6282-4e30-9bba-31564607fdcc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:23.154055 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e0de538f-c639-47b6-bbb5-9c374233e3b9 req-ef7d364e-6282-4e30-9bba-31564607fdcc service nova] Creating event network-vif-deleted:fe5983cb-efc0-4737-88bd-44a4e18e8c6f for instance e73482b8-b537-4e85-9442-e35475931895 on nch2-master-1 Sep 27 10:33:23.164236 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e0de538f-c639-47b6-bbb5-9c374233e3b9 req-ef7d364e-6282-4e30-9bba-31564607fdcc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037269 Sep 27 10:33:23.169233 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2908/6278] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:33:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:23.775557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:23.777030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:23.784577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:23.785104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:23.785546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:23.786269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:23.786752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:23.787245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:23.829899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:23.830393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:23.830826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:23.831285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lazy-loading 'fault' on Instance uuid e73482b8-b537-4e85-9442-e35475931895 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:33:23.880541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '64090c8fd3394980ad5409adcde4986f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '90c0fbb4a6d84767b920564584715f9d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:23.888942 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a627b3f9-83e4-4708-85dd-64fb21c6c0eb tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 200 len: 1332 microversion: 2.1 time: 0.117234 Sep 27 10:33:23.889618 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3371/6279] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:23 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 1332 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:24.920575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:24.921854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:24.928815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:24.929314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:24.929768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:24.930518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:24.930994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:24.931432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:24.945875 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 HTTP/1.1 Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'e73482b8-b537-4e85-9442-e35475931895'} Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:24.947566 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:24.953567 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:24.953567 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:24.953567 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:24.953567 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:24.953567 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:24.953567 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:24.953567 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:24.953567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:24.953567 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-00ca82d1-9d53-4868-b494-09af92b1aee1 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 404 len: 111 microversion: 2.1 time: 0.034475 Sep 27 10:33:24.953567 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2909/6280] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:24 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:24.959142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9853bb10-c06a-49e9-94bb-40ed970ce12f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '.version_select at 0x7f7f6267bd30>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:24.983766 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9853bb10-c06a-49e9-94bb-40ed970ce12f tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1852512552" status: 202 len: 0 microversion: 2.1 time: 0.028400 Sep 27 10:33:24.984359 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3372/6281] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:33:24 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1852512552 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:33:25.774283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:25.775157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:25.778116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c4b46958-623c-4eec-baaa-3d1f851ac015 req-40937a38-fe78-49d2-a5ab-fd2633676766 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a43e72b9-7453-4f8c-a959-8a5a38d3e9f3", "tag": "0f2d0535-c751-4347-b21f-ee095b753310"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:25.786253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:25.787097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:25.787772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:25.788769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:25.789432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:25.790210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:25.791446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c4b46958-623c-4eec-baaa-3d1f851ac015 req-40937a38-fe78-49d2-a5ab-fd2633676766 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:25.792187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c4b46958-623c-4eec-baaa-3d1f851ac015 req-40937a38-fe78-49d2-a5ab-fd2633676766 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:25.792644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c4b46958-623c-4eec-baaa-3d1f851ac015 req-40937a38-fe78-49d2-a5ab-fd2633676766 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:25.811507 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c4b46958-623c-4eec-baaa-3d1f851ac015 req-40937a38-fe78-49d2-a5ab-fd2633676766 service nova] Creating event network-changed:0f2d0535-c751-4347-b21f-ee095b753310 for instance a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 on nch1-master-1 Sep 27 10:33:25.821552 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c4b46958-623c-4eec-baaa-3d1f851ac015 req-40937a38-fe78-49d2-a5ab-fd2633676766 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047144 Sep 27 10:33:25.822528 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3373/6282] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:33:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:33:25.830702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:33:25.977508 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5cbf0438-8cb6-462d-b75d-1da608b35fd0 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "DELETE /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 204 len: 0 microversion: 2.1 time: 0.207161 Sep 27 10:33:25.979147 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2910/6283] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:33:25 2022] DELETE /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:33:25.989706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-318b303e-ce99-45be-bee8-ecc13d10a641 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:25.991142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-318b303e-ce99-45be-bee8-ecc13d10a641 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:25.999868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-318b303e-ce99-45be-bee8-ecc13d10a641 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:26.000281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-318b303e-ce99-45be-bee8-ecc13d10a641 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:26.000828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-318b303e-ce99-45be-bee8-ecc13d10a641 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:26.001555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-318b303e-ce99-45be-bee8-ecc13d10a641 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:26.002019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-318b303e-ce99-45be-bee8-ecc13d10a641 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:26.002447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-318b303e-ce99-45be-bee8-ecc13d10a641 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:26.178997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-318b303e-ce99-45be-bee8-ecc13d10a641 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:26.187225 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-318b303e-ce99-45be-bee8-ecc13d10a641 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1575 microversion: 2.1 time: 0.202198 Sep 27 10:33:26.188357 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3374/6284] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:25 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1575 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:26.982544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:26.983164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:26.991110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:26.991614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:26.992071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:26.992828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:26.993297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:26.993740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:27.008177 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 HTTP/1.1 Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'e73482b8-b537-4e85-9442-e35475931895'} Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:27.009926 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-da3fd778-12cd-4092-8c63-f762872f4453 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 404 len: 111 microversion: 2.1 time: 0.034381 Sep 27 10:33:27.015089 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2911/6285] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:33:26 2022] DELETE /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:27.020878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:27.021472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: e73482b8-b537-4e85-9442-e35475931895] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:27.028946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:27.029875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:27.030318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:27.031121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:27.031470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:27.031897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:27.046562 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 HTTP/1.1 Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'e73482b8-b537-4e85-9442-e35475931895'} Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:27.048282 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:27.053870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:27.053870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:27.053870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:27.053870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:27.053870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:27.053870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. Sep 27 10:33:27.053870 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:27.053870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance e73482b8-b537-4e85-9442-e35475931895 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:27.053870 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e90f9e81-4f2c-4e7d-86da-939ff196820e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895" status: 404 len: 111 microversion: 2.1 time: 0.032946 Sep 27 10:33:27.053870 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3375/6286] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:27 2022] GET /compute/v2.1/servers/e73482b8-b537-4e85-9442-e35475931895 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:27.215128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ffbc1f67-0557-4459-ad3d-8082b614f550 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:27.215907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ffbc1f67-0557-4459-ad3d-8082b614f550 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:27.222694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffbc1f67-0557-4459-ad3d-8082b614f550 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:27.223302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffbc1f67-0557-4459-ad3d-8082b614f550 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:27.223826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffbc1f67-0557-4459-ad3d-8082b614f550 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:27.224616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffbc1f67-0557-4459-ad3d-8082b614f550 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:27.225233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffbc1f67-0557-4459-ad3d-8082b614f550 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:27.225853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffbc1f67-0557-4459-ad3d-8082b614f550 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:27.370073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ffbc1f67-0557-4459-ad3d-8082b614f550 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:27.379319 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ffbc1f67-0557-4459-ad3d-8082b614f550 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1575 microversion: 2.1 time: 0.168967 Sep 27 10:33:27.379905 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2912/6287] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:27 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1575 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:28.029935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e2fce644-cc5b-4700-893a-010150cee13d req-140ae7e4-b675-40eb-85bc-c4b796224650 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a43e72b9-7453-4f8c-a959-8a5a38d3e9f3", "name": "network-vif-deleted", "tag": "0f2d0535-c751-4347-b21f-ee095b753310"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:28.039356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e2fce644-cc5b-4700-893a-010150cee13d req-140ae7e4-b675-40eb-85bc-c4b796224650 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.039882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e2fce644-cc5b-4700-893a-010150cee13d req-140ae7e4-b675-40eb-85bc-c4b796224650 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.040324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e2fce644-cc5b-4700-893a-010150cee13d req-140ae7e4-b675-40eb-85bc-c4b796224650 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.052513 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e2fce644-cc5b-4700-893a-010150cee13d req-140ae7e4-b675-40eb-85bc-c4b796224650 service nova] Creating event network-vif-deleted:0f2d0535-c751-4347-b21f-ee095b753310 for instance a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 on nch1-master-1 Sep 27 10:33:28.057829 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e2fce644-cc5b-4700-893a-010150cee13d req-140ae7e4-b675-40eb-85bc-c4b796224650 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.098891 Sep 27 10:33:28.058436 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3376/6288] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:33:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:28.204378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:28.205384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:28.212627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.213148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.213663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.214812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.216067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.217306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.228343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-603c151d-8628-4a8d-9bf1-18805cfe9928 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:28.228969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-603c151d-8628-4a8d-9bf1-18805cfe9928 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:28.235869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-603c151d-8628-4a8d-9bf1-18805cfe9928 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.237075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-603c151d-8628-4a8d-9bf1-18805cfe9928 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.237413 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:33:28.237996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-603c151d-8628-4a8d-9bf1-18805cfe9928 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.238974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-603c151d-8628-4a8d-9bf1-18805cfe9928 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 HTTP/1.1 Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '15bd0cb2-4701-4992-9f64-895716df0533'} Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:28.239109 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8fb98f01-ba46-4e4e-99da-8de77ba8dc1d tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 404 len: 111 microversion: 2.1 time: 0.042154 Sep 27 10:33:28.248689 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2913/6289] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:33:28 2022] DELETE /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:28.251274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-603c151d-8628-4a8d-9bf1-18805cfe9928 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.251274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-603c151d-8628-4a8d-9bf1-18805cfe9928 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.261674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:28.263127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 15bd0cb2-4701-4992-9f64-895716df0533] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:28.270858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.271572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.272238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.273072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.273645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.274163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.296996 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 HTTP/1.1 Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '15bd0cb2-4701-4992-9f64-895716df0533'} Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:28.298745 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.303514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.303514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:28.303514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:28.303514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:28.303514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:28.303514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. Sep 27 10:33:28.303514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.307381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance 15bd0cb2-4701-4992-9f64-895716df0533 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:28.308854 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-71548800-60e8-4f74-bbcf-20097c0f4c43 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533" status: 404 len: 111 microversion: 2.1 time: 0.062110 Sep 27 10:33:28.310587 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2914/6290] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:28 2022] GET /compute/v2.1/servers/15bd0cb2-4701-4992-9f64-895716df0533 => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:33:28.317964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:28.319135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:28.325778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.326420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.327004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.327794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.328491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.329319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.342895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-603c151d-8628-4a8d-9bf1-18805cfe9928 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lazy-loading 'services' on Instance uuid dc9ddb88-bfb7-40d7-b615-9755d1e824d5 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:33:28.345168 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 HTTP/1.1 Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'd5319326-fc6f-4d93-85ae-98a8b422dc14'} Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:28.346790 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a63f11d6-3a97-46f1-9a37-6570c9910c4e tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 404 len: 111 microversion: 2.1 time: 0.034186 Sep 27 10:33:28.351360 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2915/6291] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:33:28 2022] DELETE /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:33:28.356946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:28.357577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: d5319326-fc6f-4d93-85ae-98a8b422dc14] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:28.363933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.364435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.364919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.365653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.366127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.366626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.381340 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 HTTP/1.1 Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'd5319326-fc6f-4d93-85ae-98a8b422dc14'} Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:28.382918 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.391363 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:28.391363 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:28.391363 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:28.391363 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:28.391363 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:28.391363 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. Sep 27 10:33:28.391363 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:28.391363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance d5319326-fc6f-4d93-85ae-98a8b422dc14 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:28.391363 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1799fab5-9640-4e69-a489-515de290c68a tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14" status: 404 len: 111 microversion: 2.1 time: 0.031322 Sep 27 10:33:28.391363 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2916/6292] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:28 2022] GET /compute/v2.1/servers/d5319326-fc6f-4d93-85ae-98a8b422dc14 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:33:28.406635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:28.407548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:28.415803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.415993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.416377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.418055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.418542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.418987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.437245 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-603c151d-8628-4a8d-9bf1-18805cfe9928 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "DELETE /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-volume_attachments/a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3" status: 202 len: 0 microversion: 2.1 time: 0.213032 Sep 27 10:33:28.438258 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3377/6293] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:33:28 2022] DELETE /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-volume_attachments/a4827cab-0b00-4b0c-9cc5-1bd4a614c5f3 => generated 0 bytes in 215 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:33:28.451249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-57ae4bc7-82d3-4297-b624-aa52e6ef9d83 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:28.454322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-57ae4bc7-82d3-4297-b624-aa52e6ef9d83 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:28.462144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57ae4bc7-82d3-4297-b624-aa52e6ef9d83 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.462788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57ae4bc7-82d3-4297-b624-aa52e6ef9d83 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.463283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57ae4bc7-82d3-4297-b624-aa52e6ef9d83 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.464404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57ae4bc7-82d3-4297-b624-aa52e6ef9d83 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.465312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57ae4bc7-82d3-4297-b624-aa52e6ef9d83 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.465901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57ae4bc7-82d3-4297-b624-aa52e6ef9d83 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.478251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:28.478927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:28.479340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:28.479894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lazy-loading 'fault' on Instance uuid a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:33:28.527129 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-57ae4bc7-82d3-4297-b624-aa52e6ef9d83 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083451 Sep 27 10:33:28.528232 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3378/6294] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:33:28 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:33:28.532909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:28.544704 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c92b029e-2017-4db9-a4db-79dee76be5ac tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 200 len: 1228 microversion: 2.1 time: 0.145030 Sep 27 10:33:28.555348 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2917/6295] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:28 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 1228 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:33:29.540985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f2faa311-ef73-4287-b389-4fea9716feda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:29.543378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f2faa311-ef73-4287-b389-4fea9716feda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:29.551302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2faa311-ef73-4287-b389-4fea9716feda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.551792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2faa311-ef73-4287-b389-4fea9716feda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.552243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2faa311-ef73-4287-b389-4fea9716feda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.552993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2faa311-ef73-4287-b389-4fea9716feda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.553457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2faa311-ef73-4287-b389-4fea9716feda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.553890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2faa311-ef73-4287-b389-4fea9716feda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.567631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:29.568275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: a43e72b9-7453-4f8c-a959-8a5a38d3e9f3] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:29.575876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.576307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.577008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.577940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.578404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.578817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.588102 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f2faa311-ef73-4287-b389-4fea9716feda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051269 Sep 27 10:33:29.588816 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3379/6296] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:33:29 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:33:29.594207 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 could not be found. Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 HTTP/1.1 Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 could not be found. Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'a43e72b9-7453-4f8c-a959-8a5a38d3e9f3'} Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 could not be found. Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 could not be found. Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:29.595723 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.602009 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.602009 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:29.602009 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:29.602009 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:29.602009 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:29.602009 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 could not be found. Sep 27 10:33:29.602009 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.602009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:29.602009 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c1f948ea-808c-4289-8b9c-048dc3484868 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3" status: 404 len: 111 microversion: 2.1 time: 0.033601 Sep 27 10:33:29.602009 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2918/6297] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:29 2022] GET /compute/v2.1/servers/a43e72b9-7453-4f8c-a959-8a5a38d3e9f3 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:29.616800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:29.617191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:29.624443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.625032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.625737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.626591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.627060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.627493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.642957 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 HTTP/1.1 Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '403bfd67-4e7f-4918-baf9-4d14086a4598'} Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:29.645035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:29.650788 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:33:29.650788 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.650788 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:29.650788 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.650788 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.650788 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:29.650788 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:29.650788 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:29.650788 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:29.650788 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:33:29.650788 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.652804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:29.653879 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fe51b994-38e7-4bf6-801b-99b34312c0ee tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "DELETE /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 404 len: 111 microversion: 2.1 time: 0.041860 Sep 27 10:33:29.654461 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3380/6298] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:33:29 2022] DELETE /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:29.663262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:29.663962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] [instance: 403bfd67-4e7f-4918-baf9-4d14086a4598] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:29.671214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.671712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.672155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.673119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.674709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.675153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.690587 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] HTTP exception thrown: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:33:29.691856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Request method failure captured: Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 HTTP/1.1 Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs_6pZCSLQ9qdSIxbmw159b2wqZRIYpQCjYhvF_TWsW4JGcdIFRyZfU0tZVWA_OD0EofhByBwWQc_6lKfQbDA3RWo7LzHLqzw7S9V7wXo9-9CcXbUBDTVncKOnCknJ-WPkib_axB2K4_HpryHqNErNiqaGqJEL83XlDDPCbKBkR6-cU Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/90c0fbb4a6d84767b920564584715f9d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_90c0fbb4a6d84767b920564584715f9d"}]}] Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 90c0fbb4a6d84767b920564584715f9d Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1091756803 Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 64090c8fd3394980ad5409adcde4986f Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-AttachVolumeNegativeTest-1091756803-project Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '403bfd67-4e7f-4918-baf9-4d14086a4598'} Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:29.693228 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.702806 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.702806 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:29.702806 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:29.702806 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:29.702806 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:29.702806 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. Sep 27 10:33:29.702806 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.702806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] Returning 404 to user: Instance 403bfd67-4e7f-4918-baf9-4d14086a4598 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:29.702806 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c767a619-a683-4d5d-9edc-137ba136ed74 tempest-AttachVolumeNegativeTest-1091756803 tempest-AttachVolumeNegativeTest-1091756803-project] 10.222.0.26 "GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598" status: 404 len: 111 microversion: 2.1 time: 0.035939 Sep 27 10:33:29.704901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:29.705215 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2919/6299] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:29 2022] GET /compute/v2.1/servers/403bfd67-4e7f-4918-baf9-4d14086a4598 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:29.706158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.706306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.706984 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.708531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.709195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.709803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.749679 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] HTTP exception thrown: Cannot 'reboot' instance 6537d086-a852-4b27-a032-807cecd97b86 while it is in vm_state rescued Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Request method failure captured: Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86/action HTTP/1.1 Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 28 Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtB5lZw26WfhuIIxDLfEGtJx8GdKac9mUecPJWt47RWTVgFEQNt2CRnQRU62Kxhrd_5bXczg0Q9QWsT4c85-tEEw3L_ZoXg7ttqo2fTxJvLnAMIld1Q9kkM9ItLXE6S_aDTtz0h96WsfSIhdls5sa9AFKLBsHqf09n27PObWbpVe7TQ Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_40247edd5a444fd8b1c937c13d39393a"}]}] Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6771df45b4ee4426b2f76a09eab2998a Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: {"reboot": {"type": "HARD"}} Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: exception: Cannot 'reboot' instance 6537d086-a852-4b27-a032-807cecd97b86 while it is in vm_state rescued Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '6537d086-a852-4b27-a032-807cecd97b86', 'body': {'reboot': {'type': 'HARD'}}} Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1036, in _action_reboot Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.reboot(context, instance, reboot_type) Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3552, in reboot Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._hard_reboot(context, instance) Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 6537d086-a852-4b27-a032-807cecd97b86 in vm_state rescued. Cannot _hard_reboot while the instance is in this state. Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1040, in _action_reboot Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:33:29.751881 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'reboot' instance 6537d086-a852-4b27-a032-807cecd97b86 while it is in vm_state rescued Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'reboot' instance 6537d086-a852-4b27-a032-807cecd97b86 while it is in vm_state rescued Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Returning 409 to user: Cannot 'reboot' instance 6537d086-a852-4b27-a032-807cecd97b86 while it is in vm_state rescued {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-145bd2a7-c37d-44e4-93c1-3126b983e4d1 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86/action" status: 409 len: 145 microversion: 2.1 time: 0.067120 Sep 27 10:33:29.756684 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3381/6300] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:33:29 2022] POST /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86/action => generated 145 bytes in 71 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:33:29.764358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:29.767830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:29.774406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.775208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.775870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.776938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.777646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.778325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.807163 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] HTTP exception thrown: Cannot 'rebuild' instance 6537d086-a852-4b27-a032-807cecd97b86 while it is in vm_state rescued Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Request method failure captured: Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86/action HTTP/1.1 Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 65 Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtB5lZw26WfhuIIxDLfEGtJx8GdKac9mUecPJWt47RWTVgFEQNt2CRnQRU62Kxhrd_5bXczg0Q9QWsT4c85-tEEw3L_ZoXg7ttqo2fTxJvLnAMIld1Q9kkM9ItLXE6S_aDTtz0h96WsfSIhdls5sa9AFKLBsHqf09n27PObWbpVe7TQ Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_40247edd5a444fd8b1c937c13d39393a"}]}] Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 6771df45b4ee4426b2f76a09eab2998a Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: exception: Cannot 'rebuild' instance 6537d086-a852-4b27-a032-807cecd97b86 while it is in vm_state rescued Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '6537d086-a852-4b27-a032-807cecd97b86', 'body': {'rebuild': {'imageRef': 'c1623d3a-c1bb-4649-9910-5deadb8a1ff6'}}} Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1223, in _action_rebuild Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.compute_api.rebuild(context, Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 278, in inner Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 6537d086-a852-4b27-a032-807cecd97b86 in vm_state rescued. Cannot rebuild while the instance is in this state. Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:33:29.809376 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi [Previous line repeated 4 more times] Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1231, in _action_rebuild Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'rebuild' instance 6537d086-a852-4b27-a032-807cecd97b86 while it is in vm_state rescued Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'rebuild' instance 6537d086-a852-4b27-a032-807cecd97b86 while it is in vm_state rescued Sep 27 10:33:29.814154 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:29.815400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Returning 409 to user: Cannot 'rebuild' instance 6537d086-a852-4b27-a032-807cecd97b86 while it is in vm_state rescued {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:29.815834 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea260b35-1a19-4a66-baf6-359776cd9037 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "POST /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86/action" status: 409 len: 146 microversion: 2.1 time: 0.055664 Sep 27 10:33:29.816732 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2920/6301] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:33:29 2022] POST /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86/action => generated 146 bytes in 57 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 27 10:33:29.830041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-281671b0-3929-4079-9a7c-1455fde06a1f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:29.830568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-281671b0-3929-4079-9a7c-1455fde06a1f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:29.837024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-281671b0-3929-4079-9a7c-1455fde06a1f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.837734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-281671b0-3929-4079-9a7c-1455fde06a1f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.838385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-281671b0-3929-4079-9a7c-1455fde06a1f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.839544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-281671b0-3929-4079-9a7c-1455fde06a1f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:29.840177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-281671b0-3929-4079-9a7c-1455fde06a1f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:29.840618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-281671b0-3929-4079-9a7c-1455fde06a1f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:29.870201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-281671b0-3929-4079-9a7c-1455fde06a1f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:33:29.983890 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-281671b0-3929-4079-9a7c-1455fde06a1f tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "DELETE /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 204 len: 0 microversion: 2.1 time: 0.155862 Sep 27 10:33:29.985164 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3382/6302] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:33:29 2022] DELETE /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:33:29.994958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-33c1ba51-9041-4dab-9fdd-42b1c7e6a807 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:29.995841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-33c1ba51-9041-4dab-9fdd-42b1c7e6a807 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:30.003494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33c1ba51-9041-4dab-9fdd-42b1c7e6a807 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:30.004271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33c1ba51-9041-4dab-9fdd-42b1c7e6a807 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:30.004824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33c1ba51-9041-4dab-9fdd-42b1c7e6a807 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:30.005747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33c1ba51-9041-4dab-9fdd-42b1c7e6a807 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:30.006273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33c1ba51-9041-4dab-9fdd-42b1c7e6a807 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:30.006780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-33c1ba51-9041-4dab-9fdd-42b1c7e6a807 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:30.176332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-33c1ba51-9041-4dab-9fdd-42b1c7e6a807 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:30.180607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=4,nova_cell1:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:30.187548 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-33c1ba51-9041-4dab-9fdd-42b1c7e6a807 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1691 microversion: 2.1 time: 0.197416 Sep 27 10:33:30.189110 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2921/6303] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:29 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1691 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:31.213388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d2295ee-4c47-4a35-ae16-955c0a8b92bc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:31.214327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d2295ee-4c47-4a35-ae16-955c0a8b92bc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:31.221440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d2295ee-4c47-4a35-ae16-955c0a8b92bc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:31.221920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d2295ee-4c47-4a35-ae16-955c0a8b92bc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:31.222351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d2295ee-4c47-4a35-ae16-955c0a8b92bc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:31.223044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d2295ee-4c47-4a35-ae16-955c0a8b92bc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:31.223502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d2295ee-4c47-4a35-ae16-955c0a8b92bc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:31.223927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d2295ee-4c47-4a35-ae16-955c0a8b92bc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:31.243423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:31.452824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7d2295ee-4c47-4a35-ae16-955c0a8b92bc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:31.461321 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d2295ee-4c47-4a35-ae16-955c0a8b92bc tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1691 microversion: 2.1 time: 0.252328 Sep 27 10:33:31.462205 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3383/6304] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:31 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1691 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:32.493775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6cfc283f-3ed8-4341-9eac-bbfabaa35253 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:32.494440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6cfc283f-3ed8-4341-9eac-bbfabaa35253 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:32.503519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cfc283f-3ed8-4341-9eac-bbfabaa35253 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:32.504155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cfc283f-3ed8-4341-9eac-bbfabaa35253 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:32.504816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cfc283f-3ed8-4341-9eac-bbfabaa35253 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:32.505525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cfc283f-3ed8-4341-9eac-bbfabaa35253 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:32.506362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cfc283f-3ed8-4341-9eac-bbfabaa35253 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:32.506880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cfc283f-3ed8-4341-9eac-bbfabaa35253 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:32.658413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6cfc283f-3ed8-4341-9eac-bbfabaa35253 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:32.666423 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6cfc283f-3ed8-4341-9eac-bbfabaa35253 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1691 microversion: 2.1 time: 0.183363 Sep 27 10:33:32.667054 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2922/6305] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:32 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1691 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:33.690259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30907049-300f-438f-8641-4224344c41da tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:33.690811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-30907049-300f-438f-8641-4224344c41da tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:33.699883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30907049-300f-438f-8641-4224344c41da tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:33.700832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30907049-300f-438f-8641-4224344c41da tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:33.701414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30907049-300f-438f-8641-4224344c41da tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:33.702141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30907049-300f-438f-8641-4224344c41da tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:33.702956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30907049-300f-438f-8641-4224344c41da tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:33.703764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30907049-300f-438f-8641-4224344c41da tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:33.859193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-30907049-300f-438f-8641-4224344c41da tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:33.868914 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-30907049-300f-438f-8641-4224344c41da tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1691 microversion: 2.1 time: 0.182472 Sep 27 10:33:33.871230 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3384/6306] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:33 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1691 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:34.405627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ffb7d215-3271-4be1-8c67-ddfb8b01956d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:34.408283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ffb7d215-3271-4be1-8c67-ddfb8b01956d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:34.415104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb7d215-3271-4be1-8c67-ddfb8b01956d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:34.415710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb7d215-3271-4be1-8c67-ddfb8b01956d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:34.416306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb7d215-3271-4be1-8c67-ddfb8b01956d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:34.417158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb7d215-3271-4be1-8c67-ddfb8b01956d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:34.417692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb7d215-3271-4be1-8c67-ddfb8b01956d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:34.418202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffb7d215-3271-4be1-8c67-ddfb8b01956d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:34.491431 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ffb7d215-3271-4be1-8c67-ddfb8b01956d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed/action" status: 200 len: 62940 microversion: 2.1 time: 0.372885 Sep 27 10:33:34.500091 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2923/6307] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:33:34 2022] POST /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed/action => generated 62940 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:33:34.739732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Action: 'create', calling method: .version_select at 0x7f7f6297f9d0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-2118323264"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:34.870196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-47b2c713-e5ee-4581-ab43-15ac4557bf1c req-0a6decb8-143b-43c9-abe7-5aa73883d2c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dc9ddb88-bfb7-40d7-b615-9755d1e824d5", "name": "network-vif-deleted", "tag": "b77efabf-15d1-4744-b2d6-473b94c9fb51"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:34.885831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-47b2c713-e5ee-4581-ab43-15ac4557bf1c req-0a6decb8-143b-43c9-abe7-5aa73883d2c9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:34.886333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-47b2c713-e5ee-4581-ab43-15ac4557bf1c req-0a6decb8-143b-43c9-abe7-5aa73883d2c9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:34.886787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-47b2c713-e5ee-4581-ab43-15ac4557bf1c req-0a6decb8-143b-43c9-abe7-5aa73883d2c9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:34.910468 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-47b2c713-e5ee-4581-ab43-15ac4557bf1c req-0a6decb8-143b-43c9-abe7-5aa73883d2c9 service nova] Creating event network-vif-deleted:b77efabf-15d1-4744-b2d6-473b94c9fb51 for instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 on nch2-master-1 Sep 27 10:33:34.915975 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-47b2c713-e5ee-4581-ab43-15ac4557bf1c req-0a6decb8-143b-43c9-abe7-5aa73883d2c9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049914 Sep 27 10:33:34.916647 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2924/6308] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:33:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:33:34.922164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-80e9365d-d336-45fd-8a8c-5dde328678b3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:34.922830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-80e9365d-d336-45fd-8a8c-5dde328678b3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:34.929356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80e9365d-d336-45fd-8a8c-5dde328678b3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:34.930152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80e9365d-d336-45fd-8a8c-5dde328678b3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:34.930800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80e9365d-d336-45fd-8a8c-5dde328678b3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:34.931835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80e9365d-d336-45fd-8a8c-5dde328678b3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:34.932224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80e9365d-d336-45fd-8a8c-5dde328678b3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:34.932891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-80e9365d-d336-45fd-8a8c-5dde328678b3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:35.020339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-80e9365d-d336-45fd-8a8c-5dde328678b3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:35.040313 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-80e9365d-d336-45fd-8a8c-5dde328678b3 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 200 len: 1331 microversion: 2.1 time: 0.122178 Sep 27 10:33:35.040967 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2925/6309] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:34 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 1331 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:35.141354 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-602aa730-fd6a-4df4-b6a3-6c4ab5fa15d5 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 200 len: 338 microversion: 2.1 time: 0.552493 Sep 27 10:33:35.154254 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3385/6310] 10.222.0.26 () {60 vars in 1155 bytes} [Tue Sep 27 10:33:34 2022] POST /compute/v2.1/os-volumes => generated 338 bytes in 566 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 27 10:33:35.156533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b698bd3-9839-4e4e-bc3f-2be293b19ee7 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f62b81280>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:35.232552 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0b698bd3-9839-4e4e-bc3f-2be293b19ee7 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde" status: 200 len: 338 microversion: 2.1 time: 0.081342 Sep 27 10:33:35.233181 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2926/6311] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:33:35 2022] GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde => generated 338 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:36.066178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:36.066794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: dc9ddb88-bfb7-40d7-b615-9755d1e824d5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:36.073906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:36.074283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:36.074715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:36.075457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:36.075910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:36.076352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:36.090660 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] HTTP exception thrown: Instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 could not be found. Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Request method failure captured: Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 HTTP/1.1 Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtB5lZw26WfhuIIxDLfEGtJx8GdKac9mUecPJWt47RWTVgFEQNt2CRnQRU62Kxhrd_5bXczg0Q9QWsT4c85-tEEw3L_ZoXg7ttqo2fTxJvLnAMIld1Q9kkM9ItLXE6S_aDTtz0h96WsfSIhdls5sa9AFKLBsHqf09n27PObWbpVe7TQ Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_40247edd5a444fd8b1c937c13d39393a"}]}] Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6771df45b4ee4426b2f76a09eab2998a Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 could not be found. Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'dc9ddb88-bfb7-40d7-b615-9755d1e824d5'} Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 could not be found. Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 could not be found. Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:36.092346 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:36.097703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:36.097703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:36.097703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:36.097703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:36.097703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:36.097703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:36.097703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 could not be found. Sep 27 10:33:36.097703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:36.097703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Returning 404 to user: Instance dc9ddb88-bfb7-40d7-b615-9755d1e824d5 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:36.097703 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-11e602d5-71b2-4c98-9c22-a0a882688701 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5" status: 404 len: 111 microversion: 2.1 time: 0.032804 Sep 27 10:33:36.097703 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3386/6312] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:36 2022] GET /compute/v2.1/servers/dc9ddb88-bfb7-40d7-b615-9755d1e824d5 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:36.103425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6639a572-7ba1-4d3a-9368-d04234f9691d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '.version_select at 0x7f7f63629c10>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:36.125394 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6639a572-7ba1-4d3a-9368-d04234f9691d tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1650579745" status: 202 len: 0 microversion: 2.1 time: 0.026149 Sep 27 10:33:36.126041 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2927/6313] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:33:36 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1650579745 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:33:36.247478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c5c113b1-e42c-416c-968b-af8fd753e2cd tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f6267b0d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:36.303153 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c5c113b1-e42c-416c-968b-af8fd753e2cd tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde" status: 200 len: 339 microversion: 2.1 time: 0.059729 Sep 27 10:33:36.303861 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3387/6314] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:33:36 2022] GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde => generated 339 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:36.318027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-36cfd086-aa63-4fc5-af33-b93de8753768 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Action: 'create', calling method: .version_select at 0x7f7f636294c0>, body: {"snapshot": {"volume_id": "b974c5cb-3e23-4eef-87cd-fc18b0b71dde", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-2111774099"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:36.495906 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-36cfd086-aa63-4fc5-af33-b93de8753768 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.182177 Sep 27 10:33:36.496980 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2928/6315] 10.222.0.26 () {60 vars in 1162 bytes} [Tue Sep 27 10:33:36 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:36.508934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90ad817b-f08b-4ae6-a9e5-2fb90857b8c3 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f6267b3a0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:36.544168 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-90ad817b-f08b-4ae6-a9e5-2fb90857b8c3 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb" status: 200 len: 287 microversion: 2.1 time: 0.039720 Sep 27 10:33:36.556876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4a739b4b-4915-42a3-bd00-c9b24d90a8ba tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f62b8baf0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:36.561723 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3388/6316] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:33:36 2022] GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb => generated 287 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:36.595166 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4a739b4b-4915-42a3-bd00-c9b24d90a8ba tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.042391 Sep 27 10:33:36.595804 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2929/6317] 10.222.0.26 () {58 vars in 1140 bytes} [Tue Sep 27 10:33:36 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:36.609985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-34181928-b9d9-4336-8402-97769bfc2270 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f6301d670>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:36.645476 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-34181928-b9d9-4336-8402-97769bfc2270 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb" status: 200 len: 287 microversion: 2.1 time: 0.041771 Sep 27 10:33:36.646095 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3389/6318] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:33:36 2022] GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb => generated 287 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:37.660430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-079c4994-11aa-45a1-a016-06c14a40998d tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f62eafaf0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:37.693764 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-079c4994-11aa-45a1-a016-06c14a40998d tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb" status: 200 len: 287 microversion: 2.1 time: 0.038802 Sep 27 10:33:37.694446 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2930/6319] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:33:37 2022] GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb => generated 287 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:38.708692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed606a4c-36f1-4485-9d3f-6949e9828f63 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f6267bd30>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:38.740137 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed606a4c-36f1-4485-9d3f-6949e9828f63 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb" status: 200 len: 288 microversion: 2.1 time: 0.036356 Sep 27 10:33:38.740895 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3390/6320] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:33:38 2022] GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb => generated 288 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:38.756824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5eb0b9e9-0257-40c0-8e8e-116836cc01c6 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f6366d5e0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:38.819279 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5eb0b9e9-0257-40c0-8e8e-116836cc01c6 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "DELETE /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb" status: 202 len: 0 microversion: 2.1 time: 0.069304 Sep 27 10:33:38.820055 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2931/6321] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 10:33:38 2022] DELETE /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb => generated 0 bytes in 72 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:33:38.830466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8d06413-b053-4c1e-ad64-429a5f5eacaf tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f623eb940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:38.864679 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c8d06413-b053-4c1e-ad64-429a5f5eacaf tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb" status: 200 len: 287 microversion: 2.1 time: 0.038630 Sep 27 10:33:38.865401 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3391/6322] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:33:38 2022] GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb => generated 287 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:39.117674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd4b94a2-d0b7-4a81-a5fe-8d85b3003fda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:39.118157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cd4b94a2-d0b7-4a81-a5fe-8d85b3003fda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:39.125298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4b94a2-d0b7-4a81-a5fe-8d85b3003fda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:39.125802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4b94a2-d0b7-4a81-a5fe-8d85b3003fda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:39.126252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4b94a2-d0b7-4a81-a5fe-8d85b3003fda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:39.126963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4b94a2-d0b7-4a81-a5fe-8d85b3003fda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:39.127431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4b94a2-d0b7-4a81-a5fe-8d85b3003fda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:39.127935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd4b94a2-d0b7-4a81-a5fe-8d85b3003fda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:39.153634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cd4b94a2-d0b7-4a81-a5fe-8d85b3003fda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:33:39.243730 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd4b94a2-d0b7-4a81-a5fe-8d85b3003fda tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "DELETE /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 204 len: 0 microversion: 2.1 time: 0.129549 Sep 27 10:33:39.244320 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2932/6323] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:33:39 2022] DELETE /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:33:39.253058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad9f5f7a-c7f5-489d-bf87-ac41e8f345fe tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:39.253774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ad9f5f7a-c7f5-489d-bf87-ac41e8f345fe tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:39.263435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad9f5f7a-c7f5-489d-bf87-ac41e8f345fe tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:39.263932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad9f5f7a-c7f5-489d-bf87-ac41e8f345fe tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:39.264376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad9f5f7a-c7f5-489d-bf87-ac41e8f345fe tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:39.265134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad9f5f7a-c7f5-489d-bf87-ac41e8f345fe tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:39.265597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad9f5f7a-c7f5-489d-bf87-ac41e8f345fe tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:39.266001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad9f5f7a-c7f5-489d-bf87-ac41e8f345fe tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:39.404367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ad9f5f7a-c7f5-489d-bf87-ac41e8f345fe tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:39.412458 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ad9f5f7a-c7f5-489d-bf87-ac41e8f345fe tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1509 microversion: 2.1 time: 0.163288 Sep 27 10:33:39.413072 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3392/6324] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:39 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1509 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:39.879018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-001929f8-d9a7-4082-a697-7492987c1235 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f62ba2310>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:39.905999 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-001929f8-d9a7-4082-a697-7492987c1235 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] HTTP exception thrown: Snapshot 62092bd6-9a66-44d4-8fcb-d3d911809cdb could not be found. Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-001929f8-d9a7-4082-a697-7492987c1235 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Request method failure captured: Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb HTTP/1.1 Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtF-ikMpfR7UqpokMajeg69FMwbfIlugm0s143c9g7adg-xuSW_Lx_Jdwi1rP4bKk6nGTeJ9H523Dgx-Y65eJONNXVBNGBCYIFwQ09yjbdkAKhu6-nQnBkpDfip3pPiJQxcD99N6Ck5Y7juB2itMOQ1nd6FPDvAOKamsjuHDDp_cMoA Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 78c18800a5fc4f40994fade5d422694d Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_78c18800a5fc4f40994fade5d422694d"}]}] Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 78c18800a5fc4f40994fade5d422694d Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-VolumesSnapshotsTestJSON-1036287016-project Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: f827322737bf4518b0b262ae61797687 Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1036287016-project Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: exception: Snapshot 62092bd6-9a66-44d4-8fcb-d3d911809cdb could not be found. Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '62092bd6-9a66-44d4-8fcb-d3d911809cdb'} Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 62092bd6-9a66-44d4-8fcb-d3d911809cdb could not be found. (HTTP 404) (Request-ID: req-1745dfde-0d96-448f-a872-517ba35a9d74) Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 10:33:39.910200 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 62092bd6-9a66-44d4-8fcb-d3d911809cdb could not be found. Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 62092bd6-9a66-44d4-8fcb-d3d911809cdb could not be found. Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 62092bd6-9a66-44d4-8fcb-d3d911809cdb could not be found. Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-001929f8-d9a7-4082-a697-7492987c1235 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Returning 404 to user: Snapshot 62092bd6-9a66-44d4-8fcb-d3d911809cdb could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-001929f8-d9a7-4082-a697-7492987c1235 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb" status: 404 len: 111 microversion: 2.1 time: 0.037072 Sep 27 10:33:39.915921 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2933/6325] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 10:33:39 2022] GET /compute/v2.1/os-snapshots/62092bd6-9a66-44d4-8fcb-d3d911809cdb => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:39.920908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f6297ff70>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:39.989707 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-62fd9e1a-eb99-4f10-a128-b6b64fa95a05 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde" status: 202 len: 0 microversion: 2.1 time: 0.072909 Sep 27 10:33:39.990392 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3393/6326] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:33:39 2022] DELETE /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde => generated 0 bytes in 74 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:33:40.000052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b9c1a592-e9b7-474f-a405-1d8e1653b9fc tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f636d7700>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:40.052420 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b9c1a592-e9b7-474f-a405-1d8e1653b9fc tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde" status: 200 len: 338 microversion: 2.1 time: 0.056369 Sep 27 10:33:40.053077 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2934/6327] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:33:39 2022] GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde => generated 338 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:40.435796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-be43e156-e252-44a9-bf89-10931abf9dd8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:40.436796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-be43e156-e252-44a9-bf89-10931abf9dd8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:40.446225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be43e156-e252-44a9-bf89-10931abf9dd8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:40.446915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be43e156-e252-44a9-bf89-10931abf9dd8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:40.447539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be43e156-e252-44a9-bf89-10931abf9dd8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:40.448593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be43e156-e252-44a9-bf89-10931abf9dd8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:40.449536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be43e156-e252-44a9-bf89-10931abf9dd8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:40.450160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be43e156-e252-44a9-bf89-10931abf9dd8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:40.594592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-be43e156-e252-44a9-bf89-10931abf9dd8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:40.607532 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-be43e156-e252-44a9-bf89-10931abf9dd8 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1509 microversion: 2.1 time: 0.175572 Sep 27 10:33:40.608413 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3394/6328] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:40 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1509 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:41.068621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-06d6a20e-3a8f-4e38-a528-e2f0d61c01dd tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f62b9c1f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:41.097069 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-06d6a20e-3a8f-4e38-a528-e2f0d61c01dd tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] HTTP exception thrown: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-06d6a20e-3a8f-4e38-a528-e2f0d61c01dd tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Request method failure captured: Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde HTTP/1.1 Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtF-ikMpfR7UqpokMajeg69FMwbfIlugm0s143c9g7adg-xuSW_Lx_Jdwi1rP4bKk6nGTeJ9H523Dgx-Y65eJONNXVBNGBCYIFwQ09yjbdkAKhu6-nQnBkpDfip3pPiJQxcD99N6Ck5Y7juB2itMOQ1nd6FPDvAOKamsjuHDDp_cMoA Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 78c18800a5fc4f40994fade5d422694d Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_78c18800a5fc4f40994fade5d422694d"}]}] Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 78c18800a5fc4f40994fade5d422694d Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-VolumesSnapshotsTestJSON-1036287016-project Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: f827322737bf4518b0b262ae61797687 Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1036287016-project Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: exception: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'b974c5cb-3e23-4eef-87cd-fc18b0b71dde'} Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. (HTTP 404) (Request-ID: req-bb7e073e-7ee8-40a0-b1e5-e510ff43cb47) Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 10:33:41.099060 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-06d6a20e-3a8f-4e38-a528-e2f0d61c01dd tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Returning 404 to user: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-06d6a20e-3a8f-4e38-a528-e2f0d61c01dd tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde" status: 404 len: 109 microversion: 2.1 time: 0.036949 Sep 27 10:33:41.104206 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2935/6329] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:33:41 2022] GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:41.111202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9496c56-0240-4336-8784-f54fcdcc3352 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f623ebee0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:41.140686 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-c9496c56-0240-4336-8784-f54fcdcc3352 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] HTTP exception thrown: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9496c56-0240-4336-8784-f54fcdcc3352 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Request method failure captured: Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde HTTP/1.1 Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtF-ikMpfR7UqpokMajeg69FMwbfIlugm0s143c9g7adg-xuSW_Lx_Jdwi1rP4bKk6nGTeJ9H523Dgx-Y65eJONNXVBNGBCYIFwQ09yjbdkAKhu6-nQnBkpDfip3pPiJQxcD99N6Ck5Y7juB2itMOQ1nd6FPDvAOKamsjuHDDp_cMoA Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 78c18800a5fc4f40994fade5d422694d Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_78c18800a5fc4f40994fade5d422694d"}]}] Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 78c18800a5fc4f40994fade5d422694d Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesSnapshotsTestJSON-1036287016-project Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: f827322737bf4518b0b262ae61797687 Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1036287016-project Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: method: Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: exception: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'b974c5cb-3e23-4eef-87cd-fc18b0b71dde'} Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. (HTTP 404) (Request-ID: req-4fe4fd01-7f28-494b-b1da-7dbca65fe368) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 10:33:41.142646 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9496c56-0240-4336-8784-f54fcdcc3352 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Returning 404 to user: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9496c56-0240-4336-8784-f54fcdcc3352 tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde" status: 404 len: 109 microversion: 2.1 time: 0.036778 Sep 27 10:33:41.148489 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3395/6330] 10.222.0.26 () {58 vars in 1248 bytes} [Tue Sep 27 10:33:41 2022] DELETE /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde => generated 109 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:41.156218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c425613a-9aeb-4f0f-bfbc-da0c8075b05d tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Calling method '.version_select at 0x7f7f636413a0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:41.185876 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-c425613a-9aeb-4f0f-bfbc-da0c8075b05d tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] HTTP exception thrown: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c425613a-9aeb-4f0f-bfbc-da0c8075b05d tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Request method failure captured: Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde HTTP/1.1 Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtF-ikMpfR7UqpokMajeg69FMwbfIlugm0s143c9g7adg-xuSW_Lx_Jdwi1rP4bKk6nGTeJ9H523Dgx-Y65eJONNXVBNGBCYIFwQ09yjbdkAKhu6-nQnBkpDfip3pPiJQxcD99N6Ck5Y7juB2itMOQ1nd6FPDvAOKamsjuHDDp_cMoA Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 78c18800a5fc4f40994fade5d422694d Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/78c18800a5fc4f40994fade5d422694d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_78c18800a5fc4f40994fade5d422694d"}]}] Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 78c18800a5fc4f40994fade5d422694d Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1036287016 Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-VolumesSnapshotsTestJSON-1036287016-project Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: f827322737bf4518b0b262ae61797687 Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1036287016-project Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: method: Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: exception: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'b974c5cb-3e23-4eef-87cd-fc18b0b71dde'} Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. (HTTP 404) (Request-ID: req-db035bc9-1f75-4c1c-b2c5-74ea6f696735) Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 27 10:33:41.187974 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c425613a-9aeb-4f0f-bfbc-da0c8075b05d tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] Returning 404 to user: Volume b974c5cb-3e23-4eef-87cd-fc18b0b71dde could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c425613a-9aeb-4f0f-bfbc-da0c8075b05d tempest-VolumesSnapshotsTestJSON-1036287016 tempest-VolumesSnapshotsTestJSON-1036287016-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde" status: 404 len: 109 microversion: 2.1 time: 0.040824 Sep 27 10:33:41.192960 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2936/6331] 10.222.0.26 () {58 vars in 1245 bytes} [Tue Sep 27 10:33:41 2022] GET /compute/v2.1/os-volumes/b974c5cb-3e23-4eef-87cd-fc18b0b71dde => generated 109 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:41.633090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ea7ab8cd-aad7-4766-87f1-7054dc7537d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:41.634757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ea7ab8cd-aad7-4766-87f1-7054dc7537d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:41.641797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea7ab8cd-aad7-4766-87f1-7054dc7537d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:41.642556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea7ab8cd-aad7-4766-87f1-7054dc7537d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:41.643220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea7ab8cd-aad7-4766-87f1-7054dc7537d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:41.643924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea7ab8cd-aad7-4766-87f1-7054dc7537d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:41.644592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea7ab8cd-aad7-4766-87f1-7054dc7537d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:41.645044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea7ab8cd-aad7-4766-87f1-7054dc7537d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:41.904434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ab716322-49ae-4f92-ae10-6dbaef877c2a req-e5dccaad-848f-4575-a9f6-65ec9e82fa29 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6537d086-a852-4b27-a032-807cecd97b86", "name": "network-vif-deleted", "tag": "d9fe8fff-a04f-429f-9369-933e6efad0ea"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:41.932017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ea7ab8cd-aad7-4766-87f1-7054dc7537d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:41.933696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ab716322-49ae-4f92-ae10-6dbaef877c2a req-e5dccaad-848f-4575-a9f6-65ec9e82fa29 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:41.936968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ab716322-49ae-4f92-ae10-6dbaef877c2a req-e5dccaad-848f-4575-a9f6-65ec9e82fa29 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:41.938036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ab716322-49ae-4f92-ae10-6dbaef877c2a req-e5dccaad-848f-4575-a9f6-65ec9e82fa29 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:41.943369 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ea7ab8cd-aad7-4766-87f1-7054dc7537d9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 200 len: 1509 microversion: 2.1 time: 0.315152 Sep 27 10:33:41.944581 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3396/6332] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:41 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 1509 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:41.956070 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ab716322-49ae-4f92-ae10-6dbaef877c2a req-e5dccaad-848f-4575-a9f6-65ec9e82fa29 service nova] Creating event network-vif-deleted:d9fe8fff-a04f-429f-9369-933e6efad0ea for instance 6537d086-a852-4b27-a032-807cecd97b86 on nch1-master-1 Sep 27 10:33:41.962894 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ab716322-49ae-4f92-ae10-6dbaef877c2a req-e5dccaad-848f-4575-a9f6-65ec9e82fa29 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.066442 Sep 27 10:33:41.966150 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2937/6333] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:33:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:42.967439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:42.968055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 6537d086-a852-4b27-a032-807cecd97b86] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:42.975206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:42.975756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:42.976211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:42.976975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:42.977442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:42.977870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:42.992926 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] HTTP exception thrown: Instance 6537d086-a852-4b27-a032-807cecd97b86 could not be found. Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Request method failure captured: Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 HTTP/1.1 Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtB5lZw26WfhuIIxDLfEGtJx8GdKac9mUecPJWt47RWTVgFEQNt2CRnQRU62Kxhrd_5bXczg0Q9QWsT4c85-tEEw3L_ZoXg7ttqo2fTxJvLnAMIld1Q9kkM9ItLXE6S_aDTtz0h96WsfSIhdls5sa9AFKLBsHqf09n27PObWbpVe7TQ Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_40247edd5a444fd8b1c937c13d39393a"}]}] Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6771df45b4ee4426b2f76a09eab2998a Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 6537d086-a852-4b27-a032-807cecd97b86 could not be found. Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '6537d086-a852-4b27-a032-807cecd97b86'} Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6537d086-a852-4b27-a032-807cecd97b86 could not be found. Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6537d086-a852-4b27-a032-807cecd97b86 could not be found. Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:42.995492 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:43.000614 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:43.000614 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:43.000614 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:43.000614 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:43.000614 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:43.000614 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:43.000614 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6537d086-a852-4b27-a032-807cecd97b86 could not be found. Sep 27 10:33:43.000614 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:43.002212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Returning 404 to user: Instance 6537d086-a852-4b27-a032-807cecd97b86 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:43.003109 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5599ac43-7d1d-4b88-bf9d-5f8907215913 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86" status: 404 len: 111 microversion: 2.1 time: 0.040210 Sep 27 10:33:43.003900 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3397/6334] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:42 2022] GET /compute/v2.1/servers/6537d086-a852-4b27-a032-807cecd97b86 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:43.014441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-988156ad-4a15-4951-9abb-a61d57e4518e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:43.015056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-988156ad-4a15-4951-9abb-a61d57e4518e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:43.021637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-988156ad-4a15-4951-9abb-a61d57e4518e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:43.022148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-988156ad-4a15-4951-9abb-a61d57e4518e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:43.022610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-988156ad-4a15-4951-9abb-a61d57e4518e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:43.023371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-988156ad-4a15-4951-9abb-a61d57e4518e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:43.023857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-988156ad-4a15-4951-9abb-a61d57e4518e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:43.024363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-988156ad-4a15-4951-9abb-a61d57e4518e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:43.052114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-988156ad-4a15-4951-9abb-a61d57e4518e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:33:43.135243 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-988156ad-4a15-4951-9abb-a61d57e4518e tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "DELETE /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 204 len: 0 microversion: 2.1 time: 0.126584 Sep 27 10:33:43.135925 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2938/6335] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:33:43 2022] DELETE /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:33:43.144036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b6efd272-c56d-406d-afd9-7cc8a1219f9c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:43.144688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b6efd272-c56d-406d-afd9-7cc8a1219f9c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:43.150786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6efd272-c56d-406d-afd9-7cc8a1219f9c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:43.151544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6efd272-c56d-406d-afd9-7cc8a1219f9c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:43.152001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6efd272-c56d-406d-afd9-7cc8a1219f9c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:43.152825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6efd272-c56d-406d-afd9-7cc8a1219f9c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:43.153295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6efd272-c56d-406d-afd9-7cc8a1219f9c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:43.153729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b6efd272-c56d-406d-afd9-7cc8a1219f9c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:43.308387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b6efd272-c56d-406d-afd9-7cc8a1219f9c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:43.321932 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b6efd272-c56d-406d-afd9-7cc8a1219f9c tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.181774 Sep 27 10:33:43.323174 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3398/6336] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:43 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:44.346381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-84b02529-7886-4b86-84a3-acd848bd0a34 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:44.347174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-84b02529-7886-4b86-84a3-acd848bd0a34 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:44.354272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b02529-7886-4b86-84a3-acd848bd0a34 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:44.354774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b02529-7886-4b86-84a3-acd848bd0a34 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:44.355250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b02529-7886-4b86-84a3-acd848bd0a34 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:44.356123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b02529-7886-4b86-84a3-acd848bd0a34 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:44.356594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b02529-7886-4b86-84a3-acd848bd0a34 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:44.357051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-84b02529-7886-4b86-84a3-acd848bd0a34 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:44.498525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-84b02529-7886-4b86-84a3-acd848bd0a34 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:44.506695 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-84b02529-7886-4b86-84a3-acd848bd0a34 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.164701 Sep 27 10:33:44.507268 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2939/6337] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:44 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:45.530360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1434bf93-4ca9-4a21-b3db-68c50aa2936b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:45.530962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1434bf93-4ca9-4a21-b3db-68c50aa2936b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:45.534026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=59,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:45.539237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1434bf93-4ca9-4a21-b3db-68c50aa2936b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:45.539863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1434bf93-4ca9-4a21-b3db-68c50aa2936b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:45.540304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1434bf93-4ca9-4a21-b3db-68c50aa2936b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:45.541137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1434bf93-4ca9-4a21-b3db-68c50aa2936b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:45.541598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1434bf93-4ca9-4a21-b3db-68c50aa2936b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:45.542011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1434bf93-4ca9-4a21-b3db-68c50aa2936b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:45.691543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1434bf93-4ca9-4a21-b3db-68c50aa2936b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:45.699953 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1434bf93-4ca9-4a21-b3db-68c50aa2936b tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1521 microversion: 2.1 time: 0.173535 Sep 27 10:33:45.701290 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3399/6338] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:45 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1521 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:46.725350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1489a5a8-cd05-4a03-a618-843a08a94ad9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:46.727918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1489a5a8-cd05-4a03-a618-843a08a94ad9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:46.733295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=78,nova_api:UPDATE=4,nova_api:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:33:46.744081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-17388f41-b48d-4317-8945-56c7c1713a80 req-c84773e7-7815-48ed-920e-acd53fa5c3bb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "15833e30-ec8c-4756-b9df-5ca3061b4ed9", "name": "network-vif-deleted", "tag": "77c15aca-e4ea-4c69-9903-2fba830c8336"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:46.749437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1489a5a8-cd05-4a03-a618-843a08a94ad9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:46.750169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1489a5a8-cd05-4a03-a618-843a08a94ad9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:46.751580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1489a5a8-cd05-4a03-a618-843a08a94ad9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:46.752713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1489a5a8-cd05-4a03-a618-843a08a94ad9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:46.754024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1489a5a8-cd05-4a03-a618-843a08a94ad9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:46.754630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1489a5a8-cd05-4a03-a618-843a08a94ad9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:46.757548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-17388f41-b48d-4317-8945-56c7c1713a80 req-c84773e7-7815-48ed-920e-acd53fa5c3bb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:46.758282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-17388f41-b48d-4317-8945-56c7c1713a80 req-c84773e7-7815-48ed-920e-acd53fa5c3bb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:46.758808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-17388f41-b48d-4317-8945-56c7c1713a80 req-c84773e7-7815-48ed-920e-acd53fa5c3bb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:46.772290 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-17388f41-b48d-4317-8945-56c7c1713a80 req-c84773e7-7815-48ed-920e-acd53fa5c3bb service nova] Creating event network-vif-deleted:77c15aca-e4ea-4c69-9903-2fba830c8336 for instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 on nch1-master-1 Sep 27 10:33:46.783925 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-17388f41-b48d-4317-8945-56c7c1713a80 req-c84773e7-7815-48ed-920e-acd53fa5c3bb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044986 Sep 27 10:33:46.786376 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3400/6339] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:33:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:46.863781 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1489a5a8-cd05-4a03-a618-843a08a94ad9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6771df45b4ee4426b2f76a09eab2998a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '40247edd5a444fd8b1c937c13d39393a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:46.879542 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1489a5a8-cd05-4a03-a618-843a08a94ad9 tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 200 len: 1306 microversion: 2.1 time: 0.157611 Sep 27 10:33:46.880589 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2940/6340] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:46 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 1306 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:47.905144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:47.905881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] [instance: 15833e30-ec8c-4756-b9df-5ca3061b4ed9] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:47.912297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:47.912820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:47.913246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:47.913954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:47.914686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:47.915455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:47.934993 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] HTTP exception thrown: Instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 could not be found. Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Request method failure captured: Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 HTTP/1.1 Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtB5lZw26WfhuIIxDLfEGtJx8GdKac9mUecPJWt47RWTVgFEQNt2CRnQRU62Kxhrd_5bXczg0Q9QWsT4c85-tEEw3L_ZoXg7ttqo2fTxJvLnAMIld1Q9kkM9ItLXE6S_aDTtz0h96WsfSIhdls5sa9AFKLBsHqf09n27PObWbpVe7TQ Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/40247edd5a444fd8b1c937c13d39393a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_40247edd5a444fd8b1c937c13d39393a"}]}] Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 40247edd5a444fd8b1c937c13d39393a Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1875151076 Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 6771df45b4ee4426b2f76a09eab2998a Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1875151076-project Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 could not be found. Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '15833e30-ec8c-4756-b9df-5ca3061b4ed9'} Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 could not be found. Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 could not be found. Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:47.936738 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:47.943118 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:47.943118 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:47.943118 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:47.943118 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:47.943118 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:47.943118 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:47.943118 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 could not be found. Sep 27 10:33:47.943118 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:47.943118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] Returning 404 to user: Instance 15833e30-ec8c-4756-b9df-5ca3061b4ed9 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:47.943118 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-82f2e536-1a73-46b3-8df0-6c2c8c76cd2a tempest-ServerRescueNegativeTestJSON-1875151076 tempest-ServerRescueNegativeTestJSON-1875151076-project] 10.222.0.26 "GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9" status: 404 len: 111 microversion: 2.1 time: 0.038336 Sep 27 10:33:47.943118 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3401/6341] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:47 2022] GET /compute/v2.1/servers/15833e30-ec8c-4756-b9df-5ca3061b4ed9 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:33:50.956562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7bda9dd9-ea5f-4910-8394-0c0e53e3b257 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:50.958283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7bda9dd9-ea5f-4910-8394-0c0e53e3b257 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:50.965806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bda9dd9-ea5f-4910-8394-0c0e53e3b257 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:50.966470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bda9dd9-ea5f-4910-8394-0c0e53e3b257 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:50.966967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bda9dd9-ea5f-4910-8394-0c0e53e3b257 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:50.967659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bda9dd9-ea5f-4910-8394-0c0e53e3b257 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:50.972957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bda9dd9-ea5f-4910-8394-0c0e53e3b257 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:50.972957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7bda9dd9-ea5f-4910-8394-0c0e53e3b257 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:51.252081 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7bda9dd9-ea5f-4910-8394-0c0e53e3b257 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1834 microversion: 2.1 time: 0.582436 Sep 27 10:33:51.268510 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2941/6342] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:50 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1834 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:52.667734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-6cf0373c-29dd-4543-bd5e-a2c55f9a232f req-2c758ff5-06f1-47ff-8fe6-10368a675fa8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0565ee6c-196a-4ca9-981f-24e019cd89ed", "tag": "dde5d266-f09e-421c-b470-aacdbb240d15"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:52.669194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62cfe972-5498-46a2-833a-a96740875d4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:52.669709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-62cfe972-5498-46a2-833a-a96740875d4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:52.677949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62cfe972-5498-46a2-833a-a96740875d4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:52.678448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62cfe972-5498-46a2-833a-a96740875d4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:52.678887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62cfe972-5498-46a2-833a-a96740875d4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:52.679820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62cfe972-5498-46a2-833a-a96740875d4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:52.680766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62cfe972-5498-46a2-833a-a96740875d4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:52.681188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62cfe972-5498-46a2-833a-a96740875d4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:52.684165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6cf0373c-29dd-4543-bd5e-a2c55f9a232f req-2c758ff5-06f1-47ff-8fe6-10368a675fa8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:52.691469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6cf0373c-29dd-4543-bd5e-a2c55f9a232f req-2c758ff5-06f1-47ff-8fe6-10368a675fa8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:52.695831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6cf0373c-29dd-4543-bd5e-a2c55f9a232f req-2c758ff5-06f1-47ff-8fe6-10368a675fa8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:52.712266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-62cfe972-5498-46a2-833a-a96740875d4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:33:52.727309 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-6cf0373c-29dd-4543-bd5e-a2c55f9a232f req-2c758ff5-06f1-47ff-8fe6-10368a675fa8 service nova] Creating event network-changed:dde5d266-f09e-421c-b470-aacdbb240d15 for instance 0565ee6c-196a-4ca9-981f-24e019cd89ed on nch2-master-1 Sep 27 10:33:52.735354 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-6cf0373c-29dd-4543-bd5e-a2c55f9a232f req-2c758ff5-06f1-47ff-8fe6-10368a675fa8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071158 Sep 27 10:33:52.736567 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2942/6343] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:33:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:33:52.824702 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-62cfe972-5498-46a2-833a-a96740875d4d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 204 len: 0 microversion: 2.1 time: 0.162987 Sep 27 10:33:52.825605 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3402/6344] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:33:52 2022] DELETE /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 0 bytes in 166 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:33:52.836242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d99bae24-ad84-4cf9-8c0f-5cccab3cb8cd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:52.837802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d99bae24-ad84-4cf9-8c0f-5cccab3cb8cd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:52.850651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99bae24-ad84-4cf9-8c0f-5cccab3cb8cd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:52.852685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99bae24-ad84-4cf9-8c0f-5cccab3cb8cd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:52.853822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99bae24-ad84-4cf9-8c0f-5cccab3cb8cd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:52.857376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99bae24-ad84-4cf9-8c0f-5cccab3cb8cd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:52.858081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99bae24-ad84-4cf9-8c0f-5cccab3cb8cd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:52.859573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d99bae24-ad84-4cf9-8c0f-5cccab3cb8cd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:53.008474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7f194b15-f577-44d9-bdcf-2b87dee7303f tempest-ExtensionsTest-561315341 tempest-ExtensionsTest-561315341-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:53.010145 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7f194b15-f577-44d9-bdcf-2b87dee7303f tempest-ExtensionsTest-561315341 tempest-ExtensionsTest-561315341-project] 10.222.0.26 "GET /compute/v2.1/extensions/os-consoles" status: 200 len: 217 microversion: 2.1 time: 0.170736 Sep 27 10:33:53.010689 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3403/6345] 10.222.0.26 () {58 vars in 1170 bytes} [Tue Sep 27 10:33:52 2022] GET /compute/v2.1/extensions/os-consoles => generated 217 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:33:53.020037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-839aff18-4624-4def-8cbb-9de7f453623e tempest-ExtensionsTest-561315341 tempest-ExtensionsTest-561315341-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:53.022823 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-839aff18-4624-4def-8cbb-9de7f453623e tempest-ExtensionsTest-561315341 tempest-ExtensionsTest-561315341-project] 10.222.0.26 "GET /compute/v2.1/extensions" status: 200 len: 22700 microversion: 2.1 time: 0.006837 Sep 27 10:33:53.024513 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3404/6346] 10.222.0.26 () {58 vars in 1134 bytes} [Tue Sep 27 10:33:53 2022] GET /compute/v2.1/extensions => generated 22700 bytes in 9 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 10:33:53.124902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d99bae24-ad84-4cf9-8c0f-5cccab3cb8cd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:53.132954 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d99bae24-ad84-4cf9-8c0f-5cccab3cb8cd tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1689 microversion: 2.1 time: 0.302193 Sep 27 10:33:53.134303 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2943/6347] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:52 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1689 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:54.159880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-567bcbad-6541-4f44-873b-2514d569f607 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:54.161731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-567bcbad-6541-4f44-873b-2514d569f607 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:54.171824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567bcbad-6541-4f44-873b-2514d569f607 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:54.172775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567bcbad-6541-4f44-873b-2514d569f607 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:54.173504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567bcbad-6541-4f44-873b-2514d569f607 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:54.174386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567bcbad-6541-4f44-873b-2514d569f607 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:54.174897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567bcbad-6541-4f44-873b-2514d569f607 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:54.175571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567bcbad-6541-4f44-873b-2514d569f607 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:54.343973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-567bcbad-6541-4f44-873b-2514d569f607 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:54.352553 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-567bcbad-6541-4f44-873b-2514d569f607 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1689 microversion: 2.1 time: 0.197059 Sep 27 10:33:54.353485 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3405/6348] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:54 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1689 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:55.381922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-34c252b7-39de-421d-a475-cadd0d4ecd09 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:55.385696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-34c252b7-39de-421d-a475-cadd0d4ecd09 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:55.395629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c252b7-39de-421d-a475-cadd0d4ecd09 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:55.396696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c252b7-39de-421d-a475-cadd0d4ecd09 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:55.397422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c252b7-39de-421d-a475-cadd0d4ecd09 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:55.398335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c252b7-39de-421d-a475-cadd0d4ecd09 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:55.399026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c252b7-39de-421d-a475-cadd0d4ecd09 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:55.399978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34c252b7-39de-421d-a475-cadd0d4ecd09 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:55.419165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-21402575-73a7-4633-8b64-6d12503bf073 req-c4d74f8d-b8f6-40c5-8ada-9d3ba5ebfe82 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0565ee6c-196a-4ca9-981f-24e019cd89ed", "name": "network-vif-deleted", "tag": "dde5d266-f09e-421c-b470-aacdbb240d15"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:33:55.432309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-21402575-73a7-4633-8b64-6d12503bf073 req-c4d74f8d-b8f6-40c5-8ada-9d3ba5ebfe82 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:55.432899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-21402575-73a7-4633-8b64-6d12503bf073 req-c4d74f8d-b8f6-40c5-8ada-9d3ba5ebfe82 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:55.433742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-21402575-73a7-4633-8b64-6d12503bf073 req-c4d74f8d-b8f6-40c5-8ada-9d3ba5ebfe82 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:55.453582 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-21402575-73a7-4633-8b64-6d12503bf073 req-c4d74f8d-b8f6-40c5-8ada-9d3ba5ebfe82 service nova] Creating event network-vif-deleted:dde5d266-f09e-421c-b470-aacdbb240d15 for instance 0565ee6c-196a-4ca9-981f-24e019cd89ed on nch2-master-1 Sep 27 10:33:55.459398 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-21402575-73a7-4633-8b64-6d12503bf073 req-c4d74f8d-b8f6-40c5-8ada-9d3ba5ebfe82 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044440 Sep 27 10:33:55.460518 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3406/6349] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:33:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:33:55.491406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-34c252b7-39de-421d-a475-cadd0d4ecd09 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:33:55.505999 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-34c252b7-39de-421d-a475-cadd0d4ecd09 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 200 len: 1502 microversion: 2.1 time: 0.127324 Sep 27 10:33:55.507133 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2944/6350] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:55 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 1502 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:33:56.532625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:33:56.533381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 0565ee6c-196a-4ca9-981f-24e019cd89ed] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:33:56.539788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:56.540497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:56.541112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:56.541747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:33:56.542216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:33:56.542733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:33:56.559616 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] HTTP exception thrown: Instance 0565ee6c-196a-4ca9-981f-24e019cd89ed could not be found. Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Request method failure captured: Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed HTTP/1.1 Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtAz-I5R9pkEmM8ENwgEXlrfwhpK5QJgpteQwRSpBNpzmPCQ6_jXCbvGwjb68dGwzfz0hV_obx0eVwNZLFOTRd61-pMOQhq1CSGhYJOTDYOHjRjmEKJs84qfkSC88em1BS9SfLSGMC-OPias42e-aueDGLNj17wWnJe-vn54z9Dzb2o Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 86e71212d96f46a78680afaacd8709ca Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_86e71212d96f46a78680afaacd8709ca"}]}] Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 86e71212d96f46a78680afaacd8709ca Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestNetworkAdvancedServerOps-1611618676-project Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 0bd8a19d0ab44011b5430f26757a0efe Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1611618676-project Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 0565ee6c-196a-4ca9-981f-24e019cd89ed could not be found. Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '0565ee6c-196a-4ca9-981f-24e019cd89ed'} Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0565ee6c-196a-4ca9-981f-24e019cd89ed could not be found. Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0565ee6c-196a-4ca9-981f-24e019cd89ed could not be found. Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:56.561713 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:33:56.568090 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:56.568090 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:33:56.568090 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:33:56.568090 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:33:56.568090 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:33:56.568090 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:33:56.568090 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0565ee6c-196a-4ca9-981f-24e019cd89ed could not be found. Sep 27 10:33:56.568090 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:33:56.568090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Returning 404 to user: Instance 0565ee6c-196a-4ca9-981f-24e019cd89ed could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:33:56.568090 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d88144ee-2c8f-4425-92d2-0895714fff29 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed" status: 404 len: 111 microversion: 2.1 time: 0.036512 Sep 27 10:33:56.568090 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3407/6351] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:33:56 2022] GET /compute/v2.1/servers/0565ee6c-196a-4ca9-981f-24e019cd89ed => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:34:00.697409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6cd3be92-6b83-4513-a59c-934c2a628310 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '.version_select at 0x7f7f6366d1f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:00.733326 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6cd3be92-6b83-4513-a59c-934c2a628310 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1344904965" status: 202 len: 0 microversion: 2.1 time: 0.041679 Sep 27 10:34:00.734637 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2945/6352] 10.222.0.26 () {58 vars in 1284 bytes} [Tue Sep 27 10:34:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1344904965 => generated 0 bytes in 44 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:34:00.778523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-df7481d3-7376-4f6f-bb18-4ba245cc0285 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'create', calling method: .version_select at 0x7f7f628b6b80>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-1604356436", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMPSuz2vP/2sePEcd64he1DIYV3CbW5QdXN7PiGwzpQG7XpMBMJfYSDNFo3qFnPLbE5C2TqdE4oGskWK07Odnr3wS442aBK2x0b+a+PKdQLIz7ygzdp+Nv0tnH1W1XpRCw=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:34:00.798407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-df7481d3-7376-4f6f-bb18-4ba245cc0285 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for project 86e71212d96f46a78680afaacd8709ca. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:34:00.804085 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-df7481d3-7376-4f6f-bb18-4ba245cc0285 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for user 0bd8a19d0ab44011b5430f26757a0efe and project 86e71212d96f46a78680afaacd8709ca. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:34:00.833799 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-df7481d3-7376-4f6f-bb18-4ba245cc0285 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.059816 Sep 27 10:34:00.834693 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3408/6353] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:34:00 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:34:03.837811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:03.838720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:03.847187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c9298313-a979-4e47-8de8-2b18a8d42007 req-c7352181-d026-4a6f-bb9d-4d39e883b463 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9f14e82f-c065-4b4e-bb03-c94b57bc21e7", "tag": "86766b79-cf12-4620-813b-644b95940fbc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:34:03.857254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:03.857766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:03.860018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:03.862639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:03.865273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:03.866059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:03.867206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9298313-a979-4e47-8de8-2b18a8d42007 req-c7352181-d026-4a6f-bb9d-4d39e883b463 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:03.867735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9298313-a979-4e47-8de8-2b18a8d42007 req-c7352181-d026-4a6f-bb9d-4d39e883b463 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:03.868812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9298313-a979-4e47-8de8-2b18a8d42007 req-c7352181-d026-4a6f-bb9d-4d39e883b463 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:03.882778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=71,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:03.907343 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c9298313-a979-4e47-8de8-2b18a8d42007 req-c7352181-d026-4a6f-bb9d-4d39e883b463 service nova] Creating event network-changed:86766b79-cf12-4620-813b-644b95940fbc for instance 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 on nch2-master-1 Sep 27 10:34:03.920534 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c9298313-a979-4e47-8de8-2b18a8d42007 req-c7352181-d026-4a6f-bb9d-4d39e883b463 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.078282 Sep 27 10:34:03.923134 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3409/6354] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:34:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:34:03.946170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:03.975793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:34:04.165967 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1a8ebe36-2bd9-4adb-9dd3-6df31798d3cf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 204 len: 0 microversion: 2.1 time: 0.334730 Sep 27 10:34:04.172086 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2946/6355] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:34:03 2022] DELETE /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:34:04.178363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b2050b68-8a2a-4062-9e1c-1f66196ee7a9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:04.179113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b2050b68-8a2a-4062-9e1c-1f66196ee7a9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:04.191244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2050b68-8a2a-4062-9e1c-1f66196ee7a9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:04.191753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2050b68-8a2a-4062-9e1c-1f66196ee7a9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:04.192201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2050b68-8a2a-4062-9e1c-1f66196ee7a9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:04.192956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2050b68-8a2a-4062-9e1c-1f66196ee7a9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:04.193423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2050b68-8a2a-4062-9e1c-1f66196ee7a9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:04.193856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2050b68-8a2a-4062-9e1c-1f66196ee7a9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:04.422261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b2050b68-8a2a-4062-9e1c-1f66196ee7a9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:04.432367 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b2050b68-8a2a-4062-9e1c-1f66196ee7a9 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1578 microversion: 2.1 time: 0.258589 Sep 27 10:34:04.433511 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3410/6356] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:04 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1578 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:05.473664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e9285768-1e85-48fd-b669-2a5809b8663f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:05.477649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e9285768-1e85-48fd-b669-2a5809b8663f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:05.500857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9285768-1e85-48fd-b669-2a5809b8663f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:05.502581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9285768-1e85-48fd-b669-2a5809b8663f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:05.503331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9285768-1e85-48fd-b669-2a5809b8663f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:05.504398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9285768-1e85-48fd-b669-2a5809b8663f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:05.505037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9285768-1e85-48fd-b669-2a5809b8663f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:05.505839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e9285768-1e85-48fd-b669-2a5809b8663f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:05.964303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e9285768-1e85-48fd-b669-2a5809b8663f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:05.979448 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e9285768-1e85-48fd-b669-2a5809b8663f tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1578 microversion: 2.1 time: 0.523970 Sep 27 10:34:05.981839 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2947/6357] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:05 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1578 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:07.005595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9380374b-b03c-4266-b338-94381a7c81e6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:07.006825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9380374b-b03c-4266-b338-94381a7c81e6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:07.019039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9380374b-b03c-4266-b338-94381a7c81e6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:07.019958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9380374b-b03c-4266-b338-94381a7c81e6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:07.020534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9380374b-b03c-4266-b338-94381a7c81e6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:07.021558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9380374b-b03c-4266-b338-94381a7c81e6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:07.023040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9380374b-b03c-4266-b338-94381a7c81e6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:07.023635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9380374b-b03c-4266-b338-94381a7c81e6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:07.275022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9380374b-b03c-4266-b338-94381a7c81e6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:07.287888 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9380374b-b03c-4266-b338-94381a7c81e6 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1459 microversion: 2.1 time: 0.287623 Sep 27 10:34:07.288527 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3411/6358] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:06 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1459 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:07.540783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-bc8abdee-9460-4877-af7e-07f90fd3d56a req-c57acc0c-afbe-4fcd-892c-f72b2406f373 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9f14e82f-c065-4b4e-bb03-c94b57bc21e7", "name": "network-vif-deleted", "tag": "86766b79-cf12-4620-813b-644b95940fbc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:34:07.566698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bc8abdee-9460-4877-af7e-07f90fd3d56a req-c57acc0c-afbe-4fcd-892c-f72b2406f373 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:07.571368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bc8abdee-9460-4877-af7e-07f90fd3d56a req-c57acc0c-afbe-4fcd-892c-f72b2406f373 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:07.572175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-bc8abdee-9460-4877-af7e-07f90fd3d56a req-c57acc0c-afbe-4fcd-892c-f72b2406f373 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:07.606896 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-bc8abdee-9460-4877-af7e-07f90fd3d56a req-c57acc0c-afbe-4fcd-892c-f72b2406f373 service nova] Creating event network-vif-deleted:86766b79-cf12-4620-813b-644b95940fbc for instance 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 on nch2-master-1 Sep 27 10:34:07.616761 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-bc8abdee-9460-4877-af7e-07f90fd3d56a req-c57acc0c-afbe-4fcd-892c-f72b2406f373 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082273 Sep 27 10:34:07.621837 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2948/6359] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:34:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:34:08.312277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:08.312932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:08.333325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:08.333834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:08.334282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:08.335013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:08.335476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:08.335908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:08.421809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:08.423121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:08.423592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:08.426343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lazy-loading 'fault' on Instance uuid 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:34:08.528939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '178956f6c8154a06a0710b295aff3086', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '64f8fe93425d4e97b1ae5661172f4846', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:08.538642 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-29b06b45-9041-47f1-87d9-04d3f46252bf tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 200 len: 1230 microversion: 2.1 time: 0.231384 Sep 27 10:34:08.543416 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3412/6360] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:08 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 1230 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:09.571580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:09.573303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 9f14e82f-c065-4b4e-bb03-c94b57bc21e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:09.589078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:09.589836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:09.590527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:09.591590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:09.593676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:09.594388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:09.631369 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 could not be found. Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 HTTP/1.1 Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 could not be found. Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '9f14e82f-c065-4b4e-bb03-c94b57bc21e7'} Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 could not be found. Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 could not be found. Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:34:09.637769 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:34:09.640856 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:34:09.640856 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:34:09.640856 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:34:09.640856 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:34:09.640856 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:34:09.640856 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 could not be found. Sep 27 10:34:09.640856 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:34:09.640856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance 9f14e82f-c065-4b4e-bb03-c94b57bc21e7 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:34:09.640856 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4bc4edfb-26f4-493c-97f6-4ffc2654e3f3 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7" status: 404 len: 111 microversion: 2.1 time: 0.074548 Sep 27 10:34:09.645565 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2949/6361] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:09 2022] GET /compute/v2.1/servers/9f14e82f-c065-4b4e-bb03-c94b57bc21e7 => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:34:10.474928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:10.476463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:10.491550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:10.492305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:10.493338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:10.495512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:10.495993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:10.497027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:10.524123 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b HTTP/1.1 Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '82b31a6f-a898-4796-a3ad-1628306fbe7b'} Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:34:10.526756 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e4069bcb-aca6-4612-88c2-017874c5ff90 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "DELETE /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 404 len: 111 microversion: 2.1 time: 0.058104 Sep 27 10:34:10.530630 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3413/6362] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:34:10 2022] DELETE /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:34:10.537515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:10.538865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] [instance: 82b31a6f-a898-4796-a3ad-1628306fbe7b] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:10.548024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:10.548530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:10.549121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:10.549739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:10.551842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:10.553154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:10.620558 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] HTTP exception thrown: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Request method failure captured: Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b HTTP/1.1 Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMs89PyvJ5jKqg8DS_l4hT9xn78lkkq6O-R4dNHKA5Bb3O9vH97Xj-Mytrr1ikuF8egRO_hiXpjfHPQhBKN0_ZxhR8nyAX8mLqiKqmN6nXPImmnWl3hkrHOck8AhPAfiUChskXhKodWfvAtTW6ZfXqVvkAlDYbjo4ushCyvlzvKnoRws Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/64f8fe93425d4e97b1ae5661172f4846"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_64f8fe93425d4e97b1ae5661172f4846"}]}] Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestVolumeBootPattern-4600220 Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 64f8fe93425d4e97b1ae5661172f4846 Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestVolumeBootPattern-4600220 Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 178956f6c8154a06a0710b295aff3086 Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestVolumeBootPattern-4600220-project Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '82b31a6f-a898-4796-a3ad-1628306fbe7b'} Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:34:10.622377 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:34:10.625455 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:34:10.625455 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:34:10.625455 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:34:10.625455 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:34:10.625455 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:34:10.625455 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. Sep 27 10:34:10.625455 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:34:10.625455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Returning 404 to user: Instance 82b31a6f-a898-4796-a3ad-1628306fbe7b could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:34:10.625939 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eb8d672b-423e-491e-8800-f888d9f93fc4 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b" status: 404 len: 111 microversion: 2.1 time: 0.092408 Sep 27 10:34:10.626340 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2950/6363] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:10 2022] GET /compute/v2.1/servers/82b31a6f-a898-4796-a3ad-1628306fbe7b => generated 111 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:34:10.787004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-953919813", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "af828c92-0e6c-42c7-b25a-81bbdb6e8d9a"}], "key_name": "tempest-TestNetworkAdvancedServerOps-1604356436", "security_groups": [{"name": "tempest-secgroup-smoke-398254893"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:34:10.842545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:11.020508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] validate_networks() for [('af828c92-0e6c-42c7-b25a-81bbdb6e8d9a', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:34:12.289160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:34:12.290843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:34:12.291352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:34:12.291993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:34:12.292563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:34:12.305745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:12.306438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:12.307140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:12.308139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:12.308944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:12.309583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:12.331087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for project 86e71212d96f46a78680afaacd8709ca. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:34:12.337589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for user 0bd8a19d0ab44011b5430f26757a0efe and project 86e71212d96f46a78680afaacd8709ca. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:34:12.351823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:34:12.355127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:34:12.408241 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3bd49c0-596f-4e21-907a-ffe64c5f2564 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.625339 Sep 27 10:34:12.408965 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3414/6364] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:34:10 2022] POST /compute/v2.1/servers => generated 405 bytes in 1627 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:34:12.420457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0d18e530-edad-4561-8d74-51f68daceb68 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:12.421141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0d18e530-edad-4561-8d74-51f68daceb68 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:12.523017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0d18e530-edad-4561-8d74-51f68daceb68 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:12.531354 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0d18e530-edad-4561-8d74-51f68daceb68 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1266 microversion: 2.1 time: 0.114866 Sep 27 10:34:12.538387 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2951/6365] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:12 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1266 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:34:13.360265 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0ba6b860-28d5-418d-8262-49f3aa30ac61 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] Calling method '.version_select at 0x7f7f623daaf0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:13.394467 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0ba6b860-28d5-418d-8262-49f3aa30ac61 tempest-TestVolumeBootPattern-4600220 tempest-TestVolumeBootPattern-4600220-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-362079488" status: 202 len: 0 microversion: 2.1 time: 0.043113 Sep 27 10:34:13.404532 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3415/6366] 10.222.0.26 () {58 vars in 1260 bytes} [Tue Sep 27 10:34:13 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-362079488 => generated 0 bytes in 54 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:34:13.562498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b995cca1-4cb3-4ffd-a039-6c9f7183632b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:13.563131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b995cca1-4cb3-4ffd-a039-6c9f7183632b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:13.579017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b995cca1-4cb3-4ffd-a039-6c9f7183632b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:13.579804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b995cca1-4cb3-4ffd-a039-6c9f7183632b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:13.580403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b995cca1-4cb3-4ffd-a039-6c9f7183632b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:13.583423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b995cca1-4cb3-4ffd-a039-6c9f7183632b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:13.583926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b995cca1-4cb3-4ffd-a039-6c9f7183632b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:13.584634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b995cca1-4cb3-4ffd-a039-6c9f7183632b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:13.672490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b995cca1-4cb3-4ffd-a039-6c9f7183632b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:13.681047 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b995cca1-4cb3-4ffd-a039-6c9f7183632b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1262 microversion: 2.1 time: 0.122865 Sep 27 10:34:13.696702 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2952/6367] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:13 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1262 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:14.707481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62b4afcd-94a2-4c9d-9017-27b08f0686b8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:14.708911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-62b4afcd-94a2-4c9d-9017-27b08f0686b8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:14.721116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b4afcd-94a2-4c9d-9017-27b08f0686b8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:14.721787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b4afcd-94a2-4c9d-9017-27b08f0686b8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:14.722390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b4afcd-94a2-4c9d-9017-27b08f0686b8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:14.723419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b4afcd-94a2-4c9d-9017-27b08f0686b8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:14.724446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b4afcd-94a2-4c9d-9017-27b08f0686b8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:14.725418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62b4afcd-94a2-4c9d-9017-27b08f0686b8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:14.853409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-62b4afcd-94a2-4c9d-9017-27b08f0686b8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:14.872333 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-62b4afcd-94a2-4c9d-9017-27b08f0686b8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1324 microversion: 2.1 time: 0.170350 Sep 27 10:34:14.873537 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3416/6368] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:14 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1324 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:15.899973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-97f305fd-f146-4bdd-a958-c846e6c94dfb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:15.900938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-97f305fd-f146-4bdd-a958-c846e6c94dfb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:15.915784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97f305fd-f146-4bdd-a958-c846e6c94dfb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:15.917995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97f305fd-f146-4bdd-a958-c846e6c94dfb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:15.918613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97f305fd-f146-4bdd-a958-c846e6c94dfb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:15.919896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97f305fd-f146-4bdd-a958-c846e6c94dfb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:15.920369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97f305fd-f146-4bdd-a958-c846e6c94dfb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:15.921001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-97f305fd-f146-4bdd-a958-c846e6c94dfb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:16.127346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-97f305fd-f146-4bdd-a958-c846e6c94dfb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:16.137455 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-97f305fd-f146-4bdd-a958-c846e6c94dfb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1391 microversion: 2.1 time: 0.241858 Sep 27 10:34:16.158706 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2953/6369] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:15 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1391 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:16.567341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a46a67b9-cb85-4e97-a81a-23b571bd4303 req-caabb11f-56eb-46a3-a2ea-0cf580c7ae16 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7036fba1-2704-49b3-9106-b4eda4d25d8c", "tag": "daa7ce1e-3299-491a-a2c8-08296813b09c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:34:16.579439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=62,nova_api:UPDATE=1,nova_api:INSERT=4,nova_api:DELETE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:16.593799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a46a67b9-cb85-4e97-a81a-23b571bd4303 req-caabb11f-56eb-46a3-a2ea-0cf580c7ae16 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:16.595469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a46a67b9-cb85-4e97-a81a-23b571bd4303 req-caabb11f-56eb-46a3-a2ea-0cf580c7ae16 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:16.596079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a46a67b9-cb85-4e97-a81a-23b571bd4303 req-caabb11f-56eb-46a3-a2ea-0cf580c7ae16 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:16.643157 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a46a67b9-cb85-4e97-a81a-23b571bd4303 req-caabb11f-56eb-46a3-a2ea-0cf580c7ae16 service nova] Creating event network-changed:daa7ce1e-3299-491a-a2c8-08296813b09c for instance 7036fba1-2704-49b3-9106-b4eda4d25d8c on nch2-master-1 Sep 27 10:34:16.656236 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a46a67b9-cb85-4e97-a81a-23b571bd4303 req-caabb11f-56eb-46a3-a2ea-0cf580c7ae16 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.094036 Sep 27 10:34:16.658595 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3417/6370] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:34:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:34:17.160168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b4e1920d-922e-4176-a5bb-49715aea2b4c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:17.160851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b4e1920d-922e-4176-a5bb-49715aea2b4c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:17.167737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=36,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:17.171773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4e1920d-922e-4176-a5bb-49715aea2b4c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:17.172299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4e1920d-922e-4176-a5bb-49715aea2b4c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:17.172775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4e1920d-922e-4176-a5bb-49715aea2b4c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:17.173651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4e1920d-922e-4176-a5bb-49715aea2b4c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:17.174258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4e1920d-922e-4176-a5bb-49715aea2b4c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:17.174846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4e1920d-922e-4176-a5bb-49715aea2b4c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:17.446911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b4e1920d-922e-4176-a5bb-49715aea2b4c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:17.486749 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b4e1920d-922e-4176-a5bb-49715aea2b4c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1391 microversion: 2.1 time: 0.329242 Sep 27 10:34:17.501015 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2954/6371] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:17 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1391 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:34:18.514747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0602e29c-09ee-4db4-b459-2b03825d4814 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:18.518138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0602e29c-09ee-4db4-b459-2b03825d4814 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:18.537220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0602e29c-09ee-4db4-b459-2b03825d4814 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:18.538482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0602e29c-09ee-4db4-b459-2b03825d4814 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:18.539348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0602e29c-09ee-4db4-b459-2b03825d4814 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:18.540430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0602e29c-09ee-4db4-b459-2b03825d4814 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:18.541788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0602e29c-09ee-4db4-b459-2b03825d4814 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:18.542588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0602e29c-09ee-4db4-b459-2b03825d4814 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:18.817842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0602e29c-09ee-4db4-b459-2b03825d4814 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:18.838889 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0602e29c-09ee-4db4-b459-2b03825d4814 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1391 microversion: 2.1 time: 0.325029 Sep 27 10:34:18.839877 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3418/6372] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:18 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1391 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:19.866547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fffffa15-e6c0-4f40-bda2-c0c5d3e4fb17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:19.867386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fffffa15-e6c0-4f40-bda2-c0c5d3e4fb17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:19.878830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fffffa15-e6c0-4f40-bda2-c0c5d3e4fb17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:19.880376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fffffa15-e6c0-4f40-bda2-c0c5d3e4fb17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:19.880865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fffffa15-e6c0-4f40-bda2-c0c5d3e4fb17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:19.882162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fffffa15-e6c0-4f40-bda2-c0c5d3e4fb17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:19.884518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fffffa15-e6c0-4f40-bda2-c0c5d3e4fb17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:19.887015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fffffa15-e6c0-4f40-bda2-c0c5d3e4fb17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:20.209617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fffffa15-e6c0-4f40-bda2-c0c5d3e4fb17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:20.225786 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fffffa15-e6c0-4f40-bda2-c0c5d3e4fb17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1391 microversion: 2.1 time: 0.363211 Sep 27 10:34:20.228534 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2955/6373] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:19 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1391 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:21.251292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6b05dfbf-5369-478c-a250-fc2ef5d5edb0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:21.252075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6b05dfbf-5369-478c-a250-fc2ef5d5edb0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:21.265417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b05dfbf-5369-478c-a250-fc2ef5d5edb0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:21.266391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b05dfbf-5369-478c-a250-fc2ef5d5edb0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:21.268293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b05dfbf-5369-478c-a250-fc2ef5d5edb0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:21.270888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b05dfbf-5369-478c-a250-fc2ef5d5edb0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:21.271502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b05dfbf-5369-478c-a250-fc2ef5d5edb0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:21.273214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6b05dfbf-5369-478c-a250-fc2ef5d5edb0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:21.593183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6b05dfbf-5369-478c-a250-fc2ef5d5edb0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:21.604715 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6b05dfbf-5369-478c-a250-fc2ef5d5edb0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1391 microversion: 2.1 time: 0.354056 Sep 27 10:34:21.604715 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3419/6374] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:21 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1391 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:22.632031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-873e2a15-8a45-4117-881a-93e3de77618f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:22.633611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-873e2a15-8a45-4117-881a-93e3de77618f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:22.639944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:22.642133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873e2a15-8a45-4117-881a-93e3de77618f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:22.642848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873e2a15-8a45-4117-881a-93e3de77618f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:22.643551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873e2a15-8a45-4117-881a-93e3de77618f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:22.644521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873e2a15-8a45-4117-881a-93e3de77618f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:22.645252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873e2a15-8a45-4117-881a-93e3de77618f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:22.649120 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-873e2a15-8a45-4117-881a-93e3de77618f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:22.985154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-873e2a15-8a45-4117-881a-93e3de77618f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:22.995738 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-873e2a15-8a45-4117-881a-93e3de77618f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1391 microversion: 2.1 time: 0.368805 Sep 27 10:34:22.996957 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2956/6375] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:22 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1391 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:23.343104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-058c33d4-db5d-4b52-9c27-6c6a9686532f req-951884df-85db-430c-889b-9610ce9e45bb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7036fba1-2704-49b3-9106-b4eda4d25d8c", "name": "network-vif-plugged", "status": "completed", "tag": "daa7ce1e-3299-491a-a2c8-08296813b09c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:34:23.350095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:23.351091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:34:23.358961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-058c33d4-db5d-4b52-9c27-6c6a9686532f req-951884df-85db-430c-889b-9610ce9e45bb service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:23.359635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-058c33d4-db5d-4b52-9c27-6c6a9686532f req-951884df-85db-430c-889b-9610ce9e45bb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:23.360280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-058c33d4-db5d-4b52-9c27-6c6a9686532f req-951884df-85db-430c-889b-9610ce9e45bb service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:23.374998 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-058c33d4-db5d-4b52-9c27-6c6a9686532f req-951884df-85db-430c-889b-9610ce9e45bb service nova] Creating event network-vif-plugged:daa7ce1e-3299-491a-a2c8-08296813b09c for instance 7036fba1-2704-49b3-9106-b4eda4d25d8c on nch2-master-1 Sep 27 10:34:23.380778 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-058c33d4-db5d-4b52-9c27-6c6a9686532f req-951884df-85db-430c-889b-9610ce9e45bb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042474 Sep 27 10:34:23.382088 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3420/6376] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:34:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:34:24.020179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a7a548a0-cefd-4681-81ce-00d12bf63754 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:24.020920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a7a548a0-cefd-4681-81ce-00d12bf63754 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:24.029079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7a548a0-cefd-4681-81ce-00d12bf63754 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:24.029909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7a548a0-cefd-4681-81ce-00d12bf63754 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:24.030781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7a548a0-cefd-4681-81ce-00d12bf63754 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:24.032018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7a548a0-cefd-4681-81ce-00d12bf63754 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:24.032554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7a548a0-cefd-4681-81ce-00d12bf63754 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:24.033335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a7a548a0-cefd-4681-81ce-00d12bf63754 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:24.197015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a7a548a0-cefd-4681-81ce-00d12bf63754 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:24.214379 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a7a548a0-cefd-4681-81ce-00d12bf63754 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1391 microversion: 2.1 time: 0.198342 Sep 27 10:34:24.215333 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3421/6377] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:24 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1391 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:24.450514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8a997301-b15c-40c8-a399-8350de0b8b07 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250-project-admin] Calling method '.version_select at 0x7f7f629bc700>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:24.452819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a997301-b15c-40c8-a399-8350de0b8b07 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:24.453341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a997301-b15c-40c8-a399-8350de0b8b07 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:24.453842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a997301-b15c-40c8-a399-8350de0b8b07 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:24.470269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a997301-b15c-40c8-a399-8350de0b8b07 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:24.470776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a997301-b15c-40c8-a399-8350de0b8b07 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:24.471241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a997301-b15c-40c8-a399-8350de0b8b07 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:24.482563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a997301-b15c-40c8-a399-8350de0b8b07 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:24.483293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a997301-b15c-40c8-a399-8350de0b8b07 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:24.483910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a997301-b15c-40c8-a399-8350de0b8b07 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:24.490044 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8a997301-b15c-40c8-a399-8350de0b8b07 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250 tempest-PortsAdminExtendedAttrsIpV6TestJSON-308772250-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.480147 Sep 27 10:34:24.490792 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2957/6378] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:34:24 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:34:25.241270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0f1ca18-512e-4737-98d8-81fdbc2c510d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:25.242823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b0f1ca18-512e-4737-98d8-81fdbc2c510d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:25.257981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f1ca18-512e-4737-98d8-81fdbc2c510d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:25.258613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f1ca18-512e-4737-98d8-81fdbc2c510d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:25.259279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f1ca18-512e-4737-98d8-81fdbc2c510d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:25.260246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f1ca18-512e-4737-98d8-81fdbc2c510d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:25.261071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f1ca18-512e-4737-98d8-81fdbc2c510d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:25.261847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0f1ca18-512e-4737-98d8-81fdbc2c510d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:25.475340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b0f1ca18-512e-4737-98d8-81fdbc2c510d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:25.484756 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b0f1ca18-512e-4737-98d8-81fdbc2c510d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1563 microversion: 2.1 time: 0.249688 Sep 27 10:34:25.485618 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3422/6379] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:25 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1563 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:25.508706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-82796d16-7485-4508-bdc8-a9932488b3d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:25.510442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-82796d16-7485-4508-bdc8-a9932488b3d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:25.518085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82796d16-7485-4508-bdc8-a9932488b3d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:25.518599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82796d16-7485-4508-bdc8-a9932488b3d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:25.519111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82796d16-7485-4508-bdc8-a9932488b3d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:25.520557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82796d16-7485-4508-bdc8-a9932488b3d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:25.521101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82796d16-7485-4508-bdc8-a9932488b3d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:25.521550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82796d16-7485-4508-bdc8-a9932488b3d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:25.794476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-82796d16-7485-4508-bdc8-a9932488b3d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:25.808105 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-82796d16-7485-4508-bdc8-a9932488b3d1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1563 microversion: 2.1 time: 0.303090 Sep 27 10:34:25.809157 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2958/6380] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:25 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1563 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:28.452778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f766073f-0f42-48a5-8574-3b9654f9b7c9 req-3efbcc77-42f3-4cef-8320-6e272f205e9b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7036fba1-2704-49b3-9106-b4eda4d25d8c", "tag": "daa7ce1e-3299-491a-a2c8-08296813b09c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:34:28.457228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-08e67363-51a5-4746-adf1-ef1b855ae1d0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:28.457978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-08e67363-51a5-4746-adf1-ef1b855ae1d0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:34:28.467997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08e67363-51a5-4746-adf1-ef1b855ae1d0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:28.469333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f766073f-0f42-48a5-8574-3b9654f9b7c9 req-3efbcc77-42f3-4cef-8320-6e272f205e9b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:28.470164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f766073f-0f42-48a5-8574-3b9654f9b7c9 req-3efbcc77-42f3-4cef-8320-6e272f205e9b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:28.470929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f766073f-0f42-48a5-8574-3b9654f9b7c9 req-3efbcc77-42f3-4cef-8320-6e272f205e9b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:28.471563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08e67363-51a5-4746-adf1-ef1b855ae1d0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:28.476154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08e67363-51a5-4746-adf1-ef1b855ae1d0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:28.477348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08e67363-51a5-4746-adf1-ef1b855ae1d0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:28.477917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08e67363-51a5-4746-adf1-ef1b855ae1d0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:28.478369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-08e67363-51a5-4746-adf1-ef1b855ae1d0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:28.495704 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f766073f-0f42-48a5-8574-3b9654f9b7c9 req-3efbcc77-42f3-4cef-8320-6e272f205e9b service nova] Creating event network-changed:daa7ce1e-3299-491a-a2c8-08296813b09c for instance 7036fba1-2704-49b3-9106-b4eda4d25d8c on nch2-master-1 Sep 27 10:34:28.512504 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f766073f-0f42-48a5-8574-3b9654f9b7c9 req-3efbcc77-42f3-4cef-8320-6e272f205e9b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063682 Sep 27 10:34:28.513148 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2959/6381] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:34:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:34:28.634694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-08e67363-51a5-4746-adf1-ef1b855ae1d0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:34:28.651956 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-08e67363-51a5-4746-adf1-ef1b855ae1d0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1563 microversion: 2.1 time: 0.205828 Sep 27 10:34:28.653088 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3423/6382] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:34:28 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1563 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:34:33.906264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c8199030-233f-4a86-9f9d-cb7bf045caf7 tempest-PortsAdminExtendedAttrsTestJSON-1618217608 tempest-PortsAdminExtendedAttrsTestJSON-1618217608-project-admin] Calling method '.version_select at 0x7f7f6c08e550>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:34:33.908274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8199030-233f-4a86-9f9d-cb7bf045caf7 tempest-PortsAdminExtendedAttrsTestJSON-1618217608 tempest-PortsAdminExtendedAttrsTestJSON-1618217608-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:33.908842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8199030-233f-4a86-9f9d-cb7bf045caf7 tempest-PortsAdminExtendedAttrsTestJSON-1618217608 tempest-PortsAdminExtendedAttrsTestJSON-1618217608-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:33.909294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8199030-233f-4a86-9f9d-cb7bf045caf7 tempest-PortsAdminExtendedAttrsTestJSON-1618217608 tempest-PortsAdminExtendedAttrsTestJSON-1618217608-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:33.924617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8199030-233f-4a86-9f9d-cb7bf045caf7 tempest-PortsAdminExtendedAttrsTestJSON-1618217608 tempest-PortsAdminExtendedAttrsTestJSON-1618217608-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:33.925164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8199030-233f-4a86-9f9d-cb7bf045caf7 tempest-PortsAdminExtendedAttrsTestJSON-1618217608 tempest-PortsAdminExtendedAttrsTestJSON-1618217608-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:33.925688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8199030-233f-4a86-9f9d-cb7bf045caf7 tempest-PortsAdminExtendedAttrsTestJSON-1618217608 tempest-PortsAdminExtendedAttrsTestJSON-1618217608-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:33.937042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8199030-233f-4a86-9f9d-cb7bf045caf7 tempest-PortsAdminExtendedAttrsTestJSON-1618217608 tempest-PortsAdminExtendedAttrsTestJSON-1618217608-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:34:33.937543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8199030-233f-4a86-9f9d-cb7bf045caf7 tempest-PortsAdminExtendedAttrsTestJSON-1618217608 tempest-PortsAdminExtendedAttrsTestJSON-1618217608-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:34:33.937991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8199030-233f-4a86-9f9d-cb7bf045caf7 tempest-PortsAdminExtendedAttrsTestJSON-1618217608 tempest-PortsAdminExtendedAttrsTestJSON-1618217608-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:34:33.944230 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c8199030-233f-4a86-9f9d-cb7bf045caf7 tempest-PortsAdminExtendedAttrsTestJSON-1618217608 tempest-PortsAdminExtendedAttrsTestJSON-1618217608-project-admin] 10.222.0.26 "GET /compute/v2.1/os-hypervisors" status: 200 len: 191 microversion: 2.1 time: 0.174857 Sep 27 10:34:33.944892 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2960/6383] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:34:33 2022] GET /compute/v2.1/os-hypervisors => generated 191 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:35:01.071203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a9bdce2-9b7e-4756-81ef-e4562214bfc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:35:01.074520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a9bdce2-9b7e-4756-81ef-e4562214bfc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:01.078255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=13 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:01.080163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=65 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:01.105252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a9bdce2-9b7e-4756-81ef-e4562214bfc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:01.105967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a9bdce2-9b7e-4756-81ef-e4562214bfc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:01.106763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a9bdce2-9b7e-4756-81ef-e4562214bfc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:01.107691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a9bdce2-9b7e-4756-81ef-e4562214bfc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:01.108342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a9bdce2-9b7e-4756-81ef-e4562214bfc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:01.108991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a9bdce2-9b7e-4756-81ef-e4562214bfc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:01.190226 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a9bdce2-9b7e-4756-81ef-e4562214bfc1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c/action" status: 200 len: 24573 microversion: 2.1 time: 0.122544 Sep 27 10:35:01.193608 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3424/6384] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:35:01 2022] POST /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c/action => generated 24573 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:35:40.214309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c532aed4-af1a-4954-aa02-a0eb3f67fad1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:40.216945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c532aed4-af1a-4954-aa02-a0eb3f67fad1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:40.220307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=17 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:40.221310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=75,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:40.223041 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 10:35:40.223461 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 10:35:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c (ip 10.222.0.26) !!! Sep 27 10:35:40.246168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c532aed4-af1a-4954-aa02-a0eb3f67fad1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:40.246690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c532aed4-af1a-4954-aa02-a0eb3f67fad1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:40.247143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c532aed4-af1a-4954-aa02-a0eb3f67fad1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:40.247933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c532aed4-af1a-4954-aa02-a0eb3f67fad1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:40.248983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c532aed4-af1a-4954-aa02-a0eb3f67fad1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:40.249682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c532aed4-af1a-4954-aa02-a0eb3f67fad1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:40.603994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c532aed4-af1a-4954-aa02-a0eb3f67fad1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:40.612624 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c532aed4-af1a-4954-aa02-a0eb3f67fad1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1682 microversion: 2.1 time: 0.405590 Sep 27 10:35:40.613542 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2961/6385] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:40 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1682 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:40.720279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15e4a340-675d-4ee2-a89c-d58eebc0fc0c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:35:40.722975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-15e4a340-675d-4ee2-a89c-d58eebc0fc0c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:40.726044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:40.727054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:35:40.735864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15e4a340-675d-4ee2-a89c-d58eebc0fc0c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:40.736595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15e4a340-675d-4ee2-a89c-d58eebc0fc0c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:40.737224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15e4a340-675d-4ee2-a89c-d58eebc0fc0c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:40.738090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15e4a340-675d-4ee2-a89c-d58eebc0fc0c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:40.738686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15e4a340-675d-4ee2-a89c-d58eebc0fc0c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:40.739254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15e4a340-675d-4ee2-a89c-d58eebc0fc0c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:40.812474 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-15e4a340-675d-4ee2-a89c-d58eebc0fc0c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c/action" status: 200 len: 32637 microversion: 2.1 time: 0.096667 Sep 27 10:35:40.813273 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3425/6386] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:35:40 2022] POST /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c/action => generated 32637 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:35:40.969914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f90517ce-fcde-4682-923f-b2668f29d2dc tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:40.970954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f90517ce-fcde-4682-923f-b2668f29d2dc tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:40.978015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90517ce-fcde-4682-923f-b2668f29d2dc tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:40.979032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90517ce-fcde-4682-923f-b2668f29d2dc tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:40.979689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90517ce-fcde-4682-923f-b2668f29d2dc tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:40.980625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90517ce-fcde-4682-923f-b2668f29d2dc tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:40.981306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90517ce-fcde-4682-923f-b2668f29d2dc tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:40.981940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f90517ce-fcde-4682-923f-b2668f29d2dc tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:41.230369 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f90517ce-fcde-4682-923f-b2668f29d2dc tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1833 microversion: 2.1 time: 0.264365 Sep 27 10:35:41.241543 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2962/6387] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:40 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1833 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:41.264165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-09bf8b16-3bb5-463c-8391-b2e288392adb tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Action: 'action', calling method: >, body: {"os-migrateLive": {"host": null, "block_migration": true, "disk_over_commit": false}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:35:41.267649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-09bf8b16-3bb5-463c-8391-b2e288392adb tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:41.283072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09bf8b16-3bb5-463c-8391-b2e288392adb tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:41.283805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09bf8b16-3bb5-463c-8391-b2e288392adb tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:41.284251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09bf8b16-3bb5-463c-8391-b2e288392adb tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:41.285029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09bf8b16-3bb5-463c-8391-b2e288392adb tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:41.285492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09bf8b16-3bb5-463c-8391-b2e288392adb tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:41.285923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09bf8b16-3bb5-463c-8391-b2e288392adb tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:41.334828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-09bf8b16-3bb5-463c-8391-b2e288392adb tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lazy-loading 'flavor' on Instance uuid 7036fba1-2704-49b3-9106-b4eda4d25d8c {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:35:41.369743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-09bf8b16-3bb5-463c-8391-b2e288392adb tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Going to try to live migrate instance to another host {{(pid=145968) live_migrate /opt/stack/nova/nova/compute/api.py:5482}} Sep 27 10:35:43.835094 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-09bf8b16-3bb5-463c-8391-b2e288392adb tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c/action" status: 202 len: 0 microversion: 2.1 time: 2.576370 Sep 27 10:35:43.835649 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3426/6388] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:35:41 2022] POST /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c/action => generated 0 bytes in 2578 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:35:43.845447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4411a627-c5c2-499c-917c-b2afb9e41817 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:43.846890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4411a627-c5c2-499c-917c-b2afb9e41817 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:43.857809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4411a627-c5c2-499c-917c-b2afb9e41817 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:43.858695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4411a627-c5c2-499c-917c-b2afb9e41817 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:43.859542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4411a627-c5c2-499c-917c-b2afb9e41817 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:43.862994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4411a627-c5c2-499c-917c-b2afb9e41817 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:43.863713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4411a627-c5c2-499c-917c-b2afb9e41817 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:43.864689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4411a627-c5c2-499c-917c-b2afb9e41817 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:44.195712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4411a627-c5c2-499c-917c-b2afb9e41817 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:44.212210 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4411a627-c5c2-499c-917c-b2afb9e41817 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1692 microversion: 2.1 time: 0.371092 Sep 27 10:35:44.227006 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2963/6389] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:43 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1692 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:45.241232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62e0a7a4-9eea-458a-a768-fd1b7750e778 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:45.241862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-62e0a7a4-9eea-458a-a768-fd1b7750e778 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:45.249154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e0a7a4-9eea-458a-a768-fd1b7750e778 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:45.249901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e0a7a4-9eea-458a-a768-fd1b7750e778 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:45.250745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e0a7a4-9eea-458a-a768-fd1b7750e778 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:45.253201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e0a7a4-9eea-458a-a768-fd1b7750e778 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:45.253843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e0a7a4-9eea-458a-a768-fd1b7750e778 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:45.254446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62e0a7a4-9eea-458a-a768-fd1b7750e778 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:45.631213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-62e0a7a4-9eea-458a-a768-fd1b7750e778 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:45.640349 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-62e0a7a4-9eea-458a-a768-fd1b7750e778 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1692 microversion: 2.1 time: 0.402817 Sep 27 10:35:45.641122 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3427/6390] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:45 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1692 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:46.141565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7576f841-7827-4268-9b34-f47bb582a0d2 req-4f2c2fbc-6a93-40fe-8396-8cce0fcfc1cd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7036fba1-2704-49b3-9106-b4eda4d25d8c", "name": "network-vif-unplugged", "status": "completed", "tag": "daa7ce1e-3299-491a-a2c8-08296813b09c"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:35:46.158106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7576f841-7827-4268-9b34-f47bb582a0d2 req-4f2c2fbc-6a93-40fe-8396-8cce0fcfc1cd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:46.158607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7576f841-7827-4268-9b34-f47bb582a0d2 req-4f2c2fbc-6a93-40fe-8396-8cce0fcfc1cd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:46.166855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7576f841-7827-4268-9b34-f47bb582a0d2 req-4f2c2fbc-6a93-40fe-8396-8cce0fcfc1cd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:46.184324 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7576f841-7827-4268-9b34-f47bb582a0d2 req-4f2c2fbc-6a93-40fe-8396-8cce0fcfc1cd service nova] Creating event network-vif-unplugged:daa7ce1e-3299-491a-a2c8-08296813b09c for instance 7036fba1-2704-49b3-9106-b4eda4d25d8c on nch2-master-1 Sep 27 10:35:46.198470 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7576f841-7827-4268-9b34-f47bb582a0d2 req-4f2c2fbc-6a93-40fe-8396-8cce0fcfc1cd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.063209 Sep 27 10:35:46.199570 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2964/6391] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:35:46 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:35:46.664519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a3710016-491e-40f3-8017-01ab3403ad33 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:46.665199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a3710016-491e-40f3-8017-01ab3403ad33 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:46.674855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3710016-491e-40f3-8017-01ab3403ad33 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:46.675393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3710016-491e-40f3-8017-01ab3403ad33 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:46.675854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3710016-491e-40f3-8017-01ab3403ad33 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:46.676892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3710016-491e-40f3-8017-01ab3403ad33 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:46.677385 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3710016-491e-40f3-8017-01ab3403ad33 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:46.678095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3710016-491e-40f3-8017-01ab3403ad33 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:47.033827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a3710016-491e-40f3-8017-01ab3403ad33 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:47.048980 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a3710016-491e-40f3-8017-01ab3403ad33 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1692 microversion: 2.1 time: 0.388896 Sep 27 10:35:47.083300 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3428/6392] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:46 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1692 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:48.080306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b9982218-637c-4bbe-b273-cde57a17927b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:48.081459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b9982218-637c-4bbe-b273-cde57a17927b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:48.089569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9982218-637c-4bbe-b273-cde57a17927b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:48.090152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9982218-637c-4bbe-b273-cde57a17927b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:48.090812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9982218-637c-4bbe-b273-cde57a17927b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:48.091592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9982218-637c-4bbe-b273-cde57a17927b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:48.092426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9982218-637c-4bbe-b273-cde57a17927b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:48.092909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9982218-637c-4bbe-b273-cde57a17927b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:48.227194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f88d4a4a-7b56-4ad4-b3fc-7c2c91abd529 req-61eb6e23-6eb1-45d6-88f0-d82b4edf830a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7036fba1-2704-49b3-9106-b4eda4d25d8c", "tag": "daa7ce1e-3299-491a-a2c8-08296813b09c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:35:48.243652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f88d4a4a-7b56-4ad4-b3fc-7c2c91abd529 req-61eb6e23-6eb1-45d6-88f0-d82b4edf830a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:48.246065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f88d4a4a-7b56-4ad4-b3fc-7c2c91abd529 req-61eb6e23-6eb1-45d6-88f0-d82b4edf830a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:48.246493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f88d4a4a-7b56-4ad4-b3fc-7c2c91abd529 req-61eb6e23-6eb1-45d6-88f0-d82b4edf830a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:48.274450 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f88d4a4a-7b56-4ad4-b3fc-7c2c91abd529 req-61eb6e23-6eb1-45d6-88f0-d82b4edf830a service nova] Creating event network-changed:daa7ce1e-3299-491a-a2c8-08296813b09c for instance 7036fba1-2704-49b3-9106-b4eda4d25d8c on nch2-master-1 Sep 27 10:35:48.281034 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f88d4a4a-7b56-4ad4-b3fc-7c2c91abd529 req-61eb6e23-6eb1-45d6-88f0-d82b4edf830a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057472 Sep 27 10:35:48.282236 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3429/6393] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:35:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:35:48.385837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b9982218-637c-4bbe-b273-cde57a17927b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:48.408821 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b9982218-637c-4bbe-b273-cde57a17927b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1692 microversion: 2.1 time: 0.333678 Sep 27 10:35:48.409417 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2965/6394] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:48 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1692 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:49.436864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5be8af82-b148-4f28-8b4b-de12500721b3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:49.443049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5be8af82-b148-4f28-8b4b-de12500721b3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:49.456907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5be8af82-b148-4f28-8b4b-de12500721b3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:49.458164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5be8af82-b148-4f28-8b4b-de12500721b3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:49.458854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5be8af82-b148-4f28-8b4b-de12500721b3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:49.459816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5be8af82-b148-4f28-8b4b-de12500721b3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:49.460290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5be8af82-b148-4f28-8b4b-de12500721b3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:49.460754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5be8af82-b148-4f28-8b4b-de12500721b3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:49.758118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5be8af82-b148-4f28-8b4b-de12500721b3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:49.771772 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5be8af82-b148-4f28-8b4b-de12500721b3 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1692 microversion: 2.1 time: 0.342578 Sep 27 10:35:49.773392 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3430/6395] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:49 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1692 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:50.803573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-65d40302-8c94-4549-91db-0ca3295d04be tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:50.804982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-65d40302-8c94-4549-91db-0ca3295d04be tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:50.814179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d40302-8c94-4549-91db-0ca3295d04be tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:50.814683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d40302-8c94-4549-91db-0ca3295d04be tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:50.815133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d40302-8c94-4549-91db-0ca3295d04be tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:50.815864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d40302-8c94-4549-91db-0ca3295d04be tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:50.816460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d40302-8c94-4549-91db-0ca3295d04be tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:50.816894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65d40302-8c94-4549-91db-0ca3295d04be tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:51.181184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-65d40302-8c94-4549-91db-0ca3295d04be tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:51.195212 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-65d40302-8c94-4549-91db-0ca3295d04be tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1692 microversion: 2.1 time: 0.395244 Sep 27 10:35:51.196921 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2966/6396] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:50 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1692 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:52.222299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-79af8fa1-7447-49b9-b177-c3719f0b23d8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:52.224838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-79af8fa1-7447-49b9-b177-c3719f0b23d8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:52.235390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79af8fa1-7447-49b9-b177-c3719f0b23d8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:52.235893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79af8fa1-7447-49b9-b177-c3719f0b23d8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:52.236342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79af8fa1-7447-49b9-b177-c3719f0b23d8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:52.237105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79af8fa1-7447-49b9-b177-c3719f0b23d8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:52.237646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79af8fa1-7447-49b9-b177-c3719f0b23d8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:52.238005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79af8fa1-7447-49b9-b177-c3719f0b23d8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:52.562633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-79af8fa1-7447-49b9-b177-c3719f0b23d8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:52.574264 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-79af8fa1-7447-49b9-b177-c3719f0b23d8 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1692 microversion: 2.1 time: 0.356517 Sep 27 10:35:52.579870 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3431/6397] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:52 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1692 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:53.606288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-16e5fc14-fc60-4a31-a6eb-175ca8022f9f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:53.606962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-16e5fc14-fc60-4a31-a6eb-175ca8022f9f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:53.616707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16e5fc14-fc60-4a31-a6eb-175ca8022f9f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:53.617516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16e5fc14-fc60-4a31-a6eb-175ca8022f9f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:53.618039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16e5fc14-fc60-4a31-a6eb-175ca8022f9f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:53.618837 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16e5fc14-fc60-4a31-a6eb-175ca8022f9f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:53.619673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16e5fc14-fc60-4a31-a6eb-175ca8022f9f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:53.620270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-16e5fc14-fc60-4a31-a6eb-175ca8022f9f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:53.851039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-16e5fc14-fc60-4a31-a6eb-175ca8022f9f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:53.860691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-16e5fc14-fc60-4a31-a6eb-175ca8022f9f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1692 microversion: 2.1 time: 0.260151 Sep 27 10:35:53.861618 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2967/6398] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:53 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1692 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:54.889338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0c903f36-67ea-4dd4-8d88-bf36f877e63b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:54.890750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0c903f36-67ea-4dd4-8d88-bf36f877e63b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:54.900579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c903f36-67ea-4dd4-8d88-bf36f877e63b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:54.901535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c903f36-67ea-4dd4-8d88-bf36f877e63b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:54.902308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c903f36-67ea-4dd4-8d88-bf36f877e63b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:54.903425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c903f36-67ea-4dd4-8d88-bf36f877e63b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:54.905058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c903f36-67ea-4dd4-8d88-bf36f877e63b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:54.905659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c903f36-67ea-4dd4-8d88-bf36f877e63b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:55.262576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0c903f36-67ea-4dd4-8d88-bf36f877e63b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:55.278484 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0c903f36-67ea-4dd4-8d88-bf36f877e63b tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1692 microversion: 2.1 time: 0.394282 Sep 27 10:35:55.279923 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3432/6399] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:54 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1692 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:56.307184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea19ca58-5ecf-479a-bf5a-98d2e8260968 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:56.307791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ea19ca58-5ecf-479a-bf5a-98d2e8260968 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:56.326975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea19ca58-5ecf-479a-bf5a-98d2e8260968 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:56.327902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea19ca58-5ecf-479a-bf5a-98d2e8260968 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:56.328579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea19ca58-5ecf-479a-bf5a-98d2e8260968 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:56.331034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea19ca58-5ecf-479a-bf5a-98d2e8260968 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:56.332343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea19ca58-5ecf-479a-bf5a-98d2e8260968 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:56.332851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea19ca58-5ecf-479a-bf5a-98d2e8260968 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:56.521434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ea19ca58-5ecf-479a-bf5a-98d2e8260968 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:56.532701 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea19ca58-5ecf-479a-bf5a-98d2e8260968 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1682 microversion: 2.1 time: 0.229096 Sep 27 10:35:56.533890 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2968/6400] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:56 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1682 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:56.557825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e190543f-a7cd-49bf-89eb-5ebe0e8a54f6 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:56.558440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e190543f-a7cd-49bf-89eb-5ebe0e8a54f6 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:56.565945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e190543f-a7cd-49bf-89eb-5ebe0e8a54f6 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:56.566602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e190543f-a7cd-49bf-89eb-5ebe0e8a54f6 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:56.567092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e190543f-a7cd-49bf-89eb-5ebe0e8a54f6 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:56.568183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e190543f-a7cd-49bf-89eb-5ebe0e8a54f6 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:56.568640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e190543f-a7cd-49bf-89eb-5ebe0e8a54f6 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:56.569234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e190543f-a7cd-49bf-89eb-5ebe0e8a54f6 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:56.885453 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e190543f-a7cd-49bf-89eb-5ebe0e8a54f6 tempest-TestNetworkAdvancedServerOps-1633971983 tempest-TestNetworkAdvancedServerOps-1633971983-project-admin] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1833 microversion: 2.1 time: 0.332554 Sep 27 10:35:56.889145 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3433/6401] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:56 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1833 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:56.921371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-059c118a-0ac2-461b-9b38-c8d27f80cd63 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:35:56.922705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-059c118a-0ac2-461b-9b38-c8d27f80cd63 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:56.931926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-059c118a-0ac2-461b-9b38-c8d27f80cd63 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:56.932794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-059c118a-0ac2-461b-9b38-c8d27f80cd63 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:56.933404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-059c118a-0ac2-461b-9b38-c8d27f80cd63 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:56.934337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-059c118a-0ac2-461b-9b38-c8d27f80cd63 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:56.934954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-059c118a-0ac2-461b-9b38-c8d27f80cd63 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:56.935535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-059c118a-0ac2-461b-9b38-c8d27f80cd63 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:57.292135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-059c118a-0ac2-461b-9b38-c8d27f80cd63 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:35:57.300609 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-059c118a-0ac2-461b-9b38-c8d27f80cd63 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1682 microversion: 2.1 time: 0.390922 Sep 27 10:35:57.305769 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2969/6402] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:35:56 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1682 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:35:59.895919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3c2f7dc9-cc01-4840-8051-70774da90748 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:35:59.899004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3c2f7dc9-cc01-4840-8051-70774da90748 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:35:59.916598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c2f7dc9-cc01-4840-8051-70774da90748 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:59.917349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c2f7dc9-cc01-4840-8051-70774da90748 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:59.921244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c2f7dc9-cc01-4840-8051-70774da90748 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:35:59.922193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c2f7dc9-cc01-4840-8051-70774da90748 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:35:59.922829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c2f7dc9-cc01-4840-8051-70774da90748 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:35:59.924047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c2f7dc9-cc01-4840-8051-70774da90748 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:00.022905 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3c2f7dc9-cc01-4840-8051-70774da90748 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c/action" status: 200 len: 38405 microversion: 2.1 time: 0.131292 Sep 27 10:36:00.023588 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3434/6403] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:35:59 2022] POST /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c/action => generated 38405 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 27 10:36:02.139413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2f8408e5-2471-40ac-9201-c3ec4a43d482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:02.139943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2f8408e5-2471-40ac-9201-c3ec4a43d482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:02.144580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-026ae68b-7d06-4dba-95c2-bf34285c902f req-3a5def50-7445-46b3-972c-72e4b6b23326 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7036fba1-2704-49b3-9106-b4eda4d25d8c", "tag": "daa7ce1e-3299-491a-a2c8-08296813b09c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:36:02.148498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8408e5-2471-40ac-9201-c3ec4a43d482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:02.149046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8408e5-2471-40ac-9201-c3ec4a43d482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:02.149510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8408e5-2471-40ac-9201-c3ec4a43d482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:02.150243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8408e5-2471-40ac-9201-c3ec4a43d482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:02.150720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8408e5-2471-40ac-9201-c3ec4a43d482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:02.151168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2f8408e5-2471-40ac-9201-c3ec4a43d482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:02.171843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-026ae68b-7d06-4dba-95c2-bf34285c902f req-3a5def50-7445-46b3-972c-72e4b6b23326 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:02.179168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-026ae68b-7d06-4dba-95c2-bf34285c902f req-3a5def50-7445-46b3-972c-72e4b6b23326 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:02.179887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-026ae68b-7d06-4dba-95c2-bf34285c902f req-3a5def50-7445-46b3-972c-72e4b6b23326 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:02.205082 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-026ae68b-7d06-4dba-95c2-bf34285c902f req-3a5def50-7445-46b3-972c-72e4b6b23326 service nova] Creating event network-changed:daa7ce1e-3299-491a-a2c8-08296813b09c for instance 7036fba1-2704-49b3-9106-b4eda4d25d8c on nch1-master-1 Sep 27 10:36:02.217096 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-026ae68b-7d06-4dba-95c2-bf34285c902f req-3a5def50-7445-46b3-972c-72e4b6b23326 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077071 Sep 27 10:36:02.219451 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3435/6404] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:36:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:36:02.241882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2f8408e5-2471-40ac-9201-c3ec4a43d482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:36:02.467707 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2f8408e5-2471-40ac-9201-c3ec4a43d482 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 204 len: 0 microversion: 2.1 time: 0.332556 Sep 27 10:36:02.468887 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2970/6405] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:36:02 2022] DELETE /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 0 bytes in 335 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:36:02.478539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f1869713-7aa8-454b-8ba6-1e9c1e085a17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:02.479328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f1869713-7aa8-454b-8ba6-1e9c1e085a17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:02.504891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1869713-7aa8-454b-8ba6-1e9c1e085a17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:02.506485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1869713-7aa8-454b-8ba6-1e9c1e085a17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:02.507154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1869713-7aa8-454b-8ba6-1e9c1e085a17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:02.509306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1869713-7aa8-454b-8ba6-1e9c1e085a17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:02.515744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1869713-7aa8-454b-8ba6-1e9c1e085a17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:02.516463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f1869713-7aa8-454b-8ba6-1e9c1e085a17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:02.846603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f1869713-7aa8-454b-8ba6-1e9c1e085a17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:02.858337 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f1869713-7aa8-454b-8ba6-1e9c1e085a17 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1688 microversion: 2.1 time: 0.384548 Sep 27 10:36:02.859361 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3436/6406] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:02 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1688 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:03.890042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0fc22c96-5eb1-46cd-8fd9-19e060247283 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:03.891170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0fc22c96-5eb1-46cd-8fd9-19e060247283 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:03.917368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fc22c96-5eb1-46cd-8fd9-19e060247283 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:03.918315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fc22c96-5eb1-46cd-8fd9-19e060247283 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:03.919346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fc22c96-5eb1-46cd-8fd9-19e060247283 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:03.920350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fc22c96-5eb1-46cd-8fd9-19e060247283 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:03.921062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fc22c96-5eb1-46cd-8fd9-19e060247283 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:03.921658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fc22c96-5eb1-46cd-8fd9-19e060247283 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:04.329560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0fc22c96-5eb1-46cd-8fd9-19e060247283 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:04.333366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-04ad78b2-e881-4161-983b-92cff41e2700 req-a698664c-c6cc-4b0a-a7d0-c8567ed28433 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7036fba1-2704-49b3-9106-b4eda4d25d8c", "name": "network-vif-deleted", "tag": "daa7ce1e-3299-491a-a2c8-08296813b09c"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:36:04.366409 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0fc22c96-5eb1-46cd-8fd9-19e060247283 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 200 len: 1569 microversion: 2.1 time: 0.482879 Sep 27 10:36:04.382122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04ad78b2-e881-4161-983b-92cff41e2700 req-a698664c-c6cc-4b0a-a7d0-c8567ed28433 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:04.382618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04ad78b2-e881-4161-983b-92cff41e2700 req-a698664c-c6cc-4b0a-a7d0-c8567ed28433 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:04.383105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-04ad78b2-e881-4161-983b-92cff41e2700 req-a698664c-c6cc-4b0a-a7d0-c8567ed28433 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:04.399336 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-04ad78b2-e881-4161-983b-92cff41e2700 req-a698664c-c6cc-4b0a-a7d0-c8567ed28433 service nova] Creating event network-vif-deleted:daa7ce1e-3299-491a-a2c8-08296813b09c for instance 7036fba1-2704-49b3-9106-b4eda4d25d8c on nch1-master-1 Sep 27 10:36:04.403452 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2971/6407] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:03 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 1569 bytes in 521 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:04.407594 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-04ad78b2-e881-4161-983b-92cff41e2700 req-a698664c-c6cc-4b0a-a7d0-c8567ed28433 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084464 Sep 27 10:36:04.408824 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3437/6408] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:36:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:36:05.401929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:05.402558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 7036fba1-2704-49b3-9106-b4eda4d25d8c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:05.419899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:05.420771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:05.421456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:05.422974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:05.423668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:05.424274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:05.471097 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] HTTP exception thrown: Instance 7036fba1-2704-49b3-9106-b4eda4d25d8c could not be found. Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Request method failure captured: Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c HTTP/1.1 Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtAz-I5R9pkEmM8ENwgEXlrfwhpK5QJgpteQwRSpBNpzmPCQ6_jXCbvGwjb68dGwzfz0hV_obx0eVwNZLFOTRd61-pMOQhq1CSGhYJOTDYOHjRjmEKJs84qfkSC88em1BS9SfLSGMC-OPias42e-aueDGLNj17wWnJe-vn54z9Dzb2o Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 86e71212d96f46a78680afaacd8709ca Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_86e71212d96f46a78680afaacd8709ca"}]}] Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 86e71212d96f46a78680afaacd8709ca Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestNetworkAdvancedServerOps-1611618676-project Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 0bd8a19d0ab44011b5430f26757a0efe Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1611618676-project Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 7036fba1-2704-49b3-9106-b4eda4d25d8c could not be found. Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '7036fba1-2704-49b3-9106-b4eda4d25d8c'} Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7036fba1-2704-49b3-9106-b4eda4d25d8c could not be found. Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7036fba1-2704-49b3-9106-b4eda4d25d8c could not be found. Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:36:05.472776 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:36:05.476404 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:36:05.476404 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:36:05.476404 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:36:05.476404 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:36:05.476404 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:36:05.476404 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:36:05.476404 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7036fba1-2704-49b3-9106-b4eda4d25d8c could not be found. Sep 27 10:36:05.476404 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:36:05.481337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Returning 404 to user: Instance 7036fba1-2704-49b3-9106-b4eda4d25d8c could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:36:05.482309 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6776af57-0d6f-48bb-bb55-ccedb466a723 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c" status: 404 len: 111 microversion: 2.1 time: 0.084447 Sep 27 10:36:05.482913 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2972/6409] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:05 2022] GET /compute/v2.1/servers/7036fba1-2704-49b3-9106-b4eda4d25d8c => generated 111 bytes in 86 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:36:11.220596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d320d46-6768-4105-bc1c-efaeda4b6095 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '.version_select at 0x7f7f6242a670>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:11.236959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=29 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:11.260713 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d320d46-6768-4105-bc1c-efaeda4b6095 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1604356436" status: 202 len: 0 microversion: 2.1 time: 0.042028 Sep 27 10:36:11.260713 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3438/6410] 10.222.0.26 () {58 vars in 1284 bytes} [Tue Sep 27 10:36:11 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1604356436 => generated 0 bytes in 49 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:36:11.287753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ff75d420-0973-4eec-904c-37ca11fa162d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'create', calling method: .version_select at 0x7f7f6360a670>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-955922546", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBN1om6z289EZebukJRTk39emYEytky52N7/SytjgHRkhq6mCSowVLo6asdf88UoFdsBNPBe6qWo08VyX+/l+ALLLbNr2n2FArZgTBQ7RMjo2V0cEGZwbvPdU1twrWBIIbg=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:36:11.294586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=30,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:11.349315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ff75d420-0973-4eec-904c-37ca11fa162d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for project 86e71212d96f46a78680afaacd8709ca. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:36:11.364859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-ff75d420-0973-4eec-904c-37ca11fa162d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for user 0bd8a19d0ab44011b5430f26757a0efe and project 86e71212d96f46a78680afaacd8709ca. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:36:11.401410 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ff75d420-0973-4eec-904c-37ca11fa162d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 0.117253 Sep 27 10:36:11.402020 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2973/6411] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:36:11 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:36:25.562290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-67026761", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "be577d47-0fd7-4172-9cac-67e47426f894"}], "key_name": "tempest-TestNetworkAdvancedServerOps-955922546", "security_groups": [{"name": "tempest-secgroup-smoke-1867028513"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:36:25.583422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=69,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:25.584808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=2,nova_api:DELETE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:25.766967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] validate_networks() for [('be577d47-0fd7-4172-9cac-67e47426f894', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:36:26.959947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:36:26.962095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:36:26.962751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:36:26.963855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:36:26.964670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:36:26.980002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:26.980897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:26.981532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:26.982494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:26.983161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:26.983792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:27.005032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for project 86e71212d96f46a78680afaacd8709ca. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:36:27.011958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Getting quotas for user 0bd8a19d0ab44011b5430f26757a0efe and project 86e71212d96f46a78680afaacd8709ca. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:36:27.027912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:36:27.031024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:36:27.091734 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7a62a33a-5836-4763-9e00-a636c087b1fb tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.532265 Sep 27 10:36:27.092906 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3439/6412] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:36:25 2022] POST /compute/v2.1/servers => generated 406 bytes in 1535 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:36:27.103987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9b39a578-891a-44fc-a337-4b24f5405051 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:27.105683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9b39a578-891a-44fc-a337-4b24f5405051 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:27.109296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=72,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:27.110128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:27.111056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=10,nova_api:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:27.182144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9b39a578-891a-44fc-a337-4b24f5405051 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:27.192942 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9b39a578-891a-44fc-a337-4b24f5405051 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1264 microversion: 2.1 time: 0.090657 Sep 27 10:36:27.211822 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2974/6413] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:27 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1264 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:28.220018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a5ee2238-17f7-425a-9fee-7e9d3085b51a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:28.220646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a5ee2238-17f7-425a-9fee-7e9d3085b51a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:28.233836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5ee2238-17f7-425a-9fee-7e9d3085b51a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:28.234321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5ee2238-17f7-425a-9fee-7e9d3085b51a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:28.234779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5ee2238-17f7-425a-9fee-7e9d3085b51a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:28.235509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5ee2238-17f7-425a-9fee-7e9d3085b51a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:28.235976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5ee2238-17f7-425a-9fee-7e9d3085b51a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:28.236417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a5ee2238-17f7-425a-9fee-7e9d3085b51a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:28.357636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a5ee2238-17f7-425a-9fee-7e9d3085b51a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:28.367710 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a5ee2238-17f7-425a-9fee-7e9d3085b51a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1316 microversion: 2.1 time: 0.151597 Sep 27 10:36:28.369641 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3440/6414] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:28 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1316 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:29.393173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a2694cef-1590-4087-b4ae-92134b9b1476 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:29.394028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a2694cef-1590-4087-b4ae-92134b9b1476 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:29.402005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2694cef-1590-4087-b4ae-92134b9b1476 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:29.402665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2694cef-1590-4087-b4ae-92134b9b1476 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:29.403238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2694cef-1590-4087-b4ae-92134b9b1476 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:29.404352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2694cef-1590-4087-b4ae-92134b9b1476 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:29.404640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2694cef-1590-4087-b4ae-92134b9b1476 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:29.405187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2694cef-1590-4087-b4ae-92134b9b1476 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:29.546848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a2694cef-1590-4087-b4ae-92134b9b1476 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:29.556154 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a2694cef-1590-4087-b4ae-92134b9b1476 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1322 microversion: 2.1 time: 0.167629 Sep 27 10:36:29.572551 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2975/6415] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:29 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1322 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:30.580751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4b325f77-f819-4ea6-8244-1712ac50bd48 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:30.581497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4b325f77-f819-4ea6-8244-1712ac50bd48 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:30.590003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b325f77-f819-4ea6-8244-1712ac50bd48 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:30.590733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b325f77-f819-4ea6-8244-1712ac50bd48 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:30.591181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b325f77-f819-4ea6-8244-1712ac50bd48 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:30.591913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b325f77-f819-4ea6-8244-1712ac50bd48 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:30.592376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b325f77-f819-4ea6-8244-1712ac50bd48 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:30.592853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b325f77-f819-4ea6-8244-1712ac50bd48 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:30.671667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4b325f77-f819-4ea6-8244-1712ac50bd48 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:30.686200 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4b325f77-f819-4ea6-8244-1712ac50bd48 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1322 microversion: 2.1 time: 0.110092 Sep 27 10:36:30.686802 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3441/6416] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:30 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1322 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:31.710443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-557e5ff4-f584-4947-9988-588a77b5eeb4 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:31.711791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-557e5ff4-f584-4947-9988-588a77b5eeb4 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:31.723045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557e5ff4-f584-4947-9988-588a77b5eeb4 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:31.723816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557e5ff4-f584-4947-9988-588a77b5eeb4 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:31.724883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557e5ff4-f584-4947-9988-588a77b5eeb4 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:31.726087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557e5ff4-f584-4947-9988-588a77b5eeb4 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:31.727199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557e5ff4-f584-4947-9988-588a77b5eeb4 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:31.728459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-557e5ff4-f584-4947-9988-588a77b5eeb4 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:32.133571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-557e5ff4-f584-4947-9988-588a77b5eeb4 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:32.142034 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-557e5ff4-f584-4947-9988-588a77b5eeb4 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1390 microversion: 2.1 time: 0.435971 Sep 27 10:36:32.157609 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2976/6417] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:31 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1390 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:32.837791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-25cf42e6-ed39-4aa3-b35b-aa8065f3a42b req-84bb624b-ba02-4a8b-b116-74c440f3d122 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "38097746-54a6-4433-924a-477fcd11f84a", "tag": "3ece88ca-1593-4172-b7bb-d9d53cd7b4d3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:36:32.853146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-25cf42e6-ed39-4aa3-b35b-aa8065f3a42b req-84bb624b-ba02-4a8b-b116-74c440f3d122 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:32.853436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-25cf42e6-ed39-4aa3-b35b-aa8065f3a42b req-84bb624b-ba02-4a8b-b116-74c440f3d122 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:32.854028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-25cf42e6-ed39-4aa3-b35b-aa8065f3a42b req-84bb624b-ba02-4a8b-b116-74c440f3d122 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:32.881780 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-25cf42e6-ed39-4aa3-b35b-aa8065f3a42b req-84bb624b-ba02-4a8b-b116-74c440f3d122 service nova] Creating event network-changed:3ece88ca-1593-4172-b7bb-d9d53cd7b4d3 for instance 38097746-54a6-4433-924a-477fcd11f84a on nch2-master-1 Sep 27 10:36:32.891041 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-25cf42e6-ed39-4aa3-b35b-aa8065f3a42b req-84bb624b-ba02-4a8b-b116-74c440f3d122 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057182 Sep 27 10:36:32.892283 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3442/6418] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:36:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:36:33.167176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a4a066cb-c689-4808-b209-3cdbbfe7de6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:33.167826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a4a066cb-c689-4808-b209-3cdbbfe7de6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:33.181624 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4a066cb-c689-4808-b209-3cdbbfe7de6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:33.182142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4a066cb-c689-4808-b209-3cdbbfe7de6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:33.182607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4a066cb-c689-4808-b209-3cdbbfe7de6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:33.183349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4a066cb-c689-4808-b209-3cdbbfe7de6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:33.183999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4a066cb-c689-4808-b209-3cdbbfe7de6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:33.184464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4a066cb-c689-4808-b209-3cdbbfe7de6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:33.550519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a4a066cb-c689-4808-b209-3cdbbfe7de6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:33.560314 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a4a066cb-c689-4808-b209-3cdbbfe7de6d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1390 microversion: 2.1 time: 0.397426 Sep 27 10:36:33.560914 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2977/6419] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:33 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1390 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:34.590144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-22db3fd1-2d25-4822-a678-f01da301e435 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:34.590966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-22db3fd1-2d25-4822-a678-f01da301e435 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:34.598263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22db3fd1-2d25-4822-a678-f01da301e435 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:34.599142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22db3fd1-2d25-4822-a678-f01da301e435 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:34.599683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22db3fd1-2d25-4822-a678-f01da301e435 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:34.600690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22db3fd1-2d25-4822-a678-f01da301e435 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:34.601350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22db3fd1-2d25-4822-a678-f01da301e435 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:34.601979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-22db3fd1-2d25-4822-a678-f01da301e435 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:34.913202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-22db3fd1-2d25-4822-a678-f01da301e435 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:34.932716 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-22db3fd1-2d25-4822-a678-f01da301e435 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1390 microversion: 2.1 time: 0.347645 Sep 27 10:36:34.953654 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3443/6420] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:34 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1390 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:35.962600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-42252b92-4262-4d24-b3a7-9f078ad474e9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:35.963776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-42252b92-4262-4d24-b3a7-9f078ad474e9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:35.971038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42252b92-4262-4d24-b3a7-9f078ad474e9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:36.001199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42252b92-4262-4d24-b3a7-9f078ad474e9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:36.001454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42252b92-4262-4d24-b3a7-9f078ad474e9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:36.002393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42252b92-4262-4d24-b3a7-9f078ad474e9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:36.003204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42252b92-4262-4d24-b3a7-9f078ad474e9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:36.005376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42252b92-4262-4d24-b3a7-9f078ad474e9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:36.392426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-42252b92-4262-4d24-b3a7-9f078ad474e9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:36.420639 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-42252b92-4262-4d24-b3a7-9f078ad474e9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1390 microversion: 2.1 time: 0.463835 Sep 27 10:36:36.421422 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2978/6421] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:35 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1390 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:37.444836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b5cebd8-8b4b-4c9f-a14e-ea633fcf0be1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:37.445914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3b5cebd8-8b4b-4c9f-a14e-ea633fcf0be1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:37.450370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:37.451415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:37.458045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b5cebd8-8b4b-4c9f-a14e-ea633fcf0be1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:37.458834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b5cebd8-8b4b-4c9f-a14e-ea633fcf0be1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:37.459373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b5cebd8-8b4b-4c9f-a14e-ea633fcf0be1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:37.460206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b5cebd8-8b4b-4c9f-a14e-ea633fcf0be1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:37.460799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b5cebd8-8b4b-4c9f-a14e-ea633fcf0be1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:37.461332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b5cebd8-8b4b-4c9f-a14e-ea633fcf0be1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:37.710296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3b5cebd8-8b4b-4c9f-a14e-ea633fcf0be1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:37.725626 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3b5cebd8-8b4b-4c9f-a14e-ea633fcf0be1 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1390 microversion: 2.1 time: 0.285507 Sep 27 10:36:37.737563 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3444/6422] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:37 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1390 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:38.750884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d706196d-004f-4af3-868a-498e53f0a3cc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:38.752216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d706196d-004f-4af3-868a-498e53f0a3cc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:38.759144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:36:38.761531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d706196d-004f-4af3-868a-498e53f0a3cc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:38.762399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d706196d-004f-4af3-868a-498e53f0a3cc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:38.763293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d706196d-004f-4af3-868a-498e53f0a3cc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:38.764281 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d706196d-004f-4af3-868a-498e53f0a3cc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:38.764996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d706196d-004f-4af3-868a-498e53f0a3cc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:38.765652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d706196d-004f-4af3-868a-498e53f0a3cc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:39.297259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d706196d-004f-4af3-868a-498e53f0a3cc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:39.316304 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d706196d-004f-4af3-868a-498e53f0a3cc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1390 microversion: 2.1 time: 0.569184 Sep 27 10:36:39.317161 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2979/6423] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:38 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1390 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:40.125387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8df50dda-4e74-478e-95ac-dc4dc3c70ed1 req-3be18182-69d3-4ef5-b5c0-71fb36fd1bd2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38097746-54a6-4433-924a-477fcd11f84a", "name": "network-vif-plugged", "status": "completed", "tag": "3ece88ca-1593-4172-b7bb-d9d53cd7b4d3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:36:40.143557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8df50dda-4e74-478e-95ac-dc4dc3c70ed1 req-3be18182-69d3-4ef5-b5c0-71fb36fd1bd2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:40.144476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8df50dda-4e74-478e-95ac-dc4dc3c70ed1 req-3be18182-69d3-4ef5-b5c0-71fb36fd1bd2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:40.145164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8df50dda-4e74-478e-95ac-dc4dc3c70ed1 req-3be18182-69d3-4ef5-b5c0-71fb36fd1bd2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:40.162213 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8df50dda-4e74-478e-95ac-dc4dc3c70ed1 req-3be18182-69d3-4ef5-b5c0-71fb36fd1bd2 service nova] Creating event network-vif-plugged:3ece88ca-1593-4172-b7bb-d9d53cd7b4d3 for instance 38097746-54a6-4433-924a-477fcd11f84a on nch2-master-1 Sep 27 10:36:40.170357 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8df50dda-4e74-478e-95ac-dc4dc3c70ed1 req-3be18182-69d3-4ef5-b5c0-71fb36fd1bd2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047221 Sep 27 10:36:40.171964 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3445/6424] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:36:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:36:40.340003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d3d2e83d-73f3-4af3-9a26-d292e1e42e51 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:40.340720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d3d2e83d-73f3-4af3-9a26-d292e1e42e51 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:40.349613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3d2e83d-73f3-4af3-9a26-d292e1e42e51 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:40.350457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3d2e83d-73f3-4af3-9a26-d292e1e42e51 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:40.351183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3d2e83d-73f3-4af3-9a26-d292e1e42e51 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:40.352784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3d2e83d-73f3-4af3-9a26-d292e1e42e51 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:40.353529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3d2e83d-73f3-4af3-9a26-d292e1e42e51 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:40.354179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d3d2e83d-73f3-4af3-9a26-d292e1e42e51 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:40.690942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d3d2e83d-73f3-4af3-9a26-d292e1e42e51 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:40.703111 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d3d2e83d-73f3-4af3-9a26-d292e1e42e51 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1390 microversion: 2.1 time: 0.367308 Sep 27 10:36:40.719812 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2980/6425] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:40 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1390 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:41.727527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6af5e6e9-7a0f-4db1-b4b8-696f6f537901 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:41.728979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6af5e6e9-7a0f-4db1-b4b8-696f6f537901 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:41.735952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6af5e6e9-7a0f-4db1-b4b8-696f6f537901 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:41.736775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6af5e6e9-7a0f-4db1-b4b8-696f6f537901 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:41.737484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6af5e6e9-7a0f-4db1-b4b8-696f6f537901 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:41.738423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6af5e6e9-7a0f-4db1-b4b8-696f6f537901 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:41.739092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6af5e6e9-7a0f-4db1-b4b8-696f6f537901 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:41.739732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6af5e6e9-7a0f-4db1-b4b8-696f6f537901 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:42.154563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6af5e6e9-7a0f-4db1-b4b8-696f6f537901 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:42.183026 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6af5e6e9-7a0f-4db1-b4b8-696f6f537901 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1390 microversion: 2.1 time: 0.456831 Sep 27 10:36:42.186421 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3446/6426] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:41 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1390 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:43.212555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ad4674d8-e4ce-4cf5-94b5-03d4634bbbda tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:43.218290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ad4674d8-e4ce-4cf5-94b5-03d4634bbbda tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:43.227323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad4674d8-e4ce-4cf5-94b5-03d4634bbbda tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:43.228918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad4674d8-e4ce-4cf5-94b5-03d4634bbbda tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:43.229692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad4674d8-e4ce-4cf5-94b5-03d4634bbbda tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:43.233002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad4674d8-e4ce-4cf5-94b5-03d4634bbbda tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:43.233496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad4674d8-e4ce-4cf5-94b5-03d4634bbbda tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:43.233944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad4674d8-e4ce-4cf5-94b5-03d4634bbbda tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:43.494682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ad4674d8-e4ce-4cf5-94b5-03d4634bbbda tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:43.503039 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ad4674d8-e4ce-4cf5-94b5-03d4634bbbda tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1562 microversion: 2.1 time: 0.295701 Sep 27 10:36:43.503629 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2981/6427] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:43 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1562 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:43.528208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d89fc2ea-d430-432f-820c-c3e45dc31ae9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:43.528948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d89fc2ea-d430-432f-820c-c3e45dc31ae9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:43.539516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d89fc2ea-d430-432f-820c-c3e45dc31ae9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:43.540102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d89fc2ea-d430-432f-820c-c3e45dc31ae9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:43.540812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d89fc2ea-d430-432f-820c-c3e45dc31ae9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:43.542012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d89fc2ea-d430-432f-820c-c3e45dc31ae9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:43.542862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d89fc2ea-d430-432f-820c-c3e45dc31ae9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:43.543511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d89fc2ea-d430-432f-820c-c3e45dc31ae9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:43.981790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d89fc2ea-d430-432f-820c-c3e45dc31ae9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:43.993078 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d89fc2ea-d430-432f-820c-c3e45dc31ae9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1562 microversion: 2.1 time: 0.469868 Sep 27 10:36:43.994293 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3447/6428] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:43 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1562 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:36:46.469606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-30973a01-8258-4c12-8dfe-9f50bfe8b07f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:36:46.470778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-30973a01-8258-4c12-8dfe-9f50bfe8b07f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:36:46.473554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3e8091ae-84f9-48e8-8dcf-9c46b189b3f4 req-dd6a0f3f-4380-4eae-bd22-260adb43c132 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "38097746-54a6-4433-924a-477fcd11f84a", "tag": "3ece88ca-1593-4172-b7bb-d9d53cd7b4d3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:36:46.479350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30973a01-8258-4c12-8dfe-9f50bfe8b07f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:46.480266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30973a01-8258-4c12-8dfe-9f50bfe8b07f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:46.480904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30973a01-8258-4c12-8dfe-9f50bfe8b07f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:46.481787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30973a01-8258-4c12-8dfe-9f50bfe8b07f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:46.482404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30973a01-8258-4c12-8dfe-9f50bfe8b07f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:46.482991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30973a01-8258-4c12-8dfe-9f50bfe8b07f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:46.491453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3e8091ae-84f9-48e8-8dcf-9c46b189b3f4 req-dd6a0f3f-4380-4eae-bd22-260adb43c132 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:36:46.492211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3e8091ae-84f9-48e8-8dcf-9c46b189b3f4 req-dd6a0f3f-4380-4eae-bd22-260adb43c132 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:36:46.492857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3e8091ae-84f9-48e8-8dcf-9c46b189b3f4 req-dd6a0f3f-4380-4eae-bd22-260adb43c132 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:36:46.509517 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3e8091ae-84f9-48e8-8dcf-9c46b189b3f4 req-dd6a0f3f-4380-4eae-bd22-260adb43c132 service nova] Creating event network-changed:3ece88ca-1593-4172-b7bb-d9d53cd7b4d3 for instance 38097746-54a6-4433-924a-477fcd11f84a on nch2-master-1 Sep 27 10:36:46.516634 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3e8091ae-84f9-48e8-8dcf-9c46b189b3f4 req-dd6a0f3f-4380-4eae-bd22-260adb43c132 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046131 Sep 27 10:36:46.518138 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3448/6429] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:36:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:36:46.723215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-30973a01-8258-4c12-8dfe-9f50bfe8b07f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:36:46.735570 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-30973a01-8258-4c12-8dfe-9f50bfe8b07f tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1562 microversion: 2.1 time: 0.269638 Sep 27 10:36:46.736827 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2982/6430] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:36:46 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1562 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:37:18.124897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:18.126572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=44 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:18.132098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0ef46e9-1102-428a-bd50-901980e98158 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:37:18.134598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b0ef46e9-1102-428a-bd50-901980e98158 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:37:18.142131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ef46e9-1102-428a-bd50-901980e98158 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:18.142968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ef46e9-1102-428a-bd50-901980e98158 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:18.143649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ef46e9-1102-428a-bd50-901980e98158 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:18.144473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ef46e9-1102-428a-bd50-901980e98158 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:18.145215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ef46e9-1102-428a-bd50-901980e98158 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:18.145374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0ef46e9-1102-428a-bd50-901980e98158 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:18.218050 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b0ef46e9-1102-428a-bd50-901980e98158 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a/action" status: 200 len: 24568 microversion: 2.1 time: 0.095119 Sep 27 10:37:18.219414 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3449/6431] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:37:18 2022] POST /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a/action => generated 24568 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:37:56.018897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=20 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:56.019655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=48 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:56.144102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-249187256", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "df2ed96c-c50f-4235-8c9f-b6939a56508c"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:37:56.202988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] validate_networks() for [('df2ed96c-c50f-4235-8c9f-b6939a56508c', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:37:56.724918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:37:56.725362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:37:56.725857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:37:56.726282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:37:56.726987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:37:56.737398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:56.737904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:56.738359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:56.739110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:56.739582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:56.740021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:56.758092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Getting quotas for project 60f3e5639c294a53899e29af88c9c38b. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:37:56.763929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Getting quotas for user 88dbdf0286c34c179364a772529dbfe8 and project 60f3e5639c294a53899e29af88c9c38b. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:37:56.777111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:37:56.779491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:37:56.829382 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fe2266b6-9716-4e45-ac8f-25e19bfcf275 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.815975 Sep 27 10:37:56.830374 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2983/6432] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:37:56 2022] POST /compute/v2.1/servers => generated 380 bytes in 817 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:37:56.842621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3fe24caf-cd3d-4498-b6da-e72227350c5b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:37:56.843339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3fe24caf-cd3d-4498-b6da-e72227350c5b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:37:56.847556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:56.848367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:37:56.905490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3fe24caf-cd3d-4498-b6da-e72227350c5b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:37:56.912639 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3fe24caf-cd3d-4498-b6da-e72227350c5b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1213 microversion: 2.1 time: 0.075006 Sep 27 10:37:56.913255 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3450/6433] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:37:56 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1213 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:37:57.265488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"rebuild": {"imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:37:57.268457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:37:57.276560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:57.277108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:57.277717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:57.278457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:57.278937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:57.279385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:57.358545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:37:57.359008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:37:57.359502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:37:57.359927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:37:57.360359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:37:57.426773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:37:57.433433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:57.434131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:57.434779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:57.435567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:57.436267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:57.436642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:57.480564 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-631a28a2-b8f5-4bfa-863c-726ef2970487 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a/action" status: 202 len: 1136 microversion: 2.1 time: 0.219117 Sep 27 10:37:57.491038 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2984/6434] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:37:57 2022] POST /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a/action => generated 1136 bytes in 221 msecs (HTTP/1.1 202) 10 headers in 452 bytes (1 switches on core 0) Sep 27 10:37:57.500575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3fda7b40-f34e-478f-871e-8687ad94135d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:37:57.501454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3fda7b40-f34e-478f-871e-8687ad94135d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:37:57.509752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fda7b40-f34e-478f-871e-8687ad94135d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:57.510499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fda7b40-f34e-478f-871e-8687ad94135d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:57.510937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fda7b40-f34e-478f-871e-8687ad94135d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:57.512250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fda7b40-f34e-478f-871e-8687ad94135d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:57.512921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fda7b40-f34e-478f-871e-8687ad94135d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:57.513439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fda7b40-f34e-478f-871e-8687ad94135d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:57.667207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3fda7b40-f34e-478f-871e-8687ad94135d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:37:57.675935 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3fda7b40-f34e-478f-871e-8687ad94135d tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1690 microversion: 2.1 time: 0.179259 Sep 27 10:37:57.676587 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3451/6435] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:37:57 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1690 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:37:57.936234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd096665-b426-4eb3-9139-a652dca3dc17 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:37:57.936913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cd096665-b426-4eb3-9139-a652dca3dc17 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:37:57.998878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cd096665-b426-4eb3-9139-a652dca3dc17 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:37:58.007650 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd096665-b426-4eb3-9139-a652dca3dc17 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1213 microversion: 2.1 time: 0.075305 Sep 27 10:37:58.009590 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2985/6436] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:37:57 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1213 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:37:58.702390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fb346e94-6cf7-4a16-aba6-b4f7f406d53c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:37:58.703417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fb346e94-6cf7-4a16-aba6-b4f7f406d53c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:37:58.711779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb346e94-6cf7-4a16-aba6-b4f7f406d53c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:58.712276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb346e94-6cf7-4a16-aba6-b4f7f406d53c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:58.712745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb346e94-6cf7-4a16-aba6-b4f7f406d53c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:58.713469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb346e94-6cf7-4a16-aba6-b4f7f406d53c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:58.713930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb346e94-6cf7-4a16-aba6-b4f7f406d53c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:58.714355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fb346e94-6cf7-4a16-aba6-b4f7f406d53c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:58.865142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fb346e94-6cf7-4a16-aba6-b4f7f406d53c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:37:58.874502 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fb346e94-6cf7-4a16-aba6-b4f7f406d53c tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1690 microversion: 2.1 time: 0.177511 Sep 27 10:37:58.875071 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3452/6437] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:37:58 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1690 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:37:59.035775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce1583f7-8664-456e-bd99-fd028535cd4e tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:37:59.036357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ce1583f7-8664-456e-bd99-fd028535cd4e tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:37:59.045154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce1583f7-8664-456e-bd99-fd028535cd4e tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:59.045659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce1583f7-8664-456e-bd99-fd028535cd4e tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:59.046107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce1583f7-8664-456e-bd99-fd028535cd4e tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:59.046830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce1583f7-8664-456e-bd99-fd028535cd4e tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:59.047300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce1583f7-8664-456e-bd99-fd028535cd4e tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:59.047742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce1583f7-8664-456e-bd99-fd028535cd4e tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:59.113134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ce1583f7-8664-456e-bd99-fd028535cd4e tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:37:59.121604 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ce1583f7-8664-456e-bd99-fd028535cd4e tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1217 microversion: 2.1 time: 0.090145 Sep 27 10:37:59.122155 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2986/6438] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:37:59 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1217 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:37:59.898724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8cb0bc09-1d18-44e3-87ed-6ebdd6435420 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:37:59.899357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8cb0bc09-1d18-44e3-87ed-6ebdd6435420 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:37:59.906751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cb0bc09-1d18-44e3-87ed-6ebdd6435420 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:59.907245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cb0bc09-1d18-44e3-87ed-6ebdd6435420 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:59.907697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cb0bc09-1d18-44e3-87ed-6ebdd6435420 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:37:59.908406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cb0bc09-1d18-44e3-87ed-6ebdd6435420 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:37:59.908905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cb0bc09-1d18-44e3-87ed-6ebdd6435420 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:37:59.909343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8cb0bc09-1d18-44e3-87ed-6ebdd6435420 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:00.062202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8cb0bc09-1d18-44e3-87ed-6ebdd6435420 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:00.070963 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8cb0bc09-1d18-44e3-87ed-6ebdd6435420 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1690 microversion: 2.1 time: 0.176354 Sep 27 10:38:00.071547 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3453/6439] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:37:59 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1690 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:00.144888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90269567-d2c5-4da0-ba65-6476f11bd243 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:00.146315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-90269567-d2c5-4da0-ba65-6476f11bd243 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:00.153766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90269567-d2c5-4da0-ba65-6476f11bd243 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:00.154480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90269567-d2c5-4da0-ba65-6476f11bd243 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:00.155138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90269567-d2c5-4da0-ba65-6476f11bd243 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:00.155901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90269567-d2c5-4da0-ba65-6476f11bd243 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:00.156354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90269567-d2c5-4da0-ba65-6476f11bd243 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:00.157094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90269567-d2c5-4da0-ba65-6476f11bd243 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:00.251226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-90269567-d2c5-4da0-ba65-6476f11bd243 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:00.259718 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-90269567-d2c5-4da0-ba65-6476f11bd243 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1273 microversion: 2.1 time: 0.118754 Sep 27 10:38:00.260308 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2987/6440] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:00 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1273 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:01.095073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-558ba81d-c09a-4267-8810-f3a4b7fda393 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:01.095691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-558ba81d-c09a-4267-8810-f3a4b7fda393 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:01.103665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-558ba81d-c09a-4267-8810-f3a4b7fda393 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:01.104628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-558ba81d-c09a-4267-8810-f3a4b7fda393 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:01.104628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-558ba81d-c09a-4267-8810-f3a4b7fda393 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:01.105336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-558ba81d-c09a-4267-8810-f3a4b7fda393 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:01.105787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-558ba81d-c09a-4267-8810-f3a4b7fda393 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:01.106226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-558ba81d-c09a-4267-8810-f3a4b7fda393 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:01.246264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-558ba81d-c09a-4267-8810-f3a4b7fda393 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:01.254499 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-558ba81d-c09a-4267-8810-f3a4b7fda393 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1690 microversion: 2.1 time: 0.163468 Sep 27 10:38:01.255101 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3454/6441] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:01 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1690 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:01.283890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-38761e87-b578-4b4d-a4f3-f2c40009bf62 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:01.284514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-38761e87-b578-4b4d-a4f3-f2c40009bf62 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:01.291808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38761e87-b578-4b4d-a4f3-f2c40009bf62 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:01.292307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38761e87-b578-4b4d-a4f3-f2c40009bf62 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:01.292777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38761e87-b578-4b4d-a4f3-f2c40009bf62 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:01.293499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38761e87-b578-4b4d-a4f3-f2c40009bf62 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:01.294010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38761e87-b578-4b4d-a4f3-f2c40009bf62 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:01.294443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-38761e87-b578-4b4d-a4f3-f2c40009bf62 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:01.537817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-38761e87-b578-4b4d-a4f3-f2c40009bf62 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:01.547217 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-38761e87-b578-4b4d-a4f3-f2c40009bf62 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1313 microversion: 2.1 time: 0.267280 Sep 27 10:38:01.547962 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2988/6442] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:01 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1313 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:01.828692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-20efcd0a-d8d7-433c-a996-941aaec51425 req-6a727b19-0ecd-4090-9032-5b26c45e92fd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c4351c7-1ec8-4ea7-b167-bd2b66b838eb", "tag": "ddfd6cbe-d240-40ad-b814-26eb9248dc25"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:38:01.838662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-20efcd0a-d8d7-433c-a996-941aaec51425 req-6a727b19-0ecd-4090-9032-5b26c45e92fd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:01.839131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-20efcd0a-d8d7-433c-a996-941aaec51425 req-6a727b19-0ecd-4090-9032-5b26c45e92fd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:01.839554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-20efcd0a-d8d7-433c-a996-941aaec51425 req-6a727b19-0ecd-4090-9032-5b26c45e92fd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:01.850636 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-20efcd0a-d8d7-433c-a996-941aaec51425 req-6a727b19-0ecd-4090-9032-5b26c45e92fd service nova] Creating event network-changed:ddfd6cbe-d240-40ad-b814-26eb9248dc25 for instance 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb on nch1-master-1 Sep 27 10:38:01.856964 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-20efcd0a-d8d7-433c-a996-941aaec51425 req-6a727b19-0ecd-4090-9032-5b26c45e92fd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.169554 Sep 27 10:38:01.857534 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3455/6443] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:38:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:38:02.281213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f76dec5d-5489-4d19-a79c-4cf53f619832 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:02.282032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f76dec5d-5489-4d19-a79c-4cf53f619832 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:02.291486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f76dec5d-5489-4d19-a79c-4cf53f619832 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:02.292191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f76dec5d-5489-4d19-a79c-4cf53f619832 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:02.292684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f76dec5d-5489-4d19-a79c-4cf53f619832 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:02.293586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f76dec5d-5489-4d19-a79c-4cf53f619832 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:02.294185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f76dec5d-5489-4d19-a79c-4cf53f619832 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:02.294737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f76dec5d-5489-4d19-a79c-4cf53f619832 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:02.449398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f76dec5d-5489-4d19-a79c-4cf53f619832 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:02.458864 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f76dec5d-5489-4d19-a79c-4cf53f619832 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1690 microversion: 2.1 time: 0.182814 Sep 27 10:38:02.459450 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2989/6444] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:02 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1690 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:38:02.575235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-94246527-b48e-4bd6-9054-2d522031b1ea tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:02.575929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-94246527-b48e-4bd6-9054-2d522031b1ea tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:02.582812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94246527-b48e-4bd6-9054-2d522031b1ea tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:02.583302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94246527-b48e-4bd6-9054-2d522031b1ea tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:02.583736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94246527-b48e-4bd6-9054-2d522031b1ea tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:02.584431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94246527-b48e-4bd6-9054-2d522031b1ea tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:02.584891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94246527-b48e-4bd6-9054-2d522031b1ea tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:02.585347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94246527-b48e-4bd6-9054-2d522031b1ea tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:02.725647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-94246527-b48e-4bd6-9054-2d522031b1ea tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:02.734962 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-94246527-b48e-4bd6-9054-2d522031b1ea tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1313 microversion: 2.1 time: 0.165354 Sep 27 10:38:02.735554 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3456/6445] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:02 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1313 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:03.482465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c2093845-3458-4960-8465-6695878981a9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:03.483004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c2093845-3458-4960-8465-6695878981a9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:03.490108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2093845-3458-4960-8465-6695878981a9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:03.490612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2093845-3458-4960-8465-6695878981a9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:03.491062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2093845-3458-4960-8465-6695878981a9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:03.491768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2093845-3458-4960-8465-6695878981a9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:03.492339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2093845-3458-4960-8465-6695878981a9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:03.492956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c2093845-3458-4960-8465-6695878981a9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:03.647599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c2093845-3458-4960-8465-6695878981a9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:03.659081 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c2093845-3458-4960-8465-6695878981a9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1690 microversion: 2.1 time: 0.180319 Sep 27 10:38:03.659983 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2990/6446] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:03 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1690 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:03.760147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b7503c6a-23cb-4f90-8ec4-05e3e9b103fa tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:03.760715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b7503c6a-23cb-4f90-8ec4-05e3e9b103fa tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:03.768087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7503c6a-23cb-4f90-8ec4-05e3e9b103fa tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:03.768578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7503c6a-23cb-4f90-8ec4-05e3e9b103fa tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:03.769052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7503c6a-23cb-4f90-8ec4-05e3e9b103fa tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:03.769753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7503c6a-23cb-4f90-8ec4-05e3e9b103fa tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:03.770212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7503c6a-23cb-4f90-8ec4-05e3e9b103fa tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:03.770641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7503c6a-23cb-4f90-8ec4-05e3e9b103fa tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:03.904293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b7503c6a-23cb-4f90-8ec4-05e3e9b103fa tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:03.912283 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b7503c6a-23cb-4f90-8ec4-05e3e9b103fa tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1313 microversion: 2.1 time: 0.156314 Sep 27 10:38:03.912892 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3457/6447] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:03 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1313 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:04.690673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9a33c8c6-2e47-4be0-a889-2b8a63b342ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:04.691633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9a33c8c6-2e47-4be0-a889-2b8a63b342ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:04.699468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a33c8c6-2e47-4be0-a889-2b8a63b342ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:04.699956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a33c8c6-2e47-4be0-a889-2b8a63b342ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:04.700419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a33c8c6-2e47-4be0-a889-2b8a63b342ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:04.701313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a33c8c6-2e47-4be0-a889-2b8a63b342ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:04.701788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a33c8c6-2e47-4be0-a889-2b8a63b342ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:04.702222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9a33c8c6-2e47-4be0-a889-2b8a63b342ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:04.838969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9a33c8c6-2e47-4be0-a889-2b8a63b342ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:04.847803 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9a33c8c6-2e47-4be0-a889-2b8a63b342ba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1690 microversion: 2.1 time: 0.161120 Sep 27 10:38:04.848374 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2991/6448] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:04 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1690 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:04.936078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-85122723-15c9-4eb5-bf58-25dfa649dd9b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:04.936618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-85122723-15c9-4eb5-bf58-25dfa649dd9b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:04.944492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85122723-15c9-4eb5-bf58-25dfa649dd9b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:04.945035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85122723-15c9-4eb5-bf58-25dfa649dd9b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:04.945682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85122723-15c9-4eb5-bf58-25dfa649dd9b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:04.946405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85122723-15c9-4eb5-bf58-25dfa649dd9b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:04.946869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85122723-15c9-4eb5-bf58-25dfa649dd9b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:04.947302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85122723-15c9-4eb5-bf58-25dfa649dd9b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:05.086258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-85122723-15c9-4eb5-bf58-25dfa649dd9b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:05.094319 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-85122723-15c9-4eb5-bf58-25dfa649dd9b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1313 microversion: 2.1 time: 0.162385 Sep 27 10:38:05.094991 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3458/6449] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:04 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:05.872028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-980846fc-4564-48e1-9868-40a69bcf9720 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:05.872562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-980846fc-4564-48e1-9868-40a69bcf9720 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:05.879903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-980846fc-4564-48e1-9868-40a69bcf9720 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:05.880411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-980846fc-4564-48e1-9868-40a69bcf9720 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:05.880905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-980846fc-4564-48e1-9868-40a69bcf9720 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:05.881606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-980846fc-4564-48e1-9868-40a69bcf9720 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:05.882077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-980846fc-4564-48e1-9868-40a69bcf9720 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:05.882518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-980846fc-4564-48e1-9868-40a69bcf9720 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:06.118792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bef8495d-88c8-490a-9dc7-654e6ed6bf2a tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:06.119376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bef8495d-88c8-490a-9dc7-654e6ed6bf2a tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:06.127295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bef8495d-88c8-490a-9dc7-654e6ed6bf2a tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:06.127698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bef8495d-88c8-490a-9dc7-654e6ed6bf2a tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:06.128112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bef8495d-88c8-490a-9dc7-654e6ed6bf2a tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:06.128905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bef8495d-88c8-490a-9dc7-654e6ed6bf2a tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:06.129362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bef8495d-88c8-490a-9dc7-654e6ed6bf2a tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:06.129791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bef8495d-88c8-490a-9dc7-654e6ed6bf2a tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:06.266119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-980846fc-4564-48e1-9868-40a69bcf9720 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:06.270239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bef8495d-88c8-490a-9dc7-654e6ed6bf2a tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:06.274595 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-980846fc-4564-48e1-9868-40a69bcf9720 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1690 microversion: 2.1 time: 0.406425 Sep 27 10:38:06.275983 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2992/6450] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:05 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1690 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:06.288980 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bef8495d-88c8-490a-9dc7-654e6ed6bf2a tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1313 microversion: 2.1 time: 0.174628 Sep 27 10:38:06.290187 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3459/6451] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:06 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1313 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:06.747037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:06.773803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-33216ac5-d61f-4825-bce3-e40de8c48459 req-760fbb3b-9830-4ac1-8aa0-ee81ae478614 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c4351c7-1ec8-4ea7-b167-bd2b66b838eb", "name": "network-vif-plugged", "status": "completed", "tag": "ddfd6cbe-d240-40ad-b814-26eb9248dc25"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:38:06.784535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-33216ac5-d61f-4825-bce3-e40de8c48459 req-760fbb3b-9830-4ac1-8aa0-ee81ae478614 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:06.785041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-33216ac5-d61f-4825-bce3-e40de8c48459 req-760fbb3b-9830-4ac1-8aa0-ee81ae478614 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:06.785484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-33216ac5-d61f-4825-bce3-e40de8c48459 req-760fbb3b-9830-4ac1-8aa0-ee81ae478614 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:06.796974 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-33216ac5-d61f-4825-bce3-e40de8c48459 req-760fbb3b-9830-4ac1-8aa0-ee81ae478614 service nova] Creating event network-vif-plugged:ddfd6cbe-d240-40ad-b814-26eb9248dc25 for instance 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb on nch1-master-1 Sep 27 10:38:06.800524 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [req-33216ac5-d61f-4825-bce3-e40de8c48459 req-760fbb3b-9830-4ac1-8aa0-ee81ae478614 service nova] [ffac6b73-531c-4f9b-aa18-c01a364670aa] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 27 10:38:07.299274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d849a9a3-b091-430b-b420-0c37b3c70eba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:07.299959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d849a9a3-b091-430b-b420-0c37b3c70eba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:07.302055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:07.308859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d849a9a3-b091-430b-b420-0c37b3c70eba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:07.309374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d849a9a3-b091-430b-b420-0c37b3c70eba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:07.309845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d849a9a3-b091-430b-b420-0c37b3c70eba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:07.310557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d849a9a3-b091-430b-b420-0c37b3c70eba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:07.311007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d849a9a3-b091-430b-b420-0c37b3c70eba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:07.311433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d849a9a3-b091-430b-b420-0c37b3c70eba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:07.454854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d849a9a3-b091-430b-b420-0c37b3c70eba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:07.463274 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d849a9a3-b091-430b-b420-0c37b3c70eba tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1690 microversion: 2.1 time: 0.167974 Sep 27 10:38:07.463872 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3460/6452] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:07 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1690 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:07.468596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9ebe719b-1a62-4441-a7ab-f3aec0de2257 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:07.469239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ebe719b-1a62-4441-a7ab-f3aec0de2257 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:07.475098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe719b-1a62-4441-a7ab-f3aec0de2257 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:07.475589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe719b-1a62-4441-a7ab-f3aec0de2257 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:07.476038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe719b-1a62-4441-a7ab-f3aec0de2257 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:07.476934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe719b-1a62-4441-a7ab-f3aec0de2257 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:07.477431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe719b-1a62-4441-a7ab-f3aec0de2257 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:07.477868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ebe719b-1a62-4441-a7ab-f3aec0de2257 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:07.613366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9ebe719b-1a62-4441-a7ab-f3aec0de2257 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:07.624633 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9ebe719b-1a62-4441-a7ab-f3aec0de2257 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1313 microversion: 2.1 time: 0.159420 Sep 27 10:38:07.625403 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3461/6453] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:07 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1313 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:07.807986 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [req-33216ac5-d61f-4825-bce3-e40de8c48459 req-760fbb3b-9830-4ac1-8aa0-ee81ae478614 service nova] [ffac6b73-531c-4f9b-aa18-c01a364670aa] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 52910. Sep 27 10:38:07.812171 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-33216ac5-d61f-4825-bce3-e40de8c48459 req-760fbb3b-9830-4ac1-8aa0-ee81ae478614 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 1.118539 Sep 27 10:38:07.812772 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2993/6454] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:38:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 1120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:38:08.487387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a3059ebc-52b1-4d3c-87da-cc133875a342 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:08.488063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a3059ebc-52b1-4d3c-87da-cc133875a342 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:08.495363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3059ebc-52b1-4d3c-87da-cc133875a342 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:08.495900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3059ebc-52b1-4d3c-87da-cc133875a342 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:08.496341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3059ebc-52b1-4d3c-87da-cc133875a342 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:08.497075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3059ebc-52b1-4d3c-87da-cc133875a342 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:08.497513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3059ebc-52b1-4d3c-87da-cc133875a342 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:08.497939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a3059ebc-52b1-4d3c-87da-cc133875a342 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:08.637210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a3059ebc-52b1-4d3c-87da-cc133875a342 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:08.645324 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a3059ebc-52b1-4d3c-87da-cc133875a342 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1690 microversion: 2.1 time: 0.162035 Sep 27 10:38:08.645941 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3462/6455] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:08 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1690 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:08.648710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:08.652706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7a680dfc-89df-4d25-9ddc-60afb820d173 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:08.653443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7a680dfc-89df-4d25-9ddc-60afb820d173 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:08.660274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a680dfc-89df-4d25-9ddc-60afb820d173 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:08.660807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a680dfc-89df-4d25-9ddc-60afb820d173 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:08.661255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a680dfc-89df-4d25-9ddc-60afb820d173 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:08.661961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a680dfc-89df-4d25-9ddc-60afb820d173 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:08.662428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a680dfc-89df-4d25-9ddc-60afb820d173 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:08.662864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7a680dfc-89df-4d25-9ddc-60afb820d173 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:08.805986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7a680dfc-89df-4d25-9ddc-60afb820d173 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:08.814416 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7a680dfc-89df-4d25-9ddc-60afb820d173 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1313 microversion: 2.1 time: 0.167984 Sep 27 10:38:08.815027 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2994/6456] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:08 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1313 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:09.668615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-194800ae-3f7e-49a5-ac06-866b54049ddc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:09.669284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-194800ae-3f7e-49a5-ac06-866b54049ddc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:09.678659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-194800ae-3f7e-49a5-ac06-866b54049ddc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:09.679253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-194800ae-3f7e-49a5-ac06-866b54049ddc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:09.679732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-194800ae-3f7e-49a5-ac06-866b54049ddc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:09.680438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-194800ae-3f7e-49a5-ac06-866b54049ddc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:09.680925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-194800ae-3f7e-49a5-ac06-866b54049ddc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:09.681393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-194800ae-3f7e-49a5-ac06-866b54049ddc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:09.818002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-194800ae-3f7e-49a5-ac06-866b54049ddc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:09.827629 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-194800ae-3f7e-49a5-ac06-866b54049ddc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1696 microversion: 2.1 time: 0.163196 Sep 27 10:38:09.828178 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3463/6457] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:09 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1696 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:09.840915 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b4cbb4d-2254-4dd4-a001-ae808752301b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:09.841595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0b4cbb4d-2254-4dd4-a001-ae808752301b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:09.848497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4cbb4d-2254-4dd4-a001-ae808752301b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:09.849057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4cbb4d-2254-4dd4-a001-ae808752301b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:09.849509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4cbb4d-2254-4dd4-a001-ae808752301b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:09.850241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4cbb4d-2254-4dd4-a001-ae808752301b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:09.850704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4cbb4d-2254-4dd4-a001-ae808752301b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:09.851263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b4cbb4d-2254-4dd4-a001-ae808752301b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:09.990166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0b4cbb4d-2254-4dd4-a001-ae808752301b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:09.999533 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0b4cbb4d-2254-4dd4-a001-ae808752301b tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1497 microversion: 2.1 time: 0.163031 Sep 27 10:38:10.000115 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2995/6458] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:09 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:10.854964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9297acb-8559-463f-b954-af59acb9f397 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:10.855666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c9297acb-8559-463f-b954-af59acb9f397 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:10.864347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9297acb-8559-463f-b954-af59acb9f397 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:10.864970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9297acb-8559-463f-b954-af59acb9f397 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:10.865489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9297acb-8559-463f-b954-af59acb9f397 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:10.866305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9297acb-8559-463f-b954-af59acb9f397 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:10.866842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9297acb-8559-463f-b954-af59acb9f397 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:10.867391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9297acb-8559-463f-b954-af59acb9f397 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:11.013946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9297acb-8559-463f-b954-af59acb9f397 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:11.024025 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9297acb-8559-463f-b954-af59acb9f397 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1696 microversion: 2.1 time: 0.174844 Sep 27 10:38:11.024642 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3464/6459] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:10 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1696 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:12.048077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e2febb8b-ea6c-48d5-8fe9-e9c4c011ecfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:12.048738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e2febb8b-ea6c-48d5-8fe9-e9c4c011ecfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:12.056196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2febb8b-ea6c-48d5-8fe9-e9c4c011ecfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:12.056737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2febb8b-ea6c-48d5-8fe9-e9c4c011ecfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:12.057198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2febb8b-ea6c-48d5-8fe9-e9c4c011ecfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:12.058061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2febb8b-ea6c-48d5-8fe9-e9c4c011ecfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:12.058714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2febb8b-ea6c-48d5-8fe9-e9c4c011ecfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:12.059157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e2febb8b-ea6c-48d5-8fe9-e9c4c011ecfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:12.203440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e2febb8b-ea6c-48d5-8fe9-e9c4c011ecfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:12.212603 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e2febb8b-ea6c-48d5-8fe9-e9c4c011ecfc tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1696 microversion: 2.1 time: 0.168379 Sep 27 10:38:12.213321 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2996/6460] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:12 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1696 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:12.371548 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06e50ab6-c362-45ad-9944-620810c8ac13 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:12.372078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-06e50ab6-c362-45ad-9944-620810c8ac13 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:12.379516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06e50ab6-c362-45ad-9944-620810c8ac13 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:12.380002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06e50ab6-c362-45ad-9944-620810c8ac13 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:12.380442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06e50ab6-c362-45ad-9944-620810c8ac13 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:12.381168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06e50ab6-c362-45ad-9944-620810c8ac13 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:12.381622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06e50ab6-c362-45ad-9944-620810c8ac13 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:12.382050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06e50ab6-c362-45ad-9944-620810c8ac13 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:12.409568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-06e50ab6-c362-45ad-9944-620810c8ac13 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:38:12.494866 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-06e50ab6-c362-45ad-9944-620810c8ac13 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 204 len: 0 microversion: 2.1 time: 0.127231 Sep 27 10:38:12.495592 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3465/6461] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:38:12 2022] DELETE /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:38:12.504346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-243d9088-2d33-4908-aa68-6d6f7ef4d9b6 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:12.505008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-243d9088-2d33-4908-aa68-6d6f7ef4d9b6 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:12.511787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-243d9088-2d33-4908-aa68-6d6f7ef4d9b6 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:12.512287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-243d9088-2d33-4908-aa68-6d6f7ef4d9b6 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:12.512769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-243d9088-2d33-4908-aa68-6d6f7ef4d9b6 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:12.513487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-243d9088-2d33-4908-aa68-6d6f7ef4d9b6 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:12.513954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-243d9088-2d33-4908-aa68-6d6f7ef4d9b6 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:12.514397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-243d9088-2d33-4908-aa68-6d6f7ef4d9b6 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:12.652786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-243d9088-2d33-4908-aa68-6d6f7ef4d9b6 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:12.660814 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-243d9088-2d33-4908-aa68-6d6f7ef4d9b6 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1503 microversion: 2.1 time: 0.160541 Sep 27 10:38:12.661422 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2997/6462] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:12 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1503 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:13.238998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a243e629-ed5c-4e0b-9a97-0842f7589243 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:13.239731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a243e629-ed5c-4e0b-9a97-0842f7589243 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:13.246735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243e629-ed5c-4e0b-9a97-0842f7589243 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:13.247236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243e629-ed5c-4e0b-9a97-0842f7589243 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:13.247685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243e629-ed5c-4e0b-9a97-0842f7589243 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:13.248399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243e629-ed5c-4e0b-9a97-0842f7589243 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:13.248910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243e629-ed5c-4e0b-9a97-0842f7589243 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:13.249347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a243e629-ed5c-4e0b-9a97-0842f7589243 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:13.271157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:13.404387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a243e629-ed5c-4e0b-9a97-0842f7589243 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:13.412717 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a243e629-ed5c-4e0b-9a97-0842f7589243 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1696 microversion: 2.1 time: 0.179165 Sep 27 10:38:13.413317 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3466/6463] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:13 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1696 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:13.688860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4df972f3-9b0e-46cb-8662-df5afcd84810 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:13.689946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4df972f3-9b0e-46cb-8662-df5afcd84810 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:13.699310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4df972f3-9b0e-46cb-8662-df5afcd84810 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:13.699817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4df972f3-9b0e-46cb-8662-df5afcd84810 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:13.700274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4df972f3-9b0e-46cb-8662-df5afcd84810 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:13.701043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4df972f3-9b0e-46cb-8662-df5afcd84810 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:13.701501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4df972f3-9b0e-46cb-8662-df5afcd84810 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:13.701957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4df972f3-9b0e-46cb-8662-df5afcd84810 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:13.912873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4df972f3-9b0e-46cb-8662-df5afcd84810 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '88dbdf0286c34c179364a772529dbfe8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '60f3e5639c294a53899e29af88c9c38b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:13.921420 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4df972f3-9b0e-46cb-8662-df5afcd84810 tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 200 len: 1503 microversion: 2.1 time: 0.239324 Sep 27 10:38:13.922006 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2998/6464] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:13 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 1503 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:14.308762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-2c7e9db1-c98c-41e9-ab8d-f52e989a8dc3 req-7e6fc6dc-34b0-4697-9558-3825b47ce5ac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c4351c7-1ec8-4ea7-b167-bd2b66b838eb", "name": "network-vif-deleted", "tag": "ddfd6cbe-d240-40ad-b814-26eb9248dc25"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:38:14.318883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2c7e9db1-c98c-41e9-ab8d-f52e989a8dc3 req-7e6fc6dc-34b0-4697-9558-3825b47ce5ac service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:14.319368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2c7e9db1-c98c-41e9-ab8d-f52e989a8dc3 req-7e6fc6dc-34b0-4697-9558-3825b47ce5ac service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:14.320087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2c7e9db1-c98c-41e9-ab8d-f52e989a8dc3 req-7e6fc6dc-34b0-4697-9558-3825b47ce5ac service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:14.333712 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-2c7e9db1-c98c-41e9-ab8d-f52e989a8dc3 req-7e6fc6dc-34b0-4697-9558-3825b47ce5ac service nova] Creating event network-vif-deleted:ddfd6cbe-d240-40ad-b814-26eb9248dc25 for instance 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb on nch1-master-1 Sep 27 10:38:14.339916 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-2c7e9db1-c98c-41e9-ab8d-f52e989a8dc3 req-7e6fc6dc-34b0-4697-9558-3825b47ce5ac service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035599 Sep 27 10:38:14.340825 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3467/6465] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:38:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:38:14.438168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b7f2d964-f51d-4342-b0de-ca83ef8e3604 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:14.438807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b7f2d964-f51d-4342-b0de-ca83ef8e3604 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:14.445672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7f2d964-f51d-4342-b0de-ca83ef8e3604 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:14.446825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7f2d964-f51d-4342-b0de-ca83ef8e3604 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:14.447250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7f2d964-f51d-4342-b0de-ca83ef8e3604 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:14.448056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7f2d964-f51d-4342-b0de-ca83ef8e3604 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:14.448604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7f2d964-f51d-4342-b0de-ca83ef8e3604 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:14.449178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7f2d964-f51d-4342-b0de-ca83ef8e3604 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:14.491171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-d1c194d3-0a72-4280-9c80-094f62ee0b3f req-2d7d4556-4220-4a86-98a1-fcbe6a99335d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38097746-54a6-4433-924a-477fcd11f84a", "name": "network-vif-unplugged", "status": "completed", "tag": "3ece88ca-1593-4172-b7bb-d9d53cd7b4d3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:38:14.509350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d1c194d3-0a72-4280-9c80-094f62ee0b3f req-2d7d4556-4220-4a86-98a1-fcbe6a99335d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:14.509863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d1c194d3-0a72-4280-9c80-094f62ee0b3f req-2d7d4556-4220-4a86-98a1-fcbe6a99335d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:14.510315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-d1c194d3-0a72-4280-9c80-094f62ee0b3f req-2d7d4556-4220-4a86-98a1-fcbe6a99335d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:14.522676 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-d1c194d3-0a72-4280-9c80-094f62ee0b3f req-2d7d4556-4220-4a86-98a1-fcbe6a99335d service nova] Creating event network-vif-unplugged:3ece88ca-1593-4172-b7bb-d9d53cd7b4d3 for instance 38097746-54a6-4433-924a-477fcd11f84a on nch2-master-1 Sep 27 10:38:14.530312 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-d1c194d3-0a72-4280-9c80-094f62ee0b3f req-2d7d4556-4220-4a86-98a1-fcbe6a99335d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.044651 Sep 27 10:38:14.532106 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3468/6466] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:38:14 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:38:14.601885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b7f2d964-f51d-4342-b0de-ca83ef8e3604 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:14.606844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:14.612810 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b7f2d964-f51d-4342-b0de-ca83ef8e3604 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1681 microversion: 2.1 time: 0.179345 Sep 27 10:38:14.613379 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 2999/6467] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:14 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1681 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:14.947879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:14.948856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] [instance: 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:14.957536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:14.958190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:14.958780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:14.959718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:14.960341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:14.961030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:14.974822 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] HTTP exception thrown: Instance 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb could not be found. Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Request method failure captured: Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb HTTP/1.1 Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtKCUKv50xPcf4t1UhNhRHA8sVjJOCShAwmw22dqLEg4Wm_bX6vupZ0djaHMjgomvn0kddjHFOix3gC0dVu9InXu5XahBaLBetqK8SnNcWuYradCJ_yelumo9qdq_MeORp5uw8iR4Mn0ACgulmzY6ICy0hWrPNVA5fGlIog2kNDsyZY Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 60f3e5639c294a53899e29af88c9c38b Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-VolumesActionsTest-1137661699 Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/60f3e5639c294a53899e29af88c9c38b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/60f3e5639c294a53899e29af88c9c38b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/60f3e5639c294a53899e29af88c9c38b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_60f3e5639c294a53899e29af88c9c38b"}]}] Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-VolumesActionsTest-1137661699 Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 60f3e5639c294a53899e29af88c9c38b Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-VolumesActionsTest-1137661699 Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-VolumesActionsTest-1137661699-project Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 88dbdf0286c34c179364a772529dbfe8 Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-VolumesActionsTest-1137661699-project Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb could not be found. Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '1c4351c7-1ec8-4ea7-b167-bd2b66b838eb'} Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb could not be found. Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb could not be found. Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:38:14.976521 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:38:14.981130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:38:14.981130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:38:14.981130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:38:14.981130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:38:14.981130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:38:14.981130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb could not be found. Sep 27 10:38:14.981130 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:38:14.981130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] Returning 404 to user: Instance 1c4351c7-1ec8-4ea7-b167-bd2b66b838eb could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:38:14.981130 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1c249509-f20f-4051-aa64-545755b9446d tempest-VolumesActionsTest-1137661699 tempest-VolumesActionsTest-1137661699-project] 10.222.0.26 "GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb" status: 404 len: 111 microversion: 2.1 time: 0.036127 Sep 27 10:38:14.981130 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3469/6468] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:14 2022] GET /compute/v2.1/servers/1c4351c7-1ec8-4ea7-b167-bd2b66b838eb => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:38:16.774654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-da1575e5-79b0-4290-8a53-6ca7637b1cb4 req-53d3712c-d46d-4c99-b0e9-1cb0b784fe5d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38097746-54a6-4433-924a-477fcd11f84a", "name": "network-vif-plugged", "status": "completed", "tag": "3ece88ca-1593-4172-b7bb-d9d53cd7b4d3"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:38:16.786977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-da1575e5-79b0-4290-8a53-6ca7637b1cb4 req-53d3712c-d46d-4c99-b0e9-1cb0b784fe5d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:16.787538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-da1575e5-79b0-4290-8a53-6ca7637b1cb4 req-53d3712c-d46d-4c99-b0e9-1cb0b784fe5d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:16.788059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-da1575e5-79b0-4290-8a53-6ca7637b1cb4 req-53d3712c-d46d-4c99-b0e9-1cb0b784fe5d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:16.801921 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-da1575e5-79b0-4290-8a53-6ca7637b1cb4 req-53d3712c-d46d-4c99-b0e9-1cb0b784fe5d service nova] Creating event network-vif-plugged:3ece88ca-1593-4172-b7bb-d9d53cd7b4d3 for instance 38097746-54a6-4433-924a-477fcd11f84a on nch2-master-1 Sep 27 10:38:16.808796 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-da1575e5-79b0-4290-8a53-6ca7637b1cb4 req-53d3712c-d46d-4c99-b0e9-1cb0b784fe5d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038702 Sep 27 10:38:16.809696 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3000/6469] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:38:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:38:44.181654 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Sep 27 10:38:44.184323 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Sep 27 10:38:44.185132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:44.185903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:44.307601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestServerAdvancedOps-server-321128026", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "621b5363-ce18-4194-8e52-bcfca4b8b3f9"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:38:44.367434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] validate_networks() for [('621b5363-ce18-4194-8e52-bcfca4b8b3f9', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:38:44.731307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:38:44.731734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:38:44.732208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:38:44.732621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:38:44.733092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:38:44.743560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:44.744047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:44.744482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:44.745246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:44.745705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:44.746128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:44.763815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Getting quotas for project 91f4b185afc0495eaba8f2ece9562eb2. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:38:44.769365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Getting quotas for user ee4c3d891eb9466c8cc00cf5d28b3924 and project 91f4b185afc0495eaba8f2ece9562eb2. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:38:44.782731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:38:44.785028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:38:44.839555 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0925091e-f940-457c-97d2-20baf5b33f9b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.663952 Sep 27 10:38:44.840219 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3470/6470] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:38:44 2022] POST /compute/v2.1/servers => generated 380 bytes in 665 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:38:44.850087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=53,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:44.850998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:44.855228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-04eb1a10-927a-4b0f-af3e-25a7ce948743 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:44.856046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-04eb1a10-927a-4b0f-af3e-25a7ce948743 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:44.911943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-04eb1a10-927a-4b0f-af3e-25a7ce948743 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:44.919104 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-04eb1a10-927a-4b0f-af3e-25a7ce948743 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1214 microversion: 2.1 time: 0.071224 Sep 27 10:38:44.919654 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3001/6471] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:44 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1214 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:45.943840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dd976746-8dd9-4d9f-89f8-51ef6c8e5b24 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:45.945335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dd976746-8dd9-4d9f-89f8-51ef6c8e5b24 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:45.952775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd976746-8dd9-4d9f-89f8-51ef6c8e5b24 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:45.953327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd976746-8dd9-4d9f-89f8-51ef6c8e5b24 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:45.954164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd976746-8dd9-4d9f-89f8-51ef6c8e5b24 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:45.955129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd976746-8dd9-4d9f-89f8-51ef6c8e5b24 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:45.955694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd976746-8dd9-4d9f-89f8-51ef6c8e5b24 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:45.956257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dd976746-8dd9-4d9f-89f8-51ef6c8e5b24 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:46.025291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dd976746-8dd9-4d9f-89f8-51ef6c8e5b24 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:46.039643 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dd976746-8dd9-4d9f-89f8-51ef6c8e5b24 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1284 microversion: 2.1 time: 0.101178 Sep 27 10:38:46.040230 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3471/6472] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:45 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1284 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:47.063508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f9785963-e7d0-4973-9b4b-49b763b210f7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:47.064148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f9785963-e7d0-4973-9b4b-49b763b210f7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:47.072274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9785963-e7d0-4973-9b4b-49b763b210f7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:47.072797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9785963-e7d0-4973-9b4b-49b763b210f7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:47.073333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9785963-e7d0-4973-9b4b-49b763b210f7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:47.074052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9785963-e7d0-4973-9b4b-49b763b210f7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:47.074523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9785963-e7d0-4973-9b4b-49b763b210f7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:47.075024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f9785963-e7d0-4973-9b4b-49b763b210f7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:47.263329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f9785963-e7d0-4973-9b4b-49b763b210f7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:47.271704 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f9785963-e7d0-4973-9b4b-49b763b210f7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1314 microversion: 2.1 time: 0.212524 Sep 27 10:38:47.272313 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3002/6473] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:47 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1314 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:47.350412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6dd0d143-ce43-4912-bd30-00ec9787c6c0 req-86caa6d8-149e-4bc2-82ba-d93c8b480e3d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8352fb20-d2c7-487e-ae9d-3406cfb8d9e0", "tag": "a9128636-2d6b-4f91-b749-9c426271dfbc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:38:47.366197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6dd0d143-ce43-4912-bd30-00ec9787c6c0 req-86caa6d8-149e-4bc2-82ba-d93c8b480e3d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:47.366682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6dd0d143-ce43-4912-bd30-00ec9787c6c0 req-86caa6d8-149e-4bc2-82ba-d93c8b480e3d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:47.367101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6dd0d143-ce43-4912-bd30-00ec9787c6c0 req-86caa6d8-149e-4bc2-82ba-d93c8b480e3d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:47.378982 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6dd0d143-ce43-4912-bd30-00ec9787c6c0 req-86caa6d8-149e-4bc2-82ba-d93c8b480e3d service nova] Creating event network-changed:a9128636-2d6b-4f91-b749-9c426271dfbc for instance 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 on nch1-master-1 Sep 27 10:38:47.383950 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6dd0d143-ce43-4912-bd30-00ec9787c6c0 req-86caa6d8-149e-4bc2-82ba-d93c8b480e3d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038942 Sep 27 10:38:47.384594 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3472/6474] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:38:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:38:48.297112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3a946da5-1c4c-4f34-befc-80683cf00034 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:48.297751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3a946da5-1c4c-4f34-befc-80683cf00034 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:48.305269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a946da5-1c4c-4f34-befc-80683cf00034 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:48.305822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a946da5-1c4c-4f34-befc-80683cf00034 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:48.306205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a946da5-1c4c-4f34-befc-80683cf00034 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:48.306923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a946da5-1c4c-4f34-befc-80683cf00034 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:48.307411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a946da5-1c4c-4f34-befc-80683cf00034 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:48.307816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3a946da5-1c4c-4f34-befc-80683cf00034 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:48.442324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3a946da5-1c4c-4f34-befc-80683cf00034 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:48.450485 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3a946da5-1c4c-4f34-befc-80683cf00034 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1314 microversion: 2.1 time: 0.157498 Sep 27 10:38:48.451064 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3003/6475] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:48 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:49.001242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da9e6568-e80d-4dfa-ba2b-30b698a9c6f7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:38:49.003520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-da9e6568-e80d-4dfa-ba2b-30b698a9c6f7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:49.010433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9e6568-e80d-4dfa-ba2b-30b698a9c6f7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:49.010922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9e6568-e80d-4dfa-ba2b-30b698a9c6f7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:49.011362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9e6568-e80d-4dfa-ba2b-30b698a9c6f7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:49.012068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9e6568-e80d-4dfa-ba2b-30b698a9c6f7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:49.012524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9e6568-e80d-4dfa-ba2b-30b698a9c6f7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:49.013006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da9e6568-e80d-4dfa-ba2b-30b698a9c6f7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:49.081278 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da9e6568-e80d-4dfa-ba2b-30b698a9c6f7 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "POST /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a/action" status: 200 len: 24572 microversion: 2.1 time: 0.219917 Sep 27 10:38:49.082329 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3473/6476] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:38:48 2022] POST /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a/action => generated 24572 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 10:38:49.477991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2e050c0d-2742-4933-92d1-750e9a4a71ee tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:49.479133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2e050c0d-2742-4933-92d1-750e9a4a71ee tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:49.488919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e050c0d-2742-4933-92d1-750e9a4a71ee tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:49.489431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e050c0d-2742-4933-92d1-750e9a4a71ee tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:49.489888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e050c0d-2742-4933-92d1-750e9a4a71ee tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:49.490619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e050c0d-2742-4933-92d1-750e9a4a71ee tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:49.491158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e050c0d-2742-4933-92d1-750e9a4a71ee tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:49.491611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2e050c0d-2742-4933-92d1-750e9a4a71ee tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:49.626656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2e050c0d-2742-4933-92d1-750e9a4a71ee tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:49.634990 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2e050c0d-2742-4933-92d1-750e9a4a71ee tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1314 microversion: 2.1 time: 0.164147 Sep 27 10:38:49.635577 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3004/6477] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:49 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1314 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:50.661168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f940b6a2-8df5-483f-843d-55f23471a2f4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:50.662052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f940b6a2-8df5-483f-843d-55f23471a2f4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:50.670937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f940b6a2-8df5-483f-843d-55f23471a2f4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:50.671598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f940b6a2-8df5-483f-843d-55f23471a2f4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:50.672135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f940b6a2-8df5-483f-843d-55f23471a2f4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:50.673052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f940b6a2-8df5-483f-843d-55f23471a2f4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:50.673640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f940b6a2-8df5-483f-843d-55f23471a2f4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:50.674175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f940b6a2-8df5-483f-843d-55f23471a2f4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:50.839242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f940b6a2-8df5-483f-843d-55f23471a2f4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:50.847593 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f940b6a2-8df5-483f-843d-55f23471a2f4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1314 microversion: 2.1 time: 0.192092 Sep 27 10:38:50.848207 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3474/6478] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:50 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1314 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:51.869895 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [-] [1986a9fd-1b4d-44f5-a8ac-515eb5bb435f] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 27 10:38:51.873431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d69cf268-fdf6-4630-871f-3f45a314a4d0 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:51.874252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d69cf268-fdf6-4630-871f-3f45a314a4d0 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:51.881740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d69cf268-fdf6-4630-871f-3f45a314a4d0 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:51.882273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d69cf268-fdf6-4630-871f-3f45a314a4d0 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:51.882750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d69cf268-fdf6-4630-871f-3f45a314a4d0 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:51.883508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d69cf268-fdf6-4630-871f-3f45a314a4d0 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:51.883959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d69cf268-fdf6-4630-871f-3f45a314a4d0 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:51.884396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d69cf268-fdf6-4630-871f-3f45a314a4d0 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:52.047647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d69cf268-fdf6-4630-871f-3f45a314a4d0 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:52.058150 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d69cf268-fdf6-4630-871f-3f45a314a4d0 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1314 microversion: 2.1 time: 0.189767 Sep 27 10:38:52.058845 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3005/6479] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:51 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1314 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:53.007128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-308d8ca4-2435-4654-b418-6b1c8c401adb req-9a652dd8-17a4-477c-8c50-f3ac656bf9ef service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8352fb20-d2c7-487e-ae9d-3406cfb8d9e0", "name": "network-vif-plugged", "status": "completed", "tag": "a9128636-2d6b-4f91-b749-9c426271dfbc"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:38:53.018649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-308d8ca4-2435-4654-b418-6b1c8c401adb req-9a652dd8-17a4-477c-8c50-f3ac656bf9ef service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:53.019217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-308d8ca4-2435-4654-b418-6b1c8c401adb req-9a652dd8-17a4-477c-8c50-f3ac656bf9ef service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:53.019572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-308d8ca4-2435-4654-b418-6b1c8c401adb req-9a652dd8-17a4-477c-8c50-f3ac656bf9ef service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:53.032279 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-308d8ca4-2435-4654-b418-6b1c8c401adb req-9a652dd8-17a4-477c-8c50-f3ac656bf9ef service nova] Creating event network-vif-plugged:a9128636-2d6b-4f91-b749-9c426271dfbc for instance 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 on nch1-master-1 Sep 27 10:38:53.037401 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-308d8ca4-2435-4654-b418-6b1c8c401adb req-9a652dd8-17a4-477c-8c50-f3ac656bf9ef service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034061 Sep 27 10:38:53.038012 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3475/6480] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:38:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:38:53.083856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4adee90d-5548-476a-ba0e-4ce3e727c0af tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:53.085069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4adee90d-5548-476a-ba0e-4ce3e727c0af tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:53.097983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4adee90d-5548-476a-ba0e-4ce3e727c0af tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:53.098648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4adee90d-5548-476a-ba0e-4ce3e727c0af tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:53.099324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4adee90d-5548-476a-ba0e-4ce3e727c0af tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:53.100391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4adee90d-5548-476a-ba0e-4ce3e727c0af tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:53.101173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4adee90d-5548-476a-ba0e-4ce3e727c0af tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:53.101917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4adee90d-5548-476a-ba0e-4ce3e727c0af tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:53.112046 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] [1986a9fd-1b4d-44f5-a8ac-515eb5bb435f] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 53646. Sep 27 10:38:53.264028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4adee90d-5548-476a-ba0e-4ce3e727c0af tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:53.272381 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4adee90d-5548-476a-ba0e-4ce3e727c0af tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1314 microversion: 2.1 time: 0.192611 Sep 27 10:38:53.273024 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3006/6481] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:53 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1314 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:54.298705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5ddeb6ff-f1f7-44e4-b791-60ee8385d0b8 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:54.299428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5ddeb6ff-f1f7-44e4-b791-60ee8385d0b8 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:54.307779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ddeb6ff-f1f7-44e4-b791-60ee8385d0b8 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:54.308291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ddeb6ff-f1f7-44e4-b791-60ee8385d0b8 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:54.308722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ddeb6ff-f1f7-44e4-b791-60ee8385d0b8 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:54.309549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ddeb6ff-f1f7-44e4-b791-60ee8385d0b8 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:54.310003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ddeb6ff-f1f7-44e4-b791-60ee8385d0b8 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:54.310442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ddeb6ff-f1f7-44e4-b791-60ee8385d0b8 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:54.461813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5ddeb6ff-f1f7-44e4-b791-60ee8385d0b8 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:54.472448 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5ddeb6ff-f1f7-44e4-b791-60ee8385d0b8 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1314 microversion: 2.1 time: 0.178991 Sep 27 10:38:54.473173 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3476/6482] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:54 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1314 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:55.500212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6360b600-66ce-43a6-8bda-d6304717305c tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:55.501399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6360b600-66ce-43a6-8bda-d6304717305c tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:55.505670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:55.514626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6360b600-66ce-43a6-8bda-d6304717305c tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:55.515377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6360b600-66ce-43a6-8bda-d6304717305c tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:55.516098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6360b600-66ce-43a6-8bda-d6304717305c tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:55.517315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6360b600-66ce-43a6-8bda-d6304717305c tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:55.518084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6360b600-66ce-43a6-8bda-d6304717305c tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:55.518764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6360b600-66ce-43a6-8bda-d6304717305c tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:55.659408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6360b600-66ce-43a6-8bda-d6304717305c tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:55.667714 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6360b600-66ce-43a6-8bda-d6304717305c tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1499 microversion: 2.1 time: 0.174273 Sep 27 10:38:55.668307 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3007/6483] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:55 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1499 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:55.690110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-95efb4e5-b1c6-46b6-a00b-d485cbbd08bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:55.690719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-95efb4e5-b1c6-46b6-a00b-d485cbbd08bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:55.692924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:55.693611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:38:55.700260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95efb4e5-b1c6-46b6-a00b-d485cbbd08bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:55.700775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95efb4e5-b1c6-46b6-a00b-d485cbbd08bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:55.701207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95efb4e5-b1c6-46b6-a00b-d485cbbd08bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:55.701904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95efb4e5-b1c6-46b6-a00b-d485cbbd08bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:55.702353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95efb4e5-b1c6-46b6-a00b-d485cbbd08bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:55.702771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95efb4e5-b1c6-46b6-a00b-d485cbbd08bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:55.839353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-95efb4e5-b1c6-46b6-a00b-d485cbbd08bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:55.847545 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-95efb4e5-b1c6-46b6-a00b-d485cbbd08bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1499 microversion: 2.1 time: 0.161478 Sep 27 10:38:55.848309 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3477/6484] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:55 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1499 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:55.873348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-50cb55e2-44bb-47c6-9623-b6f40bbb0798 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:38:55.874063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-50cb55e2-44bb-47c6-9623-b6f40bbb0798 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:55.882222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50cb55e2-44bb-47c6-9623-b6f40bbb0798 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:55.882724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50cb55e2-44bb-47c6-9623-b6f40bbb0798 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:55.883172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50cb55e2-44bb-47c6-9623-b6f40bbb0798 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:55.883896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50cb55e2-44bb-47c6-9623-b6f40bbb0798 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:55.884355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50cb55e2-44bb-47c6-9623-b6f40bbb0798 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:55.884828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50cb55e2-44bb-47c6-9623-b6f40bbb0798 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:55.907985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-50cb55e2-44bb-47c6-9623-b6f40bbb0798 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lazy-loading 'flavor' on Instance uuid 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:38:55.984844 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-50cb55e2-44bb-47c6-9623-b6f40bbb0798 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "POST /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0/action" status: 202 len: 0 microversion: 2.1 time: 0.117979 Sep 27 10:38:55.987990 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3008/6485] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:38:55 2022] POST /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0/action => generated 0 bytes in 120 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:38:55.994893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4651162-708e-460f-b42d-bebd5bf394dc tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:55.995536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d4651162-708e-460f-b42d-bebd5bf394dc tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:56.001719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4651162-708e-460f-b42d-bebd5bf394dc tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:56.002200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4651162-708e-460f-b42d-bebd5bf394dc tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:56.002703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4651162-708e-460f-b42d-bebd5bf394dc tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:56.003387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4651162-708e-460f-b42d-bebd5bf394dc tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:56.003967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4651162-708e-460f-b42d-bebd5bf394dc tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:56.004357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4651162-708e-460f-b42d-bebd5bf394dc tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:56.155826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d4651162-708e-460f-b42d-bebd5bf394dc tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:56.166708 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4651162-708e-460f-b42d-bebd5bf394dc tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1507 microversion: 2.1 time: 0.175537 Sep 27 10:38:56.167491 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3478/6486] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:55 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1507 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:57.194895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-929b9e1f-28f4-48e3-9b77-03e87e83b51a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:57.195621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-929b9e1f-28f4-48e3-9b77-03e87e83b51a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:57.202521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929b9e1f-28f4-48e3-9b77-03e87e83b51a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:57.203027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929b9e1f-28f4-48e3-9b77-03e87e83b51a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:57.203486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929b9e1f-28f4-48e3-9b77-03e87e83b51a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:57.204209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929b9e1f-28f4-48e3-9b77-03e87e83b51a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:57.204749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929b9e1f-28f4-48e3-9b77-03e87e83b51a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:57.205196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-929b9e1f-28f4-48e3-9b77-03e87e83b51a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:57.339407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-929b9e1f-28f4-48e3-9b77-03e87e83b51a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:57.347932 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-929b9e1f-28f4-48e3-9b77-03e87e83b51a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1490 microversion: 2.1 time: 0.159035 Sep 27 10:38:57.348529 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3009/6487] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:57 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1490 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:57.370716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2b6e3e3f-7bfd-43a6-a247-44f17a8ca01b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:38:57.371305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2b6e3e3f-7bfd-43a6-a247-44f17a8ca01b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:57.379108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b6e3e3f-7bfd-43a6-a247-44f17a8ca01b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:57.379597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b6e3e3f-7bfd-43a6-a247-44f17a8ca01b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:57.380031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b6e3e3f-7bfd-43a6-a247-44f17a8ca01b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:57.380756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b6e3e3f-7bfd-43a6-a247-44f17a8ca01b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:57.381214 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b6e3e3f-7bfd-43a6-a247-44f17a8ca01b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:57.381641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2b6e3e3f-7bfd-43a6-a247-44f17a8ca01b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:57.456132 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2b6e3e3f-7bfd-43a6-a247-44f17a8ca01b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "POST /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0/action" status: 202 len: 0 microversion: 2.1 time: 0.089221 Sep 27 10:38:57.456876 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3479/6488] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:38:57 2022] POST /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0/action => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:38:57.465521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e71ccd3a-595d-4175-a9d7-cbf60d5f44bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:57.466257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e71ccd3a-595d-4175-a9d7-cbf60d5f44bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:57.472709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e71ccd3a-595d-4175-a9d7-cbf60d5f44bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:57.473253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e71ccd3a-595d-4175-a9d7-cbf60d5f44bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:57.473740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e71ccd3a-595d-4175-a9d7-cbf60d5f44bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:57.474465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e71ccd3a-595d-4175-a9d7-cbf60d5f44bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:57.474954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e71ccd3a-595d-4175-a9d7-cbf60d5f44bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:57.475384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e71ccd3a-595d-4175-a9d7-cbf60d5f44bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:57.950807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e71ccd3a-595d-4175-a9d7-cbf60d5f44bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:57.959034 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e71ccd3a-595d-4175-a9d7-cbf60d5f44bd tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1496 microversion: 2.1 time: 0.497680 Sep 27 10:38:57.959661 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3010/6489] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:57 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1496 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:38:58.984414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b31fe3ce-0a4c-4228-94f6-6d1c2342e551 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:38:58.985147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b31fe3ce-0a4c-4228-94f6-6d1c2342e551 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:38:58.992303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31fe3ce-0a4c-4228-94f6-6d1c2342e551 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:58.992888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31fe3ce-0a4c-4228-94f6-6d1c2342e551 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:58.993368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31fe3ce-0a4c-4228-94f6-6d1c2342e551 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:58.994079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31fe3ce-0a4c-4228-94f6-6d1c2342e551 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:38:58.994529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31fe3ce-0a4c-4228-94f6-6d1c2342e551 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:38:58.994952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b31fe3ce-0a4c-4228-94f6-6d1c2342e551 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:38:59.130619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b31fe3ce-0a4c-4228-94f6-6d1c2342e551 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:38:59.141817 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b31fe3ce-0a4c-4228-94f6-6d1c2342e551 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1496 microversion: 2.1 time: 0.162942 Sep 27 10:38:59.142397 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3480/6490] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:38:58 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1496 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:00.169665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a3943566-cd00-484d-b43b-25a097d3c146 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:00.171835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a3943566-cd00-484d-b43b-25a097d3c146 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:00.190634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3943566-cd00-484d-b43b-25a097d3c146 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:00.191928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3943566-cd00-484d-b43b-25a097d3c146 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:00.192242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3943566-cd00-484d-b43b-25a097d3c146 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:00.193012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3943566-cd00-484d-b43b-25a097d3c146 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:00.193481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3943566-cd00-484d-b43b-25a097d3c146 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:00.193920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a3943566-cd00-484d-b43b-25a097d3c146 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:00.332528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a3943566-cd00-484d-b43b-25a097d3c146 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:00.340860 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a3943566-cd00-484d-b43b-25a097d3c146 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1499 microversion: 2.1 time: 0.178019 Sep 27 10:39:00.341447 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3011/6491] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:00 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1499 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:00.364853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-430c8d9b-20e2-47fa-9bd2-d9b4e43331a4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:39:00.365409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-430c8d9b-20e2-47fa-9bd2-d9b4e43331a4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:00.373123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430c8d9b-20e2-47fa-9bd2-d9b4e43331a4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:00.373613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430c8d9b-20e2-47fa-9bd2-d9b4e43331a4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:00.374048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430c8d9b-20e2-47fa-9bd2-d9b4e43331a4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:00.374779 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430c8d9b-20e2-47fa-9bd2-d9b4e43331a4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:00.375257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430c8d9b-20e2-47fa-9bd2-d9b4e43331a4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:00.375684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430c8d9b-20e2-47fa-9bd2-d9b4e43331a4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:00.399118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-430c8d9b-20e2-47fa-9bd2-d9b4e43331a4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lazy-loading 'flavor' on Instance uuid 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:39:00.475588 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-430c8d9b-20e2-47fa-9bd2-d9b4e43331a4 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "POST /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0/action" status: 202 len: 0 microversion: 2.1 time: 0.115136 Sep 27 10:39:00.476245 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3481/6492] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:39:00 2022] POST /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0/action => generated 0 bytes in 117 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:39:00.484702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b34b51d2-bb03-4612-a475-3594253161ad tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:00.485411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b34b51d2-bb03-4612-a475-3594253161ad tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:00.492349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34b51d2-bb03-4612-a475-3594253161ad tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:00.492909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34b51d2-bb03-4612-a475-3594253161ad tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:00.493433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34b51d2-bb03-4612-a475-3594253161ad tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:00.494183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34b51d2-bb03-4612-a475-3594253161ad tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:00.494664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34b51d2-bb03-4612-a475-3594253161ad tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:00.495130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b34b51d2-bb03-4612-a475-3594253161ad tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:00.631158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b34b51d2-bb03-4612-a475-3594253161ad tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:00.639602 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b34b51d2-bb03-4612-a475-3594253161ad tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1507 microversion: 2.1 time: 0.158798 Sep 27 10:39:00.640199 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3012/6493] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:00 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1507 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:01.665587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5b2fa469-0d80-4a70-964e-4929ccf7b5c1 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:01.666295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5b2fa469-0d80-4a70-964e-4929ccf7b5c1 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:01.673435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b2fa469-0d80-4a70-964e-4929ccf7b5c1 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:01.673927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b2fa469-0d80-4a70-964e-4929ccf7b5c1 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:01.674367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b2fa469-0d80-4a70-964e-4929ccf7b5c1 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:01.675069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b2fa469-0d80-4a70-964e-4929ccf7b5c1 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:01.675531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b2fa469-0d80-4a70-964e-4929ccf7b5c1 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:01.675955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5b2fa469-0d80-4a70-964e-4929ccf7b5c1 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:01.815675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5b2fa469-0d80-4a70-964e-4929ccf7b5c1 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:01.823809 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5b2fa469-0d80-4a70-964e-4929ccf7b5c1 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1490 microversion: 2.1 time: 0.164084 Sep 27 10:39:01.824354 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3482/6494] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:01 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1490 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:01.846616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-738eaff5-5fbc-4721-8f2d-abe286b44c4e tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:39:01.847165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-738eaff5-5fbc-4721-8f2d-abe286b44c4e tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:01.854874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-738eaff5-5fbc-4721-8f2d-abe286b44c4e tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:01.855371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-738eaff5-5fbc-4721-8f2d-abe286b44c4e tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:01.855819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-738eaff5-5fbc-4721-8f2d-abe286b44c4e tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:01.856537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-738eaff5-5fbc-4721-8f2d-abe286b44c4e tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:01.857038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-738eaff5-5fbc-4721-8f2d-abe286b44c4e tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:01.857475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-738eaff5-5fbc-4721-8f2d-abe286b44c4e tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:01.932683 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-738eaff5-5fbc-4721-8f2d-abe286b44c4e tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "POST /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0/action" status: 202 len: 0 microversion: 2.1 time: 0.089699 Sep 27 10:39:01.933303 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3013/6495] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:39:01 2022] POST /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0/action => generated 0 bytes in 91 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:39:01.943829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-61259282-9def-46c9-9d3b-795f7a70d33b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:01.944590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-61259282-9def-46c9-9d3b-795f7a70d33b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:01.951172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61259282-9def-46c9-9d3b-795f7a70d33b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:01.951672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61259282-9def-46c9-9d3b-795f7a70d33b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:01.952099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61259282-9def-46c9-9d3b-795f7a70d33b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:01.952844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61259282-9def-46c9-9d3b-795f7a70d33b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:01.953307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61259282-9def-46c9-9d3b-795f7a70d33b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:01.953733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61259282-9def-46c9-9d3b-795f7a70d33b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:02.091560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-61259282-9def-46c9-9d3b-795f7a70d33b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:02.099898 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-61259282-9def-46c9-9d3b-795f7a70d33b tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1496 microversion: 2.1 time: 0.161108 Sep 27 10:39:02.100479 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3483/6496] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:01 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1496 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:03.130585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6ba44458-e132-413a-8e63-6b22e45be64f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:03.131283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6ba44458-e132-413a-8e63-6b22e45be64f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:03.137772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ba44458-e132-413a-8e63-6b22e45be64f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:03.138275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ba44458-e132-413a-8e63-6b22e45be64f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:03.138722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ba44458-e132-413a-8e63-6b22e45be64f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:03.139720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ba44458-e132-413a-8e63-6b22e45be64f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:03.140187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ba44458-e132-413a-8e63-6b22e45be64f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:03.140620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ba44458-e132-413a-8e63-6b22e45be64f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:03.286913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6ba44458-e132-413a-8e63-6b22e45be64f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:03.295182 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6ba44458-e132-413a-8e63-6b22e45be64f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1496 microversion: 2.1 time: 0.173621 Sep 27 10:39:03.295822 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3014/6497] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:03 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1496 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:04.324476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-af184213-187a-4f8c-b34f-1a878bb773a7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:04.325544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-af184213-187a-4f8c-b34f-1a878bb773a7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:04.336455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af184213-187a-4f8c-b34f-1a878bb773a7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:04.336991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af184213-187a-4f8c-b34f-1a878bb773a7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:04.337466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af184213-187a-4f8c-b34f-1a878bb773a7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:04.338176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af184213-187a-4f8c-b34f-1a878bb773a7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:04.338638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af184213-187a-4f8c-b34f-1a878bb773a7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:04.339071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-af184213-187a-4f8c-b34f-1a878bb773a7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:04.479827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-af184213-187a-4f8c-b34f-1a878bb773a7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:04.487857 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-af184213-187a-4f8c-b34f-1a878bb773a7 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1499 microversion: 2.1 time: 0.171066 Sep 27 10:39:04.488424 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3484/6498] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:04 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1499 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:04.511566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ead31bfb-0bce-4c1a-b8b8-f5ee34b48fb3 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:04.512228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ead31bfb-0bce-4c1a-b8b8-f5ee34b48fb3 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:04.519908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ead31bfb-0bce-4c1a-b8b8-f5ee34b48fb3 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:04.520408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ead31bfb-0bce-4c1a-b8b8-f5ee34b48fb3 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:04.520936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ead31bfb-0bce-4c1a-b8b8-f5ee34b48fb3 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:04.521650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ead31bfb-0bce-4c1a-b8b8-f5ee34b48fb3 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:04.522117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ead31bfb-0bce-4c1a-b8b8-f5ee34b48fb3 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:04.522554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ead31bfb-0bce-4c1a-b8b8-f5ee34b48fb3 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:04.549311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ead31bfb-0bce-4c1a-b8b8-f5ee34b48fb3 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:39:04.552108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:04.643294 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ead31bfb-0bce-4c1a-b8b8-f5ee34b48fb3 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "DELETE /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 204 len: 0 microversion: 2.1 time: 0.135478 Sep 27 10:39:04.643902 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3015/6499] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:39:04 2022] DELETE /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:39:04.654943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ac2bb5a2-26cc-4d42-bef5-3e129871481a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:04.655591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ac2bb5a2-26cc-4d42-bef5-3e129871481a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:04.662225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2bb5a2-26cc-4d42-bef5-3e129871481a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:04.662709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2bb5a2-26cc-4d42-bef5-3e129871481a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:04.663139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2bb5a2-26cc-4d42-bef5-3e129871481a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:04.663841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2bb5a2-26cc-4d42-bef5-3e129871481a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:04.664296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2bb5a2-26cc-4d42-bef5-3e129871481a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:04.664751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac2bb5a2-26cc-4d42-bef5-3e129871481a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:04.807721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ac2bb5a2-26cc-4d42-bef5-3e129871481a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:04.816422 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ac2bb5a2-26cc-4d42-bef5-3e129871481a tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1505 microversion: 2.1 time: 0.167143 Sep 27 10:39:04.817011 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3485/6500] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:04 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1505 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:05.841887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-20f7592e-60bb-4859-acf0-2c203a6ae05f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:05.842651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-20f7592e-60bb-4859-acf0-2c203a6ae05f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:05.850816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20f7592e-60bb-4859-acf0-2c203a6ae05f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:05.851372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20f7592e-60bb-4859-acf0-2c203a6ae05f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:05.851828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20f7592e-60bb-4859-acf0-2c203a6ae05f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:05.852622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20f7592e-60bb-4859-acf0-2c203a6ae05f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:05.853164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20f7592e-60bb-4859-acf0-2c203a6ae05f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:05.853642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20f7592e-60bb-4859-acf0-2c203a6ae05f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:05.996769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-20f7592e-60bb-4859-acf0-2c203a6ae05f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:06.008010 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-20f7592e-60bb-4859-acf0-2c203a6ae05f tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1505 microversion: 2.1 time: 0.170407 Sep 27 10:39:06.008607 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3016/6501] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:05 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1505 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:07.037465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c934dcae-c87e-42cf-8f19-a7e940690a81 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:07.038236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c934dcae-c87e-42cf-8f19-a7e940690a81 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:07.050245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934dcae-c87e-42cf-8f19-a7e940690a81 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:07.050764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934dcae-c87e-42cf-8f19-a7e940690a81 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:07.051197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934dcae-c87e-42cf-8f19-a7e940690a81 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:07.051902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934dcae-c87e-42cf-8f19-a7e940690a81 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:07.052421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934dcae-c87e-42cf-8f19-a7e940690a81 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:07.052802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c934dcae-c87e-42cf-8f19-a7e940690a81 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:07.060484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:07.112012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-98c0f606-0b7c-40d7-954a-b4a71d70ef50 req-9348a383-96d8-46cf-aac9-917aba2bbb0c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8352fb20-d2c7-487e-ae9d-3406cfb8d9e0", "name": "network-vif-deleted", "tag": "a9128636-2d6b-4f91-b749-9c426271dfbc"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:39:07.124630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-98c0f606-0b7c-40d7-954a-b4a71d70ef50 req-9348a383-96d8-46cf-aac9-917aba2bbb0c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:07.125259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-98c0f606-0b7c-40d7-954a-b4a71d70ef50 req-9348a383-96d8-46cf-aac9-917aba2bbb0c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:07.125721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-98c0f606-0b7c-40d7-954a-b4a71d70ef50 req-9348a383-96d8-46cf-aac9-917aba2bbb0c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:07.128387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c934dcae-c87e-42cf-8f19-a7e940690a81 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'ee4c3d891eb9466c8cc00cf5d28b3924', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '91f4b185afc0495eaba8f2ece9562eb2', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:07.137466 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c934dcae-c87e-42cf-8f19-a7e940690a81 tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 200 len: 1463 microversion: 2.1 time: 0.104042 Sep 27 10:39:07.138150 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3486/6502] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:07 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 1463 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:07.138790 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-98c0f606-0b7c-40d7-954a-b4a71d70ef50 req-9348a383-96d8-46cf-aac9-917aba2bbb0c service nova] Creating event network-vif-deleted:a9128636-2d6b-4f91-b749-9c426271dfbc for instance 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 on nch1-master-1 Sep 27 10:39:07.144233 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-98c0f606-0b7c-40d7-954a-b4a71d70ef50 req-9348a383-96d8-46cf-aac9-917aba2bbb0c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036653 Sep 27 10:39:07.144930 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3017/6503] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:39:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:39:08.167683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:08.168755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] [instance: 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:08.180259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:08.180785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:08.181221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:08.181904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:08.182387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:08.183143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:08.199000 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] HTTP exception thrown: Instance 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 could not be found. Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Request method failure captured: Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 HTTP/1.1 Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtKyFIndVdPBgyy12-XQuanqeAtG6W18jz9W50XxiHuajNOlLKkephw98IvqJVVhggtwTfVIZo1NQfKs_NVoYO6sNJDNGooG48KSREF9E6qAauILH8ORFtfYlE2-fHMFyqeGq48a6gNKDZJz5VzybVpRrdZ2KWw1LuOK8Ry3Jk86j_U Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 91f4b185afc0495eaba8f2ece9562eb2 Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestServerAdvancedOps-359605092 Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/91f4b185afc0495eaba8f2ece9562eb2"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/91f4b185afc0495eaba8f2ece9562eb2"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/91f4b185afc0495eaba8f2ece9562eb2"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_91f4b185afc0495eaba8f2ece9562eb2"}]}] Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestServerAdvancedOps-359605092 Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 91f4b185afc0495eaba8f2ece9562eb2 Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestServerAdvancedOps-359605092 Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestServerAdvancedOps-359605092-project Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: ee4c3d891eb9466c8cc00cf5d28b3924 Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestServerAdvancedOps-359605092-project Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 could not be found. Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '8352fb20-d2c7-487e-ae9d-3406cfb8d9e0'} Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 could not be found. Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 could not be found. Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:39:08.200693 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:39:08.205019 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:39:08.205019 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:39:08.205019 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:39:08.205019 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:39:08.205019 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:39:08.205019 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 could not be found. Sep 27 10:39:08.205019 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:39:08.205019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] Returning 404 to user: Instance 8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:39:08.205019 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b76c88ca-7636-41cf-978c-dafc42ebaebe tempest-TestServerAdvancedOps-359605092 tempest-TestServerAdvancedOps-359605092-project] 10.222.0.26 "GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0" status: 404 len: 111 microversion: 2.1 time: 0.042023 Sep 27 10:39:08.205019 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3487/6504] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:08 2022] GET /compute/v2.1/servers/8352fb20-d2c7-487e-ae9d-3406cfb8d9e0 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:39:20.650568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c4fbdd9d-df57-4325-b6d2-bf6a451b446a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:20.651145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c4fbdd9d-df57-4325-b6d2-bf6a451b446a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:20.652250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5ad8aeee-17ec-46de-a7e2-17f5d9d21b8f req-3a3530e8-3a97-4108-95ec-16fd3b490690 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "38097746-54a6-4433-924a-477fcd11f84a", "tag": "3ece88ca-1593-4172-b7bb-d9d53cd7b4d3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:39:20.654718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=41,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:20.655372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=16,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:20.659292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:20.660036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:20.662258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4fbdd9d-df57-4325-b6d2-bf6a451b446a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:20.662952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4fbdd9d-df57-4325-b6d2-bf6a451b446a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:20.663599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4fbdd9d-df57-4325-b6d2-bf6a451b446a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:20.664345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5ad8aeee-17ec-46de-a7e2-17f5d9d21b8f req-3a3530e8-3a97-4108-95ec-16fd3b490690 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:20.664485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4fbdd9d-df57-4325-b6d2-bf6a451b446a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:20.664794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5ad8aeee-17ec-46de-a7e2-17f5d9d21b8f req-3a3530e8-3a97-4108-95ec-16fd3b490690 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:20.664919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4fbdd9d-df57-4325-b6d2-bf6a451b446a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:20.665333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5ad8aeee-17ec-46de-a7e2-17f5d9d21b8f req-3a3530e8-3a97-4108-95ec-16fd3b490690 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:20.665456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c4fbdd9d-df57-4325-b6d2-bf6a451b446a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:20.679267 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5ad8aeee-17ec-46de-a7e2-17f5d9d21b8f req-3a3530e8-3a97-4108-95ec-16fd3b490690 service nova] Creating event network-changed:3ece88ca-1593-4172-b7bb-d9d53cd7b4d3 for instance 38097746-54a6-4433-924a-477fcd11f84a on nch2-master-1 Sep 27 10:39:20.685203 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5ad8aeee-17ec-46de-a7e2-17f5d9d21b8f req-3a3530e8-3a97-4108-95ec-16fd3b490690 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036522 Sep 27 10:39:20.686724 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3488/6505] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:39:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:39:20.696075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c4fbdd9d-df57-4325-b6d2-bf6a451b446a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:39:20.798734 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c4fbdd9d-df57-4325-b6d2-bf6a451b446a tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "DELETE /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 204 len: 0 microversion: 2.1 time: 0.152409 Sep 27 10:39:20.799352 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3018/6506] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:39:20 2022] DELETE /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:39:20.808688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3532efb6-6f32-4f91-8edc-5ba008a76ab0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:20.809392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3532efb6-6f32-4f91-8edc-5ba008a76ab0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:20.820472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3532efb6-6f32-4f91-8edc-5ba008a76ab0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:20.821016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3532efb6-6f32-4f91-8edc-5ba008a76ab0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:20.821455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3532efb6-6f32-4f91-8edc-5ba008a76ab0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:20.822173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3532efb6-6f32-4f91-8edc-5ba008a76ab0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:20.822628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3532efb6-6f32-4f91-8edc-5ba008a76ab0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:20.823050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3532efb6-6f32-4f91-8edc-5ba008a76ab0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:20.965898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3532efb6-6f32-4f91-8edc-5ba008a76ab0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:20.974127 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3532efb6-6f32-4f91-8edc-5ba008a76ab0 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1687 microversion: 2.1 time: 0.170040 Sep 27 10:39:20.974805 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3489/6507] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:20 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1687 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:22.000190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9569fdd2-16ed-4aba-b214-cd94eed2a3d9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:22.001610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9569fdd2-16ed-4aba-b214-cd94eed2a3d9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:22.008962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9569fdd2-16ed-4aba-b214-cd94eed2a3d9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:22.009474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9569fdd2-16ed-4aba-b214-cd94eed2a3d9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:22.010092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9569fdd2-16ed-4aba-b214-cd94eed2a3d9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:22.010824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9569fdd2-16ed-4aba-b214-cd94eed2a3d9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:22.011364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9569fdd2-16ed-4aba-b214-cd94eed2a3d9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:22.011829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9569fdd2-16ed-4aba-b214-cd94eed2a3d9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:22.221712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9569fdd2-16ed-4aba-b214-cd94eed2a3d9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:22.232102 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9569fdd2-16ed-4aba-b214-cd94eed2a3d9 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1687 microversion: 2.1 time: 0.236228 Sep 27 10:39:22.232933 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3019/6508] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:21 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1687 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:23.200154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f4a9d445-6028-416e-a991-8454081e869b req-356ee6eb-f8dd-4940-9920-736b6d0e8e73 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38097746-54a6-4433-924a-477fcd11f84a", "name": "network-vif-deleted", "tag": "3ece88ca-1593-4172-b7bb-d9d53cd7b4d3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:39:23.211365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f4a9d445-6028-416e-a991-8454081e869b req-356ee6eb-f8dd-4940-9920-736b6d0e8e73 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:23.211838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f4a9d445-6028-416e-a991-8454081e869b req-356ee6eb-f8dd-4940-9920-736b6d0e8e73 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:23.212269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f4a9d445-6028-416e-a991-8454081e869b req-356ee6eb-f8dd-4940-9920-736b6d0e8e73 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:23.223840 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f4a9d445-6028-416e-a991-8454081e869b req-356ee6eb-f8dd-4940-9920-736b6d0e8e73 service nova] Creating event network-vif-deleted:3ece88ca-1593-4172-b7bb-d9d53cd7b4d3 for instance 38097746-54a6-4433-924a-477fcd11f84a on nch2-master-1 Sep 27 10:39:23.228754 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f4a9d445-6028-416e-a991-8454081e869b req-356ee6eb-f8dd-4940-9920-736b6d0e8e73 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032731 Sep 27 10:39:23.232648 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3490/6509] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:39:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:39:23.259483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c01f2559-1d7f-41c1-9cd6-32c9f40da262 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:23.260118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c01f2559-1d7f-41c1-9cd6-32c9f40da262 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:23.267002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c01f2559-1d7f-41c1-9cd6-32c9f40da262 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:23.267541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c01f2559-1d7f-41c1-9cd6-32c9f40da262 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:23.267995 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c01f2559-1d7f-41c1-9cd6-32c9f40da262 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:23.268747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c01f2559-1d7f-41c1-9cd6-32c9f40da262 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:23.269232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c01f2559-1d7f-41c1-9cd6-32c9f40da262 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:23.269683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c01f2559-1d7f-41c1-9cd6-32c9f40da262 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:23.338473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c01f2559-1d7f-41c1-9cd6-32c9f40da262 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0bd8a19d0ab44011b5430f26757a0efe', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '86e71212d96f46a78680afaacd8709ca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:23.347467 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c01f2559-1d7f-41c1-9cd6-32c9f40da262 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 200 len: 1349 microversion: 2.1 time: 0.092023 Sep 27 10:39:23.348256 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3020/6510] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:23 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 1349 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:39:24.378497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:24.379558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [instance: 38097746-54a6-4433-924a-477fcd11f84a] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:24.388372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:24.389086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:24.389683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:24.390618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:24.391235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:24.391819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:24.409031 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] HTTP exception thrown: Instance 38097746-54a6-4433-924a-477fcd11f84a could not be found. Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Request method failure captured: Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a HTTP/1.1 Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtAz-I5R9pkEmM8ENwgEXlrfwhpK5QJgpteQwRSpBNpzmPCQ6_jXCbvGwjb68dGwzfz0hV_obx0eVwNZLFOTRd61-pMOQhq1CSGhYJOTDYOHjRjmEKJs84qfkSC88em1BS9SfLSGMC-OPias42e-aueDGLNj17wWnJe-vn54z9Dzb2o Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 86e71212d96f46a78680afaacd8709ca Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/86e71212d96f46a78680afaacd8709ca"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_86e71212d96f46a78680afaacd8709ca"}]}] Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 86e71212d96f46a78680afaacd8709ca Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1611618676 Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestNetworkAdvancedServerOps-1611618676-project Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 0bd8a19d0ab44011b5430f26757a0efe Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1611618676-project Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 38097746-54a6-4433-924a-477fcd11f84a could not be found. Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '38097746-54a6-4433-924a-477fcd11f84a'} Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 38097746-54a6-4433-924a-477fcd11f84a could not be found. Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 38097746-54a6-4433-924a-477fcd11f84a could not be found. Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:39:24.410703 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:39:24.415273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:39:24.415273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:39:24.415273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:39:24.415273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:39:24.415273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:39:24.415273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:39:24.415273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 38097746-54a6-4433-924a-477fcd11f84a could not be found. Sep 27 10:39:24.415273 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:39:24.415273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Returning 404 to user: Instance 38097746-54a6-4433-924a-477fcd11f84a could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:39:24.415273 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6685c0e4-9778-4605-a771-c0c47f13c055 tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a" status: 404 len: 111 microversion: 2.1 time: 0.040761 Sep 27 10:39:24.415273 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3491/6511] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:24 2022] GET /compute/v2.1/servers/38097746-54a6-4433-924a-477fcd11f84a => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:39:27.803803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4b54f598-69e3-4370-a6dd-82c05e367dea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] Calling method '.version_select at 0x7f7f62ba28b0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:27.808419 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [None req-4b54f598-69e3-4370-a6dd-82c05e367dea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [9107af64-6546-47f5-bc26-9366673aaaa8] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 27 10:39:28.821926 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [None req-4b54f598-69e3-4370-a6dd-82c05e367dea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] [9107af64-6546-47f5-bc26-9366673aaaa8] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 53772. Sep 27 10:39:28.847766 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4b54f598-69e3-4370-a6dd-82c05e367dea tempest-TestNetworkAdvancedServerOps-1611618676 tempest-TestNetworkAdvancedServerOps-1611618676-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-955922546" status: 202 len: 0 microversion: 2.1 time: 1.048097 Sep 27 10:39:28.848366 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3021/6512] 10.222.0.26 () {58 vars in 1281 bytes} [Tue Sep 27 10:39:27 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-955922546 => generated 0 bytes in 1049 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:39:56.296291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f834683b-b653-4988-91a0-6d2b7ad49e72 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Action: 'create', calling method: .version_select at 0x7f7f628fef70>, body: {"keypair": {"name": "tempest-TestVolumeBackupRestore-62547702", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCHOR3p28j5PdMQGZ4xIVz+C4vrNERvtyqVy082ukXgdxvSMbPjq65RLvzHbMHeG6QBTs3AsnONUJ1LrZY3pwwOfJqjlNzAb6RyyoAVegScBnKWUrvXaN+0DAyq0pXpsAA=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:39:56.304535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=8 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:56.305357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=12 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:56.319968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f834683b-b653-4988-91a0-6d2b7ad49e72 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Getting quotas for project 874d3b71bb7f44379f7efb4f68c7a14f. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:39:56.325653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-f834683b-b653-4988-91a0-6d2b7ad49e72 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Getting quotas for user 43ffa23d4c3843539e4be19528794e3c and project 874d3b71bb7f44379f7efb4f68c7a14f. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:39:56.360095 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f834683b-b653-4988-91a0-6d2b7ad49e72 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.069161 Sep 27 10:39:56.360744 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3492/6513] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:39:56 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:39:57.763057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:57.764476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=14,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:39:57.770093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBackupRestore-server-206860570", "imageRef": "", "flavorRef": "452", "block_device_mapping_v2": [{"uuid": "2b68bff0-c4ca-4aa3-b043-274fe56530cb", "source_type": "volume", "destination_type": "volume", "boot_index": 0}], "key_name": "tempest-TestVolumeBackupRestore-62547702", "security_groups": [{"name": "tempest-secgroup-smoke-1420624473"}], "networks": [{"uuid": "5ae5d702-34d3-43f2-be17-81f93904176e"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:39:57.907366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] validate_networks() for [('5ae5d702-34d3-43f2-be17-81f93904176e', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:39:58.335262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:39:58.335777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:39:58.336279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:39:58.336770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:39:58.337201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:39:58.346912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:58.347444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:58.347902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:58.348677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:58.349193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:58.349637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:58.367447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Getting quotas for project 874d3b71bb7f44379f7efb4f68c7a14f. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:39:58.373122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Getting quotas for user 43ffa23d4c3843539e4be19528794e3c and project 874d3b71bb7f44379f7efb4f68c7a14f. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:39:58.432449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:39:58.435009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='2b68bff0-c4ca-4aa3-b043-274fe56530cb',volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:39:58.576337 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c698d642-5e40-4c49-9e79-caf7834c34c8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.815232 Sep 27 10:39:58.576970 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3022/6514] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:39:57 2022] POST /compute/v2.1/servers => generated 406 bytes in 817 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:39:58.588669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b77e6ac6-e238-44f4-9fa3-6acf8c9bdc78 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:58.589308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b77e6ac6-e238-44f4-9fa3-6acf8c9bdc78 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:58.651343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b77e6ac6-e238-44f4-9fa3-6acf8c9bdc78 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:58.658491 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b77e6ac6-e238-44f4-9fa3-6acf8c9bdc78 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1096 microversion: 2.1 time: 0.073967 Sep 27 10:39:58.659073 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3493/6515] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:58 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1096 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:39:59.681682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1cee0ffe-353c-42ed-bdb6-61f47797f17a tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:39:59.682446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1cee0ffe-353c-42ed-bdb6-61f47797f17a tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:39:59.692684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cee0ffe-353c-42ed-bdb6-61f47797f17a tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:59.693232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cee0ffe-353c-42ed-bdb6-61f47797f17a tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:59.693697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cee0ffe-353c-42ed-bdb6-61f47797f17a tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:59.694420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cee0ffe-353c-42ed-bdb6-61f47797f17a tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:39:59.694910 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cee0ffe-353c-42ed-bdb6-61f47797f17a tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:39:59.695340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1cee0ffe-353c-42ed-bdb6-61f47797f17a tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:39:59.807405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1cee0ffe-353c-42ed-bdb6-61f47797f17a tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:39:59.819455 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1cee0ffe-353c-42ed-bdb6-61f47797f17a tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1202 microversion: 2.1 time: 0.141798 Sep 27 10:39:59.820047 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3023/6516] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:39:59 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1202 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:00.842955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f11dd055-77c2-4e29-b467-89bd9a58498c tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:00.844254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f11dd055-77c2-4e29-b467-89bd9a58498c tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:00.852412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f11dd055-77c2-4e29-b467-89bd9a58498c tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:00.852941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f11dd055-77c2-4e29-b467-89bd9a58498c tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:00.853585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f11dd055-77c2-4e29-b467-89bd9a58498c tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:00.854523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f11dd055-77c2-4e29-b467-89bd9a58498c tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:00.855442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f11dd055-77c2-4e29-b467-89bd9a58498c tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:00.855442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f11dd055-77c2-4e29-b467-89bd9a58498c tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:01.001571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f11dd055-77c2-4e29-b467-89bd9a58498c tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:01.011710 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f11dd055-77c2-4e29-b467-89bd9a58498c tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1280 microversion: 2.1 time: 0.172639 Sep 27 10:40:01.012324 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3494/6517] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:00 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1280 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:01.523996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d5fae223-c2de-4050-bfc9-051224f35c9e req-cdd58022-d37f-4b6b-a192-b7401bf4cfe8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b364c9c1-5d59-42aa-ae75-fd70ed2584b4", "tag": "2b55d139-718f-4b96-9ee9-8269bf85d14e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:40:01.536288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d5fae223-c2de-4050-bfc9-051224f35c9e req-cdd58022-d37f-4b6b-a192-b7401bf4cfe8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:01.536728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d5fae223-c2de-4050-bfc9-051224f35c9e req-cdd58022-d37f-4b6b-a192-b7401bf4cfe8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:01.537376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d5fae223-c2de-4050-bfc9-051224f35c9e req-cdd58022-d37f-4b6b-a192-b7401bf4cfe8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:01.549399 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d5fae223-c2de-4050-bfc9-051224f35c9e req-cdd58022-d37f-4b6b-a192-b7401bf4cfe8 service nova] Creating event network-changed:2b55d139-718f-4b96-9ee9-8269bf85d14e for instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 on nch2-master-1 Sep 27 10:40:01.554691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d5fae223-c2de-4050-bfc9-051224f35c9e req-cdd58022-d37f-4b6b-a192-b7401bf4cfe8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034836 Sep 27 10:40:01.555296 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3024/6518] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:40:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:40:02.035282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fbf69110-180d-48b6-840b-4cff3994c288 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:02.035832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fbf69110-180d-48b6-840b-4cff3994c288 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:02.042991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf69110-180d-48b6-840b-4cff3994c288 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:02.043499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf69110-180d-48b6-840b-4cff3994c288 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:02.043946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf69110-180d-48b6-840b-4cff3994c288 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:02.044679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf69110-180d-48b6-840b-4cff3994c288 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:02.045147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf69110-180d-48b6-840b-4cff3994c288 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:02.045581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fbf69110-180d-48b6-840b-4cff3994c288 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:02.204466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fbf69110-180d-48b6-840b-4cff3994c288 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:02.212786 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fbf69110-180d-48b6-840b-4cff3994c288 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1268 microversion: 2.1 time: 0.181586 Sep 27 10:40:02.213370 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3495/6519] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:02 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1268 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:03.240177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4e73af14-dc58-4360-8cf9-37ff26c3abfe tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:03.241238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4e73af14-dc58-4360-8cf9-37ff26c3abfe tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:03.247667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e73af14-dc58-4360-8cf9-37ff26c3abfe tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:03.248168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e73af14-dc58-4360-8cf9-37ff26c3abfe tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:03.248618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e73af14-dc58-4360-8cf9-37ff26c3abfe tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:03.249382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e73af14-dc58-4360-8cf9-37ff26c3abfe tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:03.249852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e73af14-dc58-4360-8cf9-37ff26c3abfe tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:03.250285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4e73af14-dc58-4360-8cf9-37ff26c3abfe tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:03.486084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4e73af14-dc58-4360-8cf9-37ff26c3abfe tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:03.494603 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4e73af14-dc58-4360-8cf9-37ff26c3abfe tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1268 microversion: 2.1 time: 0.261135 Sep 27 10:40:03.495433 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3025/6520] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:03 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1268 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:04.521035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1d11e627-c3e1-45a8-85bb-d182784261eb tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:04.521731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1d11e627-c3e1-45a8-85bb-d182784261eb tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:04.530257 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d11e627-c3e1-45a8-85bb-d182784261eb tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:04.530831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d11e627-c3e1-45a8-85bb-d182784261eb tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:04.531349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d11e627-c3e1-45a8-85bb-d182784261eb tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:04.532174 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d11e627-c3e1-45a8-85bb-d182784261eb tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:04.532707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d11e627-c3e1-45a8-85bb-d182784261eb tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:04.533249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1d11e627-c3e1-45a8-85bb-d182784261eb tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:04.675716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1d11e627-c3e1-45a8-85bb-d182784261eb tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:04.684352 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1d11e627-c3e1-45a8-85bb-d182784261eb tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1268 microversion: 2.1 time: 0.169685 Sep 27 10:40:04.684996 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3496/6521] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:04 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:05.711078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c27087ac-f5a3-420d-b76a-afd64f2a9601 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:05.711934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c27087ac-f5a3-420d-b76a-afd64f2a9601 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:05.722220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c27087ac-f5a3-420d-b76a-afd64f2a9601 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:05.722756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c27087ac-f5a3-420d-b76a-afd64f2a9601 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:05.723230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c27087ac-f5a3-420d-b76a-afd64f2a9601 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:05.723931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c27087ac-f5a3-420d-b76a-afd64f2a9601 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:05.724422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c27087ac-f5a3-420d-b76a-afd64f2a9601 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:05.724889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c27087ac-f5a3-420d-b76a-afd64f2a9601 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:05.875743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c27087ac-f5a3-420d-b76a-afd64f2a9601 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:05.884670 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c27087ac-f5a3-420d-b76a-afd64f2a9601 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1268 microversion: 2.1 time: 0.179574 Sep 27 10:40:05.885293 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3026/6522] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:05 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:06.910276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2efd4e57-094e-482f-89db-caa1d1e71ed8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:06.910950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2efd4e57-094e-482f-89db-caa1d1e71ed8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:06.920304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2efd4e57-094e-482f-89db-caa1d1e71ed8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:06.920972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2efd4e57-094e-482f-89db-caa1d1e71ed8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:06.921527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2efd4e57-094e-482f-89db-caa1d1e71ed8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:06.922408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2efd4e57-094e-482f-89db-caa1d1e71ed8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:06.922976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2efd4e57-094e-482f-89db-caa1d1e71ed8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:06.923517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2efd4e57-094e-482f-89db-caa1d1e71ed8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:07.073840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2efd4e57-094e-482f-89db-caa1d1e71ed8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:07.082124 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2efd4e57-094e-482f-89db-caa1d1e71ed8 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1268 microversion: 2.1 time: 0.177490 Sep 27 10:40:07.082700 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3497/6523] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:06 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1268 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:08.109381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b4971182-f8b0-4976-977d-4a3ab9b4f996 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:08.110401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b4971182-f8b0-4976-977d-4a3ab9b4f996 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:08.122026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4971182-f8b0-4976-977d-4a3ab9b4f996 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:08.122852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4971182-f8b0-4976-977d-4a3ab9b4f996 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:08.123596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4971182-f8b0-4976-977d-4a3ab9b4f996 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:08.124813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4971182-f8b0-4976-977d-4a3ab9b4f996 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:08.125596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4971182-f8b0-4976-977d-4a3ab9b4f996 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:08.126360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4971182-f8b0-4976-977d-4a3ab9b4f996 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:08.276133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b4971182-f8b0-4976-977d-4a3ab9b4f996 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:08.284087 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b4971182-f8b0-4976-977d-4a3ab9b4f996 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1268 microversion: 2.1 time: 0.181334 Sep 27 10:40:08.284743 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3027/6524] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:08 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1268 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:09.203370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-9cf373b2-d00c-4ec3-ab62-9ea72a552a9c req-5154224e-07f8-4a0e-a07f-4a49ef73788b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b364c9c1-5d59-42aa-ae75-fd70ed2584b4", "name": "network-vif-plugged", "status": "completed", "tag": "2b55d139-718f-4b96-9ee9-8269bf85d14e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:40:09.209243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:09.215676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9cf373b2-d00c-4ec3-ab62-9ea72a552a9c req-5154224e-07f8-4a0e-a07f-4a49ef73788b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:09.216292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9cf373b2-d00c-4ec3-ab62-9ea72a552a9c req-5154224e-07f8-4a0e-a07f-4a49ef73788b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:09.216849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9cf373b2-d00c-4ec3-ab62-9ea72a552a9c req-5154224e-07f8-4a0e-a07f-4a49ef73788b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:09.231530 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-9cf373b2-d00c-4ec3-ab62-9ea72a552a9c req-5154224e-07f8-4a0e-a07f-4a49ef73788b service nova] Creating event network-vif-plugged:2b55d139-718f-4b96-9ee9-8269bf85d14e for instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 on nch2-master-1 Sep 27 10:40:09.237225 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-9cf373b2-d00c-4ec3-ab62-9ea72a552a9c req-5154224e-07f8-4a0e-a07f-4a49ef73788b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037744 Sep 27 10:40:09.237993 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3498/6525] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:40:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:40:09.306906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d7280ee8-f6ee-424d-9d50-a09dd5ebfecf tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:09.307526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d7280ee8-f6ee-424d-9d50-a09dd5ebfecf tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:09.309974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:09.310802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:09.318884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7280ee8-f6ee-424d-9d50-a09dd5ebfecf tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:09.319381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7280ee8-f6ee-424d-9d50-a09dd5ebfecf tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:09.319841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7280ee8-f6ee-424d-9d50-a09dd5ebfecf tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:09.320572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7280ee8-f6ee-424d-9d50-a09dd5ebfecf tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:09.321090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7280ee8-f6ee-424d-9d50-a09dd5ebfecf tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:09.321545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7280ee8-f6ee-424d-9d50-a09dd5ebfecf tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:09.503874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d7280ee8-f6ee-424d-9d50-a09dd5ebfecf tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:09.511919 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d7280ee8-f6ee-424d-9d50-a09dd5ebfecf tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1268 microversion: 2.1 time: 0.209066 Sep 27 10:40:09.512477 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3028/6526] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:09 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1268 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:10.542251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c64f0b2f-c1dc-478f-818d-bd52582941a1 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:10.543363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c64f0b2f-c1dc-478f-818d-bd52582941a1 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:10.553000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c64f0b2f-c1dc-478f-818d-bd52582941a1 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:10.553567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c64f0b2f-c1dc-478f-818d-bd52582941a1 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:10.553957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c64f0b2f-c1dc-478f-818d-bd52582941a1 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:10.554643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c64f0b2f-c1dc-478f-818d-bd52582941a1 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:10.555152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c64f0b2f-c1dc-478f-818d-bd52582941a1 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:10.555516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c64f0b2f-c1dc-478f-818d-bd52582941a1 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:10.693802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c64f0b2f-c1dc-478f-818d-bd52582941a1 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:10.702225 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c64f0b2f-c1dc-478f-818d-bd52582941a1 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1268 microversion: 2.1 time: 0.168651 Sep 27 10:40:10.702806 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3499/6527] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:10 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:11.725634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aabea954-7bd8-4a0f-be90-fc19f0a879c9 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:11.726245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aabea954-7bd8-4a0f-be90-fc19f0a879c9 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:11.733697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aabea954-7bd8-4a0f-be90-fc19f0a879c9 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:11.734203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aabea954-7bd8-4a0f-be90-fc19f0a879c9 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:11.734650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aabea954-7bd8-4a0f-be90-fc19f0a879c9 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:11.735382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aabea954-7bd8-4a0f-be90-fc19f0a879c9 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:11.735836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aabea954-7bd8-4a0f-be90-fc19f0a879c9 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:11.736276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aabea954-7bd8-4a0f-be90-fc19f0a879c9 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:11.874622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aabea954-7bd8-4a0f-be90-fc19f0a879c9 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:11.885386 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aabea954-7bd8-4a0f-be90-fc19f0a879c9 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1457 microversion: 2.1 time: 0.164217 Sep 27 10:40:11.885961 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3029/6528] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:11 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1457 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:11.908077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-74605034-b25d-4b37-9f1e-69bfe3fdf2b7 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:11.908691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-74605034-b25d-4b37-9f1e-69bfe3fdf2b7 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:11.916294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74605034-b25d-4b37-9f1e-69bfe3fdf2b7 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:11.916798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74605034-b25d-4b37-9f1e-69bfe3fdf2b7 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:11.917229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74605034-b25d-4b37-9f1e-69bfe3fdf2b7 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:11.917917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74605034-b25d-4b37-9f1e-69bfe3fdf2b7 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:11.918364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74605034-b25d-4b37-9f1e-69bfe3fdf2b7 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:11.918787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74605034-b25d-4b37-9f1e-69bfe3fdf2b7 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:12.068841 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-74605034-b25d-4b37-9f1e-69bfe3fdf2b7 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:12.081127 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-74605034-b25d-4b37-9f1e-69bfe3fdf2b7 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1457 microversion: 2.1 time: 0.176647 Sep 27 10:40:12.081874 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3500/6529] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:11 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1457 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:14.021136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d8681a24-2d76-4145-af94-02a46b9f71bb req-83de7936-3df5-4573-8f34-865c81d1d55d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b364c9c1-5d59-42aa-ae75-fd70ed2584b4", "tag": "2b55d139-718f-4b96-9ee9-8269bf85d14e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:40:14.031006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d8681a24-2d76-4145-af94-02a46b9f71bb req-83de7936-3df5-4573-8f34-865c81d1d55d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:14.031519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d8681a24-2d76-4145-af94-02a46b9f71bb req-83de7936-3df5-4573-8f34-865c81d1d55d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:14.032157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d8681a24-2d76-4145-af94-02a46b9f71bb req-83de7936-3df5-4573-8f34-865c81d1d55d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:14.043748 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d8681a24-2d76-4145-af94-02a46b9f71bb req-83de7936-3df5-4573-8f34-865c81d1d55d service nova] Creating event network-changed:2b55d139-718f-4b96-9ee9-8269bf85d14e for instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 on nch2-master-1 Sep 27 10:40:14.049412 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d8681a24-2d76-4145-af94-02a46b9f71bb req-83de7936-3df5-4573-8f34-865c81d1d55d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.167463 Sep 27 10:40:14.050083 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3030/6530] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:40:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:40:15.499296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-b03fc694-fd93-4138-9603-8cde6445ffaa req-5bcbbf2a-4911-4154-bdab-eaa3d051f5e8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b364c9c1-5d59-42aa-ae75-fd70ed2584b4", "tag": "2b55d139-718f-4b96-9ee9-8269bf85d14e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:40:15.511343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b03fc694-fd93-4138-9603-8cde6445ffaa req-5bcbbf2a-4911-4154-bdab-eaa3d051f5e8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:15.512228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b03fc694-fd93-4138-9603-8cde6445ffaa req-5bcbbf2a-4911-4154-bdab-eaa3d051f5e8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:15.512888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-b03fc694-fd93-4138-9603-8cde6445ffaa req-5bcbbf2a-4911-4154-bdab-eaa3d051f5e8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:15.525291 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-b03fc694-fd93-4138-9603-8cde6445ffaa req-5bcbbf2a-4911-4154-bdab-eaa3d051f5e8 service nova] Creating event network-changed:2b55d139-718f-4b96-9ee9-8269bf85d14e for instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 on nch2-master-1 Sep 27 10:40:15.540008 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-b03fc694-fd93-4138-9603-8cde6445ffaa req-5bcbbf2a-4911-4154-bdab-eaa3d051f5e8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044385 Sep 27 10:40:15.540709 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3501/6531] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:40:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:40:17.559770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-8a31a25e-a0a7-4196-9f7b-97353d867007 req-469df687-b66f-40dd-be1d-20253fee8275 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b364c9c1-5d59-42aa-ae75-fd70ed2584b4", "tag": "2b55d139-718f-4b96-9ee9-8269bf85d14e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:40:17.570693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a31a25e-a0a7-4196-9f7b-97353d867007 req-469df687-b66f-40dd-be1d-20253fee8275 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:17.571183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a31a25e-a0a7-4196-9f7b-97353d867007 req-469df687-b66f-40dd-be1d-20253fee8275 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:17.571633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8a31a25e-a0a7-4196-9f7b-97353d867007 req-469df687-b66f-40dd-be1d-20253fee8275 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:17.584022 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8a31a25e-a0a7-4196-9f7b-97353d867007 req-469df687-b66f-40dd-be1d-20253fee8275 service nova] Creating event network-changed:2b55d139-718f-4b96-9ee9-8269bf85d14e for instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 on nch2-master-1 Sep 27 10:40:17.589324 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-8a31a25e-a0a7-4196-9f7b-97353d867007 req-469df687-b66f-40dd-be1d-20253fee8275 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034827 Sep 27 10:40:17.589917 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3031/6532] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:40:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:40:56.887536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=31,nova_api:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:56.889324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=40 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:56.889623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:56.890529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=44 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:40:56.894279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-df49c12d-1bcb-4760-9b92-08303d91f097 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:56.895019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-df49c12d-1bcb-4760-9b92-08303d91f097 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:56.895838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-fab7ae70-8cf5-4fd9-a3b4-a77e7e59ef48 req-f792634c-7170-474b-b4ae-45113c5065f6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b364c9c1-5d59-42aa-ae75-fd70ed2584b4", "tag": "2b55d139-718f-4b96-9ee9-8269bf85d14e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:40:56.908401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fab7ae70-8cf5-4fd9-a3b4-a77e7e59ef48 req-f792634c-7170-474b-b4ae-45113c5065f6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:56.908930 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fab7ae70-8cf5-4fd9-a3b4-a77e7e59ef48 req-f792634c-7170-474b-b4ae-45113c5065f6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:56.909380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fab7ae70-8cf5-4fd9-a3b4-a77e7e59ef48 req-f792634c-7170-474b-b4ae-45113c5065f6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:56.919933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df49c12d-1bcb-4760-9b92-08303d91f097 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:56.921182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df49c12d-1bcb-4760-9b92-08303d91f097 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:56.921182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df49c12d-1bcb-4760-9b92-08303d91f097 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:56.921590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df49c12d-1bcb-4760-9b92-08303d91f097 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:56.921916 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-fab7ae70-8cf5-4fd9-a3b4-a77e7e59ef48 req-f792634c-7170-474b-b4ae-45113c5065f6 service nova] Creating event network-changed:2b55d139-718f-4b96-9ee9-8269bf85d14e for instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 on nch2-master-1 Sep 27 10:40:56.922067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df49c12d-1bcb-4760-9b92-08303d91f097 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:56.922452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-df49c12d-1bcb-4760-9b92-08303d91f097 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:56.928264 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-fab7ae70-8cf5-4fd9-a3b4-a77e7e59ef48 req-f792634c-7170-474b-b4ae-45113c5065f6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041123 Sep 27 10:40:56.931103 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3032/6533] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:40:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:40:56.956098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-df49c12d-1bcb-4760-9b92-08303d91f097 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:40:57.067161 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-df49c12d-1bcb-4760-9b92-08303d91f097 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "DELETE /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 204 len: 0 microversion: 2.1 time: 0.182290 Sep 27 10:40:57.067981 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3502/6534] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:40:56 2022] DELETE /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:40:57.078380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c0d77ea6-5543-4653-be54-97b04292d537 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:57.079141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c0d77ea6-5543-4653-be54-97b04292d537 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:57.086291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d77ea6-5543-4653-be54-97b04292d537 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:57.086683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d77ea6-5543-4653-be54-97b04292d537 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:57.087342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d77ea6-5543-4653-be54-97b04292d537 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:57.088108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d77ea6-5543-4653-be54-97b04292d537 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:57.088594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d77ea6-5543-4653-be54-97b04292d537 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:57.089378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c0d77ea6-5543-4653-be54-97b04292d537 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:57.258738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c0d77ea6-5543-4653-be54-97b04292d537 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:57.268400 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c0d77ea6-5543-4653-be54-97b04292d537 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1581 microversion: 2.1 time: 0.195119 Sep 27 10:40:57.269125 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3033/6535] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:57 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1581 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:58.296293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-17352784-229a-4224-af1d-bd9e16d22c10 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:58.297034 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-17352784-229a-4224-af1d-bd9e16d22c10 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:58.304023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17352784-229a-4224-af1d-bd9e16d22c10 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:58.304565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17352784-229a-4224-af1d-bd9e16d22c10 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:58.305007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17352784-229a-4224-af1d-bd9e16d22c10 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:58.306285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17352784-229a-4224-af1d-bd9e16d22c10 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:58.306742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17352784-229a-4224-af1d-bd9e16d22c10 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:58.307172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17352784-229a-4224-af1d-bd9e16d22c10 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:58.450307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17352784-229a-4224-af1d-bd9e16d22c10 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:58.458247 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-17352784-229a-4224-af1d-bd9e16d22c10 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1581 microversion: 2.1 time: 0.166318 Sep 27 10:40:58.458763 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3503/6536] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:58 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1581 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:40:59.432961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-06ef88ff-15ac-4096-b9af-536bde335d78 req-fadf4cdc-4148-417e-878f-ec3bcc958bc2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b364c9c1-5d59-42aa-ae75-fd70ed2584b4", "name": "network-vif-deleted", "tag": "2b55d139-718f-4b96-9ee9-8269bf85d14e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:40:59.443706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-06ef88ff-15ac-4096-b9af-536bde335d78 req-fadf4cdc-4148-417e-878f-ec3bcc958bc2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:59.444206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-06ef88ff-15ac-4096-b9af-536bde335d78 req-fadf4cdc-4148-417e-878f-ec3bcc958bc2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:59.444647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-06ef88ff-15ac-4096-b9af-536bde335d78 req-fadf4cdc-4148-417e-878f-ec3bcc958bc2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:59.456524 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-06ef88ff-15ac-4096-b9af-536bde335d78 req-fadf4cdc-4148-417e-878f-ec3bcc958bc2 service nova] Creating event network-vif-deleted:2b55d139-718f-4b96-9ee9-8269bf85d14e for instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 on nch2-master-1 Sep 27 10:40:59.461739 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-06ef88ff-15ac-4096-b9af-536bde335d78 req-fadf4cdc-4148-417e-878f-ec3bcc958bc2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032833 Sep 27 10:40:59.462516 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3034/6537] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:40:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:40:59.480885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-17d5525e-8273-46be-90dd-3cd037c976c2 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:40:59.481621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-17d5525e-8273-46be-90dd-3cd037c976c2 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:40:59.489270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d5525e-8273-46be-90dd-3cd037c976c2 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:59.490051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d5525e-8273-46be-90dd-3cd037c976c2 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:59.490476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d5525e-8273-46be-90dd-3cd037c976c2 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:59.491205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d5525e-8273-46be-90dd-3cd037c976c2 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:40:59.491789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d5525e-8273-46be-90dd-3cd037c976c2 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:40:59.492218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-17d5525e-8273-46be-90dd-3cd037c976c2 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:40:59.567522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-17d5525e-8273-46be-90dd-3cd037c976c2 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '43ffa23d4c3843539e4be19528794e3c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '874d3b71bb7f44379f7efb4f68c7a14f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:40:59.576014 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-17d5525e-8273-46be-90dd-3cd037c976c2 tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 200 len: 1227 microversion: 2.1 time: 0.099009 Sep 27 10:40:59.576588 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3504/6538] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:40:59 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 1227 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:00.606161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:00.607722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] [instance: b364c9c1-5d59-42aa-ae75-fd70ed2584b4] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:00.621756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:00.622639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:00.623169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:00.623884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:00.624347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:00.624789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:00.639404 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] HTTP exception thrown: Instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 could not be found. Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Request method failure captured: Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 HTTP/1.1 Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtLnVVdXMW3V2Y-ZPWpazm6wxhX_rJ5Nys9LyunOmoVI9xmEymurN7vlBem84gUznYQSVgH9Ld0ROsKWNYYI7nIN_0Hbi0w60ERyj0Psh1dtm0r2rwjmBUclbhFLHEHSkCEsQbG6LUq5I0CUm7W-AYG2o7JS_TXXZoE52xBJeD-t1Kk Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 874d3b71bb7f44379f7efb4f68c7a14f Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestVolumeBackupRestore-883386072 Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/874d3b71bb7f44379f7efb4f68c7a14f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/874d3b71bb7f44379f7efb4f68c7a14f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/874d3b71bb7f44379f7efb4f68c7a14f"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_874d3b71bb7f44379f7efb4f68c7a14f"}]}] Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestVolumeBackupRestore-883386072 Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 874d3b71bb7f44379f7efb4f68c7a14f Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestVolumeBackupRestore-883386072 Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestVolumeBackupRestore-883386072-project Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 43ffa23d4c3843539e4be19528794e3c Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestVolumeBackupRestore-883386072-project Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 could not be found. Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'b364c9c1-5d59-42aa-ae75-fd70ed2584b4'} Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 could not be found. Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 could not be found. Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:41:00.641143 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:00.645741 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:00.645741 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:41:00.645741 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:41:00.645741 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:41:00.645741 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:41:00.645741 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 could not be found. Sep 27 10:41:00.645741 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:00.645741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Returning 404 to user: Instance b364c9c1-5d59-42aa-ae75-fd70ed2584b4 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:41:00.645741 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7d255c0f-cb4a-4e5b-9895-38ded3a0e99f tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4" status: 404 len: 111 microversion: 2.1 time: 0.043645 Sep 27 10:41:00.645741 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3035/6539] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:00 2022] GET /compute/v2.1/servers/b364c9c1-5d59-42aa-ae75-fd70ed2584b4 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:41:00.813192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2183119f-281a-4b81-a47c-431e2b4d992e tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] Calling method '.version_select at 0x7f7f6255d8b0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:00.837423 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2183119f-281a-4b81-a47c-431e2b4d992e tempest-TestVolumeBackupRestore-883386072 tempest-TestVolumeBackupRestore-883386072-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBackupRestore-62547702" status: 202 len: 0 microversion: 2.1 time: 0.028436 Sep 27 10:41:00.838043 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3505/6540] 10.222.0.26 () {58 vars in 1263 bytes} [Tue Sep 27 10:41:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBackupRestore-62547702 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:41:23.612057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:23.612840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=12 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:23.738988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersAdminNegativeTestJSON-server-1691913943", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:23.794069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] validate_networks() for None {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:41:24.073052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:41:24.073486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:41:24.073998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:41:24.074451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:41:24.074873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:41:24.084680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:24.085175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:24.085643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:24.086395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:24.086896 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:24.087337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:24.104620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Getting quotas for project dc6b14a8306e4e23a8f88b44ec0b65e9. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:41:24.110098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Getting quotas for user 1c0ef8d535824421863e905c2b3d6784 and project dc6b14a8306e4e23a8f88b44ec0b65e9. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:41:24.123491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:41:24.125877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:41:24.174818 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-82884c13-7cf7-415e-ab9d-d5046f44e416 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.567754 Sep 27 10:41:24.175468 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3036/6541] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:41:23 2022] POST /compute/v2.1/servers => generated 380 bytes in 569 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:41:24.189270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5eb595bf-5325-445c-8471-34ca57d94360 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:24.189849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5eb595bf-5325-445c-8471-34ca57d94360 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:24.192203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:24.192861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=14,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:24.250856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5eb595bf-5325-445c-8471-34ca57d94360 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:24.258445 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5eb595bf-5325-445c-8471-34ca57d94360 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1222 microversion: 2.1 time: 0.073258 Sep 27 10:41:24.259100 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3506/6542] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:24 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1222 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:25.282835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8ee9bdd3-a37c-4967-826d-2179225d434e tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:25.284039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8ee9bdd3-a37c-4967-826d-2179225d434e tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:25.337946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8ee9bdd3-a37c-4967-826d-2179225d434e tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:25.345343 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8ee9bdd3-a37c-4967-826d-2179225d434e tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1222 microversion: 2.1 time: 0.067344 Sep 27 10:41:25.345912 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3037/6543] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:25 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1222 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:26.370272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a151236e-6e49-4b78-8cc0-8baa07ec2c27 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:26.370910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a151236e-6e49-4b78-8cc0-8baa07ec2c27 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:26.381270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a151236e-6e49-4b78-8cc0-8baa07ec2c27 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:26.381914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a151236e-6e49-4b78-8cc0-8baa07ec2c27 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:26.382467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a151236e-6e49-4b78-8cc0-8baa07ec2c27 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:26.383149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a151236e-6e49-4b78-8cc0-8baa07ec2c27 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:26.383601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a151236e-6e49-4b78-8cc0-8baa07ec2c27 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:26.384026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a151236e-6e49-4b78-8cc0-8baa07ec2c27 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:26.447238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a151236e-6e49-4b78-8cc0-8baa07ec2c27 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:26.455501 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a151236e-6e49-4b78-8cc0-8baa07ec2c27 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1226 microversion: 2.1 time: 0.090947 Sep 27 10:41:26.456095 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3507/6544] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:26 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:41:27.480220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd4fd3c7-9c6e-4aa9-8a49-c68a5244b881 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:27.480871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd4fd3c7-9c6e-4aa9-8a49-c68a5244b881 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:27.488612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4fd3c7-9c6e-4aa9-8a49-c68a5244b881 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:27.489161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4fd3c7-9c6e-4aa9-8a49-c68a5244b881 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:27.489617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4fd3c7-9c6e-4aa9-8a49-c68a5244b881 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:27.490345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4fd3c7-9c6e-4aa9-8a49-c68a5244b881 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:27.490814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4fd3c7-9c6e-4aa9-8a49-c68a5244b881 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:27.491253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd4fd3c7-9c6e-4aa9-8a49-c68a5244b881 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:27.618724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fd4fd3c7-9c6e-4aa9-8a49-c68a5244b881 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:27.626829 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd4fd3c7-9c6e-4aa9-8a49-c68a5244b881 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1280 microversion: 2.1 time: 0.151204 Sep 27 10:41:27.627428 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3038/6545] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:27 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1280 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:28.649794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3ca8fefc-7230-4923-872e-bb6284714361 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:28.650425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3ca8fefc-7230-4923-872e-bb6284714361 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:28.657362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca8fefc-7230-4923-872e-bb6284714361 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:28.657885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca8fefc-7230-4923-872e-bb6284714361 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:28.658320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca8fefc-7230-4923-872e-bb6284714361 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:28.659029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca8fefc-7230-4923-872e-bb6284714361 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:28.659489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca8fefc-7230-4923-872e-bb6284714361 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:28.659923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3ca8fefc-7230-4923-872e-bb6284714361 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:28.796793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3ca8fefc-7230-4923-872e-bb6284714361 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:28.806143 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3ca8fefc-7230-4923-872e-bb6284714361 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1322 microversion: 2.1 time: 0.160265 Sep 27 10:41:28.806772 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3508/6546] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:28 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:29.077127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-27787f18-0e66-484e-bb62-d6174a564cfc req-c8fec742-aa49-4e24-839e-6c6be87af538 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19bf47c4-400c-48ba-b3e7-91132241d9e5", "tag": "430781a5-9f1f-40bc-9e94-10d2d463e895"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:29.089180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-27787f18-0e66-484e-bb62-d6174a564cfc req-c8fec742-aa49-4e24-839e-6c6be87af538 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:29.089679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-27787f18-0e66-484e-bb62-d6174a564cfc req-c8fec742-aa49-4e24-839e-6c6be87af538 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:29.090119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-27787f18-0e66-484e-bb62-d6174a564cfc req-c8fec742-aa49-4e24-839e-6c6be87af538 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:29.102753 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-27787f18-0e66-484e-bb62-d6174a564cfc req-c8fec742-aa49-4e24-839e-6c6be87af538 service nova] Creating event network-changed:430781a5-9f1f-40bc-9e94-10d2d463e895 for instance 19bf47c4-400c-48ba-b3e7-91132241d9e5 on nch1-master-1 Sep 27 10:41:29.107983 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-27787f18-0e66-484e-bb62-d6174a564cfc req-c8fec742-aa49-4e24-839e-6c6be87af538 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034990 Sep 27 10:41:29.108865 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3039/6547] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:41:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:41:29.829114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-be7b895c-5265-49c4-810c-4e3a5d429995 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:29.829707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-be7b895c-5265-49c4-810c-4e3a5d429995 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:29.837542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be7b895c-5265-49c4-810c-4e3a5d429995 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:29.838204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be7b895c-5265-49c4-810c-4e3a5d429995 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:29.838660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be7b895c-5265-49c4-810c-4e3a5d429995 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:29.839399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be7b895c-5265-49c4-810c-4e3a5d429995 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:29.839869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be7b895c-5265-49c4-810c-4e3a5d429995 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:29.840303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-be7b895c-5265-49c4-810c-4e3a5d429995 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:29.977294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-be7b895c-5265-49c4-810c-4e3a5d429995 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:29.985267 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-be7b895c-5265-49c4-810c-4e3a5d429995 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1322 microversion: 2.1 time: 0.160248 Sep 27 10:41:29.985851 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3509/6548] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:29 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1322 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:31.013210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fb75970c-3a93-4623-a422-5e6e1bfef966 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:31.013814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fb75970c-3a93-4623-a422-5e6e1bfef966 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:31.020857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb75970c-3a93-4623-a422-5e6e1bfef966 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:31.021358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb75970c-3a93-4623-a422-5e6e1bfef966 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:31.021803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb75970c-3a93-4623-a422-5e6e1bfef966 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:31.022530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb75970c-3a93-4623-a422-5e6e1bfef966 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:31.023004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb75970c-3a93-4623-a422-5e6e1bfef966 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:31.023442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fb75970c-3a93-4623-a422-5e6e1bfef966 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:31.161144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fb75970c-3a93-4623-a422-5e6e1bfef966 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:31.169492 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fb75970c-3a93-4623-a422-5e6e1bfef966 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1322 microversion: 2.1 time: 0.163378 Sep 27 10:41:31.170068 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3040/6549] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:31 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1322 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:32.192982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-96da1c40-d14c-43dc-8197-bc41b41d9847 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:32.193542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-96da1c40-d14c-43dc-8197-bc41b41d9847 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:32.200735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96da1c40-d14c-43dc-8197-bc41b41d9847 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:32.201231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96da1c40-d14c-43dc-8197-bc41b41d9847 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:32.201673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96da1c40-d14c-43dc-8197-bc41b41d9847 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:32.202415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96da1c40-d14c-43dc-8197-bc41b41d9847 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:32.202876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96da1c40-d14c-43dc-8197-bc41b41d9847 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:32.203317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96da1c40-d14c-43dc-8197-bc41b41d9847 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:32.353191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-96da1c40-d14c-43dc-8197-bc41b41d9847 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:32.361571 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-96da1c40-d14c-43dc-8197-bc41b41d9847 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1322 microversion: 2.1 time: 0.172648 Sep 27 10:41:32.362191 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3510/6550] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:32 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1322 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:33.387914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5b5dd990-25dc-48cf-95f5-042d089586f2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:33.388501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5b5dd990-25dc-48cf-95f5-042d089586f2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:33.395906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b5dd990-25dc-48cf-95f5-042d089586f2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:33.396409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b5dd990-25dc-48cf-95f5-042d089586f2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:33.396909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b5dd990-25dc-48cf-95f5-042d089586f2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:33.397627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b5dd990-25dc-48cf-95f5-042d089586f2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:33.398129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b5dd990-25dc-48cf-95f5-042d089586f2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:33.398567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5b5dd990-25dc-48cf-95f5-042d089586f2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:33.530500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5b5dd990-25dc-48cf-95f5-042d089586f2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:33.538457 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5b5dd990-25dc-48cf-95f5-042d089586f2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1322 microversion: 2.1 time: 0.156146 Sep 27 10:41:33.539040 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3041/6551] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:33 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1322 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:33.751039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-19251e09-9be5-44a2-9c6b-339fa39f3886 req-f37c952c-7224-4ff4-b899-03d54ad2710d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19bf47c4-400c-48ba-b3e7-91132241d9e5", "name": "network-vif-plugged", "status": "completed", "tag": "430781a5-9f1f-40bc-9e94-10d2d463e895"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:33.762050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-19251e09-9be5-44a2-9c6b-339fa39f3886 req-f37c952c-7224-4ff4-b899-03d54ad2710d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:33.762530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-19251e09-9be5-44a2-9c6b-339fa39f3886 req-f37c952c-7224-4ff4-b899-03d54ad2710d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:33.762961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-19251e09-9be5-44a2-9c6b-339fa39f3886 req-f37c952c-7224-4ff4-b899-03d54ad2710d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:33.774246 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-19251e09-9be5-44a2-9c6b-339fa39f3886 req-f37c952c-7224-4ff4-b899-03d54ad2710d service nova] Creating event network-vif-plugged:430781a5-9f1f-40bc-9e94-10d2d463e895 for instance 19bf47c4-400c-48ba-b3e7-91132241d9e5 on nch1-master-1 Sep 27 10:41:33.779516 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-19251e09-9be5-44a2-9c6b-339fa39f3886 req-f37c952c-7224-4ff4-b899-03d54ad2710d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032217 Sep 27 10:41:33.780057 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3511/6552] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:41:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:41:34.566348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3de0c0b6-d678-4105-bce7-fe9d452880c8 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:34.567433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3de0c0b6-d678-4105-bce7-fe9d452880c8 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:34.570650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:34.581309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3de0c0b6-d678-4105-bce7-fe9d452880c8 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:34.582148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3de0c0b6-d678-4105-bce7-fe9d452880c8 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:34.582914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3de0c0b6-d678-4105-bce7-fe9d452880c8 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:34.584095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3de0c0b6-d678-4105-bce7-fe9d452880c8 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:34.584950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3de0c0b6-d678-4105-bce7-fe9d452880c8 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:34.585692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3de0c0b6-d678-4105-bce7-fe9d452880c8 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:34.730989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3de0c0b6-d678-4105-bce7-fe9d452880c8 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:34.739046 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3de0c0b6-d678-4105-bce7-fe9d452880c8 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1322 microversion: 2.1 time: 0.179484 Sep 27 10:41:34.739651 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3042/6553] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:34 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1322 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:35.763407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c413fe85-0dbe-45c2-8627-8ef2b67f11cd tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:35.764281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c413fe85-0dbe-45c2-8627-8ef2b67f11cd tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:35.766509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:35.773082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c413fe85-0dbe-45c2-8627-8ef2b67f11cd tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:35.773571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c413fe85-0dbe-45c2-8627-8ef2b67f11cd tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:35.774012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c413fe85-0dbe-45c2-8627-8ef2b67f11cd tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:35.774721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c413fe85-0dbe-45c2-8627-8ef2b67f11cd tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:35.775207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c413fe85-0dbe-45c2-8627-8ef2b67f11cd tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:35.775638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c413fe85-0dbe-45c2-8627-8ef2b67f11cd tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:35.918585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c413fe85-0dbe-45c2-8627-8ef2b67f11cd tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:35.932610 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c413fe85-0dbe-45c2-8627-8ef2b67f11cd tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1472 microversion: 2.1 time: 0.173707 Sep 27 10:41:35.933747 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3512/6554] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:35 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1472 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:35.963482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:36.019416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ef486308-b23f-4505-bd8d-162628fc05fe tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Calling method '.version_select at 0x7f7f630bbc10>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:36.098299 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ef486308-b23f-4505-bd8d-162628fc05fe tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/1bc3c71db8db43ac92d01f72920edb9a" status: 200 len: 371 microversion: 2.1 time: 0.142348 Sep 27 10:41:36.098857 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3043/6555] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:41:35 2022] GET /compute/v2.1/os-quota-sets/1bc3c71db8db43ac92d01f72920edb9a => generated 371 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:41:36.111350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0ec6c37a-ba54-46ce-afa6-59befb88f21b tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 51201, "vcpus": 1, "disk": 5, "name": "tempest-ServersAdminNegativeTestJSON-flavor-468989538", "id": 645002659, "os-flavor-access:is_public": "True"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:36.132022 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0ec6c37a-ba54-46ce-afa6-59befb88f21b tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 434 microversion: 2.1 time: 0.024416 Sep 27 10:41:36.132530 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3513/6556] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:41:36 2022] POST /compute/v2.1/flavors => generated 434 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:41:36.145433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "645002659"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:36.147785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:36.154637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:36.155142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:36.155595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:36.156312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:36.156825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:36.157271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:36.252013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '4e8b680265904df18425b959c0bc85ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bc3c71db8db43ac92d01f72920edb9a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:36.262201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Old instance type m1.heat2, new instance type tempest-ServersAdminNegativeTestJSON-flavor-468989538 {{(pid=145969) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:41:36.269277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:36.269777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:36.270227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:36.270974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:36.271442 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:36.271913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:36.294945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Getting quotas for project dc6b14a8306e4e23a8f88b44ec0b65e9. Resources: {'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:41:36.300380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Getting quotas for user 1c0ef8d535824421863e905c2b3d6784 and project dc6b14a8306e4e23a8f88b44ec0b65e9. Resources: {'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:41:36.313311 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.compute.api [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] ram quota exceeded for 1bc3c71db8db43ac92d01f72920edb9a, tried to resize instance. Sep 27 10:41:36.313868 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] HTTP exception thrown: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Request method failure captured: Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5/action HTTP/1.1 Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 38 Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtNTewrA-6zoRpQFetIOiDwDdfqUIUGrvprSC6IamE-mo_cXnWbXeRvyLoG7Kcsp7gSuMp_atmkAL_D5H9j_E7E-JHS-xqL7PRY5rP-DW-a3w7ILqS9o8E8r2de2-SC8bUUGcCygs3AOsmvXyqVtyae5HTYX5a1oLdiQDu6W3pV96P4 Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 1bc3c71db8db43ac92d01f72920edb9a Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-932354818 Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,admin,member Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,admin,member Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1bc3c71db8db43ac92d01f72920edb9a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1bc3c71db8db43ac92d01f72920edb9a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1bc3c71db8db43ac92d01f72920edb9a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1bc3c71db8db43ac92d01f72920edb9a"}]}] Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersAdminNegativeTestJSON-932354818 Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 1bc3c71db8db43ac92d01f72920edb9a Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-932354818 Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersAdminNegativeTestJSON-932354818-project-admin Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 4e8b680265904df18425b959c0bc85ce Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersAdminNegativeTestJSON-932354818-project-admin Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: {"resize": {"flavorRef": "645002659"}} Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: exception: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '19bf47c4-400c-48ba-b3e7-91132241d9e5', 'body': {'resize': {'flavorRef': '645002659'}}} Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3878, in _check_quota_for_upsize Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, res_deltas, Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 613, in limit_check_project_and_user Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['ram'] Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1053, in _resize Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:41:36.315968 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4244, in resize Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._check_quota_for_upsize(context, instance, Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3896, in _check_quota_for_upsize Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1056, in _resize Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Returning 403 to user: Quota exceeded for ram: Requested 50177, but already used 1024 of 51200 ram {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f0a829a3-b0ab-4cee-8a26-6419b1497498 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5/action" status: 403 len: 118 microversion: 2.1 time: 0.176088 Sep 27 10:41:36.320517 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3044/6557] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:41:36 2022] POST /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5/action => generated 118 bytes in 177 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:41:36.326916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-36be39ae-871c-4df2-b5d3-2a679f1e5bf0 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:36.348258 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-36be39ae-871c-4df2-b5d3-2a679f1e5bf0 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/645002659" status: 202 len: 0 microversion: 2.1 time: 0.025358 Sep 27 10:41:36.348828 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3514/6558] 10.222.0.26 () {58 vars in 1158 bytes} [Tue Sep 27 10:41:36 2022] DELETE /compute/v2.1/flavors/645002659 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:41:36.357424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-19aabb14-f6df-4fee-bc66-dae9badbb408 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:36.377631 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-19aabb14-f6df-4fee-bc66-dae9badbb408 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.024289 Sep 27 10:41:36.378193 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3045/6559] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:41:36 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:41:36.399065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-501cd2fd-742c-46df-bb17-957027314f4f tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Calling method '.version_select at 0x7f7f62576700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:36.478458 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-501cd2fd-742c-46df-bb17-957027314f4f tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] 10.222.0.26 "GET /compute/v2.1/os-quota-sets/1bc3c71db8db43ac92d01f72920edb9a" status: 200 len: 371 microversion: 2.1 time: 0.083204 Sep 27 10:41:36.479039 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3515/6560] 10.222.0.26 () {58 vars in 1242 bytes} [Tue Sep 27 10:41:36 2022] GET /compute/v2.1/os-quota-sets/1bc3c71db8db43ac92d01f72920edb9a => generated 371 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:41:36.491333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-45ccd19d-7b64-4ef9-8373-60d9b6cff4e4 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 21, "disk": 5, "name": "tempest-ServersAdminNegativeTestJSON-flavor-1925990744", "id": 1307185935, "os-flavor-access:is_public": "True"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:36.514220 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-45ccd19d-7b64-4ef9-8373-60d9b6cff4e4 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 437 microversion: 2.1 time: 0.026576 Sep 27 10:41:36.514928 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3046/6561] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:41:36 2022] POST /compute/v2.1/flavors => generated 437 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:41:36.528011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "1307185935"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:36.530491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:36.536886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:36.537381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:36.537825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:36.538510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:36.539005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:36.539440 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:36.739182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': True, 'user_id': '4e8b680265904df18425b959c0bc85ce', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1bc3c71db8db43ac92d01f72920edb9a', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:36.745639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Old instance type m1.heat2, new instance type tempest-ServersAdminNegativeTestJSON-flavor-1925990744 {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:41:36.753377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:36.753902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:36.754389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:36.755244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:36.755736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:36.756242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:36.780413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Getting quotas for project dc6b14a8306e4e23a8f88b44ec0b65e9. Resources: {'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:41:36.785651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Getting quotas for user 1c0ef8d535824421863e905c2b3d6784 and project dc6b14a8306e4e23a8f88b44ec0b65e9. Resources: {'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:41:36.798454 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.compute.api [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] cores quota exceeded for 1bc3c71db8db43ac92d01f72920edb9a, tried to resize instance. Sep 27 10:41:36.799295 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] HTTP exception thrown: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Request method failure captured: Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5/action HTTP/1.1 Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 39 Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtNTewrA-6zoRpQFetIOiDwDdfqUIUGrvprSC6IamE-mo_cXnWbXeRvyLoG7Kcsp7gSuMp_atmkAL_D5H9j_E7E-JHS-xqL7PRY5rP-DW-a3w7ILqS9o8E8r2de2-SC8bUUGcCygs3AOsmvXyqVtyae5HTYX5a1oLdiQDu6W3pV96P4 Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 1bc3c71db8db43ac92d01f72920edb9a Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-932354818 Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,admin,member Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,admin,member Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1bc3c71db8db43ac92d01f72920edb9a"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1bc3c71db8db43ac92d01f72920edb9a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1bc3c71db8db43ac92d01f72920edb9a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1bc3c71db8db43ac92d01f72920edb9a"}]}] Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersAdminNegativeTestJSON-932354818 Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 1bc3c71db8db43ac92d01f72920edb9a Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-932354818 Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersAdminNegativeTestJSON-932354818-project-admin Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 4e8b680265904df18425b959c0bc85ce Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersAdminNegativeTestJSON-932354818-project-admin Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: {"resize": {"flavorRef": "1307185935"}} Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: exception: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '19bf47c4-400c-48ba-b3e7-91132241d9e5', 'body': {'resize': {'flavorRef': '1307185935'}}} Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3878, in _check_quota_for_upsize Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi objects.Quotas.check_deltas(context, res_deltas, Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 370, in check_deltas Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 365, in check_deltas Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi cls.limit_check_project_and_user(context, **check_kwargs) Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/quotas.py", line 298, in limit_check_project_and_user Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return quota.QUOTAS.limit_check_project_and_user(context, Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 1150, in limit_check_project_and_user Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return self._driver.limit_check_project_and_user( Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/quota.py", line 613, in limit_check_project_and_user Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.OverQuota(overs=sorted(overs), Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.OverQuota: Quota exceeded for resources: ['cores'] Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1053, in _resize Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:41:36.822017 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4244, in resize Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._check_quota_for_upsize(context, instance, Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 3896, in _check_quota_for_upsize Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.TooManyInstances(overs=overs, Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.TooManyInstances: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1056, in _resize Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPForbidden( Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPForbidden: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Returning 403 to user: Quota exceeded for cores: Requested 20, but already used 1 of 20 cores {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-54d00b82-5cf4-4ba2-ad15-b253713b9e4c tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] 10.222.0.26 "POST /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5/action" status: 403 len: 113 microversion: 2.1 time: 0.299697 Sep 27 10:41:36.826135 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3516/6562] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:41:36 2022] POST /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5/action => generated 113 bytes in 301 msecs (HTTP/1.1 403) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:41:36.832989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a6538b70-b6bc-4bed-819b-1eafb6cfa3a6 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:36.853427 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a6538b70-b6bc-4bed-819b-1eafb6cfa3a6 tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] 10.222.0.26 "DELETE /compute/v2.1/flavors/1307185935" status: 202 len: 0 microversion: 2.1 time: 0.024287 Sep 27 10:41:36.853973 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3047/6563] 10.222.0.26 () {58 vars in 1161 bytes} [Tue Sep 27 10:41:36 2022] DELETE /compute/v2.1/flavors/1307185935 => generated 0 bytes in 26 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:41:36.862335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2fc791ff-2ece-4a63-8289-f8cbb900578f tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:36.882592 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2fc791ff-2ece-4a63-8289-f8cbb900578f tempest-ServersAdminNegativeTestJSON-932354818 tempest-ServersAdminNegativeTestJSON-932354818-project-admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 4991 microversion: 2.1 time: 0.024114 Sep 27 10:41:36.883174 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3517/6564] 10.222.0.26 () {58 vars in 1146 bytes} [Tue Sep 27 10:41:36 2022] GET /compute/v2.1/flavors/detail => generated 4991 bytes in 25 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:36.904034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-20acaa62-03b7-45af-a5f2-23cd7fe698a6 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:36.904558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-20acaa62-03b7-45af-a5f2-23cd7fe698a6 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:36.911194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20acaa62-03b7-45af-a5f2-23cd7fe698a6 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:36.911735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20acaa62-03b7-45af-a5f2-23cd7fe698a6 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:36.912238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20acaa62-03b7-45af-a5f2-23cd7fe698a6 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:36.913066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20acaa62-03b7-45af-a5f2-23cd7fe698a6 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:36.913525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20acaa62-03b7-45af-a5f2-23cd7fe698a6 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:36.913968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20acaa62-03b7-45af-a5f2-23cd7fe698a6 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:36.941543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-20acaa62-03b7-45af-a5f2-23cd7fe698a6 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:41:37.024061 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-20acaa62-03b7-45af-a5f2-23cd7fe698a6 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "DELETE /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 204 len: 0 microversion: 2.1 time: 0.123684 Sep 27 10:41:37.024835 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3048/6565] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:41:36 2022] DELETE /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:41:37.033374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a316564c-7c48-4a65-83c7-6b336f02e8a2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:37.033992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a316564c-7c48-4a65-83c7-6b336f02e8a2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:37.042974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a316564c-7c48-4a65-83c7-6b336f02e8a2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:37.044114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a316564c-7c48-4a65-83c7-6b336f02e8a2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:37.044951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a316564c-7c48-4a65-83c7-6b336f02e8a2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:37.045722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a316564c-7c48-4a65-83c7-6b336f02e8a2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:37.046189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a316564c-7c48-4a65-83c7-6b336f02e8a2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:37.046623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a316564c-7c48-4a65-83c7-6b336f02e8a2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:37.186677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a316564c-7c48-4a65-83c7-6b336f02e8a2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:37.195046 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a316564c-7c48-4a65-83c7-6b336f02e8a2 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1478 microversion: 2.1 time: 0.165847 Sep 27 10:41:37.195622 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3518/6566] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:37 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1478 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:38.220684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-81763356-3bc9-43cf-9131-e0dbd054207f tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:38.221420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-81763356-3bc9-43cf-9131-e0dbd054207f tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:38.233581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81763356-3bc9-43cf-9131-e0dbd054207f tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:38.234095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81763356-3bc9-43cf-9131-e0dbd054207f tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:38.234557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81763356-3bc9-43cf-9131-e0dbd054207f tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:38.235317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81763356-3bc9-43cf-9131-e0dbd054207f tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:38.235794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81763356-3bc9-43cf-9131-e0dbd054207f tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:38.236244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81763356-3bc9-43cf-9131-e0dbd054207f tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:38.446058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-81763356-3bc9-43cf-9131-e0dbd054207f tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1c0ef8d535824421863e905c2b3d6784', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'dc6b14a8306e4e23a8f88b44ec0b65e9', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:38.455154 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-81763356-3bc9-43cf-9131-e0dbd054207f tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 200 len: 1478 microversion: 2.1 time: 0.238881 Sep 27 10:41:38.455707 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3049/6567] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:38 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 1478 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:38.827311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-bfe7803b-0e59-43aa-b546-db15e37bde55 req-d5774f6c-325c-4df9-8417-7a6632f0c418 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19bf47c4-400c-48ba-b3e7-91132241d9e5", "name": "network-vif-deleted", "tag": "430781a5-9f1f-40bc-9e94-10d2d463e895"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:38.837862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bfe7803b-0e59-43aa-b546-db15e37bde55 req-d5774f6c-325c-4df9-8417-7a6632f0c418 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:38.838348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bfe7803b-0e59-43aa-b546-db15e37bde55 req-d5774f6c-325c-4df9-8417-7a6632f0c418 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:38.838780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bfe7803b-0e59-43aa-b546-db15e37bde55 req-d5774f6c-325c-4df9-8417-7a6632f0c418 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:38.856869 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-bfe7803b-0e59-43aa-b546-db15e37bde55 req-d5774f6c-325c-4df9-8417-7a6632f0c418 service nova] Creating event network-vif-deleted:430781a5-9f1f-40bc-9e94-10d2d463e895 for instance 19bf47c4-400c-48ba-b3e7-91132241d9e5 on nch1-master-1 Sep 27 10:41:38.861921 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-bfe7803b-0e59-43aa-b546-db15e37bde55 req-d5774f6c-325c-4df9-8417-7a6632f0c418 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039289 Sep 27 10:41:38.862579 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3519/6568] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:41:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:41:39.482838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:39.484095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] [instance: 19bf47c4-400c-48ba-b3e7-91132241d9e5] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:39.496972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:39.497815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:39.498634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:39.499556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:39.500034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:39.500477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:39.515092 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] HTTP exception thrown: Instance 19bf47c4-400c-48ba-b3e7-91132241d9e5 could not be found. Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Request method failure captured: Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 HTTP/1.1 Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtNSK_18__Dn-KECDWCtU407R7-Tczvn_8CXCVwsURszhVh96VGWhdDkx580jYjwKcIWptd0vPvWpJrAjpHRIDowa3yXMogXRjX23CUwZUAUuxkeJD7JgyXazigQVnlWVNhxn6HVKX7NkVSHGKPY797-xTRejvtyi0bJVp_UpHDP_-M Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: dc6b14a8306e4e23a8f88b44ec0b65e9 Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersAdminNegativeTestJSON-831395528 Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dc6b14a8306e4e23a8f88b44ec0b65e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/dc6b14a8306e4e23a8f88b44ec0b65e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/dc6b14a8306e4e23a8f88b44ec0b65e9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_dc6b14a8306e4e23a8f88b44ec0b65e9"}]}] Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersAdminNegativeTestJSON-831395528 Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: dc6b14a8306e4e23a8f88b44ec0b65e9 Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersAdminNegativeTestJSON-831395528 Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersAdminNegativeTestJSON-831395528-project Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 1c0ef8d535824421863e905c2b3d6784 Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersAdminNegativeTestJSON-831395528-project Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 19bf47c4-400c-48ba-b3e7-91132241d9e5 could not be found. Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '19bf47c4-400c-48ba-b3e7-91132241d9e5'} Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 19bf47c4-400c-48ba-b3e7-91132241d9e5 could not be found. Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 19bf47c4-400c-48ba-b3e7-91132241d9e5 could not be found. Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:39.516827 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:41:39.524514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:39.524514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:41:39.524514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:41:39.524514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:41:39.524514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:41:39.524514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:41:39.524514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 19bf47c4-400c-48ba-b3e7-91132241d9e5 could not be found. Sep 27 10:41:39.524514 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:41:39.524514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] Returning 404 to user: Instance 19bf47c4-400c-48ba-b3e7-91132241d9e5 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:41:39.524514 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3fbd4aee-03bb-4295-ad34-55fa561fb425 tempest-ServersAdminNegativeTestJSON-831395528 tempest-ServersAdminNegativeTestJSON-831395528-project] 10.222.0.26 "GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5" status: 404 len: 111 microversion: 2.1 time: 0.042285 Sep 27 10:41:39.524514 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3050/6569] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:39 2022] GET /compute/v2.1/servers/19bf47c4-400c-48ba-b3e7-91132241d9e5 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:41:46.023154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1139073284", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "88138dcf-dee2-4736-b0cf-25ef529c74fc"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:46.212266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] validate_networks() for [('88138dcf-dee2-4736-b0cf-25ef529c74fc', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:41:46.470636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:46.652920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:41:46.653797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:41:46.654582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:41:46.655004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:41:46.655630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:41:46.665709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:46.666217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:46.666667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:46.667413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:46.667875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:46.668307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:46.687897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Getting quotas for project 8ef200fc20ce4b0da3c92b5179a40e63. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:41:46.693602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Getting quotas for user 8e0f9f1c851743c7b466be4e42d4a11b and project 8ef200fc20ce4b0da3c92b5179a40e63. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:41:46.708317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:41:46.710794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:41:46.757814 ncdv-master-1 devstack@n-api.service[145968]: ERROR oslo.messaging._drivers.impl_rabbit [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [9cc59313-1d73-4114-86d7-0092980c6242] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 27 10:41:47.768516 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [9cc59313-1d73-4114-86d7-0092980c6242] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 54632. Sep 27 10:41:47.781492 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-44fa0e20-a05b-412b-a19b-46a390d9fa7f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.887068 Sep 27 10:41:47.782461 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3520/6570] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:41:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 1889 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:41:47.794045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b0775010-55d2-4208-b78d-2e79414f6b4d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:47.794672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b0775010-55d2-4208-b78d-2e79414f6b4d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:47.797717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:47.798435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:47.928101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b0775010-55d2-4208-b78d-2e79414f6b4d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:47.937870 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b0775010-55d2-4208-b78d-2e79414f6b4d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1215 microversion: 2.1 time: 0.147623 Sep 27 10:41:47.938606 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3051/6571] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:47 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1215 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:48.962005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1a6ccc44-3006-43e4-bbe6-eba39abe3e2b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:48.962630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1a6ccc44-3006-43e4-bbe6-eba39abe3e2b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:48.969505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a6ccc44-3006-43e4-bbe6-eba39abe3e2b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:48.970435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a6ccc44-3006-43e4-bbe6-eba39abe3e2b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:48.971042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a6ccc44-3006-43e4-bbe6-eba39abe3e2b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:48.972084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a6ccc44-3006-43e4-bbe6-eba39abe3e2b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:48.972515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a6ccc44-3006-43e4-bbe6-eba39abe3e2b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:48.972985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a6ccc44-3006-43e4-bbe6-eba39abe3e2b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:49.058790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1a6ccc44-3006-43e4-bbe6-eba39abe3e2b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:49.067188 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1a6ccc44-3006-43e4-bbe6-eba39abe3e2b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1285 microversion: 2.1 time: 0.110951 Sep 27 10:41:49.069389 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3521/6572] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:48 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1285 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:50.093924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-095dc8a8-23a7-46f6-b722-13e98ab3e2ba tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:50.096169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-095dc8a8-23a7-46f6-b722-13e98ab3e2ba tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:50.098391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=63,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:50.103588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-095dc8a8-23a7-46f6-b722-13e98ab3e2ba tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:50.104096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-095dc8a8-23a7-46f6-b722-13e98ab3e2ba tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:50.104555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-095dc8a8-23a7-46f6-b722-13e98ab3e2ba tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:50.105320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-095dc8a8-23a7-46f6-b722-13e98ab3e2ba tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:50.105800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-095dc8a8-23a7-46f6-b722-13e98ab3e2ba tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:50.106247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-095dc8a8-23a7-46f6-b722-13e98ab3e2ba tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:50.338932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-095dc8a8-23a7-46f6-b722-13e98ab3e2ba tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:50.347076 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-095dc8a8-23a7-46f6-b722-13e98ab3e2ba tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1315 microversion: 2.1 time: 0.258013 Sep 27 10:41:50.347694 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3052/6573] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:50 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1315 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:50.485500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-19c6cc9d-38d6-49e0-b72b-576ee9464b90 req-12ff0d50-0698-4435-9ca3-0735a47201e0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3af889fc-12e1-40c4-a5a4-589a49c1534c", "tag": "202e9028-fb3b-4379-80ba-fe8d0c00b6bf"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:50.512788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-19c6cc9d-38d6-49e0-b72b-576ee9464b90 req-12ff0d50-0698-4435-9ca3-0735a47201e0 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:50.513298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-19c6cc9d-38d6-49e0-b72b-576ee9464b90 req-12ff0d50-0698-4435-9ca3-0735a47201e0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:50.516851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-19c6cc9d-38d6-49e0-b72b-576ee9464b90 req-12ff0d50-0698-4435-9ca3-0735a47201e0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:50.534089 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-19c6cc9d-38d6-49e0-b72b-576ee9464b90 req-12ff0d50-0698-4435-9ca3-0735a47201e0 service nova] Creating event network-changed:202e9028-fb3b-4379-80ba-fe8d0c00b6bf for instance 3af889fc-12e1-40c4-a5a4-589a49c1534c on nch2-master-1 Sep 27 10:41:50.540727 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-19c6cc9d-38d6-49e0-b72b-576ee9464b90 req-12ff0d50-0698-4435-9ca3-0735a47201e0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061198 Sep 27 10:41:50.541903 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3522/6574] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:41:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:41:51.375344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dbeaa089-1362-4ecc-81b2-afd28d5df1a3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:51.376359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dbeaa089-1362-4ecc-81b2-afd28d5df1a3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:51.388363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbeaa089-1362-4ecc-81b2-afd28d5df1a3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:51.389360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbeaa089-1362-4ecc-81b2-afd28d5df1a3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:51.389832 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbeaa089-1362-4ecc-81b2-afd28d5df1a3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:51.390561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbeaa089-1362-4ecc-81b2-afd28d5df1a3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:51.391043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbeaa089-1362-4ecc-81b2-afd28d5df1a3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:51.391496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dbeaa089-1362-4ecc-81b2-afd28d5df1a3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:51.531070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dbeaa089-1362-4ecc-81b2-afd28d5df1a3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:51.539130 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dbeaa089-1362-4ecc-81b2-afd28d5df1a3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1315 microversion: 2.1 time: 0.170874 Sep 27 10:41:51.539712 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3053/6575] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:51 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1315 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:52.570018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5798af57-65a7-460f-9b73-6774184ed44f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:52.571148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5798af57-65a7-460f-9b73-6774184ed44f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:52.579863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5798af57-65a7-460f-9b73-6774184ed44f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:52.580349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5798af57-65a7-460f-9b73-6774184ed44f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:52.580814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5798af57-65a7-460f-9b73-6774184ed44f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:52.581521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5798af57-65a7-460f-9b73-6774184ed44f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:52.581976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5798af57-65a7-460f-9b73-6774184ed44f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:52.582403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5798af57-65a7-460f-9b73-6774184ed44f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:52.715615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5798af57-65a7-460f-9b73-6774184ed44f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:52.724120 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5798af57-65a7-460f-9b73-6774184ed44f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1315 microversion: 2.1 time: 0.163266 Sep 27 10:41:52.724695 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3523/6576] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:52 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1315 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:53.750598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ddf2a65f-c060-4b59-b85a-19f0106544fd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:53.751192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ddf2a65f-c060-4b59-b85a-19f0106544fd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:53.753681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=4,nova_api:DELETE=3,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:53.761654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddf2a65f-c060-4b59-b85a-19f0106544fd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:53.762414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddf2a65f-c060-4b59-b85a-19f0106544fd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:53.763011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddf2a65f-c060-4b59-b85a-19f0106544fd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:53.763958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddf2a65f-c060-4b59-b85a-19f0106544fd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:53.764480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddf2a65f-c060-4b59-b85a-19f0106544fd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:53.765126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ddf2a65f-c060-4b59-b85a-19f0106544fd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:53.913184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ddf2a65f-c060-4b59-b85a-19f0106544fd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:53.921007 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ddf2a65f-c060-4b59-b85a-19f0106544fd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1315 microversion: 2.1 time: 0.176835 Sep 27 10:41:53.921577 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3054/6577] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:53 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1315 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:54.946863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aea5cb0a-3b47-4fa8-acbb-8a20e2e3478f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:54.947474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aea5cb0a-3b47-4fa8-acbb-8a20e2e3478f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:54.951276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=65,nova_api:INSERT=4,nova_api:DELETE=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:54.957605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea5cb0a-3b47-4fa8-acbb-8a20e2e3478f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:54.958158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea5cb0a-3b47-4fa8-acbb-8a20e2e3478f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:54.958597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea5cb0a-3b47-4fa8-acbb-8a20e2e3478f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:54.959319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea5cb0a-3b47-4fa8-acbb-8a20e2e3478f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:54.959787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea5cb0a-3b47-4fa8-acbb-8a20e2e3478f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:54.960218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aea5cb0a-3b47-4fa8-acbb-8a20e2e3478f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:54.963769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=70 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:55.107360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aea5cb0a-3b47-4fa8-acbb-8a20e2e3478f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:55.115961 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aea5cb0a-3b47-4fa8-acbb-8a20e2e3478f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1315 microversion: 2.1 time: 0.173945 Sep 27 10:41:55.116552 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3524/6578] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:54 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1315 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:56.139945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f63d3bd9-4e57-4180-bef0-e47127821591 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:56.140705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f63d3bd9-4e57-4180-bef0-e47127821591 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:56.148499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63d3bd9-4e57-4180-bef0-e47127821591 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:56.149043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63d3bd9-4e57-4180-bef0-e47127821591 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:56.149506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63d3bd9-4e57-4180-bef0-e47127821591 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:56.150256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63d3bd9-4e57-4180-bef0-e47127821591 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:56.150735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63d3bd9-4e57-4180-bef0-e47127821591 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:56.151185 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f63d3bd9-4e57-4180-bef0-e47127821591 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:56.294402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f63d3bd9-4e57-4180-bef0-e47127821591 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:56.302702 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f63d3bd9-4e57-4180-bef0-e47127821591 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1315 microversion: 2.1 time: 0.166993 Sep 27 10:41:56.303562 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3055/6579] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:56 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1315 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:57.330314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed7bf40a-b3fb-4354-9d7e-cc9cd8fb2b76 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:57.330986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed7bf40a-b3fb-4354-9d7e-cc9cd8fb2b76 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:57.334038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=5 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:57.334993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:57.341197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed7bf40a-b3fb-4354-9d7e-cc9cd8fb2b76 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:57.341584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed7bf40a-b3fb-4354-9d7e-cc9cd8fb2b76 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:57.342073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed7bf40a-b3fb-4354-9d7e-cc9cd8fb2b76 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:57.342865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed7bf40a-b3fb-4354-9d7e-cc9cd8fb2b76 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:57.343381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed7bf40a-b3fb-4354-9d7e-cc9cd8fb2b76 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:57.343858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed7bf40a-b3fb-4354-9d7e-cc9cd8fb2b76 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:57.497103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ed7bf40a-b3fb-4354-9d7e-cc9cd8fb2b76 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:57.505292 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed7bf40a-b3fb-4354-9d7e-cc9cd8fb2b76 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1315 microversion: 2.1 time: 0.180539 Sep 27 10:41:57.506100 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3525/6580] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:57 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1315 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:57.650628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a863cc31-3191-478e-ba53-d4f7223d8c6c req-be0a9b5c-751c-42fe-b8f1-09b85fcb0b69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3af889fc-12e1-40c4-a5a4-589a49c1534c", "name": "network-vif-plugged", "status": "completed", "tag": "202e9028-fb3b-4379-80ba-fe8d0c00b6bf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:57.661878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a863cc31-3191-478e-ba53-d4f7223d8c6c req-be0a9b5c-751c-42fe-b8f1-09b85fcb0b69 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:57.662469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a863cc31-3191-478e-ba53-d4f7223d8c6c req-be0a9b5c-751c-42fe-b8f1-09b85fcb0b69 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:57.662819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a863cc31-3191-478e-ba53-d4f7223d8c6c req-be0a9b5c-751c-42fe-b8f1-09b85fcb0b69 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:57.676620 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a863cc31-3191-478e-ba53-d4f7223d8c6c req-be0a9b5c-751c-42fe-b8f1-09b85fcb0b69 service nova] Creating event network-vif-plugged:202e9028-fb3b-4379-80ba-fe8d0c00b6bf for instance 3af889fc-12e1-40c4-a5a4-589a49c1534c on nch2-master-1 Sep 27 10:41:57.682395 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a863cc31-3191-478e-ba53-d4f7223d8c6c req-be0a9b5c-751c-42fe-b8f1-09b85fcb0b69 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035813 Sep 27 10:41:57.683019 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3056/6581] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:41:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:41:58.527755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0857a5f5-9335-4867-bf23-83492b614436 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:58.528363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0857a5f5-9335-4867-bf23-83492b614436 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:58.535844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0857a5f5-9335-4867-bf23-83492b614436 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:58.536342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0857a5f5-9335-4867-bf23-83492b614436 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:58.536826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0857a5f5-9335-4867-bf23-83492b614436 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:58.537539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0857a5f5-9335-4867-bf23-83492b614436 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:58.538000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0857a5f5-9335-4867-bf23-83492b614436 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:58.538432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0857a5f5-9335-4867-bf23-83492b614436 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:58.673556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0857a5f5-9335-4867-bf23-83492b614436 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:58.682275 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0857a5f5-9335-4867-bf23-83492b614436 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1315 microversion: 2.1 time: 0.158488 Sep 27 10:41:58.682859 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3526/6582] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:58 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1315 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:59.709572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4a02a141-0d21-4767-8d7d-d7e7b4ce39fb tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:41:59.710487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4a02a141-0d21-4767-8d7d-d7e7b4ce39fb tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:59.713385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:41:59.721338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a02a141-0d21-4767-8d7d-d7e7b4ce39fb tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:59.721836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a02a141-0d21-4767-8d7d-d7e7b4ce39fb tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:59.722288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a02a141-0d21-4767-8d7d-d7e7b4ce39fb tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:59.722997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a02a141-0d21-4767-8d7d-d7e7b4ce39fb tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:59.723448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a02a141-0d21-4767-8d7d-d7e7b4ce39fb tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:59.723883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a02a141-0d21-4767-8d7d-d7e7b4ce39fb tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:59.865891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4a02a141-0d21-4767-8d7d-d7e7b4ce39fb tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:59.876810 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4a02a141-0d21-4767-8d7d-d7e7b4ce39fb tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1503 microversion: 2.1 time: 0.174067 Sep 27 10:41:59.877430 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3057/6583] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:41:59 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1503 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:41:59.899492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:41:59.901881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:41:59.909584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:59.910082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:59.910533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:59.911235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:41:59.911698 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:41:59.912129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:41:59.943849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:41:59.949730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Old instance type m1.heat2, new instance type m1.heat {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:41:59.956822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:41:59.957253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:41:59.957740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:41:59.958195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:41:59.958622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:42:00.024453 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b274c30b-7c33-4019-a6da-ea7460006ab6 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "POST /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c/action" status: 202 len: 0 microversion: 2.1 time: 0.128973 Sep 27 10:42:00.025102 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3527/6584] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:41:59 2022] POST /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c/action => generated 0 bytes in 130 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:42:00.033558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd382b06-a229-4b29-aadf-722e50581ba5 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:00.034352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cd382b06-a229-4b29-aadf-722e50581ba5 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:00.040806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd382b06-a229-4b29-aadf-722e50581ba5 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:00.041305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd382b06-a229-4b29-aadf-722e50581ba5 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:00.041763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd382b06-a229-4b29-aadf-722e50581ba5 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:00.042537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd382b06-a229-4b29-aadf-722e50581ba5 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:00.043018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd382b06-a229-4b29-aadf-722e50581ba5 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:00.043464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd382b06-a229-4b29-aadf-722e50581ba5 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:00.192017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cd382b06-a229-4b29-aadf-722e50581ba5 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:00.201132 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd382b06-a229-4b29-aadf-722e50581ba5 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1512 microversion: 2.1 time: 0.171592 Sep 27 10:42:00.201707 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3058/6585] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:00 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1512 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:42:01.225978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d40e48f1-95e3-40df-b0a6-cc6412bc454e tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:01.226594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d40e48f1-95e3-40df-b0a6-cc6412bc454e tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:01.233935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d40e48f1-95e3-40df-b0a6-cc6412bc454e tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:01.234424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d40e48f1-95e3-40df-b0a6-cc6412bc454e tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:01.234863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d40e48f1-95e3-40df-b0a6-cc6412bc454e tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:01.235571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d40e48f1-95e3-40df-b0a6-cc6412bc454e tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:01.236030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d40e48f1-95e3-40df-b0a6-cc6412bc454e tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:01.236456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d40e48f1-95e3-40df-b0a6-cc6412bc454e tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:01.377550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d40e48f1-95e3-40df-b0a6-cc6412bc454e tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:01.385745 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d40e48f1-95e3-40df-b0a6-cc6412bc454e tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1512 microversion: 2.1 time: 0.165177 Sep 27 10:42:01.386327 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3528/6586] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:01 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1512 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:02.413231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ae5c9e4d-df95-45bb-9065-0e6b55d3c6d3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:02.414130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ae5c9e4d-df95-45bb-9065-0e6b55d3c6d3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:02.425729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae5c9e4d-df95-45bb-9065-0e6b55d3c6d3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:02.426247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae5c9e4d-df95-45bb-9065-0e6b55d3c6d3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:02.426703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae5c9e4d-df95-45bb-9065-0e6b55d3c6d3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:02.427438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae5c9e4d-df95-45bb-9065-0e6b55d3c6d3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:02.428250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae5c9e4d-df95-45bb-9065-0e6b55d3c6d3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:02.428686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae5c9e4d-df95-45bb-9065-0e6b55d3c6d3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:02.573375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ae5c9e4d-df95-45bb-9065-0e6b55d3c6d3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:02.581508 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ae5c9e4d-df95-45bb-9065-0e6b55d3c6d3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.175234 Sep 27 10:42:02.582122 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3059/6587] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:02 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:03.605562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-96c39f93-db27-4c54-a5d0-57f6dfe7fb2f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:03.606222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-96c39f93-db27-4c54-a5d0-57f6dfe7fb2f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:03.613883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c39f93-db27-4c54-a5d0-57f6dfe7fb2f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:03.614362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c39f93-db27-4c54-a5d0-57f6dfe7fb2f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:03.614805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c39f93-db27-4c54-a5d0-57f6dfe7fb2f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:03.615511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c39f93-db27-4c54-a5d0-57f6dfe7fb2f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:03.615970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c39f93-db27-4c54-a5d0-57f6dfe7fb2f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:03.616424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96c39f93-db27-4c54-a5d0-57f6dfe7fb2f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:03.751664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-96c39f93-db27-4c54-a5d0-57f6dfe7fb2f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:03.760474 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-96c39f93-db27-4c54-a5d0-57f6dfe7fb2f tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.159001 Sep 27 10:42:03.761241 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3529/6588] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:03 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:04.786514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dae46d09-14f4-424f-815c-8f70a51bf6dd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:04.787107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dae46d09-14f4-424f-815c-8f70a51bf6dd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:04.794425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dae46d09-14f4-424f-815c-8f70a51bf6dd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:04.794926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dae46d09-14f4-424f-815c-8f70a51bf6dd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:04.795374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dae46d09-14f4-424f-815c-8f70a51bf6dd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:04.796091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dae46d09-14f4-424f-815c-8f70a51bf6dd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:04.796560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dae46d09-14f4-424f-815c-8f70a51bf6dd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:04.797037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dae46d09-14f4-424f-815c-8f70a51bf6dd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:04.941101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dae46d09-14f4-424f-815c-8f70a51bf6dd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:04.949278 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dae46d09-14f4-424f-815c-8f70a51bf6dd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.167006 Sep 27 10:42:04.949832 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3060/6589] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:04 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:05.974048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a77f096b-41c8-457b-bff9-b35220a6954c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:05.974637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a77f096b-41c8-457b-bff9-b35220a6954c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:05.982133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a77f096b-41c8-457b-bff9-b35220a6954c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:05.982627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a77f096b-41c8-457b-bff9-b35220a6954c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:05.983073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a77f096b-41c8-457b-bff9-b35220a6954c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:05.983789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a77f096b-41c8-457b-bff9-b35220a6954c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:05.984249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a77f096b-41c8-457b-bff9-b35220a6954c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:05.984693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a77f096b-41c8-457b-bff9-b35220a6954c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:06.141622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a77f096b-41c8-457b-bff9-b35220a6954c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:06.149945 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a77f096b-41c8-457b-bff9-b35220a6954c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.180184 Sep 27 10:42:06.150530 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3530/6590] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:05 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:07.178147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-855e76ae-1648-49cc-8165-77f7eeaa5348 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:07.179222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-855e76ae-1648-49cc-8165-77f7eeaa5348 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:07.190495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-855e76ae-1648-49cc-8165-77f7eeaa5348 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:07.190994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-855e76ae-1648-49cc-8165-77f7eeaa5348 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:07.191447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-855e76ae-1648-49cc-8165-77f7eeaa5348 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:07.192161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-855e76ae-1648-49cc-8165-77f7eeaa5348 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:07.192631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-855e76ae-1648-49cc-8165-77f7eeaa5348 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:07.193105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-855e76ae-1648-49cc-8165-77f7eeaa5348 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:07.325811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-855e76ae-1648-49cc-8165-77f7eeaa5348 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:07.334050 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-855e76ae-1648-49cc-8165-77f7eeaa5348 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.162862 Sep 27 10:42:07.334624 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3061/6591] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:07 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:08.361847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a1696b50-a609-41a3-8cc5-b850be03f418 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:08.362923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a1696b50-a609-41a3-8cc5-b850be03f418 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:08.370847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1696b50-a609-41a3-8cc5-b850be03f418 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:08.371389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1696b50-a609-41a3-8cc5-b850be03f418 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:08.371882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1696b50-a609-41a3-8cc5-b850be03f418 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:08.372647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1696b50-a609-41a3-8cc5-b850be03f418 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:08.373188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1696b50-a609-41a3-8cc5-b850be03f418 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:08.373658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1696b50-a609-41a3-8cc5-b850be03f418 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:08.516998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a1696b50-a609-41a3-8cc5-b850be03f418 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:08.525203 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a1696b50-a609-41a3-8cc5-b850be03f418 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.170223 Sep 27 10:42:08.525796 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3531/6592] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:08 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:09.554282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6bda5348-9273-40fa-863c-79ec8685906c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:09.555389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6bda5348-9273-40fa-863c-79ec8685906c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:09.566833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bda5348-9273-40fa-863c-79ec8685906c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:09.567320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bda5348-9273-40fa-863c-79ec8685906c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:09.567793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bda5348-9273-40fa-863c-79ec8685906c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:09.568510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bda5348-9273-40fa-863c-79ec8685906c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:09.569013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bda5348-9273-40fa-863c-79ec8685906c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:09.569451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bda5348-9273-40fa-863c-79ec8685906c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:09.712434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6bda5348-9273-40fa-863c-79ec8685906c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:09.720589 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6bda5348-9273-40fa-863c-79ec8685906c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.174094 Sep 27 10:42:09.721632 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3062/6593] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:09 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:10.749185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-93568f92-9845-470a-a3d5-aa5137b7ee93 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:10.750041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-93568f92-9845-470a-a3d5-aa5137b7ee93 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:10.762920 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93568f92-9845-470a-a3d5-aa5137b7ee93 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:10.763676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93568f92-9845-470a-a3d5-aa5137b7ee93 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:10.764396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93568f92-9845-470a-a3d5-aa5137b7ee93 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:10.765624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93568f92-9845-470a-a3d5-aa5137b7ee93 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:10.766374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93568f92-9845-470a-a3d5-aa5137b7ee93 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:10.767087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-93568f92-9845-470a-a3d5-aa5137b7ee93 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:10.909224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-93568f92-9845-470a-a3d5-aa5137b7ee93 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:10.917701 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-93568f92-9845-470a-a3d5-aa5137b7ee93 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.175815 Sep 27 10:42:10.918316 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3532/6594] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:10 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:11.954288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e5c33ee2-f5fd-4f3d-9dd4-399c4f5d77d7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:11.955183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e5c33ee2-f5fd-4f3d-9dd4-399c4f5d77d7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:11.963940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c33ee2-f5fd-4f3d-9dd4-399c4f5d77d7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:11.964471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c33ee2-f5fd-4f3d-9dd4-399c4f5d77d7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:11.964958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c33ee2-f5fd-4f3d-9dd4-399c4f5d77d7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:11.965677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c33ee2-f5fd-4f3d-9dd4-399c4f5d77d7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:11.966144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c33ee2-f5fd-4f3d-9dd4-399c4f5d77d7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:11.966580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5c33ee2-f5fd-4f3d-9dd4-399c4f5d77d7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:12.108111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e5c33ee2-f5fd-4f3d-9dd4-399c4f5d77d7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:12.116540 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e5c33ee2-f5fd-4f3d-9dd4-399c4f5d77d7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.169146 Sep 27 10:42:12.117150 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3063/6595] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:11 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:13.140283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b908a1ff-7ea9-4cb2-b551-9b1021241800 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:13.140986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b908a1ff-7ea9-4cb2-b551-9b1021241800 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:13.148773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b908a1ff-7ea9-4cb2-b551-9b1021241800 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:13.149290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b908a1ff-7ea9-4cb2-b551-9b1021241800 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:13.149737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b908a1ff-7ea9-4cb2-b551-9b1021241800 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:13.150456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b908a1ff-7ea9-4cb2-b551-9b1021241800 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:13.150922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b908a1ff-7ea9-4cb2-b551-9b1021241800 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:13.151357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b908a1ff-7ea9-4cb2-b551-9b1021241800 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:13.298172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b908a1ff-7ea9-4cb2-b551-9b1021241800 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:13.307596 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b908a1ff-7ea9-4cb2-b551-9b1021241800 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.171284 Sep 27 10:42:13.308248 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3533/6596] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:13 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:14.334687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dc737b03-8527-40e7-adfe-9e674348bec2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:14.335269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dc737b03-8527-40e7-adfe-9e674348bec2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:14.342473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc737b03-8527-40e7-adfe-9e674348bec2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:14.342974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc737b03-8527-40e7-adfe-9e674348bec2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:14.343390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc737b03-8527-40e7-adfe-9e674348bec2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:14.344134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc737b03-8527-40e7-adfe-9e674348bec2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:14.344600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc737b03-8527-40e7-adfe-9e674348bec2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:14.345075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dc737b03-8527-40e7-adfe-9e674348bec2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:14.491734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dc737b03-8527-40e7-adfe-9e674348bec2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:14.503743 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dc737b03-8527-40e7-adfe-9e674348bec2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.174193 Sep 27 10:42:14.504419 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3064/6597] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:14 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:15.531048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d296a9af-1054-47e3-81a0-a628f3c20661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:15.532012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d296a9af-1054-47e3-81a0-a628f3c20661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:15.542619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d296a9af-1054-47e3-81a0-a628f3c20661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:15.543262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d296a9af-1054-47e3-81a0-a628f3c20661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:15.543919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d296a9af-1054-47e3-81a0-a628f3c20661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:15.545063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d296a9af-1054-47e3-81a0-a628f3c20661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:15.545712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d296a9af-1054-47e3-81a0-a628f3c20661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:15.546358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d296a9af-1054-47e3-81a0-a628f3c20661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:15.695569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d296a9af-1054-47e3-81a0-a628f3c20661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:15.708093 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d296a9af-1054-47e3-81a0-a628f3c20661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.183477 Sep 27 10:42:15.708775 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3534/6598] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:15 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:16.734152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-70a4928f-9cae-4166-bbf7-1245f0ffcffd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:16.734904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-70a4928f-9cae-4166-bbf7-1245f0ffcffd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:16.744607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4928f-9cae-4166-bbf7-1245f0ffcffd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:16.745332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4928f-9cae-4166-bbf7-1245f0ffcffd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:16.745932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4928f-9cae-4166-bbf7-1245f0ffcffd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:16.746884 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4928f-9cae-4166-bbf7-1245f0ffcffd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:16.747516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4928f-9cae-4166-bbf7-1245f0ffcffd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:16.748109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-70a4928f-9cae-4166-bbf7-1245f0ffcffd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:16.893023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-70a4928f-9cae-4166-bbf7-1245f0ffcffd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:16.904267 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-70a4928f-9cae-4166-bbf7-1245f0ffcffd tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.175282 Sep 27 10:42:16.905008 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3065/6599] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:16 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:17.928967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bbc96d48-1855-42ac-a9bc-2e8a24b0fb42 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:17.929566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bbc96d48-1855-42ac-a9bc-2e8a24b0fb42 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:17.937077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bbc96d48-1855-42ac-a9bc-2e8a24b0fb42 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:17.937575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bbc96d48-1855-42ac-a9bc-2e8a24b0fb42 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:17.938018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bbc96d48-1855-42ac-a9bc-2e8a24b0fb42 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:17.938731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bbc96d48-1855-42ac-a9bc-2e8a24b0fb42 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:17.939197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bbc96d48-1855-42ac-a9bc-2e8a24b0fb42 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:17.939630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bbc96d48-1855-42ac-a9bc-2e8a24b0fb42 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:18.078772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bbc96d48-1855-42ac-a9bc-2e8a24b0fb42 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:18.087120 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bbc96d48-1855-42ac-a9bc-2e8a24b0fb42 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.162689 Sep 27 10:42:18.087714 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3535/6600] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:17 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:19.116149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c8524860-8a47-4599-bf92-ee182095c66c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:19.117156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c8524860-8a47-4599-bf92-ee182095c66c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:19.128882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8524860-8a47-4599-bf92-ee182095c66c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:19.129769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8524860-8a47-4599-bf92-ee182095c66c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:19.130620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8524860-8a47-4599-bf92-ee182095c66c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:19.131138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8524860-8a47-4599-bf92-ee182095c66c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:19.131608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8524860-8a47-4599-bf92-ee182095c66c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:19.132048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8524860-8a47-4599-bf92-ee182095c66c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:19.287445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c8524860-8a47-4599-bf92-ee182095c66c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:19.295663 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c8524860-8a47-4599-bf92-ee182095c66c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.186751 Sep 27 10:42:19.296239 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3066/6601] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:19 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:20.324632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9cfafd1c-9071-4f96-98a3-66f7e6992fb2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:20.325723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9cfafd1c-9071-4f96-98a3-66f7e6992fb2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:20.336184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cfafd1c-9071-4f96-98a3-66f7e6992fb2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:20.336697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cfafd1c-9071-4f96-98a3-66f7e6992fb2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:20.337167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cfafd1c-9071-4f96-98a3-66f7e6992fb2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:20.337849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cfafd1c-9071-4f96-98a3-66f7e6992fb2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:20.338304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cfafd1c-9071-4f96-98a3-66f7e6992fb2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:20.338742 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9cfafd1c-9071-4f96-98a3-66f7e6992fb2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:20.485375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9cfafd1c-9071-4f96-98a3-66f7e6992fb2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:20.494577 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9cfafd1c-9071-4f96-98a3-66f7e6992fb2 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.176863 Sep 27 10:42:20.495171 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3536/6602] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:20 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:21.529354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-189b0522-eb09-46d7-9033-e5de0b7ddce7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:21.530211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-189b0522-eb09-46d7-9033-e5de0b7ddce7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:21.540292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-189b0522-eb09-46d7-9033-e5de0b7ddce7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:21.541054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-189b0522-eb09-46d7-9033-e5de0b7ddce7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:21.541694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-189b0522-eb09-46d7-9033-e5de0b7ddce7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:21.542709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-189b0522-eb09-46d7-9033-e5de0b7ddce7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:21.543383 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-189b0522-eb09-46d7-9033-e5de0b7ddce7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:21.544004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-189b0522-eb09-46d7-9033-e5de0b7ddce7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:21.547656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=87 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:21.685264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-189b0522-eb09-46d7-9033-e5de0b7ddce7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:21.694059 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-189b0522-eb09-46d7-9033-e5de0b7ddce7 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.170634 Sep 27 10:42:21.694611 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3067/6603] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:21 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:22.722023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b096a019-8165-4e17-ba52-7fc4ee37e5dc tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:22.722889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b096a019-8165-4e17-ba52-7fc4ee37e5dc tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:22.735241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b096a019-8165-4e17-ba52-7fc4ee37e5dc tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:22.736057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b096a019-8165-4e17-ba52-7fc4ee37e5dc tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:22.736868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b096a019-8165-4e17-ba52-7fc4ee37e5dc tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:22.738035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b096a019-8165-4e17-ba52-7fc4ee37e5dc tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:22.738801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b096a019-8165-4e17-ba52-7fc4ee37e5dc tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:22.739517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b096a019-8165-4e17-ba52-7fc4ee37e5dc tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:22.876329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b096a019-8165-4e17-ba52-7fc4ee37e5dc tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:22.884619 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b096a019-8165-4e17-ba52-7fc4ee37e5dc tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.169406 Sep 27 10:42:22.885239 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3537/6604] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:22 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:23.912536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-48570702-17cc-4b0c-a19d-916a6b45a9fa tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:23.913153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-48570702-17cc-4b0c-a19d-916a6b45a9fa tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:23.915125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=28 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:23.920812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48570702-17cc-4b0c-a19d-916a6b45a9fa tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:23.921283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48570702-17cc-4b0c-a19d-916a6b45a9fa tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:23.921736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48570702-17cc-4b0c-a19d-916a6b45a9fa tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:23.922449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48570702-17cc-4b0c-a19d-916a6b45a9fa tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:23.922923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48570702-17cc-4b0c-a19d-916a6b45a9fa tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:23.923387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-48570702-17cc-4b0c-a19d-916a6b45a9fa tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:24.058092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-48570702-17cc-4b0c-a19d-916a6b45a9fa tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:24.066731 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-48570702-17cc-4b0c-a19d-916a6b45a9fa tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.161600 Sep 27 10:42:24.067319 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3068/6605] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:23 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:25.096188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b870d73a-3a88-459f-abe3-d27940d97a06 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:25.096943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b870d73a-3a88-459f-abe3-d27940d97a06 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:25.099709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=32 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:25.105512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b870d73a-3a88-459f-abe3-d27940d97a06 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:25.105997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b870d73a-3a88-459f-abe3-d27940d97a06 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:25.106428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b870d73a-3a88-459f-abe3-d27940d97a06 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:25.107156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b870d73a-3a88-459f-abe3-d27940d97a06 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:25.107584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b870d73a-3a88-459f-abe3-d27940d97a06 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:25.108010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b870d73a-3a88-459f-abe3-d27940d97a06 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:25.110770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:25.246722 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b870d73a-3a88-459f-abe3-d27940d97a06 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:25.254587 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b870d73a-3a88-459f-abe3-d27940d97a06 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.162483 Sep 27 10:42:25.255196 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3538/6606] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:25 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:26.278717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-56b41e09-6cb4-4bf3-b2de-92b4b7c973d8 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:26.279415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-56b41e09-6cb4-4bf3-b2de-92b4b7c973d8 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:26.288200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56b41e09-6cb4-4bf3-b2de-92b4b7c973d8 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:26.288739 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56b41e09-6cb4-4bf3-b2de-92b4b7c973d8 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:26.289246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56b41e09-6cb4-4bf3-b2de-92b4b7c973d8 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:26.289977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56b41e09-6cb4-4bf3-b2de-92b4b7c973d8 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:26.290449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56b41e09-6cb4-4bf3-b2de-92b4b7c973d8 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:26.290888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-56b41e09-6cb4-4bf3-b2de-92b4b7c973d8 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:26.434974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-56b41e09-6cb4-4bf3-b2de-92b4b7c973d8 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:26.447234 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-56b41e09-6cb4-4bf3-b2de-92b4b7c973d8 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.172238 Sep 27 10:42:26.448005 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3069/6607] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:26 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:27.479283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-287e5276-6f4b-4013-b84d-dfdd124c228d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:27.480320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-287e5276-6f4b-4013-b84d-dfdd124c228d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:27.492241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-287e5276-6f4b-4013-b84d-dfdd124c228d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:27.492760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-287e5276-6f4b-4013-b84d-dfdd124c228d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:27.493206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-287e5276-6f4b-4013-b84d-dfdd124c228d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:27.493914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-287e5276-6f4b-4013-b84d-dfdd124c228d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:27.494375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-287e5276-6f4b-4013-b84d-dfdd124c228d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:27.494831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-287e5276-6f4b-4013-b84d-dfdd124c228d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:27.628588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-287e5276-6f4b-4013-b84d-dfdd124c228d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:27.636956 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-287e5276-6f4b-4013-b84d-dfdd124c228d tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.164230 Sep 27 10:42:27.637533 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3539/6608] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:27 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:28.664992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-39b908cb-c7e9-4772-a35b-93ad1dc75c75 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:28.666023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-39b908cb-c7e9-4772-a35b-93ad1dc75c75 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:28.678202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39b908cb-c7e9-4772-a35b-93ad1dc75c75 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:28.679019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39b908cb-c7e9-4772-a35b-93ad1dc75c75 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:28.679750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39b908cb-c7e9-4772-a35b-93ad1dc75c75 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:28.680441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39b908cb-c7e9-4772-a35b-93ad1dc75c75 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:28.681012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39b908cb-c7e9-4772-a35b-93ad1dc75c75 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:28.681535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39b908cb-c7e9-4772-a35b-93ad1dc75c75 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:28.828789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-39b908cb-c7e9-4772-a35b-93ad1dc75c75 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:28.837376 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-39b908cb-c7e9-4772-a35b-93ad1dc75c75 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1517 microversion: 2.1 time: 0.179252 Sep 27 10:42:28.837966 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3070/6609] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:28 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1517 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:29.758692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-580095f5-2b0e-4f36-95d4-ef538f0a557e req-0c11ae61-0de8-41f1-ad66-779c0f5cf92c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3af889fc-12e1-40c4-a5a4-589a49c1534c", "name": "network-vif-unplugged", "status": "completed", "tag": "202e9028-fb3b-4379-80ba-fe8d0c00b6bf"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:42:29.769963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-580095f5-2b0e-4f36-95d4-ef538f0a557e req-0c11ae61-0de8-41f1-ad66-779c0f5cf92c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:29.770488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-580095f5-2b0e-4f36-95d4-ef538f0a557e req-0c11ae61-0de8-41f1-ad66-779c0f5cf92c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:29.770977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-580095f5-2b0e-4f36-95d4-ef538f0a557e req-0c11ae61-0de8-41f1-ad66-779c0f5cf92c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:29.783392 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-580095f5-2b0e-4f36-95d4-ef538f0a557e req-0c11ae61-0de8-41f1-ad66-779c0f5cf92c service nova] Creating event network-vif-unplugged:202e9028-fb3b-4379-80ba-fe8d0c00b6bf for instance 3af889fc-12e1-40c4-a5a4-589a49c1534c on nch1-master-1 Sep 27 10:42:29.789459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-580095f5-2b0e-4f36-95d4-ef538f0a557e req-0c11ae61-0de8-41f1-ad66-779c0f5cf92c service nova] Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:42:29.800320 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-580095f5-2b0e-4f36-95d4-ef538f0a557e req-0c11ae61-0de8-41f1-ad66-779c0f5cf92c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.045864 Sep 27 10:42:29.800993 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3540/6610] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:42:29 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:42:29.860833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da2b6ef8-5c3b-4e7c-8733-80d2dbb98658 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:29.861457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-da2b6ef8-5c3b-4e7c-8733-80d2dbb98658 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:29.869640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da2b6ef8-5c3b-4e7c-8733-80d2dbb98658 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:29.870199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da2b6ef8-5c3b-4e7c-8733-80d2dbb98658 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:29.870641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da2b6ef8-5c3b-4e7c-8733-80d2dbb98658 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:29.871395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da2b6ef8-5c3b-4e7c-8733-80d2dbb98658 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:29.871898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da2b6ef8-5c3b-4e7c-8733-80d2dbb98658 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:29.872340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da2b6ef8-5c3b-4e7c-8733-80d2dbb98658 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:30.057985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da2b6ef8-5c3b-4e7c-8733-80d2dbb98658 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:30.066119 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-da2b6ef8-5c3b-4e7c-8733-80d2dbb98658 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1516 microversion: 2.1 time: 0.209278 Sep 27 10:42:30.067030 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3071/6611] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:29 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1516 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:31.099261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-921cc5ab-6111-4438-b1c9-37319d6e5f8c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:31.100099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-921cc5ab-6111-4438-b1c9-37319d6e5f8c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:31.108305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-921cc5ab-6111-4438-b1c9-37319d6e5f8c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:31.108852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-921cc5ab-6111-4438-b1c9-37319d6e5f8c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:31.109296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-921cc5ab-6111-4438-b1c9-37319d6e5f8c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:31.110044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-921cc5ab-6111-4438-b1c9-37319d6e5f8c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:31.110506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-921cc5ab-6111-4438-b1c9-37319d6e5f8c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:31.110996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-921cc5ab-6111-4438-b1c9-37319d6e5f8c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:31.259582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-921cc5ab-6111-4438-b1c9-37319d6e5f8c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:31.267674 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-921cc5ab-6111-4438-b1c9-37319d6e5f8c tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1514 microversion: 2.1 time: 0.177294 Sep 27 10:42:31.268267 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3541/6612] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:31 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1514 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:31.837265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-48c69218-5858-49fa-bc12-d42cd9fcdc32 req-bc3cfeb6-3673-4b8b-8e23-fa2f3d1544a9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3af889fc-12e1-40c4-a5a4-589a49c1534c", "tag": "202e9028-fb3b-4379-80ba-fe8d0c00b6bf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:42:31.848042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-48c69218-5858-49fa-bc12-d42cd9fcdc32 req-bc3cfeb6-3673-4b8b-8e23-fa2f3d1544a9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:31.848536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-48c69218-5858-49fa-bc12-d42cd9fcdc32 req-bc3cfeb6-3673-4b8b-8e23-fa2f3d1544a9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:31.849015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-48c69218-5858-49fa-bc12-d42cd9fcdc32 req-bc3cfeb6-3673-4b8b-8e23-fa2f3d1544a9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:31.861286 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-48c69218-5858-49fa-bc12-d42cd9fcdc32 req-bc3cfeb6-3673-4b8b-8e23-fa2f3d1544a9 service nova] Creating event network-changed:202e9028-fb3b-4379-80ba-fe8d0c00b6bf for instance 3af889fc-12e1-40c4-a5a4-589a49c1534c on nch1-master-1 Sep 27 10:42:31.867471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-48c69218-5858-49fa-bc12-d42cd9fcdc32 req-bc3cfeb6-3673-4b8b-8e23-fa2f3d1544a9 service nova] Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:42:31.879264 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-48c69218-5858-49fa-bc12-d42cd9fcdc32 req-bc3cfeb6-3673-4b8b-8e23-fa2f3d1544a9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047920 Sep 27 10:42:31.879885 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3072/6613] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:42:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:42:32.292913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b675f053-dccc-45b1-880e-74d84f7fe661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:32.294447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b675f053-dccc-45b1-880e-74d84f7fe661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:32.302325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b675f053-dccc-45b1-880e-74d84f7fe661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:32.302924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b675f053-dccc-45b1-880e-74d84f7fe661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:32.303422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b675f053-dccc-45b1-880e-74d84f7fe661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:32.304182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b675f053-dccc-45b1-880e-74d84f7fe661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:32.304576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b675f053-dccc-45b1-880e-74d84f7fe661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:32.305028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b675f053-dccc-45b1-880e-74d84f7fe661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:32.447564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b675f053-dccc-45b1-880e-74d84f7fe661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:32.455554 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b675f053-dccc-45b1-880e-74d84f7fe661 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1511 microversion: 2.1 time: 0.167796 Sep 27 10:42:32.456154 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3542/6614] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:32 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1511 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:32.479497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6f75e3d8-b69e-40ab-b2f9-1820b0a87250 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:32.480178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6f75e3d8-b69e-40ab-b2f9-1820b0a87250 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:32.486768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f75e3d8-b69e-40ab-b2f9-1820b0a87250 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:32.487365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f75e3d8-b69e-40ab-b2f9-1820b0a87250 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:32.487888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f75e3d8-b69e-40ab-b2f9-1820b0a87250 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:32.488675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f75e3d8-b69e-40ab-b2f9-1820b0a87250 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:32.489287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f75e3d8-b69e-40ab-b2f9-1820b0a87250 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:32.489787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f75e3d8-b69e-40ab-b2f9-1820b0a87250 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:32.516512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6f75e3d8-b69e-40ab-b2f9-1820b0a87250 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:42:32.570615 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.compute.api [None req-6f75e3d8-b69e-40ab-b2f9-1820b0a87250 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Found an unconfirmed migration during delete, id: 16, status: finished Sep 27 10:42:34.469839 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6f75e3d8-b69e-40ab-b2f9-1820b0a87250 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 204 len: 0 microversion: 2.1 time: 1.994283 Sep 27 10:42:34.471200 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3073/6615] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:42:32 2022] DELETE /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 0 bytes in 1997 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:42:34.479970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c22dcdc3-2154-4498-8139-139ddf946912 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:34.480624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c22dcdc3-2154-4498-8139-139ddf946912 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:34.488064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22dcdc3-2154-4498-8139-139ddf946912 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:34.488561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22dcdc3-2154-4498-8139-139ddf946912 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:34.489036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22dcdc3-2154-4498-8139-139ddf946912 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:34.489752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22dcdc3-2154-4498-8139-139ddf946912 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:34.490218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22dcdc3-2154-4498-8139-139ddf946912 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:34.490652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c22dcdc3-2154-4498-8139-139ddf946912 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:34.632112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c22dcdc3-2154-4498-8139-139ddf946912 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:34.640204 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c22dcdc3-2154-4498-8139-139ddf946912 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1503 microversion: 2.1 time: 0.164568 Sep 27 10:42:34.640842 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3543/6616] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:34 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1503 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:35.664458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bbddcbd7-c8bf-43f0-adee-9125f165e9c4 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:35.665202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bbddcbd7-c8bf-43f0-adee-9125f165e9c4 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:35.673154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbddcbd7-c8bf-43f0-adee-9125f165e9c4 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:35.673659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbddcbd7-c8bf-43f0-adee-9125f165e9c4 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:35.674119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbddcbd7-c8bf-43f0-adee-9125f165e9c4 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:35.674841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbddcbd7-c8bf-43f0-adee-9125f165e9c4 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:35.675314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbddcbd7-c8bf-43f0-adee-9125f165e9c4 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:35.675757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bbddcbd7-c8bf-43f0-adee-9125f165e9c4 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:35.819210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bbddcbd7-c8bf-43f0-adee-9125f165e9c4 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8e0f9f1c851743c7b466be4e42d4a11b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8ef200fc20ce4b0da3c92b5179a40e63', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:35.828681 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bbddcbd7-c8bf-43f0-adee-9125f165e9c4 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 200 len: 1503 microversion: 2.1 time: 0.168666 Sep 27 10:42:35.829482 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3074/6617] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:35 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 1503 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:35.996202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f8f34802-2b7c-426c-b42a-65d568f7cb60 req-5c4a1205-6afb-4c2d-b58e-6222dfc0a2e7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3af889fc-12e1-40c4-a5a4-589a49c1534c", "name": "network-vif-plugged", "status": "completed", "tag": "202e9028-fb3b-4379-80ba-fe8d0c00b6bf"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:42:36.010152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f8f34802-2b7c-426c-b42a-65d568f7cb60 req-5c4a1205-6afb-4c2d-b58e-6222dfc0a2e7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:36.010889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f8f34802-2b7c-426c-b42a-65d568f7cb60 req-5c4a1205-6afb-4c2d-b58e-6222dfc0a2e7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:36.011328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f8f34802-2b7c-426c-b42a-65d568f7cb60 req-5c4a1205-6afb-4c2d-b58e-6222dfc0a2e7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:36.023197 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f8f34802-2b7c-426c-b42a-65d568f7cb60 req-5c4a1205-6afb-4c2d-b58e-6222dfc0a2e7 service nova] Creating event network-vif-plugged:202e9028-fb3b-4379-80ba-fe8d0c00b6bf for instance 3af889fc-12e1-40c4-a5a4-589a49c1534c on nch1-master-1 Sep 27 10:42:36.029085 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f8f34802-2b7c-426c-b42a-65d568f7cb60 req-5c4a1205-6afb-4c2d-b58e-6222dfc0a2e7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037803 Sep 27 10:42:36.029860 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3544/6618] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:42:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:42:36.300114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2c27704b-7f79-4347-888e-b892715ff094 req-c8a3316d-afdb-472d-baaa-7423f0b84181 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3af889fc-12e1-40c4-a5a4-589a49c1534c", "name": "network-vif-deleted", "tag": "202e9028-fb3b-4379-80ba-fe8d0c00b6bf"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:42:36.311786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2c27704b-7f79-4347-888e-b892715ff094 req-c8a3316d-afdb-472d-baaa-7423f0b84181 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:36.312285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2c27704b-7f79-4347-888e-b892715ff094 req-c8a3316d-afdb-472d-baaa-7423f0b84181 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:36.316905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2c27704b-7f79-4347-888e-b892715ff094 req-c8a3316d-afdb-472d-baaa-7423f0b84181 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:36.333081 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2c27704b-7f79-4347-888e-b892715ff094 req-c8a3316d-afdb-472d-baaa-7423f0b84181 service nova] Creating event network-vif-deleted:202e9028-fb3b-4379-80ba-fe8d0c00b6bf for instance 3af889fc-12e1-40c4-a5a4-589a49c1534c on nch1-master-1 Sep 27 10:42:36.338203 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2c27704b-7f79-4347-888e-b892715ff094 req-c8a3316d-afdb-472d-baaa-7423f0b84181 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041795 Sep 27 10:42:36.338890 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3075/6619] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:42:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:42:36.857413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:36.858558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:36.868986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:36.869498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:36.869937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:36.870631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:36.871082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:36.871508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:36.886332 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] HTTP exception thrown: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Request method failure captured: Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c HTTP/1.1 Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtNpl-PTIB3yAqxQHPNqHvAoPJAerCP8ydJh74Hz7oPQY-UPy64QOabe4CpisT1xjV9euMjSkPH4zqnpM1knmQE5GeBIXh0SQx7Q7cGv75-FDdAJ1tGBCMYXWJzgw7lZcXl5IITWUfprDx_Jsx7i2vkegYBP1RrhwbCBcGcdx80YEMI Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 8ef200fc20ce4b0da3c92b5179a40e63 Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-188432339 Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ef200fc20ce4b0da3c92b5179a40e63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ef200fc20ce4b0da3c92b5179a40e63"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8ef200fc20ce4b0da3c92b5179a40e63"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8ef200fc20ce4b0da3c92b5179a40e63"}]}] Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-188432339 Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 8ef200fc20ce4b0da3c92b5179a40e63 Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-188432339 Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-188432339-project Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 8e0f9f1c851743c7b466be4e42d4a11b Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-188432339-project Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3af889fc-12e1-40c4-a5a4-589a49c1534c'} Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:42:36.888141 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.892776 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:42:36.892776 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:42:36.892776 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:42:36.892776 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:42:36.892776 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:42:36.892776 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.892776 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.892776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Returning 404 to user: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:42:36.892776 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-866fbcd8-ecbf-463a-9546-0fcc5c50271b tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 404 len: 111 microversion: 2.1 time: 0.040393 Sep 27 10:42:36.892776 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3545/6620] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:36 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:42:36.906317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:36.906912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:36.913606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:36.914101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:36.914541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:36.915250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:36.915715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:36.916264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:36.930232 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] HTTP exception thrown: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Request method failure captured: Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c HTTP/1.1 Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtNpl-PTIB3yAqxQHPNqHvAoPJAerCP8ydJh74Hz7oPQY-UPy64QOabe4CpisT1xjV9euMjSkPH4zqnpM1knmQE5GeBIXh0SQx7Q7cGv75-FDdAJ1tGBCMYXWJzgw7lZcXl5IITWUfprDx_Jsx7i2vkegYBP1RrhwbCBcGcdx80YEMI Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 8ef200fc20ce4b0da3c92b5179a40e63 Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-DeleteServersTestJSON-188432339 Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ef200fc20ce4b0da3c92b5179a40e63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ef200fc20ce4b0da3c92b5179a40e63"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8ef200fc20ce4b0da3c92b5179a40e63"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8ef200fc20ce4b0da3c92b5179a40e63"}]}] Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-DeleteServersTestJSON-188432339 Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 8ef200fc20ce4b0da3c92b5179a40e63 Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-DeleteServersTestJSON-188432339 Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-DeleteServersTestJSON-188432339-project Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 8e0f9f1c851743c7b466be4e42d4a11b Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-DeleteServersTestJSON-188432339-project Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '3af889fc-12e1-40c4-a5a4-589a49c1534c'} Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:42:36.932156 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Returning 404 to user: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e3cf71b6-17fa-4d92-9750-5b1638b53804 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 404 len: 111 microversion: 2.1 time: 0.032394 Sep 27 10:42:36.936617 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3076/6621] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:42:36 2022] DELETE /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:42:36.942430 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:36.943048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] [instance: 3af889fc-12e1-40c4-a5a4-589a49c1534c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:36.949933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:36.950415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:36.950849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:36.951547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:36.952004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:36.952433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:36.969742 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] HTTP exception thrown: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Request method failure captured: Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c HTTP/1.1 Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtNpl-PTIB3yAqxQHPNqHvAoPJAerCP8ydJh74Hz7oPQY-UPy64QOabe4CpisT1xjV9euMjSkPH4zqnpM1knmQE5GeBIXh0SQx7Q7cGv75-FDdAJ1tGBCMYXWJzgw7lZcXl5IITWUfprDx_Jsx7i2vkegYBP1RrhwbCBcGcdx80YEMI Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 8ef200fc20ce4b0da3c92b5179a40e63 Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-DeleteServersTestJSON-188432339 Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ef200fc20ce4b0da3c92b5179a40e63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8ef200fc20ce4b0da3c92b5179a40e63"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8ef200fc20ce4b0da3c92b5179a40e63"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8ef200fc20ce4b0da3c92b5179a40e63"}]}] Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-DeleteServersTestJSON-188432339 Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 8ef200fc20ce4b0da3c92b5179a40e63 Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-DeleteServersTestJSON-188432339 Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-DeleteServersTestJSON-188432339-project Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 8e0f9f1c851743c7b466be4e42d4a11b Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-DeleteServersTestJSON-188432339-project Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '3af889fc-12e1-40c4-a5a4-589a49c1534c'} Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:42:36.971313 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.975912 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:42:36.975912 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:42:36.975912 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:42:36.975912 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:42:36.975912 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:42:36.975912 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. Sep 27 10:42:36.975912 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:42:36.975912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] Returning 404 to user: Instance 3af889fc-12e1-40c4-a5a4-589a49c1534c could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:42:36.975912 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3b643a39-3735-40d0-8207-19ef28c6c8e3 tempest-DeleteServersTestJSON-188432339 tempest-DeleteServersTestJSON-188432339-project] 10.222.0.26 "GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c" status: 404 len: 111 microversion: 2.1 time: 0.034227 Sep 27 10:42:36.975912 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3546/6622] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:36 2022] GET /compute/v2.1/servers/3af889fc-12e1-40c4-a5a4-589a49c1534c => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:42:45.887428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-1119562684", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "861c59c3-0005-427d-877a-900443c2925b"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:42:45.944111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] validate_networks() for [('861c59c3-0005-427d-877a-900443c2925b', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:42:46.344199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:42:46.344639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:42:46.345133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:42:46.345777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:42:46.346171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:42:46.356358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:46.356933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:46.357386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:46.358138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:46.358786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:46.359233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:46.376755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Getting quotas for project 063971e11eb044b4a844a563c6ccbc5d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:42:46.382196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Getting quotas for user 74e30240d7c840edba43dddf3a862c4f and project 063971e11eb044b4a844a563c6ccbc5d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:42:46.395570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:42:46.397974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:42:46.446124 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9def0884-bd9e-4e2e-a334-aeafcb65a4cc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.687189 Sep 27 10:42:46.447195 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3077/6623] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:42:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 689 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:42:46.460619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7b726e16-1eba-4bee-bca0-0df2dad157ee tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:46.465288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7b726e16-1eba-4bee-bca0-0df2dad157ee tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:46.553186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7b726e16-1eba-4bee-bca0-0df2dad157ee tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:46.565763 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7b726e16-1eba-4bee-bca0-0df2dad157ee tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1218 microversion: 2.1 time: 0.110228 Sep 27 10:42:46.566688 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3547/6624] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:46 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1218 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:47.592195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b5661018-7543-4b39-9510-b5df7f9b0f3f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:47.594030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b5661018-7543-4b39-9510-b5df7f9b0f3f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:47.601652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5661018-7543-4b39-9510-b5df7f9b0f3f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:47.602387 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5661018-7543-4b39-9510-b5df7f9b0f3f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:47.603071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5661018-7543-4b39-9510-b5df7f9b0f3f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:47.604038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5661018-7543-4b39-9510-b5df7f9b0f3f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:47.604676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5661018-7543-4b39-9510-b5df7f9b0f3f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:47.605415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b5661018-7543-4b39-9510-b5df7f9b0f3f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:47.726756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b5661018-7543-4b39-9510-b5df7f9b0f3f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:47.737661 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b5661018-7543-4b39-9510-b5df7f9b0f3f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1288 microversion: 2.1 time: 0.150646 Sep 27 10:42:47.741303 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3078/6625] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:47 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1288 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:48.763605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d93529e5-7806-4b6a-bef7-85c00898cf31 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:48.764228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d93529e5-7806-4b6a-bef7-85c00898cf31 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:48.766601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=39 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:48.774239 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93529e5-7806-4b6a-bef7-85c00898cf31 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:48.774801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93529e5-7806-4b6a-bef7-85c00898cf31 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:48.779842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93529e5-7806-4b6a-bef7-85c00898cf31 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:48.781358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93529e5-7806-4b6a-bef7-85c00898cf31 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:48.782124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93529e5-7806-4b6a-bef7-85c00898cf31 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:48.782805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d93529e5-7806-4b6a-bef7-85c00898cf31 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:48.967099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d93529e5-7806-4b6a-bef7-85c00898cf31 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:48.979777 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d93529e5-7806-4b6a-bef7-85c00898cf31 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1318 microversion: 2.1 time: 0.222170 Sep 27 10:42:48.984714 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3548/6626] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:48 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1318 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:49.138712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b3b5de78-d699-4368-bc66-4a176ae38554 req-6270af97-e523-4e7c-b083-f8050d52db89 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "25c83c81-2c6e-4d30-81b4-8c71b4d6dce7", "tag": "9687b9f1-54e8-4a5d-a2e8-1007041e00be"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:42:49.164574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b3b5de78-d699-4368-bc66-4a176ae38554 req-6270af97-e523-4e7c-b083-f8050d52db89 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:49.165430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b3b5de78-d699-4368-bc66-4a176ae38554 req-6270af97-e523-4e7c-b083-f8050d52db89 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:49.166158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b3b5de78-d699-4368-bc66-4a176ae38554 req-6270af97-e523-4e7c-b083-f8050d52db89 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:49.179142 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b3b5de78-d699-4368-bc66-4a176ae38554 req-6270af97-e523-4e7c-b083-f8050d52db89 service nova] Creating event network-changed:9687b9f1-54e8-4a5d-a2e8-1007041e00be for instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 on nch1-master-1 Sep 27 10:42:49.184617 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b3b5de78-d699-4368-bc66-4a176ae38554 req-6270af97-e523-4e7c-b083-f8050d52db89 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052556 Sep 27 10:42:49.185932 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3079/6627] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:42:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:42:50.003475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-49529932-91b9-4279-9ada-88413f653203 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:50.005021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-49529932-91b9-4279-9ada-88413f653203 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:50.013314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49529932-91b9-4279-9ada-88413f653203 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:50.013801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49529932-91b9-4279-9ada-88413f653203 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:50.014294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49529932-91b9-4279-9ada-88413f653203 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:50.014998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49529932-91b9-4279-9ada-88413f653203 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:50.015453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49529932-91b9-4279-9ada-88413f653203 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:50.015877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49529932-91b9-4279-9ada-88413f653203 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:50.156261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-49529932-91b9-4279-9ada-88413f653203 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:50.164374 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-49529932-91b9-4279-9ada-88413f653203 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1318 microversion: 2.1 time: 0.164861 Sep 27 10:42:50.164956 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3549/6628] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:49 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1318 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:51.191917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-debf555c-5cab-424f-8d68-4eba828aa86d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:51.192760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-debf555c-5cab-424f-8d68-4eba828aa86d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:51.201277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-debf555c-5cab-424f-8d68-4eba828aa86d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:51.201880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-debf555c-5cab-424f-8d68-4eba828aa86d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:51.202525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-debf555c-5cab-424f-8d68-4eba828aa86d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:51.203598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-debf555c-5cab-424f-8d68-4eba828aa86d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:51.204280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-debf555c-5cab-424f-8d68-4eba828aa86d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:51.205001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-debf555c-5cab-424f-8d68-4eba828aa86d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:51.344513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-debf555c-5cab-424f-8d68-4eba828aa86d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:51.355125 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-debf555c-5cab-424f-8d68-4eba828aa86d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1318 microversion: 2.1 time: 0.169509 Sep 27 10:42:51.355894 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3080/6629] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:51 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1318 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:52.386074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9a8da43-d793-465f-964d-57357b90138b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:52.386656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c9a8da43-d793-465f-964d-57357b90138b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:52.397321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a8da43-d793-465f-964d-57357b90138b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:52.397718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a8da43-d793-465f-964d-57357b90138b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:52.398150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a8da43-d793-465f-964d-57357b90138b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:52.398876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a8da43-d793-465f-964d-57357b90138b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:52.399331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a8da43-d793-465f-964d-57357b90138b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:52.399754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9a8da43-d793-465f-964d-57357b90138b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:52.542081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9a8da43-d793-465f-964d-57357b90138b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:52.550041 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9a8da43-d793-465f-964d-57357b90138b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1318 microversion: 2.1 time: 0.170129 Sep 27 10:42:52.550952 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3550/6630] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:52 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1318 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:53.575605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-730d1252-1e02-480c-9191-766de3b831c9 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:53.576284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-730d1252-1e02-480c-9191-766de3b831c9 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:53.584707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730d1252-1e02-480c-9191-766de3b831c9 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:53.585352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730d1252-1e02-480c-9191-766de3b831c9 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:53.585819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730d1252-1e02-480c-9191-766de3b831c9 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:53.586540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730d1252-1e02-480c-9191-766de3b831c9 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:53.587021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730d1252-1e02-480c-9191-766de3b831c9 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:53.587467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-730d1252-1e02-480c-9191-766de3b831c9 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:53.590488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:53.734202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-730d1252-1e02-480c-9191-766de3b831c9 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:53.742611 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-730d1252-1e02-480c-9191-766de3b831c9 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1318 microversion: 2.1 time: 0.172214 Sep 27 10:42:53.743197 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3081/6631] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:53 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1318 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:54.072435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0a6e9386-0e47-4f61-b35a-bbed3ea04f14 req-2625ce0d-3961-435e-807f-eb01a87a301b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "25c83c81-2c6e-4d30-81b4-8c71b4d6dce7", "name": "network-vif-plugged", "status": "completed", "tag": "9687b9f1-54e8-4a5d-a2e8-1007041e00be"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:42:54.083331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0a6e9386-0e47-4f61-b35a-bbed3ea04f14 req-2625ce0d-3961-435e-807f-eb01a87a301b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:54.083803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0a6e9386-0e47-4f61-b35a-bbed3ea04f14 req-2625ce0d-3961-435e-807f-eb01a87a301b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:54.084292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0a6e9386-0e47-4f61-b35a-bbed3ea04f14 req-2625ce0d-3961-435e-807f-eb01a87a301b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:54.096049 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0a6e9386-0e47-4f61-b35a-bbed3ea04f14 req-2625ce0d-3961-435e-807f-eb01a87a301b service nova] Creating event network-vif-plugged:9687b9f1-54e8-4a5d-a2e8-1007041e00be for instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 on nch1-master-1 Sep 27 10:42:54.101272 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0a6e9386-0e47-4f61-b35a-bbed3ea04f14 req-2625ce0d-3961-435e-807f-eb01a87a301b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032919 Sep 27 10:42:54.101887 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3551/6632] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:42:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:42:54.771119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1a37fdea-0964-4621-aecc-54b577480957 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:54.772024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1a37fdea-0964-4621-aecc-54b577480957 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:54.776393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=44,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:54.783336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a37fdea-0964-4621-aecc-54b577480957 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:54.783877 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a37fdea-0964-4621-aecc-54b577480957 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:54.784381 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a37fdea-0964-4621-aecc-54b577480957 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:54.785137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a37fdea-0964-4621-aecc-54b577480957 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:54.785609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a37fdea-0964-4621-aecc-54b577480957 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:54.786050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a37fdea-0964-4621-aecc-54b577480957 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:54.936627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1a37fdea-0964-4621-aecc-54b577480957 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:54.944890 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1a37fdea-0964-4621-aecc-54b577480957 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1318 microversion: 2.1 time: 0.181149 Sep 27 10:42:54.945475 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3082/6633] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:54 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1318 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:55.969713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-490812fd-909a-4b00-ad41-1f8c0c4dcbdc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:55.971226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-490812fd-909a-4b00-ad41-1f8c0c4dcbdc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:55.972448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=30 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:55.981089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-490812fd-909a-4b00-ad41-1f8c0c4dcbdc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:55.981825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-490812fd-909a-4b00-ad41-1f8c0c4dcbdc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:55.983327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-490812fd-909a-4b00-ad41-1f8c0c4dcbdc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:55.986628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-490812fd-909a-4b00-ad41-1f8c0c4dcbdc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:55.987137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-490812fd-909a-4b00-ad41-1f8c0c4dcbdc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:55.987575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-490812fd-909a-4b00-ad41-1f8c0c4dcbdc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:56.140547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-490812fd-909a-4b00-ad41-1f8c0c4dcbdc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:56.149198 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-490812fd-909a-4b00-ad41-1f8c0c4dcbdc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1508 microversion: 2.1 time: 0.184828 Sep 27 10:42:56.150779 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3552/6634] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:55 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1508 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:56.181716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5c22dfb9-f068-4a16-b4d6-bfb59bcda787 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:56.182656 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5c22dfb9-f068-4a16-b4d6-bfb59bcda787 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:56.193314 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c22dfb9-f068-4a16-b4d6-bfb59bcda787 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:56.193826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c22dfb9-f068-4a16-b4d6-bfb59bcda787 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:56.194283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c22dfb9-f068-4a16-b4d6-bfb59bcda787 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:56.195037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c22dfb9-f068-4a16-b4d6-bfb59bcda787 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:56.195504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c22dfb9-f068-4a16-b4d6-bfb59bcda787 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:56.195949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5c22dfb9-f068-4a16-b4d6-bfb59bcda787 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:56.348819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5c22dfb9-f068-4a16-b4d6-bfb59bcda787 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:56.358480 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5c22dfb9-f068-4a16-b4d6-bfb59bcda787 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1508 microversion: 2.1 time: 0.182853 Sep 27 10:42:56.358588 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3083/6635] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:56 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1508 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:56.380972 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-35221031-9818-46a8-b371-e60bd11fbfb3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Action: 'update', calling method: >, body: {"server": {"OS-DCF:diskConfig": "AUTO"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:42:56.383390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-35221031-9818-46a8-b371-e60bd11fbfb3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:56.390546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35221031-9818-46a8-b371-e60bd11fbfb3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:56.390974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35221031-9818-46a8-b371-e60bd11fbfb3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:56.391416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35221031-9818-46a8-b371-e60bd11fbfb3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:56.392112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35221031-9818-46a8-b371-e60bd11fbfb3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:56.392568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35221031-9818-46a8-b371-e60bd11fbfb3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:56.393038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-35221031-9818-46a8-b371-e60bd11fbfb3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:56.479269 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-35221031-9818-46a8-b371-e60bd11fbfb3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "PUT /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1080 microversion: 2.1 time: 0.102208 Sep 27 10:42:56.479909 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3553/6636] 10.222.0.26 () {60 vars in 1256 bytes} [Tue Sep 27 10:42:56 2022] PUT /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1080 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:56.499255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-481eb662-51cd-4607-98a5-b3bf51cb270b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:56.499875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-481eb662-51cd-4607-98a5-b3bf51cb270b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:56.501810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:56.502502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:56.509768 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-481eb662-51cd-4607-98a5-b3bf51cb270b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:56.510293 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-481eb662-51cd-4607-98a5-b3bf51cb270b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:56.510754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-481eb662-51cd-4607-98a5-b3bf51cb270b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:56.511485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-481eb662-51cd-4607-98a5-b3bf51cb270b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:56.511962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-481eb662-51cd-4607-98a5-b3bf51cb270b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:56.512463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-481eb662-51cd-4607-98a5-b3bf51cb270b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:56.654654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-481eb662-51cd-4607-98a5-b3bf51cb270b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:56.662856 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-481eb662-51cd-4607-98a5-b3bf51cb270b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1506 microversion: 2.1 time: 0.167617 Sep 27 10:42:56.663441 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3084/6637] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:56 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1506 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:56.687443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bcede017-7ef5-40e3-9b75-30e62f59eb41 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:56.688054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bcede017-7ef5-40e3-9b75-30e62f59eb41 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:56.690037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:42:56.697340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcede017-7ef5-40e3-9b75-30e62f59eb41 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:56.698222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcede017-7ef5-40e3-9b75-30e62f59eb41 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:56.698970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcede017-7ef5-40e3-9b75-30e62f59eb41 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:56.700079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcede017-7ef5-40e3-9b75-30e62f59eb41 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:56.700599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcede017-7ef5-40e3-9b75-30e62f59eb41 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:56.701069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bcede017-7ef5-40e3-9b75-30e62f59eb41 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:56.853097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bcede017-7ef5-40e3-9b75-30e62f59eb41 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:56.861897 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bcede017-7ef5-40e3-9b75-30e62f59eb41 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1506 microversion: 2.1 time: 0.179802 Sep 27 10:42:56.862476 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3554/6638] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:56 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1506 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:56.884174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451", "OS-DCF:diskConfig": "MANUAL"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:42:56.886765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:56.893297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:56.893799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:56.894257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:56.894974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:56.895454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:56.895898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:56.975471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:56.981402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Old instance type m1.heat2, new instance type m1.heat {{(pid=145969) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:42:56.988008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:42:56.988437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:42:56.988949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:42:56.989370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:42:56.989911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:42:57.103218 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-05731435-21eb-43cf-8377-80581f6f9df0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "POST /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7/action" status: 202 len: 0 microversion: 2.1 time: 0.222970 Sep 27 10:42:57.103826 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3085/6639] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:42:56 2022] POST /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7/action => generated 0 bytes in 224 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:42:57.112118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7e26bf8e-b034-46a8-8b8c-87e9a7de78bc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:57.112756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7e26bf8e-b034-46a8-8b8c-87e9a7de78bc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:57.119775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e26bf8e-b034-46a8-8b8c-87e9a7de78bc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:57.120280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e26bf8e-b034-46a8-8b8c-87e9a7de78bc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:57.120732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e26bf8e-b034-46a8-8b8c-87e9a7de78bc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:57.121419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e26bf8e-b034-46a8-8b8c-87e9a7de78bc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:57.121978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e26bf8e-b034-46a8-8b8c-87e9a7de78bc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:57.122434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7e26bf8e-b034-46a8-8b8c-87e9a7de78bc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:57.288220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7e26bf8e-b034-46a8-8b8c-87e9a7de78bc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:57.297085 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7e26bf8e-b034-46a8-8b8c-87e9a7de78bc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1517 microversion: 2.1 time: 0.188781 Sep 27 10:42:57.300880 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3555/6640] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:57 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1517 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:58.325834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3db6e3c9-6aca-48d2-af1b-7e16c6ce73fa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:58.326417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3db6e3c9-6aca-48d2-af1b-7e16c6ce73fa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:58.333666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3db6e3c9-6aca-48d2-af1b-7e16c6ce73fa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:58.334159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3db6e3c9-6aca-48d2-af1b-7e16c6ce73fa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:58.334607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3db6e3c9-6aca-48d2-af1b-7e16c6ce73fa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:58.335324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3db6e3c9-6aca-48d2-af1b-7e16c6ce73fa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:58.335780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3db6e3c9-6aca-48d2-af1b-7e16c6ce73fa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:58.336216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3db6e3c9-6aca-48d2-af1b-7e16c6ce73fa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:58.479048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3db6e3c9-6aca-48d2-af1b-7e16c6ce73fa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:58.487056 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3db6e3c9-6aca-48d2-af1b-7e16c6ce73fa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1517 microversion: 2.1 time: 0.168790 Sep 27 10:42:58.487638 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3086/6641] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:58 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1517 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:42:59.515402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8b36b69a-b31c-4552-a8b7-8486c412210c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:42:59.516377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8b36b69a-b31c-4552-a8b7-8486c412210c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:42:59.528193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b36b69a-b31c-4552-a8b7-8486c412210c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:59.529327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b36b69a-b31c-4552-a8b7-8486c412210c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:59.529775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b36b69a-b31c-4552-a8b7-8486c412210c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:59.530482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b36b69a-b31c-4552-a8b7-8486c412210c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:42:59.530940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b36b69a-b31c-4552-a8b7-8486c412210c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:42:59.531371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b36b69a-b31c-4552-a8b7-8486c412210c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:42:59.672325 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8b36b69a-b31c-4552-a8b7-8486c412210c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:42:59.680741 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8b36b69a-b31c-4552-a8b7-8486c412210c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.172696 Sep 27 10:42:59.681332 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3556/6642] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:42:59 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:00.710886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7943a329-4ffa-4a34-a0c4-4b35a3c9a283 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:00.711459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7943a329-4ffa-4a34-a0c4-4b35a3c9a283 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:00.718922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7943a329-4ffa-4a34-a0c4-4b35a3c9a283 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:00.719422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7943a329-4ffa-4a34-a0c4-4b35a3c9a283 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:00.719881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7943a329-4ffa-4a34-a0c4-4b35a3c9a283 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:00.720603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7943a329-4ffa-4a34-a0c4-4b35a3c9a283 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:00.721105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7943a329-4ffa-4a34-a0c4-4b35a3c9a283 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:00.721551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7943a329-4ffa-4a34-a0c4-4b35a3c9a283 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:00.855511 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7943a329-4ffa-4a34-a0c4-4b35a3c9a283 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:00.864382 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7943a329-4ffa-4a34-a0c4-4b35a3c9a283 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.157601 Sep 27 10:43:00.865010 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3087/6643] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:00 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:01.891444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d333f244-471c-47f5-a272-cc83338d026c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:01.892105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d333f244-471c-47f5-a272-cc83338d026c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:01.898745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d333f244-471c-47f5-a272-cc83338d026c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:01.899243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d333f244-471c-47f5-a272-cc83338d026c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:01.899682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d333f244-471c-47f5-a272-cc83338d026c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:01.900389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d333f244-471c-47f5-a272-cc83338d026c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:01.900955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d333f244-471c-47f5-a272-cc83338d026c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:01.901405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d333f244-471c-47f5-a272-cc83338d026c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:02.053083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d333f244-471c-47f5-a272-cc83338d026c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:02.062193 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d333f244-471c-47f5-a272-cc83338d026c tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.176626 Sep 27 10:43:02.062983 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3557/6644] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:01 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:03.087816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2a3bbdf0-02a7-4046-b1a9-29d90f7546b4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:03.088400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2a3bbdf0-02a7-4046-b1a9-29d90f7546b4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:03.095985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3bbdf0-02a7-4046-b1a9-29d90f7546b4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:03.096484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3bbdf0-02a7-4046-b1a9-29d90f7546b4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:03.096968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3bbdf0-02a7-4046-b1a9-29d90f7546b4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:03.097679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3bbdf0-02a7-4046-b1a9-29d90f7546b4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:03.098127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3bbdf0-02a7-4046-b1a9-29d90f7546b4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:03.098567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3bbdf0-02a7-4046-b1a9-29d90f7546b4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:03.232391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2a3bbdf0-02a7-4046-b1a9-29d90f7546b4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:03.240349 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2a3bbdf0-02a7-4046-b1a9-29d90f7546b4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.156617 Sep 27 10:43:03.240952 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3088/6645] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:03 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:04.268741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a8ae2229-2ced-4e3e-9fd9-b8f91f48c372 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:04.269806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a8ae2229-2ced-4e3e-9fd9-b8f91f48c372 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:04.281833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8ae2229-2ced-4e3e-9fd9-b8f91f48c372 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:04.282647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8ae2229-2ced-4e3e-9fd9-b8f91f48c372 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:04.283387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8ae2229-2ced-4e3e-9fd9-b8f91f48c372 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:04.284557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8ae2229-2ced-4e3e-9fd9-b8f91f48c372 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:04.285388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8ae2229-2ced-4e3e-9fd9-b8f91f48c372 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:04.286117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a8ae2229-2ced-4e3e-9fd9-b8f91f48c372 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:04.439930 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a8ae2229-2ced-4e3e-9fd9-b8f91f48c372 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:04.447925 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a8ae2229-2ced-4e3e-9fd9-b8f91f48c372 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.186295 Sep 27 10:43:04.448546 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3558/6646] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:04 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:05.472355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3e4a6a1b-5f15-476f-a0bc-724c4378a6a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:05.472967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e4a6a1b-5f15-476f-a0bc-724c4378a6a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:05.480539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e4a6a1b-5f15-476f-a0bc-724c4378a6a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:05.481062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e4a6a1b-5f15-476f-a0bc-724c4378a6a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:05.481730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e4a6a1b-5f15-476f-a0bc-724c4378a6a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:05.482515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e4a6a1b-5f15-476f-a0bc-724c4378a6a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:05.482983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e4a6a1b-5f15-476f-a0bc-724c4378a6a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:05.483420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e4a6a1b-5f15-476f-a0bc-724c4378a6a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:05.624584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3e4a6a1b-5f15-476f-a0bc-724c4378a6a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:05.632782 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3e4a6a1b-5f15-476f-a0bc-724c4378a6a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.164472 Sep 27 10:43:05.633359 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3089/6647] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:05 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:06.657292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b64ef52e-7ab6-4aaf-b132-f66dab7ccd57 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:06.657909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b64ef52e-7ab6-4aaf-b132-f66dab7ccd57 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:06.665466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b64ef52e-7ab6-4aaf-b132-f66dab7ccd57 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:06.665956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b64ef52e-7ab6-4aaf-b132-f66dab7ccd57 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:06.666392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b64ef52e-7ab6-4aaf-b132-f66dab7ccd57 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:06.667161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b64ef52e-7ab6-4aaf-b132-f66dab7ccd57 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:06.667626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b64ef52e-7ab6-4aaf-b132-f66dab7ccd57 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:06.668054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b64ef52e-7ab6-4aaf-b132-f66dab7ccd57 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:06.820583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b64ef52e-7ab6-4aaf-b132-f66dab7ccd57 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:06.832532 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b64ef52e-7ab6-4aaf-b132-f66dab7ccd57 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.179227 Sep 27 10:43:06.833432 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3559/6648] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:06 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:07.862480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec789623-a17a-4a3a-8388-81f1da6c7a88 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:07.863124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ec789623-a17a-4a3a-8388-81f1da6c7a88 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:07.872148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec789623-a17a-4a3a-8388-81f1da6c7a88 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:07.872802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec789623-a17a-4a3a-8388-81f1da6c7a88 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:07.873344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec789623-a17a-4a3a-8388-81f1da6c7a88 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:07.874202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec789623-a17a-4a3a-8388-81f1da6c7a88 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:07.875026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec789623-a17a-4a3a-8388-81f1da6c7a88 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:07.875553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec789623-a17a-4a3a-8388-81f1da6c7a88 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:08.009960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ec789623-a17a-4a3a-8388-81f1da6c7a88 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:08.018480 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ec789623-a17a-4a3a-8388-81f1da6c7a88 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.161226 Sep 27 10:43:08.019068 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3090/6649] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:07 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:09.045945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-78f2acf2-d899-48d9-8fb0-d7970e2de5e8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:09.047036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-78f2acf2-d899-48d9-8fb0-d7970e2de5e8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:09.058768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78f2acf2-d899-48d9-8fb0-d7970e2de5e8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:09.059591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78f2acf2-d899-48d9-8fb0-d7970e2de5e8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:09.060355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78f2acf2-d899-48d9-8fb0-d7970e2de5e8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:09.062010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78f2acf2-d899-48d9-8fb0-d7970e2de5e8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:09.062585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78f2acf2-d899-48d9-8fb0-d7970e2de5e8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:09.063037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78f2acf2-d899-48d9-8fb0-d7970e2de5e8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:09.205308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-78f2acf2-d899-48d9-8fb0-d7970e2de5e8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:09.217063 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-78f2acf2-d899-48d9-8fb0-d7970e2de5e8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.177124 Sep 27 10:43:09.218036 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3560/6650] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:09 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:10.242275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1c416dde-2b66-4ba5-89f9-594db1228015 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:10.242925 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1c416dde-2b66-4ba5-89f9-594db1228015 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:10.251687 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c416dde-2b66-4ba5-89f9-594db1228015 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:10.252248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c416dde-2b66-4ba5-89f9-594db1228015 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:10.252807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c416dde-2b66-4ba5-89f9-594db1228015 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:10.253587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c416dde-2b66-4ba5-89f9-594db1228015 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:10.254116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c416dde-2b66-4ba5-89f9-594db1228015 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:10.254644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1c416dde-2b66-4ba5-89f9-594db1228015 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:10.400774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1c416dde-2b66-4ba5-89f9-594db1228015 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:10.409147 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1c416dde-2b66-4ba5-89f9-594db1228015 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.171352 Sep 27 10:43:10.409704 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3091/6651] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:10 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:11.436799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c6ee644e-411e-4273-8160-fbfbf8eff1f0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:11.437603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c6ee644e-411e-4273-8160-fbfbf8eff1f0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:11.445318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ee644e-411e-4273-8160-fbfbf8eff1f0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:11.445825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ee644e-411e-4273-8160-fbfbf8eff1f0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:11.446279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ee644e-411e-4273-8160-fbfbf8eff1f0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:11.447013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ee644e-411e-4273-8160-fbfbf8eff1f0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:11.447487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ee644e-411e-4273-8160-fbfbf8eff1f0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:11.447934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c6ee644e-411e-4273-8160-fbfbf8eff1f0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:11.586629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c6ee644e-411e-4273-8160-fbfbf8eff1f0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:11.595170 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c6ee644e-411e-4273-8160-fbfbf8eff1f0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.165458 Sep 27 10:43:11.595748 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3561/6652] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:11 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:12.621104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f02fe57-71f1-43bc-b026-1cdb90f3d2e0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:12.621790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f02fe57-71f1-43bc-b026-1cdb90f3d2e0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:12.629935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f02fe57-71f1-43bc-b026-1cdb90f3d2e0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:12.630432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f02fe57-71f1-43bc-b026-1cdb90f3d2e0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:12.630886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f02fe57-71f1-43bc-b026-1cdb90f3d2e0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:12.631608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f02fe57-71f1-43bc-b026-1cdb90f3d2e0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:12.632075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f02fe57-71f1-43bc-b026-1cdb90f3d2e0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:12.632517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f02fe57-71f1-43bc-b026-1cdb90f3d2e0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:12.771000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8f02fe57-71f1-43bc-b026-1cdb90f3d2e0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:12.779127 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f02fe57-71f1-43bc-b026-1cdb90f3d2e0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.164427 Sep 27 10:43:12.779720 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3092/6653] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:12 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:13.810230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aca106bd-9937-4ac4-b847-c36dc3e52a7d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:13.811421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aca106bd-9937-4ac4-b847-c36dc3e52a7d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:13.823561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aca106bd-9937-4ac4-b847-c36dc3e52a7d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:13.824363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aca106bd-9937-4ac4-b847-c36dc3e52a7d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:13.825125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aca106bd-9937-4ac4-b847-c36dc3e52a7d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:13.826280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aca106bd-9937-4ac4-b847-c36dc3e52a7d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:13.827036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aca106bd-9937-4ac4-b847-c36dc3e52a7d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:13.827732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aca106bd-9937-4ac4-b847-c36dc3e52a7d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:13.969905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aca106bd-9937-4ac4-b847-c36dc3e52a7d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:13.978100 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aca106bd-9937-4ac4-b847-c36dc3e52a7d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.177329 Sep 27 10:43:13.978683 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3562/6654] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:13 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:15.005774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f3548888-2a75-4d54-b8bc-76c032310d64 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:15.006632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f3548888-2a75-4d54-b8bc-76c032310d64 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:15.016133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3548888-2a75-4d54-b8bc-76c032310d64 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:15.016879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3548888-2a75-4d54-b8bc-76c032310d64 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:15.017474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3548888-2a75-4d54-b8bc-76c032310d64 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:15.018420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3548888-2a75-4d54-b8bc-76c032310d64 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:15.019043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3548888-2a75-4d54-b8bc-76c032310d64 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:15.019667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f3548888-2a75-4d54-b8bc-76c032310d64 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:15.161260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f3548888-2a75-4d54-b8bc-76c032310d64 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:15.169559 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f3548888-2a75-4d54-b8bc-76c032310d64 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.169732 Sep 27 10:43:15.170266 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3093/6655] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:14 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:16.197739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f2ea331e-e78e-4f0c-aba9-b2d3bb77acf8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:16.198796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f2ea331e-e78e-4f0c-aba9-b2d3bb77acf8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:16.208269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2ea331e-e78e-4f0c-aba9-b2d3bb77acf8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:16.208791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2ea331e-e78e-4f0c-aba9-b2d3bb77acf8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:16.209230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2ea331e-e78e-4f0c-aba9-b2d3bb77acf8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:16.209945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2ea331e-e78e-4f0c-aba9-b2d3bb77acf8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:16.210403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2ea331e-e78e-4f0c-aba9-b2d3bb77acf8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:16.210831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2ea331e-e78e-4f0c-aba9-b2d3bb77acf8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:16.351938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f2ea331e-e78e-4f0c-aba9-b2d3bb77acf8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:16.364281 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f2ea331e-e78e-4f0c-aba9-b2d3bb77acf8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.172882 Sep 27 10:43:16.365123 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3563/6656] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:16 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:17.396311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-364323c6-c448-49ff-ba9d-382c5d2a0e18 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:17.397376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-364323c6-c448-49ff-ba9d-382c5d2a0e18 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:17.409148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-364323c6-c448-49ff-ba9d-382c5d2a0e18 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:17.410021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-364323c6-c448-49ff-ba9d-382c5d2a0e18 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:17.410721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-364323c6-c448-49ff-ba9d-382c5d2a0e18 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:17.411900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-364323c6-c448-49ff-ba9d-382c5d2a0e18 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:17.412560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-364323c6-c448-49ff-ba9d-382c5d2a0e18 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:17.413223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-364323c6-c448-49ff-ba9d-382c5d2a0e18 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:17.574757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-364323c6-c448-49ff-ba9d-382c5d2a0e18 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:17.583386 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-364323c6-c448-49ff-ba9d-382c5d2a0e18 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.193820 Sep 27 10:43:17.583981 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3094/6657] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:17 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:18.611058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8398904e-71a7-4583-a50f-618b84b12cad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:18.612038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8398904e-71a7-4583-a50f-618b84b12cad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:18.624118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8398904e-71a7-4583-a50f-618b84b12cad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:18.625024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8398904e-71a7-4583-a50f-618b84b12cad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:18.625769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8398904e-71a7-4583-a50f-618b84b12cad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:18.626502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8398904e-71a7-4583-a50f-618b84b12cad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:18.626961 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8398904e-71a7-4583-a50f-618b84b12cad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:18.627417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8398904e-71a7-4583-a50f-618b84b12cad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:18.650235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:18.791473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8398904e-71a7-4583-a50f-618b84b12cad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:18.799552 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8398904e-71a7-4583-a50f-618b84b12cad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.195372 Sep 27 10:43:18.800506 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3564/6658] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:18 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:19.824319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e155596c-3bd9-4966-bd57-8b6e9f52c4c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:19.825026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e155596c-3bd9-4966-bd57-8b6e9f52c4c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:19.834170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e155596c-3bd9-4966-bd57-8b6e9f52c4c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:19.834675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e155596c-3bd9-4966-bd57-8b6e9f52c4c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:19.835125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e155596c-3bd9-4966-bd57-8b6e9f52c4c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:19.835845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e155596c-3bd9-4966-bd57-8b6e9f52c4c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:19.836315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e155596c-3bd9-4966-bd57-8b6e9f52c4c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:19.836806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e155596c-3bd9-4966-bd57-8b6e9f52c4c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:19.985301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e155596c-3bd9-4966-bd57-8b6e9f52c4c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:19.993728 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e155596c-3bd9-4966-bd57-8b6e9f52c4c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.173910 Sep 27 10:43:19.994279 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3095/6659] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:19 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:21.021757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f94e620d-1037-450d-b6a6-caba83a055ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:21.022662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f94e620d-1037-450d-b6a6-caba83a055ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:21.033605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94e620d-1037-450d-b6a6-caba83a055ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:21.034094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94e620d-1037-450d-b6a6-caba83a055ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:21.034534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94e620d-1037-450d-b6a6-caba83a055ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:21.035235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94e620d-1037-450d-b6a6-caba83a055ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:21.035694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94e620d-1037-450d-b6a6-caba83a055ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:21.036126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f94e620d-1037-450d-b6a6-caba83a055ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:21.182973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f94e620d-1037-450d-b6a6-caba83a055ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:21.191612 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f94e620d-1037-450d-b6a6-caba83a055ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.176812 Sep 27 10:43:21.192188 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3565/6660] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:21 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:22.221899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6f11d840-85f1-4624-b908-e37350845cca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:22.223037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6f11d840-85f1-4624-b908-e37350845cca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:22.234404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f11d840-85f1-4624-b908-e37350845cca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:22.235219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f11d840-85f1-4624-b908-e37350845cca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:22.235975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f11d840-85f1-4624-b908-e37350845cca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:22.237202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f11d840-85f1-4624-b908-e37350845cca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:22.237966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f11d840-85f1-4624-b908-e37350845cca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:22.238534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6f11d840-85f1-4624-b908-e37350845cca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:22.253838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:22.393938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6f11d840-85f1-4624-b908-e37350845cca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:22.406385 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6f11d840-85f1-4624-b908-e37350845cca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.190969 Sep 27 10:43:22.407176 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3096/6661] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:22 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:23.433526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2537fd41-47ea-4c26-84f2-27c0d56e7cfc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:23.434222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2537fd41-47ea-4c26-84f2-27c0d56e7cfc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:23.443437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2537fd41-47ea-4c26-84f2-27c0d56e7cfc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:23.444049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2537fd41-47ea-4c26-84f2-27c0d56e7cfc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:23.444605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2537fd41-47ea-4c26-84f2-27c0d56e7cfc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:23.445560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2537fd41-47ea-4c26-84f2-27c0d56e7cfc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:23.446150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2537fd41-47ea-4c26-84f2-27c0d56e7cfc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:23.446711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2537fd41-47ea-4c26-84f2-27c0d56e7cfc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:23.584279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2537fd41-47ea-4c26-84f2-27c0d56e7cfc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:23.592706 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2537fd41-47ea-4c26-84f2-27c0d56e7cfc tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.164237 Sep 27 10:43:23.593304 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3566/6662] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:23 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:24.621943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ad34eed9-7c1e-4ac7-965a-a56073e6f5c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:24.622935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ad34eed9-7c1e-4ac7-965a-a56073e6f5c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:24.633121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad34eed9-7c1e-4ac7-965a-a56073e6f5c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:24.633638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad34eed9-7c1e-4ac7-965a-a56073e6f5c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:24.634086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad34eed9-7c1e-4ac7-965a-a56073e6f5c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:24.634822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad34eed9-7c1e-4ac7-965a-a56073e6f5c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:24.635290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad34eed9-7c1e-4ac7-965a-a56073e6f5c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:24.635717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad34eed9-7c1e-4ac7-965a-a56073e6f5c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:24.783065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ad34eed9-7c1e-4ac7-965a-a56073e6f5c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:24.792273 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ad34eed9-7c1e-4ac7-965a-a56073e6f5c4 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.177279 Sep 27 10:43:24.792910 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3097/6663] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:24 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:25.817342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-85d83f3f-954c-4ef7-a7b2-1edb43434dae tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:25.818150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-85d83f3f-954c-4ef7-a7b2-1edb43434dae tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:25.827136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85d83f3f-954c-4ef7-a7b2-1edb43434dae tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:25.827741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85d83f3f-954c-4ef7-a7b2-1edb43434dae tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:25.828282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85d83f3f-954c-4ef7-a7b2-1edb43434dae tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:25.829210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85d83f3f-954c-4ef7-a7b2-1edb43434dae tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:25.829783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85d83f3f-954c-4ef7-a7b2-1edb43434dae tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:25.830308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85d83f3f-954c-4ef7-a7b2-1edb43434dae tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:25.971813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-85d83f3f-954c-4ef7-a7b2-1edb43434dae tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:25.979914 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-85d83f3f-954c-4ef7-a7b2-1edb43434dae tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.167658 Sep 27 10:43:25.980476 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3567/6664] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:25 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:27.006407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-73b8ac91-9bfe-43fd-bb83-8c7438f277f5 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:27.007017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-73b8ac91-9bfe-43fd-bb83-8c7438f277f5 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:27.009807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=36 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:27.015018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73b8ac91-9bfe-43fd-bb83-8c7438f277f5 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:27.015519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73b8ac91-9bfe-43fd-bb83-8c7438f277f5 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:27.015946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73b8ac91-9bfe-43fd-bb83-8c7438f277f5 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:27.016684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73b8ac91-9bfe-43fd-bb83-8c7438f277f5 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:27.017170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73b8ac91-9bfe-43fd-bb83-8c7438f277f5 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:27.017614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-73b8ac91-9bfe-43fd-bb83-8c7438f277f5 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:27.163533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-73b8ac91-9bfe-43fd-bb83-8c7438f277f5 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:27.175680 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-73b8ac91-9bfe-43fd-bb83-8c7438f277f5 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.174571 Sep 27 10:43:27.176483 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3098/6665] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:26 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:28.199877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4499182a-417c-4066-8930-ac71da6bf806 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:28.200487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4499182a-417c-4066-8930-ac71da6bf806 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:28.203237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=32 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:28.209083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4499182a-417c-4066-8930-ac71da6bf806 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:28.209570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4499182a-417c-4066-8930-ac71da6bf806 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:28.210031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4499182a-417c-4066-8930-ac71da6bf806 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:28.210700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4499182a-417c-4066-8930-ac71da6bf806 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:28.211147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4499182a-417c-4066-8930-ac71da6bf806 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:28.211572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4499182a-417c-4066-8930-ac71da6bf806 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:28.345683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4499182a-417c-4066-8930-ac71da6bf806 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:28.353755 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4499182a-417c-4066-8930-ac71da6bf806 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.157988 Sep 27 10:43:28.354339 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3568/6666] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:28 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:29.382125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ed95a198-081e-4cbf-86d9-99287cdf38a7 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:29.383173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ed95a198-081e-4cbf-86d9-99287cdf38a7 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:29.394855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed95a198-081e-4cbf-86d9-99287cdf38a7 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:29.395810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed95a198-081e-4cbf-86d9-99287cdf38a7 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:29.396384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed95a198-081e-4cbf-86d9-99287cdf38a7 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:29.397184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed95a198-081e-4cbf-86d9-99287cdf38a7 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:29.397662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed95a198-081e-4cbf-86d9-99287cdf38a7 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:29.398101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ed95a198-081e-4cbf-86d9-99287cdf38a7 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:29.817637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ed95a198-081e-4cbf-86d9-99287cdf38a7 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:29.826375 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ed95a198-081e-4cbf-86d9-99287cdf38a7 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.450876 Sep 27 10:43:29.826964 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3099/6667] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:29 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:30.853234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7fdb1a22-8e7d-4691-95b5-6eb52aca03ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:30.854132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7fdb1a22-8e7d-4691-95b5-6eb52aca03ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:30.865138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fdb1a22-8e7d-4691-95b5-6eb52aca03ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:30.865624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fdb1a22-8e7d-4691-95b5-6eb52aca03ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:30.866060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fdb1a22-8e7d-4691-95b5-6eb52aca03ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:30.866783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fdb1a22-8e7d-4691-95b5-6eb52aca03ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:30.867241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fdb1a22-8e7d-4691-95b5-6eb52aca03ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:30.867673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7fdb1a22-8e7d-4691-95b5-6eb52aca03ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:31.009427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7fdb1a22-8e7d-4691-95b5-6eb52aca03ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:31.017398 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7fdb1a22-8e7d-4691-95b5-6eb52aca03ad tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1522 microversion: 2.1 time: 0.170386 Sep 27 10:43:31.018077 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3569/6668] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:30 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:31.734584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-841afb92-4835-4b37-a748-9535b552e59c req-c3885c9c-3090-4046-aa23-1c7a0b553049 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "25c83c81-2c6e-4d30-81b4-8c71b4d6dce7", "name": "network-vif-unplugged", "status": "completed", "tag": "9687b9f1-54e8-4a5d-a2e8-1007041e00be"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:43:31.744862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-841afb92-4835-4b37-a748-9535b552e59c req-c3885c9c-3090-4046-aa23-1c7a0b553049 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:31.745359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-841afb92-4835-4b37-a748-9535b552e59c req-c3885c9c-3090-4046-aa23-1c7a0b553049 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:31.745809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-841afb92-4835-4b37-a748-9535b552e59c req-c3885c9c-3090-4046-aa23-1c7a0b553049 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:31.758560 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-841afb92-4835-4b37-a748-9535b552e59c req-c3885c9c-3090-4046-aa23-1c7a0b553049 service nova] Creating event network-vif-unplugged:9687b9f1-54e8-4a5d-a2e8-1007041e00be for instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 on nch2-master-1 Sep 27 10:43:31.764149 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-841afb92-4835-4b37-a748-9535b552e59c req-c3885c9c-3090-4046-aa23-1c7a0b553049 service nova] Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:43:31.776441 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-841afb92-4835-4b37-a748-9535b552e59c req-c3885c9c-3090-4046-aa23-1c7a0b553049 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.115116 Sep 27 10:43:31.777546 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3100/6669] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:43:31 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:43:32.042021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5fb58067-65a3-46de-9b86-5dd7bdbbb47f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:32.042630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5fb58067-65a3-46de-9b86-5dd7bdbbb47f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:32.048937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb58067-65a3-46de-9b86-5dd7bdbbb47f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:32.049762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb58067-65a3-46de-9b86-5dd7bdbbb47f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:32.050202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb58067-65a3-46de-9b86-5dd7bdbbb47f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:32.050929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb58067-65a3-46de-9b86-5dd7bdbbb47f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:32.051388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb58067-65a3-46de-9b86-5dd7bdbbb47f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:32.051819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5fb58067-65a3-46de-9b86-5dd7bdbbb47f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:32.188303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5fb58067-65a3-46de-9b86-5dd7bdbbb47f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:32.196441 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5fb58067-65a3-46de-9b86-5dd7bdbbb47f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1521 microversion: 2.1 time: 0.160201 Sep 27 10:43:32.205758 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3570/6670] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:32 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:33.227019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-53068c94-f27f-408e-bff3-820d92c7fa40 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:33.227894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-53068c94-f27f-408e-bff3-820d92c7fa40 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:33.237541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53068c94-f27f-408e-bff3-820d92c7fa40 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:33.238333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53068c94-f27f-408e-bff3-820d92c7fa40 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:33.239034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53068c94-f27f-408e-bff3-820d92c7fa40 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:33.240139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53068c94-f27f-408e-bff3-820d92c7fa40 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:33.240880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53068c94-f27f-408e-bff3-820d92c7fa40 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:33.241559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-53068c94-f27f-408e-bff3-820d92c7fa40 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:33.378078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-53068c94-f27f-408e-bff3-820d92c7fa40 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:33.387093 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-53068c94-f27f-408e-bff3-820d92c7fa40 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1519 microversion: 2.1 time: 0.169334 Sep 27 10:43:33.387646 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3101/6671] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:33 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1519 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:33.799512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-1e22f142-7824-4ce7-bd79-de59d0a23390 req-d6f5d442-d66a-4fa9-aeaf-a69bb7633efa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "25c83c81-2c6e-4d30-81b4-8c71b4d6dce7", "tag": "9687b9f1-54e8-4a5d-a2e8-1007041e00be"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:43:33.810543 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1e22f142-7824-4ce7-bd79-de59d0a23390 req-d6f5d442-d66a-4fa9-aeaf-a69bb7633efa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:33.811015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1e22f142-7824-4ce7-bd79-de59d0a23390 req-d6f5d442-d66a-4fa9-aeaf-a69bb7633efa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:33.811467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1e22f142-7824-4ce7-bd79-de59d0a23390 req-d6f5d442-d66a-4fa9-aeaf-a69bb7633efa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:33.823316 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-1e22f142-7824-4ce7-bd79-de59d0a23390 req-d6f5d442-d66a-4fa9-aeaf-a69bb7633efa service nova] Creating event network-changed:9687b9f1-54e8-4a5d-a2e8-1007041e00be for instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 on nch2-master-1 Sep 27 10:43:33.829168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-1e22f142-7824-4ce7-bd79-de59d0a23390 req-d6f5d442-d66a-4fa9-aeaf-a69bb7633efa service nova] Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:43:33.840485 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-1e22f142-7824-4ce7-bd79-de59d0a23390 req-d6f5d442-d66a-4fa9-aeaf-a69bb7633efa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044994 Sep 27 10:43:33.841096 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3571/6672] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:43:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:43:34.411304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bd397e88-2199-481f-8338-fee63252844f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:34.411927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bd397e88-2199-481f-8338-fee63252844f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:34.419854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd397e88-2199-481f-8338-fee63252844f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:34.420358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd397e88-2199-481f-8338-fee63252844f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:34.420854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd397e88-2199-481f-8338-fee63252844f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:34.421580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd397e88-2199-481f-8338-fee63252844f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:34.422121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd397e88-2199-481f-8338-fee63252844f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:34.422503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd397e88-2199-481f-8338-fee63252844f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:34.565986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bd397e88-2199-481f-8338-fee63252844f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:34.574524 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bd397e88-2199-481f-8338-fee63252844f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1519 microversion: 2.1 time: 0.167271 Sep 27 10:43:34.575106 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3102/6673] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:34 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1519 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:35.600396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4c57bc7-95d9-45f3-bf1b-423707ab4326 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:35.602213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d4c57bc7-95d9-45f3-bf1b-423707ab4326 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:35.609595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c57bc7-95d9-45f3-bf1b-423707ab4326 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:35.610482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c57bc7-95d9-45f3-bf1b-423707ab4326 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:35.611167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c57bc7-95d9-45f3-bf1b-423707ab4326 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:35.612266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c57bc7-95d9-45f3-bf1b-423707ab4326 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:35.613133 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c57bc7-95d9-45f3-bf1b-423707ab4326 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:35.613994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4c57bc7-95d9-45f3-bf1b-423707ab4326 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:35.761148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d4c57bc7-95d9-45f3-bf1b-423707ab4326 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:35.769894 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4c57bc7-95d9-45f3-bf1b-423707ab4326 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1516 microversion: 2.1 time: 0.175628 Sep 27 10:43:35.770465 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3572/6674] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:35 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1516 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:35.792846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2652cd8e-1718-4192-8f56-a911ae8deafe tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:43:35.794139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2652cd8e-1718-4192-8f56-a911ae8deafe tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:35.801008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2652cd8e-1718-4192-8f56-a911ae8deafe tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:35.801513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2652cd8e-1718-4192-8f56-a911ae8deafe tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:35.801969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2652cd8e-1718-4192-8f56-a911ae8deafe tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:35.802697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2652cd8e-1718-4192-8f56-a911ae8deafe tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:35.803170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2652cd8e-1718-4192-8f56-a911ae8deafe tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:35.803615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2652cd8e-1718-4192-8f56-a911ae8deafe tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:35.867578 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2652cd8e-1718-4192-8f56-a911ae8deafe tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "POST /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7/action" status: 204 len: 0 microversion: 2.1 time: 0.078530 Sep 27 10:43:35.868352 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3103/6675] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:43:35 2022] POST /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7/action => generated 0 bytes in 80 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:43:35.877262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-14b4f5d5-e625-4bf4-b3fc-dbd7f8597a84 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:35.877874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-14b4f5d5-e625-4bf4-b3fc-dbd7f8597a84 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:35.884406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14b4f5d5-e625-4bf4-b3fc-dbd7f8597a84 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:35.884995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14b4f5d5-e625-4bf4-b3fc-dbd7f8597a84 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:35.885458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14b4f5d5-e625-4bf4-b3fc-dbd7f8597a84 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:35.886157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14b4f5d5-e625-4bf4-b3fc-dbd7f8597a84 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:35.886797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14b4f5d5-e625-4bf4-b3fc-dbd7f8597a84 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:35.887891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14b4f5d5-e625-4bf4-b3fc-dbd7f8597a84 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:36.026184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-14b4f5d5-e625-4bf4-b3fc-dbd7f8597a84 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:36.035015 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-14b4f5d5-e625-4bf4-b3fc-dbd7f8597a84 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1516 microversion: 2.1 time: 0.161836 Sep 27 10:43:36.035926 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3573/6676] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:35 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1516 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:43:37.059667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-befec709-c996-41c1-bd66-a1d18a8b0080 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:37.060386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-befec709-c996-41c1-bd66-a1d18a8b0080 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:37.067101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-befec709-c996-41c1-bd66-a1d18a8b0080 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:37.067612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-befec709-c996-41c1-bd66-a1d18a8b0080 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:37.068069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-befec709-c996-41c1-bd66-a1d18a8b0080 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:37.068826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-befec709-c996-41c1-bd66-a1d18a8b0080 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:37.069305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-befec709-c996-41c1-bd66-a1d18a8b0080 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:37.069752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-befec709-c996-41c1-bd66-a1d18a8b0080 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:37.206308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-befec709-c996-41c1-bd66-a1d18a8b0080 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:37.214924 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-befec709-c996-41c1-bd66-a1d18a8b0080 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1508 microversion: 2.1 time: 0.160375 Sep 27 10:43:37.215737 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3104/6677] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:37 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1508 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:37.238347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7bea421b-158e-4c5e-a42d-8eca7a606518 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:37.238965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7bea421b-158e-4c5e-a42d-8eca7a606518 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:37.246168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bea421b-158e-4c5e-a42d-8eca7a606518 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:37.246653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bea421b-158e-4c5e-a42d-8eca7a606518 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:37.247102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bea421b-158e-4c5e-a42d-8eca7a606518 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:37.247801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bea421b-158e-4c5e-a42d-8eca7a606518 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:37.248255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bea421b-158e-4c5e-a42d-8eca7a606518 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:37.248705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7bea421b-158e-4c5e-a42d-8eca7a606518 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:37.394332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7bea421b-158e-4c5e-a42d-8eca7a606518 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:37.406705 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7bea421b-158e-4c5e-a42d-8eca7a606518 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1508 microversion: 2.1 time: 0.172080 Sep 27 10:43:37.407465 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3574/6678] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:37 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1508 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:37.435570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-48efcda7-beae-4f21-a495-5fa784d77273 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Action: 'create', calling method: .version_select at 0x7f7f6251c160>, body: {"keypair": {"name": "tempest-keypair-1855325960", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK0FJWfjnE4NCD4x8gQor9RBTtdWna1u+VVtPZSSbcvtzlCOsDDzhtt3C5FIBNxtlpG0PvZ0oStXMmCAtTPnw7chLdzNPAszBrWf0qTQwgadRWQ0sP3U3PC0WwP3HeHUkg=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:43:37.452099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-48efcda7-beae-4f21-a495-5fa784d77273 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Getting quotas for project 063971e11eb044b4a844a563c6ccbc5d. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:43:37.457834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-48efcda7-beae-4f21-a495-5fa784d77273 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Getting quotas for user 74e30240d7c840edba43dddf3a862c4f and project 063971e11eb044b4a844a563c6ccbc5d. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:43:37.488950 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-48efcda7-beae-4f21-a495-5fa784d77273 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.059376 Sep 27 10:43:37.489550 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3105/6679] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:43:37 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:43:38.405491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-440258c1-af70-454d-99db-1534ad5b7f9b req-aa880167-9f69-487b-9301-a1b1bd32ae14 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "25c83c81-2c6e-4d30-81b4-8c71b4d6dce7", "name": "network-vif-plugged", "status": "completed", "tag": "9687b9f1-54e8-4a5d-a2e8-1007041e00be"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:43:38.415347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-440258c1-af70-454d-99db-1534ad5b7f9b req-aa880167-9f69-487b-9301-a1b1bd32ae14 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:38.415832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-440258c1-af70-454d-99db-1534ad5b7f9b req-aa880167-9f69-487b-9301-a1b1bd32ae14 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:38.416264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-440258c1-af70-454d-99db-1534ad5b7f9b req-aa880167-9f69-487b-9301-a1b1bd32ae14 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:38.428041 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-440258c1-af70-454d-99db-1534ad5b7f9b req-aa880167-9f69-487b-9301-a1b1bd32ae14 service nova] Creating event network-vif-plugged:9687b9f1-54e8-4a5d-a2e8-1007041e00be for instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 on nch2-master-1 Sep 27 10:43:38.433416 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-440258c1-af70-454d-99db-1534ad5b7f9b req-aa880167-9f69-487b-9301-a1b1bd32ae14 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120731 Sep 27 10:43:38.434133 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3575/6680] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:43:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:43:38.796895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec1ef479-621d-4e9d-acc4-b50910bb3b08 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:38.797545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ec1ef479-621d-4e9d-acc4-b50910bb3b08 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:38.807543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec1ef479-621d-4e9d-acc4-b50910bb3b08 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:38.808044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec1ef479-621d-4e9d-acc4-b50910bb3b08 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:38.808701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec1ef479-621d-4e9d-acc4-b50910bb3b08 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:38.809591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec1ef479-621d-4e9d-acc4-b50910bb3b08 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:38.810069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec1ef479-621d-4e9d-acc4-b50910bb3b08 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:38.810509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec1ef479-621d-4e9d-acc4-b50910bb3b08 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:38.955759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ec1ef479-621d-4e9d-acc4-b50910bb3b08 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:38.964087 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ec1ef479-621d-4e9d-acc4-b50910bb3b08 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 200 len: 1508 microversion: 2.1 time: 0.172929 Sep 27 10:43:38.964724 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3106/6681] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:38 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 1508 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:38.988316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-580f61ee-a0d8-4a1e-9e91-5a414c04b08f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:38.988946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-580f61ee-a0d8-4a1e-9e91-5a414c04b08f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:38.995573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-580f61ee-a0d8-4a1e-9e91-5a414c04b08f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:38.996710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-580f61ee-a0d8-4a1e-9e91-5a414c04b08f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:38.997191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-580f61ee-a0d8-4a1e-9e91-5a414c04b08f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:38.997902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-580f61ee-a0d8-4a1e-9e91-5a414c04b08f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:38.998361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-580f61ee-a0d8-4a1e-9e91-5a414c04b08f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:38.998786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-580f61ee-a0d8-4a1e-9e91-5a414c04b08f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:39.025675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-580f61ee-a0d8-4a1e-9e91-5a414c04b08f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:43:39.111937 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-580f61ee-a0d8-4a1e-9e91-5a414c04b08f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "DELETE /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 204 len: 0 microversion: 2.1 time: 0.127767 Sep 27 10:43:39.113539 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3576/6682] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:43:38 2022] DELETE /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:43:39.177157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerDiskConfigTestJSON-server-1913994864", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "861c59c3-0005-427d-877a-900443c2925b"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:43:39.234977 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] validate_networks() for [('861c59c3-0005-427d-877a-900443c2925b', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:43:39.750661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:43:39.751362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:43:39.752110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:43:39.752870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:43:39.753505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:43:39.767543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:39.768020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:39.768881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:39.770181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:39.770678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:39.771436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:39.796203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Getting quotas for project 063971e11eb044b4a844a563c6ccbc5d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:43:39.802376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Getting quotas for user 74e30240d7c840edba43dddf3a862c4f and project 063971e11eb044b4a844a563c6ccbc5d. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:43:39.815816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:43:39.818250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:43:39.865792 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5cbccbb4-75a9-471e-82fa-878553318e25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.695182 Sep 27 10:43:39.866481 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3107/6683] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:43:39 2022] POST /compute/v2.1/servers => generated 380 bytes in 697 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:43:39.878540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90259ec2-e76d-447d-9eda-2ffc62b1a386 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:39.879542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-90259ec2-e76d-447d-9eda-2ffc62b1a386 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:39.939419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-90259ec2-e76d-447d-9eda-2ffc62b1a386 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:39.947797 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-90259ec2-e76d-447d-9eda-2ffc62b1a386 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1218 microversion: 2.1 time: 0.073308 Sep 27 10:43:39.948050 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3577/6684] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:39 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1218 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:40.972262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-07201d39-dc23-4639-8cf7-32f48bb6a132 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:40.973002 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-07201d39-dc23-4639-8cf7-32f48bb6a132 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:40.980515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07201d39-dc23-4639-8cf7-32f48bb6a132 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:40.981144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07201d39-dc23-4639-8cf7-32f48bb6a132 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:40.981664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07201d39-dc23-4639-8cf7-32f48bb6a132 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:40.982444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07201d39-dc23-4639-8cf7-32f48bb6a132 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:40.982979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07201d39-dc23-4639-8cf7-32f48bb6a132 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:40.983480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-07201d39-dc23-4639-8cf7-32f48bb6a132 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:41.054683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-07201d39-dc23-4639-8cf7-32f48bb6a132 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:41.063173 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-07201d39-dc23-4639-8cf7-32f48bb6a132 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1270 microversion: 2.1 time: 0.096213 Sep 27 10:43:41.063721 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3108/6685] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:40 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1270 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:41.227185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ea578966-0ed6-4e14-9536-ebc707f024c3 req-de98dbe2-ed3a-4bc7-b635-368952a77f7d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "25c83c81-2c6e-4d30-81b4-8c71b4d6dce7", "name": "network-vif-deleted", "tag": "9687b9f1-54e8-4a5d-a2e8-1007041e00be"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:43:41.239715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ea578966-0ed6-4e14-9536-ebc707f024c3 req-de98dbe2-ed3a-4bc7-b635-368952a77f7d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:41.240238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ea578966-0ed6-4e14-9536-ebc707f024c3 req-de98dbe2-ed3a-4bc7-b635-368952a77f7d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:41.240771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ea578966-0ed6-4e14-9536-ebc707f024c3 req-de98dbe2-ed3a-4bc7-b635-368952a77f7d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:41.254429 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ea578966-0ed6-4e14-9536-ebc707f024c3 req-de98dbe2-ed3a-4bc7-b635-368952a77f7d service nova] Creating event network-vif-deleted:9687b9f1-54e8-4a5d-a2e8-1007041e00be for instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 on nch2-master-1 Sep 27 10:43:41.259691 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ea578966-0ed6-4e14-9536-ebc707f024c3 req-de98dbe2-ed3a-4bc7-b635-368952a77f7d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036582 Sep 27 10:43:41.260313 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3578/6686] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:43:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:43:42.086956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-72fb73ec-dc5d-469b-9487-e3a1958d1efa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:42.087741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-72fb73ec-dc5d-469b-9487-e3a1958d1efa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:42.095270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72fb73ec-dc5d-469b-9487-e3a1958d1efa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:42.095807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72fb73ec-dc5d-469b-9487-e3a1958d1efa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:42.096266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72fb73ec-dc5d-469b-9487-e3a1958d1efa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:42.097044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72fb73ec-dc5d-469b-9487-e3a1958d1efa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:42.097636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72fb73ec-dc5d-469b-9487-e3a1958d1efa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:42.098000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72fb73ec-dc5d-469b-9487-e3a1958d1efa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:42.287028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-72fb73ec-dc5d-469b-9487-e3a1958d1efa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:42.296092 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-72fb73ec-dc5d-469b-9487-e3a1958d1efa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1318 microversion: 2.1 time: 0.213078 Sep 27 10:43:42.296728 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3109/6687] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:42 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1318 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:43.278792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c9dd78b4-675a-44d0-858e-09cc5fe697e0 req-6ca3eedd-2585-457c-9cd0-37f0871f8e5a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ccbfee66-bd3c-41ec-b09c-9b2b384d1229", "tag": "0681f2f1-ab1f-4cc4-930b-e48076441ec4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:43:43.288540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9dd78b4-675a-44d0-858e-09cc5fe697e0 req-6ca3eedd-2585-457c-9cd0-37f0871f8e5a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:43.289282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9dd78b4-675a-44d0-858e-09cc5fe697e0 req-6ca3eedd-2585-457c-9cd0-37f0871f8e5a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:43.289935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c9dd78b4-675a-44d0-858e-09cc5fe697e0 req-6ca3eedd-2585-457c-9cd0-37f0871f8e5a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:43.301590 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c9dd78b4-675a-44d0-858e-09cc5fe697e0 req-6ca3eedd-2585-457c-9cd0-37f0871f8e5a service nova] Creating event network-changed:0681f2f1-ab1f-4cc4-930b-e48076441ec4 for instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 on nch2-master-1 Sep 27 10:43:43.306844 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c9dd78b4-675a-44d0-858e-09cc5fe697e0 req-6ca3eedd-2585-457c-9cd0-37f0871f8e5a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032691 Sep 27 10:43:43.308083 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3579/6688] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:43:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:43:43.319374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a0a73d75-774b-421e-a3a3-f9a688253a9d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:43.320585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a0a73d75-774b-421e-a3a3-f9a688253a9d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:43.327867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a73d75-774b-421e-a3a3-f9a688253a9d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:43.328377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a73d75-774b-421e-a3a3-f9a688253a9d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:43.328869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a73d75-774b-421e-a3a3-f9a688253a9d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:43.329598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a73d75-774b-421e-a3a3-f9a688253a9d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:43.330073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a73d75-774b-421e-a3a3-f9a688253a9d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:43.330516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a0a73d75-774b-421e-a3a3-f9a688253a9d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:43.760419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a0a73d75-774b-421e-a3a3-f9a688253a9d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:43.768599 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a0a73d75-774b-421e-a3a3-f9a688253a9d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1318 microversion: 2.1 time: 0.453042 Sep 27 10:43:43.769174 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3110/6689] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:43 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1318 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:44.795619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-250f1cec-0475-439f-a477-945ba5cd26d1 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:44.796311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-250f1cec-0475-439f-a477-945ba5cd26d1 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:44.804113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-250f1cec-0475-439f-a477-945ba5cd26d1 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:44.804588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-250f1cec-0475-439f-a477-945ba5cd26d1 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:44.805051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-250f1cec-0475-439f-a477-945ba5cd26d1 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:44.805752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-250f1cec-0475-439f-a477-945ba5cd26d1 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:44.806195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-250f1cec-0475-439f-a477-945ba5cd26d1 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:44.806615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-250f1cec-0475-439f-a477-945ba5cd26d1 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:44.964826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-250f1cec-0475-439f-a477-945ba5cd26d1 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:44.973161 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-250f1cec-0475-439f-a477-945ba5cd26d1 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1318 microversion: 2.1 time: 0.183522 Sep 27 10:43:44.973772 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3580/6690] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:44 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1318 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:45.999009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-913e1da3-5885-45b8-918b-db03aab4fc78 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:45.999564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-913e1da3-5885-45b8-918b-db03aab4fc78 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:46.007371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-913e1da3-5885-45b8-918b-db03aab4fc78 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:46.007881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-913e1da3-5885-45b8-918b-db03aab4fc78 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:46.008324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-913e1da3-5885-45b8-918b-db03aab4fc78 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:46.009104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-913e1da3-5885-45b8-918b-db03aab4fc78 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:46.009551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-913e1da3-5885-45b8-918b-db03aab4fc78 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:46.009984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-913e1da3-5885-45b8-918b-db03aab4fc78 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:46.149246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-913e1da3-5885-45b8-918b-db03aab4fc78 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:46.157459 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-913e1da3-5885-45b8-918b-db03aab4fc78 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1318 microversion: 2.1 time: 0.163507 Sep 27 10:43:46.158039 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3111/6691] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:45 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1318 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:47.184966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb799717-6687-492c-8e27-705e72acd45d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:47.185577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cb799717-6687-492c-8e27-705e72acd45d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:47.193163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb799717-6687-492c-8e27-705e72acd45d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:47.193654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb799717-6687-492c-8e27-705e72acd45d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:47.194116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb799717-6687-492c-8e27-705e72acd45d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:47.194857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb799717-6687-492c-8e27-705e72acd45d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:47.195335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb799717-6687-492c-8e27-705e72acd45d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:47.195763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb799717-6687-492c-8e27-705e72acd45d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:47.326690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb799717-6687-492c-8e27-705e72acd45d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:47.335138 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cb799717-6687-492c-8e27-705e72acd45d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1318 microversion: 2.1 time: 0.156724 Sep 27 10:43:47.335720 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3581/6692] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:47 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1318 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:48.270646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-88c079f0-1704-4a44-850f-665b05fc00e3 req-2632abd8-dd79-4c3e-b3c6-25d9a2239d28 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ccbfee66-bd3c-41ec-b09c-9b2b384d1229", "name": "network-vif-plugged", "status": "completed", "tag": "0681f2f1-ab1f-4cc4-930b-e48076441ec4"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:43:48.284202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-88c079f0-1704-4a44-850f-665b05fc00e3 req-2632abd8-dd79-4c3e-b3c6-25d9a2239d28 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:48.284723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-88c079f0-1704-4a44-850f-665b05fc00e3 req-2632abd8-dd79-4c3e-b3c6-25d9a2239d28 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:48.285177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-88c079f0-1704-4a44-850f-665b05fc00e3 req-2632abd8-dd79-4c3e-b3c6-25d9a2239d28 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:48.296595 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-88c079f0-1704-4a44-850f-665b05fc00e3 req-2632abd8-dd79-4c3e-b3c6-25d9a2239d28 service nova] Creating event network-vif-plugged:0681f2f1-ab1f-4cc4-930b-e48076441ec4 for instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 on nch2-master-1 Sep 27 10:43:48.302299 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-88c079f0-1704-4a44-850f-665b05fc00e3 req-2632abd8-dd79-4c3e-b3c6-25d9a2239d28 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035491 Sep 27 10:43:48.303257 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3112/6693] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:43:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:43:48.359577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a2486236-e18c-4b8a-b890-fb782a05de4d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:48.360305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a2486236-e18c-4b8a-b890-fb782a05de4d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:48.367492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2486236-e18c-4b8a-b890-fb782a05de4d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:48.368072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2486236-e18c-4b8a-b890-fb782a05de4d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:48.368577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2486236-e18c-4b8a-b890-fb782a05de4d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:48.369316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2486236-e18c-4b8a-b890-fb782a05de4d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:48.369818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2486236-e18c-4b8a-b890-fb782a05de4d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:48.370293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a2486236-e18c-4b8a-b890-fb782a05de4d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:48.512892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a2486236-e18c-4b8a-b890-fb782a05de4d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:48.517037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:48.521959 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a2486236-e18c-4b8a-b890-fb782a05de4d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1318 microversion: 2.1 time: 0.167871 Sep 27 10:43:48.522546 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3582/6694] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:48 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1318 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:49.551699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-845ecd66-003c-4221-b2eb-0755b1556fba tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:49.552298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-845ecd66-003c-4221-b2eb-0755b1556fba tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:49.563705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-845ecd66-003c-4221-b2eb-0755b1556fba tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:49.564209 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-845ecd66-003c-4221-b2eb-0755b1556fba tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:49.564641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-845ecd66-003c-4221-b2eb-0755b1556fba tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:49.565389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-845ecd66-003c-4221-b2eb-0755b1556fba tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:49.565856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-845ecd66-003c-4221-b2eb-0755b1556fba tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:49.566292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-845ecd66-003c-4221-b2eb-0755b1556fba tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:49.709838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-845ecd66-003c-4221-b2eb-0755b1556fba tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:49.722205 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-845ecd66-003c-4221-b2eb-0755b1556fba tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1318 microversion: 2.1 time: 0.179072 Sep 27 10:43:49.722962 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3113/6695] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:49 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1318 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:50.752606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53fe3053-0b6b-497d-9046-3b8eb83ea72e tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:50.753616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53fe3053-0b6b-497d-9046-3b8eb83ea72e tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:50.758424 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:50.767706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53fe3053-0b6b-497d-9046-3b8eb83ea72e tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:50.768223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53fe3053-0b6b-497d-9046-3b8eb83ea72e tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:50.768629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53fe3053-0b6b-497d-9046-3b8eb83ea72e tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:50.769378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53fe3053-0b6b-497d-9046-3b8eb83ea72e tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:50.769909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53fe3053-0b6b-497d-9046-3b8eb83ea72e tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:50.770357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53fe3053-0b6b-497d-9046-3b8eb83ea72e tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:50.912804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-53fe3053-0b6b-497d-9046-3b8eb83ea72e tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:50.923235 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53fe3053-0b6b-497d-9046-3b8eb83ea72e tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1507 microversion: 2.1 time: 0.176979 Sep 27 10:43:50.924317 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3583/6696] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:50 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1507 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:50.947036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eeb61999-2547-4c39-90cb-ab4707d9dd4f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:50.947664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eeb61999-2547-4c39-90cb-ab4707d9dd4f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:50.949683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:50.950358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=10 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:50.957184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb61999-2547-4c39-90cb-ab4707d9dd4f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:50.957677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb61999-2547-4c39-90cb-ab4707d9dd4f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:50.958124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb61999-2547-4c39-90cb-ab4707d9dd4f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:50.958822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb61999-2547-4c39-90cb-ab4707d9dd4f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:50.959279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb61999-2547-4c39-90cb-ab4707d9dd4f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:50.959715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeb61999-2547-4c39-90cb-ab4707d9dd4f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:50.978218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:51.100797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eeb61999-2547-4c39-90cb-ab4707d9dd4f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:51.108739 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eeb61999-2547-4c39-90cb-ab4707d9dd4f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1507 microversion: 2.1 time: 0.165596 Sep 27 10:43:51.109463 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3114/6697] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:50 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1507 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:51.131255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451", "OS-DCF:diskConfig": "AUTO"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:43:51.133679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:51.140035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:51.140565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:51.141038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:51.141739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:51.142195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:51.142622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:51.171599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:51.179555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Old instance type m1.heat2, new instance type m1.heat {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:43:51.186387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:43:51.186812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:43:51.187332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:43:51.187749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:43:51.188161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:43:51.257254 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-53bead77-dd2f-446d-ad83-c67739593ef6 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "POST /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229/action" status: 202 len: 0 microversion: 2.1 time: 0.130011 Sep 27 10:43:51.261237 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3584/6698] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:43:51 2022] POST /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229/action => generated 0 bytes in 135 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:43:51.266603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c5e43f4-4a32-4b7f-b5db-89778ed5f21f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:51.267227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c5e43f4-4a32-4b7f-b5db-89778ed5f21f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:51.275690 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c5e43f4-4a32-4b7f-b5db-89778ed5f21f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:51.276194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c5e43f4-4a32-4b7f-b5db-89778ed5f21f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:51.276647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c5e43f4-4a32-4b7f-b5db-89778ed5f21f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:51.277397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c5e43f4-4a32-4b7f-b5db-89778ed5f21f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:51.277866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c5e43f4-4a32-4b7f-b5db-89778ed5f21f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:51.278304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c5e43f4-4a32-4b7f-b5db-89778ed5f21f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:51.417057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c5e43f4-4a32-4b7f-b5db-89778ed5f21f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:51.425973 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c5e43f4-4a32-4b7f-b5db-89778ed5f21f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1514 microversion: 2.1 time: 0.163541 Sep 27 10:43:51.426557 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3115/6699] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:51 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1514 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:43:52.450598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5884721f-9cb1-4283-90cc-57d4140c6dce tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:52.451305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5884721f-9cb1-4283-90cc-57d4140c6dce tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:52.459241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5884721f-9cb1-4283-90cc-57d4140c6dce tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:52.459730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5884721f-9cb1-4283-90cc-57d4140c6dce tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:52.460491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5884721f-9cb1-4283-90cc-57d4140c6dce tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:52.461303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5884721f-9cb1-4283-90cc-57d4140c6dce tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:52.461872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5884721f-9cb1-4283-90cc-57d4140c6dce tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:52.462300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5884721f-9cb1-4283-90cc-57d4140c6dce tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:52.600570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5884721f-9cb1-4283-90cc-57d4140c6dce tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:52.609066 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5884721f-9cb1-4283-90cc-57d4140c6dce tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1514 microversion: 2.1 time: 0.163340 Sep 27 10:43:52.609983 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3585/6700] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:52 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1514 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:53.632417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-762a7125-0f1c-4e7f-a730-9e387ee40ebb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:53.633324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-762a7125-0f1c-4e7f-a730-9e387ee40ebb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:53.643823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-762a7125-0f1c-4e7f-a730-9e387ee40ebb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:53.644444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-762a7125-0f1c-4e7f-a730-9e387ee40ebb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:53.644945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-762a7125-0f1c-4e7f-a730-9e387ee40ebb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:53.645658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-762a7125-0f1c-4e7f-a730-9e387ee40ebb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:53.646195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-762a7125-0f1c-4e7f-a730-9e387ee40ebb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:53.646634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-762a7125-0f1c-4e7f-a730-9e387ee40ebb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:53.791647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-762a7125-0f1c-4e7f-a730-9e387ee40ebb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:53.803311 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-762a7125-0f1c-4e7f-a730-9e387ee40ebb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.175220 Sep 27 10:43:53.804044 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3116/6701] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:53 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:54.836574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e9216340-a47b-440f-bdb1-430452976ef2 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:54.837815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e9216340-a47b-440f-bdb1-430452976ef2 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:54.848954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9216340-a47b-440f-bdb1-430452976ef2 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:54.849438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9216340-a47b-440f-bdb1-430452976ef2 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:54.849869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9216340-a47b-440f-bdb1-430452976ef2 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:54.850565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9216340-a47b-440f-bdb1-430452976ef2 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:54.851022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9216340-a47b-440f-bdb1-430452976ef2 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:54.851446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9216340-a47b-440f-bdb1-430452976ef2 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:54.996364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e9216340-a47b-440f-bdb1-430452976ef2 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:55.008825 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e9216340-a47b-440f-bdb1-430452976ef2 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.178945 Sep 27 10:43:55.009720 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3586/6702] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:54 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:56.040183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1ef6ecbd-8810-45e7-9d1a-45203fec48a0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:56.041037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1ef6ecbd-8810-45e7-9d1a-45203fec48a0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:56.050749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ef6ecbd-8810-45e7-9d1a-45203fec48a0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:56.051642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ef6ecbd-8810-45e7-9d1a-45203fec48a0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:56.052510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ef6ecbd-8810-45e7-9d1a-45203fec48a0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:56.053566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ef6ecbd-8810-45e7-9d1a-45203fec48a0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:56.054170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ef6ecbd-8810-45e7-9d1a-45203fec48a0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:56.054760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ef6ecbd-8810-45e7-9d1a-45203fec48a0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:56.203085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1ef6ecbd-8810-45e7-9d1a-45203fec48a0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:56.212985 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1ef6ecbd-8810-45e7-9d1a-45203fec48a0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.178222 Sep 27 10:43:56.213594 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3117/6703] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:56 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:57.243078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6f5aec37-9c35-495f-a920-853e66faee76 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:57.244970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6f5aec37-9c35-495f-a920-853e66faee76 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:57.259322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5aec37-9c35-495f-a920-853e66faee76 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:57.259816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5aec37-9c35-495f-a920-853e66faee76 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:57.260918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5aec37-9c35-495f-a920-853e66faee76 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:57.262720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5aec37-9c35-495f-a920-853e66faee76 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:57.263925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5aec37-9c35-495f-a920-853e66faee76 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:57.265154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6f5aec37-9c35-495f-a920-853e66faee76 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:57.439246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6f5aec37-9c35-495f-a920-853e66faee76 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:57.447641 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6f5aec37-9c35-495f-a920-853e66faee76 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.210122 Sep 27 10:43:57.448225 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3587/6704] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:57 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:58.474453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2a23f7f9-dece-41e9-bde7-b2aaf1245090 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:58.475085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2a23f7f9-dece-41e9-bde7-b2aaf1245090 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:58.478547 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=60,nova_api:INSERT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:58.484839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a23f7f9-dece-41e9-bde7-b2aaf1245090 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:58.485388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a23f7f9-dece-41e9-bde7-b2aaf1245090 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:58.485883 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a23f7f9-dece-41e9-bde7-b2aaf1245090 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:58.486669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a23f7f9-dece-41e9-bde7-b2aaf1245090 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:58.487198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a23f7f9-dece-41e9-bde7-b2aaf1245090 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:58.487680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a23f7f9-dece-41e9-bde7-b2aaf1245090 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:58.625919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2a23f7f9-dece-41e9-bde7-b2aaf1245090 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:58.637528 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2a23f7f9-dece-41e9-bde7-b2aaf1245090 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.168685 Sep 27 10:43:58.638299 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3118/6705] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:58 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:43:59.662584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-834ee8e3-794b-446f-b967-57766f62f007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:43:59.663155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-834ee8e3-794b-446f-b967-57766f62f007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:43:59.665938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:43:59.671128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-834ee8e3-794b-446f-b967-57766f62f007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:59.671620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-834ee8e3-794b-446f-b967-57766f62f007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:59.672059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-834ee8e3-794b-446f-b967-57766f62f007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:59.672790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-834ee8e3-794b-446f-b967-57766f62f007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:43:59.673602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-834ee8e3-794b-446f-b967-57766f62f007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:43:59.674032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-834ee8e3-794b-446f-b967-57766f62f007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:43:59.817068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-834ee8e3-794b-446f-b967-57766f62f007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:43:59.825507 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-834ee8e3-794b-446f-b967-57766f62f007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.167559 Sep 27 10:43:59.826088 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3588/6706] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:43:59 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:00.858989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1a3dc1ec-232d-4f66-920d-6c3f9b96d90f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:00.859946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1a3dc1ec-232d-4f66-920d-6c3f9b96d90f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:00.868778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a3dc1ec-232d-4f66-920d-6c3f9b96d90f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:00.869285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a3dc1ec-232d-4f66-920d-6c3f9b96d90f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:00.869737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a3dc1ec-232d-4f66-920d-6c3f9b96d90f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:00.870446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a3dc1ec-232d-4f66-920d-6c3f9b96d90f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:00.870914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a3dc1ec-232d-4f66-920d-6c3f9b96d90f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:00.871352 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1a3dc1ec-232d-4f66-920d-6c3f9b96d90f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:01.009881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1a3dc1ec-232d-4f66-920d-6c3f9b96d90f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:01.017931 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1a3dc1ec-232d-4f66-920d-6c3f9b96d90f tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.165786 Sep 27 10:44:01.018516 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3119/6707] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:00 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:02.040937 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4a4d20d7-c1d0-43a7-8ad0-34fdbe6ee6a3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:02.041555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4a4d20d7-c1d0-43a7-8ad0-34fdbe6ee6a3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:02.049463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4d20d7-c1d0-43a7-8ad0-34fdbe6ee6a3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:02.049980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4d20d7-c1d0-43a7-8ad0-34fdbe6ee6a3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:02.050413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4d20d7-c1d0-43a7-8ad0-34fdbe6ee6a3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:02.051185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4d20d7-c1d0-43a7-8ad0-34fdbe6ee6a3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:02.051644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4d20d7-c1d0-43a7-8ad0-34fdbe6ee6a3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:02.052067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4a4d20d7-c1d0-43a7-8ad0-34fdbe6ee6a3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:02.197866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4a4d20d7-c1d0-43a7-8ad0-34fdbe6ee6a3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:02.206282 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4a4d20d7-c1d0-43a7-8ad0-34fdbe6ee6a3 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.169515 Sep 27 10:44:02.207002 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3589/6708] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:02 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:03.230555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5649b790-e214-4a6b-9e18-34d864761c21 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:03.231155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5649b790-e214-4a6b-9e18-34d864761c21 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:03.238738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5649b790-e214-4a6b-9e18-34d864761c21 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:03.239237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5649b790-e214-4a6b-9e18-34d864761c21 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:03.239680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5649b790-e214-4a6b-9e18-34d864761c21 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:03.240403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5649b790-e214-4a6b-9e18-34d864761c21 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:03.240911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5649b790-e214-4a6b-9e18-34d864761c21 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:03.241345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5649b790-e214-4a6b-9e18-34d864761c21 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:03.390465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5649b790-e214-4a6b-9e18-34d864761c21 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:03.400027 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5649b790-e214-4a6b-9e18-34d864761c21 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.173475 Sep 27 10:44:03.400598 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3120/6709] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:03 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:04.425522 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-138b0200-4ccc-45a5-8e8e-285d62b87551 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:04.426371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-138b0200-4ccc-45a5-8e8e-285d62b87551 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:04.435048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-138b0200-4ccc-45a5-8e8e-285d62b87551 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:04.435619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-138b0200-4ccc-45a5-8e8e-285d62b87551 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:04.436134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-138b0200-4ccc-45a5-8e8e-285d62b87551 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:04.437007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-138b0200-4ccc-45a5-8e8e-285d62b87551 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:04.437546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-138b0200-4ccc-45a5-8e8e-285d62b87551 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:04.438047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-138b0200-4ccc-45a5-8e8e-285d62b87551 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:04.586756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-138b0200-4ccc-45a5-8e8e-285d62b87551 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:04.600292 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-138b0200-4ccc-45a5-8e8e-285d62b87551 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.179892 Sep 27 10:44:04.601169 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3590/6710] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:04 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:05.623811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b4d647a2-ee5a-48cb-b0ed-36a04867f43d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:05.624451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b4d647a2-ee5a-48cb-b0ed-36a04867f43d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:05.631938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4d647a2-ee5a-48cb-b0ed-36a04867f43d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:05.632465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4d647a2-ee5a-48cb-b0ed-36a04867f43d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:05.632971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4d647a2-ee5a-48cb-b0ed-36a04867f43d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:05.633697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4d647a2-ee5a-48cb-b0ed-36a04867f43d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:05.634162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4d647a2-ee5a-48cb-b0ed-36a04867f43d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:05.634604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b4d647a2-ee5a-48cb-b0ed-36a04867f43d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:05.786439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b4d647a2-ee5a-48cb-b0ed-36a04867f43d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:05.794601 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b4d647a2-ee5a-48cb-b0ed-36a04867f43d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.174815 Sep 27 10:44:05.795231 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3121/6711] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:05 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:06.818965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8b10452e-6bc8-4d76-9b13-a9d07489a8ab tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:06.819547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8b10452e-6bc8-4d76-9b13-a9d07489a8ab tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:06.826963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b10452e-6bc8-4d76-9b13-a9d07489a8ab tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:06.827447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b10452e-6bc8-4d76-9b13-a9d07489a8ab tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:06.827876 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b10452e-6bc8-4d76-9b13-a9d07489a8ab tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:06.828575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b10452e-6bc8-4d76-9b13-a9d07489a8ab tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:06.829062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b10452e-6bc8-4d76-9b13-a9d07489a8ab tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:06.829484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8b10452e-6bc8-4d76-9b13-a9d07489a8ab tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:06.983834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8b10452e-6bc8-4d76-9b13-a9d07489a8ab tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:06.992236 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8b10452e-6bc8-4d76-9b13-a9d07489a8ab tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.177284 Sep 27 10:44:06.992852 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3591/6712] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:06 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:08.020286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-370277ca-4ad8-4108-8c5d-671ed08f27aa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:08.021905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-370277ca-4ad8-4108-8c5d-671ed08f27aa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:08.033670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-370277ca-4ad8-4108-8c5d-671ed08f27aa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:08.034498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-370277ca-4ad8-4108-8c5d-671ed08f27aa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:08.035258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-370277ca-4ad8-4108-8c5d-671ed08f27aa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:08.036543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-370277ca-4ad8-4108-8c5d-671ed08f27aa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:08.037052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-370277ca-4ad8-4108-8c5d-671ed08f27aa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:08.037496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-370277ca-4ad8-4108-8c5d-671ed08f27aa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:08.177875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-370277ca-4ad8-4108-8c5d-671ed08f27aa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:08.185877 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-370277ca-4ad8-4108-8c5d-671ed08f27aa tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.172373 Sep 27 10:44:08.186489 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3122/6713] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:08 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:09.212581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3fd1c4c4-5350-4b36-a585-a36f65e33100 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:09.213238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3fd1c4c4-5350-4b36-a585-a36f65e33100 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:09.220190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fd1c4c4-5350-4b36-a585-a36f65e33100 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:09.220702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fd1c4c4-5350-4b36-a585-a36f65e33100 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:09.221157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fd1c4c4-5350-4b36-a585-a36f65e33100 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:09.221884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fd1c4c4-5350-4b36-a585-a36f65e33100 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:09.222332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fd1c4c4-5350-4b36-a585-a36f65e33100 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:09.222754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3fd1c4c4-5350-4b36-a585-a36f65e33100 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:09.368570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3fd1c4c4-5350-4b36-a585-a36f65e33100 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:09.376502 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3fd1c4c4-5350-4b36-a585-a36f65e33100 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.167845 Sep 27 10:44:09.377099 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3592/6714] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:09 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:10.404202 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c22a92da-86f3-4ae0-8b57-03eb948e8aff tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:10.405374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c22a92da-86f3-4ae0-8b57-03eb948e8aff tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:10.414844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22a92da-86f3-4ae0-8b57-03eb948e8aff tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:10.415339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22a92da-86f3-4ae0-8b57-03eb948e8aff tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:10.415786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22a92da-86f3-4ae0-8b57-03eb948e8aff tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:10.416501 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22a92da-86f3-4ae0-8b57-03eb948e8aff tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:10.417022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22a92da-86f3-4ae0-8b57-03eb948e8aff tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:10.417440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c22a92da-86f3-4ae0-8b57-03eb948e8aff tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:10.564606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c22a92da-86f3-4ae0-8b57-03eb948e8aff tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:10.573044 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c22a92da-86f3-4ae0-8b57-03eb948e8aff tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.175302 Sep 27 10:44:10.573650 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3123/6715] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:10 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:11.600390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1ef5a0e6-8509-461f-a1d6-e98cdc370586 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:11.601437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1ef5a0e6-8509-461f-a1d6-e98cdc370586 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:11.613249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef5a0e6-8509-461f-a1d6-e98cdc370586 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:11.614999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef5a0e6-8509-461f-a1d6-e98cdc370586 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:11.615456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef5a0e6-8509-461f-a1d6-e98cdc370586 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:11.616180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef5a0e6-8509-461f-a1d6-e98cdc370586 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:11.616642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef5a0e6-8509-461f-a1d6-e98cdc370586 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:11.617106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1ef5a0e6-8509-461f-a1d6-e98cdc370586 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:11.765318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1ef5a0e6-8509-461f-a1d6-e98cdc370586 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:11.777183 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1ef5a0e6-8509-461f-a1d6-e98cdc370586 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.183293 Sep 27 10:44:11.777967 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3593/6716] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:11 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:12.808219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-21159eec-c517-4035-baad-4918624f7427 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:12.808998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-21159eec-c517-4035-baad-4918624f7427 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:12.816670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21159eec-c517-4035-baad-4918624f7427 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:12.817189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21159eec-c517-4035-baad-4918624f7427 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:12.818201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21159eec-c517-4035-baad-4918624f7427 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:12.818584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21159eec-c517-4035-baad-4918624f7427 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:12.819051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21159eec-c517-4035-baad-4918624f7427 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:12.819489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-21159eec-c517-4035-baad-4918624f7427 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:12.966468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-21159eec-c517-4035-baad-4918624f7427 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:12.974386 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-21159eec-c517-4035-baad-4918624f7427 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.175390 Sep 27 10:44:12.974962 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3124/6717] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:12 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:14.002358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-620f4088-3ee5-4b37-b90c-5081bdbaae25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:14.003261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-620f4088-3ee5-4b37-b90c-5081bdbaae25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:14.011511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620f4088-3ee5-4b37-b90c-5081bdbaae25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:14.011996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620f4088-3ee5-4b37-b90c-5081bdbaae25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:14.012428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620f4088-3ee5-4b37-b90c-5081bdbaae25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:14.013167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620f4088-3ee5-4b37-b90c-5081bdbaae25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:14.013621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620f4088-3ee5-4b37-b90c-5081bdbaae25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:14.014047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-620f4088-3ee5-4b37-b90c-5081bdbaae25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:14.162446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-620f4088-3ee5-4b37-b90c-5081bdbaae25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:14.170758 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-620f4088-3ee5-4b37-b90c-5081bdbaae25 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.174976 Sep 27 10:44:14.171346 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3594/6718] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:13 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:15.195993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-605fad12-9757-4b55-a582-e25aaace3747 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:15.196649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-605fad12-9757-4b55-a582-e25aaace3747 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:15.204136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-605fad12-9757-4b55-a582-e25aaace3747 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:15.204694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-605fad12-9757-4b55-a582-e25aaace3747 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:15.205142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-605fad12-9757-4b55-a582-e25aaace3747 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:15.205870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-605fad12-9757-4b55-a582-e25aaace3747 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:15.206343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-605fad12-9757-4b55-a582-e25aaace3747 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:15.206778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-605fad12-9757-4b55-a582-e25aaace3747 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:15.439962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-605fad12-9757-4b55-a582-e25aaace3747 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:15.448072 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-605fad12-9757-4b55-a582-e25aaace3747 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.257135 Sep 27 10:44:15.448635 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3125/6719] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:15 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:16.475721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9914e02c-20d1-4ab9-93a3-d8042a998219 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:16.476370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9914e02c-20d1-4ab9-93a3-d8042a998219 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:16.484728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9914e02c-20d1-4ab9-93a3-d8042a998219 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:16.485324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9914e02c-20d1-4ab9-93a3-d8042a998219 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:16.485831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9914e02c-20d1-4ab9-93a3-d8042a998219 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:16.486640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9914e02c-20d1-4ab9-93a3-d8042a998219 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:16.487212 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9914e02c-20d1-4ab9-93a3-d8042a998219 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:16.487707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9914e02c-20d1-4ab9-93a3-d8042a998219 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:16.635273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9914e02c-20d1-4ab9-93a3-d8042a998219 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:16.643565 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9914e02c-20d1-4ab9-93a3-d8042a998219 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.174484 Sep 27 10:44:16.644157 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3595/6720] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:16 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:17.667820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-68dc4e52-7680-4b1b-aa93-72085efe0f19 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:17.668453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-68dc4e52-7680-4b1b-aa93-72085efe0f19 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:17.676009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68dc4e52-7680-4b1b-aa93-72085efe0f19 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:17.676505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68dc4e52-7680-4b1b-aa93-72085efe0f19 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:17.677207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68dc4e52-7680-4b1b-aa93-72085efe0f19 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:17.677924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68dc4e52-7680-4b1b-aa93-72085efe0f19 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:17.678391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68dc4e52-7680-4b1b-aa93-72085efe0f19 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:17.678829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68dc4e52-7680-4b1b-aa93-72085efe0f19 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:17.827251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-68dc4e52-7680-4b1b-aa93-72085efe0f19 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:17.839475 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-68dc4e52-7680-4b1b-aa93-72085efe0f19 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.175291 Sep 27 10:44:17.840309 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3126/6721] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:17 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:18.868584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1e3d73b9-2548-418e-9528-3b33bb452963 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:18.869319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1e3d73b9-2548-418e-9528-3b33bb452963 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:18.871879 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 10:44:18.872068 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 10:44:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 (ip 10.222.0.26) !!! Sep 27 10:44:18.883416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e3d73b9-2548-418e-9528-3b33bb452963 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:18.884122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e3d73b9-2548-418e-9528-3b33bb452963 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:18.884851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e3d73b9-2548-418e-9528-3b33bb452963 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:18.886124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e3d73b9-2548-418e-9528-3b33bb452963 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:18.886821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e3d73b9-2548-418e-9528-3b33bb452963 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:18.887389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1e3d73b9-2548-418e-9528-3b33bb452963 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:19.038735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1e3d73b9-2548-418e-9528-3b33bb452963 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:19.047073 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1e3d73b9-2548-418e-9528-3b33bb452963 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.183616 Sep 27 10:44:19.048007 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3596/6722] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:18 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:20.071386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ea18c8fd-0292-421a-a56a-cbf703129f87 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:20.072024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ea18c8fd-0292-421a-a56a-cbf703129f87 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:20.079390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea18c8fd-0292-421a-a56a-cbf703129f87 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:20.079893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea18c8fd-0292-421a-a56a-cbf703129f87 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:20.080355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea18c8fd-0292-421a-a56a-cbf703129f87 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:20.081150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea18c8fd-0292-421a-a56a-cbf703129f87 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:20.081611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea18c8fd-0292-421a-a56a-cbf703129f87 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:20.082058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ea18c8fd-0292-421a-a56a-cbf703129f87 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:20.220422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ea18c8fd-0292-421a-a56a-cbf703129f87 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:20.229455 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ea18c8fd-0292-421a-a56a-cbf703129f87 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1519 microversion: 2.1 time: 0.162103 Sep 27 10:44:20.230085 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3127/6723] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:20 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1519 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:20.643745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-1bfdd813-4b68-4ecb-82d9-b690a04aff66 req-ebe03853-6dbb-4eab-95c3-d3a8c90d7213 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ccbfee66-bd3c-41ec-b09c-9b2b384d1229", "name": "network-vif-unplugged", "status": "completed", "tag": "0681f2f1-ab1f-4cc4-930b-e48076441ec4"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:44:20.654925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1bfdd813-4b68-4ecb-82d9-b690a04aff66 req-ebe03853-6dbb-4eab-95c3-d3a8c90d7213 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:20.655483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1bfdd813-4b68-4ecb-82d9-b690a04aff66 req-ebe03853-6dbb-4eab-95c3-d3a8c90d7213 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:20.655996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1bfdd813-4b68-4ecb-82d9-b690a04aff66 req-ebe03853-6dbb-4eab-95c3-d3a8c90d7213 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:20.668755 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-1bfdd813-4b68-4ecb-82d9-b690a04aff66 req-ebe03853-6dbb-4eab-95c3-d3a8c90d7213 service nova] Creating event network-vif-unplugged:0681f2f1-ab1f-4cc4-930b-e48076441ec4 for instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 on nch1-master-1 Sep 27 10:44:20.674582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-1bfdd813-4b68-4ecb-82d9-b690a04aff66 req-ebe03853-6dbb-4eab-95c3-d3a8c90d7213 service nova] Instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:44:20.687206 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-1bfdd813-4b68-4ecb-82d9-b690a04aff66 req-ebe03853-6dbb-4eab-95c3-d3a8c90d7213 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.047335 Sep 27 10:44:20.687824 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3597/6724] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:44:20 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:44:21.253815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ac28d814-0069-4d28-bf59-1ba9128e29b0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:21.254654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ac28d814-0069-4d28-bf59-1ba9128e29b0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:21.263533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac28d814-0069-4d28-bf59-1ba9128e29b0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:21.264040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac28d814-0069-4d28-bf59-1ba9128e29b0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:21.264500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac28d814-0069-4d28-bf59-1ba9128e29b0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:21.265267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac28d814-0069-4d28-bf59-1ba9128e29b0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:21.265744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac28d814-0069-4d28-bf59-1ba9128e29b0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:21.266191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ac28d814-0069-4d28-bf59-1ba9128e29b0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:21.270393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=81 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:21.506817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ac28d814-0069-4d28-bf59-1ba9128e29b0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:21.516049 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ac28d814-0069-4d28-bf59-1ba9128e29b0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1518 microversion: 2.1 time: 0.266266 Sep 27 10:44:21.516638 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3128/6725] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:21 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1518 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:22.540414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a607e34-aba7-4fbc-9465-f08c90fd11ca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:22.541168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a607e34-aba7-4fbc-9465-f08c90fd11ca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:22.549005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a607e34-aba7-4fbc-9465-f08c90fd11ca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:22.549481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a607e34-aba7-4fbc-9465-f08c90fd11ca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:22.549910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a607e34-aba7-4fbc-9465-f08c90fd11ca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:22.550600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a607e34-aba7-4fbc-9465-f08c90fd11ca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:22.551053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a607e34-aba7-4fbc-9465-f08c90fd11ca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:22.551472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a607e34-aba7-4fbc-9465-f08c90fd11ca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:22.554215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:22.703908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-6b8eeb58-05e3-4457-8774-9fd17c238245 req-9b9ba4b2-4882-4d72-b375-0d38b021f3f0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ccbfee66-bd3c-41ec-b09c-9b2b384d1229", "tag": "0681f2f1-ab1f-4cc4-930b-e48076441ec4"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:44:22.714903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6b8eeb58-05e3-4457-8774-9fd17c238245 req-9b9ba4b2-4882-4d72-b375-0d38b021f3f0 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:22.715410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6b8eeb58-05e3-4457-8774-9fd17c238245 req-9b9ba4b2-4882-4d72-b375-0d38b021f3f0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:22.715836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6b8eeb58-05e3-4457-8774-9fd17c238245 req-9b9ba4b2-4882-4d72-b375-0d38b021f3f0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:22.729371 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-6b8eeb58-05e3-4457-8774-9fd17c238245 req-9b9ba4b2-4882-4d72-b375-0d38b021f3f0 service nova] Creating event network-changed:0681f2f1-ab1f-4cc4-930b-e48076441ec4 for instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 on nch1-master-1 Sep 27 10:44:22.736212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-6b8eeb58-05e3-4457-8774-9fd17c238245 req-9b9ba4b2-4882-4d72-b375-0d38b021f3f0 service nova] Instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:44:22.748310 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-6b8eeb58-05e3-4457-8774-9fd17c238245 req-9b9ba4b2-4882-4d72-b375-0d38b021f3f0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048260 Sep 27 10:44:22.749447 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3129/6726] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:44:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:44:22.972408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8a607e34-aba7-4fbc-9465-f08c90fd11ca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:22.983604 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a607e34-aba7-4fbc-9465-f08c90fd11ca tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1516 microversion: 2.1 time: 0.447414 Sep 27 10:44:22.984526 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3598/6727] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:22 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1516 bytes in 449 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:24.010587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5590368-d878-49f0-9417-40c4a4350de0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:24.011292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c5590368-d878-49f0-9417-40c4a4350de0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:24.018066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5590368-d878-49f0-9417-40c4a4350de0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:24.018562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5590368-d878-49f0-9417-40c4a4350de0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:24.019008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5590368-d878-49f0-9417-40c4a4350de0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:24.019717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5590368-d878-49f0-9417-40c4a4350de0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:24.020190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5590368-d878-49f0-9417-40c4a4350de0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:24.020713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5590368-d878-49f0-9417-40c4a4350de0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:24.170136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c5590368-d878-49f0-9417-40c4a4350de0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:24.179225 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c5590368-d878-49f0-9417-40c4a4350de0 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1513 microversion: 2.1 time: 0.174275 Sep 27 10:44:24.179817 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3130/6728] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:24 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1513 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:24.202466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-567cac91-5073-4811-9d8a-a9fd2f65710a tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:44:24.203023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-567cac91-5073-4811-9d8a-a9fd2f65710a tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:24.212859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567cac91-5073-4811-9d8a-a9fd2f65710a tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:24.213592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567cac91-5073-4811-9d8a-a9fd2f65710a tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:24.214344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567cac91-5073-4811-9d8a-a9fd2f65710a tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:24.215470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567cac91-5073-4811-9d8a-a9fd2f65710a tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:24.216215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567cac91-5073-4811-9d8a-a9fd2f65710a tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:24.216995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-567cac91-5073-4811-9d8a-a9fd2f65710a tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:24.289069 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-567cac91-5073-4811-9d8a-a9fd2f65710a tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "POST /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229/action" status: 204 len: 0 microversion: 2.1 time: 0.089981 Sep 27 10:44:24.289459 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3599/6729] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:44:24 2022] POST /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229/action => generated 0 bytes in 92 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:44:24.298447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a556bca2-91d3-4965-9843-ed5a69e5ce66 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:24.299194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a556bca2-91d3-4965-9843-ed5a69e5ce66 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:24.306369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a556bca2-91d3-4965-9843-ed5a69e5ce66 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:24.307146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a556bca2-91d3-4965-9843-ed5a69e5ce66 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:24.307791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a556bca2-91d3-4965-9843-ed5a69e5ce66 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:24.308751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a556bca2-91d3-4965-9843-ed5a69e5ce66 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:24.309243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a556bca2-91d3-4965-9843-ed5a69e5ce66 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:24.309699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a556bca2-91d3-4965-9843-ed5a69e5ce66 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:24.515204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a556bca2-91d3-4965-9843-ed5a69e5ce66 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:24.524725 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a556bca2-91d3-4965-9843-ed5a69e5ce66 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1513 microversion: 2.1 time: 0.230676 Sep 27 10:44:24.541401 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3131/6730] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:24 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1513 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:25.552604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c8012ddb-5b46-4c61-88e9-e8c8cb8ad2fd tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:25.553355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c8012ddb-5b46-4c61-88e9-e8c8cb8ad2fd tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:25.559807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8012ddb-5b46-4c61-88e9-e8c8cb8ad2fd tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:25.560292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8012ddb-5b46-4c61-88e9-e8c8cb8ad2fd tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:25.560759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8012ddb-5b46-4c61-88e9-e8c8cb8ad2fd tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:25.561484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8012ddb-5b46-4c61-88e9-e8c8cb8ad2fd tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:25.561938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8012ddb-5b46-4c61-88e9-e8c8cb8ad2fd tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:25.562359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c8012ddb-5b46-4c61-88e9-e8c8cb8ad2fd tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:25.701288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c8012ddb-5b46-4c61-88e9-e8c8cb8ad2fd tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:25.710551 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c8012ddb-5b46-4c61-88e9-e8c8cb8ad2fd tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1513 microversion: 2.1 time: 0.163471 Sep 27 10:44:25.711195 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3600/6731] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:25 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1513 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:26.509562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-6c0f32ae-0d1b-4a8b-822b-faaf9c6d570a req-bea192e3-1704-44a7-b7e6-b5a2f74bfd6f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ccbfee66-bd3c-41ec-b09c-9b2b384d1229", "name": "network-vif-plugged", "status": "completed", "tag": "0681f2f1-ab1f-4cc4-930b-e48076441ec4"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:44:26.523262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6c0f32ae-0d1b-4a8b-822b-faaf9c6d570a req-bea192e3-1704-44a7-b7e6-b5a2f74bfd6f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:26.523810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6c0f32ae-0d1b-4a8b-822b-faaf9c6d570a req-bea192e3-1704-44a7-b7e6-b5a2f74bfd6f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:26.524319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6c0f32ae-0d1b-4a8b-822b-faaf9c6d570a req-bea192e3-1704-44a7-b7e6-b5a2f74bfd6f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:26.536605 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-6c0f32ae-0d1b-4a8b-822b-faaf9c6d570a req-bea192e3-1704-44a7-b7e6-b5a2f74bfd6f service nova] Creating event network-vif-plugged:0681f2f1-ab1f-4cc4-930b-e48076441ec4 for instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 on nch1-master-1 Sep 27 10:44:26.541847 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-6c0f32ae-0d1b-4a8b-822b-faaf9c6d570a req-bea192e3-1704-44a7-b7e6-b5a2f74bfd6f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036288 Sep 27 10:44:26.542421 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3132/6732] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:44:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:44:26.738831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bd2d19c9-25a7-4375-b48a-940760b7115b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:26.739900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bd2d19c9-25a7-4375-b48a-940760b7115b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:26.751090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd2d19c9-25a7-4375-b48a-940760b7115b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:26.751873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd2d19c9-25a7-4375-b48a-940760b7115b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:26.752590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd2d19c9-25a7-4375-b48a-940760b7115b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:26.753690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd2d19c9-25a7-4375-b48a-940760b7115b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:26.754436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd2d19c9-25a7-4375-b48a-940760b7115b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:26.755140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bd2d19c9-25a7-4375-b48a-940760b7115b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:26.903125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bd2d19c9-25a7-4375-b48a-940760b7115b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:26.911558 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bd2d19c9-25a7-4375-b48a-940760b7115b tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1505 microversion: 2.1 time: 0.179116 Sep 27 10:44:26.912128 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3601/6733] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:26 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1505 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:26.935814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d1a2fed0-74c2-4c3f-a1fd-5b3424e97b61 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:26.936531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d1a2fed0-74c2-4c3f-a1fd-5b3424e97b61 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:26.943004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1a2fed0-74c2-4c3f-a1fd-5b3424e97b61 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:26.943531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1a2fed0-74c2-4c3f-a1fd-5b3424e97b61 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:26.943968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1a2fed0-74c2-4c3f-a1fd-5b3424e97b61 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:26.944694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1a2fed0-74c2-4c3f-a1fd-5b3424e97b61 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:26.945161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1a2fed0-74c2-4c3f-a1fd-5b3424e97b61 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:26.945599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1a2fed0-74c2-4c3f-a1fd-5b3424e97b61 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:27.086058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d1a2fed0-74c2-4c3f-a1fd-5b3424e97b61 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:27.094069 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d1a2fed0-74c2-4c3f-a1fd-5b3424e97b61 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1505 microversion: 2.1 time: 0.162560 Sep 27 10:44:27.094646 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3133/6734] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:26 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1505 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:27.117225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ace52b6f-373e-4ab1-8542-12564f9fe2a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:27.117763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ace52b6f-373e-4ab1-8542-12564f9fe2a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:27.124426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ace52b6f-373e-4ab1-8542-12564f9fe2a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:27.124955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ace52b6f-373e-4ab1-8542-12564f9fe2a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:27.125402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ace52b6f-373e-4ab1-8542-12564f9fe2a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:27.126109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ace52b6f-373e-4ab1-8542-12564f9fe2a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:27.126565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ace52b6f-373e-4ab1-8542-12564f9fe2a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:27.126990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ace52b6f-373e-4ab1-8542-12564f9fe2a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:27.262794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ace52b6f-373e-4ab1-8542-12564f9fe2a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:27.270720 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ace52b6f-373e-4ab1-8542-12564f9fe2a8 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1505 microversion: 2.1 time: 0.158043 Sep 27 10:44:27.271425 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3602/6735] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:27 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1505 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:27.293683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ff23cbf3-1507-4f57-9ce1-fbcde6e5d191 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:27.294266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ff23cbf3-1507-4f57-9ce1-fbcde6e5d191 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:27.301204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff23cbf3-1507-4f57-9ce1-fbcde6e5d191 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:27.301703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff23cbf3-1507-4f57-9ce1-fbcde6e5d191 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:27.302323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff23cbf3-1507-4f57-9ce1-fbcde6e5d191 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:27.302990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff23cbf3-1507-4f57-9ce1-fbcde6e5d191 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:27.303492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff23cbf3-1507-4f57-9ce1-fbcde6e5d191 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:27.303933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ff23cbf3-1507-4f57-9ce1-fbcde6e5d191 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:27.329116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ff23cbf3-1507-4f57-9ce1-fbcde6e5d191 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:44:27.408155 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ff23cbf3-1507-4f57-9ce1-fbcde6e5d191 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "DELETE /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 204 len: 0 microversion: 2.1 time: 0.117935 Sep 27 10:44:27.408963 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3134/6736] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:44:27 2022] DELETE /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:44:27.425830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d4552a5-613b-490a-8c05-6a67ed3b9a79 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:27.426333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d4552a5-613b-490a-8c05-6a67ed3b9a79 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:27.432814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4552a5-613b-490a-8c05-6a67ed3b9a79 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:27.433307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4552a5-613b-490a-8c05-6a67ed3b9a79 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:27.433750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4552a5-613b-490a-8c05-6a67ed3b9a79 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:27.434450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4552a5-613b-490a-8c05-6a67ed3b9a79 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:27.434911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4552a5-613b-490a-8c05-6a67ed3b9a79 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:27.435336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d4552a5-613b-490a-8c05-6a67ed3b9a79 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:27.460782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d4552a5-613b-490a-8c05-6a67ed3b9a79 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:44:27.504339 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.compute.api [None req-7d4552a5-613b-490a-8c05-6a67ed3b9a79 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Instance is already in deleting state, ignoring this request Sep 27 10:44:27.505312 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d4552a5-613b-490a-8c05-6a67ed3b9a79 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "DELETE /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 204 len: 0 microversion: 2.1 time: 0.083749 Sep 27 10:44:27.505954 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3603/6737] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:44:27 2022] DELETE /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:44:27.514964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f83b7dc0-2f1c-4984-8cd6-fa09b6309744 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:27.515670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f83b7dc0-2f1c-4984-8cd6-fa09b6309744 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:27.522887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f83b7dc0-2f1c-4984-8cd6-fa09b6309744 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:27.523365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f83b7dc0-2f1c-4984-8cd6-fa09b6309744 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:27.523807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f83b7dc0-2f1c-4984-8cd6-fa09b6309744 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:27.524512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f83b7dc0-2f1c-4984-8cd6-fa09b6309744 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:27.525011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f83b7dc0-2f1c-4984-8cd6-fa09b6309744 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:27.525448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f83b7dc0-2f1c-4984-8cd6-fa09b6309744 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:27.664336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f83b7dc0-2f1c-4984-8cd6-fa09b6309744 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:27.672559 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f83b7dc0-2f1c-4984-8cd6-fa09b6309744 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1511 microversion: 2.1 time: 0.161564 Sep 27 10:44:27.673163 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3135/6738] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:27 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1511 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:28.700114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7eac7a2a-7aaa-47d5-8c0c-43d319fd4c2d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:28.701277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7eac7a2a-7aaa-47d5-8c0c-43d319fd4c2d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:28.711270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7eac7a2a-7aaa-47d5-8c0c-43d319fd4c2d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:28.711777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7eac7a2a-7aaa-47d5-8c0c-43d319fd4c2d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:28.712208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7eac7a2a-7aaa-47d5-8c0c-43d319fd4c2d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:28.712945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7eac7a2a-7aaa-47d5-8c0c-43d319fd4c2d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:28.713405 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7eac7a2a-7aaa-47d5-8c0c-43d319fd4c2d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:28.713838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7eac7a2a-7aaa-47d5-8c0c-43d319fd4c2d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:28.856282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7eac7a2a-7aaa-47d5-8c0c-43d319fd4c2d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '74e30240d7c840edba43dddf3a862c4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '063971e11eb044b4a844a563c6ccbc5d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:28.864851 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7eac7a2a-7aaa-47d5-8c0c-43d319fd4c2d tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 200 len: 1511 microversion: 2.1 time: 0.171225 Sep 27 10:44:28.872987 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3604/6739] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:28 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 1511 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:29.228950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b77f3fb2-7664-406f-b10d-2e4ada3be54b req-0ad33442-76ba-4c04-873b-7759a0075c3d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ccbfee66-bd3c-41ec-b09c-9b2b384d1229", "name": "network-vif-deleted", "tag": "0681f2f1-ab1f-4cc4-930b-e48076441ec4"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:44:29.235072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=41,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:29.241082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b77f3fb2-7664-406f-b10d-2e4ada3be54b req-0ad33442-76ba-4c04-873b-7759a0075c3d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:29.241593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b77f3fb2-7664-406f-b10d-2e4ada3be54b req-0ad33442-76ba-4c04-873b-7759a0075c3d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:29.242013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b77f3fb2-7664-406f-b10d-2e4ada3be54b req-0ad33442-76ba-4c04-873b-7759a0075c3d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:29.257320 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b77f3fb2-7664-406f-b10d-2e4ada3be54b req-0ad33442-76ba-4c04-873b-7759a0075c3d service nova] Creating event network-vif-deleted:0681f2f1-ab1f-4cc4-930b-e48076441ec4 for instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 on nch1-master-1 Sep 27 10:44:29.262803 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b77f3fb2-7664-406f-b10d-2e4ada3be54b req-0ad33442-76ba-4c04-873b-7759a0075c3d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037434 Sep 27 10:44:29.263478 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3136/6740] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:44:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:44:29.891787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:29.892933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: ccbfee66-bd3c-41ec-b09c-9b2b384d1229] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:29.897624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=34 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:29.906705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:29.907509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:29.908245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:29.909420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:29.910290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:29.911002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:29.926415 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] HTTP exception thrown: Instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 could not be found. Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Request method failure captured: Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 HTTP/1.1 Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtOlWfxeK3mTR2eKwyq4265ckFp3X5nf3Z46qDJZSXT5dKOr0MdDRk_agSk6WEkwm05LkaUtdHmcGnC_f1F8-x-YlykEz3zdDVpcbddRWayHOb_vBw7TOfdT2prYMOcPYpGH1ndc9xkJ8hz7PwLFdfNtt6xyzPtYi8AD89KGm2ohQnE Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 063971e11eb044b4a844a563c6ccbc5d Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerDiskConfigTestJSON-2062980909 Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/063971e11eb044b4a844a563c6ccbc5d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/063971e11eb044b4a844a563c6ccbc5d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/063971e11eb044b4a844a563c6ccbc5d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_063971e11eb044b4a844a563c6ccbc5d"}]}] Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerDiskConfigTestJSON-2062980909 Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 063971e11eb044b4a844a563c6ccbc5d Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-2062980909 Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerDiskConfigTestJSON-2062980909-project Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 74e30240d7c840edba43dddf3a862c4f Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerDiskConfigTestJSON-2062980909-project Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 could not be found. Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'ccbfee66-bd3c-41ec-b09c-9b2b384d1229'} Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 could not be found. Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 could not be found. Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:44:29.928156 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:44:29.932452 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:44:29.932452 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:44:29.932452 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:44:29.932452 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:44:29.932452 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:44:29.932452 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 could not be found. Sep 27 10:44:29.932452 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:44:29.932452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Returning 404 to user: Instance ccbfee66-bd3c-41ec-b09c-9b2b384d1229 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:44:29.932452 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f018e2dd-8db5-4476-bc8c-a8527c956007 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229" status: 404 len: 111 microversion: 2.1 time: 0.044402 Sep 27 10:44:29.932452 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3605/6741] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:29 2022] GET /compute/v2.1/servers/ccbfee66-bd3c-41ec-b09c-9b2b384d1229 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:44:29.939292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c52de462-f004-4599-a290-ef3bcc1738bb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '.version_select at 0x7f7f6255f4c0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:29.963292 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c52de462-f004-4599-a290-ef3bcc1738bb tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1855325960" status: 202 len: 0 microversion: 2.1 time: 0.028290 Sep 27 10:44:29.963887 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3137/6742] 10.222.0.26 () {58 vars in 1221 bytes} [Tue Sep 27 10:44:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1855325960 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:44:30.961186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:30.961820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:30.969197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:30.969699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:30.970147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:30.970865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:30.971351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:30.971788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:30.985827 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] HTTP exception thrown: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Request method failure captured: Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 HTTP/1.1 Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtOlWfxeK3mTR2eKwyq4265ckFp3X5nf3Z46qDJZSXT5dKOr0MdDRk_agSk6WEkwm05LkaUtdHmcGnC_f1F8-x-YlykEz3zdDVpcbddRWayHOb_vBw7TOfdT2prYMOcPYpGH1ndc9xkJ8hz7PwLFdfNtt6xyzPtYi8AD89KGm2ohQnE Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 063971e11eb044b4a844a563c6ccbc5d Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerDiskConfigTestJSON-2062980909 Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/063971e11eb044b4a844a563c6ccbc5d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/063971e11eb044b4a844a563c6ccbc5d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/063971e11eb044b4a844a563c6ccbc5d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_063971e11eb044b4a844a563c6ccbc5d"}]}] Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerDiskConfigTestJSON-2062980909 Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 063971e11eb044b4a844a563c6ccbc5d Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-2062980909 Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerDiskConfigTestJSON-2062980909-project Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 74e30240d7c840edba43dddf3a862c4f Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerDiskConfigTestJSON-2062980909-project Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '25c83c81-2c6e-4d30-81b4-8c71b4d6dce7'} Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:44:30.987418 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Returning 404 to user: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ef8687b9-769f-4e2f-8b3e-b832dd21e269 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "DELETE /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 404 len: 111 microversion: 2.1 time: 0.032657 Sep 27 10:44:30.992075 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3606/6743] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:44:30 2022] DELETE /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:44:30.999037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:30.999818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] [instance: 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:31.006989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:31.007483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:31.007931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:31.008640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:31.009145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:31.009584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:31.022953 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] HTTP exception thrown: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Request method failure captured: Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 HTTP/1.1 Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtOlWfxeK3mTR2eKwyq4265ckFp3X5nf3Z46qDJZSXT5dKOr0MdDRk_agSk6WEkwm05LkaUtdHmcGnC_f1F8-x-YlykEz3zdDVpcbddRWayHOb_vBw7TOfdT2prYMOcPYpGH1ndc9xkJ8hz7PwLFdfNtt6xyzPtYi8AD89KGm2ohQnE Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 063971e11eb044b4a844a563c6ccbc5d Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerDiskConfigTestJSON-2062980909 Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/063971e11eb044b4a844a563c6ccbc5d"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/063971e11eb044b4a844a563c6ccbc5d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/063971e11eb044b4a844a563c6ccbc5d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_063971e11eb044b4a844a563c6ccbc5d"}]}] Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerDiskConfigTestJSON-2062980909 Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 063971e11eb044b4a844a563c6ccbc5d Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerDiskConfigTestJSON-2062980909 Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerDiskConfigTestJSON-2062980909-project Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 74e30240d7c840edba43dddf3a862c4f Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerDiskConfigTestJSON-2062980909-project Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '25c83c81-2c6e-4d30-81b4-8c71b4d6dce7'} Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:44:31.024573 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:44:31.029347 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:44:31.029347 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:44:31.029347 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:44:31.029347 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:44:31.029347 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:44:31.029347 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. Sep 27 10:44:31.029347 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:44:31.029347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] Returning 404 to user: Instance 25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:44:31.029347 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-560cf531-0cfc-4f49-86ab-e93e383bc931 tempest-ServerDiskConfigTestJSON-2062980909 tempest-ServerDiskConfigTestJSON-2062980909-project] 10.222.0.26 "GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7" status: 404 len: 111 microversion: 2.1 time: 0.031801 Sep 27 10:44:31.029347 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3138/6744] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:30 2022] GET /compute/v2.1/servers/25c83c81-2c6e-4d30-81b4-8c71b4d6dce7 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:44:39.671253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b120b0f2-52b2-41fe-ab26-0ccdd636f599 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'create', calling method: .version_select at 0x7f7f6232b940>, body: {"keypair": {"name": "tempest-keypair-107890057", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxsvnw74n24A9nB5y6h76rNq3Bjup4aiqjyK5yfeaZD7bI8thzBlMSTwH5eNNAHJN1haOIRQ9NxoDsr4MdCRNlhEXPzH7ahwWiYSOdnzqTSHoytoqQnx3wO1hrAALfViw=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:44:39.688733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-b120b0f2-52b2-41fe-ab26-0ccdd636f599 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:44:39.694071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-b120b0f2-52b2-41fe-ab26-0ccdd636f599 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for user 396fe9ce0f7542c382b01d1f5779c13f and project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:44:39.725221 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b120b0f2-52b2-41fe-ab26-0ccdd636f599 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.194255 Sep 27 10:44:39.725735 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3607/6745] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:44:39 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:44:41.144324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-1463041461", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8ae6d002-3910-401a-89df-d8a9ff70cdbf"}], "security_groups": [{"name": "tempest-securitygroup--404672176"}], "key_name": "tempest-keypair-107890057", "user_data": "}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:44:41.158216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=6,nova_api:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:41.158963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=53,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:41.269505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] validate_networks() for [('8ae6d002-3910-401a-89df-d8a9ff70cdbf', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:44:41.667321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:44:41.669023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:44:41.669813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:44:41.670521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:44:41.671069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:44:41.682166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:41.682694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:41.683153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:41.683924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:41.684407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:41.684879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:41.704833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:44:41.714639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for user 396fe9ce0f7542c382b01d1f5779c13f and project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:44:41.730447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:44:41.733096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:44:41.786926 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-64e38adc-0996-4b04-a8d8-523247c6dbb5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 0.648336 Sep 27 10:44:41.793851 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3139/6746] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:44:41 2022] POST /compute/v2.1/servers => generated 408 bytes in 656 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:44:41.797324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=57,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:41.801540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bfefb241-4d91-40f9-ba1f-6031b4c604d8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:41.802832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bfefb241-4d91-40f9-ba1f-6031b4c604d8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:41.878710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bfefb241-4d91-40f9-ba1f-6031b4c604d8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:41.886213 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bfefb241-4d91-40f9-ba1f-6031b4c604d8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1238 microversion: 2.1 time: 0.090946 Sep 27 10:44:41.886798 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3608/6747] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:41 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1238 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:42.913523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-88934317-cd94-43f5-812d-393e9ef2b19c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:42.914151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-88934317-cd94-43f5-812d-393e9ef2b19c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:42.922086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88934317-cd94-43f5-812d-393e9ef2b19c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:42.922604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88934317-cd94-43f5-812d-393e9ef2b19c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:42.923042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88934317-cd94-43f5-812d-393e9ef2b19c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:42.924150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88934317-cd94-43f5-812d-393e9ef2b19c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:42.924907 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88934317-cd94-43f5-812d-393e9ef2b19c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:42.925684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88934317-cd94-43f5-812d-393e9ef2b19c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:42.993134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-88934317-cd94-43f5-812d-393e9ef2b19c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:43.001266 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-88934317-cd94-43f5-812d-393e9ef2b19c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1290 microversion: 2.1 time: 0.094770 Sep 27 10:44:43.008753 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3140/6748] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:42 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1290 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:44.025462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cd5d5724-6e27-4f58-9370-49ebb8030af1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:44.026283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cd5d5724-6e27-4f58-9370-49ebb8030af1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:44.033775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5d5724-6e27-4f58-9370-49ebb8030af1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:44.034280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5d5724-6e27-4f58-9370-49ebb8030af1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:44.034729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5d5724-6e27-4f58-9370-49ebb8030af1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:44.035762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5d5724-6e27-4f58-9370-49ebb8030af1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:44.037069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5d5724-6e27-4f58-9370-49ebb8030af1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:44.037671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cd5d5724-6e27-4f58-9370-49ebb8030af1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:44.182922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cd5d5724-6e27-4f58-9370-49ebb8030af1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:44.190910 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cd5d5724-6e27-4f58-9370-49ebb8030af1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1363 microversion: 2.1 time: 0.169879 Sep 27 10:44:44.191481 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3609/6749] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:44 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1363 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:44.714080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ec494a41-2c0e-453c-b991-12af5bcb6aca req-1ee74676-cf7c-40d4-a5e2-8b0fb17dd20c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409", "tag": "b7b9fb46-008c-4668-82ac-5a4994a396a3"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:44:44.725937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ec494a41-2c0e-453c-b991-12af5bcb6aca req-1ee74676-cf7c-40d4-a5e2-8b0fb17dd20c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:44.726616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ec494a41-2c0e-453c-b991-12af5bcb6aca req-1ee74676-cf7c-40d4-a5e2-8b0fb17dd20c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:44.727316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ec494a41-2c0e-453c-b991-12af5bcb6aca req-1ee74676-cf7c-40d4-a5e2-8b0fb17dd20c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:44.740344 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ec494a41-2c0e-453c-b991-12af5bcb6aca req-1ee74676-cf7c-40d4-a5e2-8b0fb17dd20c service nova] Creating event network-changed:b7b9fb46-008c-4668-82ac-5a4994a396a3 for instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 on nch2-master-1 Sep 27 10:44:44.745619 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ec494a41-2c0e-453c-b991-12af5bcb6aca req-1ee74676-cf7c-40d4-a5e2-8b0fb17dd20c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035552 Sep 27 10:44:44.746592 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3141/6750] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:44:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:44:45.215058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0143566-4cdf-4bf5-ac82-26841aefac45 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:45.215840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b0143566-4cdf-4bf5-ac82-26841aefac45 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:45.223596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0143566-4cdf-4bf5-ac82-26841aefac45 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:45.224124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0143566-4cdf-4bf5-ac82-26841aefac45 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:45.224597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0143566-4cdf-4bf5-ac82-26841aefac45 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:45.225369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0143566-4cdf-4bf5-ac82-26841aefac45 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:45.226178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0143566-4cdf-4bf5-ac82-26841aefac45 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:45.226555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0143566-4cdf-4bf5-ac82-26841aefac45 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:45.364170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b0143566-4cdf-4bf5-ac82-26841aefac45 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:45.372454 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b0143566-4cdf-4bf5-ac82-26841aefac45 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1363 microversion: 2.1 time: 0.161462 Sep 27 10:44:45.373074 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3610/6751] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:45 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1363 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:46.402443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cd473cf5-c539-41bc-8cf6-28f7c5149c1b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:46.403534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cd473cf5-c539-41bc-8cf6-28f7c5149c1b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:46.412987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd473cf5-c539-41bc-8cf6-28f7c5149c1b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:46.413483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd473cf5-c539-41bc-8cf6-28f7c5149c1b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:46.413931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd473cf5-c539-41bc-8cf6-28f7c5149c1b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:46.414637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd473cf5-c539-41bc-8cf6-28f7c5149c1b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:46.415131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd473cf5-c539-41bc-8cf6-28f7c5149c1b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:46.415563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cd473cf5-c539-41bc-8cf6-28f7c5149c1b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:46.561429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cd473cf5-c539-41bc-8cf6-28f7c5149c1b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:46.570043 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cd473cf5-c539-41bc-8cf6-28f7c5149c1b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1363 microversion: 2.1 time: 0.176449 Sep 27 10:44:46.570627 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3142/6752] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:46 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1363 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:47.593343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d7e45ebd-de8a-47e1-9ab8-ae28d4e0caae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:47.593968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d7e45ebd-de8a-47e1-9ab8-ae28d4e0caae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:47.601637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7e45ebd-de8a-47e1-9ab8-ae28d4e0caae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:47.602141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7e45ebd-de8a-47e1-9ab8-ae28d4e0caae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:47.602586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7e45ebd-de8a-47e1-9ab8-ae28d4e0caae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:47.603307 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7e45ebd-de8a-47e1-9ab8-ae28d4e0caae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:47.603767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7e45ebd-de8a-47e1-9ab8-ae28d4e0caae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:47.604195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7e45ebd-de8a-47e1-9ab8-ae28d4e0caae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:47.747959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d7e45ebd-de8a-47e1-9ab8-ae28d4e0caae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:47.756359 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d7e45ebd-de8a-47e1-9ab8-ae28d4e0caae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1363 microversion: 2.1 time: 0.167326 Sep 27 10:44:47.756978 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3611/6753] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:47 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1363 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:48.783099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6ddcb2f0-5363-4c37-a9b6-efd1226b35e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:48.783978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6ddcb2f0-5363-4c37-a9b6-efd1226b35e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:48.793392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ddcb2f0-5363-4c37-a9b6-efd1226b35e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:48.794064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ddcb2f0-5363-4c37-a9b6-efd1226b35e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:48.794673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ddcb2f0-5363-4c37-a9b6-efd1226b35e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:48.795649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ddcb2f0-5363-4c37-a9b6-efd1226b35e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:48.796282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ddcb2f0-5363-4c37-a9b6-efd1226b35e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:48.797292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ddcb2f0-5363-4c37-a9b6-efd1226b35e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:48.934618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6ddcb2f0-5363-4c37-a9b6-efd1226b35e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:48.943272 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6ddcb2f0-5363-4c37-a9b6-efd1226b35e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1363 microversion: 2.1 time: 0.165789 Sep 27 10:44:48.943815 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3143/6754] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:48 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1363 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:49.970476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-65c9ec77-5438-411a-bebf-61d169f06577 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:49.971536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-65c9ec77-5438-411a-bebf-61d169f06577 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:49.980028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65c9ec77-5438-411a-bebf-61d169f06577 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:49.980520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65c9ec77-5438-411a-bebf-61d169f06577 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:49.980994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65c9ec77-5438-411a-bebf-61d169f06577 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:49.981705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65c9ec77-5438-411a-bebf-61d169f06577 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:49.982160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65c9ec77-5438-411a-bebf-61d169f06577 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:49.982588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-65c9ec77-5438-411a-bebf-61d169f06577 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:50.122505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-65c9ec77-5438-411a-bebf-61d169f06577 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:50.133631 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-65c9ec77-5438-411a-bebf-61d169f06577 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1363 microversion: 2.1 time: 0.169706 Sep 27 10:44:50.134306 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3612/6755] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:49 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1363 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:50.607516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1cdcf9d0-de29-47ae-baf4-d702427eee76 req-a2281e21-62cc-414e-b24a-87454b3e273f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409", "name": "network-vif-plugged", "status": "completed", "tag": "b7b9fb46-008c-4668-82ac-5a4994a396a3"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:44:50.618894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1cdcf9d0-de29-47ae-baf4-d702427eee76 req-a2281e21-62cc-414e-b24a-87454b3e273f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:50.619382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1cdcf9d0-de29-47ae-baf4-d702427eee76 req-a2281e21-62cc-414e-b24a-87454b3e273f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:50.619823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1cdcf9d0-de29-47ae-baf4-d702427eee76 req-a2281e21-62cc-414e-b24a-87454b3e273f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:50.632467 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1cdcf9d0-de29-47ae-baf4-d702427eee76 req-a2281e21-62cc-414e-b24a-87454b3e273f service nova] Creating event network-vif-plugged:b7b9fb46-008c-4668-82ac-5a4994a396a3 for instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 on nch2-master-1 Sep 27 10:44:50.637690 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1cdcf9d0-de29-47ae-baf4-d702427eee76 req-a2281e21-62cc-414e-b24a-87454b3e273f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034331 Sep 27 10:44:50.638367 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3144/6756] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:44:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:44:51.161466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2fca809c-b9a2-4e49-9d0b-3d6184b2c0d9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:51.163200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2fca809c-b9a2-4e49-9d0b-3d6184b2c0d9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:51.175588 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fca809c-b9a2-4e49-9d0b-3d6184b2c0d9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:51.176418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fca809c-b9a2-4e49-9d0b-3d6184b2c0d9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:51.177221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fca809c-b9a2-4e49-9d0b-3d6184b2c0d9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:51.177994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fca809c-b9a2-4e49-9d0b-3d6184b2c0d9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:51.178443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fca809c-b9a2-4e49-9d0b-3d6184b2c0d9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:51.178871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fca809c-b9a2-4e49-9d0b-3d6184b2c0d9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:51.327171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2fca809c-b9a2-4e49-9d0b-3d6184b2c0d9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:51.335415 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2fca809c-b9a2-4e49-9d0b-3d6184b2c0d9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1363 microversion: 2.1 time: 0.180776 Sep 27 10:44:51.335983 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3613/6757] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:51 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1363 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:52.360122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-067b2414-6022-4ee4-9074-c1e105dda832 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:52.360888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-067b2414-6022-4ee4-9074-c1e105dda832 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:52.363982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:52.364744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:52.370707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-067b2414-6022-4ee4-9074-c1e105dda832 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:52.371212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-067b2414-6022-4ee4-9074-c1e105dda832 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:52.371698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-067b2414-6022-4ee4-9074-c1e105dda832 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:52.372424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-067b2414-6022-4ee4-9074-c1e105dda832 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:52.372921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-067b2414-6022-4ee4-9074-c1e105dda832 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:52.373367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-067b2414-6022-4ee4-9074-c1e105dda832 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:52.523077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-067b2414-6022-4ee4-9074-c1e105dda832 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:44:52.532373 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-067b2414-6022-4ee4-9074-c1e105dda832 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1550 microversion: 2.1 time: 0.176325 Sep 27 10:44:52.533027 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3145/6758] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:44:52 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1550 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:44:52.556755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-78d0b6b9-3bf4-4f4c-9803-7688b612c1d5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:44:52.557296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-78d0b6b9-3bf4-4f4c-9803-7688b612c1d5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:44:52.559327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:44:52.566400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78d0b6b9-3bf4-4f4c-9803-7688b612c1d5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:52.567012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78d0b6b9-3bf4-4f4c-9803-7688b612c1d5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:52.567696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78d0b6b9-3bf4-4f4c-9803-7688b612c1d5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:52.568474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78d0b6b9-3bf4-4f4c-9803-7688b612c1d5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:52.569015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78d0b6b9-3bf4-4f4c-9803-7688b612c1d5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:52.569818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78d0b6b9-3bf4-4f4c-9803-7688b612c1d5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:52.655802 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-78d0b6b9-3bf4-4f4c-9803-7688b612c1d5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.103464 Sep 27 10:44:52.656413 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3614/6759] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:44:52 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409/os-interface => generated 284 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:44:53.886788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-06a1907f-e378-4d9f-8400-ace951d32ce7 req-ed3c3dbd-2e06-413e-b0f5-e4d143a5a5f8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409", "tag": "b7b9fb46-008c-4668-82ac-5a4994a396a3"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:44:53.898235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-06a1907f-e378-4d9f-8400-ace951d32ce7 req-ed3c3dbd-2e06-413e-b0f5-e4d143a5a5f8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:44:53.898658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-06a1907f-e378-4d9f-8400-ace951d32ce7 req-ed3c3dbd-2e06-413e-b0f5-e4d143a5a5f8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:44:53.899086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-06a1907f-e378-4d9f-8400-ace951d32ce7 req-ed3c3dbd-2e06-413e-b0f5-e4d143a5a5f8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:44:53.911075 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-06a1907f-e378-4d9f-8400-ace951d32ce7 req-ed3c3dbd-2e06-413e-b0f5-e4d143a5a5f8 service nova] Creating event network-changed:b7b9fb46-008c-4668-82ac-5a4994a396a3 for instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 on nch2-master-1 Sep 27 10:44:53.916834 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-06a1907f-e378-4d9f-8400-ace951d32ce7 req-ed3c3dbd-2e06-413e-b0f5-e4d143a5a5f8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033918 Sep 27 10:44:53.917475 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3146/6760] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:44:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:45:58.716075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:45:58.717114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=34 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:45:58.722650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1108f712-2064-4c73-9c4b-439cefcbc7fe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:45:58.723279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1108f712-2064-4c73-9c4b-439cefcbc7fe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:45:58.730657 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1108f712-2064-4c73-9c4b-439cefcbc7fe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:45:58.731170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1108f712-2064-4c73-9c4b-439cefcbc7fe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:45:58.731638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1108f712-2064-4c73-9c4b-439cefcbc7fe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:45:58.732366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1108f712-2064-4c73-9c4b-439cefcbc7fe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:45:58.732867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1108f712-2064-4c73-9c4b-439cefcbc7fe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:45:58.733291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1108f712-2064-4c73-9c4b-439cefcbc7fe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:45:58.900322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1108f712-2064-4c73-9c4b-439cefcbc7fe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:45:58.910235 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1108f712-2064-4c73-9c4b-439cefcbc7fe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1668 microversion: 2.1 time: 0.197360 Sep 27 10:45:58.910830 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3615/6761] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:45:58 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1668 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:45:58.945359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:45:58.946251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=32 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:45:58.952587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bd8f182f-dd0d-46fa-9e8b-7cf4e8d34f2a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:45:58.953428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bd8f182f-dd0d-46fa-9e8b-7cf4e8d34f2a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:45:58.961975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd8f182f-dd0d-46fa-9e8b-7cf4e8d34f2a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:45:58.962476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd8f182f-dd0d-46fa-9e8b-7cf4e8d34f2a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:45:58.963046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd8f182f-dd0d-46fa-9e8b-7cf4e8d34f2a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:45:58.963774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd8f182f-dd0d-46fa-9e8b-7cf4e8d34f2a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:45:58.964243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd8f182f-dd0d-46fa-9e8b-7cf4e8d34f2a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:45:58.964685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bd8f182f-dd0d-46fa-9e8b-7cf4e8d34f2a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:45:59.127370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bd8f182f-dd0d-46fa-9e8b-7cf4e8d34f2a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:45:59.135774 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bd8f182f-dd0d-46fa-9e8b-7cf4e8d34f2a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1668 microversion: 2.1 time: 0.194102 Sep 27 10:45:59.136364 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3147/6762] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:45:58 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1668 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:45:59.168574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6598c363-781d-4c64-b12a-cc7b1a54fe46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:45:59.169252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6598c363-781d-4c64-b12a-cc7b1a54fe46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:45:59.176237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6598c363-781d-4c64-b12a-cc7b1a54fe46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:45:59.176751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6598c363-781d-4c64-b12a-cc7b1a54fe46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:45:59.177230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6598c363-781d-4c64-b12a-cc7b1a54fe46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:45:59.177924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6598c363-781d-4c64-b12a-cc7b1a54fe46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:45:59.178404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6598c363-781d-4c64-b12a-cc7b1a54fe46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:45:59.178830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6598c363-781d-4c64-b12a-cc7b1a54fe46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:45:59.337721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6598c363-781d-4c64-b12a-cc7b1a54fe46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:45:59.346567 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6598c363-781d-4c64-b12a-cc7b1a54fe46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1668 microversion: 2.1 time: 0.184036 Sep 27 10:45:59.347125 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3616/6763] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:45:59 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1668 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:45:59.372271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:45:59.374642 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:45:59.381161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:45:59.381651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:45:59.382096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:45:59.382800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:45:59.383261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:45:59.383694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:45:59.413047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:45:59.418710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Old instance type m1.heat2, new instance type m1.heat {{(pid=145969) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:45:59.425845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:45:59.426270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:45:59.426754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:45:59.427176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:45:59.427592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:45:59.498691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5348f97b-ffe8-4b7f-8838-4aa56eaf2da6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409/action" status: 202 len: 0 microversion: 2.1 time: 0.130465 Sep 27 10:45:59.499694 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3148/6764] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:45:59 2022] POST /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409/action => generated 0 bytes in 132 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:45:59.513327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6e3c47bb-3235-4651-8677-bcb1ddc5d836 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:45:59.514462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6e3c47bb-3235-4651-8677-bcb1ddc5d836 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:45:59.524778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e3c47bb-3235-4651-8677-bcb1ddc5d836 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:45:59.525857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e3c47bb-3235-4651-8677-bcb1ddc5d836 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:45:59.526692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e3c47bb-3235-4651-8677-bcb1ddc5d836 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:45:59.528092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e3c47bb-3235-4651-8677-bcb1ddc5d836 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:45:59.529036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e3c47bb-3235-4651-8677-bcb1ddc5d836 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:45:59.529825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e3c47bb-3235-4651-8677-bcb1ddc5d836 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:45:59.705199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6e3c47bb-3235-4651-8677-bcb1ddc5d836 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:45:59.713890 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6e3c47bb-3235-4651-8677-bcb1ddc5d836 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1677 microversion: 2.1 time: 0.207993 Sep 27 10:45:59.714449 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3617/6765] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:45:59 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1677 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:00.739273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a7815399-4650-4d2a-92d2-167e340927d3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:00.739810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a7815399-4650-4d2a-92d2-167e340927d3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:00.747109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7815399-4650-4d2a-92d2-167e340927d3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:00.747608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7815399-4650-4d2a-92d2-167e340927d3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:00.748056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7815399-4650-4d2a-92d2-167e340927d3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:00.748811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7815399-4650-4d2a-92d2-167e340927d3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:00.749283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7815399-4650-4d2a-92d2-167e340927d3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:00.749721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7815399-4650-4d2a-92d2-167e340927d3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:00.890010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a7815399-4650-4d2a-92d2-167e340927d3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:00.898050 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a7815399-4650-4d2a-92d2-167e340927d3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1677 microversion: 2.1 time: 0.162840 Sep 27 10:46:00.898619 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3149/6766] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:00 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1677 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:01.924647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb23bf1e-f11e-4bef-9ca8-e9e71821354b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:01.925306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cb23bf1e-f11e-4bef-9ca8-e9e71821354b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:01.933062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb23bf1e-f11e-4bef-9ca8-e9e71821354b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:01.933576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb23bf1e-f11e-4bef-9ca8-e9e71821354b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:01.934051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb23bf1e-f11e-4bef-9ca8-e9e71821354b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:01.935533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb23bf1e-f11e-4bef-9ca8-e9e71821354b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:01.936005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb23bf1e-f11e-4bef-9ca8-e9e71821354b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:01.936441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb23bf1e-f11e-4bef-9ca8-e9e71821354b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:02.076164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb23bf1e-f11e-4bef-9ca8-e9e71821354b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:02.084639 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cb23bf1e-f11e-4bef-9ca8-e9e71821354b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1682 microversion: 2.1 time: 0.164908 Sep 27 10:46:02.085259 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3618/6767] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:01 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1682 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:03.109376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ad032605-ddfd-42af-a2c9-a5c8ff04fbf4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:03.109990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ad032605-ddfd-42af-a2c9-a5c8ff04fbf4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:03.117477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad032605-ddfd-42af-a2c9-a5c8ff04fbf4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:03.117983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad032605-ddfd-42af-a2c9-a5c8ff04fbf4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:03.118440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad032605-ddfd-42af-a2c9-a5c8ff04fbf4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:03.119164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad032605-ddfd-42af-a2c9-a5c8ff04fbf4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:03.119640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad032605-ddfd-42af-a2c9-a5c8ff04fbf4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:03.120095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ad032605-ddfd-42af-a2c9-a5c8ff04fbf4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:03.267850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ad032605-ddfd-42af-a2c9-a5c8ff04fbf4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:03.276193 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ad032605-ddfd-42af-a2c9-a5c8ff04fbf4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1682 microversion: 2.1 time: 0.171004 Sep 27 10:46:03.276927 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3150/6768] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:03 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1682 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:04.305852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2ae5d671-f45f-4218-880f-d88aba251933 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:04.306895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2ae5d671-f45f-4218-880f-d88aba251933 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:04.319296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5d671-f45f-4218-880f-d88aba251933 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:04.320090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5d671-f45f-4218-880f-d88aba251933 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:04.320965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5d671-f45f-4218-880f-d88aba251933 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:04.322128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5d671-f45f-4218-880f-d88aba251933 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:04.322908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5d671-f45f-4218-880f-d88aba251933 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:04.323635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2ae5d671-f45f-4218-880f-d88aba251933 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:04.466342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2ae5d671-f45f-4218-880f-d88aba251933 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:04.474267 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2ae5d671-f45f-4218-880f-d88aba251933 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1682 microversion: 2.1 time: 0.175236 Sep 27 10:46:04.474842 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3619/6769] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:04 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1682 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:05.500870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-72cb5018-4c30-4e90-b57f-0a2bd53129a5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:05.501482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-72cb5018-4c30-4e90-b57f-0a2bd53129a5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:05.508902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72cb5018-4c30-4e90-b57f-0a2bd53129a5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:05.509402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72cb5018-4c30-4e90-b57f-0a2bd53129a5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:05.509848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72cb5018-4c30-4e90-b57f-0a2bd53129a5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:05.510558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72cb5018-4c30-4e90-b57f-0a2bd53129a5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:05.511032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72cb5018-4c30-4e90-b57f-0a2bd53129a5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:05.511469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-72cb5018-4c30-4e90-b57f-0a2bd53129a5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:05.650294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-72cb5018-4c30-4e90-b57f-0a2bd53129a5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:05.658679 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-72cb5018-4c30-4e90-b57f-0a2bd53129a5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1682 microversion: 2.1 time: 0.162137 Sep 27 10:46:05.659275 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3151/6770] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:05 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1682 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:06.687685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-42260ebb-c305-4760-a680-a0569b62d6e5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:06.688715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-42260ebb-c305-4760-a680-a0569b62d6e5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:06.700476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42260ebb-c305-4760-a680-a0569b62d6e5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:06.701115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42260ebb-c305-4760-a680-a0569b62d6e5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:06.701561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42260ebb-c305-4760-a680-a0569b62d6e5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:06.702274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42260ebb-c305-4760-a680-a0569b62d6e5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:06.702743 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42260ebb-c305-4760-a680-a0569b62d6e5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:06.703180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42260ebb-c305-4760-a680-a0569b62d6e5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:06.852938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-42260ebb-c305-4760-a680-a0569b62d6e5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:06.862294 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-42260ebb-c305-4760-a680-a0569b62d6e5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1682 microversion: 2.1 time: 0.181658 Sep 27 10:46:06.862840 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3620/6771] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:06 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1682 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:07.890523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b1a4187d-c898-4ac6-9bc1-0e4881aa15b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:07.891394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b1a4187d-c898-4ac6-9bc1-0e4881aa15b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:07.903478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1a4187d-c898-4ac6-9bc1-0e4881aa15b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:07.904056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1a4187d-c898-4ac6-9bc1-0e4881aa15b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:07.904502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1a4187d-c898-4ac6-9bc1-0e4881aa15b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:07.905251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1a4187d-c898-4ac6-9bc1-0e4881aa15b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:07.905683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1a4187d-c898-4ac6-9bc1-0e4881aa15b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:07.906152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b1a4187d-c898-4ac6-9bc1-0e4881aa15b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:08.047363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b1a4187d-c898-4ac6-9bc1-0e4881aa15b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:08.055475 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b1a4187d-c898-4ac6-9bc1-0e4881aa15b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1682 microversion: 2.1 time: 0.171851 Sep 27 10:46:08.056056 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3152/6772] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:07 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1682 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:09.084833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-528dd7e7-a882-4f04-bdc2-0300845b4f39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:09.085517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-528dd7e7-a882-4f04-bdc2-0300845b4f39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:09.094442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-528dd7e7-a882-4f04-bdc2-0300845b4f39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:09.095254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-528dd7e7-a882-4f04-bdc2-0300845b4f39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:09.095810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-528dd7e7-a882-4f04-bdc2-0300845b4f39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:09.096781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-528dd7e7-a882-4f04-bdc2-0300845b4f39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:09.097271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-528dd7e7-a882-4f04-bdc2-0300845b4f39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:09.097775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-528dd7e7-a882-4f04-bdc2-0300845b4f39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:09.243255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-528dd7e7-a882-4f04-bdc2-0300845b4f39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:09.251679 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-528dd7e7-a882-4f04-bdc2-0300845b4f39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1682 microversion: 2.1 time: 0.175093 Sep 27 10:46:09.252290 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3621/6773] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:09 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1682 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:10.277820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-45ec6f54-93c6-4897-a727-5e813fdeee36 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:10.278486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-45ec6f54-93c6-4897-a727-5e813fdeee36 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:10.289728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ec6f54-93c6-4897-a727-5e813fdeee36 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:10.290349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ec6f54-93c6-4897-a727-5e813fdeee36 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:10.290956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ec6f54-93c6-4897-a727-5e813fdeee36 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:10.291933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ec6f54-93c6-4897-a727-5e813fdeee36 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:10.292552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ec6f54-93c6-4897-a727-5e813fdeee36 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:10.293146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-45ec6f54-93c6-4897-a727-5e813fdeee36 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:10.437471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-45ec6f54-93c6-4897-a727-5e813fdeee36 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:10.449673 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-45ec6f54-93c6-4897-a727-5e813fdeee36 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1682 microversion: 2.1 time: 0.177265 Sep 27 10:46:10.450682 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3153/6774] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:10 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1682 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:11.474101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-30c2f2c8-b032-49fd-a791-ce5621bd9d1d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:11.474891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-30c2f2c8-b032-49fd-a791-ce5621bd9d1d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:11.482134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30c2f2c8-b032-49fd-a791-ce5621bd9d1d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:11.482624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30c2f2c8-b032-49fd-a791-ce5621bd9d1d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:11.483061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30c2f2c8-b032-49fd-a791-ce5621bd9d1d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:11.483911 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30c2f2c8-b032-49fd-a791-ce5621bd9d1d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:11.484701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30c2f2c8-b032-49fd-a791-ce5621bd9d1d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:11.485151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-30c2f2c8-b032-49fd-a791-ce5621bd9d1d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:11.623341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-30c2f2c8-b032-49fd-a791-ce5621bd9d1d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:11.631460 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-30c2f2c8-b032-49fd-a791-ce5621bd9d1d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1682 microversion: 2.1 time: 0.161439 Sep 27 10:46:11.632076 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3622/6775] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:11 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1682 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:12.658799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1bd612fb-95d2-48ab-b039-d5396b9063af tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:12.659747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1bd612fb-95d2-48ab-b039-d5396b9063af tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:12.670933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd612fb-95d2-48ab-b039-d5396b9063af tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:12.671694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd612fb-95d2-48ab-b039-d5396b9063af tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:12.672568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd612fb-95d2-48ab-b039-d5396b9063af tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:12.673326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd612fb-95d2-48ab-b039-d5396b9063af tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:12.673796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd612fb-95d2-48ab-b039-d5396b9063af tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:12.674227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bd612fb-95d2-48ab-b039-d5396b9063af tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:12.826980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1bd612fb-95d2-48ab-b039-d5396b9063af tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:12.839512 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1bd612fb-95d2-48ab-b039-d5396b9063af tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1682 microversion: 2.1 time: 0.186918 Sep 27 10:46:12.849474 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3154/6776] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:12 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1682 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:13.863996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c1e1de1c-706a-4313-b36d-58c0a2438edd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:13.864562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c1e1de1c-706a-4313-b36d-58c0a2438edd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:13.872010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1e1de1c-706a-4313-b36d-58c0a2438edd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:13.872579 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1e1de1c-706a-4313-b36d-58c0a2438edd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:13.873101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1e1de1c-706a-4313-b36d-58c0a2438edd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:13.873893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1e1de1c-706a-4313-b36d-58c0a2438edd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:13.874366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1e1de1c-706a-4313-b36d-58c0a2438edd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:13.874850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1e1de1c-706a-4313-b36d-58c0a2438edd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:14.009659 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c1e1de1c-706a-4313-b36d-58c0a2438edd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:14.018302 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c1e1de1c-706a-4313-b36d-58c0a2438edd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1681 microversion: 2.1 time: 0.158656 Sep 27 10:46:14.018879 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3623/6777] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:13 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1681 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:14.282374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e8b5ed27-2b0d-4247-8e47-68e430adce5e req-78677dc2-0f2b-44a5-a107-eeb6b1387e7c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409", "name": "network-vif-unplugged", "status": "completed", "tag": "b7b9fb46-008c-4668-82ac-5a4994a396a3"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:46:14.293159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e8b5ed27-2b0d-4247-8e47-68e430adce5e req-78677dc2-0f2b-44a5-a107-eeb6b1387e7c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:14.293754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e8b5ed27-2b0d-4247-8e47-68e430adce5e req-78677dc2-0f2b-44a5-a107-eeb6b1387e7c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:14.294763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e8b5ed27-2b0d-4247-8e47-68e430adce5e req-78677dc2-0f2b-44a5-a107-eeb6b1387e7c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:14.308229 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e8b5ed27-2b0d-4247-8e47-68e430adce5e req-78677dc2-0f2b-44a5-a107-eeb6b1387e7c service nova] Creating event network-vif-unplugged:b7b9fb46-008c-4668-82ac-5a4994a396a3 for instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 on nch1-master-1 Sep 27 10:46:14.314009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-e8b5ed27-2b0d-4247-8e47-68e430adce5e req-78677dc2-0f2b-44a5-a107-eeb6b1387e7c service nova] Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:46:14.325216 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e8b5ed27-2b0d-4247-8e47-68e430adce5e req-78677dc2-0f2b-44a5-a107-eeb6b1387e7c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.047662 Sep 27 10:46:14.326320 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3155/6778] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:46:14 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:46:15.043170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f973fc87-682d-4750-9661-2f72856947e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:15.043820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f973fc87-682d-4750-9661-2f72856947e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:15.051865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f973fc87-682d-4750-9661-2f72856947e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:15.052326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f973fc87-682d-4750-9661-2f72856947e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:15.052810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f973fc87-682d-4750-9661-2f72856947e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:15.053526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f973fc87-682d-4750-9661-2f72856947e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:15.053979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f973fc87-682d-4750-9661-2f72856947e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:15.054473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f973fc87-682d-4750-9661-2f72856947e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:15.188705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f973fc87-682d-4750-9661-2f72856947e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:15.197131 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f973fc87-682d-4750-9661-2f72856947e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1681 microversion: 2.1 time: 0.158163 Sep 27 10:46:15.197735 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3624/6779] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:15 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1681 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:16.225707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6ae12eac-b58d-480a-be4f-6ca89c496917 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:16.226725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6ae12eac-b58d-480a-be4f-6ca89c496917 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:16.235871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ae12eac-b58d-480a-be4f-6ca89c496917 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:16.236369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ae12eac-b58d-480a-be4f-6ca89c496917 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:16.236863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ae12eac-b58d-480a-be4f-6ca89c496917 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:16.237583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ae12eac-b58d-480a-be4f-6ca89c496917 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:16.238057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ae12eac-b58d-480a-be4f-6ca89c496917 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:16.238480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ae12eac-b58d-480a-be4f-6ca89c496917 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:16.363593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-32c142e4-1f56-4f98-85cb-49781d2d8c3e req-9877323c-f435-4a88-9a25-334e4ca4baba service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409", "tag": "b7b9fb46-008c-4668-82ac-5a4994a396a3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:46:16.374457 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-32c142e4-1f56-4f98-85cb-49781d2d8c3e req-9877323c-f435-4a88-9a25-334e4ca4baba service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:16.374950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-32c142e4-1f56-4f98-85cb-49781d2d8c3e req-9877323c-f435-4a88-9a25-334e4ca4baba service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:16.375389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-32c142e4-1f56-4f98-85cb-49781d2d8c3e req-9877323c-f435-4a88-9a25-334e4ca4baba service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:16.387551 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-32c142e4-1f56-4f98-85cb-49781d2d8c3e req-9877323c-f435-4a88-9a25-334e4ca4baba service nova] Creating event network-changed:b7b9fb46-008c-4668-82ac-5a4994a396a3 for instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 on nch1-master-1 Sep 27 10:46:16.393250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-32c142e4-1f56-4f98-85cb-49781d2d8c3e req-9877323c-f435-4a88-9a25-334e4ca4baba service nova] Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:46:16.393509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6ae12eac-b58d-480a-be4f-6ca89c496917 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:16.403232 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6ae12eac-b58d-480a-be4f-6ca89c496917 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1679 microversion: 2.1 time: 0.184362 Sep 27 10:46:16.403951 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3156/6780] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:16 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1679 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:16.404380 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-32c142e4-1f56-4f98-85cb-49781d2d8c3e req-9877323c-f435-4a88-9a25-334e4ca4baba service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044888 Sep 27 10:46:16.405197 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3625/6781] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:46:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:46:17.427740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-41a82475-c000-4b23-a9ad-4abf50c2ae2c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:17.429034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-41a82475-c000-4b23-a9ad-4abf50c2ae2c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:17.436333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41a82475-c000-4b23-a9ad-4abf50c2ae2c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:17.436888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41a82475-c000-4b23-a9ad-4abf50c2ae2c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:17.437347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41a82475-c000-4b23-a9ad-4abf50c2ae2c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:17.438181 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41a82475-c000-4b23-a9ad-4abf50c2ae2c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:17.438640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41a82475-c000-4b23-a9ad-4abf50c2ae2c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:17.439110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-41a82475-c000-4b23-a9ad-4abf50c2ae2c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:17.676175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-41a82475-c000-4b23-a9ad-4abf50c2ae2c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:17.684267 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-41a82475-c000-4b23-a9ad-4abf50c2ae2c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1676 microversion: 2.1 time: 0.260548 Sep 27 10:46:17.684892 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3157/6782] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:17 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1676 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:17.710316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-11546c5b-ec1b-4ed2-8202-fc0eafbd253f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:46:17.710939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-11546c5b-ec1b-4ed2-8202-fc0eafbd253f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:17.717676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11546c5b-ec1b-4ed2-8202-fc0eafbd253f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:17.718222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11546c5b-ec1b-4ed2-8202-fc0eafbd253f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:17.718700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11546c5b-ec1b-4ed2-8202-fc0eafbd253f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:17.719840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11546c5b-ec1b-4ed2-8202-fc0eafbd253f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:17.720298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11546c5b-ec1b-4ed2-8202-fc0eafbd253f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:17.720768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11546c5b-ec1b-4ed2-8202-fc0eafbd253f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:17.785180 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-11546c5b-ec1b-4ed2-8202-fc0eafbd253f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409/action" status: 204 len: 0 microversion: 2.1 time: 0.080917 Sep 27 10:46:17.785899 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3626/6783] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:46:17 2022] POST /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409/action => generated 0 bytes in 83 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:46:17.796358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25506f59-9a8e-41a6-aaf7-8f0b5c1cf6c7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:17.797195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-25506f59-9a8e-41a6-aaf7-8f0b5c1cf6c7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:17.804184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25506f59-9a8e-41a6-aaf7-8f0b5c1cf6c7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:17.804771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25506f59-9a8e-41a6-aaf7-8f0b5c1cf6c7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:17.805285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25506f59-9a8e-41a6-aaf7-8f0b5c1cf6c7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:17.806167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25506f59-9a8e-41a6-aaf7-8f0b5c1cf6c7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:17.806694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25506f59-9a8e-41a6-aaf7-8f0b5c1cf6c7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:17.807210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25506f59-9a8e-41a6-aaf7-8f0b5c1cf6c7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:18.018396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-25506f59-9a8e-41a6-aaf7-8f0b5c1cf6c7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:18.026692 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-25506f59-9a8e-41a6-aaf7-8f0b5c1cf6c7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1676 microversion: 2.1 time: 0.235374 Sep 27 10:46:18.027279 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3158/6784] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:17 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1676 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:19.052162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e2c2d45b-3a7c-4314-81af-f7b2db835e4e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:19.053702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e2c2d45b-3a7c-4314-81af-f7b2db835e4e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:19.062225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c2d45b-3a7c-4314-81af-f7b2db835e4e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:19.063403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c2d45b-3a7c-4314-81af-f7b2db835e4e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:19.064276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c2d45b-3a7c-4314-81af-f7b2db835e4e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:19.065286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c2d45b-3a7c-4314-81af-f7b2db835e4e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:19.065828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c2d45b-3a7c-4314-81af-f7b2db835e4e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:19.066266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e2c2d45b-3a7c-4314-81af-f7b2db835e4e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:19.218484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e2c2d45b-3a7c-4314-81af-f7b2db835e4e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:19.228610 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e2c2d45b-3a7c-4314-81af-f7b2db835e4e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1676 microversion: 2.1 time: 0.180943 Sep 27 10:46:19.230563 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3627/6785] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:19 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1676 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:20.259167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-534c050a-88d3-47b7-a29b-4adf4b66a8be tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:20.260131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-534c050a-88d3-47b7-a29b-4adf4b66a8be tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:20.269792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-534c050a-88d3-47b7-a29b-4adf4b66a8be tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:20.270298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-534c050a-88d3-47b7-a29b-4adf4b66a8be tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:20.270754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-534c050a-88d3-47b7-a29b-4adf4b66a8be tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:20.271476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-534c050a-88d3-47b7-a29b-4adf4b66a8be tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:20.271950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-534c050a-88d3-47b7-a29b-4adf4b66a8be tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:20.272392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-534c050a-88d3-47b7-a29b-4adf4b66a8be tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:20.421945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-534c050a-88d3-47b7-a29b-4adf4b66a8be tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:20.433548 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-534c050a-88d3-47b7-a29b-4adf4b66a8be tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1668 microversion: 2.1 time: 0.180959 Sep 27 10:46:20.434338 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3159/6786] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:20 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1668 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:20.457135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5c0895df-77e7-4b99-82f8-a1fb294c6d3f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:20.457877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5c0895df-77e7-4b99-82f8-a1fb294c6d3f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:20.465329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c0895df-77e7-4b99-82f8-a1fb294c6d3f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:20.465824 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c0895df-77e7-4b99-82f8-a1fb294c6d3f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:20.466263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c0895df-77e7-4b99-82f8-a1fb294c6d3f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:20.466973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c0895df-77e7-4b99-82f8-a1fb294c6d3f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:20.467431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c0895df-77e7-4b99-82f8-a1fb294c6d3f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:20.467897 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5c0895df-77e7-4b99-82f8-a1fb294c6d3f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:20.613154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5c0895df-77e7-4b99-82f8-a1fb294c6d3f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:20.621760 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5c0895df-77e7-4b99-82f8-a1fb294c6d3f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1668 microversion: 2.1 time: 0.168641 Sep 27 10:46:20.622340 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3628/6787] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:20 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1668 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:20.645550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c5f3e412-252b-49f0-b508-fbda8d275ea4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:20.646177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c5f3e412-252b-49f0-b508-fbda8d275ea4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:20.652493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5f3e412-252b-49f0-b508-fbda8d275ea4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:20.653049 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5f3e412-252b-49f0-b508-fbda8d275ea4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:20.653498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5f3e412-252b-49f0-b508-fbda8d275ea4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:20.654223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5f3e412-252b-49f0-b508-fbda8d275ea4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:20.654696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5f3e412-252b-49f0-b508-fbda8d275ea4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:20.655138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c5f3e412-252b-49f0-b508-fbda8d275ea4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:20.811476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c5f3e412-252b-49f0-b508-fbda8d275ea4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:20.820618 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c5f3e412-252b-49f0-b508-fbda8d275ea4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1668 microversion: 2.1 time: 0.179096 Sep 27 10:46:20.821333 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3160/6788] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:20 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1668 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:20.843999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-72a4a3e9-486a-46ad-a40d-af8cc6a72fa6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:20.844997 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-72a4a3e9-486a-46ad-a40d-af8cc6a72fa6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:20.851697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72a4a3e9-486a-46ad-a40d-af8cc6a72fa6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:20.852201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72a4a3e9-486a-46ad-a40d-af8cc6a72fa6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:20.852646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72a4a3e9-486a-46ad-a40d-af8cc6a72fa6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:20.853394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72a4a3e9-486a-46ad-a40d-af8cc6a72fa6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:20.853861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72a4a3e9-486a-46ad-a40d-af8cc6a72fa6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:20.854270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72a4a3e9-486a-46ad-a40d-af8cc6a72fa6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:20.881523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-72a4a3e9-486a-46ad-a40d-af8cc6a72fa6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:46:20.915441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:20.968362 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-72a4a3e9-486a-46ad-a40d-af8cc6a72fa6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "DELETE /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 204 len: 0 microversion: 2.1 time: 0.127988 Sep 27 10:46:20.969280 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3629/6789] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:46:20 2022] DELETE /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:46:20.978989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9455617-44ec-436b-ae5c-bad7489fabfb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:20.979623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9455617-44ec-436b-ae5c-bad7489fabfb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:20.986159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9455617-44ec-436b-ae5c-bad7489fabfb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:20.987067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9455617-44ec-436b-ae5c-bad7489fabfb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:20.987517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9455617-44ec-436b-ae5c-bad7489fabfb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:20.988207 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9455617-44ec-436b-ae5c-bad7489fabfb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:20.988921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9455617-44ec-436b-ae5c-bad7489fabfb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:20.989039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-8a664d05-896e-419c-831e-55741ba72794 req-d28618fc-f98b-4b1a-a083-d567c03389e8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409", "name": "network-vif-plugged", "status": "completed", "tag": "b7b9fb46-008c-4668-82ac-5a4994a396a3"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:46:20.989375 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9455617-44ec-436b-ae5c-bad7489fabfb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:21.002010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8a664d05-896e-419c-831e-55741ba72794 req-d28618fc-f98b-4b1a-a083-d567c03389e8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:21.002589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8a664d05-896e-419c-831e-55741ba72794 req-d28618fc-f98b-4b1a-a083-d567c03389e8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:21.003102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-8a664d05-896e-419c-831e-55741ba72794 req-d28618fc-f98b-4b1a-a083-d567c03389e8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:21.015488 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-8a664d05-896e-419c-831e-55741ba72794 req-d28618fc-f98b-4b1a-a083-d567c03389e8 service nova] Creating event network-vif-plugged:b7b9fb46-008c-4668-82ac-5a4994a396a3 for instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 on nch1-master-1 Sep 27 10:46:21.020775 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-8a664d05-896e-419c-831e-55741ba72794 req-d28618fc-f98b-4b1a-a083-d567c03389e8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036494 Sep 27 10:46:21.021710 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3630/6790] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:46:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:46:21.123541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c9455617-44ec-436b-ae5c-bad7489fabfb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:21.131601 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9455617-44ec-436b-ae5c-bad7489fabfb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1674 microversion: 2.1 time: 0.157718 Sep 27 10:46:21.132181 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3161/6791] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:20 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1674 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:22.156799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-626a7efe-314f-49e1-a17b-178a6570d076 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:22.157507 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-626a7efe-314f-49e1-a17b-178a6570d076 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:22.165236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-626a7efe-314f-49e1-a17b-178a6570d076 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:22.165733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-626a7efe-314f-49e1-a17b-178a6570d076 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:22.166185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-626a7efe-314f-49e1-a17b-178a6570d076 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:22.166891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-626a7efe-314f-49e1-a17b-178a6570d076 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:22.167356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-626a7efe-314f-49e1-a17b-178a6570d076 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:22.167823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-626a7efe-314f-49e1-a17b-178a6570d076 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:22.312923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-626a7efe-314f-49e1-a17b-178a6570d076 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:22.325335 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-626a7efe-314f-49e1-a17b-178a6570d076 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1674 microversion: 2.1 time: 0.173554 Sep 27 10:46:22.326120 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3631/6792] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:22 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1674 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:23.271397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e5b16116-3a6f-4ca6-a1cb-108a13b6ca55 req-6f4ee299-74d1-40c1-a5f9-929de371747b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409", "name": "network-vif-deleted", "tag": "b7b9fb46-008c-4668-82ac-5a4994a396a3"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:46:23.284211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e5b16116-3a6f-4ca6-a1cb-108a13b6ca55 req-6f4ee299-74d1-40c1-a5f9-929de371747b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:23.284803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e5b16116-3a6f-4ca6-a1cb-108a13b6ca55 req-6f4ee299-74d1-40c1-a5f9-929de371747b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:23.285253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e5b16116-3a6f-4ca6-a1cb-108a13b6ca55 req-6f4ee299-74d1-40c1-a5f9-929de371747b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:23.301704 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e5b16116-3a6f-4ca6-a1cb-108a13b6ca55 req-6f4ee299-74d1-40c1-a5f9-929de371747b service nova] Creating event network-vif-deleted:b7b9fb46-008c-4668-82ac-5a4994a396a3 for instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 on nch1-master-1 Sep 27 10:46:23.306726 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e5b16116-3a6f-4ca6-a1cb-108a13b6ca55 req-6f4ee299-74d1-40c1-a5f9-929de371747b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039070 Sep 27 10:46:23.307367 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3162/6793] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:46:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:46:23.351114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:23.352150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:23.362591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:23.363351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:23.364106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:23.365048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:23.365509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:23.365936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:23.401131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:23.401633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:23.402075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:23.402584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lazy-loading 'fault' on Instance uuid 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:46:23.446765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:23.456387 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c813a7bc-ce3d-4be9-bbc3-b98283080074 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 200 len: 1328 microversion: 2.1 time: 0.110144 Sep 27 10:46:23.457418 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3632/6794] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:23 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 1328 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:24.486663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:24.487226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:24.494134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:24.494631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:24.495084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:24.495787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:24.496257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:24.496695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:24.500849 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:24.511723 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 HTTP/1.1 Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409'} Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:46:24.513555 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:46:24.518816 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:46:24.518816 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:46:24.518816 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:46:24.518816 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:46:24.518816 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:46:24.518816 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:46:24.518816 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:46:24.518816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:46:24.518816 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-baab6156-1015-4f53-8e76-e2e191ca3ca4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 404 len: 111 microversion: 2.1 time: 0.036447 Sep 27 10:46:24.518816 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3163/6795] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:24 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:46:24.530992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:24.531746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:24.542603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:24.543286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:24.544165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:24.545273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:24.545916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:24.546544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:24.560265 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 HTTP/1.1 Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409'} Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:46:24.562004 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:46:24.566213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:46:24.566213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:46:24.566213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:46:24.566213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:46:24.566213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:46:24.566213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:46:24.566213 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:46:24.566213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:46:24.566213 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cb69fe4f-6dbe-461b-b879-7f2e082f98d4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 404 len: 111 microversion: 2.1 time: 0.038721 Sep 27 10:46:24.566213 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3633/6796] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:24 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:46:24.614322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-343840091", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "8ae6d002-3910-401a-89df-d8a9ff70cdbf"}], "security_groups": [{"name": "tempest-securitygroup--404672176"}], "key_name": "tempest-keypair-107890057", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:46:24.729184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] validate_networks() for [('8ae6d002-3910-401a-89df-d8a9ff70cdbf', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:46:25.124711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:46:25.125265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:46:25.126086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:46:25.126550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:46:25.126993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:46:25.136673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:25.137195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:25.137652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:25.138408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:25.138885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:25.139329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:25.157256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:46:25.163221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for user 396fe9ce0f7542c382b01d1f5779c13f and project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:46:25.177140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:46:25.179567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:46:25.236682 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-63d28b25-48ac-4633-9c49-df2df060b463 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.626306 Sep 27 10:46:25.237361 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3164/6797] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:46:24 2022] POST /compute/v2.1/servers => generated 405 bytes in 628 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:46:25.249524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-77a31c59-67c1-4e2e-b1e8-d97d56663b27 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:25.250870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-77a31c59-67c1-4e2e-b1e8-d97d56663b27 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:25.309220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-77a31c59-67c1-4e2e-b1e8-d97d56663b27 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:25.317094 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-77a31c59-67c1-4e2e-b1e8-d97d56663b27 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1237 microversion: 2.1 time: 0.071940 Sep 27 10:46:25.317662 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3634/6798] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:25 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1237 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:46:26.345361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fa139dd2-e840-4d9c-8c57-3d72cd1329c5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:26.346172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa139dd2-e840-4d9c-8c57-3d72cd1329c5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:26.357658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa139dd2-e840-4d9c-8c57-3d72cd1329c5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:26.358275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa139dd2-e840-4d9c-8c57-3d72cd1329c5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:26.358764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa139dd2-e840-4d9c-8c57-3d72cd1329c5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:26.359486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa139dd2-e840-4d9c-8c57-3d72cd1329c5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:26.360001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa139dd2-e840-4d9c-8c57-3d72cd1329c5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:26.360779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa139dd2-e840-4d9c-8c57-3d72cd1329c5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:26.452461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fa139dd2-e840-4d9c-8c57-3d72cd1329c5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:26.461165 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fa139dd2-e840-4d9c-8c57-3d72cd1329c5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1297 microversion: 2.1 time: 0.122384 Sep 27 10:46:26.469208 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3165/6799] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:26 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1297 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:27.486038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b07adbd6-4eda-46b4-ac75-375afd73acec tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:27.486638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b07adbd6-4eda-46b4-ac75-375afd73acec tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:27.497278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b07adbd6-4eda-46b4-ac75-375afd73acec tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:27.497766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b07adbd6-4eda-46b4-ac75-375afd73acec tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:27.498213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b07adbd6-4eda-46b4-ac75-375afd73acec tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:27.499028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b07adbd6-4eda-46b4-ac75-375afd73acec tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:27.499493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b07adbd6-4eda-46b4-ac75-375afd73acec tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:27.499919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b07adbd6-4eda-46b4-ac75-375afd73acec tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:27.645326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b07adbd6-4eda-46b4-ac75-375afd73acec tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:27.654394 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b07adbd6-4eda-46b4-ac75-375afd73acec tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1362 microversion: 2.1 time: 0.172332 Sep 27 10:46:27.655073 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3635/6800] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:27 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1362 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:27.918936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-c733e52e-47ef-4517-a6b2-869b4bb90442 req-e71bc321-d23b-4eab-b39a-2b03a9466515 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4b7833f5-ef39-4414-92b8-0eaaa26eb4a7", "tag": "7579c2c0-f784-4f9e-b818-50eb66f8013f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:46:27.930269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c733e52e-47ef-4517-a6b2-869b4bb90442 req-e71bc321-d23b-4eab-b39a-2b03a9466515 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:27.930760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c733e52e-47ef-4517-a6b2-869b4bb90442 req-e71bc321-d23b-4eab-b39a-2b03a9466515 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:27.931200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c733e52e-47ef-4517-a6b2-869b4bb90442 req-e71bc321-d23b-4eab-b39a-2b03a9466515 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:27.942666 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c733e52e-47ef-4517-a6b2-869b4bb90442 req-e71bc321-d23b-4eab-b39a-2b03a9466515 service nova] Creating event network-changed:7579c2c0-f784-4f9e-b818-50eb66f8013f for instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 on nch2-master-1 Sep 27 10:46:27.947731 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-c733e52e-47ef-4517-a6b2-869b4bb90442 req-e71bc321-d23b-4eab-b39a-2b03a9466515 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038962 Sep 27 10:46:27.948427 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3166/6801] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:46:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:46:28.679866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-95a06ed7-420d-4bfb-83eb-8f21fb9820b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:28.680568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-95a06ed7-420d-4bfb-83eb-8f21fb9820b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:28.687902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95a06ed7-420d-4bfb-83eb-8f21fb9820b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:28.688622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95a06ed7-420d-4bfb-83eb-8f21fb9820b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:28.688856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95a06ed7-420d-4bfb-83eb-8f21fb9820b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:28.689573 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95a06ed7-420d-4bfb-83eb-8f21fb9820b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:28.690039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95a06ed7-420d-4bfb-83eb-8f21fb9820b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:28.690479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-95a06ed7-420d-4bfb-83eb-8f21fb9820b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:28.826116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-95a06ed7-420d-4bfb-83eb-8f21fb9820b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:28.834428 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-95a06ed7-420d-4bfb-83eb-8f21fb9820b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1362 microversion: 2.1 time: 0.158633 Sep 27 10:46:28.835023 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3636/6802] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:28 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1362 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:29.865790 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af7bf220-f6e4-4b2d-8dad-db336a846ff9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:29.866577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-af7bf220-f6e4-4b2d-8dad-db336a846ff9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:29.868793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=58,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:29.875746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7bf220-f6e4-4b2d-8dad-db336a846ff9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:29.876462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7bf220-f6e4-4b2d-8dad-db336a846ff9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:29.876993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7bf220-f6e4-4b2d-8dad-db336a846ff9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:29.877888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7bf220-f6e4-4b2d-8dad-db336a846ff9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:29.878417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7bf220-f6e4-4b2d-8dad-db336a846ff9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:29.878898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7bf220-f6e4-4b2d-8dad-db336a846ff9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:30.016795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af7bf220-f6e4-4b2d-8dad-db336a846ff9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:30.025267 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af7bf220-f6e4-4b2d-8dad-db336a846ff9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1362 microversion: 2.1 time: 0.165963 Sep 27 10:46:30.025837 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3167/6803] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:29 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1362 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:31.051483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cffe7d2e-ceed-493b-8094-ea7501c566bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:31.053224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cffe7d2e-ceed-493b-8094-ea7501c566bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:31.057638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=54,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:31.062172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffe7d2e-ceed-493b-8094-ea7501c566bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:31.062963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffe7d2e-ceed-493b-8094-ea7501c566bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:31.063562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffe7d2e-ceed-493b-8094-ea7501c566bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:31.064292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffe7d2e-ceed-493b-8094-ea7501c566bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:31.064979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffe7d2e-ceed-493b-8094-ea7501c566bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:31.065419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cffe7d2e-ceed-493b-8094-ea7501c566bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:31.209333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cffe7d2e-ceed-493b-8094-ea7501c566bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:31.217315 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cffe7d2e-ceed-493b-8094-ea7501c566bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1362 microversion: 2.1 time: 0.171135 Sep 27 10:46:31.218713 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3637/6804] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:31 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1362 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:32.244731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-df40790b-c758-44d9-a81a-4132ca2e8ba8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:32.245869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-df40790b-c758-44d9-a81a-4132ca2e8ba8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:32.255554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df40790b-c758-44d9-a81a-4132ca2e8ba8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:32.256058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df40790b-c758-44d9-a81a-4132ca2e8ba8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:32.256485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df40790b-c758-44d9-a81a-4132ca2e8ba8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:32.257259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df40790b-c758-44d9-a81a-4132ca2e8ba8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:32.257722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df40790b-c758-44d9-a81a-4132ca2e8ba8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:32.258158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df40790b-c758-44d9-a81a-4132ca2e8ba8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:32.388436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-df40790b-c758-44d9-a81a-4132ca2e8ba8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:32.396937 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-df40790b-c758-44d9-a81a-4132ca2e8ba8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1362 microversion: 2.1 time: 0.158933 Sep 27 10:46:32.397550 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3168/6805] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:32 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1362 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:33.082026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c33d230f-6bae-4ca1-b136-d1066131ec98 req-b4f24754-1ead-4e2b-95b5-91444d40b66e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b7833f5-ef39-4414-92b8-0eaaa26eb4a7", "name": "network-vif-plugged", "status": "completed", "tag": "7579c2c0-f784-4f9e-b818-50eb66f8013f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:46:33.093348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c33d230f-6bae-4ca1-b136-d1066131ec98 req-b4f24754-1ead-4e2b-95b5-91444d40b66e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:33.093869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c33d230f-6bae-4ca1-b136-d1066131ec98 req-b4f24754-1ead-4e2b-95b5-91444d40b66e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:33.094311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c33d230f-6bae-4ca1-b136-d1066131ec98 req-b4f24754-1ead-4e2b-95b5-91444d40b66e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:33.106140 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c33d230f-6bae-4ca1-b136-d1066131ec98 req-b4f24754-1ead-4e2b-95b5-91444d40b66e service nova] Creating event network-vif-plugged:7579c2c0-f784-4f9e-b818-50eb66f8013f for instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 on nch2-master-1 Sep 27 10:46:33.111272 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c33d230f-6bae-4ca1-b136-d1066131ec98 req-b4f24754-1ead-4e2b-95b5-91444d40b66e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033113 Sep 27 10:46:33.111876 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3638/6806] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:46:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:46:33.424497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f29a4663-6a47-4e8c-bc69-b70fcae29e5b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:33.425192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f29a4663-6a47-4e8c-bc69-b70fcae29e5b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:33.432933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29a4663-6a47-4e8c-bc69-b70fcae29e5b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:33.433459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29a4663-6a47-4e8c-bc69-b70fcae29e5b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:33.433932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29a4663-6a47-4e8c-bc69-b70fcae29e5b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:33.434684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29a4663-6a47-4e8c-bc69-b70fcae29e5b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:33.435174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29a4663-6a47-4e8c-bc69-b70fcae29e5b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:33.435637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f29a4663-6a47-4e8c-bc69-b70fcae29e5b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:33.575454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f29a4663-6a47-4e8c-bc69-b70fcae29e5b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:33.584379 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f29a4663-6a47-4e8c-bc69-b70fcae29e5b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1362 microversion: 2.1 time: 0.164155 Sep 27 10:46:33.584991 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3169/6807] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:33 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1362 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:34.609726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c82ca631-f6d3-4404-970c-cc8037513801 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:34.610551 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c82ca631-f6d3-4404-970c-cc8037513801 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:34.618647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c82ca631-f6d3-4404-970c-cc8037513801 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:34.619127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c82ca631-f6d3-4404-970c-cc8037513801 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:34.619572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c82ca631-f6d3-4404-970c-cc8037513801 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:34.620290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c82ca631-f6d3-4404-970c-cc8037513801 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:34.620796 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c82ca631-f6d3-4404-970c-cc8037513801 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:34.621235 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c82ca631-f6d3-4404-970c-cc8037513801 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:34.763603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c82ca631-f6d3-4404-970c-cc8037513801 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:34.771643 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c82ca631-f6d3-4404-970c-cc8037513801 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1362 microversion: 2.1 time: 0.166712 Sep 27 10:46:34.772393 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3639/6808] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:34 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1362 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:35.795593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:35.796435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:35.801231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-daf41414-cf4b-4ec3-8f1c-bb5a2160f5e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:35.801861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-daf41414-cf4b-4ec3-8f1c-bb5a2160f5e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:35.808886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daf41414-cf4b-4ec3-8f1c-bb5a2160f5e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:35.809414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daf41414-cf4b-4ec3-8f1c-bb5a2160f5e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:35.809861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daf41414-cf4b-4ec3-8f1c-bb5a2160f5e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:35.810567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daf41414-cf4b-4ec3-8f1c-bb5a2160f5e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:35.811032 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daf41414-cf4b-4ec3-8f1c-bb5a2160f5e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:35.811465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-daf41414-cf4b-4ec3-8f1c-bb5a2160f5e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:35.956218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-daf41414-cf4b-4ec3-8f1c-bb5a2160f5e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:46:35.964809 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-daf41414-cf4b-4ec3-8f1c-bb5a2160f5e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1550 microversion: 2.1 time: 0.171273 Sep 27 10:46:35.965396 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3170/6809] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:46:35 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1550 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:46:35.989026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6179b64a-a668-4b92-afde-395c6f70ed9c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:46:35.989569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6179b64a-a668-4b92-afde-395c6f70ed9c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:46:35.991607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:46:35.999043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6179b64a-a668-4b92-afde-395c6f70ed9c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:35.999168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6179b64a-a668-4b92-afde-395c6f70ed9c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:35.999564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6179b64a-a668-4b92-afde-395c6f70ed9c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:36.000254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6179b64a-a668-4b92-afde-395c6f70ed9c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:36.000732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6179b64a-a668-4b92-afde-395c6f70ed9c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:36.001192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6179b64a-a668-4b92-afde-395c6f70ed9c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:36.082475 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6179b64a-a668-4b92-afde-395c6f70ed9c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.097431 Sep 27 10:46:36.083310 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3640/6810] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:46:35 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7/os-interface => generated 285 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:46:37.067923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ef6df8af-11f5-4f54-910c-b64753af8c2e req-da0ea6b2-7965-4349-b511-7e8e60f7bdf6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4b7833f5-ef39-4414-92b8-0eaaa26eb4a7", "tag": "7579c2c0-f784-4f9e-b818-50eb66f8013f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:46:37.078306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ef6df8af-11f5-4f54-910c-b64753af8c2e req-da0ea6b2-7965-4349-b511-7e8e60f7bdf6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:46:37.078798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ef6df8af-11f5-4f54-910c-b64753af8c2e req-da0ea6b2-7965-4349-b511-7e8e60f7bdf6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:46:37.079238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ef6df8af-11f5-4f54-910c-b64753af8c2e req-da0ea6b2-7965-4349-b511-7e8e60f7bdf6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:46:37.090696 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ef6df8af-11f5-4f54-910c-b64753af8c2e req-da0ea6b2-7965-4349-b511-7e8e60f7bdf6 service nova] Creating event network-changed:7579c2c0-f784-4f9e-b818-50eb66f8013f for instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 on nch2-master-1 Sep 27 10:46:37.096412 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ef6df8af-11f5-4f54-910c-b64753af8c2e req-da0ea6b2-7965-4349-b511-7e8e60f7bdf6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032181 Sep 27 10:46:37.097045 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3171/6811] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:46:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:47:49.100005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:47:49.101276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=52,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:47:49.103633 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 10:47:49.103949 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 10:47:49 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7/action (ip 10.222.0.26) !!! Sep 27 10:47:49.112517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bedb8e84-a801-4fe0-b35c-6adc9be8bd53 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:47:49.120883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bedb8e84-a801-4fe0-b35c-6adc9be8bd53 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:49.121545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bedb8e84-a801-4fe0-b35c-6adc9be8bd53 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:49.122013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-bedb8e84-a801-4fe0-b35c-6adc9be8bd53 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:49.146289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bedb8e84-a801-4fe0-b35c-6adc9be8bd53 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Going to try to stop instance {{(pid=145968) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 10:47:49.198730 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bedb8e84-a801-4fe0-b35c-6adc9be8bd53 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7/action" status: 202 len: 0 microversion: 2.1 time: 0.103080 Sep 27 10:47:49.199714 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3641/6812] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:47:49 2022] POST /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7/action => generated 0 bytes in 106 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:47:49.207170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=9 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:47:49.208003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=36 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:47:49.208895 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 10:47:49.209251 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 10:47:49 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 (ip 10.222.0.26) !!! Sep 27 10:47:49.215395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0ed304f0-451f-4d57-abc7-82040e76ae76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:47:49.216092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0ed304f0-451f-4d57-abc7-82040e76ae76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:47:49.224160 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ed304f0-451f-4d57-abc7-82040e76ae76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:49.224671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ed304f0-451f-4d57-abc7-82040e76ae76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:49.225127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ed304f0-451f-4d57-abc7-82040e76ae76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:49.225850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ed304f0-451f-4d57-abc7-82040e76ae76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:49.226312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ed304f0-451f-4d57-abc7-82040e76ae76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:49.226745 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0ed304f0-451f-4d57-abc7-82040e76ae76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:49.442354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0ed304f0-451f-4d57-abc7-82040e76ae76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:47:49.450922 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0ed304f0-451f-4d57-abc7-82040e76ae76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1678 microversion: 2.1 time: 0.246546 Sep 27 10:47:49.451480 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3172/6813] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:47:49 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1678 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:47:50.480104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-60e7b2b7-2712-447e-8ca7-dd4e742b9be0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:47:50.481165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-60e7b2b7-2712-447e-8ca7-dd4e742b9be0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:47:50.493397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60e7b2b7-2712-447e-8ca7-dd4e742b9be0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:50.494201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60e7b2b7-2712-447e-8ca7-dd4e742b9be0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:50.494926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60e7b2b7-2712-447e-8ca7-dd4e742b9be0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:50.496086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60e7b2b7-2712-447e-8ca7-dd4e742b9be0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:50.496891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60e7b2b7-2712-447e-8ca7-dd4e742b9be0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:50.497603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-60e7b2b7-2712-447e-8ca7-dd4e742b9be0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:50.738287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-60e7b2b7-2712-447e-8ca7-dd4e742b9be0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:47:50.746862 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-60e7b2b7-2712-447e-8ca7-dd4e742b9be0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1678 microversion: 2.1 time: 0.274408 Sep 27 10:47:50.747454 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3642/6814] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:47:50 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1678 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:47:51.775331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2d55c9bc-54c3-4d4f-b9e7-8af55e2df60e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:47:51.776390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2d55c9bc-54c3-4d4f-b9e7-8af55e2df60e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:47:51.785880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d55c9bc-54c3-4d4f-b9e7-8af55e2df60e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:51.786388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d55c9bc-54c3-4d4f-b9e7-8af55e2df60e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:51.786850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d55c9bc-54c3-4d4f-b9e7-8af55e2df60e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:51.787560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d55c9bc-54c3-4d4f-b9e7-8af55e2df60e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:51.788034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d55c9bc-54c3-4d4f-b9e7-8af55e2df60e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:51.788479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d55c9bc-54c3-4d4f-b9e7-8af55e2df60e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:51.921988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2d55c9bc-54c3-4d4f-b9e7-8af55e2df60e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:47:51.931028 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2d55c9bc-54c3-4d4f-b9e7-8af55e2df60e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1678 microversion: 2.1 time: 0.162415 Sep 27 10:47:51.931614 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3173/6815] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:47:51 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1678 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:47:52.955975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6db73869-4f65-44b6-86eb-d28da6f2ddd1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:47:52.956585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6db73869-4f65-44b6-86eb-d28da6f2ddd1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:47:52.964073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db73869-4f65-44b6-86eb-d28da6f2ddd1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:52.964572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db73869-4f65-44b6-86eb-d28da6f2ddd1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:52.965058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db73869-4f65-44b6-86eb-d28da6f2ddd1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:52.965780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db73869-4f65-44b6-86eb-d28da6f2ddd1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:52.966246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db73869-4f65-44b6-86eb-d28da6f2ddd1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:52.966683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6db73869-4f65-44b6-86eb-d28da6f2ddd1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:53.108567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6db73869-4f65-44b6-86eb-d28da6f2ddd1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:47:53.117564 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6db73869-4f65-44b6-86eb-d28da6f2ddd1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1678 microversion: 2.1 time: 0.165905 Sep 27 10:47:53.118149 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3643/6816] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:47:52 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1678 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:47:54.153971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-49caa9c6-c6b2-43fd-8125-f06ff1107d32 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:47:54.154928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-49caa9c6-c6b2-43fd-8125-f06ff1107d32 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:47:54.162640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49caa9c6-c6b2-43fd-8125-f06ff1107d32 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:54.163114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49caa9c6-c6b2-43fd-8125-f06ff1107d32 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:54.163556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49caa9c6-c6b2-43fd-8125-f06ff1107d32 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:54.164266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49caa9c6-c6b2-43fd-8125-f06ff1107d32 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:54.164771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49caa9c6-c6b2-43fd-8125-f06ff1107d32 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:54.165214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49caa9c6-c6b2-43fd-8125-f06ff1107d32 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:54.312491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-49caa9c6-c6b2-43fd-8125-f06ff1107d32 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:47:54.324800 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-49caa9c6-c6b2-43fd-8125-f06ff1107d32 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1678 microversion: 2.1 time: 0.177553 Sep 27 10:47:54.325753 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3174/6817] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:47:54 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1678 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:47:55.352867 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a333e803-c717-4d09-a610-81c54944bd01 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:47:55.353759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a333e803-c717-4d09-a610-81c54944bd01 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:47:55.363772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a333e803-c717-4d09-a610-81c54944bd01 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:55.364261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a333e803-c717-4d09-a610-81c54944bd01 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:55.364736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a333e803-c717-4d09-a610-81c54944bd01 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:55.365459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a333e803-c717-4d09-a610-81c54944bd01 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:55.365918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a333e803-c717-4d09-a610-81c54944bd01 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:55.366353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a333e803-c717-4d09-a610-81c54944bd01 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:55.513956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a333e803-c717-4d09-a610-81c54944bd01 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:47:55.522212 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a333e803-c717-4d09-a610-81c54944bd01 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1678 microversion: 2.1 time: 0.176247 Sep 27 10:47:55.522789 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3644/6818] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:47:55 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1678 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:47:56.550238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5aab357f-8f4b-49c6-a185-320578d9f16d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:47:56.550864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5aab357f-8f4b-49c6-a185-320578d9f16d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:47:56.558396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5aab357f-8f4b-49c6-a185-320578d9f16d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:56.558901 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5aab357f-8f4b-49c6-a185-320578d9f16d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:56.559356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5aab357f-8f4b-49c6-a185-320578d9f16d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:56.560111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5aab357f-8f4b-49c6-a185-320578d9f16d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:56.560746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5aab357f-8f4b-49c6-a185-320578d9f16d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:56.561197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5aab357f-8f4b-49c6-a185-320578d9f16d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:56.693933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5aab357f-8f4b-49c6-a185-320578d9f16d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:47:56.706025 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5aab357f-8f4b-49c6-a185-320578d9f16d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1678 microversion: 2.1 time: 0.162173 Sep 27 10:47:56.706594 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3175/6819] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:47:56 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1678 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:47:57.729863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-199cedcf-1585-4745-bc48-b5f673007d76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:47:57.730484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-199cedcf-1585-4745-bc48-b5f673007d76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:47:57.737879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-199cedcf-1585-4745-bc48-b5f673007d76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:57.738148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-199cedcf-1585-4745-bc48-b5f673007d76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:57.738591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-199cedcf-1585-4745-bc48-b5f673007d76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:57.739303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-199cedcf-1585-4745-bc48-b5f673007d76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:57.739768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-199cedcf-1585-4745-bc48-b5f673007d76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:57.740199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-199cedcf-1585-4745-bc48-b5f673007d76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:57.898066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-199cedcf-1585-4745-bc48-b5f673007d76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:47:57.906339 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-199cedcf-1585-4745-bc48-b5f673007d76 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1678 microversion: 2.1 time: 0.180454 Sep 27 10:47:57.906925 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3645/6820] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:47:57 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1678 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:47:58.933400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-30de8319-fcff-4a31-90cf-419676bcaade tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:47:58.934147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-30de8319-fcff-4a31-90cf-419676bcaade tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:47:58.944946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30de8319-fcff-4a31-90cf-419676bcaade tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:58.945659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30de8319-fcff-4a31-90cf-419676bcaade tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:58.946306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30de8319-fcff-4a31-90cf-419676bcaade tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:58.947333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30de8319-fcff-4a31-90cf-419676bcaade tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:47:58.948010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30de8319-fcff-4a31-90cf-419676bcaade tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:47:58.948644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-30de8319-fcff-4a31-90cf-419676bcaade tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:47:59.087529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-30de8319-fcff-4a31-90cf-419676bcaade tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:47:59.095862 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-30de8319-fcff-4a31-90cf-419676bcaade tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1678 microversion: 2.1 time: 0.168255 Sep 27 10:47:59.096431 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3176/6821] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:47:58 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1678 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:00.126166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b9434866-ee8b-4f11-89c9-5bd5a1c9cf67 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:00.126766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b9434866-ee8b-4f11-89c9-5bd5a1c9cf67 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:00.134642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9434866-ee8b-4f11-89c9-5bd5a1c9cf67 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:00.135159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9434866-ee8b-4f11-89c9-5bd5a1c9cf67 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:00.135582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9434866-ee8b-4f11-89c9-5bd5a1c9cf67 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:00.136301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9434866-ee8b-4f11-89c9-5bd5a1c9cf67 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:00.136798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9434866-ee8b-4f11-89c9-5bd5a1c9cf67 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:00.137237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b9434866-ee8b-4f11-89c9-5bd5a1c9cf67 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:00.301506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b9434866-ee8b-4f11-89c9-5bd5a1c9cf67 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:00.311482 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b9434866-ee8b-4f11-89c9-5bd5a1c9cf67 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1655 microversion: 2.1 time: 0.190349 Sep 27 10:48:00.311986 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3646/6822] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:00 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1655 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:00.339698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:48:00.343151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:00.354625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:00.355377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:00.356079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:00.357265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:00.358000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:00.358749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:00.391932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:00.399241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Old instance type m1.heat2, new instance type m1.heat {{(pid=145969) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:48:00.406283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:48:00.406722 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:48:00.407225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:48:00.407647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:48:00.408079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:48:00.477700 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c8f4004b-22e3-47ee-8a67-11033a7cdcde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7/action" status: 202 len: 0 microversion: 2.1 time: 0.145133 Sep 27 10:48:00.478327 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3177/6823] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:48:00 2022] POST /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7/action => generated 0 bytes in 147 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:48:00.487515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-034314a3-301e-4904-afee-ef6fc3399412 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:00.488317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-034314a3-301e-4904-afee-ef6fc3399412 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:00.494666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-034314a3-301e-4904-afee-ef6fc3399412 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:00.495168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-034314a3-301e-4904-afee-ef6fc3399412 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:00.495618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-034314a3-301e-4904-afee-ef6fc3399412 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:00.496342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-034314a3-301e-4904-afee-ef6fc3399412 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:00.496828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-034314a3-301e-4904-afee-ef6fc3399412 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:00.497283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-034314a3-301e-4904-afee-ef6fc3399412 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:00.675119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-034314a3-301e-4904-afee-ef6fc3399412 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:00.683576 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-034314a3-301e-4904-afee-ef6fc3399412 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1678 microversion: 2.1 time: 0.200623 Sep 27 10:48:00.684204 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3647/6824] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:00 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1678 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:01.713295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25818806-170e-4466-a82c-73c47247c9ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:01.714151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-25818806-170e-4466-a82c-73c47247c9ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:01.724633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25818806-170e-4466-a82c-73c47247c9ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:01.725173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25818806-170e-4466-a82c-73c47247c9ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:01.725603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25818806-170e-4466-a82c-73c47247c9ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:01.726349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25818806-170e-4466-a82c-73c47247c9ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:01.726825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25818806-170e-4466-a82c-73c47247c9ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:01.727271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25818806-170e-4466-a82c-73c47247c9ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:01.864425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-25818806-170e-4466-a82c-73c47247c9ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:01.872753 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-25818806-170e-4466-a82c-73c47247c9ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1678 microversion: 2.1 time: 0.167650 Sep 27 10:48:01.873347 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3178/6825] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:01 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1678 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:02.897371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b7072d5c-0249-492b-b9e9-a8c6ef4b67aa tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:02.897959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b7072d5c-0249-492b-b9e9-a8c6ef4b67aa tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:02.905028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7072d5c-0249-492b-b9e9-a8c6ef4b67aa tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:02.905523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7072d5c-0249-492b-b9e9-a8c6ef4b67aa tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:02.905992 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7072d5c-0249-492b-b9e9-a8c6ef4b67aa tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:02.906699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7072d5c-0249-492b-b9e9-a8c6ef4b67aa tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:02.907155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7072d5c-0249-492b-b9e9-a8c6ef4b67aa tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:02.907584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7072d5c-0249-492b-b9e9-a8c6ef4b67aa tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:03.046347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b7072d5c-0249-492b-b9e9-a8c6ef4b67aa tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:03.055514 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b7072d5c-0249-492b-b9e9-a8c6ef4b67aa tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1683 microversion: 2.1 time: 0.162364 Sep 27 10:48:03.056092 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3648/6826] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:02 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1683 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:04.080408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b932faab-ea4e-44b5-91c6-1f7d5359608a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:04.081051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b932faab-ea4e-44b5-91c6-1f7d5359608a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:04.088491 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b932faab-ea4e-44b5-91c6-1f7d5359608a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:04.089027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b932faab-ea4e-44b5-91c6-1f7d5359608a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:04.089474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b932faab-ea4e-44b5-91c6-1f7d5359608a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:04.090187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b932faab-ea4e-44b5-91c6-1f7d5359608a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:04.090651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b932faab-ea4e-44b5-91c6-1f7d5359608a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:04.091088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b932faab-ea4e-44b5-91c6-1f7d5359608a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:04.232510 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b932faab-ea4e-44b5-91c6-1f7d5359608a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:04.243019 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b932faab-ea4e-44b5-91c6-1f7d5359608a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1683 microversion: 2.1 time: 0.166564 Sep 27 10:48:04.243618 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3179/6827] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:04 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1683 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:05.268144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b89f10fc-324c-4d4b-a056-b1f862af217d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:05.268839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b89f10fc-324c-4d4b-a056-b1f862af217d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:05.277455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b89f10fc-324c-4d4b-a056-b1f862af217d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:05.277455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b89f10fc-324c-4d4b-a056-b1f862af217d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:05.278033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b89f10fc-324c-4d4b-a056-b1f862af217d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:05.278416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b89f10fc-324c-4d4b-a056-b1f862af217d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:05.278894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b89f10fc-324c-4d4b-a056-b1f862af217d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:05.279330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b89f10fc-324c-4d4b-a056-b1f862af217d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:05.428141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b89f10fc-324c-4d4b-a056-b1f862af217d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:05.439101 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b89f10fc-324c-4d4b-a056-b1f862af217d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1682 microversion: 2.1 time: 0.175328 Sep 27 10:48:05.439792 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3649/6828] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:05 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1682 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:05.542560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e60c6731-fa69-4e12-ae15-aa6cb5a09e6b req-7b17c972-cf8e-48eb-a954-6fa3f28d1d4f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b7833f5-ef39-4414-92b8-0eaaa26eb4a7", "name": "network-vif-unplugged", "status": "completed", "tag": "7579c2c0-f784-4f9e-b818-50eb66f8013f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:48:05.553591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e60c6731-fa69-4e12-ae15-aa6cb5a09e6b req-7b17c972-cf8e-48eb-a954-6fa3f28d1d4f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:05.554194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e60c6731-fa69-4e12-ae15-aa6cb5a09e6b req-7b17c972-cf8e-48eb-a954-6fa3f28d1d4f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:05.554637 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e60c6731-fa69-4e12-ae15-aa6cb5a09e6b req-7b17c972-cf8e-48eb-a954-6fa3f28d1d4f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:05.567653 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e60c6731-fa69-4e12-ae15-aa6cb5a09e6b req-7b17c972-cf8e-48eb-a954-6fa3f28d1d4f service nova] Creating event network-vif-unplugged:7579c2c0-f784-4f9e-b818-50eb66f8013f for instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 on nch1-master-1 Sep 27 10:48:05.573660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-e60c6731-fa69-4e12-ae15-aa6cb5a09e6b req-7b17c972-cf8e-48eb-a954-6fa3f28d1d4f service nova] Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:48:05.586201 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e60c6731-fa69-4e12-ae15-aa6cb5a09e6b req-7b17c972-cf8e-48eb-a954-6fa3f28d1d4f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.048123 Sep 27 10:48:05.586998 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3180/6829] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:48:05 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:48:06.463923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9858a871-50a2-4870-91cf-d0e8f4098bc4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:06.464786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9858a871-50a2-4870-91cf-d0e8f4098bc4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:06.472164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9858a871-50a2-4870-91cf-d0e8f4098bc4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:06.472747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9858a871-50a2-4870-91cf-d0e8f4098bc4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:06.473186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9858a871-50a2-4870-91cf-d0e8f4098bc4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:06.473902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9858a871-50a2-4870-91cf-d0e8f4098bc4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:06.474368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9858a871-50a2-4870-91cf-d0e8f4098bc4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:06.474802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9858a871-50a2-4870-91cf-d0e8f4098bc4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:06.614155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9858a871-50a2-4870-91cf-d0e8f4098bc4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:06.623108 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9858a871-50a2-4870-91cf-d0e8f4098bc4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1682 microversion: 2.1 time: 0.163243 Sep 27 10:48:06.624055 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3650/6830] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:06 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1682 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:07.634490 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-3d672606-52e0-4bd8-8214-066439fd683d req-8dc12e27-1cf5-49b3-a3c2-26ffd0343c3e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4b7833f5-ef39-4414-92b8-0eaaa26eb4a7", "tag": "7579c2c0-f784-4f9e-b818-50eb66f8013f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:48:07.644650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3d672606-52e0-4bd8-8214-066439fd683d req-8dc12e27-1cf5-49b3-a3c2-26ffd0343c3e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:07.645263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3d672606-52e0-4bd8-8214-066439fd683d req-8dc12e27-1cf5-49b3-a3c2-26ffd0343c3e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:07.646033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3d672606-52e0-4bd8-8214-066439fd683d req-8dc12e27-1cf5-49b3-a3c2-26ffd0343c3e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:07.650612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25b29f86-dfd7-4825-9b8a-e4e693d6b16a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:07.651276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-25b29f86-dfd7-4825-9b8a-e4e693d6b16a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:07.657865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b29f86-dfd7-4825-9b8a-e4e693d6b16a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:07.658384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b29f86-dfd7-4825-9b8a-e4e693d6b16a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:07.658830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b29f86-dfd7-4825-9b8a-e4e693d6b16a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:07.659555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b29f86-dfd7-4825-9b8a-e4e693d6b16a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:07.659957 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-3d672606-52e0-4bd8-8214-066439fd683d req-8dc12e27-1cf5-49b3-a3c2-26ffd0343c3e service nova] Creating event network-changed:7579c2c0-f784-4f9e-b818-50eb66f8013f for instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 on nch1-master-1 Sep 27 10:48:07.660128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b29f86-dfd7-4825-9b8a-e4e693d6b16a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:07.660453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25b29f86-dfd7-4825-9b8a-e4e693d6b16a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:07.665647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-3d672606-52e0-4bd8-8214-066439fd683d req-8dc12e27-1cf5-49b3-a3c2-26ffd0343c3e service nova] Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:48:07.676678 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-3d672606-52e0-4bd8-8214-066439fd683d req-8dc12e27-1cf5-49b3-a3c2-26ffd0343c3e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049662 Sep 27 10:48:07.677293 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3181/6831] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:48:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:48:07.820408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-25b29f86-dfd7-4825-9b8a-e4e693d6b16a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:07.830547 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-25b29f86-dfd7-4825-9b8a-e4e693d6b16a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1680 microversion: 2.1 time: 0.184948 Sep 27 10:48:07.831295 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3651/6832] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:07 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1680 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:08.861569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-17a087c1-eb12-4fbf-9b07-2585676c0493 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:08.862594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-17a087c1-eb12-4fbf-9b07-2585676c0493 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:08.870477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17a087c1-eb12-4fbf-9b07-2585676c0493 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:08.870988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17a087c1-eb12-4fbf-9b07-2585676c0493 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:08.871434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17a087c1-eb12-4fbf-9b07-2585676c0493 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:08.872142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17a087c1-eb12-4fbf-9b07-2585676c0493 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:08.872608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17a087c1-eb12-4fbf-9b07-2585676c0493 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:08.873078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-17a087c1-eb12-4fbf-9b07-2585676c0493 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:09.015180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-17a087c1-eb12-4fbf-9b07-2585676c0493 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:09.023214 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-17a087c1-eb12-4fbf-9b07-2585676c0493 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1676 microversion: 2.1 time: 0.170392 Sep 27 10:48:09.023806 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3182/6833] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:08 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1676 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:09.050407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d5f750b-75a8-4006-b805-941ed963861f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:48:09.050949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9d5f750b-75a8-4006-b805-941ed963861f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:09.058868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5f750b-75a8-4006-b805-941ed963861f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:09.059368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5f750b-75a8-4006-b805-941ed963861f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:09.059817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5f750b-75a8-4006-b805-941ed963861f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:09.060533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5f750b-75a8-4006-b805-941ed963861f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:09.061032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5f750b-75a8-4006-b805-941ed963861f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:09.061469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5f750b-75a8-4006-b805-941ed963861f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:09.132994 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d5f750b-75a8-4006-b805-941ed963861f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7/action" status: 204 len: 0 microversion: 2.1 time: 0.088408 Sep 27 10:48:09.133671 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3652/6834] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:48:09 2022] POST /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7/action => generated 0 bytes in 91 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:48:09.144197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e5ca66ba-7af9-4ef6-a7fd-ea64731ecdb0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:09.145311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e5ca66ba-7af9-4ef6-a7fd-ea64731ecdb0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:09.152418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca66ba-7af9-4ef6-a7fd-ea64731ecdb0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:09.153199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca66ba-7af9-4ef6-a7fd-ea64731ecdb0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:09.153666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca66ba-7af9-4ef6-a7fd-ea64731ecdb0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:09.154392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca66ba-7af9-4ef6-a7fd-ea64731ecdb0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:09.154899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca66ba-7af9-4ef6-a7fd-ea64731ecdb0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:09.155444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e5ca66ba-7af9-4ef6-a7fd-ea64731ecdb0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:09.377471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e5ca66ba-7af9-4ef6-a7fd-ea64731ecdb0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:09.386252 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e5ca66ba-7af9-4ef6-a7fd-ea64731ecdb0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1676 microversion: 2.1 time: 0.246706 Sep 27 10:48:09.386880 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3183/6835] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:09 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1676 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:10.413312 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-71ccc12e-3771-4c10-bbe4-044931345605 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:10.414339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-71ccc12e-3771-4c10-bbe4-044931345605 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:10.422132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71ccc12e-3771-4c10-bbe4-044931345605 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:10.422641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71ccc12e-3771-4c10-bbe4-044931345605 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:10.423076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71ccc12e-3771-4c10-bbe4-044931345605 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:10.423790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71ccc12e-3771-4c10-bbe4-044931345605 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:10.424256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71ccc12e-3771-4c10-bbe4-044931345605 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:10.424715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-71ccc12e-3771-4c10-bbe4-044931345605 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:10.568025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-71ccc12e-3771-4c10-bbe4-044931345605 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:10.576218 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-71ccc12e-3771-4c10-bbe4-044931345605 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1676 microversion: 2.1 time: 0.169449 Sep 27 10:48:10.576826 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3653/6836] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:10 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1676 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:11.601239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2d3fdc4a-41ab-4fd3-9a92-9de36bf0e5e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:11.602601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2d3fdc4a-41ab-4fd3-9a92-9de36bf0e5e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:11.611332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3fdc4a-41ab-4fd3-9a92-9de36bf0e5e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:11.611332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3fdc4a-41ab-4fd3-9a92-9de36bf0e5e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:11.611689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3fdc4a-41ab-4fd3-9a92-9de36bf0e5e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:11.612433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3fdc4a-41ab-4fd3-9a92-9de36bf0e5e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:11.612960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3fdc4a-41ab-4fd3-9a92-9de36bf0e5e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:11.613395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3fdc4a-41ab-4fd3-9a92-9de36bf0e5e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:11.680284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-70a6dbb6-7e77-4f09-be11-8b3c8893bef6 req-34735507-4f5c-4e61-9902-52c74d285950 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b7833f5-ef39-4414-92b8-0eaaa26eb4a7", "name": "network-vif-plugged", "status": "completed", "tag": "7579c2c0-f784-4f9e-b818-50eb66f8013f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:48:11.692024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-70a6dbb6-7e77-4f09-be11-8b3c8893bef6 req-34735507-4f5c-4e61-9902-52c74d285950 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:11.692620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-70a6dbb6-7e77-4f09-be11-8b3c8893bef6 req-34735507-4f5c-4e61-9902-52c74d285950 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:11.693288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-70a6dbb6-7e77-4f09-be11-8b3c8893bef6 req-34735507-4f5c-4e61-9902-52c74d285950 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:11.708324 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-70a6dbb6-7e77-4f09-be11-8b3c8893bef6 req-34735507-4f5c-4e61-9902-52c74d285950 service nova] Creating event network-vif-plugged:7579c2c0-f784-4f9e-b818-50eb66f8013f for instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 on nch1-master-1 Sep 27 10:48:11.713795 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-70a6dbb6-7e77-4f09-be11-8b3c8893bef6 req-34735507-4f5c-4e61-9902-52c74d285950 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037879 Sep 27 10:48:11.714632 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3654/6837] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:48:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:48:11.754593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2d3fdc4a-41ab-4fd3-9a92-9de36bf0e5e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:11.762808 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2d3fdc4a-41ab-4fd3-9a92-9de36bf0e5e0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1655 microversion: 2.1 time: 0.165730 Sep 27 10:48:11.763392 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3184/6838] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:11 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1655 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:11.788986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b7b75f31-c47b-48e3-b9e3-27db77aaa5b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:11.789852 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b7b75f31-c47b-48e3-b9e3-27db77aaa5b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:11.797011 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7b75f31-c47b-48e3-b9e3-27db77aaa5b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:11.797631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7b75f31-c47b-48e3-b9e3-27db77aaa5b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:11.798146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7b75f31-c47b-48e3-b9e3-27db77aaa5b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:11.798964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7b75f31-c47b-48e3-b9e3-27db77aaa5b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:11.799541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7b75f31-c47b-48e3-b9e3-27db77aaa5b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:11.800040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7b75f31-c47b-48e3-b9e3-27db77aaa5b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:11.946652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b7b75f31-c47b-48e3-b9e3-27db77aaa5b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:11.954784 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b7b75f31-c47b-48e3-b9e3-27db77aaa5b1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1655 microversion: 2.1 time: 0.171117 Sep 27 10:48:11.956689 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3655/6839] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:11 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1655 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:11.978053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4dfa623d-8843-456c-970e-0fd92892697b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:48:11.986184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dfa623d-8843-456c-970e-0fd92892697b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:11.986684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dfa623d-8843-456c-970e-0fd92892697b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:11.987136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4dfa623d-8843-456c-970e-0fd92892697b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:12.009621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4dfa623d-8843-456c-970e-0fd92892697b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Going to try to start instance {{(pid=145969) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 27 10:48:12.056301 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4dfa623d-8843-456c-970e-0fd92892697b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7/action" status: 202 len: 0 microversion: 2.1 time: 0.082335 Sep 27 10:48:12.056953 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3185/6840] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:48:11 2022] POST /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7/action => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:48:12.069142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-172f7b8f-00a6-4ae6-b0d2-e008f82b3c86 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:12.070431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-172f7b8f-00a6-4ae6-b0d2-e008f82b3c86 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:12.077494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172f7b8f-00a6-4ae6-b0d2-e008f82b3c86 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:12.078067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172f7b8f-00a6-4ae6-b0d2-e008f82b3c86 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:12.078511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172f7b8f-00a6-4ae6-b0d2-e008f82b3c86 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:12.079231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172f7b8f-00a6-4ae6-b0d2-e008f82b3c86 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:12.079705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172f7b8f-00a6-4ae6-b0d2-e008f82b3c86 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:12.080149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172f7b8f-00a6-4ae6-b0d2-e008f82b3c86 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:12.083115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:48:12.217751 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-172f7b8f-00a6-4ae6-b0d2-e008f82b3c86 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:12.226121 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-172f7b8f-00a6-4ae6-b0d2-e008f82b3c86 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1664 microversion: 2.1 time: 0.161963 Sep 27 10:48:12.226728 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3656/6841] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:12 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1664 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:13.250700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-650b7bfe-ac48-4aa5-a560-be99e5fb4eea tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:13.251507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-650b7bfe-ac48-4aa5-a560-be99e5fb4eea tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:13.261236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-650b7bfe-ac48-4aa5-a560-be99e5fb4eea tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:13.262004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-650b7bfe-ac48-4aa5-a560-be99e5fb4eea tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:13.262655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-650b7bfe-ac48-4aa5-a560-be99e5fb4eea tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:13.263385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-650b7bfe-ac48-4aa5-a560-be99e5fb4eea tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:13.263852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-650b7bfe-ac48-4aa5-a560-be99e5fb4eea tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:13.264278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-650b7bfe-ac48-4aa5-a560-be99e5fb4eea tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:13.417882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-650b7bfe-ac48-4aa5-a560-be99e5fb4eea tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:13.422001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:48:13.428134 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-650b7bfe-ac48-4aa5-a560-be99e5fb4eea tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1668 microversion: 2.1 time: 0.182662 Sep 27 10:48:13.428917 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3186/6842] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:13 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1668 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:13.453182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9ba5fc32-8bf0-443b-9367-6bb6a31c423f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:13.453757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ba5fc32-8bf0-443b-9367-6bb6a31c423f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:13.461155 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ba5fc32-8bf0-443b-9367-6bb6a31c423f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:13.461648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ba5fc32-8bf0-443b-9367-6bb6a31c423f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:13.462084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ba5fc32-8bf0-443b-9367-6bb6a31c423f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:13.462768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ba5fc32-8bf0-443b-9367-6bb6a31c423f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:13.463226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ba5fc32-8bf0-443b-9367-6bb6a31c423f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:13.463660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9ba5fc32-8bf0-443b-9367-6bb6a31c423f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:13.489086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9ba5fc32-8bf0-443b-9367-6bb6a31c423f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:48:13.569949 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9ba5fc32-8bf0-443b-9367-6bb6a31c423f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 204 len: 0 microversion: 2.1 time: 0.120550 Sep 27 10:48:13.570536 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3657/6843] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:48:13 2022] DELETE /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:48:13.579341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-27db2797-e5ca-4514-b9ce-5dc124b06d39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:13.579964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-27db2797-e5ca-4514-b9ce-5dc124b06d39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:13.586672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27db2797-e5ca-4514-b9ce-5dc124b06d39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:13.587255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27db2797-e5ca-4514-b9ce-5dc124b06d39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:13.587800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27db2797-e5ca-4514-b9ce-5dc124b06d39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:13.588795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27db2797-e5ca-4514-b9ce-5dc124b06d39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:13.589342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27db2797-e5ca-4514-b9ce-5dc124b06d39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:13.589850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-27db2797-e5ca-4514-b9ce-5dc124b06d39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:13.737550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-27db2797-e5ca-4514-b9ce-5dc124b06d39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:13.747156 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-27db2797-e5ca-4514-b9ce-5dc124b06d39 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1674 microversion: 2.1 time: 0.172004 Sep 27 10:48:13.747736 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3187/6844] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:13 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1674 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:14.774458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-81efd9cc-b065-4fd0-b51e-7ee2492f6448 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:14.775054 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-81efd9cc-b065-4fd0-b51e-7ee2492f6448 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:14.782330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81efd9cc-b065-4fd0-b51e-7ee2492f6448 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:14.782817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81efd9cc-b065-4fd0-b51e-7ee2492f6448 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:14.783248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81efd9cc-b065-4fd0-b51e-7ee2492f6448 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:14.783952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81efd9cc-b065-4fd0-b51e-7ee2492f6448 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:14.784409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81efd9cc-b065-4fd0-b51e-7ee2492f6448 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:14.784873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-81efd9cc-b065-4fd0-b51e-7ee2492f6448 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:14.923316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-81efd9cc-b065-4fd0-b51e-7ee2492f6448 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:14.932186 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-81efd9cc-b065-4fd0-b51e-7ee2492f6448 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1674 microversion: 2.1 time: 0.163185 Sep 27 10:48:14.932814 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3658/6845] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:14 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1674 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:15.956648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0b68f8d1-f49b-40be-af6b-fdad3dd6d3ee tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:15.957986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0b68f8d1-f49b-40be-af6b-fdad3dd6d3ee tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:15.965693 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b68f8d1-f49b-40be-af6b-fdad3dd6d3ee tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:15.966196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b68f8d1-f49b-40be-af6b-fdad3dd6d3ee tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:15.966641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b68f8d1-f49b-40be-af6b-fdad3dd6d3ee tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:15.967354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b68f8d1-f49b-40be-af6b-fdad3dd6d3ee tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:15.967820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b68f8d1-f49b-40be-af6b-fdad3dd6d3ee tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:15.968256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0b68f8d1-f49b-40be-af6b-fdad3dd6d3ee tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:16.052700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0b68f8d1-f49b-40be-af6b-fdad3dd6d3ee tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:16.063646 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0b68f8d1-f49b-40be-af6b-fdad3dd6d3ee tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 200 len: 1607 microversion: 2.1 time: 0.111003 Sep 27 10:48:16.063874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f539dee1-42ce-429b-9141-8ea81166535a req-77d0f83b-6879-4887-9730-8ff61fac552d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4b7833f5-ef39-4414-92b8-0eaaa26eb4a7", "name": "network-vif-deleted", "tag": "7579c2c0-f784-4f9e-b818-50eb66f8013f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:48:16.065649 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3188/6846] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:15 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 1607 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:16.074981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f539dee1-42ce-429b-9141-8ea81166535a req-77d0f83b-6879-4887-9730-8ff61fac552d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:16.075708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f539dee1-42ce-429b-9141-8ea81166535a req-77d0f83b-6879-4887-9730-8ff61fac552d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:16.076398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f539dee1-42ce-429b-9141-8ea81166535a req-77d0f83b-6879-4887-9730-8ff61fac552d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:16.088835 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f539dee1-42ce-429b-9141-8ea81166535a req-77d0f83b-6879-4887-9730-8ff61fac552d service nova] Creating event network-vif-deleted:7579c2c0-f784-4f9e-b818-50eb66f8013f for instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 on nch1-master-1 Sep 27 10:48:16.095516 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f539dee1-42ce-429b-9141-8ea81166535a req-77d0f83b-6879-4887-9730-8ff61fac552d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035160 Sep 27 10:48:16.097009 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3659/6847] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:48:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:48:17.088583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:17.089362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:17.096839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:17.097300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:17.097965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:17.098684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:17.099184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:17.099621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:17.113977 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 HTTP/1.1 Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '4b7833f5-ef39-4414-92b8-0eaaa26eb4a7'} Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:48:17.115574 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:48:17.120700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:48:17.120700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:48:17.120700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:48:17.120700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:48:17.120700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:48:17.120700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:48:17.120700 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:48:17.120700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:48:17.120700 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fd9f3212-e69d-410b-9206-5644d4c19673 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 404 len: 111 microversion: 2.1 time: 0.032803 Sep 27 10:48:17.120700 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3189/6848] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:17 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:48:17.134181 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:17.134924 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:17.141454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:17.141941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:17.142383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:17.143126 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:17.143549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:17.143978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:17.158069 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 HTTP/1.1 Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '4b7833f5-ef39-4414-92b8-0eaaa26eb4a7'} Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:48:17.160120 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:48:17.164192 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:48:17.164192 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:48:17.164192 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:48:17.164192 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:48:17.164192 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:48:17.164192 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:48:17.164192 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:48:17.164192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:48:17.164192 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ae8498ac-a6e3-4366-99fb-f8362381c63c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 404 len: 111 microversion: 2.1 time: 0.032769 Sep 27 10:48:17.164192 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3660/6849] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:17 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:48:17.217847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-2005390488", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "8ae6d002-3910-401a-89df-d8a9ff70cdbf"}], "security_groups": [{"name": "tempest-securitygroup--404672176"}], "key_name": "tempest-keypair-107890057", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:48:17.328063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] validate_networks() for [('8ae6d002-3910-401a-89df-d8a9ff70cdbf', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:48:17.735096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:48:17.735711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:48:17.736570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:48:17.737279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:48:17.737946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:48:17.748346 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:17.748887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:17.749323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:17.750065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:17.750529 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:17.750962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:17.768347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:48:17.773997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for user 396fe9ce0f7542c382b01d1f5779c13f and project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:48:17.787420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:48:17.790110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:48:17.856348 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-96638097-ce19-4ecc-bb0c-bc94bcc91030 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.642479 Sep 27 10:48:17.857421 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3190/6850] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:48:17 2022] POST /compute/v2.1/servers => generated 405 bytes in 644 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:48:17.868987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3d30e9a6-8e8f-4fa4-a6b5-ac581487eacd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:17.869603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3d30e9a6-8e8f-4fa4-a6b5-ac581487eacd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:17.926991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3d30e9a6-8e8f-4fa4-a6b5-ac581487eacd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:17.934339 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3d30e9a6-8e8f-4fa4-a6b5-ac581487eacd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1238 microversion: 2.1 time: 0.069592 Sep 27 10:48:17.934920 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3661/6851] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:17 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1238 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:18.959918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-311eac62-f0e8-44fd-92a1-0f83f82a0ffb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:18.960463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-311eac62-f0e8-44fd-92a1-0f83f82a0ffb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:18.971558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-311eac62-f0e8-44fd-92a1-0f83f82a0ffb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:18.972517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-311eac62-f0e8-44fd-92a1-0f83f82a0ffb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:18.973351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-311eac62-f0e8-44fd-92a1-0f83f82a0ffb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:18.974077 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-311eac62-f0e8-44fd-92a1-0f83f82a0ffb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:18.974584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-311eac62-f0e8-44fd-92a1-0f83f82a0ffb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:18.975137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-311eac62-f0e8-44fd-92a1-0f83f82a0ffb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:19.067707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-311eac62-f0e8-44fd-92a1-0f83f82a0ffb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:19.077164 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-311eac62-f0e8-44fd-92a1-0f83f82a0ffb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1298 microversion: 2.1 time: 0.122241 Sep 27 10:48:19.086416 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3191/6852] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:18 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1298 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:20.102017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-51f7c10a-2da5-4ff8-aa97-a4f6c2026aa9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:20.102833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-51f7c10a-2da5-4ff8-aa97-a4f6c2026aa9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:20.108077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=50,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:48:20.112268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51f7c10a-2da5-4ff8-aa97-a4f6c2026aa9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:20.112931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51f7c10a-2da5-4ff8-aa97-a4f6c2026aa9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:20.113697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51f7c10a-2da5-4ff8-aa97-a4f6c2026aa9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:20.114788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51f7c10a-2da5-4ff8-aa97-a4f6c2026aa9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:20.115506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51f7c10a-2da5-4ff8-aa97-a4f6c2026aa9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:20.116043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-51f7c10a-2da5-4ff8-aa97-a4f6c2026aa9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:20.251699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-51f7c10a-2da5-4ff8-aa97-a4f6c2026aa9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:20.260210 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-51f7c10a-2da5-4ff8-aa97-a4f6c2026aa9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1363 microversion: 2.1 time: 0.162914 Sep 27 10:48:20.260832 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3662/6853] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:20 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1363 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:20.615370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d2309c85-89c3-4525-b5b5-51b22e658e25 req-e75e1b1e-d33a-4e8f-a4f6-2f0a6e6903aa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1577dded-7b7f-4041-9666-da60c77e78ec", "tag": "fef319cf-605a-4f0c-9994-d7cb071dbb0f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:48:20.622173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=58,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:48:20.628408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d2309c85-89c3-4525-b5b5-51b22e658e25 req-e75e1b1e-d33a-4e8f-a4f6-2f0a6e6903aa service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:20.628916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d2309c85-89c3-4525-b5b5-51b22e658e25 req-e75e1b1e-d33a-4e8f-a4f6-2f0a6e6903aa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:20.629496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d2309c85-89c3-4525-b5b5-51b22e658e25 req-e75e1b1e-d33a-4e8f-a4f6-2f0a6e6903aa service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:20.642287 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d2309c85-89c3-4525-b5b5-51b22e658e25 req-e75e1b1e-d33a-4e8f-a4f6-2f0a6e6903aa service nova] Creating event network-changed:fef319cf-605a-4f0c-9994-d7cb071dbb0f for instance 1577dded-7b7f-4041-9666-da60c77e78ec on nch2-master-1 Sep 27 10:48:20.647545 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d2309c85-89c3-4525-b5b5-51b22e658e25 req-e75e1b1e-d33a-4e8f-a4f6-2f0a6e6903aa service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036256 Sep 27 10:48:20.648715 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3192/6854] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:48:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:48:21.285710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-753bf7d4-7f8b-4527-8981-691cc0a7d989 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:21.286352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-753bf7d4-7f8b-4527-8981-691cc0a7d989 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:21.294396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-753bf7d4-7f8b-4527-8981-691cc0a7d989 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:21.294899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-753bf7d4-7f8b-4527-8981-691cc0a7d989 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:21.295344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-753bf7d4-7f8b-4527-8981-691cc0a7d989 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:21.296066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-753bf7d4-7f8b-4527-8981-691cc0a7d989 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:21.296535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-753bf7d4-7f8b-4527-8981-691cc0a7d989 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:21.297010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-753bf7d4-7f8b-4527-8981-691cc0a7d989 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:21.450944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-753bf7d4-7f8b-4527-8981-691cc0a7d989 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:21.458876 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-753bf7d4-7f8b-4527-8981-691cc0a7d989 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1363 microversion: 2.1 time: 0.177856 Sep 27 10:48:21.459465 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3663/6855] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:21 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1363 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:22.488104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0fda236d-5817-4d5e-8e93-a89942b92a61 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:22.489136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0fda236d-5817-4d5e-8e93-a89942b92a61 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:22.498114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fda236d-5817-4d5e-8e93-a89942b92a61 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:22.498728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fda236d-5817-4d5e-8e93-a89942b92a61 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:22.499512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fda236d-5817-4d5e-8e93-a89942b92a61 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:22.500399 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fda236d-5817-4d5e-8e93-a89942b92a61 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:22.501017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fda236d-5817-4d5e-8e93-a89942b92a61 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:22.501564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0fda236d-5817-4d5e-8e93-a89942b92a61 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:22.643380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0fda236d-5817-4d5e-8e93-a89942b92a61 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:22.651716 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0fda236d-5817-4d5e-8e93-a89942b92a61 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1363 microversion: 2.1 time: 0.171919 Sep 27 10:48:22.652306 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3193/6856] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:22 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1363 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:23.675705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a1fd1a0-eab5-4b66-992a-c8b3f7946a48 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:23.676331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3a1fd1a0-eab5-4b66-992a-c8b3f7946a48 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:23.684234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1fd1a0-eab5-4b66-992a-c8b3f7946a48 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:23.684755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1fd1a0-eab5-4b66-992a-c8b3f7946a48 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:23.685203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1fd1a0-eab5-4b66-992a-c8b3f7946a48 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:23.685917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1fd1a0-eab5-4b66-992a-c8b3f7946a48 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:23.686380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1fd1a0-eab5-4b66-992a-c8b3f7946a48 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:23.686810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a1fd1a0-eab5-4b66-992a-c8b3f7946a48 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:23.821177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3a1fd1a0-eab5-4b66-992a-c8b3f7946a48 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:23.829429 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3a1fd1a0-eab5-4b66-992a-c8b3f7946a48 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1363 microversion: 2.1 time: 0.157811 Sep 27 10:48:23.829933 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3664/6857] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:23 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1363 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:24.854550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aa250c1a-9711-49f6-b9ac-b5b81a32fdbe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:24.855235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aa250c1a-9711-49f6-b9ac-b5b81a32fdbe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:24.863573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa250c1a-9711-49f6-b9ac-b5b81a32fdbe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:24.864088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa250c1a-9711-49f6-b9ac-b5b81a32fdbe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:24.864615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa250c1a-9711-49f6-b9ac-b5b81a32fdbe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:24.865493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa250c1a-9711-49f6-b9ac-b5b81a32fdbe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:24.866088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa250c1a-9711-49f6-b9ac-b5b81a32fdbe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:24.866588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aa250c1a-9711-49f6-b9ac-b5b81a32fdbe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:25.013053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-aa250c1a-9711-49f6-b9ac-b5b81a32fdbe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:25.025188 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aa250c1a-9711-49f6-b9ac-b5b81a32fdbe tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1363 microversion: 2.1 time: 0.175518 Sep 27 10:48:25.025817 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3194/6858] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:24 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1363 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:25.596044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-013d5e8b-9b1b-4165-a93a-3284ff2a109c req-064e8015-a78b-4f04-9df1-c98ea8f347c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1577dded-7b7f-4041-9666-da60c77e78ec", "name": "network-vif-plugged", "status": "completed", "tag": "fef319cf-605a-4f0c-9994-d7cb071dbb0f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:48:25.608178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-013d5e8b-9b1b-4165-a93a-3284ff2a109c req-064e8015-a78b-4f04-9df1-c98ea8f347c3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:25.608729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-013d5e8b-9b1b-4165-a93a-3284ff2a109c req-064e8015-a78b-4f04-9df1-c98ea8f347c3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:25.609290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-013d5e8b-9b1b-4165-a93a-3284ff2a109c req-064e8015-a78b-4f04-9df1-c98ea8f347c3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:25.624281 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-013d5e8b-9b1b-4165-a93a-3284ff2a109c req-064e8015-a78b-4f04-9df1-c98ea8f347c3 service nova] Creating event network-vif-plugged:fef319cf-605a-4f0c-9994-d7cb071dbb0f for instance 1577dded-7b7f-4041-9666-da60c77e78ec on nch2-master-1 Sep 27 10:48:25.629599 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-013d5e8b-9b1b-4165-a93a-3284ff2a109c req-064e8015-a78b-4f04-9df1-c98ea8f347c3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037495 Sep 27 10:48:25.630478 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3665/6859] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:48:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:48:26.054024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8a748b3d-2e37-45e3-ac6f-236afcf271f0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:26.054646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8a748b3d-2e37-45e3-ac6f-236afcf271f0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:26.062125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a748b3d-2e37-45e3-ac6f-236afcf271f0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:26.062654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a748b3d-2e37-45e3-ac6f-236afcf271f0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:26.063106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a748b3d-2e37-45e3-ac6f-236afcf271f0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:26.063831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a748b3d-2e37-45e3-ac6f-236afcf271f0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:26.064297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a748b3d-2e37-45e3-ac6f-236afcf271f0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:26.064773 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8a748b3d-2e37-45e3-ac6f-236afcf271f0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:26.204590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8a748b3d-2e37-45e3-ac6f-236afcf271f0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:26.213054 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8a748b3d-2e37-45e3-ac6f-236afcf271f0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1363 microversion: 2.1 time: 0.165572 Sep 27 10:48:26.213798 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3195/6860] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:26 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1363 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:27.239022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b8f44f16-b02b-4157-ad79-2e7b5ee6ce19 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:27.239834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b8f44f16-b02b-4157-ad79-2e7b5ee6ce19 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:27.246904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8f44f16-b02b-4157-ad79-2e7b5ee6ce19 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:27.247415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8f44f16-b02b-4157-ad79-2e7b5ee6ce19 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:27.247845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8f44f16-b02b-4157-ad79-2e7b5ee6ce19 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:27.248556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8f44f16-b02b-4157-ad79-2e7b5ee6ce19 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:27.249046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8f44f16-b02b-4157-ad79-2e7b5ee6ce19 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:27.249477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8f44f16-b02b-4157-ad79-2e7b5ee6ce19 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:27.383694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b8f44f16-b02b-4157-ad79-2e7b5ee6ce19 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:48:27.391780 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b8f44f16-b02b-4157-ad79-2e7b5ee6ce19 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1551 microversion: 2.1 time: 0.158106 Sep 27 10:48:27.392366 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3666/6861] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:48:27 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1551 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:48:27.415397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b94d4f9d-2d49-4a3b-ac1e-cea129bf2b46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:48:27.416306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b94d4f9d-2d49-4a3b-ac1e-cea129bf2b46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:48:27.423938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b94d4f9d-2d49-4a3b-ac1e-cea129bf2b46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:27.424432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b94d4f9d-2d49-4a3b-ac1e-cea129bf2b46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:27.424924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b94d4f9d-2d49-4a3b-ac1e-cea129bf2b46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:27.425661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b94d4f9d-2d49-4a3b-ac1e-cea129bf2b46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:27.426128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b94d4f9d-2d49-4a3b-ac1e-cea129bf2b46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:27.426623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b94d4f9d-2d49-4a3b-ac1e-cea129bf2b46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:27.509295 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b94d4f9d-2d49-4a3b-ac1e-cea129bf2b46 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.096803 Sep 27 10:48:27.510116 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3196/6862] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:48:27 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec/os-interface => generated 285 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:48:28.793419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-885f96d1-db89-437a-a781-9949d0b3c90d req-7ff1067f-c7f3-4816-8779-690cdc2587bf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1577dded-7b7f-4041-9666-da60c77e78ec", "tag": "fef319cf-605a-4f0c-9994-d7cb071dbb0f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:48:28.798469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:48:28.805687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-885f96d1-db89-437a-a781-9949d0b3c90d req-7ff1067f-c7f3-4816-8779-690cdc2587bf service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:48:28.806166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-885f96d1-db89-437a-a781-9949d0b3c90d req-7ff1067f-c7f3-4816-8779-690cdc2587bf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:48:28.806665 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-885f96d1-db89-437a-a781-9949d0b3c90d req-7ff1067f-c7f3-4816-8779-690cdc2587bf service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:48:28.818137 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-885f96d1-db89-437a-a781-9949d0b3c90d req-7ff1067f-c7f3-4816-8779-690cdc2587bf service nova] Creating event network-changed:fef319cf-605a-4f0c-9994-d7cb071dbb0f for instance 1577dded-7b7f-4041-9666-da60c77e78ec on nch2-master-1 Sep 27 10:48:28.823543 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-885f96d1-db89-437a-a781-9949d0b3c90d req-7ff1067f-c7f3-4816-8779-690cdc2587bf service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034399 Sep 27 10:48:28.824207 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3667/6863] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:48:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:49:40.782661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:40.783570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:40.786382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=9 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:40.787146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=46 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:40.856912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:49:40.859344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:40.866488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:40.866982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:40.867435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:40.868147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:40.868621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:40.869096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:40.898497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:40.904692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Old instance type m1.heat2, new instance type m1.heat {{(pid=145969) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:49:40.911482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:49:40.911918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:49:40.912404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:49:40.912858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:49:40.913277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:49:40.987343 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4c6ade60-9f40-448e-a512-2faa1e0a65fb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec/action" status: 202 len: 0 microversion: 2.1 time: 0.206227 Sep 27 10:49:40.987955 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3197/6864] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:49:40 2022] POST /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec/action => generated 0 bytes in 208 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:49:40.996250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=11 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:40.997093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=57,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:49:40.998149 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 10:49:40.998494 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 10:49:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec (ip 10.222.0.26) !!! Sep 27 10:49:41.004761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67aae07e-58f8-4214-810b-1be81de27d0d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:41.005484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67aae07e-58f8-4214-810b-1be81de27d0d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:41.015233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67aae07e-58f8-4214-810b-1be81de27d0d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:41.015785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67aae07e-58f8-4214-810b-1be81de27d0d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:41.016238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67aae07e-58f8-4214-810b-1be81de27d0d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:41.017187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67aae07e-58f8-4214-810b-1be81de27d0d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:41.017562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67aae07e-58f8-4214-810b-1be81de27d0d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:41.017996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67aae07e-58f8-4214-810b-1be81de27d0d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:41.181895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-67aae07e-58f8-4214-810b-1be81de27d0d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:41.190430 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67aae07e-58f8-4214-810b-1be81de27d0d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1678 microversion: 2.1 time: 0.197766 Sep 27 10:49:41.190975 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3668/6865] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:40 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1678 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:42.213975 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6ea10211-0705-459e-88bc-5f4bac4dea9b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:42.215113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6ea10211-0705-459e-88bc-5f4bac4dea9b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:42.221810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ea10211-0705-459e-88bc-5f4bac4dea9b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:42.222347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ea10211-0705-459e-88bc-5f4bac4dea9b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:42.222791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ea10211-0705-459e-88bc-5f4bac4dea9b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:42.223499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ea10211-0705-459e-88bc-5f4bac4dea9b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:42.223963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ea10211-0705-459e-88bc-5f4bac4dea9b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:42.224437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6ea10211-0705-459e-88bc-5f4bac4dea9b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:42.784393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6ea10211-0705-459e-88bc-5f4bac4dea9b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:42.793408 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6ea10211-0705-459e-88bc-5f4bac4dea9b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1678 microversion: 2.1 time: 0.583440 Sep 27 10:49:42.804687 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3198/6866] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:42 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1678 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:43.824750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b01f215b-376f-4c05-8cc1-7ea529a1aa30 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:43.825426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b01f215b-376f-4c05-8cc1-7ea529a1aa30 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:43.833001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01f215b-376f-4c05-8cc1-7ea529a1aa30 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:43.833500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01f215b-376f-4c05-8cc1-7ea529a1aa30 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:43.833950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01f215b-376f-4c05-8cc1-7ea529a1aa30 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:43.834670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01f215b-376f-4c05-8cc1-7ea529a1aa30 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:43.835122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01f215b-376f-4c05-8cc1-7ea529a1aa30 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:43.835563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b01f215b-376f-4c05-8cc1-7ea529a1aa30 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:43.971825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b01f215b-376f-4c05-8cc1-7ea529a1aa30 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:43.980602 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b01f215b-376f-4c05-8cc1-7ea529a1aa30 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1683 microversion: 2.1 time: 0.163149 Sep 27 10:49:43.981299 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3669/6867] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:43 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1683 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:45.010070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-529079d4-1ed0-4472-b2a4-b0e2ffea04d1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:45.010954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-529079d4-1ed0-4472-b2a4-b0e2ffea04d1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:45.022659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-529079d4-1ed0-4472-b2a4-b0e2ffea04d1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:45.023498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-529079d4-1ed0-4472-b2a4-b0e2ffea04d1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:45.024251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-529079d4-1ed0-4472-b2a4-b0e2ffea04d1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:45.025469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-529079d4-1ed0-4472-b2a4-b0e2ffea04d1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:45.026256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-529079d4-1ed0-4472-b2a4-b0e2ffea04d1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:45.027182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-529079d4-1ed0-4472-b2a4-b0e2ffea04d1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:45.171277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-529079d4-1ed0-4472-b2a4-b0e2ffea04d1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:45.184437 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-529079d4-1ed0-4472-b2a4-b0e2ffea04d1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1683 microversion: 2.1 time: 0.180344 Sep 27 10:49:45.185302 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3199/6868] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:45 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1683 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:46.213653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4a90f20-ea6e-4433-8006-6710172d7591 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:46.214486 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d4a90f20-ea6e-4433-8006-6710172d7591 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:46.223973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4a90f20-ea6e-4433-8006-6710172d7591 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:46.224777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4a90f20-ea6e-4433-8006-6710172d7591 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:46.225402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4a90f20-ea6e-4433-8006-6710172d7591 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:46.226368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4a90f20-ea6e-4433-8006-6710172d7591 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:46.227024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4a90f20-ea6e-4433-8006-6710172d7591 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:46.227595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4a90f20-ea6e-4433-8006-6710172d7591 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:46.372832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d4a90f20-ea6e-4433-8006-6710172d7591 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:46.384967 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4a90f20-ea6e-4433-8006-6710172d7591 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1683 microversion: 2.1 time: 0.176406 Sep 27 10:49:46.385865 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3670/6869] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:46 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1683 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:47.414826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c6958c03-f873-44ff-a8d4-a7311936373f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:47.415917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c6958c03-f873-44ff-a8d4-a7311936373f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:47.428412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6958c03-f873-44ff-a8d4-a7311936373f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:47.429313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6958c03-f873-44ff-a8d4-a7311936373f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:47.430076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6958c03-f873-44ff-a8d4-a7311936373f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:47.431266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6958c03-f873-44ff-a8d4-a7311936373f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:47.432061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6958c03-f873-44ff-a8d4-a7311936373f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:47.432845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c6958c03-f873-44ff-a8d4-a7311936373f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:47.574074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c6958c03-f873-44ff-a8d4-a7311936373f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:47.582435 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c6958c03-f873-44ff-a8d4-a7311936373f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1683 microversion: 2.1 time: 0.174777 Sep 27 10:49:47.583441 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3200/6870] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:47 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1683 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:48.610688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba405c38-8a3f-47ba-a054-eb80028fc3e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:48.611344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba405c38-8a3f-47ba-a054-eb80028fc3e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:48.618746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba405c38-8a3f-47ba-a054-eb80028fc3e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:48.619244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba405c38-8a3f-47ba-a054-eb80028fc3e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:48.619687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba405c38-8a3f-47ba-a054-eb80028fc3e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:48.620407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba405c38-8a3f-47ba-a054-eb80028fc3e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:48.620919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba405c38-8a3f-47ba-a054-eb80028fc3e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:48.621359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba405c38-8a3f-47ba-a054-eb80028fc3e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:48.758567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba405c38-8a3f-47ba-a054-eb80028fc3e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:48.766716 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba405c38-8a3f-47ba-a054-eb80028fc3e1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1683 microversion: 2.1 time: 0.160524 Sep 27 10:49:48.767295 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3671/6871] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:48 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1683 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:49.794889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-827d20a2-358c-4b1c-b774-91866de553e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:49.795532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-827d20a2-358c-4b1c-b774-91866de553e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:49.802979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-827d20a2-358c-4b1c-b774-91866de553e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:49.803487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-827d20a2-358c-4b1c-b774-91866de553e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:49.803948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-827d20a2-358c-4b1c-b774-91866de553e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:49.804716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-827d20a2-358c-4b1c-b774-91866de553e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:49.805948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-827d20a2-358c-4b1c-b774-91866de553e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:49.806610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-827d20a2-358c-4b1c-b774-91866de553e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:49.944563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-827d20a2-358c-4b1c-b774-91866de553e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:49.954527 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-827d20a2-358c-4b1c-b774-91866de553e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1683 microversion: 2.1 time: 0.167520 Sep 27 10:49:49.955115 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3201/6872] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:49 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1683 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:50.983313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c1a34687-6ff6-45fe-8b72-fc117e27f750 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:50.983880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c1a34687-6ff6-45fe-8b72-fc117e27f750 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:50.991666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1a34687-6ff6-45fe-8b72-fc117e27f750 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:50.992154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1a34687-6ff6-45fe-8b72-fc117e27f750 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:50.992556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1a34687-6ff6-45fe-8b72-fc117e27f750 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:50.993318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1a34687-6ff6-45fe-8b72-fc117e27f750 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:50.993771 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1a34687-6ff6-45fe-8b72-fc117e27f750 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:50.994203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1a34687-6ff6-45fe-8b72-fc117e27f750 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:51.131013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c1a34687-6ff6-45fe-8b72-fc117e27f750 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:51.139018 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c1a34687-6ff6-45fe-8b72-fc117e27f750 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1683 microversion: 2.1 time: 0.162944 Sep 27 10:49:51.139607 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3672/6873] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:50 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1683 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:52.167958 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-78150533-0515-49c7-936c-13810f04b020 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:52.169103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-78150533-0515-49c7-936c-13810f04b020 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:52.177702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78150533-0515-49c7-936c-13810f04b020 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:52.178205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78150533-0515-49c7-936c-13810f04b020 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:52.178664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78150533-0515-49c7-936c-13810f04b020 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:52.179385 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78150533-0515-49c7-936c-13810f04b020 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:52.179860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78150533-0515-49c7-936c-13810f04b020 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:52.180300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-78150533-0515-49c7-936c-13810f04b020 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:52.326684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-78150533-0515-49c7-936c-13810f04b020 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:52.335108 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-78150533-0515-49c7-936c-13810f04b020 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1683 microversion: 2.1 time: 0.174229 Sep 27 10:49:52.335663 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3202/6874] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:52 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1683 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:53.363420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b8fc37a5-3baf-4de8-8750-e721d7aedcf3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:53.364297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b8fc37a5-3baf-4de8-8750-e721d7aedcf3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:53.375909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8fc37a5-3baf-4de8-8750-e721d7aedcf3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:53.376396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8fc37a5-3baf-4de8-8750-e721d7aedcf3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:53.376869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8fc37a5-3baf-4de8-8750-e721d7aedcf3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:53.377576 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8fc37a5-3baf-4de8-8750-e721d7aedcf3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:53.378036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8fc37a5-3baf-4de8-8750-e721d7aedcf3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:53.378459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b8fc37a5-3baf-4de8-8750-e721d7aedcf3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:53.533494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b8fc37a5-3baf-4de8-8750-e721d7aedcf3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:53.542059 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b8fc37a5-3baf-4de8-8750-e721d7aedcf3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1683 microversion: 2.1 time: 0.185708 Sep 27 10:49:53.542611 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3673/6875] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:53 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1683 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:54.578316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d01ec919-aae8-4b71-97ba-85effbbd597f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:54.579079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d01ec919-aae8-4b71-97ba-85effbbd597f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:54.585984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01ec919-aae8-4b71-97ba-85effbbd597f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:54.586502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01ec919-aae8-4b71-97ba-85effbbd597f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:54.586954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01ec919-aae8-4b71-97ba-85effbbd597f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:54.587670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01ec919-aae8-4b71-97ba-85effbbd597f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:54.588141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01ec919-aae8-4b71-97ba-85effbbd597f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:54.588580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d01ec919-aae8-4b71-97ba-85effbbd597f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:54.723008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d01ec919-aae8-4b71-97ba-85effbbd597f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:54.731504 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d01ec919-aae8-4b71-97ba-85effbbd597f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1683 microversion: 2.1 time: 0.160184 Sep 27 10:49:54.732087 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3203/6876] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:54 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1683 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:55.727673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-893d8632-31d3-4299-b152-0aa359f1230e req-dee59dfe-6dc8-4b8b-9e41-28fb2b28ac71 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1577dded-7b7f-4041-9666-da60c77e78ec", "name": "network-vif-unplugged", "status": "completed", "tag": "fef319cf-605a-4f0c-9994-d7cb071dbb0f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:49:55.737636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-893d8632-31d3-4299-b152-0aa359f1230e req-dee59dfe-6dc8-4b8b-9e41-28fb2b28ac71 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:55.738114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-893d8632-31d3-4299-b152-0aa359f1230e req-dee59dfe-6dc8-4b8b-9e41-28fb2b28ac71 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:55.738544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-893d8632-31d3-4299-b152-0aa359f1230e req-dee59dfe-6dc8-4b8b-9e41-28fb2b28ac71 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:55.751326 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-893d8632-31d3-4299-b152-0aa359f1230e req-dee59dfe-6dc8-4b8b-9e41-28fb2b28ac71 service nova] Creating event network-vif-unplugged:fef319cf-605a-4f0c-9994-d7cb071dbb0f for instance 1577dded-7b7f-4041-9666-da60c77e78ec on nch1-master-1 Sep 27 10:49:55.754414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b19adfcd-ee00-45fe-be0c-f7a4bf91eb70 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:55.755042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b19adfcd-ee00-45fe-be0c-f7a4bf91eb70 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:55.757907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-893d8632-31d3-4299-b152-0aa359f1230e req-dee59dfe-6dc8-4b8b-9e41-28fb2b28ac71 service nova] Instance 1577dded-7b7f-4041-9666-da60c77e78ec is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:49:55.763829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b19adfcd-ee00-45fe-be0c-f7a4bf91eb70 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:55.764475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b19adfcd-ee00-45fe-be0c-f7a4bf91eb70 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:55.765012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b19adfcd-ee00-45fe-be0c-f7a4bf91eb70 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:55.766330 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b19adfcd-ee00-45fe-be0c-f7a4bf91eb70 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:55.767036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b19adfcd-ee00-45fe-be0c-f7a4bf91eb70 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:55.767692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b19adfcd-ee00-45fe-be0c-f7a4bf91eb70 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:55.769910 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-893d8632-31d3-4299-b152-0aa359f1230e req-dee59dfe-6dc8-4b8b-9e41-28fb2b28ac71 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.114625 Sep 27 10:49:55.770835 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3674/6877] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:49:55 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:49:55.980808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b19adfcd-ee00-45fe-be0c-f7a4bf91eb70 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:55.989238 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b19adfcd-ee00-45fe-be0c-f7a4bf91eb70 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1682 microversion: 2.1 time: 0.238840 Sep 27 10:49:55.989807 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3204/6878] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:55 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1682 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:57.020820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c5de7740-29b4-43ca-9939-fd5ddba5a98d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:57.021793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c5de7740-29b4-43ca-9939-fd5ddba5a98d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:57.032450 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5de7740-29b4-43ca-9939-fd5ddba5a98d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:57.032974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5de7740-29b4-43ca-9939-fd5ddba5a98d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:57.033427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5de7740-29b4-43ca-9939-fd5ddba5a98d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:57.034299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5de7740-29b4-43ca-9939-fd5ddba5a98d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:57.034777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5de7740-29b4-43ca-9939-fd5ddba5a98d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:57.035259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c5de7740-29b4-43ca-9939-fd5ddba5a98d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:57.170520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c5de7740-29b4-43ca-9939-fd5ddba5a98d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:57.178720 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c5de7740-29b4-43ca-9939-fd5ddba5a98d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1680 microversion: 2.1 time: 0.167524 Sep 27 10:49:57.179313 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3675/6879] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:57 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1680 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:57.797464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-56eea7f7-9247-48ad-90d5-188bbbd08e8b req-4a17ac8f-2099-4dec-8dd5-614b66de88f8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1577dded-7b7f-4041-9666-da60c77e78ec", "tag": "fef319cf-605a-4f0c-9994-d7cb071dbb0f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:49:57.815101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-56eea7f7-9247-48ad-90d5-188bbbd08e8b req-4a17ac8f-2099-4dec-8dd5-614b66de88f8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:57.815699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-56eea7f7-9247-48ad-90d5-188bbbd08e8b req-4a17ac8f-2099-4dec-8dd5-614b66de88f8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:57.816182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-56eea7f7-9247-48ad-90d5-188bbbd08e8b req-4a17ac8f-2099-4dec-8dd5-614b66de88f8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:57.829478 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-56eea7f7-9247-48ad-90d5-188bbbd08e8b req-4a17ac8f-2099-4dec-8dd5-614b66de88f8 service nova] Creating event network-changed:fef319cf-605a-4f0c-9994-d7cb071dbb0f for instance 1577dded-7b7f-4041-9666-da60c77e78ec on nch1-master-1 Sep 27 10:49:57.835136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-56eea7f7-9247-48ad-90d5-188bbbd08e8b req-4a17ac8f-2099-4dec-8dd5-614b66de88f8 service nova] Instance 1577dded-7b7f-4041-9666-da60c77e78ec is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:49:57.846100 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-56eea7f7-9247-48ad-90d5-188bbbd08e8b req-4a17ac8f-2099-4dec-8dd5-614b66de88f8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055418 Sep 27 10:49:57.846695 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3205/6880] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:49:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:49:58.202827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a03f9266-77af-47d3-b58e-768a29d7d6a7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:58.203585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a03f9266-77af-47d3-b58e-768a29d7d6a7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:58.210714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a03f9266-77af-47d3-b58e-768a29d7d6a7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:58.211210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a03f9266-77af-47d3-b58e-768a29d7d6a7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:58.211648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a03f9266-77af-47d3-b58e-768a29d7d6a7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:58.212363 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a03f9266-77af-47d3-b58e-768a29d7d6a7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:58.212848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a03f9266-77af-47d3-b58e-768a29d7d6a7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:58.213279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a03f9266-77af-47d3-b58e-768a29d7d6a7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:58.359917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a03f9266-77af-47d3-b58e-768a29d7d6a7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:58.367913 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a03f9266-77af-47d3-b58e-768a29d7d6a7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1680 microversion: 2.1 time: 0.169151 Sep 27 10:49:58.368515 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3676/6881] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:58 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1680 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:59.398733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b9f453e8-c455-495d-beb8-0b71ab903419 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:59.399775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b9f453e8-c455-495d-beb8-0b71ab903419 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:59.409145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9f453e8-c455-495d-beb8-0b71ab903419 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:59.409643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9f453e8-c455-495d-beb8-0b71ab903419 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:59.410093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9f453e8-c455-495d-beb8-0b71ab903419 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:59.410800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9f453e8-c455-495d-beb8-0b71ab903419 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:59.411271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9f453e8-c455-495d-beb8-0b71ab903419 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:59.411710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9f453e8-c455-495d-beb8-0b71ab903419 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:59.555396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b9f453e8-c455-495d-beb8-0b71ab903419 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:49:59.563742 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b9f453e8-c455-495d-beb8-0b71ab903419 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1677 microversion: 2.1 time: 0.174266 Sep 27 10:49:59.564300 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3206/6882] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:59 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1677 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:49:59.586438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:49:59.587051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:59.594645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:59.595145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:59.595586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:59.596287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:59.596745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:59.597228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:59.636069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:59.636550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:59.637013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:59.637726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:59.638184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:59.638602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:59.659494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:49:59.665056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for user 396fe9ce0f7542c382b01d1f5779c13f and project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:49:59.774400 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0160a3e6-9d3f-478d-9d92-d53d3afbf637 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec/action" status: 202 len: 0 microversion: 2.1 time: 0.191697 Sep 27 10:49:59.776067 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3677/6883] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:49:59 2022] POST /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec/action => generated 0 bytes in 193 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:49:59.783523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-da807f5b-9bcd-4822-8118-c842232bdc6a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:49:59.784157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-da807f5b-9bcd-4822-8118-c842232bdc6a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:49:59.790911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da807f5b-9bcd-4822-8118-c842232bdc6a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:59.791439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da807f5b-9bcd-4822-8118-c842232bdc6a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:59.791914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da807f5b-9bcd-4822-8118-c842232bdc6a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:59.792766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da807f5b-9bcd-4822-8118-c842232bdc6a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:49:59.793154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da807f5b-9bcd-4822-8118-c842232bdc6a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:49:59.793592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-da807f5b-9bcd-4822-8118-c842232bdc6a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:49:59.993650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-da807f5b-9bcd-4822-8118-c842232bdc6a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:00.002769 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-da807f5b-9bcd-4822-8118-c842232bdc6a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1676 microversion: 2.1 time: 0.223171 Sep 27 10:50:00.003886 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3207/6884] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:49:59 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1676 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:01.034815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4b645250-c36b-4f83-b53a-0c71581f805e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:01.035707 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4b645250-c36b-4f83-b53a-0c71581f805e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:01.045203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b645250-c36b-4f83-b53a-0c71581f805e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:01.045787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b645250-c36b-4f83-b53a-0c71581f805e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:01.046318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b645250-c36b-4f83-b53a-0c71581f805e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:01.047213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b645250-c36b-4f83-b53a-0c71581f805e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:01.047996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b645250-c36b-4f83-b53a-0c71581f805e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:01.048512 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b645250-c36b-4f83-b53a-0c71581f805e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:01.190580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4b645250-c36b-4f83-b53a-0c71581f805e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:01.202266 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4b645250-c36b-4f83-b53a-0c71581f805e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1676 microversion: 2.1 time: 0.175414 Sep 27 10:50:01.203014 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3678/6885] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:01 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1676 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:02.226556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1bbaf627-348f-4f7f-ae9d-1c1fbed95dbf tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:02.227232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1bbaf627-348f-4f7f-ae9d-1c1fbed95dbf tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:02.253239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbaf627-348f-4f7f-ae9d-1c1fbed95dbf tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:02.253842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbaf627-348f-4f7f-ae9d-1c1fbed95dbf tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:02.254304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbaf627-348f-4f7f-ae9d-1c1fbed95dbf tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:02.255109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbaf627-348f-4f7f-ae9d-1c1fbed95dbf tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:02.255805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbaf627-348f-4f7f-ae9d-1c1fbed95dbf tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:02.258113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbaf627-348f-4f7f-ae9d-1c1fbed95dbf tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:02.322821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6180f347-86fc-4c75-a7a0-6cab43f795d0 req-4327a9b8-f91c-4ba9-864b-32a117284276 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1577dded-7b7f-4041-9666-da60c77e78ec", "name": "network-vif-plugged", "status": "completed", "tag": "fef319cf-605a-4f0c-9994-d7cb071dbb0f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:50:02.335255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6180f347-86fc-4c75-a7a0-6cab43f795d0 req-4327a9b8-f91c-4ba9-864b-32a117284276 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:02.335739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6180f347-86fc-4c75-a7a0-6cab43f795d0 req-4327a9b8-f91c-4ba9-864b-32a117284276 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:02.336175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6180f347-86fc-4c75-a7a0-6cab43f795d0 req-4327a9b8-f91c-4ba9-864b-32a117284276 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:02.350088 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6180f347-86fc-4c75-a7a0-6cab43f795d0 req-4327a9b8-f91c-4ba9-864b-32a117284276 service nova] Creating event network-vif-plugged:fef319cf-605a-4f0c-9994-d7cb071dbb0f for instance 1577dded-7b7f-4041-9666-da60c77e78ec on nch1-master-1 Sep 27 10:50:02.356141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-6180f347-86fc-4c75-a7a0-6cab43f795d0 req-4327a9b8-f91c-4ba9-864b-32a117284276 service nova] Instance 1577dded-7b7f-4041-9666-da60c77e78ec is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:50:02.368545 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6180f347-86fc-4c75-a7a0-6cab43f795d0 req-4327a9b8-f91c-4ba9-864b-32a117284276 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.138788 Sep 27 10:50:02.369475 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3679/6886] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:50:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:50:02.404590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1bbaf627-348f-4f7f-ae9d-1c1fbed95dbf tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:02.413244 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1bbaf627-348f-4f7f-ae9d-1c1fbed95dbf tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1676 microversion: 2.1 time: 0.190679 Sep 27 10:50:02.413843 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3208/6887] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:02 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1676 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:03.273121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-21f00215-d60d-4f3a-b251-2acb339f0f43 req-9069c707-bd04-44f5-ac58-224c48b5a4f2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1577dded-7b7f-4041-9666-da60c77e78ec", "name": "network-vif-unplugged", "status": "completed", "tag": "fef319cf-605a-4f0c-9994-d7cb071dbb0f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:50:03.283725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-21f00215-d60d-4f3a-b251-2acb339f0f43 req-9069c707-bd04-44f5-ac58-224c48b5a4f2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:03.284246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-21f00215-d60d-4f3a-b251-2acb339f0f43 req-9069c707-bd04-44f5-ac58-224c48b5a4f2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:03.284800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-21f00215-d60d-4f3a-b251-2acb339f0f43 req-9069c707-bd04-44f5-ac58-224c48b5a4f2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:03.297333 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-21f00215-d60d-4f3a-b251-2acb339f0f43 req-9069c707-bd04-44f5-ac58-224c48b5a4f2 service nova] Creating event network-vif-unplugged:fef319cf-605a-4f0c-9994-d7cb071dbb0f for instance 1577dded-7b7f-4041-9666-da60c77e78ec on nch2-master-1 Sep 27 10:50:03.302932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-21f00215-d60d-4f3a-b251-2acb339f0f43 req-9069c707-bd04-44f5-ac58-224c48b5a4f2 service nova] Instance 1577dded-7b7f-4041-9666-da60c77e78ec is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:50:03.314177 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-21f00215-d60d-4f3a-b251-2acb339f0f43 req-9069c707-bd04-44f5-ac58-224c48b5a4f2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.046824 Sep 27 10:50:03.315252 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3680/6888] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:50:03 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:50:03.437909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9d5b2851-5921-4091-9f60-77b989615df3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:03.438468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9d5b2851-5921-4091-9f60-77b989615df3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:03.446090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d5b2851-5921-4091-9f60-77b989615df3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:03.446603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d5b2851-5921-4091-9f60-77b989615df3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:03.447125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d5b2851-5921-4091-9f60-77b989615df3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:03.447895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d5b2851-5921-4091-9f60-77b989615df3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:03.448368 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d5b2851-5921-4091-9f60-77b989615df3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:03.448855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9d5b2851-5921-4091-9f60-77b989615df3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:03.650619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9d5b2851-5921-4091-9f60-77b989615df3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:03.660123 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9d5b2851-5921-4091-9f60-77b989615df3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1676 microversion: 2.1 time: 0.226109 Sep 27 10:50:03.660828 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3209/6889] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:03 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1676 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:04.689353 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b45ec9cb-8876-429f-b5a7-c5aaf44bb752 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:04.690509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b45ec9cb-8876-429f-b5a7-c5aaf44bb752 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:04.701860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b45ec9cb-8876-429f-b5a7-c5aaf44bb752 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:04.702601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b45ec9cb-8876-429f-b5a7-c5aaf44bb752 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:04.703296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b45ec9cb-8876-429f-b5a7-c5aaf44bb752 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:04.704431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b45ec9cb-8876-429f-b5a7-c5aaf44bb752 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:04.705284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b45ec9cb-8876-429f-b5a7-c5aaf44bb752 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:04.706018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b45ec9cb-8876-429f-b5a7-c5aaf44bb752 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:04.843049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b45ec9cb-8876-429f-b5a7-c5aaf44bb752 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:04.852764 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b45ec9cb-8876-429f-b5a7-c5aaf44bb752 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1676 microversion: 2.1 time: 0.170668 Sep 27 10:50:04.853351 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3681/6890] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:04 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1676 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:05.355756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-944a4729-7487-4884-b9cd-116000d2f25d req-7071b108-118d-48d9-8644-9d535f075b62 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1577dded-7b7f-4041-9666-da60c77e78ec", "tag": "fef319cf-605a-4f0c-9994-d7cb071dbb0f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:50:05.367899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-944a4729-7487-4884-b9cd-116000d2f25d req-7071b108-118d-48d9-8644-9d535f075b62 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:05.368403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-944a4729-7487-4884-b9cd-116000d2f25d req-7071b108-118d-48d9-8644-9d535f075b62 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:05.368905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-944a4729-7487-4884-b9cd-116000d2f25d req-7071b108-118d-48d9-8644-9d535f075b62 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:05.394294 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-944a4729-7487-4884-b9cd-116000d2f25d req-7071b108-118d-48d9-8644-9d535f075b62 service nova] Creating event network-changed:fef319cf-605a-4f0c-9994-d7cb071dbb0f for instance 1577dded-7b7f-4041-9666-da60c77e78ec on nch2-master-1 Sep 27 10:50:05.400250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-944a4729-7487-4884-b9cd-116000d2f25d req-7071b108-118d-48d9-8644-9d535f075b62 service nova] Instance 1577dded-7b7f-4041-9666-da60c77e78ec is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:50:05.411429 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-944a4729-7487-4884-b9cd-116000d2f25d req-7071b108-118d-48d9-8644-9d535f075b62 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060699 Sep 27 10:50:05.412054 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3210/6891] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:50:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:50:05.876791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7598d68d-c543-46d4-a36f-7e90f760c498 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:05.877413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7598d68d-c543-46d4-a36f-7e90f760c498 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:05.891166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7598d68d-c543-46d4-a36f-7e90f760c498 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:05.891654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7598d68d-c543-46d4-a36f-7e90f760c498 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:05.892090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7598d68d-c543-46d4-a36f-7e90f760c498 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:05.892832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7598d68d-c543-46d4-a36f-7e90f760c498 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:05.893319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7598d68d-c543-46d4-a36f-7e90f760c498 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:05.893938 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7598d68d-c543-46d4-a36f-7e90f760c498 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:05.918549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:06.081243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7598d68d-c543-46d4-a36f-7e90f760c498 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:06.089724 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7598d68d-c543-46d4-a36f-7e90f760c498 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1676 microversion: 2.1 time: 0.216934 Sep 27 10:50:06.090544 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3682/6892] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:05 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1676 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:07.113280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ee8ba899-f44a-4b28-b68f-3d1277d852f1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:07.114006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ee8ba899-f44a-4b28-b68f-3d1277d852f1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:07.121515 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee8ba899-f44a-4b28-b68f-3d1277d852f1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:07.122014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee8ba899-f44a-4b28-b68f-3d1277d852f1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:07.122467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee8ba899-f44a-4b28-b68f-3d1277d852f1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:07.123189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee8ba899-f44a-4b28-b68f-3d1277d852f1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:07.123660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee8ba899-f44a-4b28-b68f-3d1277d852f1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:07.124100 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee8ba899-f44a-4b28-b68f-3d1277d852f1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:07.291336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ee8ba899-f44a-4b28-b68f-3d1277d852f1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:07.299565 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ee8ba899-f44a-4b28-b68f-3d1277d852f1 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1676 microversion: 2.1 time: 0.190695 Sep 27 10:50:07.300122 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3211/6893] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:07 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1676 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:08.323605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ff192b96-561d-400f-945e-4711abff7252 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:08.324328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ff192b96-561d-400f-945e-4711abff7252 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:08.331783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff192b96-561d-400f-945e-4711abff7252 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:08.332269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff192b96-561d-400f-945e-4711abff7252 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:08.332715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff192b96-561d-400f-945e-4711abff7252 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:08.333449 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff192b96-561d-400f-945e-4711abff7252 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:08.333907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff192b96-561d-400f-945e-4711abff7252 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:08.334333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff192b96-561d-400f-945e-4711abff7252 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:08.473186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ff192b96-561d-400f-945e-4711abff7252 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:08.481759 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ff192b96-561d-400f-945e-4711abff7252 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1669 microversion: 2.1 time: 0.162316 Sep 27 10:50:08.482378 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3683/6894] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:08 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1669 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:08.513258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-09badd7d-99f1-424f-940b-609378915681 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:08.513802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-09badd7d-99f1-424f-940b-609378915681 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:08.521412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09badd7d-99f1-424f-940b-609378915681 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:08.521913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09badd7d-99f1-424f-940b-609378915681 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:08.522367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09badd7d-99f1-424f-940b-609378915681 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:08.523080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09badd7d-99f1-424f-940b-609378915681 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:08.523581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09badd7d-99f1-424f-940b-609378915681 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:08.524146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-09badd7d-99f1-424f-940b-609378915681 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:08.661661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-09badd7d-99f1-424f-940b-609378915681 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:08.665785 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:08.670888 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-09badd7d-99f1-424f-940b-609378915681 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1669 microversion: 2.1 time: 0.161816 Sep 27 10:50:08.671513 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3212/6895] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:08 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1669 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:08.694655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b2adbe33-9b85-4edb-b64f-d3bacd0fad6c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:08.695275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b2adbe33-9b85-4edb-b64f-d3bacd0fad6c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:08.701623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2adbe33-9b85-4edb-b64f-d3bacd0fad6c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:08.702119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2adbe33-9b85-4edb-b64f-d3bacd0fad6c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:08.702560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2adbe33-9b85-4edb-b64f-d3bacd0fad6c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:08.703302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2adbe33-9b85-4edb-b64f-d3bacd0fad6c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:08.703732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2adbe33-9b85-4edb-b64f-d3bacd0fad6c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:08.704160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b2adbe33-9b85-4edb-b64f-d3bacd0fad6c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:08.838162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b2adbe33-9b85-4edb-b64f-d3bacd0fad6c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:08.846188 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b2adbe33-9b85-4edb-b64f-d3bacd0fad6c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1669 microversion: 2.1 time: 0.155814 Sep 27 10:50:08.846771 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3684/6896] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:08 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1669 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:08.869727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7160e4d9-b2c8-4bd1-b0e9-aa4a62e44edb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:08.870244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7160e4d9-b2c8-4bd1-b0e9-aa4a62e44edb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:08.877106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7160e4d9-b2c8-4bd1-b0e9-aa4a62e44edb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:08.877614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7160e4d9-b2c8-4bd1-b0e9-aa4a62e44edb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:08.878073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7160e4d9-b2c8-4bd1-b0e9-aa4a62e44edb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:08.879356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7160e4d9-b2c8-4bd1-b0e9-aa4a62e44edb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:08.879836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7160e4d9-b2c8-4bd1-b0e9-aa4a62e44edb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:08.880283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7160e4d9-b2c8-4bd1-b0e9-aa4a62e44edb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:08.909688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7160e4d9-b2c8-4bd1-b0e9-aa4a62e44edb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:50:09.002453 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7160e4d9-b2c8-4bd1-b0e9-aa4a62e44edb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 204 len: 0 microversion: 2.1 time: 0.136588 Sep 27 10:50:09.003359 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3213/6897] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:50:08 2022] DELETE /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 0 bytes in 139 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:50:09.011768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-34600a51-776b-4b83-ba3c-dd20a7b297d0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:09.012391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-34600a51-776b-4b83-ba3c-dd20a7b297d0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:09.018734 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34600a51-776b-4b83-ba3c-dd20a7b297d0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:09.019229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34600a51-776b-4b83-ba3c-dd20a7b297d0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:09.019689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34600a51-776b-4b83-ba3c-dd20a7b297d0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:09.020389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34600a51-776b-4b83-ba3c-dd20a7b297d0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:09.020921 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34600a51-776b-4b83-ba3c-dd20a7b297d0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:09.021381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-34600a51-776b-4b83-ba3c-dd20a7b297d0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:09.167940 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-34600a51-776b-4b83-ba3c-dd20a7b297d0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:09.180194 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-34600a51-776b-4b83-ba3c-dd20a7b297d0 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1675 microversion: 2.1 time: 0.172020 Sep 27 10:50:09.181058 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3685/6898] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:09 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1675 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:10.193213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-3293178b-d4dc-4d54-adde-3f759ec5c6c4 req-efecbdee-cf8f-43de-9333-18e7c2970e50 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1577dded-7b7f-4041-9666-da60c77e78ec", "name": "network-vif-plugged", "status": "completed", "tag": "fef319cf-605a-4f0c-9994-d7cb071dbb0f"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:50:10.206561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3293178b-d4dc-4d54-adde-3f759ec5c6c4 req-efecbdee-cf8f-43de-9333-18e7c2970e50 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:10.208090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3293178b-d4dc-4d54-adde-3f759ec5c6c4 req-efecbdee-cf8f-43de-9333-18e7c2970e50 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:10.208090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-3293178b-d4dc-4d54-adde-3f759ec5c6c4 req-efecbdee-cf8f-43de-9333-18e7c2970e50 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:10.209236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a9a219cf-0b36-4394-882e-666b0dc06077 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:10.209993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a9a219cf-0b36-4394-882e-666b0dc06077 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:10.212517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:10.218991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9a219cf-0b36-4394-882e-666b0dc06077 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:10.219478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9a219cf-0b36-4394-882e-666b0dc06077 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:10.219916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9a219cf-0b36-4394-882e-666b0dc06077 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:10.220209 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-3293178b-d4dc-4d54-adde-3f759ec5c6c4 req-efecbdee-cf8f-43de-9333-18e7c2970e50 service nova] Creating event network-vif-plugged:fef319cf-605a-4f0c-9994-d7cb071dbb0f for instance 1577dded-7b7f-4041-9666-da60c77e78ec on nch2-master-1 Sep 27 10:50:10.220625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9a219cf-0b36-4394-882e-666b0dc06077 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:10.221110 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9a219cf-0b36-4394-882e-666b0dc06077 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:10.221627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a9a219cf-0b36-4394-882e-666b0dc06077 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:10.225593 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-3293178b-d4dc-4d54-adde-3f759ec5c6c4 req-efecbdee-cf8f-43de-9333-18e7c2970e50 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037154 Sep 27 10:50:10.226387 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3214/6899] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:50:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:50:10.356800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a9a219cf-0b36-4394-882e-666b0dc06077 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:10.365103 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a9a219cf-0b36-4394-882e-666b0dc06077 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1675 microversion: 2.1 time: 0.161173 Sep 27 10:50:10.365673 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3686/6900] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:10 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1675 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:11.390272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7730d85d-70fd-4fbf-a54c-dccb8cb79ac7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:11.391639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7730d85d-70fd-4fbf-a54c-dccb8cb79ac7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:11.394267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=46,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:11.399967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7730d85d-70fd-4fbf-a54c-dccb8cb79ac7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:11.400478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7730d85d-70fd-4fbf-a54c-dccb8cb79ac7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:11.400966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7730d85d-70fd-4fbf-a54c-dccb8cb79ac7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:11.401918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7730d85d-70fd-4fbf-a54c-dccb8cb79ac7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:11.402397 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7730d85d-70fd-4fbf-a54c-dccb8cb79ac7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:11.402844 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7730d85d-70fd-4fbf-a54c-dccb8cb79ac7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:11.502295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7730d85d-70fd-4fbf-a54c-dccb8cb79ac7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:11.511878 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7730d85d-70fd-4fbf-a54c-dccb8cb79ac7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 200 len: 1608 microversion: 2.1 time: 0.126792 Sep 27 10:50:11.512475 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3215/6901] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:11 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 1608 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:11.587704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a5e5bff4-15e8-4dcd-8faf-229f52529c8e req-89800a89-4066-4f82-8130-8384641c7f05 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1577dded-7b7f-4041-9666-da60c77e78ec", "name": "network-vif-deleted", "tag": "fef319cf-605a-4f0c-9994-d7cb071dbb0f"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:50:11.593999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=53,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:11.600303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a5e5bff4-15e8-4dcd-8faf-229f52529c8e req-89800a89-4066-4f82-8130-8384641c7f05 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:11.600830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a5e5bff4-15e8-4dcd-8faf-229f52529c8e req-89800a89-4066-4f82-8130-8384641c7f05 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:11.601487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a5e5bff4-15e8-4dcd-8faf-229f52529c8e req-89800a89-4066-4f82-8130-8384641c7f05 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:11.624123 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a5e5bff4-15e8-4dcd-8faf-229f52529c8e req-89800a89-4066-4f82-8130-8384641c7f05 service nova] Creating event network-vif-deleted:fef319cf-605a-4f0c-9994-d7cb071dbb0f for instance 1577dded-7b7f-4041-9666-da60c77e78ec on nch2-master-1 Sep 27 10:50:11.631115 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a5e5bff4-15e8-4dcd-8faf-229f52529c8e req-89800a89-4066-4f82-8130-8384641c7f05 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048781 Sep 27 10:50:11.631867 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3687/6902] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:50:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:50:12.536039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:12.536696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:12.544959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:12.545476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:12.545976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:12.546709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:12.547192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:12.547644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:12.562164 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec HTTP/1.1 Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1577dded-7b7f-4041-9666-da60c77e78ec'} Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:50:12.563888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:50:12.568929 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:50:12.568929 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:50:12.568929 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:50:12.568929 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:50:12.568929 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:50:12.568929 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:50:12.568929 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:50:12.568929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:50:12.568929 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0aac5f66-e2ad-46a4-a9c6-af16aa2e5b3d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 404 len: 111 microversion: 2.1 time: 0.033666 Sep 27 10:50:12.570725 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3216/6903] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:12 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:50:12.580413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:12.581297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:12.587833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:12.589051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:12.589499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:12.590217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:12.590680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:12.591119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:12.605543 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec HTTP/1.1 Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '1577dded-7b7f-4041-9666-da60c77e78ec'} Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:50:12.607699 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:50:12.612270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:50:12.612270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:50:12.612270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:50:12.612270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:50:12.612270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:50:12.612270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:50:12.612270 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:50:12.612270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:50:12.612270 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-23cc1547-1450-425a-bd2e-474ab60c187b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 404 len: 111 microversion: 2.1 time: 0.033219 Sep 27 10:50:12.612270 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3688/6904] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:12 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:50:12.661041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerActionsTestJSON-server-1773797759", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "8ae6d002-3910-401a-89df-d8a9ff70cdbf"}], "security_groups": [{"name": "tempest-securitygroup--404672176"}], "key_name": "tempest-keypair-107890057", "user_data": "IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIHVidW50dSB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH51YnVudHUvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:50:12.770434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] validate_networks() for [('8ae6d002-3910-401a-89df-d8a9ff70cdbf', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:50:13.230780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:50:13.231284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:50:13.231809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:50:13.232241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:50:13.232703 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:50:13.241913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:13.242421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:13.242878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:13.243633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:13.244109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:13.244552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:13.262116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:50:13.267653 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for user 396fe9ce0f7542c382b01d1f5779c13f and project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:50:13.280400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:50:13.282808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:50:13.334564 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-76c10ef7-24ca-4f22-965c-3f4f3e9e70dc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.677543 Sep 27 10:50:13.335193 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3217/6905] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:50:12 2022] POST /compute/v2.1/servers => generated 405 bytes in 679 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:50:13.346788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75400516-b2d5-46a1-9021-a397d526f037 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:13.347410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-75400516-b2d5-46a1-9021-a397d526f037 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:13.406684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-75400516-b2d5-46a1-9021-a397d526f037 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:13.414723 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-75400516-b2d5-46a1-9021-a397d526f037 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1238 microversion: 2.1 time: 0.071791 Sep 27 10:50:13.415277 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3689/6906] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:13 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1238 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:14.441151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d28da79b-66c1-4c30-a028-d85fc0e08130 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:14.441824 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d28da79b-66c1-4c30-a028-d85fc0e08130 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:14.449483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d28da79b-66c1-4c30-a028-d85fc0e08130 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:14.450014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d28da79b-66c1-4c30-a028-d85fc0e08130 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:14.450571 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d28da79b-66c1-4c30-a028-d85fc0e08130 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:14.451588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d28da79b-66c1-4c30-a028-d85fc0e08130 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:14.452396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d28da79b-66c1-4c30-a028-d85fc0e08130 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:14.453215 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d28da79b-66c1-4c30-a028-d85fc0e08130 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:14.850394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d28da79b-66c1-4c30-a028-d85fc0e08130 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:14.858561 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d28da79b-66c1-4c30-a028-d85fc0e08130 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1298 microversion: 2.1 time: 0.422902 Sep 27 10:50:14.859111 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3218/6907] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:14 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1298 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:15.883284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-518b806a-9c79-4451-a0e8-a95c3fe984e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:15.884048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-518b806a-9c79-4451-a0e8-a95c3fe984e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:15.892574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-518b806a-9c79-4451-a0e8-a95c3fe984e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:15.893069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-518b806a-9c79-4451-a0e8-a95c3fe984e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:15.893595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-518b806a-9c79-4451-a0e8-a95c3fe984e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:15.894396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-518b806a-9c79-4451-a0e8-a95c3fe984e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:15.894859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-518b806a-9c79-4451-a0e8-a95c3fe984e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:15.895292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-518b806a-9c79-4451-a0e8-a95c3fe984e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:16.036798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-518b806a-9c79-4451-a0e8-a95c3fe984e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:16.045825 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-518b806a-9c79-4451-a0e8-a95c3fe984e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1363 microversion: 2.1 time: 0.166709 Sep 27 10:50:16.046405 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3690/6908] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:15 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1363 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:16.233105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ed6daa7d-bd72-4bbe-8d9f-aaafb5c95c34 req-8c673f78-378c-4387-9ff9-5a830b2acf68 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8934bb4a-3501-480f-a0af-6ff5156f83fb", "tag": "318657ab-0eb5-43d1-a2b3-f4217b82903a"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:50:16.246025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ed6daa7d-bd72-4bbe-8d9f-aaafb5c95c34 req-8c673f78-378c-4387-9ff9-5a830b2acf68 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:16.246663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ed6daa7d-bd72-4bbe-8d9f-aaafb5c95c34 req-8c673f78-378c-4387-9ff9-5a830b2acf68 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:16.247588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ed6daa7d-bd72-4bbe-8d9f-aaafb5c95c34 req-8c673f78-378c-4387-9ff9-5a830b2acf68 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:16.261586 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ed6daa7d-bd72-4bbe-8d9f-aaafb5c95c34 req-8c673f78-378c-4387-9ff9-5a830b2acf68 service nova] Creating event network-changed:318657ab-0eb5-43d1-a2b3-f4217b82903a for instance 8934bb4a-3501-480f-a0af-6ff5156f83fb on nch1-master-1 Sep 27 10:50:16.267078 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ed6daa7d-bd72-4bbe-8d9f-aaafb5c95c34 req-8c673f78-378c-4387-9ff9-5a830b2acf68 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037686 Sep 27 10:50:16.267759 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3219/6909] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:50:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:50:17.075105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f91c524d-49f2-448e-86cf-1ef961191bde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:17.076092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f91c524d-49f2-448e-86cf-1ef961191bde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:17.085662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f91c524d-49f2-448e-86cf-1ef961191bde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:17.086157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f91c524d-49f2-448e-86cf-1ef961191bde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:17.086596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f91c524d-49f2-448e-86cf-1ef961191bde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:17.087308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f91c524d-49f2-448e-86cf-1ef961191bde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:17.087766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f91c524d-49f2-448e-86cf-1ef961191bde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:17.088194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f91c524d-49f2-448e-86cf-1ef961191bde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:17.236137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f91c524d-49f2-448e-86cf-1ef961191bde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:17.248237 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f91c524d-49f2-448e-86cf-1ef961191bde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1363 microversion: 2.1 time: 0.179432 Sep 27 10:50:17.249409 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3691/6910] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:17 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1363 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:18.276047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bfd50620-2a3c-419b-8f13-e7092421ab1c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:18.276681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bfd50620-2a3c-419b-8f13-e7092421ab1c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:18.283921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfd50620-2a3c-419b-8f13-e7092421ab1c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:18.284454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfd50620-2a3c-419b-8f13-e7092421ab1c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:18.285029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfd50620-2a3c-419b-8f13-e7092421ab1c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:18.285794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfd50620-2a3c-419b-8f13-e7092421ab1c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:18.286463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfd50620-2a3c-419b-8f13-e7092421ab1c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:18.286938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bfd50620-2a3c-419b-8f13-e7092421ab1c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:18.426301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bfd50620-2a3c-419b-8f13-e7092421ab1c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:18.434809 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bfd50620-2a3c-419b-8f13-e7092421ab1c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1363 microversion: 2.1 time: 0.165330 Sep 27 10:50:18.435379 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3220/6911] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:18 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1363 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:19.463413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-572f0841-78ba-495a-8803-e8db854afc02 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:19.464376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-572f0841-78ba-495a-8803-e8db854afc02 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:19.475062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-572f0841-78ba-495a-8803-e8db854afc02 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:19.475554 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-572f0841-78ba-495a-8803-e8db854afc02 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:19.475993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-572f0841-78ba-495a-8803-e8db854afc02 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:19.476708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-572f0841-78ba-495a-8803-e8db854afc02 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:19.477180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-572f0841-78ba-495a-8803-e8db854afc02 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:19.477604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-572f0841-78ba-495a-8803-e8db854afc02 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:19.624603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-572f0841-78ba-495a-8803-e8db854afc02 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:19.632902 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-572f0841-78ba-495a-8803-e8db854afc02 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1363 microversion: 2.1 time: 0.177701 Sep 27 10:50:19.633480 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3692/6912] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:19 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1363 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:20.657014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a7e0a891-9282-4577-99d9-ffae592163e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:20.657636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a7e0a891-9282-4577-99d9-ffae592163e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:20.665161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7e0a891-9282-4577-99d9-ffae592163e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:20.665692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7e0a891-9282-4577-99d9-ffae592163e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:20.666142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7e0a891-9282-4577-99d9-ffae592163e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:20.666859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7e0a891-9282-4577-99d9-ffae592163e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:20.667319 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7e0a891-9282-4577-99d9-ffae592163e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:20.667831 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a7e0a891-9282-4577-99d9-ffae592163e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:20.807168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a7e0a891-9282-4577-99d9-ffae592163e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:20.815327 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a7e0a891-9282-4577-99d9-ffae592163e8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1363 microversion: 2.1 time: 0.162379 Sep 27 10:50:20.816059 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3221/6913] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:20 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1363 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:21.847728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-117191ae-1671-423e-9139-0af493cbb608 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:21.848542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-117191ae-1671-423e-9139-0af493cbb608 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:21.855886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117191ae-1671-423e-9139-0af493cbb608 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:21.856374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117191ae-1671-423e-9139-0af493cbb608 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:21.856855 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117191ae-1671-423e-9139-0af493cbb608 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:21.857566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117191ae-1671-423e-9139-0af493cbb608 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:21.858024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117191ae-1671-423e-9139-0af493cbb608 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:21.858453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-117191ae-1671-423e-9139-0af493cbb608 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:21.988198 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-117191ae-1671-423e-9139-0af493cbb608 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:21.996434 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-117191ae-1671-423e-9139-0af493cbb608 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1363 microversion: 2.1 time: 0.155369 Sep 27 10:50:21.997057 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3693/6914] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:21 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1363 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:22.330294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-69acbb3a-72dc-4614-98e3-4bc427d91b76 req-8d8242d1-c048-4ce6-a4fd-7989fef7db6c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8934bb4a-3501-480f-a0af-6ff5156f83fb", "name": "network-vif-plugged", "status": "completed", "tag": "318657ab-0eb5-43d1-a2b3-f4217b82903a"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:50:22.340911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-69acbb3a-72dc-4614-98e3-4bc427d91b76 req-8d8242d1-c048-4ce6-a4fd-7989fef7db6c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:22.341421 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-69acbb3a-72dc-4614-98e3-4bc427d91b76 req-8d8242d1-c048-4ce6-a4fd-7989fef7db6c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:22.341867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-69acbb3a-72dc-4614-98e3-4bc427d91b76 req-8d8242d1-c048-4ce6-a4fd-7989fef7db6c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:22.354675 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-69acbb3a-72dc-4614-98e3-4bc427d91b76 req-8d8242d1-c048-4ce6-a4fd-7989fef7db6c service nova] Creating event network-vif-plugged:318657ab-0eb5-43d1-a2b3-f4217b82903a for instance 8934bb4a-3501-480f-a0af-6ff5156f83fb on nch1-master-1 Sep 27 10:50:22.360407 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-69acbb3a-72dc-4614-98e3-4bc427d91b76 req-8d8242d1-c048-4ce6-a4fd-7989fef7db6c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033708 Sep 27 10:50:22.361084 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3222/6915] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:50:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:50:23.023419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ff7266e9-841b-4179-93a3-0bc63fd2cdde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:23.024039 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ff7266e9-841b-4179-93a3-0bc63fd2cdde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:23.031412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff7266e9-841b-4179-93a3-0bc63fd2cdde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:23.031902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff7266e9-841b-4179-93a3-0bc63fd2cdde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:23.032338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff7266e9-841b-4179-93a3-0bc63fd2cdde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:23.033057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff7266e9-841b-4179-93a3-0bc63fd2cdde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:23.033523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff7266e9-841b-4179-93a3-0bc63fd2cdde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:23.033951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff7266e9-841b-4179-93a3-0bc63fd2cdde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:23.175390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ff7266e9-841b-4179-93a3-0bc63fd2cdde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:23.188398 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ff7266e9-841b-4179-93a3-0bc63fd2cdde tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1363 microversion: 2.1 time: 0.169252 Sep 27 10:50:23.189462 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3694/6916] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:23 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1363 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:24.214861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-20c93c7e-4f17-4d6a-8c0c-524c44d35791 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:24.215620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-20c93c7e-4f17-4d6a-8c0c-524c44d35791 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:24.217921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:24.218621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:24.226027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20c93c7e-4f17-4d6a-8c0c-524c44d35791 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:24.226548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20c93c7e-4f17-4d6a-8c0c-524c44d35791 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:24.227001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20c93c7e-4f17-4d6a-8c0c-524c44d35791 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:24.227731 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20c93c7e-4f17-4d6a-8c0c-524c44d35791 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:24.228194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20c93c7e-4f17-4d6a-8c0c-524c44d35791 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:24.228643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-20c93c7e-4f17-4d6a-8c0c-524c44d35791 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:24.370094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-20c93c7e-4f17-4d6a-8c0c-524c44d35791 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:50:24.383024 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-20c93c7e-4f17-4d6a-8c0c-524c44d35791 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1550 microversion: 2.1 time: 0.172242 Sep 27 10:50:24.383802 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3223/6917] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:50:24 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1550 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:50:24.414222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f9999a3e-bd1b-4a78-b406-5a3eada73510 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:50:24.414788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f9999a3e-bd1b-4a78-b406-5a3eada73510 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:50:24.417650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:50:24.426805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9999a3e-bd1b-4a78-b406-5a3eada73510 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:24.427537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9999a3e-bd1b-4a78-b406-5a3eada73510 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:24.428295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9999a3e-bd1b-4a78-b406-5a3eada73510 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:24.429444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9999a3e-bd1b-4a78-b406-5a3eada73510 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:24.429904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9999a3e-bd1b-4a78-b406-5a3eada73510 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:24.430333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f9999a3e-bd1b-4a78-b406-5a3eada73510 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:24.518747 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f9999a3e-bd1b-4a78-b406-5a3eada73510 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.110999 Sep 27 10:50:24.519351 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3695/6918] 10.222.0.26 () {58 vars in 1275 bytes} [Tue Sep 27 10:50:24 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb/os-interface => generated 284 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:50:25.803944 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-6acebb56-4dcb-4bcf-9663-e8b8e9c66982 req-6ad5ad15-6b0f-4b69-a268-3b8b27fa134b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8934bb4a-3501-480f-a0af-6ff5156f83fb", "tag": "318657ab-0eb5-43d1-a2b3-f4217b82903a"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:50:25.813851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6acebb56-4dcb-4bcf-9663-e8b8e9c66982 req-6ad5ad15-6b0f-4b69-a268-3b8b27fa134b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:50:25.814265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6acebb56-4dcb-4bcf-9663-e8b8e9c66982 req-6ad5ad15-6b0f-4b69-a268-3b8b27fa134b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:50:25.814710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-6acebb56-4dcb-4bcf-9663-e8b8e9c66982 req-6ad5ad15-6b0f-4b69-a268-3b8b27fa134b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:50:25.826391 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-6acebb56-4dcb-4bcf-9663-e8b8e9c66982 req-6ad5ad15-6b0f-4b69-a268-3b8b27fa134b service nova] Creating event network-changed:318657ab-0eb5-43d1-a2b3-f4217b82903a for instance 8934bb4a-3501-480f-a0af-6ff5156f83fb on nch1-master-1 Sep 27 10:50:25.831370 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-6acebb56-4dcb-4bcf-9663-e8b8e9c66982 req-6ad5ad15-6b0f-4b69-a268-3b8b27fa134b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.100581 Sep 27 10:50:25.832275 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3224/6919] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:50:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:51:02.432945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=19 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:02.433766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=64 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:02.437963 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0554bed5-1d56-432a-9c77-f6b761b4ccb8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:02.438660 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0554bed5-1d56-432a-9c77-f6b761b4ccb8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:02.445844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0554bed5-1d56-432a-9c77-f6b761b4ccb8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:02.446337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0554bed5-1d56-432a-9c77-f6b761b4ccb8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:02.446781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0554bed5-1d56-432a-9c77-f6b761b4ccb8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:02.447485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0554bed5-1d56-432a-9c77-f6b761b4ccb8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:02.447975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0554bed5-1d56-432a-9c77-f6b761b4ccb8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:02.448408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0554bed5-1d56-432a-9c77-f6b761b4ccb8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:02.588277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0554bed5-1d56-432a-9c77-f6b761b4ccb8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:02.600315 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0554bed5-1d56-432a-9c77-f6b761b4ccb8 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1668 microversion: 2.1 time: 0.169218 Sep 27 10:51:02.601216 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3696/6920] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:02 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1668 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:02.625342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=32,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:02.626317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:02.631159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b7e7255e-a594-4b33-8042-1027c4a0e712 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c5d7e76d-768e-416c-a698-6f718f9340c4"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:02.633767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b7e7255e-a594-4b33-8042-1027c4a0e712 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:02.642084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7e7255e-a594-4b33-8042-1027c4a0e712 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:02.642619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7e7255e-a594-4b33-8042-1027c4a0e712 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:02.643079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7e7255e-a594-4b33-8042-1027c4a0e712 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:02.643806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7e7255e-a594-4b33-8042-1027c4a0e712 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:02.644280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7e7255e-a594-4b33-8042-1027c4a0e712 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:02.644893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b7e7255e-a594-4b33-8042-1027c4a0e712 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:02.922082 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b7e7255e-a594-4b33-8042-1027c4a0e712 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.299483 Sep 27 10:51:02.922910 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3225/6921] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:51:02 2022] POST /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb/os-volume_attachments => generated 194 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:51:06.270456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:06.272791 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:06.279770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:06.280262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:06.280717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:06.281438 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:06.281901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:06.282332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:06.311672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:06.317893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Old instance type m1.heat2, new instance type m1.heat {{(pid=145968) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 10:51:06.325493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:51:06.325918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:51:06.326396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:51:06.326814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:51:06.327230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:51:06.403867 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9238330c-7598-4fd2-9f18-1a9a94c715cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb/action" status: 202 len: 0 microversion: 2.1 time: 0.137844 Sep 27 10:51:06.404439 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3697/6922] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:51:06 2022] POST /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb/action => generated 0 bytes in 139 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:51:06.414794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0c7227c0-667f-40f8-b9ca-0c465b7a5684 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:06.415536 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0c7227c0-667f-40f8-b9ca-0c465b7a5684 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:06.422336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c7227c0-667f-40f8-b9ca-0c465b7a5684 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:06.422850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c7227c0-667f-40f8-b9ca-0c465b7a5684 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:06.423311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c7227c0-667f-40f8-b9ca-0c465b7a5684 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:06.424041 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c7227c0-667f-40f8-b9ca-0c465b7a5684 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:06.424522 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c7227c0-667f-40f8-b9ca-0c465b7a5684 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:06.425024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0c7227c0-667f-40f8-b9ca-0c465b7a5684 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:06.570116 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0c7227c0-667f-40f8-b9ca-0c465b7a5684 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:06.578684 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0c7227c0-667f-40f8-b9ca-0c465b7a5684 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1723 microversion: 2.1 time: 0.169618 Sep 27 10:51:06.579257 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3226/6923] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:06 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1723 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:07.603585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-88185524-8ffa-4856-8574-98a15487289f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:07.604966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-88185524-8ffa-4856-8574-98a15487289f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:07.611580 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-88185524-8ffa-4856-8574-98a15487289f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:07.612084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-88185524-8ffa-4856-8574-98a15487289f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:07.612532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-88185524-8ffa-4856-8574-98a15487289f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:07.613288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-88185524-8ffa-4856-8574-98a15487289f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:07.613799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-88185524-8ffa-4856-8574-98a15487289f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:07.614241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-88185524-8ffa-4856-8574-98a15487289f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:07.755412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-88185524-8ffa-4856-8574-98a15487289f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:07.764257 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-88185524-8ffa-4856-8574-98a15487289f tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1723 microversion: 2.1 time: 0.165582 Sep 27 10:51:07.764869 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3698/6924] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:07 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1723 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:08.789711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-048dc39b-361b-4782-aa9d-87c4dda14181 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:08.790376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-048dc39b-361b-4782-aa9d-87c4dda14181 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:08.797847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-048dc39b-361b-4782-aa9d-87c4dda14181 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:08.798351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-048dc39b-361b-4782-aa9d-87c4dda14181 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:08.798799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-048dc39b-361b-4782-aa9d-87c4dda14181 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:08.799512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-048dc39b-361b-4782-aa9d-87c4dda14181 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:08.799980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-048dc39b-361b-4782-aa9d-87c4dda14181 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:08.800410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-048dc39b-361b-4782-aa9d-87c4dda14181 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:08.951444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-048dc39b-361b-4782-aa9d-87c4dda14181 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:08.960875 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-048dc39b-361b-4782-aa9d-87c4dda14181 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1728 microversion: 2.1 time: 0.175782 Sep 27 10:51:08.961747 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3227/6925] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:08 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1728 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:09.989950 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d3faf17d-b3dc-4ab2-8aea-ae68233f014a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:09.990503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d3faf17d-b3dc-4ab2-8aea-ae68233f014a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:09.997470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3faf17d-b3dc-4ab2-8aea-ae68233f014a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:09.997965 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3faf17d-b3dc-4ab2-8aea-ae68233f014a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:09.998404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3faf17d-b3dc-4ab2-8aea-ae68233f014a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:09.999112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3faf17d-b3dc-4ab2-8aea-ae68233f014a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:09.999586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3faf17d-b3dc-4ab2-8aea-ae68233f014a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:10.000018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d3faf17d-b3dc-4ab2-8aea-ae68233f014a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:10.151431 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d3faf17d-b3dc-4ab2-8aea-ae68233f014a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:10.160312 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d3faf17d-b3dc-4ab2-8aea-ae68233f014a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1728 microversion: 2.1 time: 0.177096 Sep 27 10:51:10.160892 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3699/6926] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:09 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1728 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:11.188038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fe9028aa-6c0f-43d5-b77e-e5f4c9a09a4c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:11.188940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fe9028aa-6c0f-43d5-b77e-e5f4c9a09a4c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:11.198786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe9028aa-6c0f-43d5-b77e-e5f4c9a09a4c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:11.199485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe9028aa-6c0f-43d5-b77e-e5f4c9a09a4c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:11.200208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe9028aa-6c0f-43d5-b77e-e5f4c9a09a4c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:11.201196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe9028aa-6c0f-43d5-b77e-e5f4c9a09a4c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:11.201874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe9028aa-6c0f-43d5-b77e-e5f4c9a09a4c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:11.202504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fe9028aa-6c0f-43d5-b77e-e5f4c9a09a4c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:11.345485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fe9028aa-6c0f-43d5-b77e-e5f4c9a09a4c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:11.359121 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fe9028aa-6c0f-43d5-b77e-e5f4c9a09a4c tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1728 microversion: 2.1 time: 0.176317 Sep 27 10:51:11.359987 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3228/6927] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:11 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1728 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:12.387031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d4ef30d7-1b20-4160-b273-26f5d3365bc2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:12.387789 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d4ef30d7-1b20-4160-b273-26f5d3365bc2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:12.396858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4ef30d7-1b20-4160-b273-26f5d3365bc2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:12.397466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4ef30d7-1b20-4160-b273-26f5d3365bc2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:12.398014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4ef30d7-1b20-4160-b273-26f5d3365bc2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:12.398888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4ef30d7-1b20-4160-b273-26f5d3365bc2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:12.399461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4ef30d7-1b20-4160-b273-26f5d3365bc2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:12.399996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d4ef30d7-1b20-4160-b273-26f5d3365bc2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:12.547347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d4ef30d7-1b20-4160-b273-26f5d3365bc2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:12.556599 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d4ef30d7-1b20-4160-b273-26f5d3365bc2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1728 microversion: 2.1 time: 0.174505 Sep 27 10:51:12.560733 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3700/6928] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:12 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1728 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:13.584864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a39aa04d-c7be-4946-8cac-e3f478d5af26 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:13.585479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a39aa04d-c7be-4946-8cac-e3f478d5af26 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:13.593485 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a39aa04d-c7be-4946-8cac-e3f478d5af26 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:13.594023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a39aa04d-c7be-4946-8cac-e3f478d5af26 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:13.594504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a39aa04d-c7be-4946-8cac-e3f478d5af26 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:13.595273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a39aa04d-c7be-4946-8cac-e3f478d5af26 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:13.597094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a39aa04d-c7be-4946-8cac-e3f478d5af26 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:13.597592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a39aa04d-c7be-4946-8cac-e3f478d5af26 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:13.743939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a39aa04d-c7be-4946-8cac-e3f478d5af26 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:13.756698 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a39aa04d-c7be-4946-8cac-e3f478d5af26 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1728 microversion: 2.1 time: 0.178362 Sep 27 10:51:13.757260 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3229/6929] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:13 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1728 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:14.784926 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1642c944-5d66-4a63-ac73-26a6ece8c1bc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:14.785939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1642c944-5d66-4a63-ac73-26a6ece8c1bc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:14.793978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1642c944-5d66-4a63-ac73-26a6ece8c1bc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:14.794461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1642c944-5d66-4a63-ac73-26a6ece8c1bc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:14.794904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1642c944-5d66-4a63-ac73-26a6ece8c1bc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:14.795597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1642c944-5d66-4a63-ac73-26a6ece8c1bc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:14.796052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1642c944-5d66-4a63-ac73-26a6ece8c1bc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:14.796476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1642c944-5d66-4a63-ac73-26a6ece8c1bc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:14.933569 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1642c944-5d66-4a63-ac73-26a6ece8c1bc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:14.942050 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1642c944-5d66-4a63-ac73-26a6ece8c1bc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1728 microversion: 2.1 time: 0.163768 Sep 27 10:51:14.942599 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3701/6930] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:14 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1728 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:15.966367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-81a0f8df-4745-475c-b003-25976313d623 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:15.966933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-81a0f8df-4745-475c-b003-25976313d623 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:15.974279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0f8df-4745-475c-b003-25976313d623 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:15.974828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0f8df-4745-475c-b003-25976313d623 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:15.975287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0f8df-4745-475c-b003-25976313d623 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:15.976117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0f8df-4745-475c-b003-25976313d623 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:15.976605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0f8df-4745-475c-b003-25976313d623 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:15.977150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-81a0f8df-4745-475c-b003-25976313d623 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:16.119733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-81a0f8df-4745-475c-b003-25976313d623 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:16.128942 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-81a0f8df-4745-475c-b003-25976313d623 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1728 microversion: 2.1 time: 0.166682 Sep 27 10:51:16.129532 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3230/6931] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:15 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1728 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:17.157675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4326dd77-b456-48c8-b49e-3a9ca921cddb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:17.158511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4326dd77-b456-48c8-b49e-3a9ca921cddb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:17.170888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4326dd77-b456-48c8-b49e-3a9ca921cddb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:17.171688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4326dd77-b456-48c8-b49e-3a9ca921cddb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:17.172419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4326dd77-b456-48c8-b49e-3a9ca921cddb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:17.173644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4326dd77-b456-48c8-b49e-3a9ca921cddb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:17.174564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4326dd77-b456-48c8-b49e-3a9ca921cddb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:17.175229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4326dd77-b456-48c8-b49e-3a9ca921cddb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:17.316436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4326dd77-b456-48c8-b49e-3a9ca921cddb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:17.325483 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4326dd77-b456-48c8-b49e-3a9ca921cddb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1728 microversion: 2.1 time: 0.174805 Sep 27 10:51:17.326064 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3702/6932] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:17 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1728 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:18.353102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cf302fd2-ae5d-4b0b-a49d-10d90649e1e6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:18.353850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cf302fd2-ae5d-4b0b-a49d-10d90649e1e6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:18.362410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf302fd2-ae5d-4b0b-a49d-10d90649e1e6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:18.363000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf302fd2-ae5d-4b0b-a49d-10d90649e1e6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:18.363537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf302fd2-ae5d-4b0b-a49d-10d90649e1e6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:18.364384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf302fd2-ae5d-4b0b-a49d-10d90649e1e6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:18.364978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf302fd2-ae5d-4b0b-a49d-10d90649e1e6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:18.365502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cf302fd2-ae5d-4b0b-a49d-10d90649e1e6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:18.505912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cf302fd2-ae5d-4b0b-a49d-10d90649e1e6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:18.515436 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cf302fd2-ae5d-4b0b-a49d-10d90649e1e6 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1728 microversion: 2.1 time: 0.168456 Sep 27 10:51:18.516034 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3231/6933] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:18 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1728 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:19.544845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6e312a37-82da-4a1e-8ab6-8c5bb3889b41 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:19.545922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6e312a37-82da-4a1e-8ab6-8c5bb3889b41 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:19.556277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e312a37-82da-4a1e-8ab6-8c5bb3889b41 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:19.556808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e312a37-82da-4a1e-8ab6-8c5bb3889b41 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:19.557246 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e312a37-82da-4a1e-8ab6-8c5bb3889b41 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:19.557947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e312a37-82da-4a1e-8ab6-8c5bb3889b41 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:19.558404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e312a37-82da-4a1e-8ab6-8c5bb3889b41 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:19.558836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6e312a37-82da-4a1e-8ab6-8c5bb3889b41 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:19.695955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6e312a37-82da-4a1e-8ab6-8c5bb3889b41 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:19.706460 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6e312a37-82da-4a1e-8ab6-8c5bb3889b41 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1728 microversion: 2.1 time: 0.168786 Sep 27 10:51:19.707045 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3703/6934] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:19 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1728 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:20.733255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-783caad2-09c8-4531-a211-9a0021d599f3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:20.733723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-783caad2-09c8-4531-a211-9a0021d599f3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:20.742351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-783caad2-09c8-4531-a211-9a0021d599f3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:20.742760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-783caad2-09c8-4531-a211-9a0021d599f3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:20.743392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-783caad2-09c8-4531-a211-9a0021d599f3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:20.744201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-783caad2-09c8-4531-a211-9a0021d599f3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:20.744804 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-783caad2-09c8-4531-a211-9a0021d599f3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:20.745735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-783caad2-09c8-4531-a211-9a0021d599f3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:20.792519 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 10:51:20.792862 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 10:51:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb (ip 10.222.0.26) !!! Sep 27 10:51:20.887544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-783caad2-09c8-4531-a211-9a0021d599f3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:20.897078 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-783caad2-09c8-4531-a211-9a0021d599f3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1728 microversion: 2.1 time: 0.169161 Sep 27 10:51:20.897901 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3232/6935] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:20 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1728 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:21.671416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7a6fc418-8a0f-42f2-89a3-9aa0d76da505 req-a65bf6a9-604b-4ff9-94e2-7c5d6e6d8b95 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8934bb4a-3501-480f-a0af-6ff5156f83fb", "name": "network-vif-unplugged", "status": "completed", "tag": "318657ab-0eb5-43d1-a2b3-f4217b82903a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:21.680496 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 10:51:21.680632 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 10:51:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.26) !!! Sep 27 10:51:21.686104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a6fc418-8a0f-42f2-89a3-9aa0d76da505 req-a65bf6a9-604b-4ff9-94e2-7c5d6e6d8b95 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:21.686590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a6fc418-8a0f-42f2-89a3-9aa0d76da505 req-a65bf6a9-604b-4ff9-94e2-7c5d6e6d8b95 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:21.687026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a6fc418-8a0f-42f2-89a3-9aa0d76da505 req-a65bf6a9-604b-4ff9-94e2-7c5d6e6d8b95 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:21.700169 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7a6fc418-8a0f-42f2-89a3-9aa0d76da505 req-a65bf6a9-604b-4ff9-94e2-7c5d6e6d8b95 service nova] Creating event network-vif-unplugged:318657ab-0eb5-43d1-a2b3-f4217b82903a for instance 8934bb4a-3501-480f-a0af-6ff5156f83fb on nch2-master-1 Sep 27 10:51:21.706477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-7a6fc418-8a0f-42f2-89a3-9aa0d76da505 req-a65bf6a9-604b-4ff9-94e2-7c5d6e6d8b95 service nova] Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:51:21.718359 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7a6fc418-8a0f-42f2-89a3-9aa0d76da505 req-a65bf6a9-604b-4ff9-94e2-7c5d6e6d8b95 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.050850 Sep 27 10:51:21.719275 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3704/6936] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:51:21 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:51:21.922542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-64c54230-f8f0-471c-8f06-6eb35fb3300b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:21.923101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-64c54230-f8f0-471c-8f06-6eb35fb3300b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:21.930973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64c54230-f8f0-471c-8f06-6eb35fb3300b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:21.931481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64c54230-f8f0-471c-8f06-6eb35fb3300b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:21.931935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64c54230-f8f0-471c-8f06-6eb35fb3300b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:21.932695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64c54230-f8f0-471c-8f06-6eb35fb3300b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:21.933170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64c54230-f8f0-471c-8f06-6eb35fb3300b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:21.933587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64c54230-f8f0-471c-8f06-6eb35fb3300b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:22.140268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-64c54230-f8f0-471c-8f06-6eb35fb3300b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:22.149573 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-64c54230-f8f0-471c-8f06-6eb35fb3300b tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1727 microversion: 2.1 time: 0.231150 Sep 27 10:51:22.150168 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3233/6937] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:21 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1727 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:23.174969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-908899d6-2e76-4bdd-932e-e7eaac5deb59 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:23.175670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-908899d6-2e76-4bdd-932e-e7eaac5deb59 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:23.182531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-908899d6-2e76-4bdd-932e-e7eaac5deb59 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:23.183082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-908899d6-2e76-4bdd-932e-e7eaac5deb59 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:23.183527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-908899d6-2e76-4bdd-932e-e7eaac5deb59 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:23.184293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-908899d6-2e76-4bdd-932e-e7eaac5deb59 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:23.184723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-908899d6-2e76-4bdd-932e-e7eaac5deb59 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:23.185157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-908899d6-2e76-4bdd-932e-e7eaac5deb59 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:23.325898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-908899d6-2e76-4bdd-932e-e7eaac5deb59 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:23.334621 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-908899d6-2e76-4bdd-932e-e7eaac5deb59 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1725 microversion: 2.1 time: 0.164972 Sep 27 10:51:23.335239 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3705/6938] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:23 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1725 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:23.758189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-9acbbc7e-2c39-487f-bb76-9401ef747e68 req-869acf6e-7597-4c5b-a91b-4cff1224b375 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8934bb4a-3501-480f-a0af-6ff5156f83fb", "tag": "318657ab-0eb5-43d1-a2b3-f4217b82903a"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:23.769273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9acbbc7e-2c39-487f-bb76-9401ef747e68 req-869acf6e-7597-4c5b-a91b-4cff1224b375 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:23.769796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9acbbc7e-2c39-487f-bb76-9401ef747e68 req-869acf6e-7597-4c5b-a91b-4cff1224b375 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:23.770480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9acbbc7e-2c39-487f-bb76-9401ef747e68 req-869acf6e-7597-4c5b-a91b-4cff1224b375 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:23.783324 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-9acbbc7e-2c39-487f-bb76-9401ef747e68 req-869acf6e-7597-4c5b-a91b-4cff1224b375 service nova] Creating event network-changed:318657ab-0eb5-43d1-a2b3-f4217b82903a for instance 8934bb4a-3501-480f-a0af-6ff5156f83fb on nch2-master-1 Sep 27 10:51:23.789391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-9acbbc7e-2c39-487f-bb76-9401ef747e68 req-869acf6e-7597-4c5b-a91b-4cff1224b375 service nova] Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:51:23.800897 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-9acbbc7e-2c39-487f-bb76-9401ef747e68 req-869acf6e-7597-4c5b-a91b-4cff1224b375 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046804 Sep 27 10:51:23.801775 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3234/6939] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:51:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:51:24.359231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4fe54c10-a408-4f25-ba61-78ff0341f2b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:24.359812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4fe54c10-a408-4f25-ba61-78ff0341f2b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:24.367705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fe54c10-a408-4f25-ba61-78ff0341f2b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:24.368203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fe54c10-a408-4f25-ba61-78ff0341f2b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:24.368643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fe54c10-a408-4f25-ba61-78ff0341f2b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:24.369400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fe54c10-a408-4f25-ba61-78ff0341f2b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:24.369861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fe54c10-a408-4f25-ba61-78ff0341f2b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:24.370291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4fe54c10-a408-4f25-ba61-78ff0341f2b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:24.515798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4fe54c10-a408-4f25-ba61-78ff0341f2b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:24.525305 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4fe54c10-a408-4f25-ba61-78ff0341f2b5 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1725 microversion: 2.1 time: 0.170068 Sep 27 10:51:24.525892 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3706/6940] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:24 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1725 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:25.550810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-230067ee-44f2-48b0-96ff-a621731b9027 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:25.552173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-230067ee-44f2-48b0-96ff-a621731b9027 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:25.558997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-230067ee-44f2-48b0-96ff-a621731b9027 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:25.559503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-230067ee-44f2-48b0-96ff-a621731b9027 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:25.559952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-230067ee-44f2-48b0-96ff-a621731b9027 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:25.560679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-230067ee-44f2-48b0-96ff-a621731b9027 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:25.561176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-230067ee-44f2-48b0-96ff-a621731b9027 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:25.561619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-230067ee-44f2-48b0-96ff-a621731b9027 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:25.778353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-230067ee-44f2-48b0-96ff-a621731b9027 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:25.787360 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-230067ee-44f2-48b0-96ff-a621731b9027 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1725 microversion: 2.1 time: 0.241815 Sep 27 10:51:25.788302 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3235/6941] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:25 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1725 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:26.812466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-42c71780-0d69-42b7-82fd-215ca9460449 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:26.813152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-42c71780-0d69-42b7-82fd-215ca9460449 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:26.820986 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42c71780-0d69-42b7-82fd-215ca9460449 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:26.821534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42c71780-0d69-42b7-82fd-215ca9460449 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:26.821983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42c71780-0d69-42b7-82fd-215ca9460449 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:26.822748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42c71780-0d69-42b7-82fd-215ca9460449 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:26.823244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42c71780-0d69-42b7-82fd-215ca9460449 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:26.823710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-42c71780-0d69-42b7-82fd-215ca9460449 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:26.967810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-42c71780-0d69-42b7-82fd-215ca9460449 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:26.980242 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-42c71780-0d69-42b7-82fd-215ca9460449 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1725 microversion: 2.1 time: 0.171699 Sep 27 10:51:26.981148 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3707/6942] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:26 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1725 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:28.005290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1979a46b-2985-4e56-baa7-97e7f4b6b320 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:28.005865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1979a46b-2985-4e56-baa7-97e7f4b6b320 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:28.013630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1979a46b-2985-4e56-baa7-97e7f4b6b320 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:28.014137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1979a46b-2985-4e56-baa7-97e7f4b6b320 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:28.014593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1979a46b-2985-4e56-baa7-97e7f4b6b320 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:28.015321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1979a46b-2985-4e56-baa7-97e7f4b6b320 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:28.015796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1979a46b-2985-4e56-baa7-97e7f4b6b320 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:28.016243 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1979a46b-2985-4e56-baa7-97e7f4b6b320 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:28.153395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1979a46b-2985-4e56-baa7-97e7f4b6b320 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:28.162419 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1979a46b-2985-4e56-baa7-97e7f4b6b320 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1725 microversion: 2.1 time: 0.161347 Sep 27 10:51:28.163364 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3236/6943] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:27 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1725 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:29.191448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0e590a8-bfff-4360-8703-5a725d373433 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:29.192130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f0e590a8-bfff-4360-8703-5a725d373433 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:29.203656 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0e590a8-bfff-4360-8703-5a725d373433 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:29.204452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0e590a8-bfff-4360-8703-5a725d373433 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:29.205171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0e590a8-bfff-4360-8703-5a725d373433 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:29.206270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0e590a8-bfff-4360-8703-5a725d373433 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:29.206934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0e590a8-bfff-4360-8703-5a725d373433 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:29.207654 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0e590a8-bfff-4360-8703-5a725d373433 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:29.352081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f0e590a8-bfff-4360-8703-5a725d373433 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:29.361153 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f0e590a8-bfff-4360-8703-5a725d373433 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1722 microversion: 2.1 time: 0.176450 Sep 27 10:51:29.361732 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3708/6944] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:29 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1722 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:29.385028 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Action: 'action', calling method: >, body: {"revertResize": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:29.385606 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:29.393124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:29.393633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:29.394092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:29.394819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:29.395301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:29.395775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:29.437329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:29.437828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:29.438283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:29.439033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:29.439498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:29.439940 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:29.461103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:51:29.466456 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Getting quotas for user 396fe9ce0f7542c382b01d1f5779c13f and project 32845fb8bd064f55bb99d01e5aeff6f3. Resources: {'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:51:29.552414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=4,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:29.581078 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d1f52421-44c1-4a35-9d89-eb2219269a1e tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "POST /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb/action" status: 202 len: 0 microversion: 2.1 time: 0.200174 Sep 27 10:51:29.581763 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3237/6945] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:51:29 2022] POST /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb/action => generated 0 bytes in 202 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:51:29.590562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-803da069-8905-4990-a651-27b8bb81c877 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:29.591264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-803da069-8905-4990-a651-27b8bb81c877 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:29.597879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803da069-8905-4990-a651-27b8bb81c877 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:29.598456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803da069-8905-4990-a651-27b8bb81c877 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:29.599090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803da069-8905-4990-a651-27b8bb81c877 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:29.600057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803da069-8905-4990-a651-27b8bb81c877 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:29.600979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803da069-8905-4990-a651-27b8bb81c877 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:29.601638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-803da069-8905-4990-a651-27b8bb81c877 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:29.742339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-803da069-8905-4990-a651-27b8bb81c877 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:29.751636 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-803da069-8905-4990-a651-27b8bb81c877 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1721 microversion: 2.1 time: 0.165383 Sep 27 10:51:29.752220 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3709/6946] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:29 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1721 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:51:30.483854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1f59d105-64b8-425c-a2f5-e0c49820dd35 req-34e9df1f-5823-4219-9be0-7584c23ce47f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8934bb4a-3501-480f-a0af-6ff5156f83fb", "name": "network-vif-plugged", "status": "completed", "tag": "318657ab-0eb5-43d1-a2b3-f4217b82903a"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:30.495080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1f59d105-64b8-425c-a2f5-e0c49820dd35 req-34e9df1f-5823-4219-9be0-7584c23ce47f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:30.495555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1f59d105-64b8-425c-a2f5-e0c49820dd35 req-34e9df1f-5823-4219-9be0-7584c23ce47f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:30.496175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1f59d105-64b8-425c-a2f5-e0c49820dd35 req-34e9df1f-5823-4219-9be0-7584c23ce47f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:30.507633 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1f59d105-64b8-425c-a2f5-e0c49820dd35 req-34e9df1f-5823-4219-9be0-7584c23ce47f service nova] Creating event network-vif-plugged:318657ab-0eb5-43d1-a2b3-f4217b82903a for instance 8934bb4a-3501-480f-a0af-6ff5156f83fb on nch2-master-1 Sep 27 10:51:30.513106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-1f59d105-64b8-425c-a2f5-e0c49820dd35 req-34e9df1f-5823-4219-9be0-7584c23ce47f service nova] Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:51:30.523974 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1f59d105-64b8-425c-a2f5-e0c49820dd35 req-34e9df1f-5823-4219-9be0-7584c23ce47f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045485 Sep 27 10:51:30.524586 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3238/6947] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:51:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:51:30.779123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cf8de582-0ffb-4254-97fb-2a2d9071bfc9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:30.779787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cf8de582-0ffb-4254-97fb-2a2d9071bfc9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:30.786765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8de582-0ffb-4254-97fb-2a2d9071bfc9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:30.787256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8de582-0ffb-4254-97fb-2a2d9071bfc9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:30.788043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8de582-0ffb-4254-97fb-2a2d9071bfc9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:30.788782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8de582-0ffb-4254-97fb-2a2d9071bfc9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:30.789241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8de582-0ffb-4254-97fb-2a2d9071bfc9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:30.789666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf8de582-0ffb-4254-97fb-2a2d9071bfc9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:30.940974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cf8de582-0ffb-4254-97fb-2a2d9071bfc9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:30.949761 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cf8de582-0ffb-4254-97fb-2a2d9071bfc9 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1721 microversion: 2.1 time: 0.176644 Sep 27 10:51:30.950355 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3710/6948] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:30 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1721 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:31.979140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90d22a23-e4ed-4d9a-b1a4-10678a24c5cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:31.979942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-90d22a23-e4ed-4d9a-b1a4-10678a24c5cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:31.987299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90d22a23-e4ed-4d9a-b1a4-10678a24c5cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:31.987798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90d22a23-e4ed-4d9a-b1a4-10678a24c5cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:31.988250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90d22a23-e4ed-4d9a-b1a4-10678a24c5cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:31.989016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90d22a23-e4ed-4d9a-b1a4-10678a24c5cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:31.989486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90d22a23-e4ed-4d9a-b1a4-10678a24c5cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:31.989929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90d22a23-e4ed-4d9a-b1a4-10678a24c5cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:32.135632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-90d22a23-e4ed-4d9a-b1a4-10678a24c5cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:32.144799 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-90d22a23-e4ed-4d9a-b1a4-10678a24c5cd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1721 microversion: 2.1 time: 0.173298 Sep 27 10:51:32.145417 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3239/6949] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:31 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1721 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:33.172029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b546de71-4fa4-4a4d-9e36-654ea51c34ca tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:33.176199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b546de71-4fa4-4a4d-9e36-654ea51c34ca tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:33.181043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=35 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:33.188192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b546de71-4fa4-4a4d-9e36-654ea51c34ca tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:33.188977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b546de71-4fa4-4a4d-9e36-654ea51c34ca tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:33.189242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b546de71-4fa4-4a4d-9e36-654ea51c34ca tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:33.190073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b546de71-4fa4-4a4d-9e36-654ea51c34ca tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:33.190592 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b546de71-4fa4-4a4d-9e36-654ea51c34ca tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:33.191128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b546de71-4fa4-4a4d-9e36-654ea51c34ca tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:33.196302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:33.335048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b546de71-4fa4-4a4d-9e36-654ea51c34ca tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:33.343970 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b546de71-4fa4-4a4d-9e36-654ea51c34ca tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1721 microversion: 2.1 time: 0.178048 Sep 27 10:51:33.344650 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3711/6950] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:33 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1721 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:33.622417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-9b853cb4-a6c1-4b21-bd82-f60b4710f46a req-7f009ad3-d42e-436f-9919-d8af1097fe0a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8934bb4a-3501-480f-a0af-6ff5156f83fb", "name": "network-vif-unplugged", "status": "completed", "tag": "318657ab-0eb5-43d1-a2b3-f4217b82903a"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:33.630139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:33.634611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9b853cb4-a6c1-4b21-bd82-f60b4710f46a req-7f009ad3-d42e-436f-9919-d8af1097fe0a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:33.635373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9b853cb4-a6c1-4b21-bd82-f60b4710f46a req-7f009ad3-d42e-436f-9919-d8af1097fe0a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:33.635867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-9b853cb4-a6c1-4b21-bd82-f60b4710f46a req-7f009ad3-d42e-436f-9919-d8af1097fe0a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:33.660422 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-9b853cb4-a6c1-4b21-bd82-f60b4710f46a req-7f009ad3-d42e-436f-9919-d8af1097fe0a service nova] Creating event network-vif-unplugged:318657ab-0eb5-43d1-a2b3-f4217b82903a for instance 8934bb4a-3501-480f-a0af-6ff5156f83fb on nch1-master-1 Sep 27 10:51:33.672081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-9b853cb4-a6c1-4b21-bd82-f60b4710f46a req-7f009ad3-d42e-436f-9919-d8af1097fe0a service nova] Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:51:33.683030 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-9b853cb4-a6c1-4b21-bd82-f60b4710f46a req-7f009ad3-d42e-436f-9919-d8af1097fe0a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.064464 Sep 27 10:51:33.683867 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3240/6951] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:51:33 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:51:34.369116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0d41a689-37be-4041-8ef6-5d723d045c80 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:34.370068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0d41a689-37be-4041-8ef6-5d723d045c80 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:34.378218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d41a689-37be-4041-8ef6-5d723d045c80 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:34.378745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d41a689-37be-4041-8ef6-5d723d045c80 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:34.379186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d41a689-37be-4041-8ef6-5d723d045c80 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:34.379891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d41a689-37be-4041-8ef6-5d723d045c80 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:34.380352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d41a689-37be-4041-8ef6-5d723d045c80 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:34.380815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0d41a689-37be-4041-8ef6-5d723d045c80 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:34.515408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0d41a689-37be-4041-8ef6-5d723d045c80 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:34.525301 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0d41a689-37be-4041-8ef6-5d723d045c80 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1721 microversion: 2.1 time: 0.160886 Sep 27 10:51:34.526100 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3712/6952] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:34 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1721 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:35.551025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-40226da2-63a4-41c3-a03f-c97901190dbc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:35.551986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-40226da2-63a4-41c3-a03f-c97901190dbc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:35.558663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40226da2-63a4-41c3-a03f-c97901190dbc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:35.559203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40226da2-63a4-41c3-a03f-c97901190dbc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:35.559663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40226da2-63a4-41c3-a03f-c97901190dbc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:35.560394 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40226da2-63a4-41c3-a03f-c97901190dbc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:35.560960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40226da2-63a4-41c3-a03f-c97901190dbc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:35.561414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-40226da2-63a4-41c3-a03f-c97901190dbc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:35.699280 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-40226da2-63a4-41c3-a03f-c97901190dbc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:35.708442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-5e2c9057-1fbc-4e26-8327-e474472f8a55 req-baeadf3b-4f05-440d-ae64-24651f9b4871 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "8934bb4a-3501-480f-a0af-6ff5156f83fb", "tag": "318657ab-0eb5-43d1-a2b3-f4217b82903a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:35.710158 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-40226da2-63a4-41c3-a03f-c97901190dbc tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1721 microversion: 2.1 time: 0.164468 Sep 27 10:51:35.710662 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3241/6953] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:35 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1721 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:35.720842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5e2c9057-1fbc-4e26-8327-e474472f8a55 req-baeadf3b-4f05-440d-ae64-24651f9b4871 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:35.721340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5e2c9057-1fbc-4e26-8327-e474472f8a55 req-baeadf3b-4f05-440d-ae64-24651f9b4871 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:35.721781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-5e2c9057-1fbc-4e26-8327-e474472f8a55 req-baeadf3b-4f05-440d-ae64-24651f9b4871 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:35.734308 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-5e2c9057-1fbc-4e26-8327-e474472f8a55 req-baeadf3b-4f05-440d-ae64-24651f9b4871 service nova] Creating event network-changed:318657ab-0eb5-43d1-a2b3-f4217b82903a for instance 8934bb4a-3501-480f-a0af-6ff5156f83fb on nch1-master-1 Sep 27 10:51:35.739914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [req-5e2c9057-1fbc-4e26-8327-e474472f8a55 req-baeadf3b-4f05-440d-ae64-24651f9b4871 service nova] Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145968) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 10:51:35.751681 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-5e2c9057-1fbc-4e26-8327-e474472f8a55 req-baeadf3b-4f05-440d-ae64-24651f9b4871 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047411 Sep 27 10:51:35.752332 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3713/6954] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:51:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:51:36.736793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-defd4db6-a225-45d6-ab73-b3d504523285 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:36.737518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-defd4db6-a225-45d6-ab73-b3d504523285 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:36.745471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-defd4db6-a225-45d6-ab73-b3d504523285 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:36.745983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-defd4db6-a225-45d6-ab73-b3d504523285 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:36.746446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-defd4db6-a225-45d6-ab73-b3d504523285 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:36.747206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-defd4db6-a225-45d6-ab73-b3d504523285 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:36.747748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-defd4db6-a225-45d6-ab73-b3d504523285 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:36.748326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-defd4db6-a225-45d6-ab73-b3d504523285 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:37.167548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-defd4db6-a225-45d6-ab73-b3d504523285 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:37.176476 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-defd4db6-a225-45d6-ab73-b3d504523285 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1721 microversion: 2.1 time: 0.444622 Sep 27 10:51:37.177153 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3242/6955] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:36 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1721 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:38.210137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-10cece07-af39-4c6a-a84d-d6d96d37aa0a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:38.210846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-10cece07-af39-4c6a-a84d-d6d96d37aa0a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:38.217927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10cece07-af39-4c6a-a84d-d6d96d37aa0a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:38.218241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10cece07-af39-4c6a-a84d-d6d96d37aa0a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:38.218677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10cece07-af39-4c6a-a84d-d6d96d37aa0a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:38.219386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10cece07-af39-4c6a-a84d-d6d96d37aa0a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:38.219851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10cece07-af39-4c6a-a84d-d6d96d37aa0a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:38.220280 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-10cece07-af39-4c6a-a84d-d6d96d37aa0a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:38.368037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-10cece07-af39-4c6a-a84d-d6d96d37aa0a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:38.377870 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-10cece07-af39-4c6a-a84d-d6d96d37aa0a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1714 microversion: 2.1 time: 0.174370 Sep 27 10:51:38.378456 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3714/6956] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:38 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1714 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:38.404464 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0602367d-44b0-4b26-a4e6-e49da93fa901 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:38.405523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0602367d-44b0-4b26-a4e6-e49da93fa901 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:38.414008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0602367d-44b0-4b26-a4e6-e49da93fa901 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:38.414516 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0602367d-44b0-4b26-a4e6-e49da93fa901 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:38.414968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0602367d-44b0-4b26-a4e6-e49da93fa901 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:38.415686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0602367d-44b0-4b26-a4e6-e49da93fa901 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:38.416158 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0602367d-44b0-4b26-a4e6-e49da93fa901 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:38.416599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0602367d-44b0-4b26-a4e6-e49da93fa901 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:38.559929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0602367d-44b0-4b26-a4e6-e49da93fa901 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:38.568897 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0602367d-44b0-4b26-a4e6-e49da93fa901 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1714 microversion: 2.1 time: 0.171225 Sep 27 10:51:38.569498 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3243/6957] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:38 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1714 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:38.593382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4f915b76-626b-42f7-91d0-96decd99d709 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:38.594389 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4f915b76-626b-42f7-91d0-96decd99d709 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:38.603078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f915b76-626b-42f7-91d0-96decd99d709 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:38.603577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f915b76-626b-42f7-91d0-96decd99d709 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:38.604022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f915b76-626b-42f7-91d0-96decd99d709 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:38.604763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f915b76-626b-42f7-91d0-96decd99d709 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:38.605223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f915b76-626b-42f7-91d0-96decd99d709 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:38.605829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4f915b76-626b-42f7-91d0-96decd99d709 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:38.744413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4f915b76-626b-42f7-91d0-96decd99d709 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:38.753184 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4f915b76-626b-42f7-91d0-96decd99d709 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1714 microversion: 2.1 time: 0.165000 Sep 27 10:51:38.754015 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3715/6958] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:38 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1714 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:38.776845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7c64898-1e24-4169-8f17-82fdc34efbb4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:38.777378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7c64898-1e24-4169-8f17-82fdc34efbb4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:38.784291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7c64898-1e24-4169-8f17-82fdc34efbb4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:38.784855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7c64898-1e24-4169-8f17-82fdc34efbb4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:38.785298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7c64898-1e24-4169-8f17-82fdc34efbb4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:38.786026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7c64898-1e24-4169-8f17-82fdc34efbb4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:38.786559 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7c64898-1e24-4169-8f17-82fdc34efbb4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:38.787012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7c64898-1e24-4169-8f17-82fdc34efbb4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:38.816715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7c64898-1e24-4169-8f17-82fdc34efbb4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:51:38.910054 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c7c64898-1e24-4169-8f17-82fdc34efbb4 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "DELETE /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 204 len: 0 microversion: 2.1 time: 0.136160 Sep 27 10:51:38.911071 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3244/6959] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:51:38 2022] DELETE /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 0 bytes in 139 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:51:38.921859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5de503eb-14a5-4635-90fd-c3bb7d66f973 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:38.922787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5de503eb-14a5-4635-90fd-c3bb7d66f973 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:38.929291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5de503eb-14a5-4635-90fd-c3bb7d66f973 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:38.929774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5de503eb-14a5-4635-90fd-c3bb7d66f973 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:38.930213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5de503eb-14a5-4635-90fd-c3bb7d66f973 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:38.930915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5de503eb-14a5-4635-90fd-c3bb7d66f973 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:38.931377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5de503eb-14a5-4635-90fd-c3bb7d66f973 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:38.931870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5de503eb-14a5-4635-90fd-c3bb7d66f973 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:39.070129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5de503eb-14a5-4635-90fd-c3bb7d66f973 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:39.078724 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5de503eb-14a5-4635-90fd-c3bb7d66f973 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1720 microversion: 2.1 time: 0.162514 Sep 27 10:51:39.079362 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3716/6960] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:38 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1720 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:40.104756 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6fff7ab1-6360-4588-8a93-232fca33e4bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:40.105470 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6fff7ab1-6360-4588-8a93-232fca33e4bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:40.108326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:40.114144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fff7ab1-6360-4588-8a93-232fca33e4bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:40.114652 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fff7ab1-6360-4588-8a93-232fca33e4bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:40.115112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fff7ab1-6360-4588-8a93-232fca33e4bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:40.115839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fff7ab1-6360-4588-8a93-232fca33e4bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:40.116315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fff7ab1-6360-4588-8a93-232fca33e4bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:40.116796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6fff7ab1-6360-4588-8a93-232fca33e4bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:40.254638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6fff7ab1-6360-4588-8a93-232fca33e4bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:40.263528 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6fff7ab1-6360-4588-8a93-232fca33e4bd tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1720 microversion: 2.1 time: 0.163527 Sep 27 10:51:40.264095 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3245/6961] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:40 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1720 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:40.705316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-651123c4-f9a2-498c-99e8-25b0c8402b31 req-2e041c52-e3a7-49be-99cb-4fdd4ac13780 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8934bb4a-3501-480f-a0af-6ff5156f83fb", "name": "network-vif-plugged", "status": "completed", "tag": "318657ab-0eb5-43d1-a2b3-f4217b82903a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:40.716702 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-651123c4-f9a2-498c-99e8-25b0c8402b31 req-2e041c52-e3a7-49be-99cb-4fdd4ac13780 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:40.717273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-651123c4-f9a2-498c-99e8-25b0c8402b31 req-2e041c52-e3a7-49be-99cb-4fdd4ac13780 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:40.717785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-651123c4-f9a2-498c-99e8-25b0c8402b31 req-2e041c52-e3a7-49be-99cb-4fdd4ac13780 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:40.730169 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-651123c4-f9a2-498c-99e8-25b0c8402b31 req-2e041c52-e3a7-49be-99cb-4fdd4ac13780 service nova] Creating event network-vif-plugged:318657ab-0eb5-43d1-a2b3-f4217b82903a for instance 8934bb4a-3501-480f-a0af-6ff5156f83fb on nch1-master-1 Sep 27 10:51:40.735383 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-651123c4-f9a2-498c-99e8-25b0c8402b31 req-2e041c52-e3a7-49be-99cb-4fdd4ac13780 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033892 Sep 27 10:51:40.736084 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3717/6962] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:51:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:51:41.287902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eae9923e-3809-44a3-8465-fa3e348f2485 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:41.289407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eae9923e-3809-44a3-8465-fa3e348f2485 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:41.296603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eae9923e-3809-44a3-8465-fa3e348f2485 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:41.297140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eae9923e-3809-44a3-8465-fa3e348f2485 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:41.297601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eae9923e-3809-44a3-8465-fa3e348f2485 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:41.298339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eae9923e-3809-44a3-8465-fa3e348f2485 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:41.298964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eae9923e-3809-44a3-8465-fa3e348f2485 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:41.299408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eae9923e-3809-44a3-8465-fa3e348f2485 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:41.395890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eae9923e-3809-44a3-8465-fa3e348f2485 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '396fe9ce0f7542c382b01d1f5779c13f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '32845fb8bd064f55bb99d01e5aeff6f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:41.404563 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eae9923e-3809-44a3-8465-fa3e348f2485 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 200 len: 1653 microversion: 2.1 time: 0.120988 Sep 27 10:51:41.405242 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3246/6963] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:41 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 1653 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:41.475314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-f6ecd6e4-0499-4777-bd71-6b12dfe9cbf9 req-aa358d64-b239-48ce-a7cc-5e91640f9c7d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "8934bb4a-3501-480f-a0af-6ff5156f83fb", "name": "network-vif-deleted", "tag": "318657ab-0eb5-43d1-a2b3-f4217b82903a"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:41.488145 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f6ecd6e4-0499-4777-bd71-6b12dfe9cbf9 req-aa358d64-b239-48ce-a7cc-5e91640f9c7d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:41.488919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f6ecd6e4-0499-4777-bd71-6b12dfe9cbf9 req-aa358d64-b239-48ce-a7cc-5e91640f9c7d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:41.489377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-f6ecd6e4-0499-4777-bd71-6b12dfe9cbf9 req-aa358d64-b239-48ce-a7cc-5e91640f9c7d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:41.502581 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-f6ecd6e4-0499-4777-bd71-6b12dfe9cbf9 req-aa358d64-b239-48ce-a7cc-5e91640f9c7d service nova] Creating event network-vif-deleted:318657ab-0eb5-43d1-a2b3-f4217b82903a for instance 8934bb4a-3501-480f-a0af-6ff5156f83fb on nch1-master-1 Sep 27 10:51:41.508760 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-f6ecd6e4-0499-4777-bd71-6b12dfe9cbf9 req-aa358d64-b239-48ce-a7cc-5e91640f9c7d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038191 Sep 27 10:51:41.509387 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3718/6964] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:51:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:51:42.429364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:42.430129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:42.437438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:42.438144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:42.438548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:42.439284 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:42.439763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:42.440216 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:42.455097 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb HTTP/1.1 Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '8934bb4a-3501-480f-a0af-6ff5156f83fb'} Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:42.456880 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:42.461159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:42.461159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:42.461159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:42.461159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:51:42.461159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:51:42.461159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:42.461159 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:42.461159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:51:42.461159 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5a301038-f04a-4ce7-b4a9-b06f5255779a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 404 len: 111 microversion: 2.1 time: 0.033008 Sep 27 10:51:42.461159 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3247/6965] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:42 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:51:42.525112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:42.527435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:42.534836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:42.535331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:42.535776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:42.536479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:42.536968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:42.537724 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:42.550865 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb/os-volume_attachments HTTP/1.1 Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'server_id': '8934bb4a-3501-480f-a0af-6ff5156f83fb'} Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:51:42.552557 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-088719e3-9307-43a8-a10f-d564c40496e7 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.033461 Sep 27 10:51:42.557053 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3719/6966] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:51:42 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb/os-volume_attachments => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:51:43.774353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:43.774911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:43.782938 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.783437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.783889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.784599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.785110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.785557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.800036 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb HTTP/1.1 Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '8934bb4a-3501-480f-a0af-6ff5156f83fb'} Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:51:43.801696 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cc07071d-de16-470f-ac2d-42c953749518 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "DELETE /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 404 len: 111 microversion: 2.1 time: 0.032973 Sep 27 10:51:43.806111 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3248/6967] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:51:43 2022] DELETE /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:51:43.812064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:43.812692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 8934bb4a-3501-480f-a0af-6ff5156f83fb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:43.820029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.820526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.821007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.821719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.822222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.822655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.837548 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb HTTP/1.1 Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '8934bb4a-3501-480f-a0af-6ff5156f83fb'} Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.839160 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.843305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.843305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.843305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.843305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:51:43.843305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:51:43.843305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. Sep 27 10:51:43.843305 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.843305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 8934bb4a-3501-480f-a0af-6ff5156f83fb could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:51:43.843305 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-15abceea-86f0-4758-a29b-d6cc44d9d1f2 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb" status: 404 len: 111 microversion: 2.1 time: 0.032541 Sep 27 10:51:43.843305 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3720/6968] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:43 2022] GET /compute/v2.1/servers/8934bb4a-3501-480f-a0af-6ff5156f83fb => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:51:43.849497 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:43.850008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:43.857888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.858441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.858891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.859599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.860159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.860599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.874436 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec HTTP/1.1 Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1577dded-7b7f-4041-9666-da60c77e78ec'} Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:51:43.875963 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-839993a2-f406-4f17-af4f-963391f5151a tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 404 len: 111 microversion: 2.1 time: 0.031941 Sep 27 10:51:43.879936 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3249/6969] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:51:43 2022] DELETE /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:51:43.888723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:43.889570 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 1577dded-7b7f-4041-9666-da60c77e78ec] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:43.897248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.897738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.898183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.898885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.899347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.899776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.914745 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec HTTP/1.1 Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '1577dded-7b7f-4041-9666-da60c77e78ec'} Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.916299 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.921356 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.921356 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.921356 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.921356 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:51:43.921356 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:51:43.921356 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. Sep 27 10:51:43.921356 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.921356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 1577dded-7b7f-4041-9666-da60c77e78ec could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:51:43.921356 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a50c7899-24b1-4fd2-b70f-4bb8fe540990 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec" status: 404 len: 111 microversion: 2.1 time: 0.035237 Sep 27 10:51:43.921356 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3721/6970] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:43 2022] GET /compute/v2.1/servers/1577dded-7b7f-4041-9666-da60c77e78ec => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:51:43.926619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:43.927140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:43.933765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.934265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.934717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.935428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.936016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.936463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.951204 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 HTTP/1.1 Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '4b7833f5-ef39-4414-92b8-0eaaa26eb4a7'} Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:51:43.952808 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d89cdf7d-5277-47ff-9a80-859b7a6059ae tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 404 len: 111 microversion: 2.1 time: 0.031806 Sep 27 10:51:43.957888 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3250/6971] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:51:43 2022] DELETE /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:51:43.962988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:43.963615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:43.969886 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.970377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.970820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.971558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:43.972030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:43.972466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:43.986541 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 HTTP/1.1 Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '4b7833f5-ef39-4414-92b8-0eaaa26eb4a7'} Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:43.988095 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.992310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:43.992310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:43.992310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:43.992310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:51:43.992310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:51:43.992310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. Sep 27 10:51:43.992310 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:43.992310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:51:43.992310 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d9fe177a-db52-45aa-a5b5-8ee4321524eb tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7" status: 404 len: 111 microversion: 2.1 time: 0.030447 Sep 27 10:51:43.992310 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3722/6972] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:43 2022] GET /compute/v2.1/servers/4b7833f5-ef39-4414-92b8-0eaaa26eb4a7 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:51:43.998488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:43.999008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:44.005610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:44.006107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:44.006556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:44.007266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:44.007755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:44.008195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:44.022497 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 HTTP/1.1 Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409'} Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:51:44.024074 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d5808f91-e197-4c12-a295-37d84a81fc9d tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "DELETE /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 404 len: 111 microversion: 2.1 time: 0.031065 Sep 27 10:51:44.028798 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3251/6973] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:51:43 2022] DELETE /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:51:44.034087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:44.034712 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] [instance: 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:44.041260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:44.041753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:44.042201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:44.042905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:44.043369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:44.043805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:44.057683 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] HTTP exception thrown: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Request method failure captured: Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 HTTP/1.1 Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtQXuaLQnYK5wpgIo_rtAZgcZyFsazqkpH5SPKA5c8WnDsFzMTQhIOv6y_1p1OUXaqsAqj63G1L7XFdj5YNifmaV7M8Or-iRIweJDqmNFn48pASe2LRrrsOdsfWg0b5KHmbTtf0bfVVY8PtByZkDESV3-_k77Jb4PdG8xQhNJg6PlC0 Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/32845fb8bd064f55bb99d01e5aeff6f3"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_32845fb8bd064f55bb99d01e5aeff6f3"}]}] Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 32845fb8bd064f55bb99d01e5aeff6f3 Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServerActionsTestJSON-429271553 Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 396fe9ce0f7542c382b01d1f5779c13f Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServerActionsTestJSON-429271553-project Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409'} Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:51:44.059281 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:44.063976 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:51:44.063976 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:51:44.063976 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:51:44.063976 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:51:44.063976 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:51:44.063976 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. Sep 27 10:51:44.063976 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:51:44.063976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Returning 404 to user: Instance 05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:51:44.063976 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0c002f79-52d9-4c74-b8e8-2a827da439c3 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409" status: 404 len: 111 microversion: 2.1 time: 0.030540 Sep 27 10:51:44.063976 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3723/6974] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:44 2022] GET /compute/v2.1/servers/05cd6b9f-7ed8-45d3-b6f8-7d5d28d48409 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:51:44.069443 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5eb80bac-ea5b-49e6-a1de-f86f5ce79e31 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] Calling method '.version_select at 0x7f7f635df820>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:44.095762 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5eb80bac-ea5b-49e6-a1de-f86f5ce79e31 tempest-ServerActionsTestJSON-429271553 tempest-ServerActionsTestJSON-429271553-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-107890057" status: 202 len: 0 microversion: 2.1 time: 0.030127 Sep 27 10:51:44.096356 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3252/6975] 10.222.0.26 () {58 vars in 1218 bytes} [Tue Sep 27 10:51:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-107890057 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:51:53.739489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1167243262", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "29934d6a-9abf-4327-b7c9-50388a209ed4"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:53.794816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] validate_networks() for [('29934d6a-9abf-4327-b7c9-50388a209ed4', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:51:54.049555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=47 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:54.253763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:51:54.254334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:51:54.254817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:51:54.255223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:51:54.255634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:51:54.265219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:54.265703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:54.266143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:54.266870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:54.267324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:54.267746 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:54.285977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Getting quotas for project b4eda9ec9a0a4216a8b70d63470e6838. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:51:54.292350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Getting quotas for user b9b3c7e824e74cbba45639dfed007e4d and project b4eda9ec9a0a4216a8b70d63470e6838. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:51:54.306744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:51:54.309213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:51:54.361027 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fef616ba-f760-4db3-a7c0-51356e5923cb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.748037 Sep 27 10:51:54.367779 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3724/6976] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:51:53 2022] POST /compute/v2.1/servers => generated 380 bytes in 756 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:51:54.371038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=69,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:54.371822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=31,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:51:54.377899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7d7a54d3-9d92-4f7f-8891-41242fc7fa5a tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:54.378634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7d7a54d3-9d92-4f7f-8891-41242fc7fa5a tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:54.440627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7d7a54d3-9d92-4f7f-8891-41242fc7fa5a tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:54.448109 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7d7a54d3-9d92-4f7f-8891-41242fc7fa5a tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1217 microversion: 2.1 time: 0.079094 Sep 27 10:51:54.448759 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3253/6977] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:54 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1217 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:55.472829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-47d9e3f9-c7f0-475d-bcc9-36faa59dc02e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:55.473595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-47d9e3f9-c7f0-475d-bcc9-36faa59dc02e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:55.482939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d9e3f9-c7f0-475d-bcc9-36faa59dc02e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:55.483555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d9e3f9-c7f0-475d-bcc9-36faa59dc02e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:55.484228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d9e3f9-c7f0-475d-bcc9-36faa59dc02e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:55.485045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d9e3f9-c7f0-475d-bcc9-36faa59dc02e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:55.485517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d9e3f9-c7f0-475d-bcc9-36faa59dc02e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:55.485951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-47d9e3f9-c7f0-475d-bcc9-36faa59dc02e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:55.583983 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-47d9e3f9-c7f0-475d-bcc9-36faa59dc02e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:55.595918 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-47d9e3f9-c7f0-475d-bcc9-36faa59dc02e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1277 microversion: 2.1 time: 0.127923 Sep 27 10:51:55.607692 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3725/6978] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:55 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1277 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:56.625947 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dca187f5-8761-43ba-bfd6-fca356405f2d tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:56.627300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dca187f5-8761-43ba-bfd6-fca356405f2d tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:56.635468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dca187f5-8761-43ba-bfd6-fca356405f2d tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:56.636064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dca187f5-8761-43ba-bfd6-fca356405f2d tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:56.636591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dca187f5-8761-43ba-bfd6-fca356405f2d tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:56.637423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dca187f5-8761-43ba-bfd6-fca356405f2d tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:56.637962 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dca187f5-8761-43ba-bfd6-fca356405f2d tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:56.638488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dca187f5-8761-43ba-bfd6-fca356405f2d tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:56.895304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dca187f5-8761-43ba-bfd6-fca356405f2d tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:56.904878 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dca187f5-8761-43ba-bfd6-fca356405f2d tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1317 microversion: 2.1 time: 0.284626 Sep 27 10:51:56.905502 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3254/6979] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:56 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1317 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:57.169090 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-bb699efe-d024-42c2-b013-2530a7c24a4d req-c395967f-3550-4d0a-8f26-2f1871606130 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "34c36fdc-0be7-4b70-b32d-cd0d45d0381c", "tag": "0867c694-80a2-4a22-86cd-045c6e37cde7"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:51:57.182471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bb699efe-d024-42c2-b013-2530a7c24a4d req-c395967f-3550-4d0a-8f26-2f1871606130 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:57.182957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bb699efe-d024-42c2-b013-2530a7c24a4d req-c395967f-3550-4d0a-8f26-2f1871606130 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:57.183590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bb699efe-d024-42c2-b013-2530a7c24a4d req-c395967f-3550-4d0a-8f26-2f1871606130 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:57.221599 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-bb699efe-d024-42c2-b013-2530a7c24a4d req-c395967f-3550-4d0a-8f26-2f1871606130 service nova] Creating event network-changed:0867c694-80a2-4a22-86cd-045c6e37cde7 for instance 34c36fdc-0be7-4b70-b32d-cd0d45d0381c on nch2-master-1 Sep 27 10:51:57.228758 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-bb699efe-d024-42c2-b013-2530a7c24a4d req-c395967f-3550-4d0a-8f26-2f1871606130 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063059 Sep 27 10:51:57.230196 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3726/6980] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:51:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:51:57.931277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a2bcdb8d-e952-4049-a2ef-f671ef317397 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:57.932325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a2bcdb8d-e952-4049-a2ef-f671ef317397 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:57.941254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2bcdb8d-e952-4049-a2ef-f671ef317397 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:57.941787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2bcdb8d-e952-4049-a2ef-f671ef317397 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:57.942208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2bcdb8d-e952-4049-a2ef-f671ef317397 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:57.942961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2bcdb8d-e952-4049-a2ef-f671ef317397 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:57.943487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2bcdb8d-e952-4049-a2ef-f671ef317397 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:57.943932 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2bcdb8d-e952-4049-a2ef-f671ef317397 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:58.090665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a2bcdb8d-e952-4049-a2ef-f671ef317397 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:58.099420 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a2bcdb8d-e952-4049-a2ef-f671ef317397 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1317 microversion: 2.1 time: 0.173411 Sep 27 10:51:58.100005 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3255/6981] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:57 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1317 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:51:59.133124 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-72b11a7f-0221-4f77-8122-48461e2a7059 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:51:59.134279 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-72b11a7f-0221-4f77-8122-48461e2a7059 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:51:59.143593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72b11a7f-0221-4f77-8122-48461e2a7059 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:59.144087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72b11a7f-0221-4f77-8122-48461e2a7059 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:59.144520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72b11a7f-0221-4f77-8122-48461e2a7059 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:59.145282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72b11a7f-0221-4f77-8122-48461e2a7059 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:51:59.145697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72b11a7f-0221-4f77-8122-48461e2a7059 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:51:59.146102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-72b11a7f-0221-4f77-8122-48461e2a7059 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:51:59.289289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-72b11a7f-0221-4f77-8122-48461e2a7059 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:51:59.297633 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-72b11a7f-0221-4f77-8122-48461e2a7059 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1317 microversion: 2.1 time: 0.173362 Sep 27 10:51:59.298260 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3727/6982] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:51:59 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1317 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:00.321124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-88582be2-2d1d-40eb-80f9-4d5de7418dab tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:00.321763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-88582be2-2d1d-40eb-80f9-4d5de7418dab tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:00.329379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88582be2-2d1d-40eb-80f9-4d5de7418dab tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:00.329889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88582be2-2d1d-40eb-80f9-4d5de7418dab tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:00.330360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88582be2-2d1d-40eb-80f9-4d5de7418dab tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:00.331735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88582be2-2d1d-40eb-80f9-4d5de7418dab tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:00.332270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88582be2-2d1d-40eb-80f9-4d5de7418dab tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:00.332795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-88582be2-2d1d-40eb-80f9-4d5de7418dab tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:00.487621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-88582be2-2d1d-40eb-80f9-4d5de7418dab tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:00.495968 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-88582be2-2d1d-40eb-80f9-4d5de7418dab tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1317 microversion: 2.1 time: 0.178826 Sep 27 10:52:00.496529 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3256/6983] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:00 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1317 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:01.520845 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c308365a-6b28-4ff3-95e4-0012fe640d60 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:01.521482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c308365a-6b28-4ff3-95e4-0012fe640d60 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:01.529112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c308365a-6b28-4ff3-95e4-0012fe640d60 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:01.529615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c308365a-6b28-4ff3-95e4-0012fe640d60 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:01.530062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c308365a-6b28-4ff3-95e4-0012fe640d60 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:01.530788 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c308365a-6b28-4ff3-95e4-0012fe640d60 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:01.531328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c308365a-6b28-4ff3-95e4-0012fe640d60 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:01.531767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c308365a-6b28-4ff3-95e4-0012fe640d60 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:01.692473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c308365a-6b28-4ff3-95e4-0012fe640d60 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:01.701009 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c308365a-6b28-4ff3-95e4-0012fe640d60 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1317 microversion: 2.1 time: 0.184200 Sep 27 10:52:01.701679 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3728/6984] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:01 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1317 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:02.616101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-21941ef7-ade2-47e3-9dc6-4a2a02519fc1 req-2e73cb10-c943-49b5-b647-f969cfe6d1b3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34c36fdc-0be7-4b70-b32d-cd0d45d0381c", "name": "network-vif-plugged", "status": "completed", "tag": "0867c694-80a2-4a22-86cd-045c6e37cde7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:52:02.627957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-21941ef7-ade2-47e3-9dc6-4a2a02519fc1 req-2e73cb10-c943-49b5-b647-f969cfe6d1b3 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:02.628525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-21941ef7-ade2-47e3-9dc6-4a2a02519fc1 req-2e73cb10-c943-49b5-b647-f969cfe6d1b3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:02.629064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-21941ef7-ade2-47e3-9dc6-4a2a02519fc1 req-2e73cb10-c943-49b5-b647-f969cfe6d1b3 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:02.641575 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-21941ef7-ade2-47e3-9dc6-4a2a02519fc1 req-2e73cb10-c943-49b5-b647-f969cfe6d1b3 service nova] Creating event network-vif-plugged:0867c694-80a2-4a22-86cd-045c6e37cde7 for instance 34c36fdc-0be7-4b70-b32d-cd0d45d0381c on nch2-master-1 Sep 27 10:52:02.646922 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-21941ef7-ade2-47e3-9dc6-4a2a02519fc1 req-2e73cb10-c943-49b5-b647-f969cfe6d1b3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034625 Sep 27 10:52:02.647525 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3257/6985] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:52:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:52:02.728046 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4949ffd4-1b92-4a01-99de-7006d5289166 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:02.729412 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4949ffd4-1b92-4a01-99de-7006d5289166 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:02.737168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4949ffd4-1b92-4a01-99de-7006d5289166 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:02.737708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4949ffd4-1b92-4a01-99de-7006d5289166 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:02.738131 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4949ffd4-1b92-4a01-99de-7006d5289166 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:02.738843 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4949ffd4-1b92-4a01-99de-7006d5289166 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:02.739297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4949ffd4-1b92-4a01-99de-7006d5289166 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:02.739718 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4949ffd4-1b92-4a01-99de-7006d5289166 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:02.878600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4949ffd4-1b92-4a01-99de-7006d5289166 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:02.887413 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4949ffd4-1b92-4a01-99de-7006d5289166 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1317 microversion: 2.1 time: 0.163753 Sep 27 10:52:02.888090 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3729/6986] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:02 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1317 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:03.915141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b23ea914-76db-4e8b-b155-be628eb589aa tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:03.915994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b23ea914-76db-4e8b-b155-be628eb589aa tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:03.923953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b23ea914-76db-4e8b-b155-be628eb589aa tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:03.924555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b23ea914-76db-4e8b-b155-be628eb589aa tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:03.925139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b23ea914-76db-4e8b-b155-be628eb589aa tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:03.926006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b23ea914-76db-4e8b-b155-be628eb589aa tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:03.926564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b23ea914-76db-4e8b-b155-be628eb589aa tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:03.927095 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b23ea914-76db-4e8b-b155-be628eb589aa tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:04.089419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b23ea914-76db-4e8b-b155-be628eb589aa tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:04.102365 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b23ea914-76db-4e8b-b155-be628eb589aa tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1317 microversion: 2.1 time: 0.193835 Sep 27 10:52:04.103207 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3258/6987] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:03 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1317 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:05.132455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a746470f-fb2a-494a-ab18-4abbebbebbde tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:05.133422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a746470f-fb2a-494a-ab18-4abbebbebbde tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:05.137878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:05.139223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:05.140229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=49,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:05.146790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a746470f-fb2a-494a-ab18-4abbebbebbde tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:05.147289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a746470f-fb2a-494a-ab18-4abbebbebbde tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:05.147831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a746470f-fb2a-494a-ab18-4abbebbebbde tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:05.148566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a746470f-fb2a-494a-ab18-4abbebbebbde tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:05.149072 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a746470f-fb2a-494a-ab18-4abbebbebbde tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:05.149505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a746470f-fb2a-494a-ab18-4abbebbebbde tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:05.287866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a746470f-fb2a-494a-ab18-4abbebbebbde tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:05.296318 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a746470f-fb2a-494a-ab18-4abbebbebbde tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1505 microversion: 2.1 time: 0.170778 Sep 27 10:52:05.297139 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3730/6988] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:05 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1505 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:05.362552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServersNegativeTestJSON-server-1189062718", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "29934d6a-9abf-4327-b7c9-50388a209ed4"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:52:05.371279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:05.419587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] validate_networks() for [('29934d6a-9abf-4327-b7c9-50388a209ed4', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:52:05.816054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:52:05.816493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:52:05.817011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:52:05.817436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:52:05.817866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:52:05.828082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:05.828639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:05.829121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:05.829936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:05.830326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:05.830820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:05.849020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Getting quotas for project b4eda9ec9a0a4216a8b70d63470e6838. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:52:05.854668 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Getting quotas for user b9b3c7e824e74cbba45639dfed007e4d and project b4eda9ec9a0a4216a8b70d63470e6838. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:52:05.867879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:52:05.870313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:52:05.917407 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2022d7ef-c86a-4f33-8690-aedf3b8aeefb tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.559042 Sep 27 10:52:05.918094 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3259/6989] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:52:05 2022] POST /compute/v2.1/servers => generated 380 bytes in 560 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:52:05.929763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-bb6df1ca-b494-414b-9601-b77bad6b269e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:05.930471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-bb6df1ca-b494-414b-9601-b77bad6b269e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:05.985850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-bb6df1ca-b494-414b-9601-b77bad6b269e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:05.993111 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-bb6df1ca-b494-414b-9601-b77bad6b269e tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1217 microversion: 2.1 time: 0.067533 Sep 27 10:52:05.993903 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3731/6990] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:05 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1217 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:07.021807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-62c242af-dd54-4b6d-8e4c-50833a772457 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:07.023863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-62c242af-dd54-4b6d-8e4c-50833a772457 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:07.033128 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62c242af-dd54-4b6d-8e4c-50833a772457 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:07.033609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62c242af-dd54-4b6d-8e4c-50833a772457 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:07.034118 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62c242af-dd54-4b6d-8e4c-50833a772457 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:07.034836 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62c242af-dd54-4b6d-8e4c-50833a772457 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:07.035301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62c242af-dd54-4b6d-8e4c-50833a772457 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:07.035758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-62c242af-dd54-4b6d-8e4c-50833a772457 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:07.106141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-62c242af-dd54-4b6d-8e4c-50833a772457 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:07.115061 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-62c242af-dd54-4b6d-8e4c-50833a772457 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1269 microversion: 2.1 time: 0.101586 Sep 27 10:52:07.115676 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3260/6991] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:07 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1269 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:08.139167 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-23c624eb-d1d4-4f84-ab74-89806d10114b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:08.139865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-23c624eb-d1d4-4f84-ab74-89806d10114b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:08.151192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c624eb-d1d4-4f84-ab74-89806d10114b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:08.152038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c624eb-d1d4-4f84-ab74-89806d10114b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:08.152713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c624eb-d1d4-4f84-ab74-89806d10114b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:08.153663 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c624eb-d1d4-4f84-ab74-89806d10114b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:08.154329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c624eb-d1d4-4f84-ab74-89806d10114b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:08.154976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-23c624eb-d1d4-4f84-ab74-89806d10114b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:08.320146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-23c624eb-d1d4-4f84-ab74-89806d10114b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:08.328631 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-23c624eb-d1d4-4f84-ab74-89806d10114b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1317 microversion: 2.1 time: 0.193480 Sep 27 10:52:08.329265 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3732/6992] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:08 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1317 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:08.915859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-aef1db98-0b7f-4a37-ac9f-a56df8173012 req-fab1b167-cadc-4ea4-b49a-1d768616ed16 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "83c3fc9b-c1c1-4946-96a0-3af64a14bd44", "tag": "16acf3e1-d208-4f34-98cc-b27f67f4ddb8"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:52:08.927001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-aef1db98-0b7f-4a37-ac9f-a56df8173012 req-fab1b167-cadc-4ea4-b49a-1d768616ed16 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:08.927486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-aef1db98-0b7f-4a37-ac9f-a56df8173012 req-fab1b167-cadc-4ea4-b49a-1d768616ed16 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:08.927939 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-aef1db98-0b7f-4a37-ac9f-a56df8173012 req-fab1b167-cadc-4ea4-b49a-1d768616ed16 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:08.943889 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-aef1db98-0b7f-4a37-ac9f-a56df8173012 req-fab1b167-cadc-4ea4-b49a-1d768616ed16 service nova] Creating event network-changed:16acf3e1-d208-4f34-98cc-b27f67f4ddb8 for instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 on nch1-master-1 Sep 27 10:52:08.949155 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-aef1db98-0b7f-4a37-ac9f-a56df8173012 req-fab1b167-cadc-4ea4-b49a-1d768616ed16 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037375 Sep 27 10:52:08.950058 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3261/6993] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:52:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:52:09.352024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6ed98958-4370-4ee0-9efd-6011c1a5d43c tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:09.352613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6ed98958-4370-4ee0-9efd-6011c1a5d43c tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:09.359863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ed98958-4370-4ee0-9efd-6011c1a5d43c tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:09.360368 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ed98958-4370-4ee0-9efd-6011c1a5d43c tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:09.360849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ed98958-4370-4ee0-9efd-6011c1a5d43c tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:09.361636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ed98958-4370-4ee0-9efd-6011c1a5d43c tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:09.362136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ed98958-4370-4ee0-9efd-6011c1a5d43c tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:09.362466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6ed98958-4370-4ee0-9efd-6011c1a5d43c tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:09.494326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6ed98958-4370-4ee0-9efd-6011c1a5d43c tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:09.502592 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6ed98958-4370-4ee0-9efd-6011c1a5d43c tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1317 microversion: 2.1 time: 0.154732 Sep 27 10:52:09.503221 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3733/6994] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:09 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1317 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:10.530503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6adbe468-a67f-4d54-bdc5-42f9080c90c5 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:10.531310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6adbe468-a67f-4d54-bdc5-42f9080c90c5 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:10.539036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6adbe468-a67f-4d54-bdc5-42f9080c90c5 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:10.539508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6adbe468-a67f-4d54-bdc5-42f9080c90c5 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:10.539961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6adbe468-a67f-4d54-bdc5-42f9080c90c5 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:10.540716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6adbe468-a67f-4d54-bdc5-42f9080c90c5 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:10.541196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6adbe468-a67f-4d54-bdc5-42f9080c90c5 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:10.541639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6adbe468-a67f-4d54-bdc5-42f9080c90c5 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:10.739819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6adbe468-a67f-4d54-bdc5-42f9080c90c5 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:10.750674 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6adbe468-a67f-4d54-bdc5-42f9080c90c5 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1317 microversion: 2.1 time: 0.228132 Sep 27 10:52:10.751581 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3262/6995] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:10 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1317 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:11.778397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2fae55ea-2545-473d-930e-1dcb7a69b8d1 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:11.779398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2fae55ea-2545-473d-930e-1dcb7a69b8d1 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:11.791340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fae55ea-2545-473d-930e-1dcb7a69b8d1 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:11.792165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fae55ea-2545-473d-930e-1dcb7a69b8d1 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:11.792967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fae55ea-2545-473d-930e-1dcb7a69b8d1 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:11.794144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fae55ea-2545-473d-930e-1dcb7a69b8d1 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:11.794903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fae55ea-2545-473d-930e-1dcb7a69b8d1 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:11.795629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2fae55ea-2545-473d-930e-1dcb7a69b8d1 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:11.943122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2fae55ea-2545-473d-930e-1dcb7a69b8d1 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:11.951133 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2fae55ea-2545-473d-930e-1dcb7a69b8d1 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1317 microversion: 2.1 time: 0.179525 Sep 27 10:52:11.951732 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3734/6996] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:11 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1317 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:12.974502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-eeadf9c3-10f5-4c7a-9948-85f47b5628b9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:12.975144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-eeadf9c3-10f5-4c7a-9948-85f47b5628b9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:12.983401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeadf9c3-10f5-4c7a-9948-85f47b5628b9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:12.984235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeadf9c3-10f5-4c7a-9948-85f47b5628b9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:12.985009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeadf9c3-10f5-4c7a-9948-85f47b5628b9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:12.986080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeadf9c3-10f5-4c7a-9948-85f47b5628b9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:12.987325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeadf9c3-10f5-4c7a-9948-85f47b5628b9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:12.987880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-eeadf9c3-10f5-4c7a-9948-85f47b5628b9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:13.128919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-eeadf9c3-10f5-4c7a-9948-85f47b5628b9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:13.136930 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-eeadf9c3-10f5-4c7a-9948-85f47b5628b9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1317 microversion: 2.1 time: 0.166639 Sep 27 10:52:13.137570 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3263/6997] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:12 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1317 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:14.162893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-57c513ca-1fb4-406c-99a4-e822b8afaec6 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:14.163555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-57c513ca-1fb4-406c-99a4-e822b8afaec6 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:14.171857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57c513ca-1fb4-406c-99a4-e822b8afaec6 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:14.172408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57c513ca-1fb4-406c-99a4-e822b8afaec6 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:14.173135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57c513ca-1fb4-406c-99a4-e822b8afaec6 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:14.173818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57c513ca-1fb4-406c-99a4-e822b8afaec6 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:14.174334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57c513ca-1fb4-406c-99a4-e822b8afaec6 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:14.174809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57c513ca-1fb4-406c-99a4-e822b8afaec6 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:14.323045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-57c513ca-1fb4-406c-99a4-e822b8afaec6 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:14.331274 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-57c513ca-1fb4-406c-99a4-e822b8afaec6 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1317 microversion: 2.1 time: 0.173213 Sep 27 10:52:14.331852 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3735/6998] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:14 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1317 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:14.907924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-adb5ba21-c720-4297-8243-bd42e624f4c8 req-98ab1166-dcf2-4527-84a5-d2cc04227641 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "83c3fc9b-c1c1-4946-96a0-3af64a14bd44", "name": "network-vif-plugged", "status": "completed", "tag": "16acf3e1-d208-4f34-98cc-b27f67f4ddb8"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:52:14.919287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-adb5ba21-c720-4297-8243-bd42e624f4c8 req-98ab1166-dcf2-4527-84a5-d2cc04227641 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:14.919794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-adb5ba21-c720-4297-8243-bd42e624f4c8 req-98ab1166-dcf2-4527-84a5-d2cc04227641 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:14.920254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-adb5ba21-c720-4297-8243-bd42e624f4c8 req-98ab1166-dcf2-4527-84a5-d2cc04227641 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:14.933647 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-adb5ba21-c720-4297-8243-bd42e624f4c8 req-98ab1166-dcf2-4527-84a5-d2cc04227641 service nova] Creating event network-vif-plugged:16acf3e1-d208-4f34-98cc-b27f67f4ddb8 for instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 on nch1-master-1 Sep 27 10:52:14.938908 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-adb5ba21-c720-4297-8243-bd42e624f4c8 req-98ab1166-dcf2-4527-84a5-d2cc04227641 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035018 Sep 27 10:52:14.939493 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3264/6999] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:52:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:52:15.359010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-216317d2-c8f0-4a20-b5a9-fd381a593fae tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:15.360047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-216317d2-c8f0-4a20-b5a9-fd381a593fae tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:15.368836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-216317d2-c8f0-4a20-b5a9-fd381a593fae tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:15.369330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-216317d2-c8f0-4a20-b5a9-fd381a593fae tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:15.369781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-216317d2-c8f0-4a20-b5a9-fd381a593fae tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:15.370490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-216317d2-c8f0-4a20-b5a9-fd381a593fae tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:15.370988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-216317d2-c8f0-4a20-b5a9-fd381a593fae tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:15.371426 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-216317d2-c8f0-4a20-b5a9-fd381a593fae tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:15.508425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-216317d2-c8f0-4a20-b5a9-fd381a593fae tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:15.516326 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-216317d2-c8f0-4a20-b5a9-fd381a593fae tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1317 microversion: 2.1 time: 0.164065 Sep 27 10:52:15.516976 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3736/7000] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:15 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1317 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:16.541013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8424ba2a-9a59-4414-bec5-b86ac4cbf10b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:16.541747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8424ba2a-9a59-4414-bec5-b86ac4cbf10b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:16.543770 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:16.544465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:16.551857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8424ba2a-9a59-4414-bec5-b86ac4cbf10b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:16.552356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8424ba2a-9a59-4414-bec5-b86ac4cbf10b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:16.552869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8424ba2a-9a59-4414-bec5-b86ac4cbf10b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:16.553566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8424ba2a-9a59-4414-bec5-b86ac4cbf10b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:16.554035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8424ba2a-9a59-4414-bec5-b86ac4cbf10b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:16.554473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8424ba2a-9a59-4414-bec5-b86ac4cbf10b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:16.695846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8424ba2a-9a59-4414-bec5-b86ac4cbf10b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:16.707281 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8424ba2a-9a59-4414-bec5-b86ac4cbf10b tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1505 microversion: 2.1 time: 0.170959 Sep 27 10:52:16.707867 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3265/7001] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:16 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1505 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:16.729713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7cb79b83-f79d-40df-91ea-591aef419dc2 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:16.730259 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7cb79b83-f79d-40df-91ea-591aef419dc2 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:16.732146 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:16.738711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cb79b83-f79d-40df-91ea-591aef419dc2 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:16.739236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cb79b83-f79d-40df-91ea-591aef419dc2 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:16.739684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cb79b83-f79d-40df-91ea-591aef419dc2 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:16.740391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cb79b83-f79d-40df-91ea-591aef419dc2 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:16.740896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cb79b83-f79d-40df-91ea-591aef419dc2 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:16.741318 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7cb79b83-f79d-40df-91ea-591aef419dc2 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:16.767272 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7cb79b83-f79d-40df-91ea-591aef419dc2 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:52:16.859685 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7cb79b83-f79d-40df-91ea-591aef419dc2 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "DELETE /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 204 len: 0 microversion: 2.1 time: 0.133552 Sep 27 10:52:16.860189 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3737/7002] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:52:16 2022] DELETE /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:52:16.869449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7473900-44a9-48d0-89c2-ee1b13c1aa2f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:16.870073 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7473900-44a9-48d0-89c2-ee1b13c1aa2f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:16.876446 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7473900-44a9-48d0-89c2-ee1b13c1aa2f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:16.876968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7473900-44a9-48d0-89c2-ee1b13c1aa2f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:16.877420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7473900-44a9-48d0-89c2-ee1b13c1aa2f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:16.878137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7473900-44a9-48d0-89c2-ee1b13c1aa2f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:16.878651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7473900-44a9-48d0-89c2-ee1b13c1aa2f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:16.879139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7473900-44a9-48d0-89c2-ee1b13c1aa2f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:17.016306 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c7473900-44a9-48d0-89c2-ee1b13c1aa2f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:17.025456 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c7473900-44a9-48d0-89c2-ee1b13c1aa2f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1511 microversion: 2.1 time: 0.160086 Sep 27 10:52:17.026037 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3266/7003] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:16 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1511 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:18.053635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-85e9be69-6853-4317-9bf6-17eeac541f21 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:18.054856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-85e9be69-6853-4317-9bf6-17eeac541f21 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:18.066684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85e9be69-6853-4317-9bf6-17eeac541f21 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:18.067340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85e9be69-6853-4317-9bf6-17eeac541f21 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:18.067612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85e9be69-6853-4317-9bf6-17eeac541f21 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:18.068321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85e9be69-6853-4317-9bf6-17eeac541f21 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:18.068812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85e9be69-6853-4317-9bf6-17eeac541f21 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:18.069241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85e9be69-6853-4317-9bf6-17eeac541f21 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:18.205005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-85e9be69-6853-4317-9bf6-17eeac541f21 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:18.214446 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-85e9be69-6853-4317-9bf6-17eeac541f21 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 200 len: 1511 microversion: 2.1 time: 0.168094 Sep 27 10:52:18.215040 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3738/7004] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:18 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 1511 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:18.672025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d46b3046-f09c-4dd9-9b1a-c3ca52883f02 req-33f41fc6-6fb2-414d-a044-fceffb75d707 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "83c3fc9b-c1c1-4946-96a0-3af64a14bd44", "name": "network-vif-deleted", "tag": "16acf3e1-d208-4f34-98cc-b27f67f4ddb8"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:52:18.685983 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d46b3046-f09c-4dd9-9b1a-c3ca52883f02 req-33f41fc6-6fb2-414d-a044-fceffb75d707 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:18.687021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d46b3046-f09c-4dd9-9b1a-c3ca52883f02 req-33f41fc6-6fb2-414d-a044-fceffb75d707 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:18.687743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d46b3046-f09c-4dd9-9b1a-c3ca52883f02 req-33f41fc6-6fb2-414d-a044-fceffb75d707 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:18.702582 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d46b3046-f09c-4dd9-9b1a-c3ca52883f02 req-33f41fc6-6fb2-414d-a044-fceffb75d707 service nova] Creating event network-vif-deleted:16acf3e1-d208-4f34-98cc-b27f67f4ddb8 for instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 on nch1-master-1 Sep 27 10:52:18.710247 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d46b3046-f09c-4dd9-9b1a-c3ca52883f02 req-33f41fc6-6fb2-414d-a044-fceffb75d707 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041085 Sep 27 10:52:18.710869 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3267/7005] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:52:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:52:19.239483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:19.240282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:19.247585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.248075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.248510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:19.249277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.249733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.250156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:19.264806 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] HTTP exception thrown: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Request method failure captured: Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 HTTP/1.1 Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtXJPw8paDe2AviMhb85V32XVVdU5SlJX6VG57dTgYJZq150haFBQopFrIzbVcm7quh9hlwp_8Y4YEMvxT8Jjb3x1W_WZOJ1TPkKdHxVnsZflLPA8dBYHcXVLcd2G_pY9DNS64LITZp4Ln0rOcGdZ1pmBRDkOH5S5hiz6_-XFXX8vss Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b4eda9ec9a0a4216a8b70d63470e6838"}]}] Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b9b3c7e824e74cbba45639dfed007e4d Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '83c3fc9b-c1c1-4946-96a0-3af64a14bd44'} Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:19.266505 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.271786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:19.271786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:19.271786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:19.271786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:52:19.271786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:52:19.271786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:19.271786 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.271786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Returning 404 to user: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:52:19.271786 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8c22c7d5-4dab-4670-aad3-904a87adb847 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 404 len: 111 microversion: 2.1 time: 0.033987 Sep 27 10:52:19.271786 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3739/7006] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:19 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:52:19.278576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f1f8f0c7-b09b-4b57-a22c-dd30761f8230 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:19.279336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f1f8f0c7-b09b-4b57-a22c-dd30761f8230 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:19.286324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1f8f0c7-b09b-4b57-a22c-dd30761f8230 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.287514 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1f8f0c7-b09b-4b57-a22c-dd30761f8230 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.287970 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1f8f0c7-b09b-4b57-a22c-dd30761f8230 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:19.288719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1f8f0c7-b09b-4b57-a22c-dd30761f8230 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.289186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1f8f0c7-b09b-4b57-a22c-dd30761f8230 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.289630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f1f8f0c7-b09b-4b57-a22c-dd30761f8230 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:19.443115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f1f8f0c7-b09b-4b57-a22c-dd30761f8230 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:19.451467 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f1f8f0c7-b09b-4b57-a22c-dd30761f8230 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1505 microversion: 2.1 time: 0.177279 Sep 27 10:52:19.452052 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3268/7007] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:19 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1505 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:19.474652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6c288ecf-d0c1-40dd-bcd4-a8276a3ca0da tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "452"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:52:19.477104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6c288ecf-d0c1-40dd-bcd4-a8276a3ca0da tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 8af19fc3-196a-49f1-80fe-43f799a736d1] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:19.483135 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-6c288ecf-d0c1-40dd-bcd4-a8276a3ca0da tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] HTTP exception thrown: Instance 8af19fc3-196a-49f1-80fe-43f799a736d1 could not be found. Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6c288ecf-d0c1-40dd-bcd4-a8276a3ca0da tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Request method failure captured: Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/8af19fc3-196a-49f1-80fe-43f799a736d1/action HTTP/1.1 Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 32 Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtXJPw8paDe2AviMhb85V32XVVdU5SlJX6VG57dTgYJZq150haFBQopFrIzbVcm7quh9hlwp_8Y4YEMvxT8Jjb3x1W_WZOJ1TPkKdHxVnsZflLPA8dBYHcXVLcd2G_pY9DNS64LITZp4Ln0rOcGdZ1pmBRDkOH5S5hiz6_-XFXX8vss Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b4eda9ec9a0a4216a8b70d63470e6838"}]}] Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b9b3c7e824e74cbba45639dfed007e4d Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: {"resize": {"flavorRef": "452"}} Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 8af19fc3-196a-49f1-80fe-43f799a736d1 could not be found. Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '8af19fc3-196a-49f1-80fe-43f799a736d1', 'body': {'resize': {'flavorRef': '452'}}} Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2953, in _get_instance Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=instance_uuid) Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 8af19fc3-196a-49f1-80fe-43f799a736d1 could not be found. Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1046, in _resize Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_id, Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:52:19.484706 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 8af19fc3-196a-49f1-80fe-43f799a736d1 could not be found. Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 8af19fc3-196a-49f1-80fe-43f799a736d1 could not be found. Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6c288ecf-d0c1-40dd-bcd4-a8276a3ca0da tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Returning 404 to user: Instance 8af19fc3-196a-49f1-80fe-43f799a736d1 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6c288ecf-d0c1-40dd-bcd4-a8276a3ca0da tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "POST /compute/v2.1/servers/8af19fc3-196a-49f1-80fe-43f799a736d1/action" status: 404 len: 111 microversion: 2.1 time: 0.016320 Sep 27 10:52:19.489476 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3740/7008] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:52:19 2022] POST /compute/v2.1/servers/8af19fc3-196a-49f1-80fe-43f799a736d1/action => generated 111 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 27 10:52:19.498123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a5b0f11d-8cb2-4cf4-b705-eae1e8ca21a4 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:19.498762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a5b0f11d-8cb2-4cf4-b705-eae1e8ca21a4 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:19.506834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5b0f11d-8cb2-4cf4-b705-eae1e8ca21a4 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.507307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5b0f11d-8cb2-4cf4-b705-eae1e8ca21a4 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.507783 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5b0f11d-8cb2-4cf4-b705-eae1e8ca21a4 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:19.508493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5b0f11d-8cb2-4cf4-b705-eae1e8ca21a4 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.509031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5b0f11d-8cb2-4cf4-b705-eae1e8ca21a4 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.509445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5b0f11d-8cb2-4cf4-b705-eae1e8ca21a4 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:19.656240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a5b0f11d-8cb2-4cf4-b705-eae1e8ca21a4 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:19.665500 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a5b0f11d-8cb2-4cf4-b705-eae1e8ca21a4 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1505 microversion: 2.1 time: 0.173547 Sep 27 10:52:19.666087 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3269/7009] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:19 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1505 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:19.689324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7680e7f0-d10c-4799-9c66-c786c6a23755 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:19.689944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7680e7f0-d10c-4799-9c66-c786c6a23755 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:19.700847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7680e7f0-d10c-4799-9c66-c786c6a23755 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.701335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7680e7f0-d10c-4799-9c66-c786c6a23755 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.701770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7680e7f0-d10c-4799-9c66-c786c6a23755 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:19.702454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7680e7f0-d10c-4799-9c66-c786c6a23755 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.702915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7680e7f0-d10c-4799-9c66-c786c6a23755 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.703340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7680e7f0-d10c-4799-9c66-c786c6a23755 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:19.839684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7680e7f0-d10c-4799-9c66-c786c6a23755 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:19.847810 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7680e7f0-d10c-4799-9c66-c786c6a23755 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1505 microversion: 2.1 time: 0.162509 Sep 27 10:52:19.848469 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3741/7010] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:19 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1505 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:19.872017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "9cc7cf48-a1a6-47fb-995c-7db274277849"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:52:19.874378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:19.883034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.883584 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.884039 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:19.884914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.885388 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.885835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:19.931127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:19.936569 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] HTTP exception thrown: Flavor 9cc7cf48-a1a6-47fb-995c-7db274277849 could not be found. Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Request method failure captured: Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: request: POST /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c/action HTTP/1.1 Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 65 Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtXJPw8paDe2AviMhb85V32XVVdU5SlJX6VG57dTgYJZq150haFBQopFrIzbVcm7quh9hlwp_8Y4YEMvxT8Jjb3x1W_WZOJ1TPkKdHxVnsZflLPA8dBYHcXVLcd2G_pY9DNS64LITZp4Ln0rOcGdZ1pmBRDkOH5S5hiz6_-XFXX8vss Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b4eda9ec9a0a4216a8b70d63470e6838"}]}] Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b9b3c7e824e74cbba45639dfed007e4d Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: {"resize": {"flavorRef": "9cc7cf48-a1a6-47fb-995c-7db274277849"}} Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: exception: Flavor 9cc7cf48-a1a6-47fb-995c-7db274277849 could not be found. Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '34c36fdc-0be7-4b70-b32d-cd0d45d0381c', 'body': {'resize': {'flavorRef': '9cc7cf48-a1a6-47fb-995c-7db274277849'}}} Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1053, in _resize Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self.compute_api.resize(context, instance, flavor_id, Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 388, in inner Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 373, in wrapper Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 356, in wrapper Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(self, context, instance, *args, **kwargs) Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4200, in resize Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi new_flavor = flavors.get_flavor_by_flavor_id( Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:52:19.938578 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 9cc7cf48-a1a6-47fb-995c-7db274277849 could not be found. Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1139, in _action_resize Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi self._resize(req, id, flavor_ref, **kwargs) Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1083, in _resize Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Flavor 9cc7cf48-a1a6-47fb-995c-7db274277849 could not be found. Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 9cc7cf48-a1a6-47fb-995c-7db274277849 could not be found. Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Returning 400 to user: Flavor 9cc7cf48-a1a6-47fb-995c-7db274277849 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b741160-fb0f-4783-8381-4311bf3eac96 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "POST /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c/action" status: 400 len: 107 microversion: 2.1 time: 0.072154 Sep 27 10:52:19.943931 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3270/7011] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:52:19 2022] POST /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c/action => generated 107 bytes in 73 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:52:19.949797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6c9b9bc7-e0e1-4a8c-ad17-d8a02f955b62 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:19.950425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6c9b9bc7-e0e1-4a8c-ad17-d8a02f955b62 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:19.956913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c9b9bc7-e0e1-4a8c-ad17-d8a02f955b62 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.957399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c9b9bc7-e0e1-4a8c-ad17-d8a02f955b62 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.958508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c9b9bc7-e0e1-4a8c-ad17-d8a02f955b62 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:19.959237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c9b9bc7-e0e1-4a8c-ad17-d8a02f955b62 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:19.959705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c9b9bc7-e0e1-4a8c-ad17-d8a02f955b62 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:19.960143 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c9b9bc7-e0e1-4a8c-ad17-d8a02f955b62 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.112298 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6c9b9bc7-e0e1-4a8c-ad17-d8a02f955b62 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:20.116848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:20.122005 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6c9b9bc7-e0e1-4a8c-ad17-d8a02f955b62 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1505 microversion: 2.1 time: 0.176157 Sep 27 10:52:20.122587 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3742/7012] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:19 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1505 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:20.145715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d40b3c5f-9d10-4d0c-a7d3-9eda9449caa9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:20.146310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d40b3c5f-9d10-4d0c-a7d3-9eda9449caa9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:20.153208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d40b3c5f-9d10-4d0c-a7d3-9eda9449caa9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.153721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d40b3c5f-9d10-4d0c-a7d3-9eda9449caa9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.154430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d40b3c5f-9d10-4d0c-a7d3-9eda9449caa9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.155203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d40b3c5f-9d10-4d0c-a7d3-9eda9449caa9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.155680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d40b3c5f-9d10-4d0c-a7d3-9eda9449caa9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.156125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d40b3c5f-9d10-4d0c-a7d3-9eda9449caa9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.305493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d40b3c5f-9d10-4d0c-a7d3-9eda9449caa9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:20.317836 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d40b3c5f-9d10-4d0c-a7d3-9eda9449caa9 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1505 microversion: 2.1 time: 0.175812 Sep 27 10:52:20.318689 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3271/7013] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:20 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1505 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:20.352129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f2c8bed7-5ab9-4ad9-99d2-a7c82c5f2e55 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": ""}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f2c8bed7-5ab9-4ad9-99d2-a7c82c5f2e55 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Request method failure captured: Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: request: POST /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c/action HTTP/1.1 Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 29 Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtXJPw8paDe2AviMhb85V32XVVdU5SlJX6VG57dTgYJZq150haFBQopFrIzbVcm7quh9hlwp_8Y4YEMvxT8Jjb3x1W_WZOJ1TPkKdHxVnsZflLPA8dBYHcXVLcd2G_pY9DNS64LITZp4Ln0rOcGdZ1pmBRDkOH5S5hiz6_-XFXX8vss Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b4eda9ec9a0a4216a8b70d63470e6838"}]}] Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b9b3c7e824e74cbba45639dfed007e4d Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: {"resize": {"flavorRef": ""}} Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: exception: Invalid input for field/attribute flavorRef. Value: . '' is too short Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '34c36fdc-0be7-4b70-b32d-cd0d45d0381c', 'body': {'resize': {'flavorRef': ''}}} Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise error Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' is too short Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Failed validating 'minLength' in schema['properties']['resize']['properties']['flavorRef']: Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi {'minLength': 1, 'type': ['string', 'integer']} Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi On instance['resize']: Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi '' Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute flavorRef. Value: . '' is too short Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.355769 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:20.361113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.361113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:20.361113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:20.361113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:20.361113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 27 10:52:20.361113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 27 10:52:20.361113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute flavorRef. Value: . '' is too short Sep 27 10:52:20.361113 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.361113 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f2c8bed7-5ab9-4ad9-99d2-a7c82c5f2e55 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Returning 400 to user: Invalid input for field/attribute flavorRef. Value: . '' is too short {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:52:20.361113 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f2c8bed7-5ab9-4ad9-99d2-a7c82c5f2e55 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "POST /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c/action" status: 400 len: 113 microversion: 2.1 time: 0.010292 Sep 27 10:52:20.361113 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3743/7014] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:52:20 2022] POST /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c/action => generated 113 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 27 10:52:20.366407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-64cbf514-38bd-49e3-b643-ec51d9c9e290 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:20.367044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-64cbf514-38bd-49e3-b643-ec51d9c9e290 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:20.373599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64cbf514-38bd-49e3-b643-ec51d9c9e290 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.374098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64cbf514-38bd-49e3-b643-ec51d9c9e290 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.374548 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64cbf514-38bd-49e3-b643-ec51d9c9e290 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.375263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64cbf514-38bd-49e3-b643-ec51d9c9e290 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.375733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64cbf514-38bd-49e3-b643-ec51d9c9e290 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.376170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-64cbf514-38bd-49e3-b643-ec51d9c9e290 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.509218 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-64cbf514-38bd-49e3-b643-ec51d9c9e290 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:20.517197 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-64cbf514-38bd-49e3-b643-ec51d9c9e290 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1505 microversion: 2.1 time: 0.154856 Sep 27 10:52:20.517777 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3272/7015] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:20 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1505 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:20.549253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:20.549888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:20.566354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.566894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.567345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.568264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.568772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.569362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.587150 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] HTTP exception thrown: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Request method failure captured: Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 HTTP/1.1 Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtXJPw8paDe2AviMhb85V32XVVdU5SlJX6VG57dTgYJZq150haFBQopFrIzbVcm7quh9hlwp_8Y4YEMvxT8Jjb3x1W_WZOJ1TPkKdHxVnsZflLPA8dBYHcXVLcd2G_pY9DNS64LITZp4Ln0rOcGdZ1pmBRDkOH5S5hiz6_-XFXX8vss Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b4eda9ec9a0a4216a8b70d63470e6838"}]}] Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: b9b3c7e824e74cbba45639dfed007e4d Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '83c3fc9b-c1c1-4946-96a0-3af64a14bd44'} Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:52:20.588767 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Returning 404 to user: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d0c9600-b696-4a2c-b1a9-1fa82a337a34 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "DELETE /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 404 len: 111 microversion: 2.1 time: 0.049989 Sep 27 10:52:20.593336 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3744/7016] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:52:20 2022] DELETE /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:52:20.602080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:20.602774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 83c3fc9b-c1c1-4946-96a0-3af64a14bd44] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:20.611424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.611927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.612392 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.613153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.613620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.614140 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.628521 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] HTTP exception thrown: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Request method failure captured: Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 HTTP/1.1 Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtXJPw8paDe2AviMhb85V32XVVdU5SlJX6VG57dTgYJZq150haFBQopFrIzbVcm7quh9hlwp_8Y4YEMvxT8Jjb3x1W_WZOJ1TPkKdHxVnsZflLPA8dBYHcXVLcd2G_pY9DNS64LITZp4Ln0rOcGdZ1pmBRDkOH5S5hiz6_-XFXX8vss Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b4eda9ec9a0a4216a8b70d63470e6838"}]}] Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b9b3c7e824e74cbba45639dfed007e4d Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '83c3fc9b-c1c1-4946-96a0-3af64a14bd44'} Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:20.630483 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.635796 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:20.635796 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:20.635796 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:20.635796 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:52:20.635796 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:52:20.635796 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. Sep 27 10:52:20.635796 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:20.635796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Returning 404 to user: Instance 83c3fc9b-c1c1-4946-96a0-3af64a14bd44 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:52:20.635796 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-75a2924e-066f-4bd1-9f55-3d5edb52e571 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44" status: 404 len: 111 microversion: 2.1 time: 0.036124 Sep 27 10:52:20.635796 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3273/7017] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:20 2022] GET /compute/v2.1/servers/83c3fc9b-c1c1-4946-96a0-3af64a14bd44 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:52:20.640917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2a400fc9-05b0-4842-8156-a9c9eed5f19f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:20.641419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2a400fc9-05b0-4842-8156-a9c9eed5f19f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:20.647901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a400fc9-05b0-4842-8156-a9c9eed5f19f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.648382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a400fc9-05b0-4842-8156-a9c9eed5f19f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.648932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a400fc9-05b0-4842-8156-a9c9eed5f19f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.649624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a400fc9-05b0-4842-8156-a9c9eed5f19f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.650075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a400fc9-05b0-4842-8156-a9c9eed5f19f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.650495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2a400fc9-05b0-4842-8156-a9c9eed5f19f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.676348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2a400fc9-05b0-4842-8156-a9c9eed5f19f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:52:20.760264 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2a400fc9-05b0-4842-8156-a9c9eed5f19f tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "DELETE /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 204 len: 0 microversion: 2.1 time: 0.123073 Sep 27 10:52:20.760864 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3745/7018] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:52:20 2022] DELETE /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 0 bytes in 125 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 27 10:52:20.770221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-44f744a3-777d-46b0-b324-ff3b3cb7a713 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:20.770861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-44f744a3-777d-46b0-b324-ff3b3cb7a713 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:20.776922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44f744a3-777d-46b0-b324-ff3b3cb7a713 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.777418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44f744a3-777d-46b0-b324-ff3b3cb7a713 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.777867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44f744a3-777d-46b0-b324-ff3b3cb7a713 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.778579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44f744a3-777d-46b0-b324-ff3b3cb7a713 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:20.779043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44f744a3-777d-46b0-b324-ff3b3cb7a713 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:20.779481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-44f744a3-777d-46b0-b324-ff3b3cb7a713 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:20.915275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-44f744a3-777d-46b0-b324-ff3b3cb7a713 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:20.918931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=100 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:52:20.923897 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-44f744a3-777d-46b0-b324-ff3b3cb7a713 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1511 microversion: 2.1 time: 0.157856 Sep 27 10:52:20.924488 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3274/7019] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:20 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1511 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:21.948501 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-79215fbb-f2e7-4aef-83cd-7e8a9d28beca tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:21.949281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-79215fbb-f2e7-4aef-83cd-7e8a9d28beca tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:21.956840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79215fbb-f2e7-4aef-83cd-7e8a9d28beca tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:21.957328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79215fbb-f2e7-4aef-83cd-7e8a9d28beca tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:21.957907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79215fbb-f2e7-4aef-83cd-7e8a9d28beca tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:21.958755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79215fbb-f2e7-4aef-83cd-7e8a9d28beca tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:21.959216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79215fbb-f2e7-4aef-83cd-7e8a9d28beca tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:21.959647 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-79215fbb-f2e7-4aef-83cd-7e8a9d28beca tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:22.102798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-79215fbb-f2e7-4aef-83cd-7e8a9d28beca tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:22.111177 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-79215fbb-f2e7-4aef-83cd-7e8a9d28beca tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1511 microversion: 2.1 time: 0.167084 Sep 27 10:52:22.111753 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3746/7020] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:21 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1511 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:22.902452 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f5d3c2a1-dd82-49b4-881d-2bf487c32bc5 req-ff2f5f44-d9fb-4e34-91cf-6894876c43c1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "34c36fdc-0be7-4b70-b32d-cd0d45d0381c", "name": "network-vif-deleted", "tag": "0867c694-80a2-4a22-86cd-045c6e37cde7"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:52:22.915102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f5d3c2a1-dd82-49b4-881d-2bf487c32bc5 req-ff2f5f44-d9fb-4e34-91cf-6894876c43c1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:22.915572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f5d3c2a1-dd82-49b4-881d-2bf487c32bc5 req-ff2f5f44-d9fb-4e34-91cf-6894876c43c1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:22.916217 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f5d3c2a1-dd82-49b4-881d-2bf487c32bc5 req-ff2f5f44-d9fb-4e34-91cf-6894876c43c1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:22.928566 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f5d3c2a1-dd82-49b4-881d-2bf487c32bc5 req-ff2f5f44-d9fb-4e34-91cf-6894876c43c1 service nova] Creating event network-vif-deleted:0867c694-80a2-4a22-86cd-045c6e37cde7 for instance 34c36fdc-0be7-4b70-b32d-cd0d45d0381c on nch2-master-1 Sep 27 10:52:22.935186 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f5d3c2a1-dd82-49b4-881d-2bf487c32bc5 req-ff2f5f44-d9fb-4e34-91cf-6894876c43c1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036255 Sep 27 10:52:22.935663 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3275/7021] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:52:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:52:23.134421 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:23.135031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:23.142062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:23.142546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:23.142976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:23.143669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:23.144121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:23.144541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:23.180757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:23.181249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:23.181703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:23.182158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lazy-loading 'fault' on Instance uuid 34c36fdc-0be7-4b70-b32d-cd0d45d0381c {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:52:23.226818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b9b3c7e824e74cbba45639dfed007e4d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b4eda9ec9a0a4216a8b70d63470e6838', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:52:23.235173 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eaf1954b-394d-4cc5-a0a8-d5d48fc6d450 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 200 len: 1307 microversion: 2.1 time: 0.105085 Sep 27 10:52:23.235746 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3747/7022] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:23 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 1307 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:52:24.265554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:52:24.266728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] [instance: 34c36fdc-0be7-4b70-b32d-cd0d45d0381c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:52:24.275172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:24.275677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:24.276130 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:24.276889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:52:24.277362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:52:24.277800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:52:24.292234 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] HTTP exception thrown: Instance 34c36fdc-0be7-4b70-b32d-cd0d45d0381c could not be found. Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Request method failure captured: Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c HTTP/1.1 Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtXJPw8paDe2AviMhb85V32XVVdU5SlJX6VG57dTgYJZq150haFBQopFrIzbVcm7quh9hlwp_8Y4YEMvxT8Jjb3x1W_WZOJ1TPkKdHxVnsZflLPA8dBYHcXVLcd2G_pY9DNS64LITZp4Ln0rOcGdZ1pmBRDkOH5S5hiz6_-XFXX8vss Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/b4eda9ec9a0a4216a8b70d63470e6838"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_b4eda9ec9a0a4216a8b70d63470e6838"}]}] Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: b4eda9ec9a0a4216a8b70d63470e6838 Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-ServersNegativeTestJSON-1701345014 Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: b9b3c7e824e74cbba45639dfed007e4d Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-ServersNegativeTestJSON-1701345014-project Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 34c36fdc-0be7-4b70-b32d-cd0d45d0381c could not be found. Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '34c36fdc-0be7-4b70-b32d-cd0d45d0381c'} Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 34c36fdc-0be7-4b70-b32d-cd0d45d0381c could not be found. Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 34c36fdc-0be7-4b70-b32d-cd0d45d0381c could not be found. Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:52:24.293933 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:24.298701 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:52:24.298701 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:52:24.298701 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:52:24.298701 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:52:24.298701 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:52:24.298701 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 34c36fdc-0be7-4b70-b32d-cd0d45d0381c could not be found. Sep 27 10:52:24.298701 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:52:24.298701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] Returning 404 to user: Instance 34c36fdc-0be7-4b70-b32d-cd0d45d0381c could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:52:24.298701 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4b2e44ab-1673-49b8-8714-a14ee2e535b3 tempest-ServersNegativeTestJSON-1701345014 tempest-ServersNegativeTestJSON-1701345014-project] 10.222.0.26 "GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c" status: 404 len: 111 microversion: 2.1 time: 0.038961 Sep 27 10:52:24.298701 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3276/7023] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:52:24 2022] GET /compute/v2.1/servers/34c36fdc-0be7-4b70-b32d-cd0d45d0381c => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:53:05.428127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6c784158-ec41-43b0-8866-6eaf20c5c321 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'create', calling method: .version_select at 0x7f7f63b8a9d0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-570772346", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJ80eSAbuXyaE+IYVzjcXUBdexi9oK6y0SH/y+dJciFI8QfIYe22Bc7MJwhX315D1KU7R+AOc4l2zv4EXwB7JEe0RVzScsB7VpdKlr4t/RljXL1dEzFBrXElRvgJEQTqKw=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:53:05.445923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:05.446604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=30,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:05.463183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6c784158-ec41-43b0-8866-6eaf20c5c321 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Getting quotas for project 714fa95d9be140018a54ece37d204047. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:53:05.468572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6c784158-ec41-43b0-8866-6eaf20c5c321 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Getting quotas for user 906ca88c8f544940b06705711ae7f1af and project 714fa95d9be140018a54ece37d204047. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:53:05.499745 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6c784158-ec41-43b0-8866-6eaf20c5c321 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.076264 Sep 27 10:53:05.500630 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3748/7024] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:53:05 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:53:05.563272 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-2134686488", "imageRef": "454087f3-c8e7-43d9-946c-9fd2f913496a", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-570772346", "networks": [{"uuid": "bc6ee60c-393d-4434-9103-3f26aa0212f4"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:53:05.573195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=63,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:05.573872 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:05.760356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] validate_networks() for [('bc6ee60c-393d-4434-9103-3f26aa0212f4', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:53:06.187898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:53:06.188339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:53:06.188865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:53:06.189313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:53:06.189741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:53:06.199386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:06.199890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:06.200343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:06.201125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:06.201594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:06.202033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:06.219740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Getting quotas for project 714fa95d9be140018a54ece37d204047. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:53:06.225495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Getting quotas for user 906ca88c8f544940b06705711ae7f1af and project 714fa95d9be140018a54ece37d204047. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:53:06.238253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:53:06.240588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='454087f3-c8e7-43d9-946c-9fd2f913496a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:53:06.287456 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-61dcf009-0ffc-4694-ad1f-23c79e86d059 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.728500 Sep 27 10:53:06.288152 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3277/7025] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:53:05 2022] POST /compute/v2.1/servers => generated 380 bytes in 730 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:53:06.299338 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dfa3e78a-74dd-4780-9559-2bfdcffce6ec tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:06.299957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dfa3e78a-74dd-4780-9559-2bfdcffce6ec tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:06.475284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dfa3e78a-74dd-4780-9559-2bfdcffce6ec tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:06.482913 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dfa3e78a-74dd-4780-9559-2bfdcffce6ec tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1258 microversion: 2.1 time: 0.187683 Sep 27 10:53:06.483493 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3749/7026] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:06 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1258 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:07.507688 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2a3c1610-b07c-4a51-831c-576df7e8a82a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:07.508384 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2a3c1610-b07c-4a51-831c-576df7e8a82a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:07.518482 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3c1610-b07c-4a51-831c-576df7e8a82a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:07.519417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3c1610-b07c-4a51-831c-576df7e8a82a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:07.520056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3c1610-b07c-4a51-831c-576df7e8a82a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:07.520845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3c1610-b07c-4a51-831c-576df7e8a82a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:07.521597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3c1610-b07c-4a51-831c-576df7e8a82a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:07.522060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2a3c1610-b07c-4a51-831c-576df7e8a82a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:07.634769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2a3c1610-b07c-4a51-831c-576df7e8a82a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:07.645705 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2a3c1610-b07c-4a51-831c-576df7e8a82a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1328 microversion: 2.1 time: 0.143188 Sep 27 10:53:07.646713 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3278/7027] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:07 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1328 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:08.669993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d9a68420-cb1e-4779-8ac4-eaf9f1749302 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:08.670631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d9a68420-cb1e-4779-8ac4-eaf9f1749302 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:08.679103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a68420-cb1e-4779-8ac4-eaf9f1749302 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:08.679605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a68420-cb1e-4779-8ac4-eaf9f1749302 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:08.680055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a68420-cb1e-4779-8ac4-eaf9f1749302 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:08.680828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a68420-cb1e-4779-8ac4-eaf9f1749302 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:08.681291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a68420-cb1e-4779-8ac4-eaf9f1749302 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:08.681729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d9a68420-cb1e-4779-8ac4-eaf9f1749302 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:08.823601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d9a68420-cb1e-4779-8ac4-eaf9f1749302 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:08.832176 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d9a68420-cb1e-4779-8ac4-eaf9f1749302 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.166262 Sep 27 10:53:08.832800 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3750/7028] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:08 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:09.858946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b39289e6-2862-4a13-980b-a4723ebdb876 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:09.859600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b39289e6-2862-4a13-980b-a4723ebdb876 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:09.867644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b39289e6-2862-4a13-980b-a4723ebdb876 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:09.868224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b39289e6-2862-4a13-980b-a4723ebdb876 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:09.868787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b39289e6-2862-4a13-980b-a4723ebdb876 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:09.869535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b39289e6-2862-4a13-980b-a4723ebdb876 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:09.870016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b39289e6-2862-4a13-980b-a4723ebdb876 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:09.870651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b39289e6-2862-4a13-980b-a4723ebdb876 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:10.027868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b39289e6-2862-4a13-980b-a4723ebdb876 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:10.036728 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b39289e6-2862-4a13-980b-a4723ebdb876 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.181445 Sep 27 10:53:10.037609 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3279/7029] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:09 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:11.061123 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-14da9d30-7b3e-425a-9292-32e12685ba85 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:11.061799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-14da9d30-7b3e-425a-9292-32e12685ba85 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:11.070333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14da9d30-7b3e-425a-9292-32e12685ba85 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:11.070836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14da9d30-7b3e-425a-9292-32e12685ba85 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:11.071276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14da9d30-7b3e-425a-9292-32e12685ba85 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:11.071995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14da9d30-7b3e-425a-9292-32e12685ba85 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:11.073114 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14da9d30-7b3e-425a-9292-32e12685ba85 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:11.073774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-14da9d30-7b3e-425a-9292-32e12685ba85 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:11.234770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-14da9d30-7b3e-425a-9292-32e12685ba85 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:11.243508 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-14da9d30-7b3e-425a-9292-32e12685ba85 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.186301 Sep 27 10:53:11.244360 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3751/7030] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:11 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:12.269369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d074f3a1-42b3-48d5-bb8a-c8119500743d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:12.270030 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d074f3a1-42b3-48d5-bb8a-c8119500743d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:12.277613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d074f3a1-42b3-48d5-bb8a-c8119500743d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:12.278119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d074f3a1-42b3-48d5-bb8a-c8119500743d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:12.278576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d074f3a1-42b3-48d5-bb8a-c8119500743d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:12.279296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d074f3a1-42b3-48d5-bb8a-c8119500743d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:12.279769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d074f3a1-42b3-48d5-bb8a-c8119500743d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:12.280225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d074f3a1-42b3-48d5-bb8a-c8119500743d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:12.450949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d074f3a1-42b3-48d5-bb8a-c8119500743d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:12.460896 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d074f3a1-42b3-48d5-bb8a-c8119500743d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.195617 Sep 27 10:53:12.461780 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3280/7031] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:12 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:13.484443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-76e4669b-7810-4275-ace6-a06ebfb904ce tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:13.485141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-76e4669b-7810-4275-ace6-a06ebfb904ce tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:13.501783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76e4669b-7810-4275-ace6-a06ebfb904ce tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:13.502289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76e4669b-7810-4275-ace6-a06ebfb904ce tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:13.503232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76e4669b-7810-4275-ace6-a06ebfb904ce tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:13.503978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76e4669b-7810-4275-ace6-a06ebfb904ce tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:13.504461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76e4669b-7810-4275-ace6-a06ebfb904ce tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:13.504905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-76e4669b-7810-4275-ace6-a06ebfb904ce tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:13.699118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-76e4669b-7810-4275-ace6-a06ebfb904ce tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:13.708187 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-76e4669b-7810-4275-ace6-a06ebfb904ce tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.227930 Sep 27 10:53:13.721400 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3752/7032] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:13 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:14.735774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-865e1913-a388-4fbf-a92a-43b6ac2690c6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:14.737105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-865e1913-a388-4fbf-a92a-43b6ac2690c6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:14.744617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-865e1913-a388-4fbf-a92a-43b6ac2690c6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:14.745838 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-865e1913-a388-4fbf-a92a-43b6ac2690c6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:14.746311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-865e1913-a388-4fbf-a92a-43b6ac2690c6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:14.747068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-865e1913-a388-4fbf-a92a-43b6ac2690c6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:14.747569 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-865e1913-a388-4fbf-a92a-43b6ac2690c6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:14.747979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-865e1913-a388-4fbf-a92a-43b6ac2690c6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:14.893978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-865e1913-a388-4fbf-a92a-43b6ac2690c6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:14.902897 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-865e1913-a388-4fbf-a92a-43b6ac2690c6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.171572 Sep 27 10:53:14.903500 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3281/7033] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:14 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:15.929104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f309cb6e-4dea-4ed9-b07e-fc02a3e50e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:15.929764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f309cb6e-4dea-4ed9-b07e-fc02a3e50e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:15.937231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f309cb6e-4dea-4ed9-b07e-fc02a3e50e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:15.937723 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f309cb6e-4dea-4ed9-b07e-fc02a3e50e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:15.938164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f309cb6e-4dea-4ed9-b07e-fc02a3e50e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:15.938901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f309cb6e-4dea-4ed9-b07e-fc02a3e50e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:15.939365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f309cb6e-4dea-4ed9-b07e-fc02a3e50e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:15.939792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f309cb6e-4dea-4ed9-b07e-fc02a3e50e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:16.076354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f309cb6e-4dea-4ed9-b07e-fc02a3e50e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:16.084965 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f309cb6e-4dea-4ed9-b07e-fc02a3e50e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.161737 Sep 27 10:53:16.085548 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3753/7034] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:15 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:17.110340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a9fe9e66-a84b-4b00-ad27-98c75c9568a7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:17.110969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a9fe9e66-a84b-4b00-ad27-98c75c9568a7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:17.113776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:17.114477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:17.123246 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9fe9e66-a84b-4b00-ad27-98c75c9568a7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:17.123963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9fe9e66-a84b-4b00-ad27-98c75c9568a7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:17.124712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9fe9e66-a84b-4b00-ad27-98c75c9568a7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:17.125894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9fe9e66-a84b-4b00-ad27-98c75c9568a7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:17.126636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9fe9e66-a84b-4b00-ad27-98c75c9568a7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:17.127292 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a9fe9e66-a84b-4b00-ad27-98c75c9568a7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:17.273101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a9fe9e66-a84b-4b00-ad27-98c75c9568a7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:17.281701 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a9fe9e66-a84b-4b00-ad27-98c75c9568a7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.175514 Sep 27 10:53:17.282286 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3282/7035] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:17 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:18.310476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9d5488bc-4588-41b9-9dc2-9be6e9b07e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:18.311623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9d5488bc-4588-41b9-9dc2-9be6e9b07e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:18.314581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:18.325666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5488bc-4588-41b9-9dc2-9be6e9b07e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:18.326534 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5488bc-4588-41b9-9dc2-9be6e9b07e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:18.327365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5488bc-4588-41b9-9dc2-9be6e9b07e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:18.328530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5488bc-4588-41b9-9dc2-9be6e9b07e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:18.329360 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5488bc-4588-41b9-9dc2-9be6e9b07e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:18.330086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9d5488bc-4588-41b9-9dc2-9be6e9b07e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:18.472860 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9d5488bc-4588-41b9-9dc2-9be6e9b07e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:18.482082 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9d5488bc-4588-41b9-9dc2-9be6e9b07e07 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.178302 Sep 27 10:53:18.482660 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3754/7036] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:18 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:19.505603 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23f8fa6f-78ae-4b5d-bc62-108de43ac405 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:19.506619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-23f8fa6f-78ae-4b5d-bc62-108de43ac405 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:19.514275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f8fa6f-78ae-4b5d-bc62-108de43ac405 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:19.514787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f8fa6f-78ae-4b5d-bc62-108de43ac405 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:19.515233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f8fa6f-78ae-4b5d-bc62-108de43ac405 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:19.516131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f8fa6f-78ae-4b5d-bc62-108de43ac405 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:19.516891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f8fa6f-78ae-4b5d-bc62-108de43ac405 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:19.517357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-23f8fa6f-78ae-4b5d-bc62-108de43ac405 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:19.725490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-aef74eaa-733d-4e27-85f5-45165d3010c7 req-11e5e1f6-c3ce-458a-8517-f48c260cfa37 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2339e0b8-c309-4e2c-9850-26ca144391cb", "tag": "4063d595-a22f-45b7-aca9-4797b656d330"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:53:19.728636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-23f8fa6f-78ae-4b5d-bc62-108de43ac405 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:19.737330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-aef74eaa-733d-4e27-85f5-45165d3010c7 req-11e5e1f6-c3ce-458a-8517-f48c260cfa37 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:19.737793 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-aef74eaa-733d-4e27-85f5-45165d3010c7 req-11e5e1f6-c3ce-458a-8517-f48c260cfa37 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:19.738428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-aef74eaa-733d-4e27-85f5-45165d3010c7 req-11e5e1f6-c3ce-458a-8517-f48c260cfa37 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:19.739397 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-23f8fa6f-78ae-4b5d-bc62-108de43ac405 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.237732 Sep 27 10:53:19.741123 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3283/7037] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:19 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:19.750733 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-aef74eaa-733d-4e27-85f5-45165d3010c7 req-11e5e1f6-c3ce-458a-8517-f48c260cfa37 service nova] Creating event network-changed:4063d595-a22f-45b7-aca9-4797b656d330 for instance 2339e0b8-c309-4e2c-9850-26ca144391cb on nch2-master-1 Sep 27 10:53:19.756362 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-aef74eaa-733d-4e27-85f5-45165d3010c7 req-11e5e1f6-c3ce-458a-8517-f48c260cfa37 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035207 Sep 27 10:53:19.757539 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3755/7038] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:53:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:53:20.763808 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-df240768-b97d-41fc-89ff-97e75e3fd7a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:20.764483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-df240768-b97d-41fc-89ff-97e75e3fd7a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:20.772242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df240768-b97d-41fc-89ff-97e75e3fd7a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:20.772784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df240768-b97d-41fc-89ff-97e75e3fd7a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:20.773252 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df240768-b97d-41fc-89ff-97e75e3fd7a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:20.773987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df240768-b97d-41fc-89ff-97e75e3fd7a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:20.774478 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df240768-b97d-41fc-89ff-97e75e3fd7a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:20.774928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df240768-b97d-41fc-89ff-97e75e3fd7a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:20.917145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-df240768-b97d-41fc-89ff-97e75e3fd7a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:20.927507 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-df240768-b97d-41fc-89ff-97e75e3fd7a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.167534 Sep 27 10:53:20.928617 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3284/7039] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:20 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:21.960362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4ba29fff-28b7-4dda-8d54-e38bd754dbf4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:21.961147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4ba29fff-28b7-4dda-8d54-e38bd754dbf4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:21.968163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ba29fff-28b7-4dda-8d54-e38bd754dbf4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:21.968684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ba29fff-28b7-4dda-8d54-e38bd754dbf4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:21.969128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ba29fff-28b7-4dda-8d54-e38bd754dbf4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:21.969844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ba29fff-28b7-4dda-8d54-e38bd754dbf4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:21.970330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ba29fff-28b7-4dda-8d54-e38bd754dbf4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:21.970759 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ba29fff-28b7-4dda-8d54-e38bd754dbf4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:22.114695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4ba29fff-28b7-4dda-8d54-e38bd754dbf4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:22.122860 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4ba29fff-28b7-4dda-8d54-e38bd754dbf4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.170499 Sep 27 10:53:22.123453 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3756/7040] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:21 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:23.154989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6d644980-b677-481a-92a8-f99f73435659 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:23.155614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6d644980-b677-481a-92a8-f99f73435659 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:23.162777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d644980-b677-481a-92a8-f99f73435659 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:23.163283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d644980-b677-481a-92a8-f99f73435659 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:23.163735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d644980-b677-481a-92a8-f99f73435659 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:23.164447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d644980-b677-481a-92a8-f99f73435659 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:23.164979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d644980-b677-481a-92a8-f99f73435659 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:23.165413 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d644980-b677-481a-92a8-f99f73435659 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:23.304298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6d644980-b677-481a-92a8-f99f73435659 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:23.312248 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6d644980-b677-481a-92a8-f99f73435659 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.161591 Sep 27 10:53:23.312875 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3285/7041] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:23 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:24.339346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b7064e1f-af5b-4766-8109-73f7d8680459 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:24.340182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b7064e1f-af5b-4766-8109-73f7d8680459 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:24.349762 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7064e1f-af5b-4766-8109-73f7d8680459 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:24.350399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7064e1f-af5b-4766-8109-73f7d8680459 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:24.351022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7064e1f-af5b-4766-8109-73f7d8680459 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:24.351954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7064e1f-af5b-4766-8109-73f7d8680459 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:24.352571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7064e1f-af5b-4766-8109-73f7d8680459 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:24.353180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b7064e1f-af5b-4766-8109-73f7d8680459 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:24.502027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b7064e1f-af5b-4766-8109-73f7d8680459 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:24.510581 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b7064e1f-af5b-4766-8109-73f7d8680459 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.177137 Sep 27 10:53:24.511161 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3757/7042] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:24 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:25.535786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-68ed636a-97df-4fe1-b263-e0a19585fed4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:25.536945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-68ed636a-97df-4fe1-b263-e0a19585fed4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:25.544615 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68ed636a-97df-4fe1-b263-e0a19585fed4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:25.545151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68ed636a-97df-4fe1-b263-e0a19585fed4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:25.545604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68ed636a-97df-4fe1-b263-e0a19585fed4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:25.546332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68ed636a-97df-4fe1-b263-e0a19585fed4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:25.546798 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68ed636a-97df-4fe1-b263-e0a19585fed4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:25.547237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-68ed636a-97df-4fe1-b263-e0a19585fed4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:25.685349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-68ed636a-97df-4fe1-b263-e0a19585fed4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:25.693579 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-68ed636a-97df-4fe1-b263-e0a19585fed4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.161864 Sep 27 10:53:25.694130 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3286/7043] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:25 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:26.721908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-12fe1062-3498-496a-b70f-e3ea49262e03 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:26.722755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-12fe1062-3498-496a-b70f-e3ea49262e03 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:26.734763 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12fe1062-3498-496a-b70f-e3ea49262e03 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:26.735649 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12fe1062-3498-496a-b70f-e3ea49262e03 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:26.736400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12fe1062-3498-496a-b70f-e3ea49262e03 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:26.737655 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12fe1062-3498-496a-b70f-e3ea49262e03 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:26.738435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12fe1062-3498-496a-b70f-e3ea49262e03 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:26.739161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-12fe1062-3498-496a-b70f-e3ea49262e03 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:26.880161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-12fe1062-3498-496a-b70f-e3ea49262e03 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:26.889236 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-12fe1062-3498-496a-b70f-e3ea49262e03 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.174210 Sep 27 10:53:26.889848 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3758/7044] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:26 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:27.023402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f0121ff0-283c-4986-b418-50737e12ec6d req-b85122ab-c347-4fb8-b5c2-fc51c6d2ee60 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2339e0b8-c309-4e2c-9850-26ca144391cb", "name": "network-vif-plugged", "status": "completed", "tag": "4063d595-a22f-45b7-aca9-4797b656d330"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:53:27.034436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f0121ff0-283c-4986-b418-50737e12ec6d req-b85122ab-c347-4fb8-b5c2-fc51c6d2ee60 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:27.034942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f0121ff0-283c-4986-b418-50737e12ec6d req-b85122ab-c347-4fb8-b5c2-fc51c6d2ee60 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:27.035533 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f0121ff0-283c-4986-b418-50737e12ec6d req-b85122ab-c347-4fb8-b5c2-fc51c6d2ee60 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:27.047987 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f0121ff0-283c-4986-b418-50737e12ec6d req-b85122ab-c347-4fb8-b5c2-fc51c6d2ee60 service nova] Creating event network-vif-plugged:4063d595-a22f-45b7-aca9-4797b656d330 for instance 2339e0b8-c309-4e2c-9850-26ca144391cb on nch2-master-1 Sep 27 10:53:27.053391 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f0121ff0-283c-4986-b418-50737e12ec6d req-b85122ab-c347-4fb8-b5c2-fc51c6d2ee60 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033955 Sep 27 10:53:27.054174 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3287/7045] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:53:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:53:27.915493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-58cba618-5a43-416d-a3b6-04612eab3780 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:27.916063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-58cba618-5a43-416d-a3b6-04612eab3780 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:27.923436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cba618-5a43-416d-a3b6-04612eab3780 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:27.923931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cba618-5a43-416d-a3b6-04612eab3780 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:27.924375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cba618-5a43-416d-a3b6-04612eab3780 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:27.925127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cba618-5a43-416d-a3b6-04612eab3780 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:27.925589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cba618-5a43-416d-a3b6-04612eab3780 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:27.926025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-58cba618-5a43-416d-a3b6-04612eab3780 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:28.085701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-58cba618-5a43-416d-a3b6-04612eab3780 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:28.094263 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-58cba618-5a43-416d-a3b6-04612eab3780 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1358 microversion: 2.1 time: 0.184259 Sep 27 10:53:28.094857 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3759/7046] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:27 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1358 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:29.122951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-120d3976-6a53-4e14-b0e8-9cfa61b215ed tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:29.123737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-120d3976-6a53-4e14-b0e8-9cfa61b215ed tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:29.130868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120d3976-6a53-4e14-b0e8-9cfa61b215ed tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:29.131405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120d3976-6a53-4e14-b0e8-9cfa61b215ed tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:29.131887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120d3976-6a53-4e14-b0e8-9cfa61b215ed tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:29.132680 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120d3976-6a53-4e14-b0e8-9cfa61b215ed tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:29.133203 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120d3976-6a53-4e14-b0e8-9cfa61b215ed tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:29.133665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120d3976-6a53-4e14-b0e8-9cfa61b215ed tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:29.278630 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-120d3976-6a53-4e14-b0e8-9cfa61b215ed tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:29.291507 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-120d3976-6a53-4e14-b0e8-9cfa61b215ed tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1548 microversion: 2.1 time: 0.175008 Sep 27 10:53:29.292109 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3288/7047] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:29 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1548 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:29.314391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f0104f67-0270-440d-bc87-4682acaa1b60 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:29.315021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f0104f67-0270-440d-bc87-4682acaa1b60 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:29.322533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0104f67-0270-440d-bc87-4682acaa1b60 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:29.323021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0104f67-0270-440d-bc87-4682acaa1b60 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:29.323462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0104f67-0270-440d-bc87-4682acaa1b60 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:29.324171 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0104f67-0270-440d-bc87-4682acaa1b60 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:29.324631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0104f67-0270-440d-bc87-4682acaa1b60 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:29.325098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f0104f67-0270-440d-bc87-4682acaa1b60 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:29.474454 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f0104f67-0270-440d-bc87-4682acaa1b60 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:29.482599 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f0104f67-0270-440d-bc87-4682acaa1b60 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1548 microversion: 2.1 time: 0.172186 Sep 27 10:53:29.483151 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3760/7048] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:29 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1548 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:29.507087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d22eeaa8-188d-4689-bf54-a8f3cbd1fb55 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:29.512017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d22eeaa8-188d-4689-bf54-a8f3cbd1fb55 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:29.512863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d22eeaa8-188d-4689-bf54-a8f3cbd1fb55 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Searching by: {'deleted': False, 'project_id': '714fa95d9be140018a54ece37d204047'} {{(pid=145969) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:53:29.518540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d22eeaa8-188d-4689-bf54-a8f3cbd1fb55 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:29.519035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d22eeaa8-188d-4689-bf54-a8f3cbd1fb55 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:29.519488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d22eeaa8-188d-4689-bf54-a8f3cbd1fb55 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:29.520232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d22eeaa8-188d-4689-bf54-a8f3cbd1fb55 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:29.520708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d22eeaa8-188d-4689-bf54-a8f3cbd1fb55 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:29.521166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d22eeaa8-188d-4689-bf54-a8f3cbd1fb55 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:29.549593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.multi_cell_list [None req-d22eeaa8-188d-4689-bf54-a8f3cbd1fb55 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145969) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:53:29.577209 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d22eeaa8-188d-4689-bf54-a8f3cbd1fb55 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.074914 Sep 27 10:53:29.577793 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3289/7049] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:53:29 2022] GET /compute/v2.1/servers => generated 343 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:53:29.589929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e629b2d6-6304-4ca7-9f38-4e9775dc728d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:29.590572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e629b2d6-6304-4ca7-9f38-4e9775dc728d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:29.596888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e629b2d6-6304-4ca7-9f38-4e9775dc728d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:29.597350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e629b2d6-6304-4ca7-9f38-4e9775dc728d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:29.598042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e629b2d6-6304-4ca7-9f38-4e9775dc728d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:29.598910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e629b2d6-6304-4ca7-9f38-4e9775dc728d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:29.599676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e629b2d6-6304-4ca7-9f38-4e9775dc728d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:29.600080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e629b2d6-6304-4ca7-9f38-4e9775dc728d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:29.741939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e629b2d6-6304-4ca7-9f38-4e9775dc728d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:29.750274 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e629b2d6-6304-4ca7-9f38-4e9775dc728d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1548 microversion: 2.1 time: 0.164358 Sep 27 10:53:29.750872 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3761/7050] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:29 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1548 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 10:53:31.348454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-826e7898-c713-46c0-b0c5-d273a4a671a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ff248593-4867-4300-83fa-11e79e5e1405"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:53:31.350889 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-826e7898-c713-46c0-b0c5-d273a4a671a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:31.358465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-826e7898-c713-46c0-b0c5-d273a4a671a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:31.358966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-826e7898-c713-46c0-b0c5-d273a4a671a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:31.359417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-826e7898-c713-46c0-b0c5-d273a4a671a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:31.360123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-826e7898-c713-46c0-b0c5-d273a4a671a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:31.360593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-826e7898-c713-46c0-b0c5-d273a4a671a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:31.361062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-826e7898-c713-46c0-b0c5-d273a4a671a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:31.650525 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-826e7898-c713-46c0-b0c5-d273a4a671a5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.305936 Sep 27 10:53:31.651151 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3290/7051] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:53:31 2022] POST /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/os-volume_attachments => generated 194 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:53:37.384292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fdf35812-6b95-43a6-8658-7ca8491e740d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:37.384923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fdf35812-6b95-43a6-8658-7ca8491e740d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:37.388053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:37.394105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdf35812-6b95-43a6-8658-7ca8491e740d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:37.394625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdf35812-6b95-43a6-8658-7ca8491e740d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:37.395086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdf35812-6b95-43a6-8658-7ca8491e740d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:37.395797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdf35812-6b95-43a6-8658-7ca8491e740d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:37.396261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdf35812-6b95-43a6-8658-7ca8491e740d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:37.396708 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fdf35812-6b95-43a6-8658-7ca8491e740d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:37.399840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=69 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:37.536321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fdf35812-6b95-43a6-8658-7ca8491e740d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:37.544956 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fdf35812-6b95-43a6-8658-7ca8491e740d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1594 microversion: 2.1 time: 0.164859 Sep 27 10:53:37.545544 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3762/7052] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:37 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1594 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:53:39.590026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1c52cc04-3d0f-4726-98aa-e61001e47981 req-c25e312d-a428-4db0-82bf-8fdee518c8c0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2339e0b8-c309-4e2c-9850-26ca144391cb", "tag": "4063d595-a22f-45b7-aca9-4797b656d330"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:53:39.602192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:39.603376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=42,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:39.616168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1c52cc04-3d0f-4726-98aa-e61001e47981 req-c25e312d-a428-4db0-82bf-8fdee518c8c0 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:39.616725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1c52cc04-3d0f-4726-98aa-e61001e47981 req-c25e312d-a428-4db0-82bf-8fdee518c8c0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:39.617300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1c52cc04-3d0f-4726-98aa-e61001e47981 req-c25e312d-a428-4db0-82bf-8fdee518c8c0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:39.623265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=84,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:53:39.635546 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1c52cc04-3d0f-4726-98aa-e61001e47981 req-c25e312d-a428-4db0-82bf-8fdee518c8c0 service nova] Creating event network-changed:4063d595-a22f-45b7-aca9-4797b656d330 for instance 2339e0b8-c309-4e2c-9850-26ca144391cb on nch2-master-1 Sep 27 10:53:39.641278 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1c52cc04-3d0f-4726-98aa-e61001e47981 req-c25e312d-a428-4db0-82bf-8fdee518c8c0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.055540 Sep 27 10:53:39.642308 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3291/7053] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:53:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:53:40.719735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75e14517-ab30-4fe7-bec4-783e48260639 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1476663105"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:53:40.720319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-75e14517-ab30-4fe7-bec4-783e48260639 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:40.727461 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75e14517-ab30-4fe7-bec4-783e48260639 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:40.727919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75e14517-ab30-4fe7-bec4-783e48260639 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:40.728356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75e14517-ab30-4fe7-bec4-783e48260639 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:40.729425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75e14517-ab30-4fe7-bec4-783e48260639 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:40.729887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75e14517-ab30-4fe7-bec4-783e48260639 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:40.730309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75e14517-ab30-4fe7-bec4-783e48260639 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:40.878656 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.network.security_group_api [None req-75e14517-ab30-4fe7-bec4-783e48260639 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Adding security group 31a21e92-bb41-4cde-ab56-73811933762a to port 4063d595-a22f-45b7-aca9-4797b656d330 Sep 27 10:53:41.220003 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-75e14517-ab30-4fe7-bec4-783e48260639 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/action" status: 202 len: 0 microversion: 2.1 time: 0.504282 Sep 27 10:53:41.220621 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3763/7054] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:53:40 2022] POST /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/action => generated 0 bytes in 506 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:53:41.231802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c908f93f-70fe-49a2-ad91-aa85d18f7ece tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:53:41.232613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c908f93f-70fe-49a2-ad91-aa85d18f7ece tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:53:41.236582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6ec47bb1-d707-4e11-9b5b-91f522dfeacf req-baf7dd2d-3fff-4dbd-98a6-47fd398ee67c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2339e0b8-c309-4e2c-9850-26ca144391cb", "tag": "4063d595-a22f-45b7-aca9-4797b656d330"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:53:41.241419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c908f93f-70fe-49a2-ad91-aa85d18f7ece tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:41.242001 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c908f93f-70fe-49a2-ad91-aa85d18f7ece tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:41.242524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c908f93f-70fe-49a2-ad91-aa85d18f7ece tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:41.243466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c908f93f-70fe-49a2-ad91-aa85d18f7ece tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:41.244176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c908f93f-70fe-49a2-ad91-aa85d18f7ece tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:41.244728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c908f93f-70fe-49a2-ad91-aa85d18f7ece tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:41.248975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6ec47bb1-d707-4e11-9b5b-91f522dfeacf req-baf7dd2d-3fff-4dbd-98a6-47fd398ee67c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:53:41.249859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6ec47bb1-d707-4e11-9b5b-91f522dfeacf req-baf7dd2d-3fff-4dbd-98a6-47fd398ee67c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:53:41.250310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6ec47bb1-d707-4e11-9b5b-91f522dfeacf req-baf7dd2d-3fff-4dbd-98a6-47fd398ee67c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:53:41.263417 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6ec47bb1-d707-4e11-9b5b-91f522dfeacf req-baf7dd2d-3fff-4dbd-98a6-47fd398ee67c service nova] Creating event network-changed:4063d595-a22f-45b7-aca9-4797b656d330 for instance 2339e0b8-c309-4e2c-9850-26ca144391cb on nch2-master-1 Sep 27 10:53:41.271385 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6ec47bb1-d707-4e11-9b5b-91f522dfeacf req-baf7dd2d-3fff-4dbd-98a6-47fd398ee67c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040563 Sep 27 10:53:41.272079 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3764/7055] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:53:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:53:41.514325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c908f93f-70fe-49a2-ad91-aa85d18f7ece tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:53:41.524126 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c908f93f-70fe-49a2-ad91-aa85d18f7ece tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1760 microversion: 2.1 time: 0.298297 Sep 27 10:53:41.524849 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3292/7056] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:53:41 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1760 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:56:34.925014 ncdv-master-1 devstack@n-api.service[145968]: ERROR oslo.messaging._drivers.impl_rabbit [-] [caeac69d-7b1f-444e-94c7-8c629d37bafc] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 27 10:56:34.926060 ncdv-master-1 devstack@n-api.service[145968]: ERROR oslo.messaging._drivers.impl_rabbit [-] [eb0e6821-e1de-45db-a339-eb2fe392322e] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 27 10:56:34.926643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=5 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:56:34.927397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=11 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:56:34.928808 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Sep 27 10:56:34.928974 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 10:56:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/action (ip 10.222.0.26) !!! Sep 27 10:56:34.930553 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 10:56:34.930709 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 10:56:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/action (ip 10.222.0.26) !!! Sep 27 10:56:34.932820 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Sep 27 10:56:34.932938 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 10:56:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/action (ip 10.222.0.26) !!! Sep 27 10:56:34.945293 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d3c5db2-ca99-4f89-9c75-b658576638a3 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'action', calling method: >, body: {"reboot": {"type": "HARD"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:56:34.947676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5d3c5db2-ca99-4f89-9c75-b658576638a3 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:56:34.961128 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d3c5db2-ca99-4f89-9c75-b658576638a3 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:56:34.961625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d3c5db2-ca99-4f89-9c75-b658576638a3 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:56:34.962068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d3c5db2-ca99-4f89-9c75-b658576638a3 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:56:34.962783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d3c5db2-ca99-4f89-9c75-b658576638a3 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:56:34.963245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d3c5db2-ca99-4f89-9c75-b658576638a3 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:56:34.963675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d3c5db2-ca99-4f89-9c75-b658576638a3 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:56:35.049876 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5d3c5db2-ca99-4f89-9c75-b658576638a3 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/action" status: 202 len: 0 microversion: 2.1 time: 0.126992 Sep 27 10:56:35.050578 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3765/7057] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:56:34 2022] POST /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/action => generated 0 bytes in 129 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:56:35.058973 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [-] [1986a9fd-1b4d-44f5-a8ac-515eb5bb435f] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 27 10:56:35.060182 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [-] [d9169ee3-829d-41de-8bb4-82814efca059] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 27 10:56:35.060727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:56:35.061626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=7 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:56:35.062803 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Sep 27 10:56:35.062924 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 10:56:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb (ip 10.222.0.26) !!! Sep 27 10:56:35.066015 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Sep 27 10:56:35.066353 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 10:56:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb (ip 10.222.0.26) !!! Sep 27 10:56:35.068169 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 10:56:35.068314 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 10:56:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb (ip 10.222.0.26) !!! Sep 27 10:56:35.077834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-af7b646d-b323-4a2f-a080-3b5bfdc86093 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:56:35.078461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-af7b646d-b323-4a2f-a080-3b5bfdc86093 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:56:35.099513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7b646d-b323-4a2f-a080-3b5bfdc86093 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:56:35.100017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7b646d-b323-4a2f-a080-3b5bfdc86093 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:56:35.100472 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7b646d-b323-4a2f-a080-3b5bfdc86093 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:56:35.101231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7b646d-b323-4a2f-a080-3b5bfdc86093 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:56:35.101707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7b646d-b323-4a2f-a080-3b5bfdc86093 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:56:35.102150 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-af7b646d-b323-4a2f-a080-3b5bfdc86093 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:56:35.496279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-af7b646d-b323-4a2f-a080-3b5bfdc86093 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:56:35.508057 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-af7b646d-b323-4a2f-a080-3b5bfdc86093 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1762 microversion: 2.1 time: 0.450777 Sep 27 10:56:35.509131 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3293/7058] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:56:35 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1762 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:56:36.536276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-808c84ba-11f6-489c-b799-25a52ded2670 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:56:36.537029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-808c84ba-11f6-489c-b799-25a52ded2670 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:56:36.545366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-808c84ba-11f6-489c-b799-25a52ded2670 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:56:36.545858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-808c84ba-11f6-489c-b799-25a52ded2670 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:56:36.546295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-808c84ba-11f6-489c-b799-25a52ded2670 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:56:36.546993 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-808c84ba-11f6-489c-b799-25a52ded2670 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:56:36.547451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-808c84ba-11f6-489c-b799-25a52ded2670 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:56:36.547912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-808c84ba-11f6-489c-b799-25a52ded2670 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:56:36.552599 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] [eb0e6821-e1de-45db-a339-eb2fe392322e] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 57254. Sep 27 10:56:36.555716 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] [caeac69d-7b1f-444e-94c7-8c629d37bafc] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 57252. Sep 27 10:56:36.840989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-808c84ba-11f6-489c-b799-25a52ded2670 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:56:36.851325 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-808c84ba-11f6-489c-b799-25a52ded2670 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1760 microversion: 2.1 time: 0.322185 Sep 27 10:56:36.851921 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3766/7059] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:56:36 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1760 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:13.683070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:13.683853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:13.695224 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] [d9169ee3-829d-41de-8bb4-82814efca059] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 57332. Sep 27 10:58:13.697165 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] [1986a9fd-1b4d-44f5-a8ac-515eb5bb435f] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 57330. Sep 27 10:58:13.753590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-de090ced-b49d-431c-8b4b-66f2608c0d20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1476663105"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:13.754156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-de090ced-b49d-431c-8b4b-66f2608c0d20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:13.763297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de090ced-b49d-431c-8b4b-66f2608c0d20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:13.763805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de090ced-b49d-431c-8b4b-66f2608c0d20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:13.764258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de090ced-b49d-431c-8b4b-66f2608c0d20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:13.764991 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de090ced-b49d-431c-8b4b-66f2608c0d20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:13.765465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de090ced-b49d-431c-8b4b-66f2608c0d20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:13.765906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de090ced-b49d-431c-8b4b-66f2608c0d20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:13.914025 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.network.security_group_api [None req-de090ced-b49d-431c-8b4b-66f2608c0d20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Removing security group 31a21e92-bb41-4cde-ab56-73811933762a from port 4063d595-a22f-45b7-aca9-4797b656d330 Sep 27 10:58:14.221276 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-de090ced-b49d-431c-8b4b-66f2608c0d20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/action" status: 202 len: 0 microversion: 2.1 time: 0.543340 Sep 27 10:58:14.221902 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3294/7060] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:58:13 2022] POST /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/action => generated 0 bytes in 545 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:58:14.286390 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:14.287689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=16,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:14.357989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-eee1e8a3-1433-4f66-b4d6-bf52dd8a9790 req-28a59f9f-7b7c-4b90-b82e-d52e4c8b9ae4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2339e0b8-c309-4e2c-9850-26ca144391cb", "tag": "4063d595-a22f-45b7-aca9-4797b656d330"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:14.370859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-eee1e8a3-1433-4f66-b4d6-bf52dd8a9790 req-28a59f9f-7b7c-4b90-b82e-d52e4c8b9ae4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:14.371341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-eee1e8a3-1433-4f66-b4d6-bf52dd8a9790 req-28a59f9f-7b7c-4b90-b82e-d52e4c8b9ae4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:14.372103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-eee1e8a3-1433-4f66-b4d6-bf52dd8a9790 req-28a59f9f-7b7c-4b90-b82e-d52e4c8b9ae4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:14.384174 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-eee1e8a3-1433-4f66-b4d6-bf52dd8a9790 req-28a59f9f-7b7c-4b90-b82e-d52e4c8b9ae4 service nova] Creating event network-changed:4063d595-a22f-45b7-aca9-4797b656d330 for instance 2339e0b8-c309-4e2c-9850-26ca144391cb on nch2-master-1 Sep 27 10:58:14.390298 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-eee1e8a3-1433-4f66-b4d6-bf52dd8a9790 req-28a59f9f-7b7c-4b90-b82e-d52e4c8b9ae4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.105875 Sep 27 10:58:14.390908 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3767/7061] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:58:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:58:15.461885 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:15.462470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:15.470177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:15.470673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:15.471169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:15.472169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:15.472587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:15.473040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:15.514454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-dd2c2bf0-20d9-4929-9bd7-9d69d5f9ea87 req-5c98b9a4-2150-4fc8-afd8-acee81ff743a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2339e0b8-c309-4e2c-9850-26ca144391cb", "tag": "4063d595-a22f-45b7-aca9-4797b656d330"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:15.529567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dd2c2bf0-20d9-4929-9bd7-9d69d5f9ea87 req-5c98b9a4-2150-4fc8-afd8-acee81ff743a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:15.530064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dd2c2bf0-20d9-4929-9bd7-9d69d5f9ea87 req-5c98b9a4-2150-4fc8-afd8-acee81ff743a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:15.530751 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-dd2c2bf0-20d9-4929-9bd7-9d69d5f9ea87 req-5c98b9a4-2150-4fc8-afd8-acee81ff743a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:15.542956 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-dd2c2bf0-20d9-4929-9bd7-9d69d5f9ea87 req-5c98b9a4-2150-4fc8-afd8-acee81ff743a service nova] Creating event network-changed:4063d595-a22f-45b7-aca9-4797b656d330 for instance 2339e0b8-c309-4e2c-9850-26ca144391cb on nch2-master-1 Sep 27 10:58:15.549612 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-dd2c2bf0-20d9-4929-9bd7-9d69d5f9ea87 req-5c98b9a4-2150-4fc8-afd8-acee81ff743a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.098763 Sep 27 10:58:15.550831 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3295/7062] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:58:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:58:15.571233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lazy-loading 'services' on Instance uuid 2339e0b8-c309-4e2c-9850-26ca144391cb {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 10:58:15.656735 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a8b7f9c-3414-4865-846a-bb2ce4a7204f tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "DELETE /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/os-volume_attachments/ff248593-4867-4300-83fa-11e79e5e1405" status: 202 len: 0 microversion: 2.1 time: 0.203108 Sep 27 10:58:15.657671 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3768/7063] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:58:15 2022] DELETE /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/os-volume_attachments/ff248593-4867-4300-83fa-11e79e5e1405 => generated 0 bytes in 205 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 10:58:15.667957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-90874768-8764-40f6-b9d2-01460626ce96 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:15.670801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-90874768-8764-40f6-b9d2-01460626ce96 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:15.682659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90874768-8764-40f6-b9d2-01460626ce96 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:15.683473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90874768-8764-40f6-b9d2-01460626ce96 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:15.684112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90874768-8764-40f6-b9d2-01460626ce96 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:15.685072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90874768-8764-40f6-b9d2-01460626ce96 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:15.685714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90874768-8764-40f6-b9d2-01460626ce96 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:15.686343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-90874768-8764-40f6-b9d2-01460626ce96 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:15.728997 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-90874768-8764-40f6-b9d2-01460626ce96 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065694 Sep 27 10:58:15.729698 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3296/7064] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:58:15 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:58:16.745827 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-830c3516-45d6-4991-8fc5-222b5bbecc6a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:16.748245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-830c3516-45d6-4991-8fc5-222b5bbecc6a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:16.758086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-830c3516-45d6-4991-8fc5-222b5bbecc6a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:16.758603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-830c3516-45d6-4991-8fc5-222b5bbecc6a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:16.759024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-830c3516-45d6-4991-8fc5-222b5bbecc6a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:16.759904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-830c3516-45d6-4991-8fc5-222b5bbecc6a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:16.760213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-830c3516-45d6-4991-8fc5-222b5bbecc6a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:16.760683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-830c3516-45d6-4991-8fc5-222b5bbecc6a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:16.793949 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-830c3516-45d6-4991-8fc5-222b5bbecc6a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.054527 Sep 27 10:58:16.794744 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3769/7065] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 10:58:16 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/os-volume_attachments => generated 25 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 10:58:16.808219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:16.808886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:16.816193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:16.816692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:16.817188 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:16.817886 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:16.818367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:16.818819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:16.896711 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] HTTP exception thrown: Instance 2339e0b8-c309-4e2c-9850-26ca144391cb is not attached to volume ff248593-4867-4300-83fa-11e79e5e1405 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Request method failure captured: Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: request: DELETE /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/os-volume_attachments/ff248593-4867-4300-83fa-11e79e5e1405 HTTP/1.1 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtYCD54ftHDthm7BCdV6AkQb8HODztxEb10C_SCo0B541ecNSCGEym3jk9y9sS1-_6UOL0lMYfx5iyQtcgBulEcndBXxotppQB_qD75QcYmJXla1b-2-rjVLpqTjTA3li4n_CEXD4Cc26PZ_YhunxBrjuC1a9L4ks6u1DI3EakP93Ac Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 714fa95d9be140018a54ece37d204047 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestMinimumBasicScenario-1586443371 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/714fa95d9be140018a54ece37d204047"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/714fa95d9be140018a54ece37d204047"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/714fa95d9be140018a54ece37d204047"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_714fa95d9be140018a54ece37d204047"}]}] Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestMinimumBasicScenario-1586443371 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 714fa95d9be140018a54ece37d204047 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1586443371 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestMinimumBasicScenario-1586443371-project Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 906ca88c8f544940b06705711ae7f1af Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestMinimumBasicScenario-1586443371-project Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 2339e0b8-c309-4e2c-9850-26ca144391cb is not attached to volume ff248593-4867-4300-83fa-11e79e5e1405 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'server_id': '2339e0b8-c309-4e2c-9850-26ca144391cb', 'id': 'ff248593-4867-4300-83fa-11e79e5e1405'} Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id ff248593-4867-4300-83fa-11e79e5e1405. Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2339e0b8-c309-4e2c-9850-26ca144391cb is not attached to volume ff248593-4867-4300-83fa-11e79e5e1405 Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:58:16.905936 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:58:16.911474 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:58:16.911474 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:58:16.911474 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2339e0b8-c309-4e2c-9850-26ca144391cb is not attached to volume ff248593-4867-4300-83fa-11e79e5e1405 Sep 27 10:58:16.911474 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 10:58:16.911474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Returning 404 to user: Instance 2339e0b8-c309-4e2c-9850-26ca144391cb is not attached to volume ff248593-4867-4300-83fa-11e79e5e1405 {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:58:16.911474 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ea5e922-2fd0-420f-bcec-7ba3eba60e09 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "DELETE /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/os-volume_attachments/ff248593-4867-4300-83fa-11e79e5e1405" status: 404 len: 154 microversion: 2.1 time: 0.105368 Sep 27 10:58:16.911474 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3297/7066] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 10:58:16 2022] DELETE /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb/os-volume_attachments/ff248593-4867-4300-83fa-11e79e5e1405 => generated 154 bytes in 107 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:58:18.073905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13509691-6a41-461b-be46-fee30acb19c9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:18.074434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-13509691-6a41-461b-be46-fee30acb19c9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:18.081760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13509691-6a41-461b-be46-fee30acb19c9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:18.082252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13509691-6a41-461b-be46-fee30acb19c9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:18.082693 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13509691-6a41-461b-be46-fee30acb19c9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:18.083864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13509691-6a41-461b-be46-fee30acb19c9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:18.083864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13509691-6a41-461b-be46-fee30acb19c9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:18.084270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13509691-6a41-461b-be46-fee30acb19c9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:18.110609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-13509691-6a41-461b-be46-fee30acb19c9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 10:58:18.194872 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13509691-6a41-461b-be46-fee30acb19c9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "DELETE /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 204 len: 0 microversion: 2.1 time: 0.124714 Sep 27 10:58:18.195810 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3770/7067] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 10:58:18 2022] DELETE /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 10:58:18.204242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bf0ede3e-57d2-4ba4-9d0a-7b09de1fc5fa tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:18.204922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bf0ede3e-57d2-4ba4-9d0a-7b09de1fc5fa tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:18.213581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf0ede3e-57d2-4ba4-9d0a-7b09de1fc5fa tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:18.214420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf0ede3e-57d2-4ba4-9d0a-7b09de1fc5fa tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:18.215061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf0ede3e-57d2-4ba4-9d0a-7b09de1fc5fa tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:18.215819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf0ede3e-57d2-4ba4-9d0a-7b09de1fc5fa tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:18.216301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf0ede3e-57d2-4ba4-9d0a-7b09de1fc5fa tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:18.216780 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bf0ede3e-57d2-4ba4-9d0a-7b09de1fc5fa tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:18.361854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bf0ede3e-57d2-4ba4-9d0a-7b09de1fc5fa tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:18.370611 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bf0ede3e-57d2-4ba4-9d0a-7b09de1fc5fa tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1554 microversion: 2.1 time: 0.170360 Sep 27 10:58:18.371205 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3298/7068] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:18 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1554 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:19.400204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82e47d97-a153-411c-afff-d94b425a4cef tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:19.401609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-82e47d97-a153-411c-afff-d94b425a4cef tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:19.415395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e47d97-a153-411c-afff-d94b425a4cef tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:19.416306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e47d97-a153-411c-afff-d94b425a4cef tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:19.416943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e47d97-a153-411c-afff-d94b425a4cef tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:19.417661 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e47d97-a153-411c-afff-d94b425a4cef tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:19.418121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e47d97-a153-411c-afff-d94b425a4cef tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:19.418547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82e47d97-a153-411c-afff-d94b425a4cef tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:19.554644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-82e47d97-a153-411c-afff-d94b425a4cef tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:19.565743 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-82e47d97-a153-411c-afff-d94b425a4cef tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1554 microversion: 2.1 time: 0.172603 Sep 27 10:58:19.566331 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3771/7069] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:19 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1554 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:20.590778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f950cae2-5b5b-47fd-a433-3526c6387edc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:20.591519 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f950cae2-5b5b-47fd-a433-3526c6387edc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:20.591971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7fd43369-7058-44eb-9259-7d90aa657415 req-e093f02d-4a48-4d4a-b5e4-8c0406646b24 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2339e0b8-c309-4e2c-9850-26ca144391cb", "name": "network-vif-deleted", "tag": "4063d595-a22f-45b7-aca9-4797b656d330"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:20.599974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f950cae2-5b5b-47fd-a433-3526c6387edc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:20.600503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f950cae2-5b5b-47fd-a433-3526c6387edc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:20.600999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f950cae2-5b5b-47fd-a433-3526c6387edc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:20.601762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f950cae2-5b5b-47fd-a433-3526c6387edc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:20.602248 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f950cae2-5b5b-47fd-a433-3526c6387edc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:20.602707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f950cae2-5b5b-47fd-a433-3526c6387edc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:20.611042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7fd43369-7058-44eb-9259-7d90aa657415 req-e093f02d-4a48-4d4a-b5e4-8c0406646b24 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:20.611042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7fd43369-7058-44eb-9259-7d90aa657415 req-e093f02d-4a48-4d4a-b5e4-8c0406646b24 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:20.611042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7fd43369-7058-44eb-9259-7d90aa657415 req-e093f02d-4a48-4d4a-b5e4-8c0406646b24 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:20.620511 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7fd43369-7058-44eb-9259-7d90aa657415 req-e093f02d-4a48-4d4a-b5e4-8c0406646b24 service nova] Creating event network-vif-deleted:4063d595-a22f-45b7-aca9-4797b656d330 for instance 2339e0b8-c309-4e2c-9850-26ca144391cb on nch2-master-1 Sep 27 10:58:20.625905 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7fd43369-7058-44eb-9259-7d90aa657415 req-e093f02d-4a48-4d4a-b5e4-8c0406646b24 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038036 Sep 27 10:58:20.626510 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3772/7070] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:58:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:58:20.685068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f950cae2-5b5b-47fd-a433-3526c6387edc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:20.693667 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f950cae2-5b5b-47fd-a433-3526c6387edc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 200 len: 1512 microversion: 2.1 time: 0.107463 Sep 27 10:58:20.694500 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3299/7071] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:20 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 1512 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:21.720326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:21.720990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: 2339e0b8-c309-4e2c-9850-26ca144391cb] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:21.727821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:21.728358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:21.728786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:21.729578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:21.730038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:21.730497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:21.745027 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] HTTP exception thrown: Instance 2339e0b8-c309-4e2c-9850-26ca144391cb could not be found. Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Request method failure captured: Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb HTTP/1.1 Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtYCD54ftHDthm7BCdV6AkQb8HODztxEb10C_SCo0B541ecNSCGEym3jk9y9sS1-_6UOL0lMYfx5iyQtcgBulEcndBXxotppQB_qD75QcYmJXla1b-2-rjVLpqTjTA3li4n_CEXD4Cc26PZ_YhunxBrjuC1a9L4ks6u1DI3EakP93Ac Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 714fa95d9be140018a54ece37d204047 Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestMinimumBasicScenario-1586443371 Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/714fa95d9be140018a54ece37d204047"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/714fa95d9be140018a54ece37d204047"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/714fa95d9be140018a54ece37d204047"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_714fa95d9be140018a54ece37d204047"}]}] Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestMinimumBasicScenario-1586443371 Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 714fa95d9be140018a54ece37d204047 Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1586443371 Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestMinimumBasicScenario-1586443371-project Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 906ca88c8f544940b06705711ae7f1af Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestMinimumBasicScenario-1586443371-project Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 2339e0b8-c309-4e2c-9850-26ca144391cb could not be found. Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '2339e0b8-c309-4e2c-9850-26ca144391cb'} Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2339e0b8-c309-4e2c-9850-26ca144391cb could not be found. Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2339e0b8-c309-4e2c-9850-26ca144391cb could not be found. Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 10:58:21.746681 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:58:21.751959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 10:58:21.751959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 10:58:21.751959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 10:58:21.751959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 10:58:21.751959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 10:58:21.751959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2339e0b8-c309-4e2c-9850-26ca144391cb could not be found. Sep 27 10:58:21.751959 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 10:58:21.751959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Returning 404 to user: Instance 2339e0b8-c309-4e2c-9850-26ca144391cb could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 10:58:21.751959 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ecb6babb-6244-4dea-b2e7-e780e94c2af9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb" status: 404 len: 111 microversion: 2.1 time: 0.033139 Sep 27 10:58:21.751959 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3773/7072] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:21 2022] GET /compute/v2.1/servers/2339e0b8-c309-4e2c-9850-26ca144391cb => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 10:58:21.757098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-23918a72-29e8-46a7-9371-769205c00e97 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '.version_select at 0x7f7f62acf310>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:21.785811 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-23918a72-29e8-46a7-9371-769205c00e97 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-570772346" status: 202 len: 0 microversion: 2.1 time: 0.032610 Sep 27 10:58:21.786405 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3300/7073] 10.222.0.26 () {58 vars in 1269 bytes} [Tue Sep 27 10:58:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-570772346 => generated 0 bytes in 34 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:58:31.145980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1fca0360-6754-41e3-b5c0-50e66b6ef073 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'create', calling method: .version_select at 0x7f7f6267b310>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1247088124", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDJ2pE8Y/xP8WKoBp0X9Y0cukkRiSPnzf4Uy3aDB7RAm/hkUzioIR20yZXtmmSaHyAV5ad52XTdR8B7LW5aRrfFBqTzYHYWP6pstZ3QWu1K2Zl6zxSHD48BLBG/GgqtdAw=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:31.167593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-1fca0360-6754-41e3-b5c0-50e66b6ef073 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Getting quotas for project 714fa95d9be140018a54ece37d204047. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:58:31.172873 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-1fca0360-6754-41e3-b5c0-50e66b6ef073 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Getting quotas for user 906ca88c8f544940b06705711ae7f1af and project 714fa95d9be140018a54ece37d204047. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:58:31.202317 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1fca0360-6754-41e3-b5c0-50e66b6ef073 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.060295 Sep 27 10:58:31.202910 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3774/7074] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 10:58:31 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:58:31.264315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=30 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:31.268618 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1399934434", "imageRef": "bd7e19d1-f06f-4e7e-a8f8-506a39397441", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1247088124", "networks": [{"uuid": "bc6ee60c-393d-4434-9103-3f26aa0212f4"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:31.321706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] validate_networks() for [('bc6ee60c-393d-4434-9103-3f26aa0212f4', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 10:58:31.781713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 10:58:31.782133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 10:58:31.782629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 10:58:31.783057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 10:58:31.783829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 10:58:31.796171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:31.796686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:31.797173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:31.797934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:31.798408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:31.798853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:31.819362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Getting quotas for project 714fa95d9be140018a54ece37d204047. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 10:58:31.825174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Getting quotas for user 906ca88c8f544940b06705711ae7f1af and project 714fa95d9be140018a54ece37d204047. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 10:58:31.837629 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 10:58:31.840152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='bd7e19d1-f06f-4e7e-a8f8-506a39397441',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 10:58:31.892744 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-32b077ec-b2bc-4322-a08d-ac6278e736c4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.630161 Sep 27 10:58:31.893317 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3301/7075] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 10:58:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 632 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 10:58:31.901821 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=41,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:31.906327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d92778c2-4981-42fa-a077-0d2604c31a22 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:31.906946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d92778c2-4981-42fa-a077-0d2604c31a22 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:32.220245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d92778c2-4981-42fa-a077-0d2604c31a22 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:32.228237 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d92778c2-4981-42fa-a077-0d2604c31a22 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1259 microversion: 2.1 time: 0.327616 Sep 27 10:58:32.228844 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3775/7076] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:31 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1259 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:33.251473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7fbf8910-b323-4d70-9382-49a1c3a7edf0 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:33.252085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7fbf8910-b323-4d70-9382-49a1c3a7edf0 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:33.260554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fbf8910-b323-4d70-9382-49a1c3a7edf0 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:33.261332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fbf8910-b323-4d70-9382-49a1c3a7edf0 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:33.262417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fbf8910-b323-4d70-9382-49a1c3a7edf0 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:33.263355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fbf8910-b323-4d70-9382-49a1c3a7edf0 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:33.263985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fbf8910-b323-4d70-9382-49a1c3a7edf0 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:33.264430 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fbf8910-b323-4d70-9382-49a1c3a7edf0 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:33.362718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7fbf8910-b323-4d70-9382-49a1c3a7edf0 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:33.371233 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7fbf8910-b323-4d70-9382-49a1c3a7edf0 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1317 microversion: 2.1 time: 0.123631 Sep 27 10:58:33.371797 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3302/7077] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:33 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1317 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:34.396781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3cc6937e-f6ae-43b0-bf1a-8b21052ec035 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:34.397351 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3cc6937e-f6ae-43b0-bf1a-8b21052ec035 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:34.404716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cc6937e-f6ae-43b0-bf1a-8b21052ec035 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:34.405484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cc6937e-f6ae-43b0-bf1a-8b21052ec035 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:34.406111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cc6937e-f6ae-43b0-bf1a-8b21052ec035 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:34.407156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cc6937e-f6ae-43b0-bf1a-8b21052ec035 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:34.407629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cc6937e-f6ae-43b0-bf1a-8b21052ec035 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:34.408411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cc6937e-f6ae-43b0-bf1a-8b21052ec035 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:34.571744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3cc6937e-f6ae-43b0-bf1a-8b21052ec035 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:34.580833 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3cc6937e-f6ae-43b0-bf1a-8b21052ec035 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.189075 Sep 27 10:58:34.581757 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3776/7078] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:34 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:35.606500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-25b31259-8a5d-4f11-a451-0fd49209d1f6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:35.608143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-25b31259-8a5d-4f11-a451-0fd49209d1f6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:35.617151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25b31259-8a5d-4f11-a451-0fd49209d1f6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:35.617885 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25b31259-8a5d-4f11-a451-0fd49209d1f6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:35.618345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25b31259-8a5d-4f11-a451-0fd49209d1f6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:35.619409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25b31259-8a5d-4f11-a451-0fd49209d1f6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:35.619762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25b31259-8a5d-4f11-a451-0fd49209d1f6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:35.620240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-25b31259-8a5d-4f11-a451-0fd49209d1f6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:35.788374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-25b31259-8a5d-4f11-a451-0fd49209d1f6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:35.797533 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-25b31259-8a5d-4f11-a451-0fd49209d1f6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.196223 Sep 27 10:58:35.798894 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3303/7079] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:35 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:36.822403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e25ded6c-0143-418d-bc51-118de4f15e95 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:36.823062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e25ded6c-0143-418d-bc51-118de4f15e95 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:36.829832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e25ded6c-0143-418d-bc51-118de4f15e95 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:36.830328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e25ded6c-0143-418d-bc51-118de4f15e95 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:36.830774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e25ded6c-0143-418d-bc51-118de4f15e95 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:36.831494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e25ded6c-0143-418d-bc51-118de4f15e95 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:36.832098 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e25ded6c-0143-418d-bc51-118de4f15e95 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:36.832535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e25ded6c-0143-418d-bc51-118de4f15e95 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:36.984285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e25ded6c-0143-418d-bc51-118de4f15e95 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:36.998581 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e25ded6c-0143-418d-bc51-118de4f15e95 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.180221 Sep 27 10:58:36.999192 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3777/7080] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:36 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:38.022772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b55ccf70-1cfc-4844-b21f-bbb0ee3942e6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:38.023632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b55ccf70-1cfc-4844-b21f-bbb0ee3942e6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:38.032401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b55ccf70-1cfc-4844-b21f-bbb0ee3942e6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:38.032936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b55ccf70-1cfc-4844-b21f-bbb0ee3942e6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:38.033948 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b55ccf70-1cfc-4844-b21f-bbb0ee3942e6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:38.034914 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b55ccf70-1cfc-4844-b21f-bbb0ee3942e6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:38.035776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b55ccf70-1cfc-4844-b21f-bbb0ee3942e6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:38.036386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b55ccf70-1cfc-4844-b21f-bbb0ee3942e6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:38.188937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b55ccf70-1cfc-4844-b21f-bbb0ee3942e6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:38.197815 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b55ccf70-1cfc-4844-b21f-bbb0ee3942e6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.179151 Sep 27 10:58:38.212684 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3304/7081] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:38 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:39.222542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f40fb814-54ec-43c1-ac6d-44ddd89a8520 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:39.223221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f40fb814-54ec-43c1-ac6d-44ddd89a8520 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:39.230914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f40fb814-54ec-43c1-ac6d-44ddd89a8520 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:39.231853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f40fb814-54ec-43c1-ac6d-44ddd89a8520 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:39.232487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f40fb814-54ec-43c1-ac6d-44ddd89a8520 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:39.233403 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f40fb814-54ec-43c1-ac6d-44ddd89a8520 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:39.234162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f40fb814-54ec-43c1-ac6d-44ddd89a8520 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:39.234515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f40fb814-54ec-43c1-ac6d-44ddd89a8520 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:39.410305 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f40fb814-54ec-43c1-ac6d-44ddd89a8520 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:39.418756 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f40fb814-54ec-43c1-ac6d-44ddd89a8520 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.200132 Sep 27 10:58:39.419546 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3778/7082] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:39 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:40.446552 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-60c8ebea-b2f3-495c-8e0f-4c7095c6f8f9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:40.447608 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-60c8ebea-b2f3-495c-8e0f-4c7095c6f8f9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:40.456087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60c8ebea-b2f3-495c-8e0f-4c7095c6f8f9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:40.456605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60c8ebea-b2f3-495c-8e0f-4c7095c6f8f9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:40.457113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60c8ebea-b2f3-495c-8e0f-4c7095c6f8f9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:40.457848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60c8ebea-b2f3-495c-8e0f-4c7095c6f8f9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:40.458335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60c8ebea-b2f3-495c-8e0f-4c7095c6f8f9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:40.458787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-60c8ebea-b2f3-495c-8e0f-4c7095c6f8f9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:40.607850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-60c8ebea-b2f3-495c-8e0f-4c7095c6f8f9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:40.616168 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-60c8ebea-b2f3-495c-8e0f-4c7095c6f8f9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.176404 Sep 27 10:58:40.616790 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3305/7083] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:40 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:41.525225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-58145958-c82f-4b4a-b66c-7b9becd8ad46 req-02b1fd40-8c9b-43ab-a82b-b6a76bbe1f37 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fc6294ad-74fb-4566-b31f-e3fb4879642c", "tag": "0e2271e7-bbbe-42fa-9451-f968a8bd73ab"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:41.537337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-58145958-c82f-4b4a-b66c-7b9becd8ad46 req-02b1fd40-8c9b-43ab-a82b-b6a76bbe1f37 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:41.538245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-58145958-c82f-4b4a-b66c-7b9becd8ad46 req-02b1fd40-8c9b-43ab-a82b-b6a76bbe1f37 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:41.538849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-58145958-c82f-4b4a-b66c-7b9becd8ad46 req-02b1fd40-8c9b-43ab-a82b-b6a76bbe1f37 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:41.551280 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-58145958-c82f-4b4a-b66c-7b9becd8ad46 req-02b1fd40-8c9b-43ab-a82b-b6a76bbe1f37 service nova] Creating event network-changed:0e2271e7-bbbe-42fa-9451-f968a8bd73ab for instance fc6294ad-74fb-4566-b31f-e3fb4879642c on nch1-master-1 Sep 27 10:58:41.557210 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-58145958-c82f-4b4a-b66c-7b9becd8ad46 req-02b1fd40-8c9b-43ab-a82b-b6a76bbe1f37 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035963 Sep 27 10:58:41.558411 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3779/7084] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:58:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:58:41.638979 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4a1725ff-3a47-4393-b464-865239a81a9e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:41.639661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4a1725ff-3a47-4393-b464-865239a81a9e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:41.646795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a1725ff-3a47-4393-b464-865239a81a9e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:41.647294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a1725ff-3a47-4393-b464-865239a81a9e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:41.647749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a1725ff-3a47-4393-b464-865239a81a9e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:41.648462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a1725ff-3a47-4393-b464-865239a81a9e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:41.648953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a1725ff-3a47-4393-b464-865239a81a9e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:41.649391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4a1725ff-3a47-4393-b464-865239a81a9e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:41.810174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:41.831876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:41.904809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4a1725ff-3a47-4393-b464-865239a81a9e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:41.913303 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4a1725ff-3a47-4393-b464-865239a81a9e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.278335 Sep 27 10:58:41.913987 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3306/7085] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:41 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:42.934955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:42.940206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0222bd2-4680-4f7f-8671-634649c4679d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:42.941068 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b0222bd2-4680-4f7f-8671-634649c4679d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:42.947954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0222bd2-4680-4f7f-8671-634649c4679d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:42.948468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0222bd2-4680-4f7f-8671-634649c4679d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:42.948946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0222bd2-4680-4f7f-8671-634649c4679d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:42.949653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0222bd2-4680-4f7f-8671-634649c4679d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:42.950106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0222bd2-4680-4f7f-8671-634649c4679d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:42.950735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0222bd2-4680-4f7f-8671-634649c4679d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:43.084070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b0222bd2-4680-4f7f-8671-634649c4679d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:43.092677 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b0222bd2-4680-4f7f-8671-634649c4679d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.158632 Sep 27 10:58:43.093505 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3780/7086] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:42 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:44.120708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6661e8e8-47d8-4eca-b377-010c07026aae tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:44.121856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6661e8e8-47d8-4eca-b377-010c07026aae tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:44.124310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=41,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:44.130341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6661e8e8-47d8-4eca-b377-010c07026aae tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:44.130854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6661e8e8-47d8-4eca-b377-010c07026aae tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:44.131300 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6661e8e8-47d8-4eca-b377-010c07026aae tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:44.132023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6661e8e8-47d8-4eca-b377-010c07026aae tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:44.132496 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6661e8e8-47d8-4eca-b377-010c07026aae tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:44.132994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6661e8e8-47d8-4eca-b377-010c07026aae tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:44.271161 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6661e8e8-47d8-4eca-b377-010c07026aae tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:44.280477 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6661e8e8-47d8-4eca-b377-010c07026aae tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.166399 Sep 27 10:58:44.281094 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3307/7087] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:44 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:45.311825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9a0108c0-4259-4fd6-bea4-298b0c38b997 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:45.312452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9a0108c0-4259-4fd6-bea4-298b0c38b997 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:45.315248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:58:45.321025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a0108c0-4259-4fd6-bea4-298b0c38b997 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:45.321584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a0108c0-4259-4fd6-bea4-298b0c38b997 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:45.322086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a0108c0-4259-4fd6-bea4-298b0c38b997 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:45.322878 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a0108c0-4259-4fd6-bea4-298b0c38b997 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:45.323406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a0108c0-4259-4fd6-bea4-298b0c38b997 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:45.323834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9a0108c0-4259-4fd6-bea4-298b0c38b997 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:45.469773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9a0108c0-4259-4fd6-bea4-298b0c38b997 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:45.478291 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9a0108c0-4259-4fd6-bea4-298b0c38b997 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.172571 Sep 27 10:58:45.478865 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3781/7088] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:45 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:46.502968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-774592c2-fae8-4d9d-ba59-a297b5efeed5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:46.503802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-774592c2-fae8-4d9d-ba59-a297b5efeed5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:46.511338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-774592c2-fae8-4d9d-ba59-a297b5efeed5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:46.511840 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-774592c2-fae8-4d9d-ba59-a297b5efeed5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:46.512304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-774592c2-fae8-4d9d-ba59-a297b5efeed5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:46.513070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-774592c2-fae8-4d9d-ba59-a297b5efeed5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:46.513718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-774592c2-fae8-4d9d-ba59-a297b5efeed5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:46.514096 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-774592c2-fae8-4d9d-ba59-a297b5efeed5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:46.669014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-774592c2-fae8-4d9d-ba59-a297b5efeed5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:46.677455 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-774592c2-fae8-4d9d-ba59-a297b5efeed5 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.179452 Sep 27 10:58:46.678505 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3308/7089] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:46 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:46.734769 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-7a8fcdaf-1e23-499e-ad5a-56218d7005ff req-29d817fd-9ab2-4fb9-ad61-20eb67e6f9c9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fc6294ad-74fb-4566-b31f-e3fb4879642c", "name": "network-vif-plugged", "status": "completed", "tag": "0e2271e7-bbbe-42fa-9451-f968a8bd73ab"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:46.744422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a8fcdaf-1e23-499e-ad5a-56218d7005ff req-29d817fd-9ab2-4fb9-ad61-20eb67e6f9c9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:46.744929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a8fcdaf-1e23-499e-ad5a-56218d7005ff req-29d817fd-9ab2-4fb9-ad61-20eb67e6f9c9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:46.745361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-7a8fcdaf-1e23-499e-ad5a-56218d7005ff req-29d817fd-9ab2-4fb9-ad61-20eb67e6f9c9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:46.756511 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-7a8fcdaf-1e23-499e-ad5a-56218d7005ff req-29d817fd-9ab2-4fb9-ad61-20eb67e6f9c9 service nova] Creating event network-vif-plugged:0e2271e7-bbbe-42fa-9451-f968a8bd73ab for instance fc6294ad-74fb-4566-b31f-e3fb4879642c on nch1-master-1 Sep 27 10:58:46.761784 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-7a8fcdaf-1e23-499e-ad5a-56218d7005ff req-29d817fd-9ab2-4fb9-ad61-20eb67e6f9c9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093882 Sep 27 10:58:46.762375 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3782/7090] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:58:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 10:58:47.705534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f505cde0-b3c3-4199-8593-9158c63458da tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:47.706454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f505cde0-b3c3-4199-8593-9158c63458da tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:47.717978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505cde0-b3c3-4199-8593-9158c63458da tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:47.718466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505cde0-b3c3-4199-8593-9158c63458da tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:47.718916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505cde0-b3c3-4199-8593-9158c63458da tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:47.719633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505cde0-b3c3-4199-8593-9158c63458da tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:47.720099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505cde0-b3c3-4199-8593-9158c63458da tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:47.720543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f505cde0-b3c3-4199-8593-9158c63458da tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:47.875800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f505cde0-b3c3-4199-8593-9158c63458da tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:47.884490 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f505cde0-b3c3-4199-8593-9158c63458da tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1359 microversion: 2.1 time: 0.185888 Sep 27 10:58:47.885332 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3309/7091] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:47 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1359 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:48.918018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4b05bf02-4738-40e4-88f1-a1bda738a4df tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:48.919061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4b05bf02-4738-40e4-88f1-a1bda738a4df tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:48.926027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b05bf02-4738-40e4-88f1-a1bda738a4df tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:48.926547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b05bf02-4738-40e4-88f1-a1bda738a4df tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:48.926990 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b05bf02-4738-40e4-88f1-a1bda738a4df tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:48.927705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b05bf02-4738-40e4-88f1-a1bda738a4df tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:48.928162 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b05bf02-4738-40e4-88f1-a1bda738a4df tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:48.928593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b05bf02-4738-40e4-88f1-a1bda738a4df tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:49.061137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4b05bf02-4738-40e4-88f1-a1bda738a4df tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:49.069097 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4b05bf02-4738-40e4-88f1-a1bda738a4df tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1549 microversion: 2.1 time: 0.162449 Sep 27 10:58:49.069678 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3783/7092] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:48 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1549 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:49.091874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-42263480-a764-44dc-8e28-a43b16d526ea tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:49.092476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-42263480-a764-44dc-8e28-a43b16d526ea tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:49.099894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42263480-a764-44dc-8e28-a43b16d526ea tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:49.100400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42263480-a764-44dc-8e28-a43b16d526ea tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:49.100898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42263480-a764-44dc-8e28-a43b16d526ea tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:49.101620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42263480-a764-44dc-8e28-a43b16d526ea tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:49.102091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42263480-a764-44dc-8e28-a43b16d526ea tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:49.102594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42263480-a764-44dc-8e28-a43b16d526ea tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:49.255349 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-42263480-a764-44dc-8e28-a43b16d526ea tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:49.267556 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-42263480-a764-44dc-8e28-a43b16d526ea tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1549 microversion: 2.1 time: 0.179859 Sep 27 10:58:49.268428 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3310/7093] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:49 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1549 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:49.290948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-90120150-a686-4e81-9b22-1670252021bf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:49.296331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-90120150-a686-4e81-9b22-1670252021bf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:49.297140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-90120150-a686-4e81-9b22-1670252021bf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Searching by: {'deleted': False, 'project_id': '714fa95d9be140018a54ece37d204047'} {{(pid=145968) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 27 10:58:49.302980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90120150-a686-4e81-9b22-1670252021bf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:49.303465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90120150-a686-4e81-9b22-1670252021bf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:49.303904 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90120150-a686-4e81-9b22-1670252021bf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:49.304639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90120150-a686-4e81-9b22-1670252021bf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:49.305137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90120150-a686-4e81-9b22-1670252021bf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:49.305564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-90120150-a686-4e81-9b22-1670252021bf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:49.340012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.multi_cell_list [None req-90120150-a686-4e81-9b22-1670252021bf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=145968) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 27 10:58:49.367272 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-90120150-a686-4e81-9b22-1670252021bf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.080234 Sep 27 10:58:49.367861 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3784/7094] 10.222.0.26 () {58 vars in 1125 bytes} [Tue Sep 27 10:58:49 2022] GET /compute/v2.1/servers => generated 343 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:58:49.383146 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-29352dc2-3fca-485e-a1b1-da2ce6ced5c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:49.384072 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-29352dc2-3fca-485e-a1b1-da2ce6ced5c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:49.392867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29352dc2-3fca-485e-a1b1-da2ce6ced5c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:49.393651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29352dc2-3fca-485e-a1b1-da2ce6ced5c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:49.394339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29352dc2-3fca-485e-a1b1-da2ce6ced5c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:49.395411 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29352dc2-3fca-485e-a1b1-da2ce6ced5c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:49.396179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29352dc2-3fca-485e-a1b1-da2ce6ced5c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:49.396928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29352dc2-3fca-485e-a1b1-da2ce6ced5c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:49.538750 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-29352dc2-3fca-485e-a1b1-da2ce6ced5c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:49.547520 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-29352dc2-3fca-485e-a1b1-da2ce6ced5c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1549 microversion: 2.1 time: 0.170868 Sep 27 10:58:49.548107 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3311/7095] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:49 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1549 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:51.100934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7968931f-a63a-4376-95c8-fd49b159f3e4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:51.103322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7968931f-a63a-4376-95c8-fd49b159f3e4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:51.110815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7968931f-a63a-4376-95c8-fd49b159f3e4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:51.111291 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7968931f-a63a-4376-95c8-fd49b159f3e4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:51.111733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7968931f-a63a-4376-95c8-fd49b159f3e4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:51.112466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7968931f-a63a-4376-95c8-fd49b159f3e4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:51.112941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7968931f-a63a-4376-95c8-fd49b159f3e4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:51.113367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7968931f-a63a-4376-95c8-fd49b159f3e4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:51.382095 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7968931f-a63a-4376-95c8-fd49b159f3e4 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.285195 Sep 27 10:58:51.382749 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3785/7096] 10.222.0.26 () {60 vars in 1323 bytes} [Tue Sep 27 10:58:51 2022] POST /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/os-volume_attachments => generated 194 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:58:54.779698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f19fdc9a-398a-40e0-9e2b-582ee2a67b67 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:54.780335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f19fdc9a-398a-40e0-9e2b-582ee2a67b67 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:54.787985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f19fdc9a-398a-40e0-9e2b-582ee2a67b67 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:54.788487 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f19fdc9a-398a-40e0-9e2b-582ee2a67b67 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:54.788964 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f19fdc9a-398a-40e0-9e2b-582ee2a67b67 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:54.789681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f19fdc9a-398a-40e0-9e2b-582ee2a67b67 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:54.790155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f19fdc9a-398a-40e0-9e2b-582ee2a67b67 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:54.790596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f19fdc9a-398a-40e0-9e2b-582ee2a67b67 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:54.926159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f19fdc9a-398a-40e0-9e2b-582ee2a67b67 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:54.936466 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f19fdc9a-398a-40e0-9e2b-582ee2a67b67 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1595 microversion: 2.1 time: 0.160711 Sep 27 10:58:54.937091 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3312/7097] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:54 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1595 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:58:56.680582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-4eef23cd-dc13-46db-aa0a-3d1afcb28428 req-9947c9d9-9f19-4fdb-a0d3-50f39395a1a1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fc6294ad-74fb-4566-b31f-e3fb4879642c", "tag": "0e2271e7-bbbe-42fa-9451-f968a8bd73ab"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:56.690646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4eef23cd-dc13-46db-aa0a-3d1afcb28428 req-9947c9d9-9f19-4fdb-a0d3-50f39395a1a1 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:56.691137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4eef23cd-dc13-46db-aa0a-3d1afcb28428 req-9947c9d9-9f19-4fdb-a0d3-50f39395a1a1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:56.691575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-4eef23cd-dc13-46db-aa0a-3d1afcb28428 req-9947c9d9-9f19-4fdb-a0d3-50f39395a1a1 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:56.704357 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-4eef23cd-dc13-46db-aa0a-3d1afcb28428 req-9947c9d9-9f19-4fdb-a0d3-50f39395a1a1 service nova] Creating event network-changed:0e2271e7-bbbe-42fa-9451-f968a8bd73ab for instance fc6294ad-74fb-4566-b31f-e3fb4879642c on nch1-master-1 Sep 27 10:58:56.709839 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-4eef23cd-dc13-46db-aa0a-3d1afcb28428 req-9947c9d9-9f19-4fdb-a0d3-50f39395a1a1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034612 Sep 27 10:58:56.715925 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3786/7098] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:58:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:58:57.802725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-764a85d6-88d0-4656-be9d-b2c51772cd15 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-961291422"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:57.803317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-764a85d6-88d0-4656-be9d-b2c51772cd15 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:57.810576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-764a85d6-88d0-4656-be9d-b2c51772cd15 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:57.811323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-764a85d6-88d0-4656-be9d-b2c51772cd15 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:57.811789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-764a85d6-88d0-4656-be9d-b2c51772cd15 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:57.812506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-764a85d6-88d0-4656-be9d-b2c51772cd15 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:57.813025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-764a85d6-88d0-4656-be9d-b2c51772cd15 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:57.813512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-764a85d6-88d0-4656-be9d-b2c51772cd15 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:57.954963 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.network.security_group_api [None req-764a85d6-88d0-4656-be9d-b2c51772cd15 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Adding security group 99fb0314-bdee-406a-aecf-755edb4de60d to port 0e2271e7-bbbe-42fa-9451-f968a8bd73ab Sep 27 10:58:58.584776 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-764a85d6-88d0-4656-be9d-b2c51772cd15 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/action" status: 202 len: 0 microversion: 2.1 time: 0.786002 Sep 27 10:58:58.585874 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3313/7099] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:58:57 2022] POST /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/action => generated 0 bytes in 788 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:58:58.595844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e9112c35-0578-4af4-87b6-ef100fa4a0ca tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:58:58.596927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e9112c35-0578-4af4-87b6-ef100fa4a0ca tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:58:58.606188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9112c35-0578-4af4-87b6-ef100fa4a0ca tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:58.607185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9112c35-0578-4af4-87b6-ef100fa4a0ca tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:58.607810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9112c35-0578-4af4-87b6-ef100fa4a0ca tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:58.608706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9112c35-0578-4af4-87b6-ef100fa4a0ca tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:58.609347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9112c35-0578-4af4-87b6-ef100fa4a0ca tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:58.610018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e9112c35-0578-4af4-87b6-ef100fa4a0ca tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:58.729583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a9ed352e-9d69-420d-a302-50840403851b req-8f6a000b-f039-4e0f-b66c-0577761633fd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fc6294ad-74fb-4566-b31f-e3fb4879642c", "tag": "0e2271e7-bbbe-42fa-9451-f968a8bd73ab"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:58:58.742957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a9ed352e-9d69-420d-a302-50840403851b req-8f6a000b-f039-4e0f-b66c-0577761633fd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:58:58.744047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a9ed352e-9d69-420d-a302-50840403851b req-8f6a000b-f039-4e0f-b66c-0577761633fd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:58:58.744581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a9ed352e-9d69-420d-a302-50840403851b req-8f6a000b-f039-4e0f-b66c-0577761633fd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:58:58.757274 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a9ed352e-9d69-420d-a302-50840403851b req-8f6a000b-f039-4e0f-b66c-0577761633fd service nova] Creating event network-changed:0e2271e7-bbbe-42fa-9451-f968a8bd73ab for instance fc6294ad-74fb-4566-b31f-e3fb4879642c on nch1-master-1 Sep 27 10:58:58.762935 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a9ed352e-9d69-420d-a302-50840403851b req-8f6a000b-f039-4e0f-b66c-0577761633fd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037983 Sep 27 10:58:58.763516 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3314/7100] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 10:58:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 10:58:58.780411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e9112c35-0578-4af4-87b6-ef100fa4a0ca tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:58:58.790087 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e9112c35-0578-4af4-87b6-ef100fa4a0ca tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1759 microversion: 2.1 time: 0.199533 Sep 27 10:58:58.790681 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3787/7101] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:58:58 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1759 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:59:22.120950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=15 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:59:22.121936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=74 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:59:22.127890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1f232a95-7ffa-4ea2-9903-651f68877d3e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 10:59:22.131082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1f232a95-7ffa-4ea2-9903-651f68877d3e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:22.139194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f232a95-7ffa-4ea2-9903-651f68877d3e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:22.139692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f232a95-7ffa-4ea2-9903-651f68877d3e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:22.140142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f232a95-7ffa-4ea2-9903-651f68877d3e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:22.140890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f232a95-7ffa-4ea2-9903-651f68877d3e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:22.141361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f232a95-7ffa-4ea2-9903-651f68877d3e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:22.141795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f232a95-7ffa-4ea2-9903-651f68877d3e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:22.225969 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1f232a95-7ffa-4ea2-9903-651f68877d3e tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/action" status: 202 len: 0 microversion: 2.1 time: 0.107519 Sep 27 10:59:22.226859 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3315/7102] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 10:59:22 2022] POST /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/action => generated 0 bytes in 111 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 10:59:22.235918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8597017a-561a-4cfa-9caf-6c62cbc7a729 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:59:22.236834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8597017a-561a-4cfa-9caf-6c62cbc7a729 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:22.239087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:59:22.239968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=13 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:59:22.240529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=67,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 10:59:22.246587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8597017a-561a-4cfa-9caf-6c62cbc7a729 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:22.247230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8597017a-561a-4cfa-9caf-6c62cbc7a729 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:22.247800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8597017a-561a-4cfa-9caf-6c62cbc7a729 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:22.248511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8597017a-561a-4cfa-9caf-6c62cbc7a729 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:22.248994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8597017a-561a-4cfa-9caf-6c62cbc7a729 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:22.249481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8597017a-561a-4cfa-9caf-6c62cbc7a729 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:22.402036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8597017a-561a-4cfa-9caf-6c62cbc7a729 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:59:22.411165 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8597017a-561a-4cfa-9caf-6c62cbc7a729 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1751 microversion: 2.1 time: 0.179418 Sep 27 10:59:22.411750 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3788/7103] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:59:22 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1751 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:59:23.441468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6bb3a567-b7aa-4c5d-b173-17b28d56d3f2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:59:23.442541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6bb3a567-b7aa-4c5d-b173-17b28d56d3f2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:23.453286 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bb3a567-b7aa-4c5d-b173-17b28d56d3f2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:23.454132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bb3a567-b7aa-4c5d-b173-17b28d56d3f2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:23.454899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bb3a567-b7aa-4c5d-b173-17b28d56d3f2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:23.456089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bb3a567-b7aa-4c5d-b173-17b28d56d3f2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:23.456934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bb3a567-b7aa-4c5d-b173-17b28d56d3f2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:23.457689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6bb3a567-b7aa-4c5d-b173-17b28d56d3f2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:23.617148 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6bb3a567-b7aa-4c5d-b173-17b28d56d3f2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:59:23.625802 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6bb3a567-b7aa-4c5d-b173-17b28d56d3f2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1756 microversion: 2.1 time: 0.193335 Sep 27 10:59:23.626350 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3316/7104] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:59:23 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1756 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:59:24.649249 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-00e5187c-05cd-42cf-be6a-7bf1ea0795d9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:59:24.649871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-00e5187c-05cd-42cf-be6a-7bf1ea0795d9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:24.657373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00e5187c-05cd-42cf-be6a-7bf1ea0795d9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:24.658217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00e5187c-05cd-42cf-be6a-7bf1ea0795d9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:24.658653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00e5187c-05cd-42cf-be6a-7bf1ea0795d9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:24.659355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00e5187c-05cd-42cf-be6a-7bf1ea0795d9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:24.659811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00e5187c-05cd-42cf-be6a-7bf1ea0795d9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:24.660233 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-00e5187c-05cd-42cf-be6a-7bf1ea0795d9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:24.822115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-00e5187c-05cd-42cf-be6a-7bf1ea0795d9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:59:24.830695 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-00e5187c-05cd-42cf-be6a-7bf1ea0795d9 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1756 microversion: 2.1 time: 0.185488 Sep 27 10:59:24.831475 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3789/7105] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:59:24 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1756 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:59:25.857302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d7178b0f-839c-4b58-9945-1289df7aeb20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:59:25.857905 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d7178b0f-839c-4b58-9945-1289df7aeb20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:25.865348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7178b0f-839c-4b58-9945-1289df7aeb20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:25.865851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7178b0f-839c-4b58-9945-1289df7aeb20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:25.866302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7178b0f-839c-4b58-9945-1289df7aeb20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:25.867016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7178b0f-839c-4b58-9945-1289df7aeb20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:25.867488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7178b0f-839c-4b58-9945-1289df7aeb20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:25.867928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7178b0f-839c-4b58-9945-1289df7aeb20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:26.023504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d7178b0f-839c-4b58-9945-1289df7aeb20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:59:26.032577 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d7178b0f-839c-4b58-9945-1289df7aeb20 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1756 microversion: 2.1 time: 0.180258 Sep 27 10:59:26.033516 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3317/7106] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:59:25 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1756 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:59:27.058026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3a79df3f-ea1e-479f-a5ce-08a0e926fae2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:59:27.058672 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3a79df3f-ea1e-479f-a5ce-08a0e926fae2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:27.066313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a79df3f-ea1e-479f-a5ce-08a0e926fae2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:27.066795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a79df3f-ea1e-479f-a5ce-08a0e926fae2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:27.067231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a79df3f-ea1e-479f-a5ce-08a0e926fae2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:27.067931 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a79df3f-ea1e-479f-a5ce-08a0e926fae2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:27.068388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a79df3f-ea1e-479f-a5ce-08a0e926fae2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:27.068848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3a79df3f-ea1e-479f-a5ce-08a0e926fae2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:27.221134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3a79df3f-ea1e-479f-a5ce-08a0e926fae2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:59:27.233883 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3a79df3f-ea1e-479f-a5ce-08a0e926fae2 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1756 microversion: 2.1 time: 0.180038 Sep 27 10:59:27.234688 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3790/7107] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:59:27 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1756 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:59:28.262354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3373531a-ce77-4c75-b0b0-12301f841ecc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:59:28.263099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3373531a-ce77-4c75-b0b0-12301f841ecc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:28.271176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3373531a-ce77-4c75-b0b0-12301f841ecc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:28.271742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3373531a-ce77-4c75-b0b0-12301f841ecc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:28.272257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3373531a-ce77-4c75-b0b0-12301f841ecc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:28.273099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3373531a-ce77-4c75-b0b0-12301f841ecc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:28.273634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3373531a-ce77-4c75-b0b0-12301f841ecc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:28.274126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3373531a-ce77-4c75-b0b0-12301f841ecc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:28.423742 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3373531a-ce77-4c75-b0b0-12301f841ecc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:59:28.435725 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3373531a-ce77-4c75-b0b0-12301f841ecc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1756 microversion: 2.1 time: 0.179893 Sep 27 10:59:28.436303 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3318/7108] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:59:28 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1756 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:59:29.462705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b3e35cab-2942-49de-b914-37e755aa3aa8 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:59:29.463539 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b3e35cab-2942-49de-b914-37e755aa3aa8 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:29.471948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e35cab-2942-49de-b914-37e755aa3aa8 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:29.472470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e35cab-2942-49de-b914-37e755aa3aa8 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:29.472912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e35cab-2942-49de-b914-37e755aa3aa8 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:29.473618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e35cab-2942-49de-b914-37e755aa3aa8 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:29.474069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e35cab-2942-49de-b914-37e755aa3aa8 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:29.474494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b3e35cab-2942-49de-b914-37e755aa3aa8 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:29.634952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b3e35cab-2942-49de-b914-37e755aa3aa8 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:59:29.643500 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b3e35cab-2942-49de-b914-37e755aa3aa8 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1756 microversion: 2.1 time: 0.186830 Sep 27 10:59:29.644577 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3791/7109] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:59:29 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1756 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:59:30.668111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e94d5eac-7dc4-4bea-bf48-a945c76e870a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:59:30.668795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e94d5eac-7dc4-4bea-bf48-a945c76e870a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:30.676351 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e94d5eac-7dc4-4bea-bf48-a945c76e870a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:30.676899 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e94d5eac-7dc4-4bea-bf48-a945c76e870a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:30.677362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e94d5eac-7dc4-4bea-bf48-a945c76e870a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:30.678086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e94d5eac-7dc4-4bea-bf48-a945c76e870a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:30.678565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e94d5eac-7dc4-4bea-bf48-a945c76e870a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:30.679597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e94d5eac-7dc4-4bea-bf48-a945c76e870a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:30.826061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e94d5eac-7dc4-4bea-bf48-a945c76e870a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:59:30.834862 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e94d5eac-7dc4-4bea-bf48-a945c76e870a tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1756 microversion: 2.1 time: 0.170989 Sep 27 10:59:30.835642 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3319/7110] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:59:30 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1756 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:59:31.860703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-96cb082c-d311-4b11-abe3-42ae42bee773 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:59:31.861303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-96cb082c-d311-4b11-abe3-42ae42bee773 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:31.868798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96cb082c-d311-4b11-abe3-42ae42bee773 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:31.869288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96cb082c-d311-4b11-abe3-42ae42bee773 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:31.869727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96cb082c-d311-4b11-abe3-42ae42bee773 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:31.870456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96cb082c-d311-4b11-abe3-42ae42bee773 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:31.870865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96cb082c-d311-4b11-abe3-42ae42bee773 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:31.871314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-96cb082c-d311-4b11-abe3-42ae42bee773 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:32.029711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-96cb082c-d311-4b11-abe3-42ae42bee773 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:59:32.038660 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-96cb082c-d311-4b11-abe3-42ae42bee773 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1756 microversion: 2.1 time: 0.182914 Sep 27 10:59:32.039405 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3792/7111] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:59:31 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1756 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:59:33.064208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3b701805-d181-4a2f-913d-d1f91ee66fcc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:59:33.064898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3b701805-d181-4a2f-913d-d1f91ee66fcc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:33.072327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b701805-d181-4a2f-913d-d1f91ee66fcc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:33.072863 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b701805-d181-4a2f-913d-d1f91ee66fcc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:33.073317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b701805-d181-4a2f-913d-d1f91ee66fcc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:33.074033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b701805-d181-4a2f-913d-d1f91ee66fcc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:33.074505 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b701805-d181-4a2f-913d-d1f91ee66fcc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:33.074943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3b701805-d181-4a2f-913d-d1f91ee66fcc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:33.228775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3b701805-d181-4a2f-913d-d1f91ee66fcc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:59:33.242323 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3b701805-d181-4a2f-913d-d1f91ee66fcc tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1756 microversion: 2.1 time: 0.182575 Sep 27 10:59:33.243081 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3320/7112] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:59:33 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1756 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 10:59:34.268513 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-75c268a3-a94e-45f5-851c-4ed963259faf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 10:59:34.269194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-75c268a3-a94e-45f5-851c-4ed963259faf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 10:59:34.277610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75c268a3-a94e-45f5-851c-4ed963259faf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:34.278150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75c268a3-a94e-45f5-851c-4ed963259faf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:34.278636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75c268a3-a94e-45f5-851c-4ed963259faf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:34.279415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75c268a3-a94e-45f5-851c-4ed963259faf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 10:59:34.279925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75c268a3-a94e-45f5-851c-4ed963259faf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 10:59:34.280428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-75c268a3-a94e-45f5-851c-4ed963259faf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 10:59:34.436776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-75c268a3-a94e-45f5-851c-4ed963259faf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 10:59:34.448623 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-75c268a3-a94e-45f5-851c-4ed963259faf tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1759 microversion: 2.1 time: 0.184335 Sep 27 10:59:34.449376 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3793/7113] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 10:59:34 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1759 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:06.833370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1411b482-3858-4f73-b53b-df37cea09b8b tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-961291422"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:00:06.834024 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1411b482-3858-4f73-b53b-df37cea09b8b tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:06.836313 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=12 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:06.837404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=40,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:06.845033 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1411b482-3858-4f73-b53b-df37cea09b8b tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:06.845539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1411b482-3858-4f73-b53b-df37cea09b8b tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:06.845993 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1411b482-3858-4f73-b53b-df37cea09b8b tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:06.846715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1411b482-3858-4f73-b53b-df37cea09b8b tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:06.847206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1411b482-3858-4f73-b53b-df37cea09b8b tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:06.847635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1411b482-3858-4f73-b53b-df37cea09b8b tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:06.978659 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.network.security_group_api [None req-1411b482-3858-4f73-b53b-df37cea09b8b tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Removing security group 99fb0314-bdee-406a-aecf-755edb4de60d from port 0e2271e7-bbbe-42fa-9451-f968a8bd73ab Sep 27 11:00:07.316734 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1411b482-3858-4f73-b53b-df37cea09b8b tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "POST /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/action" status: 202 len: 0 microversion: 2.1 time: 0.487148 Sep 27 11:00:07.316981 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3321/7114] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:00:06 2022] POST /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/action => generated 0 bytes in 489 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:00:07.325524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-9f493405-c6eb-4fe6-8dc6-99db388be2d8 req-0e4d5743-5e7f-4ab1-aefc-e6e5b6be058f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fc6294ad-74fb-4566-b31f-e3fb4879642c", "tag": "0e2271e7-bbbe-42fa-9451-f968a8bd73ab"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:00:07.335686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=12 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:07.337341 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=36 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:07.339299 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:00:07.339685 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 11:00:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.26) !!! Sep 27 11:00:07.353000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f493405-c6eb-4fe6-8dc6-99db388be2d8 req-0e4d5743-5e7f-4ab1-aefc-e6e5b6be058f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:07.353903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f493405-c6eb-4fe6-8dc6-99db388be2d8 req-0e4d5743-5e7f-4ab1-aefc-e6e5b6be058f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:07.355226 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-9f493405-c6eb-4fe6-8dc6-99db388be2d8 req-0e4d5743-5e7f-4ab1-aefc-e6e5b6be058f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:07.372912 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-9f493405-c6eb-4fe6-8dc6-99db388be2d8 req-0e4d5743-5e7f-4ab1-aefc-e6e5b6be058f service nova] Creating event network-changed:0e2271e7-bbbe-42fa-9451-f968a8bd73ab for instance fc6294ad-74fb-4566-b31f-e3fb4879642c on nch1-master-1 Sep 27 11:00:07.380214 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-9f493405-c6eb-4fe6-8dc6-99db388be2d8 req-0e4d5743-5e7f-4ab1-aefc-e6e5b6be058f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059126 Sep 27 11:00:07.381881 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3794/7115] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:00:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:00:08.406600 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-49c79232-841f-4850-b1ce-61d51110a069 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:08.407274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-49c79232-841f-4850-b1ce-61d51110a069 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:08.409807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-408764c8-7c31-430d-82d3-8a370a1573fe req-c3804348-4a6a-46f7-83c4-c45ffe46067e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fc6294ad-74fb-4566-b31f-e3fb4879642c", "tag": "0e2271e7-bbbe-42fa-9451-f968a8bd73ab"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:00:08.415212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c79232-841f-4850-b1ce-61d51110a069 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:08.415812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c79232-841f-4850-b1ce-61d51110a069 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:08.416570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c79232-841f-4850-b1ce-61d51110a069 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:08.417625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c79232-841f-4850-b1ce-61d51110a069 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:08.418108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c79232-841f-4850-b1ce-61d51110a069 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:08.418554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49c79232-841f-4850-b1ce-61d51110a069 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:08.420903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-408764c8-7c31-430d-82d3-8a370a1573fe req-c3804348-4a6a-46f7-83c4-c45ffe46067e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:08.421379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-408764c8-7c31-430d-82d3-8a370a1573fe req-c3804348-4a6a-46f7-83c4-c45ffe46067e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:08.422061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-408764c8-7c31-430d-82d3-8a370a1573fe req-c3804348-4a6a-46f7-83c4-c45ffe46067e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:08.434011 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-408764c8-7c31-430d-82d3-8a370a1573fe req-c3804348-4a6a-46f7-83c4-c45ffe46067e service nova] Creating event network-changed:0e2271e7-bbbe-42fa-9451-f968a8bd73ab for instance fc6294ad-74fb-4566-b31f-e3fb4879642c on nch1-master-1 Sep 27 11:00:08.439218 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-408764c8-7c31-430d-82d3-8a370a1573fe req-c3804348-4a6a-46f7-83c4-c45ffe46067e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034923 Sep 27 11:00:08.439825 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3795/7116] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:00:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:00:08.503984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-49c79232-841f-4850-b1ce-61d51110a069 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lazy-loading 'services' on Instance uuid fc6294ad-74fb-4566-b31f-e3fb4879642c {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 11:00:08.586015 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-49c79232-841f-4850-b1ce-61d51110a069 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "DELETE /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/os-volume_attachments/022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97" status: 202 len: 0 microversion: 2.1 time: 0.184468 Sep 27 11:00:08.586726 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3322/7117] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 11:00:08 2022] DELETE /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/os-volume_attachments/022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97 => generated 0 bytes in 186 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:00:08.597359 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-01e987c7-0baa-45c1-9a12-007cc411ec36 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:08.600275 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-01e987c7-0baa-45c1-9a12-007cc411ec36 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:08.607689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01e987c7-0baa-45c1-9a12-007cc411ec36 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:08.608596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01e987c7-0baa-45c1-9a12-007cc411ec36 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:08.609309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01e987c7-0baa-45c1-9a12-007cc411ec36 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:08.610673 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01e987c7-0baa-45c1-9a12-007cc411ec36 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:08.611378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01e987c7-0baa-45c1-9a12-007cc411ec36 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:08.612836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-01e987c7-0baa-45c1-9a12-007cc411ec36 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:08.658670 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-01e987c7-0baa-45c1-9a12-007cc411ec36 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066232 Sep 27 11:00:08.659818 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3796/7118] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 11:00:08 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:00:09.673302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-aba306df-3a67-4057-9020-41929ab666eb tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:09.675760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-aba306df-3a67-4057-9020-41929ab666eb tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:09.682774 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aba306df-3a67-4057-9020-41929ab666eb tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:09.683266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aba306df-3a67-4057-9020-41929ab666eb tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:09.683715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aba306df-3a67-4057-9020-41929ab666eb tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:09.684420 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aba306df-3a67-4057-9020-41929ab666eb tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:09.684927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aba306df-3a67-4057-9020-41929ab666eb tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:09.685365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-aba306df-3a67-4057-9020-41929ab666eb tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:09.716274 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-aba306df-3a67-4057-9020-41929ab666eb tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048341 Sep 27 11:00:09.716876 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3323/7119] 10.222.0.26 () {58 vars in 1302 bytes} [Tue Sep 27 11:00:09 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 27 11:00:09.729870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:09.730418 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:09.737541 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:09.738023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:09.738466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:09.739180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:09.739643 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:09.740061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:09.821569 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] HTTP exception thrown: Instance fc6294ad-74fb-4566-b31f-e3fb4879642c is not attached to volume 022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Request method failure captured: Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: request: DELETE /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/os-volume_attachments/022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97 HTTP/1.1 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtYCD54ftHDthm7BCdV6AkQb8HODztxEb10C_SCo0B541ecNSCGEym3jk9y9sS1-_6UOL0lMYfx5iyQtcgBulEcndBXxotppQB_qD75QcYmJXla1b-2-rjVLpqTjTA3li4n_CEXD4Cc26PZ_YhunxBrjuC1a9L4ks6u1DI3EakP93Ac Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 714fa95d9be140018a54ece37d204047 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestMinimumBasicScenario-1586443371 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/714fa95d9be140018a54ece37d204047"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/714fa95d9be140018a54ece37d204047"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/714fa95d9be140018a54ece37d204047"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_714fa95d9be140018a54ece37d204047"}]}] Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestMinimumBasicScenario-1586443371 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 714fa95d9be140018a54ece37d204047 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1586443371 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestMinimumBasicScenario-1586443371-project Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 906ca88c8f544940b06705711ae7f1af Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestMinimumBasicScenario-1586443371-project Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance fc6294ad-74fb-4566-b31f-e3fb4879642c is not attached to volume 022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'server_id': 'fc6294ad-74fb-4566-b31f-e3fb4879642c', 'id': '022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97'} Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97. Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fc6294ad-74fb-4566-b31f-e3fb4879642c is not attached to volume 022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97 Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:00:09.823520 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:00:09.828831 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 11:00:09.828831 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 11:00:09.828831 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fc6294ad-74fb-4566-b31f-e3fb4879642c is not attached to volume 022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97 Sep 27 11:00:09.828831 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:00:09.828831 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Returning 404 to user: Instance fc6294ad-74fb-4566-b31f-e3fb4879642c is not attached to volume 022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97 {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 11:00:09.828831 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e3eafb48-9577-424c-844f-320eeca31e58 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "DELETE /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/os-volume_attachments/022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97" status: 404 len: 154 microversion: 2.1 time: 0.100433 Sep 27 11:00:09.828831 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3797/7120] 10.222.0.26 () {58 vars in 1416 bytes} [Tue Sep 27 11:00:09 2022] DELETE /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c/os-volume_attachments/022ef6ed-0a3e-4b4e-a149-9c6b1bfcee97 => generated 154 bytes in 102 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 11:00:10.981228 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fa8c3517-0cc5-452e-a180-ae4f4d8c0409 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:10.981769 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa8c3517-0cc5-452e-a180-ae4f4d8c0409 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:10.989071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8c3517-0cc5-452e-a180-ae4f4d8c0409 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:10.989572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8c3517-0cc5-452e-a180-ae4f4d8c0409 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:10.990010 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8c3517-0cc5-452e-a180-ae4f4d8c0409 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:10.990716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8c3517-0cc5-452e-a180-ae4f4d8c0409 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:10.991190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8c3517-0cc5-452e-a180-ae4f4d8c0409 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:10.991626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fa8c3517-0cc5-452e-a180-ae4f4d8c0409 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:11.019107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fa8c3517-0cc5-452e-a180-ae4f4d8c0409 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 11:00:11.106462 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fa8c3517-0cc5-452e-a180-ae4f4d8c0409 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "DELETE /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 204 len: 0 microversion: 2.1 time: 0.128720 Sep 27 11:00:11.107334 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3324/7121] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 11:00:10 2022] DELETE /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 11:00:11.117362 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cb1fb9e7-7da5-4307-aed3-e9e80378bf0c tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:11.118355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cb1fb9e7-7da5-4307-aed3-e9e80378bf0c tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:11.125865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb1fb9e7-7da5-4307-aed3-e9e80378bf0c tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:11.126348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb1fb9e7-7da5-4307-aed3-e9e80378bf0c tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:11.126777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb1fb9e7-7da5-4307-aed3-e9e80378bf0c tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:11.127475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb1fb9e7-7da5-4307-aed3-e9e80378bf0c tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:11.127919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb1fb9e7-7da5-4307-aed3-e9e80378bf0c tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:11.128345 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cb1fb9e7-7da5-4307-aed3-e9e80378bf0c tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:11.263117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cb1fb9e7-7da5-4307-aed3-e9e80378bf0c tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:11.271393 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cb1fb9e7-7da5-4307-aed3-e9e80378bf0c tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1555 microversion: 2.1 time: 0.158097 Sep 27 11:00:11.271982 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3798/7122] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:11 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1555 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:12.300986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b86045a7-b816-47c1-9fdd-e1faac18c0d6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:12.303144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b86045a7-b816-47c1-9fdd-e1faac18c0d6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:12.312524 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b86045a7-b816-47c1-9fdd-e1faac18c0d6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:12.313068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b86045a7-b816-47c1-9fdd-e1faac18c0d6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:12.313525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b86045a7-b816-47c1-9fdd-e1faac18c0d6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:12.314285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b86045a7-b816-47c1-9fdd-e1faac18c0d6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:12.314738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b86045a7-b816-47c1-9fdd-e1faac18c0d6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:12.315206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b86045a7-b816-47c1-9fdd-e1faac18c0d6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:12.471752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b86045a7-b816-47c1-9fdd-e1faac18c0d6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:12.485067 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b86045a7-b816-47c1-9fdd-e1faac18c0d6 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1555 microversion: 2.1 time: 0.190896 Sep 27 11:00:12.485673 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3325/7123] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:12 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1555 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:13.304639 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-158911d3-98ef-403d-bf59-2d23ae19fea5 req-20a3edb8-45c6-4708-8cb2-0c6e60097b26 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fc6294ad-74fb-4566-b31f-e3fb4879642c", "name": "network-vif-deleted", "tag": "0e2271e7-bbbe-42fa-9451-f968a8bd73ab"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:00:13.317203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-158911d3-98ef-403d-bf59-2d23ae19fea5 req-20a3edb8-45c6-4708-8cb2-0c6e60097b26 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:13.318764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-158911d3-98ef-403d-bf59-2d23ae19fea5 req-20a3edb8-45c6-4708-8cb2-0c6e60097b26 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:13.319369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-158911d3-98ef-403d-bf59-2d23ae19fea5 req-20a3edb8-45c6-4708-8cb2-0c6e60097b26 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:13.332809 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-158911d3-98ef-403d-bf59-2d23ae19fea5 req-20a3edb8-45c6-4708-8cb2-0c6e60097b26 service nova] Creating event network-vif-deleted:0e2271e7-bbbe-42fa-9451-f968a8bd73ab for instance fc6294ad-74fb-4566-b31f-e3fb4879642c on nch1-master-1 Sep 27 11:00:13.338731 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-158911d3-98ef-403d-bf59-2d23ae19fea5 req-20a3edb8-45c6-4708-8cb2-0c6e60097b26 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037490 Sep 27 11:00:13.339772 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3799/7124] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:00:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:00:13.509541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:13.511003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:13.518127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:13.518627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:13.519085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:13.519859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:13.520336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:13.520828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:13.552858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:13.553357 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:13.553818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:13.554275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lazy-loading 'fault' on Instance uuid fc6294ad-74fb-4566-b31f-e3fb4879642c {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 11:00:13.601980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '906ca88c8f544940b06705711ae7f1af', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '714fa95d9be140018a54ece37d204047', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:13.610652 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-46d6238f-b086-42f9-9506-97196497097d tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 200 len: 1349 microversion: 2.1 time: 0.105715 Sep 27 11:00:13.612112 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3326/7125] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:13 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 1349 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:14.637267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:14.637978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] [instance: fc6294ad-74fb-4566-b31f-e3fb4879642c] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:14.644807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:14.645301 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:14.645741 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:14.646475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:14.646934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:14.647364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:14.664313 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] HTTP exception thrown: Instance fc6294ad-74fb-4566-b31f-e3fb4879642c could not be found. Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Request method failure captured: Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c HTTP/1.1 Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtYCD54ftHDthm7BCdV6AkQb8HODztxEb10C_SCo0B541ecNSCGEym3jk9y9sS1-_6UOL0lMYfx5iyQtcgBulEcndBXxotppQB_qD75QcYmJXla1b-2-rjVLpqTjTA3li4n_CEXD4Cc26PZ_YhunxBrjuC1a9L4ks6u1DI3EakP93Ac Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 714fa95d9be140018a54ece37d204047 Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestMinimumBasicScenario-1586443371 Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/714fa95d9be140018a54ece37d204047"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/714fa95d9be140018a54ece37d204047"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/714fa95d9be140018a54ece37d204047"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_714fa95d9be140018a54ece37d204047"}]}] Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestMinimumBasicScenario-1586443371 Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 714fa95d9be140018a54ece37d204047 Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1586443371 Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestMinimumBasicScenario-1586443371-project Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 906ca88c8f544940b06705711ae7f1af Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestMinimumBasicScenario-1586443371-project Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance fc6294ad-74fb-4566-b31f-e3fb4879642c could not be found. Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': 'fc6294ad-74fb-4566-b31f-e3fb4879642c'} Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fc6294ad-74fb-4566-b31f-e3fb4879642c could not be found. Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fc6294ad-74fb-4566-b31f-e3fb4879642c could not be found. Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:00:14.666545 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:00:14.672195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:00:14.672195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:00:14.672195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:00:14.672195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 11:00:14.672195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 11:00:14.672195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fc6294ad-74fb-4566-b31f-e3fb4879642c could not be found. Sep 27 11:00:14.672195 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:00:14.672195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Returning 404 to user: Instance fc6294ad-74fb-4566-b31f-e3fb4879642c could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 11:00:14.672195 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2072bde8-d1a4-4c3d-aa1f-fb138c9810c7 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c" status: 404 len: 111 microversion: 2.1 time: 0.036299 Sep 27 11:00:14.672195 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3800/7126] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:14 2022] GET /compute/v2.1/servers/fc6294ad-74fb-4566-b31f-e3fb4879642c => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 11:00:14.677212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e9d696f6-1a8e-40ce-9235-6d5935b2bcb3 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] Calling method '.version_select at 0x7f7f623db670>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:14.701416 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e9d696f6-1a8e-40ce-9235-6d5935b2bcb3 tempest-TestMinimumBasicScenario-1586443371 tempest-TestMinimumBasicScenario-1586443371-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1247088124" status: 202 len: 0 microversion: 2.1 time: 0.028212 Sep 27 11:00:14.702008 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3327/7127] 10.222.0.26 () {58 vars in 1272 bytes} [Tue Sep 27 11:00:14 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1247088124 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 11:00:24.428383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3c2bc081-d100-46cd-bab8-f62ad905f37b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'create', calling method: .version_select at 0x7f7f630a6d30>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-921838727", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFfTORO9pXnE7kdTOJ7l/nC/38U09XghuHc9wN3xn87xsk3QLZM+QA3rbWIrkajOxA+XoqV8V6RQOvpYibs301HI3TijJZiyDGne+M3gB/jaVWRJd6cr/trEkZVxjq6hqg=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:00:24.451701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3c2bc081-d100-46cd-bab8-f62ad905f37b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:00:24.458448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-3c2bc081-d100-46cd-bab8-f62ad905f37b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for user 3ec60ad271544bc99d7f394b9b06dd88 and project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:00:24.488914 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3c2bc081-d100-46cd-bab8-f62ad905f37b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 0.239555 Sep 27 11:00:24.489742 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3801/7128] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 11:00:24 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:00:30.638828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=47,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:30.640167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=22,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:30.644595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-785919682", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "06f2c411-a218-42b2-92f1-c357bf353b81"}], "key_name": "tempest-TestNetworkAdvancedServerOps-921838727", "security_groups": [{"name": "tempest-secgroup-smoke-572917147"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:00:30.805694 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] validate_networks() for [('06f2c411-a218-42b2-92f1-c357bf353b81', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 11:00:31.287984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 11:00:31.288793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 11:00:31.289981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 11:00:31.290847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 11:00:31.291506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 11:00:31.301764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:31.302675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:31.303338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:31.304303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:31.305054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:31.306753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:31.326580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:00:31.332043 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for user 3ec60ad271544bc99d7f394b9b06dd88 and project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:00:31.345860 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 11:00:31.348249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 11:00:31.399022 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6193b875-78df-4c6d-894e-fb726c9b2608 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.762443 Sep 27 11:00:31.399694 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3328/7129] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 11:00:30 2022] POST /compute/v2.1/servers => generated 405 bytes in 764 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 11:00:31.410581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=26 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:31.424306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-32d997b6-b093-44e0-a25a-3a047c47dc45 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:31.425636 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-32d997b6-b093-44e0-a25a-3a047c47dc45 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:31.501899 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-32d997b6-b093-44e0-a25a-3a047c47dc45 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:31.510053 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-32d997b6-b093-44e0-a25a-3a047c47dc45 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1265 microversion: 2.1 time: 0.100709 Sep 27 11:00:31.512759 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3802/7130] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:31 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1265 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 11:00:32.536259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-65826cb9-d422-4107-a102-5ed5adac49df tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:32.536936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-65826cb9-d422-4107-a102-5ed5adac49df tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:32.545277 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65826cb9-d422-4107-a102-5ed5adac49df tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:32.545787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65826cb9-d422-4107-a102-5ed5adac49df tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:32.546238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65826cb9-d422-4107-a102-5ed5adac49df tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:32.547009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65826cb9-d422-4107-a102-5ed5adac49df tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:32.547477 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65826cb9-d422-4107-a102-5ed5adac49df tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:32.547919 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-65826cb9-d422-4107-a102-5ed5adac49df tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:32.621084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-65826cb9-d422-4107-a102-5ed5adac49df tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:32.629690 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-65826cb9-d422-4107-a102-5ed5adac49df tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1317 microversion: 2.1 time: 0.098514 Sep 27 11:00:32.641602 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3329/7131] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:32 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1317 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:33.656775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b75fc07b-a8f8-4bd6-a7a7-4c994f37dfb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:33.658971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b75fc07b-a8f8-4bd6-a7a7-4c994f37dfb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:33.669083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b75fc07b-a8f8-4bd6-a7a7-4c994f37dfb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:33.669599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b75fc07b-a8f8-4bd6-a7a7-4c994f37dfb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:33.670040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b75fc07b-a8f8-4bd6-a7a7-4c994f37dfb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:33.671025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b75fc07b-a8f8-4bd6-a7a7-4c994f37dfb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:33.671492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b75fc07b-a8f8-4bd6-a7a7-4c994f37dfb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:33.671927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b75fc07b-a8f8-4bd6-a7a7-4c994f37dfb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:33.827971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b75fc07b-a8f8-4bd6-a7a7-4c994f37dfb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:33.837995 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b75fc07b-a8f8-4bd6-a7a7-4c994f37dfb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1390 microversion: 2.1 time: 0.188150 Sep 27 11:00:33.838832 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3803/7132] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:33 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1390 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:34.356851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-ee8e98c5-d15b-4df3-8219-89eac63a8465 req-09b94d07-9c19-40b1-ae8a-bc0e6b1715c8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9e167e09-70fd-4dc7-88c3-5daee95760fd", "tag": "bd035c17-9e6c-448f-85b2-455380ab3b8d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:00:34.371488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ee8e98c5-d15b-4df3-8219-89eac63a8465 req-09b94d07-9c19-40b1-ae8a-bc0e6b1715c8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:34.373401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ee8e98c5-d15b-4df3-8219-89eac63a8465 req-09b94d07-9c19-40b1-ae8a-bc0e6b1715c8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:34.374170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-ee8e98c5-d15b-4df3-8219-89eac63a8465 req-09b94d07-9c19-40b1-ae8a-bc0e6b1715c8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:34.389541 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-ee8e98c5-d15b-4df3-8219-89eac63a8465 req-09b94d07-9c19-40b1-ae8a-bc0e6b1715c8 service nova] Creating event network-changed:bd035c17-9e6c-448f-85b2-455380ab3b8d for instance 9e167e09-70fd-4dc7-88c3-5daee95760fd on nch2-master-1 Sep 27 11:00:34.396776 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-ee8e98c5-d15b-4df3-8219-89eac63a8465 req-09b94d07-9c19-40b1-ae8a-bc0e6b1715c8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051932 Sep 27 11:00:34.397451 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3330/7133] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:00:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:00:34.861467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-94991935-6d58-46a2-9028-c517a9430dc5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:34.862037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-94991935-6d58-46a2-9028-c517a9430dc5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:34.870589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94991935-6d58-46a2-9028-c517a9430dc5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:34.871151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94991935-6d58-46a2-9028-c517a9430dc5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:34.871733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94991935-6d58-46a2-9028-c517a9430dc5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:34.873020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94991935-6d58-46a2-9028-c517a9430dc5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:34.873645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94991935-6d58-46a2-9028-c517a9430dc5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:34.874156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-94991935-6d58-46a2-9028-c517a9430dc5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:35.024355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-94991935-6d58-46a2-9028-c517a9430dc5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:35.033480 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-94991935-6d58-46a2-9028-c517a9430dc5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1390 microversion: 2.1 time: 0.176244 Sep 27 11:00:35.034074 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3804/7134] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:34 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1390 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:36.060029 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-35e98ddc-e0e6-4739-9223-4dd1e04948a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:36.060803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-35e98ddc-e0e6-4739-9223-4dd1e04948a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:36.068801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35e98ddc-e0e6-4739-9223-4dd1e04948a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:36.069321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35e98ddc-e0e6-4739-9223-4dd1e04948a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:36.069779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35e98ddc-e0e6-4739-9223-4dd1e04948a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:36.070509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35e98ddc-e0e6-4739-9223-4dd1e04948a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:36.070986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35e98ddc-e0e6-4739-9223-4dd1e04948a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:36.071431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-35e98ddc-e0e6-4739-9223-4dd1e04948a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:36.209538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-35e98ddc-e0e6-4739-9223-4dd1e04948a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:36.219712 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-35e98ddc-e0e6-4739-9223-4dd1e04948a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1390 microversion: 2.1 time: 0.165816 Sep 27 11:00:36.220278 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3331/7135] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:36 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1390 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:37.243681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-78db292f-c86b-4b8f-8687-8c9eca50356a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:37.244300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-78db292f-c86b-4b8f-8687-8c9eca50356a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:37.250839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78db292f-c86b-4b8f-8687-8c9eca50356a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:37.251328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78db292f-c86b-4b8f-8687-8c9eca50356a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:37.251764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78db292f-c86b-4b8f-8687-8c9eca50356a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:37.252475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78db292f-c86b-4b8f-8687-8c9eca50356a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:37.253031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78db292f-c86b-4b8f-8687-8c9eca50356a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:37.253459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-78db292f-c86b-4b8f-8687-8c9eca50356a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:37.395745 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-78db292f-c86b-4b8f-8687-8c9eca50356a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:37.403620 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-78db292f-c86b-4b8f-8687-8c9eca50356a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1390 microversion: 2.1 time: 0.164246 Sep 27 11:00:37.404850 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3805/7136] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:37 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1390 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:38.433409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3274d4aa-d13b-4df8-b6f2-101c9a418923 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:38.434582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3274d4aa-d13b-4df8-b6f2-101c9a418923 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:38.441855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3274d4aa-d13b-4df8-b6f2-101c9a418923 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:38.442360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3274d4aa-d13b-4df8-b6f2-101c9a418923 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:38.442819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3274d4aa-d13b-4df8-b6f2-101c9a418923 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:38.443537 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3274d4aa-d13b-4df8-b6f2-101c9a418923 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:38.444012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3274d4aa-d13b-4df8-b6f2-101c9a418923 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:38.444450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3274d4aa-d13b-4df8-b6f2-101c9a418923 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:38.581726 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3274d4aa-d13b-4df8-b6f2-101c9a418923 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:38.590121 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3274d4aa-d13b-4df8-b6f2-101c9a418923 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1390 microversion: 2.1 time: 0.164896 Sep 27 11:00:38.590709 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3332/7137] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:38 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1390 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:39.614094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4c6d2ad2-9acf-404d-a8c8-8dcff3059ba2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:39.615118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4c6d2ad2-9acf-404d-a8c8-8dcff3059ba2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:39.617962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=36,nova_api:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:39.623922 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c6d2ad2-9acf-404d-a8c8-8dcff3059ba2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:39.624443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c6d2ad2-9acf-404d-a8c8-8dcff3059ba2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:39.625216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c6d2ad2-9acf-404d-a8c8-8dcff3059ba2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:39.625951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c6d2ad2-9acf-404d-a8c8-8dcff3059ba2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:39.626481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c6d2ad2-9acf-404d-a8c8-8dcff3059ba2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:39.626936 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4c6d2ad2-9acf-404d-a8c8-8dcff3059ba2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:39.767706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4c6d2ad2-9acf-404d-a8c8-8dcff3059ba2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:39.775700 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4c6d2ad2-9acf-404d-a8c8-8dcff3059ba2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1390 microversion: 2.1 time: 0.165682 Sep 27 11:00:39.776312 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3806/7138] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:39 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1390 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:40.804289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4d8b25f8-26f0-4828-baa9-9e45c7e1db65 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:40.805718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4d8b25f8-26f0-4828-baa9-9e45c7e1db65 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:40.817875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8b25f8-26f0-4828-baa9-9e45c7e1db65 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:40.818754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8b25f8-26f0-4828-baa9-9e45c7e1db65 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:40.819520 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8b25f8-26f0-4828-baa9-9e45c7e1db65 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:40.820746 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8b25f8-26f0-4828-baa9-9e45c7e1db65 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:40.821563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8b25f8-26f0-4828-baa9-9e45c7e1db65 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:40.822316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4d8b25f8-26f0-4828-baa9-9e45c7e1db65 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:40.977764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4d8b25f8-26f0-4828-baa9-9e45c7e1db65 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:40.986844 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4d8b25f8-26f0-4828-baa9-9e45c7e1db65 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1390 microversion: 2.1 time: 0.189300 Sep 27 11:00:40.987442 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3333/7139] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:40 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1390 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:41.157627 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-eeaa2f58-97bb-4704-9df8-6846c9fc3acc req-4099cea8-ac82-4b2f-8e11-0f0484b86b04 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9e167e09-70fd-4dc7-88c3-5daee95760fd", "name": "network-vif-plugged", "status": "completed", "tag": "bd035c17-9e6c-448f-85b2-455380ab3b8d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:00:41.168717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-eeaa2f58-97bb-4704-9df8-6846c9fc3acc req-4099cea8-ac82-4b2f-8e11-0f0484b86b04 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:41.169296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-eeaa2f58-97bb-4704-9df8-6846c9fc3acc req-4099cea8-ac82-4b2f-8e11-0f0484b86b04 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:41.169733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-eeaa2f58-97bb-4704-9df8-6846c9fc3acc req-4099cea8-ac82-4b2f-8e11-0f0484b86b04 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:41.181316 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-eeaa2f58-97bb-4704-9df8-6846c9fc3acc req-4099cea8-ac82-4b2f-8e11-0f0484b86b04 service nova] Creating event network-vif-plugged:bd035c17-9e6c-448f-85b2-455380ab3b8d for instance 9e167e09-70fd-4dc7-88c3-5daee95760fd on nch2-master-1 Sep 27 11:00:41.186575 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-eeaa2f58-97bb-4704-9df8-6846c9fc3acc req-4099cea8-ac82-4b2f-8e11-0f0484b86b04 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033424 Sep 27 11:00:41.187262 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3807/7140] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:00:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:00:42.012308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9910c98c-9030-453f-be49-11150773455b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:42.013106 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9910c98c-9030-453f-be49-11150773455b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:42.015065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:42.015820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:42.023315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910c98c-9030-453f-be49-11150773455b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:42.023828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910c98c-9030-453f-be49-11150773455b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:42.024295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910c98c-9030-453f-be49-11150773455b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:42.025057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910c98c-9030-453f-be49-11150773455b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:42.025539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910c98c-9030-453f-be49-11150773455b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:42.025990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9910c98c-9030-453f-be49-11150773455b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:42.173700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9910c98c-9030-453f-be49-11150773455b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:42.186419 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9910c98c-9030-453f-be49-11150773455b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1390 microversion: 2.1 time: 0.178161 Sep 27 11:00:42.186928 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3334/7141] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:42 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1390 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:43.207905 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:00:43.215373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7ce6ced2-77e7-41ce-bcf0-eaa8ef1b6e98 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:43.216349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7ce6ced2-77e7-41ce-bcf0-eaa8ef1b6e98 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:43.225667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ce6ced2-77e7-41ce-bcf0-eaa8ef1b6e98 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:43.226122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ce6ced2-77e7-41ce-bcf0-eaa8ef1b6e98 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:43.226644 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ce6ced2-77e7-41ce-bcf0-eaa8ef1b6e98 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:43.227631 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ce6ced2-77e7-41ce-bcf0-eaa8ef1b6e98 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:43.228255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ce6ced2-77e7-41ce-bcf0-eaa8ef1b6e98 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:43.228839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7ce6ced2-77e7-41ce-bcf0-eaa8ef1b6e98 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:43.386423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7ce6ced2-77e7-41ce-bcf0-eaa8ef1b6e98 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:43.394503 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7ce6ced2-77e7-41ce-bcf0-eaa8ef1b6e98 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1562 microversion: 2.1 time: 0.188340 Sep 27 11:00:43.395105 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3808/7142] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:43 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1562 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:43.421045 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6c61d9ec-a053-459d-9cf0-7224ee6f639d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:43.421721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6c61d9ec-a053-459d-9cf0-7224ee6f639d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:43.428857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c61d9ec-a053-459d-9cf0-7224ee6f639d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:43.429362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c61d9ec-a053-459d-9cf0-7224ee6f639d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:43.429835 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c61d9ec-a053-459d-9cf0-7224ee6f639d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:43.430553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c61d9ec-a053-459d-9cf0-7224ee6f639d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:43.431020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c61d9ec-a053-459d-9cf0-7224ee6f639d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:43.431455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6c61d9ec-a053-459d-9cf0-7224ee6f639d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:43.562848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6c61d9ec-a053-459d-9cf0-7224ee6f639d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:43.570928 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6c61d9ec-a053-459d-9cf0-7224ee6f639d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1562 microversion: 2.1 time: 0.153887 Sep 27 11:00:43.571514 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3335/7143] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:43 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1562 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:00:45.476624 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-11480c6b-35f4-4f64-923f-9f691b8c22e0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:00:45.477274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-11480c6b-35f4-4f64-923f-9f691b8c22e0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:00:45.481686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-fb85661a-efa7-4581-8c98-d741efd30d4a req-f5fd39e3-1ca3-4b38-97ab-9456bfa86058 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9e167e09-70fd-4dc7-88c3-5daee95760fd", "tag": "bd035c17-9e6c-448f-85b2-455380ab3b8d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:00:45.484838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11480c6b-35f4-4f64-923f-9f691b8c22e0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:45.486021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11480c6b-35f4-4f64-923f-9f691b8c22e0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:45.486750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11480c6b-35f4-4f64-923f-9f691b8c22e0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:45.487755 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11480c6b-35f4-4f64-923f-9f691b8c22e0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:45.488420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11480c6b-35f4-4f64-923f-9f691b8c22e0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:45.488887 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-11480c6b-35f4-4f64-923f-9f691b8c22e0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:45.491777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fb85661a-efa7-4581-8c98-d741efd30d4a req-f5fd39e3-1ca3-4b38-97ab-9456bfa86058 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:00:45.492282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fb85661a-efa7-4581-8c98-d741efd30d4a req-f5fd39e3-1ca3-4b38-97ab-9456bfa86058 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:00:45.492755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fb85661a-efa7-4581-8c98-d741efd30d4a req-f5fd39e3-1ca3-4b38-97ab-9456bfa86058 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:00:45.511869 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-fb85661a-efa7-4581-8c98-d741efd30d4a req-f5fd39e3-1ca3-4b38-97ab-9456bfa86058 service nova] Creating event network-changed:bd035c17-9e6c-448f-85b2-455380ab3b8d for instance 9e167e09-70fd-4dc7-88c3-5daee95760fd on nch2-master-1 Sep 27 11:00:45.520271 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-fb85661a-efa7-4581-8c98-d741efd30d4a req-f5fd39e3-1ca3-4b38-97ab-9456bfa86058 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042263 Sep 27 11:00:45.522498 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3336/7144] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:00:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:00:45.650202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-11480c6b-35f4-4f64-923f-9f691b8c22e0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:00:45.659529 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-11480c6b-35f4-4f64-923f-9f691b8c22e0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1562 microversion: 2.1 time: 0.186795 Sep 27 11:00:45.660596 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3809/7145] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:00:45 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1562 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:01:19.016593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=31,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:01:19.017365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=42 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:01:19.022105 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-29d93744-b25c-4d66-906c-a123def9d358 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:01:19.024540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-29d93744-b25c-4d66-906c-a123def9d358 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:19.032389 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d93744-b25c-4d66-906c-a123def9d358 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:19.033076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d93744-b25c-4d66-906c-a123def9d358 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:19.033535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d93744-b25c-4d66-906c-a123def9d358 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:19.034261 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d93744-b25c-4d66-906c-a123def9d358 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:19.034735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d93744-b25c-4d66-906c-a123def9d358 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:19.035177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29d93744-b25c-4d66-906c-a123def9d358 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:19.115571 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-29d93744-b25c-4d66-906c-a123def9d358 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd/action" status: 200 len: 24567 microversion: 2.1 time: 0.100804 Sep 27 11:01:19.116240 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3337/7146] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:01:19 2022] POST /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd/action => generated 24567 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:01:49.600079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:01:49.600976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=38 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:01:49.605987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1c87aec1-5dfc-4d44-8afa-dfc9ca849c93 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"pause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:01:49.606552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1c87aec1-5dfc-4d44-8afa-dfc9ca849c93 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:49.614125 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c87aec1-5dfc-4d44-8afa-dfc9ca849c93 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:49.614634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c87aec1-5dfc-4d44-8afa-dfc9ca849c93 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:49.615117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c87aec1-5dfc-4d44-8afa-dfc9ca849c93 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:49.616194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c87aec1-5dfc-4d44-8afa-dfc9ca849c93 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:49.616685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c87aec1-5dfc-4d44-8afa-dfc9ca849c93 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:49.617151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1c87aec1-5dfc-4d44-8afa-dfc9ca849c93 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:49.687713 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1c87aec1-5dfc-4d44-8afa-dfc9ca849c93 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd/action" status: 202 len: 0 microversion: 2.1 time: 0.090626 Sep 27 11:01:49.688390 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3810/7147] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:01:49 2022] POST /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd/action => generated 0 bytes in 93 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:01:49.698934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7b24ec90-5296-4af9-aa2b-021b6aa047fa tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:01:49.699645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7b24ec90-5296-4af9-aa2b-021b6aa047fa tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:49.703441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:01:49.704132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:01:49.710558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b24ec90-5296-4af9-aa2b-021b6aa047fa tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:49.711104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b24ec90-5296-4af9-aa2b-021b6aa047fa tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:49.711577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b24ec90-5296-4af9-aa2b-021b6aa047fa tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:49.713022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b24ec90-5296-4af9-aa2b-021b6aa047fa tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:49.713648 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b24ec90-5296-4af9-aa2b-021b6aa047fa tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:49.714094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7b24ec90-5296-4af9-aa2b-021b6aa047fa tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:49.962671 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7b24ec90-5296-4af9-aa2b-021b6aa047fa tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:01:49.971765 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7b24ec90-5296-4af9-aa2b-021b6aa047fa tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1686 microversion: 2.1 time: 0.277173 Sep 27 11:01:49.972351 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3338/7148] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:01:49 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1686 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:01:51.002236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0c3806e-bfcf-44e4-b2a1-7fe72a94a200 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:01:51.003084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b0c3806e-bfcf-44e4-b2a1-7fe72a94a200 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:51.010802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0c3806e-bfcf-44e4-b2a1-7fe72a94a200 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:51.011296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0c3806e-bfcf-44e4-b2a1-7fe72a94a200 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:51.011748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0c3806e-bfcf-44e4-b2a1-7fe72a94a200 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:51.012443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0c3806e-bfcf-44e4-b2a1-7fe72a94a200 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:51.012945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0c3806e-bfcf-44e4-b2a1-7fe72a94a200 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:51.013376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0c3806e-bfcf-44e4-b2a1-7fe72a94a200 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:51.217058 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b0c3806e-bfcf-44e4-b2a1-7fe72a94a200 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:01:51.225639 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b0c3806e-bfcf-44e4-b2a1-7fe72a94a200 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1666 microversion: 2.1 time: 0.231189 Sep 27 11:01:51.226236 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3811/7149] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:01:50 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1666 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:01:52.338224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9ea399db-d84f-4860-88aa-ee1221515619 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:01:52.341913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9ea399db-d84f-4860-88aa-ee1221515619 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:52.353609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ea399db-d84f-4860-88aa-ee1221515619 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:52.354114 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ea399db-d84f-4860-88aa-ee1221515619 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:52.354570 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ea399db-d84f-4860-88aa-ee1221515619 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:52.355299 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ea399db-d84f-4860-88aa-ee1221515619 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:52.355776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ea399db-d84f-4860-88aa-ee1221515619 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:52.356220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ea399db-d84f-4860-88aa-ee1221515619 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:52.424330 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9ea399db-d84f-4860-88aa-ee1221515619 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd/action" status: 200 len: 32639 microversion: 2.1 time: 0.092510 Sep 27 11:01:52.425352 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3339/7150] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:01:52 2022] POST /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd/action => generated 32639 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:01:52.443836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ff37ecda-29aa-4020-8d0c-ddcc3235b36f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"unpause": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:01:52.444635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ff37ecda-29aa-4020-8d0c-ddcc3235b36f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:52.452107 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff37ecda-29aa-4020-8d0c-ddcc3235b36f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:52.452615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff37ecda-29aa-4020-8d0c-ddcc3235b36f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:52.453082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff37ecda-29aa-4020-8d0c-ddcc3235b36f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:52.453801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff37ecda-29aa-4020-8d0c-ddcc3235b36f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:52.454266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff37ecda-29aa-4020-8d0c-ddcc3235b36f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:52.454703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ff37ecda-29aa-4020-8d0c-ddcc3235b36f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:52.537060 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ff37ecda-29aa-4020-8d0c-ddcc3235b36f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd/action" status: 202 len: 0 microversion: 2.1 time: 0.099314 Sep 27 11:01:52.538244 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3812/7151] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:01:52 2022] POST /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd/action => generated 0 bytes in 102 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:01:52.547479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-67e8f688-e450-4c8c-88a6-0f72253f5fac tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:01:52.548227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-67e8f688-e450-4c8c-88a6-0f72253f5fac tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:52.556329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e8f688-e450-4c8c-88a6-0f72253f5fac tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:52.557015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e8f688-e450-4c8c-88a6-0f72253f5fac tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:52.557541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e8f688-e450-4c8c-88a6-0f72253f5fac tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:52.558340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e8f688-e450-4c8c-88a6-0f72253f5fac tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:52.558852 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e8f688-e450-4c8c-88a6-0f72253f5fac tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:52.559294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-67e8f688-e450-4c8c-88a6-0f72253f5fac tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:52.711123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-67e8f688-e450-4c8c-88a6-0f72253f5fac tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:01:52.719744 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-67e8f688-e450-4c8c-88a6-0f72253f5fac tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1673 microversion: 2.1 time: 0.176728 Sep 27 11:01:52.720329 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3340/7152] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:01:52 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1673 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:01:53.757985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aa34a59b-b204-47a5-8a84-e197a8b253c3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:01:53.759151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aa34a59b-b204-47a5-8a84-e197a8b253c3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:53.770104 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa34a59b-b204-47a5-8a84-e197a8b253c3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:53.770980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa34a59b-b204-47a5-8a84-e197a8b253c3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:53.771701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa34a59b-b204-47a5-8a84-e197a8b253c3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:53.772973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa34a59b-b204-47a5-8a84-e197a8b253c3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:53.773261 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa34a59b-b204-47a5-8a84-e197a8b253c3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:53.773686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aa34a59b-b204-47a5-8a84-e197a8b253c3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:53.920381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-aa34a59b-b204-47a5-8a84-e197a8b253c3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:01:53.929229 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aa34a59b-b204-47a5-8a84-e197a8b253c3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1673 microversion: 2.1 time: 0.179806 Sep 27 11:01:53.929836 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3813/7153] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:01:53 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1673 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:01:54.957544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9765bf95-3d18-4ff5-bca0-0b34d9246708 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:01:54.958460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9765bf95-3d18-4ff5-bca0-0b34d9246708 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:54.971417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9765bf95-3d18-4ff5-bca0-0b34d9246708 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:54.972256 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9765bf95-3d18-4ff5-bca0-0b34d9246708 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:54.973085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9765bf95-3d18-4ff5-bca0-0b34d9246708 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:54.974291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9765bf95-3d18-4ff5-bca0-0b34d9246708 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:54.975084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9765bf95-3d18-4ff5-bca0-0b34d9246708 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:54.976264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9765bf95-3d18-4ff5-bca0-0b34d9246708 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:55.121986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9765bf95-3d18-4ff5-bca0-0b34d9246708 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:01:55.130017 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9765bf95-3d18-4ff5-bca0-0b34d9246708 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1673 microversion: 2.1 time: 0.179418 Sep 27 11:01:55.130604 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3341/7154] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:01:54 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1673 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:01:56.156284 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a344f650-f554-40fc-97b8-a38636482515 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:01:56.157487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a344f650-f554-40fc-97b8-a38636482515 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:56.167074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a344f650-f554-40fc-97b8-a38636482515 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:56.167725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a344f650-f554-40fc-97b8-a38636482515 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:56.168302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a344f650-f554-40fc-97b8-a38636482515 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:56.169372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a344f650-f554-40fc-97b8-a38636482515 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:56.169978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a344f650-f554-40fc-97b8-a38636482515 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:56.170545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a344f650-f554-40fc-97b8-a38636482515 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:56.311204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a344f650-f554-40fc-97b8-a38636482515 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:01:56.319603 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a344f650-f554-40fc-97b8-a38636482515 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1673 microversion: 2.1 time: 0.168650 Sep 27 11:01:56.320150 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3814/7155] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:01:56 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1673 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:01:57.347710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-449f183a-cba3-48ef-ad25-1f288a609457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:01:57.348794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-449f183a-cba3-48ef-ad25-1f288a609457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:57.357620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-449f183a-cba3-48ef-ad25-1f288a609457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:57.358131 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-449f183a-cba3-48ef-ad25-1f288a609457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:57.358590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-449f183a-cba3-48ef-ad25-1f288a609457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:57.359309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-449f183a-cba3-48ef-ad25-1f288a609457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:57.359796 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-449f183a-cba3-48ef-ad25-1f288a609457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:57.360233 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-449f183a-cba3-48ef-ad25-1f288a609457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:57.498447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-449f183a-cba3-48ef-ad25-1f288a609457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:01:57.510003 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-449f183a-cba3-48ef-ad25-1f288a609457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1673 microversion: 2.1 time: 0.168760 Sep 27 11:01:57.510815 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3342/7156] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:01:57 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1673 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:01:58.543033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c36b9403-e980-4b82-b013-c0facac42578 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:01:58.544003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c36b9403-e980-4b82-b013-c0facac42578 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:58.556089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c36b9403-e980-4b82-b013-c0facac42578 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:58.556584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c36b9403-e980-4b82-b013-c0facac42578 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:58.557067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c36b9403-e980-4b82-b013-c0facac42578 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:58.557912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c36b9403-e980-4b82-b013-c0facac42578 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:58.558503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c36b9403-e980-4b82-b013-c0facac42578 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:58.558976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c36b9403-e980-4b82-b013-c0facac42578 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:58.698315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c36b9403-e980-4b82-b013-c0facac42578 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:01:58.706887 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c36b9403-e980-4b82-b013-c0facac42578 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1681 microversion: 2.1 time: 0.170554 Sep 27 11:01:58.707443 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3815/7157] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:01:58 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1681 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:01:58.813814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a2365486-731d-4be1-90bb-a29db435ea18 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:01:58.816079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a2365486-731d-4be1-90bb-a29db435ea18 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:01:58.823685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2365486-731d-4be1-90bb-a29db435ea18 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:58.824189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2365486-731d-4be1-90bb-a29db435ea18 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:58.824644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2365486-731d-4be1-90bb-a29db435ea18 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:58.825462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2365486-731d-4be1-90bb-a29db435ea18 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:01:58.825934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2365486-731d-4be1-90bb-a29db435ea18 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:01:58.826414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a2365486-731d-4be1-90bb-a29db435ea18 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:01:58.890284 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a2365486-731d-4be1-90bb-a29db435ea18 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd/action" status: 200 len: 38428 microversion: 2.1 time: 0.080562 Sep 27 11:01:58.890968 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3343/7158] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:01:58 2022] POST /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd/action => generated 38428 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:02:00.216190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-57389a37-d7ff-4775-90ce-2912e723d41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:00.216753 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-57389a37-d7ff-4775-90ce-2912e723d41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:00.226978 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-72232769-61df-497d-8d00-e2a0d13df601 req-b7720d01-6b4c-473e-9360-a00b4701a9c7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9e167e09-70fd-4dc7-88c3-5daee95760fd", "tag": "bd035c17-9e6c-448f-85b2-455380ab3b8d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:02:00.234315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57389a37-d7ff-4775-90ce-2912e723d41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:00.234810 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57389a37-d7ff-4775-90ce-2912e723d41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:00.235277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57389a37-d7ff-4775-90ce-2912e723d41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:00.236038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57389a37-d7ff-4775-90ce-2912e723d41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:00.236690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57389a37-d7ff-4775-90ce-2912e723d41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:00.237183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-57389a37-d7ff-4775-90ce-2912e723d41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:00.238238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-72232769-61df-497d-8d00-e2a0d13df601 req-b7720d01-6b4c-473e-9360-a00b4701a9c7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:00.238729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-72232769-61df-497d-8d00-e2a0d13df601 req-b7720d01-6b4c-473e-9360-a00b4701a9c7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:00.239174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-72232769-61df-497d-8d00-e2a0d13df601 req-b7720d01-6b4c-473e-9360-a00b4701a9c7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:00.251278 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-72232769-61df-497d-8d00-e2a0d13df601 req-b7720d01-6b4c-473e-9360-a00b4701a9c7 service nova] Creating event network-changed:bd035c17-9e6c-448f-85b2-455380ab3b8d for instance 9e167e09-70fd-4dc7-88c3-5daee95760fd on nch2-master-1 Sep 27 11:02:00.256913 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-72232769-61df-497d-8d00-e2a0d13df601 req-b7720d01-6b4c-473e-9360-a00b4701a9c7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034818 Sep 27 11:02:00.257843 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3344/7159] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:02:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:02:00.269475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-57389a37-d7ff-4775-90ce-2912e723d41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 11:02:00.359781 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-57389a37-d7ff-4775-90ce-2912e723d41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "DELETE /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 204 len: 0 microversion: 2.1 time: 0.147295 Sep 27 11:02:00.361771 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3816/7160] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 11:02:00 2022] DELETE /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 11:02:00.371439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8caab1a6-fe22-4afd-9ed6-3d8274c50541 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:00.372488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8caab1a6-fe22-4afd-9ed6-3d8274c50541 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:00.378833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8caab1a6-fe22-4afd-9ed6-3d8274c50541 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:00.379337 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8caab1a6-fe22-4afd-9ed6-3d8274c50541 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:00.379787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8caab1a6-fe22-4afd-9ed6-3d8274c50541 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:00.380509 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8caab1a6-fe22-4afd-9ed6-3d8274c50541 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:00.381006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8caab1a6-fe22-4afd-9ed6-3d8274c50541 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:00.381447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8caab1a6-fe22-4afd-9ed6-3d8274c50541 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:00.546159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8caab1a6-fe22-4afd-9ed6-3d8274c50541 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:00.554403 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8caab1a6-fe22-4afd-9ed6-3d8274c50541 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1687 microversion: 2.1 time: 0.187983 Sep 27 11:02:00.555045 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3345/7161] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:00 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1687 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:01.585870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed107b0b-8c17-444a-8590-22bcb9326606 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:01.586805 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed107b0b-8c17-444a-8590-22bcb9326606 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:01.594231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed107b0b-8c17-444a-8590-22bcb9326606 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:01.594730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed107b0b-8c17-444a-8590-22bcb9326606 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:01.595203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed107b0b-8c17-444a-8590-22bcb9326606 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:01.595898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed107b0b-8c17-444a-8590-22bcb9326606 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:01.596364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed107b0b-8c17-444a-8590-22bcb9326606 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:01.596839 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed107b0b-8c17-444a-8590-22bcb9326606 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:01.742268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ed107b0b-8c17-444a-8590-22bcb9326606 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:01.751745 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed107b0b-8c17-444a-8590-22bcb9326606 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1687 microversion: 2.1 time: 0.174952 Sep 27 11:02:01.752332 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3817/7162] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:01 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1687 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:02.670055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e6e555b7-17df-41d3-949b-cf75056c02be req-49d35e43-1bc9-4809-a462-6141875d8d0f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9e167e09-70fd-4dc7-88c3-5daee95760fd", "name": "network-vif-deleted", "tag": "bd035c17-9e6c-448f-85b2-455380ab3b8d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:02:02.683457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e6e555b7-17df-41d3-949b-cf75056c02be req-49d35e43-1bc9-4809-a462-6141875d8d0f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:02.683945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e6e555b7-17df-41d3-949b-cf75056c02be req-49d35e43-1bc9-4809-a462-6141875d8d0f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:02.684380 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e6e555b7-17df-41d3-949b-cf75056c02be req-49d35e43-1bc9-4809-a462-6141875d8d0f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:02.697312 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e6e555b7-17df-41d3-949b-cf75056c02be req-49d35e43-1bc9-4809-a462-6141875d8d0f service nova] Creating event network-vif-deleted:bd035c17-9e6c-448f-85b2-455380ab3b8d for instance 9e167e09-70fd-4dc7-88c3-5daee95760fd on nch2-master-1 Sep 27 11:02:02.703380 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e6e555b7-17df-41d3-949b-cf75056c02be req-49d35e43-1bc9-4809-a462-6141875d8d0f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036896 Sep 27 11:02:02.703989 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3346/7163] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:02:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:02:02.776388 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:02.777080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:02.784747 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:02.785273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:02.785720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:02.786433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:02.786896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:02.787327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:02.825182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:02.825578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:02.826040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:02.826497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lazy-loading 'fault' on Instance uuid 9e167e09-70fd-4dc7-88c3-5daee95760fd {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 11:02:02.871943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:02.880418 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b71a9928-9d44-4e25-a53b-09e1717b613f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 200 len: 1355 microversion: 2.1 time: 0.108596 Sep 27 11:02:02.881037 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3818/7164] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:02 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 1355 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:03.906504 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:03.907596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 9e167e09-70fd-4dc7-88c3-5daee95760fd] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:03.917127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:03.917632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:03.918088 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:03.918809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:03.919298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:03.919705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:03.934757 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] HTTP exception thrown: Instance 9e167e09-70fd-4dc7-88c3-5daee95760fd could not be found. Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Request method failure captured: Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd HTTP/1.1 Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtfHoJ5jcAxOLtRcoV1nD5XXlTZzK0A-5egTQGd5lTVNdkOuvT5Fs0L9J-caES9KtErWFoSb_09yL5egcLFPiM4N0_ALLienspVX9-DRrIBhK2koE7PdcRR57IH0CHGGILEi4fmfkKfiXRleQCLI0R176myMkB92QmVT86Qtqhsg2-o Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 55b42a416dd942e780e533e4c6f2a8b4 Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_55b42a416dd942e780e533e4c6f2a8b4"}]}] Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 55b42a416dd942e780e533e4c6f2a8b4 Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestNetworkAdvancedServerOps-1003182190-project Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 3ec60ad271544bc99d7f394b9b06dd88 Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1003182190-project Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 9e167e09-70fd-4dc7-88c3-5daee95760fd could not be found. Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '9e167e09-70fd-4dc7-88c3-5daee95760fd'} Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9e167e09-70fd-4dc7-88c3-5daee95760fd could not be found. Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9e167e09-70fd-4dc7-88c3-5daee95760fd could not be found. Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:02:03.936431 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:02:03.942055 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:02:03.942055 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:02:03.942055 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:02:03.942055 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:02:03.942055 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 11:02:03.942055 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 11:02:03.942055 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9e167e09-70fd-4dc7-88c3-5daee95760fd could not be found. Sep 27 11:02:03.942055 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:02:03.942055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Returning 404 to user: Instance 9e167e09-70fd-4dc7-88c3-5daee95760fd could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 11:02:03.942055 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c06e37e2-3bb7-4720-b67e-2a7f33499bc7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd" status: 404 len: 111 microversion: 2.1 time: 0.038014 Sep 27 11:02:03.942055 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3347/7165] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:03 2022] GET /compute/v2.1/servers/9e167e09-70fd-4dc7-88c3-5daee95760fd => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 11:02:07.333599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7fed5af0-4648-48ba-b5e9-d89252cf3e9b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '.version_select at 0x7f7f628fe0d0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:07.368694 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7fed5af0-4648-48ba-b5e9-d89252cf3e9b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-921838727" status: 202 len: 0 microversion: 2.1 time: 0.039603 Sep 27 11:02:07.369900 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3819/7166] 10.222.0.26 () {58 vars in 1281 bytes} [Tue Sep 27 11:02:07 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-921838727 => generated 0 bytes in 43 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:02:07.402564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1651ac1a-dea3-456e-ac21-429f0513897b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'create', calling method: .version_select at 0x7f7f62b5aca0>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-1806921382", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLUkJACMnyTX7YPS53vD8MM6/l5jqq2FH8j5R/iheaUCat+70IounxTW0+ecUtdvxqD9dd81gq7yL2I1U068tn9p0j6SqIGgSSeThg9G6cU1WoYUTVvQq8SUJbJi6EYDcQ=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:02:07.426663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-1651ac1a-dea3-456e-ac21-429f0513897b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:02:07.432309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-1651ac1a-dea3-456e-ac21-429f0513897b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for user 3ec60ad271544bc99d7f394b9b06dd88 and project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:02:07.458215 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1651ac1a-dea3-456e-ac21-429f0513897b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.060599 Sep 27 11:02:07.458851 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3348/7167] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 11:02:07 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:02:13.018792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1683237456", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "af34560b-8285-4edf-8cee-725e2fa35cbb"}], "key_name": "tempest-TestNetworkAdvancedServerOps-1806921382", "security_groups": [{"name": "tempest-secgroup-smoke-13240781"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:02:13.032713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=72,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:13.151701 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] validate_networks() for [('af34560b-8285-4edf-8cee-725e2fa35cbb', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 11:02:13.720207 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 11:02:13.720645 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 11:02:13.721169 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 11:02:13.721585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 11:02:13.722040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 11:02:13.733612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:13.734094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:13.734536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:13.735271 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:13.735728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:13.736297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:13.754435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:02:13.760756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for user 3ec60ad271544bc99d7f394b9b06dd88 and project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:02:13.774607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 11:02:13.776550 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 11:02:13.828245 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d75ab2ed-fd61-4eb4-bd24-85e444971eee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 404 microversion: 2.1 time: 0.813580 Sep 27 11:02:13.829308 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3820/7168] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 11:02:13 2022] POST /compute/v2.1/servers => generated 404 bytes in 815 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 11:02:13.842410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9f7475ec-c838-492c-a33a-a7297d199099 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:13.843173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9f7475ec-c838-492c-a33a-a7297d199099 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:13.906322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9f7475ec-c838-492c-a33a-a7297d199099 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:13.917307 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9f7475ec-c838-492c-a33a-a7297d199099 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1267 microversion: 2.1 time: 0.080919 Sep 27 11:02:13.917976 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3349/7169] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:13 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1267 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 11:02:14.943467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-dab0d730-1001-44d7-b628-d30b0aaf7c6d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:14.944111 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-dab0d730-1001-44d7-b628-d30b0aaf7c6d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:14.951776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab0d730-1001-44d7-b628-d30b0aaf7c6d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:14.952281 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab0d730-1001-44d7-b628-d30b0aaf7c6d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:14.952760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab0d730-1001-44d7-b628-d30b0aaf7c6d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:14.953488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab0d730-1001-44d7-b628-d30b0aaf7c6d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:14.953952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab0d730-1001-44d7-b628-d30b0aaf7c6d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:14.954381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-dab0d730-1001-44d7-b628-d30b0aaf7c6d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:15.045015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-dab0d730-1001-44d7-b628-d30b0aaf7c6d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:15.054595 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-dab0d730-1001-44d7-b628-d30b0aaf7c6d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1327 microversion: 2.1 time: 0.117812 Sep 27 11:02:15.068730 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3821/7170] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:14 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1327 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:16.081927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ec9015cc-b150-4094-8696-7d62f8129a47 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:16.083468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ec9015cc-b150-4094-8696-7d62f8129a47 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:16.094870 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=44 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:16.108369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9015cc-b150-4094-8696-7d62f8129a47 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:16.109551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9015cc-b150-4094-8696-7d62f8129a47 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:16.110271 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9015cc-b150-4094-8696-7d62f8129a47 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:16.111432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9015cc-b150-4094-8696-7d62f8129a47 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:16.112142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9015cc-b150-4094-8696-7d62f8129a47 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:16.112858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ec9015cc-b150-4094-8696-7d62f8129a47 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:16.474112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ec9015cc-b150-4094-8696-7d62f8129a47 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:16.482433 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ec9015cc-b150-4094-8696-7d62f8129a47 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1391 microversion: 2.1 time: 0.406958 Sep 27 11:02:16.486894 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3350/7171] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:16 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1391 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:16.869141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-24425655-2d1e-4715-8993-8fc21c458e97 req-e24212ea-3421-462a-aab4-88cd795d2f61 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d02fb6df-802d-438f-be1a-a9e733427ebc", "tag": "9e2fa624-8544-4495-bc6f-ce199ad290fa"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:02:16.882413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-24425655-2d1e-4715-8993-8fc21c458e97 req-e24212ea-3421-462a-aab4-88cd795d2f61 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:16.882925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-24425655-2d1e-4715-8993-8fc21c458e97 req-e24212ea-3421-462a-aab4-88cd795d2f61 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:16.883361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-24425655-2d1e-4715-8993-8fc21c458e97 req-e24212ea-3421-462a-aab4-88cd795d2f61 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:16.902052 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-24425655-2d1e-4715-8993-8fc21c458e97 req-e24212ea-3421-462a-aab4-88cd795d2f61 service nova] Creating event network-changed:9e2fa624-8544-4495-bc6f-ce199ad290fa for instance d02fb6df-802d-438f-be1a-a9e733427ebc on nch1-master-1 Sep 27 11:02:16.908428 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-24425655-2d1e-4715-8993-8fc21c458e97 req-e24212ea-3421-462a-aab4-88cd795d2f61 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044194 Sep 27 11:02:16.909098 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3822/7172] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:02:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:02:17.506980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1f2468f1-8fa6-42a8-8b61-70448f3c5cb9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:17.507567 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1f2468f1-8fa6-42a8-8b61-70448f3c5cb9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:17.515084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f2468f1-8fa6-42a8-8b61-70448f3c5cb9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:17.515554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f2468f1-8fa6-42a8-8b61-70448f3c5cb9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:17.516047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f2468f1-8fa6-42a8-8b61-70448f3c5cb9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:17.516810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f2468f1-8fa6-42a8-8b61-70448f3c5cb9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:17.517287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f2468f1-8fa6-42a8-8b61-70448f3c5cb9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:17.517757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1f2468f1-8fa6-42a8-8b61-70448f3c5cb9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:17.663882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1f2468f1-8fa6-42a8-8b61-70448f3c5cb9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:17.672545 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1f2468f1-8fa6-42a8-8b61-70448f3c5cb9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1391 microversion: 2.1 time: 0.169798 Sep 27 11:02:17.673217 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3351/7173] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:17 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1391 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:18.703059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e6782482-e066-4fae-925b-9fbcd07ec668 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:18.704238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e6782482-e066-4fae-925b-9fbcd07ec668 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:18.714553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6782482-e066-4fae-925b-9fbcd07ec668 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:18.715048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6782482-e066-4fae-925b-9fbcd07ec668 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:18.715487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6782482-e066-4fae-925b-9fbcd07ec668 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:18.716199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6782482-e066-4fae-925b-9fbcd07ec668 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:18.716697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6782482-e066-4fae-925b-9fbcd07ec668 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:18.717116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e6782482-e066-4fae-925b-9fbcd07ec668 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:18.861892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e6782482-e066-4fae-925b-9fbcd07ec668 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:18.870703 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e6782482-e066-4fae-925b-9fbcd07ec668 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1391 microversion: 2.1 time: 0.177161 Sep 27 11:02:18.871261 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3823/7174] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:18 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1391 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:19.896323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5f339f76-ce69-4e75-a5ab-2cab6f508b76 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:19.897018 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5f339f76-ce69-4e75-a5ab-2cab6f508b76 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:19.901379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=40,nova_api:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:19.908282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f339f76-ce69-4e75-a5ab-2cab6f508b76 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:19.909087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f339f76-ce69-4e75-a5ab-2cab6f508b76 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:19.909597 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f339f76-ce69-4e75-a5ab-2cab6f508b76 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:19.910414 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f339f76-ce69-4e75-a5ab-2cab6f508b76 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:19.910923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f339f76-ce69-4e75-a5ab-2cab6f508b76 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:19.911539 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f339f76-ce69-4e75-a5ab-2cab6f508b76 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:20.059805 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5f339f76-ce69-4e75-a5ab-2cab6f508b76 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:20.067792 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5f339f76-ce69-4e75-a5ab-2cab6f508b76 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1391 microversion: 2.1 time: 0.176891 Sep 27 11:02:20.068469 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3352/7175] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:19 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1391 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:21.096555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-13ce8d7e-7443-4f62-83a2-80ef644d12f7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:21.098607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-13ce8d7e-7443-4f62-83a2-80ef644d12f7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:21.102427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=50,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:21.108589 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13ce8d7e-7443-4f62-83a2-80ef644d12f7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:21.109102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13ce8d7e-7443-4f62-83a2-80ef644d12f7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:21.109545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13ce8d7e-7443-4f62-83a2-80ef644d12f7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:21.110269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13ce8d7e-7443-4f62-83a2-80ef644d12f7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:21.110727 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13ce8d7e-7443-4f62-83a2-80ef644d12f7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:21.111154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-13ce8d7e-7443-4f62-83a2-80ef644d12f7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:21.255862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-13ce8d7e-7443-4f62-83a2-80ef644d12f7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:21.264866 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-13ce8d7e-7443-4f62-83a2-80ef644d12f7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1391 microversion: 2.1 time: 0.174891 Sep 27 11:02:21.265479 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3824/7176] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:21 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1391 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:21.571459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-e636b3e6-cd1f-49e3-b3c6-fca43f569333 req-40d5dcef-ed93-4533-a298-eee23a3e9422 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d02fb6df-802d-438f-be1a-a9e733427ebc", "name": "network-vif-plugged", "status": "completed", "tag": "9e2fa624-8544-4495-bc6f-ce199ad290fa"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:02:21.584210 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e636b3e6-cd1f-49e3-b3c6-fca43f569333 req-40d5dcef-ed93-4533-a298-eee23a3e9422 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:21.584765 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e636b3e6-cd1f-49e3-b3c6-fca43f569333 req-40d5dcef-ed93-4533-a298-eee23a3e9422 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:21.585221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-e636b3e6-cd1f-49e3-b3c6-fca43f569333 req-40d5dcef-ed93-4533-a298-eee23a3e9422 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:21.597006 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-e636b3e6-cd1f-49e3-b3c6-fca43f569333 req-40d5dcef-ed93-4533-a298-eee23a3e9422 service nova] Creating event network-vif-plugged:9e2fa624-8544-4495-bc6f-ce199ad290fa for instance d02fb6df-802d-438f-be1a-a9e733427ebc on nch1-master-1 Sep 27 11:02:21.602208 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-e636b3e6-cd1f-49e3-b3c6-fca43f569333 req-40d5dcef-ed93-4533-a298-eee23a3e9422 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034911 Sep 27 11:02:21.603112 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3353/7177] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:02:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:02:22.295448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1406afc3-992d-4cc2-9987-215d50e14e77 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:22.296536 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1406afc3-992d-4cc2-9987-215d50e14e77 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:22.308018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1406afc3-992d-4cc2-9987-215d50e14e77 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:22.308387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1406afc3-992d-4cc2-9987-215d50e14e77 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:22.308895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1406afc3-992d-4cc2-9987-215d50e14e77 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:22.309590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1406afc3-992d-4cc2-9987-215d50e14e77 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:22.310045 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1406afc3-992d-4cc2-9987-215d50e14e77 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:22.310474 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1406afc3-992d-4cc2-9987-215d50e14e77 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:22.444896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1406afc3-992d-4cc2-9987-215d50e14e77 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:22.457974 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1406afc3-992d-4cc2-9987-215d50e14e77 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1391 microversion: 2.1 time: 0.169304 Sep 27 11:02:22.458642 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3825/7178] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:22 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1391 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:23.483689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3c22d84c-ae43-4ab6-9602-3f41264e6870 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:23.484440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3c22d84c-ae43-4ab6-9602-3f41264e6870 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:23.491167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c22d84c-ae43-4ab6-9602-3f41264e6870 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:23.491673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c22d84c-ae43-4ab6-9602-3f41264e6870 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:23.492139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c22d84c-ae43-4ab6-9602-3f41264e6870 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:23.492882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c22d84c-ae43-4ab6-9602-3f41264e6870 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:23.493342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c22d84c-ae43-4ab6-9602-3f41264e6870 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:23.493800 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c22d84c-ae43-4ab6-9602-3f41264e6870 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:23.636332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c22d84c-ae43-4ab6-9602-3f41264e6870 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:23.644753 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3c22d84c-ae43-4ab6-9602-3f41264e6870 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1563 microversion: 2.1 time: 0.166811 Sep 27 11:02:23.645318 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3354/7179] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:23 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1563 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:23.667964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f2964fa8-8d80-4019-b951-82d29aeddfcb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:23.668800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f2964fa8-8d80-4019-b951-82d29aeddfcb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:23.676168 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2964fa8-8d80-4019-b951-82d29aeddfcb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:23.676674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2964fa8-8d80-4019-b951-82d29aeddfcb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:23.677120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2964fa8-8d80-4019-b951-82d29aeddfcb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:23.677819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2964fa8-8d80-4019-b951-82d29aeddfcb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:23.678274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2964fa8-8d80-4019-b951-82d29aeddfcb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:23.678700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2964fa8-8d80-4019-b951-82d29aeddfcb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:23.743077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:23.767282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:23.817227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f2964fa8-8d80-4019-b951-82d29aeddfcb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:23.829684 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f2964fa8-8d80-4019-b951-82d29aeddfcb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1563 microversion: 2.1 time: 0.165431 Sep 27 11:02:23.830454 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3826/7180] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:23 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1563 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:25.476366 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58f5ec0e-2f0a-4917-9278-4690f61e7aca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:25.479239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-58f5ec0e-2f0a-4917-9278-4690f61e7aca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:25.482679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:25.483313 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-84ba3dcb-7996-4468-beaa-d896a045c6d7 req-8d89f83a-8f91-45bd-8683-7771597b314b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d02fb6df-802d-438f-be1a-a9e733427ebc", "tag": "9e2fa624-8544-4495-bc6f-ce199ad290fa"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:02:25.487575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5ec0e-2f0a-4917-9278-4690f61e7aca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:25.488087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5ec0e-2f0a-4917-9278-4690f61e7aca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:25.488791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5ec0e-2f0a-4917-9278-4690f61e7aca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:25.489535 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5ec0e-2f0a-4917-9278-4690f61e7aca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:25.490179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5ec0e-2f0a-4917-9278-4690f61e7aca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:25.490632 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f5ec0e-2f0a-4917-9278-4690f61e7aca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:25.494254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-84ba3dcb-7996-4468-beaa-d896a045c6d7 req-8d89f83a-8f91-45bd-8683-7771597b314b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:25.494813 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-84ba3dcb-7996-4468-beaa-d896a045c6d7 req-8d89f83a-8f91-45bd-8683-7771597b314b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:25.495317 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-84ba3dcb-7996-4468-beaa-d896a045c6d7 req-8d89f83a-8f91-45bd-8683-7771597b314b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:25.508444 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-84ba3dcb-7996-4468-beaa-d896a045c6d7 req-8d89f83a-8f91-45bd-8683-7771597b314b service nova] Creating event network-changed:9e2fa624-8544-4495-bc6f-ce199ad290fa for instance d02fb6df-802d-438f-be1a-a9e733427ebc on nch1-master-1 Sep 27 11:02:25.516065 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-84ba3dcb-7996-4468-beaa-d896a045c6d7 req-8d89f83a-8f91-45bd-8683-7771597b314b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037615 Sep 27 11:02:25.517441 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3827/7181] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:02:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:02:25.748269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-58f5ec0e-2f0a-4917-9278-4690f61e7aca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:25.756891 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-58f5ec0e-2f0a-4917-9278-4690f61e7aca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1563 microversion: 2.1 time: 0.284508 Sep 27 11:02:25.757572 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3355/7182] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:25 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1563 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:47.353051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ed735d7f-08c7-4a8f-83f9-7a16bf3ea8bd tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:02:47.355427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ed735d7f-08c7-4a8f-83f9-7a16bf3ea8bd tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:47.357675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=7 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:47.358324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=36 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:47.364715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed735d7f-08c7-4a8f-83f9-7a16bf3ea8bd tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:47.365228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed735d7f-08c7-4a8f-83f9-7a16bf3ea8bd tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:47.365674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed735d7f-08c7-4a8f-83f9-7a16bf3ea8bd tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:47.366361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed735d7f-08c7-4a8f-83f9-7a16bf3ea8bd tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:47.366820 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed735d7f-08c7-4a8f-83f9-7a16bf3ea8bd tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:47.367251 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ed735d7f-08c7-4a8f-83f9-7a16bf3ea8bd tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:47.433539 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ed735d7f-08c7-4a8f-83f9-7a16bf3ea8bd tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc/action" status: 200 len: 24574 microversion: 2.1 time: 0.084505 Sep 27 11:02:47.434249 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3828/7183] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:02:47 2022] POST /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc/action => generated 24574 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:02:58.487237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5f33a400-b11b-478e-ad5f-b962412dacf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:02:58.489486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5f33a400-b11b-478e-ad5f-b962412dacf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:58.491847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=7 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:58.492573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=32 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:58.498372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f33a400-b11b-478e-ad5f-b962412dacf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:58.498895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f33a400-b11b-478e-ad5f-b962412dacf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:58.499328 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f33a400-b11b-478e-ad5f-b962412dacf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:58.500053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f33a400-b11b-478e-ad5f-b962412dacf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:58.500521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f33a400-b11b-478e-ad5f-b962412dacf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:58.500986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f33a400-b11b-478e-ad5f-b962412dacf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:58.586269 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5f33a400-b11b-478e-ad5f-b962412dacf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc/action" status: 202 len: 0 microversion: 2.1 time: 0.103019 Sep 27 11:02:58.586933 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3356/7184] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:02:58 2022] POST /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc/action => generated 0 bytes in 105 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:02:58.595903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ffc17856-9ffd-4aa7-b3d2-d35ba88e0b11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:58.596625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ffc17856-9ffd-4aa7-b3d2-d35ba88e0b11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:58.600740 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:58.601856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:02:58.608607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffc17856-9ffd-4aa7-b3d2-d35ba88e0b11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:58.608756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffc17856-9ffd-4aa7-b3d2-d35ba88e0b11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:58.609161 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffc17856-9ffd-4aa7-b3d2-d35ba88e0b11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:58.609877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffc17856-9ffd-4aa7-b3d2-d35ba88e0b11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:58.610337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffc17856-9ffd-4aa7-b3d2-d35ba88e0b11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:58.610804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ffc17856-9ffd-4aa7-b3d2-d35ba88e0b11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:58.760471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ffc17856-9ffd-4aa7-b3d2-d35ba88e0b11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:58.769395 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ffc17856-9ffd-4aa7-b3d2-d35ba88e0b11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1674 microversion: 2.1 time: 0.177772 Sep 27 11:02:58.770616 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3829/7185] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:58 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1674 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:02:59.799136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-426b50a8-33a5-4070-a26e-67329e350643 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:02:59.799934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-426b50a8-33a5-4070-a26e-67329e350643 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:02:59.807867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-426b50a8-33a5-4070-a26e-67329e350643 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:59.808435 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-426b50a8-33a5-4070-a26e-67329e350643 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:59.809005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-426b50a8-33a5-4070-a26e-67329e350643 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:59.809833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-426b50a8-33a5-4070-a26e-67329e350643 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:02:59.810386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-426b50a8-33a5-4070-a26e-67329e350643 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:02:59.810904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-426b50a8-33a5-4070-a26e-67329e350643 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:02:59.954047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-426b50a8-33a5-4070-a26e-67329e350643 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:02:59.965050 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-426b50a8-33a5-4070-a26e-67329e350643 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.172461 Sep 27 11:02:59.965642 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3357/7186] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:02:59 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:00.992795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e23f080a-1df1-4eaa-a86b-7fd46458a23c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:00.993369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e23f080a-1df1-4eaa-a86b-7fd46458a23c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:01.000517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e23f080a-1df1-4eaa-a86b-7fd46458a23c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:01.001043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e23f080a-1df1-4eaa-a86b-7fd46458a23c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:01.001483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e23f080a-1df1-4eaa-a86b-7fd46458a23c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:01.002183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e23f080a-1df1-4eaa-a86b-7fd46458a23c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:01.002638 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e23f080a-1df1-4eaa-a86b-7fd46458a23c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:01.003066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e23f080a-1df1-4eaa-a86b-7fd46458a23c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:01.146375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e23f080a-1df1-4eaa-a86b-7fd46458a23c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:01.155152 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e23f080a-1df1-4eaa-a86b-7fd46458a23c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.168044 Sep 27 11:03:01.155728 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3830/7187] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:00 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:02.182052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ae88d637-6388-4662-8251-7ba3ad9001a6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:02.182879 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ae88d637-6388-4662-8251-7ba3ad9001a6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:02.192988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae88d637-6388-4662-8251-7ba3ad9001a6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:02.193714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae88d637-6388-4662-8251-7ba3ad9001a6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:02.194440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae88d637-6388-4662-8251-7ba3ad9001a6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:02.195257 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae88d637-6388-4662-8251-7ba3ad9001a6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:02.195738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae88d637-6388-4662-8251-7ba3ad9001a6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:02.196184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ae88d637-6388-4662-8251-7ba3ad9001a6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:02.347232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ae88d637-6388-4662-8251-7ba3ad9001a6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:02.356153 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ae88d637-6388-4662-8251-7ba3ad9001a6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.180097 Sep 27 11:03:02.356963 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3358/7188] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:02 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:03.383067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-172b1828-b60f-40f7-9f63-861bab4e5501 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:03.383916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-172b1828-b60f-40f7-9f63-861bab4e5501 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:03.393343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172b1828-b60f-40f7-9f63-861bab4e5501 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:03.394002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172b1828-b60f-40f7-9f63-861bab4e5501 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:03.394595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172b1828-b60f-40f7-9f63-861bab4e5501 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:03.395540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172b1828-b60f-40f7-9f63-861bab4e5501 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:03.396163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172b1828-b60f-40f7-9f63-861bab4e5501 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:03.396785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-172b1828-b60f-40f7-9f63-861bab4e5501 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:03.543268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-172b1828-b60f-40f7-9f63-861bab4e5501 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:03.555952 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-172b1828-b60f-40f7-9f63-861bab4e5501 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.178447 Sep 27 11:03:03.556785 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3831/7189] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:03 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:04.584266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b8c0396e-655f-4ec2-be3d-68740dcd7e1f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:04.585428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b8c0396e-655f-4ec2-be3d-68740dcd7e1f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:04.597294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c0396e-655f-4ec2-be3d-68740dcd7e1f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:04.598142 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c0396e-655f-4ec2-be3d-68740dcd7e1f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:04.598900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c0396e-655f-4ec2-be3d-68740dcd7e1f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:04.600132 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c0396e-655f-4ec2-be3d-68740dcd7e1f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:04.600959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c0396e-655f-4ec2-be3d-68740dcd7e1f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:04.601454 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b8c0396e-655f-4ec2-be3d-68740dcd7e1f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:04.741221 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b8c0396e-655f-4ec2-be3d-68740dcd7e1f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:04.749883 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b8c0396e-655f-4ec2-be3d-68740dcd7e1f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.172211 Sep 27 11:03:04.750369 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3359/7190] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:04 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:05.774306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-033c4b4e-7314-4106-ab23-e603f4a5e2d6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:05.774909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-033c4b4e-7314-4106-ab23-e603f4a5e2d6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:05.783129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-033c4b4e-7314-4106-ab23-e603f4a5e2d6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:05.783610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-033c4b4e-7314-4106-ab23-e603f4a5e2d6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:05.784043 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-033c4b4e-7314-4106-ab23-e603f4a5e2d6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:05.784765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-033c4b4e-7314-4106-ab23-e603f4a5e2d6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:05.785222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-033c4b4e-7314-4106-ab23-e603f4a5e2d6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:05.785646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-033c4b4e-7314-4106-ab23-e603f4a5e2d6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:05.926530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-033c4b4e-7314-4106-ab23-e603f4a5e2d6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:05.934656 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-033c4b4e-7314-4106-ab23-e603f4a5e2d6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.164911 Sep 27 11:03:05.935221 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3832/7191] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:05 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:06.958924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2ea1c19b-f199-48cd-8363-82e3b5c1989f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:06.959538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2ea1c19b-f199-48cd-8363-82e3b5c1989f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:06.966968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea1c19b-f199-48cd-8363-82e3b5c1989f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:06.967479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea1c19b-f199-48cd-8363-82e3b5c1989f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:06.967937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea1c19b-f199-48cd-8363-82e3b5c1989f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:06.968692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea1c19b-f199-48cd-8363-82e3b5c1989f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:06.969179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea1c19b-f199-48cd-8363-82e3b5c1989f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:06.969660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2ea1c19b-f199-48cd-8363-82e3b5c1989f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:07.109017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2ea1c19b-f199-48cd-8363-82e3b5c1989f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:07.117567 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2ea1c19b-f199-48cd-8363-82e3b5c1989f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.162807 Sep 27 11:03:07.118204 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3360/7192] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:06 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:08.145515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa4dc212-e83c-4155-8a0b-b066afd75698 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:08.146617 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fa4dc212-e83c-4155-8a0b-b066afd75698 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:08.159386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4dc212-e83c-4155-8a0b-b066afd75698 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:08.160245 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4dc212-e83c-4155-8a0b-b066afd75698 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:08.161037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4dc212-e83c-4155-8a0b-b066afd75698 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:08.162321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4dc212-e83c-4155-8a0b-b066afd75698 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:08.163089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4dc212-e83c-4155-8a0b-b066afd75698 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:08.163858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa4dc212-e83c-4155-8a0b-b066afd75698 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:08.303022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fa4dc212-e83c-4155-8a0b-b066afd75698 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:08.311185 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fa4dc212-e83c-4155-8a0b-b066afd75698 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.172445 Sep 27 11:03:08.311951 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3833/7193] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:08 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:09.336307 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a053b13-e1bc-4854-8af4-ca2a0c0da0b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:09.336945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a053b13-e1bc-4854-8af4-ca2a0c0da0b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:09.344502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a053b13-e1bc-4854-8af4-ca2a0c0da0b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:09.345068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a053b13-e1bc-4854-8af4-ca2a0c0da0b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:09.345527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a053b13-e1bc-4854-8af4-ca2a0c0da0b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:09.346249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a053b13-e1bc-4854-8af4-ca2a0c0da0b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:09.346727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a053b13-e1bc-4854-8af4-ca2a0c0da0b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:09.347174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a053b13-e1bc-4854-8af4-ca2a0c0da0b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:09.492727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0a053b13-e1bc-4854-8af4-ca2a0c0da0b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:09.505029 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a053b13-e1bc-4854-8af4-ca2a0c0da0b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.173108 Sep 27 11:03:09.505820 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3361/7194] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:09 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:10.534092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9eef065c-23ef-4b5f-a482-628b9c8aaa97 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:10.535081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9eef065c-23ef-4b5f-a482-628b9c8aaa97 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:10.544691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9eef065c-23ef-4b5f-a482-628b9c8aaa97 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:10.545192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9eef065c-23ef-4b5f-a482-628b9c8aaa97 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:10.545628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9eef065c-23ef-4b5f-a482-628b9c8aaa97 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:10.546335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9eef065c-23ef-4b5f-a482-628b9c8aaa97 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:10.546783 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9eef065c-23ef-4b5f-a482-628b9c8aaa97 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:10.547209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9eef065c-23ef-4b5f-a482-628b9c8aaa97 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:10.691504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9eef065c-23ef-4b5f-a482-628b9c8aaa97 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:10.699634 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9eef065c-23ef-4b5f-a482-628b9c8aaa97 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.172589 Sep 27 11:03:10.700451 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3834/7195] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:10 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:11.725238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dd616f72-70e7-4a39-97b2-a02f876e65a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:11.725928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dd616f72-70e7-4a39-97b2-a02f876e65a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:11.733980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd616f72-70e7-4a39-97b2-a02f876e65a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:11.734494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd616f72-70e7-4a39-97b2-a02f876e65a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:11.734950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd616f72-70e7-4a39-97b2-a02f876e65a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:11.735709 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd616f72-70e7-4a39-97b2-a02f876e65a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:11.736184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd616f72-70e7-4a39-97b2-a02f876e65a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:11.736634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dd616f72-70e7-4a39-97b2-a02f876e65a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:11.879453 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dd616f72-70e7-4a39-97b2-a02f876e65a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:11.888173 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dd616f72-70e7-4a39-97b2-a02f876e65a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.167476 Sep 27 11:03:11.888790 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3362/7196] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:11 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:12.912516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7a28a47a-e42c-4166-89e3-6f29a76decee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:12.913177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7a28a47a-e42c-4166-89e3-6f29a76decee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:12.921031 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a28a47a-e42c-4166-89e3-6f29a76decee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:12.921526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a28a47a-e42c-4166-89e3-6f29a76decee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:12.921971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a28a47a-e42c-4166-89e3-6f29a76decee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:12.922683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a28a47a-e42c-4166-89e3-6f29a76decee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:12.923144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a28a47a-e42c-4166-89e3-6f29a76decee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:12.923574 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7a28a47a-e42c-4166-89e3-6f29a76decee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:13.066387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7a28a47a-e42c-4166-89e3-6f29a76decee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:13.078143 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7a28a47a-e42c-4166-89e3-6f29a76decee tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.169497 Sep 27 11:03:13.078914 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3835/7197] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:12 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:14.105695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d8d3d49f-98b5-4921-b98d-c6bdc30d69e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:14.106302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d8d3d49f-98b5-4921-b98d-c6bdc30d69e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:14.113415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8d3d49f-98b5-4921-b98d-c6bdc30d69e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:14.113929 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8d3d49f-98b5-4921-b98d-c6bdc30d69e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:14.114378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8d3d49f-98b5-4921-b98d-c6bdc30d69e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:14.115098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8d3d49f-98b5-4921-b98d-c6bdc30d69e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:14.115568 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8d3d49f-98b5-4921-b98d-c6bdc30d69e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:14.116015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d8d3d49f-98b5-4921-b98d-c6bdc30d69e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:14.250580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d8d3d49f-98b5-4921-b98d-c6bdc30d69e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:14.259734 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d8d3d49f-98b5-4921-b98d-c6bdc30d69e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1679 microversion: 2.1 time: 0.158716 Sep 27 11:03:14.260317 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3363/7198] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:14 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1679 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:15.285352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7dc25e61-37f4-4a4e-bc29-512a1f65a7c7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:15.285988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7dc25e61-37f4-4a4e-bc29-512a1f65a7c7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:15.293826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc25e61-37f4-4a4e-bc29-512a1f65a7c7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:15.294320 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc25e61-37f4-4a4e-bc29-512a1f65a7c7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:15.294758 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc25e61-37f4-4a4e-bc29-512a1f65a7c7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:15.295517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc25e61-37f4-4a4e-bc29-512a1f65a7c7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:15.295973 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc25e61-37f4-4a4e-bc29-512a1f65a7c7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:15.296427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc25e61-37f4-4a4e-bc29-512a1f65a7c7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:15.451726 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7dc25e61-37f4-4a4e-bc29-512a1f65a7c7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:15.463506 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7dc25e61-37f4-4a4e-bc29-512a1f65a7c7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1682 microversion: 2.1 time: 0.183569 Sep 27 11:03:15.464412 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3836/7199] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:15 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1682 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:39.715591 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-83c897d1-5685-4d11-86c8-ea1b083abe53 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:03:39.718080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-83c897d1-5685-4d11-86c8-ea1b083abe53 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:39.721400 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:39.722138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:39.723450 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:03:39.723592 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 11:03:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc/action (ip 10.222.0.26) !!! Sep 27 11:03:39.739222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83c897d1-5685-4d11-86c8-ea1b083abe53 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:39.739911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83c897d1-5685-4d11-86c8-ea1b083abe53 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:39.740438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83c897d1-5685-4d11-86c8-ea1b083abe53 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:39.741333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83c897d1-5685-4d11-86c8-ea1b083abe53 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:39.741817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83c897d1-5685-4d11-86c8-ea1b083abe53 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:39.742266 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83c897d1-5685-4d11-86c8-ea1b083abe53 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:39.800405 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-83c897d1-5685-4d11-86c8-ea1b083abe53 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc/action" status: 200 len: 69157 microversion: 2.1 time: 0.091152 Sep 27 11:03:39.801127 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3364/7200] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:03:39 2022] POST /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc/action => generated 69157 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:03:47.066103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-341f4bbc-0f69-4c1f-bb3f-220ab9a1797b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:47.067266 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-341f4bbc-0f69-4c1f-bb3f-220ab9a1797b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:47.069496 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=16 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:47.071463 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=48 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:47.077720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-341f4bbc-0f69-4c1f-bb3f-220ab9a1797b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:47.078757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-341f4bbc-0f69-4c1f-bb3f-220ab9a1797b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:47.078757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-341f4bbc-0f69-4c1f-bb3f-220ab9a1797b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:47.079411 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-341f4bbc-0f69-4c1f-bb3f-220ab9a1797b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:47.079870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-341f4bbc-0f69-4c1f-bb3f-220ab9a1797b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:47.080704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-341f4bbc-0f69-4c1f-bb3f-220ab9a1797b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:47.107006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-341f4bbc-0f69-4c1f-bb3f-220ab9a1797b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 11:03:47.138469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b5429806-4d8e-44b3-a893-e479587ed9c5 req-803ff64b-4c05-4ad6-8c37-790d078b7e28 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d02fb6df-802d-438f-be1a-a9e733427ebc", "tag": "9e2fa624-8544-4495-bc6f-ce199ad290fa"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:03:47.151390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b5429806-4d8e-44b3-a893-e479587ed9c5 req-803ff64b-4c05-4ad6-8c37-790d078b7e28 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:47.152060 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b5429806-4d8e-44b3-a893-e479587ed9c5 req-803ff64b-4c05-4ad6-8c37-790d078b7e28 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:47.152616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b5429806-4d8e-44b3-a893-e479587ed9c5 req-803ff64b-4c05-4ad6-8c37-790d078b7e28 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:47.165575 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b5429806-4d8e-44b3-a893-e479587ed9c5 req-803ff64b-4c05-4ad6-8c37-790d078b7e28 service nova] Creating event network-changed:9e2fa624-8544-4495-bc6f-ce199ad290fa for instance d02fb6df-802d-438f-be1a-a9e733427ebc on nch1-master-1 Sep 27 11:03:47.171802 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b5429806-4d8e-44b3-a893-e479587ed9c5 req-803ff64b-4c05-4ad6-8c37-790d078b7e28 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.107065 Sep 27 11:03:47.172965 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3365/7201] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:03:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:03:47.193931 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-341f4bbc-0f69-4c1f-bb3f-220ab9a1797b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 204 len: 0 microversion: 2.1 time: 0.132629 Sep 27 11:03:47.194960 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3837/7202] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 11:03:47 2022] DELETE /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 11:03:47.209434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-136b3cb9-0c45-4ea3-888e-7c9dab399bb1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:47.210856 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-136b3cb9-0c45-4ea3-888e-7c9dab399bb1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:47.218224 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-136b3cb9-0c45-4ea3-888e-7c9dab399bb1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:47.218741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-136b3cb9-0c45-4ea3-888e-7c9dab399bb1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:47.219193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-136b3cb9-0c45-4ea3-888e-7c9dab399bb1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:47.220164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-136b3cb9-0c45-4ea3-888e-7c9dab399bb1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:47.221110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-136b3cb9-0c45-4ea3-888e-7c9dab399bb1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:47.221747 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-136b3cb9-0c45-4ea3-888e-7c9dab399bb1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:47.399136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-136b3cb9-0c45-4ea3-888e-7c9dab399bb1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:47.408047 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-136b3cb9-0c45-4ea3-888e-7c9dab399bb1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1688 microversion: 2.1 time: 0.208118 Sep 27 11:03:47.408690 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3366/7203] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:47 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1688 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:48.437480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-786f1195-b27f-4ae4-8693-a454241d6cba tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:48.438651 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-786f1195-b27f-4ae4-8693-a454241d6cba tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:48.450560 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786f1195-b27f-4ae4-8693-a454241d6cba tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:48.451371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786f1195-b27f-4ae4-8693-a454241d6cba tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:48.452100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786f1195-b27f-4ae4-8693-a454241d6cba tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:48.453443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786f1195-b27f-4ae4-8693-a454241d6cba tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:48.453902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786f1195-b27f-4ae4-8693-a454241d6cba tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:48.454330 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-786f1195-b27f-4ae4-8693-a454241d6cba tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:48.595691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-786f1195-b27f-4ae4-8693-a454241d6cba tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:48.605215 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-786f1195-b27f-4ae4-8693-a454241d6cba tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1688 microversion: 2.1 time: 0.174961 Sep 27 11:03:48.613221 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3838/7204] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:48 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1688 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:49.450412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-cf9e0e35-9d65-4e26-98ed-12f9ca533c83 req-e33637bc-5204-4227-ae92-ab8c97e59e3b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d02fb6df-802d-438f-be1a-a9e733427ebc", "name": "network-vif-deleted", "tag": "9e2fa624-8544-4495-bc6f-ce199ad290fa"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:03:49.464480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cf9e0e35-9d65-4e26-98ed-12f9ca533c83 req-e33637bc-5204-4227-ae92-ab8c97e59e3b service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:49.465008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cf9e0e35-9d65-4e26-98ed-12f9ca533c83 req-e33637bc-5204-4227-ae92-ab8c97e59e3b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:49.465451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-cf9e0e35-9d65-4e26-98ed-12f9ca533c83 req-e33637bc-5204-4227-ae92-ab8c97e59e3b service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:49.478055 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-cf9e0e35-9d65-4e26-98ed-12f9ca533c83 req-e33637bc-5204-4227-ae92-ab8c97e59e3b service nova] Creating event network-vif-deleted:9e2fa624-8544-4495-bc6f-ce199ad290fa for instance d02fb6df-802d-438f-be1a-a9e733427ebc on nch1-master-1 Sep 27 11:03:49.482982 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-cf9e0e35-9d65-4e26-98ed-12f9ca533c83 req-e33637bc-5204-4227-ae92-ab8c97e59e3b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036354 Sep 27 11:03:49.483610 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3367/7205] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:03:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:03:49.629792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:49.630408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:49.637619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:49.638109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:49.638547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:49.639248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:49.639706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:49.640179 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:49.673047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:49.673441 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:49.673881 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:49.674327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lazy-loading 'fault' on Instance uuid d02fb6df-802d-438f-be1a-a9e733427ebc {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 11:03:49.717521 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:03:49.726179 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-61c3d84e-4dca-418b-96b5-1c13528fe49d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 200 len: 1357 microversion: 2.1 time: 0.100394 Sep 27 11:03:49.726811 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3839/7206] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:49 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 1357 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:03:50.757345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:50.758541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: d02fb6df-802d-438f-be1a-a9e733427ebc] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:03:50.767880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:50.768396 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:50.768900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:50.769633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:03:50.770138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:03:50.770566 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:03:50.785042 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] HTTP exception thrown: Instance d02fb6df-802d-438f-be1a-a9e733427ebc could not be found. Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Request method failure captured: Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc HTTP/1.1 Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtfHoJ5jcAxOLtRcoV1nD5XXlTZzK0A-5egTQGd5lTVNdkOuvT5Fs0L9J-caES9KtErWFoSb_09yL5egcLFPiM4N0_ALLienspVX9-DRrIBhK2koE7PdcRR57IH0CHGGILEi4fmfkKfiXRleQCLI0R176myMkB92QmVT86Qtqhsg2-o Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 55b42a416dd942e780e533e4c6f2a8b4 Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_55b42a416dd942e780e533e4c6f2a8b4"}]}] Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 55b42a416dd942e780e533e4c6f2a8b4 Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestNetworkAdvancedServerOps-1003182190-project Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 3ec60ad271544bc99d7f394b9b06dd88 Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1003182190-project Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance d02fb6df-802d-438f-be1a-a9e733427ebc could not be found. Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': 'd02fb6df-802d-438f-be1a-a9e733427ebc'} Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d02fb6df-802d-438f-be1a-a9e733427ebc could not be found. Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d02fb6df-802d-438f-be1a-a9e733427ebc could not be found. Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:03:50.786729 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:03:50.791857 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:03:50.791857 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:03:50.791857 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:03:50.791857 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:03:50.791857 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 11:03:50.791857 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 11:03:50.791857 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d02fb6df-802d-438f-be1a-a9e733427ebc could not be found. Sep 27 11:03:50.791857 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:03:50.791857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Returning 404 to user: Instance d02fb6df-802d-438f-be1a-a9e733427ebc could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 11:03:50.791857 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d182c7ba-5442-4e77-8210-7a8f3bac9bf5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc" status: 404 len: 111 microversion: 2.1 time: 0.039923 Sep 27 11:03:50.791857 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3368/7207] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:03:50 2022] GET /compute/v2.1/servers/d02fb6df-802d-438f-be1a-a9e733427ebc => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 11:03:54.071802 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-015b270b-d629-4c66-b04c-d83b36f604f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '.version_select at 0x7f7f6c1ebca0>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:03:54.094918 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-015b270b-d629-4c66-b04c-d83b36f604f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1806921382" status: 202 len: 0 microversion: 2.1 time: 0.027159 Sep 27 11:03:54.095611 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3840/7208] 10.222.0.26 () {58 vars in 1284 bytes} [Tue Sep 27 11:03:54 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1806921382 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 11:03:54.127700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9d7fcbd7-bc87-46c1-af7d-1b289c009935 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'create', calling method: .version_select at 0x7f7f62559ca0>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-982299882", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOU/v4zdrJ7SQArT48MZPaZrfmCe9ul2dW/uoWre8Je9a1xlD0ZKXNklYaUjpKpQK0HsORXOp7luKmCnboEkO/YYK9JkL7MQWDG+v113EZ5B152ivAMn87v1xyMO0Wc1NQ=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:03:54.145359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-9d7fcbd7-bc87-46c1-af7d-1b289c009935 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:03:54.150609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-9d7fcbd7-bc87-46c1-af7d-1b289c009935 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for user 3ec60ad271544bc99d7f394b9b06dd88 and project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:03:54.176825 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9d7fcbd7-bc87-46c1-af7d-1b289c009935 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 0.053906 Sep 27 11:03:54.177451 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3369/7209] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 11:03:54 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:03:59.844381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:03:59.849195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1620972955", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "50f8e3a4-db3d-44a9-8a43-7e9ca58bfd64"}], "key_name": "tempest-TestNetworkAdvancedServerOps-982299882", "security_groups": [{"name": "tempest-secgroup-smoke-508147660"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:03:59.971092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] validate_networks() for [('50f8e3a4-db3d-44a9-8a43-7e9ca58bfd64', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 11:04:00.370247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 11:04:00.370716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 11:04:00.371197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 11:04:00.371620 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 11:04:00.372228 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 11:04:00.384581 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:00.385488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:00.386596 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:00.387358 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:00.387823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:00.388371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:00.409041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:04:00.415985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for user 3ec60ad271544bc99d7f394b9b06dd88 and project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:04:00.434132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 11:04:00.436815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 11:04:00.492495 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6c690cea-f08b-4fec-996d-e2d48f74686c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.649909 Sep 27 11:04:00.493387 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3841/7210] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 11:03:59 2022] POST /compute/v2.1/servers => generated 405 bytes in 652 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 11:04:00.506153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f3f7a15e-46f7-4f3e-ae19-37e658ca2265 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:00.506802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f3f7a15e-46f7-4f3e-ae19-37e658ca2265 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:00.581660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f3f7a15e-46f7-4f3e-ae19-37e658ca2265 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:00.589361 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f3f7a15e-46f7-4f3e-ae19-37e658ca2265 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1266 microversion: 2.1 time: 0.088160 Sep 27 11:04:00.589839 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3370/7211] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:00 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1266 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:01.617439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0fd314d0-6bf2-4c53-bb1e-e92129a5d398 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:01.619148 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0fd314d0-6bf2-4c53-bb1e-e92129a5d398 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:01.628176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fd314d0-6bf2-4c53-bb1e-e92129a5d398 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:01.629063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fd314d0-6bf2-4c53-bb1e-e92129a5d398 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:01.629600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fd314d0-6bf2-4c53-bb1e-e92129a5d398 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:01.630357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fd314d0-6bf2-4c53-bb1e-e92129a5d398 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:01.630790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fd314d0-6bf2-4c53-bb1e-e92129a5d398 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:01.631220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fd314d0-6bf2-4c53-bb1e-e92129a5d398 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:01.707800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0fd314d0-6bf2-4c53-bb1e-e92129a5d398 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:01.720549 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0fd314d0-6bf2-4c53-bb1e-e92129a5d398 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1318 microversion: 2.1 time: 0.106942 Sep 27 11:04:01.732691 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3842/7212] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:01 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1318 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:02.749469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b63b90b5-fcc7-449f-abd5-7801748387ca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:02.750276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b63b90b5-fcc7-449f-abd5-7801748387ca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:02.770432 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:02.775220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b63b90b5-fcc7-449f-abd5-7801748387ca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:02.775926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b63b90b5-fcc7-449f-abd5-7801748387ca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:02.776551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b63b90b5-fcc7-449f-abd5-7801748387ca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:02.777871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b63b90b5-fcc7-449f-abd5-7801748387ca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:02.778449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b63b90b5-fcc7-449f-abd5-7801748387ca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:02.778988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b63b90b5-fcc7-449f-abd5-7801748387ca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:03.015109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b63b90b5-fcc7-449f-abd5-7801748387ca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:03.023657 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b63b90b5-fcc7-449f-abd5-7801748387ca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1391 microversion: 2.1 time: 0.280678 Sep 27 11:04:03.024439 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3371/7213] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:02 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1391 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:03.531055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-1127ebfe-c208-459f-b09e-1e7f7ed15d0f req-ce6ddb64-59ef-41b8-8a9b-c26b6c557d0f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03d38687-ac91-4e9a-9c02-e57430bdb8da", "tag": "ac164f43-1f65-45c8-a7d9-6cfb713a0941"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:04:03.542254 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1127ebfe-c208-459f-b09e-1e7f7ed15d0f req-ce6ddb64-59ef-41b8-8a9b-c26b6c557d0f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:03.542750 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1127ebfe-c208-459f-b09e-1e7f7ed15d0f req-ce6ddb64-59ef-41b8-8a9b-c26b6c557d0f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:03.543183 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1127ebfe-c208-459f-b09e-1e7f7ed15d0f req-ce6ddb64-59ef-41b8-8a9b-c26b6c557d0f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:03.559372 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-1127ebfe-c208-459f-b09e-1e7f7ed15d0f req-ce6ddb64-59ef-41b8-8a9b-c26b6c557d0f service nova] Creating event network-changed:ac164f43-1f65-45c8-a7d9-6cfb713a0941 for instance 03d38687-ac91-4e9a-9c02-e57430bdb8da on nch2-master-1 Sep 27 11:04:03.565586 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-1127ebfe-c208-459f-b09e-1e7f7ed15d0f req-ce6ddb64-59ef-41b8-8a9b-c26b6c557d0f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.123792 Sep 27 11:04:03.566888 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3843/7214] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:04:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:04:04.048011 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c1fb27b0-48ea-4906-ab5a-0b2e99087240 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:04.048845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c1fb27b0-48ea-4906-ab5a-0b2e99087240 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:04.056440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1fb27b0-48ea-4906-ab5a-0b2e99087240 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:04.056992 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1fb27b0-48ea-4906-ab5a-0b2e99087240 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:04.057486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1fb27b0-48ea-4906-ab5a-0b2e99087240 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:04.058229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1fb27b0-48ea-4906-ab5a-0b2e99087240 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:04.058705 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1fb27b0-48ea-4906-ab5a-0b2e99087240 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:04.059154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c1fb27b0-48ea-4906-ab5a-0b2e99087240 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:04.197199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c1fb27b0-48ea-4906-ab5a-0b2e99087240 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:04.208864 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c1fb27b0-48ea-4906-ab5a-0b2e99087240 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1391 microversion: 2.1 time: 0.164658 Sep 27 11:04:04.209684 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3372/7215] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:04 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1391 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:05.244166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f2f4b02f-61b9-4afb-b30a-46c44b7f4e48 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:05.245379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f2f4b02f-61b9-4afb-b30a-46c44b7f4e48 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:05.254943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2f4b02f-61b9-4afb-b30a-46c44b7f4e48 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:05.255427 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2f4b02f-61b9-4afb-b30a-46c44b7f4e48 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:05.255862 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2f4b02f-61b9-4afb-b30a-46c44b7f4e48 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:05.256562 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2f4b02f-61b9-4afb-b30a-46c44b7f4e48 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:05.257049 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2f4b02f-61b9-4afb-b30a-46c44b7f4e48 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:05.257477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f2f4b02f-61b9-4afb-b30a-46c44b7f4e48 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:05.397059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f2f4b02f-61b9-4afb-b30a-46c44b7f4e48 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:05.405533 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f2f4b02f-61b9-4afb-b30a-46c44b7f4e48 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1391 microversion: 2.1 time: 0.168964 Sep 27 11:04:05.406824 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3844/7216] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:05 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1391 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:06.429660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c356165a-6820-4b23-bfbb-1fec063e45e8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:06.430264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c356165a-6820-4b23-bfbb-1fec063e45e8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:06.437707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c356165a-6820-4b23-bfbb-1fec063e45e8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:06.438199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c356165a-6820-4b23-bfbb-1fec063e45e8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:06.438683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c356165a-6820-4b23-bfbb-1fec063e45e8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:06.439425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c356165a-6820-4b23-bfbb-1fec063e45e8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:06.439906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c356165a-6820-4b23-bfbb-1fec063e45e8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:06.440336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c356165a-6820-4b23-bfbb-1fec063e45e8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:06.700274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c356165a-6820-4b23-bfbb-1fec063e45e8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:06.709328 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c356165a-6820-4b23-bfbb-1fec063e45e8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1391 microversion: 2.1 time: 0.283853 Sep 27 11:04:06.709993 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3373/7217] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:06 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1391 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:07.735290 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-227a97ee-a901-4f12-80ef-c39c2bdc729e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:07.735943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-227a97ee-a901-4f12-80ef-c39c2bdc729e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:07.742893 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-227a97ee-a901-4f12-80ef-c39c2bdc729e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:07.743383 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-227a97ee-a901-4f12-80ef-c39c2bdc729e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:07.743826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-227a97ee-a901-4f12-80ef-c39c2bdc729e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:07.744559 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-227a97ee-a901-4f12-80ef-c39c2bdc729e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:07.745052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-227a97ee-a901-4f12-80ef-c39c2bdc729e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:07.745477 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-227a97ee-a901-4f12-80ef-c39c2bdc729e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:07.885471 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-227a97ee-a901-4f12-80ef-c39c2bdc729e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:07.894793 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-227a97ee-a901-4f12-80ef-c39c2bdc729e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1391 microversion: 2.1 time: 0.164899 Sep 27 11:04:07.895711 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3845/7218] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:07 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1391 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:08.920614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c9933c70-e3c2-4261-a549-39aed9f8628d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:08.921335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c9933c70-e3c2-4261-a549-39aed9f8628d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:08.933819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9933c70-e3c2-4261-a549-39aed9f8628d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:08.934531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9933c70-e3c2-4261-a549-39aed9f8628d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:08.935171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9933c70-e3c2-4261-a549-39aed9f8628d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:08.936206 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9933c70-e3c2-4261-a549-39aed9f8628d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:08.936968 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9933c70-e3c2-4261-a549-39aed9f8628d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:08.937574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c9933c70-e3c2-4261-a549-39aed9f8628d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:09.078341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c9933c70-e3c2-4261-a549-39aed9f8628d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:09.086431 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c9933c70-e3c2-4261-a549-39aed9f8628d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1391 microversion: 2.1 time: 0.171620 Sep 27 11:04:09.087018 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3374/7219] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:08 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1391 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:10.112963 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-625d2b59-abb3-4c9c-95a9-b67d79391679 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:10.113848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-625d2b59-abb3-4c9c-95a9-b67d79391679 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:10.121862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=34,nova_api:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:10.123460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-625d2b59-abb3-4c9c-95a9-b67d79391679 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:10.124137 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-625d2b59-abb3-4c9c-95a9-b67d79391679 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:10.124834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-625d2b59-abb3-4c9c-95a9-b67d79391679 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:10.125738 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-625d2b59-abb3-4c9c-95a9-b67d79391679 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:10.126239 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-625d2b59-abb3-4c9c-95a9-b67d79391679 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:10.126733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-625d2b59-abb3-4c9c-95a9-b67d79391679 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:10.155200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-eb65efd3-4e9d-419d-b49c-ead14907d10b req-55c318db-70a8-4da7-9e89-31a07d869690 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "03d38687-ac91-4e9a-9c02-e57430bdb8da", "name": "network-vif-plugged", "status": "completed", "tag": "ac164f43-1f65-45c8-a7d9-6cfb713a0941"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:04:10.169197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-eb65efd3-4e9d-419d-b49c-ead14907d10b req-55c318db-70a8-4da7-9e89-31a07d869690 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:10.169765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-eb65efd3-4e9d-419d-b49c-ead14907d10b req-55c318db-70a8-4da7-9e89-31a07d869690 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:10.170215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-eb65efd3-4e9d-419d-b49c-ead14907d10b req-55c318db-70a8-4da7-9e89-31a07d869690 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:10.184243 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-eb65efd3-4e9d-419d-b49c-ead14907d10b req-55c318db-70a8-4da7-9e89-31a07d869690 service nova] Creating event network-vif-plugged:ac164f43-1f65-45c8-a7d9-6cfb713a0941 for instance 03d38687-ac91-4e9a-9c02-e57430bdb8da on nch2-master-1 Sep 27 11:04:10.190437 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-eb65efd3-4e9d-419d-b49c-ead14907d10b req-55c318db-70a8-4da7-9e89-31a07d869690 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.101485 Sep 27 11:04:10.191557 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3846/7220] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:04:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:04:10.274928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-625d2b59-abb3-4c9c-95a9-b67d79391679 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:10.283381 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-625d2b59-abb3-4c9c-95a9-b67d79391679 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1391 microversion: 2.1 time: 0.177773 Sep 27 11:04:10.284071 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3375/7221] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:10 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1391 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:11.306816 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:11.308737 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:11.315808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-27f514ce-70d7-4ab9-916d-ee3cbb5a29ef tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:11.316584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-27f514ce-70d7-4ab9-916d-ee3cbb5a29ef tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:11.323790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27f514ce-70d7-4ab9-916d-ee3cbb5a29ef tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:11.324274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27f514ce-70d7-4ab9-916d-ee3cbb5a29ef tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:11.324730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27f514ce-70d7-4ab9-916d-ee3cbb5a29ef tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:11.325434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27f514ce-70d7-4ab9-916d-ee3cbb5a29ef tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:11.325889 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27f514ce-70d7-4ab9-916d-ee3cbb5a29ef tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:11.326315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-27f514ce-70d7-4ab9-916d-ee3cbb5a29ef tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:11.464586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-27f514ce-70d7-4ab9-916d-ee3cbb5a29ef tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:11.473263 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-27f514ce-70d7-4ab9-916d-ee3cbb5a29ef tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1391 microversion: 2.1 time: 0.167814 Sep 27 11:04:11.473802 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3847/7222] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:11 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1391 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:12.501290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-83f9d535-0b73-40ba-9488-f85d0c870a0a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:12.502201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-83f9d535-0b73-40ba-9488-f85d0c870a0a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:12.504859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:12.511484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f9d535-0b73-40ba-9488-f85d0c870a0a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:12.511999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f9d535-0b73-40ba-9488-f85d0c870a0a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:12.512445 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f9d535-0b73-40ba-9488-f85d0c870a0a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:12.513192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f9d535-0b73-40ba-9488-f85d0c870a0a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:12.513663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f9d535-0b73-40ba-9488-f85d0c870a0a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:12.514101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-83f9d535-0b73-40ba-9488-f85d0c870a0a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:12.656650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-83f9d535-0b73-40ba-9488-f85d0c870a0a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:12.664973 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-83f9d535-0b73-40ba-9488-f85d0c870a0a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1563 microversion: 2.1 time: 0.170768 Sep 27 11:04:12.665552 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3376/7223] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:12 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1563 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:12.691042 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-37b08bff-61f7-423c-be64-4cfd98f97a7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:12.691777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-37b08bff-61f7-423c-be64-4cfd98f97a7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:12.698928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37b08bff-61f7-423c-be64-4cfd98f97a7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:12.699428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37b08bff-61f7-423c-be64-4cfd98f97a7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:12.699848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37b08bff-61f7-423c-be64-4cfd98f97a7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:12.700623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37b08bff-61f7-423c-be64-4cfd98f97a7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:12.701121 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37b08bff-61f7-423c-be64-4cfd98f97a7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:12.701572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-37b08bff-61f7-423c-be64-4cfd98f97a7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:12.841891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-37b08bff-61f7-423c-be64-4cfd98f97a7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:12.852547 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-37b08bff-61f7-423c-be64-4cfd98f97a7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1563 microversion: 2.1 time: 0.168028 Sep 27 11:04:12.853182 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3848/7224] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:12 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1563 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:14.539818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5666fe03-03a4-4bc5-b534-d34aaf8f9b6c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:04:14.541106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-af42279a-2ba7-4a4e-81e9-d58b08347b59 req-4f4e623f-5581-4257-9ec5-3f45033ace5e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03d38687-ac91-4e9a-9c02-e57430bdb8da", "tag": "ac164f43-1f65-45c8-a7d9-6cfb713a0941"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:04:14.543767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5666fe03-03a4-4bc5-b534-d34aaf8f9b6c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:14.550177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5666fe03-03a4-4bc5-b534-d34aaf8f9b6c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:14.550689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5666fe03-03a4-4bc5-b534-d34aaf8f9b6c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:14.551553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5666fe03-03a4-4bc5-b534-d34aaf8f9b6c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:14.552472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-af42279a-2ba7-4a4e-81e9-d58b08347b59 req-4f4e623f-5581-4257-9ec5-3f45033ace5e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:14.552816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5666fe03-03a4-4bc5-b534-d34aaf8f9b6c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:14.552995 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-af42279a-2ba7-4a4e-81e9-d58b08347b59 req-4f4e623f-5581-4257-9ec5-3f45033ace5e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:14.553422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-af42279a-2ba7-4a4e-81e9-d58b08347b59 req-4f4e623f-5581-4257-9ec5-3f45033ace5e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:14.553585 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5666fe03-03a4-4bc5-b534-d34aaf8f9b6c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:14.554102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5666fe03-03a4-4bc5-b534-d34aaf8f9b6c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:14.565235 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-af42279a-2ba7-4a4e-81e9-d58b08347b59 req-4f4e623f-5581-4257-9ec5-3f45033ace5e service nova] Creating event network-changed:ac164f43-1f65-45c8-a7d9-6cfb713a0941 for instance 03d38687-ac91-4e9a-9c02-e57430bdb8da on nch2-master-1 Sep 27 11:04:14.570684 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-af42279a-2ba7-4a4e-81e9-d58b08347b59 req-4f4e623f-5581-4257-9ec5-3f45033ace5e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033332 Sep 27 11:04:14.571306 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3849/7225] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:04:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:04:14.786874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5666fe03-03a4-4bc5-b534-d34aaf8f9b6c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:04:14.795799 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5666fe03-03a4-4bc5-b534-d34aaf8f9b6c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1563 microversion: 2.1 time: 0.260456 Sep 27 11:04:14.796547 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3377/7226] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:04:14 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1563 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:04:48.146506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:48.147300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=38 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:04:48.151684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-52bcc5af-f11e-49a3-ad56-717bbe7677b7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:04:48.154018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-52bcc5af-f11e-49a3-ad56-717bbe7677b7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:04:48.162224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52bcc5af-f11e-49a3-ad56-717bbe7677b7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:48.162720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52bcc5af-f11e-49a3-ad56-717bbe7677b7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:48.163158 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52bcc5af-f11e-49a3-ad56-717bbe7677b7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:48.163874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52bcc5af-f11e-49a3-ad56-717bbe7677b7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:04:48.164340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52bcc5af-f11e-49a3-ad56-717bbe7677b7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:04:48.164809 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-52bcc5af-f11e-49a3-ad56-717bbe7677b7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:04:48.237226 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-52bcc5af-f11e-49a3-ad56-717bbe7677b7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da/action" status: 200 len: 24573 microversion: 2.1 time: 0.092558 Sep 27 11:04:48.237894 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3850/7227] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:04:48 2022] POST /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da/action => generated 24573 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:05:26.970386 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=5 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:05:26.971079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=42 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:05:27.024374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"resize": {"flavorRef": "451"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:05:27.026779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:27.033996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:27.034503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:27.034959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:27.035757 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:27.036211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:27.036670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:27.134254 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for compute:servers:resize:cross_cell failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:27.142924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Old instance type m1.heat2, new instance type m1.heat {{(pid=145969) resize /opt/stack/nova/nova/compute/api.py:4220}} Sep 27 11:05:27.149906 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 11:05:27.150353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 11:05:27.150855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 11:05:27.151294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 11:05:27.151735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 11:05:27.221883 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3ad9d848-ff48-42c9-bb69-ed7ae01ff677 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da/action" status: 202 len: 0 microversion: 2.1 time: 0.253630 Sep 27 11:05:27.222466 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3378/7228] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:05:26 2022] POST /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da/action => generated 0 bytes in 255 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:05:27.231667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-755a00be-ca16-4be7-b19f-4e8646a043d8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:27.232366 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-755a00be-ca16-4be7-b19f-4e8646a043d8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:27.234976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:05:27.236483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:05:27.243934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-755a00be-ca16-4be7-b19f-4e8646a043d8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:27.244469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-755a00be-ca16-4be7-b19f-4e8646a043d8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:27.244888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-755a00be-ca16-4be7-b19f-4e8646a043d8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:27.245603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-755a00be-ca16-4be7-b19f-4e8646a043d8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:27.246122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-755a00be-ca16-4be7-b19f-4e8646a043d8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:27.246523 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-755a00be-ca16-4be7-b19f-4e8646a043d8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:27.393099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-755a00be-ca16-4be7-b19f-4e8646a043d8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:27.401342 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-755a00be-ca16-4be7-b19f-4e8646a043d8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1690 microversion: 2.1 time: 0.174161 Sep 27 11:05:27.402062 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3851/7229] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:27 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1690 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:28.430294 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4b37d746-383f-46f4-85e5-b2877129dcb4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:28.431015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4b37d746-383f-46f4-85e5-b2877129dcb4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:28.439572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b37d746-383f-46f4-85e5-b2877129dcb4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:28.440084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b37d746-383f-46f4-85e5-b2877129dcb4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:28.440542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b37d746-383f-46f4-85e5-b2877129dcb4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:28.441748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b37d746-383f-46f4-85e5-b2877129dcb4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:28.441954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b37d746-383f-46f4-85e5-b2877129dcb4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:28.442444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4b37d746-383f-46f4-85e5-b2877129dcb4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:28.590692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4b37d746-383f-46f4-85e5-b2877129dcb4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:28.603276 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4b37d746-383f-46f4-85e5-b2877129dcb4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1690 microversion: 2.1 time: 0.179847 Sep 27 11:05:28.604688 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3379/7230] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:28 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1690 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 11:05:29.632234 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5d5d024e-8afe-4563-be1d-5b10ee030d26 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:29.632929 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5d5d024e-8afe-4563-be1d-5b10ee030d26 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:29.640144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d5d024e-8afe-4563-be1d-5b10ee030d26 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:29.640910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d5d024e-8afe-4563-be1d-5b10ee030d26 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:29.641343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d5d024e-8afe-4563-be1d-5b10ee030d26 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:29.642059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d5d024e-8afe-4563-be1d-5b10ee030d26 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:29.642518 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d5d024e-8afe-4563-be1d-5b10ee030d26 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:29.642945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5d5d024e-8afe-4563-be1d-5b10ee030d26 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:29.785890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5d5d024e-8afe-4563-be1d-5b10ee030d26 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:29.794343 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5d5d024e-8afe-4563-be1d-5b10ee030d26 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1695 microversion: 2.1 time: 0.167178 Sep 27 11:05:29.794923 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3852/7231] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:29 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1695 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:30.819592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-914ac5e2-2dd0-45dd-bc9b-0d0fc50b2e4b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:30.820290 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-914ac5e2-2dd0-45dd-bc9b-0d0fc50b2e4b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:30.827717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914ac5e2-2dd0-45dd-bc9b-0d0fc50b2e4b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:30.828213 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914ac5e2-2dd0-45dd-bc9b-0d0fc50b2e4b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:30.828682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914ac5e2-2dd0-45dd-bc9b-0d0fc50b2e4b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:30.829405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914ac5e2-2dd0-45dd-bc9b-0d0fc50b2e4b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:30.829871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914ac5e2-2dd0-45dd-bc9b-0d0fc50b2e4b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:30.830311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-914ac5e2-2dd0-45dd-bc9b-0d0fc50b2e4b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:30.983788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-914ac5e2-2dd0-45dd-bc9b-0d0fc50b2e4b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:30.996745 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-914ac5e2-2dd0-45dd-bc9b-0d0fc50b2e4b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1695 microversion: 2.1 time: 0.181272 Sep 27 11:05:30.997603 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3380/7232] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:30 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1695 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:32.024694 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a85efb4e-76b9-49f0-9e96-f838c8370891 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:32.025609 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a85efb4e-76b9-49f0-9e96-f838c8370891 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:32.034498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a85efb4e-76b9-49f0-9e96-f838c8370891 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:32.035236 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a85efb4e-76b9-49f0-9e96-f838c8370891 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:32.035778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a85efb4e-76b9-49f0-9e96-f838c8370891 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:32.036489 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a85efb4e-76b9-49f0-9e96-f838c8370891 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:32.036969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a85efb4e-76b9-49f0-9e96-f838c8370891 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:32.037395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a85efb4e-76b9-49f0-9e96-f838c8370891 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:32.184774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a85efb4e-76b9-49f0-9e96-f838c8370891 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:32.197502 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a85efb4e-76b9-49f0-9e96-f838c8370891 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1695 microversion: 2.1 time: 0.178315 Sep 27 11:05:32.198252 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3853/7233] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:32 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1695 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:33.222867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b69e95b0-5288-4ff4-80a8-90b844b6e352 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:33.223809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b69e95b0-5288-4ff4-80a8-90b844b6e352 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:33.231540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b69e95b0-5288-4ff4-80a8-90b844b6e352 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:33.232087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b69e95b0-5288-4ff4-80a8-90b844b6e352 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:33.232565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b69e95b0-5288-4ff4-80a8-90b844b6e352 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:33.233347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b69e95b0-5288-4ff4-80a8-90b844b6e352 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:33.233854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b69e95b0-5288-4ff4-80a8-90b844b6e352 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:33.234309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b69e95b0-5288-4ff4-80a8-90b844b6e352 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:33.426113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b69e95b0-5288-4ff4-80a8-90b844b6e352 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:33.434217 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b69e95b0-5288-4ff4-80a8-90b844b6e352 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1695 microversion: 2.1 time: 0.215507 Sep 27 11:05:33.434896 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3381/7234] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:33 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1695 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:34.461309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-280df6d3-d4bb-4bbe-bed2-31897cbf4a81 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:34.461956 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-280df6d3-d4bb-4bbe-bed2-31897cbf4a81 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:34.469371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-280df6d3-d4bb-4bbe-bed2-31897cbf4a81 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:34.469866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-280df6d3-d4bb-4bbe-bed2-31897cbf4a81 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:34.470308 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-280df6d3-d4bb-4bbe-bed2-31897cbf4a81 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:34.471019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-280df6d3-d4bb-4bbe-bed2-31897cbf4a81 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:34.471487 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-280df6d3-d4bb-4bbe-bed2-31897cbf4a81 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:34.471918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-280df6d3-d4bb-4bbe-bed2-31897cbf4a81 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:34.615490 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-280df6d3-d4bb-4bbe-bed2-31897cbf4a81 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:34.624359 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-280df6d3-d4bb-4bbe-bed2-31897cbf4a81 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1695 microversion: 2.1 time: 0.168895 Sep 27 11:05:34.624995 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3854/7235] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:34 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1695 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:35.648833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-59713cd5-4f01-4fb4-a3e2-01972d078b96 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:35.649486 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-59713cd5-4f01-4fb4-a3e2-01972d078b96 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:35.657331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59713cd5-4f01-4fb4-a3e2-01972d078b96 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:35.657854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59713cd5-4f01-4fb4-a3e2-01972d078b96 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:35.658336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59713cd5-4f01-4fb4-a3e2-01972d078b96 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:35.659067 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59713cd5-4f01-4fb4-a3e2-01972d078b96 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:35.659551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59713cd5-4f01-4fb4-a3e2-01972d078b96 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:35.660005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-59713cd5-4f01-4fb4-a3e2-01972d078b96 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:35.797853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-59713cd5-4f01-4fb4-a3e2-01972d078b96 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:35.806178 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-59713cd5-4f01-4fb4-a3e2-01972d078b96 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1695 microversion: 2.1 time: 0.161634 Sep 27 11:05:35.806770 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3382/7236] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:35 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1695 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:36.836016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-843147c0-9983-4a85-9b5a-b235e4b0b4ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:36.836989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-843147c0-9983-4a85-9b5a-b235e4b0b4ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:36.846289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-843147c0-9983-4a85-9b5a-b235e4b0b4ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:36.847032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-843147c0-9983-4a85-9b5a-b235e4b0b4ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:36.847704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-843147c0-9983-4a85-9b5a-b235e4b0b4ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:36.848823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-843147c0-9983-4a85-9b5a-b235e4b0b4ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:36.849531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-843147c0-9983-4a85-9b5a-b235e4b0b4ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:36.850194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-843147c0-9983-4a85-9b5a-b235e4b0b4ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:37.005475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-843147c0-9983-4a85-9b5a-b235e4b0b4ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:37.013737 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-843147c0-9983-4a85-9b5a-b235e4b0b4ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1695 microversion: 2.1 time: 0.186347 Sep 27 11:05:37.014347 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3855/7237] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:36 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1695 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:38.042786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-dcaba6d1-1801-4c6b-bc42-f1eb0c557452 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:38.043666 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-dcaba6d1-1801-4c6b-bc42-f1eb0c557452 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:38.055316 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcaba6d1-1801-4c6b-bc42-f1eb0c557452 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:38.055828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcaba6d1-1801-4c6b-bc42-f1eb0c557452 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:38.056301 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcaba6d1-1801-4c6b-bc42-f1eb0c557452 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:38.057071 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcaba6d1-1801-4c6b-bc42-f1eb0c557452 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:38.057557 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcaba6d1-1801-4c6b-bc42-f1eb0c557452 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:38.058009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-dcaba6d1-1801-4c6b-bc42-f1eb0c557452 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:38.211338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-dcaba6d1-1801-4c6b-bc42-f1eb0c557452 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:38.219655 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-dcaba6d1-1801-4c6b-bc42-f1eb0c557452 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1695 microversion: 2.1 time: 0.184177 Sep 27 11:05:38.220248 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3383/7238] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:38 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1695 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:39.247817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-74eaca11-9986-4830-b0e9-848369f546a3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:39.248502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-74eaca11-9986-4830-b0e9-848369f546a3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:39.258664 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74eaca11-9986-4830-b0e9-848369f546a3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:39.259150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74eaca11-9986-4830-b0e9-848369f546a3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:39.259584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74eaca11-9986-4830-b0e9-848369f546a3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:39.260285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74eaca11-9986-4830-b0e9-848369f546a3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:39.260786 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74eaca11-9986-4830-b0e9-848369f546a3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:39.261205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-74eaca11-9986-4830-b0e9-848369f546a3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:39.402178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-74eaca11-9986-4830-b0e9-848369f546a3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:39.410546 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-74eaca11-9986-4830-b0e9-848369f546a3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1695 microversion: 2.1 time: 0.169440 Sep 27 11:05:39.411128 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3856/7239] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:39 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1695 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:40.439178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f77b1dd3-837a-4a22-8819-8e4cf3954e7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:40.440094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f77b1dd3-837a-4a22-8819-8e4cf3954e7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:40.451902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77b1dd3-837a-4a22-8819-8e4cf3954e7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:40.452740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77b1dd3-837a-4a22-8819-8e4cf3954e7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:40.453586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77b1dd3-837a-4a22-8819-8e4cf3954e7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:40.454763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77b1dd3-837a-4a22-8819-8e4cf3954e7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:40.455554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77b1dd3-837a-4a22-8819-8e4cf3954e7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:40.456289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f77b1dd3-837a-4a22-8819-8e4cf3954e7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:40.609104 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f77b1dd3-837a-4a22-8819-8e4cf3954e7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:40.620491 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f77b1dd3-837a-4a22-8819-8e4cf3954e7a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1695 microversion: 2.1 time: 0.188035 Sep 27 11:05:40.621364 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3384/7240] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:40 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1695 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:41.644934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-da3f930c-96aa-449c-97db-4f68f99c2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:41.645527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-da3f930c-96aa-449c-97db-4f68f99c2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:41.652932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da3f930c-96aa-449c-97db-4f68f99c2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:41.653433 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da3f930c-96aa-449c-97db-4f68f99c2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:41.653879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da3f930c-96aa-449c-97db-4f68f99c2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:41.654598 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da3f930c-96aa-449c-97db-4f68f99c2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:41.655083 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da3f930c-96aa-449c-97db-4f68f99c2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:41.655520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-da3f930c-96aa-449c-97db-4f68f99c2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:41.963914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-da3f930c-96aa-449c-97db-4f68f99c2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:41.972227 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-da3f930c-96aa-449c-97db-4f68f99c2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1694 microversion: 2.1 time: 0.331408 Sep 27 11:05:41.972863 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3857/7241] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:41 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1694 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:42.089433 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2a4a7ccb-5391-4ea7-b40b-a8ed6be836b0 req-97c11d42-ff24-4e6c-a210-835b71accfc4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "03d38687-ac91-4e9a-9c02-e57430bdb8da", "name": "network-vif-unplugged", "status": "completed", "tag": "ac164f43-1f65-45c8-a7d9-6cfb713a0941"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:05:42.100716 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2a4a7ccb-5391-4ea7-b40b-a8ed6be836b0 req-97c11d42-ff24-4e6c-a210-835b71accfc4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:42.101220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2a4a7ccb-5391-4ea7-b40b-a8ed6be836b0 req-97c11d42-ff24-4e6c-a210-835b71accfc4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:42.101672 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2a4a7ccb-5391-4ea7-b40b-a8ed6be836b0 req-97c11d42-ff24-4e6c-a210-835b71accfc4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:42.114629 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2a4a7ccb-5391-4ea7-b40b-a8ed6be836b0 req-97c11d42-ff24-4e6c-a210-835b71accfc4 service nova] Creating event network-vif-unplugged:ac164f43-1f65-45c8-a7d9-6cfb713a0941 for instance 03d38687-ac91-4e9a-9c02-e57430bdb8da on nch1-master-1 Sep 27 11:05:42.120325 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-2a4a7ccb-5391-4ea7-b40b-a8ed6be836b0 req-97c11d42-ff24-4e6c-a210-835b71accfc4 service nova] Instance 03d38687-ac91-4e9a-9c02-e57430bdb8da is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 11:05:42.131724 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2a4a7ccb-5391-4ea7-b40b-a8ed6be836b0 req-97c11d42-ff24-4e6c-a210-835b71accfc4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.046194 Sep 27 11:05:42.132335 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3385/7242] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:05:42 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:05:43.006227 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b000b3b3-7940-4d2b-b11c-66d691846e35 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:43.006957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b000b3b3-7940-4d2b-b11c-66d691846e35 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:43.014304 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b000b3b3-7940-4d2b-b11c-66d691846e35 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:43.014795 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b000b3b3-7940-4d2b-b11c-66d691846e35 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:43.015247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b000b3b3-7940-4d2b-b11c-66d691846e35 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:43.015947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b000b3b3-7940-4d2b-b11c-66d691846e35 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:43.016407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b000b3b3-7940-4d2b-b11c-66d691846e35 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:43.016891 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b000b3b3-7940-4d2b-b11c-66d691846e35 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:43.277262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b000b3b3-7940-4d2b-b11c-66d691846e35 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:43.287063 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b000b3b3-7940-4d2b-b11c-66d691846e35 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1694 microversion: 2.1 time: 0.284812 Sep 27 11:05:43.287716 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3858/7243] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:43 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1694 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:44.173661 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a3c06e98-63a9-45ef-9bc2-3fa7b7d622d4 req-d2ba73f0-7c2e-40c6-bf73-a8184b40c8c6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03d38687-ac91-4e9a-9c02-e57430bdb8da", "tag": "ac164f43-1f65-45c8-a7d9-6cfb713a0941"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:05:44.187009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a3c06e98-63a9-45ef-9bc2-3fa7b7d622d4 req-d2ba73f0-7c2e-40c6-bf73-a8184b40c8c6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:44.187499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a3c06e98-63a9-45ef-9bc2-3fa7b7d622d4 req-d2ba73f0-7c2e-40c6-bf73-a8184b40c8c6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:44.187941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a3c06e98-63a9-45ef-9bc2-3fa7b7d622d4 req-d2ba73f0-7c2e-40c6-bf73-a8184b40c8c6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:44.200009 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a3c06e98-63a9-45ef-9bc2-3fa7b7d622d4 req-d2ba73f0-7c2e-40c6-bf73-a8184b40c8c6 service nova] Creating event network-changed:ac164f43-1f65-45c8-a7d9-6cfb713a0941 for instance 03d38687-ac91-4e9a-9c02-e57430bdb8da on nch1-master-1 Sep 27 11:05:44.205825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [req-a3c06e98-63a9-45ef-9bc2-3fa7b7d622d4 req-d2ba73f0-7c2e-40c6-bf73-a8184b40c8c6 service nova] Instance 03d38687-ac91-4e9a-9c02-e57430bdb8da is migrating within the same cell, copying events to all relevant hosts: {'nch1-master-1', 'nch2-master-1'} {{(pid=145969) _get_relevant_hosts /opt/stack/nova/nova/compute/api.py:5938}} Sep 27 11:05:44.217355 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a3c06e98-63a9-45ef-9bc2-3fa7b7d622d4 req-d2ba73f0-7c2e-40c6-bf73-a8184b40c8c6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052315 Sep 27 11:05:44.217944 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3386/7244] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:05:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:05:44.310985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9392c2ca-860c-4dd5-ac29-14c17a50e41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:44.311628 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9392c2ca-860c-4dd5-ac29-14c17a50e41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:44.319015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9392c2ca-860c-4dd5-ac29-14c17a50e41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:44.319510 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9392c2ca-860c-4dd5-ac29-14c17a50e41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:44.319948 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9392c2ca-860c-4dd5-ac29-14c17a50e41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:44.320675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9392c2ca-860c-4dd5-ac29-14c17a50e41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:44.321154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9392c2ca-860c-4dd5-ac29-14c17a50e41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:44.321585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9392c2ca-860c-4dd5-ac29-14c17a50e41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:44.459854 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9392c2ca-860c-4dd5-ac29-14c17a50e41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:44.468249 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9392c2ca-860c-4dd5-ac29-14c17a50e41a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1692 microversion: 2.1 time: 0.161753 Sep 27 11:05:44.468857 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3859/7245] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:44 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1692 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:45.494825 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0bf6da91-9c0b-49ec-80e8-58b932217c43 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:45.495563 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0bf6da91-9c0b-49ec-80e8-58b932217c43 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:45.502311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bf6da91-9c0b-49ec-80e8-58b932217c43 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:45.502900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bf6da91-9c0b-49ec-80e8-58b932217c43 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:45.503358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bf6da91-9c0b-49ec-80e8-58b932217c43 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:45.504089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bf6da91-9c0b-49ec-80e8-58b932217c43 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:45.504561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bf6da91-9c0b-49ec-80e8-58b932217c43 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:45.505038 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0bf6da91-9c0b-49ec-80e8-58b932217c43 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:45.644417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0bf6da91-9c0b-49ec-80e8-58b932217c43 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:45.653005 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0bf6da91-9c0b-49ec-80e8-58b932217c43 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1689 microversion: 2.1 time: 0.163593 Sep 27 11:05:45.653575 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3387/7246] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:45 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1689 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:45.686182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-92da374c-78a9-472a-b653-7394695c9015 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"confirmResize": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:05:45.686778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-92da374c-78a9-472a-b653-7394695c9015 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:45.694138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92da374c-78a9-472a-b653-7394695c9015 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:45.694629 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92da374c-78a9-472a-b653-7394695c9015 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:45.695070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92da374c-78a9-472a-b653-7394695c9015 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:45.695776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92da374c-78a9-472a-b653-7394695c9015 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:45.696238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92da374c-78a9-472a-b653-7394695c9015 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:45.696677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-92da374c-78a9-472a-b653-7394695c9015 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:45.759462 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-92da374c-78a9-472a-b653-7394695c9015 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da/action" status: 204 len: 0 microversion: 2.1 time: 0.077107 Sep 27 11:05:45.760393 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3860/7247] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:05:45 2022] POST /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da/action => generated 0 bytes in 79 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 11:05:45.769793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-39f49b32-c0de-4cb7-8e11-3576ab89ec39 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:45.770423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-39f49b32-c0de-4cb7-8e11-3576ab89ec39 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:45.776809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39f49b32-c0de-4cb7-8e11-3576ab89ec39 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:45.777320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39f49b32-c0de-4cb7-8e11-3576ab89ec39 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:45.777861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39f49b32-c0de-4cb7-8e11-3576ab89ec39 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:45.778662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39f49b32-c0de-4cb7-8e11-3576ab89ec39 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:45.779168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39f49b32-c0de-4cb7-8e11-3576ab89ec39 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:45.779663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-39f49b32-c0de-4cb7-8e11-3576ab89ec39 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:46.001390 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-39f49b32-c0de-4cb7-8e11-3576ab89ec39 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:46.010099 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-39f49b32-c0de-4cb7-8e11-3576ab89ec39 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1689 microversion: 2.1 time: 0.244926 Sep 27 11:05:46.010867 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3388/7248] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:45 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1689 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:46.039674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-390fb45e-6134-498b-99e0-e32fb9e229b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:46.040382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-390fb45e-6134-498b-99e0-e32fb9e229b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:46.047186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-390fb45e-6134-498b-99e0-e32fb9e229b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:46.047754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-390fb45e-6134-498b-99e0-e32fb9e229b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:46.048302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-390fb45e-6134-498b-99e0-e32fb9e229b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:46.049102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-390fb45e-6134-498b-99e0-e32fb9e229b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:46.049567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-390fb45e-6134-498b-99e0-e32fb9e229b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:46.050028 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-390fb45e-6134-498b-99e0-e32fb9e229b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:46.191062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-390fb45e-6134-498b-99e0-e32fb9e229b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:46.199516 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-390fb45e-6134-498b-99e0-e32fb9e229b1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1689 microversion: 2.1 time: 0.164304 Sep 27 11:05:46.200311 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3861/7249] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:46 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1689 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:47.226008 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3c37fb03-44fa-4180-af16-f8a28f8aac95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:05:47.226683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3c37fb03-44fa-4180-af16-f8a28f8aac95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:05:47.234311 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c37fb03-44fa-4180-af16-f8a28f8aac95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:47.234812 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c37fb03-44fa-4180-af16-f8a28f8aac95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:47.235268 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c37fb03-44fa-4180-af16-f8a28f8aac95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:47.235985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c37fb03-44fa-4180-af16-f8a28f8aac95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:47.236457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c37fb03-44fa-4180-af16-f8a28f8aac95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:47.236936 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3c37fb03-44fa-4180-af16-f8a28f8aac95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:47.383431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3c37fb03-44fa-4180-af16-f8a28f8aac95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:05:47.396139 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3c37fb03-44fa-4180-af16-f8a28f8aac95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1681 microversion: 2.1 time: 0.176295 Sep 27 11:05:47.396860 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3389/7250] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:05:47 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1681 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:05:48.658689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-afc8ca99-e839-47f7-a281-0c2408cf326f req-e89fdaf5-bfed-4630-9209-c828afccaf29 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "03d38687-ac91-4e9a-9c02-e57430bdb8da", "name": "network-vif-plugged", "status": "completed", "tag": "ac164f43-1f65-45c8-a7d9-6cfb713a0941"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:05:48.672115 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-afc8ca99-e839-47f7-a281-0c2408cf326f req-e89fdaf5-bfed-4630-9209-c828afccaf29 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:05:48.672606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-afc8ca99-e839-47f7-a281-0c2408cf326f req-e89fdaf5-bfed-4630-9209-c828afccaf29 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:05:48.673077 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-afc8ca99-e839-47f7-a281-0c2408cf326f req-e89fdaf5-bfed-4630-9209-c828afccaf29 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:05:48.684693 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-afc8ca99-e839-47f7-a281-0c2408cf326f req-e89fdaf5-bfed-4630-9209-c828afccaf29 service nova] Creating event network-vif-plugged:ac164f43-1f65-45c8-a7d9-6cfb713a0941 for instance 03d38687-ac91-4e9a-9c02-e57430bdb8da on nch1-master-1 Sep 27 11:05:48.689697 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-afc8ca99-e839-47f7-a281-0c2408cf326f req-e89fdaf5-bfed-4630-9209-c828afccaf29 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035397 Sep 27 11:05:48.690296 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3862/7251] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:05:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:06:09.612999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6053f595-3cdc-443c-b071-43240282f384 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:06:09.615367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6053f595-3cdc-443c-b071-43240282f384 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:09.618649 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=28 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:09.619333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=74,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:09.625952 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6053f595-3cdc-443c-b071-43240282f384 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:09.626468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6053f595-3cdc-443c-b071-43240282f384 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:09.627006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6053f595-3cdc-443c-b071-43240282f384 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:09.627918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6053f595-3cdc-443c-b071-43240282f384 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:09.628382 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6053f595-3cdc-443c-b071-43240282f384 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:09.628869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6053f595-3cdc-443c-b071-43240282f384 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:09.691579 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6053f595-3cdc-443c-b071-43240282f384 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da/action" status: 200 len: 24573 microversion: 2.1 time: 0.082990 Sep 27 11:06:09.692245 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3390/7252] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:06:09 2022] POST /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da/action => generated 24573 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:06:16.749989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=24 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:16.750772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=73,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:16.757085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-f3eb5d04-4538-45c2-9245-b2f0b25d1267 req-297444cf-0328-4583-8d1b-3e10d1d65802 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "03d38687-ac91-4e9a-9c02-e57430bdb8da", "tag": "ac164f43-1f65-45c8-a7d9-6cfb713a0941"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:06:16.766303 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e0fc150e-3afc-4741-b7d6-e3ec1dcbb528 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:16.766900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e0fc150e-3afc-4741-b7d6-e3ec1dcbb528 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:16.767475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f3eb5d04-4538-45c2-9245-b2f0b25d1267 req-297444cf-0328-4583-8d1b-3e10d1d65802 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:16.768163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f3eb5d04-4538-45c2-9245-b2f0b25d1267 req-297444cf-0328-4583-8d1b-3e10d1d65802 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:16.768605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-f3eb5d04-4538-45c2-9245-b2f0b25d1267 req-297444cf-0328-4583-8d1b-3e10d1d65802 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:16.775056 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0fc150e-3afc-4741-b7d6-e3ec1dcbb528 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:16.775546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0fc150e-3afc-4741-b7d6-e3ec1dcbb528 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:16.775981 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0fc150e-3afc-4741-b7d6-e3ec1dcbb528 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:16.776892 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0fc150e-3afc-4741-b7d6-e3ec1dcbb528 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:16.777606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0fc150e-3afc-4741-b7d6-e3ec1dcbb528 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:16.778674 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e0fc150e-3afc-4741-b7d6-e3ec1dcbb528 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:16.782428 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-f3eb5d04-4538-45c2-9245-b2f0b25d1267 req-297444cf-0328-4583-8d1b-3e10d1d65802 service nova] Creating event network-changed:ac164f43-1f65-45c8-a7d9-6cfb713a0941 for instance 03d38687-ac91-4e9a-9c02-e57430bdb8da on nch1-master-1 Sep 27 11:06:16.788453 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-f3eb5d04-4538-45c2-9245-b2f0b25d1267 req-297444cf-0328-4583-8d1b-3e10d1d65802 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035701 Sep 27 11:06:16.789297 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3391/7253] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:06:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:06:16.810996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e0fc150e-3afc-4741-b7d6-e3ec1dcbb528 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 11:06:16.898402 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e0fc150e-3afc-4741-b7d6-e3ec1dcbb528 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "DELETE /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 204 len: 0 microversion: 2.1 time: 0.150214 Sep 27 11:06:16.899024 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3863/7254] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 11:06:16 2022] DELETE /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 0 bytes in 152 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 11:06:16.912813 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6cc31825-cfa1-49d1-89eb-759dfa0b00bb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:16.914451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6cc31825-cfa1-49d1-89eb-759dfa0b00bb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:16.921538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cc31825-cfa1-49d1-89eb-759dfa0b00bb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:16.922474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cc31825-cfa1-49d1-89eb-759dfa0b00bb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:16.923179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cc31825-cfa1-49d1-89eb-759dfa0b00bb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:16.924187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cc31825-cfa1-49d1-89eb-759dfa0b00bb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:16.924937 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cc31825-cfa1-49d1-89eb-759dfa0b00bb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:16.925601 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6cc31825-cfa1-49d1-89eb-759dfa0b00bb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:17.068898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6cc31825-cfa1-49d1-89eb-759dfa0b00bb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:17.077101 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6cc31825-cfa1-49d1-89eb-759dfa0b00bb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1687 microversion: 2.1 time: 0.172919 Sep 27 11:06:17.077690 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3392/7255] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:16 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1687 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:18.101706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ac051a8e-0402-4b61-bc6f-c14df0cb590a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:18.102416 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ac051a8e-0402-4b61-bc6f-c14df0cb590a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:18.110193 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac051a8e-0402-4b61-bc6f-c14df0cb590a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:18.110333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac051a8e-0402-4b61-bc6f-c14df0cb590a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:18.113610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac051a8e-0402-4b61-bc6f-c14df0cb590a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:18.113610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac051a8e-0402-4b61-bc6f-c14df0cb590a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:18.113610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac051a8e-0402-4b61-bc6f-c14df0cb590a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:18.113610 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ac051a8e-0402-4b61-bc6f-c14df0cb590a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:18.375982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ac051a8e-0402-4b61-bc6f-c14df0cb590a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:18.387597 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ac051a8e-0402-4b61-bc6f-c14df0cb590a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 200 len: 1687 microversion: 2.1 time: 0.290478 Sep 27 11:06:18.388212 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3864/7256] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:18 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 1687 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:18.805676 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1f47fd7c-f4fa-46ae-aa35-044116bf453b req-f68600fa-4a56-4696-91d2-021995f81b72 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "03d38687-ac91-4e9a-9c02-e57430bdb8da", "name": "network-vif-deleted", "tag": "ac164f43-1f65-45c8-a7d9-6cfb713a0941"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:06:18.817711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1f47fd7c-f4fa-46ae-aa35-044116bf453b req-f68600fa-4a56-4696-91d2-021995f81b72 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:18.818199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1f47fd7c-f4fa-46ae-aa35-044116bf453b req-f68600fa-4a56-4696-91d2-021995f81b72 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:18.818803 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1f47fd7c-f4fa-46ae-aa35-044116bf453b req-f68600fa-4a56-4696-91d2-021995f81b72 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:18.831764 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1f47fd7c-f4fa-46ae-aa35-044116bf453b req-f68600fa-4a56-4696-91d2-021995f81b72 service nova] Creating event network-vif-deleted:ac164f43-1f65-45c8-a7d9-6cfb713a0941 for instance 03d38687-ac91-4e9a-9c02-e57430bdb8da on nch1-master-1 Sep 27 11:06:18.837515 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1f47fd7c-f4fa-46ae-aa35-044116bf453b req-f68600fa-4a56-4696-91d2-021995f81b72 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036318 Sep 27 11:06:18.838132 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3393/7257] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:06:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:06:19.413812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:19.414676 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 03d38687-ac91-4e9a-9c02-e57430bdb8da] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:19.422297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:19.422716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:19.423159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:19.423863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:19.424323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:19.424787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:19.439647 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] HTTP exception thrown: Instance 03d38687-ac91-4e9a-9c02-e57430bdb8da could not be found. Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Request method failure captured: Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da HTTP/1.1 Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtfHoJ5jcAxOLtRcoV1nD5XXlTZzK0A-5egTQGd5lTVNdkOuvT5Fs0L9J-caES9KtErWFoSb_09yL5egcLFPiM4N0_ALLienspVX9-DRrIBhK2koE7PdcRR57IH0CHGGILEi4fmfkKfiXRleQCLI0R176myMkB92QmVT86Qtqhsg2-o Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 55b42a416dd942e780e533e4c6f2a8b4 Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_55b42a416dd942e780e533e4c6f2a8b4"}]}] Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 55b42a416dd942e780e533e4c6f2a8b4 Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestNetworkAdvancedServerOps-1003182190-project Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 3ec60ad271544bc99d7f394b9b06dd88 Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1003182190-project Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 03d38687-ac91-4e9a-9c02-e57430bdb8da could not be found. Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '03d38687-ac91-4e9a-9c02-e57430bdb8da'} Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 03d38687-ac91-4e9a-9c02-e57430bdb8da could not be found. Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 03d38687-ac91-4e9a-9c02-e57430bdb8da could not be found. Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:06:19.441400 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:06:19.447035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:06:19.447035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:06:19.447035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:06:19.447035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:06:19.447035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 11:06:19.447035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 11:06:19.447035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 03d38687-ac91-4e9a-9c02-e57430bdb8da could not be found. Sep 27 11:06:19.447035 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:06:19.447035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Returning 404 to user: Instance 03d38687-ac91-4e9a-9c02-e57430bdb8da could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 11:06:19.447035 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b80cb080-8750-4b6c-9889-a51e5b94e81b tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da" status: 404 len: 111 microversion: 2.1 time: 0.033843 Sep 27 11:06:19.447035 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3865/7258] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:19 2022] GET /compute/v2.1/servers/03d38687-ac91-4e9a-9c02-e57430bdb8da => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 11:06:22.753064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5f03c5eb-35cf-4c82-bab9-89fbe137073c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '.version_select at 0x7f7f62ade310>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:22.784638 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5f03c5eb-35cf-4c82-bab9-89fbe137073c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-982299882" status: 202 len: 0 microversion: 2.1 time: 0.037084 Sep 27 11:06:22.785379 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3394/7259] 10.222.0.26 () {58 vars in 1281 bytes} [Tue Sep 27 11:06:22 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-982299882 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:06:22.814328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8988a04e-d4e5-4199-8513-26b09e8c9820 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'create', calling method: .version_select at 0x7f7f637978b0>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-1700595915", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBG/NVSAr7twMm5/jH7dKUQyKefbE6/6/UtTn23jUy3H0HWI93P0v69yxd6q2MbGVqgz41J4aIli3wCVrCeJe5Rv1wjC4Fr4vt+qtPxbHiCv6NkQXcr5NwP5CYkbx/GljmQ=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:06:22.832577 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8988a04e-d4e5-4199-8513-26b09e8c9820 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:06:22.839768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-8988a04e-d4e5-4199-8513-26b09e8c9820 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for user 3ec60ad271544bc99d7f394b9b06dd88 and project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:06:22.867747 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8988a04e-d4e5-4199-8513-26b09e8c9820 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 407 microversion: 2.1 time: 0.057528 Sep 27 11:06:22.868925 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3866/7260] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 11:06:22 2022] POST /compute/v2.1/os-keypairs => generated 407 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:06:28.649471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-488955732", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "365492c9-d9cf-4528-96dd-f90c7d4a4aa3"}], "key_name": "tempest-TestNetworkAdvancedServerOps-1700595915", "security_groups": [{"name": "tempest-secgroup-smoke-276740619"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:06:28.774778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] validate_networks() for [('365492c9-d9cf-4528-96dd-f90c7d4a4aa3', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 11:06:28.826364 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=14 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:29.195278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 11:06:29.195655 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 11:06:29.196517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 11:06:29.197154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 11:06:29.197589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 11:06:29.207675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:29.208327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:29.208916 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:29.209864 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:29.210431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:29.210990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:29.231802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:06:29.238674 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for user 3ec60ad271544bc99d7f394b9b06dd88 and project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:06:29.253595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 11:06:29.256303 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 11:06:29.312084 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7ec5a1d0-d9e4-498f-87df-c11227ae3cf3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.667328 Sep 27 11:06:29.314658 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3395/7261] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 11:06:28 2022] POST /compute/v2.1/servers => generated 405 bytes in 671 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 11:06:29.326467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5a48252f-12e4-4daa-978e-53d1dfac9382 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:29.326957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5a48252f-12e4-4daa-978e-53d1dfac9382 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:29.406908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5a48252f-12e4-4daa-978e-53d1dfac9382 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:29.414066 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5a48252f-12e4-4daa-978e-53d1dfac9382 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1266 microversion: 2.1 time: 0.092698 Sep 27 11:06:29.416461 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3867/7262] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:29 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1266 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:30.440628 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8b34c3c0-3d3f-4543-83be-2b4c3c83ccb2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:30.441500 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8b34c3c0-3d3f-4543-83be-2b4c3c83ccb2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:30.449725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b34c3c0-3d3f-4543-83be-2b4c3c83ccb2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:30.450369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b34c3c0-3d3f-4543-83be-2b4c3c83ccb2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:30.451061 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b34c3c0-3d3f-4543-83be-2b4c3c83ccb2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:30.452017 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b34c3c0-3d3f-4543-83be-2b4c3c83ccb2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:30.452484 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b34c3c0-3d3f-4543-83be-2b4c3c83ccb2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:30.452990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8b34c3c0-3d3f-4543-83be-2b4c3c83ccb2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:30.533264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8b34c3c0-3d3f-4543-83be-2b4c3c83ccb2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:30.542769 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8b34c3c0-3d3f-4543-83be-2b4c3c83ccb2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1318 microversion: 2.1 time: 0.107413 Sep 27 11:06:30.543650 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3396/7263] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:30 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1318 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:31.571557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-29be2c76-7882-45ae-8986-181036ca3d1a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:31.572822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-29be2c76-7882-45ae-8986-181036ca3d1a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:31.575060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=22,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:31.585797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29be2c76-7882-45ae-8986-181036ca3d1a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:31.586630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29be2c76-7882-45ae-8986-181036ca3d1a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:31.587262 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29be2c76-7882-45ae-8986-181036ca3d1a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:31.588018 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29be2c76-7882-45ae-8986-181036ca3d1a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:31.588531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29be2c76-7882-45ae-8986-181036ca3d1a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:31.589001 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-29be2c76-7882-45ae-8986-181036ca3d1a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:31.772322 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-29be2c76-7882-45ae-8986-181036ca3d1a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:31.782713 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-29be2c76-7882-45ae-8986-181036ca3d1a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1391 microversion: 2.1 time: 0.215554 Sep 27 11:06:31.784000 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3868/7264] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:31 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1391 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:32.463183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7c4270fe-7b49-431c-82c8-03a16a88371c req-9ccd6db2-ad6f-483c-8755-1ea6408243c8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ff68f81-0424-4089-9d47-5aab12c8b217", "tag": "d972135e-ac14-40bd-a0e9-896784309aa0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:06:32.484517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7c4270fe-7b49-431c-82c8-03a16a88371c req-9ccd6db2-ad6f-483c-8755-1ea6408243c8 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:32.485448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7c4270fe-7b49-431c-82c8-03a16a88371c req-9ccd6db2-ad6f-483c-8755-1ea6408243c8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:32.486517 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7c4270fe-7b49-431c-82c8-03a16a88371c req-9ccd6db2-ad6f-483c-8755-1ea6408243c8 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:32.512538 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7c4270fe-7b49-431c-82c8-03a16a88371c req-9ccd6db2-ad6f-483c-8755-1ea6408243c8 service nova] Creating event network-changed:d972135e-ac14-40bd-a0e9-896784309aa0 for instance 7ff68f81-0424-4089-9d47-5aab12c8b217 on nch2-master-1 Sep 27 11:06:32.519331 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7c4270fe-7b49-431c-82c8-03a16a88371c req-9ccd6db2-ad6f-483c-8755-1ea6408243c8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064505 Sep 27 11:06:32.520090 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3397/7265] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:06:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:06:32.808445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-45e0719d-bbe1-4478-b16a-e1b32e8a64e9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:32.809066 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-45e0719d-bbe1-4478-b16a-e1b32e8a64e9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:32.815260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e0719d-bbe1-4478-b16a-e1b32e8a64e9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:32.815756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e0719d-bbe1-4478-b16a-e1b32e8a64e9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:32.816204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e0719d-bbe1-4478-b16a-e1b32e8a64e9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:32.817070 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e0719d-bbe1-4478-b16a-e1b32e8a64e9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:32.817531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e0719d-bbe1-4478-b16a-e1b32e8a64e9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:32.817964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45e0719d-bbe1-4478-b16a-e1b32e8a64e9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:32.958074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-45e0719d-bbe1-4478-b16a-e1b32e8a64e9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:32.968232 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-45e0719d-bbe1-4478-b16a-e1b32e8a64e9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1391 microversion: 2.1 time: 0.165341 Sep 27 11:06:32.969260 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3869/7266] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:32 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1391 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:33.996880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-080247c3-9a75-4b31-aef6-cc445477c433 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:33.997786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-080247c3-9a75-4b31-aef6-cc445477c433 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:34.007521 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-080247c3-9a75-4b31-aef6-cc445477c433 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:34.008023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-080247c3-9a75-4b31-aef6-cc445477c433 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:34.008479 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-080247c3-9a75-4b31-aef6-cc445477c433 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:34.009274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-080247c3-9a75-4b31-aef6-cc445477c433 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:34.009741 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-080247c3-9a75-4b31-aef6-cc445477c433 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:34.010182 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-080247c3-9a75-4b31-aef6-cc445477c433 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:34.151861 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-080247c3-9a75-4b31-aef6-cc445477c433 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:34.164808 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-080247c3-9a75-4b31-aef6-cc445477c433 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1391 microversion: 2.1 time: 0.175889 Sep 27 11:06:34.165412 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3398/7267] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:33 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1391 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:35.197706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6bf1a72b-aa95-4816-97d0-b10bd39349e3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:35.198340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6bf1a72b-aa95-4816-97d0-b10bd39349e3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:35.205603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf1a72b-aa95-4816-97d0-b10bd39349e3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:35.206067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf1a72b-aa95-4816-97d0-b10bd39349e3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:35.206594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf1a72b-aa95-4816-97d0-b10bd39349e3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:35.207310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf1a72b-aa95-4816-97d0-b10bd39349e3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:35.207773 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf1a72b-aa95-4816-97d0-b10bd39349e3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:35.208210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf1a72b-aa95-4816-97d0-b10bd39349e3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:35.348695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6bf1a72b-aa95-4816-97d0-b10bd39349e3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:35.357175 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6bf1a72b-aa95-4816-97d0-b10bd39349e3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1391 microversion: 2.1 time: 0.163618 Sep 27 11:06:35.357871 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3870/7268] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:35 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1391 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:36.383689 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7207e6b-535d-4701-9701-fb18e96d3491 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:36.384338 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7207e6b-535d-4701-9701-fb18e96d3491 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:36.391675 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7207e6b-535d-4701-9701-fb18e96d3491 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:36.392186 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7207e6b-535d-4701-9701-fb18e96d3491 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:36.392638 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7207e6b-535d-4701-9701-fb18e96d3491 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:36.393462 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7207e6b-535d-4701-9701-fb18e96d3491 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:36.393941 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7207e6b-535d-4701-9701-fb18e96d3491 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:36.394437 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7207e6b-535d-4701-9701-fb18e96d3491 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:36.538980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c7207e6b-535d-4701-9701-fb18e96d3491 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:36.547327 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c7207e6b-535d-4701-9701-fb18e96d3491 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1391 microversion: 2.1 time: 0.168850 Sep 27 11:06:36.547903 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3399/7269] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:36 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1391 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:37.571830 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0e3c15e9-eb91-4084-8977-2215b61dadad tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:37.572384 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0e3c15e9-eb91-4084-8977-2215b61dadad tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:37.578874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e3c15e9-eb91-4084-8977-2215b61dadad tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:37.579373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e3c15e9-eb91-4084-8977-2215b61dadad tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:37.579812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e3c15e9-eb91-4084-8977-2215b61dadad tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:37.580509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e3c15e9-eb91-4084-8977-2215b61dadad tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:37.580998 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e3c15e9-eb91-4084-8977-2215b61dadad tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:37.581453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0e3c15e9-eb91-4084-8977-2215b61dadad tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:37.718917 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0e3c15e9-eb91-4084-8977-2215b61dadad tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:37.729583 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0e3c15e9-eb91-4084-8977-2215b61dadad tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1391 microversion: 2.1 time: 0.162465 Sep 27 11:06:37.730138 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3871/7270] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:37 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1391 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:38.755016 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a98b782-9c33-4a2f-a4da-bf2a895ed13a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:38.755728 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a98b782-9c33-4a2f-a4da-bf2a895ed13a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:38.762994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a98b782-9c33-4a2f-a4da-bf2a895ed13a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:38.763499 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a98b782-9c33-4a2f-a4da-bf2a895ed13a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:38.763953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a98b782-9c33-4a2f-a4da-bf2a895ed13a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:38.764700 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a98b782-9c33-4a2f-a4da-bf2a895ed13a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:38.765176 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a98b782-9c33-4a2f-a4da-bf2a895ed13a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:38.765622 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a98b782-9c33-4a2f-a4da-bf2a895ed13a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:38.879190 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-10e3434d-3405-442c-9023-9852397a21d1 req-327f5220-e222-4867-8537-97d34553dc81 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ff68f81-0424-4089-9d47-5aab12c8b217", "name": "network-vif-plugged", "status": "completed", "tag": "d972135e-ac14-40bd-a0e9-896784309aa0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:06:38.890840 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-10e3434d-3405-442c-9023-9852397a21d1 req-327f5220-e222-4867-8537-97d34553dc81 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:38.891396 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-10e3434d-3405-442c-9023-9852397a21d1 req-327f5220-e222-4867-8537-97d34553dc81 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:38.891847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-10e3434d-3405-442c-9023-9852397a21d1 req-327f5220-e222-4867-8537-97d34553dc81 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:38.906755 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-10e3434d-3405-442c-9023-9852397a21d1 req-327f5220-e222-4867-8537-97d34553dc81 service nova] Creating event network-vif-plugged:d972135e-ac14-40bd-a0e9-896784309aa0 for instance 7ff68f81-0424-4089-9d47-5aab12c8b217 on nch2-master-1 Sep 27 11:06:38.909814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0a98b782-9c33-4a2f-a4da-bf2a895ed13a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:38.918990 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-10e3434d-3405-442c-9023-9852397a21d1 req-327f5220-e222-4867-8537-97d34553dc81 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044821 Sep 27 11:06:38.920615 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3872/7271] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:06:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:06:38.923691 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a98b782-9c33-4a2f-a4da-bf2a895ed13a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1391 microversion: 2.1 time: 0.174154 Sep 27 11:06:38.924333 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3400/7272] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:38 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1391 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:39.947105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:39.952335 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-77c3587e-7c90-4466-935f-73d0846517a9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:39.953067 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-77c3587e-7c90-4466-935f-73d0846517a9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:39.959914 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77c3587e-7c90-4466-935f-73d0846517a9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:39.960292 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77c3587e-7c90-4466-935f-73d0846517a9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:39.960842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77c3587e-7c90-4466-935f-73d0846517a9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:39.961715 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77c3587e-7c90-4466-935f-73d0846517a9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:39.962191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77c3587e-7c90-4466-935f-73d0846517a9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:39.962595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-77c3587e-7c90-4466-935f-73d0846517a9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:40.101675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-77c3587e-7c90-4466-935f-73d0846517a9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:40.110368 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-77c3587e-7c90-4466-935f-73d0846517a9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1391 microversion: 2.1 time: 0.164380 Sep 27 11:06:40.110963 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3873/7273] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:39 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1391 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 11:06:41.139109 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-85cacc83-10c9-4083-a477-04f7b9df5391 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:41.140162 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-85cacc83-10c9-4083-a477-04f7b9df5391 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:41.143371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:41.144786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:41.147767 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=37,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:06:41.155359 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85cacc83-10c9-4083-a477-04f7b9df5391 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:41.156198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85cacc83-10c9-4083-a477-04f7b9df5391 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:41.156678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85cacc83-10c9-4083-a477-04f7b9df5391 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:41.157393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85cacc83-10c9-4083-a477-04f7b9df5391 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:41.157913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85cacc83-10c9-4083-a477-04f7b9df5391 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:41.158362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-85cacc83-10c9-4083-a477-04f7b9df5391 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:41.309902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-85cacc83-10c9-4083-a477-04f7b9df5391 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:41.318470 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-85cacc83-10c9-4083-a477-04f7b9df5391 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1563 microversion: 2.1 time: 0.186436 Sep 27 11:06:41.319051 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3401/7274] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:41 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1563 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:06:41.341966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-157a744d-7155-4030-b857-6f7f826e01a0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:41.342583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-157a744d-7155-4030-b857-6f7f826e01a0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:41.349765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-157a744d-7155-4030-b857-6f7f826e01a0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:41.350256 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-157a744d-7155-4030-b857-6f7f826e01a0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:41.350790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-157a744d-7155-4030-b857-6f7f826e01a0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:41.351398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-157a744d-7155-4030-b857-6f7f826e01a0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:41.351865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-157a744d-7155-4030-b857-6f7f826e01a0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:41.352392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-157a744d-7155-4030-b857-6f7f826e01a0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:41.503451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-157a744d-7155-4030-b857-6f7f826e01a0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:41.511787 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-157a744d-7155-4030-b857-6f7f826e01a0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1563 microversion: 2.1 time: 0.173822 Sep 27 11:06:41.512369 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3874/7275] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:41 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1563 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 11:06:43.219342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0af31a5e-5a90-4495-9f12-423392b9cdbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:06:43.220013 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0af31a5e-5a90-4495-9f12-423392b9cdbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:06:43.222137 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0b72257a-12e0-4e52-a242-902c8604d034 req-b6a719da-934e-44a2-b1c4-2351bba8182e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ff68f81-0424-4089-9d47-5aab12c8b217", "tag": "d972135e-ac14-40bd-a0e9-896784309aa0"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:06:43.227483 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0af31a5e-5a90-4495-9f12-423392b9cdbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:43.228422 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0af31a5e-5a90-4495-9f12-423392b9cdbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:43.228942 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0af31a5e-5a90-4495-9f12-423392b9cdbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:43.229789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0af31a5e-5a90-4495-9f12-423392b9cdbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:43.230708 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0af31a5e-5a90-4495-9f12-423392b9cdbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:43.231331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0af31a5e-5a90-4495-9f12-423392b9cdbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:43.232923 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0b72257a-12e0-4e52-a242-902c8604d034 req-b6a719da-934e-44a2-b1c4-2351bba8182e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:06:43.233435 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0b72257a-12e0-4e52-a242-902c8604d034 req-b6a719da-934e-44a2-b1c4-2351bba8182e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:06:43.233837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0b72257a-12e0-4e52-a242-902c8604d034 req-b6a719da-934e-44a2-b1c4-2351bba8182e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:06:43.246349 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0b72257a-12e0-4e52-a242-902c8604d034 req-b6a719da-934e-44a2-b1c4-2351bba8182e service nova] Creating event network-changed:d972135e-ac14-40bd-a0e9-896784309aa0 for instance 7ff68f81-0424-4089-9d47-5aab12c8b217 on nch2-master-1 Sep 27 11:06:43.251774 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0b72257a-12e0-4e52-a242-902c8604d034 req-b6a719da-934e-44a2-b1c4-2351bba8182e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033950 Sep 27 11:06:43.252465 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3875/7276] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:06:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 11:06:43.459518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0af31a5e-5a90-4495-9f12-423392b9cdbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:06:43.469043 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0af31a5e-5a90-4495-9f12-423392b9cdbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1563 microversion: 2.1 time: 0.254233 Sep 27 11:06:43.470075 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3402/7277] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:06:43 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1563 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:07:16.855817 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:07:16.856768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=40 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:07:16.858019 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:07:16.858360 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 11:07:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217/action (ip 10.222.0.26) !!! Sep 27 11:07:16.863163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2cfcf71e-3a54-4e70-83f6-064332d920ec tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:07:16.865782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2cfcf71e-3a54-4e70-83f6-064332d920ec tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:07:16.877213 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cfcf71e-3a54-4e70-83f6-064332d920ec tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:07:16.877814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cfcf71e-3a54-4e70-83f6-064332d920ec tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:07:16.878166 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cfcf71e-3a54-4e70-83f6-064332d920ec tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:07:16.878882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cfcf71e-3a54-4e70-83f6-064332d920ec tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:07:16.879337 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cfcf71e-3a54-4e70-83f6-064332d920ec tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:07:16.879761 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2cfcf71e-3a54-4e70-83f6-064332d920ec tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:07:16.963673 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2cfcf71e-3a54-4e70-83f6-064332d920ec tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217/action" status: 200 len: 24564 microversion: 2.1 time: 0.109676 Sep 27 11:07:16.964685 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3876/7278] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:07:16 2022] POST /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217/action => generated 24564 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:07:55.922717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:07:55.923488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=40 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:07:55.929534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-344b3266-524b-4a38-ac2d-7d4e932ee938 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:07:55.937080 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-344b3266-524b-4a38-ac2d-7d4e932ee938 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:07:55.937588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-344b3266-524b-4a38-ac2d-7d4e932ee938 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:07:55.938044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-344b3266-524b-4a38-ac2d-7d4e932ee938 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:07:55.962623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-344b3266-524b-4a38-ac2d-7d4e932ee938 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Going to try to stop instance {{(pid=145969) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 27 11:07:56.016572 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-344b3266-524b-4a38-ac2d-7d4e932ee938 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217/action" status: 202 len: 0 microversion: 2.1 time: 0.095279 Sep 27 11:07:56.017353 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3403/7279] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:07:55 2022] POST /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217/action => generated 0 bytes in 97 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:07:56.027002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fcbb2bcb-b203-4cc8-a972-dbbd17a49457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:07:56.027716 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fcbb2bcb-b203-4cc8-a972-dbbd17a49457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:07:56.030720 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:07:56.031378 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:07:56.037686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcbb2bcb-b203-4cc8-a972-dbbd17a49457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:07:56.038230 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcbb2bcb-b203-4cc8-a972-dbbd17a49457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:07:56.038679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcbb2bcb-b203-4cc8-a972-dbbd17a49457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:07:56.039514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcbb2bcb-b203-4cc8-a972-dbbd17a49457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:07:56.039979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcbb2bcb-b203-4cc8-a972-dbbd17a49457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:07:56.040415 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fcbb2bcb-b203-4cc8-a972-dbbd17a49457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:07:56.243590 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fcbb2bcb-b203-4cc8-a972-dbbd17a49457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:07:56.252006 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fcbb2bcb-b203-4cc8-a972-dbbd17a49457 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1692 microversion: 2.1 time: 0.229531 Sep 27 11:07:56.252589 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3877/7280] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:07:56 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1692 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:07:57.283723 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-782da4a2-bc70-4a3b-a687-84eb83c51188 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:07:57.284960 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-782da4a2-bc70-4a3b-a687-84eb83c51188 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:07:57.295534 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-782da4a2-bc70-4a3b-a687-84eb83c51188 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:07:57.296363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-782da4a2-bc70-4a3b-a687-84eb83c51188 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:07:57.297157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-782da4a2-bc70-4a3b-a687-84eb83c51188 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:07:57.298371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-782da4a2-bc70-4a3b-a687-84eb83c51188 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:07:57.299155 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-782da4a2-bc70-4a3b-a687-84eb83c51188 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:07:57.299908 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-782da4a2-bc70-4a3b-a687-84eb83c51188 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:07:57.498465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-782da4a2-bc70-4a3b-a687-84eb83c51188 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:07:57.507149 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-782da4a2-bc70-4a3b-a687-84eb83c51188 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1692 microversion: 2.1 time: 0.232250 Sep 27 11:07:57.507729 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3404/7281] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:07:57 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1692 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:07:58.535176 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-290c3674-1f72-456d-96ae-8129d2aab4af tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:07:58.536196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-290c3674-1f72-456d-96ae-8129d2aab4af tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:07:58.547662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-290c3674-1f72-456d-96ae-8129d2aab4af tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:07:58.548156 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-290c3674-1f72-456d-96ae-8129d2aab4af tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:07:58.548597 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-290c3674-1f72-456d-96ae-8129d2aab4af tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:07:58.549346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-290c3674-1f72-456d-96ae-8129d2aab4af tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:07:58.549808 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-290c3674-1f72-456d-96ae-8129d2aab4af tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:07:58.550248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-290c3674-1f72-456d-96ae-8129d2aab4af tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:07:58.692287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-290c3674-1f72-456d-96ae-8129d2aab4af tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:07:58.700527 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-290c3674-1f72-456d-96ae-8129d2aab4af tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1692 microversion: 2.1 time: 0.172577 Sep 27 11:07:58.701128 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3878/7282] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:07:58 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1692 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:07:59.724025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a4d01f94-400a-45bd-aa62-2e52be7b5c11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:07:59.724792 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a4d01f94-400a-45bd-aa62-2e52be7b5c11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:07:59.732417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4d01f94-400a-45bd-aa62-2e52be7b5c11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:07:59.732946 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4d01f94-400a-45bd-aa62-2e52be7b5c11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:07:59.733398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4d01f94-400a-45bd-aa62-2e52be7b5c11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:07:59.734117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4d01f94-400a-45bd-aa62-2e52be7b5c11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:07:59.734587 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4d01f94-400a-45bd-aa62-2e52be7b5c11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:07:59.735025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a4d01f94-400a-45bd-aa62-2e52be7b5c11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:07:59.888103 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-a4d01f94-400a-45bd-aa62-2e52be7b5c11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:07:59.899551 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a4d01f94-400a-45bd-aa62-2e52be7b5c11 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1692 microversion: 2.1 time: 0.179538 Sep 27 11:07:59.900145 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3405/7283] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:07:59 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1692 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:08:00.923842 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d7b951a5-9cd2-4d35-af8a-fff76244cd06 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:08:00.924425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d7b951a5-9cd2-4d35-af8a-fff76244cd06 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:08:00.931977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b951a5-9cd2-4d35-af8a-fff76244cd06 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:00.932469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b951a5-9cd2-4d35-af8a-fff76244cd06 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:00.932954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b951a5-9cd2-4d35-af8a-fff76244cd06 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:00.933662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b951a5-9cd2-4d35-af8a-fff76244cd06 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:00.934120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b951a5-9cd2-4d35-af8a-fff76244cd06 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:00.934553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d7b951a5-9cd2-4d35-af8a-fff76244cd06 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:01.069309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d7b951a5-9cd2-4d35-af8a-fff76244cd06 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:08:01.078143 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d7b951a5-9cd2-4d35-af8a-fff76244cd06 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1692 microversion: 2.1 time: 0.158189 Sep 27 11:08:01.078710 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3879/7284] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:08:00 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1692 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:08:02.104834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9ffb79fe-8876-45f9-a302-04fe0ffeafb8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:08:02.105419 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9ffb79fe-8876-45f9-a302-04fe0ffeafb8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:08:02.112577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ffb79fe-8876-45f9-a302-04fe0ffeafb8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:02.113093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ffb79fe-8876-45f9-a302-04fe0ffeafb8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:02.113579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ffb79fe-8876-45f9-a302-04fe0ffeafb8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:02.114304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ffb79fe-8876-45f9-a302-04fe0ffeafb8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:02.114777 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ffb79fe-8876-45f9-a302-04fe0ffeafb8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:02.115225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9ffb79fe-8876-45f9-a302-04fe0ffeafb8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:02.254881 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9ffb79fe-8876-45f9-a302-04fe0ffeafb8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:08:02.263711 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9ffb79fe-8876-45f9-a302-04fe0ffeafb8 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1692 microversion: 2.1 time: 0.164165 Sep 27 11:08:02.264302 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3406/7285] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:08:02 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1692 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:08:03.292446 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-44db14f1-9dd9-4d9f-9aab-23e4eda7dd1d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:08:03.293500 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-44db14f1-9dd9-4d9f-9aab-23e4eda7dd1d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:08:03.305289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44db14f1-9dd9-4d9f-9aab-23e4eda7dd1d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:03.305865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44db14f1-9dd9-4d9f-9aab-23e4eda7dd1d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:03.306328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44db14f1-9dd9-4d9f-9aab-23e4eda7dd1d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:03.307074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44db14f1-9dd9-4d9f-9aab-23e4eda7dd1d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:03.307533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44db14f1-9dd9-4d9f-9aab-23e4eda7dd1d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:03.307964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-44db14f1-9dd9-4d9f-9aab-23e4eda7dd1d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:03.455593 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-44db14f1-9dd9-4d9f-9aab-23e4eda7dd1d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:08:03.465959 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-44db14f1-9dd9-4d9f-9aab-23e4eda7dd1d tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1692 microversion: 2.1 time: 0.180290 Sep 27 11:08:03.466540 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3880/7286] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:08:03 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1692 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:08:04.494727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-265726d4-1513-41ee-a79e-7cdec5f1555c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:08:04.495799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-265726d4-1513-41ee-a79e-7cdec5f1555c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:08:04.507473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-265726d4-1513-41ee-a79e-7cdec5f1555c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:04.508309 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-265726d4-1513-41ee-a79e-7cdec5f1555c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:04.509270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-265726d4-1513-41ee-a79e-7cdec5f1555c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:04.510244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-265726d4-1513-41ee-a79e-7cdec5f1555c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:04.510715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-265726d4-1513-41ee-a79e-7cdec5f1555c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:04.511159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-265726d4-1513-41ee-a79e-7cdec5f1555c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:04.657023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-265726d4-1513-41ee-a79e-7cdec5f1555c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:08:04.665534 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-265726d4-1513-41ee-a79e-7cdec5f1555c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1692 microversion: 2.1 time: 0.178297 Sep 27 11:08:04.666094 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3407/7287] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:08:04 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1692 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:08:05.694376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3527fd52-2592-4743-ac9c-2857d46ccb31 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:08:05.695255 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3527fd52-2592-4743-ac9c-2857d46ccb31 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:08:05.707315 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3527fd52-2592-4743-ac9c-2857d46ccb31 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:05.708150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3527fd52-2592-4743-ac9c-2857d46ccb31 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:05.708944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3527fd52-2592-4743-ac9c-2857d46ccb31 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:05.710117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3527fd52-2592-4743-ac9c-2857d46ccb31 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:05.710884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3527fd52-2592-4743-ac9c-2857d46ccb31 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:05.711599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3527fd52-2592-4743-ac9c-2857d46ccb31 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:05.849398 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3527fd52-2592-4743-ac9c-2857d46ccb31 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:08:05.857603 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3527fd52-2592-4743-ac9c-2857d46ccb31 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1692 microversion: 2.1 time: 0.169972 Sep 27 11:08:05.858184 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3881/7288] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:08:05 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1692 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:08:06.885959 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c789c88c-22a5-4b30-9c6c-9a42958ba8ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:08:06.887220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c789c88c-22a5-4b30-9c6c-9a42958ba8ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:08:06.899201 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c789c88c-22a5-4b30-9c6c-9a42958ba8ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:06.900037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c789c88c-22a5-4b30-9c6c-9a42958ba8ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:06.900846 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c789c88c-22a5-4b30-9c6c-9a42958ba8ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:06.902232 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c789c88c-22a5-4b30-9c6c-9a42958ba8ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:06.902713 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c789c88c-22a5-4b30-9c6c-9a42958ba8ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:06.903151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c789c88c-22a5-4b30-9c6c-9a42958ba8ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:07.043891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c789c88c-22a5-4b30-9c6c-9a42958ba8ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:08:07.052113 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c789c88c-22a5-4b30-9c6c-9a42958ba8ab tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1669 microversion: 2.1 time: 0.173015 Sep 27 11:08:07.052691 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3408/7289] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:08:06 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1669 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:08:08.161432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-798e6d2d-0e42-401d-9cf0-62e345ea7b8e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:08:08.165048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-798e6d2d-0e42-401d-9cf0-62e345ea7b8e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:08:08.172632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798e6d2d-0e42-401d-9cf0-62e345ea7b8e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:08.173150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798e6d2d-0e42-401d-9cf0-62e345ea7b8e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:08.173586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798e6d2d-0e42-401d-9cf0-62e345ea7b8e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:08.174288 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798e6d2d-0e42-401d-9cf0-62e345ea7b8e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:08.174744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798e6d2d-0e42-401d-9cf0-62e345ea7b8e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:08.175173 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-798e6d2d-0e42-401d-9cf0-62e345ea7b8e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:08.236902 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-798e6d2d-0e42-401d-9cf0-62e345ea7b8e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217/action" status: 200 len: 38344 microversion: 2.1 time: 0.082646 Sep 27 11:08:08.237577 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3882/7290] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:08:08 2022] POST /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217/action => generated 38344 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:08:08.252378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3d89ccd-dba4-4ff1-8d5c-cccdf376906a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:08:08.259843 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3d89ccd-dba4-4ff1-8d5c-cccdf376906a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:08.260358 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3d89ccd-dba4-4ff1-8d5c-cccdf376906a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:08.261015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3d89ccd-dba4-4ff1-8d5c-cccdf376906a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:08.284645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e3d89ccd-dba4-4ff1-8d5c-cccdf376906a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Going to try to start instance {{(pid=145969) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 27 11:08:08.333212 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e3d89ccd-dba4-4ff1-8d5c-cccdf376906a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217/action" status: 202 len: 0 microversion: 2.1 time: 0.084801 Sep 27 11:08:08.334315 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3409/7291] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:08:08 2022] POST /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217/action => generated 0 bytes in 87 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:08:08.345159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-59d07fd7-1a98-4705-a4d7-7a325f719395 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:08:08.345869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-59d07fd7-1a98-4705-a4d7-7a325f719395 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:08:08.353134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d07fd7-1a98-4705-a4d7-7a325f719395 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:08.353615 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d07fd7-1a98-4705-a4d7-7a325f719395 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:08.354277 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d07fd7-1a98-4705-a4d7-7a325f719395 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:08.355646 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d07fd7-1a98-4705-a4d7-7a325f719395 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:08.356112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d07fd7-1a98-4705-a4d7-7a325f719395 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:08.356549 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59d07fd7-1a98-4705-a4d7-7a325f719395 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:08.508527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59d07fd7-1a98-4705-a4d7-7a325f719395 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:08:08.516914 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-59d07fd7-1a98-4705-a4d7-7a325f719395 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1678 microversion: 2.1 time: 0.176040 Sep 27 11:08:08.517505 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3883/7292] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:08:08 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1678 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:08:09.544163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-895a2f70-b7fa-4b86-a5e6-3ec1ee548c54 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:08:09.544996 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-895a2f70-b7fa-4b86-a5e6-3ec1ee548c54 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:08:09.552531 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-895a2f70-b7fa-4b86-a5e6-3ec1ee548c54 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:09.553066 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-895a2f70-b7fa-4b86-a5e6-3ec1ee548c54 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:09.553530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-895a2f70-b7fa-4b86-a5e6-3ec1ee548c54 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:09.554258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-895a2f70-b7fa-4b86-a5e6-3ec1ee548c54 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:09.554743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-895a2f70-b7fa-4b86-a5e6-3ec1ee548c54 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:09.555192 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-895a2f70-b7fa-4b86-a5e6-3ec1ee548c54 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:09.695362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-895a2f70-b7fa-4b86-a5e6-3ec1ee548c54 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:08:09.703754 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-895a2f70-b7fa-4b86-a5e6-3ec1ee548c54 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1678 microversion: 2.1 time: 0.165644 Sep 27 11:08:09.704468 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3410/7293] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:08:09 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1678 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:08:10.732442 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-45c845e3-b59c-4106-949b-559628734324 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:08:10.733451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-45c845e3-b59c-4106-949b-559628734324 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:08:10.746470 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45c845e3-b59c-4106-949b-559628734324 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:10.746964 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45c845e3-b59c-4106-949b-559628734324 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:10.747375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45c845e3-b59c-4106-949b-559628734324 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:10.748108 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45c845e3-b59c-4106-949b-559628734324 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:10.748566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45c845e3-b59c-4106-949b-559628734324 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:10.749025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-45c845e3-b59c-4106-949b-559628734324 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:10.897024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-45c845e3-b59c-4106-949b-559628734324 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:08:10.907204 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-45c845e3-b59c-4106-949b-559628734324 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1682 microversion: 2.1 time: 0.181850 Sep 27 11:08:10.907785 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3884/7294] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:08:10 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1682 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:08:43.193166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-696a1a4c-16a0-4743-bcf2-a5b443a96328 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:08:43.195755 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-696a1a4c-16a0-4743-bcf2-a5b443a96328 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:08:43.198554 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=16 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:08:43.199220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:08:43.205320 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-696a1a4c-16a0-4743-bcf2-a5b443a96328 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:43.205822 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-696a1a4c-16a0-4743-bcf2-a5b443a96328 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:43.206282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-696a1a4c-16a0-4743-bcf2-a5b443a96328 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:43.207026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-696a1a4c-16a0-4743-bcf2-a5b443a96328 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:08:43.207480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-696a1a4c-16a0-4743-bcf2-a5b443a96328 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:08:43.207934 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-696a1a4c-16a0-4743-bcf2-a5b443a96328 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:08:43.292820 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-696a1a4c-16a0-4743-bcf2-a5b443a96328 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217/action" status: 200 len: 62909 microversion: 2.1 time: 0.103446 Sep 27 11:08:43.293760 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3411/7295] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:08:43 2022] POST /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217/action => generated 62909 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:09:07.150604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f69a9c54-a00e-409f-9dab-69c22947e3f3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:07.151160 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f69a9c54-a00e-409f-9dab-69c22947e3f3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:07.153916 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=16 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:07.154730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=46 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:07.161967 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:07.162744 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=4 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:07.166045 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:09:07.166385 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 11:09:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 (ip 10.222.0.26) !!! Sep 27 11:09:07.182253 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f69a9c54-a00e-409f-9dab-69c22947e3f3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:07.182870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f69a9c54-a00e-409f-9dab-69c22947e3f3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:07.183437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f69a9c54-a00e-409f-9dab-69c22947e3f3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:07.184263 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f69a9c54-a00e-409f-9dab-69c22947e3f3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:07.184823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f69a9c54-a00e-409f-9dab-69c22947e3f3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:07.185447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f69a9c54-a00e-409f-9dab-69c22947e3f3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:07.213582 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f69a9c54-a00e-409f-9dab-69c22947e3f3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 11:09:07.223678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-4569ed85-990b-42da-920d-bafc9dac73d6 req-6472bd15-eba0-461a-b595-7bf6325fdcf5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ff68f81-0424-4089-9d47-5aab12c8b217", "tag": "d972135e-ac14-40bd-a0e9-896784309aa0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:09:07.235636 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4569ed85-990b-42da-920d-bafc9dac73d6 req-6472bd15-eba0-461a-b595-7bf6325fdcf5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:07.236133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4569ed85-990b-42da-920d-bafc9dac73d6 req-6472bd15-eba0-461a-b595-7bf6325fdcf5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:07.236578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-4569ed85-990b-42da-920d-bafc9dac73d6 req-6472bd15-eba0-461a-b595-7bf6325fdcf5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:07.247941 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-4569ed85-990b-42da-920d-bafc9dac73d6 req-6472bd15-eba0-461a-b595-7bf6325fdcf5 service nova] Creating event network-changed:d972135e-ac14-40bd-a0e9-896784309aa0 for instance 7ff68f81-0424-4089-9d47-5aab12c8b217 on nch2-master-1 Sep 27 11:09:07.253133 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-4569ed85-990b-42da-920d-bafc9dac73d6 req-6472bd15-eba0-461a-b595-7bf6325fdcf5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.096613 Sep 27 11:09:07.253740 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3412/7296] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:09:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:09:07.304564 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f69a9c54-a00e-409f-9dab-69c22947e3f3 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 204 len: 0 microversion: 2.1 time: 0.157542 Sep 27 11:09:07.308360 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3885/7297] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 11:09:07 2022] DELETE /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 11:09:07.316841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-df39b1ea-1eba-426c-83f4-4a44765eb172 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:07.316841 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-df39b1ea-1eba-426c-83f4-4a44765eb172 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:07.323476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df39b1ea-1eba-426c-83f4-4a44765eb172 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:07.323984 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df39b1ea-1eba-426c-83f4-4a44765eb172 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:07.324434 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df39b1ea-1eba-426c-83f4-4a44765eb172 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:07.325190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df39b1ea-1eba-426c-83f4-4a44765eb172 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:07.325725 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df39b1ea-1eba-426c-83f4-4a44765eb172 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:07.326166 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-df39b1ea-1eba-426c-83f4-4a44765eb172 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:07.464006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-df39b1ea-1eba-426c-83f4-4a44765eb172 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:07.472088 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-df39b1ea-1eba-426c-83f4-4a44765eb172 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1688 microversion: 2.1 time: 0.161573 Sep 27 11:09:07.472679 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3413/7298] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:07 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1688 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 11:09:08.501094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cbc4ef37-f039-4898-bf70-0676f981ffd1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:08.501766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cbc4ef37-f039-4898-bf70-0676f981ffd1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:08.510564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc4ef37-f039-4898-bf70-0676f981ffd1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:08.511151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc4ef37-f039-4898-bf70-0676f981ffd1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:08.511666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc4ef37-f039-4898-bf70-0676f981ffd1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:08.512495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc4ef37-f039-4898-bf70-0676f981ffd1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:08.513082 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc4ef37-f039-4898-bf70-0676f981ffd1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:08.514149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cbc4ef37-f039-4898-bf70-0676f981ffd1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:08.657408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cbc4ef37-f039-4898-bf70-0676f981ffd1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:08.665422 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cbc4ef37-f039-4898-bf70-0676f981ffd1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1688 microversion: 2.1 time: 0.171661 Sep 27 11:09:08.666050 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3886/7299] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:08 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1688 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:09.626821 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-839bd160-d5f4-47a2-bb30-282d18c6df21 req-b7062374-5646-493c-be07-615dace2931a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ff68f81-0424-4089-9d47-5aab12c8b217", "name": "network-vif-deleted", "tag": "d972135e-ac14-40bd-a0e9-896784309aa0"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:09:09.636334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-839bd160-d5f4-47a2-bb30-282d18c6df21 req-b7062374-5646-493c-be07-615dace2931a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:09.636858 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-839bd160-d5f4-47a2-bb30-282d18c6df21 req-b7062374-5646-493c-be07-615dace2931a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:09.637302 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-839bd160-d5f4-47a2-bb30-282d18c6df21 req-b7062374-5646-493c-be07-615dace2931a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:09.649126 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-839bd160-d5f4-47a2-bb30-282d18c6df21 req-b7062374-5646-493c-be07-615dace2931a service nova] Creating event network-vif-deleted:d972135e-ac14-40bd-a0e9-896784309aa0 for instance 7ff68f81-0424-4089-9d47-5aab12c8b217 on nch2-master-1 Sep 27 11:09:09.655019 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-839bd160-d5f4-47a2-bb30-282d18c6df21 req-b7062374-5646-493c-be07-615dace2931a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.095052 Sep 27 11:09:09.655871 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3414/7300] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:09:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:09:09.689787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:09.690399 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:09.697731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:09.698225 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:09.698695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:09.699448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:09.699910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:09.700344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:09.734469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:09.734962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:09.735422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:09.735880 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lazy-loading 'fault' on Instance uuid 7ff68f81-0424-4089-9d47-5aab12c8b217 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 11:09:09.778456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:09.786912 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a1242b05-e83a-4e71-816b-718d7d3fbe7f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 200 len: 1356 microversion: 2.1 time: 0.101923 Sep 27 11:09:09.787492 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3887/7301] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:09 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 1356 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:10.818888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:10.819663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 7ff68f81-0424-4089-9d47-5aab12c8b217] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:10.829086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:10.829589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:10.830044 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:10.830766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:10.831241 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:10.831683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:10.845731 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] HTTP exception thrown: Instance 7ff68f81-0424-4089-9d47-5aab12c8b217 could not be found. Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Request method failure captured: Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 HTTP/1.1 Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtfHoJ5jcAxOLtRcoV1nD5XXlTZzK0A-5egTQGd5lTVNdkOuvT5Fs0L9J-caES9KtErWFoSb_09yL5egcLFPiM4N0_ALLienspVX9-DRrIBhK2koE7PdcRR57IH0CHGGILEi4fmfkKfiXRleQCLI0R176myMkB92QmVT86Qtqhsg2-o Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 55b42a416dd942e780e533e4c6f2a8b4 Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_55b42a416dd942e780e533e4c6f2a8b4"}]}] Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 55b42a416dd942e780e533e4c6f2a8b4 Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestNetworkAdvancedServerOps-1003182190-project Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 3ec60ad271544bc99d7f394b9b06dd88 Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1003182190-project Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 7ff68f81-0424-4089-9d47-5aab12c8b217 could not be found. Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '7ff68f81-0424-4089-9d47-5aab12c8b217'} Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7ff68f81-0424-4089-9d47-5aab12c8b217 could not be found. Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7ff68f81-0424-4089-9d47-5aab12c8b217 could not be found. Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:09:10.847398 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:09:10.857089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:09:10.857089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:09:10.857089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:09:10.857089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:09:10.857089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 11:09:10.857089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 11:09:10.857089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7ff68f81-0424-4089-9d47-5aab12c8b217 could not be found. Sep 27 11:09:10.857089 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:09:10.857089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Returning 404 to user: Instance 7ff68f81-0424-4089-9d47-5aab12c8b217 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 11:09:10.857089 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c7335fbf-a220-4e73-9f8e-0450ec9b6fa6 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217" status: 404 len: 111 microversion: 2.1 time: 0.038566 Sep 27 11:09:10.857089 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3415/7302] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:10 2022] GET /compute/v2.1/servers/7ff68f81-0424-4089-9d47-5aab12c8b217 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 11:09:14.526691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9d27960-3f7c-4d86-94b3-6825d94430da tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '.version_select at 0x7f7f63730940>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:14.556541 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9d27960-3f7c-4d86-94b3-6825d94430da tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1700595915" status: 202 len: 0 microversion: 2.1 time: 0.033933 Sep 27 11:09:14.560618 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3888/7303] 10.222.0.26 () {58 vars in 1284 bytes} [Tue Sep 27 11:09:14 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-1700595915 => generated 0 bytes in 36 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:09:14.589784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-50dabbc8-65fb-407b-93df-cf94541b7d8c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'create', calling method: .version_select at 0x7f7f62eb0670>, body: {"keypair": {"name": "tempest-TestNetworkAdvancedServerOps-522270313", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMuzgMqCBfUROQNMjicOrGBtw2Di8mLYO6OjOG4lUYjR7cGebvZWMLHxQTvvnIV9oyZ64gsJvWhVjAgRSn0D+KBMGRxSyeGaHhX+Wa5SzejGo72vx194K8ovIrsIxdFtXQ=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:09:14.607208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-50dabbc8-65fb-407b-93df-cf94541b7d8c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:09:14.613904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-50dabbc8-65fb-407b-93df-cf94541b7d8c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for user 3ec60ad271544bc99d7f394b9b06dd88 and project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:09:14.627574 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [None req-50dabbc8-65fb-407b-93df-cf94541b7d8c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [9107af64-6546-47f5-bc26-9366673aaaa8] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 27 11:09:15.635257 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [None req-50dabbc8-65fb-407b-93df-cf94541b7d8c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [9107af64-6546-47f5-bc26-9366673aaaa8] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 59084. Sep 27 11:09:15.655859 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-50dabbc8-65fb-407b-93df-cf94541b7d8c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 406 microversion: 2.1 time: 1.069891 Sep 27 11:09:15.656846 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3416/7304] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 11:09:14 2022] POST /compute/v2.1/os-keypairs => generated 406 bytes in 1072 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:09:21.062957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:21.068437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestNetworkAdvancedServerOps-server-1740918681", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "networks": [{"uuid": "faf4fbbb-fd4f-459f-8a63-f5196801c220"}], "key_name": "tempest-TestNetworkAdvancedServerOps-522270313", "security_groups": [{"name": "tempest-secgroup-smoke-1319081172"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:09:21.199037 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] validate_networks() for [('faf4fbbb-fd4f-459f-8a63-f5196801c220', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 11:09:21.794282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 11:09:21.794282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 11:09:21.794608 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 11:09:21.794908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 11:09:21.795530 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 11:09:21.812327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:21.813641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:21.814321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:21.815297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:21.816419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:21.816949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:21.839201 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:09:21.848825 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Getting quotas for user 3ec60ad271544bc99d7f394b9b06dd88 and project 55b42a416dd942e780e533e4c6f2a8b4. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:09:21.868552 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 11:09:21.871572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 11:09:21.948507 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-560f7799-43be-4695-86df-0a7239a4dcb0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.887400 Sep 27 11:09:21.962935 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=12 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:21.967823 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-93e05bbe-a3fb-47b9-8923-e81eec332cbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:21.969473 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-93e05bbe-a3fb-47b9-8923-e81eec332cbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:21.970047 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3889/7305] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 11:09:21 2022] POST /compute/v2.1/servers => generated 406 bytes in 910 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 11:09:22.087348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-93e05bbe-a3fb-47b9-8923-e81eec332cbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:22.104617 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-93e05bbe-a3fb-47b9-8923-e81eec332cbe tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1266 microversion: 2.1 time: 0.139694 Sep 27 11:09:22.105534 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3417/7306] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:21 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1266 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:23.128837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c7f36071-51f3-4db4-b798-ff8ec5f830c0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:23.129484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c7f36071-51f3-4db4-b798-ff8ec5f830c0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:23.136822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7f36071-51f3-4db4-b798-ff8ec5f830c0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:23.137316 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7f36071-51f3-4db4-b798-ff8ec5f830c0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:23.137767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7f36071-51f3-4db4-b798-ff8ec5f830c0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:23.138482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7f36071-51f3-4db4-b798-ff8ec5f830c0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:23.138943 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7f36071-51f3-4db4-b798-ff8ec5f830c0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:23.139495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c7f36071-51f3-4db4-b798-ff8ec5f830c0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:23.215453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c7f36071-51f3-4db4-b798-ff8ec5f830c0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:23.224795 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c7f36071-51f3-4db4-b798-ff8ec5f830c0 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1318 microversion: 2.1 time: 0.101018 Sep 27 11:09:23.226298 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3890/7307] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:23 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1318 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:24.248695 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-24fd2639-c0f1-4b4c-be91-bf93af26f9e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:24.249471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-24fd2639-c0f1-4b4c-be91-bf93af26f9e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:24.257494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24fd2639-c0f1-4b4c-be91-bf93af26f9e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:24.258090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24fd2639-c0f1-4b4c-be91-bf93af26f9e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:24.258616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24fd2639-c0f1-4b4c-be91-bf93af26f9e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:24.259525 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24fd2639-c0f1-4b4c-be91-bf93af26f9e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:24.260081 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24fd2639-c0f1-4b4c-be91-bf93af26f9e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:24.260611 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-24fd2639-c0f1-4b4c-be91-bf93af26f9e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:24.443170 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-24fd2639-c0f1-4b4c-be91-bf93af26f9e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:24.452551 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-24fd2639-c0f1-4b4c-be91-bf93af26f9e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1392 microversion: 2.1 time: 0.207770 Sep 27 11:09:24.453355 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3418/7308] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:24 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1392 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:25.146094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e0db3188-3a4b-45f8-a298-cfdef3b09827 req-5ebda8ae-d3cd-43e7-a38c-62ea115b37bc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "58ab3107-040d-4c85-a772-619ad5864aad", "tag": "124e2529-94d2-4378-b992-dcdc9c386ba9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:09:25.158807 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e0db3188-3a4b-45f8-a298-cfdef3b09827 req-5ebda8ae-d3cd-43e7-a38c-62ea115b37bc service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:25.159285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e0db3188-3a4b-45f8-a298-cfdef3b09827 req-5ebda8ae-d3cd-43e7-a38c-62ea115b37bc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:25.159714 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e0db3188-3a4b-45f8-a298-cfdef3b09827 req-5ebda8ae-d3cd-43e7-a38c-62ea115b37bc service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:25.182258 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e0db3188-3a4b-45f8-a298-cfdef3b09827 req-5ebda8ae-d3cd-43e7-a38c-62ea115b37bc service nova] Creating event network-changed:124e2529-94d2-4378-b992-dcdc9c386ba9 for instance 58ab3107-040d-4c85-a772-619ad5864aad on nch2-master-1 Sep 27 11:09:25.187967 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e0db3188-3a4b-45f8-a298-cfdef3b09827 req-5ebda8ae-d3cd-43e7-a38c-62ea115b37bc service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045470 Sep 27 11:09:25.188594 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3891/7309] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:09:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:09:25.477753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-79ca7bd5-db94-4ed4-b59c-dd70428f2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:25.478376 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-79ca7bd5-db94-4ed4-b59c-dd70428f2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:25.485754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79ca7bd5-db94-4ed4-b59c-dd70428f2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:25.486530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79ca7bd5-db94-4ed4-b59c-dd70428f2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:25.486994 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79ca7bd5-db94-4ed4-b59c-dd70428f2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:25.487719 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79ca7bd5-db94-4ed4-b59c-dd70428f2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:25.488194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79ca7bd5-db94-4ed4-b59c-dd70428f2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:25.488633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79ca7bd5-db94-4ed4-b59c-dd70428f2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:25.637093 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-79ca7bd5-db94-4ed4-b59c-dd70428f2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:25.646198 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-79ca7bd5-db94-4ed4-b59c-dd70428f2591 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1392 microversion: 2.1 time: 0.172364 Sep 27 11:09:25.648701 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3419/7310] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:25 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1392 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:26.674760 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69eb247f-0350-47c0-bca2-8e7ed8809f95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:26.675459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69eb247f-0350-47c0-bca2-8e7ed8809f95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:26.682350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69eb247f-0350-47c0-bca2-8e7ed8809f95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:26.682836 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69eb247f-0350-47c0-bca2-8e7ed8809f95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:26.683274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69eb247f-0350-47c0-bca2-8e7ed8809f95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:26.683979 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69eb247f-0350-47c0-bca2-8e7ed8809f95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:26.684436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69eb247f-0350-47c0-bca2-8e7ed8809f95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:26.684910 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69eb247f-0350-47c0-bca2-8e7ed8809f95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:26.827147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-69eb247f-0350-47c0-bca2-8e7ed8809f95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:26.836210 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69eb247f-0350-47c0-bca2-8e7ed8809f95 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1392 microversion: 2.1 time: 0.168581 Sep 27 11:09:26.836839 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3892/7311] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:26 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1392 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:27.860078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-05c4109c-ffdb-49b6-b045-39eec7252e32 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:27.860853 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-05c4109c-ffdb-49b6-b045-39eec7252e32 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:27.868641 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05c4109c-ffdb-49b6-b045-39eec7252e32 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:27.869191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05c4109c-ffdb-49b6-b045-39eec7252e32 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:27.869759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05c4109c-ffdb-49b6-b045-39eec7252e32 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:27.870551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05c4109c-ffdb-49b6-b045-39eec7252e32 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:27.871085 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05c4109c-ffdb-49b6-b045-39eec7252e32 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:27.871589 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05c4109c-ffdb-49b6-b045-39eec7252e32 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:28.020866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-05c4109c-ffdb-49b6-b045-39eec7252e32 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:28.029202 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-05c4109c-ffdb-49b6-b045-39eec7252e32 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1392 microversion: 2.1 time: 0.173175 Sep 27 11:09:28.029797 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3420/7312] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:27 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1392 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:29.053969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-723bee9f-99c7-4363-848d-0cd0d6c638eb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:29.054605 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-723bee9f-99c7-4363-848d-0cd0d6c638eb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:29.062038 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-723bee9f-99c7-4363-848d-0cd0d6c638eb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:29.062537 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-723bee9f-99c7-4363-848d-0cd0d6c638eb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:29.062985 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-723bee9f-99c7-4363-848d-0cd0d6c638eb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:29.063782 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-723bee9f-99c7-4363-848d-0cd0d6c638eb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:29.064296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-723bee9f-99c7-4363-848d-0cd0d6c638eb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:29.064754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-723bee9f-99c7-4363-848d-0cd0d6c638eb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:29.207420 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-723bee9f-99c7-4363-848d-0cd0d6c638eb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:29.219570 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-723bee9f-99c7-4363-848d-0cd0d6c638eb tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1392 microversion: 2.1 time: 0.169283 Sep 27 11:09:29.220858 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3893/7313] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:29 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1392 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:30.246165 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6823849e-e492-43b9-9923-7d7f9b25a4b9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:30.246749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6823849e-e492-43b9-9923-7d7f9b25a4b9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:30.257339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6823849e-e492-43b9-9923-7d7f9b25a4b9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:30.258068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6823849e-e492-43b9-9923-7d7f9b25a4b9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:30.258702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6823849e-e492-43b9-9923-7d7f9b25a4b9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:30.259604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6823849e-e492-43b9-9923-7d7f9b25a4b9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:30.260295 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6823849e-e492-43b9-9923-7d7f9b25a4b9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:30.260922 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6823849e-e492-43b9-9923-7d7f9b25a4b9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:30.400999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6823849e-e492-43b9-9923-7d7f9b25a4b9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:30.409604 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6823849e-e492-43b9-9923-7d7f9b25a4b9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1392 microversion: 2.1 time: 0.169534 Sep 27 11:09:30.410173 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3421/7314] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:30 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1392 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:31.432757 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fa26c63a-909b-4809-9f52-f8dac94a6424 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:31.433151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fa26c63a-909b-4809-9f52-f8dac94a6424 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:31.441379 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa26c63a-909b-4809-9f52-f8dac94a6424 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:31.441903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa26c63a-909b-4809-9f52-f8dac94a6424 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:31.442346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa26c63a-909b-4809-9f52-f8dac94a6424 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:31.443073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa26c63a-909b-4809-9f52-f8dac94a6424 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:31.443542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa26c63a-909b-4809-9f52-f8dac94a6424 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:31.443974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fa26c63a-909b-4809-9f52-f8dac94a6424 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:31.589393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fa26c63a-909b-4809-9f52-f8dac94a6424 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:31.599234 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fa26c63a-909b-4809-9f52-f8dac94a6424 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1392 microversion: 2.1 time: 0.170291 Sep 27 11:09:31.612487 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3894/7315] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:31 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1392 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:31.635163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1cb29fee-2a06-41c0-9dc0-810f6a8cf292 req-b2a4593a-a6e4-4a8a-8795-dc091b6fd2fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "58ab3107-040d-4c85-a772-619ad5864aad", "name": "network-vif-plugged", "status": "completed", "tag": "124e2529-94d2-4378-b992-dcdc9c386ba9"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:09:31.645924 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1cb29fee-2a06-41c0-9dc0-810f6a8cf292 req-b2a4593a-a6e4-4a8a-8795-dc091b6fd2fd service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:31.646799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1cb29fee-2a06-41c0-9dc0-810f6a8cf292 req-b2a4593a-a6e4-4a8a-8795-dc091b6fd2fd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:31.647259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1cb29fee-2a06-41c0-9dc0-810f6a8cf292 req-b2a4593a-a6e4-4a8a-8795-dc091b6fd2fd service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:31.659480 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1cb29fee-2a06-41c0-9dc0-810f6a8cf292 req-b2a4593a-a6e4-4a8a-8795-dc091b6fd2fd service nova] Creating event network-vif-plugged:124e2529-94d2-4378-b992-dcdc9c386ba9 for instance 58ab3107-040d-4c85-a772-619ad5864aad on nch2-master-1 Sep 27 11:09:31.665308 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1cb29fee-2a06-41c0-9dc0-810f6a8cf292 req-b2a4593a-a6e4-4a8a-8795-dc091b6fd2fd service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.092966 Sep 27 11:09:31.665921 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3422/7316] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:09:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:09:32.627189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b7e0011-6cb2-4e21-9d23-1f7fcb4648f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:32.628075 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1b7e0011-6cb2-4e21-9d23-1f7fcb4648f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:32.630081 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:32.630907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:32.637960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b7e0011-6cb2-4e21-9d23-1f7fcb4648f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:32.638445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b7e0011-6cb2-4e21-9d23-1f7fcb4648f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:32.638879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b7e0011-6cb2-4e21-9d23-1f7fcb4648f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:32.639567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b7e0011-6cb2-4e21-9d23-1f7fcb4648f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:32.640024 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b7e0011-6cb2-4e21-9d23-1f7fcb4648f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:32.640447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b7e0011-6cb2-4e21-9d23-1f7fcb4648f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:32.779150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1b7e0011-6cb2-4e21-9d23-1f7fcb4648f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:32.787542 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1b7e0011-6cb2-4e21-9d23-1f7fcb4648f4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1392 microversion: 2.1 time: 0.164453 Sep 27 11:09:32.788099 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3895/7317] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:32 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1392 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:33.807898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:09:33.814512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c14371de-75be-4ad6-abd1-08b0cc06230a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:33.815327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c14371de-75be-4ad6-abd1-08b0cc06230a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:33.822015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c14371de-75be-4ad6-abd1-08b0cc06230a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:33.822532 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c14371de-75be-4ad6-abd1-08b0cc06230a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:33.822981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c14371de-75be-4ad6-abd1-08b0cc06230a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:33.823704 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c14371de-75be-4ad6-abd1-08b0cc06230a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:33.824177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c14371de-75be-4ad6-abd1-08b0cc06230a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:33.824620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c14371de-75be-4ad6-abd1-08b0cc06230a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:33.971893 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c14371de-75be-4ad6-abd1-08b0cc06230a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:33.980566 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c14371de-75be-4ad6-abd1-08b0cc06230a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1564 microversion: 2.1 time: 0.173652 Sep 27 11:09:33.981108 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3423/7318] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:33 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1564 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:34.003408 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f8f1a741-7922-4881-bbf4-dd68aea1a31e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:34.004475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f8f1a741-7922-4881-bbf4-dd68aea1a31e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:34.012080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f1a741-7922-4881-bbf4-dd68aea1a31e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:34.012578 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f1a741-7922-4881-bbf4-dd68aea1a31e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:34.013074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f1a741-7922-4881-bbf4-dd68aea1a31e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:34.013778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f1a741-7922-4881-bbf4-dd68aea1a31e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:34.014248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f1a741-7922-4881-bbf4-dd68aea1a31e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:34.014685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f8f1a741-7922-4881-bbf4-dd68aea1a31e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:34.172865 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f8f1a741-7922-4881-bbf4-dd68aea1a31e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:34.180938 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f8f1a741-7922-4881-bbf4-dd68aea1a31e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1564 microversion: 2.1 time: 0.181446 Sep 27 11:09:34.181507 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3896/7319] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:33 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1564 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:09:35.872506 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58ee1f4c-54d4-4ef5-8d7b-48c225fcefb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:09:35.873175 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-58ee1f4c-54d4-4ef5-8d7b-48c225fcefb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:09:35.874032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-3eac788c-f107-45ed-bf34-63d7784753fd req-c186da19-1380-460d-b07a-2d67c4690fde service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "58ab3107-040d-4c85-a772-619ad5864aad", "tag": "124e2529-94d2-4378-b992-dcdc9c386ba9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:09:35.881588 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58ee1f4c-54d4-4ef5-8d7b-48c225fcefb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:35.882360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58ee1f4c-54d4-4ef5-8d7b-48c225fcefb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:35.883718 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58ee1f4c-54d4-4ef5-8d7b-48c225fcefb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:35.884743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58ee1f4c-54d4-4ef5-8d7b-48c225fcefb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:35.885180 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58ee1f4c-54d4-4ef5-8d7b-48c225fcefb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:35.885696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3eac788c-f107-45ed-bf34-63d7784753fd req-c186da19-1380-460d-b07a-2d67c4690fde service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:09:35.886108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58ee1f4c-54d4-4ef5-8d7b-48c225fcefb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:35.886248 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3eac788c-f107-45ed-bf34-63d7784753fd req-c186da19-1380-460d-b07a-2d67c4690fde service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:09:35.886600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-3eac788c-f107-45ed-bf34-63d7784753fd req-c186da19-1380-460d-b07a-2d67c4690fde service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:09:35.898632 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-3eac788c-f107-45ed-bf34-63d7784753fd req-c186da19-1380-460d-b07a-2d67c4690fde service nova] Creating event network-changed:124e2529-94d2-4378-b992-dcdc9c386ba9 for instance 58ab3107-040d-4c85-a772-619ad5864aad on nch2-master-1 Sep 27 11:09:35.904357 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-3eac788c-f107-45ed-bf34-63d7784753fd req-c186da19-1380-460d-b07a-2d67c4690fde service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033965 Sep 27 11:09:35.905043 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3897/7320] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:09:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 11:09:35.936094 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:09:35.936094 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 11:09:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad (ip 10.222.0.26) !!! Sep 27 11:09:36.033083 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-58ee1f4c-54d4-4ef5-8d7b-48c225fcefb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:09:36.042932 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-58ee1f4c-54d4-4ef5-8d7b-48c225fcefb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1564 microversion: 2.1 time: 0.175539 Sep 27 11:09:36.043904 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3424/7321] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:09:35 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1564 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:10:09.391546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=47,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:09.392311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=42 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:09.396677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-69be9290-3af9-4d9c-b262-09d5b111ed1c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:10:09.398988 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-69be9290-3af9-4d9c-b262-09d5b111ed1c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:09.407258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69be9290-3af9-4d9c-b262-09d5b111ed1c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:09.407744 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69be9290-3af9-4d9c-b262-09d5b111ed1c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:09.408199 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69be9290-3af9-4d9c-b262-09d5b111ed1c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:09.408928 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69be9290-3af9-4d9c-b262-09d5b111ed1c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:09.409371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69be9290-3af9-4d9c-b262-09d5b111ed1c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:09.409790 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-69be9290-3af9-4d9c-b262-09d5b111ed1c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:09.483497 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-69be9290-3af9-4d9c-b262-09d5b111ed1c tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad/action" status: 200 len: 24567 microversion: 2.1 time: 0.093379 Sep 27 11:10:09.484413 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3898/7322] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:10:09 2022] POST /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad/action => generated 24567 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:10:39.891829 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:39.893115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=38 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:40.109576 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-120a85a8-2525-45ab-861a-4c1f987a8cb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"suspend": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:10:40.110154 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-120a85a8-2525-45ab-861a-4c1f987a8cb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:40.117264 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120a85a8-2525-45ab-861a-4c1f987a8cb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:40.117762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120a85a8-2525-45ab-861a-4c1f987a8cb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:40.118226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120a85a8-2525-45ab-861a-4c1f987a8cb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:40.119541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120a85a8-2525-45ab-861a-4c1f987a8cb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:40.119541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120a85a8-2525-45ab-861a-4c1f987a8cb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:40.119888 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-120a85a8-2525-45ab-861a-4c1f987a8cb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:40.144119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-120a85a8-2525-45ab-861a-4c1f987a8cb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lazy-loading 'flavor' on Instance uuid 58ab3107-040d-4c85-a772-619ad5864aad {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 11:10:40.220591 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-120a85a8-2525-45ab-861a-4c1f987a8cb5 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad/action" status: 202 len: 0 microversion: 2.1 time: 0.331825 Sep 27 11:10:40.221504 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3425/7323] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:10:39 2022] POST /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad/action => generated 0 bytes in 334 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:10:40.231086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cf12651a-42e9-4ee6-9a1b-b66b7f4ec4c4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:40.231884 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cf12651a-42e9-4ee6-9a1b-b66b7f4ec4c4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:40.234484 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=4 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:40.235119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:10:40.241084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf12651a-42e9-4ee6-9a1b-b66b7f4ec4c4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:40.242343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf12651a-42e9-4ee6-9a1b-b66b7f4ec4c4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:40.242775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf12651a-42e9-4ee6-9a1b-b66b7f4ec4c4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:40.243968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf12651a-42e9-4ee6-9a1b-b66b7f4ec4c4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:40.244462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf12651a-42e9-4ee6-9a1b-b66b7f4ec4c4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:40.244918 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cf12651a-42e9-4ee6-9a1b-b66b7f4ec4c4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:40.390297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cf12651a-42e9-4ee6-9a1b-b66b7f4ec4c4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:10:40.399345 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cf12651a-42e9-4ee6-9a1b-b66b7f4ec4c4 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1691 microversion: 2.1 time: 0.172603 Sep 27 11:10:40.399853 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3899/7324] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:10:40 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1691 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:10:41.431084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-468f4b1d-8562-40d1-bfcd-a502f06035a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:41.431817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-468f4b1d-8562-40d1-bfcd-a502f06035a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:41.439616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-468f4b1d-8562-40d1-bfcd-a502f06035a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:41.440123 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-468f4b1d-8562-40d1-bfcd-a502f06035a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:41.440580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-468f4b1d-8562-40d1-bfcd-a502f06035a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:41.441342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-468f4b1d-8562-40d1-bfcd-a502f06035a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:41.441819 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-468f4b1d-8562-40d1-bfcd-a502f06035a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:41.442273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-468f4b1d-8562-40d1-bfcd-a502f06035a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:41.594633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-468f4b1d-8562-40d1-bfcd-a502f06035a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:10:41.604697 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-468f4b1d-8562-40d1-bfcd-a502f06035a7 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1691 microversion: 2.1 time: 0.180559 Sep 27 11:10:41.605536 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3426/7325] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:10:41 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1691 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:10:42.634393 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-675137b3-0d64-40fd-accc-73218653430a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:42.635017 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-675137b3-0d64-40fd-accc-73218653430a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:42.642848 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675137b3-0d64-40fd-accc-73218653430a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:42.643342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675137b3-0d64-40fd-accc-73218653430a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:42.643781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675137b3-0d64-40fd-accc-73218653430a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:42.644481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675137b3-0d64-40fd-accc-73218653430a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:42.644966 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675137b3-0d64-40fd-accc-73218653430a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:42.645392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-675137b3-0d64-40fd-accc-73218653430a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:42.792425 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-675137b3-0d64-40fd-accc-73218653430a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:10:42.803761 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-675137b3-0d64-40fd-accc-73218653430a tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1674 microversion: 2.1 time: 0.174347 Sep 27 11:10:42.804361 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3900/7326] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:10:42 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1674 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:10:43.915474 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6a322efa-4d27-48b0-b6d3-af814700a1f9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:10:43.917869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6a322efa-4d27-48b0-b6d3-af814700a1f9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:43.924465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a322efa-4d27-48b0-b6d3-af814700a1f9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:43.925034 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a322efa-4d27-48b0-b6d3-af814700a1f9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:43.925489 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a322efa-4d27-48b0-b6d3-af814700a1f9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:43.926220 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a322efa-4d27-48b0-b6d3-af814700a1f9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:43.926691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a322efa-4d27-48b0-b6d3-af814700a1f9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:43.927125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6a322efa-4d27-48b0-b6d3-af814700a1f9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:43.993116 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6a322efa-4d27-48b0-b6d3-af814700a1f9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad/action" status: 200 len: 36753 microversion: 2.1 time: 0.081768 Sep 27 11:10:43.993793 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3427/7327] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:10:43 2022] POST /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad/action => generated 36753 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:10:44.006339 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eac4fd3d-3f77-43cd-a2ca-e3915a35ccca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"resume": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:10:44.006912 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eac4fd3d-3f77-43cd-a2ca-e3915a35ccca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:44.014545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eac4fd3d-3f77-43cd-a2ca-e3915a35ccca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:44.015036 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eac4fd3d-3f77-43cd-a2ca-e3915a35ccca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:44.015478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eac4fd3d-3f77-43cd-a2ca-e3915a35ccca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:44.016188 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eac4fd3d-3f77-43cd-a2ca-e3915a35ccca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:44.016648 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eac4fd3d-3f77-43cd-a2ca-e3915a35ccca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:44.017129 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eac4fd3d-3f77-43cd-a2ca-e3915a35ccca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:44.089818 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eac4fd3d-3f77-43cd-a2ca-e3915a35ccca tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad/action" status: 202 len: 0 microversion: 2.1 time: 0.087568 Sep 27 11:10:44.090551 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3901/7328] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:10:44 2022] POST /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad/action => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:10:44.101663 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-79c9b95b-4ca9-431c-94d2-473fc39d2f01 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:44.102378 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-79c9b95b-4ca9-431c-94d2-473fc39d2f01 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:44.111068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79c9b95b-4ca9-431c-94d2-473fc39d2f01 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:44.111659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79c9b95b-4ca9-431c-94d2-473fc39d2f01 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:44.112436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79c9b95b-4ca9-431c-94d2-473fc39d2f01 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:44.113199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79c9b95b-4ca9-431c-94d2-473fc39d2f01 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:44.113681 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79c9b95b-4ca9-431c-94d2-473fc39d2f01 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:44.114124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-79c9b95b-4ca9-431c-94d2-473fc39d2f01 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:44.260702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-79c9b95b-4ca9-431c-94d2-473fc39d2f01 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:10:44.268770 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-79c9b95b-4ca9-431c-94d2-473fc39d2f01 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1680 microversion: 2.1 time: 0.172145 Sep 27 11:10:44.269617 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3428/7329] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:10:44 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1680 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:10:45.299495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6bf74e68-0db3-4676-a6ee-540a66101257 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:45.300699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6bf74e68-0db3-4676-a6ee-540a66101257 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:45.309797 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf74e68-0db3-4676-a6ee-540a66101257 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:45.310480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf74e68-0db3-4676-a6ee-540a66101257 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:45.310989 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf74e68-0db3-4676-a6ee-540a66101257 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:45.311777 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf74e68-0db3-4676-a6ee-540a66101257 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:45.312483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf74e68-0db3-4676-a6ee-540a66101257 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:45.313194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6bf74e68-0db3-4676-a6ee-540a66101257 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:45.462502 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6bf74e68-0db3-4676-a6ee-540a66101257 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:10:45.470711 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6bf74e68-0db3-4676-a6ee-540a66101257 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1680 microversion: 2.1 time: 0.179853 Sep 27 11:10:45.473376 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3902/7330] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:10:45 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1680 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:10:46.505480 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fac38a6e-913b-45cb-9e19-d1b2f20dc851 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:46.506828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fac38a6e-913b-45cb-9e19-d1b2f20dc851 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:46.514219 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fac38a6e-913b-45cb-9e19-d1b2f20dc851 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:46.514721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fac38a6e-913b-45cb-9e19-d1b2f20dc851 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:46.515178 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fac38a6e-913b-45cb-9e19-d1b2f20dc851 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:46.515897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fac38a6e-913b-45cb-9e19-d1b2f20dc851 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:46.516493 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fac38a6e-913b-45cb-9e19-d1b2f20dc851 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:46.522574 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fac38a6e-913b-45cb-9e19-d1b2f20dc851 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:46.663826 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fac38a6e-913b-45cb-9e19-d1b2f20dc851 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:10:46.671939 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fac38a6e-913b-45cb-9e19-d1b2f20dc851 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1683 microversion: 2.1 time: 0.173273 Sep 27 11:10:46.672522 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3429/7331] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:10:46 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1683 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:10:46.784186 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ddf4c044-8965-4acd-b89c-8310ad59bca9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:10:46.786475 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ddf4c044-8965-4acd-b89c-8310ad59bca9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:46.794007 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddf4c044-8965-4acd-b89c-8310ad59bca9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:46.794491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddf4c044-8965-4acd-b89c-8310ad59bca9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:46.794932 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddf4c044-8965-4acd-b89c-8310ad59bca9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:46.795635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddf4c044-8965-4acd-b89c-8310ad59bca9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:46.796091 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddf4c044-8965-4acd-b89c-8310ad59bca9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:46.796520 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ddf4c044-8965-4acd-b89c-8310ad59bca9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:46.858989 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ddf4c044-8965-4acd-b89c-8310ad59bca9 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "POST /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad/action" status: 200 len: 36753 microversion: 2.1 time: 0.078677 Sep 27 11:10:46.859625 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3903/7332] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:10:46 2022] POST /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad/action => generated 36753 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 27 11:10:47.945758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bc250c7e-9528-4579-8491-bb0e6eff39b2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:47.946287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bc250c7e-9528-4579-8491-bb0e6eff39b2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:47.954302 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ccd150b3-8218-4600-9409-d5f6ab2ef507 req-9c19268c-fe20-4ef3-8abb-6f51f804816e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "58ab3107-040d-4c85-a772-619ad5864aad", "tag": "124e2529-94d2-4378-b992-dcdc9c386ba9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:10:47.967735 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ccd150b3-8218-4600-9409-d5f6ab2ef507 req-9c19268c-fe20-4ef3-8abb-6f51f804816e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:47.968381 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ccd150b3-8218-4600-9409-d5f6ab2ef507 req-9c19268c-fe20-4ef3-8abb-6f51f804816e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:47.969100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ccd150b3-8218-4600-9409-d5f6ab2ef507 req-9c19268c-fe20-4ef3-8abb-6f51f804816e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:47.971776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc250c7e-9528-4579-8491-bb0e6eff39b2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:47.972323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc250c7e-9528-4579-8491-bb0e6eff39b2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:47.973147 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc250c7e-9528-4579-8491-bb0e6eff39b2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:47.973912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc250c7e-9528-4579-8491-bb0e6eff39b2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:47.974407 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc250c7e-9528-4579-8491-bb0e6eff39b2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:47.974818 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bc250c7e-9528-4579-8491-bb0e6eff39b2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:47.984002 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ccd150b3-8218-4600-9409-d5f6ab2ef507 req-9c19268c-fe20-4ef3-8abb-6f51f804816e service nova] Creating event network-changed:124e2529-94d2-4378-b992-dcdc9c386ba9 for instance 58ab3107-040d-4c85-a772-619ad5864aad on nch2-master-1 Sep 27 11:10:47.989532 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ccd150b3-8218-4600-9409-d5f6ab2ef507 req-9c19268c-fe20-4ef3-8abb-6f51f804816e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039156 Sep 27 11:10:47.990398 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3904/7333] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:10:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:10:48.005817 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bc250c7e-9528-4579-8491-bb0e6eff39b2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 11:10:48.101317 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bc250c7e-9528-4579-8491-bb0e6eff39b2 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "DELETE /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 204 len: 0 microversion: 2.1 time: 0.159935 Sep 27 11:10:48.102308 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3430/7334] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 11:10:47 2022] DELETE /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 0 bytes in 162 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 11:10:48.112864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0fc19d0f-c15d-405b-93de-bbed4509a61e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:48.113717 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0fc19d0f-c15d-405b-93de-bbed4509a61e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:48.121691 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fc19d0f-c15d-405b-93de-bbed4509a61e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:48.122289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fc19d0f-c15d-405b-93de-bbed4509a61e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:48.122829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fc19d0f-c15d-405b-93de-bbed4509a61e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:48.123626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fc19d0f-c15d-405b-93de-bbed4509a61e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:48.124282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fc19d0f-c15d-405b-93de-bbed4509a61e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:48.124736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0fc19d0f-c15d-405b-93de-bbed4509a61e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:48.268977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-0fc19d0f-c15d-405b-93de-bbed4509a61e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:10:48.277908 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0fc19d0f-c15d-405b-93de-bbed4509a61e tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1689 microversion: 2.1 time: 0.170340 Sep 27 11:10:48.278495 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3905/7335] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:10:48 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1689 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:10:49.309168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-045959bb-ee31-4ba4-bfc7-15a02459c728 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:49.310365 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-045959bb-ee31-4ba4-bfc7-15a02459c728 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:49.322458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045959bb-ee31-4ba4-bfc7-15a02459c728 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:49.323372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045959bb-ee31-4ba4-bfc7-15a02459c728 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:49.324251 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045959bb-ee31-4ba4-bfc7-15a02459c728 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:49.325167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045959bb-ee31-4ba4-bfc7-15a02459c728 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:49.325507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045959bb-ee31-4ba4-bfc7-15a02459c728 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:49.325943 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-045959bb-ee31-4ba4-bfc7-15a02459c728 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:49.468334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-045959bb-ee31-4ba4-bfc7-15a02459c728 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:10:49.476841 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-045959bb-ee31-4ba4-bfc7-15a02459c728 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1689 microversion: 2.1 time: 0.175336 Sep 27 11:10:49.477436 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3431/7336] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:10:49 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1689 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:10:50.406163 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-67e6c70c-7b12-4cb1-a94b-53bd8c4700ae req-04d6bc98-70ec-49b5-8ba0-6e45082014c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "58ab3107-040d-4c85-a772-619ad5864aad", "name": "network-vif-deleted", "tag": "124e2529-94d2-4378-b992-dcdc9c386ba9"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:10:50.419361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-67e6c70c-7b12-4cb1-a94b-53bd8c4700ae req-04d6bc98-70ec-49b5-8ba0-6e45082014c2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:50.420012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-67e6c70c-7b12-4cb1-a94b-53bd8c4700ae req-04d6bc98-70ec-49b5-8ba0-6e45082014c2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:50.420765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-67e6c70c-7b12-4cb1-a94b-53bd8c4700ae req-04d6bc98-70ec-49b5-8ba0-6e45082014c2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:50.433822 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-67e6c70c-7b12-4cb1-a94b-53bd8c4700ae req-04d6bc98-70ec-49b5-8ba0-6e45082014c2 service nova] Creating event network-vif-deleted:124e2529-94d2-4378-b992-dcdc9c386ba9 for instance 58ab3107-040d-4c85-a772-619ad5864aad on nch2-master-1 Sep 27 11:10:50.439280 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-67e6c70c-7b12-4cb1-a94b-53bd8c4700ae req-04d6bc98-70ec-49b5-8ba0-6e45082014c2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037154 Sep 27 11:10:50.439902 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3906/7337] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:10:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:10:50.500986 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:50.501631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:50.509799 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:50.510331 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:50.510815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:50.511721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:50.512223 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:50.512685 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:50.546230 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:50.546737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:50.547195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:50.547696 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lazy-loading 'fault' on Instance uuid 58ab3107-040d-4c85-a772-619ad5864aad {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 11:10:50.595810 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ec60ad271544bc99d7f394b9b06dd88', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55b42a416dd942e780e533e4c6f2a8b4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:10:50.604314 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e58025a4-bdd0-4bcc-955f-c511f6e901cc tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 200 len: 1356 microversion: 2.1 time: 0.107332 Sep 27 11:10:50.604998 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3432/7338] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:10:50 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 1356 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:10:51.633187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:51.633890 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] [instance: 58ab3107-040d-4c85-a772-619ad5864aad] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:10:51.640428 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:51.641078 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:51.641494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:51.642483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:10:51.643229 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:10:51.643967 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:10:51.662013 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] HTTP exception thrown: Instance 58ab3107-040d-4c85-a772-619ad5864aad could not be found. Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Request method failure captured: Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad HTTP/1.1 Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtfHoJ5jcAxOLtRcoV1nD5XXlTZzK0A-5egTQGd5lTVNdkOuvT5Fs0L9J-caES9KtErWFoSb_09yL5egcLFPiM4N0_ALLienspVX9-DRrIBhK2koE7PdcRR57IH0CHGGILEi4fmfkKfiXRleQCLI0R176myMkB92QmVT86Qtqhsg2-o Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 55b42a416dd942e780e533e4c6f2a8b4 Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/55b42a416dd942e780e533e4c6f2a8b4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_55b42a416dd942e780e533e4c6f2a8b4"}]}] Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 55b42a416dd942e780e533e4c6f2a8b4 Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestNetworkAdvancedServerOps-1003182190 Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestNetworkAdvancedServerOps-1003182190-project Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 3ec60ad271544bc99d7f394b9b06dd88 Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestNetworkAdvancedServerOps-1003182190-project Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 58ab3107-040d-4c85-a772-619ad5864aad could not be found. Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '58ab3107-040d-4c85-a772-619ad5864aad'} Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 58ab3107-040d-4c85-a772-619ad5864aad could not be found. Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 58ab3107-040d-4c85-a772-619ad5864aad could not be found. Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:10:51.663801 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:10:51.669391 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:10:51.669391 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:10:51.669391 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:10:51.669391 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:10:51.669391 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 11:10:51.669391 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 11:10:51.669391 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 58ab3107-040d-4c85-a772-619ad5864aad could not be found. Sep 27 11:10:51.669391 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:10:51.669391 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Returning 404 to user: Instance 58ab3107-040d-4c85-a772-619ad5864aad could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 11:10:51.669391 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b0b4e657-af7e-486c-bef6-40a2060cf4e1 tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad" status: 404 len: 111 microversion: 2.1 time: 0.039476 Sep 27 11:10:51.669391 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3907/7339] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:10:51 2022] GET /compute/v2.1/servers/58ab3107-040d-4c85-a772-619ad5864aad => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 11:10:55.318078 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-097f4c25-4443-4473-8ee9-a1d19ca8320f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] Calling method '.version_select at 0x7f7f62acf1f0>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:10:55.341869 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-097f4c25-4443-4473-8ee9-a1d19ca8320f tempest-TestNetworkAdvancedServerOps-1003182190 tempest-TestNetworkAdvancedServerOps-1003182190-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-522270313" status: 202 len: 0 microversion: 2.1 time: 0.027769 Sep 27 11:10:55.347491 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3433/7340] 10.222.0.26 () {58 vars in 1281 bytes} [Tue Sep 27 11:10:55 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestNetworkAdvancedServerOps-522270313 => generated 0 bytes in 29 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 27 11:11:07.027135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=18 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:07.029180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=44,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:07.239668 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0bc66ee1-8d78-41b9-8745-a1ad8eac2af1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Action: 'create', calling method: .version_select at 0x7f7f6c17dee0>, body: {"keypair": {"name": "tempest-TestShelveInstance-2001153803", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBINQYWe2aIX5/UrUrgVd5LEoBJrCfJ63gr+Rva/lf6Qo6Tqnzza0jJhxneiL9oXEbh7vdsJSTbzc+QHC8OZstLsdIQOmRAgRbYetFl6ntUQw4kzMq7M4t+f/VD8/h+9qXw=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:11:07.256677 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0bc66ee1-8d78-41b9-8745-a1ad8eac2af1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Getting quotas for project 37db46b0eb6a4c7d938085b26d1968a7. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:11:07.262210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-0bc66ee1-8d78-41b9-8745-a1ad8eac2af1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Getting quotas for user 90cea8d7a5494c45ba299df59797633f and project 37db46b0eb6a4c7d938085b26d1968a7. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:11:07.291266 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0bc66ee1-8d78-41b9-8745-a1ad8eac2af1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.270060 Sep 27 11:11:07.291984 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3908/7341] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 11:11:07 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:11:08.359794 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=63,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:08.361965 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:08.366857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1451182455", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-2001153803", "security_groups": [{"name": "tempest-secgroup-smoke-1607829269"}], "networks": [{"uuid": "b6772c8e-2375-4108-a69f-0a6f573b8005"}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:11:08.485553 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] validate_networks() for [('b6772c8e-2375-4108-a69f-0a6f573b8005', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 11:11:08.988492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 11:11:08.989022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 11:11:08.989526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 11:11:08.989956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 11:11:08.990492 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 11:11:08.999586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:09.000091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:09.000551 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:09.001369 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:09.001876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:09.002318 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:09.020917 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Getting quotas for project 37db46b0eb6a4c7d938085b26d1968a7. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:11:09.026969 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Getting quotas for user 90cea8d7a5494c45ba299df59797633f and project 37db46b0eb6a4c7d938085b26d1968a7. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:11:09.040059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 11:11:09.042468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 11:11:09.091299 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4f74c6ba-4648-4d98-9c16-01b0383b789f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.734165 Sep 27 11:11:09.092465 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3434/7342] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 11:11:08 2022] POST /compute/v2.1/servers => generated 406 bytes in 736 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 11:11:09.108221 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b48885ca-adc5-4480-8f48-c1f6a11cfb02 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:11:09.109030 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b48885ca-adc5-4480-8f48-c1f6a11cfb02 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:11:09.177925 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b48885ca-adc5-4480-8f48-c1f6a11cfb02 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:11:09.186416 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b48885ca-adc5-4480-8f48-c1f6a11cfb02 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1247 microversion: 2.1 time: 0.086076 Sep 27 11:11:09.187005 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3909/7343] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:11:09 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1247 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:11:10.215540 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-654b1151-61a6-4722-93b5-cab442d1bc72 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:11:10.216190 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-654b1151-61a6-4722-93b5-cab442d1bc72 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:11:10.239401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-654b1151-61a6-4722-93b5-cab442d1bc72 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:10.239911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-654b1151-61a6-4722-93b5-cab442d1bc72 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:10.240868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-654b1151-61a6-4722-93b5-cab442d1bc72 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:10.241619 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-654b1151-61a6-4722-93b5-cab442d1bc72 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:10.242090 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-654b1151-61a6-4722-93b5-cab442d1bc72 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:10.246174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-654b1151-61a6-4722-93b5-cab442d1bc72 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:10.403787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-654b1151-61a6-4722-93b5-cab442d1bc72 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:11:10.417960 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-654b1151-61a6-4722-93b5-cab442d1bc72 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1307 microversion: 2.1 time: 0.211479 Sep 27 11:11:10.418594 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3435/7344] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:11:10 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1307 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:11:11.442533 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9c547748-a1c9-470a-9365-3f073f1c6725 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:11:11.443178 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9c547748-a1c9-470a-9365-3f073f1c6725 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:11:11.450749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c547748-a1c9-470a-9365-3f073f1c6725 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:11.451247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c547748-a1c9-470a-9365-3f073f1c6725 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:11.451686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c547748-a1c9-470a-9365-3f073f1c6725 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:11.452402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c547748-a1c9-470a-9365-3f073f1c6725 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:11.452894 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c547748-a1c9-470a-9365-3f073f1c6725 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:11.453326 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c547748-a1c9-470a-9365-3f073f1c6725 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:11.612141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9c547748-a1c9-470a-9365-3f073f1c6725 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:11:11.621666 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9c547748-a1c9-470a-9365-3f073f1c6725 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1373 microversion: 2.1 time: 0.183167 Sep 27 11:11:11.622427 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3910/7345] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:11:11 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1373 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:11:11.845344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-94885e7c-a139-4ded-9534-a5673b5cb0b3 req-47de1a4f-a226-4a1f-96ca-42bed2680ec6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ca7c7b0-0978-4023-b97d-2f8fc46639e7", "tag": "b1be14d2-b2c4-4a69-9a91-a4521243aaf5"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:11:11.861988 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-94885e7c-a139-4ded-9534-a5673b5cb0b3 req-47de1a4f-a226-4a1f-96ca-42bed2680ec6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:11.862326 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-94885e7c-a139-4ded-9534-a5673b5cb0b3 req-47de1a4f-a226-4a1f-96ca-42bed2680ec6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:11.863076 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-94885e7c-a139-4ded-9534-a5673b5cb0b3 req-47de1a4f-a226-4a1f-96ca-42bed2680ec6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:11.875865 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-94885e7c-a139-4ded-9534-a5673b5cb0b3 req-47de1a4f-a226-4a1f-96ca-42bed2680ec6 service nova] Creating event network-changed:b1be14d2-b2c4-4a69-9a91-a4521243aaf5 for instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 on nch1-master-1 Sep 27 11:11:11.881298 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-94885e7c-a139-4ded-9534-a5673b5cb0b3 req-47de1a4f-a226-4a1f-96ca-42bed2680ec6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048646 Sep 27 11:11:11.882155 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3436/7346] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:11:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:11:12.645906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3c6cf99e-d572-44ae-93e3-ddf98b916bd8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:11:12.646567 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3c6cf99e-d572-44ae-93e3-ddf98b916bd8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:11:12.654371 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c6cf99e-d572-44ae-93e3-ddf98b916bd8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:12.654850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c6cf99e-d572-44ae-93e3-ddf98b916bd8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:12.655289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c6cf99e-d572-44ae-93e3-ddf98b916bd8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:12.655987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c6cf99e-d572-44ae-93e3-ddf98b916bd8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:12.656445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c6cf99e-d572-44ae-93e3-ddf98b916bd8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:12.656907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3c6cf99e-d572-44ae-93e3-ddf98b916bd8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:12.792409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3c6cf99e-d572-44ae-93e3-ddf98b916bd8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:11:12.801174 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3c6cf99e-d572-44ae-93e3-ddf98b916bd8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1373 microversion: 2.1 time: 0.159143 Sep 27 11:11:12.801756 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3911/7347] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:11:12 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1373 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:11:13.833361 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ffe7d85f-38a2-4f37-93ad-4511e1581fc2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:11:13.834465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ffe7d85f-38a2-4f37-93ad-4511e1581fc2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:11:13.844778 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffe7d85f-38a2-4f37-93ad-4511e1581fc2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:13.845592 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffe7d85f-38a2-4f37-93ad-4511e1581fc2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:13.846329 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffe7d85f-38a2-4f37-93ad-4511e1581fc2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:13.847586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffe7d85f-38a2-4f37-93ad-4511e1581fc2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:13.848194 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffe7d85f-38a2-4f37-93ad-4511e1581fc2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:13.848966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ffe7d85f-38a2-4f37-93ad-4511e1581fc2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:13.999814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ffe7d85f-38a2-4f37-93ad-4511e1581fc2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:11:14.008021 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ffe7d85f-38a2-4f37-93ad-4511e1581fc2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1373 microversion: 2.1 time: 0.183002 Sep 27 11:11:14.008611 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3437/7348] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:11:13 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1373 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:11:15.038164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-09696525-a9c6-4f88-8191-fb6d06e10532 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:11:15.039053 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-09696525-a9c6-4f88-8191-fb6d06e10532 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:11:15.050022 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09696525-a9c6-4f88-8191-fb6d06e10532 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:15.050732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09696525-a9c6-4f88-8191-fb6d06e10532 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:15.051376 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09696525-a9c6-4f88-8191-fb6d06e10532 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:15.052375 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09696525-a9c6-4f88-8191-fb6d06e10532 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:15.053172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09696525-a9c6-4f88-8191-fb6d06e10532 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:15.053812 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-09696525-a9c6-4f88-8191-fb6d06e10532 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:15.191933 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-09696525-a9c6-4f88-8191-fb6d06e10532 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:11:15.200464 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-09696525-a9c6-4f88-8191-fb6d06e10532 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1373 microversion: 2.1 time: 0.168258 Sep 27 11:11:15.201067 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3912/7349] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:11:15 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1373 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:11:16.224027 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b54d21f6-29e8-4ae4-a04b-9388dabcba81 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:11:16.224684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b54d21f6-29e8-4ae4-a04b-9388dabcba81 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:11:16.233310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54d21f6-29e8-4ae4-a04b-9388dabcba81 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:16.233816 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54d21f6-29e8-4ae4-a04b-9388dabcba81 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:16.234274 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54d21f6-29e8-4ae4-a04b-9388dabcba81 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:16.234999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54d21f6-29e8-4ae4-a04b-9388dabcba81 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:16.235476 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54d21f6-29e8-4ae4-a04b-9388dabcba81 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:16.235923 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b54d21f6-29e8-4ae4-a04b-9388dabcba81 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:16.376101 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b54d21f6-29e8-4ae4-a04b-9388dabcba81 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:11:16.384464 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b54d21f6-29e8-4ae4-a04b-9388dabcba81 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1373 microversion: 2.1 time: 0.164596 Sep 27 11:11:16.385096 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3438/7350] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:11:16 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1373 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:11:17.417811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-39b9d20f-b3c4-409a-a121-57e27df766b5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:11:17.418736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-39b9d20f-b3c4-409a-a121-57e27df766b5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:11:17.430505 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39b9d20f-b3c4-409a-a121-57e27df766b5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:17.431003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39b9d20f-b3c4-409a-a121-57e27df766b5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:17.431448 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39b9d20f-b3c4-409a-a121-57e27df766b5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:17.432157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39b9d20f-b3c4-409a-a121-57e27df766b5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:17.432623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39b9d20f-b3c4-409a-a121-57e27df766b5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:17.433088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-39b9d20f-b3c4-409a-a121-57e27df766b5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:17.572444 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-39b9d20f-b3c4-409a-a121-57e27df766b5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:11:17.580789 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-39b9d20f-b3c4-409a-a121-57e27df766b5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1373 microversion: 2.1 time: 0.170299 Sep 27 11:11:17.581388 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3913/7351] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:11:17 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1373 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:11:18.114344 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-de79c445-6cc5-4c86-ac1d-1c8007d02b38 req-cff47af4-614c-49ba-8c87-a7714bd3bc17 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ca7c7b0-0978-4023-b97d-2f8fc46639e7", "name": "network-vif-plugged", "status": "completed", "tag": "b1be14d2-b2c4-4a69-9a91-a4521243aaf5"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:11:18.127333 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-de79c445-6cc5-4c86-ac1d-1c8007d02b38 req-cff47af4-614c-49ba-8c87-a7714bd3bc17 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:18.127845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-de79c445-6cc5-4c86-ac1d-1c8007d02b38 req-cff47af4-614c-49ba-8c87-a7714bd3bc17 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:18.128640 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-de79c445-6cc5-4c86-ac1d-1c8007d02b38 req-cff47af4-614c-49ba-8c87-a7714bd3bc17 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:18.140874 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-de79c445-6cc5-4c86-ac1d-1c8007d02b38 req-cff47af4-614c-49ba-8c87-a7714bd3bc17 service nova] Creating event network-vif-plugged:b1be14d2-b2c4-4a69-9a91-a4521243aaf5 for instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 on nch1-master-1 Sep 27 11:11:18.146249 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-de79c445-6cc5-4c86-ac1d-1c8007d02b38 req-cff47af4-614c-49ba-8c87-a7714bd3bc17 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035832 Sep 27 11:11:18.147174 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3439/7352] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:11:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:11:18.604329 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4b2d5f4d-e2a9-427c-903d-34bf01893222 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:11:18.604987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4b2d5f4d-e2a9-427c-903d-34bf01893222 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:11:18.612437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2d5f4d-e2a9-427c-903d-34bf01893222 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:18.612971 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2d5f4d-e2a9-427c-903d-34bf01893222 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:18.613417 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2d5f4d-e2a9-427c-903d-34bf01893222 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:18.614136 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2d5f4d-e2a9-427c-903d-34bf01893222 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:18.614625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2d5f4d-e2a9-427c-903d-34bf01893222 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:18.615061 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4b2d5f4d-e2a9-427c-903d-34bf01893222 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:18.755382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4b2d5f4d-e2a9-427c-903d-34bf01893222 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:11:18.763962 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4b2d5f4d-e2a9-427c-903d-34bf01893222 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1373 microversion: 2.1 time: 0.163710 Sep 27 11:11:18.764541 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3914/7353] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:11:18 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1373 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:11:19.801342 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d7a156b1-a607-4525-a1d5-bc584c641cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:11:19.802549 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d7a156b1-a607-4525-a1d5-bc584c641cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:11:19.805580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:19.806283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:19.813526 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a156b1-a607-4525-a1d5-bc584c641cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:19.814042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a156b1-a607-4525-a1d5-bc584c641cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:19.814503 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a156b1-a607-4525-a1d5-bc584c641cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:19.815283 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a156b1-a607-4525-a1d5-bc584c641cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:19.815762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a156b1-a607-4525-a1d5-bc584c641cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:19.816212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d7a156b1-a607-4525-a1d5-bc584c641cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:19.987438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d7a156b1-a607-4525-a1d5-bc584c641cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:11:19.995695 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d7a156b1-a607-4525-a1d5-bc584c641cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1557 microversion: 2.1 time: 0.201763 Sep 27 11:11:19.996574 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3440/7354] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:11:19 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1557 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:11:20.026092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1b0f5e25-1bcf-4229-9558-7c11856c1fb4 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:11:20.026909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1b0f5e25-1bcf-4229-9558-7c11856c1fb4 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:11:20.029600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:11:20.039060 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b0f5e25-1bcf-4229-9558-7c11856c1fb4 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:20.039853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b0f5e25-1bcf-4229-9558-7c11856c1fb4 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:20.040310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b0f5e25-1bcf-4229-9558-7c11856c1fb4 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:20.041048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b0f5e25-1bcf-4229-9558-7c11856c1fb4 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:20.041517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b0f5e25-1bcf-4229-9558-7c11856c1fb4 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:20.041951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1b0f5e25-1bcf-4229-9558-7c11856c1fb4 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:20.187879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1b0f5e25-1bcf-4229-9558-7c11856c1fb4 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:11:20.199349 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1b0f5e25-1bcf-4229-9558-7c11856c1fb4 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1557 microversion: 2.1 time: 0.180117 Sep 27 11:11:20.200431 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3915/7355] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:11:20 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1557 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:11:22.595834 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-c2c12da1-d2e9-479f-9be4-00d1e95c2531 req-a19b04f2-593b-46a4-9396-8633892b8fd0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ca7c7b0-0978-4023-b97d-2f8fc46639e7", "tag": "b1be14d2-b2c4-4a69-9a91-a4521243aaf5"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:11:22.607880 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c2c12da1-d2e9-479f-9be4-00d1e95c2531 req-a19b04f2-593b-46a4-9396-8633892b8fd0 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:11:22.608255 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c2c12da1-d2e9-479f-9be4-00d1e95c2531 req-a19b04f2-593b-46a4-9396-8633892b8fd0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:11:22.608730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-c2c12da1-d2e9-479f-9be4-00d1e95c2531 req-a19b04f2-593b-46a4-9396-8633892b8fd0 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:11:22.624275 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-c2c12da1-d2e9-479f-9be4-00d1e95c2531 req-a19b04f2-593b-46a4-9396-8633892b8fd0 service nova] Creating event network-changed:b1be14d2-b2c4-4a69-9a91-a4521243aaf5 for instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 on nch1-master-1 Sep 27 11:11:22.629907 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-c2c12da1-d2e9-479f-9be4-00d1e95c2531 req-a19b04f2-593b-46a4-9396-8633892b8fd0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037704 Sep 27 11:11:22.630683 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3441/7356] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:11:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:13:17.711725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=27,nova_api:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:17.712642 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=36 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:17.713795 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:13:17.713945 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 11:13:17 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7/action (ip 10.222.0.26) !!! Sep 27 11:13:17.719065 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4439c9b4-9410-4fb0-8bb2-cd7c0199647e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:13:17.719618 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4439c9b4-9410-4fb0-8bb2-cd7c0199647e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:17.728602 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4439c9b4-9410-4fb0-8bb2-cd7c0199647e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:17.729118 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4439c9b4-9410-4fb0-8bb2-cd7c0199647e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:17.729553 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4439c9b4-9410-4fb0-8bb2-cd7c0199647e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:17.730237 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4439c9b4-9410-4fb0-8bb2-cd7c0199647e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:17.730689 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4439c9b4-9410-4fb0-8bb2-cd7c0199647e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:17.731109 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4439c9b4-9410-4fb0-8bb2-cd7c0199647e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:17.755850 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-4439c9b4-9410-4fb0-8bb2-cd7c0199647e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lazy-loading 'flavor' on Instance uuid 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 11:13:18.128221 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4439c9b4-9410-4fb0-8bb2-cd7c0199647e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "POST /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7/action" status: 202 len: 0 microversion: 2.1 time: 0.418133 Sep 27 11:13:18.128867 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3916/7357] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:13:17 2022] POST /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7/action => generated 0 bytes in 420 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:13:18.136277 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [-] [d9169ee3-829d-41de-8bb4-82814efca059] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 27 11:13:18.137228 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [-] [1986a9fd-1b4d-44f5-a8ac-515eb5bb435f] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 27 11:13:18.138279 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=29,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:18.139269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=32 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:18.140341 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:13:18.140470 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 11:13:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 (ip 10.222.0.26) !!! Sep 27 11:13:18.141883 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Too many heartbeats missed Sep 27 11:13:18.147887 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-02967beb-0238-4b63-8cc6-f23026260cf5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:18.149179 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-02967beb-0238-4b63-8cc6-f23026260cf5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:18.160086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02967beb-0238-4b63-8cc6-f23026260cf5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:18.160918 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02967beb-0238-4b63-8cc6-f23026260cf5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:18.161377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02967beb-0238-4b63-8cc6-f23026260cf5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:18.162229 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02967beb-0238-4b63-8cc6-f23026260cf5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:18.162701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02967beb-0238-4b63-8cc6-f23026260cf5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:18.163145 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-02967beb-0238-4b63-8cc6-f23026260cf5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:18.360721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-02967beb-0238-4b63-8cc6-f23026260cf5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:18.369362 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-02967beb-0238-4b63-8cc6-f23026260cf5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1681 microversion: 2.1 time: 0.235418 Sep 27 11:13:18.369852 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3442/7358] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:18 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1681 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:19.400306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6750ded8-a286-4c56-952c-48999b9255fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:19.401616 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6750ded8-a286-4c56-952c-48999b9255fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:19.412149 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6750ded8-a286-4c56-952c-48999b9255fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:19.412635 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6750ded8-a286-4c56-952c-48999b9255fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:19.413099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6750ded8-a286-4c56-952c-48999b9255fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:19.413799 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6750ded8-a286-4c56-952c-48999b9255fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:19.414260 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6750ded8-a286-4c56-952c-48999b9255fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:19.414687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6750ded8-a286-4c56-952c-48999b9255fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:19.565432 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6750ded8-a286-4c56-952c-48999b9255fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:19.574188 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6750ded8-a286-4c56-952c-48999b9255fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1681 microversion: 2.1 time: 0.183468 Sep 27 11:13:19.574766 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3917/7359] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:19 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1681 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:20.602157 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-bdba8ebc-a718-4edb-bcb5-1d9d3fdfddcf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:20.603211 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-bdba8ebc-a718-4edb-bcb5-1d9d3fdfddcf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:20.616339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdba8ebc-a718-4edb-bcb5-1d9d3fdfddcf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:20.616909 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdba8ebc-a718-4edb-bcb5-1d9d3fdfddcf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:20.617370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdba8ebc-a718-4edb-bcb5-1d9d3fdfddcf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:20.618092 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdba8ebc-a718-4edb-bcb5-1d9d3fdfddcf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:20.618580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdba8ebc-a718-4edb-bcb5-1d9d3fdfddcf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:20.619031 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-bdba8ebc-a718-4edb-bcb5-1d9d3fdfddcf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:20.623972 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] [d9169ee3-829d-41de-8bb4-82814efca059] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 59540. Sep 27 11:13:20.638120 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] [1986a9fd-1b4d-44f5-a8ac-515eb5bb435f] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 59542. Sep 27 11:13:20.762471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-bdba8ebc-a718-4edb-bcb5-1d9d3fdfddcf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:20.773210 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-bdba8ebc-a718-4edb-bcb5-1d9d3fdfddcf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1681 microversion: 2.1 time: 0.177835 Sep 27 11:13:20.774145 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3443/7360] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:20 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1681 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:21.797599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-05ab03b0-86e1-40a5-a21c-a3450ffc4900 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:21.798210 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-05ab03b0-86e1-40a5-a21c-a3450ffc4900 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:21.805978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05ab03b0-86e1-40a5-a21c-a3450ffc4900 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:21.806473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05ab03b0-86e1-40a5-a21c-a3450ffc4900 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:21.806907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05ab03b0-86e1-40a5-a21c-a3450ffc4900 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:21.807653 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05ab03b0-86e1-40a5-a21c-a3450ffc4900 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:21.808130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05ab03b0-86e1-40a5-a21c-a3450ffc4900 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:21.808563 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05ab03b0-86e1-40a5-a21c-a3450ffc4900 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:21.945350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-05ab03b0-86e1-40a5-a21c-a3450ffc4900 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:21.953995 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-05ab03b0-86e1-40a5-a21c-a3450ffc4900 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1681 microversion: 2.1 time: 0.160527 Sep 27 11:13:21.954583 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3918/7361] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:21 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1681 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:22.982573 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-360f7e34-3951-4baf-b648-173bc5946633 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:22.983370 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-360f7e34-3951-4baf-b648-173bc5946633 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:22.992455 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-360f7e34-3951-4baf-b648-173bc5946633 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:22.992989 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-360f7e34-3951-4baf-b648-173bc5946633 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:22.993440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-360f7e34-3951-4baf-b648-173bc5946633 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:22.994164 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-360f7e34-3951-4baf-b648-173bc5946633 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:22.994633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-360f7e34-3951-4baf-b648-173bc5946633 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:22.995087 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-360f7e34-3951-4baf-b648-173bc5946633 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:23.135174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-360f7e34-3951-4baf-b648-173bc5946633 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:23.146776 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-360f7e34-3951-4baf-b648-173bc5946633 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1681 microversion: 2.1 time: 0.170943 Sep 27 11:13:23.147703 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3444/7362] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:22 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1681 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:24.173350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ba3e7242-a79a-4557-9b47-6ce9e2301676 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:24.173982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ba3e7242-a79a-4557-9b47-6ce9e2301676 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:24.181515 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba3e7242-a79a-4557-9b47-6ce9e2301676 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:24.182006 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba3e7242-a79a-4557-9b47-6ce9e2301676 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:24.182443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba3e7242-a79a-4557-9b47-6ce9e2301676 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:24.183153 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba3e7242-a79a-4557-9b47-6ce9e2301676 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:24.183606 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba3e7242-a79a-4557-9b47-6ce9e2301676 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:24.184029 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ba3e7242-a79a-4557-9b47-6ce9e2301676 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:24.339274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ba3e7242-a79a-4557-9b47-6ce9e2301676 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:24.347959 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ba3e7242-a79a-4557-9b47-6ce9e2301676 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1681 microversion: 2.1 time: 0.180161 Sep 27 11:13:24.348687 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3919/7363] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:24 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1681 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:25.376343 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c7541da4-a50d-4e3c-adc1-d077dee314fb tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:25.376980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c7541da4-a50d-4e3c-adc1-d077dee314fb tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:25.385198 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7541da4-a50d-4e3c-adc1-d077dee314fb tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:25.385706 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7541da4-a50d-4e3c-adc1-d077dee314fb tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:25.386167 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7541da4-a50d-4e3c-adc1-d077dee314fb tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:25.386892 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7541da4-a50d-4e3c-adc1-d077dee314fb tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:25.387372 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7541da4-a50d-4e3c-adc1-d077dee314fb tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:25.387848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c7541da4-a50d-4e3c-adc1-d077dee314fb tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:25.537173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c7541da4-a50d-4e3c-adc1-d077dee314fb tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:25.545385 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c7541da4-a50d-4e3c-adc1-d077dee314fb tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1681 microversion: 2.1 time: 0.175830 Sep 27 11:13:25.545973 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3445/7364] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:25 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1681 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:26.572604 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-06415d2b-7d9b-473f-aa08-97c3faee8b5e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:26.573599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-06415d2b-7d9b-473f-aa08-97c3faee8b5e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:26.584191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06415d2b-7d9b-473f-aa08-97c3faee8b5e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:26.584697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06415d2b-7d9b-473f-aa08-97c3faee8b5e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:26.585170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06415d2b-7d9b-473f-aa08-97c3faee8b5e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:26.585882 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06415d2b-7d9b-473f-aa08-97c3faee8b5e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:26.586350 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06415d2b-7d9b-473f-aa08-97c3faee8b5e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:26.586781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-06415d2b-7d9b-473f-aa08-97c3faee8b5e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:26.727903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-06415d2b-7d9b-473f-aa08-97c3faee8b5e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:26.736844 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-06415d2b-7d9b-473f-aa08-97c3faee8b5e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1681 microversion: 2.1 time: 0.170816 Sep 27 11:13:26.737393 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3920/7365] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:26 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1681 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:27.765679 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-caea226a-63a1-4b1f-8f6a-ca71cc71196e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:27.766707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-caea226a-63a1-4b1f-8f6a-ca71cc71196e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:27.779047 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-caea226a-63a1-4b1f-8f6a-ca71cc71196e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:27.779859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-caea226a-63a1-4b1f-8f6a-ca71cc71196e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:27.780692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-caea226a-63a1-4b1f-8f6a-ca71cc71196e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:27.781865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-caea226a-63a1-4b1f-8f6a-ca71cc71196e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:27.782673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-caea226a-63a1-4b1f-8f6a-ca71cc71196e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:27.783417 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-caea226a-63a1-4b1f-8f6a-ca71cc71196e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:27.921040 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-caea226a-63a1-4b1f-8f6a-ca71cc71196e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:27.929451 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-caea226a-63a1-4b1f-8f6a-ca71cc71196e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1681 microversion: 2.1 time: 0.170810 Sep 27 11:13:27.930045 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3446/7366] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:27 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1681 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:28.956370 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6d61121d-c8eb-4356-8714-9e27002905e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:28.957208 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6d61121d-c8eb-4356-8714-9e27002905e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:28.965319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d61121d-c8eb-4356-8714-9e27002905e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:28.965851 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d61121d-c8eb-4356-8714-9e27002905e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:28.966278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d61121d-c8eb-4356-8714-9e27002905e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:28.967092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d61121d-c8eb-4356-8714-9e27002905e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:28.967529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d61121d-c8eb-4356-8714-9e27002905e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:28.967957 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6d61121d-c8eb-4356-8714-9e27002905e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:29.118220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6d61121d-c8eb-4356-8714-9e27002905e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:29.126379 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6d61121d-c8eb-4356-8714-9e27002905e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1702 microversion: 2.1 time: 0.175964 Sep 27 11:13:29.126956 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3921/7367] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:28 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1702 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:30.151424 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ce0b8658-5217-4139-a72b-ca34e913498e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:30.152063 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ce0b8658-5217-4139-a72b-ca34e913498e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:30.159707 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce0b8658-5217-4139-a72b-ca34e913498e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:30.160195 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce0b8658-5217-4139-a72b-ca34e913498e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:30.160711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce0b8658-5217-4139-a72b-ca34e913498e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:30.161444 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce0b8658-5217-4139-a72b-ca34e913498e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:30.161955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce0b8658-5217-4139-a72b-ca34e913498e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:30.162404 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ce0b8658-5217-4139-a72b-ca34e913498e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:30.318075 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ce0b8658-5217-4139-a72b-ca34e913498e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:30.331143 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ce0b8658-5217-4139-a72b-ca34e913498e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1702 microversion: 2.1 time: 0.183987 Sep 27 11:13:30.332024 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3447/7368] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:30 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1702 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:31.357051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25153e51-fb5a-494a-be27-ea389ab002b3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:31.357683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-25153e51-fb5a-494a-be27-ea389ab002b3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:31.365172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25153e51-fb5a-494a-be27-ea389ab002b3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:31.365692 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25153e51-fb5a-494a-be27-ea389ab002b3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:31.366122 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25153e51-fb5a-494a-be27-ea389ab002b3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:31.366844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25153e51-fb5a-494a-be27-ea389ab002b3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:31.367314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25153e51-fb5a-494a-be27-ea389ab002b3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:31.367804 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25153e51-fb5a-494a-be27-ea389ab002b3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:31.510294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-25153e51-fb5a-494a-be27-ea389ab002b3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:31.518283 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-25153e51-fb5a-494a-be27-ea389ab002b3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1697 microversion: 2.1 time: 0.166600 Sep 27 11:13:31.519001 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3922/7369] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:31 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1697 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:32.544928 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-086e7408-5c20-4fed-a6a8-eba8786df957 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:32.546498 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-086e7408-5c20-4fed-a6a8-eba8786df957 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:32.554402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-086e7408-5c20-4fed-a6a8-eba8786df957 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:32.556151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-086e7408-5c20-4fed-a6a8-eba8786df957 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:32.557019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-086e7408-5c20-4fed-a6a8-eba8786df957 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:32.558138 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-086e7408-5c20-4fed-a6a8-eba8786df957 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:32.558683 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-086e7408-5c20-4fed-a6a8-eba8786df957 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:32.559199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-086e7408-5c20-4fed-a6a8-eba8786df957 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:32.718440 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-086e7408-5c20-4fed-a6a8-eba8786df957 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:32.730367 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-086e7408-5c20-4fed-a6a8-eba8786df957 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1697 microversion: 2.1 time: 0.190264 Sep 27 11:13:32.744687 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3448/7370] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:32 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1697 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:32.880519 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-ec171853-d913-469c-967b-7bed97ffec96 req-6f8ce2f8-bd31-4606-96a5-36473f15db8d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ca7c7b0-0978-4023-b97d-2f8fc46639e7", "name": "network-vif-plugged", "status": "completed", "tag": "b1be14d2-b2c4-4a69-9a91-a4521243aaf5"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:13:32.899002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ec171853-d913-469c-967b-7bed97ffec96 req-6f8ce2f8-bd31-4606-96a5-36473f15db8d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:32.899945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ec171853-d913-469c-967b-7bed97ffec96 req-6f8ce2f8-bd31-4606-96a5-36473f15db8d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:32.900599 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-ec171853-d913-469c-967b-7bed97ffec96 req-6f8ce2f8-bd31-4606-96a5-36473f15db8d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:32.913756 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-ec171853-d913-469c-967b-7bed97ffec96 req-6f8ce2f8-bd31-4606-96a5-36473f15db8d service nova] Creating event network-vif-plugged:b1be14d2-b2c4-4a69-9a91-a4521243aaf5 for instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 on nch1-master-1 Sep 27 11:13:32.924374 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-ec171853-d913-469c-967b-7bed97ffec96 req-6f8ce2f8-bd31-4606-96a5-36473f15db8d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044525 Sep 27 11:13:32.925969 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3923/7371] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:13:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:13:33.756124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-50285010-6b06-4224-b413-1c2417383ff2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:33.756763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-50285010-6b06-4224-b413-1c2417383ff2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:33.764350 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50285010-6b06-4224-b413-1c2417383ff2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:33.765227 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50285010-6b06-4224-b413-1c2417383ff2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:33.765903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50285010-6b06-4224-b413-1c2417383ff2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:33.766848 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50285010-6b06-4224-b413-1c2417383ff2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:33.767315 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50285010-6b06-4224-b413-1c2417383ff2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:33.767961 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-50285010-6b06-4224-b413-1c2417383ff2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:33.944727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-50285010-6b06-4224-b413-1c2417383ff2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:33.953541 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-50285010-6b06-4224-b413-1c2417383ff2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1697 microversion: 2.1 time: 0.201774 Sep 27 11:13:33.954474 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3449/7372] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:33 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1697 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:34.989354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-82c8b20e-810b-4577-8ec4-7403070f976f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:34.991352 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-82c8b20e-810b-4577-8ec4-7403070f976f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:35.000516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82c8b20e-810b-4577-8ec4-7403070f976f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:35.001613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82c8b20e-810b-4577-8ec4-7403070f976f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:35.002144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82c8b20e-810b-4577-8ec4-7403070f976f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:35.002960 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82c8b20e-810b-4577-8ec4-7403070f976f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:35.003517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82c8b20e-810b-4577-8ec4-7403070f976f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:35.004040 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-82c8b20e-810b-4577-8ec4-7403070f976f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:35.178704 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-82c8b20e-810b-4577-8ec4-7403070f976f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:35.187933 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-82c8b20e-810b-4577-8ec4-7403070f976f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1697 microversion: 2.1 time: 0.209366 Sep 27 11:13:35.200683 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3924/7373] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:34 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1697 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:36.214324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b32aaa0d-5d0e-45ac-9e99-904da107c36a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:36.218927 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b32aaa0d-5d0e-45ac-9e99-904da107c36a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:36.226282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b32aaa0d-5d0e-45ac-9e99-904da107c36a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:36.226807 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b32aaa0d-5d0e-45ac-9e99-904da107c36a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:36.227546 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b32aaa0d-5d0e-45ac-9e99-904da107c36a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:36.228626 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b32aaa0d-5d0e-45ac-9e99-904da107c36a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:36.229321 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b32aaa0d-5d0e-45ac-9e99-904da107c36a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:36.229981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b32aaa0d-5d0e-45ac-9e99-904da107c36a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:36.403665 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b32aaa0d-5d0e-45ac-9e99-904da107c36a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:36.414172 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b32aaa0d-5d0e-45ac-9e99-904da107c36a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1697 microversion: 2.1 time: 0.204198 Sep 27 11:13:36.415407 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3450/7374] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:36 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1697 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:37.441800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-e69853ae-598c-40b2-8237-8033517d43c0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:37.443140 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-e69853ae-598c-40b2-8237-8033517d43c0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:37.450189 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e69853ae-598c-40b2-8237-8033517d43c0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:37.450684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e69853ae-598c-40b2-8237-8033517d43c0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:37.451119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e69853ae-598c-40b2-8237-8033517d43c0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:37.451829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e69853ae-598c-40b2-8237-8033517d43c0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:37.452285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e69853ae-598c-40b2-8237-8033517d43c0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:37.452731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-e69853ae-598c-40b2-8237-8033517d43c0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:37.612632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-e69853ae-598c-40b2-8237-8033517d43c0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:37.621809 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-e69853ae-598c-40b2-8237-8033517d43c0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1697 microversion: 2.1 time: 0.184121 Sep 27 11:13:37.625026 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3925/7375] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:37 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1697 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:38.645732 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e0ca12e2-3fda-40d4-a65d-cfb943656969 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:38.647183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e0ca12e2-3fda-40d4-a65d-cfb943656969 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:38.656847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0ca12e2-3fda-40d4-a65d-cfb943656969 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:38.657360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0ca12e2-3fda-40d4-a65d-cfb943656969 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:38.657806 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0ca12e2-3fda-40d4-a65d-cfb943656969 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:38.658538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0ca12e2-3fda-40d4-a65d-cfb943656969 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:38.659014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0ca12e2-3fda-40d4-a65d-cfb943656969 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:38.659461 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e0ca12e2-3fda-40d4-a65d-cfb943656969 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:38.818169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e0ca12e2-3fda-40d4-a65d-cfb943656969 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:38.828060 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e0ca12e2-3fda-40d4-a65d-cfb943656969 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1697 microversion: 2.1 time: 0.186203 Sep 27 11:13:38.840872 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3451/7376] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:38 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1697 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:39.856479 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fc84191b-65df-4668-88d7-916179c48680 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:39.857159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fc84191b-65df-4668-88d7-916179c48680 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:39.864270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc84191b-65df-4668-88d7-916179c48680 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:39.864872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc84191b-65df-4668-88d7-916179c48680 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:39.865572 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc84191b-65df-4668-88d7-916179c48680 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:39.866295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc84191b-65df-4668-88d7-916179c48680 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:39.866752 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc84191b-65df-4668-88d7-916179c48680 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:39.867177 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc84191b-65df-4668-88d7-916179c48680 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:40.035991 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc84191b-65df-4668-88d7-916179c48680 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:40.044437 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fc84191b-65df-4668-88d7-916179c48680 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1697 microversion: 2.1 time: 0.195602 Sep 27 11:13:40.045255 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3926/7377] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:39 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1697 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:41.067815 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4cbd95d2-f9df-4c4f-86db-19957c217370 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:41.068471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4cbd95d2-f9df-4c4f-86db-19957c217370 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:41.076200 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cbd95d2-f9df-4c4f-86db-19957c217370 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:41.076749 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cbd95d2-f9df-4c4f-86db-19957c217370 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:41.077212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cbd95d2-f9df-4c4f-86db-19957c217370 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:41.077950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cbd95d2-f9df-4c4f-86db-19957c217370 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:41.078428 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cbd95d2-f9df-4c4f-86db-19957c217370 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:41.078862 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4cbd95d2-f9df-4c4f-86db-19957c217370 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:41.287987 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4cbd95d2-f9df-4c4f-86db-19957c217370 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:41.297048 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4cbd95d2-f9df-4c4f-86db-19957c217370 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1679 microversion: 2.1 time: 0.233243 Sep 27 11:13:41.297672 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3452/7378] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:41 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1679 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:42.323532 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7007c2d7-230b-436b-a167-7533f231de12 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:42.324165 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7007c2d7-230b-436b-a167-7533f231de12 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:42.331565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7007c2d7-230b-436b-a167-7533f231de12 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:42.332048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7007c2d7-230b-436b-a167-7533f231de12 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:42.332483 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7007c2d7-230b-436b-a167-7533f231de12 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:42.333216 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7007c2d7-230b-436b-a167-7533f231de12 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:42.333669 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7007c2d7-230b-436b-a167-7533f231de12 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:42.334097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7007c2d7-230b-436b-a167-7533f231de12 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:42.479184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7007c2d7-230b-436b-a167-7533f231de12 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:42.487378 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7007c2d7-230b-436b-a167-7533f231de12 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1679 microversion: 2.1 time: 0.167891 Sep 27 11:13:42.487941 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3927/7379] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:42 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1679 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:43.513764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5dee5d8e-3ada-4715-a4d9-db4536db8530 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:43.514560 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5dee5d8e-3ada-4715-a4d9-db4536db8530 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:43.524401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dee5d8e-3ada-4715-a4d9-db4536db8530 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:43.525102 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dee5d8e-3ada-4715-a4d9-db4536db8530 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:43.525711 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dee5d8e-3ada-4715-a4d9-db4536db8530 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:43.526590 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dee5d8e-3ada-4715-a4d9-db4536db8530 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:43.527058 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dee5d8e-3ada-4715-a4d9-db4536db8530 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:43.527523 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5dee5d8e-3ada-4715-a4d9-db4536db8530 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:43.673057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5dee5d8e-3ada-4715-a4d9-db4536db8530 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:43.684052 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5dee5d8e-3ada-4715-a4d9-db4536db8530 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1622 microversion: 2.1 time: 0.176145 Sep 27 11:13:43.684649 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3453/7380] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:43 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1622 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:43.708535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-513dbc63-11ae-4837-82ec-99ae6378e748 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:13:43.709130 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-513dbc63-11ae-4837-82ec-99ae6378e748 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:43.716250 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-513dbc63-11ae-4837-82ec-99ae6378e748 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:43.716766 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-513dbc63-11ae-4837-82ec-99ae6378e748 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:43.717200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-513dbc63-11ae-4837-82ec-99ae6378e748 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:43.717896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-513dbc63-11ae-4837-82ec-99ae6378e748 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:43.718349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-513dbc63-11ae-4837-82ec-99ae6378e748 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:43.718772 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-513dbc63-11ae-4837-82ec-99ae6378e748 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:43.763529 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-513dbc63-11ae-4837-82ec-99ae6378e748 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=145968) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 27 11:13:43.844702 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-513dbc63-11ae-4837-82ec-99ae6378e748 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "POST /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7/action" status: 202 len: 0 microversion: 2.1 time: 0.140149 Sep 27 11:13:43.845505 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3928/7381] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:13:43 2022] POST /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7/action => generated 0 bytes in 141 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:13:43.854004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-de206286-9406-4bc4-a6d6-759fd03a23f3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:43.854921 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-de206286-9406-4bc4-a6d6-759fd03a23f3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:43.862410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de206286-9406-4bc4-a6d6-759fd03a23f3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:43.862985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de206286-9406-4bc4-a6d6-759fd03a23f3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:43.863457 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de206286-9406-4bc4-a6d6-759fd03a23f3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:43.864191 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de206286-9406-4bc4-a6d6-759fd03a23f3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:43.864854 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de206286-9406-4bc4-a6d6-759fd03a23f3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:43.865297 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-de206286-9406-4bc4-a6d6-759fd03a23f3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:44.005423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-de206286-9406-4bc4-a6d6-759fd03a23f3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:44.013820 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-de206286-9406-4bc4-a6d6-759fd03a23f3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1630 microversion: 2.1 time: 0.164001 Sep 27 11:13:44.014433 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3454/7382] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:43 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1630 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:44.888135 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-a7748d16-17ef-4315-95be-43952ff43476 req-bb0402a9-c692-4fbb-9a28-e478131ff773 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ca7c7b0-0978-4023-b97d-2f8fc46639e7", "name": "network-vif-unplugged", "status": "completed", "tag": "b1be14d2-b2c4-4a69-9a91-a4521243aaf5"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:13:44.900010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a7748d16-17ef-4315-95be-43952ff43476 req-bb0402a9-c692-4fbb-9a28-e478131ff773 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:44.900488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a7748d16-17ef-4315-95be-43952ff43476 req-bb0402a9-c692-4fbb-9a28-e478131ff773 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:44.901154 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-a7748d16-17ef-4315-95be-43952ff43476 req-bb0402a9-c692-4fbb-9a28-e478131ff773 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:44.912217 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-a7748d16-17ef-4315-95be-43952ff43476 req-bb0402a9-c692-4fbb-9a28-e478131ff773 service nova] Creating event network-vif-unplugged:b1be14d2-b2c4-4a69-9a91-a4521243aaf5 for instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 on nch2-master-1 Sep 27 11:13:44.917179 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-a7748d16-17ef-4315-95be-43952ff43476 req-bb0402a9-c692-4fbb-9a28-e478131ff773 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035265 Sep 27 11:13:44.918333 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3929/7383] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:13:44 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:13:45.037159 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cbb1ecc5-067b-4e90-b622-faa912dc3ea1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:45.037793 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cbb1ecc5-067b-4e90-b622-faa912dc3ea1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:45.045593 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb1ecc5-067b-4e90-b622-faa912dc3ea1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:45.046099 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb1ecc5-067b-4e90-b622-faa912dc3ea1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:45.046555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb1ecc5-067b-4e90-b622-faa912dc3ea1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:45.047278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb1ecc5-067b-4e90-b622-faa912dc3ea1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:45.047753 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb1ecc5-067b-4e90-b622-faa912dc3ea1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:45.048196 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cbb1ecc5-067b-4e90-b622-faa912dc3ea1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:45.195393 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cbb1ecc5-067b-4e90-b622-faa912dc3ea1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:45.203785 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cbb1ecc5-067b-4e90-b622-faa912dc3ea1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.170694 Sep 27 11:13:45.204374 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3455/7384] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:45 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:46.230969 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-85f91dc0-6edc-4672-8f33-055cd24e7429 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:46.232202 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-85f91dc0-6edc-4672-8f33-055cd24e7429 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:46.239241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85f91dc0-6edc-4672-8f33-055cd24e7429 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:46.239739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85f91dc0-6edc-4672-8f33-055cd24e7429 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:46.240180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85f91dc0-6edc-4672-8f33-055cd24e7429 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:46.240974 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85f91dc0-6edc-4672-8f33-055cd24e7429 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:46.241434 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85f91dc0-6edc-4672-8f33-055cd24e7429 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:46.241864 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-85f91dc0-6edc-4672-8f33-055cd24e7429 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:46.390035 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-85f91dc0-6edc-4672-8f33-055cd24e7429 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:46.393835 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:46.399254 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-85f91dc0-6edc-4672-8f33-055cd24e7429 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.173973 Sep 27 11:13:46.399859 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3930/7385] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:46 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:46.531296 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-453d9cad-441b-435e-ac58-7b24d724945c req-2df0a40a-bc66-4472-9e31-8539cc7a8f3d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ca7c7b0-0978-4023-b97d-2f8fc46639e7", "tag": "b1be14d2-b2c4-4a69-9a91-a4521243aaf5"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:13:46.542070 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-453d9cad-441b-435e-ac58-7b24d724945c req-2df0a40a-bc66-4472-9e31-8539cc7a8f3d service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:46.542564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-453d9cad-441b-435e-ac58-7b24d724945c req-2df0a40a-bc66-4472-9e31-8539cc7a8f3d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:46.543046 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-453d9cad-441b-435e-ac58-7b24d724945c req-2df0a40a-bc66-4472-9e31-8539cc7a8f3d service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:46.554873 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-453d9cad-441b-435e-ac58-7b24d724945c req-2df0a40a-bc66-4472-9e31-8539cc7a8f3d service nova] Creating event network-changed:b1be14d2-b2c4-4a69-9a91-a4521243aaf5 for instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 on nch2-master-1 Sep 27 11:13:46.558458 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [req-453d9cad-441b-435e-ac58-7b24d724945c req-2df0a40a-bc66-4472-9e31-8539cc7a8f3d service nova] [ffac6b73-531c-4f9b-aa18-c01a364670aa] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 27 11:13:47.427571 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-b67a7caf-059a-4e2c-8b94-e7e990c3413b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:47.430294 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-b67a7caf-059a-4e2c-8b94-e7e990c3413b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:47.454041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b67a7caf-059a-4e2c-8b94-e7e990c3413b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:47.456414 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b67a7caf-059a-4e2c-8b94-e7e990c3413b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:47.457733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b67a7caf-059a-4e2c-8b94-e7e990c3413b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:47.459565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b67a7caf-059a-4e2c-8b94-e7e990c3413b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:47.460681 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b67a7caf-059a-4e2c-8b94-e7e990c3413b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:47.461765 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-b67a7caf-059a-4e2c-8b94-e7e990c3413b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:47.570387 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [req-453d9cad-441b-435e-ac58-7b24d724945c req-2df0a40a-bc66-4472-9e31-8539cc7a8f3d service nova] [ffac6b73-531c-4f9b-aa18-c01a364670aa] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 59714. Sep 27 11:13:47.574893 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-453d9cad-441b-435e-ac58-7b24d724945c req-2df0a40a-bc66-4472-9e31-8539cc7a8f3d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 1.047928 Sep 27 11:13:47.575647 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3456/7386] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:13:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 1050 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:13:47.656622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-b67a7caf-059a-4e2c-8b94-e7e990c3413b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:47.665253 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-b67a7caf-059a-4e2c-8b94-e7e990c3413b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.244252 Sep 27 11:13:47.666760 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3931/7387] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:47 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:48.691828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2c40e129-5d3c-4e2e-ac55-847c3c1cefdc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:48.692724 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2c40e129-5d3c-4e2e-ac55-847c3c1cefdc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:48.695135 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=31 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:48.701465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c40e129-5d3c-4e2e-ac55-847c3c1cefdc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:48.702423 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c40e129-5d3c-4e2e-ac55-847c3c1cefdc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:48.702855 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c40e129-5d3c-4e2e-ac55-847c3c1cefdc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:48.703558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c40e129-5d3c-4e2e-ac55-847c3c1cefdc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:48.704310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c40e129-5d3c-4e2e-ac55-847c3c1cefdc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:48.704865 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2c40e129-5d3c-4e2e-ac55-847c3c1cefdc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:48.708053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=87 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:48.860340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2c40e129-5d3c-4e2e-ac55-847c3c1cefdc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:48.869439 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2c40e129-5d3c-4e2e-ac55-847c3c1cefdc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.182687 Sep 27 11:13:48.870484 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3457/7388] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:48 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:49.894542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-9c2f4078-7917-4490-813f-5dd55f9f743d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:49.895204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-9c2f4078-7917-4490-813f-5dd55f9f743d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:49.903372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=37,nova_api:UPDATE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:13:49.904976 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c2f4078-7917-4490-813f-5dd55f9f743d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:49.905531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c2f4078-7917-4490-813f-5dd55f9f743d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:49.906347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c2f4078-7917-4490-813f-5dd55f9f743d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:49.907410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c2f4078-7917-4490-813f-5dd55f9f743d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:49.908055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c2f4078-7917-4490-813f-5dd55f9f743d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:49.908866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-9c2f4078-7917-4490-813f-5dd55f9f743d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:50.064754 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-9c2f4078-7917-4490-813f-5dd55f9f743d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:50.073011 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-9c2f4078-7917-4490-813f-5dd55f9f743d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.182629 Sep 27 11:13:50.073815 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3932/7389] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:49 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:51.097208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-fbb83101-a73b-4e42-bdb6-8440fa956b22 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:51.097900 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-fbb83101-a73b-4e42-bdb6-8440fa956b22 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:51.105139 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb83101-a73b-4e42-bdb6-8440fa956b22 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:51.105702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb83101-a73b-4e42-bdb6-8440fa956b22 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:51.106282 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb83101-a73b-4e42-bdb6-8440fa956b22 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:51.107053 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb83101-a73b-4e42-bdb6-8440fa956b22 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:51.107614 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb83101-a73b-4e42-bdb6-8440fa956b22 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:51.108107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-fbb83101-a73b-4e42-bdb6-8440fa956b22 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:51.265439 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-fbb83101-a73b-4e42-bdb6-8440fa956b22 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:51.276035 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-fbb83101-a73b-4e42-bdb6-8440fa956b22 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.183000 Sep 27 11:13:51.277113 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3458/7390] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:51 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:52.301127 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f110b545-76c9-4187-9132-91d77c8db024 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:52.301811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f110b545-76c9-4187-9132-91d77c8db024 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:52.311833 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f110b545-76c9-4187-9132-91d77c8db024 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:52.312346 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f110b545-76c9-4187-9132-91d77c8db024 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:52.312861 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f110b545-76c9-4187-9132-91d77c8db024 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:52.313586 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f110b545-76c9-4187-9132-91d77c8db024 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:52.314055 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f110b545-76c9-4187-9132-91d77c8db024 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:52.314462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f110b545-76c9-4187-9132-91d77c8db024 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:52.465949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f110b545-76c9-4187-9132-91d77c8db024 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:52.474995 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f110b545-76c9-4187-9132-91d77c8db024 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.177700 Sep 27 11:13:52.475592 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3933/7391] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:52 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:53.617542 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b57428c4-2edb-4278-8576-fff534f5479d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:53.618156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b57428c4-2edb-4278-8576-fff534f5479d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:53.625647 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b57428c4-2edb-4278-8576-fff534f5479d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:53.626152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b57428c4-2edb-4278-8576-fff534f5479d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:53.626604 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b57428c4-2edb-4278-8576-fff534f5479d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:53.627341 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b57428c4-2edb-4278-8576-fff534f5479d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:53.627801 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b57428c4-2edb-4278-8576-fff534f5479d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:53.628235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b57428c4-2edb-4278-8576-fff534f5479d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:53.770263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b57428c4-2edb-4278-8576-fff534f5479d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:53.781180 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b57428c4-2edb-4278-8576-fff534f5479d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.182750 Sep 27 11:13:53.782024 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3459/7392] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:53 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:54.808021 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-688ce3f9-0110-4c95-9f47-9bd323eb1c3f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:54.808978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-688ce3f9-0110-4c95-9f47-9bd323eb1c3f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:54.816334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-688ce3f9-0110-4c95-9f47-9bd323eb1c3f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:54.816874 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-688ce3f9-0110-4c95-9f47-9bd323eb1c3f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:54.817311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-688ce3f9-0110-4c95-9f47-9bd323eb1c3f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:54.818013 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-688ce3f9-0110-4c95-9f47-9bd323eb1c3f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:54.818468 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-688ce3f9-0110-4c95-9f47-9bd323eb1c3f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:54.818898 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-688ce3f9-0110-4c95-9f47-9bd323eb1c3f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:54.958182 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-688ce3f9-0110-4c95-9f47-9bd323eb1c3f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:54.966961 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-688ce3f9-0110-4c95-9f47-9bd323eb1c3f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.164001 Sep 27 11:13:54.967544 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3934/7393] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:54 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:55.996305 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c91d413e-db26-427a-9cbc-a590ccf48607 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:55.997172 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c91d413e-db26-427a-9cbc-a590ccf48607 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:56.008727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c91d413e-db26-427a-9cbc-a590ccf48607 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:56.009528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c91d413e-db26-427a-9cbc-a590ccf48607 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:56.010273 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c91d413e-db26-427a-9cbc-a590ccf48607 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:56.011555 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c91d413e-db26-427a-9cbc-a590ccf48607 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:56.012004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c91d413e-db26-427a-9cbc-a590ccf48607 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:56.012451 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c91d413e-db26-427a-9cbc-a590ccf48607 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:56.155820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-c91d413e-db26-427a-9cbc-a590ccf48607 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:56.168566 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c91d413e-db26-427a-9cbc-a590ccf48607 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.179586 Sep 27 11:13:56.169195 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3460/7394] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:55 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:57.194117 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7d61b015-5bd5-4442-b4f5-6b07de569c98 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:57.194682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7d61b015-5bd5-4442-b4f5-6b07de569c98 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:57.202941 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d61b015-5bd5-4442-b4f5-6b07de569c98 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:57.203488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d61b015-5bd5-4442-b4f5-6b07de569c98 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:57.204092 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d61b015-5bd5-4442-b4f5-6b07de569c98 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:57.204853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d61b015-5bd5-4442-b4f5-6b07de569c98 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:57.205328 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d61b015-5bd5-4442-b4f5-6b07de569c98 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:57.205764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7d61b015-5bd5-4442-b4f5-6b07de569c98 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:57.343499 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7d61b015-5bd5-4442-b4f5-6b07de569c98 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:57.351458 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7d61b015-5bd5-4442-b4f5-6b07de569c98 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.162283 Sep 27 11:13:57.352270 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3935/7395] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:57 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:58.379009 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-77e6d5e1-76d5-4797-851f-f579b46413e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:58.380036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-77e6d5e1-76d5-4797-851f-f579b46413e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:58.390733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77e6d5e1-76d5-4797-851f-f579b46413e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:58.391234 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77e6d5e1-76d5-4797-851f-f579b46413e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:58.391686 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77e6d5e1-76d5-4797-851f-f579b46413e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:58.392401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77e6d5e1-76d5-4797-851f-f579b46413e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:58.392891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77e6d5e1-76d5-4797-851f-f579b46413e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:58.393356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-77e6d5e1-76d5-4797-851f-f579b46413e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:58.538990 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-77e6d5e1-76d5-4797-851f-f579b46413e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:58.551838 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-77e6d5e1-76d5-4797-851f-f579b46413e7 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.179478 Sep 27 11:13:58.552675 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3461/7396] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:58 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:13:59.579633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5e360223-f45a-4872-a88d-8dd251b3101e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:13:59.580321 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5e360223-f45a-4872-a88d-8dd251b3101e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:13:59.587770 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e360223-f45a-4872-a88d-8dd251b3101e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:59.588267 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e360223-f45a-4872-a88d-8dd251b3101e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:59.588706 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e360223-f45a-4872-a88d-8dd251b3101e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:59.589451 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e360223-f45a-4872-a88d-8dd251b3101e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:13:59.589913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e360223-f45a-4872-a88d-8dd251b3101e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:13:59.590344 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5e360223-f45a-4872-a88d-8dd251b3101e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:13:59.729264 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5e360223-f45a-4872-a88d-8dd251b3101e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:13:59.737438 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5e360223-f45a-4872-a88d-8dd251b3101e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.161921 Sep 27 11:13:59.738071 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3936/7397] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:13:59 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:14:00.761673 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4f6d9a6a-0964-40b3-a7de-3803685d08d9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:14:00.762902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4f6d9a6a-0964-40b3-a7de-3803685d08d9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:14:00.770379 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f6d9a6a-0964-40b3-a7de-3803685d08d9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:14:00.770851 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f6d9a6a-0964-40b3-a7de-3803685d08d9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:14:00.771356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f6d9a6a-0964-40b3-a7de-3803685d08d9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:14:00.772069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f6d9a6a-0964-40b3-a7de-3803685d08d9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:14:00.772545 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f6d9a6a-0964-40b3-a7de-3803685d08d9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:14:00.773023 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f6d9a6a-0964-40b3-a7de-3803685d08d9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:14:00.912599 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4f6d9a6a-0964-40b3-a7de-3803685d08d9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:14:00.922296 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4f6d9a6a-0964-40b3-a7de-3803685d08d9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.164644 Sep 27 11:14:00.922910 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3462/7398] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:14:00 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:14:01.951749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3cfa5ea8-9a52-417c-ae04-ff5e3652c600 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:14:01.952377 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3cfa5ea8-9a52-417c-ae04-ff5e3652c600 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:14:01.959697 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfa5ea8-9a52-417c-ae04-ff5e3652c600 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:14:01.960215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfa5ea8-9a52-417c-ae04-ff5e3652c600 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:14:01.960675 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfa5ea8-9a52-417c-ae04-ff5e3652c600 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:14:01.961386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfa5ea8-9a52-417c-ae04-ff5e3652c600 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:14:01.961846 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfa5ea8-9a52-417c-ae04-ff5e3652c600 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:14:01.962269 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3cfa5ea8-9a52-417c-ae04-ff5e3652c600 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:14:02.100756 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3cfa5ea8-9a52-417c-ae04-ff5e3652c600 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:14:02.110400 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3cfa5ea8-9a52-417c-ae04-ff5e3652c600 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.165745 Sep 27 11:14:02.110991 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3937/7399] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:14:01 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:14:02.922354 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2f726101-9670-4aac-a307-9181557b8d7d req-68a479e3-cef4-4302-8f7e-f12259123397 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ca7c7b0-0978-4023-b97d-2f8fc46639e7", "name": "network-vif-plugged", "status": "completed", "tag": "b1be14d2-b2c4-4a69-9a91-a4521243aaf5"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:14:02.933403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2f726101-9670-4aac-a307-9181557b8d7d req-68a479e3-cef4-4302-8f7e-f12259123397 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:14:02.933912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2f726101-9670-4aac-a307-9181557b8d7d req-68a479e3-cef4-4302-8f7e-f12259123397 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:14:02.934367 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2f726101-9670-4aac-a307-9181557b8d7d req-68a479e3-cef4-4302-8f7e-f12259123397 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:14:02.947795 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2f726101-9670-4aac-a307-9181557b8d7d req-68a479e3-cef4-4302-8f7e-f12259123397 service nova] Creating event network-vif-plugged:b1be14d2-b2c4-4a69-9a91-a4521243aaf5 for instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 on nch2-master-1 Sep 27 11:14:02.954025 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2f726101-9670-4aac-a307-9181557b8d7d req-68a479e3-cef4-4302-8f7e-f12259123397 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035871 Sep 27 11:14:02.954645 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3463/7400] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:14:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:14:03.135768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7effde2e-a7d4-48cb-aadc-ca28983d7ed0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:14:03.136394 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7effde2e-a7d4-48cb-aadc-ca28983d7ed0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:14:03.145907 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7effde2e-a7d4-48cb-aadc-ca28983d7ed0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:14:03.146409 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7effde2e-a7d4-48cb-aadc-ca28983d7ed0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:14:03.146888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7effde2e-a7d4-48cb-aadc-ca28983d7ed0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:14:03.147614 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7effde2e-a7d4-48cb-aadc-ca28983d7ed0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:14:03.148472 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7effde2e-a7d4-48cb-aadc-ca28983d7ed0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:14:03.148944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7effde2e-a7d4-48cb-aadc-ca28983d7ed0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:14:03.299670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7effde2e-a7d4-48cb-aadc-ca28983d7ed0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:14:03.312188 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7effde2e-a7d4-48cb-aadc-ca28983d7ed0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.181325 Sep 27 11:14:03.313072 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3938/7401] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:14:03 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:14:04.347701 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-34eff47d-1986-4bf3-b2ae-7bc216407b07 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:14:04.348920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-34eff47d-1986-4bf3-b2ae-7bc216407b07 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:14:04.358605 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34eff47d-1986-4bf3-b2ae-7bc216407b07 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:14:04.359107 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34eff47d-1986-4bf3-b2ae-7bc216407b07 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:14:04.359562 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34eff47d-1986-4bf3-b2ae-7bc216407b07 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:14:04.360285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34eff47d-1986-4bf3-b2ae-7bc216407b07 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:14:04.360791 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34eff47d-1986-4bf3-b2ae-7bc216407b07 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:14:04.361236 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-34eff47d-1986-4bf3-b2ae-7bc216407b07 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:14:04.511754 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-34eff47d-1986-4bf3-b2ae-7bc216407b07 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:14:04.520001 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-34eff47d-1986-4bf3-b2ae-7bc216407b07 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1688 microversion: 2.1 time: 0.181082 Sep 27 11:14:04.520575 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3464/7402] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:14:04 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1688 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:14:05.548206 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-25eee7ba-4de2-43d9-b4c3-40abfcb26597 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:14:05.548834 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-25eee7ba-4de2-43d9-b4c3-40abfcb26597 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:14:05.556101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25eee7ba-4de2-43d9-b4c3-40abfcb26597 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:14:05.556584 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25eee7ba-4de2-43d9-b4c3-40abfcb26597 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:14:05.557044 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25eee7ba-4de2-43d9-b4c3-40abfcb26597 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:14:05.557787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25eee7ba-4de2-43d9-b4c3-40abfcb26597 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:14:05.558243 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25eee7ba-4de2-43d9-b4c3-40abfcb26597 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:14:05.558687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-25eee7ba-4de2-43d9-b4c3-40abfcb26597 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:14:05.712612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-25eee7ba-4de2-43d9-b4c3-40abfcb26597 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:14:05.721160 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-25eee7ba-4de2-43d9-b4c3-40abfcb26597 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1675 microversion: 2.1 time: 0.179655 Sep 27 11:14:05.721748 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3939/7403] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:14:05 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1675 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:42.607956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-a5c7b348-65cd-4447-943d-ad4de0d4610c tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:42.609253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a5c7b348-65cd-4447-943d-ad4de0d4610c tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:42.610329 ncdv-master-1 devstack@n-api.service[145968]: ERROR oslo.messaging._drivers.impl_rabbit [-] [eb0e6821-e1de-45db-a339-eb2fe392322e] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 27 11:15:42.611257 ncdv-master-1 devstack@n-api.service[145968]: ERROR oslo.messaging._drivers.impl_rabbit [-] [caeac69d-7b1f-444e-94c7-8c629d37bafc] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 27 11:15:42.612217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=15 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:15:42.613285 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=54 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:15:42.614381 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:15:42.614540 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 11:15:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.26) !!! Sep 27 11:15:42.619014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=15 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:15:42.620048 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=43 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:15:42.623179 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Sep 27 11:15:42.623301 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 11:15:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 (ip 10.222.0.26) !!! Sep 27 11:15:42.638659 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5c7b348-65cd-4447-943d-ad4de0d4610c tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:42.639177 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5c7b348-65cd-4447-943d-ad4de0d4610c tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:42.639635 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5c7b348-65cd-4447-943d-ad4de0d4610c tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:42.640377 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5c7b348-65cd-4447-943d-ad4de0d4610c tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:42.640904 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5c7b348-65cd-4447-943d-ad4de0d4610c tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:42.641355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-a5c7b348-65cd-4447-943d-ad4de0d4610c tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:42.671391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-a5c7b348-65cd-4447-943d-ad4de0d4610c tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 11:15:42.687324 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-1e6857d1-8d4b-494a-a73e-e87bae64ee69 req-64b9e382-ad3c-43c6-aa63-27fb80abe445 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4ca7c7b0-0978-4023-b97d-2f8fc46639e7", "tag": "b1be14d2-b2c4-4a69-9a91-a4521243aaf5"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:15:42.697838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1e6857d1-8d4b-494a-a73e-e87bae64ee69 req-64b9e382-ad3c-43c6-aa63-27fb80abe445 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:42.698327 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1e6857d1-8d4b-494a-a73e-e87bae64ee69 req-64b9e382-ad3c-43c6-aa63-27fb80abe445 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:42.698944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-1e6857d1-8d4b-494a-a73e-e87bae64ee69 req-64b9e382-ad3c-43c6-aa63-27fb80abe445 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:42.711723 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-1e6857d1-8d4b-494a-a73e-e87bae64ee69 req-64b9e382-ad3c-43c6-aa63-27fb80abe445 service nova] Creating event network-changed:b1be14d2-b2c4-4a69-9a91-a4521243aaf5 for instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 on nch2-master-1 Sep 27 11:15:42.717858 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-1e6857d1-8d4b-494a-a73e-e87bae64ee69 req-64b9e382-ad3c-43c6-aa63-27fb80abe445 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.111423 Sep 27 11:15:42.719052 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3940/7404] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:15:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:15:42.767933 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-a5c7b348-65cd-4447-943d-ad4de0d4610c tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 204 len: 0 microversion: 2.1 time: 0.166309 Sep 27 11:15:42.769262 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3465/7405] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 11:15:42 2022] DELETE /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 11:15:42.779314 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-7dc373d7-6b24-4de8-b246-e1024acb3fb1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:42.780101 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-7dc373d7-6b24-4de8-b246-e1024acb3fb1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:42.787340 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc373d7-6b24-4de8-b246-e1024acb3fb1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:42.788105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc373d7-6b24-4de8-b246-e1024acb3fb1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:42.788826 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc373d7-6b24-4de8-b246-e1024acb3fb1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:42.789721 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc373d7-6b24-4de8-b246-e1024acb3fb1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:42.790187 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc373d7-6b24-4de8-b246-e1024acb3fb1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:42.790626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-7dc373d7-6b24-4de8-b246-e1024acb3fb1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:42.944679 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-7dc373d7-6b24-4de8-b246-e1024acb3fb1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:15:42.953631 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-7dc373d7-6b24-4de8-b246-e1024acb3fb1 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1681 microversion: 2.1 time: 0.178840 Sep 27 11:15:42.954235 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3941/7406] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:42 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1681 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:43.980269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-29b0f7c8-33e4-4af6-b985-0ac7db11f803 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:43.981057 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-29b0f7c8-33e4-4af6-b985-0ac7db11f803 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:43.988579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b0f7c8-33e4-4af6-b985-0ac7db11f803 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:43.989094 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b0f7c8-33e4-4af6-b985-0ac7db11f803 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:43.989586 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b0f7c8-33e4-4af6-b985-0ac7db11f803 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:43.990335 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b0f7c8-33e4-4af6-b985-0ac7db11f803 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:43.990814 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b0f7c8-33e4-4af6-b985-0ac7db11f803 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:43.991259 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-29b0f7c8-33e4-4af6-b985-0ac7db11f803 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:44.171894 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-29b0f7c8-33e4-4af6-b985-0ac7db11f803 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:15:44.180968 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-29b0f7c8-33e4-4af6-b985-0ac7db11f803 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1681 microversion: 2.1 time: 0.205080 Sep 27 11:15:44.181547 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3466/7407] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:43 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1681 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:45.153748 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-bc0ff4cc-2bd5-4558-a63e-6031fdd0174f req-413b2ee3-18a4-44ba-8019-96f0782dea22 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4ca7c7b0-0978-4023-b97d-2f8fc46639e7", "name": "network-vif-deleted", "tag": "b1be14d2-b2c4-4a69-9a91-a4521243aaf5"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:15:45.169303 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] [eb0e6821-e1de-45db-a339-eb2fe392322e] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 59876. Sep 27 11:15:45.175003 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] [caeac69d-7b1f-444e-94c7-8c629d37bafc] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 59878. Sep 27 11:15:45.179732 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bc0ff4cc-2bd5-4558-a63e-6031fdd0174f req-413b2ee3-18a4-44ba-8019-96f0782dea22 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:45.180224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bc0ff4cc-2bd5-4558-a63e-6031fdd0174f req-413b2ee3-18a4-44ba-8019-96f0782dea22 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:45.180682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bc0ff4cc-2bd5-4558-a63e-6031fdd0174f req-413b2ee3-18a4-44ba-8019-96f0782dea22 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:45.194794 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-bc0ff4cc-2bd5-4558-a63e-6031fdd0174f req-413b2ee3-18a4-44ba-8019-96f0782dea22 service nova] Creating event network-vif-deleted:b1be14d2-b2c4-4a69-9a91-a4521243aaf5 for instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 on nch2-master-1 Sep 27 11:15:45.202049 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-bc0ff4cc-2bd5-4558-a63e-6031fdd0174f req-413b2ee3-18a4-44ba-8019-96f0782dea22 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052484 Sep 27 11:15:45.202662 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3942/7408] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:15:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:15:45.205620 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d39b53c1-b64f-448d-835d-06161bc653f5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:45.206403 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-d39b53c1-b64f-448d-835d-06161bc653f5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:45.213920 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d39b53c1-b64f-448d-835d-06161bc653f5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:45.214418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d39b53c1-b64f-448d-835d-06161bc653f5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:45.215084 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d39b53c1-b64f-448d-835d-06161bc653f5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:45.215953 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d39b53c1-b64f-448d-835d-06161bc653f5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:45.216550 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d39b53c1-b64f-448d-835d-06161bc653f5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:45.217670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-d39b53c1-b64f-448d-835d-06161bc653f5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:45.285518 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-d39b53c1-b64f-448d-835d-06161bc653f5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:15:45.295060 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d39b53c1-b64f-448d-835d-06161bc653f5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 200 len: 1332 microversion: 2.1 time: 0.094094 Sep 27 11:15:45.295651 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3467/7409] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:45 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 1332 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:46.321863 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:46.322736 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4ca7c7b0-0978-4023-b97d-2f8fc46639e7] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:46.330838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:46.331447 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:46.332287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:46.333051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:46.333514 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:46.333947 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:46.347796 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] HTTP exception thrown: Instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 could not be found. Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Request method failure captured: Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 HTTP/1.1 Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtpFyEAsTKQK_R_zoAYTu93KsHs-d1Iek0-OmH4eIKijvQ5I1NezJJErnSobjKpmSWEZly8UgWWnEWzglcHZKuHALei7PJwLQaLPf5EvRMNQIOGYRRk0WU40xS67gdP__QQAI1rLvn0iUt7UvDT1P63h9K9awLpalGDygX0M2F7mmlg Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 37db46b0eb6a4c7d938085b26d1968a7 Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestShelveInstance-666939746 Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37db46b0eb6a4c7d938085b26d1968a7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37db46b0eb6a4c7d938085b26d1968a7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/37db46b0eb6a4c7d938085b26d1968a7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_37db46b0eb6a4c7d938085b26d1968a7"}]}] Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestShelveInstance-666939746 Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 37db46b0eb6a4c7d938085b26d1968a7 Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestShelveInstance-666939746 Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestShelveInstance-666939746-project Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 90cea8d7a5494c45ba299df59797633f Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestShelveInstance-666939746-project Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 could not be found. Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '4ca7c7b0-0978-4023-b97d-2f8fc46639e7'} Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 could not be found. Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 could not be found. Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:15:46.349437 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:15:46.353380 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:15:46.353380 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:15:46.353380 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:15:46.353380 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 11:15:46.353380 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 11:15:46.353380 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 could not be found. Sep 27 11:15:46.353380 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:15:46.353380 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Returning 404 to user: Instance 4ca7c7b0-0978-4023-b97d-2f8fc46639e7 could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 11:15:46.353380 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-0a75a427-38ce-4b8d-adcf-e83953d4a3f6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7" status: 404 len: 111 microversion: 2.1 time: 0.035803 Sep 27 11:15:46.353380 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3943/7410] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:46 2022] GET /compute/v2.1/servers/4ca7c7b0-0978-4023-b97d-2f8fc46639e7 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 11:15:46.519643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-d3107d65-e3a8-4663-acb8-56a9651c0dbf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '.version_select at 0x7f7f6014f160>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:46.549223 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-d3107d65-e3a8-4663-acb8-56a9651c0dbf tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-2001153803" status: 202 len: 0 microversion: 2.1 time: 0.033608 Sep 27 11:15:46.553535 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3468/7411] 10.222.0.26 () {58 vars in 1254 bytes} [Tue Sep 27 11:15:46 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-2001153803 => generated 0 bytes in 35 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:15:46.569684 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a99454bc-5f54-4a13-924e-45f130453149 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Action: 'create', calling method: .version_select at 0x7f7f637dc9d0>, body: {"keypair": {"name": "tempest-TestShelveInstance-128583505", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM98N4C5/pfzDAEK4QveMTKG9yWCqgpEn85HdYJc8V+8zB6uJYovUuDBaUW6BxBrx2wmzJQH7X1QcaRjbFBtpPdfBpootsxKfyCLrsbJ0hvZXaaOBLJp+3pCTboRfhR8HQ=="}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:15:46.585196 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-a99454bc-5f54-4a13-924e-45f130453149 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Getting quotas for project 37db46b0eb6a4c7d938085b26d1968a7. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:15:46.590478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-a99454bc-5f54-4a13-924e-45f130453149 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Getting quotas for user 90cea8d7a5494c45ba299df59797633f and project 37db46b0eb6a4c7d938085b26d1968a7. Resources: {'key_pairs'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:15:46.623300 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a99454bc-5f54-4a13-924e-45f130453149 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.057325 Sep 27 11:15:46.624238 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3944/7412] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 11:15:46 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 11:15:50.238426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1835370031", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-128583505", "security_groups": [{"name": "tempest-secgroup-smoke-1677790147"}], "networks": [{"uuid": "b6772c8e-2375-4108-a69f-0a6f573b8005"}], "block_device_mapping_v2": [{"uuid": "570e326c-0f5d-44bd-a289-2e89c4df917f", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:15:50.366289 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.network.neutron [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] validate_networks() for [('b6772c8e-2375-4108-a69f-0a6f573b8005', None, None, None, None, None)] {{(pid=145969) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 11:15:50.765874 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Flavor limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 11:15:50.766465 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Image limits 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 11:15:50.767276 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Flavor pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 11:15:50.767981 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Image pref 0:0:0 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 11:15:50.768714 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.virt.hardware [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145969) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 11:15:50.782678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:50.783468 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:50.784245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:50.785530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:50.786226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:50.786691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:50.804507 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Getting quotas for project 37db46b0eb6a4c7d938085b26d1968a7. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:15:50.809740 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Getting quotas for user 90cea8d7a5494c45ba299df59797633f and project 37db46b0eb6a4c7d938085b26d1968a7. Resources: {'instances', 'ram', 'cores'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:15:50.871097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Going to run 1 instances... {{(pid=145969) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 11:15:50.873395 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='570e326c-0f5d-44bd-a289-2e89c4df917f',volume_size=None,volume_type=None)] {{(pid=145969) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 11:15:51.018655 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5d532368-f821-43fc-a8b2-bfa911df8308 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.785817 Sep 27 11:15:51.019274 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3469/7413] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 11:15:50 2022] POST /compute/v2.1/servers => generated 406 bytes in 787 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 11:15:51.032184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-87719059-cf22-4881-9d54-c0f068dc07dd tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:51.033003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-87719059-cf22-4881-9d54-c0f068dc07dd tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:51.090138 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-87719059-cf22-4881-9d54-c0f068dc07dd tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:15:51.097275 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-87719059-cf22-4881-9d54-c0f068dc07dd tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1088 microversion: 2.1 time: 0.070022 Sep 27 11:15:51.097867 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3945/7414] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:51 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1088 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:52.120327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1bbb4807-f73d-475b-bff2-916a1405be24 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:52.121267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1bbb4807-f73d-475b-bff2-916a1405be24 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:52.129558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbb4807-f73d-475b-bff2-916a1405be24 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:52.131098 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbb4807-f73d-475b-bff2-916a1405be24 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:52.131973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbb4807-f73d-475b-bff2-916a1405be24 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:52.132847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbb4807-f73d-475b-bff2-916a1405be24 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:52.133441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbb4807-f73d-475b-bff2-916a1405be24 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:52.134212 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1bbb4807-f73d-475b-bff2-916a1405be24 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:52.219348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1bbb4807-f73d-475b-bff2-916a1405be24 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:15:52.227537 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1bbb4807-f73d-475b-bff2-916a1405be24 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1194 microversion: 2.1 time: 0.111112 Sep 27 11:15:52.228346 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3470/7415] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:52 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1194 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:53.252102 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-05344faf-77f7-427d-b015-2cfa1d53b926 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:53.252838 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-05344faf-77f7-427d-b015-2cfa1d53b926 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:53.263879 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05344faf-77f7-427d-b015-2cfa1d53b926 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:53.264423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05344faf-77f7-427d-b015-2cfa1d53b926 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:53.265270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05344faf-77f7-427d-b015-2cfa1d53b926 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:53.265640 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05344faf-77f7-427d-b015-2cfa1d53b926 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:53.266106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05344faf-77f7-427d-b015-2cfa1d53b926 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:53.266564 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-05344faf-77f7-427d-b015-2cfa1d53b926 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:53.420311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-05344faf-77f7-427d-b015-2cfa1d53b926 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:15:53.435087 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-05344faf-77f7-427d-b015-2cfa1d53b926 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1272 microversion: 2.1 time: 0.187932 Sep 27 11:15:53.435691 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3946/7416] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:53 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1272 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:53.684240 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1a792387-b000-4abe-ae42-ffe1094056e5 req-b3b7a7fd-7f22-4477-b59e-f418ca9b9e65 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4f793bbd-0d6e-434e-a84f-552abcb0ba11", "tag": "128e0956-38c1-48fc-b832-009f7e0de308"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:15:53.695152 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1a792387-b000-4abe-ae42-ffe1094056e5 req-b3b7a7fd-7f22-4477-b59e-f418ca9b9e65 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:53.695664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1a792387-b000-4abe-ae42-ffe1094056e5 req-b3b7a7fd-7f22-4477-b59e-f418ca9b9e65 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:53.696113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1a792387-b000-4abe-ae42-ffe1094056e5 req-b3b7a7fd-7f22-4477-b59e-f418ca9b9e65 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:53.707554 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1a792387-b000-4abe-ae42-ffe1094056e5 req-b3b7a7fd-7f22-4477-b59e-f418ca9b9e65 service nova] Creating event network-changed:128e0956-38c1-48fc-b832-009f7e0de308 for instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 on nch1-master-1 Sep 27 11:15:53.713418 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1a792387-b000-4abe-ae42-ffe1094056e5 req-b3b7a7fd-7f22-4477-b59e-f418ca9b9e65 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.117970 Sep 27 11:15:53.714032 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3471/7417] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:15:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:15:54.459241 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-48975b6b-71b1-431c-ab80-00830f7a450b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:54.459781 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-48975b6b-71b1-431c-ab80-00830f7a450b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:54.467240 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48975b6b-71b1-431c-ab80-00830f7a450b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:54.467730 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48975b6b-71b1-431c-ab80-00830f7a450b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:54.468164 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48975b6b-71b1-431c-ab80-00830f7a450b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:54.468915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48975b6b-71b1-431c-ab80-00830f7a450b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:54.469374 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48975b6b-71b1-431c-ab80-00830f7a450b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:54.469798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-48975b6b-71b1-431c-ab80-00830f7a450b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:54.620197 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-48975b6b-71b1-431c-ab80-00830f7a450b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:15:54.628699 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-48975b6b-71b1-431c-ab80-00830f7a450b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1260 microversion: 2.1 time: 0.173735 Sep 27 11:15:54.629410 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3947/7418] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:54 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1260 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:55.660543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-05a7a93a-d281-47c3-90dc-4e2249bfada8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:55.662409 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-05a7a93a-d281-47c3-90dc-4e2249bfada8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:55.675488 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05a7a93a-d281-47c3-90dc-4e2249bfada8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:55.675651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05a7a93a-d281-47c3-90dc-4e2249bfada8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:55.676898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05a7a93a-d281-47c3-90dc-4e2249bfada8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:55.677667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05a7a93a-d281-47c3-90dc-4e2249bfada8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:55.678134 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05a7a93a-d281-47c3-90dc-4e2249bfada8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:55.678692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-05a7a93a-d281-47c3-90dc-4e2249bfada8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:55.827980 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-05a7a93a-d281-47c3-90dc-4e2249bfada8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:15:55.836611 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-05a7a93a-d281-47c3-90dc-4e2249bfada8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1260 microversion: 2.1 time: 0.185202 Sep 27 11:15:55.837198 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3472/7419] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:55 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1260 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:56.862407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-62eba31c-af8f-4a1b-8912-de9f31b043ae tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:56.862962 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-62eba31c-af8f-4a1b-8912-de9f31b043ae tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:56.873621 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62eba31c-af8f-4a1b-8912-de9f31b043ae tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:56.874369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62eba31c-af8f-4a1b-8912-de9f31b043ae tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:56.875014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62eba31c-af8f-4a1b-8912-de9f31b043ae tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:56.876094 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62eba31c-af8f-4a1b-8912-de9f31b043ae tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:56.876859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62eba31c-af8f-4a1b-8912-de9f31b043ae tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:56.877459 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-62eba31c-af8f-4a1b-8912-de9f31b043ae tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:57.014555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-62eba31c-af8f-4a1b-8912-de9f31b043ae tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:15:57.023569 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-62eba31c-af8f-4a1b-8912-de9f31b043ae tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1260 microversion: 2.1 time: 0.167551 Sep 27 11:15:57.024137 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3948/7420] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:56 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1260 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:58.047847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3118bde1-9b88-4c51-925f-884372dece05 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:58.048625 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3118bde1-9b88-4c51-925f-884372dece05 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:58.056720 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3118bde1-9b88-4c51-925f-884372dece05 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:58.057348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3118bde1-9b88-4c51-925f-884372dece05 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:58.057776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3118bde1-9b88-4c51-925f-884372dece05 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:58.058678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3118bde1-9b88-4c51-925f-884372dece05 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:58.059163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3118bde1-9b88-4c51-925f-884372dece05 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:58.059607 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3118bde1-9b88-4c51-925f-884372dece05 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:58.201631 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3118bde1-9b88-4c51-925f-884372dece05 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:15:58.209870 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3118bde1-9b88-4c51-925f-884372dece05 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1260 microversion: 2.1 time: 0.166107 Sep 27 11:15:58.210462 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3473/7421] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:58 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1260 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:59.234749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c9178118-846c-4475-bada-f5a412262ad3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:15:59.235516 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c9178118-846c-4475-bada-f5a412262ad3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:15:59.245286 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9178118-846c-4475-bada-f5a412262ad3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:59.245776 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9178118-846c-4475-bada-f5a412262ad3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:59.246209 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9178118-846c-4475-bada-f5a412262ad3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:59.246909 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9178118-846c-4475-bada-f5a412262ad3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:59.247367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9178118-846c-4475-bada-f5a412262ad3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:59.247792 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c9178118-846c-4475-bada-f5a412262ad3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:59.402945 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c9178118-846c-4475-bada-f5a412262ad3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:15:59.411140 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c9178118-846c-4475-bada-f5a412262ad3 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1260 microversion: 2.1 time: 0.181721 Sep 27 11:15:59.411726 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3949/7422] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:15:59 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1260 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:15:59.633875 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-664d155c-3849-4ac7-bcf9-4e1a035c2bd9 req-d1ab967f-9d2b-4183-ba8a-698b97336e09 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f793bbd-0d6e-434e-a84f-552abcb0ba11", "name": "network-vif-plugged", "status": "completed", "tag": "128e0956-38c1-48fc-b832-009f7e0de308"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:15:59.644113 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-664d155c-3849-4ac7-bcf9-4e1a035c2bd9 req-d1ab967f-9d2b-4183-ba8a-698b97336e09 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:15:59.644729 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-664d155c-3849-4ac7-bcf9-4e1a035c2bd9 req-d1ab967f-9d2b-4183-ba8a-698b97336e09 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:15:59.645238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-664d155c-3849-4ac7-bcf9-4e1a035c2bd9 req-d1ab967f-9d2b-4183-ba8a-698b97336e09 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:15:59.657521 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-664d155c-3849-4ac7-bcf9-4e1a035c2bd9 req-d1ab967f-9d2b-4183-ba8a-698b97336e09 service nova] Creating event network-vif-plugged:128e0956-38c1-48fc-b832-009f7e0de308 for instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 on nch1-master-1 Sep 27 11:15:59.662795 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-664d155c-3849-4ac7-bcf9-4e1a035c2bd9 req-d1ab967f-9d2b-4183-ba8a-698b97336e09 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093055 Sep 27 11:15:59.663520 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3474/7423] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:15:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:16:00.436952 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2d5b9741-6692-4820-b4d5-2d6e6ee724d8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:16:00.437678 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2d5b9741-6692-4820-b4d5-2d6e6ee724d8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:16:00.447939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d5b9741-6692-4820-b4d5-2d6e6ee724d8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:16:00.448688 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d5b9741-6692-4820-b4d5-2d6e6ee724d8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:16:00.449180 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d5b9741-6692-4820-b4d5-2d6e6ee724d8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:16:00.450142 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d5b9741-6692-4820-b4d5-2d6e6ee724d8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:16:00.450739 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d5b9741-6692-4820-b4d5-2d6e6ee724d8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:16:00.451803 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2d5b9741-6692-4820-b4d5-2d6e6ee724d8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:16:00.587005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-2d5b9741-6692-4820-b4d5-2d6e6ee724d8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:16:00.595115 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2d5b9741-6692-4820-b4d5-2d6e6ee724d8 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1260 microversion: 2.1 time: 0.163867 Sep 27 11:16:00.595702 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3950/7424] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:16:00 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1260 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 11:16:01.619332 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:16:01.621250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:16:01.628748 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0d9709ee-1845-4bcb-9ff2-b513ec72ffe0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:16:01.629972 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0d9709ee-1845-4bcb-9ff2-b513ec72ffe0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:16:01.639035 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d9709ee-1845-4bcb-9ff2-b513ec72ffe0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:16:01.639544 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d9709ee-1845-4bcb-9ff2-b513ec72ffe0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:16:01.639999 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d9709ee-1845-4bcb-9ff2-b513ec72ffe0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:16:01.640727 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d9709ee-1845-4bcb-9ff2-b513ec72ffe0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:16:01.641231 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d9709ee-1845-4bcb-9ff2-b513ec72ffe0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:16:01.641677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0d9709ee-1845-4bcb-9ff2-b513ec72ffe0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:16:01.777572 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0d9709ee-1845-4bcb-9ff2-b513ec72ffe0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:16:01.788538 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0d9709ee-1845-4bcb-9ff2-b513ec72ffe0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1444 microversion: 2.1 time: 0.172205 Sep 27 11:16:01.789425 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3475/7425] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:16:01 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1444 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:16:01.811485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-5ae651db-1b75-41e5-acbb-35c1efe63487 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:16:01.812097 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-5ae651db-1b75-41e5-acbb-35c1efe63487 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:16:01.814120 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:16:01.820015 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ae651db-1b75-41e5-acbb-35c1efe63487 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:16:01.820495 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ae651db-1b75-41e5-acbb-35c1efe63487 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:16:01.821404 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ae651db-1b75-41e5-acbb-35c1efe63487 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:16:01.821900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ae651db-1b75-41e5-acbb-35c1efe63487 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:16:01.822392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ae651db-1b75-41e5-acbb-35c1efe63487 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:16:01.822815 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-5ae651db-1b75-41e5-acbb-35c1efe63487 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:16:01.970422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-5ae651db-1b75-41e5-acbb-35c1efe63487 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:16:01.979192 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-5ae651db-1b75-41e5-acbb-35c1efe63487 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1444 microversion: 2.1 time: 0.171482 Sep 27 11:16:01.979979 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3951/7426] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:16:01 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1444 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:16:03.666733 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-8d855167-4821-4d66-94cf-8b020ed49641 req-2f5e17f7-8908-4cdd-96c4-5cb0240c7090 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4f793bbd-0d6e-434e-a84f-552abcb0ba11", "tag": "128e0956-38c1-48fc-b832-009f7e0de308"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:16:03.677866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8d855167-4821-4d66-94cf-8b020ed49641 req-2f5e17f7-8908-4cdd-96c4-5cb0240c7090 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:16:03.678391 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8d855167-4821-4d66-94cf-8b020ed49641 req-2f5e17f7-8908-4cdd-96c4-5cb0240c7090 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:16:03.679004 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-8d855167-4821-4d66-94cf-8b020ed49641 req-2f5e17f7-8908-4cdd-96c4-5cb0240c7090 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:16:03.691632 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-8d855167-4821-4d66-94cf-8b020ed49641 req-2f5e17f7-8908-4cdd-96c4-5cb0240c7090 service nova] Creating event network-changed:128e0956-38c1-48fc-b832-009f7e0de308 for instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 on nch1-master-1 Sep 27 11:16:03.696881 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-8d855167-4821-4d66-94cf-8b020ed49641 req-2f5e17f7-8908-4cdd-96c4-5cb0240c7090 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034515 Sep 27 11:16:03.697671 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3476/7427] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:16:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:18:08.830794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:18:08.832205 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=48 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:18:08.834113 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:18:08.834364 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 11:18:08 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11/action (ip 10.222.0.26) !!! Sep 27 11:18:08.899274 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:18:08.899832 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:08.906422 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:08.906913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:08.907356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:08.908064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:08.908527 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:08.908999 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:08.932392 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lazy-loading 'flavor' on Instance uuid 4f793bbd-0d6e-434e-a84f-552abcb0ba11 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 11:18:09.015373 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-04495fe0-1e91-4567-b224-c6d0b0fd4bc5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "POST /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11/action" status: 202 len: 0 microversion: 2.1 time: 0.187980 Sep 27 11:18:09.015988 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3952/7428] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:18:08 2022] POST /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11/action => generated 0 bytes in 190 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:18:09.025447 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=43,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:18:09.026334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=58,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:18:09.032737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6e068786-ba80-4041-96cc-bd5e6db7bb2f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:09.033730 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6e068786-ba80-4041-96cc-bd5e6db7bb2f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:09.046036 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e068786-ba80-4041-96cc-bd5e6db7bb2f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:09.046743 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e068786-ba80-4041-96cc-bd5e6db7bb2f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:09.047193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e068786-ba80-4041-96cc-bd5e6db7bb2f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:09.047931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e068786-ba80-4041-96cc-bd5e6db7bb2f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:09.048410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e068786-ba80-4041-96cc-bd5e6db7bb2f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:09.048882 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6e068786-ba80-4041-96cc-bd5e6db7bb2f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:09.191660 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6e068786-ba80-4041-96cc-bd5e6db7bb2f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:09.199770 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6e068786-ba80-4041-96cc-bd5e6db7bb2f tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.177426 Sep 27 11:18:09.200357 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3477/7429] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:09 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 27 11:18:10.227822 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3dcd439f-b26e-4de0-9ca8-e5509d141433 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:10.229023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3dcd439f-b26e-4de0-9ca8-e5509d141433 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:10.238231 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3dcd439f-b26e-4de0-9ca8-e5509d141433 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:10.238749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3dcd439f-b26e-4de0-9ca8-e5509d141433 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:10.239184 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3dcd439f-b26e-4de0-9ca8-e5509d141433 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:10.239906 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3dcd439f-b26e-4de0-9ca8-e5509d141433 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:10.240342 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3dcd439f-b26e-4de0-9ca8-e5509d141433 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:10.240787 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3dcd439f-b26e-4de0-9ca8-e5509d141433 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:10.385331 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3dcd439f-b26e-4de0-9ca8-e5509d141433 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:10.393730 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3dcd439f-b26e-4de0-9ca8-e5509d141433 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.173100 Sep 27 11:18:10.394308 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3953/7430] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:10 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:11.422275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4369ecce-b1c3-4075-a7ce-bf5de77cfea0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:11.423323 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4369ecce-b1c3-4075-a7ce-bf5de77cfea0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:11.436037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4369ecce-b1c3-4075-a7ce-bf5de77cfea0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:11.437065 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4369ecce-b1c3-4075-a7ce-bf5de77cfea0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:11.437581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4369ecce-b1c3-4075-a7ce-bf5de77cfea0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:11.438208 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4369ecce-b1c3-4075-a7ce-bf5de77cfea0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:11.438667 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4369ecce-b1c3-4075-a7ce-bf5de77cfea0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:11.439115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4369ecce-b1c3-4075-a7ce-bf5de77cfea0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:11.577998 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4369ecce-b1c3-4075-a7ce-bf5de77cfea0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:11.588826 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4369ecce-b1c3-4075-a7ce-bf5de77cfea0 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.173361 Sep 27 11:18:11.589606 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3478/7431] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:11 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:12.616093 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1f3c6e34-3aa0-41c0-ae9d-1bda39759290 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:12.616819 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1f3c6e34-3aa0-41c0-ae9d-1bda39759290 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:12.624059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f3c6e34-3aa0-41c0-ae9d-1bda39759290 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:12.624542 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f3c6e34-3aa0-41c0-ae9d-1bda39759290 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:12.625026 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f3c6e34-3aa0-41c0-ae9d-1bda39759290 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:12.625699 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f3c6e34-3aa0-41c0-ae9d-1bda39759290 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:12.626583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f3c6e34-3aa0-41c0-ae9d-1bda39759290 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:12.626583 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1f3c6e34-3aa0-41c0-ae9d-1bda39759290 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:12.761278 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1f3c6e34-3aa0-41c0-ae9d-1bda39759290 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:12.773572 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1f3c6e34-3aa0-41c0-ae9d-1bda39759290 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.161152 Sep 27 11:18:12.774362 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3954/7432] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:12 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:13.798401 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-b9c1f3bd-9f85-433f-88a9-21dbd18cafef tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:13.798898 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-b9c1f3bd-9f85-433f-88a9-21dbd18cafef tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:13.805974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9c1f3bd-9f85-433f-88a9-21dbd18cafef tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:13.806508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9c1f3bd-9f85-433f-88a9-21dbd18cafef tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:13.806931 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9c1f3bd-9f85-433f-88a9-21dbd18cafef tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:13.807658 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9c1f3bd-9f85-433f-88a9-21dbd18cafef tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:13.808127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9c1f3bd-9f85-433f-88a9-21dbd18cafef tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:13.808561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-b9c1f3bd-9f85-433f-88a9-21dbd18cafef tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:13.955012 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-b9c1f3bd-9f85-433f-88a9-21dbd18cafef tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:13.963501 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-b9c1f3bd-9f85-433f-88a9-21dbd18cafef tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.169665 Sep 27 11:18:13.964230 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3479/7433] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:13 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:14.987780 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-3382dece-b696-456c-91eb-f6faed73c0ca tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:14.988348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-3382dece-b696-456c-91eb-f6faed73c0ca tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:14.995794 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3382dece-b696-456c-91eb-f6faed73c0ca tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:14.996295 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3382dece-b696-456c-91eb-f6faed73c0ca tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:14.996768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3382dece-b696-456c-91eb-f6faed73c0ca tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:14.997476 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3382dece-b696-456c-91eb-f6faed73c0ca tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:14.997939 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3382dece-b696-456c-91eb-f6faed73c0ca tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:14.998372 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-3382dece-b696-456c-91eb-f6faed73c0ca tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:15.154690 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-3382dece-b696-456c-91eb-f6faed73c0ca tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:15.164340 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-3382dece-b696-456c-91eb-f6faed73c0ca tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.180648 Sep 27 11:18:15.165151 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3955/7434] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:14 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:16.193141 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5e57d6a3-bce1-4034-8039-bf26e1be8216 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:16.194169 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5e57d6a3-bce1-4034-8039-bf26e1be8216 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:16.205692 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e57d6a3-bce1-4034-8039-bf26e1be8216 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:16.206193 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e57d6a3-bce1-4034-8039-bf26e1be8216 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:16.206644 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e57d6a3-bce1-4034-8039-bf26e1be8216 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:16.207355 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e57d6a3-bce1-4034-8039-bf26e1be8216 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:16.207820 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e57d6a3-bce1-4034-8039-bf26e1be8216 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:16.208260 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5e57d6a3-bce1-4034-8039-bf26e1be8216 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:16.356055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5e57d6a3-bce1-4034-8039-bf26e1be8216 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:16.368446 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5e57d6a3-bce1-4034-8039-bf26e1be8216 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.181855 Sep 27 11:18:16.369298 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3480/7435] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:16 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:17.393492 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d62c8a8b-c21d-4c3d-98d7-80c8f92bb14a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:17.394089 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d62c8a8b-c21d-4c3d-98d7-80c8f92bb14a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:17.401361 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d62c8a8b-c21d-4c3d-98d7-80c8f92bb14a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:17.401847 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d62c8a8b-c21d-4c3d-98d7-80c8f92bb14a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:17.402282 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d62c8a8b-c21d-4c3d-98d7-80c8f92bb14a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:17.402982 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d62c8a8b-c21d-4c3d-98d7-80c8f92bb14a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:17.403437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d62c8a8b-c21d-4c3d-98d7-80c8f92bb14a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:17.403858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d62c8a8b-c21d-4c3d-98d7-80c8f92bb14a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:17.589139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d62c8a8b-c21d-4c3d-98d7-80c8f92bb14a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:17.597917 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d62c8a8b-c21d-4c3d-98d7-80c8f92bb14a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.208973 Sep 27 11:18:17.598517 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3956/7436] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:17 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:18.622222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3af06e83-f73e-446e-b38e-adbae51b2d21 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:18.622876 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3af06e83-f73e-446e-b38e-adbae51b2d21 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:18.630287 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3af06e83-f73e-446e-b38e-adbae51b2d21 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:18.630787 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3af06e83-f73e-446e-b38e-adbae51b2d21 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:18.631238 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3af06e83-f73e-446e-b38e-adbae51b2d21 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:18.631949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3af06e83-f73e-446e-b38e-adbae51b2d21 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:18.632416 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3af06e83-f73e-446e-b38e-adbae51b2d21 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:18.632890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3af06e83-f73e-446e-b38e-adbae51b2d21 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:18.828857 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3af06e83-f73e-446e-b38e-adbae51b2d21 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:18.839667 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3af06e83-f73e-446e-b38e-adbae51b2d21 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.221212 Sep 27 11:18:18.840500 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3481/7437] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:18 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:19.865996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-fc9b6ff2-6529-4f4f-96bb-e4bf433e45a9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:19.866540 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-fc9b6ff2-6529-4f4f-96bb-e4bf433e45a9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:19.874079 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9b6ff2-6529-4f4f-96bb-e4bf433e45a9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:19.874575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9b6ff2-6529-4f4f-96bb-e4bf433e45a9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:19.875019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9b6ff2-6529-4f4f-96bb-e4bf433e45a9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:19.875738 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9b6ff2-6529-4f4f-96bb-e4bf433e45a9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:19.876232 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9b6ff2-6529-4f4f-96bb-e4bf433e45a9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:19.876683 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-fc9b6ff2-6529-4f4f-96bb-e4bf433e45a9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:20.014413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-fc9b6ff2-6529-4f4f-96bb-e4bf433e45a9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:20.023641 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-fc9b6ff2-6529-4f4f-96bb-e4bf433e45a9 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.161747 Sep 27 11:18:20.024247 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3957/7438] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:19 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:21.048064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-6d3858dd-5fe5-40fe-9031-ef231538fc90 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:21.048809 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-6d3858dd-5fe5-40fe-9031-ef231538fc90 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:21.055360 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d3858dd-5fe5-40fe-9031-ef231538fc90 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:21.055867 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d3858dd-5fe5-40fe-9031-ef231538fc90 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:21.056324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d3858dd-5fe5-40fe-9031-ef231538fc90 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:21.057082 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d3858dd-5fe5-40fe-9031-ef231538fc90 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:21.057558 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d3858dd-5fe5-40fe-9031-ef231538fc90 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:21.058003 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-6d3858dd-5fe5-40fe-9031-ef231538fc90 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:21.197580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-6d3858dd-5fe5-40fe-9031-ef231538fc90 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:21.206103 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-6d3858dd-5fe5-40fe-9031-ef231538fc90 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.162467 Sep 27 11:18:21.206690 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3482/7439] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:21 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:22.233915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-59028501-4c39-483e-afaa-58aaf2438442 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:22.235014 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-59028501-4c39-483e-afaa-58aaf2438442 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:22.243105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59028501-4c39-483e-afaa-58aaf2438442 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:22.243611 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59028501-4c39-483e-afaa-58aaf2438442 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:22.244033 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59028501-4c39-483e-afaa-58aaf2438442 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:22.244767 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59028501-4c39-483e-afaa-58aaf2438442 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:22.245223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59028501-4c39-483e-afaa-58aaf2438442 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:22.245650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-59028501-4c39-483e-afaa-58aaf2438442 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:22.390896 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-59028501-4c39-483e-afaa-58aaf2438442 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:22.399927 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-59028501-4c39-483e-afaa-58aaf2438442 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1510 microversion: 2.1 time: 0.172722 Sep 27 11:18:22.400505 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3958/7440] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:22 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1510 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:22.422784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:18:22.423405 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:22.431054 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:22.431565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:22.432020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:22.432782 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:22.433263 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:22.433715 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:22.517736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=145969) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 27 11:18:22.600086 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-c46dbd23-8cd3-4a93-9438-6c56d0a31994 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "POST /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11/action" status: 202 len: 0 microversion: 2.1 time: 0.181361 Sep 27 11:18:22.601125 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3483/7441] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:18:22 2022] POST /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11/action => generated 0 bytes in 183 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:18:22.609561 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-430d3734-c6c7-437d-b696-435366b0373a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:22.611080 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-430d3734-c6c7-437d-b696-435366b0373a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:22.617488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430d3734-c6c7-437d-b696-435366b0373a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:22.617980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430d3734-c6c7-437d-b696-435366b0373a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:22.618423 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430d3734-c6c7-437d-b696-435366b0373a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:22.619211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430d3734-c6c7-437d-b696-435366b0373a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:22.619671 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430d3734-c6c7-437d-b696-435366b0373a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:22.620100 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-430d3734-c6c7-437d-b696-435366b0373a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:22.812445 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-430d3734-c6c7-437d-b696-435366b0373a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:22.821667 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-430d3734-c6c7-437d-b696-435366b0373a tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1518 microversion: 2.1 time: 0.215996 Sep 27 11:18:22.822266 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3959/7442] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:22 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1518 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:23.847583 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8faf3440-9c1d-4fe2-bc9b-2c1cd0447cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:23.848222 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8faf3440-9c1d-4fe2-bc9b-2c1cd0447cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:23.854828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8faf3440-9c1d-4fe2-bc9b-2c1cd0447cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:23.855334 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8faf3440-9c1d-4fe2-bc9b-2c1cd0447cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:23.855789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8faf3440-9c1d-4fe2-bc9b-2c1cd0447cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:23.856575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8faf3440-9c1d-4fe2-bc9b-2c1cd0447cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:23.857021 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8faf3440-9c1d-4fe2-bc9b-2c1cd0447cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:23.857448 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8faf3440-9c1d-4fe2-bc9b-2c1cd0447cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:23.994253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-8faf3440-9c1d-4fe2-bc9b-2c1cd0447cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:24.002394 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8faf3440-9c1d-4fe2-bc9b-2c1cd0447cea tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1520 microversion: 2.1 time: 0.159980 Sep 27 11:18:24.003533 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3484/7443] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:23 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1520 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:25.027601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-64c74940-88d6-4308-905f-25f1a2e596c5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:25.028203 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-64c74940-88d6-4308-905f-25f1a2e596c5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:25.035218 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64c74940-88d6-4308-905f-25f1a2e596c5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:25.035695 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64c74940-88d6-4308-905f-25f1a2e596c5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:25.036147 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64c74940-88d6-4308-905f-25f1a2e596c5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:25.036908 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64c74940-88d6-4308-905f-25f1a2e596c5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:25.037365 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64c74940-88d6-4308-905f-25f1a2e596c5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:25.037806 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-64c74940-88d6-4308-905f-25f1a2e596c5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:25.261705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-64c74940-88d6-4308-905f-25f1a2e596c5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:25.269960 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-64c74940-88d6-4308-905f-25f1a2e596c5 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1576 microversion: 2.1 time: 0.246402 Sep 27 11:18:25.270857 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3960/7444] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:25 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1576 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:25.355761 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-b4d75acb-17c8-4e10-8581-84b13beaf15e req-3f7c2971-072f-4b11-98bd-3fb3af715534 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f793bbd-0d6e-434e-a84f-552abcb0ba11", "name": "network-vif-unplugged", "status": "completed", "tag": "128e0956-38c1-48fc-b832-009f7e0de308"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:18:25.369184 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b4d75acb-17c8-4e10-8581-84b13beaf15e req-3f7c2971-072f-4b11-98bd-3fb3af715534 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:25.369699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b4d75acb-17c8-4e10-8581-84b13beaf15e req-3f7c2971-072f-4b11-98bd-3fb3af715534 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:25.370125 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-b4d75acb-17c8-4e10-8581-84b13beaf15e req-3f7c2971-072f-4b11-98bd-3fb3af715534 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:25.382106 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-b4d75acb-17c8-4e10-8581-84b13beaf15e req-3f7c2971-072f-4b11-98bd-3fb3af715534 service nova] Creating event network-vif-unplugged:128e0956-38c1-48fc-b832-009f7e0de308 for instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 on nch2-master-1 Sep 27 11:18:25.387684 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-b4d75acb-17c8-4e10-8581-84b13beaf15e req-3f7c2971-072f-4b11-98bd-3fb3af715534 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035763 Sep 27 11:18:25.388387 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3485/7445] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:18:25 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:18:26.294041 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ad94e4a8-44a8-4a4a-9d6c-393da565b678 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:26.294662 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ad94e4a8-44a8-4a4a-9d6c-393da565b678 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:26.302364 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad94e4a8-44a8-4a4a-9d6c-393da565b678 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:26.302856 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad94e4a8-44a8-4a4a-9d6c-393da565b678 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:26.305531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad94e4a8-44a8-4a4a-9d6c-393da565b678 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:26.305531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad94e4a8-44a8-4a4a-9d6c-393da565b678 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:26.305531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad94e4a8-44a8-4a4a-9d6c-393da565b678 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:26.305531 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ad94e4a8-44a8-4a4a-9d6c-393da565b678 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:26.518703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ad94e4a8-44a8-4a4a-9d6c-393da565b678 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:26.530326 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ad94e4a8-44a8-4a4a-9d6c-393da565b678 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1576 microversion: 2.1 time: 0.240292 Sep 27 11:18:26.530910 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3961/7446] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:26 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1576 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:27.418949 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-7a338f4f-90e6-48fd-a408-ef982f8fd594 req-0631da87-c240-4221-ba7d-503aac4870b9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4f793bbd-0d6e-434e-a84f-552abcb0ba11", "tag": "128e0956-38c1-48fc-b832-009f7e0de308"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:18:27.430956 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7a338f4f-90e6-48fd-a408-ef982f8fd594 req-0631da87-c240-4221-ba7d-503aac4870b9 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:27.431458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7a338f4f-90e6-48fd-a408-ef982f8fd594 req-0631da87-c240-4221-ba7d-503aac4870b9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:27.431913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-7a338f4f-90e6-48fd-a408-ef982f8fd594 req-0631da87-c240-4221-ba7d-503aac4870b9 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:27.443481 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-7a338f4f-90e6-48fd-a408-ef982f8fd594 req-0631da87-c240-4221-ba7d-503aac4870b9 service nova] Creating event network-changed:128e0956-38c1-48fc-b832-009f7e0de308 for instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 on nch2-master-1 Sep 27 11:18:27.448707 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-7a338f4f-90e6-48fd-a408-ef982f8fd594 req-0631da87-c240-4221-ba7d-503aac4870b9 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036775 Sep 27 11:18:27.449337 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3486/7447] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:18:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:18:27.554705 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4995af7f-e928-45f7-b3bd-4d29259d8dec tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:27.555323 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4995af7f-e928-45f7-b3bd-4d29259d8dec tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:27.562667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4995af7f-e928-45f7-b3bd-4d29259d8dec tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:27.563157 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4995af7f-e928-45f7-b3bd-4d29259d8dec tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:27.563591 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4995af7f-e928-45f7-b3bd-4d29259d8dec tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:27.564289 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4995af7f-e928-45f7-b3bd-4d29259d8dec tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:27.564785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4995af7f-e928-45f7-b3bd-4d29259d8dec tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:27.565211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4995af7f-e928-45f7-b3bd-4d29259d8dec tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:27.710680 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4995af7f-e928-45f7-b3bd-4d29259d8dec tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:27.719837 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4995af7f-e928-45f7-b3bd-4d29259d8dec tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1576 microversion: 2.1 time: 0.169108 Sep 27 11:18:27.720427 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3962/7448] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:27 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1576 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:28.751317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-902a6e00-48fe-4a3c-bb9d-56c0e961a2de tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:28.752502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-902a6e00-48fe-4a3c-bb9d-56c0e961a2de tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:28.760037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-902a6e00-48fe-4a3c-bb9d-56c0e961a2de tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:28.760541 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-902a6e00-48fe-4a3c-bb9d-56c0e961a2de tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:28.761026 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-902a6e00-48fe-4a3c-bb9d-56c0e961a2de tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:28.761771 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-902a6e00-48fe-4a3c-bb9d-56c0e961a2de tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:28.762245 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-902a6e00-48fe-4a3c-bb9d-56c0e961a2de tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:28.762684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-902a6e00-48fe-4a3c-bb9d-56c0e961a2de tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:28.906613 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-902a6e00-48fe-4a3c-bb9d-56c0e961a2de tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:28.915203 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-902a6e00-48fe-4a3c-bb9d-56c0e961a2de tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1576 microversion: 2.1 time: 0.172682 Sep 27 11:18:28.915791 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3487/7449] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:28 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1576 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:29.943048 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-6885d55f-a00b-49ae-94fc-827ee8f1c1a6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:29.943687 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-6885d55f-a00b-49ae-94fc-827ee8f1c1a6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:29.951247 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6885d55f-a00b-49ae-94fc-827ee8f1c1a6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:29.951764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6885d55f-a00b-49ae-94fc-827ee8f1c1a6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:29.952172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6885d55f-a00b-49ae-94fc-827ee8f1c1a6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:29.952946 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6885d55f-a00b-49ae-94fc-827ee8f1c1a6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:29.953439 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6885d55f-a00b-49ae-94fc-827ee8f1c1a6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:29.953872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-6885d55f-a00b-49ae-94fc-827ee8f1c1a6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:30.101087 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-6885d55f-a00b-49ae-94fc-827ee8f1c1a6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:30.109562 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-6885d55f-a00b-49ae-94fc-827ee8f1c1a6 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1576 microversion: 2.1 time: 0.172875 Sep 27 11:18:30.110151 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3963/7450] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:29 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1576 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:31.136839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-42f3355c-b874-48ee-87c2-864835ec81a2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:31.137594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-42f3355c-b874-48ee-87c2-864835ec81a2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:31.147412 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42f3355c-b874-48ee-87c2-864835ec81a2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:31.147911 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42f3355c-b874-48ee-87c2-864835ec81a2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:31.148362 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42f3355c-b874-48ee-87c2-864835ec81a2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:31.149112 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42f3355c-b874-48ee-87c2-864835ec81a2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:31.149581 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42f3355c-b874-48ee-87c2-864835ec81a2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:31.150020 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-42f3355c-b874-48ee-87c2-864835ec81a2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:31.292250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-42f3355c-b874-48ee-87c2-864835ec81a2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:31.300287 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-42f3355c-b874-48ee-87c2-864835ec81a2 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1576 microversion: 2.1 time: 0.168804 Sep 27 11:18:31.300845 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3488/7451] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:31 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1576 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:32.160469 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-934bee9b-eab4-4dc3-a467-52ec9c98ec57 req-e3545158-f82e-43b5-a225-cc4f37933994 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f793bbd-0d6e-434e-a84f-552abcb0ba11", "name": "network-vif-plugged", "status": "completed", "tag": "128e0956-38c1-48fc-b832-009f7e0de308"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:18:32.171944 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-934bee9b-eab4-4dc3-a467-52ec9c98ec57 req-e3545158-f82e-43b5-a225-cc4f37933994 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:32.172517 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-934bee9b-eab4-4dc3-a467-52ec9c98ec57 req-e3545158-f82e-43b5-a225-cc4f37933994 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:32.173020 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-934bee9b-eab4-4dc3-a467-52ec9c98ec57 req-e3545158-f82e-43b5-a225-cc4f37933994 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:32.186016 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-934bee9b-eab4-4dc3-a467-52ec9c98ec57 req-e3545158-f82e-43b5-a225-cc4f37933994 service nova] Creating event network-vif-plugged:128e0956-38c1-48fc-b832-009f7e0de308 for instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 on nch2-master-1 Sep 27 11:18:32.191252 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-934bee9b-eab4-4dc3-a467-52ec9c98ec57 req-e3545158-f82e-43b5-a225-cc4f37933994 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034684 Sep 27 11:18:32.192560 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3964/7452] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:18:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:18:32.323698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7fadb8ba-e7f4-40b0-9bb4-ad6e401c753b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:32.324291 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7fadb8ba-e7f4-40b0-9bb4-ad6e401c753b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:32.331734 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fadb8ba-e7f4-40b0-9bb4-ad6e401c753b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:32.332244 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fadb8ba-e7f4-40b0-9bb4-ad6e401c753b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:32.332759 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fadb8ba-e7f4-40b0-9bb4-ad6e401c753b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:32.333508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fadb8ba-e7f4-40b0-9bb4-ad6e401c753b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:32.333985 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fadb8ba-e7f4-40b0-9bb4-ad6e401c753b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:32.334425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7fadb8ba-e7f4-40b0-9bb4-ad6e401c753b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:32.471735 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-7fadb8ba-e7f4-40b0-9bb4-ad6e401c753b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:32.480041 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7fadb8ba-e7f4-40b0-9bb4-ad6e401c753b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1576 microversion: 2.1 time: 0.160286 Sep 27 11:18:32.480575 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3489/7453] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:32 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1576 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:33.509866 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-16cd7124-11c9-463d-a5f0-288ce33ac195 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:33.510870 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-16cd7124-11c9-463d-a5f0-288ce33ac195 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:33.521798 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cd7124-11c9-463d-a5f0-288ce33ac195 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:33.522287 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cd7124-11c9-463d-a5f0-288ce33ac195 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:33.522725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cd7124-11c9-463d-a5f0-288ce33ac195 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:33.523458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cd7124-11c9-463d-a5f0-288ce33ac195 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:33.523919 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cd7124-11c9-463d-a5f0-288ce33ac195 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:33.524395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-16cd7124-11c9-463d-a5f0-288ce33ac195 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:33.673494 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-16cd7124-11c9-463d-a5f0-288ce33ac195 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:33.687756 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-16cd7124-11c9-463d-a5f0-288ce33ac195 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1576 microversion: 2.1 time: 0.186408 Sep 27 11:18:33.688630 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3965/7454] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:33 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1576 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:18:34.718353 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-99fb1c11-7e37-454c-9626-195f96e9690b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:18:34.719406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-99fb1c11-7e37-454c-9626-195f96e9690b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:18:34.729110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99fb1c11-7e37-454c-9626-195f96e9690b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:34.729616 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99fb1c11-7e37-454c-9626-195f96e9690b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:34.730069 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99fb1c11-7e37-454c-9626-195f96e9690b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:34.730784 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99fb1c11-7e37-454c-9626-195f96e9690b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:18:34.731258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99fb1c11-7e37-454c-9626-195f96e9690b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:18:34.731699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-99fb1c11-7e37-454c-9626-195f96e9690b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:18:34.872890 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-99fb1c11-7e37-454c-9626-195f96e9690b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:18:34.885950 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-99fb1c11-7e37-454c-9626-195f96e9690b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1563 microversion: 2.1 time: 0.173947 Sep 27 11:18:34.886811 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3490/7455] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:18:34 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1563 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:19:59.304626 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=28 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:19:59.311117 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-2e412121-d51f-4393-b9d2-0becb3cde58a req-8888ff6d-bcab-48d1-8ce6-ee10f7abb744 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4f793bbd-0d6e-434e-a84f-552abcb0ba11", "tag": "128e0956-38c1-48fc-b832-009f7e0de308"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:19:59.313071 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:19:59.314361 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:19:59.314731 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 11:19:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 (ip 10.222.0.26) !!! Sep 27 11:19:59.319288 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=32,nova_api:UPDATE=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:19:59.320068 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=80,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:19:59.321615 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:19:59.321751 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 11:19:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.26) !!! Sep 27 11:19:59.322204 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a329da7f-f36c-40a4-8f64-d850ae537a88 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:19:59.324069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a329da7f-f36c-40a4-8f64-d850ae537a88 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:19:59.331657 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2e412121-d51f-4393-b9d2-0becb3cde58a req-8888ff6d-bcab-48d1-8ce6-ee10f7abb744 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:19:59.332214 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2e412121-d51f-4393-b9d2-0becb3cde58a req-8888ff6d-bcab-48d1-8ce6-ee10f7abb744 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:19:59.332623 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-2e412121-d51f-4393-b9d2-0becb3cde58a req-8888ff6d-bcab-48d1-8ce6-ee10f7abb744 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:19:59.334686 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a329da7f-f36c-40a4-8f64-d850ae537a88 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:19:59.335355 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a329da7f-f36c-40a4-8f64-d850ae537a88 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:19:59.335951 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a329da7f-f36c-40a4-8f64-d850ae537a88 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:19:59.337000 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a329da7f-f36c-40a4-8f64-d850ae537a88 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:19:59.337467 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a329da7f-f36c-40a4-8f64-d850ae537a88 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:19:59.337895 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a329da7f-f36c-40a4-8f64-d850ae537a88 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:19:59.347637 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-2e412121-d51f-4393-b9d2-0becb3cde58a req-8888ff6d-bcab-48d1-8ce6-ee10f7abb744 service nova] Creating event network-changed:128e0956-38c1-48fc-b832-009f7e0de308 for instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 on nch2-master-1 Sep 27 11:19:59.353019 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-2e412121-d51f-4393-b9d2-0becb3cde58a req-8888ff6d-bcab-48d1-8ce6-ee10f7abb744 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046400 Sep 27 11:19:59.354008 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3491/7456] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:19:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:19:59.377622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a329da7f-f36c-40a4-8f64-d850ae537a88 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 11:19:59.472229 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a329da7f-f36c-40a4-8f64-d850ae537a88 tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "DELETE /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 204 len: 0 microversion: 2.1 time: 0.169523 Sep 27 11:19:59.477884 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3966/7457] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 11:19:59 2022] DELETE /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 11:19:59.482582 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-49809949-f6c7-497e-9412-b2e022ef598b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:19:59.483431 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-49809949-f6c7-497e-9412-b2e022ef598b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:19:59.491127 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49809949-f6c7-497e-9412-b2e022ef598b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:19:59.491868 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49809949-f6c7-497e-9412-b2e022ef598b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:19:59.492327 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49809949-f6c7-497e-9412-b2e022ef598b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:19:59.493091 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49809949-f6c7-497e-9412-b2e022ef598b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:19:59.493776 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49809949-f6c7-497e-9412-b2e022ef598b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:19:59.494427 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-49809949-f6c7-497e-9412-b2e022ef598b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:19:59.644121 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-49809949-f6c7-497e-9412-b2e022ef598b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:19:59.652673 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-49809949-f6c7-497e-9412-b2e022ef598b tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.174635 Sep 27 11:19:59.653252 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3492/7458] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:19:59 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:00.678557 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ea05e0fc-352a-4af4-a7b6-2fd1a0aac9fe tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:00.679258 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ea05e0fc-352a-4af4-a7b6-2fd1a0aac9fe tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:00.687073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea05e0fc-352a-4af4-a7b6-2fd1a0aac9fe tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:00.687575 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea05e0fc-352a-4af4-a7b6-2fd1a0aac9fe tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:00.688152 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea05e0fc-352a-4af4-a7b6-2fd1a0aac9fe tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:00.688901 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea05e0fc-352a-4af4-a7b6-2fd1a0aac9fe tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:00.689357 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea05e0fc-352a-4af4-a7b6-2fd1a0aac9fe tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:00.689811 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ea05e0fc-352a-4af4-a7b6-2fd1a0aac9fe tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:00.904401 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ea05e0fc-352a-4af4-a7b6-2fd1a0aac9fe tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:00.913240 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ea05e0fc-352a-4af4-a7b6-2fd1a0aac9fe tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1569 microversion: 2.1 time: 0.239146 Sep 27 11:20:00.913833 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3967/7459] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:00 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1569 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:01.865602 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-a98e7959-a93b-42ba-92bc-4a9ee4fcc294 req-0f1492bc-24e3-4b16-9e30-c889fc76b628 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4f793bbd-0d6e-434e-a84f-552abcb0ba11", "name": "network-vif-deleted", "tag": "128e0956-38c1-48fc-b832-009f7e0de308"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:20:01.877144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a98e7959-a93b-42ba-92bc-4a9ee4fcc294 req-0f1492bc-24e3-4b16-9e30-c889fc76b628 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:01.877669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a98e7959-a93b-42ba-92bc-4a9ee4fcc294 req-0f1492bc-24e3-4b16-9e30-c889fc76b628 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:01.878119 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-a98e7959-a93b-42ba-92bc-4a9ee4fcc294 req-0f1492bc-24e3-4b16-9e30-c889fc76b628 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:01.889954 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-a98e7959-a93b-42ba-92bc-4a9ee4fcc294 req-0f1492bc-24e3-4b16-9e30-c889fc76b628 service nova] Creating event network-vif-deleted:128e0956-38c1-48fc-b832-009f7e0de308 for instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 on nch2-master-1 Sep 27 11:20:01.895542 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-a98e7959-a93b-42ba-92bc-4a9ee4fcc294 req-0f1492bc-24e3-4b16-9e30-c889fc76b628 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033964 Sep 27 11:20:01.896175 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3493/7460] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:20:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:20:01.937319 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-a1fa6ec0-e2ea-48ee-8a9f-b776ff4f23fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:01.937927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-a1fa6ec0-e2ea-48ee-8a9f-b776ff4f23fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:01.945382 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fa6ec0-e2ea-48ee-8a9f-b776ff4f23fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:01.945872 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fa6ec0-e2ea-48ee-8a9f-b776ff4f23fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:01.946306 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fa6ec0-e2ea-48ee-8a9f-b776ff4f23fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:01.947010 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fa6ec0-e2ea-48ee-8a9f-b776ff4f23fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:01.947466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fa6ec0-e2ea-48ee-8a9f-b776ff4f23fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:01.947871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-a1fa6ec0-e2ea-48ee-8a9f-b776ff4f23fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:02.015725 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-a1fa6ec0-e2ea-48ee-8a9f-b776ff4f23fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '90cea8d7a5494c45ba299df59797633f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37db46b0eb6a4c7d938085b26d1968a7', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:02.025924 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-a1fa6ec0-e2ea-48ee-8a9f-b776ff4f23fc tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 200 len: 1219 microversion: 2.1 time: 0.092857 Sep 27 11:20:02.026511 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3968/7461] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:01 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 1219 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:03.055111 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:03.055878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] [instance: 4f793bbd-0d6e-434e-a84f-552abcb0ba11] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:03.065779 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:03.066495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:03.067108 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:03.068129 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:03.068833 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:03.069466 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:03.085972 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] HTTP exception thrown: Instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 could not be found. Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Request method failure captured: Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 HTTP/1.1 Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtpFyEAsTKQK_R_zoAYTu93KsHs-d1Iek0-OmH4eIKijvQ5I1NezJJErnSobjKpmSWEZly8UgWWnEWzglcHZKuHALei7PJwLQaLPf5EvRMNQIOGYRRk0WU40xS67gdP__QQAI1rLvn0iUt7UvDT1P63h9K9awLpalGDygX0M2F7mmlg Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 37db46b0eb6a4c7d938085b26d1968a7 Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestShelveInstance-666939746 Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37db46b0eb6a4c7d938085b26d1968a7"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37db46b0eb6a4c7d938085b26d1968a7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/37db46b0eb6a4c7d938085b26d1968a7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_37db46b0eb6a4c7d938085b26d1968a7"}]}] Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestShelveInstance-666939746 Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 37db46b0eb6a4c7d938085b26d1968a7 Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestShelveInstance-666939746 Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestShelveInstance-666939746-project Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 90cea8d7a5494c45ba299df59797633f Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestShelveInstance-666939746-project Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 could not be found. Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '4f793bbd-0d6e-434e-a84f-552abcb0ba11'} Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 could not be found. Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 could not be found. Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:20:03.087643 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:20:03.091449 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:20:03.091449 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:20:03.091449 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:20:03.091449 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 11:20:03.091449 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 11:20:03.091449 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 could not be found. Sep 27 11:20:03.091449 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:20:03.091449 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Returning 404 to user: Instance 4f793bbd-0d6e-434e-a84f-552abcb0ba11 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 11:20:03.091449 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ca08f30c-a4d0-447b-b18e-70a19e2db30d tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11" status: 404 len: 111 microversion: 2.1 time: 0.041713 Sep 27 11:20:03.091449 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3494/7462] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:03 2022] GET /compute/v2.1/servers/4f793bbd-0d6e-434e-a84f-552abcb0ba11 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 11:20:03.261074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-893358d4-3aec-434f-8a73-4e54364f0a4e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] Calling method '.version_select at 0x7f7f6c178700>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:03.291635 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-893358d4-3aec-434f-8a73-4e54364f0a4e tempest-TestShelveInstance-666939746 tempest-TestShelveInstance-666939746-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-128583505" status: 202 len: 0 microversion: 2.1 time: 0.035162 Sep 27 11:20:03.292438 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3969/7463] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 11:20:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-128583505 => generated 0 bytes in 37 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 27 11:20:21.086866 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=8 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:21.088594 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=12 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:21.312079 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7f0e4d31-e0be-4363-b13f-ca387818a012 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Action: 'create', calling method: .version_select at 0x7f7f6c071ee0>, body: {"keypair": {"name": "tempest-TestSnapshotPattern-83732341", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGQzJoOZa8I88d1DYzlinOZ7GzoDMcX09gBxK2L242oXnkVBkyQGRuVANEB9E+YQTC+AK47RK3CtTvLus7oLeZ0v8OA+Ss7z4KMRVHW5mhm4oiHEiMwg5WSEr8+l4gN6+Q=="}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:20:21.329580 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7f0e4d31-e0be-4363-b13f-ca387818a012 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Getting quotas for project 34d2d2279cd14736ba71123e751c9aeb. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:20:21.336187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.quota [None req-7f0e4d31-e0be-4363-b13f-ca387818a012 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Getting quotas for user 8bf66c7321304c91840efc2cbeb52a4f and project 34d2d2279cd14736ba71123e751c9aeb. Resources: {'key_pairs'} {{(pid=145969) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:20:21.366743 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7f0e4d31-e0be-4363-b13f-ca387818a012 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.287523 Sep 27 11:20:21.367325 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3495/7464] 10.222.0.26 () {60 vars in 1159 bytes} [Tue Sep 27 11:20:21 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:20:22.417568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=26,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:22.418829 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=14,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:22.424084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestSnapshotPattern-server-521740738", "imageRef": "c1623d3a-c1bb-4649-9910-5deadb8a1ff6", "flavorRef": "452", "key_name": "tempest-TestSnapshotPattern-83732341", "security_groups": [{"name": "tempest-secgroup-smoke-1847565083"}], "networks": [{"uuid": "8313513f-6210-4a72-a6b6-882b7b1b37be"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:20:22.541980 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] validate_networks() for [('8313513f-6210-4a72-a6b6-882b7b1b37be', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 11:20:23.047299 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 11:20:23.047733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 11:20:23.048220 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 11:20:23.048634 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 11:20:23.049088 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 11:20:23.058222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:23.058700 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:23.059132 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:23.059859 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:23.060310 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:23.060778 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:23.079224 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Getting quotas for project 34d2d2279cd14736ba71123e751c9aeb. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:20:23.084774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Getting quotas for user 8bf66c7321304c91840efc2cbeb52a4f and project 34d2d2279cd14736ba71123e751c9aeb. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:20:23.098194 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 11:20:23.100682 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='c1623d3a-c1bb-4649-9910-5deadb8a1ff6',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 11:20:23.153285 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-86eca594-1ef0-4add-9f71-e39494981524 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.737642 Sep 27 11:20:23.153964 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3970/7465] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 11:20:22 2022] POST /compute/v2.1/servers => generated 406 bytes in 739 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 27 11:20:23.166339 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-68f30154-210d-4cde-afb5-6124fdf14055 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:23.166950 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-68f30154-210d-4cde-afb5-6124fdf14055 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:23.225406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-68f30154-210d-4cde-afb5-6124fdf14055 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:23.245449 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-68f30154-210d-4cde-afb5-6124fdf14055 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1246 microversion: 2.1 time: 0.083826 Sep 27 11:20:23.246441 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3496/7466] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:23 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1246 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:24.273934 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-04b5f21e-b261-4c4a-8c6a-65feca1dc6a6 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:24.275555 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-04b5f21e-b261-4c4a-8c6a-65feca1dc6a6 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:24.289356 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04b5f21e-b261-4c4a-8c6a-65feca1dc6a6 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:24.290300 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04b5f21e-b261-4c4a-8c6a-65feca1dc6a6 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:24.290968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04b5f21e-b261-4c4a-8c6a-65feca1dc6a6 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:24.291975 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04b5f21e-b261-4c4a-8c6a-65feca1dc6a6 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:24.292685 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04b5f21e-b261-4c4a-8c6a-65feca1dc6a6 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:24.293585 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-04b5f21e-b261-4c4a-8c6a-65feca1dc6a6 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:24.384508 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-04b5f21e-b261-4c4a-8c6a-65feca1dc6a6 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:24.398897 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-04b5f21e-b261-4c4a-8c6a-65feca1dc6a6 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1306 microversion: 2.1 time: 0.131970 Sep 27 11:20:24.401283 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3971/7467] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:24 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1306 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:25.424912 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-13368662-4f98-4c02-a4df-f487e6af06d0 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:25.425869 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-13368662-4f98-4c02-a4df-f487e6af06d0 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:25.437736 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13368662-4f98-4c02-a4df-f487e6af06d0 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:25.438322 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13368662-4f98-4c02-a4df-f487e6af06d0 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:25.438839 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13368662-4f98-4c02-a4df-f487e6af06d0 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:25.439789 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13368662-4f98-4c02-a4df-f487e6af06d0 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:25.440324 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13368662-4f98-4c02-a4df-f487e6af06d0 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:25.441000 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-13368662-4f98-4c02-a4df-f487e6af06d0 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:25.665275 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-13368662-4f98-4c02-a4df-f487e6af06d0 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:25.673806 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-13368662-4f98-4c02-a4df-f487e6af06d0 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1372 microversion: 2.1 time: 0.253069 Sep 27 11:20:25.674898 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3497/7468] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:25 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1372 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:25.845670 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-147bb1bd-3ca0-4f35-8cbb-8ed85066b0f9 req-e6eeff34-0dc1-4bbe-8d1d-293dc1f090e5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7dda007a-6f7c-4b57-a910-480eedbf967f", "tag": "da201a4d-39e0-43c4-b336-8ccd6da2e34e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:20:25.865978 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-147bb1bd-3ca0-4f35-8cbb-8ed85066b0f9 req-e6eeff34-0dc1-4bbe-8d1d-293dc1f090e5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:25.866652 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-147bb1bd-3ca0-4f35-8cbb-8ed85066b0f9 req-e6eeff34-0dc1-4bbe-8d1d-293dc1f090e5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:25.867238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-147bb1bd-3ca0-4f35-8cbb-8ed85066b0f9 req-e6eeff34-0dc1-4bbe-8d1d-293dc1f090e5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:25.880577 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-147bb1bd-3ca0-4f35-8cbb-8ed85066b0f9 req-e6eeff34-0dc1-4bbe-8d1d-293dc1f090e5 service nova] Creating event network-changed:da201a4d-39e0-43c4-b336-8ccd6da2e34e for instance 7dda007a-6f7c-4b57-a910-480eedbf967f on nch1-master-1 Sep 27 11:20:25.886475 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-147bb1bd-3ca0-4f35-8cbb-8ed85066b0f9 req-e6eeff34-0dc1-4bbe-8d1d-293dc1f090e5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044902 Sep 27 11:20:25.887275 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3972/7469] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:20:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:20:26.697712 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-cc7090fc-da78-45bc-a4ce-9879bff4f157 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:26.698348 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-cc7090fc-da78-45bc-a4ce-9879bff4f157 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:26.706402 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc7090fc-da78-45bc-a4ce-9879bff4f157 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:26.706926 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc7090fc-da78-45bc-a4ce-9879bff4f157 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:26.707418 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc7090fc-da78-45bc-a4ce-9879bff4f157 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:26.708174 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc7090fc-da78-45bc-a4ce-9879bff4f157 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:26.708702 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc7090fc-da78-45bc-a4ce-9879bff4f157 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:26.709143 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-cc7090fc-da78-45bc-a4ce-9879bff4f157 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:26.855037 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-cc7090fc-da78-45bc-a4ce-9879bff4f157 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:26.868369 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-cc7090fc-da78-45bc-a4ce-9879bff4f157 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1372 microversion: 2.1 time: 0.174437 Sep 27 11:20:26.869194 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3498/7470] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:26 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1372 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:27.896386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-ec4a037e-f79c-4f67-93f3-eb88167ea547 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:27.897112 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-ec4a037e-f79c-4f67-93f3-eb88167ea547 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:27.903900 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec4a037e-f79c-4f67-93f3-eb88167ea547 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:27.904407 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec4a037e-f79c-4f67-93f3-eb88167ea547 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:27.904875 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec4a037e-f79c-4f67-93f3-eb88167ea547 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:27.905729 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec4a037e-f79c-4f67-93f3-eb88167ea547 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:27.906191 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec4a037e-f79c-4f67-93f3-eb88167ea547 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:27.906623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-ec4a037e-f79c-4f67-93f3-eb88167ea547 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:28.044103 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-ec4a037e-f79c-4f67-93f3-eb88167ea547 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:28.052504 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-ec4a037e-f79c-4f67-93f3-eb88167ea547 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1372 microversion: 2.1 time: 0.161734 Sep 27 11:20:28.053126 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3973/7471] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:27 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1372 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:29.075764 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0a0289f1-c92e-4796-887e-d4ada3691054 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:29.076363 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0a0289f1-c92e-4796-887e-d4ada3691054 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:29.083973 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a0289f1-c92e-4796-887e-d4ada3691054 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:29.084495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a0289f1-c92e-4796-887e-d4ada3691054 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:29.084976 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a0289f1-c92e-4796-887e-d4ada3691054 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:29.085699 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a0289f1-c92e-4796-887e-d4ada3691054 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:29.086171 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a0289f1-c92e-4796-887e-d4ada3691054 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:29.086609 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0a0289f1-c92e-4796-887e-d4ada3691054 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:29.243265 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0a0289f1-c92e-4796-887e-d4ada3691054 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:29.251783 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0a0289f1-c92e-4796-887e-d4ada3691054 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1372 microversion: 2.1 time: 0.179966 Sep 27 11:20:29.252393 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3499/7472] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:29 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1372 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:30.275774 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8ca432b0-50a2-45df-83d7-d7c7d97669e9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:30.276354 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8ca432b0-50a2-45df-83d7-d7c7d97669e9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:30.283853 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ca432b0-50a2-45df-83d7-d7c7d97669e9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:30.284348 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ca432b0-50a2-45df-83d7-d7c7d97669e9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:30.284814 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ca432b0-50a2-45df-83d7-d7c7d97669e9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:30.285526 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ca432b0-50a2-45df-83d7-d7c7d97669e9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:30.285977 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ca432b0-50a2-45df-83d7-d7c7d97669e9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:30.286397 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8ca432b0-50a2-45df-83d7-d7c7d97669e9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:30.426076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8ca432b0-50a2-45df-83d7-d7c7d97669e9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:30.434260 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8ca432b0-50a2-45df-83d7-d7c7d97669e9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1372 microversion: 2.1 time: 0.162517 Sep 27 11:20:30.434812 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3974/7473] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:30 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1372 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:30.795577 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-45a667a3-c0e9-4c32-8ff6-6f4e17e78dd8 req-84011096-cfdc-4741-9c88-b7dc26e6eae4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7dda007a-6f7c-4b57-a910-480eedbf967f", "name": "network-vif-plugged", "status": "completed", "tag": "da201a4d-39e0-43c4-b336-8ccd6da2e34e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:20:30.807621 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-45a667a3-c0e9-4c32-8ff6-6f4e17e78dd8 req-84011096-cfdc-4741-9c88-b7dc26e6eae4 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:30.808168 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-45a667a3-c0e9-4c32-8ff6-6f4e17e78dd8 req-84011096-cfdc-4741-9c88-b7dc26e6eae4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:30.808617 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-45a667a3-c0e9-4c32-8ff6-6f4e17e78dd8 req-84011096-cfdc-4741-9c88-b7dc26e6eae4 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:30.821027 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-45a667a3-c0e9-4c32-8ff6-6f4e17e78dd8 req-84011096-cfdc-4741-9c88-b7dc26e6eae4 service nova] Creating event network-vif-plugged:da201a4d-39e0-43c4-b336-8ccd6da2e34e for instance 7dda007a-6f7c-4b57-a910-480eedbf967f on nch1-master-1 Sep 27 11:20:30.826243 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-45a667a3-c0e9-4c32-8ff6-6f4e17e78dd8 req-84011096-cfdc-4741-9c88-b7dc26e6eae4 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034687 Sep 27 11:20:30.826854 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3500/7474] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:20:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:20:31.462222 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8a937a70-eaa8-4342-9ae3-e08bd17f5a30 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:31.463270 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8a937a70-eaa8-4342-9ae3-e08bd17f5a30 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:31.473062 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a937a70-eaa8-4342-9ae3-e08bd17f5a30 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:31.473556 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a937a70-eaa8-4342-9ae3-e08bd17f5a30 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:31.474005 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a937a70-eaa8-4342-9ae3-e08bd17f5a30 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:31.474711 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a937a70-eaa8-4342-9ae3-e08bd17f5a30 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:31.475172 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a937a70-eaa8-4342-9ae3-e08bd17f5a30 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:31.475601 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8a937a70-eaa8-4342-9ae3-e08bd17f5a30 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:31.627504 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8a937a70-eaa8-4342-9ae3-e08bd17f5a30 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:31.636306 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8a937a70-eaa8-4342-9ae3-e08bd17f5a30 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1372 microversion: 2.1 time: 0.180734 Sep 27 11:20:31.636938 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3975/7475] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:31 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1372 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:32.668460 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1b177bd8-e5db-4e20-b55f-57ab60ecfb34 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:32.669634 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1b177bd8-e5db-4e20-b55f-57ab60ecfb34 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:32.680575 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b177bd8-e5db-4e20-b55f-57ab60ecfb34 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:32.681502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b177bd8-e5db-4e20-b55f-57ab60ecfb34 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:32.682226 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b177bd8-e5db-4e20-b55f-57ab60ecfb34 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:32.683408 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b177bd8-e5db-4e20-b55f-57ab60ecfb34 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:32.684205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b177bd8-e5db-4e20-b55f-57ab60ecfb34 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:32.685110 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1b177bd8-e5db-4e20-b55f-57ab60ecfb34 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:32.836436 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1b177bd8-e5db-4e20-b55f-57ab60ecfb34 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:32.845130 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1b177bd8-e5db-4e20-b55f-57ab60ecfb34 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1557 microversion: 2.1 time: 0.185700 Sep 27 11:20:32.845733 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3501/7476] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:32 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1557 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:32.868630 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-8e3ce6c1-e545-4f12-aa15-2a36b2c198cb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:20:32.869297 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-8e3ce6c1-e545-4f12-aa15-2a36b2c198cb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:20:32.876498 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e3ce6c1-e545-4f12-aa15-2a36b2c198cb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:32.877019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e3ce6c1-e545-4f12-aa15-2a36b2c198cb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:32.877456 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e3ce6c1-e545-4f12-aa15-2a36b2c198cb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:32.878150 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e3ce6c1-e545-4f12-aa15-2a36b2c198cb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:32.878600 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e3ce6c1-e545-4f12-aa15-2a36b2c198cb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:32.879025 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-8e3ce6c1-e545-4f12-aa15-2a36b2c198cb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:33.034511 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-8e3ce6c1-e545-4f12-aa15-2a36b2c198cb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:20:33.042584 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-8e3ce6c1-e545-4f12-aa15-2a36b2c198cb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1557 microversion: 2.1 time: 0.178238 Sep 27 11:20:33.043177 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3976/7477] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:20:32 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1557 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:20:35.071871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d2397114-7630-4ec4-9c9f-792d1824a730 req-502611ac-fc33-4bbc-ba69-b69f56c1e162 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7dda007a-6f7c-4b57-a910-480eedbf967f", "tag": "da201a4d-39e0-43c4-b336-8ccd6da2e34e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:20:35.077144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=6 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:20:35.084646 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d2397114-7630-4ec4-9c9f-792d1824a730 req-502611ac-fc33-4bbc-ba69-b69f56c1e162 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:20:35.085670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d2397114-7630-4ec4-9c9f-792d1824a730 req-502611ac-fc33-4bbc-ba69-b69f56c1e162 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:20:35.085670 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d2397114-7630-4ec4-9c9f-792d1824a730 req-502611ac-fc33-4bbc-ba69-b69f56c1e162 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:20:35.097888 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d2397114-7630-4ec4-9c9f-792d1824a730 req-502611ac-fc33-4bbc-ba69-b69f56c1e162 service nova] Creating event network-changed:da201a4d-39e0-43c4-b336-8ccd6da2e34e for instance 7dda007a-6f7c-4b57-a910-480eedbf967f on nch1-master-1 Sep 27 11:20:35.103559 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d2397114-7630-4ec4-9c9f-792d1824a730 req-502611ac-fc33-4bbc-ba69-b69f56c1e162 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035489 Sep 27 11:20:35.104512 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3502/7478] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:20:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 27 11:22:30.400406 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:30.401731 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:30.403027 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=27,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:30.404242 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=34 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:30.412785 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-2558568f-803c-4273-8e42-cab7c0feb276 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-TestSnapshotPatternsnapshot-1519404472"}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:22:30.415334 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-2558568f-803c-4273-8e42-cab7c0feb276 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:30.424052 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2558568f-803c-4273-8e42-cab7c0feb276 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:30.424566 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2558568f-803c-4273-8e42-cab7c0feb276 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:30.425023 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2558568f-803c-4273-8e42-cab7c0feb276 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:30.425719 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2558568f-803c-4273-8e42-cab7c0feb276 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:30.426170 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2558568f-803c-4273-8e42-cab7c0feb276 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:30.426594 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-2558568f-803c-4273-8e42-cab7c0feb276 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:30.661901 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-2558568f-803c-4273-8e42-cab7c0feb276 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "POST /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f/action" status: 202 len: 0 microversion: 2.1 time: 0.264717 Sep 27 11:22:30.662571 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3977/7479] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:22:30 2022] POST /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f/action => generated 0 bytes in 267 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 11:22:35.504822 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [-] [1986a9fd-1b4d-44f5-a8ac-515eb5bb435f] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 27 11:22:35.506706 ncdv-master-1 devstack@n-api.service[145969]: ERROR oslo.messaging._drivers.impl_rabbit [-] [d9169ee3-829d-41de-8bb4-82814efca059] AMQP server on 10.222.0.26:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 27 11:22:35.508308 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=27,nova_api:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:35.510543 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=28 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:35.511868 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:22:35.512228 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 11:22:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.26) !!! Sep 27 11:22:35.634850 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-89e8772b-d279-4022-8583-e31363e18733 req-e210523a-6dd8-4e0d-8889-c9d251d9c4be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7dda007a-6f7c-4b57-a910-480eedbf967f", "name": "network-vif-plugged", "status": "completed", "tag": "da201a4d-39e0-43c4-b336-8ccd6da2e34e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:22:35.646561 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-89e8772b-d279-4022-8583-e31363e18733 req-e210523a-6dd8-4e0d-8889-c9d251d9c4be service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:35.647156 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-89e8772b-d279-4022-8583-e31363e18733 req-e210523a-6dd8-4e0d-8889-c9d251d9c4be service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:35.647795 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-89e8772b-d279-4022-8583-e31363e18733 req-e210523a-6dd8-4e0d-8889-c9d251d9c4be service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:35.659641 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-89e8772b-d279-4022-8583-e31363e18733 req-e210523a-6dd8-4e0d-8889-c9d251d9c4be service nova] Creating event network-vif-plugged:da201a4d-39e0-43c4-b336-8ccd6da2e34e for instance 7dda007a-6f7c-4b57-a910-480eedbf967f on nch1-master-1 Sep 27 11:22:35.665874 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-89e8772b-d279-4022-8583-e31363e18733 req-e210523a-6dd8-4e0d-8889-c9d251d9c4be service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.163405 Sep 27 11:22:35.666779 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3503/7480] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:22:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:22:44.282844 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:44.283696 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=12,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:44.288546 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestSnapshotPattern-server-11392437", "imageRef": "cc9e2200-3188-4a80-bb75-601374966d62", "flavorRef": "452", "key_name": "tempest-TestSnapshotPattern-83732341", "security_groups": [{"name": "tempest-secgroup-smoke-1847565083"}], "networks": [{"uuid": "8313513f-6210-4a72-a6b6-882b7b1b37be"}]}} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:22:44.409903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.network.neutron [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] validate_networks() for [('8313513f-6210-4a72-a6b6-882b7b1b37be', None, None, None, None, None)] {{(pid=145968) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 27 11:22:44.808047 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Flavor limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 27 11:22:44.808478 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Image limits 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 27 11:22:44.808996 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Flavor pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 27 11:22:44.809410 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Image pref 0:0:0 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 27 11:22:44.809828 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.virt.hardware [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=145968) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 27 11:22:44.819016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:44.820185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:44.820637 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:44.821436 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:44.821902 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:44.822333 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:44.840273 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Getting quotas for project 34d2d2279cd14736ba71123e751c9aeb. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 27 11:22:44.845915 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.quota [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Getting quotas for user 8bf66c7321304c91840efc2cbeb52a4f and project 34d2d2279cd14736ba71123e751c9aeb. Resources: {'instances', 'ram', 'cores'} {{(pid=145968) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 27 11:22:44.858858 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Going to run 1 instances... {{(pid=145968) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 27 11:22:44.861386 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='cc9e2200-3188-4a80-bb75-601374966d62',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=145968) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 27 11:22:44.912071 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-aac1fe24-c2cf-4ec7-8312-463c9332531b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.631182 Sep 27 11:22:44.913053 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3978/7481] 10.222.0.26 () {60 vars in 1147 bytes} [Tue Sep 27 11:22:44 2022] POST /compute/v2.1/servers => generated 406 bytes in 633 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 27 11:22:44.927766 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e604093c-ffe8-481b-b069-5dca44444f35 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:44.928530 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e604093c-ffe8-481b-b069-5dca44444f35 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:44.944117 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] [d9169ee3-829d-41de-8bb4-82814efca059] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 60736. Sep 27 11:22:44.950735 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] [1986a9fd-1b4d-44f5-a8ac-515eb5bb435f] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 60734. Sep 27 11:22:45.001762 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e604093c-ffe8-481b-b069-5dca44444f35 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:45.010187 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e604093c-ffe8-481b-b069-5dca44444f35 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1245 microversion: 2.1 time: 0.089769 Sep 27 11:22:45.010755 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3504/7482] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:44 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1245 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:46.039768 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-4ecae67a-b7a3-4477-aa3e-27373acaa127 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:46.040883 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-4ecae67a-b7a3-4477-aa3e-27373acaa127 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:46.052050 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ecae67a-b7a3-4477-aa3e-27373acaa127 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:46.052953 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ecae67a-b7a3-4477-aa3e-27373acaa127 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:46.054057 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ecae67a-b7a3-4477-aa3e-27373acaa127 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:46.054857 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ecae67a-b7a3-4477-aa3e-27373acaa127 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:46.055347 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ecae67a-b7a3-4477-aa3e-27373acaa127 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:46.055818 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-4ecae67a-b7a3-4477-aa3e-27373acaa127 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:46.131016 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-4ecae67a-b7a3-4477-aa3e-27373acaa127 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:46.140088 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-4ecae67a-b7a3-4477-aa3e-27373acaa127 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1297 microversion: 2.1 time: 0.108430 Sep 27 11:22:46.140693 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3979/7483] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:46 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1297 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:47.166237 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-4f2b0b33-57d3-42ed-8d7f-265ffe2b4d72 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:47.166966 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-4f2b0b33-57d3-42ed-8d7f-265ffe2b4d72 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:47.169086 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:47.176450 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2b0b33-57d3-42ed-8d7f-265ffe2b4d72 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:47.177064 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2b0b33-57d3-42ed-8d7f-265ffe2b4d72 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:47.177527 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2b0b33-57d3-42ed-8d7f-265ffe2b4d72 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:47.178250 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2b0b33-57d3-42ed-8d7f-265ffe2b4d72 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:47.178913 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2b0b33-57d3-42ed-8d7f-265ffe2b4d72 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:47.179564 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-4f2b0b33-57d3-42ed-8d7f-265ffe2b4d72 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:47.265022 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-4f2b0b33-57d3-42ed-8d7f-265ffe2b4d72 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:47.273959 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-4f2b0b33-57d3-42ed-8d7f-265ffe2b4d72 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1303 microversion: 2.1 time: 0.112561 Sep 27 11:22:47.274563 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3505/7484] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:47 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1303 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:47.516349 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-e42e324b-56d0-4e82-89d9-3b1cfba3022a req-37325aab-b9d3-4ad1-83e1-0b8e1bc374f6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7dda007a-6f7c-4b57-a910-480eedbf967f", "name": "network-vif-unplugged", "status": "completed", "tag": "da201a4d-39e0-43c4-b336-8ccd6da2e34e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:22:47.529868 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e42e324b-56d0-4e82-89d9-3b1cfba3022a req-37325aab-b9d3-4ad1-83e1-0b8e1bc374f6 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:47.530607 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e42e324b-56d0-4e82-89d9-3b1cfba3022a req-37325aab-b9d3-4ad1-83e1-0b8e1bc374f6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:47.531466 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-e42e324b-56d0-4e82-89d9-3b1cfba3022a req-37325aab-b9d3-4ad1-83e1-0b8e1bc374f6 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:47.547170 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-e42e324b-56d0-4e82-89d9-3b1cfba3022a req-37325aab-b9d3-4ad1-83e1-0b8e1bc374f6 service nova] Creating event network-vif-unplugged:da201a4d-39e0-43c4-b336-8ccd6da2e34e for instance 7dda007a-6f7c-4b57-a910-480eedbf967f on nch1-master-1 Sep 27 11:22:47.553152 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-e42e324b-56d0-4e82-89d9-3b1cfba3022a req-37325aab-b9d3-4ad1-83e1-0b8e1bc374f6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.043547 Sep 27 11:22:47.554127 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3980/7485] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:22:47 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:22:48.298056 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ef68f1f1-08f4-40f1-8f2f-6a706838165b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:48.298662 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ef68f1f1-08f4-40f1-8f2f-6a706838165b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:48.306596 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef68f1f1-08f4-40f1-8f2f-6a706838165b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:48.307097 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef68f1f1-08f4-40f1-8f2f-6a706838165b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:48.307556 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef68f1f1-08f4-40f1-8f2f-6a706838165b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:48.308627 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef68f1f1-08f4-40f1-8f2f-6a706838165b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:48.309122 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef68f1f1-08f4-40f1-8f2f-6a706838165b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:48.309565 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ef68f1f1-08f4-40f1-8f2f-6a706838165b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:48.379481 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ef68f1f1-08f4-40f1-8f2f-6a706838165b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:48.387873 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ef68f1f1-08f4-40f1-8f2f-6a706838165b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1303 microversion: 2.1 time: 0.093727 Sep 27 11:22:48.388589 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3506/7486] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:48 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1303 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:49.412367 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-cc6a24e0-2791-4123-bf99-2527d6981910 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:49.413877 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-cc6a24e0-2791-4123-bf99-2527d6981910 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:49.421728 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc6a24e0-2791-4123-bf99-2527d6981910 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:49.422223 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc6a24e0-2791-4123-bf99-2527d6981910 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:49.422666 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc6a24e0-2791-4123-bf99-2527d6981910 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:49.423547 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc6a24e0-2791-4123-bf99-2527d6981910 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:49.424002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc6a24e0-2791-4123-bf99-2527d6981910 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:49.424429 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-cc6a24e0-2791-4123-bf99-2527d6981910 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:49.496069 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-cc6a24e0-2791-4123-bf99-2527d6981910 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:49.504243 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-cc6a24e0-2791-4123-bf99-2527d6981910 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1303 microversion: 2.1 time: 0.096380 Sep 27 11:22:49.505047 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3981/7487] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:49 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1303 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:50.547758 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f71ac89c-5ca9-43ca-936f-e2538ccbe143 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:50.549173 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-f71ac89c-5ca9-43ca-936f-e2538ccbe143 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:50.557235 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f71ac89c-5ca9-43ca-936f-e2538ccbe143 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:50.558059 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f71ac89c-5ca9-43ca-936f-e2538ccbe143 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:50.558528 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f71ac89c-5ca9-43ca-936f-e2538ccbe143 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:50.559269 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f71ac89c-5ca9-43ca-936f-e2538ccbe143 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:50.559752 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f71ac89c-5ca9-43ca-936f-e2538ccbe143 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:50.560199 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-f71ac89c-5ca9-43ca-936f-e2538ccbe143 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:50.632089 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-f71ac89c-5ca9-43ca-936f-e2538ccbe143 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:50.646596 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f71ac89c-5ca9-43ca-936f-e2538ccbe143 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1303 microversion: 2.1 time: 0.111163 Sep 27 11:22:50.647312 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3507/7488] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:50 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1303 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:51.674003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-49a7f674-bfaa-4f1d-b510-f23b2629615d tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:51.675106 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-49a7f674-bfaa-4f1d-b510-f23b2629615d tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:51.682343 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49a7f674-bfaa-4f1d-b510-f23b2629615d tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:51.682987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49a7f674-bfaa-4f1d-b510-f23b2629615d tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:51.683437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49a7f674-bfaa-4f1d-b510-f23b2629615d tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:51.684159 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49a7f674-bfaa-4f1d-b510-f23b2629615d tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:51.684623 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49a7f674-bfaa-4f1d-b510-f23b2629615d tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:51.685244 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-49a7f674-bfaa-4f1d-b510-f23b2629615d tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:51.814003 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-49a7f674-bfaa-4f1d-b510-f23b2629615d tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:51.822641 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-49a7f674-bfaa-4f1d-b510-f23b2629615d tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1303 microversion: 2.1 time: 0.156074 Sep 27 11:22:51.823442 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3982/7489] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:51 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1303 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:52.846538 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-9055e305-c62c-4cd1-8f34-3191fcea09f9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:52.847267 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-9055e305-c62c-4cd1-8f34-3191fcea09f9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:52.854374 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9055e305-c62c-4cd1-8f34-3191fcea09f9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:52.854878 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9055e305-c62c-4cd1-8f34-3191fcea09f9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:52.855441 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9055e305-c62c-4cd1-8f34-3191fcea09f9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:52.856513 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9055e305-c62c-4cd1-8f34-3191fcea09f9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:52.857124 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9055e305-c62c-4cd1-8f34-3191fcea09f9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:52.857664 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-9055e305-c62c-4cd1-8f34-3191fcea09f9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:52.929654 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-9055e305-c62c-4cd1-8f34-3191fcea09f9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:52.938477 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-9055e305-c62c-4cd1-8f34-3191fcea09f9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1303 microversion: 2.1 time: 0.095892 Sep 27 11:22:52.939401 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3508/7490] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:52 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1303 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:53.962481 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-f7fc2f79-8b6d-46af-8566-3eafa3f444f8 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:53.963139 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-f7fc2f79-8b6d-46af-8566-3eafa3f444f8 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:53.970485 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7fc2f79-8b6d-46af-8566-3eafa3f444f8 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:53.970955 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7fc2f79-8b6d-46af-8566-3eafa3f444f8 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:53.971402 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7fc2f79-8b6d-46af-8566-3eafa3f444f8 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:53.972119 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7fc2f79-8b6d-46af-8566-3eafa3f444f8 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:53.972587 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7fc2f79-8b6d-46af-8566-3eafa3f444f8 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:53.973443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-f7fc2f79-8b6d-46af-8566-3eafa3f444f8 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:54.041019 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-f7fc2f79-8b6d-46af-8566-3eafa3f444f8 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:54.049369 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-f7fc2f79-8b6d-46af-8566-3eafa3f444f8 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1303 microversion: 2.1 time: 0.090945 Sep 27 11:22:54.049956 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3983/7491] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:53 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1303 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:55.079019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-2d3b2165-945a-4c02-9f93-ecc8b03f7851 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:55.080005 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-2d3b2165-945a-4c02-9f93-ecc8b03f7851 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:55.084356 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell0:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:55.090610 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3b2165-945a-4c02-9f93-ecc8b03f7851 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:55.091126 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3b2165-945a-4c02-9f93-ecc8b03f7851 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:55.091579 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3b2165-945a-4c02-9f93-ecc8b03f7851 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:55.092298 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3b2165-945a-4c02-9f93-ecc8b03f7851 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:55.092811 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3b2165-945a-4c02-9f93-ecc8b03f7851 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:55.093253 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-2d3b2165-945a-4c02-9f93-ecc8b03f7851 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:55.169062 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-2d3b2165-945a-4c02-9f93-ecc8b03f7851 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:55.177397 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-2d3b2165-945a-4c02-9f93-ecc8b03f7851 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1303 microversion: 2.1 time: 0.105030 Sep 27 11:22:55.177984 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3509/7492] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:55 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1303 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:56.204927 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-d5a0ce4c-98da-4bed-a855-25e990eea101 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:56.205764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-d5a0ce4c-98da-4bed-a855-25e990eea101 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:56.209063 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:56.210105 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell0:SELECT=6 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:22:56.220413 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5a0ce4c-98da-4bed-a855-25e990eea101 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:56.221252 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5a0ce4c-98da-4bed-a855-25e990eea101 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:56.221994 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5a0ce4c-98da-4bed-a855-25e990eea101 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:56.223144 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5a0ce4c-98da-4bed-a855-25e990eea101 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:56.223913 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5a0ce4c-98da-4bed-a855-25e990eea101 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:56.224622 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-d5a0ce4c-98da-4bed-a855-25e990eea101 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:56.290633 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-d5a0ce4c-98da-4bed-a855-25e990eea101 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:56.301626 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-d5a0ce4c-98da-4bed-a855-25e990eea101 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1303 microversion: 2.1 time: 0.103481 Sep 27 11:22:56.302195 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3984/7493] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:56 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1303 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:57.325859 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-e3a4bd31-2bfc-4365-afcc-93e558787dcc tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:57.326415 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-e3a4bd31-2bfc-4365-afcc-93e558787dcc tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:57.333842 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3a4bd31-2bfc-4365-afcc-93e558787dcc tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:57.334347 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3a4bd31-2bfc-4365-afcc-93e558787dcc tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:57.334797 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3a4bd31-2bfc-4365-afcc-93e558787dcc tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:57.335678 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3a4bd31-2bfc-4365-afcc-93e558787dcc tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:57.336052 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3a4bd31-2bfc-4365-afcc-93e558787dcc tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:57.336495 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-e3a4bd31-2bfc-4365-afcc-93e558787dcc tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:57.481951 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-e3a4bd31-2bfc-4365-afcc-93e558787dcc tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:57.490015 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-e3a4bd31-2bfc-4365-afcc-93e558787dcc tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1371 microversion: 2.1 time: 0.168351 Sep 27 11:22:57.490691 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3510/7494] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:57 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1371 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:57.764215 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-bc6618c6-9311-4a95-9b48-1897e8debbc9 req-6ac6ef2e-9f03-4bcb-a8f8-0f8adc36d8d7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1f125e9f-02a8-4128-ae0c-b0a5d1d39637", "tag": "1d41a138-4d20-452c-a5cc-e8e142ab1f5d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:22:57.775073 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bc6618c6-9311-4a95-9b48-1897e8debbc9 req-6ac6ef2e-9f03-4bcb-a8f8-0f8adc36d8d7 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:57.775703 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bc6618c6-9311-4a95-9b48-1897e8debbc9 req-6ac6ef2e-9f03-4bcb-a8f8-0f8adc36d8d7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:57.776238 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-bc6618c6-9311-4a95-9b48-1897e8debbc9 req-6ac6ef2e-9f03-4bcb-a8f8-0f8adc36d8d7 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:57.790208 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-bc6618c6-9311-4a95-9b48-1897e8debbc9 req-6ac6ef2e-9f03-4bcb-a8f8-0f8adc36d8d7 service nova] Creating event network-changed:1d41a138-4d20-452c-a5cc-e8e142ab1f5d for instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 on nch2-master-1 Sep 27 11:22:57.795816 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-bc6618c6-9311-4a95-9b48-1897e8debbc9 req-6ac6ef2e-9f03-4bcb-a8f8-0f8adc36d8d7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.106128 Sep 27 11:22:57.796428 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3985/7495] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:22:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:22:58.517650 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-beffa0fa-69d1-4c2c-9147-e9bbfdee0a6e tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:58.518845 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-beffa0fa-69d1-4c2c-9147-e9bbfdee0a6e tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:58.527974 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-beffa0fa-69d1-4c2c-9147-e9bbfdee0a6e tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:58.528469 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-beffa0fa-69d1-4c2c-9147-e9bbfdee0a6e tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:58.528954 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-beffa0fa-69d1-4c2c-9147-e9bbfdee0a6e tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:58.529669 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-beffa0fa-69d1-4c2c-9147-e9bbfdee0a6e tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:58.530136 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-beffa0fa-69d1-4c2c-9147-e9bbfdee0a6e tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:58.530578 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-beffa0fa-69d1-4c2c-9147-e9bbfdee0a6e tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:58.687151 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-beffa0fa-69d1-4c2c-9147-e9bbfdee0a6e tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:58.695834 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-beffa0fa-69d1-4c2c-9147-e9bbfdee0a6e tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1371 microversion: 2.1 time: 0.185015 Sep 27 11:22:58.696700 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3511/7496] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:58 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1371 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:22:59.725458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-67ae2baf-e2a3-42e4-8ad8-ac7cce11f685 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:22:59.726565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-67ae2baf-e2a3-42e4-8ad8-ac7cce11f685 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:22:59.732949 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67ae2baf-e2a3-42e4-8ad8-ac7cce11f685 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:59.733443 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67ae2baf-e2a3-42e4-8ad8-ac7cce11f685 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:59.733888 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67ae2baf-e2a3-42e4-8ad8-ac7cce11f685 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:59.734595 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67ae2baf-e2a3-42e4-8ad8-ac7cce11f685 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:22:59.735059 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67ae2baf-e2a3-42e4-8ad8-ac7cce11f685 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:22:59.735497 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-67ae2baf-e2a3-42e4-8ad8-ac7cce11f685 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:22:59.888095 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-67ae2baf-e2a3-42e4-8ad8-ac7cce11f685 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:22:59.899900 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-67ae2baf-e2a3-42e4-8ad8-ac7cce11f685 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1371 microversion: 2.1 time: 0.179520 Sep 27 11:22:59.900709 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3986/7497] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:22:59 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1371 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:23:00.930788 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-3e9a8e96-574e-4562-a00c-449164a8e825 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:23:00.931340 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-3e9a8e96-574e-4562-a00c-449164a8e825 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:23:00.938682 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a8e96-574e-4562-a00c-449164a8e825 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:00.939189 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a8e96-574e-4562-a00c-449164a8e825 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:00.939651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a8e96-574e-4562-a00c-449164a8e825 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:00.940373 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a8e96-574e-4562-a00c-449164a8e825 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:00.940891 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a8e96-574e-4562-a00c-449164a8e825 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:00.941336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-3e9a8e96-574e-4562-a00c-449164a8e825 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:01.104971 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-3e9a8e96-574e-4562-a00c-449164a8e825 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:23:01.113228 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-3e9a8e96-574e-4562-a00c-449164a8e825 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1371 microversion: 2.1 time: 0.188475 Sep 27 11:23:01.113845 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3512/7498] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:23:00 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1371 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:23:02.142311 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-26792973-3af9-4522-beca-f9550225e976 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:23:02.143455 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-26792973-3af9-4522-beca-f9550225e976 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:23:02.154613 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26792973-3af9-4522-beca-f9550225e976 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:02.155458 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26792973-3af9-4522-beca-f9550225e976 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:02.156200 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26792973-3af9-4522-beca-f9550225e976 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:02.157437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26792973-3af9-4522-beca-f9550225e976 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:02.158211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26792973-3af9-4522-beca-f9550225e976 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:02.158935 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-26792973-3af9-4522-beca-f9550225e976 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:02.298473 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-26792973-3af9-4522-beca-f9550225e976 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:23:02.306500 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-26792973-3af9-4522-beca-f9550225e976 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1371 microversion: 2.1 time: 0.171286 Sep 27 11:23:02.307091 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3987/7499] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:23:02 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1371 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:23:03.334144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-1ad23329-f3d2-4b12-b36c-05cef22c048b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:23:03.335197 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-1ad23329-f3d2-4b12-b36c-05cef22c048b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:23:03.343721 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ad23329-f3d2-4b12-b36c-05cef22c048b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:03.344225 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ad23329-f3d2-4b12-b36c-05cef22c048b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:03.344645 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ad23329-f3d2-4b12-b36c-05cef22c048b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:03.345425 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ad23329-f3d2-4b12-b36c-05cef22c048b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:03.345895 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ad23329-f3d2-4b12-b36c-05cef22c048b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:03.346336 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-1ad23329-f3d2-4b12-b36c-05cef22c048b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:03.487955 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-1ad23329-f3d2-4b12-b36c-05cef22c048b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:23:03.496040 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-1ad23329-f3d2-4b12-b36c-05cef22c048b tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1371 microversion: 2.1 time: 0.168817 Sep 27 11:23:03.496632 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3513/7500] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:23:03 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1371 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:23:03.689419 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-2102653f-c529-450f-9cff-612da4b94ea2 req-5c7defd0-d03d-417e-970c-0a2400c81e0a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1f125e9f-02a8-4128-ae0c-b0a5d1d39637", "name": "network-vif-plugged", "status": "completed", "tag": "1d41a138-4d20-452c-a5cc-e8e142ab1f5d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:23:03.700667 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2102653f-c529-450f-9cff-612da4b94ea2 req-5c7defd0-d03d-417e-970c-0a2400c81e0a service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:03.701211 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2102653f-c529-450f-9cff-612da4b94ea2 req-5c7defd0-d03d-417e-970c-0a2400c81e0a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:03.701658 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-2102653f-c529-450f-9cff-612da4b94ea2 req-5c7defd0-d03d-417e-970c-0a2400c81e0a service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:03.713227 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-2102653f-c529-450f-9cff-612da4b94ea2 req-5c7defd0-d03d-417e-970c-0a2400c81e0a service nova] Creating event network-vif-plugged:1d41a138-4d20-452c-a5cc-e8e142ab1f5d for instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 on nch2-master-1 Sep 27 11:23:03.718414 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-2102653f-c529-450f-9cff-612da4b94ea2 req-5c7defd0-d03d-417e-970c-0a2400c81e0a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032961 Sep 27 11:23:03.719029 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3988/7501] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:23:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:23:04.524710 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-58f4d30a-d432-464d-baad-08963117c6ae tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:23:04.525772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-58f4d30a-d432-464d-baad-08963117c6ae tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:23:04.537697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f4d30a-d432-464d-baad-08963117c6ae tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:04.538775 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f4d30a-d432-464d-baad-08963117c6ae tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:04.539285 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f4d30a-d432-464d-baad-08963117c6ae tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:04.540015 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f4d30a-d432-464d-baad-08963117c6ae tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:04.540494 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f4d30a-d432-464d-baad-08963117c6ae tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:04.540982 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-58f4d30a-d432-464d-baad-08963117c6ae tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:04.683997 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-58f4d30a-d432-464d-baad-08963117c6ae tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:23:04.696368 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-58f4d30a-d432-464d-baad-08963117c6ae tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1371 microversion: 2.1 time: 0.178946 Sep 27 11:23:04.697418 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3514/7502] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:23:04 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1371 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:23:05.726296 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1fae045b-f4c0-47a1-814d-8f2296239338 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:23:05.727002 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1fae045b-f4c0-47a1-814d-8f2296239338 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:23:05.734086 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fae045b-f4c0-47a1-814d-8f2296239338 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:05.734959 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fae045b-f4c0-47a1-814d-8f2296239338 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:05.735076 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fae045b-f4c0-47a1-814d-8f2296239338 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:05.735849 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fae045b-f4c0-47a1-814d-8f2296239338 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:05.736219 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fae045b-f4c0-47a1-814d-8f2296239338 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:05.736632 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1fae045b-f4c0-47a1-814d-8f2296239338 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:05.882775 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-1fae045b-f4c0-47a1-814d-8f2296239338 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:23:05.890984 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1fae045b-f4c0-47a1-814d-8f2296239338 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1556 microversion: 2.1 time: 0.169970 Sep 27 11:23:05.891536 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3989/7503] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:23:05 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1556 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:23:05.917312 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-5f83a944-f1f3-47b5-baf7-064e94816a78 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:23:05.917945 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-5f83a944-f1f3-47b5-baf7-064e94816a78 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:23:05.920802 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=27 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:23:05.926471 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f83a944-f1f3-47b5-baf7-064e94816a78 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:05.927006 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f83a944-f1f3-47b5-baf7-064e94816a78 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:05.927463 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f83a944-f1f3-47b5-baf7-064e94816a78 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:05.928205 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f83a944-f1f3-47b5-baf7-064e94816a78 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:05.928684 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f83a944-f1f3-47b5-baf7-064e94816a78 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:05.929183 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-5f83a944-f1f3-47b5-baf7-064e94816a78 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:06.074317 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-5f83a944-f1f3-47b5-baf7-064e94816a78 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:23:06.085747 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-5f83a944-f1f3-47b5-baf7-064e94816a78 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1556 microversion: 2.1 time: 0.174148 Sep 27 11:23:06.086578 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3515/7504] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:23:05 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1556 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:23:07.826837 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-6006ae18-6857-44b0-aa09-b67711333923 req-9f72ebeb-7a4e-446a-88c9-7e93019457f2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1f125e9f-02a8-4128-ae0c-b0a5d1d39637", "tag": "1d41a138-4d20-452c-a5cc-e8e142ab1f5d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:23:07.836395 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6006ae18-6857-44b0-aa09-b67711333923 req-9f72ebeb-7a4e-446a-88c9-7e93019457f2 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:23:07.836970 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6006ae18-6857-44b0-aa09-b67711333923 req-9f72ebeb-7a4e-446a-88c9-7e93019457f2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:23:07.837462 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-6006ae18-6857-44b0-aa09-b67711333923 req-9f72ebeb-7a4e-446a-88c9-7e93019457f2 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:23:07.848686 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-6006ae18-6857-44b0-aa09-b67711333923 req-9f72ebeb-7a4e-446a-88c9-7e93019457f2 service nova] Creating event network-changed:1d41a138-4d20-452c-a5cc-e8e142ab1f5d for instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 on nch2-master-1 Sep 27 11:23:07.854150 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-6006ae18-6857-44b0-aa09-b67711333923 req-9f72ebeb-7a4e-446a-88c9-7e93019457f2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.089708 Sep 27 11:23:07.854807 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3990/7505] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:23:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:25:13.850270 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:13.851512 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=66 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:13.853203 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Sep 27 11:25:13.853507 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 11:25:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637/action (ip 10.222.0.26) !!! Sep 27 11:25:13.855295 ncdv-master-1 devstack@n-api.service[145969]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:25:13.855531 ncdv-master-1 devstack@n-api.service[145969]: Tue Sep 27 11:25:13 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637/action (ip 10.222.0.26) !!! Sep 27 11:25:13.862957 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-7e450bd6-1877-42aa-9fe7-ed4892ecb045 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Action: 'action', calling method: >, body: {"createImage": {"name": "tempest-TestSnapshotPatternsnapshot-870087525"}} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:25:13.865406 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-7e450bd6-1877-42aa-9fe7-ed4892ecb045 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:25:13.875697 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e450bd6-1877-42aa-9fe7-ed4892ecb045 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:13.876204 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e450bd6-1877-42aa-9fe7-ed4892ecb045 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:13.876677 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e450bd6-1877-42aa-9fe7-ed4892ecb045 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:13.877426 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e450bd6-1877-42aa-9fe7-ed4892ecb045 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:13.877902 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e450bd6-1877-42aa-9fe7-ed4892ecb045 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:13.878345 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-7e450bd6-1877-42aa-9fe7-ed4892ecb045 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:14.301403 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-7e450bd6-1877-42aa-9fe7-ed4892ecb045 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "POST /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637/action" status: 202 len: 0 microversion: 2.1 time: 0.454469 Sep 27 11:25:14.303030 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3516/7506] 10.222.0.26 () {60 vars in 1278 bytes} [Tue Sep 27 11:25:13 2022] POST /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637/action => generated 0 bytes in 457 msecs (HTTP/1.1 202) 10 headers in 450 bytes (1 switches on core 0) Sep 27 11:25:20.349292 ncdv-master-1 devstack@n-api.service[145968]: ERROR oslo.messaging._drivers.impl_rabbit [-] [eb0e6821-e1de-45db-a339-eb2fe392322e] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 27 11:25:20.350727 ncdv-master-1 devstack@n-api.service[145968]: ERROR oslo.messaging._drivers.impl_rabbit [-] [caeac69d-7b1f-444e-94c7-8c629d37bafc] AMQP server on 10.222.0.26:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 27 11:25:20.351493 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=3 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:20.352309 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=58 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:20.353642 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Sep 27 11:25:20.353785 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 11:25:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.26) !!! Sep 27 11:25:20.355592 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 27 11:25:20.355711 ncdv-master-1 devstack@n-api.service[145968]: Tue Sep 27 11:25:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.26) !!! Sep 27 11:25:20.362051 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-c835f497-0820-4bee-9c3d-57be096168fe req-9ccaaa67-42ff-4067-98e8-796640739b0e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1f125e9f-02a8-4128-ae0c-b0a5d1d39637", "name": "network-vif-plugged", "status": "completed", "tag": "1d41a138-4d20-452c-a5cc-e8e142ab1f5d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:25:20.378709 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c835f497-0820-4bee-9c3d-57be096168fe req-9ccaaa67-42ff-4067-98e8-796640739b0e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:20.379192 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c835f497-0820-4bee-9c3d-57be096168fe req-9ccaaa67-42ff-4067-98e8-796640739b0e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:20.379625 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-c835f497-0820-4bee-9c3d-57be096168fe req-9ccaaa67-42ff-4067-98e8-796640739b0e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:20.392757 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-c835f497-0820-4bee-9c3d-57be096168fe req-9ccaaa67-42ff-4067-98e8-796640739b0e service nova] Creating event network-vif-plugged:1d41a138-4d20-452c-a5cc-e8e142ab1f5d for instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 on nch2-master-1 Sep 27 11:25:20.397708 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-c835f497-0820-4bee-9c3d-57be096168fe req-9ccaaa67-42ff-4067-98e8-796640739b0e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050304 Sep 27 11:25:20.398512 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3991/7507] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:25:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:25:33.728452 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_cell1:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:33.729153 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-598e649c-1185-4f92-a7bc-0dc3959324bb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:25:33.729544 ncdv-master-1 devstack@n-api.service[145968]: DEBUG dbcounter [-] [145968] Writing DB stats nova_api:SELECT=2 {{(pid=145968) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:33.729633 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-598e649c-1185-4f92-a7bc-0dc3959324bb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:25:33.734242 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_api:SELECT=2 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:33.735310 ncdv-master-1 devstack@n-api.service[145969]: DEBUG dbcounter [-] [145969] Writing DB stats nova_cell1:SELECT=12,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=145969) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 27 11:25:33.736074 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-10a35054-b977-4783-b0b3-a9e26705a3fb req-f9ac64f5-189d-4144-a591-e1849421797c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1f125e9f-02a8-4128-ae0c-b0a5d1d39637", "tag": "1d41a138-4d20-452c-a5cc-e8e142ab1f5d"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:25:33.741598 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598e649c-1185-4f92-a7bc-0dc3959324bb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:33.742410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598e649c-1185-4f92-a7bc-0dc3959324bb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:33.743458 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598e649c-1185-4f92-a7bc-0dc3959324bb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:33.745698 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598e649c-1185-4f92-a7bc-0dc3959324bb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:33.746410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598e649c-1185-4f92-a7bc-0dc3959324bb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:33.746934 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] [eb0e6821-e1de-45db-a339-eb2fe392322e] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 32892. Sep 27 11:25:33.747262 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-598e649c-1185-4f92-a7bc-0dc3959324bb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:33.752411 ncdv-master-1 devstack@n-api.service[145968]: INFO oslo.messaging._drivers.impl_rabbit [-] [caeac69d-7b1f-444e-94c7-8c629d37bafc] Reconnected to AMQP server on 10.222.0.26:5672 via [amqp] client with port 32894. Sep 27 11:25:33.760509 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-10a35054-b977-4783-b0b3-a9e26705a3fb req-f9ac64f5-189d-4144-a591-e1849421797c service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:33.761276 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-10a35054-b977-4783-b0b3-a9e26705a3fb req-f9ac64f5-189d-4144-a591-e1849421797c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:33.761482 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-10a35054-b977-4783-b0b3-a9e26705a3fb req-f9ac64f5-189d-4144-a591-e1849421797c service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:33.773409 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-10a35054-b977-4783-b0b3-a9e26705a3fb req-f9ac64f5-189d-4144-a591-e1849421797c service nova] Creating event network-changed:1d41a138-4d20-452c-a5cc-e8e142ab1f5d for instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 on nch2-master-1 Sep 27 11:25:33.778752 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-10a35054-b977-4783-b0b3-a9e26705a3fb req-f9ac64f5-189d-4144-a591-e1849421797c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053745 Sep 27 11:25:33.779736 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3992/7508] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:25:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:25:33.781051 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-598e649c-1185-4f92-a7bc-0dc3959324bb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Going to try to terminate instance {{(pid=145969) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 11:25:33.871606 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-598e649c-1185-4f92-a7bc-0dc3959324bb tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "DELETE /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 204 len: 0 microversion: 2.1 time: 0.148587 Sep 27 11:25:33.872316 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3517/7509] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 11:25:33 2022] DELETE /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 11:25:33.885733 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-962bc7da-be63-4969-b580-c5afdfa3dd85 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:25:33.886503 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-962bc7da-be63-4969-b580-c5afdfa3dd85 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:25:33.907968 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-962bc7da-be63-4969-b580-c5afdfa3dd85 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:33.909800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-962bc7da-be63-4969-b580-c5afdfa3dd85 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:33.910332 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-962bc7da-be63-4969-b580-c5afdfa3dd85 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:33.911185 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-962bc7da-be63-4969-b580-c5afdfa3dd85 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:33.911650 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-962bc7da-be63-4969-b580-c5afdfa3dd85 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:33.912217 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-962bc7da-be63-4969-b580-c5afdfa3dd85 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:34.104151 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-962bc7da-be63-4969-b580-c5afdfa3dd85 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:25:34.113786 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-962bc7da-be63-4969-b580-c5afdfa3dd85 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1680 microversion: 2.1 time: 0.236182 Sep 27 11:25:34.114560 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3993/7510] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:25:33 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1680 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:25:34.382760 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-1e624dbc-e76f-4609-96af-611dfd3d9cd4 req-b5f09a9b-d1af-4253-8f12-c164b60fae2f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1f125e9f-02a8-4128-ae0c-b0a5d1d39637", "name": "network-vif-unplugged", "status": "completed", "tag": "1d41a138-4d20-452c-a5cc-e8e142ab1f5d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:25:34.402438 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1e624dbc-e76f-4609-96af-611dfd3d9cd4 req-b5f09a9b-d1af-4253-8f12-c164b60fae2f service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:34.403050 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1e624dbc-e76f-4609-96af-611dfd3d9cd4 req-b5f09a9b-d1af-4253-8f12-c164b60fae2f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:34.403459 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-1e624dbc-e76f-4609-96af-611dfd3d9cd4 req-b5f09a9b-d1af-4253-8f12-c164b60fae2f service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:34.420754 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-1e624dbc-e76f-4609-96af-611dfd3d9cd4 req-b5f09a9b-d1af-4253-8f12-c164b60fae2f service nova] Creating event network-vif-unplugged:1d41a138-4d20-452c-a5cc-e8e142ab1f5d for instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 on nch2-master-1 Sep 27 11:25:34.426390 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-1e624dbc-e76f-4609-96af-611dfd3d9cd4 req-b5f09a9b-d1af-4253-8f12-c164b60fae2f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.054812 Sep 27 11:25:34.427020 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3518/7511] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:25:34 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:25:35.142369 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-c1b5ddc2-42e1-4716-a9d1-f23af9c04d98 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:25:35.143524 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-c1b5ddc2-42e1-4716-a9d1-f23af9c04d98 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:25:35.154869 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b5ddc2-42e1-4716-a9d1-f23af9c04d98 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:35.155903 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b5ddc2-42e1-4716-a9d1-f23af9c04d98 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:35.156387 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b5ddc2-42e1-4716-a9d1-f23af9c04d98 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:35.157641 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b5ddc2-42e1-4716-a9d1-f23af9c04d98 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:35.158141 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b5ddc2-42e1-4716-a9d1-f23af9c04d98 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:35.158568 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-c1b5ddc2-42e1-4716-a9d1-f23af9c04d98 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:35.368283 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-c1b5ddc2-42e1-4716-a9d1-f23af9c04d98 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:25:35.376572 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-c1b5ddc2-42e1-4716-a9d1-f23af9c04d98 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1680 microversion: 2.1 time: 0.241452 Sep 27 11:25:35.377157 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3994/7512] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:25:35 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1680 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:25:36.211025 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-fc5277e8-e812-448d-8353-43092559f69c req-60199813-b579-4fe7-a7e8-a012a7c4155e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1f125e9f-02a8-4128-ae0c-b0a5d1d39637", "name": "network-vif-deleted", "tag": "1d41a138-4d20-452c-a5cc-e8e142ab1f5d"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:25:36.224639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fc5277e8-e812-448d-8353-43092559f69c req-60199813-b579-4fe7-a7e8-a012a7c4155e service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:36.225163 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fc5277e8-e812-448d-8353-43092559f69c req-60199813-b579-4fe7-a7e8-a012a7c4155e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:36.225612 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-fc5277e8-e812-448d-8353-43092559f69c req-60199813-b579-4fe7-a7e8-a012a7c4155e service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:36.238516 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-fc5277e8-e812-448d-8353-43092559f69c req-60199813-b579-4fe7-a7e8-a012a7c4155e service nova] Creating event network-vif-deleted:1d41a138-4d20-452c-a5cc-e8e142ab1f5d for instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 on nch2-master-1 Sep 27 11:25:36.243833 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-fc5277e8-e812-448d-8353-43092559f69c req-60199813-b579-4fe7-a7e8-a012a7c4155e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037386 Sep 27 11:25:36.244681 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3519/7513] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:25:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:25:36.401823 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:25:36.402480 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:25:36.409619 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:36.410116 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:36.410558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:36.411268 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:36.411764 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:36.412195 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:36.447612 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:36.448099 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:36.448535 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:36.449012 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.objects.instance [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lazy-loading 'fault' on Instance uuid 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 {{(pid=145968) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 11:25:36.494713 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.policy [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145968) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:25:36.503602 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-251a3334-a721-4686-bea8-9f178c8f3718 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 200 len: 1335 microversion: 2.1 time: 0.105572 Sep 27 11:25:36.504213 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3995/7514] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:25:36 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 1335 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:25:37.533830 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:25:37.535014 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 1f125e9f-02a8-4128-ae0c-b0a5d1d39637] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:25:37.545828 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:37.546651 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:37.547467 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:37.548847 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:37.549258 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:37.549737 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:37.563595 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.wsgi [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] HTTP exception thrown: Instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 could not be found. Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Request method failure captured: Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: request: GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 HTTP/1.1 Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: Accept: application/json Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: Accept-Encoding: identity Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: Connection: close Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: Content-Length: 0 Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: Content-Type: application/json Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: Host: 10.222.0.26 Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: Openstack-System-Scope: None Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: User-Agent: python-urllib3/1.26.12 Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Auth-Token: gAAAAABjMtxvKjnv6wV3GuGjckqdG1-VB06PhBdkv6peC2kyASn40W2fQsHmrRzSuXs9W-hiBYyTtJaYZg6r99sdAdYg0yiV303cwoiwMGEzLqedRGjExIEY-bfg-jbngLxy9YT9pZ5qOA69TL7jNZX7PeW_cIWuAubT5KPY8NpwEtNarR59FDs Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Id: None Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Domain-Name: None Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Identity-Status: Confirmed Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Is-Admin-Project: True Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Id: default Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Domain-Name: Default Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Id: 34d2d2279cd14736ba71123e751c9aeb Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Project-Name: tempest-TestSnapshotPattern-960493092 Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Role: reader,member Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Roles: reader,member Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/34d2d2279cd14736ba71123e751c9aeb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/34d2d2279cd14736ba71123e751c9aeb"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/34d2d2279cd14736ba71123e751c9aeb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_34d2d2279cd14736ba71123e751c9aeb"}]}] Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant: tempest-TestSnapshotPattern-960493092 Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Id: 34d2d2279cd14736ba71123e751c9aeb Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-Tenant-Name: tempest-TestSnapshotPattern-960493092 Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-User: tempest-TestSnapshotPattern-960493092-project Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Id: default Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-User-Domain-Name: Default Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-User-Id: 8bf66c7321304c91840efc2cbeb52a4f Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: X-User-Name: tempest-TestSnapshotPattern-960493092-project Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: method: > Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: exception: Instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 could not be found. Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: action_args: {'id': '1f125e9f-02a8-4128-ae0c-b0a5d1d39637'} Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 could not be found. Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 could not be found. Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:25:37.565304 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:25:37.569371 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:25:37.569371 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:25:37.569371 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:25:37.569371 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 11:25:37.569371 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 11:25:37.569371 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 could not be found. Sep 27 11:25:37.569371 ncdv-master-1 devstack@n-api.service[145969]: ERROR nova.api.openstack.wsgi Sep 27 11:25:37.569371 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Returning 404 to user: Instance 1f125e9f-02a8-4128-ae0c-b0a5d1d39637 could not be found. {{(pid=145969) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 11:25:37.569371 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-8f2b06bd-8522-4eaa-bfef-43d079a663b9 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637" status: 404 len: 111 microversion: 2.1 time: 0.041705 Sep 27 11:25:37.569371 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3520/7515] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:25:37 2022] GET /compute/v2.1/servers/1f125e9f-02a8-4128-ae0c-b0a5d1d39637 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 11:25:39.012465 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-1a22ca19-5ff7-47bf-9193-18856d1cf24f tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:25:39.013749 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1a22ca19-5ff7-47bf-9193-18856d1cf24f tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:25:39.015055 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [req-d161c233-4f70-48c6-b1d0-c8b52238f969 req-c5a0ba4a-3bc4-414e-a907-05f05cdf0d87 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7dda007a-6f7c-4b57-a910-480eedbf967f", "tag": "da201a4d-39e0-43c4-b336-8ccd6da2e34e"}]} {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:25:39.021800 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a22ca19-5ff7-47bf-9193-18856d1cf24f tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:39.022437 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a22ca19-5ff7-47bf-9193-18856d1cf24f tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:39.022987 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a22ca19-5ff7-47bf-9193-18856d1cf24f tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:39.023954 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a22ca19-5ff7-47bf-9193-18856d1cf24f tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:39.024488 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a22ca19-5ff7-47bf-9193-18856d1cf24f tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:39.025032 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-1a22ca19-5ff7-47bf-9193-18856d1cf24f tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:39.027187 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d161c233-4f70-48c6-b1d0-c8b52238f969 req-c5a0ba4a-3bc4-414e-a907-05f05cdf0d87 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:39.027763 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d161c233-4f70-48c6-b1d0-c8b52238f969 req-c5a0ba4a-3bc4-414e-a907-05f05cdf0d87 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:39.028278 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [req-d161c233-4f70-48c6-b1d0-c8b52238f969 req-c5a0ba4a-3bc4-414e-a907-05f05cdf0d87 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:39.040806 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.compute.server_external_events [req-d161c233-4f70-48c6-b1d0-c8b52238f969 req-c5a0ba4a-3bc4-414e-a907-05f05cdf0d87 service nova] Creating event network-changed:da201a4d-39e0-43c4-b336-8ccd6da2e34e for instance 7dda007a-6f7c-4b57-a910-480eedbf967f on nch1-master-1 Sep 27 11:25:39.049153 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [req-d161c233-4f70-48c6-b1d0-c8b52238f969 req-c5a0ba4a-3bc4-414e-a907-05f05cdf0d87 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037876 Sep 27 11:25:39.049914 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3521/7516] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:25:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:25:39.053506 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-1a22ca19-5ff7-47bf-9193-18856d1cf24f tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Going to try to terminate instance {{(pid=145968) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 27 11:25:39.143718 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-1a22ca19-5ff7-47bf-9193-18856d1cf24f tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "DELETE /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 204 len: 0 microversion: 2.1 time: 0.133940 Sep 27 11:25:39.148817 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3996/7517] 10.222.0.26 () {58 vars in 1239 bytes} [Tue Sep 27 11:25:39 2022] DELETE /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 27 11:25:39.155133 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-0cd9074c-cbd9-4b3e-b370-589da3fbc80c tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:25:39.155871 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-0cd9074c-cbd9-4b3e-b370-589da3fbc80c tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:25:39.162772 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cd9074c-cbd9-4b3e-b370-589da3fbc80c tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:39.163502 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cd9074c-cbd9-4b3e-b370-589da3fbc80c tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:39.164115 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cd9074c-cbd9-4b3e-b370-589da3fbc80c tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:39.164903 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cd9074c-cbd9-4b3e-b370-589da3fbc80c tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:39.165429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cd9074c-cbd9-4b3e-b370-589da3fbc80c tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:39.165897 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-0cd9074c-cbd9-4b3e-b370-589da3fbc80c tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:39.335691 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-0cd9074c-cbd9-4b3e-b370-589da3fbc80c tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:25:39.344370 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-0cd9074c-cbd9-4b3e-b370-589da3fbc80c tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1681 microversion: 2.1 time: 0.193552 Sep 27 11:25:39.344984 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3522/7518] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:25:39 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1681 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:25:40.284958 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [req-0fefc3eb-34e1-44e5-959b-86135dfb0797 req-1fd03a9d-99ea-4ea2-8126-8a6f281f63c5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7dda007a-6f7c-4b57-a910-480eedbf967f", "name": "network-vif-deleted", "tag": "da201a4d-39e0-43c4-b336-8ccd6da2e34e"}]} {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 27 11:25:40.296710 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0fefc3eb-34e1-44e5-959b-86135dfb0797 req-1fd03a9d-99ea-4ea2-8126-8a6f281f63c5 service nova] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:40.297134 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0fefc3eb-34e1-44e5-959b-86135dfb0797 req-1fd03a9d-99ea-4ea2-8126-8a6f281f63c5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:40.297565 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [req-0fefc3eb-34e1-44e5-959b-86135dfb0797 req-1fd03a9d-99ea-4ea2-8126-8a6f281f63c5 service nova] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:40.309939 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.compute.server_external_events [req-0fefc3eb-34e1-44e5-959b-86135dfb0797 req-1fd03a9d-99ea-4ea2-8126-8a6f281f63c5 service nova] Creating event network-vif-deleted:da201a4d-39e0-43c4-b336-8ccd6da2e34e for instance 7dda007a-6f7c-4b57-a910-480eedbf967f on nch1-master-1 Sep 27 11:25:40.314917 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [req-0fefc3eb-34e1-44e5-959b-86135dfb0797 req-1fd03a9d-99ea-4ea2-8126-8a6f281f63c5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034194 Sep 27 11:25:40.315774 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3997/7519] 10.222.0.26 () {64 vars in 1317 bytes} [Tue Sep 27 11:25:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 27 11:25:40.368873 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:25:40.369508 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.compute.api [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145969) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:25:40.378639 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:40.379144 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:40.379595 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:40.380410 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:40.381042 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:40.381475 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:40.416249 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:40.416786 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:40.417247 ncdv-master-1 devstack@n-api.service[145969]: DEBUG oslo_concurrency.lockutils [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145969) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:40.417717 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.objects.instance [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lazy-loading 'fault' on Instance uuid 7dda007a-6f7c-4b57-a910-480eedbf967f {{(pid=145969) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 27 11:25:40.466429 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.policy [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8bf66c7321304c91840efc2cbeb52a4f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34d2d2279cd14736ba71123e751c9aeb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=145969) authorize /opt/stack/nova/nova/policy.py:199}} Sep 27 11:25:40.474874 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-ee07084a-1680-45e3-a26a-57c32a626497 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 200 len: 1336 microversion: 2.1 time: 0.110141 Sep 27 11:25:40.475444 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3523/7520] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:25:40 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 1336 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 27 11:25:41.505373 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '>' {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:25:41.506545 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.compute.api [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] [instance: 7dda007a-6f7c-4b57-a910-480eedbf967f] Fetching instance by UUID {{(pid=145968) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 27 11:25:41.517558 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:41.518453 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:41.518871 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:41.519603 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Acquiring lock "ed1088da-caae-4426-8048-9cba90e7755c" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 27 11:25:41.520064 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 27 11:25:41.520491 ncdv-master-1 devstack@n-api.service[145968]: DEBUG oslo_concurrency.lockutils [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Lock "ed1088da-caae-4426-8048-9cba90e7755c" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=145968) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 27 11:25:41.534537 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.wsgi [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] HTTP exception thrown: Instance 7dda007a-6f7c-4b57-a910-480eedbf967f could not be found. Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Request method failure captured: Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: request: GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f HTTP/1.1 Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: Accept: application/json Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: Accept-Encoding: identity Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: Connection: close Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: Content-Length: 0 Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: Content-Type: application/json Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: Host: 10.222.0.26 Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: Openstack-System-Scope: None Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: User-Agent: python-urllib3/1.26.12 Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Auth-Token: gAAAAABjMtxvKjnv6wV3GuGjckqdG1-VB06PhBdkv6peC2kyASn40W2fQsHmrRzSuXs9W-hiBYyTtJaYZg6r99sdAdYg0yiV303cwoiwMGEzLqedRGjExIEY-bfg-jbngLxy9YT9pZ5qOA69TL7jNZX7PeW_cIWuAubT5KPY8NpwEtNarR59FDs Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Id: None Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Domain-Name: None Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Identity-Status: Confirmed Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Is-Admin-Project: True Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Id: default Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Domain-Name: Default Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Id: 34d2d2279cd14736ba71123e751c9aeb Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Project-Name: tempest-TestSnapshotPattern-960493092 Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Role: reader,member Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Roles: reader,member Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/34d2d2279cd14736ba71123e751c9aeb"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/34d2d2279cd14736ba71123e751c9aeb"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/34d2d2279cd14736ba71123e751c9aeb"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_34d2d2279cd14736ba71123e751c9aeb"}]}] Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant: tempest-TestSnapshotPattern-960493092 Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Id: 34d2d2279cd14736ba71123e751c9aeb Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-Tenant-Name: tempest-TestSnapshotPattern-960493092 Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-User: tempest-TestSnapshotPattern-960493092-project Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Id: default Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-User-Domain-Name: Default Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-User-Id: 8bf66c7321304c91840efc2cbeb52a4f Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: X-User-Name: tempest-TestSnapshotPattern-960493092-project Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: method: > Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: exception: Instance 7dda007a-6f7c-4b57-a910-480eedbf967f could not be found. Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: action_args: {'id': '7dda007a-6f7c-4b57-a910-480eedbf967f'} Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: {{(pid=145968) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7dda007a-6f7c-4b57-a910-480eedbf967f could not be found. Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7dda007a-6f7c-4b57-a910-480eedbf967f could not be found. Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 27 11:25:41.536175 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:25:41.540084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 27 11:25:41.540084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 27 11:25:41.540084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 27 11:25:41.540084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 27 11:25:41.540084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 27 11:25:41.540084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7dda007a-6f7c-4b57-a910-480eedbf967f could not be found. Sep 27 11:25:41.540084 ncdv-master-1 devstack@n-api.service[145968]: ERROR nova.api.openstack.wsgi Sep 27 11:25:41.540084 ncdv-master-1 devstack@n-api.service[145968]: DEBUG nova.api.openstack.wsgi [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Returning 404 to user: Instance 7dda007a-6f7c-4b57-a910-480eedbf967f could not be found. {{(pid=145968) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 27 11:25:41.540084 ncdv-master-1 devstack@n-api.service[145968]: INFO nova.api.openstack.requestlog [None req-eb86f161-fbb2-4385-a0d2-b7a6ebc1b0fe tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f" status: 404 len: 111 microversion: 2.1 time: 0.040980 Sep 27 11:25:41.540084 ncdv-master-1 devstack@n-api.service[145968]: [pid: 145968|app: 0|req: 3998/7521] 10.222.0.26 () {58 vars in 1236 bytes} [Tue Sep 27 11:25:41 2022] GET /compute/v2.1/servers/7dda007a-6f7c-4b57-a910-480eedbf967f => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 27 11:25:41.728019 ncdv-master-1 devstack@n-api.service[145969]: DEBUG nova.api.openstack.wsgi [None req-f6a01e4c-b929-48df-be84-5d3c9b5e5385 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] Calling method '.version_select at 0x7f7f6c08e160>' {{(pid=145969) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 27 11:25:41.755579 ncdv-master-1 devstack@n-api.service[145969]: INFO nova.api.openstack.requestlog [None req-f6a01e4c-b929-48df-be84-5d3c9b5e5385 tempest-TestSnapshotPattern-960493092 tempest-TestSnapshotPattern-960493092-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestSnapshotPattern-83732341" status: 202 len: 0 microversion: 2.1 time: 0.031502 Sep 27 11:25:41.756124 ncdv-master-1 devstack@n-api.service[145969]: [pid: 145969|app: 0|req: 3524/7522] 10.222.0.26 () {58 vars in 1251 bytes} [Tue Sep 27 11:25:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestSnapshotPattern-83732341 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)